Lawm los, koj yuav tsum muaj cov sij hawm ntau ntawv muaj tseeb paub. Nws tsis yog, loj cov ntaub ntawv yeej yog ib lub sij hawm uas yog siv ntau thiab txoj teeb ua khub. Lis ntaub ntawv loj, ib qho yuav tsum tau siv daim ntaub ntawv txawv ua cov ntaub ntawv uas tsis yog yam tshuaj uas feem ntau yog siv.
Yog li no zoo li cas raws nraim hauv daim ntaub ntawv txawv ua cov ntaub ntawv? Thaum muaj ntau yam ntawv uas cia li tu thiab ntxuav ntawm cov ntaub ntawv loj, lub moj khaum los luj tas lawm cov Apache Hadoop.
Apache Hadoop yog dab tsi?
Hadoop yog ib tug moj khaum software qhib-qhov chaw sau Java thiab comprises ntawm ob yam, uas yog cia ib sab thiab sib tau cov ntaub ntawv ua ib sab. Cia ib sab hu ua Hadoop faib cov ntaub ntawv uas (HDFS) ua ib sab yog hu ua MapReduce.
Tuaj hauv no Tshooj, peb yuav ua tibzoo saib mus rau qhov zoo uas yog saib xyuas Hadoop MapReduce programming.
Zoo rau MapReduce programming
Yog qhov zoo uas MapReduce programming-
Scalability
Hadoop cas yuav lub platform uas yog ib scalable. Qhov no yog vim nws muaj peev xwm muab cia raws li faib loj loj poob lawm cov ntaub ntawv nyob rau ntawm servers kom txaus lom zem ntau. Cov servers yuav pheej yig thiab lawv tseem muaj nyob mus tib seem. Tseem, tus muab ntawm servers tsuas ntxiv ua zog.
Contrary rau cov tsoos database paub tswj lub (RDMS) uas tsis tau scale yuav kom cov ntaub ntawv nqi loj loj ntawm cov ntaub ntawv, Hadoop MapReduce programming enables koom haum lag luam khiav daim ntaub ntawv los ntawm ib cov ntshav uas raus cov kev pab ntawm ntau nplooj terabytes ntawm cov ntaub ntawv kuj loj loj.
Cov tshuaj tsis txhob kim
Hadoop tus qauv scalable mas kuj implies tias nws los kom tsis txhob kim heev tov rau cov lag luam uas yuav tsum tau khaws cov ntaub ntawv puas tau cog.
Tus me database paub tswj lub, nws yuav massively raug nqi teev rau cov degrees tau Hadoop prohibitive, cia li mus ua ntaub ntawv cov ntaub ntawv. Zoj, muaj ntau lub lag luam yuav tau downsize tej ntaub ntawv thiab siv raws li cov kev xav tau ntawm koj li cas tej ntaub ntawv yuav tau zoo tshaj plaws classifications ntxiv. Qhov rooj, tej ntaub ntawv yuav tsum tau yuav deleted, xaiv qhov lawv yuav sib enormous nqi cia. Qhov no yeej pab raws li uas tej lub sij hawm luv luv, thiab yog ib lub lag luam yog muaj hloov txoj kev pab them nqi qhov chaw cia cov kab, cov txheej tag tej ntaub ntawv yuav tsis tau kev pab tom qab.
Rau ib tug ntawv kiag li nyias, Hadoop ntawv teev tawm architecture, nrog rau MapReduce programming, tso cai rau cov kua lwj thiab ua cov ntaub ntawv yam pheej yig heev thiab kuj yuav pab rau lub sij hawm tom qab. qhov tseeb, tus nqi nyiaj no loj heev thiab cov nqi kom koj tsis txhob tawm txhiab/10 Nplooj nuj nqis mus rau cov nuj nqis puas rau txhua txhua terabyte ntawm cov ntaub ntawv.
Yooj
Lwm lub koos haum ua hauj lwm yuav ua siv Hadoop MapReduce programming tau rau ntau yam tshiab los ntawm cov ntaub ntawv thiab tseem phais lub hom ntaub ntawv, seb lawv yuav xyaum los yog unstructured. Qhov no pub rau lawv ua kom muaj tus nqi ntawm tag nrho cov ntaub ntawv uas yuav tsum accessed los ntawm lawv.
Raws li tej kab, Hadoop muaj txhawb los lus heev heev uas yuav tau muab siv rau kev ua ntaub ntawv thiab cia. Seb cov ntaub ntawv los yog kev tawm, tug, los yog clickstream, MapReduce yuav ua hauj lwm rau txhua tus. Tseem, Hadoop MapReduce programming pub rau daim ntaub ntawv ntau, xws li lub xav, xyuas txog ntawm qhov cav, li cas rau kev tsom xam, warehousing ntawm cov ntaub ntawv thiab kev dag ntxias kom paub tias.
Cauj
Hadoop siv ib hom hu ua distributed tej ntaub ntawv kaw lus cia, uas yeej implements lub kuas lawv los nrhiav cov ntaub ntawv nyob rau hauv ib pawg. Cov cuab yeej uas siv rau cov ntaub ntawv ua, xws li MapReduce programming, muaj feem ntau yuav nyob rau lub servers qub heev, uas tso cai rau ua ceev cov ntaub ntawv no rau cov.
Txawm tias koj yuav tau soj ntsuam nrog loj tagnrho ntawm cov ntaub ntawv uas yog unstructured li, Hadoop MapReduce yuav siv sij hawm feeb rau txheej txheem ntawm cov ntaub ntawv terabytes, thiab teev rau ntawm cov ntaub ntawv petabytes.
Ruaj ntseg thiab Authentication
Ruaj ntseg yog ib nam ntawm tej ntaub ntawv tseem ceeb heev. Yog hais tias cov chav tsev neeg los yog lub koom haum tau rau ntau lub petabytes ntawm cov ntaub ntawv los ntawm koj lub koom haum, nws yuav ua tau koj loj heev tsim txom ntawd tej lag luam dealings ua hauj lwm thiab.
Hauv no xav txog, MapReduce ua haujlwm nrog HDFS thiab HBase ruaj ntseg uas tso cai rau tsuas pom zoo rau cov neeg los khiav lag luam ntawm cov ntaub ntawv nyob rau hauv lawv.
Thaum uas tig mus ua
Yog ib qhov sib nrauj thawj tus ua hauj lwm hauv MapReduce programming yog tias nws divides paub tab uas lawv tso rau hauv mus tib seem.
Thaum uas tig mus ua pub ntau processors los siv rau cov kev pab raws qib divided, xws li tias lawv kuj muaj tag nrho cov kev pab cuam nyob hauv lub sij hawm tsawg.
Raws li daim phiaj thiab resilient xwm
Thaum cov ntaub ntawv no xa mus rau ib tug ntawm ib tug neeg nyob hauv lub network tag nrho, nyiaj lub teeb heev ib yam ntawm cov ntaub ntawv yog haujlwm tseem khaw tseg mus rau lwm heev heev o uas tsim lub network. Yog li, Yog tias qhov twg tsis ua hauj lwm uas yuav pab tau ib tug ntawm, muaj ntau yeej lwm cov ntaub ntawv uas yuav tseem muaj accessed thaum twg xav tau qhov yuav tshwm sim. Qhov no ib txwm cuabyeej cov nyob rau ntawm qhov chaw.
Ib lub biggest zoo saib xyuas Hadoop yog tias nws kam rau ua txhaum. Hadoop MapReduce muaj peev xwm ceev ceev ceev faj faults uas muaj tshwm sim thiab thov tov ceev thiab tsis siv neeg rov qab ces. Qhov no ua ib changer mas thaum nws yuav loj ua ntaub ntawv tawm.
Qauv yooj yim ntawm programming
Cov qhov ntau cov chaw zoo uas muaj cov Hadoop MapReduce, ib qhov sawv daws yuav tseem ceeb tshaj plaws yog Disease fact ntawd tias nws raws li programming ib tug qauv yooj yim. Qhov no yeej pub rau cov programmers los MapReduce cov kev pab uas yuav siv kev pab raws qib uas yooj yim tshaj thiab efficiency.
Cov kev pab cuam rau MapReduce yuav tau sau ntawv siv Java, Nws yog ib hom lus uas tsis yog nyuaj rau txeej thiab nws kuj yog siv ntau. Yog li, Nws yog ib qho yooj yim rau cov neeg kom paub thiab sau cov kev pab cuam uas raws li lawv xav tau cov ntaub ntawv ua sufficiently.
Xaus
Thaum nws los txog nram AUTHORIZED uas loj teeb ua khub, Hadoop tus MapReduce programming pub rau cov kev ua ntawm tej loj tagnrho ntawm cov ntaub ntawv yam ruaj ntseg thiab tsis kim heev dhau. Hadoop thiab triumphs hla database paub tswj lub nruab thaum nws tawm los ntawm cov ntaub ntawv loj nyob AUTHORIZED. Thaum kawg, cov lag luam muaj twb pom tau hais tus cog lus tias Hadoop tuas thiab yog nws tseem hais tias nws tus nqi rau cov lag luam yuav loj hlob li unstructured cov ntaub ntawv yuav loj hlob.
