Ans : Hadoop streaming yog lub company haib uas muaj Hadoop tis.Lub tswvyim ntawm Hadoop moj khaum theem pib yog phua tau hauj lwm,txheej txheem nws nyob mus tib seem thiab ces koom nws rov qab mus rau thaum xaus no.Ces muaj muaj ob lub ntsiab cheebtsam no lub moj khaum.
ib) Daim ntawv qhia npe
b) Kom tau ntaub ntawv
Lub Hadoop streaming company tso cai rau koj sau daim ntawv qhia los txo tej kev siv tej lus uas muaj cuab kav ua hauj lwm nrog cov STDIN thiab STDOUT.
Kuv nyeem koj tsab xov xwm qhia txog tau streaming hadoop. Kuv pom tias nws yuav pab tau tiag tiag. Tab sis, kuv muaj lus nug xav paub ntxiv txog yuav siv txoj kev no.
Ib ntsiab lus nug kuv xav nug no yog kuv tsab ntawv perl xav tau ntau dua ib tug sib cav, li cas kuv kis tau los ntawm kev hais kom ua kab?
Piv txwv, Kuv siv cov lus hauv qab no hais kom ua, qhov twg kuv siv ntau inputs lis ntau yam lus. Tab sis qhov tseeb, lub tswv yim ntawv cov ntaub ntawv no yog tus thawj ib tug. Tag nrho lwm cov neeg no yog tej kev pab perl tsab ntawv yuav tsum tau nyeem ntawv rau mus pab txheem cov ntaub ntawv thawj input.
hadoop thawv /usr/lib/hadoop-mapreduce/hadoop-streaming.jar -D mapred.reduce.tasks=0 -D mapred.map.tasks.speculative.execution=false -D mapred.task.timeout=12000000-input nlp_research/edt_nlp_data/3000001.txt-input shift.txt-teev tswv yim - input dict-input nlp_research/deid-1.1/deid.config - inputformat org.apache.hadoop.mapred.lib.NLineInputFormat-zis perl_output-mapper deid_mapper.pl-thov deid_mapper.pl
Yog hais tias koj tau muab tej kev taw qhia, uas yuav poj!