Peb yuav piav Hadoop teeb rau ntawm ib qho ntawm thiab zov ntawm. Lub chaw teeb ntawm Hadoop thiab configuration yuav muaj nyob hauv cov lus. Ua ntej koj yuav tsum download tau lub qab software (rpm).
- Java JDK RPM
- Apache Hadoop 0.20.204.0 RPM
IB) Ib qho ntawm qhov system Hadoop teeb
1) Install JDK on a Red Hat or CentOS 5+ tsim.
$ ./jdk-6u26-linux-x 64-rpm.bin.sh
Java yog ntsia thiab muab cov JAVA_HOME los /usr/java/default
2) Install Apache Hadoop 0.20.204.
$ rpm -i hadoop-0.20.204.0-1.i386.rpm
3) Apache Hadoop configuration teeb thiab pib muaj dab Hadoop.
$ /usr/sbin/hadoop-setup-Single-node.sh
Lub teeb wizard mam coj koj mus rau ib daim ntawv teev cov lus yuav muaj teeb Hadoop. Hadoop yuav tsum khiav tom qab teb 'Y' rau tag nrho cov lus nug.
Ua ib tug neeg siv nyiaj ntawm HDFS rau koj tus kheej.
$ /usr/sbin/hadoop-create-user.sh -u $USER
B) O tej Hadoop teeb
1) Nruab ib ob lub JDK thiab Hadoop 0.20.204.0 RPMs nyob rau tag nrho cov ntshav
2) Ua kom muaj hadoop configuration rau tag nrho cov ntshav:
$ /usr/sbin/hadoop-setup-conf.sh \
--namenode-url = hdfs://${namenode}:9000/ \
--jobtracker-url = ${jobtracker}:9001 \
--conf dir = / yam/hadoop \
--hdfs-dir = / var/lib/hadoop/hdfs \
--namenode-dir = / var/lib/hadoop/hdfs/namenode \
--mapred-dir = / var/lib/hadoop/mapred \
--datanode-dir = / var/lib/hadoop/hdfs/ntaub ntawv \
--cav-dir = / var/cav/hadoop \
--Nws pib
Nyob qhov twg ${namenode} thiab ${jobtracker} yuav tsum muab hloov rau hostname namenode thiab jobtracker.
3) Hom ntawv namenode thiab teeb default HDFS layout.
$ /usr/sbin/hadoop-setup-hdfs.sh
4) Pib rau tag nrho cov ntaub ntawv o.
$ /etc/init.d/hadoop-datanode pib
5) Pib txoj hauj lwm tracker ntawm.
$ /etc/init.d/hadoop-jobtracker pib
6) Pib ua hauj lwm tracker o.
$ /etc/init.d/hadoop-tasktracker pib
7) Ua ib tug neeg siv nyiaj ntawm HDFS rau koj tus kheej.
$ /usr/sbin/hadoop-create-user.sh -u $USER
C) Teeb ib puag ncig rau Hadoop
$ VI ~/.bash_profile
Nyob rau hauv tsab ntawv nrog thaum hom tso kev rau JAVA_HOME
Export JAVA_HOME
Tseg tej ntaub ntawv los ntawm txhaj ESC:wq
Khiav lub .bash_profile
$ tau qhov twg los ~/.bash_profile
Koj muab JAVA_HOME kab nyob hauv Hadoop cov chaw kawm ntawv
$ /etc/hadoop/hadoop-env.sh
D) Configuration rau cov Hadoop
Siv cov nram qab no:
conf/tub ntxhais-site.xml:
| <configuration> |
| <khoom> |
| <lub npe>FS.default.name</lub npe> |
| <tus nqi>hdfs://localhost:9000</tus nqi> |
| </khoom> |
| </configuration> |
conf/hdfs-site.xml:
| <configuration> |
| <khoom> |
| <lub npe>dfs.replication</lub npe> |
| <tus nqi>1</tus nqi> |
| </khoom> |
| </configuration> |
conf/mapred-site.xml:
| <configuration> |
| <khoom> |
| <lub npe>mapred.Job.tracker</lub npe> |
| <tus nqi>localhost:9001</tus nqi> |
| </khoom> |
| </configuration> |
E) Hadoop Commands
$ hadoop
$ hadoop namenode – hom (Format lub namenode, Yog hais tias hais rau
teb xovxwm 'Y')
$ hadoop namenode (Pib lub namenode)
$ nrhiav tau / -lub npe pib-dfs.sh (nrhiav cov ntaub ntawv hauv directory)
$ CD usr/sbin (Mus duas paub directory ncaj nraim)
$ muab cov khaub noom-dfs.sh
$ muab cov khaub noom-mapred.sh
$ hadoop fs – ls / (Qhia tau hais tias tus ntawv tais ceev tseg hauv paus HDFS)
$ hadooop fs – muab tswv yim ntawv/file01 /input/file01 (Daim qauv hauv zos tawm tswv yim/file01 rau
HDFS hauv paus /input/file01)
