Nos mos describere Hadoop setup in singularibus node et multi node. The Hadoop environment setup and configuration will be described in details. Primo vos postulo ut download sequenti software (RPM).
- Java JDK RPM
- Apache Hadoop 0.20.204.0 RPM
A) Single node system Hadoop setup
1) Install JDK in Rubrum Hat aut CentOS 5+ system.
$ ./jdk-6u26-linux-x64-rpm.bin.sh
Java installatur et statuit JAVA_HOME ad / usr / Java / default
2) Install Apache Hadoop 0.20.204.
$ RPM-i hadoop-0.20.204.0-1.i386.rpm
3) Setup Apache Hadoop configuration et satus Hadoop processibus.
$ /usr / sbin / hadoop-setup-unius node.sh
Setup veneficus, docebit vos per a album of quaestiones ad setup Hadoop. Hadoop debet currente post respondens 'Y' quaestiones omnes.
Partum a user rationem in HDFS pro teipso.
$ /usr / sbin / hadoop-faceres-user.sh-u $ USUFRUCTUARIUS
B) Multi-nodes Hadoop setup
1) Install tam JDK et Hadoop 0.20.204.0 RPMs super omnes nodorum
2) Generare hadoop configuration super omnes nodorum:
$ /usr / sbin / hadoop-setup-conf.sh \
--namenode-url=hdfs://${namenode}:9000/ \
--jobtracker-url=${jobtracker}:9001 \
--conf-dir=/etc/hadoop \
--hdfs-dir=/var/lib/hadoop/hdfs \
--namenode-dir=/var/lib/hadoop/hdfs/namenode \
--mapred-dir=/var/lib/hadoop/mapred \
--datanode-dir=/var/lib/hadoop/hdfs/data \
--log-dir=/var/log/hadoop \
--auto
Ubi ${namenode} et ${jobtracker} debet reponatur cum hostname de namenode et jobtracker.
3) Format namenode et setup default HDFS layout.
$ /usr / sbin / hadoop-setup-hdfs.sh
4) Satus omnia indicia nodorum.
$ /etc / init.d / hadoop-datanode satus
5) Satus job venato node.
$ /etc / init.d / hadoop-jobtracker satus
6) Satus munus venato nodorum.
$ /etc / init.d / Hadoop-munus venato starter
7) Partum a user rationem in HDFS pro teipso.
$ /usr / sbin / hadoop-faceres-user.sh-u $ USUFRUCTUARIUS
C) Setup environment pro Hadoop
$ VI ~ /. bash_profile
In INSERT modus paro semita pro JAVA_HOME
Export JAVA_HOME
Salvum file per clicking esc:wq
Currendam. Bash_profile
$ source ~ /. bash_profile
Pone JAVA_HOME semita in Hadoop environment file
$ /etc / Hadoop / Hadoop-env.sh
D) Configuration pro Hadoop
Utor sequens:
Confessiones / core-site.xml:
| <configuration> |
| <proprietas> |
| <nomen>fs.default.name</nomen> |
| <valorem>hdfs://localhost:9000</valorem> |
| </proprietas> |
| </configuration> |
Confessiones / hdfs-site.xml:
| <configuration> |
| <proprietas> |
| <nomen>dfs.replication</nomen> |
| <valorem>1</valorem> |
| </proprietas> |
| </configuration> |
Confessiones / mapred-site.xml:
| <configuration> |
| <proprietas> |
| <nomen>mapred.job.tracker</nomen> |
| <valorem>localhost:9001</valorem> |
| </proprietas> |
| </configuration> |
Eam) Hadoop order
$ hadoop
$ hadoop namenode-format (Format in namenode, Si flagitas,
Respondeo dicendum press 'Y')
$ hadoop NameNode (Satus namenode)
$ invenire / -nominare satus-dfs.sh (invenire lima in album)
$ cd usr / sbin (Vade ad respectivis presul directe)
$ satus-dfs.sh
$ satus-mapred.sh
$ hadoop FS-LS / (Ostendit HDFS radix folder)
$ hadooop FS-posuit input/file01 / input/file01 (Effingo localis input/file01 ad
HDFS radix / input/file01)
