在阿里云临时租了三台服务器，做hadoop部署使用.

集群角色分工

第1台：hadoop-master

   角色：NameNode;ResourceManager

   Addr：172.26.73.102

第2台：hadoop-slave1

   角色：DataNode;NodeManager

   Addr：172.26.73.103

第3台：hadoop-slave2

   角色：DataNode;NodeManager

   Addr：172.26.73.104

上传文件

 hadoop2.4.1和jdk1.7目录上传到hadoop-master的／root下面

免密SSH登录配置

      分别在三台机器上执行此命令

      ssh-keygen -t rsa  

      ssh-copy-id -i .ssh/id_rsa.pub root@hadoop-slave1

      ssh-copy-id -i .ssh/id_rsa.pub root@hadoop-slave2

      ssh-copy-id -i .ssh/id_rsa.pub root@hadoop-master

配置环境变量

三台机器上配置HADOOP_HOME相关参数，修改/etc/profile

JAVA_HOME=/root/jdk1.7.0_80/

export JAVA_HOME

PATH=$PATH:$JAVA_HOME/bin

export PATH

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

HADOOP_HOME=/root/hadoop-2.4.1

PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

export HADOOP_HOME PATH

修改配置文件

hadoop-env.sh 

修改JAVA_HOME

export JAVA_HOME=/root/jdk1.7.0_80

hdfs-site.xml

<property>

    <name>dfs.replication</name>

    <value>2</value>

  </property>

  <property>

    <name>dfs.permissions</name>

    <value>false</value>

  </property>

core-site.xml

<property>

    <name>fs.defaultFS</name>

    <value>hdfs://172.26.73.102:9000</value>

  </property>

  <property>

    <name>hadoop.tmp.dir</name>

    <value>/root/hadoop-2.4.1/tmp</value>

  </property>

slaves

172.26.73.103

172.26.73.104

mapred-site.xml

<property>

    <name>mapreduce.framework.name</name>

    <value>yarn</value>

  </property>

yarn-site.xml

<property>

        <name>yarn.resourcemanager.hostname</name>

        <value>172.26.73.102</value>

</property>

<property>

        <name>yarn.nodemanager.aux-services</name>

        <value>mapreduce_shuffle</value>

</property>

格式化namenode目录

hdfs namenode -format

传送文件

把hadoop-master的JDK目录和hadoop目录，传到hadoop-slave1和hadoop-slave1的/root下面

scp -r /root/* root@hadoop-slave1:/root

 scp -r /root/* root@hadoop-slave2:/root

启动hadoop

hadoop-master上运行start-all.sh，可能出现类似下面问题：

fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack' 

Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /root/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.

请在hadoop-env.sh和yarn-env.sh添加

export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_PREFIX}/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib" 

启动成功用，可以在hadoop-master运行jps，看到：

13958 SecondaryNameNode

13782 NameNode

14478 Jps

14107 ResourceManager

hadoop-slave1和hadoop-slave2，运行jps，看到：

13958 SecondaryNameNode

13782 NameNode

14492 Jps

14107 ResourceManager