最新Hadoop-2.7.2+hbase-1.2.0+zookeeper-3.4.8 HA高可用集群配置安装

?

?

?

关于准备工作??我这里就不一一写出来了，总结一下有主机名，ip，主机名和ip的映射关系，防火墙，ssh免密码，jdk的安装及环境变量的设置。

安装zookeeper到?h5、h6、h7上面

修改?/home/zookeeper-3.4.8/conf的zoo_sample.cfg

cp zoo_sample.cfg zoo.cfg

?

?

# The number of milliseconds of each tick

tickTime=2000

# The number of ticks that the initial

# synchronization phase can take

initLimit=10

# The number of ticks that can pass between

# sending a request and getting an acknowledgement

syncLimit=5

# the directory where the snapshot is stored.

# do not use /tmp for storage, /tmp here is just

# example sakes.

dataDir=/home/zookeeper-3.4.8/data

# the port at which the clients will connect

clientPort=2181

# the maximum number of client connections.

# increase this if you need to handle more clients

#maxClientCnxns=60

#

# Be sure to read the maintenance section of the

# administrator guide before turning on autopurge.

#

# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance

#

# The number of snapshots to retain in dataDir

#autopurge.snapRetainCount=3

# Purge task interval in hours

# Set to "0" to disable auto purge feature

#autopurge.purgeInterval=1

server.1=h5:2888:3888

server.2=h6:2888:3888

server.3=h7:2888:3888

?

?

?

?

创建?data文件夹??和在里面??创建文件myid??并写入数字1

touch data/myid

?

echo 1 > data/myid

?

拷贝整个zookeeper到另外两个节点上

?

scp -r /home/zookeeper-3.4.8? h6:/home/

scp -r /home/zookeeper-3.4.8? h7:/home/

其他两个节点的myid??修改为?2? 3

安装hadoop

?

/home/hadoop-2.7.2/etc/Hadoop

?

hadoop-env.sh：

export JAVA_HOME=/home/jdk

?

core-site.xml：

?

?

<configuration>

<!-- 指定hdfs的nameservice为masters -->

<property>

<name>fs.defaultFS</name>

<value>hdfs://masters</value>

</property>

<!-- 指定hadoop临时目录 -->

<property>

<name>hadoop.tmp.dir</name>

<value>/home/hadoop-2.7.2/tmp</value>

</property>

<!-- 指定zookeeper地址 -->

<property>

<name>ha.zookeeper.quorum</name>

<value>h5:2181,h6:2181,h7:2181</value>

</property>

</configuration>

?

?

hdfs-site.xml：

?

?

?

<configuration>

<!--指定hdfs的nameservice为masters，需要和core-site.xml中的保持一致 -->

??????? <property>

??????????????? <name>dfs.nameservices</name>

??????????????? <value>masters</value>

??????? </property>

??????? <!-- h1下面有两个NameNode，分别是h1，h2 -->

??????? <property>

??????????????? <name>dfs.ha.namenodes.masters</name>

??????????????? <value>h1,h2</value>

??????? </property>

??????? <!-- h1的RPC通信地址 -->

??????? <property>

??????????????? <name>dfs.namenode.rpc-address.masters.h1</name>

??????????????? <value>h1:9000</value>

??????? </property>

??????? <!-- h1的http通信地址 -->

??????? <property>

??? ????????????<name>dfs.namenode.http-address.masters.h1</name>

??????????????? <value>h1:50070</value>

??????? </property>

??????? <!-- h2的RPC通信地址 -->

??????? <property>

??????????????? <name>dfs.namenode.rpc-address.masters.h2</name>

??????????????? <value>h2:9000</value>

??????? </property>

??????? <!-- h2的http通信地址 -->

??????? <property>

??????????????? <name>dfs.namenode.http-address.masters.h2</name>

??????????????? <value>h2:50070</value>

??????? </property>

??????? <!-- 指定NameNode的元数据在JournalNode上的存放位置 -->

??????? <property>

??????????????? <name>dfs.namenode.shared.edits.dir</name>

??????????????? <value>qjournal://h5:8485;h6:8485;h7:8485/masters</value>

??????? </property>

??????? <!-- 指定JournalNode在本地磁盘存放数据的位置 -->

??????? <property>

??????????????? <name>dfs.journalnode.edits.dir</name>

??????????????? <value>/home/hadoop-2.7.2/journal</value>

??????? </property>

??????? <!-- 开启NameNode失败自动切换 -->

??????? <property>

??????????????? <name>dfs.ha.automatic-failover.enabled</name>

??????????????? <value>true</value>

??????? </property>

??????? <!-- 配置失败自动切换实现方式 -->

??????? <property>

??????????????? <name>dfs.client.failover.proxy.provider.masters</name>

??????????????? <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>

??????? </property>

??????? <!-- 配置隔离机制方法，多个机制用换行分割，即每个机制暂用一行-->

??????? <property>

??????????????? <name>dfs.ha.fencing.methods</name>

??????????????? <value>

??????????????????????? sshfence

??????????????????????? shell(/bin/true)

??????????????? </value>

??????? </property>

??????? <!-- 使用sshfence隔离机制时需要ssh免登陆 -->

??????? <property>

??????????????? <name>dfs.ha.fencing.ssh.private-key-files</name>

??????????????? <value>/root/.ssh/id_rsa</value>

??????? </property>

??????? <!-- 配置sshfence隔离机制超时时间 -->

??????? <property>

??????????????? <name>dfs.ha.fencing.ssh.connect-timeout</name>

??????????????? <value>30000</value>

??????? </property>

</configuration>

?

?

?

mapred-site.xml：

?

?

<configuration>

<!-- 指定mr框架为yarn方式 -->

<property>

<name>mapreduce.framework.name</name>

<value>yarn</value>

</property>

</configuration>

?

yarn-site.xml：

?

<configuration>

?

<!-- 开启RM高可靠 -->

??????? <property>

??????????????? <name>yarn.resourcemanager.ha.enabled</name>

??????????????? <value>true</value>

??????? </property>

??????? <!-- 指定RM的cluster id -->

??????? <property>

??????????????? <name>yarn.resourcemanager.cluster-id</name>

??????????????? <value>RM_HA_ID</value>

??????? </property>

??????? <!-- 指定RM的名字 -->

??????? <property>

??????????????? <name>yarn.resourcemanager.ha.rm-ids</name>

??????????????? <value>rm1,rm2</value>

??????? </property>

??????? <!-- 分别指定RM的地址 -->

??????? <property>

??????????????? <name>yarn.resourcemanager.hostname.rm1</name>

??????????????? <value>h3</value>

??????? </property>

??????? <property>

?? ?????????????<name>yarn.resourcemanager.hostname.rm2</name>

??????????????? <value>h4</value>

??????? </property>

??????? <property>

??????????????? <name>yarn.resourcemanager.recovery.enabled</name>

??????????????? <value>true</value>

??????? </property>

????????

??????? <property>

??????????????? <name>yarn.resourcemanager.store.class</name>

??????????????? <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>

??????? </property>

??????? <!-- 指定zk集群地址 -->

??????? <property>

??????????????? <name>yarn.resourcemanager.zk-address</name>

??????????????? <value>h5:2181,h6:2181,h7:2181</value>

??????? </property>

??????? <property>

??????????????? <name>yarn.nodemanager.aux-services</name>

????????? ??????<value>mapreduce_shuffle</value>

??????? </property>

</configuration>

?

?

Slaves：

?

h5

h6

h7

?

然后 拷贝到其他节点

?

?

scp -r hadoop-2.7.2 h2:/home/????等等

?

?

这个地方说明一下 ?yarn 的HA ?是在 ?h3和h4 ?上面

?

?

启动顺序

###注意：严格按照下面的步骤

?

1.???????启动zookeeper集群

?

[root@h6 ~]# cd /home/zookeeper-3.4.8/bin/

[root@h6 bin]# ./zkServer.sh start

?

H5? h6? h7??都一样

[root@h6 bin]# ./zkServer.sh status

查看状态

?

2.???????启动journalnode

[root@h5 bin]# cd /home/hadoop-2.7.2/sbin/

[root@h5 sbin]# ./hadoop-daemons.sh start journalnode

h5: starting journalnode, logging to /home/hadoop-2.7.2/logs/hadoop-root-journalnode-h5.out

h7: starting journalnode, logging to /home/hadoop-2.7.2/logs/hadoop-root-journalnode-h7.out

h6: starting journalnode, logging to /home/hadoop-2.7.2/logs/hadoop-root-journalnode-h6.out

[root@h5 sbin]# jps

2420 JournalNode

2309 QuorumPeerMain

2461 Jps

[root@h5 sbin]# ^C

?

?

3.???????格式化HDFS

?

在h1上执行命令:

hdfs namenode -format

格式化后会在根据core-site.xml中的hadoop.tmp.dir配置生成个文件

拷贝tmp?到?h2

[root@h1 hadoop-2.7.2]# scp -r tmp/ h2:/home/hadoop-2.7.2/

?

4.?格式化ZK(在h1上执行即可)

?

[root@h1 hadoop-2.7.2]# hdfs zkfc -formatZK

?

5.?启动HDFS(在h1上执行)

?

[root@h1 hadoop-2.7.2]# sbin/start-dfs.sh

16/02/25 05:01:14 WARN hdfs.DFSUtil: Namenode for ns1 remains unresolved for ID null.? Check your hdfs-site.xml file to ensure namenodes are configured properly.

16/02/25 05:01:14 WARN hdfs.DFSUtil: Namenode for ns2 remains unresolved for ID null.? Check your hdfs-site.xml file to ensure namenodes are configured properly.

16/02/25 05:01:14 WARN hdfs.DFSUtil: Namenode for ns3 remains unresolved for ID null.? Check your hdfs-site.xml file to ensure namenodes are configured properly.

Starting namenodes on [h1 h2 masters masters masters]

masters: ssh: Could not resolve hostname masters: Name or service not known

masters: ssh: Could not resolve hostname masters: Name or service not known

masters: ssh: Could not resolve hostname masters: Name or service not known

h2: starting namenode, logging to /home/hadoop-2.7.2/logs/hadoop-root-namenode-h2.out

h1: starting namenode, logging to /home/hadoop-2.7.2/logs/hadoop-root-namenode-h1.out

h5: starting datanode, logging to /home/hadoop-2.7.2/logs/hadoop-root-datanode-h5.out

h7: starting datanode, logging to /home/hadoop-2.7.2/logs/hadoop-root-datanode-h7.out

h6: starting datanode, logging to /home/hadoop-2.7.2/logs/hadoop-root-datanode-h6.out

Starting journal nodes [h5 h6 h7]

h5: journalnode running as process 2420. Stop it first.

h6: journalnode running as process 2885. Stop it first.

h7: journalnode running as process 2896. Stop it first.

Starting ZK Failover Controllers on NN hosts [h1 h2 masters masters masters]

masters: ssh: Could not resolve hostname masters: Name or service not known

masters: ssh: Could not resolve hostname masters: Name or service not known

masters: ssh: Could not resolve hostname masters: Name or service not known

h2: starting zkfc, logging to /home/hadoop-2.7.2/logs/hadoop-root-zkfc-h2.out

h1: starting zkfc, logging to /home/hadoop-2.7.2/logs/hadoop-root-zkfc-h1.out

[root@h1 hadoop-2.7.2]#

?

6.?启动YARN(是在h3上执行start-yarn.sh，把namenode和resourcemanager分开是因为性能问题，因为他们都要占用大量资源，所以把他们分开了，他们分开了就要分别在不同的机器上启动)

?

[root@h3 sbin]# ./start-yarn.sh

?

[root@h4 sbin]# ./yarn-daemons.sh start resourcemanager

?

?

验证：

?

http://192.168.128.11:50070

?

Overview ‘h1:9000‘ (active)

?

?

http://192.168.128.12:50070

?

?

Overview ‘h2:9000‘ (standby)

?

上传文件

[root@h4 bin]# hadoop fs -put /etc/profile /profile

[root@h4 bin]# hadoop fs -ls

ls: `.‘: No such file or directory

[root@h4 bin]# hadoop fs -ls /

Found 1 items

-rw-r--r--?? 3 root supergroup?????? 1814 2016-02-26 19:08 /profile

[root@h4 bin]#

?

杀死h1

[root@h1 sbin]# jps

2480 NameNode

2868 Jps

2775 DFSZKFailoverController

[root@h1 sbin]# kill -9 2480

[root@h1 sbin]# jps

2880 Jps

2775 DFSZKFailoverController

[root@h1 sbin]# hadoop fs -ls /

Found 1 items

-rw-r--r--?? 3 root supergroup?????? 1814 2016-02-26 19:08 /profile

?

此时?h2??变为active

?

手动启动?h1的?namenode

?

[root@h1 sbin]# ./hadoop-daemon.sh start namenode

starting namenode, logging to /home/hadoop-2.7.2/logs/hadoop-root-namenode-h1.out

[root@h1 sbin]# hadoop jar /home/hadoop-2.7.2/s

?

观察? h1?状态为standby

?

验证yarn

?

[root@h1 sbin]# hadoop jar /home/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar wordcount /profile /out

16/02/26 19:14:23 INFO input.FileInputFormat: Total input paths to process : 1

16/02/26 19:14:23 INFO mapreduce.JobSubmitter: number of splits:1

16/02/26 19:14:23 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1456484773347_0001

16/02/26 19:14:24 INFO impl.YarnClientImpl: Submitted application application_1456484773347_0001

16/02/26 19:14:24 INFO mapreduce.Job: The url to track the job: http://h3:8088/proxy/application_1456484773347_0001/

16/02/26 19:14:24 INFO mapreduce.Job: Running job: job_1456484773347_0001

16/02/26 19:14:49 INFO mapreduce.Job: Job job_1456484773347_0001 running in uber mode : false

16/02/26 19:14:49 INFO mapreduce.Job:? map 0% reduce 0%

16/02/26 19:15:05 INFO mapreduce.Job:? map 100% reduce 0%

16/02/26 19:15:22 INFO mapreduce.Job:? map 100% reduce 100%

16/02/26 19:15:23 INFO mapreduce.Job: Job job_1456484773347_0001 completed successfully

16/02/26 19:15:23 INFO mapreduce.Job: Counters: 49

??????? File System Counters

??????????????? FILE: Number of bytes read=2099

??????????????? FILE: Number of bytes written=243781

????????????? ??FILE: Number of read operations=0

??????????????? FILE: Number of large read operations=0

??????????????? FILE: Number of write operations=0

??????????????? HDFS: Number of bytes read=1901

??????????????? HDFS: Number of bytes written=1470

????????????? ??HDFS: Number of read operations=6

??????????????? HDFS: Number of large read operations=0

??????????????? HDFS: Number of write operations=2

??????? Job Counters

??????????????? Launched map tasks=1

??????????????? Launched reduce tasks=1

????????????? ??Data-local map tasks=1

??????????????? Total time spent by all maps in occupied slots (ms)=13014

??????????????? Total time spent by all reduces in occupied slots (ms)=13470

??????????????? Total time spent by all map tasks (ms)=13014

??????????????? Total time spent by all reduce tasks (ms)=13470

??????????????? Total vcore-milliseconds taken by all map tasks=13014

??????????????? Total vcore-milliseconds taken by all reduce tasks=13470

??????????????? Total megabyte-milliseconds taken by all map tasks=13326336

??????????????? Total megabyte-milliseconds taken by all reduce tasks=13793280

??????? Map-Reduce Framework

??????????????? Map input records=80

??????????????? Map output records=256

??????????????? Map output bytes=2588

??????????????? Map output materialized bytes=2099

??????????????? Input split bytes=87

??????????????? Combine input records=256

??????????????? Combine output records=156

??????????????? Reduce input groups=156

??????????????? Reduce shuffle bytes=2099

??????????????? Reduce input records=156

??????????????? Reduce output records=156

??????????????? Spilled Records=312

??????????????? Shuffled Maps =1

??????????????? Failed Shuffles=0

??????????????? Merged Map outputs=1

??????????????? GC time elapsed (ms)=395

??????????????? CPU time spent (ms)=4100

??????????????? Physical memory (bytes) snapshot=298807296

??????????????? Virtual memory (bytes) snapshot=4201771008

??????????????? Total committed heap usage (bytes)=138964992

??????? Shuffle Errors

??????????????? BAD_ID=0

?????? ?????????CONNECTION=0

??????????????? IO_ERROR=0

??????????????? WRONG_LENGTH=0

??????????????? WRONG_MAP=0

??????????????? WRONG_REDUCE=0

??????? File Input Format Counters

??????????????? Bytes Read=1814

??????? File Output Format Counters

??????????? ????Bytes Written=1470

[root@h1 sbin]# hadoop fs -ls /

Found 3 items

drwxr-xr-x?? - root supergroup????????? 0 2016-02-26 19:15 /out

-rw-r--r--?? 3 root supergroup?????? 1814 2016-02-26 19:08 /profile

drwx------?? - root supergroup????????? 0 2016-02-26 19:14 /tmp

[root@h1 sbin]#

?

Hadoop ha??集群搭建完成

?

安装hbase

hbase-env.sh：

?

export JAVA_HOME=/home/jdk

export HBASE_MANAGES_ZK=false

?

?

?

hbase-site.xml：

?

?

<configuration>

<property>

<name>hbase.rootdir</name>

<value>hdfs://h1:9000/hbase</value>

</property>

<property>

<name>hbase.cluster.distributed</name>

<value>true</value>

</property>

?

<property>

<name>hbase.master</name>

<value>h1:60000</value>

</property>

?<property>

?<name>hbase.master.port</name>

?<value>60000</value>

?<description>The port master should bind to.</description>

?</property>

?

?

<property>

<name>hbase.zookeeper.quorum</name>

<value>h5,h6,h7</value>

</property>

<property>

<name>dfs.replication</name>

<value>3</value>

</property>

</configuration>

?

注意：$HBASE_HOME/conf/hbase-site.xml的hbase.rootdir的主机和端口号与$HADOOP_HOME/conf/core-site.xml的fs.default.name的主机和端口号一致

?

?

Regionservers：内容为:

h5

h6

h7

?

复制到h2? h5,h6,h7上面

?

整个启动顺序

?

按照上面启动hadoop? ha??的顺序??先启动好

?

然后在h1，h2上启动hbase

?

./start-hbase.sh

?

?

测试进入?hbase

?

[root@h1 bin]# hbase shell

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/home/hbase-1.2.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/home/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

HBase Shell; enter ‘help<RETURN>‘ for list of supported commands.

Type "exit<RETURN>" to leave the HBase Shell

Version 1.2.0, r25b281972df2f5b15c426c8963cbf77dd853a5ad, Thu Feb 18 23:01:49 CST 2016

?

hbase(main):001:0> esit

NameError: undefined local variable or method `esit‘ for #<Object:0x7ad1caa2>

?

hbase(main):002:0> exit

?

至此全部结束。

最新Hadoop-2.7.2+hbase-1.2.0+zookeeper-3.4.8 HA高可用集群配置安装

原文：http://weir2009.iteye.com/blog/2279118