hdfs高可用--NA HA

yhqtxy1314 · 发表于 2019-6-14 17:43

hdfs 高可用
https://www.cnblogs.com/yhq1314/p/11024590.html

yhqtxy1314 · 发表于 2019-6-14 17:43

##安装后查看和验证
[hadoop@hadoop1 hadoop]$ hdfs haadmin -getServiceState nn1
19/06/14 17:09:53 DEBUG util.Shell: setsid exited with exit code 0
19/06/14 17:09:53 DEBUG tools.DFSHAAdmin: Using NN principal:
19/06/14 17:09:53 DEBUG namenode.NameNode: Setting fs.defaultFS to hdfs://hadoop1:8020
19/06/14 17:09:53 DEBUG ipc.ProtobufRpcEngine: Call: getServiceStatus took 105ms
active
[hadoop@hadoop1 hadoop]$ hdfs haadmin -getServiceState nn2
19/06/14 17:11:29 DEBUG ipc.ProtobufRpcEngine: Call: getServiceStatus took 107ms
standby

手工切换,将active的NameNode从nn1切换到nn2
[hadoop@hadoop1 hadoop]$ hdfs haadmin -DfSHAadmin -failover nn1 nn2
19/06/14 17:20:16 DEBUG ipc.ProtobufRpcEngine: Call: gracefulFailover took 1467ms
Failover to NameNode at hadoop2/192.168.19.73:8040 successful
[hadoop@hadoop1 hadoop]$ hdfs haadmin -getServiceState nn1
19/06/14 17:29:49 DEBUG ipc.ProtobufRpcEngine: Call: getServiceStatus took 81ms
active
[hadoop@hadoop1 hadoop]$ hdfs haadmin -getServiceState nn2
19/06/14 17:30:08 DEBUG ipc.Client: Failed to connect to server: hadoop2/192.168.19.73:8040: retries get failed due to exceeded maximum allowed retries number: 1
java.net.ConnectException: Connection refused
然后启动nn2 的namenode在查看变为standby
[hadoop@hadoop2 ~]$ /opt/hadoop/sbin/hadoop-daemon.sh start namenode
starting namenode, logging to /opt/hadoop/logs/hadoop-hadoop-namenode-hadoop2.out
19/06/14 17:31:03 DEBUG ipc.Client: IPC Client (413601558) connection to hadoop2/192.168.19.73:8040 from hadoop got value #0
19/06/14 17:31:03 DEBUG ipc.ProtobufRpcEngine: Call: getServiceStatus took 106ms
standby

yhqtxy1314 · 发表于 2019-6-14 17:45

NameNode健康检查：
[hadoop@hadoop1 hadoop]$ hdfs haadmin -checkHealth nn1
将其中一台NameNode给kill后，查看健康状态：
[hadoop@hadoop2 ~]$ jps
3537 DFSZKFailoverController
3748 NameNode
3878 DataNode
3606 JournalNode
3336 QuorumPeerMain
4411 Jps
4012 ResourceManager
[hadoop@hadoop2 ~]$ kill 3748
[hadoop@hadoop1 hadoop]$ hdfs haadmin -checkHealth nn1
[hadoop@hadoop1 hadoop]$ hdfs haadmin -checkHealth nn2
查看所有的DataNode列表
[hadoop@hadoop1 hadoop]$ hdfs dfsadmin -report | more
查看正常DataNode列表
hdfs dfsadmin -report -live
查看异常DataNode列表：
hdfs dfsadmin -report -dead
获取指定DataNode信息(运行时间及版本等):
hdfs haadmin -checkHealth nn2

yhqtxy1314 · 发表于 2019-6-14 17:45

###YARN相关的命令
查看resourceManager状态的命令
[hadoop@hadoop1 hadoop]$ yarn rmadmin -getServiceState rm1
19/06/14 17:33:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
active
[hadoop@hadoop1 hadoop]$ yarn rmadmin -getServiceState rm2
19/06/14 17:33:52 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
standby