apache hadoop多节点群集失败,未显示远程datanode

dw1jzc5e  于 2021-05-31  发布在  Hadoop
关注(0)|答案(0)|浏览(323)

我很难建立一个多节点集群。我有一个运行ubuntu20.04的razer和一个运行osx catalina的imac。razer是主机名节点,razer和imac都被设置为datanodes(从工作者)。两台计算机都复制了ssh密钥,因此它们可以在没有任何密码的情况下进行ssh连接。但是,我在hadoop Jmeter 板上显示imac中的远程datanode时遇到了问题。我可以从razer现场看到datanode,我想这与我的远程机器mac无法连接到我在core-site.xml中设置为的hdfs有关hfds://hadoopmaster:9000.

RAZER = Hostname: Hadoopmaster 
IMAC = Hostname: Hadoopslave

基于一些故障排除,我查看了imac中的datanode日志,发现它拒绝连接到端口9000上的hadoopmaster。

2020-06-01 13:44:33,193 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 
hadoopmaster/192.168.1.191:8070. Already tried 6 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:35,550 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 
hadoopmaster/192.168.1.191:8070. Already tried 7 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:36,574 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 
hadoopmaster/192.168.1.191:8070. Already tried 8 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:37,597 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 
hadoopmaster/192.168.1.191:8070. Already tried 9 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:37,619 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem 
connecting to server: hadoopmaster/192.168.1.191:8070
2020-06-01 13:44:44,660 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 
hadoopmaster/192.168.1.191:8070. Already tried 0 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:45,534 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: RECEIVED 
SIGNAL 15: SIGTERM
2020-06-01 13:44:45,537 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

以下是我的设置:
hdfs-site.xml文件

<configuration>
<property>
        <name>dfs.replication</name>
        <value>1</value>
</property>
</configuration>

core-site.xml文件

<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/Cellar/hadoop/hdfs/tmp</value>
<description>A base for other temporary directories</description>             
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoopmaster:8070</value>
</property>
</configuration>

所以我认为连接到我机器上的端口9000有问题。因此,我的下一步是在终端命令窗口中测试ssh连接:

IMAC Command: ssh username@hadoopmaster -p 9000

结果:拒绝连接
所以我的下一步是在我的剃须刀机器上执行ssh命令:

Razer Command: ssh hadoopmaster -p 9000

结果:拒绝连接
所以我试着在我的剃须刀修改ufw防火墙打开端口9000,任何hadoopmaster,所有端口,但仍然没有运气。
请帮助我让我的远程机器imac连接到razer上的端口9000,这样我就可以在我的网络中创建hadoop集群,并在 Jmeter 板上将远程从机视为实时数据节点。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题