我很难建立一个多节点集群。我有一个运行ubuntu20.04的razer和一个运行osx catalina的imac。razer是主机名节点,razer和imac都被设置为datanodes(从工作者)。两台计算机都复制了ssh密钥,因此它们可以在没有任何密码的情况下进行ssh连接。但是,我在hadoop Jmeter 板上显示imac中的远程datanode时遇到了问题。我可以从razer现场看到datanode,我想这与我的远程机器mac无法连接到我在core-site.xml中设置为的hdfs有关hfds://hadoopmaster:9000.
RAZER = Hostname: Hadoopmaster
IMAC = Hostname: Hadoopslave
基于一些故障排除,我查看了imac中的datanode日志,发现它拒绝连接到端口9000上的hadoopmaster。
2020-06-01 13:44:33,193 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
hadoopmaster/192.168.1.191:8070. Already tried 6 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:35,550 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
hadoopmaster/192.168.1.191:8070. Already tried 7 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:36,574 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
hadoopmaster/192.168.1.191:8070. Already tried 8 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:37,597 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
hadoopmaster/192.168.1.191:8070. Already tried 9 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:37,619 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem
connecting to server: hadoopmaster/192.168.1.191:8070
2020-06-01 13:44:44,660 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
hadoopmaster/192.168.1.191:8070. Already tried 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2020-06-01 13:44:45,534 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: RECEIVED
SIGNAL 15: SIGTERM
2020-06-01 13:44:45,537 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
以下是我的设置:
hdfs-site.xml文件
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
core-site.xml文件
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/Cellar/hadoop/hdfs/tmp</value>
<description>A base for other temporary directories</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoopmaster:8070</value>
</property>
</configuration>
所以我认为连接到我机器上的端口9000有问题。因此,我的下一步是在终端命令窗口中测试ssh连接:
IMAC Command: ssh username@hadoopmaster -p 9000
结果:拒绝连接
所以我的下一步是在我的剃须刀机器上执行ssh命令:
Razer Command: ssh hadoopmaster -p 9000
结果:拒绝连接
所以我试着在我的剃须刀修改ufw防火墙打开端口9000,任何hadoopmaster,所有端口,但仍然没有运气。
请帮助我让我的远程机器imac连接到razer上的端口9000,这样我就可以在我的网络中创建hadoop集群,并在 Jmeter 板上将远程从机视为实时数据节点。
暂无答案!
目前还没有任何答案,快来回答吧!