hadoop集群设置：数据节点在从机上不工作

tez616oj 于 2021-06-03 发布在 Hadoop

关注(0)|答案(3)|浏览(378)

我试图在两台机器上运行hadoop。下面提到的是我的core-site.xml文件。
10.0.x.y是我的主机，10.0.x.y1是从机。

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://10.0.x.y:54310</value>
    </property>
</configuration>

下面提到的是我的hdfs-site.xml文件

<configuration>
        <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.name.dir</name>
        <value>/data/hadoop-namenode-data</value>
    </property>
    <property>
        <name>dfs.data.dir</name>
        <value>/data/hadoop-datanode-data</value>
    </property>

</configuration>

对于机器10.0.x.y上的10.0.x.y start-dfs.sh---my name node和datanode上的命令，10.0.x.y运行正常，但namenode.log文件上的错误消息是

2014-07-02 15:49:08,628 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode RPC up at: TPMOPW388/10.0.7.111:54310
2014-07-02 15:49:08,628 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state
2014-07-02 15:49:11,268 WARN org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Unresolved datanode registration from 10.0.40.18
2014-07-02 15:49:11,268 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0)
2014-07-02 15:49:11,268 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 54310, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.registerDatanode from 10.0.40.18:57523 Call#1 Retry#0: error: org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0)
org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0)
        at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:739)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3929)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:948)
        at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90)
        at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:24079)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)

感谢您的帮助。

hadoop hdfs

来源：https://stackoverflow.com/questions/24528692/hadoop-cluster-setup-data-node-not-working-on-slave-machine

3条答案

按热度按时间

eaf3rand1#

可能与主机名配置有关。检查 /etc/hosts 文件，并确保在每台计算机中都有这两台计算机的ip地址和主机名。

赞(0）回复(0）举报 2021-06-04

xdnvmnnf2#

根据这条线：

15:49:11,268 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0)

这可能是由于您的名称节点无法通过ssh访问根帐户为的数据节点。
所以，
确保您安装了ssh并配置了私钥，以便无需密码即可登录！

赞(0）回复(0）举报 2021-06-04

nom7f22z3#

在hdfs-site.xml中使用value=“number of nodes in your system”，并检查/etc/hosts。

赞(0）回复(0）举报 2021-06-04