服务器a:192.168.96.130,操作系统:centos7.x服务器b:localhost,我的电脑,操作系统:windows10
我在servera上安装hadoop3.1.2,并编写java应用程序将数据写入servera上的hdfs。当java应用程序部署在服务器a上时,它可以成功地将包含内容的文件写入hdfs。当java应用程序部署在服务器b上时,它可以将文件写入hdfs,但不能将内容写入文件中。始终获取错误:
2020-03-18 20:56:43,460 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 9000, call Call#4 Retry#0 org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.96.1:53463
java.io.IOException: File /canal/canal_1/canal_1-2020-3-19-4.txt could only be written to 0 of the 1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2121)
at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:295)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2702)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:875)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:561)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:872)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:818)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2678)
下面是我的java应用程序代码:
Configuration conf = new Configuration();
FileSystem fs= FileSystem.get(new URI("hdfs://192.168.96.1:9000/"),conf,"root");
FSDataOutputStream out = fs.create(new Path("/canal/canal_1/canal_1-2020-03-10.txt"));
out.writeBytes("15, kevin15, 2020.3.15");
out.flush();
out.close();
fs.close();
如何解决这个问题?
1条答案
按热度按时间b4wnujal1#
我想你应该先检查一下你的集群是否健康。
http://namenode1:50070
那么可能您还没有关闭iptables,所以当您在server2上时无法telnet端口。您可以尝试执行命令telnet SERVER1_IP 50020
在server1和server2上进行检查。