将文件从linux本地文件系统复制到hdfs时出错

8qgya5xd  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(480)

它的单节点集群当我开始的时候,我没有发现任何问题,我也检查了所有守护进程的日志。
但是,当我将数据从本地linux文件系统复制到hdfs文件系统时,抛出了一个异常:;org.apache.hadoop.hdfs.dfsclient:datastreamer异常:java.lang.illegalargumentexception:不包含有效的host:port authority: 0:0:0:0:0:0:0:1:50010
请查看附件日志;

2014-07-08 15:56:28,342 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: 0:0:0:0:0:0:0:1:50010
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.createBlockOutputStream(DFSClient.java:3629)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3588)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

2014-07-08 15:56:28,342 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for blk_3686864267613252482_3024 bad datanode[0] nodes == null
2014-07-08 15:56:28,342 WARN org.apache.hadoop.hdfs.DFSClient: Could not get block locations. Source file "/tmp/hadoop-user1/mapred/system/jobtracker.info" - Aborting...
2014-07-08 15:56:28,342 WARN org.apache.hadoop.mapred.JobTracker: Writing to file hdfs://localhost:9000/tmp/hadoop-user1/mapred/system/jobtracker.info failed!
2014-07-08 15:56:28,343 WARN org.apache.hadoop.mapred.JobTracker: FileSystem is not ready yet!
2014-07-08 15:56:28,351 WARN org.apache.hadoop.mapred.JobTracker: Failed to initialize recovery manager. 
java.io.IOException: Could not get block locations. Source file "/tmp/hadoop-user1/mapred/system/jobtracker.info" - Aborting...
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:3270)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2200(DFSClient.java:2783)
    at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2987)
2014-07-08 15:56:29,352 WARN org.apache.hadoop.mapred.JobTracker: Retrying...
vsaztqbk

vsaztqbk1#

检查mapred-site.xml。它应该有有效的跟踪器条目。当然,主机和端口值应该根据您的系统进行更改。

<property>
  <name>mapred.job.tracker</name>
  <value>host:port</value>
</property>

相关问题