org.apache.hadoop.fs.blockmissingeexception

brjng4g3  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(380)

我在hadoop作业中得到以下异常。我无法找出这个异常的原因。你能帮我解决这个错误吗?为什么会出现这个错误?
异常堆栈跟踪:

org.apache.hadoop.fs.BlockMissingException: Could not obtain block: blk_6488469967470272993_1110 file=/media/ephemeral0/hadoop-root/5614861051_themeToProductsPipe_F832FB7243E64F41892F20744AF6A0B7/part-00054
    at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:3158)
    at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2940)
    at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:3107)
    at java.io.DataInputStream.readFully(DataInputStream.java:195)
    at java.io.DataInputStream.readFully(DataInputStream.java:169)
    at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1576)
    at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1554)
    at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1543)
    at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1538)
    at org.apache.hadoop.mapred.SequenceFileRecordReader.(SequenceFileRecordReader.java:47)
    at org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:64)
    at cascading.tap.hadoop.io.MultiInputFormat$1.operate(MultiInputFormat.java:253)
    at cascading.tap.hadoop.io.MultiInputFormat$1.operate(MultiInputFormat.java:248)
    at cascading.util.Util.retry(Util.java:762)
    at cascading.tap.hadoop.io.MultiInputFormat.getRecordReader(MultiInputFormat.java:247)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:346)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:312)
    at org.apache.hadoop.mapred.Child$3.run(Child.java:205)
rjzwgtxy

rjzwgtxy1#

该错误意味着客户端无法按时命中包含该块的数据节点。这可能是由于网络问题,或者是因为数据节点太忙而无法响应,或者是因为它们关闭了。
这个异常不足以跟踪根本原因。

相关问题