reducer总是失败,map总是成功

nnvyjq4y  于 2021-06-02  发布在  Hadoop
关注(0)|答案(0)|浏览(205)

我正在1gb的文本文件上运行简单的wordcount作业。我的集群有8个datanode和1个namenode,每个节点的存储容量为3gb。
当我运行wordcount时,我可以看到map总是成功的,reducer抛出了一个错误并失败了。请查看下面的错误消息。

14/10/05 15:42:02 INFO mapred.JobClient:  map 100% reduce 31%
14/10/05 15:42:07 INFO mapred.JobClient: Task Id : attempt_201410051534_0002_m_000016_0, Status :  FAILED
FSError: java.io.IOException: No space left on device
14/10/05 15:42:14 INFO mapred.JobClient: Task Id : attempt_201410051534_0002_r_000000_0, Status : FAILED
java.io.IOException: Task: attempt_201410051534_0002_r_000000_0 - The reduce copier failed
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local     directory for     file:/app/hadoop/tmp/mapred/local/taskTracker/hduser/jobcache/job_201410051534_0002/attempt_201410051534_0002_r_000000_0/output/map_18.out

你能告诉我怎么解决这个问题吗?
谢谢纳瓦兹

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题