简单hadoop map reduce出错

pdsfdshx  于 2021-06-04  发布在  Hadoop
关注(0)|答案(4)|浏览(357)

我试着运行一个hadoopMap来减少字数问题。据我所知，我正确地设置了hdfs和所有内容。
当我执行的时候，
我在尝试运行一个简单的字数计算问题时遇到这个错误。我是hadoop新手。任何帮助如何解决这将不胜感激。

13/06/13 20:21:17 INFO input.FileInputFormat: Total input paths to process : 4
13/06/13 20:21:17 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/06/13 20:21:17 WARN snappy.LoadSnappy: Snappy native library not loaded
13/06/13 20:21:17 INFO mapred.JobClient: Running job: job_201306131940_0007
13/06/13 20:21:18 INFO mapred.JobClient:  map 0% reduce 0%
13/06/13 20:21:21 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_0 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_0 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:21 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_0&filter=stdout
13/06/13 20:21:21 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_0&filter=stderr
13/06/13 20:21:24 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_1, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_1 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_1 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:25 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_1&filter=stdout
13/06/13 20:21:25 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_1&filter=stderr
13/06/13 20:21:28 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000005_2, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_2 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000005_2 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:28 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_2&filter=stdout
13/06/13 20:21:28 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000005_2&filter=stderr
13/06/13 20:21:34 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_0 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_0 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:34 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_0&filter=stdout
13/06/13 20:21:34 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_0&filter=stderr
13/06/13 20:21:37 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_1, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_1 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_1 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

13/06/13 20:21:37 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_1&filter=stdout
13/06/13 20:21:37 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_1&filter=stderr
13/06/13 20:21:40 INFO mapred.JobClient: Task Id : attempt_201306131940_0007_m_000004_2, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/vraju/Desktop/hadoop/libexec/../logs/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_2 to /app/hadoop/tmp/mapred/local/userlogs/job_201306131940_0007/attempt_201306131940_0007_m_000004_2 failed.
at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)
13/06/13 20:21:40 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_2&filter=stdout
13/06/13 20:21:40 WARN mapred.JobClient: Error reading task outputhttp://ubuntu:50060/tasklog?plaintext=true&attemptid=attempt_201306131940_0007_m_000004_2&filter=stderr
13/06/13 20:21:43 INFO mapred.JobClient: Job complete: job_201306131940_0007
13/06/13 20:21:43 INFO mapred.JobClient: Counters: 4
13/06/13 20:21:43 INFO mapred.JobClient:   Job Counters 
13/06/13 20:21:43 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=24100
13/06/13 20:21:43 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/06/13 20:21:43 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/06/13 20:21:43 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
hadoopubuntumapreduceword-count

4条答案

按热度按时间
z9gpfhce

z9gpfhce1#

我有这个问题，但它是一个包名问题。
我在用maven编译。在pom.xml中，我们有 <groupId>GROUPNAME</groupId> 我的所有包都必须在groupname中，例如groupname.newpackage.classname
然后，它就起作用了。

赞(0）分享  回复(0）举报  2021-06-04
hm2xizp9

hm2xizp92#

我得到了同样的错误，而试图做同样的事情-执行简单的文本字计数。在我的例子中，我解决了更改hadoop所依赖的jdk版本的问题。我正在使用hadoop1.2.1，我在网上发现它的首选java版本是1.6

赞(0）分享  回复(0）举报  2021-06-04
nzrxty8p

nzrxty8p3#

我花了一整天的时间来解决这个问题，直到我在网上找到一个解决方案，在virtualbox的共享目录中不支持创建symlink的操作。如果将hadoop目录移动到/usr/local可以解决您的问题，请尝试。

赞(0）分享  回复(0）举报  2021-06-04
vtwuwzda

vtwuwzda4#

当你得到

java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)

似乎分配给任务跟踪器的内存多于节点的实际内存。
你可以在这里得到更多

赞(0）分享  回复(0）举报  2021-06-04
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com