我一直在寻找解决办法,但一无所获。也许这里有人遇到了这样的问题,有了一些想法。
基本上,我和hadoop一起工作,试图完成一些mapreduce任务。一个主节点和两个从节点。配置似乎还可以,因为我可以用ssh从每个节点连接到每个节点。我已经准备好了 /etc/hosts
以及hadoop的配置,例如: mapred-site.xml
, hdfs-site.xml
.
任务执行后,一切似乎都好了。没有错误或警告,但当我 grep WARN
在日志文件中,有一些条目让我担心。
在 tasktracker-master.log
我发现:
2013-06-03 14:14:47,525 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:15:00,680 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 14:17:44,352 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:17:59,182 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 14:23:49,419 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:23:55,156 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 14:27:40,619 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:27:46,382 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 14:34:15,498 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:34:22,103 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 14:36:38,551 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201306031434_0002 being deleted.
2013-06-03 14:37:17,724 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201306031434_0003 being deleted.
2013-06-03 14:59:42,395 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:59:48,158 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.`
同样的事情也发生在 slave1/slave2
:
tasktracker-slave1.log:
2013-06-03 14:59:34,439 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:59:45,954 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 15:01:29,380 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201306031459_0001 being deleted.
2013-06-03 15:03:14,848 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201306031459_0003 being deleted.
tasktracker-slave2.log:
2013-06-03 14:59:37,642 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-06-03 14:59:45,805 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled.
2013-06-03 15:02:29,489 WARN org.apache.hadoop.mapred.TaskTracker: Unknown job job_201306031459_0002 being deleted.
有人见过这些警告,知道它们是什么意思吗?
还有一件事,我怎样才能知道任务是否真的在所有假定的节点上执行 master, slave1, slave2
? 这些警告意味着 slave1/slave2
你不工作吗?
我希望我已经清楚地描述了我的问题。谢谢你的帮助。
暂无答案!
目前还没有任何答案,快来回答吧!