我向hadoop提交了一个mapreduce作业，并在屏幕上查看进度报告。对于map任务和reduce任务，进度报告应该是单调递增的（例如0%、10%、25%、60%、78%、95%和100%）。但事实上，所报告的进展并不是单调地增加：
14/01/21 11:05:37 INFO mapred.JobClient: Running job: job_201401201555_0036
14/01/21 11:05:38 INFO mapred.JobClient: map 0% reduce 0%
14/01/21 11:06:07 INFO mapred.JobClient: map 11% reduce 0%
14/01/21 11:06:10 INFO mapred.JobClient: map 0% reduce 0%
14/01/21 11:06:19 INFO mapred.JobClient: map 9% reduce 0%
14/01/21 11:06:22 INFO mapred.JobClient: map 22% reduce 0%
14/01/21 11:06:25 INFO mapred.JobClient: map 31% reduce 0%
14/01/21 11:06:28 INFO mapred.JobClient: map 39% reduce 0%
14/01/21 11:06:29 INFO mapred.JobClient: map 53% reduce 0%
14/01/21 11:06:30 INFO mapred.JobClient: map 57% reduce 0%
14/01/21 11:06:32 INFO mapred.JobClient: map 50% reduce 0%
14/01/21 11:06:33 INFO mapred.JobClient: map 55% reduce 0%
14/01/21 11:06:34 INFO mapred.JobClient: map 43% reduce 0%
14/01/21 11:06:35 INFO mapred.JobClient: map 48% reduce 0%
14/01/21 11:06:36 INFO mapred.JobClient: map 40% reduce 0%
14/01/21 11:06:38 INFO mapred.JobClient: map 30% reduce 0%
14/01/21 11:06:40 INFO mapred.JobClient: map 40% reduce 0%
14/01/21 11:06:41 INFO mapred.JobClient: map 49% reduce 0%
14/01/21 11:06:43 INFO mapred.JobClient: map 57% reduce 0%
14/01/21 11:06:44 INFO mapred.JobClient: map 70% reduce 0%
14/01/21 11:06:46 INFO mapred.JobClient: map 73% reduce 0%
14/01/21 11:06:47 INFO mapred.JobClient: map 82% reduce 0%
14/01/21 11:06:48 INFO mapred.JobClient: map 93% reduce 0%
14/01/21 11:06:50 INFO mapred.JobClient: map 94% reduce 0%
14/01/21 11:06:52 INFO mapred.JobClient: map 95% reduce 0%
14/01/21 11:06:53 INFO mapred.JobClient: map 96% reduce 0%
14/01/21 11:06:56 INFO mapred.JobClient: map 98% reduce 0%
14/01/21 11:06:59 INFO mapred.JobClient: map 99% reduce 0%
14/01/21 11:07:00 INFO mapred.JobClient: map 100% reduce 0%
14/01/21 11:07:19 INFO mapred.JobClient: map 100% reduce 4%
14/01/21 11:07:22 INFO mapred.JobClient: map 100% reduce 8%
14/01/21 11:07:25 INFO mapred.JobClient: map 100% reduce 66%
14/01/21 11:07:29 INFO mapred.JobClient: map 100% reduce 67%
14/01/21 11:07:32 INFO mapred.JobClient: map 100% reduce 68%
14/01/21 11:07:35 INFO mapred.JobClient: map 100% reduce 69%
14/01/21 11:07:41 INFO mapred.JobClient: map 100% reduce 70%
14/01/21 11:07:47 INFO mapred.JobClient: map 100% reduce 71%
14/01/21 11:07:53 INFO mapred.JobClient: map 100% reduce 72%
14/01/21 11:07:59 INFO mapred.JobClient: map 100% reduce 73%
14/01/21 11:08:02 INFO mapred.JobClient: map 100% reduce 100%
14/01/21 11:08:03 INFO mapred.JobClient: Job complete: job_201401201555_0036进度由所有输入拆分中已处理的拆分百分比表示。但为什么进展报告不是单调地增加？
1条答案
按热度按时间lpwwtiir1#
检查TaskTracker和jobtracker的日志。map阶段是否有故障？如果一台机器不能执行一项任务，或者主人再也不能到达它，那么该任务将由另一台机器从头开始重新执行；因此，进步不是单调增加的。