我在一台独立的机器上运行hive。hadoop正在伪分布式模式下运行。我正在运行连接两个表的配置单元查询(一个表有7m条记录,另一个有51m条记录,每个记录包含8列)。经过一段时间的处理后,mapper达到零百分比,然后偶尔继续打印零。你能帮我解决这个问题吗。
参考以下日志。
2016-04-12 22:52:58,469 Stage-1 map = 71%, reduce = 1%
2016-04-12 22:53:00,517 Stage-1 map = 72%, reduce = 1%
2016-04-12 22:53:02,560 Stage-1 map = 73%, reduce = 1%
2016-04-12 22:53:09,740 Stage-1 map = 74%, reduce = 1%
2016-04-12 22:53:11,796 Stage-1 map = 75%, reduce = 1%
2016-04-12 22:53:13,842 Stage-1 map = 76%, reduce = 1%
2016-04-12 22:53:21,037 Stage-1 map = 77%, reduce = 1%
2016-04-12 22:53:24,114 Stage-1 map = 78%, reduce = 1%
2016-04-12 22:53:26,156 Stage-1 map = 79%, reduce = 1%
2016-04-12 22:53:35,433 Stage-1 map = 81%, reduce = 1%
2016-04-12 22:53:38,507 Stage-1 map = 82%, reduce = 1%
2016-04-12 22:53:45,725 Stage-1 map = 82%, reduce = 0%
2016-04-12 22:53:49,925 Stage-1 map = 0%, reduce = 0%
2016-04-12 22:54:50,236 Stage-1 map = 0%, reduce = 0%
2016-04-12 22:55:50,546 Stage-1 map = 0%, reduce = 0%
2016-04-12 22:56:50,863 Stage-1 map = 0%, reduce = 0%
2016-04-12 22:57:51,128 Stage-1 map = 0%, reduce = 0%
2016-04-12 22:58:51,352 Stage-1 map = 0%, reduce = 0%
2016-04-12 22:59:51,612 Stage-1 map = 0%, reduce = 0%
2016-04-12 23:00:51,886 Stage-1 map = 0%, reduce = 0%
2016-04-12 23:01:52,131 Stage-1 map = 0%, reduce = 0%
我在追踪器里确认了状态。状态显示两次尝试,一次尝试失败,诊断消息如下。
AM Container for appattempt_1460481465127_0001_000001 exited with exitCode: -100
For more detailed output, check application tracking page:http://localhost:8088/cluster/app/application_1460481465127_0001Then, click on links to logs of each attempt.
Diagnostics: Container released on a *lost* nodeFailing this attempt
提前谢谢。
1条答案
按热度按时间frebpwbc1#
这个问题似乎是由map端的堆空间引起的。
尝试通过执行以下操作来增加Map任务堆大小:
在
mapred-site.xml
(尝试调整以下值以与您的用例匹配):