hadoop陷入“运行作业”

6l7fqoea  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(352)

我想从文档中运行hadoop字数计算程序。然而,该计划仍停留在 running job ```
16/09/02 10:51:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/09/02 10:51:13 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
16/09/02 10:51:13 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/09/02 10:51:14 INFO input.FileInputFormat: Total input paths to process : 1
16/09/02 10:51:14 INFO mapreduce.JobSubmitter: number of splits:2
16/09/02 10:51:14 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1472783047951_0003
16/09/02 10:51:14 INFO impl.YarnClientImpl: Submitted application application_1472783047951_0003
16/09/02 10:51:14 INFO mapreduce.Job: The url to track the job: http://hadoop-master:8088/proxy/application_1472783047951_0003/
16/09/02 10:51:14 INFO mapreduce.Job: Running job: job_1472783047951_0003

展示http://hadoop-master:8088/代理/应用程序\u 1472783047951 \u 0003/
![](https://i.stack.imgur.com/7d1gb.png)
它在上运行appmasterhttp://hadoop-slave2:8042,给我看看
![](https://i.stack.imgur.com/leChM.png)
但是,由于它在wordcount上有问题,所以它也一直存在 `Hive` ```
hive (default)> select a, b, count(1) as cnt from newtb group by a, b;
Query ID = hadoop_20160902110124_d2b2680b-c493-4986-aa84-f65794bfd8e4
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1472783047951_0004, Tracking URL = http://hadoop-master:8088/proxy/application_1472783047951_0004/
Kill Command = /opt/hadoop-2.6.4/bin/hadoop job  -kill job_1472783047951_0004

select*没有问题。

hive (default)> select * from newtb;
OK
1   2   3
1   3   4
2   3   4
5   6   7
8   9   0
1   8   3
Time taken: 0.101 seconds, Fetched: 6 row(s)

所以,我认为mapreduce有问题。有足够的磁盘和内存。那么,如何解决呢?

vbopmzt1

vbopmzt11#

出现问题是因为应用程序主机无法启动容器并运行作业。首先尝试重新启动系统,如果没有更改,则必须更改yarn-site.xml和mapred-site.xml中的内存分配。使用基本内存设置。
使用以下链接http://www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide/#yarn-配置1
在这里
在这里

相关问题