yarn应用程序在运行任何MapReduceJAR时提交后失败

t5zmwmid  于 2021-06-01  发布在  Hadoop
关注(0)|答案(1)|浏览(307)

以下是错误快照:

[hduser@secondary ~]$ yarn jar test_word_count.jar     com.test.wordc.WordCount /temp /test_output
18/10/11 16:13:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/10/11 16:13:02 INFO client.RMProxy: Connecting to ResourceManager at master/10.0.70.148:8032
18/10/11 16:13:02 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
18/10/11 16:13:03 INFO input.FileInputFormat: Total input paths to process : 1
18/10/11 16:13:03 INFO mapreduce.JobSubmitter: number of splits:1
18/10/11 16:13:04 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1539254507997_0001
18/10/11 16:13:04 INFO impl.YarnClientImpl: Submitted application application_1539254507997_0001
18/10/11 16:13:04 INFO mapreduce.Job: The url to track the job: http://secondary:8088/proxy/application_1539254507997_0001/
18/10/11 16:13:04 INFO mapreduce.Job: Running job: job_1539254507997_0001
18/10/11 16:13:05 INFO mapreduce.Job: Job job_1539254507997_0001 running in uber mode : false
18/10/11 16:13:05 INFO mapreduce.Job:  map 0% reduce 0%
18/10/11 16:13:05 INFO mapreduce.Job: Job job_1539254507997_0001 failed with state FAILED due to: Application application_1539254507997_0001 failed 2 times due to Error launching appattempt_1539254507997_0001_000002. Got exception: java.io.IOException: Failed on local exception: java.io.IOException: java.io.EOFException; Host Details : local host is: "secondary/10.0.70.149"; destination host is: "slave4":38102; 
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy32.startContainers(Unknown Source)
at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119)
at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.io.EOFException
at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:643)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:730)
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1521)
at org.apache.hadoop.ipc.Client.call(Client.java:1438)
... 9 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:367)
at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:553)
at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:368)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:722)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:718)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:717)
... 12 more
. Failing the application.
18/10/11 16:13:05 INFO mapreduce.Job: Counters: 0

这是/etc/hosts文件

[hduser@secondary ~]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.0.70.149 secondary 
10.0.70.148 master
10.0.70.143 slave1
10.0.70.144 slave2
10.0.70.145 slave3
10.0.70.146 slave4
10.0.70.147 slave5

所有的服务都很好。它是一个具有5个dn和1个nn的6节点群集。提交作业后,我得到一个上面列出的错误。nn hadoop版本是hadoop2.6.0,而dn中的hadoop版本是2.5.2。它是一个maven版本,hadoop版本2.6.0位于pom.xml中

6l7fqoea

6l7fqoea1#

通过在namemode iteslf上安装hadoop2.5.2修复了这个错误。似乎整个集群都需要相同的hadoop版本。

相关问题