hadoop—编译成功,但提交作业失败后

7fyelxc5  于 2021-05-29  发布在  Hadoop
关注(0)|答案(2)|浏览(372)

我自己写计划 Hadoop2.6.0 继承abstractyarnscheduler。我编译成功,但在hadoop中提交作业时,rm崩溃了。
这是登录主节点

2015-07-19 13:31:59,931 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1437327062733_0001 State change from NEW to NEW_SAVING

**2015-07-19 13:31:59,932 FATAL org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread**

java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/proto/YarnServerResourceManagerServiceProtos$ApplicationStateDataProtoOrBuilder
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
    at org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java:56)
    at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.records.ApplicationStateData.newInstance(ApplicationStateData.java:43)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.records.ApplicationStateData.newInstance(ApplicationStateData.java:56)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StoreAppTransition.transition(RMStateStore.java:131)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$StoreAppTransition.transition(RMStateStore.java:1)
    at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
    at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
    at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
    at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:787)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:839)
    at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:1)
    at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
    at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.proto.YarnServerResourceManagerServiceProtos$ApplicationStateDataProtoOrBuilder
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 20 more
2015-07-19 13:31:59,934 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..
2015-07-19 13:31:59,937 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
2015-07-19 13:31:59,938 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:8088
2015-07-19 13:31:59,938 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
2015-07-19 13:31:59,938 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
2015-07-19 13:31:59,944 WARN org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8032, call org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport from 129.10.58.155:50992 Call#42 Retry#0
java.lang.NoSuchMethodError: org.apache.hadoop.yarn.server.utils.BuilderUtils.newApplicationResourceUsageReport(IILorg/apache/hadoop/yarn/api/records/Resource;Lorg/apache/hadoop/yarn/api/records/Resource;Lorg/apache/hadoop/yarn/api/records/Resource;)Lorg/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport;
    at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.<clinit>(RMServerUtils.java:237)
    at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.createAndGetApplicationReport(RMAppImpl.java:520)
    at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:296)
    at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:170)
    at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:401)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
2015-07-19 13:32:00,039 INFO org.apache.hadoop.ipc.Server: Stopping server on 8032
2015-07-19 13:32:00,040 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8032
2015-07-19 13:32:00,040 INFO org.apache.hadoop.ipc.Server: Stopping server on 8033
2015-07-19 13:32:00,041 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8033
2015-07-19 13:32:00,041 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2015-07-19 13:32:00,041 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
2015-07-19 13:32:00,041 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2015-07-19 13:32:00,042 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2015-07-19 13:32:00,042 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2015-07-19 13:32:00,042 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2015-07-19 13:32:00,042 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2015-07-19 13:32:01,042 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:02,043 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:03,043 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:04,043 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:05,044 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:06,044 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:07,044 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
2015-07-19 13:32:08,044 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Waiting for AsyncDispatcher to drain.
wrrgggsh

wrrgggsh1#

我想可能是一些用户会话/权限问题,我在我的独立示例(ubuntudesktop16lte、jdk1.8.92和hadoop2.7.2)中遇到了相同的错误。如果我重新启动我的机器并重新启动,它会再次正常工作。如果我只是重新登录,重新启动守护进程,在同一终端会话上重新提交作业,它会不断弹出相同的错误。
你能解决这个问题吗?
在我的机器上复制的步骤是:
(1) 启动终端,使用命令su hduser(在终端上)登录到hadoop专用用户hduser(sudo用户)
(2) 使用以下命令启动hadoop守护进程:start-dfs.sh和start-yarn.sh
(3) 我可以用jps命令查看所有进程。
(4) 很少有乔布斯先生成功地完成了。我提交同样的工作后,约10-15分钟。
(5) hduser用户会话被抛出并在常规桌面用户会话中着陆。

2admgd59

2admgd592#

根据错误信息,系统找不到类 org.apache.hadoop.yarn.proto.YarnServerResourceManagerServiceProtos$ApplicationStateDataProtoOrBuilder ,是否将类添加到类路径?

相关问题