提前感谢您的帮助。我使用提供的hadoop示例运行一个yarn作业。作业永远不会完成，并停留在“接受”状态。查看正在打印的内容，似乎作业正在等待完成——客户机不断地探测作业状态。
作业示例（来自hadoop 2.6.0）：
spark-submit --master yarn-client --driver-memory 4g --executor-memory 2g --executor-cores 4 --class org.apache.spark.examples.SparkPi /home/john/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar 100输出：
....
....
disabled; ui acls disabled; users with view permissions: Set(john); users with modify permissions: Set(jogn)
16/07/27 17:36:09 INFO yarn.Client: Submitting application 1 to ResourceManager
16/07/27 17:36:09 INFO impl.YarnClientImpl: Submitted application application_1469665943738_0001
16/07/27 17:36:10 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:10 INFO yarn.Client:
client token: N/A
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1469666169333
final status: UNDEFINED
tracking URL: http://cpt-bdx021:8088/proxy/application_1469665943738_0001/
user: john
16/07/27 17:36:11 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:12 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:13 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:14 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:15 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:16 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:17 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:18 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:19 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:20 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:21 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
16/07/27 17:36:22 INFO yarn.Client: Application report for application_1469665943738_0001 (state: ACCEPTED)
...........
...........
...........更新（看起来作业已提交给resourcemanager——因此被“接受”，但resourcemanager“没有看到”任何节点或hadoop工作进程可以实际传递作业）：
$ jps
jps
12404 Jps
12211 NameNode
12315 DataNode
11743 ApplicationHistoryServer
11876 ResourceManager
11542 NodeManager
$ yarn node -list
16/07/27 23:07:53 INFO client.RMProxy: Connecting to ResourceManager at /192.168.0.5.55:8032
Total Nodes:0
Node-Id Node-State Node-Http-Address Number-of-Running-Containers更新（2）：我使用默认值 etc/container-executor.cfg file :
yarn.nodemanager.linux-container-executor.group=#configured value of yarn.nodemanager.linux-container-executor.group
banned.users=#comma separated list of users who can not run applications
min.user.id=1000#Prevent other super-users
allowed.system.users=##comma separated list of system users who CAN run applications另外，作为我的一方，我想提一下，我没有 hadoop 用户或hadoop`用户组。我使用的是登录系统时使用的默认帐户。如果有关系的话。谢谢！
更新（3）：nodemanager日志
org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at 192.168.0.5.55:8031
2016-07-28 00:23:26,083 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: []
2016-07-28 00:23:26,087 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[]
2016-07-28 00:23:26,233 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMContainerTokenSecretManager: Rolling master-key for container-tokens, got key with id -160570002
2016-07-28 00:23:26,236 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMTokenSecretManagerInNM: Rolling master-key for container-tokens, got key with id -1876215653
2016-07-28 00:23:26,237 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registered with ResourceManager as 192.168.0.5.55:53034 with total resource of <memory:8192, vCores:8>
2016-07-28 00:23:26,237 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests
2条答案
按热度按时间u5i3ibmn1#
最常见的可能性是，将作业发送到的队列没有您请求的可用资源。
典型问题可能是：
资源需求（内存和/或内核）。您要求它分配更多的内存/内核。这可能是因为集群的使用已经接近完全，或者您的设置不一致。有关此页的更多详细信息。
磁盘空间。检查节点空间，有一个运行状况检查可能会阻止您运行应用程序。
在多租户/多队列环境中，如果每个队列都有硬资源限制，则应用程序可能会遇到这些限制。您可能需要增加设置，或者在另一个具有更多资源的队列中进行测试。
bmp9r5qi2#
您的作业从未完成的原因是它从未进入运行状态（从接受状态）。有一个调度程序，负责调度哪些应用程序将获得资源，从而状态运行。
有两个调度器可用：公平调度器和容量调度器。您可以在hadoopYarn文档中找到详细信息。如果您能提供yarn-site.xml、capacity-scheduler.xml和fair-scheduler.xml文件，我会给您更好的帮助：）。