hadoop的resourcemanager在通过ansible启动时无法启动

q5iwbnjs  于 2021-07-13  发布在  Hadoop
关注(0)|答案(0)|浏览(582)

我无法开始Yarn（通过 ./start-yarn.sh )在伪分布式模式下通过ansible执行时。
这是我的剧本的一部分，从Yarn开始（哪里 hadoop_home 是 /home/hdoop/hadoop ):

  1. - name: hdoop > Start YARN
  2.   become: true
  3.   become_user: hdoop
  4.   environment:
  5.     HADOOP_HOME: "{{ hadoop_home }}"
  6.     HADOOP_HDFS_HOME: "{{ hadoop_home }}"
  7.     HADOOP_CONF_DIR: "{{ hadoop_home }}/etc/hadoop"
  8.     HADOOP_YARN_HOME: "{{ hadoop_home }}"
  9.   shell: 
  10.     cmd: "{{ hadoop_home }}/sbin/start-yarn.sh"
  11.     executable: /bin/bash

这是我的 yarn-site.xml 文件：

  1. <configuration>
  2. <property>
  3.     <name>yarn.nodemanager.aux-services</name>
  4.     <value>mapreduce_shuffle</value>
  5. </property>
  6. <property>
  7.   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  8.   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  9. </property>
  10. <property>
  11.   <name>yarn.resourcemanager.hostname</name>
  12.   <value>0.0.0.0</value>
  13. </property>
  14. <property>
  15.   <name>yarn.resourcemanager.webapp.address</name>
  16.   <value>0.0.0.0:8088</value>
  17. </property>
  18. <property>
  19.   <name>yarn.acl.enable</name>
  20.   <value>0</value>
  21. </property>
  22. <property>
  23.   <name>yarn.nodemanager.env-whitelist</name>   
  24.   <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
  25. </property>
  26. </configuration>

这将导致 NodeManager 开始但不是开始 ResourceManager . 我得到以下错误：

  1. STARTUP_MSG:   build = Unknown -r 7a3bc90b05f257c8ace2f76d74264906f0f7a932; compiled by 'hexiaoqiao' on 2021-01-03T09:26Z
  2. STARTUP_MSG:   java = 1.8.0_282
  4. ************************************************************/
  6. 2021-02-20 18:27:03,040 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: registered UNIX signal handlers for [TERM, HUP, INT]
  7. 2021-02-20 18:27:03,632 INFO org.apache.hadoop.conf.Configuration: found resource core-site.xml at file:/home/hdoop/hadoop/etc/hadoop/core-site.xml
  8. 2021-02-20 18:27:03,736 INFO org.apache.hadoop.conf.Configuration: resource-types.xml not found
  9. 2021-02-20 18:27:03,736 INFO org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'resource-types.xml'.
  10. 2021-02-20 18:27:03,799 INFO org.apache.hadoop.conf.Configuration: found resource yarn-site.xml at file:/home/hdoop/hadoop/etc/hadoop/yarn-site.xml
  11. 2021-02-20 18:27:03,808 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
  12. 2021-02-20 18:27:03,897 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
  13. 2021-02-20 18:27:03,905 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTokenSecretManager: ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
  14. 2021-02-20 18:27:03,917 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms
  15. 2021-02-20 18:27:03,965 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler
  16. 2021-02-20 18:27:03,970 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager
  17. 2021-02-20 18:27:03,970 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
  18. 2021-02-20 18:27:04,002 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.event.EventDispatcher
  19. 2021-02-20 18:27:04,003 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher
  20. 2021-02-20 18:27:04,005 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher
  21. 2021-02-20 18:27:04,005 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher
  22. 2021-02-20 18:27:04,081 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
  23. 2021-02-20 18:27:04,231 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
  24. 2021-02-20 18:27:04,231 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system started
  25. 2021-02-20 18:27:04,245 INFO org.apache.hadoop.yarn.security.YarnAuthorizationProvider: org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instantiated.
  26. 2021-02-20 18:27:04,248 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager
  27. 2021-02-20 18:27:04,254 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher
  28. 2021-02-20 18:27:04,255 INFO org.apache.hadoop.yarn.server.resourcemanager.RMNMInfo: Registered RMNMInfo MBean
  29. 2021-02-20 18:27:04,256 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.monitor.RMAppLifetimeMonitor: Application lifelime monitor interval set to 3000 ms.
  30. 2021-02-20 18:27:04,261 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.MultiNodeSortingManager: Initializing NodeSortingService=MultiNodeSortingManager
  31. 2021-02-20 18:27:04,262 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list
  32. 2021-02-20 18:27:04,285 INFO org.apache.hadoop.conf.Configuration: found resource capacity-scheduler.xml at file:/home/hdoop/hadoop/etc/hadoop/capacity-scheduler.xml
  33. 2021-02-20 18:27:04,293 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler: Minimum allocation = <memory:1024, vCores:1>
  34. 2021-02-20 18:27:04,293 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler: Maximum allocation = <memory:8192, vCores:4>
  35. 2021-02-20 18:27:04,382 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root is undefined
  36. 2021-02-20 18:27:04,383 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root is undefined
  37. 2021-02-20 18:27:04,407 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: root, capacity=1.0, absoluteCapacity=1.0, maxCapacity=1.0, absoluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APP:*ADMINISTER_QUEUE:*, labels=*,
  38. , reservationsContinueLooking=true, orderingPolicy=utilization, priority=0
  39. 2021-02-20 18:27:04,407 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Initialized parent-queue root name=root, fullname=root
  40. 2021-02-20 18:27:04,440 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root.default is undefined
  41. 2021-02-20 18:27:04,440 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root.default is undefined
  42. 2021-02-20 18:27:04,444 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Initializing default
  43. capacity = 1.0 [= (float) configuredCapacity / 100 ]
  44. absoluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ]
  45. maxCapacity = 1.0 [= configuredMaxCapacity ]
  46. absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]
  47. effectiveMinResource=<memory:0, vCores:0>
  48.  , effectiveMaxResource=<memory:0, vCores:0>
  49. userLimit = 100 [= configuredUserLimit ]
  50. userLimitFactor = 1.0 [= configuredUserLimitFactor ]
  51. maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)]
  52. maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ]
  53. usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)]
  54. absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory]
  55. maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ]
  56. minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ]
  57. maximumAllocation = <memory:8192, vCores:4> [= configuredMaxAllocation ]
  58. numContainers = 0 [= currentNumContainers ]
  59. state = RUNNING [= configuredState ]
  60. acls = SUBMIT_APP:*ADMINISTER_QUEUE:* [= configuredAcls ]
  61. nodeLocalityDelay = 40
  62. rackLocalityAdditionalDelay = -1
  63. labels=*,
  64. reservationsContinueLooking = true
  65. preemptionDisabled = true
  66. defaultAppPriorityPerQueue = 0
  67. priority = 0
  68. maxLifetime = -1 seconds
  69. defaultLifetime = -1 seconds
  70. 2021-02-20 18:27:04,445 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0, effectiveMinResource=<memory:0, vCores:0> , effectiveMaxResource=<memory:0, vCores:0>
  71. 2021-02-20 18:27:04,445 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager: Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
  72. 2021-02-20 18:27:04,452 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
  73. 2021-02-20 18:27:04,452 INFO org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule: Initialized queue mappings, override: false
  74. 2021-02-20 18:27:04,452 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.WorkflowPriorityMappingsManager: Initialized workflow priority mappings, override: false
  75. 2021-02-20 18:27:04,453 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.MultiNodeSortingManager: MultiNode scheduling is 'false', and configured policies are 
  76. 2021-02-20 18:27:04,454 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<<memory:1024, vCores:1>>, maximumAllocation=<<memory:8192, vCores:4>>, asynchronousScheduling=false, asyncScheduleInterval=5ms,multiNodePlacementEnabled=false
  77. 2021-02-20 18:27:04,476 INFO org.apache.hadoop.conf.Configuration: dynamic-resources.xml not found
  78. 2021-02-20 18:27:04,480 INFO org.apache.hadoop.yarn.server.resourcemanager.AMSProcessingChain: Initializing AMS Processing chain. Root Processor=[org.apache.hadoop.yarn.server.resourcemanager.DefaultAMSProcessor].
  79. 2021-02-20 18:27:04,480 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: disabled placement handler will be used, all scheduling requests will be rejected.
  80. 2021-02-20 18:27:04,481 INFO org.apache.hadoop.yarn.server.resourcemanager.AMSProcessingChain: Adding [org.apache.hadoop.yarn.server.resourcemanager.scheduler.constraint.processor.DisabledPlacementProcessor] tp top of AMS Processing chain. 
  81. 2021-02-20 18:27:04,502 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: TimelineServicePublisher is not configured
  82. 2021-02-20 18:27:04,584 INFO org.eclipse.jetty.util.log: Logging initialized @2275ms to org.eclipse.jetty.util.log.Slf4jLog
  83. 2021-02-20 18:27:04,943 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
  84. 2021-02-20 18:27:04,980 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
  85. 2021-02-20 18:27:05,006 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
  86. 2021-02-20 18:27:05,018 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
  87. 2021-02-20 18:27:05,018 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
  88. 2021-02-20 18:27:05,018 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
  89. 2021-02-20 18:27:05,018 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
  90. 2021-02-20 18:27:05,018 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
  91. 2021-02-20 18:27:05,018 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
  92. 2021-02-20 18:27:05,021 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /cluster/*
  93. 2021-02-20 18:27:05,021 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
  94. 2021-02-20 18:27:05,021 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /app/*
  95. 2021-02-20 18:27:06,588 INFO org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
  96. 2021-02-20 18:27:06,625 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8088
  97. 2021-02-20 18:27:06,628 INFO org.eclipse.jetty.server.Server: jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
  98. 2021-02-20 18:27:06,716 INFO org.eclipse.jetty.server.session: DefaultSessionIdManager workerName=node0
  99. 2021-02-20 18:27:06,716 INFO org.eclipse.jetty.server.session: No SessionScavenger set, using defaults
  100. 2021-02-20 18:27:06,721 INFO org.eclipse.jetty.server.session: node0 Scavenging every 660000ms
  101. 2021-02-20 18:27:06,763 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
  102. 2021-02-20 18:27:06,798 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
  103. 2021-02-20 18:27:06,819 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
  104. 2021-02-20 18:27:06,836 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
  105. 2021-02-20 18:27:06,906 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@6cb6decd{logs,/logs,file:///home/hdoop/hadoop/logs/,AVAILABLE}
  106. 2021-02-20 18:27:06,915 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@30bce90b{static,/static,jar:file:/home/hdoop/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar!/webapps/static,AVAILABLE}
  107. 2021-02-20 18:27:06,935 WARN org.eclipse.jetty.webapp.WebInfConfiguration: Can't generate resourceBase as part of webapp tmp dir name: java.lang.NullPointerException
  108. 2021-02-20 18:27:07,382 INFO org.eclipse.jetty.util.TypeUtil: JVM Runtime does not support Modules
  109. 2021-02-20 18:27:12,831 ERROR org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: RECEIVED SIGNAL 1: SIGHUP
  110. 2021-02-20 18:27:15,009 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@4016ccc1{cluster,/,file:///tmp/jetty-0_0_0_0-8088-_-any-3811097377054804183.dir/webapp/,AVAILABLE}{jar:file:/home/hdoop/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar!/webapps/cluster}
  111. 2021-02-20 18:27:15,048 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@1bd39d3c{HTTP/1.1,[http/1.1]}{0.0.0.0:8088}
  112. 2021-02-20 18:27:15,048 INFO org.eclipse.jetty.server.Server: Started @12739ms
  113. 2021-02-20 18:27:15,048 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app cluster started at 8088
  114. 2021-02-20 18:27:15,277 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 100, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
  115. 2021-02-20 18:27:15,327 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8033
  116. 2021-02-20 18:27:15,771 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server
  117. 2021-02-20 18:27:15,773 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to active state
  118. 2021-02-20 18:27:15,772 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  119. 2021-02-20 18:27:15,773 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8033: starting
  120. 2021-02-20 18:27:15,818 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Updating AMRMToken
  121. 2021-02-20 18:27:15,821 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTokenSecretManager: Rolling master-key for container-tokens
  122. 2021-02-20 18:27:15,822 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Rolling master-key for nm-tokens
  123. 2021-02-20 18:27:15,822 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
  124. 2021-02-20 18:27:15,822 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager: storing master key with keyID 1
  125. 2021-02-20 18:27:15,822 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing RMDTMasterKey.
  126. 2021-02-20 18:27:15,833 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
  127. 2021-02-20 18:27:15,833 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
  128. 2021-02-20 18:27:15,834 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager: storing master key with keyID 2
  129. 2021-02-20 18:27:15,834 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing RMDTMasterKey.
  130. 2021-02-20 18:27:15,835 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler
  131. 2021-02-20 18:27:16,227 INFO org.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStore: Created store directory :file:/tmp/hadoop-yarn-hdoop/node-attribute
  132. 2021-02-20 18:27:16,297 INFO org.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStore: Finished write mirror at:file:/tmp/hadoop-yarn-hdoop/node-attribute/nodeattribute.mirror
  133. 2021-02-20 18:27:16,297 INFO org.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStore: Finished create editlog file at:file:/tmp/hadoop-yarn-hdoop/node-attribute/nodeattribute.editlog
  134. 2021-02-20 18:27:16,319 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesManagerImpl$ForwardingEventHandler
  135. 2021-02-20 18:27:16,322 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.MultiNodeSortingManager: Starting NodeSortingService=MultiNodeSortingManager
  136. 2021-02-20 18:27:16,363 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 5000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
  137. 2021-02-20 18:27:16,364 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8031
  138. 2021-02-20 18:27:16,384 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server
  139. 2021-02-20 18:27:16,394 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  140. 2021-02-20 18:27:16,434 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8031: starting
  141. 2021-02-20 18:27:16,455 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor
  142. 2021-02-20 18:27:16,496 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 5000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
  143. 2021-02-20 18:27:16,501 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8030
  144. 2021-02-20 18:27:16,528 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server
  145. 2021-02-20 18:27:16,529 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  146. 2021-02-20 18:27:16,529 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8030: starting
  147. 2021-02-20 18:27:16,721 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 5000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
  148. 2021-02-20 18:27:16,722 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8032
  149. 2021-02-20 18:27:16,725 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server
  150. 2021-02-20 18:27:16,742 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8032: starting
  151. 2021-02-20 18:27:16,763 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to active state
  152. 2021-02-20 18:27:16,763 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
  153. 2021-02-20 18:27:16,766 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
  154. 2021-02-20 18:27:16,769 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.w.WebAppContext@4016ccc1{cluster,/,null,UNAVAILABLE}{jar:file:/home/hdoop/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar!/webapps/cluster}
  155. 2021-02-20 18:27:16,785 INFO org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@1bd39d3c{HTTP/1.1,[http/1.1]}{0.0.0.0:8088}
  156. 2021-02-20 18:27:16,786 INFO org.eclipse.jetty.server.session: node0 Stopped scavenging
  157. 2021-02-20 18:27:16,791 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@30bce90b{static,/static,jar:file:/home/hdoop/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar!/webapps/static,UNAVAILABLE}
  158. 2021-02-20 18:27:16,791 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@6cb6decd{logs,/logs,file:///home/hdoop/hadoop/logs/,UNAVAILABLE}
  159. 2021-02-20 18:27:16,795 INFO org.apache.hadoop.ipc.Server: Stopping server on 8032
  160. 2021-02-20 18:27:16,811 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8032
  161. 2021-02-20 18:27:16,821 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  162. 2021-02-20 18:27:16,822 INFO org.apache.hadoop.ipc.Server: Stopping server on 8033
  163. 2021-02-20 18:27:16,822 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8033
  164. 2021-02-20 18:27:16,825 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
  165. 2021-02-20 18:27:16,825 WARN org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning.
  166. 2021-02-20 18:27:16,826 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  167. 2021-02-20 18:27:16,827 INFO org.apache.hadoop.ipc.Server: Stopping server on 8030
  168. 2021-02-20 18:27:16,839 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8030
  169. 2021-02-20 18:27:16,841 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  170. 2021-02-20 18:27:16,847 INFO org.apache.hadoop.ipc.Server: Stopping server on 8031
  171. 2021-02-20 18:27:16,862 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8031
  172. 2021-02-20 18:27:16,863 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
  173. 2021-02-20 18:27:16,863 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: NMLivelinessMonitor thread interrupted
  174. 2021-02-20 18:27:16,868 ERROR org.apache.hadoop.yarn.event.EventDispatcher: Returning, interrupted : java.lang.InterruptedException
  175. 2021-02-20 18:27:16,868 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager: org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager thread interrupted
  176. 2021-02-20 18:27:16,869 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, ignoring any new events.
  177. 2021-02-20 18:27:16,870 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, ignoring any new events.
  178. 2021-02-20 18:27:16,870 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: org.apache.hadoop.yarn.server.resourcemanager.rmapp.monitor.RMAppLifetimeMonitor thread interrupted
  179. 2021-02-20 18:27:16,873 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: AMLivelinessMonitor thread interrupted
  180. 2021-02-20 18:27:16,873 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: AMLivelinessMonitor thread interrupted
  181. 2021-02-20 18:27:16,874 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted
  182. 2021-02-20 18:27:16,875 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
  183. 2021-02-20 18:27:16,878 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
  184. 2021-02-20 18:27:16,879 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped.
  185. 2021-02-20 18:27:16,879 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
  186. 2021-02-20 18:27:16,879 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, ignoring any new events.
  187. 2021-02-20 18:27:16,881 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state
  188. 2021-02-20 18:27:16,882 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: 
  189. /************************************************************
  190. SHUTDOWN_MSG: Shutting down ResourceManager at hadoop-1/127.0.1.1
  192. ************************************************************/

在这一点上，一个电话 jps 告诉我 ResourceManager 未启动：

  1. hdoop@hadoop-1:~/hadoop/logs$ jps
  2. 18372 DataNode
  3. 18537 SecondaryNameNode
  4. 18282 NameNode
  5. 19549 Jps
  6. 18927 NodeManager

奇怪的是，当我自己通过终端启动时，它运行得很好。
有人知道发生了什么事吗？或者为什么 ResourceManager 拒绝从ansible开始？

hadoopyarnansible

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com