我正在设置单节点群集。我在跟踪https://dev.to/awwsmm/installing-and-running-hadoop-and-spark-on-ubuntu-18-393h 辅导的。但是，我的nodemanager没有启动。

14468 SecondaryNameNode
14648 ResourceManager
14074 NameNode
15293 Jps
14223 DataNode

以下是节点管理器日志：

2020-08-07 14:13:52,796 FATAL org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Failed to initialize mapreduce_shuffle 
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:480)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:527)
2020-08-07 14:13:52,800 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed in state INITED; cause: java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:480)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:527)
2020-08-07 14:13:52,801 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl failed in state INITED; cause: java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:480)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:527)
2020-08-07 14:13:52,802 INFO org.apache.hadoop.service.AbstractService: Service NodeManager failed in state INITED; cause: java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
java.lang.IllegalArgumentException: The ServiceName: mapreduce_shuffle  set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:114)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:237)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:255)
    at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
    at 

2020-08-07 14:13:52,816 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NodeManager at es/127.0.1.1

************************************************************/

以下是my-site.xml：

<?xml version="1.0"?>

<configuration>

<!-- Site specific YARN configuration properties -->

  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>  
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>

我看过很多帖子 mapreduce.shuffle 必须改为 mapreduce_shuffle . 但是，在myyarn-site.xml中，我已经 mapreduce_shuffle .

1条答案

按热度按时间

ioekq8ef1#

从日志中 mapreduce_shuffle ，我可以看到两个空格在 mapreduce_shuffle 价值观。您是否可以正确地键入并复制到yarn-site.xml中，而不是直接从博客中复制它。这可能是由于某些特殊字符出现在 mapreduce_shuffle 一串
替换 yarn.nodemanager.auxservices.mapreduce.shuffle.class 与 yarn.nodemanager.aux-services.mapreduce_shuffle.class 标记并启动hadoop它应该可以工作。

<configuration>
<!-- Site specific YARN configuration properties -->

  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>  
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>

赞(0）回复(0）举报 2021-05-31

hadoop节点管理器未启动

1条答案

相关问题

热门标签

最新问答