运行超出虚拟内存限制的容器

vxbzzdmp  于 2021-06-01  发布在  Hadoop
关注(0)|答案(0)|浏览(288)

我正在尝试在raspberrypi3上运行一个单节点hadoop集群。我能跑 hdfs dfs -ls / ,因此我知道至少hdfs已启动,但当我将示例wordcount作业作为冒烟测试运行以查看集群是否正常工作时,出现以下错误: Container ... is running beyond virtual memory limits. Current usage: 33.8 MB of 255 MB physical memory used; 1.1 GB of 537.6 MB virtual memory used. Killing container. 我的配置如下:
hdfs-site.xml:

<property>  
<name>dfs.replication</name>  
<value>1</value>  
</property>

mapred-site.xml:

<property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.map.memory.mb</name>
    <value>256</value>
  </property>
  <property>
    <name>mapreduce.map.java.opts</name>
    <value>-Xmx210m</value>
  </property>
  <property>
    <name>mapreduce.reduce.memory.mb</name>
    <value>256</value>
  </property>
  <property>
    <name>mapreduce.reduce.java.opts</name>
    <value>-Xmx210m</value>
  </property>
  <property>
    <name>yarn.app.mapreduce.am.resource.mb</name>
    <value>256</value>
  </property>

yarn-site.xml:

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.resource.cpu-vcores</name>
    <value>4</value>
  </property>
  <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>768</value>
  </property>
  <property>
    <name>yarn.scheduler.minimum-allocation-mb</name>
    <value>128</value>
  </property>
  <property>
    <name>yarn.scheduler.maximum-allocation-mb</name>
    <value>768</value>
  </property>
  <property>
    <name>yarn.scheduler.minimum-allocation-vcores</name>
    <value>1</value>
  </property>
  <property>
    <name>yarn.scheduler.maximum-allocation-vcores</name>
    <value>4</value>
  </property>

core-site.xml:

<property>  
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
</property>  
<property>  
  <name>hadoop.tmp.dir</name>
  <value>/hdfs/tmp</value>
</property>

树莓皮有cpu:4× 手臂皮质-a53和ram:1gb
如果有帮助的话,我的配置和设置与本文中描述的完全相同:https://web.archive.org/web/20170221231927/http://www.becausewecangeek.com/building-a-raspberry-pi-hadoop-cluster-part-1/

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题