我正在尝试在raspberrypi3上运行一个单节点hadoop集群。我能跑 hdfs dfs -ls /
,因此我知道至少hdfs已启动,但当我将示例wordcount作业作为冒烟测试运行以查看集群是否正常工作时,出现以下错误: Container ... is running beyond virtual memory limits. Current usage: 33.8 MB of 255 MB physical memory used; 1.1 GB of 537.6 MB virtual memory used. Killing container.
我的配置如下:
hdfs-site.xml:
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
mapred-site.xml:
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.map.memory.mb</name>
<value>256</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx210m</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>256</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx210m</value>
</property>
<property>
<name>yarn.app.mapreduce.am.resource.mb</name>
<value>256</value>
</property>
yarn-site.xml:
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>4</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>768</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>128</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>768</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-vcores</name>
<value>4</value>
</property>
core-site.xml:
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/hdfs/tmp</value>
</property>
树莓皮有cpu:4× 手臂皮质-a53和ram:1gb
如果有帮助的话,我的配置和设置与本文中描述的完全相同:https://web.archive.org/web/20170221231927/http://www.becausewecangeek.com/building-a-raspberry-pi-hadoop-cluster-part-1/
暂无答案!
目前还没有任何答案,快来回答吧!