在docker容器中运行hdfs伪模式

emeijp43  于 2021-05-31  发布在  Hadoop
关注(0)|答案(1)|浏览(440)

我正尝试在docker容器中以伪模式运行hdfs,配置如下页面:https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/singlecluster.html#pseudo-分布式操作,但我没有使用 start-all.sh 脚本,因为它不应该能够做ssh,所以我手动运行命令 bin/hdfs --daemon start namenode|datanode 一个接一个的开始。问题是我可以看到namenode成功启动,但datanode退出时没有任何错误消息。datanode的最后一段日志是:

...
2018-04-09 21:04:03,830 INFO org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker: Scheduling a check for [DISK]file:/apps/hadoop/hdfs/data
2018-04-09 21:04:04,188 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2018-04-09 21:04:04,296 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2018-04-09 21:04:04,296 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2018-04-09 21:04:04,665 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2018-04-09 21:04:04,667 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576
2018-04-09 21:04:04,671 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is hdfs
2018-04-09 21:04:04,671 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2018-04-09 21:04:04,677 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0
2018-04-09 21:04:04,733 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:9866
2018-04-09 21:04:04,735 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwidth is 10485760 bytes/s
2018-04-09 21:04:04,735 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 50
``` `core-site.xml` 文件:
<property>
    <name>dfs.namenode.name.dir</name>
    <value>/apps/hadoop/hdfs/name</value>
</property>

<property>
    <name>dfs.datanode.data.dir</name>
    <value>/apps/hadoop/hdfs/data</value>
</property>
bcs8qyzn

bcs8qyzn1#

我认为这是基本的图像问题,我是使用阿尔卑斯山,一旦我改为centos,datanode工程!一定是阿尔卑斯山少了些什么,如果有人知道是什么,请欣赏,因为基于centos的图像最终会比阿尔卑斯山大得多。

相关问题