如何使用开源puppet5.x部署hadoop的三个datanode集群?

zour9fqk  于 2021-06-01  发布在  Hadoop
关注(0)|答案(0)|浏览(231)

我想使用puppet部署hadoop集群。我希望在webui中有三个活动节点。webui中的“datanode信息”显示只有两个节点是活动的。
我正在aws中运行RHEL7.x。我在木偶大师和五个木偶特工身上使用木偶5.x。我正在尝试这个hadoop模块:https://forge.puppet.com/cesnet/site_hadoop/
我将此site.pp文件配置为具有三个从属节点:

$clients = [
  'client.example.com',
]
$master = 'master.example.com'
$slaves = [
    'node1.example.com',
    'node2.example.com',
    'node3.example.com',
]
$zookeepers = [
  $master,
]

# set to false for initial run

$hdfs_deployed = true

class { '::hadoop':
  hdfs_hostname       => $master,
  yarn_hostname       => $master,
  slaves              => $slaves,
  frontends           => $clients,
  nfs_hostnames       => $clients,
  zookeeper_hostnames => $zookeepers,
  hdfs_deployed       => $hdfs_deployed,
}

class { '::hbase':
  hdfs_hostname       => $master,
  master_hostname     => $master,
  slaves              => $slaves,
  zookeeper_hostnames => $zookeepers,
}

class { '::hive':
  metastore_hostname  => $master,
  server2_hostname    => $master,
  zookeeper_hostnames => $zookeepers,
}

class { '::spark':
  hdfs_hostname          => $master,
  historyserver_hostname => $master,
}

class { '::site_hadoop':
  users => [
    'hawking',
  ],
}

# required for hive, oozie

class { '::mysql::bindings':
  java_enable => true,
}

node 'master.example.com' {
  class { '::zookeeper':
    hostnames => $zookeepers,
  }
  include ::site_hadoop::role::master
}

node 'node1.example.com', 'node2.example.com', 'node3.example.com' {
  include ::site_hadoop::role::slave
}

node 'client.example.com' {
  include ::site_hadoop::role::frontend
}

我在三个从属节点、客户机节点和主节点上运行了多次“sudo puppet agent-t-d”。从来没有一个错误。当我登录到hadoopwebui时,我看到只有两个数据节点(site.pp清单中的三个从属节点中的两个)存在。第三个应该像容错一样隐藏吗?我想它应该是一个活动节点。为什么活动节点不出现在群集中?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题