hadoop安装和设置

snz8szmq  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(253)

我正在尝试在centos中自己设置和安装hadoop集群环境。tasktracker未运行。它在执行start-all.sh之后消失
有人能帮我安装hadoop单节点/多节点集群吗?

mwg9r5ms

mwg9r5ms1#

Can you try this and let me know if you face any issues while installing the hadooop.

Pre-requiste
1.  Ensure Hosts file  – changes (master, slave) – no loopback , no local host
2.  Ensure Sshd runs
3.  Ensure hadoop Sudo ers – No passwd authentication
4.  Ensure selinux is disabled (not permissive) for user, policies and contexts
5.  Ensure Reboot - Pings masters and slaves.
6.  Ensure JDK 7 is installed in master and slaves.
7.  Ensure firewall is off or iptables allows ports mentioned by hadoop
8.  Ensure IPV6 is disabled or export HADOOP_OPTS=-Djava.net.preferIPv4Stack=true in hadoop_env.sh

In Master for seamless cut to slaves

Generate SHH - password less access
ssh-keygen -t rsa
$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@master
$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@slave
mkdir /opt/hadoop

# cd /opt/hadoop/

gunzip and untar the hadoop ball under /opt/hadoop/hadoop

Ensure hadoop owns the /opt directory
Changes in xml files under /opt/hadoop/hadoop/conf
core-site.xml

# Add the following inside the configuration tag

<property>
<name>fs.default.name</name>
<value>hdfs://master:9000/</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>

hdfs-site.xml

# Add the following inside the configuration tag

<property>
<name>dfs.data.dir</name>
<value>/opt/hadoop/hadoop/dfs/name/data</value>
<final>true</final>
</property>
<property>
<name>dfs.name.dir</name>
<value>/opt/hadoop/hadoop/dfs/name</value>
<final>true</final>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
mapred-site.xml
<property>
<name>mapred.job.tracker</name>
<value>hadoop-master:9001</value>
</property>
conf/hadoop-env.sh
export JAVA_HOME=/opt/jdk1.7.0_45
export HADOOP_OPTS=-Djava.net.preferIPv4Stack=true
export HADOOP_CONF_DIR=/opt/hadoop/hadoop/conf
Ship this cut to slaves ------------------------------>

In Master cut only (under conf)
$ cat masters
hadoop-master
$ cat slaves`enter code here`
Slave
$ bin/hadoop namenode –format
$ bin/start-dfs.sh`enter code here`
$ bin/start-mapred.sh

相关问题