添加ec2用户以使用hadoop

h9vpoimq  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(461)

我已经准备好了。我用ssh连接到主节点。我想把一个文件复制到hdfs系统。在我的程序中,这行代码是:

  1. os.system('/home/hadoop/bin/hdfs dfs -put %s PATH_to_HADOOP' % tmp_output)

我想输入hdfs文件系统的路径。
我愿意

  1. [ec2-user@ip-172-31-0-185 input]$ /home/hadoop/bin/hdfs dfs -ls /
  2. Found 2 items
  3. drwxr-xr-x - hadoop supergroup 0 2014-04-14 22:21 /hbase
  4. drwxrwx--- - hadoop supergroup 0 2014-04-14 22:19 /tmp

我尽力了

  1. [ec2-user@ip-172-31-0-185 input]$ /home/hadoop/bin/hdfs dfs -mkdir /tmp/stockmarkets
  2. mkdir: Permission denied: user=ec2-user, access=EXECUTE, inode="/tmp":hadoop:supergroup:drwxrwx---

因此,要添加ec2用户以使用hadoop,我遵循以下说明:
http://cloudcelebrity.wordpress.com/2013/06/05/handling-permission-denied-error-on-hdfs/
但在我写完之后(我用ubuntu代替ec2用户)

  1. sudo adduser ec2-user hadoop

我没有收到add消息,而是得到:

  1. Usage: useradd [options] LOGIN
  2. Options:
  3. -b, --base-dir BASE_DIR base directory for the home directory of the
  4. new account
  5. -c, --comment COMMENT GECOS field of the new account
  6. -d, --home-dir HOME_DIR home directory of the new account
  7. -D, --defaults print or change default useradd configuration
  8. -e, --expiredate EXPIRE_DATE expiration date of the new account
  9. -f, --inactive INACTIVE password inactivity period of the new account
  10. -g, --gid GROUP name or ID of the primary group of the new
  11. account
  12. -G, --groups GROUPS list of supplementary groups of the new
  13. account
  14. -h, --help display this help message and exit
  15. -k, --skel SKEL_DIR use this alternative skeleton directory
  16. -K, --key KEY=VALUE override /etc/login.defs defaults
  17. -l, --no-log-init do not add the user to the lastlog and
  18. faillog databases
  19. -m, --create-home create the user's home directory
  20. -M, --no-create-home do not create the user's home directory
  21. -N, --no-user-group do not create a group with the same name as
  22. the user
  23. -o, --non-unique allow to create users with duplicate
  24. (non-unique) UID
  25. -p, --password PASSWORD encrypted password of the new account
  26. -r, --system create a system account
  27. -s, --shell SHELL login shell of the new account
  28. -u, --uid UID user ID of the new account
  29. -U, --user-group create a group with the same name as the user
  30. -Z, --selinux-user SEUSER use a specific SEUSER for the SELinux user mapping

所以我很困惑,搞砸了。。请帮助>。。。。

rdlzhqv9

rdlzhqv91#

ssh作为hadoop@(publicip)用于amazon emr。
从那里你可以用hdfs做任何你想做的事情,而不必“su”。我只是做了一个mkdir,运行了distcp和流媒体作业。我做的一切都是hadoop@,按照emr的指令。

relj7zay

relj7zay2#

如果查看hdfs目录/tmp的权限,可以看到/tmp由用户hadoop和 ec2-user 没有在/tmp中创建文件/目录的权限
为目录/tmp分配适当的权限,使用以下命令

  1. [ec2-user@ip-172-31-0-185 input]$ sudo -su hadoop /home/hadoop/bin/hdfs dfs -chmod 777 /tmp

现在尝试在/tmp hdfs位置内创建目录

  1. [ec2-user@ip-172-31-0-185 input]$ /home/hadoop/bin/hdfs dfs -mkdir /tmp/stockmarkets

相关问题