我有以下设置:
$ export |grep HADOOP
declare -x HADOOP_HOME="/home/jesaremi/hadoop-3.1.3-bin"
declare -x HADOOP_OPTIONAL_TOOLS="hadoop-azure"
$ echo $PATH
/home/jesaremi/hadoop-3.1.3-bin/bin:/home/jesaremi/hadoop-3.1.3-bin/sbin:/opt/protobuf/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/jesaremi/.dotnet/tools:/opt/HP_Fortify/HP_Fortify_SCA_and_Apps_4.30/bin:/opt/HP_Fortify/HP_Fortify_SCA_and_Apps_4.30/bin
$ ls -l /home/jesaremi/hadoop-3.1.3-bin/share/hadoop/common/
total 7452
-rw-rw-r-- 1 jesaremi jesaremi 303043 Dec 19 17:30 hadoop-azure-3.1.3.jar
-rw-rw-r-- 1 jesaremi jesaremi 4092533 Dec 17 09:42 hadoop-common-3.1.3-SNAPSHOT.jar
-rw-rw-r-- 1 jesaremi jesaremi 2877294 Dec 17 09:42 hadoop-common-3.1.3-SNAPSHOT-tests.jar
-rw-rw-r-- 1 jesaremi jesaremi 130001 Dec 17 09:42 hadoop-kms-3.1.3-SNAPSHOT.jar
-rw-rw-r-- 1 jesaremi jesaremi 201637 Dec 17 09:42 hadoop-nfs-3.1.3-SNAPSHOT.jar
drwxrwxr-x 2 jesaremi jesaremi 4096 Dec 17 09:42 jdiff
drwxrwxr-x 2 jesaremi jesaremi 4096 Dec 17 09:42 lib
drwxrwxr-x 2 jesaremi jesaremi 4096 Dec 17 09:42 sources
drwxrwxr-x 3 jesaremi jesaremi 4096 Dec 17 09:42 webapps
然而,当我尝试计算abfs文件系统的数量时,我得到了以下结果:
$ hdfs dfsadmin -addMount abfs://main@test1.dfs.core.windows.net/ /mnt/abfs/main "fs.azure.abfs.account.name=test1.dfs.core.windows.net,fs.azure.account.key.test1.dfs.core.windows.net=<key removed>"
addMount: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem not found
1条答案
按热度按时间f0ofjuux1#
担心hadoop\u可选的\u工具env var不够;你需要把hadoopazurejar和其他一些东西放到common/lib中
从share/hadoop/tools/lib将hadoop azure jar、azure-*和wildfly-openssl.jar复制到share/hadoop/common/lib中
cloudstorejar带有诊断功能,因为它告诉您缺少哪个jar。