hadoop2.2.0 wordcount示例中的“no filesystem for scheme:hdfs”ioexception

zf9nrax1 于 2021-06-04 发布在 Hadoop

关注(0)|答案(1)|浏览(448)

我有一个新的hadoopyarn安装，我已经通过中给定的jar文件执行了wordcount示例 hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples... 但是当我尝试编译wordcount源代码并运行它时，它会 java.io.IOException: No FileSystem for scheme: hdfs .
上述例外情况与这一行代码有关：

FileInputFormat.addInputPath(job, new Path(args[0]));

编辑：命令和输出如下：

hduser@master-virtual-machine:~$ hadoop jar Desktop/NativeWordcount.jar /tin /tout
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [rsrc:org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:rsrc:slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
13/12/03 07:14:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:166)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:466)
    at WordCount.main(WordCount.java:55)
    ... 10 more

hadoop hdfs yarn word-count

来源：https://stackoverflow.com/questions/20355176/no-filesystem-for-scheme-hdfs-ioexception-in-hadoop-2-2-0-wordcount-example

1条答案

按热度按时间

bn31dyow1#

我今天也遇到了这个问题。您需要确保hadoop hdfs jar在您的类路径中。
我的第一次尝试是简单地在 hadoop-hdfs maven中的包，但这是不够的。最后，我听从了cloudera的建议，添加了对 hadoop-client . 你方的相关条款 pom.xml 文件是：

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-client</artifactId>
    <version>VERSION</version>
 </dependency>

当我在clojure和leiningen一起做这个的时候，我把这个加到了我的 project.clj 改为文件：

(defproject 
  ; ...
  :dependencies [[org.apache.hadoop/hadoop-client "VERSION"]
                 ; ...
                 ])

（当然，您的版本将取决于系统上安装的内容。目前2.x系列中唯一的发布版本是 2.2.0 .)

赞(0）回复(0）举报 2021-06-04

我来回答

hadoop2.2.0 wordcount示例中的“no filesystem for scheme:hdfs”ioexception

1条答案

相关问题

热门标签

最新问答