malformedurexception

i7uaboj4  于 2021-06-02  发布在  Hadoop
关注(0)|答案(3)|浏览(299)

我有下面的测试程序从hdfs读取一个文件。

public class FileReader {
    public static final String NAMENODE_IP = "172.32.17.209";
    public static final String FILE_PATH = "/notice.html";

    public static void main(String[] args) throws MalformedURLException,
            IOException {
        String url = "hdfs://" + NAMENODE_IP + FILE_PATH;

        InputStream is = new URL(url).openStream();
        InputStreamReader isr = new InputStreamReader(is);
        BufferedReader br = new BufferedReader(isr);
        String line = br.readLine();
        while(line != null) {
            System.out.println(line);
            line = br.readLine();
        }
    }
}

它是给予 java.net.MalformedURLException ```
Exception in thread "main" java.net.MalformedURLException: unknown protocol: hdfs
at java.net.URL.(URL.java:592)
at java.net.URL.(URL.java:482)
at java.net.URL.(URL.java:431)
at in.ksharma.hdfs.FileReader.main(FileReader.java:29)

wqlqzqxt

wqlqzqxt1#

在我们的案例中,我们必须将其与其他答案结合起来:
https://stackoverflow.com/a/21118824/1549135
首先在我们的hdfs设置课程中( Scala code ):

val hadoopConfig: Configuration = new Configuration()
hadoopConfig.set("fs.hdfs.impl", classOf[DistributedFileSystem].getName)
hadoopConfig.set("fs.file.impl", classOf[LocalFileSystem].getName)

后来,就像公认的答案:
https://stackoverflow.com/a/25971334/1549135

URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory)
Try(new URL(path))

旁注:
我们已经有: "org.apache.hadoop" % "hadoop-hdfs" % "2.8.0" 在我们的依赖和它没有帮助。

ny6fqffe

ny6fqffe2#

注册hadoop的url处理程序。标准的url处理程序不知道如何处理hdfs://scheme。
试试这个:

public static void main(String[] args) throws MalformedURLException,
            IOException {
        URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());

        String url = "hdfs://" + NAMENODE_IP + FILE_PATH;

        InputStream is = new URL(url).openStream();
        InputStreamReader isr = new InputStreamReader(is);
        BufferedReader br = new BufferedReader(isr);
        String line = br.readLine();
        while(line != null) {
            System.out.println(line);
            line = br.readLine();
        }
    }
g6ll5ycj

g6ll5ycj3#

在hadoop2.6上编写从hdfs读取的java应用程序时,我也遇到了同样的问题。我的解决办法是:添加

hadoop-2.X/share/hadoop/hdfs/hadoop-hdfs-2.X.jar to your classpath.

相关问题