来自eclipse的wordcount

epfja78i  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(398)

我使用的是hadoop的eclipse插件。通过创建一个hadoop服务器,我可以看到hdfs中的所有文件,但是当我尝试从eclipse运行wordcount.java文件时,它给了我一个异常,而从终端运行起来却很顺利。例外情况如下。

2/11/14 04:09:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for  your platform... using builtin-java classes where applicable
12/11/14 04:09:06 WARN mapred.JobClient: No job jar file set.  User classes may not be     found. See JobConf(Class) or JobConf#setJar(String).
12/11/14 04:09:06 WARN snappy.LoadSnappy: Snappy native library not loaded
12/11/14 04:09:06 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-hduser/mapred/staging/hduser1728681403/.staging/job_local_0001
12/11/14 04:09:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg
Exception in thread "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg
    at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
    at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:989)
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:981)
    at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1261)
    at WordCount.run(WordCount.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at WordCount.main(WordCount.java:155)
kr98yfug

kr98yfug1#

谢谢你解决了我的问题。从eclipse上我得到了同样的问题。。。使用hdfs://localhost:54310/用户/。。。而不是“/user/…”

sbtkgmzw

sbtkgmzw2#

我先来调查一下:

ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg

这似乎是问题的根源。你确定这是正确的路径吗?如果是这样的话,您可能没有访问它的特权。以后我会尽可能地消除警告。

相关问题