在hdfs上运行mapreduce程序时,输入路径不存在

nhaq1z21  于 2021-05-31  发布在  Hadoop
关注(0)|答案(0)|浏览(261)

我确信我已经把输入文件上传到hdfs了

[hadoop@hadoop000 data]$ hadoop fs -put trackinfo_20130721.data /project/input
[hadoop@hadoop000 data]$ hadoop fs -ls /project/input
Found 1 items
-rw-r--r--   1 hadoop supergroup  173555592 2020-04-07 02:33 /project/input/trackinfo_20130721.data

[hadoop@hadoop000 data]$ hadoop fs -ls hdfs://hadoop000:8020/project/input/
Found 1 items
-rw-r--r--   1 hadoop supergroup  173555592 2020-04-07 02:33 hdfs://hadoop000:8020/project/input/trackinfo_20130721.data

当我执行以下命令时:

hadoop jar /home/hadoop/lib/hadoop-train-v2-1.0.jar com.imooc.bigdata.hadoop.mr.project.mr.PVStatApp hdfs://hadoop000:8020/project/input/trackinfo_20130721.data hdfs://hadoop000:8020/project/output/v1/pvstat/

它显示以下错误:

20/04/07 02:26:45 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1583424547361_0031
20/04/07 02:26:45 WARN security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://hadoop000:8020/project/input/trackinfo_20130721.data
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://hadoop000:8020/project/input/trackinfo_20130721.data
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)
    at com.imooc.bigdata.hadoop.mr.project.mr.PVStatApp.main(PVStatApp.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:141)

这非常奇怪,因为输入路径应该在我的hdfs中。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题