我确信我已经把输入文件上传到hdfs了
[hadoop@hadoop000 data]$ hadoop fs -put trackinfo_20130721.data /project/input
[hadoop@hadoop000 data]$ hadoop fs -ls /project/input
Found 1 items
-rw-r--r-- 1 hadoop supergroup 173555592 2020-04-07 02:33 /project/input/trackinfo_20130721.data
[hadoop@hadoop000 data]$ hadoop fs -ls hdfs://hadoop000:8020/project/input/
Found 1 items
-rw-r--r-- 1 hadoop supergroup 173555592 2020-04-07 02:33 hdfs://hadoop000:8020/project/input/trackinfo_20130721.data
当我执行以下命令时:
hadoop jar /home/hadoop/lib/hadoop-train-v2-1.0.jar com.imooc.bigdata.hadoop.mr.project.mr.PVStatApp hdfs://hadoop000:8020/project/input/trackinfo_20130721.data hdfs://hadoop000:8020/project/output/v1/pvstat/
它显示以下错误:
20/04/07 02:26:45 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1583424547361_0031
20/04/07 02:26:45 WARN security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://hadoop000:8020/project/input/trackinfo_20130721.data
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://hadoop000:8020/project/input/trackinfo_20130721.data
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)
at com.imooc.bigdata.hadoop.mr.project.mr.PVStatApp.main(PVStatApp.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
这非常奇怪,因为输入路径应该在我的hdfs中。
暂无答案!
目前还没有任何答案,快来回答吧!