hadoop 2.2.0字数prg无法从远程运行

qq24tv8q  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(296)

我无法使用eclipse从远程计算机运行hadoop2.2.0中的wordcount示例。我构建了可运行的jar并使用>java-jar wordcount.jar执行。我的驾驶员等级代码如下

  1. Configuration conf = new Configuration();
  2. Job job = new Job(conf, "wordcount");
  3. conf.set("fs.defaultFS", "hdfs://192.168.117.128:8020/");
  4. conf.set("hadoop.job.ugi", "root");
  5. conf.set("mapred.job.tracker", "192.168.117.128:8021");
  6. Path inputPath = new Path("/input/FB_Bank_Comments_abi_required_original.txt");
  7. Path outputPath = new Path("/output/wordcount"+System.currentTimeMillis());
  8. job.setJobName("wordcount_analysis");
  9. job.setJarByClass(Map.class);
  10. job.setOutputKeyClass(Text.class);
  11. job.setOutputValueClass(IntWritable.class);
  12. job.setMapperClass(Map.class);
  13. job.setReducerClass(Reduce.class);
  14. job.setInputFormatClass(TextInputFormat.class);
  15. job.setOutputFormatClass(TextOutputFormat.class);
  16. FileInputFormat.addInputPath(job, inputPath);
  17. FileOutputFormat.setOutputPath(job, outputPath);
  18. job.waitForCompletion(true);

我得到以下错误

  1. Exception in thread "main" java.lang.NullPointerException
  2. at java.lang.ProcessBuilder.start(Unknown Source)
  3. at org.apache.hadoop.util.Shell.runCommand(Shell.java:404)
  4. at org.apache.hadoop.util.Shell.run(Shell.java:379)
  5. at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
  6. at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
  7. at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
  8. at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:639)
  9. at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:435)
  10. at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:277)
  11. at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:125)
  12. at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:344)
  13. at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
  14. at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
  15. at java.security.AccessController.doPrivileged(Native Method)
  16. at javax.security.auth.Subject.doAs(Unknown Source)
  17. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
  18. at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
  19. at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
  20. at org.myorg.WordCount.main(WordCount.java:69)

我错过了什么?任何帮助都将不胜感激

lf5gs5x2

lf5gs5x21#

我也犯了同样的错误。最终我找到了原因。输出目录已存在。在我删除了输出目录并再次运行之后,它工作了。

相关问题