在本地模式下运行hadoop作业时出现错误“java.lang.illegalargumentexception:无法从空字符串创建路径”

zpgglvta  于 2021-06-04  发布在  Hadoop
关注(0)|答案(0)|浏览(254)

我有一个hadoop作业,在本地模式下,在hadoop2.2.0安装的测试环境中定期运行。作业大部分时间都会成功,但偶尔会失败,并显示上述错误消息。调用堆栈如下:

  1. Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path from an empty string
  2. at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
  3. at org.apache.hadoop.fs.Path.<init>(Path.java:135)
  4. at org.apache.hadoop.fs.Path.<init>(Path.java:94)
  5. at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:150)
  6. at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:636)
  7. at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:430)
  8. at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
  9. at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
  10. at java.security.AccessController.doPrivileged(Native Method)
  11. at javax.security.auth.Subject.doAs(Subject.java:415)
  12. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
  13. at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
  14. at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
  15. at com.mathworks.hadoop.MWMapReduceDriver.run(MWMapReduceDriver.java:1443)
  16. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  17. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
  18. at com.mathworks.hadoop.MWMapReduceDriver.main(MWMapReduceDriver.java:1454)
  19. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  20. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  21. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  22. at java.lang.reflect.Method.invoke(Method.java:606)
  23. at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

在hadoop1.2.1中,同样的任务不会失败。我查看了localjobrunner.java的源代码,发现第150行如下:

  1. this.localJobDir = localFs.makeQualified(new Path(new Path(conf.getLocalPath(jobDir), user), jobid.toString()));

我在异常捕获块中打印了conf.getlocalpath(jobdir)、user和jobid.tostring(),发现当作业失败时user为空。根据源代码,通过调用获取用户 UserGroupInformation.getCurrentUser().getShortUserName() . 有人知道这个调用在什么情况下会返回空字符串吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题