在hadoop中运行作业-error security.usergroupinformation

dhxwm5r4  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(530)

我在hadoop中运行了一个作业(jar文件)(我的jar文件有一些其他jar文件和一个zip文件),但是我遇到了这个错误。如果有人能帮助我并说出这个错误的原因,我将不胜感激。我认为这个错误是因为在我运行的jar文件中存在一个zip文件,但是如何手动解压呢?

  1. ERROR security.UserGroupInformation: PriviledgedActionException
  2. as:hduser cause:java.util.zip.ZipException: error reading zip file
  3. at java.util.zip.ZipFile.read(Native Method)
  4. at java.util.zip.ZipFile.access$1400(ZipFile.java:60)
  5. at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:717)
  6. at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:419)
  7. at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
  8. at java.io.FilterInputStream.read(FilterInputStream.java:107)
  9. at org.apache.hadoop.util.RunJar.unJar(RunJar.java:55)
  10. at org.apache.hadoop.filecache.TrackerDistributedCacheManager.downloadCacheObject(TrackerDistributedCacheManager.java:438)
  11. at org.apache.hadoop.filecache.TrackerDistributedCacheManager.localizePublicCacheObject(TrackerDistributedCacheManager.java:477)
  12. at org.apache.hadoop.filecache.TrackerDistributedCacheManager.getLocalCache(TrackerDistributedCacheManager.java:192)
  13. at org.apache.hadoop.filecache.TaskDistributedCacheManager.setupCache(TaskDistributedCacheManager.java:182)
  14. at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:131)
  15. at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:592)
  16. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1013)
  17. at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
  18. at java.security.AccessController.doPrivileged(Native Method)
  19. at javax.security.auth.Subject.doAs(Subject.java:422)
  20. at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
  21. at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
  22. at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
  23. at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:259)
  24. at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
  25. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
  26. at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
  27. at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
  28. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  29. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  30. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  31. at java.lang.reflect.Method.invoke(Method.java:498)
  32. at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
lndjwyie

lndjwyie1#

根据这个日志:

  1. as:hduser cause:java.util.zip.ZipException: error reading zip file
  2. at java.util.zip.ZipFile.read(Native Method)
  3. at java.util.zip.ZipFile.access$1400(ZipFile.java:60)

你好像没有权限读取那个zip文件。
请尝试由您的用户读取该数据(sudo-u hduser head-l 1/path/of/file.zip),我猜您也会看到权限错误。您应该首先调用hadoop用户(hduser?)的读取权限
被selinux阻止是另一个可能的问题,请检查日志:/var/log/audit/audit.log
如果你想在linux文件系统中解压这个zip文件,你可以运行
$unzip/path/of/file
如果这是一个hdfs路径,您可能需要首先获取它(如果我将操作它)

相关问题