我在hadoop中运行了一个作业(jar文件)(我的jar文件有一些其他jar文件和一个zip文件),但是我遇到了这个错误。如果有人能帮助我并说出这个错误的原因,我将不胜感激。我认为这个错误是因为在我运行的jar文件中存在一个zip文件,但是如何手动解压呢?
ERROR security.UserGroupInformation: PriviledgedActionException
as:hduser cause:java.util.zip.ZipException: error reading zip file
at java.util.zip.ZipFile.read(Native Method)
at java.util.zip.ZipFile.access$1400(ZipFile.java:60)
at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:717)
at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:419)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.hadoop.util.RunJar.unJar(RunJar.java:55)
at org.apache.hadoop.filecache.TrackerDistributedCacheManager.downloadCacheObject(TrackerDistributedCacheManager.java:438)
at org.apache.hadoop.filecache.TrackerDistributedCacheManager.localizePublicCacheObject(TrackerDistributedCacheManager.java:477)
at org.apache.hadoop.filecache.TrackerDistributedCacheManager.getLocalCache(TrackerDistributedCacheManager.java:192)
at org.apache.hadoop.filecache.TaskDistributedCacheManager.setupCache(TaskDistributedCacheManager.java:182)
at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:131)
at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:592)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1013)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:259)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
1条答案
按热度按时间lndjwyie1#
根据这个日志:
你好像没有权限读取那个zip文件。
请尝试由您的用户读取该数据(sudo-u hduser head-l 1/path/of/file.zip),我猜您也会看到权限错误。您应该首先调用hadoop用户(hduser?)的读取权限
被selinux阻止是另一个可能的问题,请检查日志:/var/log/audit/audit.log
如果你想在linux文件系统中解压这个zip文件,你可以运行
$unzip/path/of/file
如果这是一个hdfs路径,您可能需要首先获取它(如果我将操作它)