我创造了一个 jar
使用 eclipse
对于 mapreduce
工作。如果你提取 jar
你可以看到在场的所有同学。当你 run
这个 jar
在 hadoop``using
hadoop命令它显示如下所示的错误。 它不能只识别一个类,即
Test_project$TwoDArrayWritables.
Test_project是
main类和
TwoDArrayWritables是一个
class内
Test_project.
TwoDArrayWritables继承
TwoDArrayWritable内置类
hadoop` .
jar提取图像:
错误:
16/04/05 15:48:28 INFO mapred.JobClient: Task Id : attempt_201604051120_0002_m_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:889)
at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:747)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:966)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:422)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881)
... 9 more
Caused by: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
... 10 more
16/04/05 15:48:34 INFO mapred.JobClient: Task Id : attempt_201604051120_0002_m_000000_2, Status : FAILED
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:889)
at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:747)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:966)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:422)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881)
... 9 more
Caused by: java.lang.ClassNotFoundException: mapreduce.Test_project$TwoDArrayWritables
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
... 10 more
谷歌搜索了许多解决方案,但都没有奏效。请帮帮我!
2条答案
按热度按时间w8rqjzmb1#
试试这个:
bin/hadoop jar /path/to/jarfile/newproj.jar Test_project.TwoDArrayWritables /user/hduser/input /user/hduser/output1
2sbarzqh2#
挣扎了大约几个星期后,我的脑海中突然出现了一些东西,即我在工作中使用了两个减速机,因此为每个减速机定义了两个jobconf:
我以前的(错误代码)
JobConf conf = new JobConf(getConf(), Test_project.class); JobConf conf2 = new JobConf(getConf());
因为我认为配置已经定义好了,所以在conf2中没有提到类test\u project.class****我的礼物(正确代码)
JobConf conf = new JobConf(getConf(), Test_project.class); JobConf conf2 = new JobConf(getConf(), Test_project.class);
引发错误的原因是,在执行时,它正在搜索test\u project$twodarraywritables,因为它没有test\u project.class类,所以找不到twodarraywritables现在可以了。