hadoop找不到joda依赖项

jecbmhm3  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(411)

我试着编写一个mapreduce应用程序,它从csv文件中获取输入并聚合每个帐户的滚动利息。我已经通过了mrunit测试,一切似乎都正常。当我尝试在我的机器上运行jar文件(我在本地运行hadoop)时,会出现以下错误

..........>>
14/07/16 09:09:05 INFO mapred.MapTask: Starting flush of map output
14/07/16 09:09:05 INFO mapred.LocalJobRunner: map task executor complete.
14/07/16 09:09:05 WARN mapred.LocalJobRunner: job_local1566570594_0001
java.lang.Exception: java.lang.NoClassDefFoundError: org/joda/time/ReadableInstant
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NoClassDefFoundError: org/joda/time/ReadableInstant
    at org.mrIntCalc.CalcInterestMapper.map(CalcInterestMapper.java:38)
    at org.mrIntCalc.CalcInterestMapper.map(CalcInterestMapper.java:12)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.joda.time.ReadableInstant
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 11 more
14/07/16 09:09:05 INFO mapreduce.Job: Job job_local1566570594_0001 running in uber mode : false
14/07/16 09:09:05 INFO mapreduce.Job:  map 0% reduce 0%
14/07/16 09:09:05 INFO mapreduce.Job: Job job_local1566570594_0001 failed with state FAILED due to: NA
14/07/16 09:09:05 INFO mapreduce.Job: Counters: 0

我使用maven构建了这个应用程序,因此joda(v2.3)在pom.xml中作为一个依赖项列出。就像我说的,我的单元测试结果很清楚。查看这个问题,我看到一些人提到在这个问题的根源上有一个类路径问题,尽管我不认为这是完全正确的。有什么想法吗?

tzdcorbm

tzdcorbm1#

将joda-time.jar复制到hadoop公共库中。也许不是最好的解决方案,但至少它是有效的。

yqyhoc1h

yqyhoc1h2#

检查类路径/构建路径中是否存在joda time*.jar如果没有提及,请添加它。否则,将jar复制到lib目录并将其添加到类路径中。

相关问题