当我尝试的时候
hadoop jar apache-nutch-2.2.jar org.apache.nutch.crawl.Crawler crawl -dir crawl -depth 3 -topN 5我得到以下错误。。。
13/07/09 09:02:46 WARN conf.Configuration: nutch-default.xml:a attempt to override final parameter: hadoop.job.history.user.location; Ignoring.
13/07/09 09:02:46 WARN conf.Configuration: nutch-default.xml:a attempt to override final parameter: hadoop.job.history.user.location; Ignoring.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/gora/persistency/impl/PersistentBase
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:218)
at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:68)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:136)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:250)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawler.main(Crawler.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
Caused by: java.lang.ClassNotFoundException: org.apache.gora.persistency.impl.PersistentBase
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 22 more有人能帮我把错误修好吗？
1条答案
按热度按时间lpwwtiir1#
你有哥拉依赖的问题。当然
apache-nutch-2.2.jar没有org/apache/gora/persistency/impl/PersistentBase.class. 您可以查看：jar tf apache-nutch-2.2.jar | grep持久化库
检查是否使用gora0.3版本编译nutch。
我猜您的hadoop节点中没有安装gora-*依赖项，因此解决方案是使用.job（而不是.jar）witch将所有依赖项绑定到hadoop中来发送它们。
如果您有此安装：
~
|-坚果/
|- apache-nutch-2.2.作业
|- 垃圾箱/
|- 坚果
以及
PATH=~/nutch/bin:.....您只需执行以下操作即可执行nutch：$ nutch inject ...
$ nutch crawl以及nutch命令在需要时调用hadoop。=============已更新==============
违规线路：http://grepcode.com/file/repo1.maven.org/maven2/org.apache.nutch/nutch/2.2.1/org/apache/nutch/crawl/injectorjob.java/#218
=============更新2=============
您正在使用命令行调用nutch：
如果你这么做，你必须保证
gora-core-0.x.jar在类路径中。如果调用.job，它必须
lib/gora-core-0.x.jar在拉链里面。hadoop解包了。job并添加了lib/*到类路径，所以应该没有必要做任何事情。