我正在部署模式下使用nutch2.0爬网URL:
我用蚂蚁做坚果
我在hadoop中使用了nutch-2.0.job。
我运行了以下命令: sudo -u hdfs hadoop jar /home/bluesky/nutch/runtime/deploy/apache-nutch-2.0.job org.apache.nutch.crawl.InjectorJob /user/bluesky/nutch/urls
sudo -u hdfs hadoop jar /home/bluesky/nutch/runtime/deploy/apache-nutch-2.0.job org.apache.nutch.crawl.GeneratorJob sudo -u hdfs hadoop jar /home/bluesky/nutch/runtime/deploy/apache-nutch-2.0.job org.apache.nutch.fetcher.FetcherJob (batch -id)
但当我运行最后一个命令时,它在日志中给了我错误:
2012-10-09 15:35:04,292 ERROR org.apache.nutch.fetcher.FetcherJob: Unexpected error for http://www.imdb.com/title/tt0327056/
java.lang.AbstractMethodError: org.apache.gora.persistency.ListGenericArray.size()I
at org.apache.avro.generic.GenericDatumWriter.getArraySize(GenericDatumWriter.java:142)
at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:128)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.gora.avro.PersistentDatumWriter.write(PersistentDatumWriter.java:61)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:102)
at org.apache.gora.avro.PersistentDatumWriter.writeRecord(PersistentDatumWriter.java:74)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:64)
at org.apache.gora.avro.PersistentDatumWriter.write(PersistentDatumWriter.java:61)
at org.apache.gora.util.IOUtils.serialize(IOUtils.java:170)
at org.apache.gora.sql.store.SqlStore.setField(SqlStore.java:718)
at org.apache.gora.sql.store.SqlStore.setObject(SqlStore.java:669)
at org.apache.gora.sql.statement.MySqlInsertUpdateStatement.toStatement(MySqlInsertUpdateStatement.java:96)
at org.apache.gora.sql.store.SqlStore.put(SqlStore.java:616)
at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:60)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:514)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.nutch.fetcher.FetcherReducer$FetcherThread.output(FetcherReducer.java:668)
at org.apache.nutch.fetcher.FetcherReducer$FetcherThread.run(FetcherReducer.java:540)
2012-10-09 15:35:04,293 ERROR org.apache.nutch.fetcher.FetcherJob: fetcher throwable caught
java.lang.AbstractMethodError: org.apache.gora.persistency.ListGenericArray.size()I
at org.apache.avro.generic.GenericDatumWriter.getArraySize(GenericDatumWriter.java:142)
at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:128)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.gora.avro.PersistentDatumWriter.write(PersistentDatumWriter.java:61)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:102)
at org.apache.gora.avro.PersistentDatumWriter.writeRecord(PersistentDatumWriter.java:74)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:64)
at org.apache.gora.avro.PersistentDatumWriter.write(PersistentDatumWriter.java:61)
at org.apache.gora.util.IOUtils.serialize(IOUtils.java:170)
at org.apache.gora.sql.store.SqlStore.setField(SqlStore.java:718)
at org.apache.gora.sql.store.SqlStore.setObject(SqlStore.java:669)
at org.apache.gora.sql.statement.MySqlInsertUpdateStatement.toStatement(MySqlInsertUpdateStatement.java:96)
at org.apache.gora.sql.store.SqlStore.put(SqlStore.java:616)
at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:60)
暂无答案!
目前还没有任何答案,快来回答吧!