我正在编写一个pig udf,并尝试使用maven将其编译成jar。代码在eclipse中注册为良好,但是当我在maven中构建它时,我得到一个找不到符号的错误,如下所示。我已经检查了openreader方法的方法签名,它与我传递给它的类相匹配,所以我不确定maven编译出了什么问题。
依赖项代码段:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>${hadoop.version}</version>
<exclusions>
<exclusion>
<groupId>hsqldb</groupId>
<artifactId>hsqldb</artifactId>
</exclusion>
<exclusion>
<groupId>net.sf.kosmosfs</groupId>
<artifactId>kfs</artifactId>
</exclusion>
<exclusion>
<groupId>org.eclipse.jdt</groupId>
<artifactId>core</artifactId>
</exclusion>
<exclusion>
<groupId>net.java.dev.jets3t</groupId>
<artifactId>jets3t</artifactId>
</exclusion>
<exclusion>
<groupId>oro</groupId>
<artifactId>oro</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.7.5</version>
</dependency>
代码段:
import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.FileReader;
import org.apache.avro.file.SeekableInput;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.io.DatumReader;
import org.apache.avro.mapred.FsInput;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
protected String getLastRecord(String location) throws IOException {
Path path = new Path(location);
Configuration config = new Configuration();
SeekableInput input = new FsInput(path, config);
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>();
FileReader<GenericRecord> fileReader = DataFileReader.openReader(input, reader);
Iterator<GenericRecord> itr = fileReader.iterator();
long initialPosition = input.tell();
itr.next();
long newPosition = input.tell();
input.seek(input.length() - (initialPosition - newPosition));
return itr.next().get("id").toString();
};
错误:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project newscred-recsys-udfs: Compilation failure
[ERROR] /Users/andrew/Code/recommender/hadoop/newscred_recsys/udfs/java/src/main/java/com/mortardata/newscred/recsys/SqoopAvroLoader.java:[104,64] cannot find symbol
[ERROR] symbol: method openReader(org.apache.avro.file.SeekableInput,org.apache.avro.io.DatumReader<org.apache.avro.generic.GenericRecord>)
[ERROR] location: class org.apache.avro.file.DataFileReader
[ERROR] -> [Help 1]
2条答案
按热度按时间chhkpiq41#
显然,org.apache.avro的jar与从apache网站分发的jar有一组不同的方法,这导致了不匹配。
y1aodyip2#
这个项目应该清理干净。项目->清理