我是hadoop世界的新手。我为我的应用程序开发了一个定制的输入格式和阅读器。我在运行单元测试时遇到问题。我跟随这篇文章开发了我的单元测试:如何对定制的recordreader和inputformat类进行单元测试?。我的单元测试代码与post中的代码完全相同,除了inputformat类:
Configuration conf = new Configuration(false);
conf.set("fs.default.name", "file:///");
File testFile = new File("/tmp/testfile");
Path path = new Path(testFile.getAbsoluteFile().toURI());
FileSplit split = new FileSplit(path, 0, testFile.length(), null);
MyFileInputFormat inputFormat = ReflectionUtils.newInstance(MyFileInputFormat.class, conf);
TaskAttemptContext context = new TaskAttemptContextImpl(conf, new TaskAttemptID());
try {
RecordReader reader = inputFormat.createRecordReader(split, context);
reader.initialize(split, context);
} catch (Exception e) {
}
当我尝试使用“mvn test”运行此测试时,出现以下错误:
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration$DeprecationDelta
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.hadoop.mapreduce.util.ConfigUtil.addDeprecatedKeys(ConfigUtil.java:54)
at org.apache.hadoop.mapreduce.util.ConfigUtil.loadResources(ConfigUtil.java:42)
at org.apache.hadoop.mapred.JobConf.(JobConf.java:119)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:80)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
有人能帮我弄明白怎么解决这个问题吗?
暂无答案!
目前还没有任何答案,快来回答吧!