如何对mongohadoop作业进行单元测试?
我目前的尝试:
public class MapperTest {
MapDriver<Object, BSONObject, Text, IntWritable> d;
@Before
public void setUp() throws IOException {
WordMapper mapper = new WordMapper();
d = MapDriver.newMapDriver(mapper);
}
@Test
public void testMapper() throws IOException {
BSONObject doc = new BasicBSONObject("sentence", "Two words");
d.withInput(new Text("anykey"), doc );
d.withOutput(new Text("Two"), new IntWritable(1));
d.withOutput(new Text("words"), new IntWritable(1));
d.runTest();
}
}
产生以下输出:
在org.bson.basicbsonobject类的io.serializations中,没有在conf中实现序列化的适用类
org.apache.hadoop.mrunit.internal.io.serialization.copy(serialization)上的java.lang.illegalstateexception。java:67)在org.apache.hadoop.mrunit.internal.io.serialization.copy(serialization。java:91)位于org.apache.hadoop.mrunit.internal.io.serialization.copywithconf(序列化)。java:104)在org.apache.hadoop.mrunit.testdriver.copy(testdriver。java:608)在org.apache.hadoop.mrunit.testdriver.copypair(testdriver。java:612)在org.apache.hadoop.mrunit.mapdriverbase.addinput(mapdriverbase。java:118)在org.apache.hadoop.mrunit.mapdriverbase.withinput(mapdriverbase。java:207) ...
1条答案
按热度按时间4bbkushb1#
您需要设置序列化程序。示例:mapdriver.getconfiguration().setstrings(“io.serializations”,“org.apache.hadoop.io.serializer.writeableserialization”,mongoserde.class.getname());
mongoserde公司:https://gist.github.com/lfrancke/01d1819a94f14da171e3
但我面临错误“org.bson.io.basicoutputbuffer.pipe(ljava/io/dataoutput;)我在使用这个(mongoserde)的时候。