用mrunit对mongohadoop作业进行单元测试

t98cgbkg  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(303)

如何对mongohadoop作业进行单元测试?
我目前的尝试:

public class MapperTest {

    MapDriver<Object, BSONObject, Text, IntWritable> d;

    @Before
    public void setUp() throws IOException {
        WordMapper mapper = new WordMapper();
        d = MapDriver.newMapDriver(mapper);
    }

    @Test
    public void testMapper() throws IOException {

        BSONObject doc = new BasicBSONObject("sentence", "Two words");
        d.withInput(new Text("anykey"), doc );

        d.withOutput(new Text("Two"), new IntWritable(1));
        d.withOutput(new Text("words"), new IntWritable(1));

        d.runTest();
    }
}

产生以下输出:
在org.bson.basicbsonobject类的io.serializations中,没有在conf中实现序列化的适用类
org.apache.hadoop.mrunit.internal.io.serialization.copy(serialization)上的java.lang.illegalstateexception。java:67)在org.apache.hadoop.mrunit.internal.io.serialization.copy(serialization。java:91)位于org.apache.hadoop.mrunit.internal.io.serialization.copywithconf(序列化)。java:104)在org.apache.hadoop.mrunit.testdriver.copy(testdriver。java:608)在org.apache.hadoop.mrunit.testdriver.copypair(testdriver。java:612)在org.apache.hadoop.mrunit.mapdriverbase.addinput(mapdriverbase。java:118)在org.apache.hadoop.mrunit.mapdriverbase.withinput(mapdriverbase。java:207) ...

4bbkushb

4bbkushb1#

您需要设置序列化程序。示例:mapdriver.getconfiguration().setstrings(“io.serializations”,“org.apache.hadoop.io.serializer.writeableserialization”,mongoserde.class.getname());
mongoserde公司:https://gist.github.com/lfrancke/01d1819a94f14da171e3
但我面临错误“org.bson.io.basicoutputbuffer.pipe(ljava/io/dataoutput;)我在使用这个(mongoserde)的时候。

相关问题