自定义对象作为Map器输出的值

kxxlusnw  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(358)

我已将对象构造为:

Class ObjExample {
     String s;
     Object[] objArray; // element in this array can be primitive type or array of primitive type.
}

我知道要使用它作为mapper或reducer的输出类型,我们必须为它实现writeablecomparable。
但是我真的很困惑如何为这种类编写readfields(),write(),compareto()?

qnakjoqk

qnakjoqk1#

您可以 Package 字段 sText 以及 objArrayArrayWritable . 每个元素 objArray 将是一个数组(也是 ArrayWritable )原语。以下是可能的实现:

public static final class ObjExample implements WritableComparable<ObjExample> {
    public final Text s = new Text(); // wrapped String
    public final ArrayOfArrays objArray = new ArrayOfArrays();

    @Override
    public int compareTo(ObjExample o) {
        // your logic here, example:
        return s.compareTo(o.s);
    }

    @Override
    public void write(DataOutput dataOutput) throws IOException {
        s.write(dataOutput);
        objArray.write(dataOutput);
    }

    @Override
    public void readFields(DataInput dataInput) throws IOException {
        s.readFields(dataInput);
        objArray.readFields(dataInput);
    }

    // set size of the objArray
    public void setSize(int n) {
        objArray.set(new IntArray[n]);
    }

    // set i-th element of the objArray to an array of elements
    public void setElement(int i, IntWritable... elements) {
        IntArray subArr = new IntArray();
        subArr.set(elements);
        objArray.get()[i] = subArr;
    }
}

您还需要另外两个类才能使其正常工作:

// array of primitives
public static final class IntArray extends ArrayWritable {
    public IntArray() {
        // you can specify any other primitive wrapper (DoubleWritable, Text, ...)
        super(IntWritable.class);
    }
}

// array of arrays
public static final class ArrayOfArrays extends ArrayWritable {
    public ArrayOfArrays() {
        super(IntArray.class);
    }
}

对象构造示例:

ObjExample o = new ObjExample();
o.s.set("hello");
o.setSize(2);
o.setElement(0, new IntWritable(0)); // single primitive
o.setElement(1, new IntWritable(1), new IntWritable(2)); // array of primitives

相关问题