hadoop在完成所有的约简之后给出输出

r3i60tvu  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(346)

在代码的第一部分中,我设置了许多数组来跟踪如下值:

@Override 
  public void configure(JobConf conf){
       top5=new String[5];
      counttop5=new int[5]
      }

现在,在从reducer获得一些代码之后,我想将top的内容输出到输出文件,但是,为了做到这一点,我创建了一个close()函数:

@Override 
      public void close(){
         //what goes here? 
      }

但是,正如您所看到的,由于输出是在reducer方法中定义的,因此没有什么可调用的。虽然代码本身很长,但这里是方法的数据签名:

public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, Writable> {
private  static IntWritable one = new IntWritable(1);
    public void map(LongWritable key, Text value, OutputCollector<Text, Writable> output, Reporter reporter) throws IOException {
public static class Reduce extends MapReduceBase implements Reducer<Text, Writable, Text, Writable> {

      static ArrayList<String> files;
          static String[] top5;
          static int[] counttop5;
          int reducedterms;
          public void configure(JobConf conf){
              files= new ArrayList<String>();
              top5=new String[5];
              reducedterms=0; 
              counttop5=new int[5];

          }
          @Override 
          public void close(){
             //what goes here? 
          }

          public void reduce(Text key, Iterator<Writable> values, OutputCollector<Text, Writable> output, Reporter reporter) throws IOException {
    }

有人知道解决办法吗?

1aaf6o9v

1aaf6o9v1#

这是一个使用 org.apache.hadoop.mapreduce 班级。api与您现在提供的代码有些不同,因为我们现在扩展了 org.apache.hadoop.mapreduce.Reducer 基类而不是实现 org.apache.hadoop.mapred.Reducer 接口,但这里有一个简单的方法来做您需要的事情:

public static class Reduce extends Reducer<Text, Writable, Text, Writable> {

  static ArrayList<String> files;
  static String[] top5;
  static int[] counttop5;
  int reducedterms;

  //setup method instead of configure
  public void setup(Context context){
          files= new ArrayList<String>();
          top5=new String[5];
          reducedterms=0; 
          counttop5=new int[5];
  }

  //In cleanup you may access the write method
  @Override 
  public void cleanup(Context context){
         // Use top5 and counttop5 the way you see fit
         // Use context.write() to pass them ahead!
  }

      public void reduce(Text key, Iterator<Writable> values, Context context) throws IOException {
}

相关问题