hadoop-remove newline from reducer输出

5w9g7ksd  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(296)

我的hadoop程序读取数千行(.log)文件并对其进行解析。之后,我使用output.collect()函数将它们写入一个文件。但是在每个reducer输出之后,它都将输出写入newline。如何将所有内容写入同一行,如何确保我的输出文件仅为一行?
减速器等级

public class Reduce extends MapReduceBase
    implements Reducer<Text, Text, Text, Text> {

    public void reduce(Text key, Iterator<Text> values,OutputCollector<Text, Text> output,
            Reporter reporter) throws IOException {
        Text t2 = new Text("");
      output.collect(key, t2);
    }
  }

Map类

public void map(LongWritable key, Text value, 
                        OutputCollector<Text, Text> output, 
                        Reporter reporter) throws IOException {

    Path("xxxx");//Location of file in HDFS
InputStreamReader(fs.open(pt)));

                String line = value.toString();

                String bePublished="";
               String patternString = "xxxx;
               Pattern pattern = Pattern.compile(patternString);
               Matcher matcher = pattern.matcher(line);

                  for(int u=0;u<48;u++){
       //Here i update my bepublished string

                  }

                 Text t2 = new Text("");
                 output.collect(new Text(bePublished), t2); 

        }

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题