treeset< text>在hadoop mapper map函数中未对值进行排序

fslejnso  于 2021-06-03  发布在  Hadoop
关注(0)|答案(0)|浏览(215)

Map类

import java.io.IOException;

import java.util.TreeSet;

import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.io.NullWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;

public class MapperTopNMR 
       extends Mapper<LongWritable, Text, NullWritable, Text> 
{   

    //TreeSet<Text> fatcats = new TreeSet<Text>(new SelComp());

    TreeSet<Text> fatcats = new TreeSet<Text>();

    @Override           
    public void map(LongWritable key, Text value, Context context)

            throws IOException, InterruptedException
    {    

           fatcats.add(new Text(value)); 

           if (fatcats.size() > 3)
            {

                fatcats.remove(fatcats.first());

            }   
    }
        @Override   
                protected void cleanup(Context context)
                          throws IOException, InterruptedException 
                {

                    for ( Text catname : fatcats ) 
                    {

                        context.write( NullWritable.get(),catname);

                    }
                }
            }

mapper类的mrunit

public class TopNMRTest 

{
  MapDriver<LongWritable, Text, NullWritable, Text> mapDriver;

  @Before
  public void setUp()
  {

    MapperTopNMR mapper = new MapperTopNMR();

    mapDriver = new MapDriver<LongWritable, Text, NullWritable, Text>();

    mapDriver.setMapper(mapper);
  }

  @Test
  public void testMapper() throws IOException 
  {

    mapDriver.withInput(new LongWritable(1), new Text("11"));

    mapDriver.withInput(new LongWritable(1), new Text("15"));

    mapDriver.withInput(new LongWritable(1), new Text("3"));

    mapDriver.withInput(new LongWritable(1), new Text("3"));

    mapDriver.withInput(new LongWritable(1), new Text("7"));

    mapDriver.withOutput(NullWritable.get(), new Text("7"));

    mapDriver.withOutput(NullWritable.get(), new Text("11"));

    mapDriver.withOutput(NullWritable.get(), new Text("15"));

    mapDriver.runTest();
  }

我期待结果
(空)7
(空)11
(空)15
但是当我打印输出时,存储在treeset中的值不在
顺序,它给出了在这个例子中插入的方式
树集包含: 11 15 3 7 (another 3 is duplicate it was eliminated). 注:树集-消除重复,不给自然秩序。甚至我尝试了treeset示例的comparator来反转顺序,结果如下

7 3 15 11.

请帮我摆脱这个问题。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题