错误:map reduce程序的java.lang.nullpointerexception错误

svdrlsy4  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(336)

这个问题在这里已经有答案了

什么是nullpointerexception,如何修复它(12个答案)
四年前关门了。
i map.java类我试图通过以下方式发送文件名
context.write(新文本(stringword),新文本(文件名))
递归.java

import java.io.IOException;

 import org.apache.hadoop.io.*;
 import org.apache.hadoop.mapreduce.*;

 public class Reduce extends Reducer<Text, Text, Text, Text> {

@Override
public void reduce(Text key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
    int sum = 0;
    MapWritable occurenceInFile = new MapWritable();
    for (Text val : values) {
        if(occurenceInFile.containsKey(val)) {
            occurenceInFile.put(val, new IntWritable(1));
        } else {
            IntWritable occurence = (IntWritable) occurenceInFile.get(val);
            occurenceInFile.put(val, new IntWritable(occurence.get() + 1));
        }
        sum += 1;
    }
    String result = key.toString() + " (";
    boolean flag = false;
    for (Writable filenameWritable : occurenceInFile.keySet()) {
        if (flag) {
            result += ", ";
        }
        String filename = ((Text) filenameWritable).toString();
        result += filename + "=" + ((IntWritable) occurenceInFile.get(filenameWritable)).toString();
        flag = true;
    }
    result += ")\nTotal occurence of \"" + key + "\": " + Integer.toString(sum);

    context.write(key, new Text(result));
}
}

错误代码此错误代码显示在stderr中

Error: java.lang.NullPointerException
at Reduce.reduce(Reduce.java:18)
at Reduce.reduce(Reduce.java:6)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:180)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:656)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:394)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:166)

我开始写一个wordcount程序,但我仍然不知道如何修复这个错误。

hec6srdp

hec6srdp1#

我猜第18行是这个: occurenceInFile.put(val, new IntWritable(occurence.get() + 1)); 所以有可能 occurence 上一行读取的变量值为 null . 另一个变种是 occurence.get() 价值是 null .
所以,很难确切地指出是什么导致了它,但是您应该调试您的程序并找出,为什么会这样 occurenceInFile.get(val); 返回 null 价值观。

相关问题