hadoop失败错误:java.lang.arrayindexoutofboundsexception:5

uxhixvfz  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(604)

我使用hadoop mapreduce来计算每年的最小值和最大值,但是当我运行程序时,我得到了错误: FAILED Error: java.lang.ArrayIndexOutOfBoundsException: 5 我认为这是因为我的数据中有空值,因为程序在没有空值时运行良好。
因此,在我的map函数中,我编写if语句来检查是否有头,是否有空值:

public static class ExposureMapper
        extends Mapper<Object, Text, Text, MinMaxExposure> {

    private Text year = new Text();
    private double minexposure;
    private Double maxexposure;

    private MinMaxExposure outPut = new MinMaxExposure();

    public void map(Object key, Text value, Context context
    ) throws IOException, InterruptedException {
        try {
            //Some condition satisfying it is header
            if (value.toString().contains("Product")) {
                return;
            } else if(value.toString()==null) {
               return;
            }
            else{
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        String[] solarFields = value.toString().split(",");

        year.set(solarFields[2]);
        minexposure = Double.parseDouble(solarFields[5]);
        maxexposure = Double.parseDouble(solarFields[5]);

        try {
            outPut.setMinExposure(minexposure);
            outPut.setMaxExposure(maxexposure);
            context.write(year, outPut);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

但同样的错误也会发生。。。是因为 value.toString()==null 不是检查空值的正确方法吗?
编辑:

19/06/07 00:14:30 INFO mapreduce.Job: Task Id : attempt_1527224104960_0812_m_000000_1, Status : FAILED
Error: java.lang.ArrayIndexOutOfBoundsException: 5
    at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:50)
    at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:23)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
pod7payv

pod7payv1#

如果 value.toString().split(","); 只有不到六个元素, solarFields[5] 将不是元素,因此您将看到 ArrayIndexOutOfBoundsException .
创建后立即 solarFields 你应该检查它的长度:

if (solarFields == null || solarFields.length < 6) {
    return;
}

你还要确保 Double.parseDouble(solarFields[5]); 不会扔垃圾 NumberFormatException :

Double exposure;
try {
    exposure = Double.parseDouble(solarFields[5]);
} catch (NumberFormatException e) {
    return;
}

相关问题