我使用hadoop mapreduce来计算每年的最小值和最大值,但是当我运行程序时,我得到了错误: FAILED Error: java.lang.ArrayIndexOutOfBoundsException: 5
我认为这是因为我的数据中有空值,因为程序在没有空值时运行良好。
因此,在我的map函数中,我编写if语句来检查是否有头,是否有空值:
public static class ExposureMapper
extends Mapper<Object, Text, Text, MinMaxExposure> {
private Text year = new Text();
private double minexposure;
private Double maxexposure;
private MinMaxExposure outPut = new MinMaxExposure();
public void map(Object key, Text value, Context context
) throws IOException, InterruptedException {
try {
//Some condition satisfying it is header
if (value.toString().contains("Product")) {
return;
} else if(value.toString()==null) {
return;
}
else{
}
} catch (Exception e) {
e.printStackTrace();
}
String[] solarFields = value.toString().split(",");
year.set(solarFields[2]);
minexposure = Double.parseDouble(solarFields[5]);
maxexposure = Double.parseDouble(solarFields[5]);
try {
outPut.setMinExposure(minexposure);
outPut.setMaxExposure(maxexposure);
context.write(year, outPut);
} catch (IOException e) {
e.printStackTrace();
}
}
但同样的错误也会发生。。。是因为 value.toString()==null
不是检查空值的正确方法吗?
编辑:
19/06/07 00:14:30 INFO mapreduce.Job: Task Id : attempt_1527224104960_0812_m_000000_1, Status : FAILED
Error: java.lang.ArrayIndexOutOfBoundsException: 5
at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:50)
at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:23)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
1条答案
按热度按时间pod7payv1#
如果
value.toString().split(",");
只有不到六个元素,solarFields[5]
将不是元素,因此您将看到ArrayIndexOutOfBoundsException
.创建后立即
solarFields
你应该检查它的长度:你还要确保
Double.parseDouble(solarFields[5]);
不会扔垃圾NumberFormatException
: