我在配置单元中有一个orc格式的表。下面是create语句
CREATE EXTERNAL TABLE IF NOT EXISTS Mydb.test(
transkey string,
acckey string,
name string,
sex string,
city string,
gentime timestamp
)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION '/data/gau/work/hive/test';
现在我想在这个表中添加两个额外的字段,所以我修改了表并添加了两个额外的字段。
alter table mydb.test添加列(location string,age int)。
因此,在修改表之后,我可以看到旧数据中新添加的列的字段值如预期的那样为null。e、 g如果我启动这个查询,从mydb.testlimit5中选择transkey,gentime,location,age;
+-----------------------------------+------------------------+-----------------+-----------------+--+
| transkey | gentime | location | age |
+-----------------------------------+------------------------+-----------------+-----------------+--+
| asxxxxx | 2019-07-xx 04:24:19.0 | india | 25 |
| rtyxxxxxxx | 2019-07-04 04:24:19.0 | afg | 45 |
| uuinxxxxxx | 2019-xx-04 04:24:20.0 | test | 67 |
| ghjkxxxxxxx | 2017-xx-06 09:01:14.0 | NULL | NULL |
| uuuixxxxxxx | 2017-xx-06 09:01:xxx | NULL | NULL |
| xxxxyyxxx | 2017-xx-06 09:01:1xxxx | NULL | NULL |
但是,当我从旧数据中获取一个特定的transkey(我可以通过查看gentime记录来知道数据是旧的还是新的)并从mydb.test启动一个类似select age,location的查询,其中transkey='yy'(根据gentime xyy是旧数据,新添加的两个字段都为null)时,它会出错如下
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row
at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMa
pper.java:170)
... 8 more
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at org.apache.hadoop.io.Text.set(Text.java:225)
at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$6.writeValue(VectorExpressionWriterFactory.java:686)
at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterBytes.writeValue(VectorExpressionWriterFactory.java:272)
在启动上述查询时,理想情况下,我应该为这些列获取空值,但我得到的却是此错误。请帮助。
p、 所有这些查询在具有这两个新列的新数据上都能很好地工作
暂无答案!
目前还没有任何答案,快来回答吧!