从配置单元表中选择行时获取“org.apache.hadoop.hive.ql.metadata.hiveexception:hive runtime error while processing row”

iaqfqrcu  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(257)

我在配置单元中有一个orc格式的表。下面是create语句

CREATE EXTERNAL TABLE IF NOT EXISTS Mydb.test(    
  transkey string,
  acckey string,
  name string,
  sex string,
  city string,
  gentime timestamp
  )
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION '/data/gau/work/hive/test';

现在我想在这个表中添加两个额外的字段,所以我修改了表并添加了两个额外的字段。
alter table mydb.test添加列(location string,age int)。
因此,在修改表之后,我可以看到旧数据中新添加的列的字段值如预期的那样为null。e、 g如果我启动这个查询,从mydb.testlimit5中选择transkey,gentime,location,age;

+-----------------------------------+------------------------+-----------------+-----------------+--+
|          transkey                 |         gentime        |      location   | age              |
+-----------------------------------+------------------------+-----------------+-----------------+--+
| asxxxxx                           | 2019-07-xx 04:24:19.0  | india           | 25               |
| rtyxxxxxxx                        | 2019-07-04 04:24:19.0  | afg             | 45               |
| uuinxxxxxx                        | 2019-xx-04 04:24:20.0  | test            | 67               |
| ghjkxxxxxxx                       | 2017-xx-06 09:01:14.0  | NULL            | NULL             |
| uuuixxxxxxx                       | 2017-xx-06 09:01:xxx   | NULL            | NULL             |
| xxxxyyxxx                         | 2017-xx-06 09:01:1xxxx | NULL            | NULL             |

但是,当我从旧数据中获取一个特定的transkey(我可以通过查看gentime记录来知道数据是旧的还是新的)并从mydb.test启动一个类似select age,location的查询,其中transkey='yy'(根据gentime xyy是旧数据,新添加的两个字段都为null)时,它会出错如下

java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row 
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row 
    at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMa

pper.java:170)
    ... 8 more
Caused by: java.lang.NullPointerException
    at java.lang.System.arraycopy(Native Method)
    at org.apache.hadoop.io.Text.set(Text.java:225)
    at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$6.writeValue(VectorExpressionWriterFactory.java:686)
    at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterBytes.writeValue(VectorExpressionWriterFactory.java:272)

在启动上述查询时,理想情况下,我应该为这些列获取空值,但我得到的却是此错误。请帮助。
p、 所有这些查询在具有这两个新列的新数据上都能很好地工作

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题