配置单元接受无效查询并引发runtimeexception

cig3rfwq  于 2021-06-29  发布在  Hive
关注(0)|答案(2)|浏览(335)

今天,我在hive(mapr-distribution-hive 0.13.0-mapr-1508-21228)上发现了一个奇怪的行为
表定义:

CREATE EXTERNAL TABLE gd_temp_test.rate_merchants_test(
ROW_KEY string,
TRANS_DESC1 string,
TRANS_DESC2 string,
TRANS_DESC3 string,
TRANS_ID string
) 
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\u0001' LINES TERMINATED BY '\n'
STORED AS textfile
LOCATION '/home/gd/tempdata';

当执行下面的查询时,它确实接受了相同的查询并抛出了runtimeexception。

select * from gd_temp_test.rate_merchants_test t1 where t1.TRANS_DESC1 limit 1;

请注意,查询中的表是一个外部表 TRANS_DESC1 属于 String 类型。
例外

at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row .................

        ... 8 more
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean
        at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:134)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:540)
        ... 9 more

我原以为它在解析或评估执行计划时会被拒绝。当我解释问题的时候,

explain select * from gd_temp_test.rate_merchants_test t1 where t1.TRANS_DESC1 limit 1;

它能够解释这个问题。

Explain
STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 is a root stage

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Map Operator Tree:
          TableScan
            alias: t1
            Statistics: Num rows: 344 Data size: 1238497 Basic stats: COMPLETE Column stats: NONE
            Filter Operator
              predicate: trans_desc1 (type: string)
              Statistics: Num rows: 172 Data size: 619248 Basic stats: COMPLETE Column stats: NONE
              Select Operator
                expressions: row_key (type: string), trans_desc1 (type: string), trans_desc2 (type: string), trans_desc3 (type: string), trans_id (type: string)
                outputColumnNames: _col0, _col1, _col2, _col3, _col4
                Statistics: Num rows: 172 Data size: 619248 Basic stats: COMPLETE Column stats: NONE
                Limit
                  Number of rows: 1
                  Statistics: Num rows: 1 Data size: 3600 Basic stats: COMPLETE Column stats: NONE
                  File Output Operator
                    compressed: false
                    Statistics: Num rows: 1 Data size: 3600 Basic stats: COMPLETE Column stats: NONE
                    table:
                        input format: org.apache.hadoop.mapred.TextInputFormat
                        output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                        serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

  Stage: Stage-0
    Fetch Operator
      limit: 1

Time taken: 0.196 seconds, Fetched: 33 row(s)

有没有解释是什么原因导致这一切要等到运行时?这是正常的行为吗。
编辑1:添加了示例表定义。

bn31dyow

bn31dyow1#

如果看到配置单元数据类型隐式转换,则不允许字符串到布尔值的转换。
在你的询问中, where t1.TRANS_DESC1 尝试将trans_desc1读取为布尔类型,这就是它抛出 java.lang.ClassCastException

r3i60tvu

r3i60tvu2#

请上传您的表结构,并确保所有数据类型必须相同,数据类型的转换不适用。

相关问题