spark sql抛出错误“java.lang.unsupportedoperationexception:未知字段类型:void”

cnwbcb6i  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(649)

在创建列值默认为null的表时,我发现spark(1.6)sql中有以下错误。例如:创建表test作为select列\u a,null作为test\u temp中的列\u b;
同样的方法也适用于配置单元并创建数据类型为“void”的列。
我使用空字符串而不是null来避免异常和获取字符串数据类型的新列。
有没有更好的方法使用sparksql在配置单元表中插入空值?

2017-12-26 07:27:59 ERROR StandardImsLogger$:177 - org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.UnsupportedOperationException: Unknown field type: void
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:789)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:746)
    at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$createTable$1.apply$mcV$sp(ClientWrapper.scala:428)
    at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$createTable$1.apply(ClientWrapper.scala:426)
    at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$createTable$1.apply(ClientWrapper.scala:426)
    at org.apache.spark.sql.hive.client.ClientWrapper$$anonfun$withHiveState$1.apply(ClientWrapper.scala:293)
    at org.apache.spark.sql.hive.client.ClientWrapper.liftedTree1$1(ClientWrapper.scala:239)
    at org.apache.spark.sql.hive.client.ClientWrapper.retryLocked(ClientWrapper.scala:238)
    at org.apache.spark.sql.hive.client.ClientWrapper.withHiveState(ClientWrapper.scala:281)
    at org.apache.spark.sql.hive.client.ClientWrapper.createTable(ClientWrapper.scala:426)
    at org.apache.spark.sql.hive.execution.CreateTableAsSelect.metastoreRelation$lzycompute$1(CreateTableAsSelect.scala:72)
    at org.apache.spark.sql.hive.execution.CreateTableAsSelect.metastoreRelation$1(CreateTableAsSelect.scala:47)
    at org.apache.spark.sql.hive.execution.CreateTableAsSelect.run(CreateTableAsSelect.scala:89)
    at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58)
    at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56)
    at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70)
    at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
    at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
    at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
    at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:56)
    at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:56)
    at org.apache.spark.sql.DataFrame.withCallback(DataFrame.scala:153)
    at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:145)
    at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:130)
    at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52)
    at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:829)
a5g8bdjr

a5g8bdjr1#

我找不到关于数据类型的太多信息 void 但它看起来有点像 Any scala中的数据类型。
本页末尾的表格说明 void 可以转换为任何其他数据类型。
这里有一些jira的问题,有点类似于你所面临的问题
hive -2901
hive -747
所以,正如评论中提到的 NULL 可以将其强制转换为任何隐式数据类型。

select cast(NULL as string) as column_b

相关问题