在pyspark中写入配置单元数据库时阶段失败

uz75evzq  于 2021-06-24  发布在  Hive
关注(0)|答案(0)|浏览(200)

我已经用所需的数据创建了Dataframe,正在尝试写入hivedb。我得到这个错误。df2的结构是这样的

root
 |-- hashval: string (nullable = true)
 |-- ftr_[0]: string (nullable = true)
 |-- ftr_[1]: string (nullable = true)
 |-- ftr_[2]: string (nullable = true)
 |-- ftr_[3]: string (nullable = true)
 |-- ftr_[4]: string (nullable = true)
 .
 .
 . 
 more

当我试图写入数据库时,我遇到了这个错误。不知道是什么引起的。数据不为空。我知道。

df2.registerTempTable("test")
spark.sql("DROP TABLE IF EXISTS  scdb.spec_test")
spark.sql("CREATE TABLE IF NOT EXISTS scdb.spec_test  as select * from test" )

这是什么错误

at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 77.0 
failed 4 times, most recent failure: Lost task 3.3 in stage 77.0 (TID 1614, hpcb04xxxx.hpc.xxx.com, 
executor 155): java.util.concurrent.ExecutionException: 
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 1220, Column 14: failed 
to dompile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 1220, Column 
 14: Expression "isNull_6" is not an rvalue     at org.spark_project.guava.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:552)

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题