Apache-Spark:如何在expr中使用自定义项?

46scxncf  于 2022-11-16  发布在  Apache
关注(0)|答案(1)|浏览(139)

如何在表达式中使用spark UDF?我有:

def power3(input: Int) = input * input * input

import org.apache.spark.sql.functions.udf
val power3UDF = udf(power3(_:Int):Int).withName("power3UDF")

numDF
  .select(expr("Number"), expr("power3UDF(Number) AS POWER3"))
  .show()

这会产生错误:

Undefined function: power3UDF. This function is neither a built-in/temporary function, nor a persistent function that is qualified as spark_catalog.default.power3udf.
eimct9ow

eimct9ow1#

需要注册函数spark的SQL函数:

spark.udf.register("power3", power3(_:Int):Int)

相关问题