如何在表达式中使用spark UDF?我有:
def power3(input: Int) = input * input * input
import org.apache.spark.sql.functions.udf
val power3UDF = udf(power3(_:Int):Int).withName("power3UDF")
numDF
.select(expr("Number"), expr("power3UDF(Number) AS POWER3"))
.show()
这会产生错误:
Undefined function: power3UDF. This function is neither a built-in/temporary function, nor a persistent function that is qualified as spark_catalog.default.power3udf.
1条答案
按热度按时间eimct9ow1#
需要注册函数spark的SQL函数: