我正在学习sparksql,并尝试在创建的数据集中应用过滤器。我定义了一个简单的employee case类,它有3个字段:name、salary和dpt。
case class Employee( name: String, salary: Double, age: Int, dpt: Dept)
最后一个字段dpt的定义如下:
sealed trait Dept extends { val name: String }
case object Accountability extends Dept { override val name = "AC"}
case object Sales extends Dept { override val name = "S"}
case object Finance extends Dept { override val name = "F"}
case object Marketing extends Dept { override val name = "M"}
case object Communication extends Dept { override val name = "C"}
case object Reception extends Dept { override val name = "R"}
case object HumanResource extends Dept { override val name = "HR"}
我已经尝试使用kryo编码器来解决它,但它不工作。
object DeptEncoders {
implicit def deptEncoder : org.apache.spark.sql.Encoder[Dept] = org.apache.spark.sql.Encoders.kryo[Dept]
}
1条答案
按热度按时间ax6ht2ek1#
根据以下文件: