学习自定义Spark RDD Partitioner，编码了一些逻辑，但是编译不了。
在Spark 2.4.3中，启动spark shell：

case class Transaction(name:String, amount:Double, country:String)
val transactions = Seq(
 Transaction("Bob", 100, "UK"),
 Transaction("James", 15, "UK"),
 Transaction("Marek", 51, "US"),
 Transaction("Paul", 57, "US")
)

import org.apache.spark.Partitioner
class CountryPartitioner(override val numPartitions: Int) extends Partitioner { 
  def getPartition(key: Any): Int = key match { 
     case s: Transaction => s.country.hashCode % numPartitions  
  }  
  override def equals(other: Any): Boolean = other.isInstanceOf[CountryPartitioner]  
  override def hashCode: Int = 0
}

val rdd = sc.parallelize(transactions).partitionBy(new CountryPartitioner(2))

误差

error: value partitionBy is not a member of org.apache.spark.rdd.RDD[Transaction]
       rdd.partitionBy(new CountryPartitioner(2))
           ^

我从网上了解到，这段代码将工作没有任何错误.我的代码和这段代码几乎一样，区别是Transaction类...我不知道为什么我的代码不工作。即使我不能在线RDD API。

import org.apache.spark.Partitioner
class TwoPartsPartitioner(override val numPartitions: Int) extends Partitioner { def getPartition(key: Any): Int = key match { case s: String => {if (s(0).toUpper > 'J') 1 else 0 } }
override def equals(other: Any): Boolean = other.isInstanceOf[TwoPartsPartitioner]
override def hashCode: Int = 0
}

var x = sc.parallelize(Array(("sandeep",1),("giri",1),("abhishek",1),("sravani",1),("jude",1)), 3)
var y = x.partitionBy(new TwoPartsPartitioner(2))

来源：https://gist.github.com/girisandeep/f90e456da6f2381f9c86e8e6bc4e8260