做你想做的事的有效而简单的方法是使用 Dataframes 而不是rdd
假设你有这样的数据-

table1 = [(1,'A'),(2,'B'),(3,'C'),(4,'B')]
table2 = [('A',10),('B',20),('D',30),('E',40)]

# create the dataframes based on the data

df1 = spark.createDataFrame(table1,schema=['k1','v1'])
df2 = spark.createDataFrame(table2,schema=['k2','v2'])

df1.show() 
+---+---+
| k1| v1|
+---+---+
|  1|  A|
|  2|  B|
|  3|  C|
|  4|  B|
+---+---+

 df2.show() 
+---+---+
| k2| v2|
+---+---+
|  A| 10|
|  B| 20|
|  D| 30|
|  E| 40|
+---+---+

# do a simple inner join and only select df2 columns

df2\
.join(df1, df1.v1==df2.k2)\
.select(df2.columns)\
.dropDuplicates()
.show()

+---+---+
| k2| v2|
+---+---+
|  B| 20|
|  A| 10|
+---+---+

只是使用RDD-

rdd1 = sc.parallelize(table1)
rdd2 = sc.parallelize(table2)

rdd2\
.join(rdd1.map(lambda x : (x[1],x[0])))\
.mapValues(lambda x: x[0])\
.distinct()\
.collect()