使用 explain() 看看会发生什么。

df.explain()

== Physical Plan ==
Union
:- *(5) Project [ID#3599L, VALUE#3600, VALUE#3604]
:  +- *(5) SortMergeJoin [ID#3599L], [ID#3603L], Inner
:     :- *(2) Sort [ID#3599L ASC NULLS FIRST], false, 0
:     :  +- Exchange hashpartitioning(ID#3599L, 200), true, [id=#1332]
:     :     +- *(1) Filter isnotnull(ID#3599L)
:     :        +- *(1) Scan ExistingRDD[ID#3599L,VALUE#3600]
:     +- *(4) Sort [ID#3603L ASC NULLS FIRST], false, 0
:        +- ReusedExchange [ID#3603L, VALUE#3604], Exchange hashpartitioning(ID#3599L, 200), true, [id=#1332]
+- *(10) Project [ID#3599L, VALUE#3600, VALUE#3609]
   +- *(10) SortMergeJoin [ID#3599L], [ID#3608L], Inner
      :- *(7) Sort [ID#3599L ASC NULLS FIRST], false, 0
      :  +- ReusedExchange [ID#3599L, VALUE#3600], Exchange hashpartitioning(ID#3599L, 200), true, [id=#1332]
      +- *(9) Sort [ID#3608L ASC NULLS FIRST], false, 0
         +- ReusedExchange [ID#3608L, VALUE#3609], Exchange hashpartitioning(ID#3599L, 200), true, [id=#1332]

sort、sortmergejoin、project和union。这发生在你采取行动时，比如 count() 在那之前，spark将计划如何继续。