下面的代码可能有助于解决您的问题，

import pyspark.sql.functions as F

input_row=[["2017-05-05","2016-10-15"],["2019-06-22","2020-02-06"],["2020-10-09","2020-10-09"]]

df = spark.createDataFrame(input_row,["date1","date2"])
df.show()

'''
+----------+----------+
|     date1|     date2|
+----------+----------+
|2017-05-05|2016-10-15|
|2019-06-22|2020-02-06|
|2020-10-09|2020-10-09|
+----------+----------+
'''

方法1：使用spark sql和 month 以及 dayofmonth 内置方法

df.createOrReplaceTempView("test_table")

spark.sql("select *,case when ((month(CAST(`date1` AS DATE)) > month(CAST(`date2` AS DATE))) and (dayofmonth(CAST(`date1` AS DATE))> dayofmonth(CAST (`date2` AS DATE)))) then 'Y' when ((month(CAST(`date1` AS DATE)) < month(CAST(`date2` AS DATE))) and (dayofmonth(CAST(`date1` AS DATE))< dayofmonth(CAST (`date2` AS DATE)))) then 'N'  else 'E' end as FLAG from test_table").show()

'''
+----------+----------+----+
|     date1|     date2|FLAG|
+----------+----------+----+
|2017-05-05|2016-10-15|   N|
|2019-06-22|2020-02-06|   Y|
|2020-10-09|2020-10-09|   E|
+----------+----------+----+
'''

方法2。使用spark sql datediff 功能

df.createOrReplaceTempView("test_table")

spark.sql("select *, case when datediff(date1, date2)>0 then 'Y' when datediff( date1, date2 )<0 then 'N' else 'E' end as FLAG from test_table").show()

'''
+----------+----------+----+
|     date1|     date2|FLAG|
+----------+----------+----+
|2017-05-05|2016-10-15|   Y|
|2019-06-22|2020-02-06|   N|
|2020-10-09|2020-10-09|   E|
+----------+----------+----+
'''

三。另一种方法是使用df表达式而不是sparksql，

df.select(df["*"], F.expr("CASE WHEN datediff( date1, date2 )>0 THEN 'Y' " + "WHEN datediff( date1, date2 )<0 then 'N' " + "else 'E' end").alias("FLAG")).show()

'''
+----------+----------+----+
|     date1|     date2|FLAG|
+----------+----------+----+
|2017-05-05|2016-10-15|   Y|
|2019-06-22|2020-02-06|   N|
|2020-10-09|2020-10-09|   E|
+----------+----------+----+
'''