你可以用 when 用于检查列 reason 匹配dict键。您可以动态生成 when 使用python的表达式 functools.reduce 通过传递列表 myDict.keys() :

from functools import reduce
from pyspark.sql import functions as F

df2 = df.withColumn(
    "new_reason",
    reduce(
        lambda c, k: c.when(F.lower(F.col("reason")).rlike(rf"\b{k.lower()}\b"), myDict[k]),
        myDict.keys(),
        F
    )
)

df2.show(truncate=False)

# +---+-------------------------+-------------------+

# |id |reason                   |new_reason         |

# +---+-------------------------+-------------------+

# |1  |Needed better Support    |Support Issue      |

# |2  |Better value from android|Left for Competitor|

# |3  |Price was to expensive   |Pricing Issue      |

# |4  |Support problems         |Support Issue      |

# +---+-------------------------+-------------------+

您可以创建关键字dataframe，并使用 rlike 条件。我补充道 \\\\b 在关键字之前和之后，只有单词边界之间的单词才会匹配，并且不会有部分单词匹配（例如，“菠萝”匹配“苹果”）。

import pyspark.sql.functions as F

keywords = spark.createDataFrame([[k,v] for (k,v) in myDict.items()]).toDF('key', 'new_reason')

result = df.join(
    keywords, 
    F.expr("lower(reason) rlike '\\\\b' || lower(key) || '\\\\b'"), 
    'left'
).drop('key')

result.show(truncate=False)
+---+-------------------------+-------------------+
|id |reason                   |new_reason         |
+---+-------------------------+-------------------+
|1  |Needed better Support    |Support Issue      |
|2  |Better value from android|Left for Competitor|
|3  |Price was to expensive   |Pricing Issue      |
|4  |Support problems         |Support Issue      |
+---+-------------------------+-------------------+