这样就可以了，你可以决定是否排除行，我添加了一个额外的列并过滤掉，但选项是你的：

from pyspark.sql import functions as f

columns = ['id', 'text']
vals = [
        (1, ['I', 'am', 'good']),
        (2, ['You', 'are', 'ok']),
        (3, ['ok'])
       ]

df = spark.createDataFrame(vals, columns)
#df.show()

df2 = df.withColumn("text_left_over", f.expr("filter(text, x -> not(length(x) < 3))"))
df2.show()

# This is the actual piece of logic you are looking for.
df3 = df.withColumn("text_left_over", f.expr("filter(text, x -> not(length(x) < 3))")).where(f.size(f.col("text_left_over")) > 0).drop("text")
df3.show()

返回：

+---+--------------+--------------+
| id|          text|text_left_over|
+---+--------------+--------------+
|  1| [I, am, good]|        [good]|
|  2|[You, are, ok]|    [You, are]|
|  3|          [ok]|            []|
+---+--------------+--------------+

+---+--------------+
| id|text_left_over|
+---+--------------+
|  1|        [good]|
|  2|    [You, are]|
+---+--------------+

这就是解决办法

filter_length_udf = udf(lambda row: [x for x in row if len(x) >= 3], ArrayType(StringType()))
df_final_words = df_stemmed.withColumn('words_filtered', filter_length_udf(col('words')))