lit 不需要，因为第二个参数应该是要匹配的字符串模式，而不是列。

df.withColumn("STREET_NAME", regexp_replace($"STREET_NAME", "STREET", "ST"))

尝试 replace （或） regexp_replace() spark内置功能。

df.show()
//+-----------+-----+
//|STREET_NAME| NAME|
//+-----------+-----+
//|     1ST ST| NICK|
//| 2ND STREET|  SAM|
//|    3RD AVE| ERIC|
//| 4TH AVENUE|SARAH|
//+-----------+-----+
df.createOrReplaceTempView("tmp")
spark.sql("select replace(replace(STREET_NAME,'STREET','ST'),'AVENUE','AVE') as STREET_NAME,NAME from tmp").show()
//+-----------+-----+
//|STREET_NAME| NAME|
//+-----------+-----+
//|     1ST ST| NICK|
//|     2ND ST|  SAM|
//|    3RD AVE| ERIC|
//|    4TH AVE|SARAH|
//+-----------+-----+
//or using regexp_replace function
df.withColumn("STREET_NAME",regexp_replace(regexp_replace(col("STREET_NAME"),"STREET","ST"),"AVENUE","AVE")).show()
//+-----------+-----+
//|STREET_NAME| NAME|
//+-----------+-----+
//|     1ST ST| NICK|
//|     2ND ST|  SAM|
//|    3RD AVE| ERIC|
//|    4TH AVE|SARAH|
//+-----------+-----+