pyspark 如何将T-SQL CASE WHEN STATEMENT转换为Databricks sparkSQL

mnemlml8  于 2023-06-21  发布在  Spark
关注(0)|答案(1)|浏览(142)

我尝试在sparkSQL中编写一个典型的T-SQL CASE WHEN语句。
T-SQL CASE WHEN语句如下所示:

SELECT
  *
 ,CASE
    WHEN DatesTestv3.as_of_date = EOMONTH(DATEADD(MONTH, -1, SYSDATETIME())) THEN 'LM'
  END AS 'Last Month End'
from dbo.DatesTestv3

我尝试将上面的T-SQL转换为sparkSQL如下:

spark.sql("select *, case when as_of_date = last_day(add_months(current_date(),-1)) then 'LM' END AS 'Last Month End" from table)

但是,我得到了ParseException错误。
我相信这可能是一个简单的语法问题。
因此,对于如何将T-SQL转换为sparkSQL有什么想法吗?

mqkwyuun

mqkwyuun1#

据我所知,我已经在我的Pyspark中尝试了相同的CASE语句。
我已经创建了下面的列表。id,as_of_date,last_month_end

from pyspark.sql.functions import to_date
spark.sql("CREATE TABLE data020 (id INT, as_of_date DATE, last_month_end STRING)
data = [
(1, "2023-06-01", "MA"),
(2, "2023-06-01", "MA"),
(3, "2023-06-01", "MA"),
(4, "2023-06-01", "MA"),
(5, "2023-06-01", "MA"),
(6, "2023-06-01", "MA"),
(7, "2023-06-01", "MA"),
(8, "2023-06-01", "MA"),
(9, "2023-06-01", "MA"),
(10, "2023-06-01", "MA")
]
df = spark.createDataFrame(data, ["id", "as_of_date", "last_month_end"])
df = df.withColumn("as_of_date", to_date(df.as_of_date))
df.write.mode("overwrite").insertInto("data020")

x1c 0d1x我试过的CASE语句是这样的。我尝试了两种不同的方式。

from pyspark.sql.functions import last_day, date_sub, current_date, when
data = spark.table("data_dry")
data = data_dry.withColumn("Last Month End", when(data.as_of_date == last_day(date_sub(current_date(), 1)), "LM").otherwise(None))
data_dry.display()

然后呢

result = spark.sql("""

SELECT *,CASE WHEN as_of_date = last_day(date_add(current_date(),-1))THEN 'LM' ELSE NULL END AS Last Month End FROM data_dry“"”)
result.display()


也可以用**Date_trunc**函数试试。

相关问题