使用Azure Databricks和Pyspark从Azure SQL表中删除行

azpvetkf  于 2023-08-02  发布在  Spark
关注(0)|答案(1)|浏览(107)

下面是我使用的片段

azuresqlOptions={ "driver":jdbcDriver, "url":jdbcUrl, "user":username, "port":jdbcPort, "password":password }

query = "(DELETE cone.address WHERE Address_ID=756 ) ad1" df1 = spark.read.format("jdbc").option("header","true").options(**azuresqlOptions).option("dbtable",query).load() display(df1)

字符串
我得到下面的错误:

com.microsoft.sqlserver.jdbc.SQLServerException: A nested INSERT, UPDATE, DELETE, or MERGE statement must have an OUTPUT clause.


有谁能帮我这个忙吗

cbjzeqam

cbjzeqam1#

在Azure数据块中从包含示例数据的SQL表中删除行时


的数据
下面的代码:

from pyspark.sql import SparkSession          
spark = SparkSession.builder.getOrCreate()    
Host = "<serverName>.database.windows.net"    
Port = 1433    
Database = "<dbName>"    
Username = "<userName>"    
Password = "<password>"    
Driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver"    
table = "<table>"    
Url = f"jdbc:sqlserver://{Host}:{Port};databaseName={Database}"

connectionProperties = {    
   "user": Username,    
    "password": Password,   
    "driver": Driver
}

query = f"(Delete {table} where Id = 1) AS subquery"
df = spark.read.jdbc(url=Url, table=query, properties=connectionProperties)
df.show()

字符串
我得到了同样的错误:



我尝试了下面的过程从表中删除行。我已经使用filter函数删除行:

from pyspark.sql.functions import *
df2 = df.filter(col("<condition>") 
df2.show()



我使用以下代码将 Dataframe 写入SQL表:

df2.write.format("jdbc").mode("overwrite").option("url", "<Url>").option("dbtable", ""<table>").option("user", "<Username>").option("password", "<Password>").save()


表已成功更新。


更新时间:

你可以使用下面的代码来执行删除查询:

import pyodbc
conn = pyodbc.connect( 'DRIVER={ODBC Driver 17 for SQL Server};'
                       'SERVER=<serverNmae>.database.windows.net;'
                       'DATABASE=<db>;UID=<userName>;'
                       'PWD=<password>')
conn.execute('DELETE <tableName> WHERE <condition>')

相关问题