使用pymysql将json数据从pandas插入到MySQL

hrirmatl  于 2023-08-01  发布在  Mysql
关注(0)|答案(1)|浏览(144)

我有一个csv文件,其中有json转储列表或dict列。
pictures["https://images1.loopnet.com/i2/-zlV--gguqB1DTY1kXtod7Y11T6NjYHuEOsdziUbkIc/110/106-Caines-St-Seward-AK-Primary-Photo-1-Large.jpg", "https://images1.loopnet.com/i2/9oRXPv2oL67ts0OBDiQq--j6wl2NaP2_fVpRG4WmPWs/110/106-Caines-St-Seward-AK-Interior-Photo-2-Large.jpg", "https://images1.loopnet.com/i2/Whv9igp7CuqG14CkzA5y6oHo7NA0mltLBMttvdhTd1Y/110/106-Caines-St-Seward-AK-Interior-Photo-3-Large.jpg", "https://images1.loopnet.com/i2/WmAM3YCIRbr7NtaP322I98DjvqGY7MMf_p8fyKsZhVs/110/106-Caines-St-Seward-AK-Interior-Photo-4-Large.jpg", "https://images1.loopnet.com/i2/dB7SmPhSBDinpHJOM5oRxg6sLSF7Bp_Pa5SdyeCQpl0/110/106-Caines-St-Seward-AK-Interior-Photo-5-Large.jpg"]的条目示例
financial_info{"Gross Rental Income": {"annual": null, "annual_per_sf": null}, "Other Income": {"annual": null, "annual_per_sf": null}, "Vacancy Loss": {"annual": null, "annual_per_sf": null}, "Effective Gross Income": {"annual": null, "annual_per_sf": null}, "Net Operating Income": {"annual": 99999.0, "annual_per_sf": 9.99}}示例
我想把它从csv加载到pandas数据框,并插入到MySQL数据库。我在MySQL中将这些列定义为json类型。
但我得到pymysql.err.ProgrammingError: (1064, 'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near \'["https://images1.loo...
我认为这是编码或引号的问题,但它是不工作时,我申请

import pandas as pd
from pymysql.converters import escape_string

data = pd.read_csv(filename, encoding="utf-8")
data["pictures"] = data["pictures"].apply(escape_string)

load_data_to_db(data)

字符串

bcs8qyzn

bcs8qyzn1#

问题可能与您将JSON数据插入MySQL数据库的方式有关。错误消息指示尝试插入JSON数据时语法有问题。
您可以直接使用pandas.to_json()方法将DataFrame中的JSON数据转换为JSON格式的字符串,而不是使用escape_string手动将JSON数据转换为字符串。
在这段代码中,我们使用pd.json.dumps将DataFrame中的JSON数据转换为JSON格式的字符串。然后,在将数据插入MySQL数据库时,我们使用%s作为JSON格式字符串的占位符,以避免任何SQL注入问题。
代码:

import pandas as pd
import pymysql

# Read the CSV file into a pandas DataFrame
data = pd.read_csv(filename, encoding="utf-8")

# Convert the JSON data in columns to JSON-formatted strings
data["pictures"] = data["pictures"].apply(pd.json.dumps)
data["financial_info"] = data["financial_info"].apply(pd.json.dumps)

# Assumed that connection was called before: "connection = pymysql.connect"
# Create a cursor object
cursor = connection.cursor()

# Loop through the DataFrame and insert each row into the MySQL database
for index, row in data.iterrows():
    sql = "INSERT INTO your_table (pictures, financial_info) VALUES (%s, %s)"
    values = (row["pictures"], row["financial_info"])
    cursor.execute(sql, values)

字符串
稍后,如果您想在Python中将JSON格式的字符串转换回JSON数据,您可以使用json模块提供的json.loads()函数。该函数解析JSON格式的字符串,并返回相应的JSON数据作为Python对象(例如字典或列表)。
从JSON格式的字符串返回到JSON数据的示例代码:

import json

# Example JSON-formatted string
json_str = '{"key": "value", "numbers": [1, 2, 3], "nested": {"foo": "bar"}}'

# Convert the JSON-formatted string to JSON data
json_data = json.loads(json_str)

# Now, json_data is a Python object containing the JSON data
print(json_data)
# Output: {'key': 'value', 'numbers': [1, 2, 3]}

# You can access the data as usual in Python
print(json_data["key"])  # Output: 'value'
print(json_data["numbers"])  # Output: [1, 2, 3]

相关问题