将 Dataframe 写入雪花错误:不支持以类型(时间戳)绑定数据

uqcuzwp8  于 2021-09-08  发布在  Java
关注(0)|答案(1)|浏览(530)

我有一个Pandas的数据框,我正试图写回雪花。但是,在尝试执行此操作时,我遇到以下错误:

(snowflake.connector.errors.ProgrammingError) 252004: Failed processing pyformat-parameters: 255001: Binding data in type (timestamp) is not supported.

我已尝试使用转换日期列 .dt.to_pydatetime() 但还是没有运气。
下面是我的代码


# write data back into table

# Fill in your SFlake details here

engine = create_engine(URL(
    account = '######',
    user = 'mike',
    authenticator = 'externalbrowser',
    database = 'DatbaseName',
    schema = 'SchemaName',
    warehouse = 'WarehouseName',
    role='RoleName',
))

connection = engine.connect()

# FIX ATTEMPT: Binding data in type (timestamp) is not supported in Snowflake, here is the work around:

df['month_end'] = df['month_end'].dt.to_pydatetime()
df['NEW_CUST_DT'] = df['NEW_CUST_DT'].dt.to_pydatetime()

df.to_sql('engagement', con=engine, index=False, if_exists='append') #make sure index is False, Snowflake doesnt accept indexes

connection.close()
engine.dispose()

Dataframe 详细信息供参考:

mm5n2pyu

mm5n2pyu1#

以下是可下载格式的代码。
我使用的csv股票数据,你可以从雅虎api。
月底和新客户被添加到数据中,以匹配您所做的转换。
我怀疑您的表上可能存在类型不匹配,或者源数据有问题。
尝试将数据发送到一个新表,看看是否有效。如果确实如此,请删除并重新创建表,或者修改 Dataframe 以匹配。您采取哪种策略将取决于您的代码库和用例的成熟程度。
“我的 Dataframe ”中的类型有:

date           datetime64[ns]
open                  float64
high                  float64
low                   float64
close                 float64
adjclose              float64
volume                  int64
ticker                 object
month_end      datetime64[ns]
NEW_CUST_DT    datetime64[ns]

测试上传的代码(对我有效)是:

import pandas as pd
import sqlalchemy as sa
import pandas as pd
from sqlalchemy import *
import snowflake.connector

def savedfToSnowflake(df):
    engine = sa.create_engine(
        'snowflake://{user}:{password}@{account}/'.format(
            user=cred['u'],
            password=cred['p'],
            account=cred['a'],
        )
    )
    try:
        connection = engine.connect()
        print("Connected to Snowflake ")
        df.to_sql('stockprice', con=engine, index=False,
                  if_exists='append')  # make sure index is False, Snowflake doesnt accept indexes
        print("Successfully saved data to snowflake")
    except Exception as ex:
        print("Exception occurred {}".format(ex))
    finally:
        # close connection
        if connection:
            connection.close()
        if engine:
            engine.dispose()

def getData():
    df = pd.read_csv('/sampledata/AA.csv')
    df = df.tail(20)
    df['date'] = pd.to_datetime(df['date'])
    df['month_end'] = df['date'].dt.to_pydatetime()
    df['NEW_CUST_DT'] = df['date'].dt.to_pydatetime()
    print(df.dtypes)
    return df

def main():
    df = getData()
    savedfToSnowflake(df)

if __name__ == '__main__':
    main()

我的结果表是

create or replace TABLE ENGAGEMENT (
DATE TIMESTAMP_NTZ(9),
OPEN FLOAT,
HIGH FLOAT,
LOW FLOAT,
CLOSE FLOAT,
ADJCLOSE FLOAT,
VOLUME NUMBER(38,0),
TICKER VARCHAR(16777216),
MONTH_END TIMESTAMP_NTZ(9),
NEW_CUST_DT TIMESTAMP_NTZ(9));

验证结果如下:

相关问题