I have a project with 10+ parsers and at the end have this code:
`
cursor = conn.cursor()
my_file = open(r'csv\file.csv')
sql_statement = """
CREATE TEMP TABLE temp
(
LIKE vhcl
)
ON COMMIT DROP;
COPY temp FROM STDIN WITH
CSV
HEADER
DELIMITER AS ',';
INSERT INTO vhcl
SELECT *
FROM temp
ON CONFLICT (id) DO UPDATE SET name= EXCLUDED.name"""
cursor.copy_expert(sql=sql_statement, file=my_file)
conn.commit()
cursor.close()
` Everything worked fine until a couple of weeks ago I started to get these errors:
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
I noticed, that if parsers works (for example) less, than 10 minutes, I won't get those errors
I tried to make a separate function, that adds data to the DB after the parser ends working. It still gives me that error. The strange thing is that I ran my parsers on my home pc, and it works fine, also, if I add data manually with the same function, but in a different file, it also works fine.
I asked about banned IP for db, but it's okay. So I have no idea why I have this error.
PostgreSQL log
2条答案
按热度按时间uidvcgyl1#
您的网络出现问题。
服务器和客户端都抱怨对方意外地挂断了它们的电话。所以是中间的某个配置错误的网络组件切断了线路。你有两个选择:
tcp_keepalives_idle
,以便操作系统频繁发送“保持活动数据包”,并且网络不会将连接视为空闲您可能需要阅读this以了解更多详细信息。
xuo3flqw2#
最后,我找到了一个解决方案。我仍然不知道是什么问题。这不是一个连接问题,因为一些解析器与相同的IP和相同的网络连接工作正常。我仍然能够添加数据与相同的脚本,但在一个单独的项目文件。
我的解决方案是在连接中添加“keepalives”设置: