Scrapy：将Javascript数组转换为Python上的Json

6mw9ycah 于 2022-11-09 发布在 Java

关注(0)|答案(2)|浏览(167)

我一直在努力与一个网站，我正在报废使用scrappy.这个网站，返回一系列的Javascript变量（数组）与产品数据.例如：

datos[0] = ["12345","3M YELLOW CAT5E CABLE","6.81","1","A","N","N","N","N","N",0,0,0,0,0,"0","0","0","0","0","P","001-0030","12","40K8957","28396","250","Due: 30-12-1899",0.0000,1,"",\'\'];
datos[1] = ["12346","3M GREEN CAT5E CABLE","7.81","1","A","N","N","N","N","N",0,0,0,0,0,"0","0","0","0","0","P","001-0030","12","40K8957","28396","250","Due: 30-12-1899",0.0000,1,"",\'\'];
...

所以......
用Scrapy把数组取到字符串中很容易，因为站点响应会打印变量。问题是我想把它转换成Json，这样我就可以处理它并把它存储在数据库表中。
通常我会使用Javascript的函数Json.stringify将其转换为Json，然后用PHP发布。
然而，当使用Python的json.loads甚至StringIO时，我无法将数组加载到json中。
可能是一个格式错误，但我无法识别它，因为我不是Json或PythonMaven。
编辑：我刚刚意识到，既然Scrapy不能执行Javascript，可能主要的问题是数据只是一个字符串。我应该把它格式化为Json格式。
任何帮助都是非常欢迎的。

谢谢-谢谢

scrapy

来源：https://stackoverflow.com/questions/72665122/scrapy-converting-javascript-array-to-json-on-python

2条答案

按热度按时间

yuvru6vn1#

如果你想接受一个数组并创建一个json对象，你可以这样做。

values = ["12345","3M YELLOW CAT5E CABLE","6.81","1","A","N","N","N","N","N",0,0,0,0,0,"0","0","0","0","0","P","001-0030","12","40K8957","28396","250","Due: 30-12-1899",0.0000,1]
keys = [x for x in range(len(values))]
d = dict(zip(keys, values))
x = json.dumps(d)

赞(0）回复(0）举报 2022-11-09

0ve6wy6x2#

在scrapy文档中有一节介绍了解析JavaScript代码的各种方法。对于您的情况，如果您只需要将其放在数组中，则可以使用正则表达式来获取数据。
由于你正在刮的网站是不存在的问题，我假设这将是一个更直接的方式来获得它，但你可以使用任何方式似乎合适。

赞(0）回复(0）举报 2022-11-09