在Python中将JSON转换为换行符分隔的JSON

6bc51xsx  于 2023-06-25  发布在  Python
关注(0)|答案(5)|浏览(137)

我的目标是用Python将JSON文件转换为可以从Cloud Storage上传到BigQuery(as described here)的格式。
我尝试使用newlineJSON包进行转换,但收到以下错误。

JSONDecodeError: Expecting value or ']': line 2 column 1 (char 5)

有人有办法解决这个问题吗?
下面是JSON代码示例:

[{
    "key01": "value01",
    "key02": "value02",
    ...
    "keyN": "valueN"
},
{
    "key01": "value01",
    "key02": "value02",
    ...
    "keyN": "valueN"
},
{
    "key01": "value01",
    "key02": "value02",
    ...
    "keyN": "valueN"
}
]

下面是现有的python脚本:

with nlj.open(url_samplejson, json_lib = "simplejson") as src_:
    with nlj.open(url_convertedjson, "w") as dst_:
        for line_ in src_:
            dst_.write(line_)
58wvjzkj

58wvjzkj1#

使用jq的答案确实很有用,但如果你仍然想用Python来实现(正如问题所显示的那样),你可以用内置的json模块来实现。

import json
from io import StringIO
in_json = StringIO("""[{
    "key01": "value01",
    "key02": "value02",

    "keyN": "valueN"
},
{
    "key01": "value01",
    "key02": "value02",

    "keyN": "valueN"
},
{
    "key01": "value01",
    "key02": "value02",

    "keyN": "valueN"
}
]""")

result = [json.dumps(record) for record in json.load(in_json)]  # the only significant line to convert the JSON to the desired format

print('\n'.join(result))

{"key01": "value01", "key02": "value02", "keyN": "valueN"}
{"key01": "value01", "key02": "value02", "keyN": "valueN"}
{"key01": "value01", "key02": "value02", "keyN": "valueN"}
  • 我在这里使用StringIOprint只是为了使示例更容易在本地测试。

作为替代方案,您可以使用Python jq binding将其与the other answer组合。

vhmi4jdf

vhmi4jdf2#

如果你想摆脱Python,可以使用jq

$ cat a.json 
[{
    "key01": "value01",
    "key02": "value02",
    "keyN": "valueN"
},
{
    "key01": "value01",
    "key02": "value02",
    "keyN": "valueN"
},
{
    "key01": "value01",
    "key02": "value02",
    "keyN": "valueN"
}
]

$ cat a.json | jq -c '.[]'
{"key01":"value01","key02":"value02","keyN":"valueN"}
{"key01":"value01","key02":"value02","keyN":"valueN"}
{"key01":"value01","key02":"value02","keyN":"valueN"}

我使用的迭代器是'.[]'来遍历数组,-c将每个JSON对象放在一行中。
资源:

gj3fmq9x

gj3fmq9x3#

获取JSON文件,转换成ND-JSON文件。

import json

with open("results-20190312-113458.json", "r") as read_file:
    data = json.load(read_file)
result = [json.dumps(record) for record in data]
with open('nd-proceesed.json', 'w') as obj:
    for i in result:
        obj.write(i+'\n')

希望这对某人有帮助。

t5fffqht

t5fffqht4#

with open('out.json', 'w') as f:
  for obj in objs:
    json.dump(obj, f)
    f.write('\n')
gz5pxeao

gz5pxeao5#

如果文件很大,你可能需要使用更快的模块,比如msgspecorjson,因为他们使用的是c(或者c++,我不完全知道,但有更快)
示例:

with open('test.json', 'rb') as f:
    j = msgspec.json.decode(f.read())
    
j = b'\n'.join(map(msgspec.json.encode, j))
with open('test.json', 'wb') as f:
    f.write(j)

相关问题