bounty将在7天后过期。回答此问题可获得+100的声誉奖励。Arun.K正在寻找来自声誉良好来源的答案:能够通过代码传递分页中的下一个链接,并从所有页面获取数据
我正在尝试使用pyspark进行分页,并遇到以下错误。我的分页链接在标题中为Key [Link]和value [rel=“next”]。错误显示在此行r1 = requests.get(response.links['next'])。问题是传递的“next”URL中缺少baseURL。
getURL = 'https://api.xxx.com/v3/direct-access/abc'
baseURL = 'https://api.xxx.com/v3/direct-access'
headers = {
"accept" : "application/json",
"Content-Type": "application/json",
"Authorization": "Bearer " + str(token)
}
results = []
response = requests.get(getURL, headers=headers)
r = response.json()
for i in r:
results.append(i)
while response.links['next']: ## != response.links['last']:
r1 = requests.get(response.links['next'])
r = r1.json()
for i in r:
results.append(i)
Error: InvalidSchema: No connection adapters were found for "{'url': '/abc? action=next&next_page=%28id%2Ccompletionid%29+%3C+%28840430000754002%2C840430413029241%29&pagesize=10000', 'rel': 'next'}"
我怎样才能把baseURL和url合并成一个链接,然后把它传递给while循环?
https://api.xxx.com/v3/direct-access/abc?action=next&next_page=%28id%2Ccompletionid%29+%3C+%28840430000754002%2C840430413029241%29&pagesize=10000
1条答案
按热度按时间egdjgwm81#
像这样: