csv 我的Python应用程序会同时从多个API端点请求数据吗?

juud5qan  于 2023-03-15  发布在  Python
关注(0)|答案(2)|浏览(137)

我们希望您开发一个小型Python应用程序,该应用程序可以同时从多个API端点请求数据,并将从每个端点接收到的数据保存在一个单独的CSV文件中。用于此目的的API位于:https://fakestoreapi.com请求#1将从相关API端点(GET /products)获取所有产品。请求#2将从相关API端点(GET /users)获取所有用户。接收到的数据将分别保存在products.csv和users.csv文件中。
我的密码正确吗?

import requests
import csv

# API endpoint URLs
products_url = "https://fakestoreapi.com/products"
users_url = "https://fakestoreapi.com/users"

# Send requests to API endpoints
products_response = requests.get(products_url)
users_response = requests.get(users_url)

# Check if requests are successful
if products_response.status_code != 200:
    print("Products request failed with status code:", products_response.status_code)
    exit()
    
if users_response.status_code != 200:
    print("Users request failed with status code:", users_response.status_code)
    exit()

# Save products data to CSV file
products_data = products_response.json()
with open("products.csv", "w", newline="") as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(products_data[0].keys())
    for product in products_data:
        writer.writerow(product.values())

# Save users data to CSV file
users_data = users_response.json()
with open("users.csv", "w", newline="") as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(users_data[0].keys())
    for user in users_data:
        writer.writerow(user.values())
qoefvg9y

qoefvg9y1#

我不确定你会如何并发地做这些,但是我认为你可以使用多处理,即使它对于这个简单的任务来说是极端的矫枉过正。
下面是我的解决方案:

import requests
import csv
from multiprocessing import Pool

# API endpoint URLs
products_url = "https://fakestoreapi.com/products"
users_url = "https://fakestoreapi.com/users"

def get_data(inp: tuple):
    url = inp[0]
    f_name = inp[1]

    r = requests.get(url)

    # Check if requests are successful
    if r.status_code != 200:
        print("Products request failed with status code:", r.status_code)
        return
    
    # Save products data to CSV file
    data = r.json()
    with open(f"{f_name}.csv", "w", newline="") as csvfile:
        writer = csv.writer(csvfile)
        writer.writerow(data[0].keys())
        for d in data:
            writer.writerow(d.values())
    
    return

if __name__ == '__main__':
    info = [(products_url, 'products',), (users_url, 'users',)]

    with Pool(2) as p: # 2 for 2 processes
        p.map(get_data, info)
neekobn8

neekobn82#

否,您的代码不符合所列出的要求。在您的代码中,您运行了以下函数:

products_response = requests.get(products_url)
users_response = requests.get(users_url)

但是,requests.get的第一次调用将停止第二次调用的执行,直到它完成。这意味着您的代码是 synchronous。您需要的是同时运行两个GET请求的 asynchronous(或多线程)代码。您可以使用threading库通过以下代码完成此操作,该库可以集成到当前代码中:

import requests
import threading

url1_response = ""
url2_response = ""

def get_url1():
    global url1_response
    response = requests.get("https://example.com/endpoint1")
    url1_response = response.text
    print(f"Response from endpoint 1: {response.text[:35]}...")

def get_url2():
    global url2_response
    response = requests.get("https://example.com/endpoint2")
    url2_response = response.text
    print(f"Response from endpoint 2: {response.text[:35]}...")

# Create threads for each URL
threads = []
thread = threading.Thread(target=get_url1)
threads.append(thread)
thread2 = threading.Thread(target=get_url2)
threads.append(thread2)

# Start threads
for thread in threads:
    thread.start()

print("Waiting for threads to finish...")
for thread in threads:
    thread.join()

# Process the data here

相关问题