我用python bs4从amazon拉数据,但它没有从某些链接拉数据

bfrts1fy  于 2022-11-21  发布在  Python
关注(0)|答案(1)|浏览(107)

image here
image here
我扫描链接,并提请价格和标题的产品,但有时在一些网页上它不吸引任何产品,我猜它没有列出链接,我该如何修复它?
我给了你两张照片,有时候有,有时候没有。这是什么原因?
`

import requests
from bs4 import BeautifulSoup

pricelist = []
titlelist = []
productlist = []
countpage = 1

sk = "/s?k=HyperX+Cloud+II+Gaming+Kulakl%C4%B1k&page=1"
while True:

    url = f"https://www.amazon.com.tr{sk}"
    countpage+=1
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 OPR/91.0.4516.106"
    }

    request = requests.get(url,headers=headers)

    soup = BeautifulSoup(request.content,"html.parser")

    result = soup.findAll("div", {"class":"sg-col-4-of-24 sg-col-4-of-12 s-result-item s-asin sg-col-4-of-16 sg-col s-widget-spacing-small sg-col-4-of-20"})
    itemcounter = 0
    for item in result:
        try:

            itemprice = item.find("span", {"class":"a-offscreen"}).text.strip()
            itemtitle = item.find("span", {"class":"a-size-base-plus a-color-base a-text-normal"}).text.strip()
            f = open("read.txt","a+",encoding="utf-8")
            f.write(f"{itemprice} / {itemtitle} \n")
            itemcounter+=1
        except:
            pass
    print(itemcounter)
    after = soup.find("a", {"class":"s-pagination-item s-pagination-next s-pagination-button s-pagination-separator"}, href=True)
    try:

        sk = after["href"]
    except TypeError:
        
        break
    print(sk)

`
这是亚马逊的功劳吗?

xytpbqjk

xytpbqjk1#

result = soup.findAll("div", {"class":"sg-col-4-of-24 sg-col-4-of-12 s-result-item s-asin sg-col-4-of-16 sg-col s-widget-spacing-small sg-col-4-of-20"})

由于Amazon更改了某些页面上的类名称,
它的工作原理如下。

result = soup.findAll("div", {"class":"s-card-container s-overflow-hidden aok-relative puis-expand-height puis-include-content-margin puis s-latency-cf-section s-card-border"})

相关问题