如何用python scrapy提取没有标记特定文本？(新问题)

nsc4cvqm 于 2022-11-09 发布在 Python

关注(0)|答案(1)|浏览(315)

我使用scrapy来提取html中的目标文本，如下所示：

我的代码是：

import scrapy
from scrapy.crawler import CrawlerProcess
class MmSpider(scrapy.Spider):
    name = 'name'
    start_urls = ['file:///Users/saihhold/Desktop/maimai.mht']

    def parse(self, response):
        for title in response.xpath('//div[@class="media-body"]/div/div[1]'):
            yield {
                title.xpath('.//text()').getall()
            }

if __name__ == "__main__":
    process = CrawlerProcess()
    process.crawl(MmSpider)
    process.start()

然后使用以下命令运行它：

scrapy runspider mmspider.py -o mm.jl

但是mm.jl文件是空，我的代码或xpath有什么问题吗？

scrapy

来源：https://stackoverflow.com/questions/71839254/how-to-extract-a-specific-text-with-no-tag-by-python-scrapy-new-problem

1条答案

按热度按时间

vltsax251#

你的代码是好的，但是xpath选择不正确。你可以按照下一个例子来使用xpath获取标题。

import scrapy
from scrapy.crawler import CrawlerProcess
class MmSpider(scrapy.Spider):
    name = 'name'
    start_urls = ['https://www.timeout.com/film/best-movies-of-all-time']

    def parse(self, response):
        for title in response.xpath('//h3[@class="_h3_cuogz_1"]'):
            yield {
                'title':title.xpath('.//text()').getall()[-1].replace('\xa0','')
            }

if __name__ == "__main__":
    process = CrawlerProcess()
    process.crawl(MmSpider)
    process.start()

输出量：

{'title': '2001: A Space Odyssey (1968)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'The Godfather (1972)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'Citizen Kane (1941)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'Jeanne Dielman, 23, Quai du Commerce, 1080 Bruxelles (1975)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'Raiders of the Lost Ark (1981)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'La Dolce Vita (1960)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'Seven Samurai (1954)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'In the Mood for Love (2000)'}
2022-04-12 15:17:36 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.timeout.com/film/best-movies-of-all-time>
{'title': 'There Will Be Blood (2007)'}

...等等

赞(0）回复(0）举报 2022-11-09

我来回答

如何用python scrapy提取没有标记特定文本？(新问题)

1条答案

相关问题

热门标签

最新问答