scrapy 如何抓取跨度内的数据

3vpjnl9f  于 2023-05-07  发布在  其他
关注(0)|答案(1)|浏览(245)

我是新的刮,并希望检索数据从一个跨度,但他们没有一个类。
它们看起来像这样:

"<span content=\"3\">3 - 4 habitaciones</span>", "<span content=\"2\">2 baños</span>", "<span>Desde  89 m²</span>"
"<span content=\"4\">4 baños</span>", "<span>1.300 m²</span>"
"<span content=\"4\">4 baños</span>", "<span>1.300 m²</span>"

这是网站https://www.properati.com.ec/s/venta这是我试图刮scrap this info
有些人可能有所有的信息:

  • habitaciones(房间),
  • baños(浴室),

但有些没有,我想知道是否有一种方法,让每一个单独从刮阶段或如果我应该做一些魔术后

def parse(self, response):
    for products in response.css('div.listing.listing-card'):
        yield {
                'name': products.css('div.listing-card__title::text').get(),
                'location': products.css('div.listing-card__location::text').get(),
                'link': products.css('div.listing.listing-card').attrib['data-href'],
                'properties': products.css('span').getall()
            }

这些产量

"name": "Casa en Venta en Calderón", "location": "Calderón, Quito, Pichincha", "link": "/detalle/14032-32-e7e7-1f62d773dd45-dc2fc640-b378-120d", "properties": ["<span content=\"3\">3 habitaciones</span>", "<span content=\"3\">3 baños</span>", "<span>152 m²</span>"]`

我希望属性更像房间*浴室平方米**

thigvfpy

thigvfpy1#

当然,您可以为每个清单div迭代每个属性div,然后您可以运行使用一些条件表达式来测试属性是否通过查看图标类描述来描述,然后使用图标描述中的键将文本添加到生成的字典中。
例如:

def parse(self, response):
    for products in response.css('div.listing.listing-card'):
        item = {
            'name': products.css('div.listing-card__title::text').get(),
            'location': products.css('div.listing-card__location::text').get(),
            'link': products.css('div.listing.listing-card').attrib['data-href']
        }
        for prop in products.css('div.listing-card__property'):
            if prop.css('div.card-icon__area').get():
                item["area"] = prop.css('span::text').get()
            elif prop.css('div.card-icon__bathrooms').get():
                item["bathrooms"] = prop.css('span::text').get()
            elif prop.css('div.card-icon__bedrooms').get():
                item["bedrooms"] = prop.css('span::text').get()
        yield item

或者,如果你想变得更聪明,你可以跳过if语句,直接提取属性divs图标类描述的键,以获得相同的结果:

def parse(self, response):
    for products in response.css('div.listing.listing-card'):
        item = {
            'name': products.css('div.listing-card__title::text').get(),
            'location': products.css('div.listing-card__location::text').get(),
            'link': products.css('div.listing.listing-card').attrib['data-href']
        }
        for prop in products.css('div.listing-card__property'):
            item[prop.xpath('./div/@class').get().split('__')[-1]] = prop.xpath("./span/text()").get()
        yield item

输出:

{'name': 'Conjunto Residencial Altamira', 'location': 'Distrito Metropolitano de Quito, Pichincha', 'link': '/proyecto/14032-32-c100-2e2644d36103-24d37585-809a-47ef', 'bedrooms': '2 -
 3 habitaciones', 'bathrooms': '2 - 3 baños', 'area': 'Desde  104 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'San Pio de Pietrelcina', 'location': 'Guayaquil, Guayas', 'link': '/proyecto/14032-32-93a-697855c16d29-d4bad5ea-8795-4935', 'bedrooms': '3 habitaciones', 'bathrooms': '4 bañ
os', 'area': 'Desde  118 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Los Lojas', 'location': 'Los Lojas, Daule, Guayas', 'link': '/detalle/14032-32-2315-b5496c787991-fd643c70-8aa8-120d', 'bedrooms': '5 habitaciones', 'bathroo
ms': '5 baños', 'area': '144 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-66d3-7be660cb0540-fc65e6e8-ac23-4a87', 'bedrooms': '3 habitaciones', 'bathrooms': '4
,5 baños', 'area': '159 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-9e00-e189ea1d44de-fc52fb5f-a0b8-406d', 'bedrooms': '2 habitaciones', 'bathrooms':
 '3 baños', 'area': '101 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-3afb-450a26e47b70-fa0693ec-bf04-4442', 'bedrooms': '4 habitaciones', 'bathrooms':
 '2 baños', 'area': '344 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Terreno en Venta en Cuenca', 'location': 'Cuenca, Azuay', 'link': '/detalle/14032-32-baed-bc973d006d88-f50dc4a7-acce-308c', 'area': '1.755 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en General Vernaza', 'location': 'General Vernaza, Salitre, Guayas', 'link': '/detalle/14032-32-e9de-13e2841b36ee-ef522710-8eb0-120d', 'bedrooms': '3 habitacio
nes', 'bathrooms': '3,5 baños', 'area': '152 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Portoviejo', 'location': 'Portoviejo, Manabí', 'link': '/detalle/14032-32-3f85-18d8fe00a459-ef4fdde4-9faa-463f', 'bedrooms': '2 habitaciones', 'bathrooms':
'2,5 baños', 'area': '137 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Terreno en Venta en Guayllabamba', 'location': 'Guayllabamba, Quito, Pichincha', 'link': '/detalle/14032-32-e954-d1af2422949-e9dadfa0-9438-120d'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Anconcito', 'location': 'Anconcito, Salinas, Santa Elena', 'link': '/detalle/14032-32-e944-3104d79522ed-e8bf02a0-bfba-120d', 'area': '420 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-4367-1fb3aa53ed81-e85785ef-b912-4c47', 'bedrooms': '3 habitaciones', 'bathrooms': '2
,5 baños', 'area': '200 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-5c8f-db8a03a69e6-e4dddcf8-b0ff-3213', 'bedrooms': '2 habitaciones', 'bathrooms':
'2 baños', 'area': '65 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Terreno en Venta en Tabacundo', 'location': 'Tabacundo, Pedro Moncayo, Pichincha', 'link': '/detalle/14032-32-1022-6eea6bb08512-e35450f5-b1a6-41a2'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-7228-df85a4c2b992-e1d7e6b4-a805-4e87', 'bedrooms': '2 habitaciones', 'bathrooms':
 '1 baño', 'area': '55.560 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-fe85-1305e21ff6e4-e199f8c1-9631-423f', 'bathrooms': '1 baño', 'area': '105 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-c7ba-9a911a65a6ad-e0f3f5a2-b0f6-49b6'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Cuenca', 'location': 'Cuenca, Azuay', 'link': '/detalle/14032-32-2835-477f50fcb9d-e0e82005-90c0-484a', 'bedrooms': '5 habitaciones', 'bathrooms': '2 baños',
 'area': '354 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-d411-690205fb726c-e00b4f30-a841-120d', 'bedrooms': '3 habitaciones', 'bathrooms': '2
 baños', 'area': '120 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-5934-4f8121111b29-dfc7b8fe-bb17-4cb9', 'bedrooms': '1 habitación', 'bathrooms': '
1 baño', 'area': '36 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Cuenca', 'location': 'Cuenca, Azuay', 'link': '/detalle/14032-32-bc13-c159cd97536f-dcd8ce25-ab34-406c', 'area': '380 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Los Lojas', 'location': 'Los Lojas, Daule, Guayas', 'link': '/detalle/14032-32-d1b-24ebe8ff2152-db522c3a-bee3-416d', 'bedrooms': '3 habitaciones', 'bathroom
s': '3,5 baños', 'area': '120 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Cuenca', 'location': 'Cuenca, Azuay', 'link': '/detalle/14032-32-94e8-556dcdf71a41-d8d648f4-8e63-39a8', 'bedrooms': '36 habitaciones', 'bathrooms': '18 baño
s', 'area': '1.650 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-a9c1-58e4b4ba0790-d6362d4d-a2c8-44b1', 'bedrooms': '3 habitaciones', 'bathrooms': '2,5 ba
ños', 'area': '169 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Local comercial en Venta en Quito', 'location': 'Quito, Pichincha', 'link': '/detalle/14032-32-b56-c894b8ba0654-d1264979-a9fd-4414', 'area': '120 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Terreno en Venta en Tenguel', 'location': 'Tenguel, Guayaquil, Guayas', 'link': '/detalle/14032-32-7213-821cdfd49ed9-d079cf3d-8ff2-398f', 'bedrooms': '1 habitación', 'bathro
oms': '1 baño', 'area': '160 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Cayambe', 'location': 'Cayambe, Pichincha', 'link': '/detalle/14032-32-7952-815d679024b7-cfab17ea-9111-4c5d'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Jadan', 'location': 'Jadan, Azuay', 'link': '/detalle/14032-32-c742-bd20679efadd-cfa8cf38-98c0-3549', 'bedrooms': '4 habitaciones', 'bathrooms': '3 baños',
'area': '220 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-69ac-7c9ac8abf614-cc4aec92-bf79-4b06', 'bedrooms': '2 habitaciones', 'bathro
oms': '2 baños', 'area': '139 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Departamento en Venta en Guayaquil', 'location': 'Guayaquil, Guayas', 'link': '/detalle/14032-32-4a60-de3924004276-cbd2304d-9a6b-4c4e', 'bedrooms': '3 habitaciones', 'bathro
oms': '6 baños', 'area': '261 m²'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Terreno en Venta en Calderón', 'location': 'Calderón, Quito, Pichincha', 'link': '/detalle/14032-32-ad6b-ce7536f8d67d-c8e83426-b5d0-494e'}
2023-05-03 12:36:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.properati.com.ec/s/venta>
{'name': 'Casa en Venta en Tenguel', 'location': 'Tenguel, Guayaquil, Guayas', 'link': '/detalle/14032-32-2cbb-378545d5e0df-c879e5b6-80f3-3819', 'bedrooms': '4 habitaciones', 'bathroo
ms': '9 baños', 'area': '420 m²'}

相关问题