Question

How to get URL page title with scrapy and save it to a JSON file in Python

Answer 1

import scrapy

class Spider(scrapy.Spider):
    name = "SPIDER_NAME"

    start_urls = [
        'https://www.example.com/'
    ]

    def parse(self, response):
        yield {
            'title': response.css('title::text').get()
        }


# Windows 10
# Visual Studio Code - In TERMINAL

# scrapy crawl SPIDER_NAME -o data.json



''' 
run:

[
{"title": "Example Domain"}
]

'''

 

70+ SQL courses for beginners and professionals