I’m trying to run below scrapy code and it’s running fine. But the problem is even after updating the code, it is returning the same output. Infact, if I delete the entire content of this file and ran the spider, it’s returning the same output as earlier. It seems like, the output is not getting updated as per the contents of the code file. Please do let me know, how to return the updated output.
import scrapy
class BookspiderSpider(scrapy.Spider):
name = "bookspider"
allowed_domains = ["books.toscrape.com"]
start_urls = ["https://books.toscrape.com"]
def parse(self, response):
books = response.css('article.product_pod')
for book in books:
yield{
'name' : book.css('h3 a::text'),
'Price' : book.css('.product_price .price_color::text'),
'URL' : book.css('h3 a').attrib['href']
}
next_page = response.css('li.next a ::attr(href)').get()
if next_page is not None:
if 'catalogue/' in next_page:
nextPageURL = 'https://books.toscrape.com/' + next_page
else:
nextPageURL = 'https://books.toscrape.com/catalogue/' + next_page
yield response.follow(nextPageURL, callback = self.parse)
Problem:
Output of the code is not getting updated even after making code changes in the file.
Expectation:
Output should be returned as per the code file