I want to create a web crawler to extract data from NSE India (Indian stock exchange website). The project’s main purpose is to fetch all company-related announcements (PDFs) for a given list of companies. This data will be later stored on the cloud for further use.

This information will be later uploaded on some other site. This site will also provide watchlist features such as notifying the user if any companies in their watchlist make new announcements.

So the first step of this project is to scrape data from the site, which is the company name and announcement pdf. For this project should I use BS4 or Scrapy? I am new to programming, however I know BS4 basics but have no clue about Scrapy. If someone can guide me through this I would be really thankful.