This code below can extract all links from the site,
from bs4 import BeautifulSoup
import requests
r=requests.get("https://www.drishtiias.com/current-affairs-news-analysis-editorials")
soup=BeautifulSoup(r.content,"html.parser")
links = soup.find_all("a")
for link in links:
print("Link:", link.get("href"), "Text:", link.string)
but I want to extract links from certain part of a site.
I tried this
links = soup.find_all("a", class_='box-hide')
but it is not working
the href i need tag is in
/html/body/section[1]/div[2]/div/article/div[1]/div[1]/div/div/ul/li[2]/a
Also is there a way to export these links to text file and stop the loop after extracting certain no. of links.
Thanks
New contributor
Abhay Malhotra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.