I was practicing web scrapers and trying to pull the top 250 movies from IMDB. For some reason it only grabs the first 25 titles though. Anyone know why?
url = "https://www.imdb.com/chart/top/?ref_=nv_mv_250"
response = response = requests.get(url, headers={
"user-agent": "*THIS WAS A USER AGENT BUT I'M HIDING IT BECAUSE IDK IF A SHOULD PUT THAT INFO IN HERE*"
"Safari/537.36"})
html = response.content
soup = BeautifulSoup(html, 'html.parser')
titles = soup.find_all('a', class_="ipc-title-link-wrapper")
print(titles)
for title in titles:
title = title.find('h3', class_='ipc-title__text').get_text()
print(title)
New contributor
SpacemanSpiff is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.