I want to scrape the inspect element data from a particular webpage, and parse through it to find the data I need.
import requests
from bs4 import BeautifulSoup
url = 'https://www.somewebpage.com'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.prettify())
This just ends up printing everything formatted rather than the raw data, and I’m trying to access the raw data and then parse through it to find what comes after the element I’m looking for.
Ethan is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
2
It is possible that the requests query didn’t get the full response. Usually, the websites send less data and the fetch the data slowly as the user browses the website to reduce page loading time. You can open in web browser and load the entire page first. Then open inspection tools and copy the code from elements or source tab
Vivid Programmer is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1