Here are my codes:
`import pandas as pd
import requests
from bs4 import BeautifulSoup
url = "https://www.payscale.com/college-salary-report/majors-that-pay-you-back/bachelors/"
response = requests.get(url)
soup_job = response.text
soup = BeautifulSoup(soup_job, "html.parser")
table = soup.find_all('table', class_="data-table")
print(table)`
`Even when I do this, it is still not working. Is there anyone who can help me, please?’
page=1
while page <= 34:
response = requests.get(
f"https://www.payscale.com/college-salary-report/majors-that-pay-you-back/bachelors/page/{page}")
page += 1
I have used selenium to get that done, but it is still not working.
I’m trying to see if someone can review and provide some hint on how to get that done.
import pandas as pd
import requests
from bs4 import BeautifulSoup
url = "https://www.payscale.com/college-salary-report/majors-that-pay-you-back/bachelors"
response = requests.get(url)
soup_job = response.text
soup = BeautifulSoup(soup_job, "html.parser")
table = soup.find_all('table', class_="data-table")
print(table)
When I print the table, I expected to get some text or data, but it gives me an empty list.
I have changed the class name to see if I can get something better; it is showing me the same issue, and when I tried to use find instead of find all, it returned none instead of an empty list.
I want to get the data inside the table there, and after that, I will be able to tract the head and the body of the table to get the data I want; nothing works for me so far.
Dave is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1