I’m trying to scrape the data from MyNeta, I want to specifically scrape the data of the table provided by MyNeta. I’m able to scrape only the headers of the table but unable to scrape information of the table.
import requests
from bs4 import BeautifulSoup
<code>def scrape_url_structure(url):
response = requests.get(url)
if response.status_code == 200:
soup = BeautifulSoup(response.content, 'html.parser')
table = soup.find("table", class_="w3-table w3-bordered")
if table:
rows = table.find_all('tr')
for row in rows:
cells = row.find_all(['th', 'td'])
for cell in cells:
print(cell.get_text(strip=True))
print('-' * 20)
else:
print("Table not found on the page.")
def main():
url = "https://www.myneta.info/LokSabha2024/index.php? action=summary&subAction=candidates_analyzed&sort=candidate&page=1"
print("Scraping URL structure:")
scrape_url_structure(url)
</code>
<code>def scrape_url_structure(url):
response = requests.get(url)
if response.status_code == 200:
soup = BeautifulSoup(response.content, 'html.parser')
table = soup.find("table", class_="w3-table w3-bordered")
if table:
rows = table.find_all('tr')
for row in rows:
cells = row.find_all(['th', 'td'])
for cell in cells:
print(cell.get_text(strip=True))
print('-' * 20)
else:
print("Table not found on the page.")
def main():
url = "https://www.myneta.info/LokSabha2024/index.php? action=summary&subAction=candidates_analyzed&sort=candidate&page=1"
print("Scraping URL structure:")
scrape_url_structure(url)
</code>
def scrape_url_structure(url):
response = requests.get(url)
if response.status_code == 200:
soup = BeautifulSoup(response.content, 'html.parser')
table = soup.find("table", class_="w3-table w3-bordered")
if table:
rows = table.find_all('tr')
for row in rows:
cells = row.find_all(['th', 'td'])
for cell in cells:
print(cell.get_text(strip=True))
print('-' * 20)
else:
print("Table not found on the page.")
def main():
url = "https://www.myneta.info/LokSabha2024/index.php? action=summary&subAction=candidates_analyzed&sort=candidate&page=1"
print("Scraping URL structure:")
scrape_url_structure(url)
if name == “main“:
main()
The output comes as :
Scraping URL structure:
Sno
Candidate∇
Constituency
Party
Criminal Case
Education
Total Assets
Liabilities
New contributor
Maitreya is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.