I need to scrap data instantly (5-10sec) but my code works 60-80sec.
if have better and faster idea, please let me know. it does not matter whether by selenium or even another language
Here is my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
import pandas as pd
import time
start = time.time()
driver = webdriver.Chrome()
driver.get("https://www.investing.com/economic-calendar/")
# Scrape the data
events = []
# Locate the rows in the table
rows = driver.find_elements(By.XPATH, '/html/body/div[6]/section/div[6]/table/tbody/tr[14]')
for row in rows:
try:
actual = row.find_element(By.XPATH, './td[5]').text
previous = row.find_element(By.XPATH, './td[7]').text
events.append([actual, previous])
except Exception as e:
print(f"Error processing row: {e}")
driver.quit()
df = pd.DataFrame(events, columns=[ 'Actual', 'Previous'])
df.head()
df.to_csv('economic_calendar.csv', index=False)
end = time.time()
print(end - start)
New contributor
Ismoiljon Jo’rayev is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.