I am scraping a job website with selenium and I am getting incorrect values when it comes to salary. All other values are correct, just a mix up with the salary.
This is my code
from selenium import webdriver
from selenium.webdriver.common.by import By
import pandas as pd
url = 'https://www.glassdoor.com/Job/remote-data-analyst-jobs-SRCH_IL.0,6_IS11047_KO7,19.htm'
driver = webdriver.Edge()
driver.get(url)
job_title = []
company_name = []
job_location = []
salary_estimate = []
container = driver.find_element(By.CLASS_NAME, "JobsList_wrapper__EyUF6")
mini_container = container.find_elements(By.CLASS_NAME, "JobCard_jobCardContainer___hKKI")
for x in mini_container:
job_title.append(x.find_element(By.XPATH, './/a[contains(@id, "job-title")]').text)
company_name.append(x.find_element(By.XPATH, './/div[contains(@class, "EmployerProfile")]').text)
job_location.append(x.find_element(By.XPATH, './/div[contains(@id, "job-location")]').text)
try:
salary_estimate.append(x.find_element(By.XPATH, './/div[contains(@id, "job-salary")]').text)
except:
pass
print(x.text)
jobs = {
'job_title' : job_title,
'company_name' : company_name,
'job_location' : job_location,
'salary_estimate' : salary_estimate
}
df = pd.DataFrame.from_dict(jobs, orient='index').transpose()
df
And this is my result
job_title company_name job_location salary_estimate
0 Data Analyst CSCI Consultingn4.7 Remote $15.00 Per Hour (Employer est.)
1 Data Reporting Analyst State of New Mexicon3.3 Santa Fe, NM 53????−
72K (Employer est.)
2 Air optimization data analyst American Express Global Business Traveln4.2 Remote 46????−
93K (Employer est.)
3 Data Analyst, Professional Services (Remote) CrowdStriken4.1 Remote 85????−
135K (Employer est.)
4 Business Analyst – Healthcare Sagiant Tech Solutions Pvt Ltd Remote 55.00−
65.00 Per Hour (Employer est.)
… … … … …
296 Sr. INFOR (M3) ERP Business Analyst VSE Careersn3.2 Remote None
297 Senior Actuarial Analyst – Tri-State P&C (remote) Millimann3.8 United States None
298 Salesforce Senior Business Analyst Valvoline Global Operationsn4.4 Remote None
299 Fiscal Analyst 2 State of Washingtonn3.6 Olympia, WA None
300 IT BUSINESS ANALYST – SENIOR/SPECIALIST State of Washingtonn3.6 Tumwater, WA None
301 rows × 4 columns