`I am able to extract house price data which is: $276,000
but I am not able to extract “Beds” or any other type of span data: example:
2
This is the script I am using:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import time
def extract_beds_from_url(url):
beds_data = {'URL': url, 'Beds': 'N/A'}
try:
# Set up the WebDriver
print("Setting up the WebDriver...")
service = Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=service)
print("WebDriver set up successfully.")
# Open the web page
print(f"Opening URL: {url}")
driver.get(url)
print("URL opened successfully.")
# Wait for the page to fully render
time.sleep(10)
# Extracting number of beds
try:
print("Attempting to find the beds element using CSS_SELECTOR...")
beds_element = driver.find_element(By.CSS_SELECTOR, 'span[data-qa="beds"].item-detail-label')
print("Beds element (CSS_SELECTOR) HTML content:", beds_element.get_attribute('outerHTML'))
beds_data['Beds'] = beds_element.text if beds_element else 'N/A'
print("Beds element found and extracted successfully using CSS_SELECTOR.")
except Exception as e:
print(f"Error while finding the beds element using CSS_SELECTOR: {e}")
# Close the WebDriver
print("Closing WebDriver...")
driver.quit()
print("WebDriver closed successfully.")
except Exception as e:
print(f"An error occurred during the WebDriver setup or execution: {e}")
return beds_data
# URL of the page to extract data from
page_url = 'https://portal.onehome.com/es-US/property/aotf~1075849001~SEFMIAMI?token=eyJPU04iOiJTRUZNSUFNSSIsInR5cGUiOiIxIiwiY29udGFjdGlkIjo5MjgxMjg5LCJzZXRrZXkiOiI3MzAiLCJlbWFpbCI6InRhYmlvMTk4NUBob3RtYWlsLmNvbSIsInJlc291cmNlaWQiOjAsImFnZW50aWQiOjEwOTc1LCJpc2RlbHRhIjpmYWxzZSwiVmlld01vZGUiOiIxIn0%3D&searchId=d198393a-22a6-33b5-a207-8db32bb8cf89'
# Extract beds from the URL
beds_data = extract_beds_from_url(page_url)
# Print the extracted data
print("Extracted Data:")
print(f"URL: {beds_data['URL']}")
print(f"Beds: {beds_data['Beds']}")
I am expecting to get Beds printed but it is coming empty.
New contributor
tabio 85 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.