I need to do web scraping on an HVAC site but I’m stuck on the form before the scraping part.
The example zipcode gets put in the input box fine, I just haven’t been able to actually do the submitting!
Here is the code I have so far, and the website
# Import required modules
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
import codecs
import re
driver=webdriver.Firefox()
zipcode_input = "35404"
#Redirect
val = 'https://www.americanstandardair.com/find-your-dealer/'
wait = WebDriverWait(driver, 10)
driver.get(val)
wait.until(EC.url_to_be(val))
page_source = driver.page_source
#waiting until input box is there then typing zipcode
send_keys = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.ID, 'zipcode'))).send_keys(zipcode_input)
#waiting until submit button is there then submitting
submit_zipcode = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR('button[data-cy="zipcode-search"]'))))
submit_zipcode.click()
soup = BeautifulSoup(page_source,features="html.parser")
print("complete")
driver.quit()
I get this error in IDLE relating to the submit_zipcode line of finding the zipcode-search:
TypeError: 'str' object is not callable
Know it means I tried to call a string like a function but I really don’t know which one that is. I tried to use the css selector “button.absolute” instead to select the button but still no good.
I’m new to python so I’m pretty stumped with what’s wrong, so any help would be appreciated.
user25774167 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.