I know there’s quite a lot in Stackoverflow (and elsewhere) about “element not interactable” but I don’t think this is a duplicate.
My issue is that I get the selenium.common.exceptions.ElementNotInteractableException ONLY if I try to run my code in headless mode.
Here’s the code. Don’t worry about what it’s trying to achieve other than provide a Minimal Reproducible Example
from selenium import webdriver
from selenium.webdriver import ChromeOptions
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
options = ChromeOptions()
options.add_argument("--headless")
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
options.add_argument(f"user-agent={user_agent}")
QUERY = "museums in london"
# This bypasses the "automation" warning
def bypass(driver):
try:
wait = WebDriverWait(driver, 5)
for button in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "button"))):
if button.text.strip() == "Reject all":
button.click()
break
except Exception:
pass
with webdriver.Chrome(options=options) as driver:
driver.get("https://www.google.com")
bypass(driver)
selector = By.CSS_SELECTOR, 'form[action="/search"] textarea[name=q]'
textarea = WebDriverWait(driver, 5).until(EC.presence_of_element_located(selector))
# The following Action sequence makes no difference to the program's behaviour
ActionChains(driver).move_to_element(textarea).click(textarea).perform()
textarea.send_keys(QUERY+Keys.RETURN)
urls = set()
selector = By.CSS_SELECTOR, "cite[role=text]"
for cite in WebDriverWait(driver, 5).until(EC.presence_of_all_elements_located(selector)):
if text := cite.text.strip():
urls.add(text)
print(*urls, sep="n")
As shown, I’m trying to do run the Chrome driver in headless mode. If I do that I consistently get the ElementNotInteractableException at the point where I try to invoke send_keys().
If you remove the addition of the –headless argument to the ChromeOptions class, the code runs without error.
I had hoped that the ActionChains code would help but it makes no difference to the program’s behaviour whether in headless mode or not. I just left it in in case anyone suggested it as a potential workaround.
Why is this only a problem when in headless mode? What can be done about it?
6
Often in my projects as I encounter such a problem I add such options. Often these are anti-bot security features.
options = Options()
options.add_argument("--headless")
options.add_argument("--disable-gpu")
options.add_argument("--window-size=1920,1080")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument(
"user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3")
driver = webdriver.Chrome(options=options)
I would also recommend using newer user agents this one is slightly outdated.
I would personally add in your code cleanly for debugging purposes.
driver.save_screenshot('your_screenshot_path')
2
Why?
TLDR; a lot of sites have at least basic anti-bot mechanisms in place to read your browser info. This will include what flags your browser has set. Often times, a website will see the ‘headless’ flag and either completely block you or at least limit what you can see.
(This is also extends to the other answer which suggested a more up-to-date useragent as sometimes sites will block common bot or just old useragents)
Solving it
Another option you have besides those already suggested is to try using the Python package pyvirtualdisplay
(pip install pyvirtualdisplay
). This will simulate a screen your driver will display to, instead of setting the headless flag.
Very basic example:
from pyvirtualdisplay import Display
options = ChromeOptions()
options.add_argument("user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36")
with Display(visible = False, size = (1920, 1080)):
with webdriver.Chrome(options = options) as driver:
driver.get("https://www.google.com")
The only caveat is that this only works on Linux due to the requirements of the pyvirtualdisplay
package. Otherwise you can run this in a Docker container (or slightly more difficultly through WSL).
Ash Olorenshaw is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.