I’m building a SeleniumBase scraper that takes a set of tasks and performs them on a website. After performing some actions on the site the script calls driver.execute_script("return window.variable;"). This works fine on all domains I’ve scraped even with any redirects after clicks.

Today I scraped a new website which includes pressing two buttons that both redirect you to a page (2 redirects total). After the redirects the script runs execute_script and throws a selenium.common.exceptions.JavascriptException.

Traceback (most recent call last):
  File "/home/victor/.pyenv/versions/3.12.3/envs/tagprotection-v3/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/victor/Webvitals/tagprotection-v3/tagprotection/worker.py", line 28, in __collect_datalayer
    datalayer = sb.execute_script("return window.variable;")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/victor/.pyenv/versions/3.12.3/envs/tagprotection-v3/lib/python3.12/site-packages/seleniumbase/fixtures/base_case.py", line 3372, in execute_script
    return self.driver.execute_script(script, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/victor/.pyenv/versions/3.12.3/envs/tagprotection-v3/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 414, in execute_script
    return self.execute(command, {"script": script, "args": converted_args})["value"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/victor/.pyenv/versions/3.12.3/envs/tagprotection-v3/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 354, in execute
    self.error_handler.check_response(response)
  File "/home/victor/.pyenv/versions/3.12.3/envs/tagprotection-v3/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.JavascriptException: Message: javascript error: {"status":10,"value":"stale element not found in the current frame"}

I’m not quite sure why this javascript variable is a stale element. Data is being pushed to it on every page (similar to Google’s dataLayer)