I’m trying to build a webscrapper to scrap through LinkedIn job postings. Jobs are listed in a unordered list, and currently my code clicks on each one. However, I keep getting

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found in the current frame

I think this because when I click on a list item, the whole DOM gets reloaded and hence the li items are now stale.

How do I overcome this?

driver.find_element(By.CSS_SELECTOR, "[href='https://www.linkedin.com/jobs/?']").click()
driver.implicitly_wait(0.5)

driver.get('https://www.linkedin.com/jobs/search?keywords=software%20engineer&location=United%20States')

job_block = driver.find_element(By.CLASS_NAME, "scaffold-layout__list-container")
jobs = job_block.find_elements(By.TAG_NAME, "li")

for job in jobs:
   time.sleep(3)
   wait = WebDriverWait(driver, 100) #This didn't work
   element = wait.until(EC.element_to_be_clickable(job))
   #This didn't work
   # ActionChains(driver).double_click(job).perform() 
   element.click()
   driver.implicitly_wait(0.5)

I think the list jobs is getting stale after a list item is clicked on