I’m trying to scrape this page: https://www.lavoro.gov.it/Pagine/Cerca-nel-sito?search=big+data
As you can see at the bottom of the page there are the number of pages and the icon with the arrow that i’m clicking with selenium.
Problem is, whenever I copy the Xpath of this object i get to the second page and my code start endlessly cycling between the first and the second page.
this is the block of code I’m trying to use for opening next pages:
try:
next_page_link = driver.find_element(By.XPATH, "//li[@class = 'page-item pager__item pager__item--next ']")
# search_results_url = next_page_link.get_attribute("href")
# driver.get(search_results_url) # <--- open next page with results using URL
next_page_link.click() # <--- or click link
except Exception as e:
print('[DEBUG] Exception:', e)
print('[DEBUG] break')
#input('Press ENTER to continue')
break # exit loop
But I can’t seem to go past the second page, someone knows how to do it?