Relative Content

Tag Archive for javascripthtmlselenium-webdriverweb-scraping

How can I scrape a dynamically rendered webpage using Selenium?

This is the page I’m trying to scrape. My goal is to check whether or not the string “Any available Pool will appear here.” is on the page.
I’ve done some research and found out that the page is dynamically generated by a script in the html which contains all the content of the website in a dictionary-like data structure.
The issue I’m running into is that, even after loading the page and scrolling down, the page source that I’m extracting with (driver.getPageSource()) still doesn’t contain the html element with the string (<p class="font-[300]">Any available Pool will appear here</p>) and the search only matches once with the string that appears in the script JSON.