This is the page I’m trying to scrape. My goal is to check whether or not the string “Any available Pool will appear here.” is on the page.
I’ve done some research and found out that the page is dynamically generated by a script in the html which contains all the content of the website in a dictionary-like data structure.
The issue I’m running into is that, even after loading the page and scrolling down, the page source that I’m extracting with (driver.getPageSource()
) still doesn’t contain the html element with the string (<p class="font-[300]">Any available Pool will appear here</p>
) and the search only matches once with the string that appears in the script JSON.
I’ve tried delaying the driver.getPageSource()
function, since I thought the page wasn’t being rendered fast enough, but that hasn’t worked.
I can clearly see the string on the page, but it’s nowhere to be found in the page source (except in the script json) so I’m very confused.