I am trying to get practice learning Selenium using Python.

I’ve come across a website (https://kwcommercial.com/agents/) that has elements that aren’t findable. For example, I’m trying to access the “load more” button which has a class of “load-more-agents” (which is visible in inspect element). However, Selenium cannot detect this button/div container (or rather the majority of divs/elements on the page) when I try using driver.find_element(). when I try to find all divs on the site, I get a small list shown below.

List:
kw-header_universal
kw-container
kw-header_universal_navigation
kw-header_primary
kw-container
kw-header_primary_logo
kw-header_primary_navigation
kw-header_primary_toggle
kw-page
kw-section kw-section–bg-is-image kw-section–bg-is-image
kw-container kw-container–md kw-container–center
kw-section kw-section–bg-is-white
kw-container kw-container–xl kw-container–left
kw-section kw-footer_primary
kw-container
kw-footer_primary_navigation
kw-container
kw-footer_primary_logo
kw-footer_secondary kw-section kw-section–min-p
kw-container kw-container–center kw-container–sm
kw-footer_secondary_navigation

My question is why are the majority of elements on this webpage hidden from selenium, and if I can somehow bypass this block to scrape the webpage.

To solve this problem I tried printing all the divs shown above and using a wait.until() call to wait for the elements to load, but this didn’t work.