I am using HtmlUnit Android to scrap data from a website, I am able to create the web client and connect it to the website, but after clicking a button I cannot obtain further elements. In the code, the 1st print is executed but 2nd isn’t. This code was working fine 3 months ago, and today I got an error which I got rid of by simpling update the html unit library from implementation("net.sourceforge.htmlunit:htmlunit-android:2.67.0") to implementation("org.htmlunit:htmlunit3-android:3.7.0").

Web scrapping code:

        val webClient = WebClient(BrowserVersion.CHROME)

        webClient.options.isCssEnabled = false

        var page: HtmlPage = webClient.getPage("https://timetables.qmul.ac.uk/default.aspx")

        val locationsBtn: HtmlAnchor = page.getHtmlElementById("LinkBtn_locations")

        println(locationsBtn)

        page = locationsBtn.click()
        println(page.webResponse.contentAsString)