I am trying to crawl on this website: https://www.softpedia.com/get/Programming/Other-Programming-Files/Apidog.shtml using html unit.

I want to “click” on “Download now” button, it should run js code and open a new windows over the page:

WebClient webClient = new WebClient(BrowserVersion.EDGE);
webClient.waitForBackgroundJavaScript(20 * 1000);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setUseInsecureSSL(true);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
HtmlPage page = webClient.getPage(patchUrl);
HtmlElement element = page.getFirstByXPath(xpath);
page = element.click();
webClient.waitForBackgroundJavaScript(10 * 1000);
return page.asXml();

I managed to get the html element by its xpath (“//*[@id=”dlbtn1″]/a”), but calling to click() from it is doing nothing. Even if I call to sleep or to waitForBackgroundJavaScript() the readyState_ field in the HtmlElement is “loading” and not “completed”.

Am I missing something? thanks.

Tried to change the code, serached for anawers here, looked into the documentation, nothing helped me.