The test server worked well, but when you try to deploy it to another server, it stops working at chromeDriverService = ChromeDriverService.createDefaultService();
even if you remove the ChromeDriverService and immediately use driver = new ChromeDriver(options);
the code stops without executing it.
Both servers are the same except for the version and memory. Changing the size of the thread pool from 10 to 1 doesn’t work because I was wondering if it was a memory problem.
It stopped before it was initialized, so the log is not even taken.
Main Server – Rocky Linux release 8.10 (Green Obsidian)
total used free shared buff/cache available
Mem: 7.5Gi 5.5Gi 926Mi 178Mi 1.1Gi 1.6Gi
Swap: 2.0Gi 1.7Gi 303Mi
Test Server – Rocky Linux release 8.9 (Green Obsidian)
total used free shared buff/cache available
Mem: 15Gi 6.6Gi 2.4Gi 477Mi 6.4Gi 8.0Gi
Swap: 2.0Gi 843Mi 1.2Gi
I use Selenium Java 4.22.0, Chrome 126, ChromeDriver 126
@Log4j2
public class SeleniumCrawlerDriver implements AutoCloseable {
private static final ChromeOptions chromeOptions = new ChromeOptions();
private static final String CHROME_DRIVER = "webdriver.chrome.driver";
private ChromeDriverService chromeDriverService;
private WebDriver driver;
public SeleniumCrawlerDriver() {
System.setProperty(CHROME_DRIVER, getChromeDriverPath());
}
public boolean create() {
ChromeOptions options = initChromeOptions();
log.debug("chrome option init");
chromeDriverService = ChromeDriverService.createDefaultService();
log.debug("create ChromeDriverService");
//// chromeDriverService = new ChromeDriverService.Builder()
//// .usingDriverExecutable(new File(getChromeDriverPath()))
//// .usingAnyFreePort()
//// .withLogFile(new File(getChromeDriverLogPath(site.getLogName())))
//// .build();
driver = new ChromeDriver(chromeDriverService, options);
// driver = new ChromeDriver(options);
driver.manage().timeouts().scriptTimeout(Duration.ofSeconds(30));
driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(30));
driver.manage().timeouts().pageLoadTimeout(Duration.ofSeconds(30));
return true;
}
private ChromeOptions initChromeOptions() {
String downloadFilepath = getDownloadLocationPath();
Map<String, Object> prefs = new HashMap<>();
prefs.put("credentials_enable_service", false);
prefs.put("profile.password_manager_enabled", false);
prefs.put("download.prompt_for_download", false);
prefs.put("download.default_directory", downloadFilepath);
prefs.put("browser.set_download_behavior", "{behavior : 'allow' , downloadPath: '" + downloadFilepath + "'}");
// chromeOptions.addArguments("--download.default_directory=/dev/null");
chromeOptions.addArguments("--disable-infobars");
chromeOptions.addArguments("--disable-notifications");
chromeOptions.addArguments("--headless=new");
chromeOptions.addArguments("--window-size=1920,1080");
chromeOptions.addArguments("--disable-extensions");
chromeOptions.addArguments("--proxy-server='direct://'");
chromeOptions.addArguments("--proxy-bypass-list=*");
chromeOptions.addArguments("--start-maximized");
chromeOptions.addArguments("--disable-gpu");
chromeOptions.addArguments("--disable-dev-shm-usage");
chromeOptions.addArguments("--disable-logging");
chromeOptions.addArguments("--log-level=2");
// chromeOptions.addArguments("--disable-popup-blocking");
chromeOptions.addArguments("--no-sandbox");
chromeOptions.addArguments("--ignore-certificate-errors");
chromeOptions.setExperimentalOption("prefs", prefs);
chromeOptions.addArguments("start-maximized");
chromeOptions.addArguments("enable-javascript");
chromeOptions.addArguments("user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36");
return chromeOptions;
}
}
Ahn Nayeong is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1