I’ve been debugging this for quite some time and I don’t know what else I should do to fix this.
Long story short: I have a rails app that needs to launch chromedriver to navigate to a specific website and get some data from there. I’m using Ubuntu and 22.04 and I installed chromedriver and google chrome stable on /usr/bin. I can launch chrome and chromedriver from the terminal. My rails app is consuming from a sidekiq queue the URL to navigate and the application fails with:
Selenium::WebDriver::Error::WebDriverError: unable to connect to /usr/bin/chromedriver 127.0.0.1:9515
It seems an issue launching chromedriver from rails app, but I did already some debug steps.
To debug this I created a standalone ruby script and executed a similar code to navigate to google and get the page title. It ran successfully. After that I created this code and added to config/initializers/selenium.rb
# config/initializers/selenium.rb
require 'selenium/webdriver'
Rails.logger.info "==== Boot up context ===="
Rails.logger.info "Rails environment: #{Rails.env}"
Rails.logger.info "User executing: #{ENV['USER']}"
Rails.logger.info "PATH: #{ENV['PATH']}"
user_agent = 'Mozilla/5.0 (X11'
args = ["--user-agent=#{user_agent}", '--no-sandbox', '--disable-dev-shm-usage', '--ignore-certificate-errors', '--window-size=1920,1080']
args << "--headless=new" if Rails.env.production?
options = Selenium::WebDriver::Chrome::Options.new(
args: args
)
begin
Rails.logger.info "Initializing ChromeDriver"
Rails.logger.info "Chrome options: #{options.args.join(' ')}"
driver = Selenium::WebDriver.for(:chrome, options: options)
driver.navigate.to "http://www.google.com"
Rails.logger.info "Page title: #{driver.title}"
driver.quit
rescue => e
Rails.logger.error "Error starting chromedriver: #{e.message}"
Rails.logger.error e.backtrace.join("n")
end
When the app boot, I can see the page title with success. So far so good!
The problem is when I add this to my main code when app get from the queue. The main difference is that I have a Module, don’t know if that makes any relevant difference for this case but I’m not able to launch chromedriver and it fails with the error Selenium::WebDriver::Error::WebDriverError: unable to connect to /usr/bin/chromedriver 127.0.0.1:9515
I will post the code below
require 'selenium-webdriver'
module MyModule
PROXY_ENABLED = true
USER_AGENT = 'Mozilla/5.0'
def proxy_enabled?
PROXY_ENABLED
end
def proxy_username
ProxyManager.instance.username
end
def proxy_password
ProxyManager.instance.password
end
def random_proxy
ProxyManager.instance.random_proxy(self.class::SOURCE_NAME)
end
def user_agent
USER_AGENT
end
def build_selenium
proxy_addr = proxy_enabled? ? random_proxy : nil
proxy = proxy_enabled? ? Selenium::WebDriver::Proxy.new(
http: proxy_addr,
ssl: proxy_addr
) : nil
args = ["--user-agent=#{user_agent}", '--no-sandbox', '--disable-dev-shm-usage', '--ignore-certificate-errors', '--window-size=1920,1080']
args << "--headless=new" if Rails.env.production?
options = Selenium::WebDriver::Chrome::Options.new(
proxy: proxy,
args: args
)
driver = Selenium::WebDriver.for(:chrome, options: options)
if proxy_enabled?
driver.register(username: proxy_username, password: proxy_password)
end
# Return a hash containing both the driver, the proxy address, and the source name
{
driver: driver,
proxy_addr: proxy_addr
}
end
I printed the PATH and running user and they are the same on both situations. I also printed chromedriver and google-chrome paths and they looks ok.