I have been trying to connect to the selenium/chrome-standalone docker container, and it is having trouble creating sessions. I keep getting the following error:

21:02:35.263 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "18a8e2283000ca2a77b5526145bdce62","eventTime": 1719867755250989883,"eventName": "exception","attributes": {"exception.message": "Unable to create session: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: Chrome failed to start: crashed.n  (disconnected: unable to connect to renderer)n  (The process started from chrome location u002fusru002fbinu002fgoogle-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) nHost info: host: 'chrome', ip: '172.19.0.2'nBuild info: version: '4.22.0', revision: 'c5f3146703'nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.6.31-linuxkit', java.version: '17.0.11'nDriver info: driver.version: unknownnBuild info: version: '4.22.0', revision: 'c5f3146703'nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.6.31-linuxkit', java.version: '17.0.11'nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: session not created: Chrome failed to start: crashed.n  (disconnected: unable to connect to renderer)n  (The process started from chrome location u002fusru002fbinu002fgoogle-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) nHost info: host: 'chrome', ip: '172.19.0.2'nBuild info: version: '4.22.0', revision: 'c5f3146703'nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.6.31-linuxkit', java.version: '17.0.11'nDriver info: driver.version: unknownnBuild info: version: '4.22.0', revision: 'c5f3146703'nSystem info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.6.31-linuxkit', java.version: '17.0.11'nDriver info: driver.version: unknownntat org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:221)ntat org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:71)ntat org.openqa.selenium.grid.node.local.SessionSlot.apply(SessionSlot.java:147)ntat org.openqa.selenium.grid.node.local.LocalNode.newSession(LocalNode.java:469)ntat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:652)ntat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:571)ntat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:834)ntat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:791)ntat java.baseu002fjava.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)ntat java.baseu002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)ntat java.baseu002fjava.lang.Thread.run(Unknown Source)n","exception.type": "org.openqa.selenium.SessionNotCreatedException","logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--headless, --no-sandbox, --disable-dev-shm-usage], extensions: []}, goog:loggingPrefs: {performance: ALL}, pageLoadStrategy: normal}]"}}

Here is the docker-compose.yml file:

services:
  chrome:
    platform: linux/x86_64
    hostname: chrome
    image: selenium/standalone-chrome:4.22.0-20240621
    shm_size: 2gb
    privileged: true
  datafetcher:
    build: .
    depends_on:
      - chrome

The code that is generating this error is the following:

options = Options()
options.set_capability("goog:loggingPrefs", {"performance": "ALL"})
options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")

driver = webdriver.Remote("http://chrome:4444/wd/hub", options=options)

I tried using different versions of selenium/chrome-standalone and using the node/hub containers instead of the standalone. Any suggestions?