I’ve built a simple multiplayer trivia app in React that uses Socket.io to communicate between client and server. To start, visitor needs to login to establish connection. If connection is established, GameLobby component is rendered. I’ve written Playwright E2E test to assert this.

  test('visitor can login', async ({ page }) => {
    const username = 'john.doe'
    await page.goto('/')

    await expect(page).toHaveURL(/http://localhost:5173//)

    const loginButton = page.getByTestId('header-login-button')

    await loginButton.click()

    const usernameInput = page.getByPlaceholder('Type your username')

    await usernameInput.fill(username)
    await page.getByRole('button', { name: 'Login!' }).click()

    await expect(page.getByTestId('game-lobby')).toBeVisible()
  })

Test passes with no problem in my local environment both on chromium and firefox, that is div tagged with test-id game-lobby is visible.

However, when code is pushed to my github repo, E2E tests fail. I logged messages that appear in console during the test runs:

// chromium
WebSocket connection to 'ws://localhost:8181/socket.io/?EIO=4&transport=websocket&sid=URltd7CxRl-ryPZuAAAB' failed: WebSocket is closed before the connection is established.
WebSocket connection to 'ws://localhost:8181/socket.io/?EIO=4&transport=websocket&sid=TElu3sE0DVJJ9905AAAC' failed: Error during WebSocket handshake: Unexpected response code: 400
// firefox
"Firefox can’t establish a connection to the server at ws://localhost:8181/socket.io/?EIO=4&transport=websocket&sid=IeUT9q1NhUwqoEWPAAAP."

Excerpt related to E2E from YAML file for the pipeline looks like so:

- name: Install Playwright Browsers
  run: npx playwright install --with-deps chromium firefox

- name: Start dev server for E2E tests
  run: npm run dev:e2e -w=server &

- name: Build front end for E2E tests
  run: npm run build -w=client
  env:
    VITE_SOCKET_PORT: 8181,
    VITE_SOCKET_ORIGIN: http://localhost

- name: Wait for server to startup
  uses: Jtalk/[email protected]
  with:
    max-attempts: 5
    retry-delay: 5s
    url: http://localhost:8181/api/health

- name: Test client E2E
  run: npm run test:e2e -w=client

During pipeline run on github, the health api step passes, so the server should be running. I also tried to increase Playwright timeouts in the playwright.config file:

  timeout: process.env.CI ? 90_000 : 10_000,
  expect: {
    timeout: process.env.CI ? 15_000 : 2_000,
  },

Can anyone explain what causes test to fail on github side?