I’m encountering persistent deployment errors on AWS Elastic Beanstalk and codePipeline, and I need some assistance to diagnose and resolve the issue. The error message I receive is:

Deployment completed, but with errors: During an aborted deployment, some instances may have deployed the new application version. To ensure all instances are running the same version, re-deploy the appropriate application version. Failed to deploy application. Unsuccessful command execution on instance id(s) 'i-0aa5817c1885fa11e'. Aborting the operation.

Here are the details of my setup and what I have tried so far:

Environment Details:

• Platform: AWS Elastic Beanstalk

• Instance Type: Amazon Linux 2

• Python Version: 3.11

Deployment Scripts:

buildspec.yml:

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.11
    commands:
      - echo Installing dependencies...
      - pip install --upgrade pip
      - pip install -r requirements.txt

  pre_build:
    commands:
      - echo Pre-build phase...

  build:
    commands:
      - echo Build phase...

  post_build:
    commands:
      - echo Post-build phase...

artifacts:
  files:
    - '**/*'
  base-directory: .

.ebextensions/install_firefox.config:

packages:
  yum:
    python3-devel: []
    libX11-devel: []
    libXtst-devel: []
    libXScrnSaver: []
    alsa-lib: []
    xorg-x11-server-Xvfb: []
    libxkbcommon-x11: []
    at-spi2-core: []
    google-noto-cjk-fonts-common: []
    google-noto-sans-cjk-ttc-fonts: []

commands:
  01_install_dependencies:
    command: yum install -y wget bzip2

  02_cleanup_previous_files:
    command: |
      rm -f /opt/firefox.tar.bz2
      rm -f /opt/geckodriver.tar.gz

  03_download_firefox:
    command: |
      wget -O /opt/firefox.tar.bz2 "https://download.mozilla.org/?product=firefox-latest&os=linux64&lang=en-US"
      tar -xjf /opt/firefox.tar.bz2 -C /opt/
      chmod +x /opt/firefox/firefox
      ln -sf /opt/firefox/firefox /usr/bin/firefox

  04_download_geckodriver:
    command: |
      wget https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz -O /opt/geckodriver.tar.gz
      tar -xzf /opt/geckodriver.tar.gz -C /opt/
      mv /opt/geckodriver /usr/local/bin/geckodriver
      chmod +x /usr/local/bin/geckodriver
      ln -sf /usr/local/bin/geckodriver /usr/bin/geckodriver

  05_install_firefox:
    command: yum install -y xorg-x11-server-Xvfb

  06_configure_xvfb:
    command: |
      Xvfb :99 -screen 0 1920x1080x24 &
      export DISPLAY=:99

  07_configure_firefox_options:
    command: |
      mkdir -p /etc/firefox
      echo "FIREFOX_ARGS=--headless" >> /etc/firefox/default

Selenium Helper Script:

import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.options import Options
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.firefox.service import Service
from selenium.common.exceptions import WebDriverException

def init_driver():
    try:
        options = webdriver.FirefoxOptions()
        options.headless = True
        options.add_argument('--no-sandbox')
        options.add_argument('--disable-dev-shm-usage')

        driver_path = '/usr/local/bin/geckodriver'
        service = Service(driver_path)
        driver = webdriver.Firefox(service=service, options=options)
        return driver

    except WebDriverException as e:
        print(f"WebDriverException occurred...: {e}")
        return None
    except Exception as e:
        print(f"Error occurred...: {e}")
        return None

def get_full_page_html(driver, base_url):
    if driver is None:
        return None

    driver.get(base_url)
    
    try:
        WebDriverWait(driver, 10).until(
            EC.presence_of_all_elements_located((By.CSS_SELECTOR, "body"))
        )
        print(f"Successfully navigated to {base_url}")

    except TimeoutException as error:
        print(f"Error occurred while waiting for the page to load: {error}")
        return None
    except WebDriverException as error:
        print(f"WebDriverException occurred while loading the page: {error}")
        return None
    
    time.sleep(2)
    page_source = driver.page_source
    return page_source

Additional Context:

• I have attempted to ensure idempotent scripts and have included cleanup commands to remove previous files before downloading new ones.

• I removed all application versions from Elastic Beanstalk, removed the entire Elastic Beanstalk application, and recreated a new EB application.

• I tried deploying just a basic sample Flask application.

• I have checked CloudWatch logs and the failing instance logs but couldn’t find a specific cause.

• From my research, I found that reusing the same ZIP file name could cause such issues but it also not working

• This issue started occurring when I began working with Selenium. Prior to that, deployments were consistently successful.