I’m working with GitLab CI and I have a job template that I want to modify. Currently, my job looks like this:
mwaa:dags:test:2.8.1:
variables:
AIRFLOW_VERSION: 2.8.1
PYTHON_VERSION: 3.11
extends:
- .template_MWAA:DAGS:Test
This job extends a template that includes before_script
, script
, and after_script
sections. I want to modify the job so that:
- If there are changes in the
airflow-mwaa/requirements/**
directory, it should run all sections (before_script
,script
, andafter_script
). - If there are no changes in the
airflow-mwaa/requirements/**
directory, it should only run thescript
section.
Here are the relevant parts of the template:
before_script:
- echo "Starting .install_dependencies template"
- pip3 install -r "$CI_PROJECT_DIR/airflow-mwaa/requirements/$AIRFLOW_VERSION/requirements.txt"
script:
- echo "Starting .template_MWAA:DAGS:Test template"
- cd "$CI_PROJECT_DIR/airflow-mwaa"
- airflow db init
- ./tests-ci.sh
after_script:
# remove older files from terraform providers cache
- find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type f -delete
- find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type d -empty -delete
stage: Test
extends:
- .template_MWAA
needs: []
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
PYTHONPATH: "$CI_PROJECT_DIR/airflow-mwaa/dags"
cache:
key: $CI_JOB_NAME_SLUG-python
paths:
- .cache/pip
before_script:
- echo "Starting .install_dependencies template"
- pip3 install -r "$CI_PROJECT_DIR/airflow-mwaa/requirements/$AIRFLOW_VERSION/requirements.txt"
script:
- echo "Starting .template_MWAA:DAGS:Test template"
- cd "$CI_PROJECT_DIR/airflow-mwaa"
- airflow db init
- ./tests-ci.sh
after_script:
# remove older files from terraform providers cache
- find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type f -delete
- find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type d -empty -delete
coverage: '/^TOTALs+d+s+d+s+(d+%)$/'
artifacts:
reports:
junit: $CI_PROJECT_DIR/airflow-mwaa/test_results/report.xml
rules:
- if: $CI_COMMIT_REF_PROTECTED == "true"
when: on_success
allow_failure: false
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- airflow-mwaa/dags/**/*
- airflow-mwaa/tests/**/*
allow_failure: false
when: on_success
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
when: manual
allow_failure: true
- when: never
How can I modify my job or template to achieve this conditional execution based on file changes? I’m not sure how to combine the changes
condition with selective execution of script sections.
Any help or guidance would be greatly appreciated!
0