AWS CodePipeline Deploys Unchanged Images in ECS
I am deploying multiple services using AWS CodePipeline and CodeBuild. The pipeline has been set up using AWS Copilot with the following commands:
copilot init pipeline
copilot pipeline deploy
Below is a snippet of my pipeline manifest file:
name: ******
version: 1
source:
provider: GitHub
properties:
branch: *******
repository: ***********
connection_name: ********
stages:
- name: ***********
deployments:
- service1
- service2
- service3:
depends_on: [service1, service2]
Buildspec.yaml
# Buildspec runs in the build stage of your pipeline.
version: 0.2
phases:
install:
commands:
- echo "cd into $CODEBUILD_SRC_DIR"
- cd $CODEBUILD_SRC_DIR
# Download the copilot linux binary.
- wget -q https://ecs-cli-v2-release.s3.amazonaws.com/copilot-linux-v1.34.0 -O copilot-linux
- chmod +x ./copilot-linux
build:
commands:
- echo "Building base image dts-npm-dependencies..."
- docker build -t base-image -f Dockerfile .
post_build:
commands:
- ls -la
- export COLOR="false"
- export CI="true"
- pipeline=$(cat $CODEBUILD_SRC_DIR/copilot/pipelines/production/manifest.yml | ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))')
- pl_envs=$(echo $pipeline | jq -r '.stages[].name')
# Find all the local services in the workspace.
- svc_ls_result=$(./copilot-linux svc ls --local --json)
- svc_list=$(echo $svc_ls_result | jq '.services')
- >
if [ ! "$svc_list" = null ]; then
svcs=$(echo $svc_ls_result | jq -r '.services[].name');
fi
# Find all the local jobs in the workspace.
- job_ls_result=$(./copilot-linux job ls --local --json)
- job_list=$(echo $job_ls_result | jq '.jobs')
- >
if [ ! "$job_list" = null ]; then
jobs=$(echo $job_ls_result | jq -r '.jobs[].name');
fi
# Raise error if no services or jobs are found.
- >
if [ "$svc_list" = null ] && [ "$job_list" = null ]; then
echo "No services or jobs found for the pipeline to deploy. Please create at least one service or job and push the manifest to the remote." 1>&2;
exit 1;
fi
# Generate the cloudformation templates.
# The tag is the build ID but we replaced the colon ':' with a dash '-'.
# We truncate the tag (from the front) to 128 characters, the limit for Docker tags
# (https://docs.docker.com/engine/reference/commandline/tag/)
# Check if the `svc package` commanded exited with a non-zero status. If so, echo error msg and exit.
- >
for env in $pl_envs; do
tag=$(echo ${CODEBUILD_BUILD_ID##*:}-$env | sed 's/:/-/g' | rev | cut -c 1-128 | rev)
for svc in $svcs; do
./copilot-linux svc package -n $svc -e $env --output-dir './infrastructure' --tag $tag --upload-assets;
if [ $? -ne 0 ]; then
echo "Cloudformation stack and config files were not generated. Please check build logs to see if there was a manifest validation error." 1>&2;
exit 1;
fi
done;
for job in $jobs; do
./copilot-linux job package -n $job -e $env --output-dir './infrastructure' --tag $tag --upload-assets;
if [ $? -ne 0 ]; then
echo "Cloudformation stack and config files were not generated. Please check build logs to see if there was a manifest validation error." 1>&2;
exit 1;
fi
done;
done;
- ls -lah ./infrastructure
artifacts:
files:
- "infrastructure/*"
The Dockerfile for the services is as follows:
FROM base-image
ARG micro_service=service1
WORKDIR /usr/src/app
ADD ./$micro_service/ ./$micro_service
EXPOSE 8000
WORKDIR /usr/src/app/$micro_service
# Build Service
RUN npm run build
RUN chmod +x start.sh
CMD ["./start.sh"]
The base image contains common code for all the services.
Issue:
When the pipeline runs, it builds the Docker images and pushes them to ECR. However, even if the images are cached and not updated in ECR, the ECS task definition is still updated and the services are redeployed.
Question:
- Is there a method to prevent the ECS task from updating when the Docker images have not changed? – How can I ensure that only modified images trigger an ECS deployment?
- If the image for service1 changes, deploy only service1 without deploying the other services?
- I am not pushing the base image to ECR; it is only cached temporarily. How can I persist the cache for this image?
I am not uploading the base-image onto ECR, it is only cached for sometime, how to persist the cache for this iamge
I’ve tried caching the base image locally to avoid rebuilding it for every pipeline execution. However, this cache is temporary and does not persist across different builds. I expected the pipeline to recognize unchanged images and skip the deployment of unmodified services, but it still updates the ECS task definition, redeploying all services.