I have 4 gitlab-ci.yml files very similar to each other, the do this:

build database:
  stage: build
  retry: 2
  script:
    - echo 'Building the database image...'
    - docker pull ${IMAGE_PREFIX}/db:latest || true
    - docker build
        --file database/docker/Dockerfile
        --build-arg IMAGE_REGISTRY=${IMAGE_REGISTRY}
        --pull
        --cache-from ${IMAGE_PREFIX}/db:latest
        --tag ${IMAGE_PREFIX}/db:${CI_COMMIT_SHA}
        .
  rules:
    - if : '$CI_COMMIT_BRANCH == "main"'

push database:
  stage: push
  retry: 2
  script:
    - docker push ${IMAGE_PREFIX}/db:${CI_COMMIT_SHA}
    - docker rmi  ${IMAGE_PREFIX}/db:latest || true
    - docker rmi  ${IMAGE_PREFIX}/db:${CI_COMMIT_SHA}
  rules:
    - if : '$CI_COMMIT_BRANCH == "main"'

In the Gitlab Pipeline, they all pass the build stage, but the push stage seems to randomly fail with an error tag does not exist what can be happening?

I have done - docker images to debug, and indeed, when it fails, I see the image has the incorrect tag:

$ docker images
REPOSITORY                TAG                                        IMAGE ID       CREATED          SIZE
db                        d5490eeb56483b580c06f1d7a028c9a39d28eff4   a8955a972823
$ echo ${IMAGE_PREFIX}/db:${CI_COMMIT_SHA}
db:c9eaa8109b4d79d012244fccebd69c964e1469b3

I have checked and the build stage used the c9eaa8109b4d79d012244fccebd69c964e1469b3 tag. What can be happening? Why is it a random fail?

And I want to push