Gitlab ci file structure:

- Main code
- gitlab-ci.yml 
- automated_trigger -> folder
   - pipeline.yml
   - reports -> folder
     - main_ones.yml
     - get_test_stats.yml
   - imports -> folder
     - dependencies.yml
     - cluster_info.yml
     - overrides -> folder
       - cases.yml
       - data.yml

The below is the code in gitlab-ci.yml file, along with other stage definitions for merge request (this MR related stages are working fine) no issues there:

include:
  - project: sample_test/gitversion
    ref: $GITLAB_LIBRARY_VERSION 
    file: lib_git_version.yml

  - local: '/automated_trigger/pipeline.yml'
    rules:
      - if: '$CI_PIPELINE_SOURCE == "schedule"'
      - if: '$CI_PIPELINE_SOURCE == "web"'

With this file structure, there are bunch of stages in gitlab-ci.yml (this has stages for merge req), pipeline.yml has stages (meant for pipelines) and via include it calls the stages from main_ones.yml and get_test_stats.yml files.

First question is: Can I have stages defined in gitlab-ci.yml file and pipelines.yml files separately? When I had stages listed in both the files, triggering a pipeline manually it did not run the stages defined in pipeline.yml.

So now I copied and listed all the stages from pipeline.yml in gitlab-ci.yml and due to above situation.

After the above change, there is an error saying one of the stage in pipeline.yml does not have the needs:get_test_stats stage which is defined in the get_test_stats.yml within reports folder.

This get_test_stats.yml is included in the pipeline.yml file like below:

workflow:
 name: blah blah

include:
  - local: '/automated_trigger/imports/overrides.yml'
    rules:
      - if: '$CI_PIPELINE_SOURCE == "schedule"'
      - if: '$CI_PIPELINE_SOURCE == "web"'

Within overrides.yml i have the below:

    rules:
      - if: '$CI_PIPELINE_SOURCE == "schedule"'
      - if: '$CI_PIPELINE_SOURCE == "web"'

I’m having trouble in running the stages that are meant for pipeline which has been nested with 6-8 yml files.

All these were running perfectly fine when the pipeline.yml was the main gitlab-ci.yml but now it complains as it is called from another yml file.

I’m hope I have explained the question well.