I have Jenkins Groovy running declarative Pipeline with a parallel block. The parallel stages run on different agents with different build processes. I want my Groovy to load variables from YAML files stored in the code bases, for use in the shell scripts that run the builds. And the variables need to be set differently for the different agents.

I’ve tried to distill the issue down to show what I’m trying to do, hopefully it’s not still too lengthy to ask for help here. In my actual projects, the .groovy files in vars are in a shared library repository, while the Jenkinsfile and the YAML files are in each code repository.

I’ve set sleeps to force a race condition. With the sleeps, I’m having it first do the reconfiguration on Patience, then the reconfiguration on Fortitude, then show the variables on Patience, and then finally show the variables on Fortitude.

It seems that when an environment variable on one agent is set using env.*, the environment variable is also set on the other agent, even though it’s an entirely different Jenkins build node. Like spooky action at a distance, there.

At the end of the post below, I have the expected output, and what I’m actually getting instead.

I have tried adding env. in front of SYSTEM_OS in the reconfiguration stages, but that seems to set a global variable that overrides the other agent. Having no env. seems to set a local variable that’s scoped the the stage only, not to the parallel stage for the agent.

In localPipelineConfig.groovy I tried using "$local_setting_item.key" =, but that gives the error "$local_setting_item.key" is a GString expression, but it should be a variable expression. So I tried using def "$local_setting_item.key" =, but that gives the error unexpected token: def.

Jenkinsfile

library identifier: 'shared-library@version', retriever: legacySCM(scm)
PatienceAndFortitudeParallelPipeline()

vars/PatienceAndFortitudeParallelPipeline.groovy

def call() {
    pipeline {
        agent { label 'ubuntu-16.04' }
        stages {
            stage('Configuration') {
                steps {
                    script {
                        globalPipelineConfig("globalSettings.yml")
                        env.SERVER = "builder.example.com"
                    }
                }
            }
            stage('Patience and Fortitude in parallel') {
                parallel {
                    stage('Patience') {
                        agent { label 'ubuntu-16.04' }
                        stages {
                            stage('Patience - Reconfiguration') {
                                steps {
                                    script {
                                        localPipelineConfig('patienceSettings.yml')
                                        SYSTEM_OS = 'ubuntu'
                                    }
                                }
                            }
                            stage('Patience - Do some work') {
                                steps {
                                    sleep 2
                                    sh '''
                                    echo "Project: $PROJECT"
                                    echo "Server: $SERVER"
                                    echo "Python version: $PYTHON_VERSION"
                                    echo "OS: $SYSTEM_OS"
                                    '''
                                }
                            }
                            stage('Patience - Do more work') {
                                steps {
                                    sh '''
                                    $WORKSPACE/project_builder_patience.sh
                                    '''
                                }
                            }
                        }
                    }
                    stage('Fortitude') {
                        agent { label 'amazon_linux2' }
                        stages {
                            stage('Fortitude - Reconfiguration') {
                                steps {
                                    sleep 1
                                    script {
                                        localPipelineConfig('fortitudeSettings.yml')
                                        SYSTEM_OS = 'al2'
                                    }
                                }
                            }
                            stage('Fortitude - Do some work') {
                                steps {
                                    sleep 2
                                    sh '''
                                    echo "Project: $PROJECT"
                                    echo "Server: $SERVER"
                                    echo "Python version: $PYTHON_VERSION"
                                    echo "OS: $SYSTEM_OS"
                                    '''
                                }
                            }
                            stage('Fortitude - Do more work') {
                                steps {
                                    sh '''
                                    $WORKSPACE/project_builder_fortitude.sh
                                    '''
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

vars/globalPipelineConfig.groovy

def call(configFileName) {
    Map pipelineCfg = readYaml(file: "${WORKSPACE}/${configFileName}")
    for (global_setting_item in pipelineCfg.global_settings) {
        env."$global_setting_item.key" = global_setting_item.value
    }
    return
}

globalSettings.yml

global_settings:
  PROJECT: "services"

vars/localPipelineConfig.groovy

def call(configFileName) {
    Map pipelineCfg = readYaml(file: "${WORKSPACE}/${configFileName}")
    for (local_setting_item in pipelineCfg.local_settings) {
        env."$local_setting_item.key" = local_setting_item.value
    }
    return
}

patienceSettings.yml

local_settings:
  PYTHON_VERSION: python3.10

fortitudeSettings.yml

local_settings:
  PYTHON_VERSION: python3.11

project_builder_patience.sh

#!/bin/bash
echo "I would build Patience."

project_builder_fortitude.sh

#!/bin/bash
echo "I would build Fortitude."

Desired output on patience node:

Project: services
Server: builder.example.com
Python version: python3.10
OS: ubuntu
I would build Patience.

Actual output on patience node:

Project: services
Server: builder.example.com
Python version: python3.11
OS: 
I would build Patience.

Desired output on fortitude node:

Project: services
Server: builder.example.com
Python version: python3.11
OS: al2
I would build Fortitude.

Actual output on fortitude node:

Project: services
Server: builder.example.com
Python version: python3.11
OS: 
I would build Fortitude.