I’m running a Docker container of Jenkins with Ansible in the same image.
Ansible is updating another Docker container on the same Docker network.
Useing an ssh key file.
Tested using root user and user jenkins
Both users tested. I can ssh from the Jenkins/Ansible container to the target container using the CLI. I can run the playbook on the CLI and it works, it’s only when I run the playbook through Jenkins pipline it failes.
If I run the same command that Jenkins tries, (copied from the Jenkins console). The only change I made was the tmp ssh key to the real key. The temp key comes from Jenkins credentials. The playbook works fine.
--private-key /var/lib/jenkins/workspace/TestJs/ssh9809173197875714932.key
Real Key
--private-key /var/lib/jenkins/.ssh/jenkins_key
/usr/bin/ansible-playbook /var/lib/jenkins/workspace/TestJs/deploy/playbooks/deployToTarget.yml -i /var/lib/jenkins/workspace/TestJs/deploy/playbooks/inventory.inv --private-key /var/lib/jenkins/.ssh/jenkins_key -u jenkins
The above call on the Jenkins Docker cli works as expected.
To rule out the Jenkins temp key and validate the Jenkins key credentials, I wrote a simple pipline that connects to the host directly and runs a shell script, that connects fine and work as expected.
Pipeline Snippet
stages {
stage('Deploy on Test') {
steps {
ansiblePlaybook( credentialsId: 'TEST_HOST_SSH_KEY',
disableHostKeyChecking: true,
installation: 'Ansible',
inventory: "${WORKSPACE}/deploy/playbooks/inventory.inv",
playbook: "${WORKSPACE}/deploy/playbooks/deployToBSI_SMA.yml",
colorized: true)
}
}
}
The console shows this error everytime. I admit I am struggling with this one.
PLAY [Deploy On Test] ***************************************************
TASK [Gathering Facts] *********************************************************
[1;31mfatal: [bsi_sma]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: muxserver_listen: link mux listener /var/lib/jenkins/.ansible/cp/119f480311.4lIaQB6LhOC2svmf => /var/lib/jenkins/.ansible/cp/119f480311: Bad file descriptor", "unreachable": true}[0m
PLAY RECAP *********************************************************************
[0;31mbsi_sma[0m : ok=0 changed=0 [1;31munreachable=1 [0m failed=0 skipped=0 rescued=0 ignored=0
Any help is much appreciated; I have spent far too many days on this issue.
UPDATE:
Originally, tested both users in the CLI, as in -u jenkins or -u root
Both work while I’m in the CLI as root
If I run su jenkins and try running the same command in the CLI, I get the same error.
So the issue appears to be who the command is run under; Jenkins is running as user Jenkins.
UPDATE fixed it:
Jenkins was not picking up the Ansible configuration in the playbook
The CLI was of course
I added this to the Jenkins pipeline and it fixed it
environment {
ANSIBLE_CONFIG = "${WORKSPACE}/deploy/playbooks/ansible.cfg"
}
Jenkins running /usr/bin/ansible-playbook, was not picking up the Ansible configuration inside the playbook directory.
The CLI was, of course picking up the configuration.
I added this to the Jenkins pipeline, and it fixed it.
environment {
ANSIBLE_CONFIG = "${WORKSPACE}/deploy/playbooks/ansible.cfg"
}
I also added this to my Ansible config.
ssh_args= -o ControlMaster=no
Now, it works as expected.
However, I’m unsure why it does not work if a control_path is set.
This is often related to the control path used by SSH for connection sharing.
It looks like an issue with SSH connection multiplexing when Ansible runs in Jenkins.
To fix it, add this to your Ansible configuration (ansible.cfg):
[ssh_connection]
control_path = %(directory)s/%%h-%%p-%%r
control_persist = no
Disabling control_persist avoids the use of control sockets that can cause “Bad file descriptor” issues inside containers.
Make sure that the Jenkins pipeline uses the correct, persistent key (/var/lib/jenkins/.ssh/jenkins_key) rather than a temporary one.
Also, the Jenkins user should have write access to the Ansible control path directory (/var/lib/jenkins/.ansible/cp/).
That should do it.
2