TeamCity Ansible Vault Integration

Strahinja Kustudic -

TeamCity (TC) is a powerful CI/CD tool which we use a lot together with Ansible for host provisioning and application deployment. However, one hurdle we have come across with TeamCity, is that there is limited support for running Ansible playbooks. What is even more important for us is that it doesn't have any support Ansible Vault, on which we rely heavily for managing secrets.

To be able to use Vault with TC, we would somehow need to pass TC the Vault password used for encrypting secrets. This is obviously far from ideal. What our developers usually did as a workaround is:

  1. Set a TC parameter with the value of the Vault password
  2. Create a file with the content of the Vault password parameter
  3. While running ansible-playbook pass --vault-password-file pointing to the created file
  4. After everything is over, delete the vault password file, just to be safe so that we don't keep it on the TC agent.

This wasn't hard to do, but it was repetitive, and there is a risk for the last step to be forgotten, and therefore a security concern. That's why we decided to add support for Ansible Vault. It turns out it's very easy to do.

On every TC agent that runs Ansible we created a Bash script /opt/teamcity-agent/bin/ansible-vault-env with the following content:

#!/bin/bash

echo "${ANSIBLE_VAULT_PASSWORD:-vault_pass_not_set}"
We needed to set the default value of this variable to vault_pass_not_set string, because Ansible doesn't allow the Vault password to be an empty string.

And make sure this file is an executable:

sudo chmod +x /opt/teamcity-agent/bin/ansible-vault-env

Then on each TC agent we set the following environment variable pointing to that script:

ANSIBLE_VAULT_PASSWORD_FILE="/opt/teamcity-agent/bin/ansible-vault-env"

You can set this in many different places, depending on what distribution you are using, but if you are using a Linux distribution that uses systemd, you can set it in the TC agent unit file. Here is our example:

# /etc/systemd/system/teamcity-agent.service
[Unit]
Description="TeamCity Agent"

[Service]
User=teamcity
EnvironmentFile=/etc/sysconfig/teamcity-agent
ExecStart=/opt/teamcity-agent/app/bin/agent.sh run
ExecStop=/opt/teamcity-agent/app/bin/agent.sh stop
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

and /etc/sysconfig/teamcity-agent contains:

# Other variables

ANSIBLE_VAULT_PASSWORD_FILE="/opt/teamcity-agent/bin/ansible-vault-env"

then restart TC agent:

sudo systemctl restart teamcity-agent

ANSIBLE_VAULT_PASSWORD_FILE is a variable which Ansible reads to get the path to the Ansible Vault file (it's basically a replacement for the --vault-password-file parameter). If that Vault file is an executable, which it is in our case, it will run, and then its output will be used as the Vault password. Because that script only prints the value of the ANSIBLE_VAULT_PASSWORD environment variable, which we will set in a TC build as the Vault password, that means Ansible will get the Vault password. The nicest thing about it is that the TC agent only knows about the Vault password during the execution of the build, so there is no need for creating, nor cleaning any Vault password files.

Now, the only thing that we need to do in a TC build to be able to use Ansible Vault, is to define an environment parameter ANSIBLE_VAULT_PASSWORD to the value of your Vault passphrase. To do that, you just need to edit the configuration of your build, click on Parameters on the sidebar and create an Environment variable, like this:

That's it! Everything else will work out of the box and you won't need to pass any Vault parameters to ansible-playbook, nor clean up any credentials. Problem solved.