Vault secret manager plugin

This is a GoCD Secrets plugin which allows users to use Vault as a secret manger for the GoCD server.

The plugin supports Version 2 of KV Secrets Engine.

Table of Contents

• Setup Vault using docker
• Configure the plugin
• Building the code base
• Troubleshooting

Setup Vault using docker

1. Run following command to start docker container for vault

docker run --cap-add=IPC_LOCK -e VAULT_DEV_ROOT_TOKEN_ID=some-token -p8200:8200  -d --name=dev-vault vault:latest

2. Once container is up and running exec to container in order to create secrets

export VAULT_ADDR='http://127.0.0.1:8200'
export VAULT_TOKEN='some-token'

The above environment variables are used by vault client to connect to the vault server.

3. Create secret

export BASE_PATH='secret/gocd'
vault kv put $BASE_PATH AWS_ACCESS_KEY=ABDASDKDLKM \
    AWS_SECRET_KEY=asdsfksfhkdfgfhfghhg; \
    GITHUB_TOKEN=97dasdd9789sd79sdf7sd9f7s9f9sd7f9sdvfd9gd9

Configure the plugin

The plugin needs to be configured with a secret config in order to connect to Vault. The configuration can be added from the Secrets Management page under Admin > Secret Management.

Alternatively, the configuration can be added directly to the config.xml using the configuration.

<secretConfigs>
    <secretConfig id="vault" pluginId="com.thoughtworks.gocd.secretmanager.vault">
      <description>All secrets for env1</description>
      <configuration>
        <property>
          <key>VaultUrl</key>
          <value>http://127.0.0.1:8200</value>
        </property>
        <property>
          <key>VaultPath</key>
          <value>secret/gocd</value>
        </property>
        <property>
          <key>AuthMethod</key>
          <value>token</value>
        </property>
        <property>
          <key>Token</key>
          <value>some-auth-token</value>
        </property>
      </configuration>
      <rules>
          <allow action="refer" type="environment">env_*</allow>
          <deny action="refer" type="pipeline_group">my_group</deny>
          <allow action="refer" type="pipeline_group">other_group</allow>
      </rules>
    </secretConfig>
</secretConfigs>

<rules> tag defines where this secretConfig is allowed/denied to be referred. For more details about rules and examples refer the GoCD Secret Management documentation

Field	Required	Description
VaultUrl	Yes	The url of the Vault server instance. If no address is explicitly set, the plugin will look to the VAULT_ADDR environment variable.
VaultPath	Yes	The root vault path which holds the secrets, and should be prepended to all secret values (e.g. secret/gocd)
ConnectionTimeout	No	The number of seconds to wait before giving up on establishing an HTTP(s) connection to the Vault server. If no openTimeout is explicitly set, then the object will look to the VAULT_OPEN_TIMEOUT environment variable. Defaults to 5 seconds.
ReadTimeout	No	Once connection has already been established, this is the number of seconds to wait for all data to finish downloading. If no readTimeout is explicitly set, then the object will look to the VAULT_READ_TIMEOUT environment variable. Defaults to 30 seconds.
ServerPem	No	An X.509 certificate, in unencrypted PEM format with UTF-8 encoding to use when communicating with Vault over HTTPS
AuthMethod	Yes	The auth method to use to authenticate with the Vault server, can be one of token, approle or cert
Token	No	Required if using token auth method. This is the token used to read secrets from Vault. Ensure this token has a longer ttl, the plugin will not be renewing the token.
RoleId	No	Required if using approle auth method. The plugins will use the configured RoleId and SecretId to authenticate with Vault.
SecretId	No	Required if using approle auth method.
ClientKeyPem	No	Required if using cert auth method. An RSA private key, in unencrypted PEM format with UTF-8 encoding.
ClientPem	No	Required if using cert auth method. An X.509 client certificate, in unencrypted PEM format with UTF-8 encoding.
Max Retries	No	Number of times to attempt to gather secrets from Vault. Defaults to 0.
Retry Interval Milliseconds	No	Duration between retry attempts (set by Max Retries). Defaults to 100 milliseconds.

Using the secret plugin in GoCD

Since version 1.3.0, the complete path used to look up the secret in Vault can be varied on individual secret retrieval, allowing a given Secret Configuration to be shared for all paths within a given root.

Sub-paths can be optionally configured by prefixing the key with subpath:, i.e the retrieved path is the concatenation of the plugin config VaultPath and the optional extra path specified in the secret key before :.

Assuming the the vault config's root VaultPath is secret/gocd, then:

• {{SECRET:[vault][my_key]}} looks up the key my_key in the secret at secret/gocd
• {{SECRET:[vault][my_server:my_key]}} looks up the key my_key in the secret at secret/gocd/my_server
• {{SECRET:[vault][/a/b/c/d:my_key]}} looks up the key my_key in the secret at secret/gocd/a/b/c/d
• {{SECRET:[vault][a:b:my_key]}} looks up the key a:b:my_key in the secret at secret/gocd (multiple :s are ignored)

Building the code base

To build the jar, run ./gradlew clean test assemble

Troubleshooting

Enable Debug Logs

If you are on GoCD version 19.6 and above:

Edit the file wrapper-properties.conf on your GoCD server and add the following options. The location of the wrapper-properties.conf can be found in the installation documentation of the GoCD server.

# We recommend that you begin with the index `100` and increment the index for each system property
wrapper.java.additional.100=-Dplugin.com.thoughtworks.gocd.secretmanager.vault.log.level=debug

If you're running with GoCD server 19.6 and above on docker using one of the supported GoCD server images, set the environment variable GOCD_SERVER_JVM_OPTIONS:

docker run -e "GOCD_SERVER_JVM_OPTIONS=-Dplugin.com.thoughtworks.gocd.secretmanager.vault.log.level=debug" ...

The plugin logs are written to LOG_DIR/plugin-com.thoughtworks.gocd.secretmanager.vault.log. The log dir

• on Linux is /var/log/go-server
• on Windows are written to C:\Program Files\Go Server\logs
• on docker images are written to /godata/logs