Config Fetcher

This repo help us to fetch all configs from vault and consul and put them in a nested directory.
With using this one we can mock all configs or change and copy them to another environment.

Our stack use vault for getting secrets and some generic configs. Before to vault we fetch the consul to load configs.

In future we will have one config server to manage that in one place.

Usage

--consul-path string     consul path to load, multiple comma or space separated (CONFIG_CONSUL_PATH)
--consul-prefix string   consul prefix to load
--consul-save string     consul save to folder
--consul-set string      consul path to set
--vault-path string      vault path to load, multiple comma or space separated (CONFIG_VAULT_PATH)
--vault-prefix string    vault prefix to load
--vault-save string      vault save to folder

Run the following command to fetch all configs from vault and consul:

# load credentials
source env/local.sh

# fetch all configs from vault
cfetcher --vault-save=./out/${CONFIG_ENV}/finops-vault
# fetch all configs from consul
cfetcher --consul-save=./out/${CONFIG_ENV}/finops-consul

Build

Create CLI binary:

make build-linux build-windows

Mocking

Use turna tool to mock the loaded configuration for vault and consul.

This mock file designed for vault and consul configs run on same turna server.

docker run --rm -it \
-e LOG_LEVEL=debug \
-p 8080:8080 \
-v $(pwd)/out/test/finops-consul:/finops-consul -v $(pwd)/out/test/finops-vault:/finops-vault \
-v $(pwd)/mock/turna.yaml:/turna.yaml \
ghcr.io/rakunlabs/turna:latest

Test mocking configs

make run-load

Development

Testing Vault

First create a vault server

make vault

After that login

export VAULT_ADDR=http://127.0.0.1:8200
TOKEN=$(docker logs vault |& grep "Root Token:" | cut -d":" -f2 | xargs) make vault-login

Create approle with a secret values, get role-id and secret in the output

make vault-role-enable vault-role vault-secret

For testing set this env values

export VAULT_ROLE_ID=xxx
export VAULT_ROLE_SECRET=xxx