Kubetruth

The CloudTruth integration for kubernetes that pushes parameter updates into kubernetes resources - usually ConfigMaps and Secrets, but any resource is allowed. The goal is to provide you a mechanism that is as hands off as possible, using naming conventions to automate the delivery of configuration so that you don't have to jump through setup hoops for each app/service/etc that you would like to configure with CloudTruth.

Installation

helm repo add cloudtruth https://packages.cloudtruth.com/charts/
helm install \
    --set appSettings.apiKey=<api_key> \
    --set projectMappings.root.environment=<environment> \
    kubetruth cloudtruth/kubetruth

Note that the helm release name (kubetruth above) is used in generating the names of the resources created at install time. Thus in the examples below, a name like kubetruth-root for the default installed CRD would be different in your system if you gave helm install a different release name.

Update

helm repo update
helm upgrade kubetruth cloudtruth/kubetruth

Uninstall

helm delete kubetruth
helm repo remove cloudtruth

Usage

Parameterize the helm install with --set * or --values yourConfig.yaml to control how kubetruth matches against your organization's naming conventions:

Parameter	Description	Type	Default	Required
appSettings.apiKey	The CloudTruth api key. Read only access is sufficient	string	n/a	yes
appSettings.pollingInterval	Interval to poll CloudTruth api for changes	integer	300	no
appSettings.debug	Debug logging and behavior	flag	false	no
secret.create	Create the kubernetes Secret containing the supplied api-key	boolean	true	no
secret.name	The name of the kubernetes Secret to create	string	<helm fullname>	no
projectMappings.root.environment	The CloudTruth environment to lookup parameter values for.	string	default	yes
projectMappings.root.project_selector	A regexp to limit the projects acted against (client-side). This performs a substring match, so use regexp anchors to match the entire string, e.g. "^myProject$". Supplies any named matches for template evaluation	string	""	no
projectMappings.root.key_selector	A regexp to limit the keys acted against (client-side). This performs a substring match, so use regexp anchors to match the entire string, e.g. "^myKey$". Supplies any named matches for template evaluation	string	""	no
projectMappings.root.tag	The version tag used when querying for parameters	string	none	no
projectMappings.root.skip	Skips the generation of resources for the selected projects	flag	false	no
projectMappings.root.suppress_namespace_inheritance	Prevents the CRD from the primary namespace from being inherited by secondary namespaces	flag	false	no
projectMappings.root.log_level	Sets the kubetruth logging level while handling the selected projects	enum(debug, info, warn, error, fatal)	as set by cli	no
projectMappings.root.included_projects	Include the parameters from other projects into the selected ones. This can be recursive in a depth first fashion, so if A imports B and B imports C, then A will get B's and C's parameters. For key conflicts, if A includes B and B includes C, then the precendence is A overrides B overrides C. If A includes [B, C], then the precendence is A overrides C overrides B.	list	[]	no
projectMappings.root.context	Additional variables made available to the resource templates. Can also be templates	map	default	no
projectMappings.root.active_templates	Selects the templates that should be active, includes all templates when nil, none when empty	list	nil	no
projectMappings.root.resource_templates	The templates to use in generating kubernetes resources (ConfigMap/Secrets/other)	map	default	no
projectMappings.<override_name>.*	Define override mappings to override settings from the root selector for specific projects. When doing this on the command-line (e.g. for helm install), it may be more convenient to use --values <file> instead of --set for large data sets	map	{}	no

With the default resource_templates, Kubetruth maps the parameters from CloudTruth Projects into ConfigMaps and Secrets of the same names as the Projects. Kubetruth will not overwrite any existing kubernetes resources that do not have the label app.kubernetes.io/managed-by: kubetruth. If you have some that you want kubetruth to manage, then either add the label or delete them manually.

For example, for a CloudTruth layout that looks like:

myProject:

oneParam=value1
twoParam=value2

otherProject:

someParam=value3
mySecret=value4 (marked as a secret within CloudTruth)

Kubetruth will generate the kubernetes resources:

ConfigMap named myProject:

    oneParam: value1
    twoParam: value2

ConfigMap named otherProject:

    someParam: value3

Secret named otherProject:

    mySecret: value4

These kubernetes resources can then be referenced in the standard ways.

To use them as environment variables in a pod:

    envFrom:
      - configMapRef:
          name: otherProject
    envFrom:
      - secretRef:
          name: otherProject

To use them as files on disk in a pod:

      containers:
        - name: myProject
          volumeMounts:
            - name: config-volume
              mountPath: /etc/myConfig
      volumes:
        - name: config-volume
          configMap:
            name: myProject

Note that config map updates don't get seen by a running pod. You can use something like Reloader to automate restarting the pod on a ConfigMap change, or read config from mounted volumes for configmaps/secrets, which do get updated automatically in a running pod. Alternatively, you make your ConfigMap's name include a checksum of its contents, and patch a pre-existing Deployment/Pod whenever that name changes. Similary, you can also make use of the checksum in the name when using kubetruth to deploy

Additional configuration

Kubetruth uses a CustomResourceDefinition called ProjectMapping(.kubetruth.cloudtruth.com) for additional configuration. The ProjectMapping CRD has two types identified by the scope property, the root scope and the override scope. The root scope is required, and there can be only one per namespace (see below). It sets up the global behavior for mapping the CloudTruth projects to kubernetes resources. You can edit it in the standard ways, e.g. kubectl edit projectmapping kubetruth-root. The override scope allows you to override the root scope's behavior for those CloudTruth projects whose names match its project_selector pattern.

For the project_selector, which is defined under the Usage section, you use a regex to match against CloudTruth project names, e.g. "^service" will match any project that starts with service. Since this does substring matching, if you want to match a specific CloudTruth project name you should use regex anchors to ensure you don't match more than expected. For example, if you have a CloudTruth project called kubernetes_secrets, you'll want the project_selector value to be "^kubernetes_secrets$" so you don't accidentally match another project named not_kubernetes_secrets.

Note that Kubetruth watches for changes to ProjectMappings, so touching any of them wakes it up from a polling sleep. This makes it quick and easy to test out configuration changes without having a short polling interval. You can also force a wakeup by execing the wakeup script in the running container:

kubectl exec deployment/kubetruth -- wakeup

To customize how the kubernetes resources are generated, edit the resource_templates property in the ProjectMappings. These templates are processed using the Liquid template language, and can reference the following liquid variables:

Liquid Variables	Description
template	The name of the template currently being rendered.
kubetruth_namespace	The namespace kubetruth is installed in.
mapping_namespace	The namespace that the current set of mappings exist in.
project	The project name.
project_heirarchy	The included_projects tree that this project includes. (useful to debug when using complex included_projects)
debug	Indicates if kubetruth is operating in debug (logging) mode.
parameters	The CloudTruth parameters from the project.
parameter_origins	The projects each parameter originates from. (useful to debug when using complex included_projects)
secrets	The CloudTruth secrets from the project.
secret_origins	The projects each secret originates from. (useful to debug when using complex included_projects)
templates	The CloudTruth templates from the project keyed by their name, e.g. templates.mytemplate . Evaluated with the environment from the active ProjectMapping
templates.names	The names of the CloudTruth templates for the project
context	A hash of context variables supplied from ProjectMappings. (useful to override portions of templates without having to replace them completely in an override)

In addition to the built in liquid filters, kubetruth also define a few custom ones:

Custom Filters	Description
dns_safe	Ensures the string is safe for use as a kubernetes resource name (i.e. Namespace/ConfigMap/Secret names)
env_safe	Ensures the string is safe for setting as a shell environment variable
key_safe	Ensures the string is safe for use as a key inside a ConfigMap/Secret data hash
indent: count	Indents each line in the argument by count spaces
nindent: count	Adds a leading newline, then indents each line in the argument by count spaces
stringify	Converts argument to a string safe to use in yaml (escapes quotes and surrounds with the quote character)
parse_yaml	Parses yaml string into a structured representation
to_yaml	Converts object to a yaml representation
parse_json	Parses json string into a structured representation
to_json	Converts object to a json representation
encode64	The argument bas64 encoded
decode64	The argument bas64 decoded
sha256	The sha256 digest of the argument
inflate	Converts a map of key/values into a nested data structure based on a delimiter in the key name, e.g. {foo.baz.bum: 2} => {foo: {bar: {baz: 2}}} Inverse of deflate
deflate	Converts a nested data structure into a single level map using a delimiter to indicate level in the key name, e.g. {foo: {bar: {baz: 2}}} => {foo.baz.bum: 2} Inverse of inflate
typify	Converts string values into primitive types (int, float, bool) where applicable for a nested data structure
merge	Combines two hashes into one, a.l.a ruby merge
re_replace	Regexp search and replace, a.l.a ruby gsub, e.g. "foo" | re_replace: "o+", "X"
re_contains	Returns a boolean for the regexp compare against the string target, e.g. "foo" | re_contains: "o+" => true

The default resource_templates make use of the context attribute to allow simpler modification of some common fields. These include:

Context Variables	Description
context.resource_name	set this in a ProjectMapping to supply a different name to the the default templates
context.resource_namespace	set this in a ProjectMapping to supply a different namespace to the default templates
context.skip_secrets	set this in a ProjectMapping to prevent output of a Secret resource even when secrets are present

Since the context is a freeform map type, you can add custom items to it, as well as architect any custom templates to make use of those custom items in the same way the default templates do. If the value of a context entry is a string, it is treated as a template, and can reference and set variables in the resource_template that is evaluating it. If the value is of some other yaml type (e.g. boolean/number/list/map), it will get passed through as that type to the template that is referencing it, so you can use it in more complex template logic like {% foreach item in context.my_items %}

The default resource_templates add the parameter_origins and project_heirarchy key as annotations on each kubernetes resource under management. This can be disabled by removing them from the template, or wrapping them in a test for debug. The data produced by these help to illustrate how project inclusion affects the project the resources were written for. It currently shows the project heirarchy and the project each parameter originates from, for example an entry like timeout: myService (commonService -> common) indicates that the timeout parameter is getting its value from the myService project, and if you removed it from there, it would then get it from the commonService project, and if you removed that, it would then get it from the common project.

Multi Instance Config

By default, Kubetruth is setup with a single set of ProjectMapping CRDs installed into the same namespace it was installed to. These are the primary CRDs. For systems that use independent kubernetes clusters per environment, this is all that you need. If, however, you'd like to be able to run multiple environments in the same cluster, you can make use of the multi-instance feature of kubetruth.

To do so, one simply needs to create ProjectMapping CRDs in namespaces other than the primary. These CRDs will automatically inherit the contents of the CRD of the same name from the primary namespace, and you can then selectively override the attributes you need to change for the supplemental instance. This allows you to reuse all the templates/logic/etc that you setup in the primary, and only have to change the differing dimension. See the environment example below

Example Config

The projectmapping resource has a shortname of pm for convenience when using kubectl.

Namespace per Project

To create kubernetes Resources in namespaces named after each Project:

kubectl edit pm kubetruth-root

and set the context.resource_namespace field:

spec:
    context:
        resource_namespace: '{{ project | dns_safe }}'

Or to do it with kubectl patch:

kubectl patch pm kubetruth-root --type json --patch '[{"op": "replace", "path": "/spec/context/resource_namespace", "value": "{{ project | dns_safe }}"}]'

Or to do it at install time, add the following to the helm install command:

--set projectMappings.root.context.resource_namespace="\{\{ project | dns_safe \}\}"

Share common data

To include the parameters from a Project named Base into all other projects, without creating Resources for Base itself:

# Set the included_project in the root mapping
kubectl patch pm kubetruth-root --type json --patch '[{"op": "replace", "path": "/spec/included_projects", "value": ["Base"]}]'

# Exclude the Base project by creating an override mapping that skips it:
kubectl apply -f - <<EOF
apiVersion: kubetruth.cloudtruth.com/v1
kind: ProjectMapping
metadata:
    name: exclude-base
spec:
    scope: override
    project_selector: "^Base$"
    skip: true
    included_projects: []
EOF

Customize naming of Resources

To override the naming of kubernetes Resources on a per-Project basis:

kubectl apply -f - <<EOF
apiVersion: kubetruth.cloudtruth.com/v1
kind: ProjectMapping
metadata:
  name: funkyproject-special-naming
spec:
    scope: override
    project_selector: funkyProject
    context:
        resource_name: notSoFunkyConfigMap
        resource_namespace: notSoFunkyNamespace
EOF

Environment per namespace

To setup an environment per namespace:

# Optionally disable output from the primary.  This is only neccessary if
# kubetruth is installed to a namespace that you don't want to generate
# resources in
kubectl patch pm kubetruth-root --type json --patch '[{"op": "replace", "path": "/spec/skip", "value": "true"}]'

# Tag each namespace that you'd like to have its own environment
# The name should be the same name as the primary root mapping created by the
# kubetruth installer
kubectl --namespace <your_namespace> apply -f - <<EOF
apiVersion: kubetruth.cloudtruth.com/v1
kind: ProjectMapping
metadata:
  name: <your_root_mapping_name>
spec:
  scope: root
  environment: <your_environment>
  skip: false
EOF

More specific project selection

To limit the Projects processed to those whose names start with service, except for serviceOddball:

kubectl patch pm kubetruth-root --type json --patch '[{"op": "replace", "path": "/spec/project_selector", "value": "^service"}]'

kubectl apply -f - <<EOF
apiVersion: kubetruth.cloudtruth.com/v1
kind: ProjectMapping
metadata:
  name: funkyproject-special-naming
spec:
  scope: override
  project_selector: serviceOddball
  skip: true
EOF

To see the ProjectMappings that have been setup

$ kubectl get pm
NAME                          SCOPE      PROJECT          AGE
exclude-base                  override   ^Base$           7m6s
funkyproject-special-naming   override   serviceOddball   13s
kubetruth-root                root       ^service         27m

$ kubectl describe pm kubetruth-root
Name:         kubetruth-root
Namespace:    default
<snipped>

Additional examples

More examples are available in the examples directory

Development

After checking out the repo, run bundle to install dependencies. Then, run bundle exec rspec to run the tests. You can also run bundle exec rake console for an interactive prompt that will allow you to experiment.

To install and run via helm in a local cluster:

mkdir local
cat > local/values.yml <<-EOF
image:
  repository: kubetruth
  pullPolicy: Never
  tag: latest
appSettings:
  debug: true
  apiKey: <your_api_key>
EOF

# requires minikube to be installed and running
rake install

# OR

# If using minikube, ensure that docker builds the image into the minikube container
# with the command:
# eval $(minikube docker-env)
#
docker build --release development -t kubetruth . && helm install \
    --set image.repository=kubetruth --set image.pullPolicy=Never --set image.tag=latest \
    --set appSettings.debug=true --set appSettings.apiKey=$CLOUDTRUTH_API_KEY \
    kubetruth ./helm/kubetruth/

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/cloudtruth/kubetruth.