Merge pull request #40 from PremiereGlobal/deploy-shell

Deploy shell

.gitignore

@@ -1 +1,2 @@
 bin
+.kube

CHANGELOG.md

@@ -2,6 +2,11 @@
 
 ## 0.1.3
 
+### Improvements
+* Added `--method` to `stim deploy` allowing optional `shell` deployment.  Will auto-select the best options if left empty.
+  * `shell` deploy will be auto-selected if it detects that it is running in a container
+* Added configuration option for cache location. See [docs/CONFIG.md](docs/CONFIG.md)
+
 ### Bugfix
 * Fixed Pagerduty request to include all CLI parameters (for example source, component, etc.) as those were not actually being sent to Pagerduty previously
 

Dockerfile

@@ -13,6 +13,12 @@ RUN CGO_ENABLED=0 GOOS=${GOOS} go build -mod vendor -ldflags "-X github.com/Prem
 
 FROM alpine:latest
 
+ENV STIM_PATH=/stim
+ENV STIM_CACHE_PATH=/cache
+
+VOLUME /stim
+VOLUME /cache
+
 RUN apk --no-cache add ca-certificates
 
 COPY --from=builder /go/src/github.com/PremiereGlobal/stim/bin/stim /usr/bin

docs/CACHE.md

@@ -0,0 +1,8 @@
+## Cache
+Certain stim commands use caching to speed up operations.  The structure of the cache is as follows.
+```
+├── ${STIM_CACHE_PATH}/   # Set via environment variable
+│   ├── bin/              # Storage for binary executables
+│   │   ├── darwin/       # Versioned MacOS binaries
+│   │   ├── linux/        # Versioned Linux binaries
+```

docs/CONFIG.md

@@ -0,0 +1,38 @@
+# Configuration
+Stim can be configured in several ways.  Many options can be configured via environment variables, config files or command-line parameters.
+
+## Order of Presedence
+For all stim options:
+
+`CLI Options` *overrides* `Environment Variables` *overrides* `Config File`
+
+## Global
+These options configure where stim looks for and stores its core configuration data.
+|Env Var| Global CLI Option | Description | Default |
+|---|---|---|---|
+| `STIM_PATH` | `--path` | Path to the stim directory.  This is the default location for configuration files. | `${HOME}/.stim`|
+| `STIM_CACHE_PATH` | `--cache-path` | Path for caching data. See [CACHE.md](CACHE.md) for more details. | `${STIM_PATH}/cache` |
+| `STIM_CONFIG_FILE` | `--config` | Path for the global stim configuration file | `${STIM_PATH}/config.yaml`|
+
+### Stim Config File
+Additional configuration can be set in the `STIM_CONFIG_FILE`.
+
+| Option | Description | Type | Default |
+|---|---|---|---|
+| `path` |  | `string` | `token` |
+| `cache-path` |  | `string` | `token` |
+| `auth.method` | Method to use for authentication.  Currently this would be the Vault auth-backend to use. | `string` | `token` |
+| `aws.default-profile` | When fetching AWS credential, set to default AWS profile (in `~/.aws/credentials`). | `bool` | `false` |
+| `aws.ttl` | Default ttl to set when fetching AWS credentials. (ex. `24h`) | `duration` | `Vault Default Setting` |
+| `aws.use-profiles` | When fetching AWS credential, store the credentials as AWS profile (in `~/.aws/credentials`). | `bool` | `false` |
+| `aws.web-ttl` | TTL for AWS web logins. | `duration` | `AWS default` |
+| `logging.file.disable` | Option to disable file logging | `boolean` | `false` |
+| `logging.file.level` | File logging verbosity | `string` | `info` |
+| `logging.file.path` | File logging path | `string` | `info` |
+| `pagerduty.vault-apikey-key` | Vault key for the Pagerduty API key | `string` | ` ` |
+| `pagerduty.vault-apikey-path` | Vault path for the Pagerduty API key | `string` | ` ` |
+| `vault-address` | Address to be used for connecting with Vault | `string` | ` ` |
+| `vault-initial-token-duration` | Default token duration to use when authenticating with Vault | `duration` | `Vault Default Setting` |
+| `vault-username` | Default username to use when logging into Vault | `string` | `Vault Default Setting` |
+| `vault-username-skip-prompt` | Skip the username prompt if `vault-username` is set | `bool` | `false` |
+| `verbose` | Use verbose logging | `bool` | `false` |

docs/DEPLOY.md

@@ -2,10 +2,8 @@
 
 Stim can be used to deploy to Kubernetes using Vault to configure a deployment environment.
 
-> WARNING: this functionality has not been tested for automated deploys.  Use at your own risk.
-
 ## Prerequisites
-To use this functionality, the [Docker](https://docs.docker.com/install/) daemon must be installed and running on the machine.
+This functionality relies on a running [Docker](https://docs.docker.com/install/) daemon. It can also be run inside a container.
 
 ## Usage
 
@@ -18,6 +16,7 @@ To use this functionality, the [Docker](https://docs.docker.com/install/) daemon
 | `-f, --deploy-file` | Location of the deployment config file to use.  Defaults to `./stim.deploy.yaml` |
 | `-e, --environment` | Environment to deploy. If no value is provided, the user will be prompted. |
 | `-i, --instance` | Instance to deploy to. The special value of "all" can be specified to deploy to all environments. If no value is provided, the user will be prompted. |
+| `-m, --method` | Method to use for deployment.  Valid values are 'auto' 'docker' or 'shell'.  Auto will use docker if it is available or fall back to shell if not. 'shell' is not recommended unless in a controlled environment. (default "auto") |
 
 ## Configuration
 `stim deploy` is configured with a YAML file (`./stim.deploy.yaml` by default) that provides an inventory of the deployment environments as well as the configuration of those environments.
@@ -128,6 +127,7 @@ The *Spec* represents a set of environment configurations that determine where t
 | `kubernetes` | Kubernetes configuration | [Kubernetes](#kubernetes) | `false` | |
 | `env` | Static environment variables | [[]EnvVar](#envvar) | `false` | |
 | `secrets` | Secret configuration specification | [[]Secret](#secret) | `false` | |
+| `tools` | Configuration for CLI tools required for deployment | [Tools](#tools) | `false` | |
 
 ### Kubernetes
 
@@ -159,3 +159,21 @@ The *SecretSpec* type represents a definition of a Vault secret being pulled int
 | `set` | Key-value mappings of environment variable names to secret field names | `map[string]string` | `true` | |
 | `version` | The version to pull for Vault kv2 secrets.  Can be negative to "go back" x number of version.  For example, `-1` will pull the last previous version.  | `unsigned int` | `true` | |
 | `ttl` | The time-to-live, in seconds, for dynamic secrets. | `int`| `false` | |
+
+### Tools
+
+The *Tools* configuration specifies which CLI tools are required.
+
+| Field | Description | Type | Required | Default |
+| ----- | ----------- | ------ | -------- | -------- |
+| `helm` | Include if `helm` is required.   | [ToolSpec](#toolspec) | `false` | |
+| `kubectl` | Include if `kubectl` is required. Will match version to the cluster if `version` is not specified. | [ToolSpec](#toolspec) | `false` | |
+| `vault` | Include if `vault` is required. Will match version to the server if `version` is not specified. | [ToolSpec](#toolspec) | `false` | |
+
+### ToolSpec
+
+Describes the requirement of the CLI tool
+
+| Field | Description | Type | Required | Default |
+| ----- | ----------- | ------ | -------- | -------- |
+| `version` | The version of the tool needed. | `string` | `only for helm` | |

examples/deploy/basic/deploy.sh

@@ -1,5 +1,4 @@
 #!/bin/bash
-
 set -eu -o pipefail
 
 # Exit if not running with `stim deploy`

examples/deploy/jenkins/Jenkinsfile

@@ -0,0 +1,25 @@
+def label = "buildpod-${env.JOB_NAME}-${env.BUILD_NUMBER}".replace('_', '-').replace('/', '-')
+
+podTemplate(
+  label: label,
+  containers: [
+    containerTemplate(
+      name: 'stim',
+      image: 'premiereglobal/stim:latest',
+      command: 'cat',
+      ttyEnabled: true,
+      envVars: [ envVar( key: 'VAULT_ADDR', value: '<Vault URL for your company>')]
+    )
+  ]
+) {
+  node(label) {
+    checkout scm
+    stage('Deploy') {
+      container(name: 'stim') {
+        withCredentials([string(credentialsId: 'vault-default', variable: 'VAULT_TOKEN')]) {
+          sh 'stim deploy --environment dev --instance all'
+        }
+      }
+    }
+  }
+}

examples/deploy/jenkins/README.md

@@ -0,0 +1,4 @@
+# Jenkins deployment with Stim
+
+In this example we're deploying a Nginx container with Jenkins. This also uses Helm 3 and the stim version 1.3 that can run without the need of a second container in Jenkins. The goal is to have a simple Jenkinsfile with the logic in the deploy.sh so you can test locally.
+To run this example, simply run `stim deploy` from this directory.

examples/deploy/jenkins/deploy.sh

@@ -0,0 +1,14 @@
+#!/bin/sh
+set -eu -o pipefail
+
+# Exit if not running with `stim deploy`
+if [ ! ${STIM_DEPLOY+x} ]; then echo "Must be run with 'stim deploy'"; exit 1; fi
+
+# This script is meant to be run from a docker container
+helm repo add bitnami https://charts.bitnami.com/bitnami
+helm upgrade \
+  --debug \
+  --install \
+  --namespace ${NAMESPACE} \
+  --set image.tag=${IMAGE_TAG} \
+  nginx-test bitnami/nginx

examples/deploy/jenkins/stim.deploy.yaml

@@ -0,0 +1,21 @@
+# Configuration for kicking off the deployment (files and directories)
+global:
+  spec:
+    kubernetes:
+      serviceAccount: sre
+    tools:
+        helm:
+          version: "v3.0.2"
+    env:
+      - name: IMAGE_TAG
+        value: "latest"
+environments:
+  - name: dev
+    instances:
+      - name: devcluster
+        spec:
+          kubernetes:
+            cluster: devcluster.my-domain.com
+          env:
+          - name: NAMESPACE
+            value: default

examples/deploy/multi-cluster/deploy/helm.sh

@@ -1,5 +1,4 @@
 #!/bin/bash
-
 set -eu -o pipefail
 
 # Exit if not running with `stim deploy`

examples/deploy/multi-cluster/stim.deploy.yaml

@@ -4,13 +4,12 @@ deployment:
   script: helm.sh
 
 global:
-
   # Global spec (applies to all environments)
   spec:
-
-    # All deployments will be deployed with the same chart version
-  - name: HELM_CHART_VERSION
-    value: 3.8.5
+    env:
+      # All deployments will be deployed with the same chart version
+      - name: HELM_CHART_VERSION
+        value: 3.8.5
 
 # List of environment definitions
 environments:

go.mod

@@ -11,25 +11,31 @@ replace (
 )
 
 require (
+	github.com/Microsoft/go-winio v0.4.14 // indirect
 	github.com/PagerDuty/go-pagerduty v0.0.0-20191002190746-f60f4fc45222
-	github.com/PremiereGlobal/vault-to-envs v0.2.1
+	github.com/PremiereGlobal/vault-to-envs v0.2.2-0.20190928170516-b94151c229ae
 	github.com/aws/aws-sdk-go v1.25.6
 	github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
 	github.com/cornelk/hashmap v1.0.0
 	github.com/docker/distribution v2.7.1+incompatible // indirect
 	github.com/docker/docker v1.13.1
 	github.com/go-ini/ini v1.48.0
+	github.com/googleapis/gnostic v0.3.1 // indirect
+	github.com/gorilla/mux v1.7.3 // indirect
+	github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 // indirect
 	github.com/hashicorp/vault v1.2.3
 	github.com/hashicorp/vault/api v1.0.5-0.20190909201928-35325e2c3262
+	//	github.com/hashicorp/vault v1.0.2
 	github.com/imdario/mergo v0.3.8
+	github.com/krolaw/zipstream v0.0.0-20180621105154-0a2661891f94
 	github.com/manifoldco/promptui v0.3.2
 	github.com/mitchellh/go-homedir v1.1.0
 	github.com/morikuni/aec v1.0.0 // indirect
 	github.com/nicksnyder/go-i18n v1.10.0 // indirect
 	github.com/nlopes/slack v0.6.0
+	github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
 	github.com/prometheus/client_golang v1.1.0
 	github.com/skratchdot/open-golang v0.0.0-20190402232053-79abb63cd66e
-	github.com/spf13/afero v1.2.2 // indirect
 	github.com/spf13/cobra v0.0.5
 	github.com/spf13/viper v1.4.0
 	github.com/stretchr/testify v1.4.0 // indirect