WIP: Vault and IAM integration

examples/vault-s3-private/Makefile

@@ -0,0 +1,49 @@
+.PHONY: init plan apply destroy clean
+
+.DEFAULT_GOAL = help
+
+# Hardcoding value of 3 minutes when we check if the plan file is stale
+STALE_PLAN_FILE := `find "tf.out" -mmin -3 | grep -q tf.out`
+
+## Check if tf.out is stale (Older than 2 minutes)
+check-plan-file:
+	@if ! ${STALE_PLAN_FILE} ; then \
+		echo "ERROR: Stale tf.out plan file (older than 3 minutes)!"; \
+		exit 1; \
+	fi
+
+## Runs terraform get and terraform init for env
+init:
+	@terraform get
+	@terraform init
+
+## use 'terraform plan' to map out updates to apply
+plan:
+	@terraform plan -out=tf.out
+
+## use 'terraform apply' to apply updates in a 'tf.out' plan file
+apply: check-plan-file
+	@terraform apply tf.out
+
+## use 'terraform destroy' to remove all resources from AWS
+destroy:
+	@terraform destroy
+
+## rm -rf all files and state
+clean:
+	@rm -f tf.out
+	@rm -f terraform.*.backup
+	@rm -f terraform.tfstate
+
+## Show help screen.
+help:
+	@echo "Please use \`make <target>' where <target> is one of\n\n"
+	@awk '/^[a-zA-Z\-\_0-9]+:/ { \
+		helpMessage = match(lastLine, /^## (.*)/); \
+		if (helpMessage) { \
+			helpCommand = substr($$1, 0, index($$1, ":")-1); \
+			helpMessage = substr(lastLine, RSTART + 3, RLENGTH); \
+			printf "%-30s %s\n", helpCommand, helpMessage; \
+		} \
+	} \
+	{ lastLine = $$0 }' $(MAKEFILE_LIST)

examples/vault-s3-private/README.md

@@ -0,0 +1,128 @@
+# Example showing Vault and IAM Integration
+
+This example creates a private s3 bucket resources. It then uses vault
+to create keys which only has access to those s3 buckets. The example
+code will create an IAM role with access to that bucket and will also
+configure vault so that we can dynamically generate credentials for
+accessing that bucket.
+
+## Requirements
+
+These are the required things for this example:
+
+* A running vault server. If you just want to experiment with this,
+  run a development server using:
+
+``` shellsession
+vault server -dev
+```
+
+* The AWS access and secret keys for an IAM user which the AWS Secret
+  Backend for Vault will use for issuing new credentials. If you don't
+  have any, you can create one using [vault-iam
+  module](../../modules/vault-iam). You need to put the access keys in
+  [variables.tf](./variables.tf)
+
+
+## Environment creation and deployment
+
+``` shellsession
+$ make init
+$ make plan
+$ make apply
+module.vault_aws_backend.vault_aws_secret_backend.aws: Creating...
+module.vault_aws_backend.vault_aws_secret_backend.aws: Creation complete after 0s [id=fpco/aws/dev/vault]
+aws_iam_role.vault_bucket_role: Creating...
+aws_s3_bucket.vault-test-bucket: Creating...
+aws_iam_role.vault_bucket_role: Still creating... [10s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [10s elapsed]
+aws_iam_role.vault_bucket_role: Still creating... [20s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [20s elapsed]
+aws_iam_role.vault_bucket_role: Creation complete after 22s [id=bucket_access_role]
+module.vault_aws_backend.vault_aws_secret_backend_role.aws_role: Creating...
+module.vault_aws_backend.vault_aws_secret_backend_role.aws_role: Creation complete after 0s [id=fpco/aws/dev/vault/roles/s3_app_user]
+aws_s3_bucket.vault-test-bucket: Still creating... [30s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [40s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [50s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [1m0s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [1m10s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [1m20s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [1m30s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [1m40s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [1m50s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [2m0s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [2m10s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [2m20s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [2m30s elapsed]
+aws_s3_bucket.vault-test-bucket: Still creating... [2m40s elapsed]
+aws_s3_bucket.vault-test-bucket: Creation complete after 2m48s [id=vault-fpco-test-bucket]
+aws_iam_role_policy.vault_bucket_policy: Creating...
+aws_iam_role_policy.vault_bucket_policy: Still creating... [10s elapsed]
+aws_iam_role_policy.vault_bucket_policy: Still creating... [20s elapsed]
+aws_iam_role_policy.vault_bucket_policy: Creation complete after 24s [id=bucket_access_role:bucket-policy]
+
+Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
+
+The state of your infrastructure has been saved to the path
+below. This state is required to modify and destroy your
+infrastructure, so keep it safe. To inspect the complete state
+use the `terraform show` command.
+
+State path: terraform.tfstate
+```
+
+## Testing
+
+Make sure you are already authorized with the vault server. If not,
+use `vault login` to do it. And then, you can dynamically create AWS
+credentials for accessing the s3 bucket you created:
+
+``` shellsession
+$ vault read fpco/aws/dev/vault/creds/s3_app_user
+Key                Value
+---                -----
+lease_id           fpco/aws/prod/vault/creds/s3_app_user/eJcLUNbpTNRFpLoTL9mEW76p
+lease_duration     14m59s
+lease_renewable    false
+access_key         xxx
+secret_key         xxx
+security_token     xxx
+```
+
+Now let's try to see all the files in our bucket:
+
+``` shellsession
+$ env AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=xxx AWS_SESSION_TOKEN=xxx aws s3 ls s3://vault-fpco-test-bucket
+```
+
+It gives you no output since there are no files. But the command
+works, which confirms us that the generated credentials are working as
+expected.
+
+Now let's try to do something for which you don't have access with the
+same credentials:
+
+``` shellsession
+$ env AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY=xxxx AWS_SESSION_TOKEN=xxx aws ec2 describe-instances --region="us-east-2"
+An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation.
+```
+
+That doesn't work, which is expected. Let's try to see if we can
+access files of some other buckets which is present:
+
+``` shellsession
+$ env AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=xxx AWS_SESSION_TOKEN=xxx aws s3 ls s3://some-other-existing-bucket
+An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied
+```
+
+## Destruction
+
+``` shellsession
+$ make destroy
+$ make clean
+```
+
+## Notes
+
+- This example was last tested with `Terraform v0.12.3`
+- This example assumes AWS credentials setup with access to the **us-east-2** region.

examples/vault-s3-private/main.tf

@@ -0,0 +1,67 @@
+resource "aws_s3_bucket" "vault-test-bucket" {
+  bucket = "vault-fpco-test-bucket"
+  acl    = "private"
+  region = "us-east-2"
+
+  tags = {
+    Name        = "Vault test bucket"
+    Environment = "Dev"
+  }
+}
+
+# Here we allow everyone to assume this role. In production systems
+# it's best to restrict it's scope so that only some IAM users are
+# able to assume this role.
+resource "aws_iam_role" "vault_bucket_role" {
+  name = "bucket_access_role"
+
+  assume_role_policy = <<EOF
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Action": "sts:AssumeRole",
+      "Principal": {
+        "AWS": "*"
+      },
+      "Effect": "Allow",
+      "Sid": ""
+    }
+  ]
+}
+EOF
+  tags = {
+    Environment = "Dev"
+  }
+}
+
+resource "aws_iam_role_policy" "vault_bucket_policy" {
+  name = "bucket-policy"
+  role = "${aws_iam_role.vault_bucket_role.id}"
+  policy = <<EOF
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": ["s3:*", "iam:CreateAccessKey"],
+      "Resource": ["${aws_s3_bucket.vault-test-bucket.arn}"]
+    }
+  ]
+}
+EOF
+}
+
+module "vault_aws_backend" {
+  source                    = "../../modules/vault-aws-backend/"
+  vault_address             = "${var.vault_address}"
+  vault_token               = "${var.vault_token}"
+  secret_backend_path       = "${var.secret_backend_path}"
+  default_lease_ttl_seconds = "${var.default_lease_ttl_seconds}"
+  max_lease_ttl_seconds     = "${var.max_lease_ttl_seconds}"
+  credential_type           = "${var.credential_type}"
+  role_name                 = "${var.role_name}"
+  role_arn                  = "${aws_iam_role.vault_bucket_role.arn}"
+  access_key                = "${var.access_key}"
+  secret_key                = "${var.secret_key}"
+}

examples/vault-s3-private/variables.tf

@@ -0,0 +1,56 @@
+variable "vault_address" {
+  description = "URL for the Vault server"
+  default = "http://127.0.0.1:8200"
+}
+variable "vault_token" {
+  description = "Vault token needed for authorization to the server"
+  default = "xxx"
+}
+
+variable "region" {
+  description = "AWS Region"
+  type        = "string"
+  default     = "us-east-2"
+}
+
+variable "secret_backend_path" {
+  description = "Unique AWS secret path for mouting"
+  type        = "string"
+  default     = "fpco/aws/dev/vault"
+}
+
+variable "default_lease_ttl_seconds" {
+  description = "The default TTL for credentials issued by this backend."
+  type    = "string"
+  default = "900"
+}
+
+variable "max_lease_ttl_seconds" {
+  description = "The maximum TTL that can be requested for credentials issued by this backend."
+  type    = "string"
+  default = "900"
+}
+
+variable "credential_type" {
+  description = "Specifies the type of credential to be used when retrieving credentials from the role."
+  type    = "string"
+  default = "assumed_role"
+}
+
+variable "access_key" {
+  description = "The AWS Access Key ID this backend should use to issue new credentials."
+  type    = "string"
+  default = "xxx"
+}
+
+variable "secret_key" {
+  description = "The AWS Secret Key this backend should use to issue new credentials."
+  type    = "string"
+  default = "xxx"
+}
+
+variable "role_name" {
+  description = "The name to identify this role within the backend. Must be unique within the backend."
+  type    = "string"
+  default = "s3_app_user"
+}

modules/vault-aws-backend/README.md

@@ -0,0 +1,6 @@
+# AWS Secret Backend for Vault
+
+This module creates an AWS secret backend and a role for the
+backend. Roles are used to map credentials to the policies that
+generated them. For an example of it's use, [see
+vault-s3-private](../../examples/vault-s3-private).

modules/vault-aws-backend/main.tf

@@ -0,0 +1,25 @@
+provider "aws" {
+  region = "${var.region}"
+}
+
+provider "vault" {
+  address = "${var.vault_address}"
+  token   = "${var.vault_token}"
+}
+
+resource "vault_aws_secret_backend" "aws" {
+  region                    = "${var.region}"
+  access_key                = "${var.access_key}"
+  secret_key                = "${var.secret_key}"
+  path                      = "${var.secret_backend_path}"
+  default_lease_ttl_seconds = "${var.default_lease_ttl_seconds}"
+  max_lease_ttl_seconds     = "${var.max_lease_ttl_seconds}"
+}
+
+resource "vault_aws_secret_backend_role" "aws_role" {
+  backend         = "${vault_aws_secret_backend.aws.path}"
+  name            = "${var.role_name}"
+  credential_type = "${var.credential_type}"
+  role_arns       = ["${var.role_arn}"]
+}
+

modules/vault-aws-backend/variables.tf

@@ -0,0 +1,55 @@
+variable "vault_address" {
+  description = "URL for the Vault server"
+  type        = "string"
+}
+
+variable "vault_token" {
+  description = "Vault token needed for authorization to the server"
+  type        = "string"
+}
+
+variable "region" {
+  description = "AWS Region"
+  type        = "string"
+  default     = "us-east-2"
+}
+
+variable "secret_backend_path" {
+  description = "Unique AWS secret path for mouting"
+  type        = "string"
+}
+
+variable "default_lease_ttl_seconds" {
+  description = "The default TTL for credentials issued by this backend."
+  type = "string"
+}
+
+variable "max_lease_ttl_seconds" {
+  description = "The maximum TTL that can be requested for credentials issued by this backend."
+  type = "string"
+}
+
+variable "credential_type" {
+  description = "Specifies the type of credential to be used when retrieving credentials from the role."
+  type = "string"
+}
+
+variable "access_key" {
+  description = "The AWS Access Key ID this backend should use to issue new credentials."
+  type = "string"
+}
+
+variable "secret_key" {
+  description = "The AWS Secret Key this backend should use to issue new credentials."
+  type = "string"
+}
+
+variable "role_name" {
+  description = "The name to identify this role within the backend. Must be unique within the backend."
+  type = "string"
+}
+
+variable "role_arn" {
+  description = "Specifies the ARN of the AWS role this Vault role is allowed to assume"
+  type    = "string"
+}