Deploy Configs: Single vs. Multiple; TOML vs. YAML

[mbklein]

We have a bunch of options when it comes to our SAM deploy config files, and all of them are better than what we're doing now.

Note

All of these options share one issue: Injecting the HoneybadgerRevision parameter into the overrides. This is because SAM doens't merge command line --parameter-overrides with the config file's parameter_overrides – if the command line option is used, it's as if the config file option doesn't exist. There are several open issues to address this, but there's been no consensus and no fix for several years.

Fortunately, each of the proposed changes involves having each parameter on its own line, so HoneybadgerRevision could be swapped in at deploy time with a simple sed command. (This is true of our current configuration as well, but there are other advantages.)

Proposed Options

1. Current Practice: Single samconfig.toml file with .parameters file per environment

Deploy command:

sam deploy \
  --no-confirm-changeset \
  --no-fail-on-empty-changeset \
  --config-env $CONFIG_ENV \
  --config-file ./samconfig.toml \
  --parameter-overrides $(while IFS='=' read -r key value; do params+=" $key=$value"; done < ./$CONFIG_ENV.parameters && echo "$params HoneybadgerRevision=$HONEYBADGER_REVISION")    

We settled on this mainly because SAM's default way of including parameters in the TOML file was ugly and we didn't know there was a better alternative:

parameter_overrides="\"Param1=Value1\" \"Param2=Value2\""

Pros: Manageable parameters with no inline escaping
Con: Ugly, complicated deploy command

2. Single TOML w/arrays of params

samconfig.toml

[staging]
[staging.deploy]
[staging.deploy.parameters]
stack_name = "dc-api-v2"
s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-foo"
s3_prefix = "dc-api-v2"
region = "us-east-1"
confirm_changeset = true
capabilities = "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
image_repositories = []
parameter_overrides = [
  "Param1=Value1",
  "Param2=Value2"
]

[production]
[production.deploy]
[production.deploy.parameters]
stack_name = "dc-api-v2"
s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-bar"
s3_prefix = "dc-api-v2"
region = "us-east-1"
confirm_changeset = true
capabilities = "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
image_repositories = []
parameter_overrides = [
  "Param1=Value1",
  "Param2=Value2"
]

Pros: Manageable parameters with no inline escaping
Cons: Harder to have one-off dev/test deploys without ballooning the file; parameter pairs still have to be quoted

3. Single YAML w/multiline string of params

samconfig.yaml

staging:
  deploy:
    parameters:
      stack_name: "dc-api-v2"
      s3_bucket: "aws-sam-cli-managed-default-samclisourcebucket-foo"
      s3_prefix: "dc-api-v2"
      region: "us-east-1"
      confirm_changeset: true
      capabilities: "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
      image_repositories: []
      parameter_overrides: >
        Param1=Value1
        Param2=Value2

production:
  deploy:
    parameters:
      stack_name: "dc-api-v2"
      s3_bucket: "aws-sam-cli-managed-default-samclisourcebucket-bar"
      s3_prefix: "dc-api-v2"
      region: "us-east-1"
      confirm_changeset: true
      capabilities: "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
      image_repositories: []
      parameter_overrides: >
        Param1=Value1
        Param2=Value2

Pros: Manageable parameters with no inline escaping; YAML > TOML
Cons: Harder to have one-off dev/test deploys without ballooning the file

4. TOML file per environment

samconfig.staging.toml

[default]
[default.deploy]
[default.deploy.parameters]
stack_name = "dc-api-v2"
s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-foo"
s3_prefix = "dc-api-v2"
region = "us-east-1"
confirm_changeset = true
capabilities = "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
image_repositories = []
parameter_overrides = [
  "Param1=Value1",
  "Param2=Value2"
]

samconfig.production.toml

[default]
[default.deploy]
[default.deploy.parameters]
stack_name = "dc-api-v2"
s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-bar"
s3_prefix = "dc-api-v2"
region = "us-east-1"
confirm_changeset = true
capabilities = "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
image_repositories = []
parameter_overrides = [
  "Param1=Value1",
  "Param2=Value2"
]

Deploy command would use --config-file samconfig.${CONFIG_ENV}.toml instead of --config-env $CONFIG_ENV

Pros: Each file is short and self-contained; easier to have one-off dev/test stacks
Cons: Parameter pairs still have to be quoted

5. YAML file per environment

samconfig.staging.yaml

default:
  deploy:
    parameters:
      stack_name: "dc-api-v2"
      s3_bucket: "aws-sam-cli-managed-default-samclisourcebucket-foo"
      s3_prefix: "dc-api-v2"
      region: "us-east-1"
      confirm_changeset: true
      capabilities: "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
      image_repositories: []
      parameter_overrides: >
        Param1=Value1
        Param2=Value2

samconfig.production.yaml

default:
  deploy:
    parameters:
      stack_name: "dc-api-v2"
      s3_bucket: "aws-sam-cli-managed-default-samclisourcebucket-bar"
      s3_prefix: "dc-api-v2"
      region: "us-east-1"
      confirm_changeset: true
      capabilities: "CAPABILITY_IAM CAPABILITY_AUTO_EXPAND"
      image_repositories: []
      parameter_overrides: >
        Param1=Value1
        Param2=Value2

Pros: Just like 4, but YAML
Cons: Maybe none?