CDK to TF migration supporting changes (#125)

* PLAT-7142: CDK to TF migration support.

.gitignore

@@ -10,6 +10,7 @@
 *nodes.outputs*
 **migrated.txt
 **/deploy-test/*
+**terraform.plan
 
 **.terraform.lock.hcl*
 **.terraform.lock.hcl

README.md

@@ -18,6 +18,10 @@
   * Purpose: Ensures the integrity and functionality of the module.
   * Contents: Contains automation-driven tests intended for validation and continuous integration (CI) checks.
 
+* bin/state-migration/:
+  * Purpose: Contains automation to perform terraform state migration, from a monolithic module to a multi-module structure.
+  * Contents: Script and documentation to perform terraform state migration.
+
 Always refer to each section's respective README or documentation for detailed information and usage guidelines.
 
 ## Prerequisites
@@ -36,7 +40,7 @@ Update the following values:
 MOD_VERSION='v3.0.0'
 DEPLOY_DIR='domino-deploy'
 ```
-:warning: Ensure the DEPLOY_DIR does not exist or is currently empty.
+:warning: Ensure the `DEPLOY_DIR` does not exist or is currently empty.
 
 ```bash
 mkdir -p "$DEPLOY_DIR"
@@ -52,6 +56,7 @@ with Terraform immediately by creating Terraform configuration files.
 ```
 
 If successful, you should get a structure similar to this:
+
 ```bash
 domino-deploy
 ├── README.md
@@ -79,7 +84,7 @@ domino-deploy
 └── tf.sh
 ```
 
-**Note**: It's recommended to go through the README.md within the DEPLOY_DIR for further details.
+**Note**: It's recommended to go through the README.md within the `DEPLOY_DIR` for further details.
 
 ### 2. Update modules version
 You can update the modules version using a script or manually.
@@ -106,14 +111,119 @@ For example if `MOD_VERSION=v3.0.0`
 * **nodes/main.tf** : Update `module.nodes.source` from `"./../../../../modules/nodes"` to `github.com/dominodatalab/terraform-aws-eks.git//modules/nodes?ref=v3.0.0`
 
 
-### 3. Review and Configure `infra.tfvars`
-Your initial setup is guided by the Terraform variables in `domino-deploy/terraform/infra.tfvars`. Ensure you review and modify this file as needed.
+### 3. Review and Configure `tfvars`
+
+Consult available variables within each of the modules `variables.tf`
+
+* `domino-deploy/terraform/infra/variables.tf`
+  * `deploy_id`
+  * `region`
+  * `tags`
+  * `network`
+  * `default_node_groups`
+  * `additional_node_groups`
+  * `storage`
+  * `kms`
+  * `eks`
+  * `ssh_pvt_key_path`
+  * `route53_hosted_zone_name`
+  * `bastion`
+
+* `domino-deploy/terraform/cluster/variables.tf`
+  * `eks`
+  * `kms_info`: :warning: Variable is only intended for migrating infrastructure, it is not recommended to set it.
+
+* `domino-deploy/terraform/nodes/variables.tf`
+  * `default_node_groups`
+  * `additional_node_groups`
+
+Configure terraform variables at:
+
+* `domino-deploy/terraform/infra.tfvars`
+* `domino-deploy/terraform/cluster.tfvars`
+* `domino-deploy/terraform/nodes.tfvars`
+
+**NOTE**: The `eks` configuration is required in both the `infra` and `cluster` modules because the Kubernetes version is used for installing the `kubectl` binary on the bastion host. Similarly, `default_node_groups` and `additional_node_groups` must be defined in both the `infra` and `nodes` modules, as the `availability zones` for the `nodes` are necessary for setting up the network infrastructure.
+The `eks` module will source its information from the `infra` outputs if it is not configured on `cluster.tfvars`, as will the `nodes` module if the variables are not configured on `nodes.tfvars`. We recommended setting the variables in `eks` and `nodes` from the beggining as future kubernetes upgrades will be driven from `cluster.tfvars` and `nodes.tfvars`.
 
 
 ### 4. Create SSH Key pair
-The deployment requires an SSH key. Update the `ssh_pvt_key_path` variable in `domino-deploy/terraform/infra.tfvars` with the full path of your key.
+The deployment requires an SSH key. Update the `ssh_pvt_key_path` variable in `domino-deploy/terraform/infra.tfvars` with the full path of your key (we recommend you place your key under the `domino-deploy/terraform` directory).
 
 If you don't have an SSH key, you can create one using:
 ```bash
  ssh-keygen -q -P '' -t rsa -b 4096 -m PEM -f domino.pem && chmod 600 domino.pem
 ```
+
+### 5. Deploy
+#### 1. Set `AWS` credentials and verify.
+```bash
+aws sts get-caller-identity
+```
+
+#### 2. Change into `domino-deploy`(or whatever your `DEPLOY_DIR` is)
+
+```bash
+cd domino-deploy
+```
+
+#### 3. Plan and Apply.
+:warning: It is recommended to become familiar with the `tf.sh` [usage](./examples/deploy/README.md#usage).
+
+At this point all requirements should be set to provision the infrastructure.
+
+For each of the modules, run `init`, `plan`, inspect the plan, then `apply` in the following order:
+
+1. `infra`
+2. `cluster`
+3. `nodes`
+
+Note: You can use `all` instead but it is recommended that the `plan`  and `apply` be done one at a time, so that the plans can be carefully examined.
+
+1. Init all
+
+```bash
+./tf.sh all init
+```
+
+2. `infra` plan.
+
+```bash
+./tf.sh infra plan
+```
+3. :exclamation: Carefully inspect the actions detailed in the `infra` plan for correctness, before proceeding.
+
+4. `infra` apply
+
+```bash
+./tf.sh infra apply
+```
+
+5. `cluster` plan
+
+```bash
+./tf.sh cluster plan
+```
+
+6. :exclamation: Carefully inspect the actions detailed in the `cluster` plan for correctness, before proceeding.
+
+7. `cluster` apply
+
+```bash
+./tf.sh cluster apply
+```
+
+8. nodes plan
+
+```bash
+./tf.sh nodes plan
+```
+9.  :exclamation: Carefully inspect the actions detailed in the `nodes` plan for correctness, before proceeding.
+
+10.  `nodes` apply
+
+```bash
+./tf.sh nodes apply
+```
+
+### At this point your deployment has been completed.

bin/state-migration/README.md

@@ -46,7 +46,7 @@ Ensure all prerequisites are met.
     * **LEGACY_STATE**: Path to the Terraform state file for the deployment you're migrating. This file is typically named `terraform.tfstate`.
 
 4. Run the script:
-  * Change onto `DEPLOY_DIR` and run the script
+  * Change into `DEPLOY_DIR` and run the script
   ```bash
   cd $DEPLOY_DIR
   ./migrate-states.sh

examples/deploy/README.md

@@ -1,7 +1,7 @@
 # Terraform Multi-Module Management
 
 ## Overview
-The `tf.sh` script provides a convenient method to manage multiple Terraform configurations for various components of a system. The primary modules managed by this script include `infra`, `cluster`, and `nodes`. These components might represent different layers of an infrastructure deployment.
+The `tf.sh` script provides a convenient method to manage multiple Terraform configurations for various components of a system. The primary modules managed by this script include `infra`, `cluster`, and `nodes`. These components might represent different layers of an infrastructure deployment. Similarly the `set-mod-version.sh` script helps to set the source module version on all three modules(`infra`, `cluster`, and `nodes`), see [README](../../README.md#Using_script).
 
 ## Pre-requisites
 * Ensure that `terraform` is installed and accessible in your path.
@@ -10,7 +10,7 @@ The `tf.sh` script provides a convenient method to manage multiple Terraform con
 ## Directory Structure
 The script expects the following directory structure:
 ```
-examples/deploy
+deploy
 ├── README.md
 ├── meta.sh
 ├── set-mod-version.sh
@@ -37,11 +37,11 @@ examples/deploy
 ```
 
 * Each subdirectory under `terraform` (e.g., `infra`, `cluster`, `nodes`) should contain its respective Terraform configurations.
-* Each component is expected to have a corresponding `.tfvars` file at the root directory. For instance, for the `infra` component, there should be an `infra.tfvars` in the root directory.
+* Each component is expected to have a corresponding `.tfvars` file at the `terraform` directory. For instance, for the `infra` component, there should be an `terraform/infra.tfvars` file.
 * Each of component's state and output(when the `output` command is invoked) is saved in the `terraform` directory:
 
 ```bash
-└── examples/deploy/terraform
+└─ deploy/terraform
    ├── cluster.outputs
    ├── cluster.tfstate
    ├── infra.outputs
@@ -51,56 +51,7 @@ examples/deploy
 ```
 ## Variables structure
 
-The modular design of this Terraform setup allows for a streamlined flow of variable values across different stages of your infrastructure, from the foundational `infra` module up to the more specialized `nodes` module.
-
-
-### Inter-module Variable Propagation
-
-1. **From `infra` to `cluster`**:
-   * The `infra` module is where most foundational variables are defined. Once provisioned, these variable values can be consumed by the `cluster` module using Terraform's [remote state data source](https://www.terraform.io/docs/language/state/remote-state-data.html).
-
-2. **From both `infra` and `cluster` to `nodes`**:
-   * The `nodes` module consumes variable values from both the `infra` and `cluster` modules. This is achieved by accessing their respective remote states.
-
-### infra.tfvars
-You can find examples in the examples/tfvars directory. This file accommodates all variables defined in modules/infra.
-
-### cluster.tfvars
-This file provides the capability to override the k8s_version variable, aiding in Kubernetes upgrades.
-
-### nodes.tfvars
-This file allows you to override two variables: default_node_groups and additional_node_groups, making it easier to update node groups.
-
-### Overriding Variables for Kubernetes Upgrades
-
-The ability to upgrade Kubernetes without affecting other infrastructure components is crucial for maintainability:
-
-```bash
-.
-├── README.md
-├── cluster.tfvars
-├── infra.tfvars
-├── nodes.tfvars
-├── terraform
-│   ├── cluster
-│   ├── infra
-│   └── nodes
-└── tf.sh
-```
-
-* The `cluster` module accepts a variable named `k8s_version` via the `cluster.tfvars`.
-* While the initial value of `k8s_version` comes from the `infra` module, you have the flexibility to overwrite it in the `cluster` module via the the `cluster.tfvars`. This facilitates Kubernetes version upgrades without making changes to the underlying infrastructure set up by the `infra` module.
-
-### Enhancing Flexibility in Node Configurations
-
-For node configurations and upgrades, the design follows a similar pattern:
-
-* The `nodes` module allows you to override the default node configurations (`default_node_groups`) and any additional node configurations (`additional_node_groups`).
-* This is done using the `merge` function, ensuring you can easily add or modify node groups as required.
-* In scenarios where only the node pool requires an update, you can simply modify the `nodes.tfvars` and run `./tf.sh nodes apply`. This avoids the need to re-apply the `infra` or `cluster` modules, streamlining node management.
-
-With this structure, the infrastructure maintains a clear hierarchy of variable propagation, ensuring ease of use, flexibility, and minimal disruptions during updates and upgrades.
-
+See [README](../../README.md#3-review-and-configure-tfvars)
 
 ## Usage
 
@@ -110,16 +61,20 @@ To use the script, invoke it with the desired command and component:
 ./tf.sh <component> <command>
 ```
 
-* **component**: The component you wish to apply the command on. Supported components are `infra`, `cluster`, `nodes`, and `all`. Using `all` will apply the command on all components.
+* **component**: The component parameter refers to the specific section of your architecture that you wish to target with a command. Supported components include `infra`, `cluster`, `nodes`, and `all`. Selecting all will execute the command across `infra`, `cluster` and `nodes`.
+  The script uses the component parameter to identify corresponding Terraform directories and to name both the Terraform variables file (`terraform/${component}.tfvars`) and the Terraform state file (`terraform/${component}.tfstate`). If you create a custom folder named `mydir` that includes your Terraform configuration, setup a terraform variables file(`terraform/mydir.tfstate`), and state file(`terraform/mydir.tfstate`) if existing, then you can utilize the tf.sh script to execute Terraform commands. For example, running `./tf.sh mydir plan`.
 
-* **command**: Supported commands include:
-  * init: Initializes the Terraform configurations.
-  * plan: Shows the execution plan of Terraform.
-  * apply: Applies the Terraform configurations.
-  * destroy: Destroys the Terraform resources.
-  * output: Shows the output values of your configurations.
-  * refresh: Refreshes the Terraform state file.
+  It's important to note that your custom directory, mydir, ***will not*** be included when using the `all` value for components.
 
+* **command**: Supported commands include:
+  * `init`: Initializes the Terraform configurations.
+  * `plan`: Shows the execution plan of Terraform.
+  * `apply`: Applies the Terraform configurations.
+  * `destroy`: Destroys the Terraform resources.
+  * `output`: Shows the output values of your configurations.
+  * `refresh`: Refreshes the Terraform state file.
+  * `plan_out`: Generates a plan and writes it to `terraform/${component}-terraform.plan`.
+  * `apply_plan`: Applies plan located at `terraform/${component}-terraform.plan`.
 
 ## Examples
 
@@ -141,6 +96,18 @@ To use the script, invoke it with the desired command and component:
 ./tf.sh all destroy
 ```
 
+* To perform a plan and write it to a file(the plan file will be stored at: `terraform/${component}-terraform.plan`):
+
+```bash
+./tf.sh cluster plan_out
+```
+
+* To apply a a previously generated plan stored at `terraform/${component}-terraform.plan` for this example `terraform/cluster-terraform.plan`:
+
+```bash
+./tf.sh cluster apply_plan
+```
+
 ## Common Operations
 
 For some frequently performed operations, follow the steps outlined below:
@@ -149,12 +116,12 @@ For some frequently performed operations, follow the steps outlined below:
 See the repo's [README](../../README.md#bootstrap-module) for how to bootstrap the module.
 
 ### Updating the modules' version
-See `README` [Update modules version](../../README.md#update-modules-version)
+See `README` [Update modules version](../../README.md#2-update-modules-version)
 
 ### Kubernetes Upgrade:
 In order to update Kubernetes we will need to update the `cluster` and the `nodes`.
 
-1. Update the `k8s-version` variable in the `cluster.tfvars` file.
+1. Set the `eks.k8s_version` variable to desired version(At most it can be 2 minor versions ahead.)
 2. Update cluster:
    1. Plan and review the changes:
       ```bash
@@ -165,6 +132,7 @@ In order to update Kubernetes we will need to update the `cluster` and the `node
       ./tf.sh cluster apply
       ```
 3. Update nodes:
+Given that the nodes source the k8s version from `eks` we just need to plan and apply.
    1. Plan and review the changes:
       ```bash
       ./tf.sh nodes plan
@@ -175,14 +143,12 @@ In order to update Kubernetes we will need to update the `cluster` and the `node
       ```
 
 ### Nodes Upgrade:
-In order to just update the nodes to the latest AMI for the existing version.
-
-1. Update nodes:
-   1. Plan and review the changes:
-      ```bash
-      ./tf.sh nodes plan
-      ```
-   2. Apply the changes:
-      ```bash
-      ./tf.sh nodes apply
-      ```
+Given that the nodes module looks for the latest AMI we just need to plan and apply:
+1. Plan and review the changes:
+   ```bash
+   ./tf.sh nodes plan
+   ```
+2. Apply the changes:
+   ```bash
+   ./tf.sh nodes apply
+   ```

examples/deploy/set-mod-version.sh

@@ -2,6 +2,11 @@
 set -euo pipefail
 
 validate_mod_version() {
+  [[ -n "${MOD_VALIDATION_OFF:-}" ]] && {
+    echo 'MOD_VALIDATION_OFF is set, skipping module version validation'
+    return
+  }
+
   url="https://api.github.com/repos/dominodatalab/terraform-aws-eks/tags"
 
   local curl_cmd=("curl" "-s")
@@ -48,7 +53,7 @@ MOD_VERSION="$1"
 [ -z "${MOD_VERSION// /}" ] && { echo "Provide a module version in the format $(vX.X.X), ie $(v3.0.0)" && exit 1; }
 
 SH_DIR="$(realpath "$(dirname "${BASH_SOURCE[0]}")")"
-
 source "${SH_DIR}/meta.sh"
+
 validate_mod_version
 set_module_version

examples/deploy/terraform/cluster.tfvars

@@ -1 +1,2 @@
-k8s_version = null
+eks      = null
+kms_info = null

examples/deploy/terraform/cluster/README.md

@@ -30,7 +30,8 @@
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_k8s_version"></a> [k8s\_version](#input\_k8s\_version) | Update k8s version. | `string` | `null` | no |
+| <a name="input_eks"></a> [eks](#input\_eks) | creation\_role\_name = Name of the role to import.<br>    k8s\_version = EKS cluster k8s version.<br>    kubeconfig = {<br>      extra\_args = Optional extra args when generating kubeconfig.<br>      path       = Fully qualified path name to write the kubeconfig file.<br>    }<br>    public\_access = {<br>      enabled = Enable EKS API public endpoint.<br>      cidrs   = List of CIDR ranges permitted for accessing the EKS public endpoint.<br>    }<br>    Custom role maps for aws auth configmap<br>    custom\_role\_maps = {<br>      rolearn  = string<br>      username = string<br>      groups   = list(string)<br>    }<br>    master\_role\_names  = IAM role names to be added as masters in eks.<br>    cluster\_addons     = EKS cluster addons. vpc-cni is installed separately.<br>    vpc\_cni            = Configuration for AWS VPC CNI<br>    ssm\_log\_group\_name = CloudWatch log group to send the SSM session logs to.<br>    identity\_providers = Configuration for IDP(Identity Provider).<br>  } | <pre>object({<br>    creation_role_name = optional(string, null)<br>    k8s_version        = optional(string)<br>    kubeconfig = optional(object({<br>      extra_args = optional(string)<br>      path       = optional(string)<br>    }), {})<br>    public_access = optional(object({<br>      enabled = optional(bool)<br>      cidrs   = optional(list(string))<br>    }), {})<br>    custom_role_maps = optional(list(object({<br>      rolearn  = string<br>      username = string<br>      groups   = list(string)<br>    })))<br>    master_role_names  = optional(list(string))<br>    cluster_addons     = optional(list(string))<br>    ssm_log_group_name = optional(string)<br>    vpc_cni = optional(object({<br>      prefix_delegation = optional(bool)<br>    }))<br>    identity_providers = optional(list(object({<br>      client_id                     = string<br>      groups_claim                  = optional(string)<br>      groups_prefix                 = optional(string)<br>      identity_provider_config_name = string<br>      issuer_url                    = optional(string)<br>      required_claims               = optional(string)<br>      username_claim                = optional(string)<br>      username_prefix               = optional(string)<br>    })))<br>  })</pre> | `null` | no |
+| <a name="input_kms_info"></a> [kms\_info](#input\_kms\_info) | Overrides the KMS key information. Meant for migrated configurations.<br>    {<br>      key\_id  = KMS key id.<br>      key\_arn = KMS key arn.<br>      enabled = KMS key is enabled.<br>    } | <pre>object({<br>    key_id  = string<br>    key_arn = string<br>    enabled = bool<br>  })</pre> | `null` | no |
 
 ## Outputs