PLAT-5622: Integrate with the deployer (#4)

Expose what is necessary for the deployer.
IAM performed by respective modules.
Cleanup paths

README.md

@@ -77,9 +77,9 @@ aws s3 rb s3://"${AWS_TERRAFORM_REMOTE_STATE_BUCKET}" --force
 
 | Name | Version |
 |------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.26.0 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.32.0 |
 | <a name="provider_null"></a> [null](#provider\_null) | 3.1.1 |
-| <a name="provider_tls"></a> [tls](#provider\_tls) | 4.0.1 |
+| <a name="provider_tls"></a> [tls](#provider\_tls) | 4.0.3 |
 
 ## Modules
 
@@ -96,9 +96,14 @@ aws s3 rb s3://"${AWS_TERRAFORM_REMOTE_STATE_BUCKET}" --force
 
 | Name | Type |
 |------|------|
+| [aws_iam_policy.route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
+| [aws_iam_role_policy_attachment.route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
 | [aws_key_pair.domino](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/key_pair) | resource |
 | [aws_availability_zones.available](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) | data source |
 | [aws_ec2_instance_type_offerings.nodes](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ec2_instance_type_offerings) | data source |
+| [aws_iam_policy_document.route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
+| [aws_iam_role.eks_master_roles](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_role) | data source |
+| [aws_route53_zone.hosted](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) | data source |
 | [null_data_source.validate_zones](https://registry.terraform.io/providers/hashicorp/null/latest/docs/data-sources/data_source) | data source |
 | [tls_public_key.domino](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/data-sources/public_key) | data source |
 
@@ -116,11 +121,12 @@ aws s3 rb s3://"${AWS_TERRAFORM_REMOTE_STATE_BUCKET}" --force
 | <a name="input_efs_access_point_path"></a> [efs\_access\_point\_path](#input\_efs\_access\_point\_path) | Filesystem path for efs. | `string` | `"/domino"` | no |
 | <a name="input_eks_master_role_names"></a> [eks\_master\_role\_names](#input\_eks\_master\_role\_names) | IAM role names to be added as masters in eks. | `list(string)` | `[]` | no |
 | <a name="input_k8s_version"></a> [k8s\_version](#input\_k8s\_version) | EKS cluster k8s version. | `string` | `"1.23"` | no |
+| <a name="input_kubeconfig_path"></a> [kubeconfig\_path](#input\_kubeconfig\_path) | fully qualified path name to write the kubeconfig file | `string` | `""` | no |
 | <a name="input_number_of_azs"></a> [number\_of\_azs](#input\_number\_of\_azs) | Number of AZ to distribute the deployment, EKS needs at least 2. | `number` | `3` | no |
 | <a name="input_private_cidr_network_bits"></a> [private\_cidr\_network\_bits](#input\_private\_cidr\_network\_bits) | Number of network bits to allocate to the private subnet. i.e /19 -> 8,192 IPs. | `number` | `19` | no |
 | <a name="input_public_cidr_network_bits"></a> [public\_cidr\_network\_bits](#input\_public\_cidr\_network\_bits) | Number of network bits to allocate to the public subnet. i.e /27 -> 32 IPs. | `number` | `27` | no |
 | <a name="input_region"></a> [region](#input\_region) | AWS region for the deployment | `string` | n/a | yes |
-| <a name="input_route53_hosted_zone_name"></a> [route53\_hosted\_zone\_name](#input\_route53\_hosted\_zone\_name) | AWS Route53 Hosted zone. | `string` | n/a | yes |
+| <a name="input_route53_hosted_zone_name"></a> [route53\_hosted\_zone\_name](#input\_route53\_hosted\_zone\_name) | Optional hosted zone for External DNSone. | `string` | `""` | no |
 | <a name="input_s3_force_destroy_on_deletion"></a> [s3\_force\_destroy\_on\_deletion](#input\_s3\_force\_destroy\_on\_deletion) | Toogle to allow recursive deletion of all objects in the s3 buckets. if 'false' terraform will NOT be able to delete non-empty buckets | `bool` | `false` | no |
 | <a name="input_ssh_pvt_key_path"></a> [ssh\_pvt\_key\_path](#input\_ssh\_pvt\_key\_path) | SSH private key filepath. | `string` | n/a | yes |
 | <a name="input_tags"></a> [tags](#input\_tags) | Deployment tags. | `map(string)` | `{}` | no |
@@ -130,14 +136,13 @@ aws s3 rb s3://"${AWS_TERRAFORM_REMOTE_STATE_BUCKET}" --force
 
 | Name | Description |
 |------|-------------|
-| <a name="output_deploy_id"></a> [deploy\_id](#output\_deploy\_id) | Deployment ID. |
-| <a name="output_efs_access_point_id"></a> [efs\_access\_point\_id](#output\_efs\_access\_point\_id) | EFS access\_point id |
-| <a name="output_efs_file_system_id"></a> [efs\_file\_system\_id](#output\_efs\_file\_system\_id) | EFS filesystem id |
-| <a name="output_efs_volume_handle"></a> [efs\_volume\_handle](#output\_efs\_volume\_handle) | EFS volume handle <filesystem id id>::<accesspoint id> |
+| <a name="output_bastion_ip"></a> [bastion\_ip](#output\_bastion\_ip) | n/a |
+| <a name="output_domino_key_pair"></a> [domino\_key\_pair](#output\_domino\_key\_pair) | Domino key pair |
+| <a name="output_efs_access_point"></a> [efs\_access\_point](#output\_efs\_access\_point) | EFS access point |
+| <a name="output_efs_file_system"></a> [efs\_file\_system](#output\_efs\_file\_system) | EFS file system |
 | <a name="output_hostname"></a> [hostname](#output\_hostname) | Domino instance URL. |
 | <a name="output_k8s_tunnel_command"></a> [k8s\_tunnel\_command](#output\_k8s\_tunnel\_command) | Command to run the k8s tunnel mallory. |
-| <a name="output_key_pair_name"></a> [key\_pair\_name](#output\_key\_pair\_name) | SSH key pair name. |
-| <a name="output_region"></a> [region](#output\_region) | Deployment region. |
-| <a name="output_s3_buckets"></a> [s3\_buckets](#output\_s3\_buckets) | S3 buckets name,arn. |
+| <a name="output_kubeconfig"></a> [kubeconfig](#output\_kubeconfig) | n/a |
+| <a name="output_s3_buckets"></a> [s3\_buckets](#output\_s3\_buckets) | S3 buckets |
 | <a name="output_ssh_bastion_command"></a> [ssh\_bastion\_command](#output\_ssh\_bastion\_command) | Command to ssh into the bastion host |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

iam.tf

@@ -0,0 +1,39 @@
+data "aws_route53_zone" "hosted" {
+  count        = var.route53_hosted_zone_name != "" ? 1 : 0
+  name         = var.route53_hosted_zone_name
+  private_zone = false
+}
+
+data "aws_iam_policy_document" "route53" {
+  count = var.route53_hosted_zone_name != "" ? 1 : 0
+  statement {
+
+    effect    = "Allow"
+    resources = ["*"]
+    actions   = ["route53:ListHostedZones"]
+  }
+
+  statement {
+
+    effect    = "Allow"
+    resources = [data.aws_route53_zone.hosted.0.arn]
+
+    actions = [
+      "route53:ChangeResourceRecordSets",
+      "route53:ListResourceRecordSets",
+    ]
+  }
+}
+
+resource "aws_iam_policy" "route53" {
+  count  = var.route53_hosted_zone_name != "" ? 1 : 0
+  name   = "${var.deploy_id}-Route53"
+  path   = "/"
+  policy = data.aws_iam_policy_document.route53.0.json
+}
+
+resource "aws_iam_role_policy_attachment" "route53" {
+  for_each   = var.route53_hosted_zone_name != "" ? toset([for r in module.eks.eks_node_roles : r.name]) : []
+  policy_arn = aws_iam_policy.route53.0.arn
+  role       = each.value
+}

main.tf

@@ -38,9 +38,8 @@ locals {
   available_azs_data = zipmap(data.aws_availability_zones.available.names, data.aws_availability_zones.available.zone_ids)
   # Getting the required azs name and id.
   bastion_user     = "ec2-user"
-  working_dir      = path.cwd
-  ssh_pvt_key_path = "${local.working_dir}/${var.ssh_pvt_key_path}"
-  kubeconfig_path  = "${local.working_dir}/kubeconfig"
+  ssh_pvt_key_path = abspath(pathexpand(var.ssh_pvt_key_path))
+  kubeconfig_path  = var.kubeconfig_path != "" ? abspath(pathexpand(var.kubeconfig_path)) : "${path.cwd}/kubeconfig"
 }
 
 # Validate that the number of offered and available zones satisfy the number of required zones. https://github.com/hashicorp/terraform/issues/31122 may result in a more elegant validation and deprecation of the null_data_source
@@ -88,28 +87,14 @@ module "network" {
   deploy_id                = var.deploy_id
   base_cidr_block          = var.base_cidr_block
   vpc_id                   = var.vpc_id
-  monitoring_s3_bucket_arn = module.storage.monitoring_s3_bucket_arn
-
+  monitoring_s3_bucket_arn = module.storage.s3_buckets["monitoring"].arn
 }
 
 locals {
   public_subnets  = module.network.public_subnets
   private_subnets = module.network.private_subnets
 }
 
-module "storage" {
-  source                       = "./submodules/storage"
-  deploy_id                    = var.deploy_id
-  efs_access_point_path        = var.efs_access_point_path
-  s3_force_destroy_on_deletion = var.s3_force_destroy_on_deletion
-  subnets = [for s in local.private_subnets : {
-    name       = s.name
-    id         = s.id
-    cidr_block = s.cidr_block
-  }]
-  vpc_id = module.network.vpc_id
-}
-
 module "bastion" {
   count = var.create_bastion ? 1 : 0
 
@@ -130,24 +115,41 @@ module "eks" {
   deploy_id                 = var.deploy_id
   private_subnets           = local.private_subnets
   ssh_pvt_key_path          = aws_key_pair.domino.key_name
-  route53_hosted_zone_name  = var.route53_hosted_zone_name
   bastion_security_group_id = try(module.bastion[0].security_group_id, "")
   create_bastion_sg         = var.create_bastion
   kubeconfig_path           = local.kubeconfig_path
   default_node_groups       = var.default_node_groups
   additional_node_groups    = var.additional_node_groups
-  s3_buckets                = module.storage.s3_buckets
+}
+
+module "storage" {
+  source                       = "./submodules/storage"
+  deploy_id                    = var.deploy_id
+  efs_access_point_path        = var.efs_access_point_path
+  s3_force_destroy_on_deletion = var.s3_force_destroy_on_deletion
+  subnets = [for s in local.private_subnets : {
+    name       = s.name
+    id         = s.id
+    cidr_block = s.cidr_block
+  }]
+  vpc_id = module.network.vpc_id
+  roles  = module.eks.eks_node_roles
+}
+
+data "aws_iam_role" "eks_master_roles" {
+  for_each = toset(var.eks_master_role_names)
+  name     = each.key
 }
 
 module "k8s_setup" {
-  source                  = "./submodules/k8s"
-  ssh_pvt_key_path        = abspath(local.ssh_pvt_key_path)
-  bastion_user            = local.bastion_user
-  bastion_public_ip       = try(module.bastion[0].public_ip, "")
-  k8s_cluster_endpoint    = module.eks.cluster_endpoint
-  managed_nodes_role_arns = module.eks.managed_nodes_role_arns
-  eks_master_role_names   = concat(var.eks_master_role_names, module.eks.eks_master_role_name)
-  kubeconfig_path         = local.kubeconfig_path
+  source               = "./submodules/k8s"
+  ssh_pvt_key_path     = abspath(local.ssh_pvt_key_path)
+  bastion_user         = local.bastion_user
+  bastion_public_ip    = try(module.bastion[0].public_ip, "")
+  k8s_cluster_endpoint = module.eks.cluster_endpoint
+  eks_node_role_arns   = [for r in module.eks.eks_node_roles : r.arn]
+  eks_master_role_arns = [for r in concat(values(data.aws_iam_role.eks_master_roles), module.eks.eks_master_roles) : r.arn]
+  kubeconfig_path      = local.kubeconfig_path
   depends_on = [
     module.eks,
     module.bastion

outputs.tf

@@ -3,6 +3,10 @@ output "ssh_bastion_command" {
   value       = "ssh -i ${local.ssh_pvt_key_path} -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no ${local.bastion_user}@${module.bastion[0].public_ip}"
 }
 
+output "bastion_ip" {
+  value = var.create_bastion ? module.bastion[0].public_ip : ""
+}
+
 output "k8s_tunnel_command" {
   description = "Command to run the k8s tunnel mallory."
   value       = module.k8s_setup.k8s_tunnel_command
@@ -13,37 +17,26 @@ output "hostname" {
   value       = "${var.deploy_id}.${var.route53_hosted_zone_name}"
 }
 
-output "efs_volume_handle" {
-  description = "EFS volume handle <filesystem id id>::<accesspoint id>"
-  value       = module.storage.efs_volume_handle
-}
-
-output "efs_access_point_id" {
-  description = "EFS access_point id"
-  value       = module.storage.efs_access_point_id
-}
-
-output "efs_file_system_id" {
-  description = "EFS filesystem id"
-  value       = module.storage.efs_file_system_id
-}
-
-output "region" {
-  description = "Deployment region."
-  value       = var.region
+output "efs_access_point" {
+  description = "EFS access point"
+  value       = module.storage.efs_access_point
 }
 
-output "deploy_id" {
-  description = "Deployment ID."
-  value       = var.deploy_id
+output "efs_file_system" {
+  description = "EFS file system"
+  value       = module.storage.efs_file_system
 }
 
 output "s3_buckets" {
-  description = "S3 buckets name,arn."
+  description = "S3 buckets"
   value       = module.storage.s3_buckets
 }
 
-output "key_pair_name" {
-  description = "SSH key pair name."
-  value       = aws_key_pair.domino.key_name
+output "domino_key_pair" {
+  description = "Domino key pair"
+  value       = aws_key_pair.domino
+}
+
+output "kubeconfig" {
+  value = local.kubeconfig_path
 }

submodules/bastion/README.md

@@ -12,7 +12,7 @@
 
 | Name | Version |
 |------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.22.0 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0 |
 
 ## Modules
 

submodules/eks/README.md

@@ -13,8 +13,8 @@
 
 | Name | Version |
 |------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.22.0 |
-| <a name="provider_null"></a> [null](#provider\_null) | 3.1.1 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.0 |
+| <a name="provider_null"></a> [null](#provider\_null) | >= 3.1.0 |
 
 ## Modules
 
@@ -34,14 +34,11 @@ No modules.
 | [aws_iam_policy.autoscaler](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
 | [aws_iam_policy.domino_ecr_restricted](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
 | [aws_iam_policy.ebs_csi](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
-| [aws_iam_policy.route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
-| [aws_iam_policy.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
 | [aws_iam_policy.snapshot](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
 | [aws_iam_role.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
 | [aws_iam_role.eks_nodes](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
 | [aws_iam_role_policy_attachment.aws_eks_nodes](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
 | [aws_iam_role_policy_attachment.custom_eks_nodes](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
-| [aws_iam_role_policy_attachment.custom_eks_nodes_route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
 | [aws_iam_role_policy_attachment.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
 | [aws_kms_key.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
 | [aws_launch_template.additional_node_groups](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) | resource |
@@ -63,11 +60,8 @@ No modules.
 | [aws_iam_policy_document.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.eks_nodes](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.kms_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
-| [aws_iam_policy_document.route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
-| [aws_iam_policy_document.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.snapshot](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
-| [aws_route53_zone.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) | data source |
 
 ## Inputs
 
@@ -84,8 +78,6 @@ No modules.
 | <a name="input_kubeconfig_path"></a> [kubeconfig\_path](#input\_kubeconfig\_path) | Kubeconfig file path. | `string` | `"kubeconfig"` | no |
 | <a name="input_private_subnets"></a> [private\_subnets](#input\_private\_subnets) | Private subnets object | <pre>list(object({<br>    cidr_block = string<br>    name       = string<br>    type       = string<br>    zone       = string<br>    zone_id    = string<br>    id         = string<br>  }))</pre> | n/a | yes |
 | <a name="input_region"></a> [region](#input\_region) | AWS region for the deployment | `string` | n/a | yes |
-| <a name="input_route53_hosted_zone_name"></a> [route53\_hosted\_zone\_name](#input\_route53\_hosted\_zone\_name) | Route53 zone | `string` | n/a | yes |
-| <a name="input_s3_buckets"></a> [s3\_buckets](#input\_s3\_buckets) | S3 buckets information that the nodegroups need access to | <pre>list(object({<br>    bucket_name = string<br>    arn         = string<br>  }))</pre> | n/a | yes |
 | <a name="input_ssh_pvt_key_path"></a> [ssh\_pvt\_key\_path](#input\_ssh\_pvt\_key\_path) | SSH private key filepath. | `string` | n/a | yes |
 | <a name="input_vpc_id"></a> [vpc\_id](#input\_vpc\_id) | VPC ID. | `string` | n/a | yes |
 
@@ -94,9 +86,8 @@ No modules.
 | Name | Description |
 |------|-------------|
 | <a name="output_cluster_endpoint"></a> [cluster\_endpoint](#output\_cluster\_endpoint) | EKS cluster API endpoint. |
-| <a name="output_eks_master_role_name"></a> [eks\_master\_role\_name](#output\_eks\_master\_role\_name) | EKS master role arns. |
-| <a name="output_hosted_zone_id"></a> [hosted\_zone\_id](#output\_hosted\_zone\_id) | DNS hosted zone ID. |
-| <a name="output_managed_nodes_role_arns"></a> [managed\_nodes\_role\_arns](#output\_managed\_nodes\_role\_arns) | EKS managed nodes arns. |
+| <a name="output_eks_master_roles"></a> [eks\_master\_roles](#output\_eks\_master\_roles) | EKS master roles. |
+| <a name="output_eks_node_roles"></a> [eks\_node\_roles](#output\_eks\_node\_roles) | EKS managed node roles |
 | <a name="output_nodes_security_group_id"></a> [nodes\_security\_group\_id](#output\_nodes\_security\_group\_id) | EKS managed nodes security group id. |
 | <a name="output_security_group_id"></a> [security\_group\_id](#output\_security\_group\_id) | EKS security group id. |
 <!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

submodules/eks/cluster.tf

@@ -116,10 +116,7 @@ resource "aws_eks_addon" "this" {
 
 resource "null_resource" "kubeconfig" {
   provisioner "local-exec" {
-    environment = {
-      KUBECONFIG = var.kubeconfig_path
-    }
-    command = "aws eks update-kubeconfig --region ${var.region} --name ${aws_eks_cluster.this.name}"
+    command = "aws eks update-kubeconfig --kubeconfig ${var.kubeconfig_path} --region ${var.region} --name ${aws_eks_cluster.this.name}"
   }
   triggers = {
     domino_eks_cluster_ca = aws_eks_cluster.this.certificate_authority[0].data