feat: Add AWS CDK project and Helm charts for Beckn-Onix deployment o…

[mozammil89]

…n AWS cloud

• Added CDK stacks for deploying Beckn-Onix components (Registry, Gateway, BAP, BPP)
• Implemented infrastructure provisioning for VPC, EKS, RDS, Redis, DocumentDB, and RabbitMQ
• Included Helm charts deployment for Registry, Gateway, BAP, and BPP on Amazon EKS cluster (or any K8s environment)
• Configured .env file for user-specific variables
• Updated README with deployment instructions and CDK stack details
• User guide for running Helm charts

Any many more...

[vbabuEM]

Moz had covered most of the architectural and high level deployment model in his tech talks on this. It all seemed fine. All the code is in a separate folder. I have not understood the code, but I am guessing the risk in merging is small. So I am approving it. I have requested @prasad-takale-eminds for a review if his time permits.
cc @faizmagic @ravi-prakash-v

[prasad-takale-eminds]

@mozammil89 Please find the below review comment

1. In aws-cdk/beckn-cdk/cdk.context.json, the Availability Zone and Account ID are hard-coded. Please make these configurable.

2. Kindly include a sample .env file.

3. In aws-cdk/beckn-cdk/lib/config.ts, the .env file is missing from the aws-cdk/beckn-cdk directory. The line dotenv.config({ path: path.resolve(__dirname, "../.env") }); expects it to be present.

4. The RDS password is being saved in AWS Secrets Manager. We need to disable secret rotation for the RDS password since we are updating the config files during the initial installation.

5. Ensure the Layer2 configuration setup is installed after the BAP and BPP deployment.

6. Regarding the pull policy set as IfNotPresent, how should we update to a new version of the application?

CC: @vbabuEM @faizmagic @ravi-prakash-v

[mozammil89]

@mozammil89 Please find the below review comment

1. In aws-cdk/beckn-cdk/cdk.context.json, the Availability Zone and Account ID are hard-coded. Please make these configurable.
2. Kindly include a sample .env file.
3. In aws-cdk/beckn-cdk/lib/config.ts, the .env file is missing from the aws-cdk/beckn-cdk directory. The line dotenv.config({ path: path.resolve(__dirname, "../.env") }); expects it to be present.
4. The RDS password is being saved in AWS Secrets Manager. We need to disable secret rotation for the RDS password since we are updating the config files during the initial installation.
5. Ensure the Layer2 configuration setup is installed after the BAP and BPP deployment.
6. Regarding the pull policy set as IfNotPresent, how should we update to a new version of the application?

CC: @vbabuEM @faizmagic @ravi-prakash-v

Thank you for the review, @vbabuEM and @prasad-takale-eminds.

Below are my responses to the comments:

1. Availability Zone and Account ID in cdk.context.json:
   The cdk.context.json file is auto-generated by CDK and is used to cache environment-specific information such as the availability zones. I’ve deleted this file and added .gitignore to exclude cdk.context.json from version control and ensure that any environment-specific information is parameterized in the CDK code for flexibility.

2. Sample .env file:
   Good catch, thank you. Somehow I forgot to include the sample .env file. It’s added now.

3. Missing .env file in lib/config.ts:
   This is addressed in point 2.

4. RDS password rotation:
   Password rotation is disabled in the secret manager by default for RDS secrets.

5. Layer2 configuration:
   Layer2 config files that come bundled in container images are copied over to EFS share during bootstrap. Check the initContainer block in the helm chart for BAP and BPP - https://github.com/mozammil89/beckn-onix/blob/feat/add-aws-cdk-beckn-onix/aws-cdk/helm/beckn-onix-bap/charts/bap-network/templates/deployment.yaml#L34

6. Pull policy IfNotPresent:
   We could set the pull policy to always but that would result in too many network calls if someone is experimenting with Beckn-Onix in development environment. An alternate option would be to use the tag. Happy to make changes to Always if that is preferred.

CC: @vbabuEM @faizmagic @ravi-prakash-v

[prasad-takale-eminds]

@mozammil89 Please find the below review comment

1. In aws-cdk/beckn-cdk/cdk.context.json, the Availability Zone and Account ID are hard-coded. Please make these configurable.
2. Kindly include a sample .env file.
3. In aws-cdk/beckn-cdk/lib/config.ts, the .env file is missing from the aws-cdk/beckn-cdk directory. The line dotenv.config({ path: path.resolve(__dirname, "../.env") }); expects it to be present.
4. The RDS password is being saved in AWS Secrets Manager. We need to disable secret rotation for the RDS password since we are updating the config files during the initial installation.
5. Ensure the Layer2 configuration setup is installed after the BAP and BPP deployment.
6. Regarding the pull policy set as IfNotPresent, how should we update to a new version of the application?

CC: @vbabuEM @faizmagic @ravi-prakash-v

Thank you for the review, @vbabuEM and @prasad-takale-eminds.

Below are my responses to the comments:

1. Availability Zone and Account ID in cdk.context.json:
   The cdk.context.json file is auto-generated by CDK and is used to cache environment-specific information such as the availability zones. I’ve deleted this file and added .gitignore to exclude cdk.context.json from version control and ensure that any environment-specific information is parameterized in the CDK code for flexibility.
2. Sample .env file:
   Good catch, thank you. Somehow I forgot to include the sample .env file. It’s added now.
3. Missing .env file in lib/config.ts:
   This is addressed in point 2.
4. RDS password rotation:
   Password rotation is disabled in the secret manager by default for RDS secrets.
5. Layer2 configuration:
   Layer2 config files that come bundled in container images are copied over to EFS share during bootstrap. Check the initContainer block in the helm chart for BAP and BPP - https://github.com/mozammil89/beckn-onix/blob/feat/add-aws-cdk-beckn-onix/aws-cdk/helm/beckn-onix-bap/charts/bap-network/templates/deployment.yaml#L34
6. Pull policy IfNotPresent:
   We could set the pull policy to always but that would result in too many network calls if someone is experimenting with Beckn-Onix in development environment. An alternate option would be to use the tag. Happy to make changes to Always if that is preferred.

CC: @vbabuEM @faizmagic @ravi-prakash-v

@mozammil89 For Layer2 if the user wants to install domain-specific layer2 config then how we can install that?
As of now, we are using following this after the installation
@vbabuEM Can you please comment on this?

[mozammil89]

@mozammil89 Please find the below review comment

1. In aws-cdk/beckn-cdk/cdk.context.json, the Availability Zone and Account ID are hard-coded. Please make these configurable.
2. Kindly include a sample .env file.
3. In aws-cdk/beckn-cdk/lib/config.ts, the .env file is missing from the aws-cdk/beckn-cdk directory. The line dotenv.config({ path: path.resolve(__dirname, "../.env") }); expects it to be present.
4. The RDS password is being saved in AWS Secrets Manager. We need to disable secret rotation for the RDS password since we are updating the config files during the initial installation.
5. Ensure the Layer2 configuration setup is installed after the BAP and BPP deployment.
6. Regarding the pull policy set as IfNotPresent, how should we update to a new version of the application?

CC: @vbabuEM @faizmagic @ravi-prakash-v

Thank you for the review, @vbabuEM and @prasad-takale-eminds.
Below are my responses to the comments:

1. Availability Zone and Account ID in cdk.context.json:
   The cdk.context.json file is auto-generated by CDK and is used to cache environment-specific information such as the availability zones. I’ve deleted this file and added .gitignore to exclude cdk.context.json from version control and ensure that any environment-specific information is parameterized in the CDK code for flexibility.
2. Sample .env file:
   Good catch, thank you. Somehow I forgot to include the sample .env file. It’s added now.
3. Missing .env file in lib/config.ts:
   This is addressed in point 2.
4. RDS password rotation:
   Password rotation is disabled in the secret manager by default for RDS secrets.
5. Layer2 configuration:
   Layer2 config files that come bundled in container images are copied over to EFS share during bootstrap. Check the initContainer block in the helm chart for BAP and BPP - https://github.com/mozammil89/beckn-onix/blob/feat/add-aws-cdk-beckn-onix/aws-cdk/helm/beckn-onix-bap/charts/bap-network/templates/deployment.yaml#L34
6. Pull policy IfNotPresent:
   We could set the pull policy to always but that would result in too many network calls if someone is experimenting with Beckn-Onix in development environment. An alternate option would be to use the tag. Happy to make changes to Always if that is preferred.

CC: @vbabuEM @faizmagic @ravi-prakash-v

@mozammil89 For Layer2 if the user wants to install domain-specific layer2 config then how we can install that? As of now, we are using following this after the installation @vbabuEM Can you please comment on this?

@prasad-takale-eminds
Amazon EFS was especially used in this scenario where you may have multiple PODs that required layer 2 config. The current script appears to be copying file into a standalone docker container created through docker compose.

The approach you need to take in Kubernetes environment is fairly straightforward. You simply mount the EFS shared volume (used for BAP or BPP) onto an EC2 machine (any micro VM could help) and copy the Layer 2 configuration file. The new files will immediately become available to the corresponding pods.

Mounting an EFS file system is an easy process:

1. Install the amazon-efs-utils package on your EC2 instance.

2. Mount the file system using its ID. You can find the fileSystemId in the EFS service, which was created by the Helm chart during the deployment of BAP or BPP services:

$ mkdir efs
$ sudo mount -t efs file-system-id efs/

3. Copy the Layer 2 configuration files to the efs directory.

I'll update the readme with above steps. Let me know if you're OK with it.

Cc: @vbabuEM

[prasad-takale-eminds]

@mozammil89 Please find the below review comment

1. In aws-cdk/beckn-cdk/cdk.context.json, the Availability Zone and Account ID are hard-coded. Please make these configurable.
2. Kindly include a sample .env file.
3. In aws-cdk/beckn-cdk/lib/config.ts, the .env file is missing from the aws-cdk/beckn-cdk directory. The line dotenv.config({ path: path.resolve(__dirname, "../.env") }); expects it to be present.
4. The RDS password is being saved in AWS Secrets Manager. We need to disable secret rotation for the RDS password since we are updating the config files during the initial installation.
5. Ensure the Layer2 configuration setup is installed after the BAP and BPP deployment.
6. Regarding the pull policy set as IfNotPresent, how should we update to a new version of the application?

CC: @vbabuEM @faizmagic @ravi-prakash-v

Thank you for the review, @vbabuEM and @prasad-takale-eminds.
Below are my responses to the comments:

1. Availability Zone and Account ID in cdk.context.json:
   The cdk.context.json file is auto-generated by CDK and is used to cache environment-specific information such as the availability zones. I’ve deleted this file and added .gitignore to exclude cdk.context.json from version control and ensure that any environment-specific information is parameterized in the CDK code for flexibility.
2. Sample .env file:
   Good catch, thank you. Somehow I forgot to include the sample .env file. It’s added now.
3. Missing .env file in lib/config.ts:
   This is addressed in point 2.
4. RDS password rotation:
   Password rotation is disabled in the secret manager by default for RDS secrets.
5. Layer2 configuration:
   Layer2 config files that come bundled in container images are copied over to EFS share during bootstrap. Check the initContainer block in the helm chart for BAP and BPP - https://github.com/mozammil89/beckn-onix/blob/feat/add-aws-cdk-beckn-onix/aws-cdk/helm/beckn-onix-bap/charts/bap-network/templates/deployment.yaml#L34
6. Pull policy IfNotPresent:
   We could set the pull policy to always but that would result in too many network calls if someone is experimenting with Beckn-Onix in development environment. An alternate option would be to use the tag. Happy to make changes to Always if that is preferred.

CC: @vbabuEM @faizmagic @ravi-prakash-v

@mozammil89 For Layer2 if the user wants to install domain-specific layer2 config then how we can install that? As of now, we are using following this after the installation @vbabuEM Can you please comment on this?

@prasad-takale-eminds Amazon EFS was especially used in this scenario where you may have multiple PODs that required layer 2 config. The current script appears to be copying file into a standalone docker container created through docker compose.

The approach you need to take in Kubernetes environment is fairly straightforward. You simply mount the EFS shared volume (used for BAP or BPP) onto an EC2 machine (any micro VM could help) and copy the Layer 2 configuration file. The new files will immediately become available to the corresponding pods.

Mounting an EFS file system is an easy process:

1. Install the amazon-efs-utils package on your EC2 instance.
2. Mount the file system using its ID. You can find the fileSystemId in the EFS service, which was created by the Helm chart during the deployment of BAP or BPP services:

$ mkdir efs
$ sudo mount -t efs file-system-id efs/

3. Copy the Layer 2 configuration files to the efs directory.

I'll update the readme with above steps. Let me know if you're OK with it.

Cc: @vbabuEM

Sure, this will help.
Please update the README with the steps.
CC: @faizmagic @ravi-prakash-v @vbabuEM