Big data lake analysis using AWS infrastructure.
Shows how to use AWS with Terraform to accomplish the following tasks:
- How to build an AWS data lake with Terraform
- You must have an AWS account, and have your default credentials and AWS Region configured
- You must have Terraform installed
- As an AWS best practice, grant this code least privilege, or only the permissions required to perform a task. For more information, see Grant Least Privilege in the AWS Identity and Access Management User Guide.
- This code has been tested in us-west-1 AWS Regions only. However it should work in any other region.
- Running this code it will result in charges to your AWS account.
The private key generated by this resource will be stored unencrypted in your Terraform state file. Use of this resource for production deployments is not recommended. Instead, generate a private key file outside of Terraform and distribute it securely to the system where Terraform will be run.
It will deploy to the account setup for the aws cli called 'default'
cd AWS_DataLake
terraform init
terraform validate
terraform apply --auto-approve
terraform destroy --auto-approve
- clone this Git repo
- cd to the the directory AWS_DataLake
- use terraform init command to initialize the Terraform working directory and to download plugins for a provider (aws)
- terraform validate command will validate the code before actual deployment
- terraform plan command will show the execution code before making the actual deployment
- terraform apply
- Alternate command : terraform apply -auto-approve
- terraform destroy
- Alternate command : terraform destroy -auto-approve
- terraform fmt -recursive # A way to format the terraform code