-
Notifications
You must be signed in to change notification settings - Fork 8
Terraform
Terraform is a framework developed by Hashicorp that helps to programmatically define your cloud infrastructure. Terraform is flexible for Google Cloud Platform, AWS, Azure, and other major cloud providers.
To make DLP scalable, we take advantage of the cloud since we pay for what we use, no need to guess capacity, and we don't have to reinvent the wheel. We use AWS due to the various options provided and that it has good community support.
We have two ways of configuring our AWS resources:
- Go in the AWS Management Console and configure the resources through the website
- Programmatically define our resources
At first glance, it seems that option 1 is the most straightforward, but it comes with a risk. Say that something went wrong with your AWS service and you had to fix it. It would be a pain to go ahead and make those changes within the console (unless you know what you're doing :)). Also, linking AWS services together isn't necessarily the most straightforward within the AWS Management Console. With Option 2, while we have to "write more code", at least we have defined our AWS resource configurations and if something were to go wrong with our cloud services or we wanted to do a migration in the future, we can simply update the configurations in the code and run a simple deploy command (and leave the annoying stuff in AWS Management Console to AWS). This is the premise behind a paradigm called Infrastructure as Code. Terraform simply acts as a vehicle to help realize Option 2.
See /dlp-terraform
directory.
Helpful Resource: This fantastic documentation!
- Follow the installation instructions from Hashicorp here
- Configure the AWS credentials for Terraform to use. Follow instructions in the README to install AWS CLI
cd ~/Deep-Learning-Playground
cd dlp-terraform
-
terraform init
- this starts terraform and loads up the cache - You're ready to go!
@thomaschin35, @farisdurrani, @karkir0003 and of course [email protected]
- Home
- Terraform
- Bearer-Token-Gen-Script
- Frontend-Backend Communication Documentation
- Backend Documentation (backend)
-
driver.py
- AWS Helper Files (backend.aws_helpers)
- Dynamo DB Utility Files (aws_helpers.dynamo_db_utils)
- AWS Secrets Utility Files (aws_secrets_utils)
- AWS Batch Utility Files (aws_batch_utils)
- Firebase Helper Files (backend.firebase_helpers)
- Common Files (backend.common)
-
constants.py
-
dataset.py
-
default_datasets.py
-
email_notifier.py
-
loss_functions.py
-
optimizer.py
-
utils.py
- Deep Learning Files (backend.dl)
- Machine Learning Files (backend.ml)
- Frontend Documentation
- Bug Manual
- Developer Runbook
- Examples to locally test DLP
- Knowledge Share