Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial Amazon Managed Workflows for Apache Airflow (MWAA) #75

Merged
merged 1 commit into from
May 28, 2024

Conversation

michalc
Copy link
Member

@michalc michalc commented May 21, 2024

Initial basic setup of MWAA.

We are currently just aiming to get a single pipeline ingesting data into Data Workspace, and in order to do that we need things fired up in the correct VPCs/subnets etc, with access to at least one bucket, and will also down the line need some credentials. So we're Terraform-ing early on in the process to not build up a debt to have to do it later, and to find out if we can do everything we do need to do via Terraform

@michalc michalc requested a review from a team as a code owner May 21, 2024 13:26
@nboyse nboyse force-pushed the feat/managed-workflows-airflow branch 4 times, most recently from 8180d96 to f73f8d6 Compare May 24, 2024 14:44
@michalc michalc force-pushed the feat/managed-workflows-airflow branch from f73f8d6 to 372b9b1 Compare May 24, 2024 14:47
@nboyse nboyse force-pushed the feat/managed-workflows-airflow branch from 372b9b1 to f73f8d6 Compare May 24, 2024 14:49
@michalc michalc force-pushed the feat/managed-workflows-airflow branch 4 times, most recently from 4c89fe4 to b210c17 Compare May 24, 2024 14:52
@michalc michalc changed the title feat: start of Airflow feat: initial Amazon Managed Workflows for Apache Airflow (MWAA) May 25, 2024
Initial basic setup of MWAA.

We are currently just aiming to get a single pipeline ingesting data into Data
Workspace, and in order to do that we need things fired up in the correct
VPCs/subnets etc, with access to at least one bucket, and will also down the
line need some credentials. So we're Terraform-ing early on in the process to
not build up a debt to have to do it later, and to find out if we can do
everything we do need to do via Terraform

Co-authored-by: Tash Boyse <[email protected]>
Co-authored-by: Michal Charemza <[email protected]>
Co-authored-by: Mohizur Khan <[email protected]>
Co-authored-by: Josh Wong <[email protected]>
@michalc michalc force-pushed the feat/managed-workflows-airflow branch from b210c17 to 73ad203 Compare May 28, 2024 10:04
@michalc michalc merged commit f18f5cb into main May 28, 2024
1 check passed
@michalc michalc deleted the feat/managed-workflows-airflow branch May 28, 2024 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant