Determining role used for terraform deploy with Atmos

[rauthur]

After completing nearly all of the identity steps in the refarch I'm hitting an issue where atmos doesn't seem to be switching from the planner IAM role to the terraform IAM role in some workflows.

For example this fails with a few permission denials since the planner role cannot actually create resources (KMS keys, etc.):

terraform deploy cloudtrail -s core-gbl-root

I've pull the above out of the deploy workflow from baseline.

Example error:

Error: creating KMS Key: operation error KMS: CreateKey, https response error StatusCode: 400, RequestID: x, api error AccessDeniedException: User: arn:aws:sts::x:assumed-role/x-core-gbl-root-planner/aws-go-sdk-1722266462231447631 is not authorized to perform: kms:TagResource because no identity-based policy allows the kms:TagResource action

The account-map seems to have the correct roles set for plan vs. apply.

[milldr]

This is by design. No AWS Teams should have access to apply Terraform in the core-root account. Instead, use SuperAdmin when applying against core-root.

We recommend against changing that pattern, but you can see and modify the AWS Team Roles granted in each account with that aws-team-roles deployment in that given stack. The default is defined in stacks/catalog/aws-team-roles.yaml and the override for core-root would be in stacks/orgs/<NAMESPACE>/core/root/global-region/identity.yaml

[milldr]

No AWS Teams should have access to apply Terraform in the core-root account.

I see now that the managers Team does have terraform access in core-root. Do you know which AWS Team you have assumed before running Terraform?

Within your infra geodesic shell, run this to check:

 √ . [foo-identity] (HOST) infrastructure ⨠ aws sts get-caller-identity
{
    "UserId": "ABCD1234:foo-identity",
    "Account": "1234567890",
    "Arn": "arn:aws:sts::1234567890:assumed-role/foo-core-gbl-identity-devops/foo-identity"
}

For example here I am using the devops team, so I would only have planner access in core-root

[rauthur]

Got it! I was running under DevOps. Using SuperAdmin makes sense in this context, I was just confused by the final step in the identity docs:

Now that roles are deployed, we can validate their setup by deploying CloudTrail and ECR. Run the following workflow: atmos workflow deploy -f baseline.

At first glance this reads to me as use the assume-role workflows we just finished setting up to deploy the final pieces of baseline.

[milldr]

hmm that phrasing is certainly misleading. I'll make a note of it. Thanks for pointing that out!

[rauthur]

One minor gotcha to add about the identity steps are that they seem to require the GitHub OIDC providers to have been created from a later phase. Easy to address but took a second to figure out.

Edit with detail: Without the providers existing in AWS an empty string is returned for the federated principle for some roles leading to a provisioning error.

[milldr]

ah yes that is tricky as well! It's a matter of order of operations and definitely could be better documented.

Much later in the journey, we want to allow GitHub permission to assume an AWS Team in order to plan and apply Terraform with our Atmos GitHub Actions (what we refer to as "gitops"). To do so, we add the trusted_github_repos input for aws-teams and specify the infrastructure repository. That enables Terraform to create an OIDC role and as you pointed out that of course requires the GitHub OIDC provider to be deployed. Yet at this point in the journey you may not have deployed that provider yet.

So yes you have 2 choices. You can deploy the GitHub OIDC providers now, or you can comment out trusted_github_repos in the aws-teams stack catalog and deploy it later with that layer.