Simplify setup and teardown of staging infrastructure

[dennyabrain]

Scope of Work

script to start and stop a staging environment
take a backup and restore aws rds postgres
put secrets somewhere and checkin all other files
kubectl comands to bring up and down deployment

[dennyabrain]

Lets start with
1: Write a script or command that can create an S3 bucket and Delete it
2: Create an IAM account programatically with suitable policy to read/write to this bucket

[dennyabrain]

Blocked at Move personal account to organization.
AWS recommendation is to not use root account for everything else.

Next Steps :

• Switch to Organization IAM center
• Create new admin who can do the restricted operations

[duggalsu]

Managed to delete personal IAM Identity Center and recreate with AWS Organizations. Successfully added admin user with AdministratorAccess permissions.
Refer: https://docs.aws.amazon.com/singlesignon/latest/userguide/quick-start-default-idc.html

[duggalsu]

Setup update

1. Enable IAM Identity Center with AWS Organizations for SSO. Any roles/users created are assigned temporary keys for security. (No need to worry about manually rotating keys on compromise).
2. Create an admin user for all tasks. Do not use the root user
3. Setup AWS CLI to allow access to user with SSO - since the access keys are temporary for security
   See instructions here - https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html
   Basically need to run $ aws configure sso
4. Use AWS CLI to create S3 buckets similar to creating a bucket via console with all non-optional settings
   Refer CLI commands - https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/index.html#cli-aws-s3api

# create bucket
$ aws s3api create-bucket \
    --region us-east-1 \
    --bucket my-bucket \
    --object-ownership BucketOwnerEnforced \
    --no-object-lock-enabled-for-bucket

# list all buckets
$ aws s3 ls

# get public access block - this defaults to enabled for all 4 settings - so no need to manually enable
$ aws s3api get-public-access-block \
    --bucket my-bucket

• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/put-bucket-versioning.html

If the versioning state has never been set on a bucket, it has no versioning state; a GetBucketVersioning request does not return a versioning state value.

# get bucket versioning
$ aws s3api get-bucket-versioning \
    --bucket my-bucket

# put bucket versioning
$ aws s3api put-bucket-versioning \
    --bucket my-bucket \
    --versioning-configuration Status=Suspended

5. Rotate all permanent keys for security - without taking down services
   Refer

• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_revoke-sessions.html
• https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds.html
• https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#rotate-credentials
• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html

[duggalsu]

Login to expired session (due to timeout)

1. Sign into account
2. Grant access to CLI by following instructions when executing

$ aws sso login --no-browser

[duggalsu]

Task: Create an IAM account programatically with suitable policy to read/write to this bucket

https://aws.amazon.com/blogs/security/use-iam-identity-center-apis-to-audit-and-manage-application-assignments/

Before creating user - enable automatic sending of verification email via IAM Identity Center console

• https://docs.aws.amazon.com/singlesignon/latest/userguide/userswithoutpwd.html

# Step 1: Get your Identity Center instance information
$ aws sso-admin list-instances

# Step 2: Create user and group in your Identity Store
$ aws identitystore create-user --identity-store-id "d-**********" --user-name "MyUser" --emails Value="[email protected]",Type="Work",Primary=true —display-name "My User" —name FamilyName="User",GivenName="My"

# Create a group in your Identity Store:
$ aws identitystore create-group --identity-store-id d-********** --display-name test-s3-app

# Run the following command to add the user to the group:
$ aws identitystore create-group-membership --identity-store-id d-********** --group-id ********-****-****-****-************ --member-id UserId=********-****-****-****-************

Create S3 policy
https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-policies-s3.html#iam-policy-ex2

Attaching the following policy to the group grants everybody in the group access to the following folder in Amazon S3: DOC-EXAMPLE-BUCKET1/share/marketing. Group members are allowed to access only the specific Amazon S3 permissions shown in the policy and only for objects in the specified folder. 

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:PutObject",
            "s3:GetObject",
            "s3:GetObjectVersion",
            "s3:DeleteObject",
            "s3:DeleteObjectVersion"
         ],
         "Resource":"arn:aws:s3:::DOC-EXAMPLE-BUCKET1/share/marketing/*"
      }
   ]
}

TODO:
https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html

IAM Identity Center assigns access to a user or group in one or more AWS accounts with permission sets. 


You can add AWS managed policies, customer managed policies, inline policies, and AWS managed policies for job functions to your permission sets. You can also assign an AWS managed policy or a customer managed policy as a permissions boundary.

• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-permission-set.html
• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-account-assignment.html
• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/put-inline-policy-to-permission-set.html

Edit: Updated with instruction link for enabling automatic sending of verification email when creating user

[duggalsu]

Create permission set and assign to group

1. Create permission set
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-permission-set.html

$ aws sso-admin create-permission-set \
    --instance-arn <value> \
    --name <value>

2. Validate policy
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/accessanalyzer/validate-policy.html

$ aws accessanalyzer validate-policy \
    --policy-document <value> \
    --policy-type <value>

3. Put inline policy to permission set
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/put-inline-policy-to-permission-set.html

$ aws sso-admin put-inline-policy-to-permission-set \
    --inline-policy <value> \
    --instance-arn <value> \
    --permission-set-arn <value>

4. List permission set provisioning status for verification
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/list-permission-set-provisioning-status.html

$ aws sso-admin list-permission-set-provisioning-status \
    --instance-arn <value>

5. Get group id for next command
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/identitystore/list-groups.html

$ aws identitystore list-groups \
    --identity-store-id <value>

6. Assign permission set to group
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/create-account-assignment.html

$ aws sso-admin create-account-assignment \
    --instance-arn <value> \
    --permission-set-arn <value> \
    --principal-id <value> \
    --principal-type <value> \
    --target-id <value> \
    --target-type <value>

7. List account assignment creation status for verification
   https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/list-account-assignment-creation-status.html

$ aws sso-admin list-account-assignment-creation-status \
    --instance-arn <value>

8. Modify existing policy (optional)
9. Re-invoke put inline policy as above
10. Provision permission set to update policy for provisioned accounts
    https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/provision-permission-set.html

$ aws sso-admin provision-permission-set \
    --instance-arn <value> \
    --permission-set-arn <value> \
    --target-id <value> \
    --target-type <value>

3. Verify provisioning status
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sso-admin/list-permission-set-provisioning-status.html

$ aws sso-admin list-permission-set-provisioning-status \
    --instance-arn <value>

[duggalsu]

Now you can verify the user via email and set a password and add the user for AWS CLI access

[duggalsu]

In Progress - IAM Identity Center (SSO) and Github Actions to upload to S3
- https://dev.to/aws-builders/aws-sso-github-setup-35ld
- https://medium.com/@alissonpdc/github-actions-how-to-authenticate-on-aws-cloud-14185f811bf0
- https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services
- https://github.com/aws-actions/configure-aws-credentials

[duggalsu]

Update:

• https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers.html

As a security best practice, we recommend you manage human users in IAM Identity Center with an external SAML identity provider instead of using SAML federation in IAM. For information about specific situations where an IAM user is required, see When to create an IAM user (instead of a role).

• IAM Identity Center - should be used to manage all humans users either using IAM Identity Center (within AWS org) OR external SAML (Security Assertion Markup Language) identity provider
• IAM (roles - NOT users) - should be used for machine-to-machine IdP using OIDC eg github actions

Steps to follow to configure Github Actions access to AWS

• https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services

Official AWS credentials github action - https://github.com/aws-actions/configure-aws-credentials
Create IAM OIDC identity - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html

• https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/list-open-id-connect-providers.html

$ aws iam list-open-id-connect-providers

$ aws iam create-open-id-connect-provider \
    --url <value> \
    --client-id-list <value> \
    --tags Key=string,Value=string

Create roles for OIDC - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html

• https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#policies_resource-based

Resource-based policies are JSON policy documents that you attach to a resource such as an Amazon S3 bucket. These policies grant the specified principal permission to perform specific actions on that resource and defines under what conditions this applies. Resource-based policies are inline policies. There are no managed resource-based policies.

Example steps - https://www.eliasbrange.dev/posts/secure-aws-deploys-from-github-actions-with-oidc/

TODO

1. Use Github environments - https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment

Environments, environment secrets, and deployment protection rules are available in public repositories for all current GitHub plans. They are not available on legacy plans, such as Bronze, Silver, or Gold. For access to environments, environment secrets, and deployment branches in private or internal repositories, you must use GitHub Pro, GitHub Team, or GitHub Enterprise. If you are on a GitHub Free, GitHub Pro, or GitHub Team plan, other deployment protection rules, such as a wait timer or required reviewers, are only available for public repositories.

2. Configure environment in role trust policy - https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services#configuring-the-role-and-trust-policy

If you use a workflow with an environment, the sub field must reference the environment name: repo:OWNER/REPOSITORY:environment:NAME. For more information, see "About security hardening with OpenID Connect."

Note: When environments are used in workflows or in OIDC policies, we recommend adding protection rules to the environment for additional security. For example, you can configure deployment rules on an environment to restrict which branches and tags can deploy to the environment or access environment secrets. For more information, see "Using environments for deployment."

3. Create IAM roles - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp.html
   Create role - https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/create-role.html

$ aws iam create-role \
    --role-name <value> \
    --assume-role-policy-document <value>

Sample role trust policy document - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html#idp_oidc_Create_GitHub

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::012345678910:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:GitHubOrg/GitHubRepo:ref:refs/heads/GitHubBranch"
        }
      }
    }
  ]
}

Create inline permission policy - https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/put-role-policy.html

$ aws iam put-role-policy \
    --role-name <value> \
    --policy-name <value> \
    --policy-document <value>

Sample role permissions policy - https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-policies-s3.html#iam-policy-ex2

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:PutObject",
            "s3:GetObject",
            "s3:GetObjectVersion",
            "s3:DeleteObject",
            "s3:DeleteObjectVersion"
         ],
         "Resource":"arn:aws:s3:::DOC-EXAMPLE-BUCKET1/share/marketing/*"
      }
   ]
}

[duggalsu]

Successfully setup github workflow to push objects to AWS S3. Refer sample workflow here

• https://github.com/duggalsu/aws-test/blob/main/.github/workflows/aws-s3-deploy-prod.yml

[duggalsu]

Validating the S3 bucket role policy

$ aws accessanalyzer validate-policy \
    --policy-document <value> \
    --policy-type RESOURCE_POLICY

[dennyabrain]

Denny's notes from Aurora's call

Create accounts/identities which have temporary credentials that get revoked every hour (default)

AWS has 3 products for identity management

1. IAM
   any user gets long lived credentials. never add or share publically
   never create users here (deprecated)
2. IAM Identity Center (called as Identity Provider, IdP)
   previously called Single Sign On. any user you create have short lived credentials. and they get rotated. standard practice is to create users from IAM identity center. if you want a subset of the users to have certain permissions, you make policies, add them to a group. add user to group.
   This is how you create human users.
3. Machine to machine interaction doesn't require a user. create roles, which have an identity and not associated with any user. role is similar to group.
   From a third party service. the third party service will act as an IdP. they will create an identity and then generally through a protocol like OIDC, you will be able to add this external identity as part of aws iam role.
   IdP doesnt allow creating roles.
4. AWS Cognito. IAM full fledged access management.

Convention
create groups with a name according to role of the group.

Root user HAS to create an admin account or restricted user via IAM iDentity center.
since these credentials are short lived

1. login through web browser
2. login through aws cli
   1. open in browser, authorize access.

users created via IdP,
when the admin user logs in they see their access key.

aws CLI setup
https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html
once setup, key rotation happens automatically.
! AWS CLI version 2

two types of policies
trust policy
defines who the user who gets access to this group or role
attached when creating a role
permissions policy
what services and applications a user gets access to

Github Environments
generally used for deployment
sets check before deployment can progress
within this environment you can add secrets
only if the checks pass, secret access is blocked.

when using environment, url in the github policy will be different.

there is a cli to validate policies.

Generate the github OIDC token and added it to IAM role

Process

1. in aws create an OIDC provider linked to github
2. you create a role, and attach a trust policy, in the policy is details of the github OIDC provider and the repo details.

in create role step
assume-role-policy-document -> is the trust policy
put-role-policy-> permissions policy

inline policy vs managed policy

AWS_ROLE_TO_ASSUME is the role ARN

[duggalsu]

Note: Permissions

Approach 1: Role-Based Access Control (RBAC)

• https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbac

The traditional authorization model used in IAM is called role-based access control (RBAC). RBAC defines permissions based on a person's job function, known outside of AWS as a role. Within AWS a role usually refers to an IAM role, which is an identity in IAM that you can assume. IAM does include managed policies for job functions that align permissions to a job function in an RBAC model.
					
In IAM, you implement RBAC by creating different policies for different job functions. You then attach the policies to identities (IAM users, groups of users, or IAM roles). As a best practice, you grant the minimum permissions necessary for the job function. This is known as granting least privilege. Do this by listing the specific resources that the job function can access. The disadvantage to using the traditional RBAC model is that when employees add new resources, you must update policies to allow access to those resources.

Approach 2: Attribute-Based Access Control (ABAC)

• https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_attribute-based-access-control.html

Attribute-based access control (ABAC) is an authorization strategy that defines permissions based on attributes. In AWS, these attributes are called tags.

• https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html#introduction_attribute-based-access-control_compare-rbac

ABAC provides the following advantages over the traditional RBAC model:

ABAC permissions scale with innovation. It's no longer necessary for an administrator to update existing policies to allow access to new resources. For example, assume that you designed your ABAC strategy with the access-project tag. A developer uses the role with the access-project = Heart tag. When people on the Heart project need additional Amazon EC2 resources, the developer can create new Amazon EC2 instances with the access-project = Heart tag. Then anyone on the Heart project can start and stop those instances because their tag values match.
					
ABAC requires fewer policies. Because you don't have to create different policies for different job functions, you create fewer policies. Those policies are easier to manage.
					
Using ABAC, teams can change and grow quickly. This is because permissions for new resources are automatically granted based on attributes. For example, if your company already supports the Heart and Star projects using ABAC, it's easy to add a new Lightning project. An IAM administrator creates a new role with the access-project = Lightning tag. It's not necessary to change the policy to support a new project. Anyone that has permissions to assume the role can create and view instances tagged with access-project = Lightning. Additionally, a team member might move from the Heart project to the Lightning project. The IAM administrator assigns the user to a different IAM role. It's not necessary to change the permissions policies.
					
Granular permissions are possible using ABAC. When you create policies, it's a best practice to grant least privilege. Using traditional RBAC, you must write a policy that allows access to only specific resources. However, when you use ABAC, you can allow actions on all resources, but only if the resource tag matches the principal's tag.
					
Use employee attributes from your corporate directory with ABAC. You can configure your SAML or OIDC provider to pass session tags to AWS. When your employees federate into AWS, their attributes are applied to their resulting principal in AWS. You can then use ABAC to allow or deny permissions based on those attributes.

ABAC in IAM Identity Center

• https://docs.aws.amazon.com/singlesignon/latest/userguide/abac.html

The following are additional benefits of using ABAC in IAM Identity Center.

ABAC requires fewer permission sets – Because you don't have to create different policies for different job functions, you create fewer permission sets. This reduces your permissions management complexity.

Using ABAC, teams can change and grow quickly – Permissions for new resources are automatically granted based on attributes when resources are appropriately tagged upon creation. 

Use employee attributes from your corporate directory with ABAC – You can use existing employee attributes from any identity source configured in IAM Identity Center to make access control decisions in AWS.

Track who is accessing resources – Security administrators can easily determine the identity of a session by reviewing the user attributes in AWS CloudTrail to track user activity in AWS.

TODO

• https://docs.aws.amazon.com/singlesignon/latest/userguide/abac-checklist.html
• https://docs.aws.amazon.com/singlesignon/latest/userguide/attributesforaccesscontrol.html
• https://docs.aws.amazon.com/singlesignon/latest/APIReference/API_CreateInstanceAccessControlAttributeConfiguration.html

[dennyabrain]

@duggalsu ok so this makes sense. Lets finalize a convention around Attributes to create some policies.
I can think of two attributes that are important for access control - Project Name and Environment.
For eg, Aatman can create/edit/delete resources under Uli and Staging environment. or Denny can create/edit/delete resources under Viral Spiral and Staging and Production environment etc.
Let me know if you can think of many other attributes.

This also means that in all of out automations and scripts we'll have to make sure we add tags to resources.