Skip to content

Commit

Permalink
Merge pull request #2215 from ASFHyP3/whitelisting-sandbox
Browse files Browse the repository at this point in the history
Whitelisting
  • Loading branch information
jtherrmann authored May 8, 2024
2 parents 3c486a4 + a9edd11 commit 739f8b3
Show file tree
Hide file tree
Showing 26 changed files with 1,093 additions and 259 deletions.
6 changes: 3 additions & 3 deletions .github/actions/deploy-hyp3/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ inputs:
DEFAULT_CREDITS_PER_USER:
description: "The default number of credits given to a new user"
required: true
RESET_CREDITS_MONTHLY:
description: "Whether to reset each user's remaining credits each month"
DEFAULT_APPLICATION_STATUS:
description: "The default status for new user applications."
required: true
COST_PROFILE:
description: "Job spec cost profile"
Expand Down Expand Up @@ -126,7 +126,7 @@ runs:
$ORIGIN_ACCESS_IDENTITY_ID \
$DISTRIBUTION_URL \
DefaultCreditsPerUser='${{ inputs.DEFAULT_CREDITS_PER_USER }}' \
ResetCreditsMonthly='${{ inputs.RESET_CREDITS_MONTHLY }}' \
DefaultApplicationStatus='${{ inputs.DEFAULT_APPLICATION_STATUS }}' \
DefaultMaxvCpus='${{ inputs.DEFAULT_MAX_VCPUS }}' \
ExpandedMaxvCpus='${{ inputs.EXPANDED_MAX_VCPUS }}' \
MonthlyBudget='${{ inputs.MONTHLY_BUDGET }}' \
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/deploy-daac.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 14
default_credits_per_user: 10000
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: EDC
deploy_ref: refs/heads/main
job_files: job_spec/AUTORIFT.yml job_spec/INSAR_GAMMA.yml job_spec/RTC_GAMMA.yml job_spec/INSAR_ISCE_BURST.yml
Expand All @@ -41,7 +41,7 @@ jobs:
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 10000
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: EDC
deploy_ref: refs/heads/develop
job_files: >-
Expand Down Expand Up @@ -90,7 +90,7 @@ jobs:
SECRET_ARN: ${{ secrets.SECRET_ARN }}
CLOUDFORMATION_ROLE_ARN: ${{ secrets.CLOUDFORMATION_ROLE_ARN }}
DEFAULT_CREDITS_PER_USER: ${{ matrix.default_credits_per_user }}
RESET_CREDITS_MONTHLY: ${{ matrix.reset_credits_monthly }}
DEFAULT_APPLICATION_STATUS: ${{ matrix.default_application_status }}
COST_PROFILE: ${{ matrix.cost_profile }}
JOB_FILES: ${{ matrix.job_files }}
DEFAULT_MAX_VCPUS: ${{ matrix.default_max_vcpus }}
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/deploy-enterprise-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: false
default_application_status: APPROVED
cost_profile: DEFAULT
deploy_ref: refs/heads/develop
job_files: >-
Expand All @@ -44,7 +44,7 @@ jobs:
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/ARIA_RAIDER.yml
Expand All @@ -63,7 +63,7 @@ jobs:
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/AUTORIFT_ITS_LIVE.yml
Expand Down Expand Up @@ -109,7 +109,7 @@ jobs:
SECRET_ARN: ${{ secrets.SECRET_ARN }}
CLOUDFORMATION_ROLE_ARN: ${{ secrets.CLOUDFORMATION_ROLE_ARN }}
DEFAULT_CREDITS_PER_USER: ${{ matrix.default_credits_per_user }}
RESET_CREDITS_MONTHLY: ${{ matrix.reset_credits_monthly }}
DEFAULT_APPLICATION_STATUS: ${{ matrix.default_application_status }}
COST_PROFILE: ${{ matrix.cost_profile }}
JOB_FILES: ${{ matrix.job_files }}
DEFAULT_MAX_VCPUS: ${{ matrix.default_max_vcpus }}
Expand Down
26 changes: 13 additions & 13 deletions .github/workflows/deploy-enterprise.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/AUTORIFT_ITS_LIVE.yml
Expand All @@ -39,7 +39,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 180
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/ARIA_RAIDER.yml
Expand All @@ -58,7 +58,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 30
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_ISCE.yml
instance_types: c6id.xlarge,c6id.2xlarge,c6id.4xlarge,c6id.8xlarge
Expand All @@ -75,7 +75,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_ISCE.yml
instance_types: c6id.xlarge,c6id.2xlarge,c6id.4xlarge,c6id.8xlarge
Expand All @@ -92,7 +92,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 365000
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_GAMMA.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -109,7 +109,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/RTC_GAMMA.yml job_spec/WATER_MAP.yml job_spec/WATER_MAP_EQ.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -126,7 +126,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 90
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/RTC_GAMMA.yml job_spec/WATER_MAP.yml job_spec/WATER_MAP_EQ.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -143,7 +143,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 30
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_GAMMA.yml job_spec/INSAR_ISCE_BURST.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -160,7 +160,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_GAMMA.yml job_spec/RTC_GAMMA.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -177,7 +177,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_GAMMA.yml job_spec/RTC_GAMMA.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -194,7 +194,7 @@ jobs:
image_tag: latest
product_lifetime_in_days: 30
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/INSAR_GAMMA.yml job_spec/RTC_GAMMA.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand All @@ -213,7 +213,7 @@ jobs:
# S3 bucket, but maybe we want to allow for a backlog of products-to-be-transferred?
product_lifetime_in_days: 14
default_credits_per_user: 0
reset_credits_monthly: true
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: job_spec/WATER_MAP.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
Expand Down Expand Up @@ -256,7 +256,7 @@ jobs:
SECRET_ARN: ${{ secrets.SECRET_ARN }}
CLOUDFORMATION_ROLE_ARN: ${{ secrets.CLOUDFORMATION_ROLE_ARN }}
DEFAULT_CREDITS_PER_USER: ${{ matrix.default_credits_per_user }}
RESET_CREDITS_MONTHLY: ${{ matrix.reset_credits_monthly }}
DEFAULT_APPLICATION_STATUS: ${{ matrix.default_application_status }}
COST_PROFILE: ${{ matrix.cost_profile }}
JOB_FILES: ${{ matrix.job_files }}
DEFAULT_MAX_VCPUS: ${{ matrix.default_max_vcpus }}
Expand Down
83 changes: 83 additions & 0 deletions .github/workflows/deploy-whitelisting-sandbox.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Deploy Whitelisting Sandbox Stack to AWS

on:
push:
branches:
- whitelisting-sandbox

concurrency: ${{ github.workflow }}-${{ github.ref }}

jobs:
deploy:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- environment: hyp3-whitelisting-sandbox
domain: hyp3-whitelisting-sandbox.asf.alaska.edu
template_bucket: cf-templates-1hz9ldhhl4ahu-us-west-2
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 10
default_application_status: NOT_STARTED
cost_profile: EDC
deploy_ref: refs/heads/whitelisting-sandbox
job_files: >-
job_spec/AUTORIFT.yml
job_spec/INSAR_GAMMA.yml
job_spec/RTC_GAMMA.yml
job_spec/INSAR_ISCE_BURST.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
default_max_vcpus: 640
expanded_max_vcpus: 640
required_surplus: 0
security_environment: ASF
ami_id: /aws/service/ecs/optimized-ami/amazon-linux-2023/recommended/image_id
distribution_url: ''

environment:
name: ${{ matrix.environment }}
url: https://${{ matrix.domain }}

steps:
- uses: actions/[email protected]

- uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.V2_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.V2_AWS_SECRET_ACCESS_KEY }}
aws-session-token: ${{ secrets.V2_AWS_SESSION_TOKEN }}
aws-region: ${{ secrets.AWS_REGION }}

- uses: actions/setup-python@v5
with:
python-version: 3.9

- uses: ./.github/actions/deploy-hyp3
with:
TEMPLATE_BUCKET: ${{ matrix.template_bucket }}
STACK_NAME: ${{ matrix.environment }}
DOMAIN_NAME: ${{ matrix.domain }}
API_NAME: ${{ matrix.environment }}
CERTIFICATE_ARN: ${{ secrets.CERTIFICATE_ARN }}
IMAGE_TAG: ${{ matrix.image_tag }}
PRODUCT_LIFETIME: ${{ matrix.product_lifetime_in_days }}
VPC_ID: ${{ secrets.VPC_ID }}
SUBNET_IDS: ${{ secrets.SUBNET_IDS }}
SECRET_ARN: ${{ secrets.SECRET_ARN }}
CLOUDFORMATION_ROLE_ARN: ${{ secrets.CLOUDFORMATION_ROLE_ARN }}
DEFAULT_CREDITS_PER_USER: ${{ matrix.default_credits_per_user }}
DEFAULT_APPLICATION_STATUS: ${{ matrix.default_application_status }}
COST_PROFILE: ${{ matrix.cost_profile }}
JOB_FILES: ${{ matrix.job_files }}
DEFAULT_MAX_VCPUS: ${{ matrix.default_max_vcpus }}
EXPANDED_MAX_VCPUS: ${{ matrix.expanded_max_vcpus }}
MONTHLY_BUDGET: ${{ secrets.MONTHLY_BUDGET }}
REQUIRED_SURPLUS: ${{ matrix.required_surplus }}
ORIGIN_ACCESS_IDENTITY_ID: ${{ secrets.ORIGIN_ACCESS_IDENTITY_ID }}
SECURITY_ENVIRONMENT: ${{ matrix.security_environment }}
AMI_ID: ${{ matrix.ami_id }}
INSTANCE_TYPES: ${{ matrix.instance_types }}
DISTRIBUTION_URL: ${{ matrix.distribution_url }}
AUTH_PUBLIC_KEY: ${{ secrets.AUTH_PUBLIC_KEY }}
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,32 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [7.2.0]

This release includes changes to support an upcoming user whitelisting feature. A new user will be required to submit an application for a monthly credit allotment and will not be able to submit HyP3 jobs until an operator has manually reviewed and approved the application. As of this release, all new and existing users are automatically approved without being required to submit an application, but this will change in the near future.

⚠️ Important notes for HyP3 deployment operators:
- Changing a user's application status (e.g. to approve or reject a new user) requires manually updating the value of the `application_status` field in the Users table.
- The response for both `/user` endpoints now automatically includes all Users table fields except those prefixed by an underscore (`_`).
- The following manual updates must be made to the Users table upon deployment of this release:
- Add field `application_status` with the appropriate value for each user.
- Rename field `month_of_last_credits_reset` to `_month_of_last_credit_reset`.
- Rename field `notes` to `_notes`.

### Added
- A new `PATCH /user` endpoint with a single `use_case` parameter allows the user to submit an application for a monthly credit allotment or update a pending application. The structure for a successful response is the same as for `GET /user`.
- A new `default_application_status` deployment parameter specifies the default status for new user applications. The parameter has been set to `APPROVED` for all deployments.

### Changed
- The `POST /jobs` endpoint now returns a `403` response if the user has not been approved for a monthly credit allotment.
- The response schema for the `GET /user` endpoint now includes:
- A required `application_status` field representing the status of the user's application: `NOT_STARTED`, `PENDING`, `APPROVED`, or `REJECTED`.
- An optional `use_case` field containing the use case submitted with the user's application.
- An optional `credits_per_month` field representing the user's monthly credit allotment, if different from the deployment default.

### Removed
- The `reset_credits_monthly` deployment parameter has been removed. Credits now reset monthly in all deployments. This only changes the behavior of the `hyp3-enterprise-test` deployment.

## [7.1.1]
### Changed
- Reduced `start_execution_manager` batch size from 600 jobs to 500 jobs. Fixes [#2241](https://github.com/ASFHyP3/hyp3/issues/2241).
Expand Down
4 changes: 2 additions & 2 deletions apps/api/api-cf.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Parameters:
DefaultCreditsPerUser:
Type: Number

ResetCreditsMonthly:
DefaultApplicationStatus:
Type: String

SystemAvailable:
Expand Down Expand Up @@ -182,7 +182,7 @@ Resources:
AUTH_PUBLIC_KEY: !Ref AuthPublicKey
AUTH_ALGORITHM: !Ref AuthAlgorithm
DEFAULT_CREDITS_PER_USER: !Ref DefaultCreditsPerUser
RESET_CREDITS_MONTHLY: !Ref ResetCreditsMonthly
DEFAULT_APPLICATION_STATUS: !Ref DefaultApplicationStatus
SYSTEM_AVAILABLE: !Ref SystemAvailable
Code: src/
Handler: hyp3_api.lambda_handler.handler
Expand Down
Loading

0 comments on commit 739f8b3

Please sign in to comment.