Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v6.4.0 #2146

Merged
merged 12 commits into from
Mar 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/deploy-daac.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ jobs:
deploy_ref: refs/heads/main
job_files: job_spec/AUTORIFT.yml job_spec/INSAR_GAMMA.yml job_spec/RTC_GAMMA.yml job_spec/INSAR_ISCE_BURST.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
default_max_vcpus: 1500
expanded_max_vcpus: 3000
required_surplus: 3000
default_max_vcpus: 1200
expanded_max_vcpus: 2400
required_surplus: 2000
security_environment: EDC
ami_id: image_id_ecs_amz2
distribution_url: 'https://d3gm2hf49xd6jj.cloudfront.net'
Expand All @@ -51,9 +51,9 @@ jobs:
job_spec/INSAR_ISCE_TEST.yml
job_spec/INSAR_ISCE_BURST.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
default_max_vcpus: 1500
expanded_max_vcpus: 3000
required_surplus: 3000
default_max_vcpus: 1200
expanded_max_vcpus: 2400
required_surplus: 2000
security_environment: EDC
ami_id: image_id_ecs_amz2
distribution_url: 'https://d1riv60tezqha9.cloudfront.net'
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [6.4.0]

### Changed
- Reduced vcpu limits for EDC deployments from 1,500/3,000 to 1,200/2,400.

### Removed
- The `disable-private-dns` lambda function added in v4.3.2 has been removed; the underlying issue has been resolved in
the Earthdata Cloud platform. Fixes [#1956](https://github.com/ASFHyP3/hyp3/issues/1956).

## [6.3.0]

### Changed
Expand Down
6 changes: 2 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,10 @@ HANDLE_BATCH_EVENT = ${PWD}/apps/handle-batch-event/src
SCALE_CLUSTER = ${PWD}/apps/scale-cluster/src
START_EXECUTION_MANAGER = ${PWD}/apps/start-execution-manager/src
START_EXECUTION_WORKER = ${PWD}/apps/start-execution-worker/src
DISABLE_PRIVATE_DNS = ${PWD}/apps/disable-private-dns/src
UPDATE_DB = ${PWD}/apps/update-db/src
UPLOAD_LOG = ${PWD}/apps/upload-log/src
DYNAMO = ${PWD}/lib/dynamo
export PYTHONPATH = ${API}:${CHECK_PROCESSING_TIME}:${GET_FILES}:${HANDLE_BATCH_EVENT}:${SCALE_CLUSTER}:${START_EXECUTION_MANAGER}:${START_EXECUTION_WORKER}:${DISABLE_PRIVATE_DNS}:${UPDATE_DB}:${UPLOAD_LOG}:${DYNAMO}
export PYTHONPATH = ${API}:${CHECK_PROCESSING_TIME}:${GET_FILES}:${HANDLE_BATCH_EVENT}:${SCALE_CLUSTER}:${START_EXECUTION_MANAGER}:${START_EXECUTION_WORKER}:${UPDATE_DB}:${UPLOAD_LOG}:${DYNAMO}


build: render
Expand All @@ -19,7 +18,6 @@ build: render
python -m pip install --upgrade -r requirements-apps-scale-cluster.txt -t ${SCALE_CLUSTER}; \
python -m pip install --upgrade -r requirements-apps-start-execution-manager.txt -t ${START_EXECUTION_MANAGER}; \
python -m pip install --upgrade -r requirements-apps-start-execution-worker.txt -t ${START_EXECUTION_WORKER}; \
python -m pip install --upgrade -r requirements-apps-disable-private-dns.txt -t ${DISABLE_PRIVATE_DNS}; \
python -m pip install --upgrade -r requirements-apps-update-db.txt -t ${UPDATE_DB}

test_file ?= tests/
Expand All @@ -44,7 +42,7 @@ render:
static: flake8 openapi-validate cfn-lint

flake8:
flake8 --ignore=E731 --max-line-length=120 --import-order-style=pycharm --statistics --application-import-names hyp3_api,get_files,handle_batch_event,check_processing_time,start_execution_manager,start_execution_worker,disable_private_dns,update_db,upload_log,dynamo,lambda_logging,scale_cluster apps tests lib
flake8 --ignore=E731 --max-line-length=120 --import-order-style=pycharm --statistics --application-import-names hyp3_api,get_files,handle_batch_event,check_processing_time,start_execution_manager,start_execution_worker,update_db,upload_log,dynamo,lambda_logging,scale_cluster apps tests lib

openapi-validate: render
openapi-spec-validator apps/api/src/hyp3_api/api-spec/openapi-spec.yml
Expand Down
91 changes: 0 additions & 91 deletions apps/disable-private-dns/disable-private-dns-cf.yml.j2

This file was deleted.

48 changes: 0 additions & 48 deletions apps/disable-private-dns/src/disable_private_dns.py

This file was deleted.

11 changes: 0 additions & 11 deletions apps/main-cf.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -355,14 +355,3 @@ Resources:
KeySchema:
- AttributeName: user_id
KeyType: HASH

{% if security_environment == 'EDC' %}
DisablePrivateDNS:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
VpcId: !Ref VpcId
SecurityGroupId: !GetAtt Cluster.Outputs.SecurityGroupId
SubnetIds: !Join [",", !Ref SubnetIds]
TemplateURL: disable-private-dns/disable-private-dns-cf.yml
{% endif %}
5 changes: 2 additions & 3 deletions requirements-all.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@
-r requirements-apps-scale-cluster.txt
-r requirements-apps-start-execution-manager.txt
-r requirements-apps-start-execution-worker.txt
-r requirements-apps-disable-private-dns.txt
-r requirements-apps-update-db.txt
boto3==1.34.60
boto3==1.34.62
jinja2==3.1.3
moto[dynamodb]==5.0.3.dev33
pytest==8.1.1
Expand All @@ -15,6 +14,6 @@ flake8==7.0.0
flake8-import-order==0.18.2
flake8-blind-except==0.2.1
flake8-builtins==2.2.0
setuptools==69.1.1
setuptools==69.2.0
openapi-spec-validator==0.7.1
cfn-lint==0.86.0
1 change: 0 additions & 1 deletion requirements-apps-disable-private-dns.txt

This file was deleted.

2 changes: 1 addition & 1 deletion requirements-apps-start-execution-manager.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
boto3==1.34.60
boto3==1.34.62
./lib/dynamo/
./lib/lambda_logging/
2 changes: 1 addition & 1 deletion requirements-apps-start-execution-worker.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
boto3==1.34.60
boto3==1.34.62
./lib/lambda_logging/
Loading