Skip to content
This repository has been archived by the owner on Aug 9, 2023. It is now read-only.

Major Release - v3.0.0
Compare
Choose a tag to compare
@wleepang wleepang released this 21 Oct 07:03
· 187 commits to master since this release
v3.0.0
5397c0c

NOTE: This release provides an improved deployment experience and extensibility as a trade-off to breaking changes with prior 2.x releases.

New

  • A generalized GWFCore that is agnostic of workflow orchestrator and can be used with multiple orchestrators at once.

    • Uses AWS Systems Manager Parameter Store for GWFCore attributes like S3 Bucket paths and AWS Batch Queue Names
    • Project namespacing for multi-tenent deployment
    • Deployments are self-contained - distributed artifacts (e.g. Amazon EBS Autoscale) are stored in S3 on initial deployment
    • Launch template based provisioning can be updated without rebuilding AWS Batch Compute Environments
      • Uses provisioning scripts stored in AWS CodeCommit with changes automatically deployed to S3 via AWS CodeBuild and AWS CodePipeline
    • AWS Batch Compute Environments uses 5th generation C, M, R instance families for cost optimization and limits max instance sizes to 4xl to reduce impact of SPOT terminations

  • Updated Cromwell architecture

    • Compatible with Cromwell v52+
    • Includes Aurora Serveless database for call-caching metadata store
    • Cromwell server instance updated to Amazon Linux 2

  • Generalized container builds for nextflow and Step Functions demo workflow containers

    • Container Dockerfiles stored in AWS CodeCommit
    • Container image builds via AWS CodeBuild

  • Tagged builds are version pinned. Templates and assets are accessible via https://<bucketname>.s3.amazonaws.com/<version>/<assetpath> with <version> being vX.Y.Z or latest. Assets are no longer available at the root bucket level.

Minor fixes and improvements

  • Nextflow container uses upstream base image for version pinning
  • Nextflow container entrypoint captures AWS Batch job termination and gracefully terminates workflow
  • Consistent naming for orchestrator templates

Documentation changes

  • AIO templates removed. These are no longer needed as the GWFCore is generalized and compatible with all documented orchestrators. Recommended installation procedure is to 1) Create or select a VPC with at least 2 public and 2 private subnets; 2) Deploy GWFCore; 3) Deploy desired orchestrators.

  • Add Cromwell and Nextflow "Troubleshooting" documentation

  • Add Cost Optimization documentation

Assets 2
Loading