Minor Release - v3.1.0

What's Changed

• Fix graceful cancellation behavior for Nextflow by @wleepang in #160
• Bump to cdk v1.102.0 by @itzhapaz in #162
• Bump cdk version by @wleepang in #163
• Change examples source files by @itzhapaz in #164
• Unified cromwell and gwf-core template by @henriqueribeiro in #166
• Cromwell Docker authentication by @henriqueribeiro in #165
• Add message about s3 bucket location by @itzhapaz in #168
• Add EFS support in Nextflow and other minor changes by @ajfriedman18 in #169
• Add nextflow helper script by @wleepang in #170
• Update README.md by @hyandell in #179
• Cromwell install docs by @markjschreiber in #180
• GWF core auto update code pipeline by @markjschreiber in #182
• FSx Lustre deployment and integration with Cromwell by @spatel-gfb in #186, #187
• Bump minimist from 1.2.5 to 1.2.6 in /src/aws-genomics-cdk by @dependabot in #189

New Contributors

• @ajfriedman18 made their first contribution in #169
• @hyandell made their first contribution in #179
• @spatel-gfb made their first contribution in #186
• @dependabot made their first contribution in #189

Full Changelog: v3.0.7...v3.1.0

Patch release - v3.0.7

Bug fixes

• #159 - add server timeout config for cromwell, specify default region in cromwell server instance

Improvements

• #138, #154, #157 - add CDK constructs
• #152 - add custom deployment instructions to docs
• #156 - add a nextflow+gwfcore combined template

Known Issues

• #141 - Globbing error staging input data, causing Step Functions workflows to fail

Patch release - v3.0.6.1

Hotfix

• fix site deployment

Patch release - v3.0.6

Bug fixes

• #144 - run get_cromwell_tools when provisioning cromwell server

Updates

• Use nextflow image from AWS ECR Public Gallery

Improvements

• #147 - improve deployment scripts for custom distributions
  • add verbosity control
  • add public asset control
• #143 - get cromwell.jar from s3, allow cromwell access to user defined private buckets

Known issues

• #141 - Globbing error staging input data, causing Step Functions workflows to fail

Patch release - v3.0.5

Bug fixes

• #119 - Move files from tmp dir on Cromwell server
• #131 - Enable 4GB swap on instances
• #134 , #136 - Use "main" branch name for CodeCommit repos
• #135 - Use aws-cli shim for Cromwell

Updates

• Use Amazon Linux 2 for instances

Improvements

• #127 - Send instance provisioning logs to CloudWatch
• Use ECR public repos for container base images

Known issues

• #141 - Globbing error staging input data, causing Step Functions workflows to fail

Patch release - v3.0.4

Bug fixes

• #128 , #132 - fixed issue running AWS CLI v2 with container images that lack glibc

Updates

• #125 - fix README instructions for self-deployment, add instructions to access Cromwell server via Amazon Systems Manager.

Improvements

• #126 - add womtool and cromshell to Cromwell server instance

Patch release - v3.0.3

Bug fixes

• #121 - Add SSM policy to Cromwell Server EC2 instance
• #116 - Change Nextflow example to use demo RNASeq workflow from nextflow-io

Improvements

• #123 - Prefer using cached container instances to reduce container image pull traffic

Patch release - v3.0.2

Bug fixes

• use lower-case stack names when launching from guide for s3 bucket naming compatibility as these are potentially used for namespaces
• fix s3 http url allowed pattern for template root url parameters
• pin "test" as a version for e2e testing

Improvements

• Check for dependencies in _scripts/make_dist

Patch release - v3.0.1

Updates

• Add check for aws-cli installation and termination if not available
• Enable ECS SPOT instance draining

Major Release - v3.0.0

NOTE: This release provides an improved deployment experience and extensibility as a trade-off to breaking changes with prior 2.x releases.

New

• A generalized GWFCore that is agnostic of workflow orchestrator and can be used with multiple orchestrators at once.

  • Uses AWS Systems Manager Parameter Store for GWFCore attributes like S3 Bucket paths and AWS Batch Queue Names
  • Project namespacing for multi-tenent deployment
  • Deployments are self-contained - distributed artifacts (e.g. Amazon EBS Autoscale) are stored in S3 on initial deployment
  • Launch template based provisioning can be updated without rebuilding AWS Batch Compute Environments
    • Uses provisioning scripts stored in AWS CodeCommit with changes automatically deployed to S3 via AWS CodeBuild and AWS CodePipeline
  • AWS Batch Compute Environments uses 5th generation C, M, R instance families for cost optimization and limits max instance sizes to 4xl to reduce impact of SPOT terminations

• Updated Cromwell architecture

  • Compatible with Cromwell v52+
  • Includes Aurora Serveless database for call-caching metadata store
  • Cromwell server instance updated to Amazon Linux 2

• Generalized container builds for nextflow and Step Functions demo workflow containers

  • Container Dockerfiles stored in AWS CodeCommit
  • Container image builds via AWS CodeBuild

• Tagged builds are version pinned. Templates and assets are accessible via https://<bucketname>.s3.amazonaws.com/<version>/<assetpath> with <version> being vX.Y.Z or latest. Assets are no longer available at the root bucket level.

Minor fixes and improvements

• Nextflow container uses upstream base image for version pinning
• Nextflow container entrypoint captures AWS Batch job termination and gracefully terminates workflow
• Consistent naming for orchestrator templates

Documentation changes

• AIO templates removed. These are no longer needed as the GWFCore is generalized and compatible with all documented orchestrators. Recommended installation procedure is to 1) Create or select a VPC with at least 2 public and 2 private subnets; 2) Deploy GWFCore; 3) Deploy desired orchestrators.

• Add Cromwell and Nextflow "Troubleshooting" documentation

• Add Cost Optimization documentation