Skip to content

5.0.0
Compare
Choose a tag to compare
@DailyDreaming DailyDreaming released this 04 Dec 03:52
· 752 commits to master since this release
releases/5.0.0
f182c64
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Caching is now turned on by default.

CWL v1.1 is also now fully supported and passing all conformance tests.

The following batch system names are now deprecated (#3225) and replaced by:

  • singleMachine -> single_machine
  • gridEngine -> grid_engine
  • LSF -> lsf
  • Mesos -> mesos
  • Slurm -> slurm
  • Torque -> torque
  • HTCondor -> htcondor
  • Kubernetes -> kubernetes
  • k8s -> kubernetes

Changelog

CWL

  • Upgrade to a CWL v1.2.0 capable cwltool (3.0.20200709181526 -> 3.0.20200807132242). #3137
  • Export to a bucket if cwl's CreateFile is specified. #3124
  • CWL v1.1 Support: loadContents if True for StepValueFrom objects. #3266
  • CWL: Fix workdir permissions. #3229
  • Test CWL secondary file from s3. #3311
  • CWL v1.1 Support: Inplace update has side effect on directory content. #3280
  • More CWL v1.2 conformance tests. #3336
  • CWL: Reformat w/ Black and better support for tmpdir_prefix with suffix. #3333

WDL

  • Add ceil() and floor() WDL functions. #3168
  • Create a stdout and stderr file for each WDL task. #3181
  • Add WDL write functions and builtin tests. #3236
    • write_lines
    • write_tsv
    • write_json
    • write_map
  • Add WDL read functions and builtin tests. #3244
    • read_lines
    • read_tsv
    • read_json
    • read_map
    • read_int
    • read_string
    • read_float
    • read_boolean
  • Add transpose() WDL function. #3273
  • Add length() WDL function. #3307
  • Add sub() WDL function. #3252
  • Add range() WDL function. #3277
  • Add size() WDL function. #3255
  • Add zip() WDL function. #3355
  • Implement WDL Pair type. #3304
  • Refactor WDL types #3335

Kubernetes

  • Adding Labels to Pods/Jobs. #3233
  • Make kubernetes & botocore packages fully optional. #3259 #3261
  • KubeWatch optimization. #3227
  • Catch other kubernetes imports. #3261
  • Automatically infer singularity for kubernetes. #3279

Misc

  • Allow Caching By Default. #3111
  • Update dependencies and loosen restrictions slightly. #3139
  • Ensure that falsey (but non-null) values are considered. #3143
  • Remove pathlib. #3169
  • Move AWS job store uploads/downloads to boto3. #3153
  • Update prom/node-exporter to use quay.io. #3186
  • Add support for TOIL_CUSTOM_INIT_COMMAND. #3183
  • Update docker module version to 4.3.1. #3222
  • Add 'stream' and 'demux' options to apiDockerCall. #3224
  • Use boto3 to create our leader node. #3145
  • Prevent job kind filenames from getting too long. #3230
  • Add a retry decorator. #3144
  • Remove mesos offer messages. #3308
  • Job Concept Unification. #3250
  • Add reporting for accessed files of failed jobs. #3309
  • Add coverage and refactor. #3314
  • Expose --statePollingWait param through toil. #3321
  • Record the maximum memory used by LSF jobs. #3327
  • Remove custom math functions and replace with built ins. #3338
  • Define a changelog process. #3316
  • SLURM: getting job details from scontrol show jobs was non-functional and has now been fixed. This behavior in the case of sacct not being configured now works. #3346
  • LSF: handle unicode configuration files #3354
  • Replace abssympath w/ builtin. #3356

Bug Fixes

  • Turn off watch code in Kubernetes. #3175
  • Check against joining the current thread during CachingFileStore destructor. #3187
  • Slurm.py error, undefined stdout - stdout should be stderr in code. #3194
  • Fix restart flag error (issue #3094). #3142
  • Batch System generates a key error instead of a usage message. #3225
  • Remove Python 2 imports in WDL. #3228
  • Move checkForDeadlocks() in the leader. #3234
  • Fix boto3 migration bugs. #3243
  • Fix None check when parsing WDL JSON file. #3272
  • Fix broken stdout/stderr in torque job. #3306
  • Add doubleMem for LSF jobs that die due to imposed memory limit. #3313
  • Add 500 status to the list of retriable SDB BotoServerErrors. #3329
  • Add a retry logic for BucketNotEmpty error. #3341
  • Fix None check when parsing WDL JSON file. #3272
  • Warn when trying to clean non-existent job store #3288
  • Only rm a statefile that exists. #3348
  • CWL: Remove unprocessed secondaryFiles from both Dict and List types.

Thank you to our contributors: @mr-c, @arostamianfar, @adamnovak, @diekhans, @w-gao, @jonathanxu18, @jeffrey856, @davidlougheed, @altairwei, @mberacochea, @drkennetz, @ionox0!

Assets 2
Loading