Skip to content

Pipelines developed and used by 4DN-DCIC in Common Workflow Language

License

Notifications You must be signed in to change notification settings

4dn-dcic/pipelines-cwl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pipelines-cwl

  • This is a collection of pipelines built by 4DN-DCIC that were used to process 4DN data on the 4DN portal using Tibanna workflow executor.

Directories

  • cwl_awsem_v1 : Tibanna AWSEM CWLs in v1.0 in json format

How to run the cwl

To run docker through CWL, you need a cwl executor - we use cwltool (https://github.com/common-workflow-language/cwltool) to run CWL with a json/yml file describing input data. Some example input data are inside the tests/test_input_json directory and you can see some cwltool (=cwl-runner) commands inside the tests/tests.sh script.

Testing CWL

To test cwls in this repo against the test files in tests, use tests/tests_v1.sh with cwl name (without .cwl).

source tests/tests_v1.sh bwa-mem

Notes on releases

Freeze 0.2.7

  • All CWL is now in the folder cwl.

The 4DN pipeline suite freeze 0.2.7 is equivalent to the following.

  • Hi-C processing pipeline v43
  • Repliseq processing pipeline v16
  • FastQC v2

Freeze >= 0.2.6

  • All CWL is converted to version 1 (no more draft3) and they are kept in the folder cwl_awsem_v1.

Freeze >=0.0.2

  • Directory cwl_awsem is manually updated starting from freeze 0.0.2 (after Sep 1, 2017), since we no longer use SevenBridges.

Freeze 0.0.1

  • Freeze 0.0.1 contains exported SBG CWLs and AWSEM CWL files auto-converted from SBG CWLs. The following is a description of how freeze 0.0.1 was generated.

    The content has been generated by the following command.

    source download.sh 
    

    For this to work, you need to have SBG_TOKEN as an environmental variable on your machine.

    To convert sbg cwl to awsem cwl in a batch, do the following

    source convert.sh
    

    This conversion uses script convert_sbgcwl_to_awsemcwl.py individually. The shell script assumes Mac (xargs -I{} instead of xargs -i).