Merge pull request #324 from ggabernet/dev

Make RAxML default, use profile for full size test
nf-core · Apr 22, 2024 · 1402385 · 1402385
2 parents e6eacdd + 43cf4d5
commit 1402385
Show file tree

Hide file tree

Showing 34 changed files with 87 additions and 113 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,7 +3,7 @@
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
-## [3.3.1] - 2024-04-11 Confringo hotfix
+## [4.0] - 2024-04-22 Ascendio
 
 ### `Added`
 
@@ -13,6 +13,18 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 - [#319](https://github.com/nf-core/airrflow/pull/319) Fix test full profile and nebnext_umi_tcr profile.
 - [#321](https://github.com/nf-core/airrflow/pull/321) Label Dowser tips by isotype instead of c_call by default.
+- [#322](https://github.com/nf-core/airrflow/pull/322) Use RAxML as the default builder for dowser. Skip lineage trees by default.
+
+### `Dependencies`
+
+| Dependency | Old version | New version |
+| ---------- | ----------- | ----------- |
+| enchantr   | 0.1.11      | 0.1.14      |
+
+### `Deprecated parameters`
+
+- `--skip_lineage_trees` is now deprecated in favor of `--lineage_trees`. Lineage trees are skipped by default.
+- `--igphyml` parameter is deprecated in favor of `--lineage_tree_exec`. All lineage tree building software part of Dowser are now supported.
 
 ## [3.3.0] - 2024-03-31 Confringo
 

diff --git a/CITATIONS.md b/CITATIONS.md
@@ -32,42 +32,10 @@
 
   > Gupta, N. T., Vander Heiden, J. A., Uduman, M., Gadala-Maria, D., Yaari, G., & Kleinstein, S. H. (2015). Change-O: a toolkit for analyzing large-scale B cell immunoglobulin repertoire sequencing data: Table 1. Bioinformatics, 31(20), 3356–3358.
 
-- [Alakazam](https://doi.org/10.1126/scitranslmed.3008879)
-
-  > Stern, J. N. H., Yaari, G., Vander Heiden, J. A., Church, G., Donahue, W. F., Hintzen, R. Q., … O’Connor, K. C. (2014). B cells populating the multiple sclerosis brain mature in the draining cervical lymph nodes. Science Translational Medicine, 6(248).
-
-- [SCOPer](https://doi.org/10.1093/bioinformatics/bty235)
-
-  > Nouri N, Kleinstein S (2018). “A spectral clustering-based method for identifying clones from high-throughput B cell repertoire sequencing data.” Bioinformatics, i341-i349.
-
-  > Nouri N, Kleinstein S (2020). “Somatic hypermutation analysis for improved identification of B cell clonal families from next-generation sequencing data.” PLOS Computational Biology, 16(6), e1007977.
-
-  > Gupta N, Adams K, Briggs A, Timberlake S, Vigneault F, Kleinstein S (2017). “Hierarchical clustering can identify B cell clones with high confidence in Ig repertoire sequencing data.” The Journal of Immunology, 2489-2499.
-
-- [Dowser](https://doi.org/10.1371/journal.pcbi.1009885)
-
-  > Hoehn K, Pybus O, Kleinstein S (2022). “Phylogenetic analysis of migration, differentiation, and class switching in B cells.” PLoS Computational Biology.
-
-- [IgPhyML](https://www.pnas.org/doi/10.1073/pnas.1906020116)
-
-  > Hoehn K, Van der Heiden J, Zhou J, Lunter G, Pybus O, Kleinstein S (2019). “Repertoire-wide phylogenetic models of B cell molecular evolution reveal evolutionary signatures of aging and vaccination.” PNAS.
-
 - [IgBLAST](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3692102/)
 
   > Ye J, Ma N, Madden TL, Ostell JM. (2013). IgBLAST: an immunoglobulin variable domain sequence analysis tool. Nucleic Acids Res.
 
-- [Fastp](https://doi.org/10.1093/bioinformatics/bty560)
-
-  > Shifu Chen, Yanqing Zhou, Yaru Chen, Jia Gu, fastp: an ultra-fast all-in-one FASTQ preprocessor, Bioinformatics. 2018 Sept 1; 34(17), i884–i890. doi: 10.1093/bioinformatics/bty560.
-
-- [pRESTO](https://doi.org/10.1093/bioinformatics/btu138)
-
-  > Vander Heiden, J. A., Yaari, G., Uduman, M., Stern, J. N. H., O’Connor, K. C., Hafler, D. A., … Kleinstein, S. H. (2014). pRESTO: a toolkit for processing high-throughput sequencing raw reads of lymphocyte receptor repertoires. Bioinformatics, 30(13), 1930–1932.
-
-- [SHazaM, Change-O](https://doi.org/10.1093/bioinformatics/btv359)
-
-  > Gupta, N. T., Vander Heiden, J. A., Uduman, M., Gadala-Maria, D., Yaari, G., & Kleinstein, S. H. (2015). Change-O: a toolkit for analyzing large-scale B cell immunoglobulin repertoire sequencing data. Bioinformatics, 31(20), 3356–3358.
-
 - [Alakazam](https://doi.org/10.1126/scitranslmed.3008879)
 
   > Stern, J. N. H., Yaari, G., Vander Heiden, J. A., Church, G., Donahue, W. F., Hintzen, R. Q., … O’Connor, K. C. (2014). B cells populating the multiple sclerosis brain mature in the draining cervical lymph nodes. Science Translational Medicine, 6(248).
@@ -88,6 +56,10 @@
 
   > Hoehn K, Van der Heiden J, Zhou J, Lunter G, Pybus O, Kleinstein S (2019). “Repertoire-wide phylogenetic models of B cell molecular evolution reveal evolutionary signatures of aging and vaccination. PNAS, 116(45) 22664-22672."
 
+- [RAxML](10.1093/bioinformatics/btu033)
+
+  > Stamatakis A. (2014) RAxML version 8: a tool for phylogenetic analysis and post-analysis of large phylogenies. Bioinformatics, 30(9): 1312-1313.
+
 - [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
 
   > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.

diff --git a/README.md b/README.md
@@ -81,8 +81,8 @@ nf-core/airrflow allows the end-to-end processing of BCR and TCR bulk and single
 4. Clonal analysis (bulk and single-cell)
 
 - Find threshold for clone definition (`SHazaM`, `EnchantR`).
-- Create germlines and define clones, repertoire analysis (`Change-O`, `EnchantR`).
-- Build lineage trees (`SCOPer`, `IgphyML`, `EnchantR`).
+- Create germlines and define clones, repertoire analysis (`SCOPer`, `EnchantR`).
+- Build lineage trees (`Dowser`, `IgphyML`, `RAxML`, `EnchantR`).
 
 5. Repertoire analysis and reporting
 
@@ -125,6 +125,16 @@ nextflow run nf-core/airrflow \
 --outdir ./results
 ```
 
+For common **bulk sequencing protocols** we provide pre-set profiles that specify primers, UMI length, etc for common commercially available sequencing protocols. Please check the [Supported protocol profiles](#supported-protocol-profiles) for a full list of available profiles. An example command running the NEBNext UMI protocol profile with docker containers is:
+
+```bash
+nextflow run nf-core/airrflow \
+-profile nebnext_umi,docker \
+--mode fastq \
+--input input_samplesheet.tsv \
+--outdir results
+```
+
 A typical command to run the pipeline from **single cell raw fastq files** (10X genomics) is:
 
 ```bash

diff --git a/assets/multiqc_config.yml b/assets/multiqc_config.yml
@@ -1,5 +1,5 @@
 report_comment: >
-  This report has been generated by the <a href="https://github.com/nf-core/airrflow/releases/tag/3.3.1" target="_blank">nf-core/airrflow</a>
+  This report has been generated by the <a href="https://github.com/nf-core/airrflow/releases/tag/4.0" target="_blank">nf-core/airrflow</a>
   analysis pipeline. For information about how to interpret these results, please see the
   <a href="https://nf-co.re/airrflow" target="_blank">documentation</a>.
 

diff --git a/assets/repertoire_comparison.Rmd b/assets/repertoire_comparison.Rmd
@@ -423,6 +423,10 @@ In addition, citations for the tools and data used in this pipeline are as follo
 
   > Hoehn K, Van der Heiden J, Zhou J, Lunter G, Pybus O, Kleinstein S (2019). “Repertoire-wide phylogenetic models of B cell molecular evolution reveal evolutionary signatures of aging and vaccination.” PNAS.
 
+- [RAxML](10.1093/bioinformatics/btu033)
+
+  > Stamatakis A. (2014) RAxML version 8: a tool for phylogenetic analysis and post-analysis of large phylogenies. Bioinformatics, 30(9): 1312-1313.
+
 - [TIgGER](https://doi.org/10.1073/pnas.1417683112)
 
   > Gadala-maria, D., Yaari, G., Uduman, M., & Kleinstein, S. H. (2015). Automated analysis of high-throughput B-cell sequencing data reveals a high frequency of novel immunoglobulin V gene segment alleles. Proceedings of the National Academy of Sciences, 112(8), 1–9.

diff --git a/conf/clontech_umi_tcr.config b/conf/clontech_umi_tcr.config
@@ -40,5 +40,4 @@ params {
 
     // TCR options
     clonal_threshold = 0
-    skip_lineage = true
 }
diff --git a/conf/modules.config b/conf/modules.config
@@ -562,8 +562,7 @@ process {
             mode: params.publish_dir_mode,
             saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
         ]
-        ext.args = ['build':'igphyml',
-                    'minseq':5,
+        ext.args = ['minseq':5,
                     'traits':'isotype',
                     'tips':'isotype']
     }

diff --git a/conf/nebnext_umi_tcr.config b/conf/nebnext_umi_tcr.config
@@ -37,5 +37,4 @@ params {
 
     //TCR options
     clonal_threshold = 0
-    skip_lineage = true
 }
diff --git a/conf/test.config b/conf/test.config
@@ -35,6 +35,7 @@ params {
     umi_position = 'R1'
     index_file = true
     isotype_column = 'c_primer'
+    lineage_trees = true
 }
 
 process{

diff --git a/conf/test_assembled_hs.config b/conf/test_assembled_hs.config
@@ -27,5 +27,6 @@ params {
     collapseby = 'filename'
     cloneby = 'subject_id'
     remove_chimeric = true
+    lineage_trees = true
 }
 
diff --git a/conf/test_assembled_immcantation_devel_hs.config b/conf/test_assembled_immcantation_devel_hs.config
@@ -21,7 +21,6 @@ params {
     input = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/testdata-reveal/test_assembled_metadata_hs.tsv'
     imgtdb_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/imgtdb_base.zip'
     igblast_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/igblast_base.zip'
-    igphyml = '/usr/local/share/igphyml/src/igphyml'
 
     reassign = true
     productive_only = true

diff --git a/conf/test_assembled_immcantation_devel_mm.config b/conf/test_assembled_immcantation_devel_mm.config
@@ -21,7 +21,6 @@ params {
     input = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/testdata-reveal/test_assembled_metadata_mm.tsv'
     imgtdb_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/imgtdb_base.zip'
     igblast_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/igblast_base.zip'
-    igphyml = '/usr/local/share/igphyml/src/igphyml'
 
     reassign = true
     productive_only = true

diff --git a/conf/test_assembled_mm.config b/conf/test_assembled_mm.config
@@ -27,5 +27,7 @@ params {
     collapseby = 'filename'
     cloneby = 'subject_id'
     remove_chimeric = true
+
+    lineage_trees = true
 }
 
diff --git a/conf/test_clontech_umi.config b/conf/test_clontech_umi.config
@@ -27,6 +27,5 @@ params {
     igblast_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/igblast_base.zip'
 
     clonal_threshold = 0.1
-    skip_lineage = true
 
 }
diff --git a/conf/test_full.config b/conf/test_full.config
@@ -9,44 +9,16 @@
 
 ----------------------------------------------------------------------------------------
 */
+includeConfig 'nebnext_umi_bcr.config'
 
 params {
     config_profile_name        = 'Full test profile'
     config_profile_description = 'Full test dataset to check pipeline function'
 
     // Input data
     input = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/testdata-bcr/metadata_pcr_umi_airr_300.tsv'
-    cprimers = 's3://ngi-igenomes/test-data/airrflow/pcr_umi/cprimers.fasta'
-    vprimers = 's3://ngi-igenomes/test-data/airrflow/pcr_umi/vprimers.fasta'
     imgtdb_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/imgtdb_base.zip'
     igblast_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/igblast_base.zip'
 
-    // Other params
-    library_generation_method = 'specific_pcr_umi'
-    cprimer_position = 'R1'
-    umi_length = 15
-    umi_start = 0
-    umi_position = 'R1'
-}
-
-process {
-    withName:DOWSER_LINEAGES{
-        ext.args = ['build':'igphyml',
-                    'minseq':5,
-                    'traits':'isotype',
-                    'tips':'isotype']
-    }
-
-    withName:DEFINE_CLONES_COMPUTE{
-        ext.args = ['outname':'', 'model':'hierarchical',
-                    'method':'nt', 'linkage':'single',
-                    'min_n':30, 'isotype_column':'c_primer']
-
-    }
-    withName:DEFINE_CLONES_REPORT{
-        ext.args = ['outname':'', 'model':'hierarchical',
-                    'method':'nt', 'linkage':'single',
-                    'min_n':30, 'isotype_column':'c_primer']
-
-    }
+    lineage_trees = true
 }
diff --git a/conf/test_nebnext_umi.config b/conf/test_nebnext_umi.config
@@ -28,6 +28,5 @@ params {
     igblast_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/igblast_base.zip'
 
     clonal_threshold = 0.1
-    skip_lineage = true
 
 }
diff --git a/conf/test_raw_immcantation_devel.config b/conf/test_raw_immcantation_devel.config
@@ -26,7 +26,6 @@ params {
 
     imgtdb_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/imgtdb_base.zip'
     igblast_base = 'https://raw.githubusercontent.com/nf-core/test-datasets/airrflow/database-cache/igblast_base.zip'
-    igphyml = '/usr/local/share/igphyml/src/igphyml'
 
     mode = 'fastq'
 

diff --git a/conf/test_tcr.config b/conf/test_tcr.config
@@ -25,7 +25,6 @@ params {
     library_generation_method = 'dt_5p_race_umi'
     cprimer_position = 'R1'
     clonal_threshold = 0
-    skip_lineage = true
 
 
     // Input data

diff --git a/modules/local/airrflow_report/airrflow_report.nf b/modules/local/airrflow_report/airrflow_report.nf
@@ -6,8 +6,8 @@ process AIRRFLOW_REPORT {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     tuple val(meta), path(tab) // sequence tsv table in AIRR format

diff --git a/modules/local/enchantr/collapse_duplicates.nf b/modules/local/enchantr/collapse_duplicates.nf
@@ -8,8 +8,8 @@ process COLLAPSE_DUPLICATES {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     tuple val(meta), path(tabs) // tuple [val(meta), sequence tsv in AIRR format ]

diff --git a/modules/local/enchantr/define_clones.nf b/modules/local/enchantr/define_clones.nf
@@ -25,8 +25,8 @@ process DEFINE_CLONES {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     tuple val(meta), path(tabs) // meta, sequence tsv in AIRR format

diff --git a/modules/local/enchantr/detect_contamination.nf b/modules/local/enchantr/detect_contamination.nf
@@ -9,8 +9,8 @@ process DETECT_CONTAMINATION {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     path(tabs)

diff --git a/modules/local/enchantr/dowser_lineages.nf b/modules/local/enchantr/dowser_lineages.nf
@@ -25,8 +25,8 @@ process DOWSER_LINEAGES {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     tuple val(meta), path(tabs)
@@ -43,7 +43,8 @@ process DOWSER_LINEAGES {
     """
     Rscript -e "enchantr::enchantr_report('dowser_lineage', \\
                                         report_params=list('input'='${tabs}', \\
-                                        'exec'='${params.igphyml}', \\
+                                        'build'='${params.lineage_tree_builder}', \\
+                                        'exec'='${params.lineage_tree_exec}', \\
                                         'outdir'=getwd(), \\
                                         'nproc'=${task.cpus},\\
                                         'log'='${id_name}_dowser_command_log' ${args}))"

diff --git a/modules/local/enchantr/find_threshold.nf b/modules/local/enchantr/find_threshold.nf
@@ -25,8 +25,8 @@ process FIND_THRESHOLD {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
 
     input:

diff --git a/modules/local/enchantr/remove_chimeric.nf b/modules/local/enchantr/remove_chimeric.nf
@@ -9,8 +9,8 @@ process REMOVE_CHIMERIC {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
 
     input:

diff --git a/modules/local/enchantr/report_file_size.nf b/modules/local/enchantr/report_file_size.nf
@@ -10,8 +10,8 @@ process REPORT_FILE_SIZE {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     path logs

diff --git a/modules/local/enchantr/single_cell_qc.nf b/modules/local/enchantr/single_cell_qc.nf
@@ -24,8 +24,8 @@ process SINGLE_CELL_QC {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     path(tabs)

diff --git a/modules/local/enchantr/validate_input.nf b/modules/local/enchantr/validate_input.nf
@@ -10,8 +10,8 @@ process VALIDATE_INPUT {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     input:
     file samplesheet

diff --git a/modules/local/reveal/add_meta_to_tab.nf b/modules/local/reveal/add_meta_to_tab.nf
@@ -7,8 +7,8 @@ process ADD_META_TO_TAB {
         error "nf-core/airrflow currently does not support Conda. Please use a container profile instead."
     }
     container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
-        'docker.io/immcantation/airrflow:3.3.0':
-        'docker.io/immcantation/airrflow:3.3.0' }"
+        'docker.io/immcantation/airrflow:4.0.0':
+        'docker.io/immcantation/airrflow:4.0.0' }"
 
     cache 'deep' // Without 'deep' this process would run when using -resume