Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorporate DSL2 features into docs #3793

Merged
merged 13 commits into from
Oct 15, 2023
Merged
192 changes: 192 additions & 0 deletions docs/dsl1.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
.. _dsl1-page:

********************
Migrating from DSL 1
********************

In Nextflow version ``22.03.0-edge``, DSL2 became the default DSL version. In version ``22.12.0-edge``,
DSL1 support was removed, and this documentation was updated to use DSL2 by default. Users who are still
bentsherman marked this conversation as resolved.
Show resolved Hide resolved
using DSL1 should migrate their pipelines to DSL2 in order to use the latest versions of Nextflow. This page
describes the differences between DSL1 and DSL2, and how to migrate to DSL2.

In Nextflow versions prior to ``22.03.0-edge``, you must enable DSL2 explicitly in order to use it. You can either
set the feature flag in your pipeline script::

nextflow.enable.dsl=2

Or set the environment variable where you launch Nextflow::

export NXF_DEFAULT_DSL=2


Processes and workflows
=======================

In DSL1, a process definition is also the process invocation. Process inputs and outputs are connected to channels
using ``from`` and ``into``. Here is the :ref:`getstarted-first` example written in DSL1::

nextflow.enable.dsl=1

params.str = 'Hello world!'

process splitLetters {

output:
file 'chunk_*' into letters

"""
printf '${params.str}' | split -b 6 - chunk_
"""
}

process convertToUpper {

input:
file x from letters.flatten()

output:
stdout result

"""
cat $x | tr '[a-z]' '[A-Z]'
"""
}

result.view { it.trim() }

To migrate this code to DSL2, you need to move all of your channel logic throughout the script
into a (nameless) ``workflow`` definition. Additionally, you must call each process explicitly,
bentsherman marked this conversation as resolved.
Show resolved Hide resolved
passing any input channels as arguments (instead of ``from ...``) and receiving any output channels
as return values (instead of ``into ...``).

Refer to the :ref:`workflow-page` page to learn how to define a workflow, as well as the original
:ref:`getstarted-first` example to see the resulting DSL2 version.
bentsherman marked this conversation as resolved.
Show resolved Hide resolved


Channel forking
===============

In DSL1, a channel can be used as an input only once; to use a channel multiple times, the channel must
be forked using the ``into`` operator.

In DSL2, channels are automatically forked when connecting two or more consumers.

For example::
bentsherman marked this conversation as resolved.
Show resolved Hide resolved

channel
.from('Hello','Hola','Ciao')
.set{ cheers }

cheers
.map{ it.toUpperCase() }
.view()

cheers
.map{ it.reverse() }
.view()

Similarly, process outputs can be consumed by multiple consumers automatically, which makes workflow scripts
much easier to read and write.


Modules
=======

In DSL1, the entire Nextflow pipeline must be defined in a single file (e.g. ``main.nf``). This
restriction becomes quite cumbersome as a pipeline becomes larger, and it hinders the sharing and
reuse of pipeline components.

DSL2 introduces the concept of "module scripts" (or "modules" for short), which are Nextflow scripts
that can be "included" by other scripts. While modules are not essential to migrating to DSL2, nor are
they mandatory in DSL2 by any means, modules can help you organize a large pipeline into multiple smaller
files, and take advantage of modules created by others. Check out the :ref:`module-page` to get started.


Deprecations
============

Processes
---------

* The ``set`` process input type is no longer supported, use :ref:`tuple <process-input-tuple>` instead.
* The ``set`` process output type is no longer supported, use :ref:`tuple <process-out-tuple>` instead.
* The ``mode flatten`` option for process outputs is no longer available. Use the :ref:`operator-flatten` operator on the corresponding output channel instead.

* Unqualified value and file elements in a tuple declaration are no longer allowed. Use an explicit
``val`` or ``path`` qualifier.

For example::

process foo {
input:
tuple X, 'some-file.sam'
output:
tuple X, 'some-file.bam'

script:
'''
bentsherman marked this conversation as resolved.
Show resolved Hide resolved
your_command --in $X some-file.sam > some-file.bam
'''
}

Use::

process foo {
input:
tuple val(X), path('some-file.sam')
output:
tuple val(X), path('some-file.bam')

script:
'''
your_command --in $X some-file.sam > some-file.bam
'''
}


Channels
--------

* Channel method ``bind`` has been deprecated in DSL2.
* Channel method ``<<`` has been deprecated in DSL2.
* Channel factory ``create`` has been deprecated in DSL2.

Operators
---------

* Operator ``choice`` has been deprecated in DSL2. Use :ref:`operator-branch` instead.
* Operator ``close`` has been deprecated in DSL2.
* Operator ``countBy`` has been deprecated in DSL2.
* Operator ``into`` has been deprecated in DSL2, as it is no longer needed.
* Operator ``fork`` has been renamed to :ref:`operator-multimap`.
* Operator ``groupBy`` has been deprecated in DSL2. Use :ref:`operator-grouptuple` instead.
* Operators ``print`` and ``println`` have been deprecated in DSL2. Use :ref:`operator-view` instead.
* Operator ``route`` has been deprecated in DSL2.
* Operator ``separate`` has been deprecated in DSL2.
* Operator ``spread`` has been deprecated in DSL2. Use :ref:`operator-combine` instead.

DSL2 Preview
------------

* The ``nextflow.preview.dsl=2`` feature flag is no longer needed.
bentsherman marked this conversation as resolved.
Show resolved Hide resolved
* Anonymous and unwrapped includes are no longer supported. Use an explicit module inclusion instead.

For example::

include './some/library'
include bar from './other/library'

workflow {
foo()
bar()
}

Should be replaced with::

include { foo } from './some/library'
include { bar } from './other/library'

workflow {
foo()
bar()
}
Loading