Welcome to the nf-neuro project ! A Nextflow modules and workflows repository for neuroimaging maintained by the SCIL team. The primary focus of the library is to provide pre-built processes and processing sequences for neuroimaging, optimized for Nextflow DSL2, based on open-source technologies and made easily available to pipeline's developers through the nf-core framework.

Let's say you develop a pipeline for neuroimaging. You want to make it the more efficient, reliable, reproducible and also be able to evaluate it and control the quality of its outputs. That's what nf-neuro provides to you, all in one repository, hosting all dependencies you need to start developing and analyzing.

The only thing we ask of you is to develop in Nextflow DSL2. We use principle and standards closely aligned with nf-core, but we'll make you adapt to them slowly as you go (we still haven't finished complying to all of them ourselves). Using nf-neuro helps accelerate development in neuroimaging and produces better research outcomes for all !

• Using modules from nf-neuro
• Developing in nf-neuro
  • Manual configuration
    • Dependencies
    • Python environment
    • Loading the project's environment
    • Global environment
    • Working with VS Code
  • Configuration via the devcontainer
• Contributing to the nf-neuro project
  • Adding a new module to nf-neuro
    • Generate the template
    • Edit the template
      • Edit main.nf
      • Edit environment.yml
      • Edit meta.yml
    • Create test cases
      • Edit tests/main.nf.test
      • Edit tests/nextflow.config
    • Generate tests snapshots
    • Request for more test resources
    • Lint your code
    • Submit your PR
  • Defining optional input parameters
  • Test data infrastructure
  • Adding a new subworkflow to nf-neuro
    • Generate the template
    • Edit the template
      • Edit main.nf
        • Define your subworkflow inputs
        • Fill the main: section
        • Define your Workflow outputs
      • Edit meta.yml
      • Create test cases
    • Lint your code
    • Submit your PR
• Running tests
• Configuring Docker for easy usage
• Installing Prettier and editorconfig

To import modules from nf-neuro, you first need to install nf-core on your system (can be done simply using pip install nf-core). Once done, nf-neuro modules are imported using this command :

nf-core modules --git-remote https://github.com/scilus/nf-neuro.git install <category>/<tool>

where you input the <tool> you want to import from the desired <category>. To get a list of the available modules, run :

nf-core modules --git-remote https://github.com/scilus/nf-neuro.git list remote

The same can be done for subworkflows, replacing modules in the nf-core command by `subworkflows :

nf-core subworkflows --git-remote https://github.com/scilus/nf-neuro.git install <category>/<subworkflow>

It can become heavy to always prepend the commands with --git-remote, more so if you need to specify a --branch where to fetch the information. You can instead define the git-remote and branch using Environment Variables :

export NFCORE_MODULES_GIT_REMOTE=https://github.com/scilus/nf-neuro.git
export NFCORE_MODULES_GIT_BRANCH=main
export NFCORE_SUBWORKFLOWS_GIT_REMOTE=https://github.com/scilus/nf-neuro.git
export NFCORE_SUBWORKFLOWS_GIT_BRANCH=main

and call all commands without specifying the --git-remote and --branch options, while still targeting the nf-neuro repository.

The nf-neuro project requires some specific tools to be installed on your system so that the development environment runs correctly. You can install them manually, but if you desire to streamline the process and start coding faster, we highly recommend using the VS Code development container to get fully configured in a matter of minutes.

• Python ≥ 3.8, < 3.13
• Docker ≥ 24 (we recommend using Docker Desktop)
• Java Runtime ≥ 11, ≤ 17
  • On Ubuntu, install openjdk-jre-<version> packages
• Nextflow ≥ 23.04.0
• nf-test ≥ 0.9.0
• Node ≥ 14, Prettier and editorconfig (see below)

The project uses poetry to manage python dependencies. To install it using pipx, run the following commands :

pip install pipx
pipx ensurepath
pipx install poetry==1.8.*

Once done, install the project with :

poetry install

The project scripts and dependencies can be accessed using :

poetry shell

which will activate the project's python environment in the current shell.

To exit the environment, simply enter the exit command in the shell.

Set the following environment variables in your .bashrc (or whatever is the equivalent for your shell) :

export NFCORE_MODULES_GIT_REMOTE=https://github.com/scilus/nf-neuro.git
export NFCORE_MODULES_GIT_BRANCH=main
export NFCORE_SUBWORKFLOWS_GIT_REMOTE=https://github.com/scilus/nf-neuro.git
export NFCORE_SUBWORKFLOWS_GIT_BRANCH=main

This will make it so the nf-core commands target the right repository by default. Else, you'll need to add --git-remote and --branch options to pretty much all commands relating to modules and subworkflows.

The nf-neuro project curates a bundle of useful extensions for Visual Studio Code, the nf-neuro-extensions package. You can find it easily on the extension marketplace.

The devcontainer definition for the project contains all required dependencies and setup steps are automatically executed. To use this installation method, you need to have Docker (refer to this section for configuration requirements or validate your configuration) and Visual Studio Code installed on your system.

Open the cloned repository in VS Code and click on the arrow box in the lower left corner, to get a prompt to Reopen in container. The procedure will start a docker build, wait for a few minutes and enjoy your fully configured development environment.

• Available in the container :

  • nf-neuro, nf-core all accessible through the terminal, which is configured to load the poetry environment in shells automatically
  • nf-neuro configured as the main repository for all nf-core commands, using NFCORE_* environment variables
  • git, github-cli
  • curl, wget, apt-get
  • nextflow, nf-test, docker, tmux

• Available in the VS Code IDE through extensions :

  • Docker images and containers management
  • Nextflow execution environemnt
  • Python and C++ linting, building and debugging tools
  • Github Pull Requests management
  • Github flavored markdown previewing
  • Test Data introspection
  • Resource monitoring
  • Remote development
  • Live sharing with collaborators

If you want to propose a new module to the repository, follow the guidelines in the module creation documentation. The same goes for subworkflows, using these guidelines instead. We follow standards closely aligned with nf-core, with some exceptions on process atomicity and how test data is handled. Modules that don't abide to them won't be accepted and PR containing them will be closed automatically.

Tests are run through nf-core, using the command :

nf-core modules test <category/tool>

The tool can be omitted to run tests for all modules in a category.

The simplest way of installing everything Docker is to use Docker Desktop. You can also go the engine way and install Docker manually.

Once installed, you need to configure some access rights to the Docker daemon. The easiest way to do this is to add your user to the docker group. This can be done with the following command :

sudo groupadd docker
sudo usermod -aG docker $USER

After running this command, you need to log out and log back in to apply the changes.

To install Prettier and editorconfig for the project, you need to have node and npm installed on your system to at least version 14. On Ubuntu, you can do it using snap :

sudo snap install node --classic

However, if you cannot install snap, or have another OS, refer to the official documentation for the installation procedure.

Under the current configuration for the Development Container, for this project, we use the following procedure, considering ${NODE_MAJOR} is at least 14 for Prettier :

curl -fsSL https://deb.nodesource.com/setup_${NODE_MAJOR}.x | bash - &&\
apt-get install -y nodejs

npm install --save-dev --save-exact prettier
npm install --save-dev --save-exact editorconfig-checker

echo "function prettier() { npm exec prettier $@; }" >> ~/.bashrc
echo "function editorconfig-checker() { npm exec editorconfig-checker $@; }" >> ~/.bashrc