NARPS Open Pipelines

[bclenet]

Authors

Affiliations

Boris Clénet*¹, Élodie Germani*¹, Arshitha Basavaraj², Remi Gau³, Yaroslav Halchenko⁴, Paul Taylor⁵, Camille Maumet¹

1. Univ Rennes, Inria, CNRS, Inserm, France
2. Data Science and Sharing Team, NIMH, National Institutes of Health, Bethesda, MD, USA
3. Origami lab, McGill University, Montréal, Québec, Canada
4. Center for Open Neuroscience, Department of Psychological and Brain Sciences, Dartmouth College, NH, USA
5. Scientific and Statistical Computing Core, NIMH, National Institutes of Health, Bethesda, MD, USA

Contacts

Boris Clénet [email protected]
Élodie Germani [email protected]
Arshitha Basavaraj [email protected]
Remi Gau [email protected]
Yaroslav Halchenko [email protected]
Paul Taylor [email protected]
Camille Maumet [email protected]

Summary

Introduction

Different analytical choices can lead to variations in the results, a phenomenon that was illustrated in neuroimaging by the NARPS project (Botvinik-Nezer et al., 2020). In NARPS, 70 teans were tasks to analyze the same dataset to answer 9 yes/no research questions. Each team share their final results as well as a textual description (COBIDAS-compliant Nichols et al., 2017) of their analysis.

The goal of NARPS Open Pipelines is to create a codebase reproducing the 70 pipelines of the NARPS project and share this as an open resource for the community.

Results

The OHBM Brainhack 2023 gave the oppurtunity to:

1. make the repository more welcoming to new contributions:

• Proof-read and test the contribution process : everyone helped in finding and fixing inconsitencies in the documentation and in the processes related to contributing. PR#66, PR#65, PR#63, PR#64, PR#52, PR#50

• Create GitHub Actions workflows for enabling continuous integration, i.e.: testing existing pipelines everytime there are changes on them. PR#47

• Develop a new GitHub Actions workflow to detect typos in code comments and documentations at each commit. PR#48

2. learn new skills:

• Learn NiPype : joining the project was an opportunity to start using Nipype.

3. advance pipeline reproductions

• Better understand AFNI pipelines : thanks to Paul Taylor, the project will benefit from a deeper understanding of AFNI pipelines, with afni_proc examples of the AFNI team's pipeline in NARPS (Paul A Taylor et al., 2023 and associated repository).

• Start new pipeline reproductions. PR#62, Issue#61, PR#59, Issue#57, Issue#60, PR#55, Issue#51, Issue#49

In the end a total of:

• 6 pull requests were merged, 4 opened ;
• 4 issues were closed, 6 opened.

References (Bibtex)

@article{botvinik2020,
  author  = "Botvinik-Nezer, R. et al.",
  title   = "Variability in the analysis of a single neuroimaging dataset by many teams",
  journal = "Nature",
  year    = 2020
}

@article{taylor2023
  author  = "Paul A Taylor et al.",
  title   = "Highlight Results, Don't Hide Them: Enhance interpretation, reduce biases and improve reproducibility",
  journal = "NeuroImage",
  year    = 2023
}

@article{nichols2017best,
  title={Best practices in data analysis and sharing in neuroimaging using MRI},
  author={Nichols, Thomas E and Das, Samir and Eickhoff, Simon B and Evans, Alan C and Glatard, Tristan and Hanke, Michael and Kriegeskorte, Nikolaus and Milham, Michael P and Poldrack, Russell A and Poline, Jean-Baptiste and others},
  journal={Nature neuroscience},
  volume={20},
  number={3},
  pages={299--303},
  year={2017},
  publisher={Nature Publishing Group US New York}
}

[cmaumet]

Thanks @bclenet! Ping: @elodiegermani @Arshitha @Remi-Gau @yarikoptic @mrneont this is our submission for the OHBM Brainhack 2023 proceedings.

[anibalsolon]

Thank you @bclenet and all, such outstanding achievements!
We are considering going for more paragraph-like content than list, what do you think? To ease the process, I've rewritten the results content as a paragraph (line breaks just to ease the read, not new paragraphs). If you'd like to make changes, please feel free to copy the content to the main text in this issue.

During the OHBM Brainhack 2023, participants had the opportunity to improve the project in various ways.
Firstly, they focused on fostering a more inclusive environment for new contributors, refining the contribution process through proofreading and testing.
Several pull requests (PR #66, PR #65, PR #63, PR #64, PR #52, PR #50) were dedicated to this effort, rectifying inconsistencies in the documentation and contributing procedures.
Additionally, they implemented GitHub Actions workflows to enable continuous integration, ensuring the testing of pipelines with each alteration (PR #47), and crafted a new workflow to detect code comments and documentation typos (PR #48).
Skill acquisition was a significant aspect, with individuals diving into NiPype and advancing pipeline reproductions.
Thanks to Paul Taylor's contributions, a deeper understanding of AFNI pipelines emerged, particularly benefiting from afni_proc examples within the NARPS project (Taylor et al., 2023).
This endeavor resulted in the initiation of new pipeline reproductions (PR #62, Issue #61, PR #59, Issue #57, Issue #60, PR #55, Issue #51, Issue #49).
Ultimately, the Hackathon yielded substantial progress: 6 pull requests merged, 4 issues opened, 4 closed, and 6 opened, showcasing a fruitful collaboration and dedication to project enhancement.

[bclenet]

Hi @anibalsolon and thanks for the comment,
I'm fine with this new shape, as long as the links are still available and lead to the right place (e.g.: #47 should not point to this repo's issue number 47 but to narps_open_pipelines' one).

[anibalsolon]

Certainly, I'll make sure the links are correct. Thank you!

[anibalsolon]

hello all, could you please provide the authors information in the following link by May 15th?
https://docs.google.com/forms/d/e/1FAIpQLSckbC4F6KOtge1KOyzwj5yIbWR7tB8HrnqQ4KPZB7Mr3UcvMw/viewform