Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor ext config as process inputs #1

Closed
wants to merge 1 commit into from
Closed

Refactor ext config as process inputs #1

wants to merge 1 commit into from

Conversation

bentsherman
Copy link
Owner

This PR demonstrates how to remove the module configs by refactoring ext config settings into process inputs.

  • the module config is only used to define process config, specifically ext and publishDir

  • this process config could be in the process definition, but isn't because it's usually pipeline-specific rather than inherent to the generic module. it is also useful to overwrite settings e.g. for different subworkflows that call the same process

  • the upcoming workflow publish definition (Workflow output definition nextflow-io/nextflow#4784) replaces publishDir with a new publish definition in the Nextflow DSL, so now the only purpose of the module config is to provide ext config to processes

  • ext config is essentially a shortcut for pipeline parameters, while it is convenient, it makes the pipeline harder to understand overall as the "inputs" to a process are split between pipeline code, params, and config

  • this PR shows how these ext config settings can be refactored as process inputs. configurability can be preserved by providing the same ext settings as pipeline params. it makes the params longer, but it is a more transparent and comprehensive view of what the pipeline "inputs" are

  • it is in line with the push to use params only in the top-level workflow and pass params as inputs to subworkflows and processes, in order to increase the modularity of the pipeline

@github-actions GitHub Actions
Copy link

nf-core lint overall result: Failed ❌

Posted for pipeline commit 4252806

+| ✅ 166 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   7 tests had warnings |!
-| ❌   4 tests failed       |-

❌ Test failures:

  • files_unchanged - .github/CONTRIBUTING.md does not match the template
  • files_unchanged - .github/PULL_REQUEST_TEMPLATE.md does not match the template
  • files_unchanged - .github/workflows/linting.yml does not match the template
  • schema_params - Param gffread_args from nextflow config not found in nextflow_schema.json

❗ Test warnings:

  • files_exist - File not found: assets/multiqc_config.yml
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • nextflow_config - Config default ignored: params.ribo_database_manifest
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/rnaseq/rnaseq/.github/workflows/awstest.yml
  • multiqc_config - 'assets/multiqc_config.yml' not found

✅ Tests passed:

Run details

  • nf-core/tools version 2.13.1
  • Run at 2024-04-23 04:05:41

@bentsherman
Copy link
Owner Author

Closing, follow nf-core/fetchngs#312 for latest developments

@bentsherman bentsherman deleted the refactor-ext-config branch September 24, 2024 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@bentsherman