Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update quantms-utils=0.0.12 #429

Merged
merged 40 commits into from
Oct 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7e77679
Merge pull request #41 from ypriverol/dev
daichengxin Oct 7, 2024
607dd93
update quantms-utils=0.0.12
ypriverol Oct 14, 2024
8b4e30e
Merge pull request #42 from bigbio/dev
daichengxin Oct 14, 2024
50515a3
change schema to https
ypriverol Oct 18, 2024
de7c4ee
change schema to https
ypriverol Oct 18, 2024
9a3a724
linting changes
ypriverol Oct 18, 2024
52054b7
linting changes
ypriverol Oct 18, 2024
d3f0539
Template update for nf-core/tools version 3.0.2
ypriverol Oct 18, 2024
b30cdd6
update template 3.0.2
ypriverol Oct 18, 2024
f5cf008
update template 3.0.2
ypriverol Oct 18, 2024
66741ad
update template 3.0.2
ypriverol Oct 18, 2024
e12efa0
update template 3.0.2
ypriverol Oct 18, 2024
8c78801
update template 3.0.2
ypriverol Oct 18, 2024
bd90705
update template 3.0.2
ypriverol Oct 18, 2024
cb77002
update template 3.0.2
ypriverol Oct 18, 2024
75b2059
update template 3.0.2
ypriverol Oct 18, 2024
02c7617
update template 3.0.2
ypriverol Oct 18, 2024
e5fe179
update template 3.0.2 - versions issues
ypriverol Oct 18, 2024
2a01350
update template 3.0.2 - versions issues
ypriverol Oct 18, 2024
1dfd40a
update template 3.0.2 - versions issues
ypriverol Oct 18, 2024
7f288e9
update template 3.0.2 - versions issues
ypriverol Oct 19, 2024
6538c0e
update template 3.0.2 - versions issues
ypriverol Oct 19, 2024
cfebb68
update template 3.0.2 - versions issues
ypriverol Oct 19, 2024
f42e31d
update template 3.0.2 - versions issues
ypriverol Oct 19, 2024
b2875d2
update template 3.0.2 - versions issues
ypriverol Oct 19, 2024
6c7a086
update template 3.0.2 - versions issues
ypriverol Oct 19, 2024
5b7fc86
remove empty versions
ypriverol Oct 19, 2024
0bcf6d3
remove empty versions
ypriverol Oct 19, 2024
f7e2d17
remove empty versions
ypriverol Oct 19, 2024
b7236ac
remove empty versions
ypriverol Oct 19, 2024
06a229d
remove empty versions
ypriverol Oct 19, 2024
fc79168
remove empty versions
ypriverol Oct 19, 2024
ca4f16a
Merge pull request #44 from ypriverol/dev
daichengxin Oct 20, 2024
94e4c9a
fixed version
daichengxin Oct 20, 2024
1835360
ignore nextflow config
daichengxin Oct 20, 2024
63f1794
Update .nf-core.yml
daichengxin Oct 20, 2024
2274b60
Merge pull request #1 from daichengxin/dev
ypriverol Oct 20, 2024
4c3e8f0
remove empty versions
ypriverol Oct 20, 2024
aab1541
Update quantms.nf
daichengxin Oct 20, 2024
9d69475
Merge branch 'dev' into dev
daichengxin Oct 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ If you'd like to write some code for nf-core/quantms and bigbio/quantms, the sta
1. Check that there isn't already an issue about your idea in the [nf-core/quantms issues](https://github.com/nf-core/quantms/issues) and [bigbio/quantms_issues](https://github.com/bigbio/quantms/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [bigbio/quantms repository](https://github.com/bigbio/quantms) to your GitHub account
3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
4. Use `nf-core schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
4. Use `nf-core pipelines schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged

If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).
Expand All @@ -40,7 +40,7 @@ There are typically two types of tests that run:
### Lint tests

`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core pipelines lint <pipeline-directory>` command.

If any failures or warnings are encountered, please follow the listed URL for more documentation.

Expand Down Expand Up @@ -75,7 +75,7 @@ If you wish to contribute a new step, please use the following coding standards:
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new parameters to `nextflow.config` with a default (see below).
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core schema build` tool).
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core pipelines schema build` tool).
6. Add sanity checks and validation for all relevant parameters.
7. Perform local tests to validate that the new code works as expected.
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
Expand All @@ -86,11 +86,11 @@ If you wish to contribute a new step, please use the following coding standards:

Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.

Once there, use `nf-core schema build` to add to `nextflow_schema.json`.
Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`.

### Default processes resource requirements

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

Expand All @@ -103,7 +103,7 @@ Please use the following naming schemes, to make it easy to understand what is g

### Nextflow version bumping

If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`
If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core pipelines bump-version --nextflow . [min-nf-version]`

### Images and figures

Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/quan
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/quantms/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/quantms _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Make sure your code lints (`nf-core pipelines lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
Expand Down
31 changes: 22 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,12 @@ on:
pull_request:
release:
types: [published]
workflow_dispatch:

env:
NXF_ANSI_LOG: false
NXF_SINGULARITY_CACHEDIR: ${{ github.workspace }}/.singularity
NXF_SINGULARITY_LIBRARYDIR: ${{ github.workspace }}/.singularity

concurrency:
group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
Expand All @@ -32,36 +35,43 @@ jobs:
matrix:
# Nextflow versions
NXF_VER:
- "23.04.0"
- "24.04.2"
- "latest-everything"
test_profile:
["test_lfq", "test_lfq_sage", "test_dia", "test_localize", "test_tmt", "test_dda_id", "test_tmt_corr"]
exec_profile: ["docker"]
exclude:
- test_profile: test_dia
exec_profile: conda
exec_profile: "conda"
- test_profile: test_localize
exec_profile: conda
exec_profile: "conda"
- NXF_VER: "latest-everything"
exec_profile: "conda"
include:
- test_profile: test_latest_dia
exec_profile: singularity
exec_profile: "singularity"
- test_profile: test_lfq
exec_profile: conda
exec_profile: "conda"
- test_profile: test_dda_id
exec_profile: conda
exec_profile: "conda"
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4

- name: Install Nextflow
- name: Set up Nextflow
uses: nf-core/setup-nextflow@v2
with:
version: "${{ matrix.NXF_VER }}"

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
- name: Set up Apptainer
if: matrix.exec_profile == 'singularity'
uses: eWaterCycle/setup-apptainer@main

- name: Set up Singularity
if: matrix.exec_profile == 'singularity'
run: |
mkdir -p $NXF_SINGULARITY_CACHEDIR
mkdir -p $NXF_SINGULARITY_LIBRARYDIR

- name: Install micromamba
env:
Expand All @@ -78,6 +88,9 @@ jobs:
if: matrix.exec_profile == 'singularity'
uses: singularityhub/install-singularity@main

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- name: Run pipeline with test data in docker/singularity profile
if: matrix.exec_profile == 'docker' || matrix.exec_profile == 'singularity'
# TODO nf-core: You can customise CI pipeline run tests as required
Expand Down
53 changes: 43 additions & 10 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Test successful pipeline download with 'nf-core download'
name: Test successful pipeline download with 'nf-core pipelines download'

# Run the workflow when:
# - dispatched manually
Expand All @@ -8,7 +8,7 @@ on:
workflow_dispatch:
inputs:
testbranch:
description: "The specific branch you wish to utilize for the test execution of nf-core download."
description: "The specific branch you wish to utilize for the test execution of nf-core pipelines download."
required: true
default: "dev"
pull_request:
Expand Down Expand Up @@ -39,9 +39,11 @@ jobs:
with:
python-version: "3.12"
architecture: "x64"
- uses: eWaterCycle/setup-singularity@931d4e31109e875b13309ae1d07c70ca8fbc8537 # v7

- name: Setup Apptainer
uses: eWaterCycle/setup-apptainer@4bb22c52d4f63406c49e94c804632975787312b3 # v2.0.0
with:
singularity-version: 3.8.3
apptainer-version: 1.3.4

- name: Install dependencies
run: |
Expand All @@ -54,33 +56,64 @@ jobs:
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> ${GITHUB_ENV}
echo "REPO_BRANCH=${{ github.event.inputs.testbranch || 'dev' }}" >> ${GITHUB_ENV}
- name: Make a cache directory for the container images
run: |
mkdir -p ./singularity_container_images
- name: Download the pipeline
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
run: |
nf-core download ${{ env.REPO_LOWERCASE }} \
nf-core pipelines download ${{ env.REPO_LOWERCASE }} \
--revision ${{ env.REPO_BRANCH }} \
--outdir ./${{ env.REPOTITLE_LOWERCASE }} \
--compress "none" \
--container-system 'singularity' \
--container-library "quay.io" -l "docker.io" -l "ghcr.io" \
--container-library "quay.io" -l "docker.io" -l "community.wave.seqera.io" \
--container-cache-utilisation 'amend' \
--download-configuration
--download-configuration 'yes'
- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}

- name: Count the downloaded number of container images
id: count_initial
run: |
image_count=$(ls -1 ./singularity_container_images | wc -l | xargs)
echo "Initial container image count: $image_count"
echo "IMAGE_COUNT_INITIAL=$image_count" >> ${GITHUB_ENV}
- name: Run the downloaded pipeline (stub)
id: stub_run_pipeline
continue-on-error: true
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
- name: Run the downloaded pipeline (stub run not supported)
id: run_pipeline
if: ${{ job.steps.stub_run_pipeline.status == failure() }}
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -profile test,singularity --outdir ./results

- name: Count the downloaded number of container images
id: count_afterwards
run: |
image_count=$(ls -1 ./singularity_container_images | wc -l | xargs)
echo "Post-pipeline run container image count: $image_count"
echo "IMAGE_COUNT_AFTER=$image_count" >> ${GITHUB_ENV}
- name: Compare container image counts
run: |
if [ "${{ env.IMAGE_COUNT_INITIAL }}" -ne "${{ env.IMAGE_COUNT_AFTER }}" ]; then
initial_count=${{ env.IMAGE_COUNT_INITIAL }}
final_count=${{ env.IMAGE_COUNT_AFTER }}
difference=$((final_count - initial_count))
echo "$difference additional container images were \n downloaded at runtime . The pipeline has no support for offline runs!"
tree ./singularity_container_images
exit 1
else
echo "The pipeline can be downloaded successfully!"
fi
23 changes: 19 additions & 4 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: nf-core linting
# This workflow is triggered on pushes and PRs to the repository.
# It runs the `nf-core lint` and markdown lint tests to ensure
# It runs the `nf-core pipelines lint` and markdown lint tests to ensure
# that the code meets the nf-core guidelines.
on:
push:
Expand Down Expand Up @@ -41,17 +41,32 @@ jobs:
python-version: "3.12"
architecture: "x64"

- name: read .nf-core.yml
uses: pietrobolcato/[email protected]
id: read_yml
with:
config: ${{ github.workspace }}/.nf-core.yml

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install nf-core
pip install nf-core==${{ steps.read_yml.outputs['nf_core_version'] }}
- name: Run nf-core pipelines lint
if: ${{ github.base_ref != 'master' }}
env:
GITHUB_COMMENTS_URL: ${{ github.event.pull_request.comments_url }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_PR_COMMIT: ${{ github.event.pull_request.head.sha }}
run: nf-core -l lint_log.txt pipelines lint --dir ${GITHUB_WORKSPACE} --markdown lint_results.md

- name: Run nf-core lint
- name: Run nf-core pipelines lint --release
if: ${{ github.base_ref == 'master' }}
env:
GITHUB_COMMENTS_URL: ${{ github.event.pull_request.comments_url }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_PR_COMMIT: ${{ github.event.pull_request.head.sha }}
run: nf-core -l lint_log.txt lint --dir ${GITHUB_WORKSPACE} --markdown lint_results.md
run: nf-core -l lint_log.txt pipelines lint --release --dir ${GITHUB_WORKSPACE} --markdown lint_results.md

- name: Save PR number
if: ${{ always() }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/linting_comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download lint results
uses: dawidd6/action-download-artifact@09f2f74827fd3a8607589e5ad7f9398816f540fe # v3
uses: dawidd6/action-download-artifact@bf251b5aa9c2f7eeb574a96ee720e24f801b7c11 # v6
with:
workflow: linting.yml
workflow_conclusion: completed
Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/template_version_comment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: nf-core template version comment
# This workflow is triggered on PRs to check if the pipeline template version matches the latest nf-core version.
# It posts a comment to the PR, even if it comes from a fork.

on: pull_request_target

jobs:
template_version:
runs-on: ubuntu-latest
steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Read template version from .nf-core.yml
uses: nichmor/[email protected]
id: read_yml
with:
config: ${{ github.workspace }}/.nf-core.yml

- name: Install nf-core
run: |
python -m pip install --upgrade pip
pip install nf-core==${{ steps.read_yml.outputs['nf_core_version'] }}
- name: Check nf-core outdated
id: nf_core_outdated
run: echo "OUTPUT=$(pip list --outdated | grep nf-core)" >> ${GITHUB_ENV}

- name: Post nf-core template version comment
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
if: |
contains(env.OUTPUT, 'nf-core')
with:
repo-token: ${{ secrets.NF_CORE_BOT_AUTH_TOKEN }}
allow-repeats: false
message: |
> [!WARNING]
> Newer version of the nf-core template is available.
>
> Your pipeline is using an old version of the nf-core template: ${{ steps.read_yml.outputs['nf_core_version'] }}.
> Please update your pipeline to the latest version.
>
> For more documentation on how to update your pipeline, please see the [nf-core documentation](https://github.com/nf-core/tools?tab=readme-ov-file#sync-a-pipeline-with-the-template) and [Synchronisation documentation](https://nf-co.re/docs/contributing/sync).
#
14 changes: 1 addition & 13 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,4 @@ results/
testing/
testing*
*.pyc
.idea/
.idea/*
*.log
/build/
results*/
venv/
node_modules
conversion_inputs
debug_dir
test_out

lint_log.txt
node_modules
null/
7 changes: 2 additions & 5 deletions .gitpod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,14 @@ tasks:
command: |
pre-commit install --install-hooks
nextflow self-update
- name: unset JAVA_TOOL_OPTIONS
command: |
unset JAVA_TOOL_OPTIONS
vscode:
extensions: # based on nf-core.nf-core-extensionpack
- esbenp.prettier-vscode # Markdown/CommonMark linting and style checking for Visual Studio Code
#- esbenp.prettier-vscode # Markdown/CommonMark linting and style checking for Visual Studio Code
- EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files
- Gruntfuggly.todo-tree # Display TODO and FIXME in a tree view in the activity bar
- mechatroner.rainbow-csv # Highlight columns in csv files in different colors
# - nextflow.nextflow # Nextflow syntax highlighting
- nextflow.nextflow # Nextflow syntax highlighting
- oderwat.indent-rainbow # Highlight indentation level
- streetsidesoftware.code-spell-checker # Spelling checker for source code
- charliermarsh.ruff # Code linter Ruff
11 changes: 8 additions & 3 deletions .nf-core.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
repository_type: pipeline
nf_core_version: "2.14.1"
bump_version: null
lint:
files_exist:
- conf/igenomes.config
- conf/test_full.config
- conf/test.config
- .github/workflows/awstest.yml
- .github/workflows/awsfulltest.yml
- .github/workflows/ci.yml
- .gitignore
files_unchanged:
- .github/PULL_REQUEST_TEMPLATE.md
- .github/CONTRIBUTING.md
- docs/README.md
multiqc_config: False
nextflow_config: false
multiqc_config: false
nf_core_version: 3.0.2
org_path: null
repository_type: pipeline
Loading
Loading