Add camera new tools #277

DamienCode404 · 2024-10-11T13:57:59Z

Addition of New CAMERA Tools for the Metabolomics Suite

Description:

This PR introduces several new tools to the CAMERA tool suite, used for metabolomics analysis with LC-MS data. These tools complement existing functionalities and enhance peak detection and annotation.

New Tools Added:

camera_groupFWHM:
- Groups peaks within a defined retention time window using Full Width at Half Maximum (FWHM).
camera_groupCorr:
- Groups peaks based on retention time and intensity correlations across samples.
camera_findIsotopes:
- Detects isotope patterns in LC-MS peak lists based on mass differences and expected isotope ratios.
- Provides isotope annotation for downstream metabolomics analysis.
camera_findAdducts:
- Identifies potential adducts in mass spectrometry data by detecting characteristic mass shifts between peaks.
- Facilitates the identification of molecular ions and their adducts for more accurate metabolite annotation.

Why These Changes?

These new tools provide greater flexibility and modularity for analyzing LC-MS data by breaking down the all-in-one annotateDiffreport tool into four distinct tools. This separation allows users to run specific tasks such as isotope detection, adduct identification, peak grouping by FWHM, or correlation independently. By decoupling these functionalities, users can better customize their workflows based on their needs, making the analysis more efficient and tailored to specific research objectives.

FOR CONTRIBUTOR:

- I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
- License permits unrestricted use (educational + commercial)
- This PR adds a new tool or tool collection
- This PR updates an existing tool or tool collection
- This PR does something else (explain below)

bgruening

Can you maybe reformat your tools using 4 spaces.

Please also consider using the https://github.com/galaxyproject/galaxy-language-server it can reformat all your tools automatically and will do the lining for you as well :)

tools/camera/CAMERA_findAdducts.R

tools/camera/macros.xml

tools/camera/groupCorr.xml

tools/camera/groupFWHM.xml

tools/camera/findAdducts.xml

…rs (&& and ||)

hechth · 2024-10-21T12:52:07Z

@DamienCode404 @bgruening @yguitton the way how the tool reads input data currently is wrong/broken. Through the image argument, hard-coded paths from the R session get loaded into the Galaxy tool ... this doesn't work.

Which data are you using? The output from XCMS fillpeaks or CAMERA annotate? Maybe we can arrange a call to talk about these things and figure them out, then I can help with the implementation.

hechth · 2024-10-21T13:09:21Z

Also, more in general, what is the purpose of these new tools? I can see that the existing CAMERA tool also seems to include all of these steps - do you want to split them or whats the plan here?

DamienCode404 · 2024-10-21T13:38:00Z

@DamienCode404 @bgruening @yguitton the way how the tool reads input data currently is wrong/broken. Through the image argument, hard-coded paths from the R session get loaded into the Galaxy tool ... this doesn't work.

Which data are you using? The output from XCMS fillpeaks or CAMERA annotate? Maybe we can arrange a call to talk about these things and figure them out, then I can help with the implementation.

Hi @hechth, I'm curious as to why this isn't working. I'm still a beginner with galaxy tools.

For the groupFWHM camera tool, we use xcms fillpeaks files as input. For the rest of the tools, we only use camera rdata output.

DamienCode404 · 2024-10-21T13:45:48Z

Also, more in general, what is the purpose of these new tools? I can see that the existing CAMERA tool also seems to include all of these steps - do you want to split them or whats the plan here?

The main aim of this tool is, as you said, to split the annotateDiffreport tool into 4 sub-tools. This is to provide users with more options when launching these tools and to skip certain steps if necessary. Also, this method seems to give more consistent results. We expect execution time to decrease as well.

hechth · 2024-10-22T06:45:34Z

You should use Rds files storing only a single variable if already using builin R datatypes. This is a serious security vulnerability. If you load an RData file, it might overwrite anything internal. Someone can store an environment where print downloads some malware and runs it. If you load a Rds file, at least you avoid that other variables might be corrupted.

jsaintvanne · 2024-10-22T07:05:48Z

Hi Helge !

This choice has been done for all XCMS workflow in W4M since the beginning I think... ! With this we should rework all this workflow that actually works with RData containing multiple variables...

Maybe we can ask @lecorguille about it ? Cause here, we just continue the workflow of XCMS in CAMERA, didn't touch the variables saved in RData files.

hechth · 2024-10-22T07:19:53Z

@jsaintvanne Yeah I just saw - this is probably something that would make sense to address - maybe also with the update to XCMS 4?

hechth · 2024-10-22T09:01:29Z

Currently planemo test fails with the following error message.

Error in retrieveRawfileInTheWorkingDir(singlefile, zipfile) : 
  Cannot access the sample: ko15.CDF located: /home/laberca/galaxy/database/objects/a/0/c/dataset_a0c47c86-be0f-4097-bd3e-68b6f5e9f04b.dat . Please, contact your administrator ... if you have one!
Execution halted
.

jsaintvanne · 2024-10-22T12:02:56Z

@jsaintvanne Yeah I just saw - this is probably something that would make sense to address - maybe also with the update to XCMS 4?

Yeah we were asking how we will go to XCMS 4, maybe that's a way... !

Currently planemo test fails with the following error message.

Error in retrieveRawfileInTheWorkingDir(singlefile, zipfile) : 
  Cannot access the sample: ko15.CDF located: /home/laberca/galaxy/database/objects/a/0/c/dataset_a0c47c86-be0f-4097-bd3e-68b6f5e9f04b.dat . Please, contact your administrator ... if you have one!
Execution halted
.

Think this come from the data test where there is the singlefile variable to keep the link between the cdf filename and their galaxy name and that has been done in local that's why we have this path hardcoded... @DamienCode404 is working on it !

We should maybe discuss about the RData and RDs files and their security cause we can't really see the problem here sorry !

…d the correct number of sample columns in tsv output.

Failed to expand inclusions [{'source': 'camera_groupfwhm.xml'}, {'source': 'camera_groupfwhm.r'}]

WARNING: Failed to expand inclusions [{'source': 'camera_groupfwhm.xml'}, {'source': 'camera_groupfwhm.r'}]

Failed Tests RData : Binary data detected, not displaying diff

- Removal of duplicate functions from scripts and lib.r files - Retrieve arguments with the W4MRUtils::parse_args function

bgruening

Can you please include your Rscript that you use with "required_files": https://docs.galaxyproject.org/en/latest/dev/schema.html#tool-required-files

bgruening · 2024-10-26T15:04:58Z

tools/camera/findAdducts.xml

+
+    <expand macro="requirements"/>
+
+    <command detect_errors="exit_code"><![CDATA[


The indentation seems to be off here and makes it hard to read.

I recommend to use the https://github.com/galaxyproject/galaxy-language-server it has an auto-format feature.

For me, the indentation seems correct. I validate the lint steps. I've already indented and formatted the code. Is it just the 4 extra spaces that bother you @bgruening ? In all the other tools I've seen, they all have this style of indentation between <tool></tool> tags. Maybe I've misunderstood.

…ion delta. - Try to correct conditions (findAdducts) - Gives the right delta to compare test file sizes (groupFWHM).

nSlaves issue with findAdducts. Use of 'xcmsClusterApply' is deprecated! Use 'BPPARAM' arguments instead. Need update of the package CAMERA to parallelize. nSlaves is set by default to 1.

DamienCode404 · 2024-11-06T15:08:11Z

Hello everyone,
We're making progress with the development of the CAMERA tool suite, but I'm getting stuck with a display error.
In my “help” section for each tool we'd like to add a global workflow diagram. Exemple here :

------------------------------------------
General schema of the metabolomic workflow
------------------------------------------

.. image:: groupFWHM.png

This code should display my image in Galaxy, but I have this result :

I think it's maybe a problem with the fact that i m developing in a local environment, or because my png files are too big (~55ko).
Let me know if you already solved this problem. Ty !

Add camera new tools

811f290

DamienCode404 requested a review from lecorguille as a code owner October 11, 2024 13:58

DamienCode404 added 2 commits October 11, 2024 16:24

Update macros file for linting

21403d7

Delete all symbolic links

9331cf5

bgruening reviewed Oct 11, 2024

View reviewed changes

DamienCode404 added 7 commits October 18, 2024 11:16

Changed indentation to 4 spaces and added profile=“23.0”

8595982

make descriptions shorter

976109a

Removal of files that are too large and unnecessary for testing purposes

75ef0a4

Remove unnecessary sections

471bd82

Remove <expand macro="stdio"/>

9ebb51a

Code indentation for R files

52883bb

Solved Error : Conditional expressions require scalar logical operato…

67ef11c

…rs (&& and ||)

DamienCode404 added 9 commits October 22, 2024 17:23

Delete outdated test files + try to fix singlefile problem

c890131

Add "singlefile_galaxyPath" and "singlefile_sampleName" arguments

ac642dd

added singlefile checks

572667c

Corrected arguments for retrieving galaxyPath and sampleName + Remove…

da3a0af

…d the correct number of sample columns in tsv output.

Add test-data + minor bug fix

5c202f0

Fix version for CAMERA_groupFWHM tool.

2007bc1

Failed to expand inclusions [{'source': 'camera_groupfwhm.xml'}, {'source': 'camera_groupfwhm.r'}]

Update .shed.yml

7aed914

WARNING: Failed to expand inclusions [{'source': 'camera_groupfwhm.xml'}, {'source': 'camera_groupfwhm.r'}]

Add test-data file for findIsotopes tool

f8e1a62

Change test method with file size Rdata + expect_num_outputs

c9de9b2

Failed Tests RData : Binary data detected, not displaying diff

DamienCode404 added 5 commits October 24, 2024 17:12

Delete @HELP_AUTHORS@ in help section

d97b434

Cleaning up the code

374edcb

- Removal of duplicate functions from scripts and lib.r files - Retrieve arguments with the W4MRUtils::parse_args function

Add source_local function to use lib.r correctly

d078329

Checks added in case of singlefile NULL

d83c1f0

Concatenate print into a single string

9d0ff32

bgruening reviewed Oct 26, 2024

View reviewed changes

DamienCode404 added 19 commits October 28, 2024 10:27

Add "required_files"

9b01961

Corrects findAdducts arguments and groupFWHM test file size verificat…

c250ae6

…ion delta. - Try to correct conditions (findAdducts) - Gives the right delta to compare test file sizes (groupFWHM).

Convert arguments with “NULL” string values into real NULL values.

fabbcbc

Add FindAdducts test files

4f0faef

Change test files to less than 1M in size

9646d78

Change test files to less than 1M in size v2

92172b2

Change test files to less than 1M in size v3

b921ff8

Try to solve the variableMetadata column problem

dd106f1

Try to solve the variableMetadata column problem v2

04a9bdd

Minor bug fix

79018cc

feat: xsAnnotate new parameters

96d4228

fix: phenoData

b16ecf1

fix: psg_list (findAdducts) and sample (xsAnnotate)

43e7c4a

nSlaves issue with findAdducts. Use of 'xcmsClusterApply' is deprecated! Use 'BPPARAM' arguments instead. Need update of the package CAMERA to parallelize. nSlaves is set by default to 1.

fix: psg_list (findAdducts) and sample (xsAnnotate) v2

cae897d

Add print to view arguments after xsAnnotate step

6bb1b0a

fix lintr and change test data files

245124e

Change test files for findAdducts

5db6857

Add psg_list option to groupCorr

bc9d3de

Add "Changelog/News" section to CAMERA tools

4e17f51

DamienCode404 added 2 commits November 8, 2024 13:52

Solves problem of creating variableMetadata with 1 sample

38cc94e

No need to perform a group step for a single file

c89c2dd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add camera new tools #277

Add camera new tools #277

DamienCode404 commented Oct 11, 2024 •

edited

Loading

bgruening left a comment

hechth commented Oct 21, 2024

hechth commented Oct 21, 2024

DamienCode404 commented Oct 21, 2024

DamienCode404 commented Oct 21, 2024

hechth commented Oct 22, 2024

jsaintvanne commented Oct 22, 2024

hechth commented Oct 22, 2024

hechth commented Oct 22, 2024

jsaintvanne commented Oct 22, 2024

bgruening left a comment

bgruening Oct 26, 2024

DamienCode404 Oct 29, 2024

DamienCode404 commented Nov 6, 2024


		<expand macro="requirements"/>

		<command detect_errors="exit_code"><![CDATA[

Add camera new tools #277

Are you sure you want to change the base?

Add camera new tools #277

Conversation

DamienCode404 commented Oct 11, 2024 • edited Loading

Addition of New CAMERA Tools for the Metabolomics Suite

Description:

New Tools Added:

Why These Changes?

bgruening left a comment

Choose a reason for hiding this comment

hechth commented Oct 21, 2024

hechth commented Oct 21, 2024

DamienCode404 commented Oct 21, 2024

DamienCode404 commented Oct 21, 2024

hechth commented Oct 22, 2024

jsaintvanne commented Oct 22, 2024

hechth commented Oct 22, 2024

hechth commented Oct 22, 2024

jsaintvanne commented Oct 22, 2024

bgruening left a comment

Choose a reason for hiding this comment

bgruening Oct 26, 2024

Choose a reason for hiding this comment

DamienCode404 Oct 29, 2024

Choose a reason for hiding this comment

DamienCode404 commented Nov 6, 2024

DamienCode404 commented Oct 11, 2024 •

edited

Loading