Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include error delta in log of FROG tools #7

Open
mihai-sysbio opened this issue Aug 25, 2023 · 1 comment
Open

Include error delta in log of FROG tools #7

mihai-sysbio opened this issue Aug 25, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@mihai-sysbio
Copy link

During the validation of the FROG spec with FBCMT, I have encountered some cases where models were reproducible, with the exception of a small number of reactions. Instead of having to read through the whole report, not being sure what exactly caused the reported errors and simply trying randomly with relaxed parameters (-a 0.01 -r 0.01), it would be easier for me as a curator if this (informal) summary would be printed out at the end. And if it’s too much for the current version of the specification, it could be implemented as a --verbose flag by different tools for now.

@exaexa exaexa added the enhancement New feature or request label Aug 26, 2023
@exaexa
Copy link
Collaborator

exaexa commented Aug 26, 2023

Hi,
I'm all in. Adding a bit of continuation here from the thread we had in mail before:

I'll need to think about how much of this should be mandated for all implementations via the spec requirements, and how much of this should be recommended (or just implemented in FBCMT&others).

Certain pros for me:

  • the tolerances are a pretty wild subject for discussion and the whole interpretation of these cannot be summarized by just putting 2 thresholds
  • reporting of the numerical imprecisions would be much better if the users could at least print out a histogram or see which categories of tests failed most

Possible problems:

  • we already adhered to a quite binding methodology, and this might cause frog to easily irreversibly adhere to an even more specific one, complicating the implementation of alternative approaches

I suggest this:

  • spec recommends that the implementations are capable of outputting the tolerance stats and test results in an interoperable format, recommending a simple TSV
  • I'll try to make a demo thereof in FBCMT and see

( Also cc @rsmsheriff )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants