Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add R2 score to metrics #8093

Open
wants to merge 9 commits into
base: dev
Choose a base branch
from

Conversation

thibaultdvx
Copy link

Fixes #8085.

Description

I created:

  • R2Metric and compute_r2_score in monai.metrics,
  • a handler called R2Score in monai.handlers,
  • three unittest modules: test_compute_r2_score.py, test_handler_r2_score.py and test_handler_r2_score_dist.py.

I also modified the docs to mention R2 score.

Note:

  • R2 score expects 1D or 2D data, i.e. I didn't allow the computation of R2 at pixel level in an image, like it can be done with the other regression metrics.
  • One needs all the data to compute R2 score (because of the mean), so tensors are simply accumulated and the metric computation is made once on all the data.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

thibaultdvx and others added 9 commits September 18, 2024 17:13
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
@thibaultdvx thibaultdvx marked this pull request as ready for review September 18, 2024 16:52
@thibaultdvx
Copy link
Author

/build

@thibaultdvx
Copy link
Author

hi @KumoLiu, could you please have a look at this PR?

Copy link
Contributor

@KumoLiu KumoLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, leave few comments inline.

from monai.handlers import R2Score


class TestHandlerR2Score(unittest.TestCase):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def _calculate(y_pred: np.ndarray, y: np.ndarray, p: int) -> float:
num_obs = len(y)
rss = np.sum((y_pred - y) ** 2)
tss = np.sum(y**2) - np.sum(y) ** 2 / num_obs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be tss = np.sum((y - np.mean(y)) ** 2) here?


class R2Score(IgniteMetricHandler):
"""
Computes :math:`R^{2}` score accumulating predictions and the ground-truth during an epoch and applying `compute_r2_score`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As elsewhere it would help to explain where this is used and what it would be for. The math is explained in the function which is fine so no need here, just a high level idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here if possible to combine this file with the previous.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Average Precision and R2 to the metrics
3 participants