Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding validation ELBO loss for pyro models #1073

Closed
vitkl opened this issue May 24, 2021 · 3 comments
Closed

adding validation ELBO loss for pyro models #1073

vitkl opened this issue May 24, 2021 · 3 comments

Comments

@vitkl
Copy link
Contributor

vitkl commented May 24, 2021

Would be great to have validation set ELBO loss for pyro models. As discussed with Adam here #1059 (comment)
@adamgayoso I can implement this, but could you point me where this belongs?

@vitkl
Copy link
Contributor Author

vitkl commented May 24, 2021

A related warning:

  /Users/vk7/anaconda3/envs/scvi-tools-dev/lib/python3.7/site-packages/pytorch_lightning/utilities/distributed.py:69: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
    warnings.warn(*args, **kwargs)

@adamgayoso
Copy link
Member

We need to add a function to this class

https://github.com/YosefLab/scvi-tools/blob/14ac97718c7d50470bf0db25aa7dd6d4f4a245c6/scvi/train/_trainingplans.py#L581

with this signature

https://github.com/YosefLab/scvi-tools/blob/14ac97718c7d50470bf0db25aa7dd6d4f4a245c6/scvi/train/_trainingplans.py#L517

that

  1. runs evaluate_loss and then self.log(..) it, or
  2. returns the loss evaluation then additionally write a method with signature

https://github.com/YosefLab/scvi-tools/blob/14ac97718c7d50470bf0db25aa7dd6d4f4a245c6/scvi/train/_trainingplans.py#L563

That sums over the accumulation of the loss. This would be analogous to what we discussed with averaging/summing during minibatching

@canergen
Copy link
Member

The implementation in #3015 is correct and can be adjusted for other models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants