Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unstable CI tests #254

Open
vpratz opened this issue Nov 21, 2024 · 3 comments
Open

Unstable CI tests #254

vpratz opened this issue Nov 21, 2024 · 3 comments
Labels
discussion Discuss a topic or question not necessarily with a clear output in mind. unit tests A new set of tests needs to be added.

Comments

@vpratz
Copy link
Collaborator

vpratz commented Nov 21, 2024

I currently observe that tests sometimes fail, and then pass on repeat. Maybe we can use this issue to identify those cases and develop a strategy to make the tests more robust. I currently observe two failure modes that seem to be temporary (e.g. here):

  • Jax on Windows sometimes produces Windows fatal exception: code 0x8001010d
  • tests/test_two_moons/test_two_moons.py::test_fit[inference_network='coupling_flow'] produces too high values or nan
@vpratz vpratz added unit tests A new set of tests needs to be added. v2 discussion Discuss a topic or question not necessarily with a clear output in mind. labels Nov 21, 2024
@paul-buerkner
Copy link
Contributor

Are these tests still unstable after the latest fixes that @stefanradev93 introduced?

@stefanradev93
Copy link
Contributor

Closing, as the tests seem to be stable now.

@vpratz
Copy link
Collaborator Author

vpratz commented Dec 13, 2024

Reopening this, as we still sometimes see tests/test_two_moons/test_two_moons.py::test_fit[inference_network='coupling_flow'] fail with too large values (but no nans). What is the best approach here? Can we safely increase the threshold for passing, or do we need a more powerful network/longer training to get more consistent results?

@vpratz vpratz reopened this Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Discuss a topic or question not necessarily with a clear output in mind. unit tests A new set of tests needs to be added.
Projects
None yet
Development

No branches or pull requests

3 participants