Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notebooks: makes notebooks deterministic and prints output of asserts #847

Merged
merged 5 commits into from
Mar 1, 2024

Conversation

fabianandresgrob
Copy link
Collaborator

PR for #842 and #843

@nickfraser
Copy link
Collaborator

nickfraser commented Feb 14, 2024

Can you test the following:

  • Running (run-all from the top) these notebooks multiple times will produce the same output
  • Running them from the commandline with in-place updating will produce the same output as above
  • Running from the commandline on a new machine will also produce the same output

I have a small script to run the notebooks like so:

#!/bin/bash
# source: https://mljar.com/blog/run-jupyter-notebook-command-line/
for f in *.ipynb; do
    cp $f $f.orig
    jupyter nbconvert --execute --allow-errors --to notebook --inplace $f
    echo nbdiff-web -b /dev/null $f.orig $f
done

My goal is that when you compare with ndiff-web and a manual notebook run through the browser, you should get the same output (ignore warnings and such). This will make it easier to verify that the notebooks work in future.

@fabianandresgrob
Copy link
Collaborator Author

Checked using that script, everything looks good. Only minor things differ like kernel id. I added one more torch.manual_seed() to the ONNX export.

@Giuseppe5
Copy link
Collaborator

When doing multiple runs to compare, restart the kernels, just to be sure

@fabianandresgrob fabianandresgrob force-pushed the improve_notebooks branch 3 times, most recently from 50ef9a0 to 468b98b Compare February 15, 2024 18:26
@capnramses capnramses linked an issue Feb 21, 2024 that may be closed by this pull request
@Giuseppe5
Copy link
Collaborator

Rebase please

@Giuseppe5 Giuseppe5 mentioned this pull request Feb 28, 2024
4 tasks
Copy link
Collaborator

@nickfraser nickfraser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nickfraser nickfraser merged commit 1b8ff28 into Xilinx:dev Mar 1, 2024
22 checks passed
@fabianandresgrob fabianandresgrob deleted the improve_notebooks branch March 1, 2024 12:32
@fabianandresgrob fabianandresgrob restored the improve_notebooks branch March 7, 2024 14:07
@fabianandresgrob fabianandresgrob deleted the improve_notebooks branch March 7, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make notebook examples deterministic
3 participants