Complete implementation of Consistency Models

[vpratz]

Completes the implementation of Consistency Models.
I tested the implementation as a drop in replacement in the two moons notebook.

[marvinschmitt]

Thanks for the PR @vpratz, that's fantastic! Since you were creating a notebook to test the implementation anyway, would you mind adding that to the examples/ folder as well (maybe with some minimal accompanying text)?

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 17.85714% with 23 lines in your changes missing coverage. Please review.

Project coverage is 53.87%. Comparing base (f27da60) to head (1037d30).
Report is 2 commits behind head on dev.

Files with missing lines	Patch %	Lines
...w/networks/consistency_models/consistency_model.py	14.81%	23 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #205      +/-   ##
==========================================
+ Coverage   53.21%   53.87%   +0.65%     
==========================================
  Files         142      142              
  Lines        3266     3265       -1     
==========================================
+ Hits         1738     1759      +21     
+ Misses       1528     1506      -22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[vpratz]

Thanks for your feedback, I have added the notebook. I included a background section that turned out to be not very minimal, feel free to modify/shorten as you see fit.

[marvinschmitt]

Thanks again for the PR. I made some changes to the tutorial notebook:

• fix a few typos
• adjust $x \to \theta$ in the initial description of consistency models to be consistent (haha) with our subsequent conditional version, where $\theta$ is the target variable $x$ is the conditioning variable.
• use hyperparameters from the Consistency Model Posterior Estimation paper:
  • empirical variance of the prior samples as sigma2
  • max_time=10

We can merge from my side.

@stefanradev93 @LarsKue before we merge:

• Any feedback or change requests from your side?
• What's the strategy for adding tests?