Skip to content

Code for the paper "Gamma-Models: Generative Temporal Difference Learning for Infinite-Horizon Prediction"

License

Notifications You must be signed in to change notification settings

jannerm/gamma-models

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gamma-Models

Open In Colab

Code release for Gamma-Models: Generative Temporal Difference Learning for Infinite-Horizon Prediction.

Run from your browser

For the quickest startup, we recommend running the notebook directly in your browser using Google Colab.

This notebook will generate a video that looks like the following:

The last cell in the notebook shows how the trained model can be used for value estimation:

visualize_values(prob_fn, n_steps=20)

as shown in Figure 4 of the paper. Increasing n_steps will improve the resolution of the value map but make the visualization take longer to generate.

Run locally

  1. Clone gamma-models
git clone https://github.com/jannerm/gamma-models.git
  1. Create a conda environment and install gamma
cd gamma-models
conda env create -f environment.yml
conda activate gamma
pip install -e .
  1. Add gamma as an IPython kernel and launch jupyter
python -m ipykernel install --user --name=gamma
jupyter notebook --port 6100 scripts

Open gamma-pendulum-local.ipynb, which matches the Colab notebook except for a bit of Colab-specific setup in the beginning.

Reference

@inproceedings{janner2020gamma,
  title={$\gamma$-Models: Generative Temporal Difference Learning for Infinite-Horizon Prediction},
  author={Michael Janner and Igor Mordatch and Sergey Levine},
  booktitle={Advances in Neural Information Processing Systems},
  year={2020}
}

Acknowledgments

The underlying neural spline flow implementation is based on Andrej Karpathy's pytorch-normalizing-flows repo, which in turn is based on Conor Durkan and Iain Murray's and nsf codebase.

About

Code for the paper "Gamma-Models: Generative Temporal Difference Learning for Infinite-Horizon Prediction"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages