This project aims to explore Bayesian approaches to deep learning by implementing Variational Inference, and also a simple HMC based sampler. The VI implementation is largely based on the Autograd Bayesian Neural Networks example which is implemented in Python. The HMC implementation is based on the example from AdvancedHMC which uses the No-U-Turn Samples (NUTS). This project compares and evaluates sampling and VI based approaches.
This codebase contains a number of notebooks for the experiments, and some Julia modules which contain the VI implementation. Details of each artefact are described in the final report's 'Artefact' section.
To run the Julia implementations in this codebase you need Julia version 1.7, and Jupyter notebook. There are several other packages that will require installing using the Pkg package. This is all detailed in the final report.
To see the results of experiments for BNN using VI check the vi_experiments notebooks.
The results of experiments using HMC are in the hmc_experiments.
- Download Julia from the official site: https://julialang.org/downloads/
- Install Julia
- Download anaconda distribution from: https://www.anaconda.com/products/distribution
- For Linux, run
jupyter notebook
- For Windows, open an anaconda prompt then run
jupyter notebook
- Open Julia prompt and execute: (you should be able to just copy the whole list into prompt and execute)
using Pkg
Pkg.add("IJulia")
Pkg.add("Plots")
Pkg.add("Distributions")
Pkg.add("Flux")
Pkg.add("OrderedCollections")
Pkg.add("OMEinsum")
Pkg.add("BenchmarkTools")
Pkg.add(Pkg.PackageSpec(;name="AdvancedHMC", version="0.3.5"))
Pkg.add("ForwardDiff")
Pkg.add("Zygote")
- Unzip the submitted codebase
- On Jupyter Notebook in your browser, navigate to where the code was extracted from step 7
- Execute all five notebooks