notebooks

Jupyter Notebooks 📓 by Naereen @ GitHub

This folder hosts some Jupyter Notebooks, to present in a nice format some numerical experiments for my SMPyBandits project.

The wonderful Jupyter tools is awesome to write interactive and nicely presented 🐍 Python simulations!

---

1. List of experiments presented with notebooks

MAB problems

• Easily creating various Multi-Armed Bandit problems, explains the interface of the Environment.MAB module.

Single-Player simulations

• A simple example of Single-Player simulation, comparing UCB1 (for two values of $\alpha$, 1 and 1/2), Thompson Sampling, BayesUCB and kl-UCB.
• Do we even need UCB? demonstrates the need for an algorithm smarter than the naive EmpiricalMeans.
• Lai-Robbins lower-bound for doubling-tricks algorithms with full restart.

Active research on Single-Player MAB

• Exploring different doubling tricks for different kinds of regret bounds.
• Experiments of statistical tests for piecewise stationary bandits
• Demonstrations of Single-Player Simulations for Non-Stationary Bandits.

Multi-Player simulations

• A simple example of Multi-Player simulation with 4 Centralized Algorithms, comparing CentralizedMultiplePlay and CentralizedIMP with UCB and Thompson Sampling.
• A simple example of Multi-Player simulation with 2 Decentralized Algorithms, comparing rhoRand and Selfish (for the "collision avoidance" part) combined with UCB and Thompson Sampling for learning the arms. Spoiler: Selfish beats rhoRand!

(Old) Experiments

• Can we use a (non-online) Unsupervised Learning algorithm for (online) Bandit problem ?
• Can we use a computationally expensive Black-Box Bayesian optimization algorithm for (online) Bandit problem ?

---

2. Question: How to read these documents?

2.a. View the notebooks statically 📝

• Either directly in GitHub: see the list of notebooks;
• Or on nbviewer.jupiter.org: list of notebooks.

2.b. Play with the notebooks dynamically (on MyBinder) 💥

Anyone can use the mybinder.org website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.

• Do_we_even_need_UCB.ipynb
• Easily_creating_MAB_problems.ipynb
• Example_of_a_small_Single-Player_Simulation.ipynb
• Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb
• Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb
• Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb
• Exploring different doubling tricks for different kinds of regret bounds.ipynb
• Experiments of statistical tests for piecewise stationary bandits.ipynb
• Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb

2.c. Play with the notebooks dynamically (on Google Colab) 💥

Anyone can use the colab.research.google.com/notebook website (by clicking on the icon above) to run the notebook in her/his web-browser. You can then play with it as long as you like, for instance by modifying the values or experimenting with the code.

• Do_we_even_need_UCB.ipynb
• Easily_creating_MAB_problems.ipynb
• Example_of_a_small_Single-Player_Simulation.ipynb
• Example_of_a_small_Multi-Player_Simulation__with_Centralized_Algorithms.ipynb
• Example_of_a_small_Multi-Player_Simulation__with_rhoRand_and_Selfish_Algorithms.ipynb
• Lai_Robbins_Lower_Bound_for_Doubling_Trick_with_Restarting_Algorithms.ipynb
• Exploring different doubling tricks for different kinds of regret bounds.ipynb
• Experiments of statistical tests for piecewise stationary bandits.ipynb
• Demonstrations of Single-Player Simulations for Non-Stationary-Bandits.ipynb

---

3. Question: Requirements to run the notebooks locally?

All the requirements can be installed with pip.

Note: if you use Python 3 instead of Python 2, you might have to replace pip and python by pip3 and python3 in the next commands (if both pip and pip3 are installed).

3.a. Jupyter Notebook and IPython

sudo pip install jupyter ipython

It will also install all the dependencies, afterward you should have a jupyter-notebook command (or a jupyter command, to be ran as jupyter notebook) available in your PATH:

$ whereis jupyter-notebook
jupyter-notebook: /usr/local/bin/jupyter-notebook
$ jupyter-notebook --version  # version >= 4 is recommended
4.4.1

3.b. My numerical environment, SMPyBandits

• First, install its dependencies (pip install -r requirements).
• Then, either install it (not yet), or be sure to work in the main folder.

Note: it's probably better to use virtualenv, if you like it. I never really understood how and why virtualenv are useful, but if you know why, you should know how to use it.

---

💁 More information?

• More information about notebooks (on the documentation of IPython) or on the FAQ on Jupyter's website.
• More information about mybinder.org: on this example repository.

📜 License ?

All the notebooks in this folder are published under the terms of the MIT License (file LICENSE.txt). © Lilian Besson, 2016-18.