Skip to content

Commit

Permalink
docs: added some links to API docs, explain which class to use. Closes
Browse files Browse the repository at this point in the history
  • Loading branch information
kjappelbaum authored Oct 25, 2020
1 parent 4af8189 commit 6cabb45
Show file tree
Hide file tree
Showing 33 changed files with 85 additions and 43 deletions.
Binary file modified docs/_build/doctrees/developer_notes.doctree
Binary file not shown.
Binary file modified docs/_build/doctrees/environment.pickle
Binary file not shown.
Binary file modified docs/_build/doctrees/getting_started.doctree
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/_build/html/.buildinfo
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: f7417dc9d569e9ca5682e495ec814785
config: eb1b58b8f9c80a0fdd55070f5fa71994
tags: 645f666f9bcd5a90fca523b33c5a78b7
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Overview: module code &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>Overview: module code &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/models/gpr.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.models.gpr &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.models.gpr &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/core.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.core &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.core &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/pal_base.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.pal_base &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.pal_base &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.pal_coregionalized &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.pal_coregionalized &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/pal_gbdt.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.pal_gbdt &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.pal_gbdt &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/pal_gpy.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.pal_gpy &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.pal_gpy &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/pal_sklearn.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.pal_sklearn &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.pal_sklearn &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/schedules.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.schedules &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.schedules &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/utils.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.utils &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.utils &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/pal/validate_inputs.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.pal.validate_inputs &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.pal.validate_inputs &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_modules/pypal/plotting.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>pypal.plotting &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>pypal.plotting &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="../../_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 2 additions & 0 deletions docs/_build/html/_sources/developer_notes.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ Currently, documentation is hosted on GitHub pages. Build it locally using :code
git subtree push --prefix docs/_build/html origin gh-pages
.. _new_pal_class:

Implementing a new PAL class
-----------------------------

Expand Down
23 changes: 17 additions & 6 deletions docs/_build/html/_sources/getting_started.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,27 @@ The latest version of PyPAL can be installed from GitHub using
pip install git+https://github.com/kjappelbaum/pypal.git
Which class do i use?
-----------------------

- For Gaussian processes built with :code:`sklearn` use :py:class:`pypal.pal.pal_sklearn.PALSklearn`
- For Gaussian processes built with :code:`GPy` use :py:class:`pypal.pal.pal_gpy.PALGPy`
- For coregionalized Gaussian processes (built with :code:`GPy`) use :py:class:`pypal.pal.pal_coregionalized.PALCoregionalized`
- For quantile regression using :code:`LightGBM` gradient boosted decision trees use :py:class:`pypal.pal.pal_gbdt.PALGBDT`

If your favorite model is not listed, you can easily implement it yourself (see :ref:`new_pal_class`)!


Running an active learning experiment
---------------------------------------

The `examples` directory contains a `Jupyter notebook with an example <https://github.com/kjappelbaum/pypal/blob/master/examples/test_pal.ipynb>`_ that can also be run on MyBinder.

If using a Gaussian process model built with :code:`sklearn` or :code:`GPy` we recommend using a pre-built class such as :code:`PALSklearn` or :code:`PALGpy` and following the subsequent steps:
If using a Gaussian process model built with :code:`sklearn` or :code:`GPy` we recommend using a pre-built class such as :py:class:`pypal.pal.pal_sklearn.PALSklearn`, :py:class:`pypal.pal.pal_coregionalized.PALCoregionalized`, :py:class:`pypal.pal.pal_gpy.PALGPy` and following the subsequent steps (for more details on which class to use see :ref:`Which class do i use?`):

1. For each objective create a model (if using a coregionalized Gaussian process model, only one model needs to be created)

2. Sample a few initial points from the design space. We provide the :code:`get_maxmin_samples` or :code:`get_kmeans_samples` utilities that can help with the sampling. Our code assumes that :code:`X` is a :code:`np.array`.
2. Sample a few initial points from the design space. We provide the :py:meth:`pypal.pal.utils.get_maxmin_samples` or :py:meth:`pypal.pal.utils.get_kmeans_samples` utilities that can help with the sampling. Our code assumes that :code:`X` is a :code:`np.array`.

.. code-block:: python
Expand Down Expand Up @@ -81,7 +92,7 @@ for a case in which we performed measurements for samples with index 1 and 2 of
Hyperparameter optimization
.............................
Usually, the hyperparameters of a machine learning model, in particular the kernel hyperparameters of a Gaussian process regression model, should be optimized as new training data is added.
However, since this is usually a computationally expensive process, it may not be desirable to perform this at every iteration of the active learning process. The iteration frequency of the hyperparameter optimization is internally set by the :code:`_should_optimize_hyperparameter` function, which by default uses a schedule that optimizes the hyperparameter every 10th iteration. This behavior can be changed by override this function.
However, since this is usually a computationally expensive process, it may not be desirable to perform this at every iteration of the active learning process. The iteration frequency of the hyperparameter optimization is internally set by the :code:`_should_optimize_hyperparameters` function, which by default uses a schedule that optimizes the hyperparameter every 10th iteration. This behavior can be changed by override this function.

Logging
........
Expand All @@ -94,7 +105,7 @@ Basic information such as the current iteration and the classification status ar
# returns: pypal at iteration 1. 10 Pareto optimal points, 1304 discarded points, 200 unclassified points.
We also provide calculation of the hypervolume enclosed by the Pareto front with the :code:`get_hypervolume` function
We also provide calculation of the hypervolume enclosed by the Pareto front with the function :py:meth:`pypal.pal.utils.get_hypervolume`

.. code:: python
Expand All @@ -121,7 +132,7 @@ Exploring a space where all objectives are known
.................................................

In some cases, we may already posess all measurements, but would like to run PAL with different settings to test how the algorithm performs.
In this case, we provide the :code:`exhaust_loop` wrapper.
In this case, we provide the :py:meth:`pypal.pal.utils.exhaust_loop` wrapper.

.. code-block:: python
Expand Down Expand Up @@ -182,7 +193,7 @@ By default, the code will run a simple cross-validation only on the first iterat
This behavior can changed with the cross-validation test being performed more frequently by overriding the :code:`should_run_crossvalidation` function.

Another way to detect overfitting is to use :code:`make_jointplot` function from the plotting subpackage. This function will plot all objectives against each other (with errorbars and different classes indicated with colors) and histograms of the objectives on the diagonal. If the majority of predicted points tend to overlap one another and get discarded by PyPAL, this may suggest that the surrogate model is overfitted.
Another way to detect overfitting is to use :py:func:`pypal.plotting.make_jointplot`function from the plotting subpackage. This function will plot all objectives against each other (with errorbars and different classes indicated with colors) and histograms of the objectives on the diagonal. If the majority of predicted points tend to overlap one another and get discarded by PyPAL, this may suggest that the surrogate model is overfitted.
.. code-block:: python
Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/_static/documentation_options.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
var DOCUMENTATION_OPTIONS = {
URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
VERSION: '0+untagged.210.gc6bd515.dirty',
VERSION: '0+untagged.212.g4af8189.dirty',
LANGUAGE: 'None',
COLLAPSE_INDEX: false,
BUILDER: 'html',
Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/api.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>The PyPAL API reference &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>The PyPAL API reference &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/background.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Background &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>Background &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
4 changes: 2 additions & 2 deletions docs/_build/html/developer_notes.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Developer notes &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>Developer notes &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down Expand Up @@ -225,7 +225,7 @@ <h3>Documentation<a class="headerlink" href="#documentation" title="Permalink to
</div>
</div>
<div class="section" id="implementing-a-new-pal-class">
<h2>Implementing a new PAL class<a class="headerlink" href="#implementing-a-new-pal-class" title="Permalink to this headline"></a></h2>
<span id="new-pal-class"></span><h2>Implementing a new PAL class<a class="headerlink" href="#implementing-a-new-pal-class" title="Permalink to this headline"></a></h2>
<p>If you want to use PyPAL with a model that we do not support yet, i.e., not <code class="code docutils literal notranslate"><span class="pre">GPy</span></code> or <code class="code docutils literal notranslate"><span class="pre">sklearn</span></code> Gaussian process regression, it is easy to write your own class. For this, you need to inherit from <cite>PALBase</cite> and implement your <code class="code docutils literal notranslate"><span class="pre">_train</span></code> and <code class="code docutils literal notranslate"><span class="pre">_predict</span></code> functions (and maybe also the <code class="code docutils literal notranslate"><span class="pre">_set_hyperparameters</span></code> and <code class="code docutils literal notranslate"><span class="pre">_should_optimize_hyperparameters</span></code> functions) using the <code class="code docutils literal notranslate"><span class="pre">design_space</span></code> and <code class="code docutils literal notranslate"><span class="pre">y</span></code> attributes of the class.</p>
<p>For instance, if we develop some multioutput model that has a <code class="code docutils literal notranslate"><span class="pre">train()</span></code> and a <code class="code docutils literal notranslate"><span class="pre">predict()</span></code> method we could simply use the following design pattern</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">pypal</span> <span class="kn">import</span> <span class="n">PALBase</span>
Expand Down
2 changes: 1 addition & 1 deletion docs/_build/html/genindex.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Index &#8212; pypal 0+untagged.210.gc6bd515.dirty documentation</title>
<title>Index &#8212; pypal 0+untagged.212.g4af8189.dirty documentation</title>

<link rel="stylesheet" href="_static/css/index.d431a4ee1c1efae0e38bdfebc22debff.css">

Expand Down
Loading

0 comments on commit 6cabb45

Please sign in to comment.