hypothesis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Hypothesis\n",
    "\n",
    "> [Hypothesis](http://hypothesis.works/) is a Python library for creating unit tests which are simpler to write and more powerful when run, finding edge cases in your code you wouldn’t have thought to look for. It is stable, powerful and easy to add to any existing test suite."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classical Test\n",
    "1. Set up some data.\n",
    "1. Perform some operations on the data.\n",
    "1. Assert something, typically by comparing the value, about the result.\n",
    "\n",
    "### Hypothesis Test\n",
    "1. For all data matching some specification.\n",
    "1. Perform some operations on the data.\n",
    "1. Assert something, namely a general property, about the result.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hypothesis\n",
    "\n",
    "- performs a **strategic random search** in the input space\n",
    "\n",
    "\n",
    "- checks if the asserted **properties** hold true\n",
    "\n",
    "\n",
    "- if not, tries to find a minimal example\n",
    "\n",
    "\n",
    "- saves that example to check if the problem gets fixed\n",
    "\n",
    "Hypothesis does **Property Based Testing** and generalizes a normal unit test."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example\n",
    "\n",
    "Compute the mean\n",
    "\n",
    "```python\n",
    "def mean(x):\n",
    "    return sum(x) / len(x)\n",
    "```\n",
    "\n",
    "What could be a **property** of this function?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can expect that\n",
    "```python\n",
    "min(x) <= mean(x) <= max(x)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need a **data specification**, i.e. a *strategy*.\n",
    "\n",
    "```python\n",
    "from hypothesis.strategies import lists, floats\n",
    "lists(elements=floats())\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing hypothesis/intro_mean_0.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile hypothesis/intro_mean_0.py\n",
    "from hypothesis import given\n",
    "from hypothesis.strategies import lists, floats\n",
    "\n",
    "\n",
    "def mean(x):\n",
    "    return sum(x) / len(x)\n",
    "\n",
    "@given(x=lists(elements=floats()))\n",
    "def test_mean(x):\n",
    "    assert min(x) <= mean(x) <= max(x)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    import pytest\n",
    "    pytest.main([__file__])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0\n",
      "rootdir: /home/claus/repo/PAULST, inifile:\n",
      "plugins: mock-1.6.0, cov-2.3.1, hypothesis-3.33.0\n",
      "collected 1 item                                                                \u001b[0m\u001b[1m\n",
      "\n",
      "hypothesis/intro_mean_0.py F\n",
      "\n",
      "=================================== FAILURES ===================================\n",
      "\u001b[31m\u001b[1m__________________________________ test_mean ___________________________________\u001b[0m\n",
      "\n",
      "\u001b[1m    @given(x=lists(elements=floats()))\u001b[0m\n",
      "\u001b[1m>   def test_mean(x):\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_0.py\u001b[0m:9: \n",
      "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/executors.py\u001b[0m:58: in default_new_style_executor\n",
      "\u001b[1m    return function(data)\u001b[0m\n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/core.py\u001b[0m:136: in run\n",
      "\u001b[1m    return test(*args, **kwargs)\u001b[0m\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_0.py\u001b[0m:9: in test_mean\n",
      "\u001b[1m    def test_mean(x):\u001b[0m\n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/core.py\u001b[0m:605: in timed_test\n",
      "\u001b[1m    result = test(*args, **kwargs)\u001b[0m\n",
      "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n",
      "\n",
      "x = []\n",
      "\n",
      "\u001b[1m    @given(x=lists(elements=floats()))\u001b[0m\n",
      "\u001b[1m    def test_mean(x):\u001b[0m\n",
      "\u001b[1m>       assert min(x) <= mean(x) <= max(x)\u001b[0m\n",
      "\u001b[1m\u001b[31mE       ValueError: min() arg is an empty sequence\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_0.py\u001b[0m:10: ValueError\n",
      "---------------------------------- Hypothesis ----------------------------------\n",
      "Falsifying example: test_mean(x=[])\n",
      "You can add @seed(154071910969116746272963620341985113718) to this test or run pytest with --hypothesis-seed=154071910969116746272963620341985113718 to reproduce this failure.\n",
      "\u001b[33m=============================== warnings summary ===============================\u001b[0m\n",
      "None\n",
      "  Module already imported so can not be re-written: hypothesis\n",
      "\n",
      "-- Docs: http://doc.pytest.org/en/latest/warnings.html\n",
      "\u001b[31m\u001b[1m===================== 1 failed, 1 warnings in 0.16 seconds =====================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!python hypothesis/intro_mean_0.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing hypothesis/intro_mean_1.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile hypothesis/intro_mean_1.py\n",
    "\n",
    "from hypothesis import given, assume  # <-- import assume\n",
    "from hypothesis.strategies import lists, floats\n",
    "\n",
    "def mean(x):\n",
    "    return sum(x) / len(x)\n",
    "\n",
    "@given(x=lists(elements=floats()))\n",
    "def test_mean(x):\n",
    "    assume(x)  # <-- assume that list is not empty\n",
    "    assert min(x) <= mean(x) <= max(x)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    import pytest\n",
    "    pytest.main([__file__])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0\n",
      "rootdir: /home/claus/repo/PAULST, inifile:\n",
      "plugins: mock-1.6.0, cov-2.3.1, hypothesis-3.33.0\n",
      "collected 1 item                                                                \u001b[0m\u001b[1m\n",
      "\n",
      "hypothesis/intro_mean_1.py F\n",
      "\n",
      "=================================== FAILURES ===================================\n",
      "\u001b[31m\u001b[1m__________________________________ test_mean ___________________________________\u001b[0m\n",
      "\n",
      "\u001b[1m    @given(x=lists(elements=floats()))\u001b[0m\n",
      "\u001b[1m>   def test_mean(x):\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_1.py\u001b[0m:9: \n",
      "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/executors.py\u001b[0m:58: in default_new_style_executor\n",
      "\u001b[1m    return function(data)\u001b[0m\n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/core.py\u001b[0m:136: in run\n",
      "\u001b[1m    return test(*args, **kwargs)\u001b[0m\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_1.py\u001b[0m:9: in test_mean\n",
      "\u001b[1m    def test_mean(x):\u001b[0m\n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/core.py\u001b[0m:605: in timed_test\n",
      "\u001b[1m    result = test(*args, **kwargs)\u001b[0m\n",
      "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n",
      "\n",
      "x = [nan]\n",
      "\n",
      "\u001b[1m    @given(x=lists(elements=floats()))\u001b[0m\n",
      "\u001b[1m    def test_mean(x):\u001b[0m\n",
      "\u001b[1m        assume(x)  # <-- assume that list is not empty\u001b[0m\n",
      "\u001b[1m>       assert min(x) <= mean(x) <= max(x)\u001b[0m\n",
      "\u001b[1m\u001b[31mE       assert nan <= nan\u001b[0m\n",
      "\u001b[1m\u001b[31mE        +  where nan = min([nan])\u001b[0m\n",
      "\u001b[1m\u001b[31mE        +  and   nan = mean([nan])\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_1.py\u001b[0m:11: AssertionError\n",
      "---------------------------------- Hypothesis ----------------------------------\n",
      "Falsifying example: test_mean(x=[nan])\n",
      "You can add @seed(234266594554593701488138895958435831724) to this test or run pytest with --hypothesis-seed=234266594554593701488138895958435831724 to reproduce this failure.\n",
      "\u001b[33m=============================== warnings summary ===============================\u001b[0m\n",
      "None\n",
      "  Module already imported so can not be re-written: hypothesis\n",
      "\n",
      "-- Docs: http://doc.pytest.org/en/latest/warnings.html\n",
      "\u001b[31m\u001b[1m===================== 1 failed, 1 warnings in 0.61 seconds =====================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!python hypothesis/intro_mean_1.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing hypothesis/intro_mean_2.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile hypothesis/intro_mean_2.py\n",
    "\n",
    "from hypothesis import given, assume  # <-- import assume\n",
    "from hypothesis.strategies import lists, floats\n",
    "\n",
    "def mean(x):\n",
    "    return sum(x) / len(x)\n",
    "\n",
    "@given(x=lists(elements=floats(allow_nan=False)))  # <-- do not allow nan\n",
    "def test_mean(x):\n",
    "    assume(x)  # <-- assume that list is not empty\n",
    "    assert min(x) <= mean(x) <= max(x)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    import pytest\n",
    "    pytest.main([__file__])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0\n",
      "rootdir: /home/claus/repo/PAULST, inifile:\n",
      "plugins: mock-1.6.0, cov-2.3.1, hypothesis-3.33.0\n",
      "collected 1 item                                                                \u001b[0m\u001b[1m\n",
      "\n",
      "hypothesis/intro_mean_2.py F\n",
      "\n",
      "=================================== FAILURES ===================================\n",
      "\u001b[31m\u001b[1m__________________________________ test_mean ___________________________________\u001b[0m\n",
      "\n",
      "\u001b[1m    @given(x=lists(elements=floats(allow_nan=False)))  # <-- do not allow nan\u001b[0m\n",
      "\u001b[1m>   def test_mean(x):\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_2.py\u001b[0m:9: \n",
      "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/executors.py\u001b[0m:58: in default_new_style_executor\n",
      "\u001b[1m    return function(data)\u001b[0m\n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/core.py\u001b[0m:136: in run\n",
      "\u001b[1m    return test(*args, **kwargs)\u001b[0m\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_2.py\u001b[0m:9: in test_mean\n",
      "\u001b[1m    def test_mean(x):\u001b[0m\n",
      "\u001b[1m\u001b[31m../../anaconda3/envs/scipytst/lib/python3.6/site-packages/hypothesis/core.py\u001b[0m:605: in timed_test\n",
      "\u001b[1m    result = test(*args, **kwargs)\u001b[0m\n",
      "_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ \n",
      "\n",
      "x = [-inf, inf]\n",
      "\n",
      "\u001b[1m    @given(x=lists(elements=floats(allow_nan=False)))  # <-- do not allow nan\u001b[0m\n",
      "\u001b[1m    def test_mean(x):\u001b[0m\n",
      "\u001b[1m        assume(x)  # <-- assume that list is not empty\u001b[0m\n",
      "\u001b[1m>       assert min(x) <= mean(x) <= max(x)\u001b[0m\n",
      "\u001b[1m\u001b[31mE       assert -inf <= nan\u001b[0m\n",
      "\u001b[1m\u001b[31mE        +  where -inf = min([-inf, inf])\u001b[0m\n",
      "\u001b[1m\u001b[31mE        +  and   nan = mean([-inf, inf])\u001b[0m\n",
      "\n",
      "\u001b[1m\u001b[31mhypothesis/intro_mean_2.py\u001b[0m:11: AssertionError\n",
      "---------------------------------- Hypothesis ----------------------------------\n",
      "Falsifying example: test_mean(x=[-inf, inf])\n",
      "You can add @seed(241023051238221796542528424773920859300) to this test or run pytest with --hypothesis-seed=241023051238221796542528424773920859300 to reproduce this failure.\n",
      "\u001b[33m=============================== warnings summary ===============================\u001b[0m\n",
      "None\n",
      "  Module already imported so can not be re-written: hypothesis\n",
      "\n",
      "-- Docs: http://doc.pytest.org/en/latest/warnings.html\n",
      "\u001b[31m\u001b[1m===================== 1 failed, 1 warnings in 0.55 seconds =====================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!python hypothesis/intro_mean_2.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing hypothesis/intro_mean_3.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile hypothesis/intro_mean_3.py\n",
    "\n",
    "from hypothesis import given, assume  # <-- import assume\n",
    "from hypothesis.strategies import lists, floats\n",
    "\n",
    "def mean(x):\n",
    "    return sum(x) / len(x)\n",
    "\n",
    "# <-- do not allow nan, do not allow infinity\n",
    "@given(x=lists(elements=floats(allow_nan=False, allow_infinity=False)))\n",
    "def test_mean(x):\n",
    "    assume(x)  # <-- assume that list is not empty\n",
    "    assert min(x) <= mean(x) <= max(x)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    import pytest\n",
    "    pytest.main([__file__])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0\n",
      "rootdir: /home/claus/repo/PAULST, inifile:\n",
      "plugins: mock-1.6.0, cov-2.3.1, hypothesis-3.33.0\n",
      "collected 1 item                                                                \u001b[0m\u001b[1m\n",
      "\n",
      "hypothesis/intro_mean_3.py .\n",
      "\n",
      "\u001b[33m=============================== warnings summary ===============================\u001b[0m\n",
      "None\n",
      "  Module already imported so can not be re-written: hypothesis\n",
      "\n",
      "-- Docs: http://doc.pytest.org/en/latest/warnings.html\n",
      "\u001b[33m\u001b[1m===================== 1 passed, 1 warnings in 0.36 seconds =====================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!python hypothesis/intro_mean_3.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you imagine any additional problems?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.30000000000000004"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How about this?\n",
    "0.1 + 0.1 + 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing hypothesis/intro_mean_4.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile hypothesis/intro_mean_4.py\n",
    "\n",
    "from hypothesis import given, assume, example\n",
    "from hypothesis.strategies import lists, floats\n",
    "\n",
    "def mean(x):\n",
    "    return sum(x) / len(x)\n",
    "\n",
    "@example([0.1, 0.1, 0.1])  # <-- add special case\n",
    "@given(x=lists(elements=floats(allow_nan=False, allow_infinity=False)))\n",
    "def test_mean(x):\n",
    "    assume(x)\n",
    "    assert min(x) - 1e-8 <= mean(x) <= max(x) + 1e-8\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    import pytest\n",
    "    pytest.main([__file__])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.6.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0\n",
      "rootdir: /home/claus/repo/PAULST, inifile:\n",
      "plugins: mock-1.6.0, cov-2.3.1, hypothesis-3.33.0\n",
      "collected 1 item                                                                \u001b[0m\u001b[1m\n",
      "\n",
      "hypothesis/intro_mean_4.py .\n",
      "\n",
      "\u001b[33m=============================== warnings summary ===============================\u001b[0m\n",
      "None\n",
      "  Module already imported so can not be re-written: hypothesis\n",
      "\n",
      "-- Docs: http://doc.pytest.org/en/latest/warnings.html\n",
      "\u001b[33m\u001b[1m===================== 1 passed, 1 warnings in 0.33 seconds =====================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!python hypothesis/intro_mean_4.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summary\n",
    "\n",
    "Hypothesis\n",
    "- managed to find edge cases that are not covered\n",
    "- explicit example cases can be passed\n",
    "\n",
    "Challenges\n",
    "- getting your data right\n",
    "- identifying properties\n",
    "\n",
    "Notes\n",
    "- Properties are very problem specific and maybe not obvious\n",
    "- Testing properties is in general not sufficient (cf. `mean` vs. `median`)\n",
    "- In above example we restricted the search space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generate Data\n",
    "\n",
    "[Hypothesis for the Scientific Stack](http://hypothesis.readthedocs.io/en/latest/numpy.html):\n",
    "- numpy\n",
    "- pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def examples(strategy, n=3):\n",
    "    for _ in range(n):\n",
    "        print(strategy.example(), '\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1e-05, 4.339153192424458e+16, -6.285711018828626e+16, -1.5, nan, 6.338751142470446e-108] \n",
      "\n",
      "[] \n",
      "\n",
      "[-1.1977964691766333e+186, 1.8906949861730064e+16, -3.402823466e+38, 3273952135995461.0, nan, 3.402823466e+38, nan, -2.9528053461762548e+16, -3.703384598018958e+16, -1.845729331567465e+142, -4.343837701191257e+16, 6.0659438183327416e+16, -7.0618361161914216e+16, 1.192092896e-07, -1.2087613502964555e+234, 2.2250738585072014e-308, -3.4089697788891177e+218, 2.8344539885747253e+189] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "from hypothesis.strategies import lists, floats\n",
    "\n",
    "strategy = lists(elements=floats())\n",
    "examples(strategy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.9633653394907686, 0.07723148456860775] \n",
      "\n",
      "[0.9331416954308236, 0.4142750850720617] \n",
      "\n",
      "[0.05793894695600567, 0.42810248775534954] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "strategy = lists(elements=floats(min_value=0, max_value=1), \n",
    "                 min_size=2, \n",
    "                 max_size=2)\n",
    "examples(strategy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n",
    "\n",
    "Start with [`arrays`](http://hypothesis.readthedocs.io/en/latest/numpy.html#hypothesis.extra.numpy.arrays) to implement the following strategies:\n",
    "\n",
    "- create integer arrays of shape (2, 3)\n",
    "- as above but with with bounded values\n",
    "- create float arrays of shape (len, 2) where len is between 2 and 5\n",
    "- as above but with first column values <= second column values\n",
    "- as above buth with <\n",
    "\n",
    "Note: [`filter`](http://hypothesis.readthedocs.io/en/latest/data.html#mapping) and [`map`](http://hypothesis.readthedocs.io/en/latest/data.html#filtering) allow to adapt a strategy.\n",
    "\n",
    "---\n",
    "\n",
    "Use [`composite`](https://hypothesis.readthedocs.io/en/latest/data.html#composite-strategies) to create:\n",
    "- an array X of shape (5, 2) and\n",
    "- an array y of shape (5,) where y is a random linear combination of the columns of X, i.e. y = X * c.T.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solutions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from hypothesis.extra.numpy import arrays\n",
    "from hypothesis.strategies import composite, just\n",
    "from hypothesis.strategies import (tuples, lists, \n",
    "                                   booleans, integers, floats)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 3, 3],\n",
       "       [1, 0, 2]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "arrays(dtype=np.int, elements=integers(min_value=0, max_value=3), shape=(2, 3)).example()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.86894353,  0.87988412],\n",
       "       [ 0.91949955,  0.48971159],\n",
       "       [ 0.22638774,  0.32773948],\n",
       "       [ 0.22638774,  0.22638774],\n",
       "       [ 0.91302933,  0.22638774]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arrays(dtype=np.float, \n",
    "       elements=floats(min_value=0, max_value=1), \n",
    "       shape=tuples(integers(min_value=2, max_value=5), \n",
    "                    just(2))).example()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.29211861,  0.29211861],\n",
       "       [ 0.5440692 ,  0.83423388],\n",
       "       [ 0.29211861,  0.29211861]])"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arrays(dtype=np.float, \n",
    "       elements=floats(min_value=0, max_value=1), \n",
    "       shape=tuples(integers(min_value=2, max_value=5), \n",
    "                    just(2))).map(lambda x: np.sort(x)).example()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.21068308,  0.45112424],\n",
       "       [ 0.06522418,  0.5403578 ]])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arrays(dtype=np.float, \n",
    "       elements=floats(min_value=0, max_value=1), \n",
    "       shape=tuples(integers(min_value=2, max_value=5), \n",
    "                    just(2))).map(lambda x: np.sort(x)).filter(lambda x: all(x[:, 0] < x[:, 1])).example()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "@composite\n",
    "def create_linear_dependency(draw):\n",
    "    X = draw(arrays(dtype=np.int, elements=integers(0, 7), shape=(5, 2)))\n",
    "    c = draw(arrays(dtype=np.int, elements=integers(0, 7), shape=(2,)))\n",
    "    return X, X @ c\n",
    "\n",
    "create_linear_dependency().example()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Summary\n",
    "\n",
    "- hypothesis allows one to explicitely state assumptions\n",
    "\n",
    "\n",
    "- hypothesis can be helpful in identifying implicit assumptions\n",
    "\n",
    "\n",
    "- hypothesis test examples may be a bit too pathological in some situations\n",
    "\n",
    "\n",
    "- encoding/deconding problems or situations where an inverse function exists can be tackled very nicely"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}