scopatz_scipy2013.ipynb

{
 "metadata": {
  "name": "scopatz_scipy2013"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center\">\n",
      "<h1>XDress - Type, But Verify</h1>\n",
      "<h2><img src=\"thief.jpg\" /><br />SciPy 2013</h2>\n",
      "</div>\n",
      "<div style=\"text-align:center\">Anthony Michael Scopatz</div>\n",
      "<div style=\"text-align:center\">The University of Wisconsin-Madison, NumFOCUS</div>\n",
      "<div style=\"text-align:center\">Austin, TX, June 26th, 2013</div>"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os\n",
      "import sys\n",
      "import shutil\n",
      "import numpy as np\n",
      "import matplotlib\n",
      "import collections\n",
      "matplotlib.rc('font', family='serif', size=16)\n",
      "import matplotlib.pyplot as plt\n",
      "from IPython.display import *\n",
      "from utils import htmltable\n",
      "from pprint import pprint"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "source": [
      "## Cheat Sheet\n",
      "\n",
      "**Center:**  ``<div style=\"text-align:center;\"></div>``\n",
      "    \n",
      "**Right:**   ``<div style=\"text-align:right;\"></div>``\n",
      "    \n",
      "**Center H3:**  ``<div style=\"text-align:center;\"><h3></h3></div>``\n",
      "\n",
      "**Right H3:**  ``<div style=\"text-align:right;\"><h3></h3></div>``\n",
      "    \n",
      "**Center Image:**  ``<div style=\"text-align:center;\"><img src=\"\" width=\"80%\" /></div>``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## What's the Big Idea\n",
      "\n",
      "<div style=\"text-align:center;\">At its core, xdress is a suite of code generation routines built on top of a dynamic type system.</div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "## Motivation\n",
      "\n",
      "* First class C++ support\n",
      "* Deep access to the type system\n",
      "* A multitude of front ends (including Cython)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## What XDress Is"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "source": [
      "Created via [this link](http://interactive.blockdiag.com/?compression=deflate&src=eJyr5lJQUHIM8FRwy8xLSS1SUoguLqnMSVWwVUhJLM5ITYm1BioISCwqTi0qVtC1Q1UL5geHKIRlFmeW5EMFnPNTUhXcU_NSixKBYsVK1ly1AOqXHj4)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<div style=\"text-align:center;\"><img src=\"workflow.svg\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "## What XDress Is Not\n",
      "\n",
      "* **Parser**\n",
      "* **Compiler**\n",
      "* **Build System**"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Type System\n",
      "\n",
      "* Since we are not doing any parsing or compiling, the type system doesn't need to know the concrete size of any types."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Rather the type system is a convient, consistent, and complete way to *spell* type names!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Type names are composed of immutable Python types:\n",
      "    \n",
      "    * ``str``\n",
      "    * ``tuple``\n",
      "    * ``int``\n",
      "    * ``TypeMatcher``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:right;\"><img src=\"zomg.jpg\" width=\"75\" /><br /><b>zOMG!</b></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Kinds of Types\n",
      "\n",
      "As a brief diversion into type theory, xdress supports:\n",
      "    \n",
      "<div style=\"text-align:center;\"><h3>Base Types</h3></div><br/>\n",
      "\n",
      "<div style=\"text-align:center;\"><h3>Dependent Types</h3></div><br/>\n",
      "\n",
      "<div style=\"text-align:center;\"><h3>Refinement Types</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Base Types\n",
      "\n",
      "Base types are the normal primitive types that we all know and lovehate.  \n",
      "\n",
      "These are represented by ``str``.  \n",
      "\n",
      "Classes and structs also become base types."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import xdress.typesystem as ts\n",
      "pprint(ts.base_types)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "set(['bool',\n",
        "     'char',\n",
        "     'complex128',\n",
        "     'file',\n",
        "     'float128',\n",
        "     'float32',\n",
        "     'float64',\n",
        "     'int128',\n",
        "     'int16',\n",
        "     'int32',\n",
        "     'int64',\n",
        "     'str',\n",
        "     'type',\n",
        "     'uchar',\n",
        "     'uint128',\n",
        "     'uint16',\n",
        "     'uint32',\n",
        "     'uint64',\n",
        "     'void'])\n"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Dependent Types\n",
      "\n",
      "Dependent types are parameterized by static values *or* types!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "These are used to represent templated types in C++."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "They are spelled with ``tuple``s."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "set_of_strings = ('set', 'str')\n",
      "map_int_double = ('map', 'int32', 'float64')"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Refined Types\n",
      "\n",
      "Refined types are defined such that all of their elements are subset of another type.  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "They are specified by the refinement type name, the super type name, and a predicate."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "__Examples:__\n",
      "    \n",
      "* Even integers are a refinement of ``int32``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Strings which begin with \"A\" are a refinement of ``str``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* ``set([\"Awesome\", \"As\", \"Anthony\"])`` is a refinement of \"A\" strings"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Refined Types\n",
      "\n",
      "These may be spelled with ``str`` or the (super type, type name) ``tuple``s."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "type_name = 'evenint'\n",
      "final_form = ('int32', 'evenint')"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "## Refined Dependent Types\n",
      "\n",
      "Recall that dependent types are fully composable with other types."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "int_range = ('range', 'int32', 1, 42)\n",
      "even_int_range = ('range', ('int32', 'evenint'), 1, 42)\n",
      "too_much_fun = ('map', 'evenint', ('range', ('int32', 'evenint'), 1, 42))"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## And More!\n",
      "\n",
      "* What you have seen so far is the shorthand form.  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* There is also a canonical form: ``ts.canon(t)``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* There are aliases: ``ts.canon('f8') == 'float64'``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* Standard refined & dependent types built-in:\n",
      "    \n",
      "    * **templates:** ``pair``, ``set``, ``map``, ``vector``\n",
      "    * **refinements:** ``enum``, ``fucntion``, ``function_pointer``"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "* There is also a ``TypeMatcher`` class - inspired by ``re`` - which may replace any type or portion of a type."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><img src=\"dinner.jpg\" width=\"80%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## API Descriptions\n",
      "\n",
      "Code is generated from descriptions of API elements (variables, functions, and classes) to produce idomatic Python wrappers."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "brienne_desc = {\n",
      "    'name': 'Brienne',\n",
      "    'parents': ['LordSelwyn'],\n",
      "    'namespace': 'westeros',\n",
      "    'attrs': {\n",
      "        'moniker': 'str',\n",
      "        },\n",
      "    'methods': {\n",
      "        ('Brienne',): None,\n",
      "        ('Brienne', ('monkier', 'str', '\"Maid of Tarth\"')): None,\n",
      "        ('~Brienne',): None, \n",
      "        ('attack', ('person', ('char' '*'), ('weapon_id', 'i2', 1))): 'float32',\n",
      "        },\n",
      "    'docstrings': {\n",
      "        'class': \"I am so tall!\",\n",
      "        'attrs': {\n",
      "            'monkier': 'extra name for character',\n",
      "            },\n",
      "        'methods': {\n",
      "            'attack': \"Attack someone with a weapon and returns damage.\"\n",
      "            },\n",
      "        },\n",
      "    'extra': {},\n",
      "    }"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``cythongen`` via Hoover Example\n",
      "\n",
      "### jehsrc/hoover.h\n",
      "\n",
      "    #if !defined(HOOVER)\n",
      "    #define HOOVER\n",
      "    namespace hoover {\n",
      "    class A {\n",
      "    public:\n",
      "      float a;\n",
      "      double power(int n=1);\n",
      "    };};\n",
      "    #endif\n",
      "    \n",
      "### jehsrc/hoover.cpp\n",
      "\n",
      "    #include \"hoover.h\"\n",
      "    double hoover::A::power(int n){\n",
      "      int i = 1;\n",
      "      double val = a;\n",
      "      while (i < n){val *= a;};\n",
      "      return val;\n",
      "    };"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "!rm -r jedgar jehbuild build"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Hoover Example\n",
      "\n",
      "### jeh_xdressrc.py\n",
      "\n",
      "    package = 'jedgar'     # top-level python package name\n",
      "    packagedir = 'jedgar'  # location of the python package\n",
      "    sourcedir = 'jehsrc'   # location of C/C++ source\n",
      "    builddir = 'jehbuild'  # build and temp dir\n",
      "\n",
      "    classes = [\n",
      "        ('A', 'hoover'),\n",
      "        ]"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "### Run & Build"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "xdress --rc jeh_xdressrc.py\n",
      "python jeh_setup.py build > /dev/null 2>&1"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "autoall: discovering API names\n",
        "autoall: no API names to discover!\n",
        "autodescribe: scraping C/C++ APIs from source\n",
        "autodescribe: describing A\n",
        "autodescribe: registering A\n",
        "cythongen: creating C/C++ API wrappers\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Hoover Example\n",
      "\n",
      "### jedgar/hoover.pyx\n",
      "\n",
      "    from libc.stdlib cimport free\n",
      "\n",
      "    cdef class A:\n",
      "        \"\"\"no docstring for A, please file a bug report!\"\"\"\n",
      "\n",
      "        # constuctors\n",
      "        def __cinit__(self, *args, **kwargs):\n",
      "            self._inst = NULL\n",
      "            self._free_inst = True\n",
      "\n",
      "        def __init__(self):\n",
      "            \"\"\"A(self)\"\"\"\n",
      "            self._inst = new cpp_hoover.A()\n",
      "\n",
      "        def __dealloc__(self):\n",
      "            if self._free_inst:\n",
      "                free(self._inst)\n",
      "\n",
      "        # attributes\n",
      "        property a:\n",
      "            \"\"\"no docstring for a, please file a bug report!\"\"\"\n",
      "            def __get__(self):\n",
      "                return float((<cpp_hoover.A *> self._inst).a)\n",
      "\n",
      "            def __set__(self, value):\n",
      "                (<cpp_hoover.A *> self._inst).a = <float> value\n",
      "\n",
      "        # methods\n",
      "        def power(self, n=1):\n",
      "            \"\"\"power(self, n=1)\n",
      "            no docstring for power, please file a bug report!\"\"\"\n",
      "            cdef double rtnval\n",
      "            rtnval = (<cpp_hoover.A *> self._inst).power(<int> n)\n",
      "            return float(rtnval)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "builddir = os.path.abspath('build/lib.linux-x86_64-2.7/')\n",
      "if builddir not in sys.path:\n",
      "    sys.path.insert(0, builddir)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Hoover Example"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from jedgar.hoover import A\n",
      "\n",
      "a = A()\n",
      "a.a = 10\n",
      "a.power(42)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "9.999999999999999e+41"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``cythongen`` can get much more complicated..."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<div style=\"text-align:center;\"><img src=\"cry_havoc.jpg\" width=\"50%\" /></div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "subslide"
      }
     },
     "source": [
      "## Function Pointer Callbacks\n",
      "\n",
      "### TwoNums Struct in C\n",
      "\n",
      "    typedef struct TwoNums\n",
      "    {\n",
      "      double a;\n",
      "      double b;\n",
      "      double (*op)(double, double);\n",
      "    } TwoNums;\n",
      "    \n",
      "### Sidecar \n",
      "\n",
      "    mod = {'TwoNums': {\n",
      "        'extra': {\n",
      "            'max_callbacks': 2,\n",
      "            'pyx': \"\"\"\n",
      "        def call_from_c(self, x, y):\n",
      "            rtn = (<c_pydevice.TwoNums *> self._inst).op(<double> x, <double> y)\n",
      "            return float(rtn)\n",
      "            \"\"\"},},}\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "d = os.path.abspath('/home/scopatz/xdress/tests/build/build/lib.linux-x86_64-2.7/')\n",
      "if d not in sys.path:\n",
      "    sys.path.insert(0, d)\n",
      "import xdtest.pydevice as mod"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "subslide"
      }
     },
     "source": [
      "## Function Pointer Callbacks"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = mod.TwoNums(5, 10)\n",
      "print \"a and b = \", x.a, x.b\n",
      "\n",
      "f = lambda a_, b_: a_ + b_\n",
      "print \"f's refcount before assignment: \", sys.getrefcount(f)\n",
      "x.op = f\n",
      "print \"f's refcount after assignment: \", sys.getrefcount(f)\n",
      "print \"x.op = \", x.op\n",
      "\n",
      "# Test that function pointer in C is getting set properly\n",
      "x._op = None\n",
      "print \"x._op is None:\", x._op is None\n",
      "print \"f's refcount after None assignment: \", sys.getrefcount(f)\n",
      "print \"x.op = \", x.op\n",
      "\n",
      "v = x.op(14, 16)\n",
      "print \"result of x.op(14, 16) = \", v\n",
      "v = x.call_from_c(14, 16)\n",
      "print \"result of x.call_from_c(14, 16) = \", v"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "a and b =  5.0 10.0\n",
        "f's refcount before assignment:  2\n",
        "f's refcount after assignment:  4\n",
        "x.op =  <function <lambda> at 0x3ca3668>\n",
        "x._op is None: True\n",
        "f's refcount after None assignment:  3\n",
        "x.op =  <cyfunction TwoNums.op.__get__.<locals>.op_proxy at 0x3acf410>\n",
        "result of x.op(14, 16) =  30.0\n",
        "result of x.call_from_c(14, 16) =  30.0\n"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "subslide"
      }
     },
     "source": [
      "## Function Pointer Callbacks"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x.op = lambda a_, b_: 2*a_ + b_**2\n",
      "print \"x.op = \", x.op\n",
      "print \"f's refcount after op re-assignment: \", sys.getrefcount(f)\n",
      "\n",
      "v = x.op(14, 16)\n",
      "print \"result of x.op(14, 16) = \", v\n",
      "v = x.call_from_c(14, 16)\n",
      "print \"result of x.call_from_c(14, 16) = \", v"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "x.op =  <function <lambda> at 0x3ca3758>\n",
        "f's refcount after op re-assignment:  3\n",
        "result of x.op(14, 16) =  284\n",
        "result of x.call_from_c(14, 16) =  284.0\n"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "subslide"
      }
     },
     "source": [
      "## Function Pointer Callbacks"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# test two instances\n",
      "y = mod.TwoNums(5, 10)\n",
      "x.op = lambda a_, b_: a_ + b_\n",
      "print \"x.op = \", x.op\n",
      "y.op = lambda a_, b_: 2*a_ + b_**2\n",
      "print \"y.op = \", y.op\n",
      "v = x.op(14, 16)\n",
      "print \"result of x.op(14, 16) = \", v\n",
      "v = x.call_from_c(14, 16)\n",
      "print \"result of x.call_from_c(14, 16) = \", v"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stderr",
       "text": [
        "-c:3: RuntimeWarning: Ran out of available callbacks for TwoNums.op, overriding existing callback.\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "x.op =  <function <lambda> at 0x3ca3848>\n",
        "y.op =  <function <lambda> at 0x3ca36e0>\n",
        "result of x.op(14, 16) =  30\n",
        "result of x.call_from_c(14, 16) =  30.0\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stderr",
       "text": [
        "-c:5: RuntimeWarning: Ran out of available callbacks for TwoNums.op, overriding existing callback.\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "subslide"
      }
     },
     "source": [
      "## Function and Method Dispatch\n",
      "\n",
      "    def _func_0():\n",
      "        \"\"\"func()\n",
      "        I am a weird function.\"\"\"\n",
      "        cdef int rtnval\n",
      "        rtnval = cpp_reprocess.func()\n",
      "        return int(rtnval)\n",
      "\n",
      "    def _func_1(x, y=10):\n",
      "        \"\"\"func(x, y=10)\n",
      "        I am a weird function.\"\"\"\n",
      "        cdef int rtnval\n",
      "        rtnval = cpp_reprocess.func(<double> x, <int> y)\n",
      "        return int(rtnval)\n",
      "\n",
      "    def _func_2(x, y=1.0e+1):\n",
      "        \"\"\"func(x, y=1.0e+1)\n",
      "        I am a weird function.\"\"\"\n",
      "        cdef char * x_proxy\n",
      "        cdef int rtnval\n",
      "        x_bytes = x.encode()\n",
      "        rtnval = cpp_reprocess.func(std_string(<char *> x_bytes), <double> y)\n",
      "        return int(rtnval)\n",
      "\n",
      "    _func_0_argtypes = frozenset()\n",
      "    _func_1_argtypes = frozenset(((0, float), (1, int), (\"x\", float), (\"y\", int)))\n",
      "    _func_2_argtypes = frozenset(((0, str), (1, float), (\"x\", str), (\"y\", float)))\n",
      "\n",
      "    def func(self, *args, **kwargs):\n",
      "        \"\"\"func(x, y=1.0e+1)\n",
      "        I am a weird function.\"\"\"\n",
      "        types = set([(i, type(a)) for i, a in enumerate(args)])\n",
      "        types.update([(k, type(v)) for k, v in kwargs.items()])\n",
      "        # vtable-like dispatch for exactly matching types\n",
      "        if types <= self._func_0_argtypes:\n",
      "            return self._func_0(*args, **kwargs)\n",
      "        if types <= self._func_1_argtypes:\n",
      "            return self._func_1(*args, **kwargs)\n",
      "        if types <= self._func_2_argtypes:\n",
      "            return self._func_2(*args, **kwargs)\n",
      "        # duck-typed dispatch based on whatever works!\n",
      "        try:\n",
      "            return self._func_0(*args, **kwargs)\n",
      "        except (RuntimeError, TypeError, NameError):\n",
      "            pass\n",
      "        try:\n",
      "            return self._func_1(*args, **kwargs)\n",
      "        except (RuntimeError, TypeError, NameError):\n",
      "            pass\n",
      "        try:\n",
      "            return self._func_2(*args, **kwargs)\n",
      "        except (RuntimeError, TypeError, NameError):\n",
      "            pass\n",
      "        raise RuntimeError('method func() could not be dispatched')"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## C++ Standard Library Container Wrappers\n",
      "\n",
      "Another code generator on top of the type system are idomatic wrappers which provide **views** into C++ containers."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "Currently this only includes ``map``, ``set``, and ``vector`` but others would be trivial to add."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "### xdressrc.py (what the user writes)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "stlcontainers_module = 'conts'\n",
      "stlcontainers = [\n",
      "    ('vector', 'float64'),\n",
      "    ('vector', 'str'),\n",
      "    ('vector', 'int32'),\n",
      "    ('vector', ('vector', 'float64')),\n",
      "    ('set', 'int'),\n",
      "    ('set', 'str'),\n",
      "    ('set', 'char'),\n",
      "    ('map', 'str', 'str'),\n",
      "    ('map', 'str', 'int'),\n",
      "    ('map', 'str', 'float'),\n",
      "    ('map', 'int', 'int'),\n",
      "    ('map', 'int', 'float'),\n",
      "    ('map', 'int', ('set', 'int')),\n",
      "    ('map', 'int', ('set', 'str')),\n",
      "    ('map', 'int', ('vector', 'str')),\n",
      "    ('map', 'int', ('vector', 'float')),\n",
      "    ('map', 'int', ('vector', ('vector', 'float64'))),\n",
      "    ('map', 'int', ('map', 'int', 'char')),\n",
      "    ('map', 'int', ('map', 'int', 'float')),\n",
      "    ('map', 'int', ('map', 'int', ('vector', 'float'))),\n",
      "    ]"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``std::map<T, U>`` Wrappers"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import xdtest.xdstlc as conts"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print [m for m in dir(conts) if m.startswith('Map')]"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['MapIntBool', 'MapIntChar', 'MapIntComplex', 'MapIntDouble', 'MapIntInt', 'MapIntMapIntBool', 'MapIntMapIntChar', 'MapIntMapIntDouble', 'MapIntMapIntVectorBool', 'MapIntMapIntVectorChar', 'MapIntMapIntVectorDouble', 'MapIntSetChar', 'MapIntSetInt', 'MapIntSetStr', 'MapIntSetUInt', 'MapIntStr', 'MapIntVectorBool', 'MapIntVectorChar', 'MapIntVectorDouble', 'MapIntVectorInt', 'MapIntVectorStr', 'MapIntVectorUInt', 'MapIntVectorVectorDouble', 'MapStrDouble', 'MapStrInt', 'MapStrStr', 'MapStrUInt', 'MapUIntDouble', 'MapUIntStr', 'MapUIntUInt']\n"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<br />"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores = conts.MapStrInt({'Joan': 42, 'Leslie': 6})\n",
      "print scores\n",
      "print \"scores is mapping:\", isinstance(scores, collections.MutableMapping)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{u'Joan': 42, u'Leslie': 6}\n",
        "scores is mapping: True\n"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<br />"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores['Leslie'] = 14\n",
      "print scores['Joan'] / scores['Leslie']"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "3\n"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Bad Types Fail\n",
      "\n",
      "<br /><br />"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores[\"Hoover\"] = [1.92e3, \"hell0\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "TypeError",
       "evalue": "an integer is required",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
        "\u001b[1;32m<ipython-input-40-117d2e157c41>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mscores\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"Hoover\"\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;36m1.92e3\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"hell0\"\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[1;32m/home/scopatz/xdress/tests/build/build/lib.linux-x86_64-2.7/xdtest/xdstlc.so\u001b[0m in \u001b[0;36mxdtest.xdstlc._MapStrInt.__setitem__ (/home/scopatz/xdress/tests/build/xdtest/xdstlc.cxx:20723)\u001b[1;34m()\u001b[0m\n",
        "\u001b[1;31mTypeError\u001b[0m: an integer is required"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## ``std::vector<T>`` Wrappers Are NumPy Arrays!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Vectors are wrapped by providing a 1D numpy view into the vector's memory with a custom dtype that represents the template type."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print conts.xd_double.name\n",
      "print \"Is dtype?\", isinstance(conts.xd_double, np.dtype)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "xd_double\n",
        "Is dtype? True\n"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<br />**Numpy array of ``std::string``s**"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([\"Joan\", 'Hoover', 'Mulan', 'Leslie'], dtype=conts.xd_str)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "array([u'Joan', u'Hoover', u'Mulan', u'Leslie'], dtype='xd_str')"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<br />**And now for something truly evil...**"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([[42, 65], [6, 28, 496, 8128]], dtype=conts.xd_vector_double)"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "array([array([ 42.,  65.]),\n",
        "       array([  6.00000000e+00,   2.80000000e+01,   4.96000000e+02,\n",
        "         8.12800000e+03])], dtype='xd_vector_double')"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## C++ Standard Library Container Wrappers\n",
      "\n",
      "All of these wrappers are composable with other types, including classes and structs wrapped from ``cythongen``.\n",
      "\n",
      "<br /><br />"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "yolo = conts.MapIntMapIntVectorBool({\n",
      "    42: {65: [1, 0, False, True]}, \n",
      "    18: {}, \n",
      "    2040: {3: [\"yes\", \"no\"]}})\n",
      "print yolo\n",
      "print \"Value is MapIntVectorBool?\", isinstance(yolo[42], conts.MapIntVectorBool)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{18: {}, 42: {65: array([ True, False, False,  True], dtype=bool)}, 2040: {3: array([ True,  True], dtype=bool)}}\n",
        "Value is MapIntVectorBool? True\n"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Plugins\n",
      "\n",
      "XDress has a plugin architeture which makes writing your own code generators super easy!"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from xdress.plugins import Plugin\n",
      "\n",
      "class XDressPlugin(Plugin):\n",
      "    requires = ('xdress.base',)\n",
      "\n",
      "    defaultrc = {'choices': ['J. Edgar Hoover', 'Hua Mulan', 'Leslie'],\n",
      "                 'answer': 'Joan of Arc',}\n",
      "        \n",
      "    def update_argparser(self, parser):\n",
      "        parser.add_argument('-c', '--choices', action='store', dest='choices',\n",
      "                            nargs=\"+\", help=\"famous people chocies\")\n",
      "        parser.add_argument('-a', '--answer', action='store', dest='answer',\n",
      "                            help=\"person who was executed\")\n",
      "\n",
      "    def setup(self, rc):\n",
      "        if 'Joan of Arc' not in rc.choices:\n",
      "            rc.choices.append('Joan of Arc')\n",
      "\n",
      "    def execute(self, rc):\n",
      "        if rc.answer == 'Joan of Arc':\n",
      "            print('Joan has met an untimely demise!')\n",
      "        else:\n",
      "            raise ValueError('Joan of Arc was executed, not ' + rc.answer)\n",
      "\n",
      "    def report_debug(self, rc):\n",
      "        return \"the possible choices were \" + str(rc.choices)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Cool Ideas\n",
      "\n",
      "<div style=\"text-align:center;\"><h3>Fix/Rewrite the Clang AST Python Bindings</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><h3>Create a Better (Than RelaxNG) Validation Engine</h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "fragment"
      }
     },
     "source": [
      "<div style=\"text-align:center;\"><h3>Wrap HDF5 - <a href=\"https://github.com/scopatz/cya\">cya project</a></h3></div><br/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Questions\n",
      "\n",
      "<div style=\"text-align:center;\"><b>http://xdress.org/</b><br /><img src=\"magician.jpg\" width=\"30%\" /><br /><b>xdress@googlegroups.com</b></div>"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "!make"
     ],
     "language": "python",
     "metadata": {
      "slideshow": {
       "slide_type": "skip"
      }
     },
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "cp -r ~/nbconvert/reveal . \r\n",
        "rm -rf reveal/.git* \r\n",
        "mkdir -p js \r\n",
        "cp ~/nbconvert/js/mathjax-onload.js js\r\n",
        "cp classy.css reveal/css/theme/ \r\n",
        "python ~/nbconvert/nbconvert.py -f reveal scopatz_scipy2013.ipynb \r\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "sed -i 's:reveal/css/theme/simple.css:reveal/css/theme/classy.css:' scopatz_scipy2013_slides.html \r\n",
        "sed -i 's:class=\"fragment\" class=\"text_cell_render:class=\"fragment text_cell_render:' scopatz_scipy2013_slides.html \r\n",
        "sed -i 's/.rendered_html ul{list-style:disc;margin:1em 2em;}/.rendered_html ul{list-style:disc;margin:0em 2em;}/' scopatz_scipy2013_slides.html \r\n",
        "sed -i '/<\\!-- Social buttons -->/,/<\\!-- End of social buttons -->/d' scopatz_scipy2013_slides.html \r\n",
        "sed -i 's/background: #3F3F3F;/background: #4A363D;/' reveal/lib/css/zenburn.css \r\n"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 22
    }
   ],
   "metadata": {}
  }
 ]
}