HW4-apm382_Townsend_Griffin.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <p style=\"text-align: center;\">MIS 382N: Advanced Predictive Modeling</p>\n",
    "# <p style=\"text-align: center;\">Assignment 4</p>\n",
    "## <p style=\"text-align: center;\">Total points: 50 </p>\n",
    "## <p style=\"text-align: center;\">Due: Mon, November 14</p>\n",
    "## <p style=\"text-align: center;\">Team: DavisTownsend & Dallas Griffin\n",
    "\n",
    "\n",
    "\n",
    "Your homework should be written in a **Jupyter notebook**. Please submit **only one** ipynb file from each group, and include the names of all the group members. Also, please make sure your code runs and the graphics (and anything else) are displayed in your notebook before submitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 1 - MNIST (15 pts)\n",
    "In this question you classify hand-written digits. We want to use MNIST data set and try Multi-layer Perceptron (MLP) classifier using sklearn package in Python. In order to simplify the problem, we classify digits into 8 classes (digits 0,1,2,...7) and ignore records for digits 8 and 9.  Use the code below to access the data set and extract data with labels 0 to 7, and split the data set into train set and test set.\n",
    "\n",
    "1. Fit a Multilayer Perceptron Classifier using the standard options on sklearn's MLP on train data. Report the root MSE for both train and test data. (5 pts)\n",
    "\n",
    "    Use these parameters for your model: \n",
    "                    {hidden_layer_sizes=(50,), max_iter=10, alpha=1e-4,\n",
    "                    solver='sgd', verbose=10, tol=1e-4, random_state=42,\n",
    "                    learning_rate_init=.1}\n",
    "                    \n",
    "2. To find better parameters for the MLP Classifier model, try an exhaustive search over all parameters of the data. Use sklearn's GridSearchCV to find the best subset of parameters from the set:\n",
    "                    { alpha = [0.1,0.01,0.001], activation : ['logistic', 'relu'] }\n",
    "    which parameters resulted in a more accurate model? Can you explain why? (5 pts)\n",
    "\n",
    "3. Select 5 misclassified images and display them. (5 pts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: pylab import has clobbered these variables: ['clf']\n",
      "`%matplotlib` prevents importing * from pylab and numpy\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import fetch_mldata\n",
    "from sklearn.model_selection import (train_test_split,KFold)\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn import metrics\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "\n",
    "%pylab inline\n",
    "\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "\n",
    "mnist = fetch_mldata(\"MNIST original\")\n",
    "# rescale the data, use the traditional train/test split\n",
    "X = (mnist.data / 255.)[:48200]\n",
    "y = mnist.target[:48200]\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.1, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1 - Fit a MLP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 0.24522635\n",
      "Iteration 2, loss = 0.10589741\n",
      "Iteration 3, loss = 0.07488529\n",
      "Iteration 4, loss = 0.05895084\n",
      "Iteration 5, loss = 0.04782934\n",
      "Iteration 6, loss = 0.03945315\n",
      "Iteration 7, loss = 0.03320940\n",
      "Iteration 8, loss = 0.02872414\n",
      "Iteration 9, loss = 0.02451143\n",
      "Iteration 10, loss = 0.02132893\n"
     ]
    }
   ],
   "source": [
    "MLP = MLPClassifier(hidden_layer_sizes=(50, ), max_iter=10, alpha=.0001, \n",
    "                    solver='sgd', verbose=10, tol=.0001, random_state=42, learning_rate_init=.1)\n",
    "\n",
    "#Fit model & predict y\n",
    "MLP.fit(X_train, y_train)\n",
    "pred_y_train = MLP.predict(X_train)\n",
    "pred_y_test = MLP.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Precision:0.99563632295\n",
      "Recall: 0.995620101429\n",
      "Accuracy: 0.995620101429\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x16696898>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUUAAAEZCAYAAAAAODfRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8U1UbwPFf0tJJAQEnU1SOioobB1KGEwfiwC1LlgwB\nBYFKLaMMEQREKKNs90JBxfGqLBFRkc0DLlAoIEUobYGSJu8fN8VYS5u2ubGpz9dPPjaX3PucO/Lk\n3HHOcXg8HpRSSlmc/3YBlFKqLNGkqJRSPjQpKqWUD02KSinlQ5OiUkr50KSolFI+wv/tAoQSY0wd\n4BdgqYg0zfdvs4C2QHUgDtggInEFLONZoDvwO+DB2gd7gO4isq0YZXkE6OtdRh3gMPCH931PEVnh\n53JOB94UkcZFfG4R8JSIbPG3jIUsKyDbwLustsBQYJOI3FLasimlSbH4jgD1jTG1ROQ3AGNMDHAt\n1hc8T2EPgL4mIr3y3hhjegCvAFf4WwgRmQfM884/C1gvIuP8Xou/lpMGFJoQvZ+7rbjLLkKpt4HX\no8BAEXklkIVT/12aFIsvF3gdeBgY6Z12F/AeVs2tJP4HjCh90f5ijPkC2A8YYArwLfAcEAGcDnwq\nIp28td8NIhLnrcHV9f57HayaZxsR2W2M+QW4G6sWnAz8DFzgXV53EVlijKkOzALqAelYtb/1IjLU\njyIf3wbGmErABO/yK3j/rZ+IuI0xR7C2dUNgF1YSrWuMOdkb+yXgYsANLMZKmL7zXYS175YDLwC3\nedepP3AvcCGwE7hdRA4bYzoAnb3lqAqMEpGp3hpqa2+cc4CjwKMisskYcyqQApyLdbxMFZEXC1sv\nP7aPChK9plh8HmAu1hcrT1usL6SjuAszxoQBHYEvAlK6v9svIheIyEtAL2CwiFwNNABaGWMu8X7O\nt1bbGLhbRM4D/gS6FLDcK4ExInIpMBNI8k5/ESvBNgDaANf4U0ifbfC5d9ILwLcicgVwKXAyf/3g\nRADvici5ItIcK9k/JSITgInAPhG5ELgcK3E+lW++80TkOyAS2CkiF2H9aEz3bqPzgSre7RPrLdct\nInIZcD8wxqfoTbB+EC4EvgL6eadPAcS7Da8BOhlj6p1gvZ70Zxup4NGaYgmIyBpjjNubVP4AKnpr\nCP4u4n5jTGOsJFoB+A7oZENRl/n83Q5oaYwZiFWDiQYqYtUmfX0pIlnev9dg1Y7y2y4i671/f4/1\nowBwC3AJgLd2+VYhZStoG3T2/tttwBXGmMe876OwamR5ludbVt6P0S14E7GIHDPGpABPYNWQC5rv\nHe//f8Kq0e4G8NaKq4pIljHmduA2Y8w5WDXQWJ/5v/NefsjbDq29f7fAm4xFJAOrdooxpqD10na2\nZYwmxZKbBzyClRTnFXPev11POxFjzHSsGo8HSBGRacWMk+nz93KsJLcYeANoRME128M+f3uK+RlX\nvs/nFlK2wraBE7hXRATAGFOZvyfFzALn+ueZjxMr4Z5ovqM+fx/LvzBjTA1gJTAV6wfmLeBWn48U\nth2OJztjzJnAPv65XpXQpFjm6Olz8eUd+POxrkG1wbpBcKLPlZiIdBKRS0Tk0hIkxOOMMVWwTtee\nFpEFQE3gbCCstGXMZxHW6SbGmGpYNaeSfOk/xnu6bIyJBN4Hevgx32Ksu9p583UGPilB/DyXA3tF\nJFlEPgVu9y67qH37KdDe+9nKWNcOz+af67UQ/9ZLBZHWFIvPAyAiu4wxm4ADInLA99+8YowxGd6/\nHd5/u9rOMp1omogcMMaMBNYYY/Zh1VqWY31Rfy5FjPz6AjOMMWuxbrT8CmT7uXxfTwDjjTHrsY7R\nT/nrFDh/OTz55nvRO18FrCQ5wo/58sv7t4+BDsYYwaplfoN1ZnB2EeXvCUzxbgcHkOy95FLYeqky\nwqFdh6lAMcZ0A74XkVXGmAisU85EEfn4Xy6aUn7TmqIKpE3AJO/d5ArAG5oQVajRmqJSSvnQmqJS\nqswzxgwA7sA6A5kMLAVmYz2VsEFE8m6wdcK6wXYM61ruB8aYKKwbo6cAGUBbEUk/USy9+6yUKtOM\nMfHA1SJyDdAUqA2MAwaJSDzgNMa08rYk6ol1Q/NmYKQxpgLQDVgnIk2wHp8bXFi8MltTvPOSR4J6\nXv/GkvHBDKdUmRVRqVqpHye7qE6839/fdduXFBXvJmCDMWYBfzXJfExE8honfATciFVrXC4iLiDD\nGLMNq1VTY2C0z2dDMykqpUKXw1HqvOqrOlbt8DasdvXv8/ez3ENAJayEedBneiZQOd/0vM+ekCZF\npVTAORwBvTKXDmz21gC3ejv3qOnz73HAAazrhZXyTf/TOz0u32dPSK8pKqXKuuVY1wgxxpyB1f78\nf95rjWC1eV8GrAYaG2MivC2JzgU2YHXW0dL72Zb8vU+Af9CaolIq4MICWFP03kG+zhjzDVYLoW5Y\nraVmeG+kbAbeEhGPMWYiVhJ1YN2IyTHGTAHmGGOWYbV3f7CweJoUlVIB5wzs6TMiMqCAyU0L+Fwq\nkJpv2mGsPgr8oklRKRVwAb7RElR6TVEppXxoTVEpFXCO0vec96/RpKiUCrhAX1MMppBJimNfHkpW\nptXR8d5df7Bg7oc8/kwHAHbt2M1LQ1PxeDzc/tDNNL6xER48fL98LW9Mf+/4MmrUPZ3n5j5L2+bd\ncbkK6xTaP+s2bGT8pCnMTJlU6mX5I33/fu5/tCPTX5pA3Tq1bYvj8XgYPvp5ZOs2IiMjSEoYSK2a\nNWyL53a7SUoexa/bd+BwOkgc0J+z6p1pW7w8wdx/M2bP5culy3G5XNx3z120viPQgyP+U7CPT1+h\nfE0xJJJieAWrmIldRh6fNmDsE8yd+Dpb1m6jZ1Inroi/hF+3/sZ1N19F/0eSABgx8xm+/vw7dvz0\nO1ExUbTr8wA5R//R63yJzJr3Mgs/XExMdHRAllcUl8vFsJFjiIqKtD3W518uJScnh/kzp7Fuw0bG\njJ/IxOdHFz1jCX25bDkOh4O5M1JY/d0aJkxOsTUeBHf/rf5uDWvXb2D+zGlkHz7MnPmv2h4z2Mdn\nfs4QTopBqeMaY0oV58z6tYmMjuTZl/oxJOVpzrngLEY9OYEta7cRHh5GleqVyT6Uzb7d+xja/a/B\n1sLDw8nJyQHg8cEdmP/iGxw9klO6lfGqXbMmE8aMLPqDATJ2wiTa3NOaU06ubnus739Yy7VXXwXA\nRRc0YOPmLbbGax7fhGcHPQ3ArrQ0KsXFFTFH6QVz/3319SrOOasevZ56ml59+xN/3bW2xwz28Zmf\nA6ffr7LGthIZY+oZYxYYY34HfjbG7DDGfGCMqV/cZR09ksOCuR8ypPsYUkbMpm9yNxwOB9VPq8aE\nt0YSV7kiv2zdgdvtITPDGoiube/7+WnLr+z+bS/3d2nNt0t/YPuPvwesWt+iWTxhYYEe4qRgCxZ+\nQNWqJ3FNoysJRveXWVlZxFX8a9C68LAw3G57hyZ2Op0kJA1n9Njx3HrzTbbGguDuvz8PHGDTZmHc\nqGSeGdCPp5951vaYwVy/gjgcDr9fZY2daXoGMFJEaopIXRGpDQzDGh+5WHZtT2PJh18BkLZjD4cO\nZnJS9Srs251O9zv788nbX9DxKWsY5vAK4fRJ7kZUdCRTR8wGoEnLa7j+zniGTRtIlWqVSZrydIBW\nMTgWLPyAlatW06FrD7Zs3UZC0jDS9+cfmTRwYmNjycr+a2gVt9uD02n/L3py0jMsfPs1kpJHcuTI\n0aJnCBFVKlfmmqsbER4eTt06tYmMiOTPA4U2vw15TofD71dZY+eRHiUiq3wniMjXJVlQizvjad/X\naplz0slViI6Notsz7Tmt1ikAHM46fLwmkzC+D7/IdqaOnHN8/sdb9SOxy0gGdx7JgfSDPNstcNer\nPEEYoXL2tMnMTJnEzJRJnFv/HJKTBlOtakHDMQfGJQ0vYtmKlQCsXb+Bc86uZ1ssgIUfLmbG7LkA\nREZE4nQ6cTqD82UJxv679OKLWLHSOvT3/vEHR44eoUrlyrbHheCsX0EcxfivrLHzRstaY8xMrBHV\nDmL1TtESWFfcBX22YAk9kzoxIvUZ3B4PLyZNB6DXkM64clwcPZLDpKEzaNT0Ms6/xBAWHs5ljRvi\n8cC8F99g24afji/L4/EEdDcEe6cG44e1RbN4Vn6zmkc6dgFgWGKCrfGub96UwUOSadf5cXJzcxnw\nZB8iIiJsjZknGPuvSeNr+W7NWh5o2xGPx0NC/6eCdtpYFpNOWWfbGC3esXHvxOrgsRJW9z0rgHdF\npMig2smsUv+OQHQy2+L8u/3+/v5v09tlKnPbVlP0Jr53vS+l1H9IWBCuQdslJJ5TVEqFllA+bQ/d\ndK6UUjbQmqJSKuC07bNSSvkoiw9l+0uTolIq4MriQ9n+0qSolAq4UL7RoklRKRVwevqslFI+9PRZ\nKaV86OmzDYLd7O7yC+8Karxv178T1HhKBVMoP5ITuiVXSikblNmaolIqdOmNFqWU8hEWwqfPmhSV\nUgEXynefQzedK6WUDbSmqJQKOL2mqJRSPkL59FmTolIq4PThbaWU8qE1xTLA4/EwfPTzyNZtREZG\nkJQwkFo1a5R4eR26PUjTG64lPDyc1+ct4LtVaxk+diBut5sft/7CiMFWi5v+iT24+PILyc6yxknu\n9dggXK5cRo5PoGq1k8jMzOKZviM5eCCjxGVxuVwkDhvBzrQ0XMdcdGrflqZNGpd4eUUJ9LYsSnlf\nvzzrNmxk/KQpzEyZZGsct9tNUvIoft2+A4fTQeKA/pxV70xbY+YXytcUy83d58+/XEpOTg7zZ07j\nie7dGDN+YomXdVmjhjS8tAGP3tWdjvc9welnnEK/wd2Z+Nx0Otz3BE6Hk6Y3XAvA+Rcauj7yFI89\n0IfHHuhDdtZh2jzciq2bf6Z9m14seucTuvR6tFTrtuijj6lSpTJzpk1h8oSxjBgztlTLK0ogt6U/\nyvv6Acya9zJJyaPIycmxPdaXy5bjcDiYOyOFHl06M2Fyiu0x8zvRwPcFvcqacpMUv/9hLddefRUA\nF13QgI2bt5R4Wdc2uZIft/7C+GnDmZg6giX/W8l5F9Tn+9XWkNXLv/yaqxpfDkDtujV4dtRTzH7r\nRVrdewsAl15xISuWrPJ+dhVXNb6sNKvGTTe0oEfXzgB43B7Cw+2t4AdyW/qjvK8fQO2aNZkwZqTt\ncQCaxzfh2UFPA7ArLY1KcXFBievrRAPfF/RfWVNuTp+zsrKIqxh7/H14WBhutxtnCYZarFK1Mqef\ncQo9OgykZu3TmThjBA7nXzsvK+swcXGxREdH8crsd5g74w3CwsKY8eoLbFonxFaMJfNQlvXZzGxi\nfcpVEtFRUcfX8cmBCfTq1qVUyytKILelP8r7+gG0aBbPrrQ025afn9PpJCFpOF8sWcrYUclBi3s8\nfhmsAfqr3NQUY2NjycrOPv7e7faU+CA/8OdBVixdTW5uLtt/+Z2jR3OIi/vrSxQbG01GRiaHDx/h\n5Vlvk3M0h8PZh1m9cg31zz+LzEOZxMTGWJ+tGMOhjMzSrRywe/ceOnbryR23tuTmG68v9fIKE8ht\n6a/yvn7/huSkZ1j49mskJY/kyJGj/3ZxQoZtR4Ix5gtjzFf5XiuNMV/ZEe+ShhexbMVKANau38A5\nZ9cr8bLWfLuea+OvBODkU6oRHRPNqhXfc1mjhgA0bnoV33+zjrpn1Wbu29ZF8/DwMC654kI2rd/K\nmm830KS5dXp2XbOrjp92l9S+9P106dWHvr260+q2lqValj8CuS39Ud7Xz5cHj+0xFn64mBmz5wIQ\nGRGJ0+nE6Qxuzc3hcPj9KmvsPH0eAEwHWgMuG+MA1unJym9W80hH69RrWGJCiZe17POvufSKi3j5\nvRQcDgfDE8ax6/fdJI3uR3iFcH75cTuffvglAAvf+ZiX30vh2LFjvP/Wx/zy43Z2/ZbG8HGDmP3m\ni+Tk5DCg17BSrVvq7LkcOpTJ1NRZpMyYhcMBUyaMIyIiolTLPZFAbkt/lPf18xWMa2jXN2/K4CHJ\ntOv8OLm5uQx4so9t2/JEQvn02eHx2PfLZYzpB/woIu8Wd96cjHT7f1J9aCezSlkiKlUrdUZ7vElv\nv7+/k5eOL1MZ1NYbLSIyxs7lK6VUoJWbu89KqbIjyJcwA0qTolIq4MriDRR/aVJUSgVcKN9o0aSo\nlAq4UK4plr8nVpVSqhS0pqiUCjgduEoppXwE+pqiMeYU4FvgeiAGWARs9f7zFBF50xjTCegMHAOS\nReQDY0wUMB84BcgA2opIemGxNCkqpQIukDnRGBMOpAB5DdYvA8aKyAs+nzkV6AlcipU0lxtjPgG6\nAetEZKgx5j5gMNC7sHiaFJVSZd3zwBRgoPf9ZUB9Y8ydWLXFPsCVwHIRcQEZxphtQEOgMTDaO99H\nWEmxUJoUvYLd7O6eJk8ELdZbSycELZZSELjTZ2NMO2CviHxqjBkEOIBVwHQRWWOMGQg8C/wAHPSZ\nNROoDMT5TD8EVCqy7AEpuVJK+QhgJ7PtgRuMMV8AFwNzgI9EZI333xd4px/k7wkvDvgT6zpinM+0\nA0UF1JqiUirgAvWcoojE5/1tjPkc6Aq8b4zpKSKrgRbAd8BqINkYEwFEA+cCG4CvgJZYN2laAsuK\niqlJUSkVcGH2Nn7uCkwyxuQAu4HOIpJpjJkILMc6xR4kIjnGmCnAHGPMMuAo8GBRC9ekqJQKCSLS\n3OftP4Z7FJFUIDXftMNAm+LE0aSolAo4bfuslFI+yuIoff7SpKiUCjitKSqllI8QzonlJym6XC4S\nkoazKy2NsLAwkhIGULdObVtirduwkfGTpjAzZRJbZCsjx75AeFgYFSpEMGLIYKqedFKxlznulWFk\nZ1qtmPbs/IN3535I98EdAUjbsZsXh8zA4/HwWL+HOe/i+hzOOgxAcu8XOJx9BIAadU9nzLwkHm32\nOC5XbqnXLc8Hiz/h1TfeYv7MaSVaZlFcLheJw0awMy0N1zEXndq3pWmTf1xH13h++Df2X0FCueuw\ncpMUl61Yidudy7zUqaxctZqJk1MYN3pEwOPMmvcyCz9cTEx0NACjx00gof9T1D/7LN58ZwGps+fR\nr0+vYi0zvIK1G57pPPL4tIHjejN34mts/mEbvYZ05sr4S1j15fecdd6ZPNttNJkZWX9bRnRMFB36\nPsixo8cCtm4Am0V49/1FJV6mPxZ99DFVqlRmxJBEDmZkcO9D9iap8hrv39p/BQnl0+ciW7QYYy4K\nRkFKq06dWuTm5uLxeMjMyiS8QgVb4tSuWZMJY/5KXmNGDKP+2WcBkJubS2RUZLGXeaapTVR0JEmT\n+zM0ZQD1LziLkX3Hs/mHbYSHh3FStcpkZVo1wzNqn0r3wR0ZNWswLVo1Ob6M7oM7Mnfi6xw9khOw\ndTtw4CAvTpnGgCcLbT9fajfd0IIeXTsD4HF7CA+397e6vMb7t/ZfQRwO/19ljT9753XgvEAEM8ZE\nisjRQCwrv5joGH7flcYd9zzAgYMHmfSCPQMJtmgWz660tOPvq1erCsAPa9fz2pvvMHva5GIv8+jh\nHN6d8wGfLljC6bVPJWlSf7q2eopqp1Zl2NQBZB3K5hfZTlR0JIte/YQF8z4iLMzJ8OmD+HHjz1zd\n4gpWL1vD9h9/L9VB5rtubrebZ5NH0q93LyIiImwdxD06KgqArKwsnhyYQK9uXWyLVZ7j/Vv7ryCh\nXFP0JyluMsYkYjXCPpw3UUSWnmgGY8ztwCSsfs0SROR17z99BDQ/0XylMe+V12h8dSN6Pd6VPXv/\noGO3Hrz72nwq2FRj9LX4k8+YMXsekyc8T5UqlYs9/67taaT9tgeAtB17yDh4iJNOrsK+3el0a9WP\nG+6M57F+DzMhcRoLX/mYYznHOAasX72JM+vXIb7lNaTv2c8NrZtSpVoVhqQMIOGx5FKt0+Ytwo7f\nfmf46DEcOXqUX37ZznMvTKB/H3s6sti9ew+9+w/kgTb3cPON19sS478UL9j7rzzxJylWBZp5X3k8\nFJ7cErAaaTuBN40xUSIyB+x7eKly5UrHT0vi4iricuWSm+vG7py48MPFvLXgPWZOnUSluLiiZyjA\n9XfGU+ecWkwdOYeqJ1chJjaa7s90YMaY+aT9tofs7CO4c93UqHs6/Uf34In7EggLD+P8Swz/e38Z\n3Vr1O76s6R+MI7HrqFKtkwcPDc4/j3dfmw/ArrQ0+ic8a9sXal/6frr06kNC/ye58vLLbInxX4oX\n7P1XkHL9nKKINCvqMwXIEZE/AYwxrYDPjTE7wL46/MMP3Efi0BG07dwNl8tF7+5diSrB9b3icLvd\njB43njNOO43e/QbicMDll15Ct04di7WcTxcs4YkhnRk5czAet5uJz1p3CZ8Y2pljOS6OHjnKpCEz\nOLA/gy8WLef5+UNwHXPx+cJl/P7Lrr8ty+Mp/S9PsA/o1NlzOXQok6mps0iZMQuHA6ZMGEdERITG\nK4GykJBC+e6zw+MpPE8ZY+oAM4C6wHXAK0AHEfm1kHnmAvuAwSKSZYypBXwMVBGRM/wpWE5GenAv\nggSZ9qeoyqqIStVKndHG3T3M7+9v37cHl6kM6k9/ilOBMVidNu4BXgXmFjFPB2Ad3pqhiPyGdfr9\nRolLqpRSQeDPNcXqIvKJMWa0iHiA6caY7oXN4O0SfHa+aXsoYmwEpVT5EMqnz/4kxcPGmJp4a33G\nmMZY/ZIppVSB7O1O0V7+JMU+WMMJnmWM+QHrbvS9tpZKKRXSynVNUUS+NcZcAdTHugYpIlLyZhNK\nqXIvhHOiX838KgOjsG6uTAcGG2Ni7C6YUkr9G/y5+5wKuIB2QGesEbGC192GUirkhDmcfr/KGn+u\nKZ4tIvf4vO9tjFlnV4GUUqGvXJ8+A2KMuTrvjTGmIbDNviIppUKd0+Hw+1XWnLCmaIz5BesxnGjg\nHmPMFiAXq8ccTYpKqXKpsNPnpsEqhFKqfCmXj+SIyHYAY0wEcBOQv4/9opr6qUIEsz3y7dd0C1os\ngIVfTQlqPIpovx9wIfyFD5ZQ3kT+3Gj5CKvjle0+0zxoUlRKnUC5rCn6qC4iDW0viVKq3AjlZn7+\n3H3+3BhzvTGm7D1QpJQqkxwOh9+vssafmuIO4BPAY4wB61TaIyJhdhZMKRW6ymCu85s/SfEJoK6I\n7LC7MEqp8qEsPn/oL3+S4k4g3e6CKKXKj7J4Wuwvf5PiBmPMCuB47zgi0sG2Uiml1L/En6T4gfel\nlFJ+CeGKol9J8QvbSxEALpeLxGEj2JmWhuuYi07t29K0SWNbYq3bsJHxk6YwM2US/RMSSd+/H4/H\nGkryogsv4LnhQ2yJm75/P/c/2pHpL02gbp3apVrWpNdHkZWZDcDunXt5ITEFgGYtr+WOB2+mz8OD\nAWjToRXxt1xDVmY2b816n2+WrgFg/mdT2LndGnh989qtzJ74WonLEox9Z8UYacVwHaNT+7Z8+PGn\npKfvx4OHXbt2e/ddUkDj+vI9buzkcrlISBrOrrQ0wsLCSEoYUOrjpbicIfxMjj9JcQnWw9oOoAJw\nGrAGuMLGchXboo8+pkqVyowYksjBjAzufciepDhr3sss/HAxMdHRADyXPBSAjEOHeKxbT57ua88o\nfS6Xi2EjxwRk2NYKFazd/nTHoX+bfta5dbmp9V8j2tY5uxbxt1xDrwcG4XQ4eGH+cNZ8vYHqp1bl\nx00/k9RrTKnLAsHZd4sW58UYbMV4uB2fvP8O4N13j/eybd/BP48bOy1bsRK3O5d5qVNZuWo1Eyen\nMG70CNvj+grlGy1FPnsoImeKSD3v/2sC1wCb7C9a8dx0Qwt6dO0MgMftITzcn3xffLVr1mTCmJH/\nmD556gweaHMP1apWtSXu2AmTaHNPa045uXqpl1XP1CEqOpLkqYMYNf0ZzIVnE1cplra97mfKqNnH\nP1e7Xg3Wrd5EriuXY8dc7NyRRr36tTnn/HpUP7Uao1MTGfrS09Soc3qpyhOMfXfT9S3o0bVTgTEm\nT0v17rv8LVkD50THjR3q1KlFbm4uHo+HzKxMwitUCErc8qLYD2SLyDfAZcWdzxgTbYyxbXT66Kgo\nYqKjycrK4smBCfTq1sWWOC2axRMW9vdHNPf/+Servv2OO2+/1ZaYCxZ+QNWqJ3FNoysD0sz3yJEc\n3pz9PgldRjBx2AwGPvcET43owbTn5nDk8FHwDqb+67YdXHjZeURGRxJXuSLnX2yIio4ife9+Xpv+\nLk93HMprMxbw9KiepSpPMPbd32M8Q69uVhI+vu9uaxnwmL4KOm7sEhMdw++70rjjngcYOuI5Hrov\n+EMqORz+v8qaIn+SjTGJPm8dwPlY4z8XNd/5wAjgT+BlYAaQa4x5QkQWlay4hdu9ew+9+w/kgTb3\ncPON19sRokCf/u8Lbr3pRtseQ1iw8AOcTicrV61my9ZtJCQNY+LY0SWule78dRe7duwGYNeO3Zxy\nxsm43W56Du5ERGQEtevVoHO/R5k2Zi7vv/YxI1IGsTdtH1vWbuPggQx27dhNbq4bgE1rhKonVyn1\nOgZj3+3es4fe/QfxwL13c/MNVoxP//clt950Q0g/QpLfvFdeo/HVjej1eFf27P2Djt168O5r86kQ\nxBpjKG9Pf85TfNfOg3WN0Z+r6inAYKAu8BbWwFdHsDqYCHhS3Je+ny69+pDQ/0muvLzYFdli8/BX\nle3rb76lS8d2tsWaPW3y8b87dO1B4sD+pTpNv+mu5tQ9pzYvJadS9eST+P3XXXRu1ReAU06vzsAx\nTzBtzFwqVYkjJjaKJ9s+S0xsNMlTE/h122906P0gGQcP8dashdQzdfhjd+keYw3GvrNi9CWhX9+/\nxfh69Wpb911+vseNXSpXrnT88kBcXEVcrlxyc90E8yw6hHOiX6P5lfRWqlNElgBLjDHNRGQvgDHG\nVcLlFSp19lwOHcpkauosUmbMwuGAKRPGERERYUc4HD6/Fb/u2EHNGjVsifOPuAE42Ba/8zlPDn+c\nsXOG4HZ7GDe44K6+Mg4cona9mkx8dQTHco4xY+w8AF5PtU6Zr2xyKbmuXJ5PmFzg/P4Kxr5LnTOX\nQ4cOMTUBiDhEAAAgAElEQVR1Nimps3DgYPL45/l1x2/UPCM4+w7+ftzY5eEH7iNx6Ajadu6Gy+Wi\nd/euAblBVxyhXFN0eE5wkcqn5+0CiUi9whZsjEn1zt9ZRNzeaQOAS0TkvqIKlpORHuRO8sov7U8x\nwEL4C++PiErVSr2Cb3ef4PdOufulJ8rUBi1uz9sPAgnAeD+W3Qm4PS8hev0OTPS7dEqpkBTKj+QU\n2fM2gDHmZGAqcDYQLyLfFbVgbzJ8L9+0+SUvqlIqVIRwTiz6kRxjzAPAemAjcJk/CVEp9d9WLvtT\n9NYOU4BzgJYi8n3QSqWUUv+Swq4pbgZigXeAHt4OZo/TXnKUUidSBiuAfissKT4ZtFIopcqVctkh\nhIjMCWZBlFLlR1m8Vugve3pNUEqpAPEOmjcdMIAb6AocBWZ7328Qke7ez3YCOgPHgGQR+cAYEwXM\nB04BMoC2InLCZlg6Qp9SKuAC3CHE7ViD5TXGajo8AhgHDBKReMBpjGlljDkV6AlcDdwMjDTGVAC6\nAetEpAkwz7uME9KkqJQKuEA+kiMi72HV/gDqYHUyc6mILPNO+wi4AbgSWC4iLhHJALYBDYHGwGKf\nzxba40hhj+S4+Xszv2NYVdVIIENE7Ot8TikV0gJ9SVFE3MaY2cCdwL1YSTDPIaASEAcc9JmeCVTO\nNz3vsydU2I0WJ4AxZgqwAnhZRDzGmLuxqqYqRAS7LfLDzfsFNd78zwPTA7gKHDua+YlIO2PMKcBq\nwLcL8zjgANb1wkr5pv/pnR6X77Mn5M/pcyMRmS8iHm/B3qaMDUWglCpbAnlN0RjzsLczGbC6H8wF\nvjXGxHun3QIsw0qWjY0xEcaYysC5wAbgKyCvF+GW3s+ekD93n7OMMe2BN7CS6CPoONBKqUIE+JGc\nd4BZxpglWDmrF7AFmOG9kbIZeMt7JjsRWI7VD+wgEcnxnu3OMcYsw7pr/WBhwfxJig8Dk7B6t3ED\nn2ElRqWUKlAgc6KIZAMFdTfYtIDPpgKp+aYdBtr4G8+fTma3A7cbY6qKyH5/F6yU+u9ylMcWLXmM\nMRdjDT8QY4y5ClgKtNEOIpRSJxLCDVr8utEyEWgNpIvILqwHIVNsLZVSSv1L/EmKMSKyOe+NiHyK\n9ayiUkoVqFz2p+hjvzGmId4HuY0xDwF6bVEpdULlspccH92AOUADY8wBrKYzD9laqhLweDwMH/08\nsnUbkZERJCUMpFZNe0dpS9+/n/sf7cj0lyZQt05tW2MBrNuwkfGTpjAzZZKtcezYlqPmJZKdeRiA\nvbv28dFrn9G+34O4c90cy3Ex6dkZHDpwiBZ3NqFF63hyXbm8M3Mha1asp1XbW2h49QXggdhKMVSu\nWpmut/QtU+vnj1Def8VVBiuAfvMnKUaJSGNjTCwQJiIZ3hsuZcrnXy4lJyeH+TOnsW7DRsaMn8jE\n50fbFs/lcjFs5JigDR05a97LLPxwMTHR0UV/uJQCvS3DK1iH2dBuf7U8eXZqf1JHz+e3n3bSonU8\nd7a9hffnLebm+1rw9MNDiIyKYOiMgaz7ehPvzfmI9+Z8BMDT43oxb/wbZWr9/BHK+++/5oTXFI0x\n1xpjmgDvGmOuAy4DLjbGNAfmBquA/vr+h7Vce7WVqy+6oAEbN2+xNd7YCZNoc09rTjm5uq1x8tSu\nWZMJY0YGJVagt2Wd+rWIjI5k0It9eealpzi7QT3GD0zht592AhAW5iQn5xhnNziTLT9sw53r5nDW\nEXb/tofa59Q8vpwrm11KZkYWG1ZvPlEovwT7WIHQ3n8lEuBucoKpsBstNwBDgNOBod6/hwADsUb2\n85u3vaKtsrKyiKsYe/x9eFgYbre7kDlKbsHCD6ha9SSuaXRl0IYcbtEsnrCwsKDECvS2zDmSw/vz\nFjOi5zhmjJpLr+GdyPjzEAD1LzqLm+5tzgevfEJ0bPTxU2yAI9lHian4V83qznYteXP6+yUuR55g\nHit5Qnn/lUS5vNEiIkkAxphHgFdFxOVtUhMhIlmFLdQYUz/fpLnGmEe9y91auiIXLDY2lqzs7OPv\n3W4PTqc9PaMtWPgBTqeTlatWs2XrNhKShjFx7GiqVa1qS7xgC/S23LV9N7t/2wPA7t/2cuhgFlWq\nV+bci8+hdbtbGfnEeDIPZnE46zAxsX8lwaiYKLIPWeWoUfd0MjOy2bvzjxKXI08wj5V/Q1lYvzKY\n6/zmz5Y6Cqzx/l0b2GKMaVXEPJ8B72M9zzgVq8fcqdj4fOMlDS9i2YqVAKxdv4Fzzq5nVyhmT5vM\nzJRJzEyZxLn1zyE5aXDQEqIH+6umgd6WzVtdx6O97wfgpOpViI6JosFl53LTvc1J6jKafbutpvQ/\nbvwFc/E5hIWHER0bTY26p7PDe4p9YaPz+eGr9aUqR55gHiv5heL+KwmH0+H3q6zx50bLM3g7ZRSR\nn4wxlwGfkG+g+3wux0qAU0TkU2PMFyLSrNSlLUSLZvGs/GY1j3TsAsCwxAQ7wx0X7F9EB/YHDPS2\n/HzBMh5/tgNDpg/A4/YwZdgsBrzwBH/sTuep53uABzZ9L7w1/X0+ev0zhqUOBBy8+tLb5LpyATi9\n9qmsW7WptKsG/HvHCoTm/iuJUK4pOjxFXBQzxmwRkXPzTVsrIg2LmC8ceB7YC9xQ3KSYk5EepKt1\nKtC0P8XQFlGpWqlT2tcjZ/n9/b1qYPsylUL9qSkuN8a8Crzsfd8GWFnUTCLiAnobY9qhwx4o9Z8S\nyjVFf5Jid6zBYLpgDUmwFJjsbwARmY016pZS6j+iLN5V9ldhY7ScJiK7gVOxOpj1fWL2NGCHzWVT\nSqmgK6ymOAO4DViC1e7Zke//wb+lpZQKCSFcUSz0OcXbvP8/M3jFUUqVB46w0M2KhZ0+zyxsRhHp\nEPjiKKXKg1C+pljYXeEl3lcccAbwOdbziScVMZ9SSoWswk6f5wAYYx4HrhYRt/f9G8DXwSmeUioU\nhXBF0a9HcioDVYF93venAhVtK5FSKuSF8umzP0kxGVhnjFkBhAGNsJ5bVEqpAoVwTvRriNN5xpjP\ngGuwHsXpKiJ7bS+ZUip0hXBW9GeI0wigPXAuVg3xCWPMKBHJsbtwKjQFuy3yE7cNDWq8CYsSgxov\nFJXF3m/85c9d5JewriFeitXM72wg1c5CKaVCWwh3vO1XUrxMRAYBx0QkG2gLXGJvsZRSoaxc9rzt\nw+M9hc7rCqi6z99KKfUPZTDX+c2fmuJ4rJ60TzPGjAe+BV6wtVRKKfUv8aem+BHwHdAM65Gc20Vk\nna2lUkqFthCuKvqTFJeJyHlAYPqCV0qVe6F899mfpLjWO6LfN8Dx8SdFRPtTVEoVqLwnxUbely/t\nT1EpVS7506IlJPpT9Hg8DB/9PLJ1G5GRESQlDKRWzRoarwRcLheJw0awMy0N1zEXndq3pWmTxrbE\nAvvWLa5KRQZMf4oJfV6iQmQFHnyyDbm5uez97Q/mP/caAPGtG3PVzVficXv47PUv+P7LH4iKiaRj\nUjsioyM4luNi9rB5HDqQWaIyrNuwkfGTpjAzZRJbZCvd+/ajbu1aALS5+y5uur55qdczv2AfmwUJ\n4UuKhfaneAYwCTgHWA4MFJEDwSpYcX3+5VJycnKYP3Ma6zZsZMz4iUx8frTGK4FFH31MlSqVGTEk\nkYMZGdz7kL1J0Y51c4Y5eeCpNuQcsRpe3dr+ZhbNWszm1Vto98wjXHDV+fyy6Vca33EtIzo8R0RU\nBRLnDuL7L3/g6lsasfOnXSyYupBrb72KGx5swTuTCxvRt2Cz5r3Mwg8XExMdDcCmLULbhx7g0Qfv\nL9W6FSXYx2ZBQvn0ubBHcmYBW4B+QBSleAzHGOM0xtQwxtjWD+P3P6zl2quvAuCiCxqwcfMWu0KV\n+3g33dCCHl07A+BxewgP9+cqS8nZsW53P96KpQtWcDD9IAC/bf2dilViAYiKiSQ3N5esjGxGdHgO\nj8dDpWqVOXbUSqA7f95FVEyU9dnYKHKP5ZaoDLVr1mTCmJHH32/avIWly7+iXefHeXb4SLIPHy5k\n7pIL9rFZkFB+eLuwJFVDRAaJyGKgM3BlcRZsjEn1/r8RsBV4B9hgjLmqpIUtTFZWFnEVY4+/Dw8L\nw+122xGq3MeLjooiJjqarKwsnhyYQK9uXWyJkyfQ63bVzVdy6M9MtnwrgPXF++P3P2jT624S5w4k\nrkpFtq75EbBONeNbN6bf5N6s+uRbqzwHsznvCsPgOQO4/v7mrPigyBF9C9SiWTxhYWHH3194QQOe\nfKI7s6dNpmaNM5gyzZ7WssE+NgvkKMarjCmsCnC8wwcROWaMKW4HEHnXIpOBW0Rkm/eU/FUgvpjL\nKlJsbCxZ2dnH37vdHpxO+zoIL+/xdu/eQ+/+A3mgzT3cfOP1tsWBwK/b1S0b4XF7OPdyQ82zz6Bt\nwkPUPLsmyR1Gs2fHXprc2Zh7etzJ6+PfBmDJu8tZ9v5X9BzTlR/X/Uyzu5vwySv/Y8WilZxR73S6\nDO9IcofnSr2ezZs2Ia6i1RVpi6bxjHzenjYQwT5WypvibKmSNu3LFZFtACKyq5gx/XZJw4tYtsL6\nRV+7fgPnnG3vzfHyHG9f+n669OpD317daXVbS9vi5An0ur3Q60XG957E+N6T+P3HncwePp99u/Zx\nJPsoAAf3HSS6Ygyn1DyZzsOsoYbcuW6O5bjwuN1kZWRxOMs6tc08kElUTGSpypOna88+bNy0GYCv\nV3/L+eeagCw3v2AfmwVxOp1+v8qawmqKDYwxP/u8r+F97wA8IlLUlq5sjPkOiDXGdAReBsYC20tV\n4hNo0Syeld+s5pGO1qnesMQEO8L8J+Klzp7LoUOZTE2dRcqMWTgcMGXCOCIiImyJF4x1m//cqzyW\n1I5cVy65Lhfzn3udP/f+yW/bfqff5N54PB42rtrMj+t+Zu/OfTzc/37iW19HWJjz+J3q0npmwFOM\nHPMCFSqEU61aNZIGPR2Q5eYX7GOzQGUv1/nN4fEUXAE0xtQpbEYRKTK5GWMigYZANtZ1xQ5Aqogc\nK2renIx07XRC+UX7UwysiErVSn2l78dX3vH7+3v2g3eVqSuLhQ1cVeoanYgcxWoJkyeltMtUSik7\n2fushVLqP6ksPmrjL02KSqnAC92cqElRKRV4odyiRZOiUirw9PRZKaX+EsI5UZOiUirw9EaLUkr5\nCvA1RW8fCqNEpJkx5mJgEdazzwBTRORNY0wnrH4ajgHJIvKBMSYKmA+cAmQAbUUkvbBYmhSVUgEX\nyJqiMaYf8AiQ16nlZcBYEXnB5zOnAj2xxqePAZYbYz4BugHrRGSoMeY+YDDQu7B4mhSVUmXdj0Br\nYJ73/WVAfWPMnVi1xT5YvXgtFxEXkGGM2YbVmq4xkNeZ5EdYSbFQmhRVyAt2s7sHmj4Z1Hivfjk2\nqPECIZCP5IjIu/maHa8CpovIGmPMQOBZ4AfgoM9nMoHKQJzP9ENApaLihXCzbaVUWeVwOvx+lcAC\nEVmT9zdwMVbi8014ccCfWNcR43ymFTl6gCZFpVTgORz+v4rvY2PM5d6/W2CNS78aaGyMiTDGVAbO\nBTYAXwF5/d+1BJYVtXA9fVZKBZzNj+R0A170dny9G+gsIpnGmIlY40k5gEEikmOMmQLMMcYsA44C\nDxa1cE2KSqkyz9tr1zXev9dg3UDJ/5lUIDXftMNAm+LE0qSolAq80H12W5OiUirwtEMIpZTy4SiD\nY6/4K3RLrpRSNtCaolIq8PT0uWxo80j744OA1zjjDIYOHmRrvHUbNjJ+0hRmpkyyNQ5Yg7YPH/08\nsnUbkZERJCUMpFbNGiEfC8DlcpE4bAQ709JwHXPRqX1bmjb5x83FgHG73SQlj+LX7TtwOB0kDujP\nWfXOLHrGQjz3chLZmdZYy3t37uODVz/lsacfJjc3l2M5Ll4cPJ2MA4e4/eGbaXxzI9y5bt6ZuYjV\nS9ZwZ9uWXHLNhXjwEBsXS5Vqleh0U58SlyXY+68g2ktOGZCTkwNA6hT7ExTArHkvs/DDxcRERwcl\n3udfLiUnJ4f5M6exbsNGxoyfyMTnRxc9YxmPBbDoo4+pUqUyI4YkcjAjg3sfsjcpfrlsOQ6Hg7kz\nUlj93RomTE4p1fqFV7C+Rkldnjs+bci0AUwfNY8dP/7ODXfFc2e7lrw1431ufeB6Hr+9H1Ex0Yx9\nbSirl6xhwZwPWTDnQwAGju/N3PGlG1I12PuvQKGbE4OXFI0x1YF0EbFl6FLZ9iOHDx+mS8/e5Oa6\n6fV4Fy66oIEdoQCoXbMmE8aMZGBicIbX/P6HtVx79VUAXHRBAzZu3lIuYgHcdEMLbry+OQAet4fw\ncHsPy+bxTWh6nZV0d6WlUSkurog5Cle3fi2ioiIZ/NKTOJ1OXnnpbcYNmMzB/RkAOMPCOJZzjCOH\nj7J31z6iYqKJionEnev+23IaNb+MQwczWf/N5lKVJ9j7ryBaUyyAMaY9UAur37NXgCNAjDHmcRH5\nLNDxoiIjaf/IQ9zV6na27/iNbk/0ZdHbr+O06S5Yi2bx7EpLs2XZBcnKyjp+aQAgPCwMt9tty/oF\nMxZAdFTU8bhPDkygV7cutsTx5XQ6SUgazhdLljJ2VHKplnX0SA7vzf2I/723lNNrnUrCpL70vHMA\nAOais7mlTQueeWwEAOl7/mTC28k4HE7enbXob8tp3e5WXhg4pVRlgeDvv/LGzp/kx4GmwPvAHSKy\n1RhzBvAeEPCkWLdObWrXqglAndq1qFK5Mn/sS+fUU04OdKh/RWxsLFnZ2cffu90e2w7yYMbKs3v3\nHnr3H8gDbe7h5huvtzVWnuSkZ0jfv58H2z3Ge2+8SlRUZImWk7Z9N7t/22v9/dseDh3I5KTqlTn3\nkvrc1f42knuNI/NgFpc3uZgq1SvT9dancACDJ/djyw/b+Gnzr9Q483SyDmWxZ+cfpV6vf2P/5ecI\nC90EbGfJj4lIFlZ3PT8DiMguwJbT53ffX8SY8S8CsPePP8jKzubk6tXsCPU3HntW5x8uaXgRy1as\nBGDt+g2cc3a9chELYF/6frr06kPfXt1pdVvLomcopYUfLmbG7LkAREZE4nQ6cZbibmnzVk1o2/d+\nAE6qXoXo2CgaXH4et7RpQWKnUfyRZnX0nJmRRc7RHHJdubhcuWQfyiY2LgaAi65swPcr1pdyzSzB\n3n8FsrdDCFvZWVN83xjzHlZPFYuMMR8DNwOf2xGsdavbGTxkOG07dcPhcDB08KCg/Do6gnRFuUWz\neFZ+s5pHOlqnlsMSE8pFLIDU2XM5dCiTqamzSJkxC4cDpkwYR0REhC3xrm/elMFDkmnX+XFyc3MZ\n8GSfUsX633tL6Z7UkWGpA/G4PUweOpNB4/vwR9o+nh7bEw8eNn4nvDntPX7a9Csj5wzGnetm8w9b\nWffNJgDOqHMaa7/eEJD1C/b+K0goX1N0eDz21XSMMfHATUB1IB2rZ9wP/Jk3JyM9OFUwpYqpvHcy\nG1GpWqkz2t4VS/3+/p5ybZMylUFtvc0nIkuAJXbGUEqVQfrwtlJK/SWUT581KSqlAk+TolJK/UW7\nDlNKKV9aU1RKqb/oNUWllPKlSVEppf4SytcUQ7eBolJK2UBrikqpwAvh02dbm/mVhjbzU8pyU6PH\nghrvi83vljqj/bn+O7+/vyddeFmZyqBaU1RKBZ5eU1RKqfJBa4pKqYBzOEK3vqVJUSkVeCF8o0WT\nolIq4LRFi1JK+QrhGy2aFJVSAac1RaWU8qVJUSmlfOjd53+fx+Nh+Ojnka3biIyMIClhILVq1rAt\nnsvlIiFpOLvS0ggLCyMpYQB169S2LZ7b7SYpeRS/bt+Bw+kgcUB/zqp3pi2xXC4XicNGsDMtDdcx\nF53at6Vpk8a2xPKVvn8/9z/akekvTbB1Wwb7WLEj3tS3nifzkDW28+6de5g2dh5PDX2cipVicTqd\njBwwgd0793Lng7dwY6tmeDxu3pj1Pks+/gqAN76Yzm+/pgGw6QchdcLLpVvJfEK5Q4hykxQ//3Ip\nOTk5zJ85jXUbNjJm/EQmPj/atnjLVqzE7c5lXupUVq5azcTJKYwbPcK2eF8uW47D4WDujBRWf7eG\nCZNTbFu/RR99TJUqlRkxJJGDGRnc+5D9SdHlcjFs5JgSD0hfHME+VgIdr0IF62v7ZPvE49P6J/fg\n04VLWPrJShpe0YDa9WqSnXWY29rcSKfWfYmMjmT2woks+fgrzqh1Gls3/swzPUaWet3KI9uSojGm\nkohk2LX8/L7/YS3XXn0VABdd0ICNm7fYGq9OnVrk5ubi8XjIzMokvEIFW+M1j29C0+usxLQrLY1K\ncXG2xbrphhbceH1zADxuD+Hh9v92jp0wiTb3tCbVO0i9nYJ9rAQ63lnnnklUdCTPTU/EGeYkdfzL\nXHDpefwkvzIm9Vl2/76XF0ekknM0h06t++LxeKh28kkcPZoDQP0G9Tj5tGqMnTWUo0eOMnn0LH7/\ndVep1/NvQviaop0n/ruNMR1tXP7fZGVlEVcx9vj78LAw3G63bfFiomP4fVcad9zzAENHPMdD991r\nW6w8TqeThKThjB47nltvvsm2ONFRUcRER5OVlcWTAxPo1a2LbbEAFiz8gKpVT+KaRlcSjP5Jgn2s\nBDrekSNHeX3mAvp3GsoLQ6aS8Fwfzqh1KocOZtKv4xD27t7Hg53uAqxT9zsfvIVJr4zks4XWaMPp\nf/zJy1Pf4sn2ibwy7W0Gje5duhUsgMMZ5verrLEzKa4FLjHGfG6MibcxDgCxsbFkZWcff+92e3A6\n7Vu9ea+8RuOrG7Hw7dd465W5JCQN49ixY7bFy5Oc9AwL336NpOSRHDly1LY4u3fvoWO3ntxxa0tu\nvvF62+KAlRRXrlpNh6492LJ1GwlJw0jfv9+2eME+VgId7/dfd/HpwqUA7NyeRsaBQ3jcHr76YjUA\nX32xmvoNzjr++QWvfMTdTTrQ8IoGNLyiAbLhJ1Z4P7thzRaqnXxSictyIg6nw+9XWWNnUjwsIj2A\n/kAvY8x6Y8x4Y0wvO4Jd0vAilq1YCcDa9Rs45+x6doQ5rnLlSlSsWBGAuLiKuFy55ObaV9tY+OFi\nZnhPLSMjInE6nThtOqD2pe+nS68+9O3VnVa3tbQlhq/Z0yYzM2USM1MmcW79c0hOGky1qlVtixfs\nYyXQ8W65qwWPP90egGonn0RMxRiWf7aKq+IvB6Dh5Q349ccd1Kx7BkMm9AfAnesm5+gx3G43bbvf\nxz2P3g7AWaYue3fvK1V5yhvb+lM0xnwhIs183lcGmgBGRJ4vav7i9qeYd4dv67YfARiWmGDrHczs\nw4dJHDqCP9L34XK5eOT++2ytUR0+coTBQ5LZl55Obm4uj7V7lPjrrrUl1uix4/n4s885s25tPB7r\n8tCUCeOIiIiwJZ6vjt16MHhA/6DcfQ7WsVLaePn7UwwLD+Pp5J6cesbJeDwepj4/h/Q//uSpYd2J\niookKzOL4U+9QFZmNo90u5dGTS7D43azatn3zE95i9i4GBKe60N0TBQul4sJw6b/7ZpiIPpTzNr5\ns9/f39ga9cpUddHOpNhWROaUdH7tZFYpSyh2Mpu96xe/v78xZ5xZppKibbcVS5MQlVIhTh/eVkop\nH2XwBoq/NCkqpQJOO4RQSilfevqslFJ/0ZqiUkr50pqiUkrZwxjjACYDDYEjwGMi8rNd8UI3nSul\nyqwAN/O7E4gUkWuAgcA4O8uuSVEpFXgOh/+vojUGFgOIyCrgcjuLrklRKRVwAe4lpxJw0Oe9yxhj\nW+4qs9cUIypVC93bV0oF0Beb3/23i1BsAf7+ZgC+HYg6RcS23le0pqiUKutWAC0BjDFXAevtDFZm\na4pKKeX1LnCDMWaF9317O4PZ1kuOUkqFIj19VkopH5oUlVLKhyZFpZTyUW5utAS7KZA3ZiNglO+w\nCzbFCQdmAnWBCCBZRBbaGM8JTAcM4Aa6isgmu+L5xD0F+Ba4XkS22hzrO/569u0XEbF15EljzADg\nDqACMFlEZtkYqy3QDvAA0VjfidOCOeRwKCtPNcWgNgUyxvTDShz2j94ODwP7RKQJcAswyeZ4twMe\nEWkMDAZG2BwvL/GnANlFfTYAsSIBRKS592V3QowHrvYem02BWnbGE5E5ItJMRJoD3wE9NSH6rzwl\nxaA2BQJ+BFrbHCPPG1jJCax9ZutYqiLyHtDZ+7Yu8Ked8byeB6YAAR6VvUANgVhjzMfGmM+8NX47\n3QRsMMYsAN4HFtkcDwBjzOXA+SKSGox45UV5SopBbQokIu8CLruWny9WtohkGWPigDeBhCDEdBtj\nZgMTgJftjGWMaQfsFZFPgWC0ZMoGxojITUA34GU7jxWgOnAZcI833is2xvI1EBgSpFjlRnlKikFt\nChRsxphawOfAHBF5PRgxRaQdUB+YYYyJtjFUe6yHc78ALgbmeq8v2mUr3kQvItuAdOB0G+OlAx+L\niMt7rfSIMaa6jfHyhhSuLyJL7IxTHpWnpBjUpkA+bK/ZGGNOBT4G+gdjlERjzMPeGwNg3bTKxbrh\nYgsRifdeA2sG/AA8KiJ77YoHdADGAhhjzsD6MU2zMd5y4GafeDFYidJOTYD/2RyjXCo3d58JclMg\nH8FoEjQQqAIMNsYkemPeIiJHbYr3DjDLGLME6xh5wsZY+QVje6Zird8yrGTfwc6zChH5wBhznTHm\nG6wf0cdFxO71NICtT1+UV9rMTymlfJSn02ellCo1TYpKKeVDk6JSSvnQpKiUUj40KSqllA9Nikop\n5UOTYjljjLnAGOM2xhTaLtsYU9cYM6MUcf7xXJ8xpq0xZp8x5mSfaXWMMb+UNE6gl6NUUTQplj/t\nsNpHdy3ic3WBeqWIc6IHXGOxervx57OBiqlUwJSnFi3/ecaYMKxuxhoDK40xZ4rIL8aY67F6oXEA\n24GHsDp6ONMY8yLwFpCU1y+kMWYW8IWIzDXGJAPNgZOAfcBdRTTBewe40BjzgIi8mq98z2J1STbU\n+xfrf8cAAAK5SURBVP4XIB5oBtwK1PC+JgC1vXH3YXWXBhBtjHkdq7XGj0BHETno7Q3mBay+A/cB\nXURku7ct9X7gfOA+EVlX3G2q/nu0pli+3Ab8KiI/YjV77GKMiQDmA4+ISENgHfAo0BP4VkR6euf9\nRy3MGHMWVqcCV4vIucBPWAkVTtzm+yhWbfUF39PoE/CNeQVwI1ab3bHAB97yOrC63gI4BRgvIhd7\ny5JojKkAzAAeEJHLsfrR9L0ssFZEztOEqPylNcXypR2QVzt7EysZvg38LiLrgf+3d/euUQVhFId/\nUbQSP4LBVKKxOKYQJQGxMNjb2whBEMXCQgSxshJEm4CNiEmjaBFbCxtTCSIWfqKYF8l/kDYIkRCL\nd6KvgZvdymI5TzXcndmd6jD3XjhLRNyCP8WnW4qIJUk3JF0mT2enyBNar3UfJM0Bj4DrW0ytwfom\nIlaAFUnrZCMQ5Ml2XxsvRsTbNn4GPCZbfI4AL1r7OsCu8r3veu3XrHIoDoh2KjsLTEq6Rt4F7OXv\nrefGvN38W7EGeWKrAbWjzZ0gQ3aGDNk1+m8Fuk22Pp/v9TvNal3cUdCwVsZDZNnuNmApIibanoeA\nA2Xezz73awb49nmQTAMLEXEwIsYi4hBwhwzFEUlH27ybwBWyIHcjlJaBMUk7JQ0DU+36GfLZ4iyw\nSN7ebu9nMxHxi2wqqoW4y+TzPSSdBEY7lncF77ik4218EXgFBDAs6XS7fon/V+JqA8ihODguAA82\nXXsIHCNfvjyV9AkYB+4B34E9kp60P6V6CXwDngOv2/p54ERbtwB8Bg63z3q+CY6I98D9cmke2C/p\nK3AV+NixdL1j/IN8jviFbLO+GxGrwDlgpu1zmgzMvvZotpmrw8zMCp8UzcwKh6KZWeFQNDMrHIpm\nZoVD0cyscCiamRUORTOzwqFoZlb8BnOVJQ2h9/+pAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x16465400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Model accuracy - training data\n",
    "print\n",
    "print 'Precision:' + str(metrics.precision_score(y_train, pred_y_train, average = 'weighted', pos_label='high'))\n",
    "print 'Recall: ' + str(metrics.recall_score(y_train, pred_y_train, average = 'weighted', pos_label='high'))\n",
    "print 'Accuracy: ' + str(metrics.accuracy_score(y_train, pred_y_train))\n",
    "\n",
    "# Fancy boy confusion matrix - training data\n",
    "mat = metrics.confusion_matrix(y_train, pred_y_train)\n",
    "sns.heatmap(mat.T, square=True, annot=True, fmt='d', cbar=True)\n",
    "plt.xlabel('Actual Number')\n",
    "plt.ylabel('Predicted Number')\n",
    "plt.title('MLP - Training Performance')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Precision:0.980387484364\n",
      "Recall: 0.980290456432\n",
      "Accuracy: 0.980290456432\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x16fa3908>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAAEZCAYAAADmEfyVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8FNXawPHfJpAKoYqNJqKPioKICgqCiKLXa3mt14aA\niIAUFa9IpEvvRZQmRRC96lXxigWvF6WJlyISgvAAKqhUAWlBCMnu+8dsuBFTFjKzYZPny2c/ZDY7\n5zmzO/vkzJmZc3yBQABjjCmOogq7AsYYU1gsARpjii1LgMaYYssSoDGm2LIEaIwptiwBGmOKrRKF\nXYGiRkSqAT8CC1X1+hN+Nx1oCVQESgOpqlo6hzL6AB2BX4AAzue0E+ioqhtPoi4tgK7BMqoBvwO/\nBpc7q+qSU9i+fsB/VfXjE54/H1AgJfiULxhnrKq+dpIxqgIfA+nAE6q64mTraUwoLAF64whwoYhU\nUdWfAUQkAWiIkxSy5HUR5j9UtUvWgoh0At4Argq1Eqo6C5gVXH86sEZVR4W8FTm7AViZy+8OqOoV\nWQsiUhlYIyLLVHXdScS4EfhJVW8tQD2NyZclQG9kAm8BjwCDg8/dDXyA0yI7Ff8BBhW8av8jImWB\nscAlQEng30A3VQ2IyADgduAosBtoBfwNuBwYLSJ+VZ2bV/mq+ouI/ABcAKwTkbZAO5zW4a9AJ1Xd\nJCKzgCSgBrAfqAKUEZHPVLW5iHQAngQygO3B9X44Yb0PcFq5+4EGQCXgH8HlvwJnAI+p6iIREWA8\nkACci5PQHwD8wdePApoBZwGjVXV88P3qifOZpuO0dlur6qEctqvzybTUTeGxPkBvBICZOF+WLC2B\n6ThfkpMiItFAG+ALV2r3P2OBr1T1KuAKnGTwlIhUBzoA9VT1amA+cJWqvgR8CzyTX/IL1rsRTlJa\nJiJNgQeBhqpaLxj73WwvL6mql6lqI+BFYH4w+TUHugCNVbVucJ05OazXM7hcB6eV3AD4O/Crql4L\nTACeD77mCWCKqjYEagIXAbcEf5cA/Bz83YPAMBGJFpG7cZLkVapaG6d7okMu2/XP/N4bc3qwFqBH\nVHWViPhFpC5Oq6CUqn7nND5C8kAwgfhwWmcrgbYuV/M2oK6ItA8uxwGHgXHAWmCViHwCfKyqX4ZQ\nXpKIfBOscwlgF/CAqu4QkeeAC4GlIpL1R6CCiGT1gS7OpcybcboDfgNQ1akiMlpEzs1lvX+pagDY\nJiJHgXnB578H7g3+/Bxwk4h0C9apElAqexnB/78BYnGSYjPgbVU9GKzHMwAiMvKE7QpkbVfWa83p\nyxKgt2YBLXAS4KyTXPcPfYC5EZEpwJU4X7yJqjr5JGJEAXep6vfBssoCmarqBxqLyJU4/XEvicin\nqvpcPuX9oQ/wBNHAdFXtFYzlA85W1YPBPwqH8qjjibL+KOS03tETlo/lsP47OIfT7+Aku/P4Y8v8\n9+D/WX20vuDrj/fZBt+rpFy26xxLfpHBDoG9kfVleh24D7gf5wRGbq87ZaraVlXrquoVJ5n8wGkd\ndQUQkThgLtBeROqKSAqwVlWH4BzW1Qmuk8H/ks+J8tqeecDDIlIpuNwZ+CzEOj4oIuWD9WwLbFPV\nzSGsm5vmQF9V/SdOI+AqnESWk6xt+hy4V0SyWor9cQ7NP+XP2zUPExGsBeiNAICqbhOR74B9qrov\n+++CEkTkQPDnrMOna7ys0wk6AeNEZA1OUvsUGKmqfhF5D/hGRA4BacHXgtNiGiMisap6YlLP9ay2\nqn4iIqOA/4iIH/gN58RQfut9KiIvAwuCLcVdOCdnclovv+UsycBcEdmDs21f4vQF5lqGqn4YPHmy\nNFj/NUBbVf09j+0ypzmfDYdljCmu7BDYGFNsWQI0xhRblgCNMcWWJUBjTLF12p4FvrX2g2E9OzNn\n8fhwhjPmtBWTVKHAl2fVrtYk5O9vypYFBY53qk7bBGiMiVw+X6HltJNiCdAY4zqfLzJ61yKjlsYY\n4wFrARpjXBcdIS1AS4DGGNdFWQI0xhRXkXISJDLStDHGeMBagMYY1/kKPtJbWFgCNMa4zu0+QBHp\nDtyBM2zbK8BCYAbOPC6pqtox+Lq2OFMeHAMGqupHeZUbkQlw7D8GcviQM2jvjl92ERsfS7kKZQA4\n89wzWL96I8O6j+fKRnV4sJ0zNNum735kwuAZrtUhEAgwYOgIdMNGYmNj6NsjmSqVz81/xQiJB5CS\nupYx4ycwbaK3d8lkZGTQu/8gtm7fTsaxDNq2bsn1jRtZvFNUGPvKidzsAxSRJsA1qnqtiCTizPUy\nCnghOMnVBBG5E/gaZ0DaK3CmMVgcnFgrp1HBgQhMgCVKOlVOfnzAn36XWDqBwa/2ZNKwmcTFx/LY\nMw/RrfWLHDqQxt0t/0rpMqU4uD+3kddPzvwvF5Kens7r0yaTkrqW4WPGMW7EUFfKPh3iTZ81mw8/\n/pSE+HjPYmSZ+8k8ypYtw6B+vdl/4AD3Pextgijq8cK9r+Qkyt2TIDcDqSIyB2c+7W7A46q6KPj7\nT3BG+fYDi1U1AzggIhuB2uQ+jWt4EqCIRAXnmSiwGlKNuPhY+k/oTlR0FDNfegtd8z0Ajzx5Lx++\nMY/9ew9Q95rL2LzxZ9o+14KzK1fi03fnu5b8AL75djUNr2kAQO1La7F23XrXyj4d4lWtXJmxwweT\n3PtFT+MA3HxTM5rfeAMAAX+AEiW83S2Lerxw7ys58bl7frUiUBVnEq8aOKOSZw9wEGd+ltI405pm\nOQSUyatgzz4JEamB00y9EsgQkSicYcSfUdUNp1ru0SNHeXfGXD57/0vOqXoWL77yPG1v70rpsqWo\nc3UtJg2dCUCZcqW57MpL6Hjf8xw9ks7wGX1Yt3oj23/e6cbmkZaWRulSiceXS0RH4/f7iYry5sR6\nuOM1a9qEbdu3e1L2ieLj4gBnG59N7kGXDu0sXgGEe1/JicuXwewB1gVbdhtE5AhQOdvvSwP7gAM4\nifDE53Pl5TvyKjBYVSuranVVrYozkcz0ghS6dfN2vvhoCQDbftrBwf2HKH9GWRrdVJ8vP/7q+OsO\n7DvEhrXfc+C3gxz9/SipK9dT46JqBQn9B4mJiaQdPnx82e8PeLqDhTteuO3YsZM2HTpzx19v5Zbm\nN1q8Ajgd9pUony/kRwgWE5y3WUTOARJx5mBpEvz9X4BFwHKgkYjEiEgZnPmeU/Os5yluXyjiVPW/\n2Z9Q1a8LWmjzu5rS9u/OfOPlzyhHfEIce3/dR90Gl7Ji8bfHX7dp3Y9Ur1mFUkmJREVHcVHtmvz0\n/daChj+ubp3aLFqyFIDVa1K5oGYN18o+HeJlCeQ+X5Frdu/ZS7suz9C1S0fuvO1Wi1dAhbWvZOc7\niX/5CZ7JXSUiy4APgA7As0A/EVmCc2b4n6q6E2dO68U4s/i9oKrpeZXtZWfEahGZhjPT2H6c5uit\nQEpBCp33/hc882J7hs3oQ8AfYHTviQQCAc6tdjbbf9l1/HUHfjvIjLH/YOCkFwgEAiyct5Sff3Av\nATZr2oSly5bToo1zONO/dw/Xyj4d4mUJx/VcU2fM5ODBQ0yaOp2Jr07H54MJY0cRExNj8U5BYe0r\nXlLV7jk8fX0Or5sKTA21XM9mhQtOEP1/QCOc4/IDwBLgfVXNN6gNiGpM4XBjQNRml9wT8vf3P9+9\nW/QGRA0mufeDD2NMMRIdIf3TEXcdoDHm9Bcpt8JFRpo2xhgPWAvQGOM6Gw/QGFNsRcp4gJYAjTGu\nc/leYM9YAjTGuC5SToJYAjTGuM4OgY0xxZYdAhtjii07BC6gcN+aduVld4c13oo174U1njHhFCmX\nwURGLY0xxgOnbQvQGBO57CSIMabYio6QQ2BLgMYY10XKWeDISNPGGOMBawEaY1xnfYDGmGIrUg6B\nLQEaY1xnF0IbY4otawGGWSAQYMDQEeiGjcTGxtC3RzJVKp/rStmPdXiI629qSIkSJXhr1hxSVn1H\nn8F/B2DL5l/o220YgUCA1u0f5JY7mnHowCFmTP4Hi+YXeBbQP0hJXcuY8ROYNtHbu2S8fC8tnvfx\nMjIy6N1/EFu3byfjWAZtW7fk+saNPIuXk0jpAywyZ4Hnf7mQ9PR0Xp82mac6dmD4mHGulFuvfh3q\nXFGLR+/uSJu/PcXZ51Si83OPM2boJFrd1xkfPprceC01LzyPW+5oxsN3tKP9o8/RsetjxMS6N+3h\n9Fmz6TtwCOnpeU5z6gqv3kuLF554cz+ZR9myZXht8gReGTuSQcNHehovJy5PjO5dPQs1uou++XY1\nDa9pAEDtS2uxdt16V8pt2PhqNm34kTGTBzBu6iAW/OcrurbrxbcrUilRsgQVK5Xn0ME0zqtZjRVL\nV5GRkcmx9GNs+fEXLrzIvQmpq1auzNjhg10rLy9evZcWLzzxbr6pGZ3aPwFAwB+gRInwH+i5OTG6\nl4pMAkxLS6N0qcTjyyWio/H7/QUut2z5Mlxy6YU826EPA3qMYsjYXgCcdU4l3vtsBmXKJqHfbWKj\n/kC9+nWIj4+jTNkkLq93KfEJ8QWOn6VZ0yZER0e7Vl5evHovLV544sXHxZEQH09aWhrPJvegS4d2\nnsXKTaS0AItMH2BiYiJphw8fX/b7A0S5MDfpvt/288OmLWRmZrLlx184ejSdsuXKsGPbLu5o+gh3\n/e1WuvXuRK+/D+EfM+cwYeZwtm/bScqq79i3d3+B4xcGr95LixeeeAA7duzk6W7JPHj/vdzS/EZP\nY0Uyzz4FEflCRL464bFURL7yIl7dOrVZtGQpAKvXpHJBTXcOP1etWEPDJlcDcEalCsQnxNNvWDeq\nVHM6sdMO/U6m30+ZskkkJsbT6r7ODOgxirPOqcRG/cGVOmQXIOB6mSfy6r20eOGJt3vPXtp1eYau\nXTpy5223ehorNz6fL+RHYfKyBdgdmALcBWR4GAdwDhGXLltOizZOc79/7x6ulLto/tdccVVtZn8w\nEZ/Px8Ceozmcdpj+I7tzLP0YR34/Sp/nh7F/3wHOq1mN2R9M5Fj6MUYNnOBK/BOFo8/Eq/fS4oUn\n3tQZMzl48BCTpk5n4qvT8flgwthRxMS4d1IuP4V9aBsqXyDgXYtCRJ4DNqnq+ye7bvqBPd43dbKx\nAVGNccQkVShw9nqy8dMhf39fWTim0LKlp32Aqjrcy/KNMaYgisxJEGPM6SMqMo6ALQEaY9xX2Cc3\nQmUJ0Bjjukg5CWIJ0BjjukhpARaZO0GMMeZkWQvQGOM6mxTJGFNsWR+gMabYipD8Z32Axpjiy1qA\nQStS3g1rvDsbdgxbrA8WezuCdKGLlObGKQr4Mwu7CifNDoGNMcVWYQ90GipLgMYY10XKdYCWAI0x\nrouOkJuB7SSIMabYshagMcZ1dhLEGFNsuX0SREQqASuAG4EEYC6wIfjrCar6joi0BZ4AjgEDVfWj\n/Mq1BGiMcZ2bLUARKQFMBLJmlqoHjFTV0dlecybQGbgCJ0EuFpHPVPVYXmVbAjTGuM7lI+ARwAQg\nObhcD7hQRP4PpxX4DHA1sFhVM4ADIrIRqA2szKvgIpMAA4EAA4aOQDdsJDY2hr49kqlS+VxPY+7Z\n+xsPtGzDlPFjqF6tqmvljntrEGkHnT92O7f+SmxcDOUqlgXgzHPOYF3KRoY9/xL3PXYHTW65hrRD\nv/PP6R+yfNEq1+oA3m3fifx+P30HDmXzTz/h8/no3f05zq9xnmfxsqSkrmXM+AlMm+jtheKFsW/+\n7dE2lCpVCoDK55xNv57dPY13IrcugxGRVsAuVf23iLwA+ID/AlNUdZWIJAN9gG+B7PPQHgLK5Fd+\nkUmA879cSHp6Oq9Pm0xK6lqGjxnHuBFDPYuXkZFB/yHDiIuLdbXckiWdjyT58QF/+l1i6QSGvNqL\nSUNfo1rNyjS55RqeeqgnUT4fI19/kW//m8qx9Dxb/CHzavty8uWiJfh8MHPKBJZ/s4qxr0xi3Igh\nnsacPms2H378KQnx7k1en5tw75vp6ekATH1lrGcx8uPiIXBrwC8iNwGXA68Bd6jqruDv5wDjgAVA\nUrb1SgP78q1nfi8QkdonW+PC8M23q2l4TQMAal9ai7Xr1nsab+S48dx/z11UqljR1XLPk2rExccy\nYGIyg6b0QC6refx3jzx5H/9681P27z1AlRrnkrJiHZkZmRw7lsG2LTs470L3WmlebV9ObmhyHX1e\neB6Abdu2k5RU2vOYVStXZuzwwZ7HgfDvm7pxE78fOUL7Ll1p2+lpUlLXehovJz5f6I+8qGoTVW2q\nqk2BVcCjwL9E5KrgS5rhHOYuBxqJSIyIlAEuAlLzq2co1wG+FcJrQiIinjUn0tLSKF0q8fhyieho\n/H6/J7HmzP2I8uXKcW39q12fqPzokaP8c/pcerYfzPj+U+k2pBM+n48y5UpTp34t/j1nAQCbN/7M\npfUuIjY+ltJlSnHx5RcSF+/O2+vl9uUmKiqKHv0GMHTUWP56S3PP4zVr2oTo6GjP40B4902AuLg4\nWj3yIBPHjaJnt2dJ7tPf03g5ifL5Qn6cgvbAaBGZD1wLDFDVnTgtwcXA58ALqpqeX0GhHAJ/JyK9\ncY67f896UlUX5raCiNwOjMc5Hd1DVbOS6CfADSHEPGmJiYmkHT58fNnvDxAV5c113nM+/JioKB9L\nly1n/YZN9OjXn3EjhlGhfLkCl71183a2/bQTgG0/7eDAvkOUP6MsDa6/ki8/WnL8db/8uI25b37G\ngAnJ/LpjN+tTNnJg38ECxwdvty8vA/v0ZM/e33io9eN88NYbYTn8Dodw7psA1atWoWqwj7Fa1SqU\nKZPEr7v3cGalMzyLGQ6qmj13NMrh91OBqSdTZigJsDzQNPjIEiDvRNYD53g9CnhHROJU9TXw7g7p\nunVqs2DxEpo3u4HVa1K5oGYNr0IxY9LLx39+rEMnenfv5lpyaH5XU6pfWIVXBk6n/BnlSEiMY++v\n+7i8waW8Oel/k6knlS1NfGI8z7XqS0JiPAMmJbN548+u1MHL7cvJh5/MY+euXTzesgWxMTFERUUR\nFaZbqcLRwg3nvgnw/ocfsXHTD/To1pVdv+7mcNphzqhYwdOYJyoygyEEj71PVrqq/gYgIncC80Xk\nJ/Bub2vWtAlLly2nRZt2APTv3cOrUH/g9gc97/0v6Nq/PcNn9CUQ8DOq10QCgQDnVj+bHb/sOv66\nA/sOUqXGOYx5YwDH0jOYOnK2q/XIEo4d+camTej14kBatetIZmYm3bs+TUxMjOdxITzbF+598647\nbqN3/0G0fKIjUVE++vVM9rTFmZNIGQzBFwjknZNEpBrwKlAduA54A3hMVTfnsc5MYDfQS1XTRKQK\nMA8oq6rnhFKx9AN7wtP5lCWf98FtdzbqFLZYNh5gZAv3eICxZSsV+A0ddU//kL9QXd/tVWgfYCh/\nFiYBw3Guq9kJvAnMzGedx4AUgi0+Vf0Z5xD67VOuqTHGuCyUPsCKqvqZiAxV1QAwRUTyHM44eDX2\njBOe2wk8fco1NcZEjEg5BA4lAf4uIpUJtuZEpBFw1NNaGWMiWoQMBxhSAnwGZ+SF80XkW5yzwvd5\nWitjTEQrMi1AVV0RvOr6Qpw+Qw3lAkNjTPEVIfkvpFvhygBDcE58TAF6iUiC1xUzxhivhXIWeCqQ\nAbTCGWywNDDZwzoZYyJctC8q5EdhCqUPsKaq3ptt+WkRSfGqQsaYyFdkDoEBFZFrshZEpA6w0bsq\nGWMinceDIbgm1xagiPyIc+lLPHCviKwHMoGLsQRojCkC8joEvj5clTDGFC0RfxmMqm4BEJEY4Gbg\nxOFA8rsdzuQhnPfn3li/TdhiAXy+bFpY4wUyM8Iazxcd3oHUfVHhGbfQTRGS/0I6CfIJzjBWW7I9\nF8ASoDEmFxHfAsymoqrW8bwmxpgiI1JuhQvlLPB8EblRRAr3gh1jTMTw+XwhPwpTKC3An4DPgICI\ngHM4HFDVyOuYMMaERYQcAYeUAJ8CqqvqT15XxhhTNBT29X2hCiUBbgX2eF0RY0zRUdiHtqEKNQGm\nisgS4PgoMKr6mGe1MsaYMAglAX4UfBhjTEgipAEYUgL8wvNauCAjI4Pe/Qexdft2Mo5l0LZ1S65v\n/KepQ13j9/vpO3Aom3/6CZ/PR+/uz3F+jfM8iwewZ+9vPNCyDVPGj6F6taqulftw23toeMPVRJeI\nZs6bn7B6+VqSB3chEAjw44afGN1/EgC33XcTt99/M5kZmcyc+DZfL1jpWh0AUlLXMmb8BKZN9PYi\n8WPHjtGr/2B+2badUqUS6fH3Z6gSnEfXC4FAgAFDR6AbNhIbG0PfHsmexgv3dyEn4ZrWtKBCSYAL\ncC589gElgbOAVcBVHtbrpM39ZB5ly5ZhUL/e7D9wgPse9vZD/3LREnw+mDllAsu/WcXYVyYxbsQQ\nz+JlZGTQf8gw1ycLr3NVLWpdLjz54PPExcfywGP/R6fujzFl9OukrPyOrn3a0+iGq1m7Wrnnkdt4\n/O5niI2L5eU3hrB8ybdkZrgzY9n0WbP58ONPSYiPd6W8vLz7wYckJibw+qsT2PzTzwwcMZqJY0Z4\nFm/+lwtJT0/n9WmTSUldy/Ax4xg3Yqhn8cL9XchJkTkJoqp/aNaIyNVAnpMiFYabb2pG8xududoD\n/gAlSnh7u9INTa7j+usaArBt23aSkkp7Gm/kuPHcf89dTJ0xy9Vyr25Ulx82/sTA8ckkJMYzYcRr\n3HZvc1JWfgfAfxd+w1WNLscfCJCy8jsyM/0cTvudX7Zs43ypzoa137tSj6qVKzN2+GCSe7/oSnl5\n+f7HzTS6pj4A1atW4cfNW/JZo2C++XY1Da9pAEDtS2uxdt16T+OF+7sQyU764mZVXQbUO9n1RCRe\nRNxtvmQTHxdHQnw8aWlpPJvcgy4d2nkV6rioqCh69BvA0FFj+estzT2LM2fuR5QvV45r619NwOW5\n5cuUS0JqnU/vp4Yyst9Eeg3vii/b4cvhw7+TkJhAQmI8aYcOH3/+98NHKFXKvYHBmzVtQnR0eC4t\nlQsuYMHipQCsTl3Lr7/uJr/5sQsiLS2N0qUSjy+XiI7G7/d7Fq8wvgsn8vlCfxSmfP80iEjvbIs+\n4BKc+YHzW+8SYBDwGzAbZ3L1TBF5SlXnnlp187Zjx06e7pbMg/ffyy3Nb/QixJ8M7NOTPXt/46HW\nj/PBW2+4fogKMOfDj4mK8rF02XLWb9hEj379GTdiGBXKnzg+xck7sO8gW77/hcxMP79s3kb60XTO\nOLPC8d8nJMRz6GAaaYcOk5iY8KfnI9Fdt9/Kj5u30Kp9J+rWvoyLLxJPL9tITEwk7fD//nj4/QGi\nory9saowvgvZRcplMKF8Cr5sjwBOn2Aos8JNBEYDXwL/BK4G6gLJp1LR/Ozes5d2XZ6ha5eO3Hnb\nrV6E+IMPP5nHq685h6OxMTFERUV51vE7Y9LLTJswnmkTxnPRhTUZ2KeXK8kPIGXld9S/7goAKlQq\nT3x8HN98nUKdq2oBUL/xFaSsWMv6NRu5rN7FlChZgsRSCVStUZkfNrp/bbzbLdycpK5bT/2r6jFj\n4nhuuuF6Kp97jqfx6tapzaIlwRbnmlQuqFnD03jh/i7kpMi0AFW13ymWHaWqC4AFItJUVXcBiIgn\nYxdNnTGTgwcPMWnqdCa+Oh2fDyaMHUVMTIwX4bixaRN6vTiQVu06kpmZSfeuT3sWKzsf7u4xXy9Y\nSZ16tZj09nDw+RjZbyI7tu6k24BOlChRgi3f/8KX874C4N1Zc3l59mDw+Zg8ehYZx9z/KN3evpxU\nq1KZbpP6MWXGTJJKl6Zfj+c9jdesaROWLltOizbOoWj/3j08jRfu70JOIqUF6Mut7yPbiNA5UtU8\n/4yJyNTg+k+oqj/4XHegrqr+Lb+KpR/Y431TIDsP+4AKm40H6K5wjwcYbjFJFQqcvd7tODbkL9Q9\nLz9VaNnyZEeEfgjoAYwJoey2wO1ZyS/oF2BcyLUzxkSkiL8MJmtEaAAROQOYBNQEmqhqvlfABhPf\nByc89/qpV9UYEykiJP+FNDH6g8AaYC1QL5TkZ4wp3iJ+PMBgq28icAFwq6p+E7ZaGWNMGOTVB7gO\nSATeAzoFB0M9zkaDMcbkJlIOgfNKgM+GrRbGmCIl4gdDUNXXwlkRY0zRUdh9e6GyiY6MMcVW0b6i\n0xhTKCKkAWgJ0Bjjvkg5BM7rMhg/f7wV7hjgB2KBA6rqzt34xpgiJ0LyX54nQaIARGQCsASYraoB\nEbkHuCVM9QufSPnETkG478294aqWYY03f7mdrzvdRMqtcKGcBKmvqq+ragBAVd/lNBsO3xhzeiky\nw2EBaSLSGngbJ2G2wOYJNsbkwc0+QBGJAqYAgtMN1x44CswILqeqasfga9sCT+B02Q1U1TxntAyl\nBfgIcDewA2c0l2Y4SdAYY3LkcgvwdiCgqo2AXjgjzY8CXlDVJkCUiNwpImcCnYFrcLrpBotIybwK\nDmVA1C3A7SJSXlX3hlRdY0yx5nPxThBV/UBEPgwuVsOZZuNGVV0UfO4ToDlOa3CxqmYAB0RkI1Ab\nyHUAl1DmBLkc+AeQICINgIXA/TY4gjEmN2737amqX0RmAP+HMyXHTdl+fRBIAkoD+7M9fwgok1e5\noRwCjwPuAvao6jagA84oMcYYEzaq2gq4EGeCtewTSJcG9gEHcBLhic/nKpQEmKCq67JV4t841wIa\nY0yO3BwPUEQeCU6nAXAEyARWiEiT4HN/ARYBy4FGIhIjImWAi4DUvMoO5SzwXhGpQ/CiaBF5GLC+\nQGNMrlweDeY9YLqILMDJWV2A9cCrwZMc64B/Bq9THgcsxpnF8gVVTc+r4FASYAfgNaCWiOwDNgIP\nn/KmeCQQCDBg6Ah0w0ZiY2Po2yOZKpXPLTLxAFJS1zJm/ASmTRzvaZyMjAx69x/E1u3byTiWQdvW\nLbm+cSPXyn+k7b00vKE+JUpG8/4bH/Pt8lR6DHkGv9/Pjxu3MOpFp4elQeN6tHryQQB07SZG93en\n56UwPjsI3+dXWPGyc7MPUFUPAzlNpHZ9Dq+dCkwNtexQDoHjgqefywNVVfWq4M+nlflfLiQ9PZ3X\np03mqY4/m8iMAAAbgUlEQVQdGD7G27mXwh1v+qzZ9B04hPT0PP+guWLuJ/MoW7YMr02ewCtjRzJo\n+EjXyr78qkupVfciOjz4HJ1bJHPm2WfQOflxJo2aSecWyfiiomjUrD7xCXF0+HtrnmvXl/YP/J0d\nW3eSVLa0K3UI92cH4f38CiNepMo1AYpIQxFpDLwvItcB9YDLReQGYGa4Khiqb75dTcNrGgBQ+9Ja\nrF23vkjFq1q5MmOHD/Y0Rpabb2pGp/ZPABDwByhRwr0xM65udAU/btjCoJd7MGRCb776chkXXnI+\nKSvXAvD1whVcdW1dLq17MT9s2Ezn7o8z/vUh7N2zjwP7DrpSh3B/dhDez68w4v1JhNwKkteefRPQ\nBDgbeDHb8xk4M8SFTEQqZU2M7pW0tDRKl0o8vlwiOhq/309UlDdDHoY7XrOmTdi2fbsnZZ8oPi4O\ncLbx2eQedOnQzrWyy5ZL4sxzzqBbu36cU+Ushkzo9Yf37HDa7ySWSqBsuSTq1r+MVnd05siRo7w8\neyipq9az9aeCvwfh/uwgvJ9fYcQ7UcSPBqOqfQFEpAXwpqpmBDscY1Q1La9CReTCE56aKSKPBsvd\nULAq5ywxMZG0w4ePL/v9AU936HDHC7cdO3bydLdkHrz/Xm5pfqNr5e7fd5DN3/9MZqafnzdvI/3o\nMc44M+H47xMS4zl44BD79x1k3ZqN7PvtAACrV6zlgotruJIAi/pndzqIkPwXUh/gUWBV8OeqwHoR\nuTOfdT4H/oVzveAknHv4JuHh9YN169Rm0ZKlAKxek8oFNWt4FapQ4mUJ/GGEMm/s3rOXdl2eoWuX\njtx5262ulp2yci31r6sHQIVK5YmLj2Pl16u5/KpLAWjQ+EpSVq5lw9pN1LigGqXLlCI6OopadYTN\nm35ypQ6F9dlBeD6/woyXxRflC/lRmELp3OkJ3Aigqt+LSD3gM06Y9PwEV+Ikuwmq+m8R+UJVmxa4\ntnlo1rQJS5ctp0Ub53Ctf+8eXoYLe7wsPrzfYabOmMnBg4eYNHU6E1+djs8HE8aOIiYmpsBlL12w\ngjpX1mLyO6Pw+WBk35fZvnUX3Qd0IbpkNFu+/5kvPl0CwKRRrzF6Wn8CgQDzP17E5u9/LnB8KLzP\nDsLz+RVmvONxI6QF6AsE8v4LISLrVfWiE55brap18lmvBDAC2AXcdLIJMP3AnsL502UKzMYDjGwx\nSRUKnL6+Hjw95O9vg+TWhZYuQ2kBLhaRN4HZweX7gaX5rRS8IflpEWmFTb5kTLESKS3AUBJgR5wh\nZtrhjLG1EHgl1ACqOgNn3C5jTDER8WeBReQsVd0BnIkzGOrb2X59FuBOj7QxxhSSvFqArwK3AQtw\n7gP2nfB/+E6dGWMiSoQ0APO8DvC24P/nha86xpiiwBcdGRkwr0PgPKcSU9XH3K+OMaYoiJQ+wLzO\nzi4IPkoD5wDzca7/K5fPesYYExHyOgR+DUBEngSuUVV/cPlt4OvwVM8YE4kipAEY0mUwZXCGv9od\nXD4TKOVZjYwxES9SDoFDSYADgRQRWQJEA/Vxrgs0xpgcRUj+C2lazFki8jlwLc7lL+29HtrKGBPh\nIiQDhjItZgzQGmeCkc7AUyIyJL+x9gsq4M/0svg/8UVFhzVeOAUyM8IaL9z35rZtHr7BDAAmf/pi\n/i9yUSTum4U9ykuoQjmb+zJOn98VOLfC1eQkxtw3xhQ/ETIgdEgJsJ6qvgAcC05O0hKo6221jDGR\nzM1pMb0UykmQQPAwOGt4m4rZfjbGmD8p7JZdqEJpAY7BGeH5LBEZA6wARntaK2OMCYNQWoCfACuB\npjiXwdyuqime1soYE9kipAkYSgJcpKoXA995XRljTNEQKWeBQ0mAq4Mzwy0Dfs96UlVtPEBjTI6K\nUgKsH3xkZ+MBGmMiXih3gkTMeIB/e7QNpUo5tylXPuds+vXs7mm8lNS1jBk/gWkTx3saByAQCDBg\n6Ah0w0ZiY2Po2yOZKpXP9SRWRkYGPV8cxLbtO4guEU2f5G5Ur1rFk1jg/bb1m5HM4UPOwcuv2/bw\nyRv/pnX3hwHY+fMupg16neyTg3Ud1YlvFnzLlx8sdq0O4dw3w7mv5CZCugDzHA/wHGA8cAGwGEhW\n1X3hqtjJSk93bkyZ+srYsMSbPms2H378KQnx8WGJN//LhaSnp/P6tMmkpK5l+JhxjBsx1JNYi776\nGr/fz8wpr/D1shWMmzCZUYP7exILvN22EiWdXXxopzHHn+sypD3vvDKHjSnf83jPR7m8UW1WLVoN\nwD3t7yShlLufabj3zXDuK7mJlEPgvC6DmQ6sB54D4ijApS8iEiUi54qIZ+MI6sZN/H7kCO27dKVt\np6dJSV3rVSgAqlauzNjhgz2Nkd03366m4TUNAKh9aS3WrlvvWazqVauQkZlJIBDg4KFDlCxZ0rNY\n4O22Vb2gMjFxsfx9TGe6vfQUNWpVZ1z3iWxM+Z7oEtGUqZDE72lO6/DKpnXxZ/pZ87W7+064981w\n7iu5iZQLofNKSOeq6guq+inwBHD1yRQsIlOD/9cHNgDvAaki0uBUK5uXuLg4Wj3yIBPHjaJnt2dJ\n7tMfv9/vRSjAmVw7Ojp892impaVRulTi8eUS0dGebV98Qjxbt23njr89Qv+hI3no/ns8iZPFy207\neiSdT2Z/xoinX+K1YW/Svu9j+Hw+yp9ZjkFv9KZUUil+2vgL59Y4hwbNr+L9KR+6/qUM974Zzn0l\nV76TeBSivPoAjw92oKrHRORkBz/I6jscCPxFVTcGD6vfBJqcZFn5ql61ClWD/RzVqlahTJkkft29\nhzMrneF2qEKRmJhI2uHDx5f9/gBRUd40qF9/820aNqhPlw5t2bnrVx7v+BTvvfGaZy1BL7dtx087\n2fWLM3jRzp93cehAGmUrlmHvzt94/v4+NL79Wh56+j7279lPuYpl6f7yM1Q8uwLH0jPYvX0PqcvW\nFbgO4d43w7mvRLqTeVdO9fa3TFXdCKCq204yZsje//AjRox9GYBdv+7mcNphzqhYwYtQfxAI012B\ndevUZtESZz761WtSuaCmdyfhk5KSKBVsQZQuXYqMzEwyM71rQXi5bY1vv5YHutwLQNmKZYhPjKPV\n8w9RqbKTfI4cPoo/0887r8yhf9thDOk4msUfLWXem5+7kvwg/PtmOPeV3ERFRYX8KEx5tQBricgP\n2ZbPDS77gICq5veulhGRlUCiiLQBZgMjgS0FqnEu7rrjNnr3H0TLJzoSFeWjX8/ksLy5vjC14Zs1\nbcLSZctp0aYdAP17ezcE1CMP3EefAUNo1b4TGRmZPNXhCeLiYj2L5+W2LfzXEh7v1ZIXJj5LwB/g\n1f4zAWjbqyXH0jNIP5rOtEGzXIuXk3Dvm+HcV3IVIQ1OX/bT/9mJSLW8VlTVfBOZiMQCdYDDOP2A\njwFTVfVYfuse3bcrrAMuROKYa6EK93iAvuhQLi91j40H6K6YpAoF/qu+6Y33Qv7+1nzo7kLrCcxr\nUqQCt9RU9SjOHSRZJha0TGOMcUt4/1QbY4qFwr68JVSWAI0x7ouM/GcJ0Bjjvki5E8QSoDHGfXYI\nbIwpriIk/1kCNMa4z06CGGOKL+sDNMYUV5HSAoyQG1aMMcZ9p20LsCjfmhZu4b41LdymzBsQ1njN\nGzwe1nif/XdqWOO5we3LYILD6g1R1aYicjkwF+f2WoAJqvqOiLTFGbrvGDBQVT/Kr9yi/c0wxhQK\nNxOgiDwHtAAOBZ+qB4xU1dHZXnMm0Bm4AkgAFovIZ/mNO2AJ0BjjPnf7ADcBdwFZw/bUAy4Ukf/D\naQU+gzNg82JVzQAOiMhGoDbOnOa5sj5AY4zr3BwSX1XfB7IPafRf4DlVbQL8APQBkoD92V5zCCiT\nX9mWAI0xkWaOqq7K+hm4HCf5JWV7TWkg30ncLAEaY9zn7Zwg80TkyuDPzXAOc5cDjUQkRkTKABcB\nqfkVZH2AxhjXeTwYQgfgpeA8RTuAJ1T1kIiMw5nC1we8oKr5zmNkCdAY4zqfy0P+Bwdovjb48yqg\nUQ6vmQqc1DVDdghsjCm2rAVojHGf3QscXoFAgAFDR6AbNhIbG0PfHslUCc7FWhTiAaSkrmXM+AlM\nmzje0zjF4b0E2LP3Nx5o2YYp48dQvVpV18qd9M+RpB1MA2D71l28P/sjuvbpQHp6OpvW/cj4wVM5\nX6rTKbkNgUAAn8/HJXUupEfHwaz46tsCxy+s9zO7SLkXuMgkwPlfLiQ9PZ3Xp00mJXUtw8eMY9yI\noUUm3vRZs/nw409JiI/3LEaWov5eAmRkZNB/yDDXp/ssWdL5SnVt3fv4cxPfHs7YAZNZl7KR1p0f\npNlfG/OfjxbyTKteADRpfg2/7tzjSvKDwnk//yQy8l/4+gBFpKKIePa2fPPtahpe0wCA2pfWYu26\n9V6FKpR4VStXZuzwwZ7GyFLU30uAkePGc/89d1GpYkVXyz3/ovOIj49l2JQ+jJzWj4trX0DFMyuw\nLmUjAGtXreeyehcff31sXAytOj/IS4Neda0OhfF+nsjNC6G95FkLUERaA1Vwblp+AzgCJIjIk6r6\nudvx0tLSKF0q8fhyieho/H6/ZxNQhztes6ZN2LZ9uydln6iov5dz5n5E+XLluLb+1bw6Y6arZR89\ncpR/TJvDx+9+zrnVzmbYpN5s/3kHta+8hJQV33FN06uIi/9fq/PWe27iy0+WcHD/oTxKPTnhfj8j\nmZeHwE8C1wP/Au5Q1Q0icg7wAeB6AkxMTCTt8OHjy35/wNMPPNzxwqmov5dzPvyYqCgfS5ctZ/2G\nTfTo159xI4ZRoXy5Apf98+ZtbN3i/KHaumU7+/cdZPLo13nkiXt4tMP9pKz8jvSj/0tON93emN5d\n3D08PR32TV90ZHwXvKzlMVVNAw7i3K+Hqm4DQp4x/mTUrVObRUuWArB6TSoX1KzhRZhCi5cl4M3b\n9wdF/b2cMellpk0Yz7QJ47nowpoM7NPLleQHcOvdzejwfGsAKpxRjoTEeC6pfQH9nxvF39v0pUzZ\nJFYG+/oSEuMpUbIEu3ftdSV2lsLaN//A5wv9UYi8bAH+S0Q+wLkdZa6IzANuAeZ7EaxZ0yYsXbac\nFm3aAdC/dw8vwhRavCy+MPQuF5f3Etx/Pz9693O6D+zMuFmD8Af8DOv5EmXKJTF6en+O/H6EVctS\nWbbYuY21SvVz2LF1l6vxoXDfzyyF3bcXKl8g4F2LQkSaADcDFYE9OMPV5DtIIUD6gT3eN3VM0eDh\nPpyToj4gakxShQJnr11LFob8oVRq2LjQsqWnl8Go6gJggZcxjDGnIbsQ2hhTXEXKIbAlQGOM+ywB\nGmOKK4+Hw3KNJUBjjPusBWiMKa6sD9AYU3xZAjTGFFeR0gcYGTfsGWOMB6wFaIxxnx0CmzyF+fat\ncAr4M8Mazxcd3t043Lem3Vw/vLfefbHu/QKX4fakSF6xBGiMcZ/1ARpjzOnNWoDGGNf5fJHRtrIE\naIxxn50EMcYUV3YniDGm+IqQkyCWAI0xrrMWoDGm+LIEaIwptuwscPilpK5lzPgJTJs43vNYGRkZ\n9O4/iK3bt5NxLIO2rVtyfeNGnsbcs/c3HmjZhinjx1C9WlXP4vj9fvoOHMrmn37C5/PRu/tznF/j\nPM/iZWRk0PPFQWzbvoPoEtH0Se5G9apVPIsXCAQYMHQEumEjsbEx9O2RTJXK50ZkvEn/HMGhg84c\nwDu27uT92R/TtU970tOPsWndj4wf/Me7VgZP6smSz//L3Hf+7Ur83NhgCGE2fdZs+g4cQnp6elji\nzf1kHmXLluG1yRN4ZexIBg0f6Wm8jIwM+g8ZRlxcrKdxAL5ctASfD2ZOmUCn9m0Z+8okT+Mt+upr\n/H4/M6e8QrvWLRk3YbKn8eZ/uZD09HRenzaZpzp2YPiYcREZr2RJp/3ybOvePNu6N8N7vsyz/Trw\n0qBXefrRnqQdOkyzv153/PVtnnqYUqUTcyuuWPIsAYpIkldl56Rq5cqMHT44bPFuvqkZndo/AUDA\nH6BECW8b0yPHjef+e+6iUsWKnsYBuKHJdfR54XkAtm3bTlJSaU/jVa9ahYzMTAKBAAcPHaJkyZKe\nxvvm29U0vKYBALUvrcXadesjMt75F51HXHwsw6b0ZsS0vlxc+wIqnlmBdSkbAUhdtZ7L6l0MQOOb\nGuD3+1kenJPYcxEyMbqXLcAdItLGw/L/oFnTJkRHR4crHPFxcSTEx5OWlsazyT3o0qGdZ7HmzP2I\n8uXKcW39qwkQnkEUoqKi6NFvAENHjeWvtzT3NFZ8Qjxbt23njr89Qv+hI3no/ns8jZeWlkbpUv9r\nCZWIjsbv90dcvCNHjvLWtDl0a/sio/tNosewZ9j+805qX3kJANc2vYq4+Diq16xCs9saM/2lNwsc\nM1S+qOiQH4XJy2bLaqCuiMwH+gXnCC5SduzYydPdknnw/nu5pfmNnsWZ8+HHREX5WLpsOes3bKJH\nv/6MGzGMCuXLeRYTYGCfnuzZ+xsPtX6cD956w7PD79fffJuGDerTpUNbdu76lcc7PsV7b7zmWUsw\nMTGRtMOHjy/7/QGiPBy9xKt4v2zextYt2wHYumU7B/YdZMroWTz8xL206HAfa1auI/1oAs3vvJ4K\nlcozasaLnHVuJY6lH2PH1l2s+Gp1geuQm0jpA/QyAf6uqp1E5EogWUTGA/8BflBVzzpdwtVC2r1n\nL+26PEOPbs9y9ZX1PI01Y9LLx39+rEMnenfv5mny+/CTeezctYvHW7YgNiaGqKgoojzcoZOSko53\nIZQuXYqMzEwyM/14dSRct05tFixeQvNmN7B6TSoX1KzhTSCP4/3l7mbUuLAaY/tPpsIZ5YhPjOfi\n2hcy4LlRHDqQRucX2vD1wm/+cNj76JP3s/fX3zxNfpHEywToA1DVFcA9IlIGaAyIhzHxEZ6/PFNn\nzOTgwUNMmjqdia9Ox+eDCWNHERMT42nccGzfjU2b0OvFgbRq15HMzEy6d33a0+165IH76DNgCK3a\ndyIjI5OnOjzh6cmeZk2bsHTZclq0cbot+vfu4VksL+N9/O7nPD+wM2NnDSQQCDCs53jKlEti1PQX\nOfL7UVYtWxO+Pr8TRch1gL6ARwNzikhLVX3tVNdPP7Cn6I4YCjYgqovCPSBquBXCgKgFzl6Ht/0Y\n8g6ecM55hZYtPdtzCpL8jDERzi6ENsYUW3YSxBhTXNlgCMaY4ssOgY0xxZW1AI0xxZe1AI0xpuBE\nxAe8AtQBjgCPq+oPbpQdGWnaGBNRfFG+kB8h+D8gVlWvBZKBUW7V0xKgMcZ97o4G0wj4FEBV/wtc\n6VY1LQEaY1zn8mgwScD+bMsZIuJK7jpt+wBjkipExmkkYzz2xbr3C7sKJ83l7+8BIPuglFGq6sr4\nZdYCNMac7pYAtwKISANgjVsFn7YtQGOMCXofuElElgSXW7tVsGejwRhjzOnODoGNMcWWJUBjTLFl\nCdAYU2wVmZMgXt4uk0fM+sAQVW3qcZwSwDSgOhADDFTVDz2MFwVMwZm+wA+0V9XvvIqXLW4lYAVw\no6pu8DjWSv53bdmPqurpDIYi0h24AygJvKKq0z2M1RJoBQSAeJzvxFmqesCrmJGqKLUAPbtdJici\n8hxOkvB+pnJ4BNitqo2BvwDjPY53OxBQ1UZAL2CQx/GykvxE4HB+r3UhViyAqt4QfHid/JoA1wT3\nzeuBKl7GU9XXVLWpqt4ArAQ6W/LLWVFKgJ7dLpOLTcBdHsfI8jZOIgLnMzvmZTBV/QB4IrhYHfjN\ny3hBI4AJwLYwxKoDJIrIPBH5PNiS99LNQKqIzAH+Bcz1OB4AwRkZL1HVqeGIF4mKUgL07HaZnKjq\n+0CGV+WfEOuwqqaJSGngHcDbacycmH4RmQGMBWZ7GUtEWgG7VPXfEJZp/Q4Dw1X1ZqADMNvLfQWo\nCNQD7g3Ge8PDWNklA/3CFCsiFaUE6NntMqcDEakCzAdeU9W3whFTVVsBFwKviki8h6Fa41zo+gVw\nOTAz2B/olQ0Ek7qqbgT2AGd7GG8PME9VM4J9m0dEpKKH8QhOQ3uhqi7wMk6kK0oJ0LPbZfLheYtF\nRM4E5gHdwjHbnog8Euy0B+eEUibOyRBPqGqTYJ9VU+Bb4FFV3eVVPOAxYCSAiJyD84dzu4fxFgO3\nZIuXgJMUvdQY+I/HMSJekTkLjIe3y+QjHLfSJANlgV4i0jsY8y+qetSjeO8B00VkAc4+8pSHsU4U\njvdzKs72LcJJ7I95ebSgqh+JyHUisgznD+aTqur1dgrg6VUQRYHdCmeMKbaK0iGwMcacFEuAxphi\nyxKgMabYsgRojCm2LAEaY4otS4DGmGLLEmARIyKXiohfRPK8T1lEqovIqwWI86fr5kSkpYjsFpEz\nsj1XTUR+PNU4bpdjTHaWAIueVjj3C7fP53XVgRoFiJPbBaSJOKO6hPJat2Iac0qK0p0gxZ6IROMM\nndUIWCoi56nqjyJyI85oKz5gC/AwziAH54nIS8A/gb5Z4xqKyHTgC1WdKSIDgRuAcsBu4O58blN7\nD7hMRB5U1TdPqF8fnGG2Xgwu/wg0AZoCfwXODT7GAlWDcXfjDAEGEC8ib+Hc5bAJaKOq+4OjnozG\nGftuN9BOVbcE7y3eC1wC/E1VU072PTVFm7UAi5bbgM2qugnn1sB2IhIDvA60UNU6QArwKNAZWKGq\nnYPr/ql1JSLn49xQf42qXgR8j5M8Ifd7oI/itEJHZz8UzkX2mFcBzXHuYR0JfBSsrw9nOCmASsAY\nVb08WJfeIlISeBV4UFWvxBkHMvuh/WpVvdiSn8mJtQCLllZAVqvrHZzE9y7wi6quAVDVnnB8kM48\nqer3IvJ3EWmL0+pqgNPyym+9b0RkCjAJeCaPl2ZPoktUNQ1IE5EAzsg34LRYywV/Xq+qS4M/vw7M\nwBmt5nzgX8FRwQFKZSv3v/nV1xRflgCLiGBr61agnog8hdO6L8v/Dh+zXpfEH4cNA6cllj0ZlQy+\n9gqchDoSJ6FmEvroNy/ijEb8UH5xgtKzr5zL4ASZ2X724QwMGwV8r6pXBOvsA87M9rrfQ6yvKYbs\nELjoaAF8rqpVVbWGqlYHBuIkwDNE5KLg67oB7XAGc81KQLuBGiISIyLlgeuCzzfB6QucDKzHOUSN\nDqUyqnoMZ0Se7IO37sbpj0NErgbOymX13JLsxSJSJ/jzY8C/AQXKi0ij4POPE74BR02EswRYdLQE\nXj7huQnAZTgnRmaJyLfAxcAQYB1QRkReC0549DGwFngLWBhc/x/A5cH1PgdWA+cFf5fvGVlVXQmM\nyfbUP4CKIpIKdARW5bJqIJefN+L0+6XgjLI8WFXTgfuAkcF6tsBJjiHV0RRvNhyWMabYshagMabY\nsgRojCm2LAEaY4otS4DGmGLLEqAxptiyBGiMKbYsARpjii1LgMaYYuv/AVnNIeShiXigAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x164652b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Model accuracy - training data\n",
    "print 'Precision:' + str(metrics.precision_score(y_test, pred_y_test, average = 'weighted', pos_label='high'))\n",
    "print 'Recall: ' + str(metrics.recall_score(y_test, pred_y_test, average = 'weighted', pos_label='high'))\n",
    "print 'Accuracy: ' + str(metrics.accuracy_score(y_test, pred_y_test))\n",
    "\n",
    "# Fancy boy confusion matrix - training data\n",
    "mat = metrics.confusion_matrix(y_test, pred_y_test)\n",
    "sns.heatmap(mat.T, square=True, annot=True, fmt='d', cbar=True)\n",
    "plt.xlabel('Actual Number')\n",
    "plt.ylabel('Predicted Number')\n",
    "plt.title('MLP - Test Performance')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2 - Optimize MLP Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 0.63780467\n",
      "Iteration 2, loss = 0.26993412\n",
      "Iteration 3, loss = 0.24531053\n",
      "Iteration 4, loss = 0.23249797\n",
      "Iteration 5, loss = 0.22329679\n",
      "Iteration 6, loss = 0.21626242\n",
      "Iteration 7, loss = 0.21060256\n",
      "Iteration 8, loss = 0.20636226\n",
      "Iteration 9, loss = 0.20275525\n",
      "Iteration 10, loss = 0.19956171\n",
      "Iteration 1, loss = 0.63829233\n",
      "Iteration 2, loss = 0.27025643\n",
      "Iteration 3, loss = 0.24654183\n",
      "Iteration 4, loss = 0.23410992\n",
      "Iteration 5, loss = 0.22552123\n",
      "Iteration 6, loss = 0.21804481\n",
      "Iteration 7, loss = 0.21292405\n",
      "Iteration 8, loss = 0.20858724\n",
      "Iteration 9, loss = 0.20470124\n",
      "Iteration 10, loss = 0.20241500\n",
      "Iteration 1, loss = 0.63081829\n",
      "Iteration 2, loss = 0.26626237\n",
      "Iteration 3, loss = 0.24312374\n",
      "Iteration 4, loss = 0.22986850\n",
      "Iteration 5, loss = 0.22091677\n",
      "Iteration 6, loss = 0.21371822\n",
      "Iteration 7, loss = 0.20887361\n",
      "Iteration 8, loss = 0.20509427\n",
      "Iteration 9, loss = 0.20130855\n",
      "Iteration 10, loss = 0.19839854\n",
      "Iteration 1, loss = 0.60657526\n",
      "Iteration 2, loss = 0.22005063\n",
      "Iteration 3, loss = 0.18682238\n",
      "Iteration 4, loss = 0.16716691\n",
      "Iteration 5, loss = 0.15196110\n",
      "Iteration 6, loss = 0.13962892\n",
      "Iteration 7, loss = 0.12912051\n",
      "Iteration 8, loss = 0.12056776\n",
      "Iteration 9, loss = 0.11294147\n",
      "Iteration 10, loss = 0.10638375\n",
      "Iteration 1, loss = 0.60690770\n",
      "Iteration 2, loss = 0.22004360\n",
      "Iteration 3, loss = 0.18788125\n",
      "Iteration 4, loss = 0.16889224\n",
      "Iteration 5, loss = 0.15457528\n",
      "Iteration 6, loss = 0.14202926\n",
      "Iteration 7, loss = 0.13243389\n",
      "Iteration 8, loss = 0.12403781\n",
      "Iteration 9, loss = 0.11650343\n",
      "Iteration 10, loss = 0.11071412\n",
      "Iteration 1, loss = 0.59941831\n",
      "Iteration 2, loss = 0.21620908\n",
      "Iteration 3, loss = 0.18488508\n",
      "Iteration 4, loss = 0.16496688\n",
      "Iteration 5, loss = 0.15008771\n",
      "Iteration 6, loss = 0.13782905\n",
      "Iteration 7, loss = 0.12843622\n",
      "Iteration 8, loss = 0.12048553\n",
      "Iteration 9, loss = 0.11278761\n",
      "Iteration 10, loss = 0.10651958\n",
      "Iteration 1, loss = 0.60334773\n",
      "Iteration 2, loss = 0.21464833\n",
      "Iteration 3, loss = 0.18020430\n",
      "Iteration 4, loss = 0.15948147\n",
      "Iteration 5, loss = 0.14328362\n",
      "Iteration 6, loss = 0.13001486\n",
      "Iteration 7, loss = 0.11860901\n",
      "Iteration 8, loss = 0.10920139\n",
      "Iteration 9, loss = 0.10075090\n",
      "Iteration 10, loss = 0.09343127\n",
      "Iteration 1, loss = 0.60366405\n",
      "Iteration 2, loss = 0.21460377\n",
      "Iteration 3, loss = 0.18124167\n",
      "Iteration 4, loss = 0.16122386\n",
      "Iteration 5, loss = 0.14594845\n",
      "Iteration 6, loss = 0.13250119\n",
      "Iteration 7, loss = 0.12205182\n",
      "Iteration 8, loss = 0.11283304\n",
      "Iteration 9, loss = 0.10451663\n",
      "Iteration 10, loss = 0.09794638\n",
      "Iteration 1, loss = 0.59617305\n",
      "Iteration 2, loss = 0.21078484\n",
      "Iteration 3, loss = 0.17828764\n",
      "Iteration 4, loss = 0.15732605\n",
      "Iteration 5, loss = 0.14145815\n",
      "Iteration 6, loss = 0.12829625\n",
      "Iteration 7, loss = 0.11803868\n",
      "Iteration 8, loss = 0.10926194\n",
      "Iteration 9, loss = 0.10074601\n",
      "Iteration 10, loss = 0.09373623\n",
      "Iteration 1, loss = 0.32769650\n",
      "Iteration 2, loss = 0.16996961\n",
      "Iteration 3, loss = 0.13890580\n",
      "Iteration 4, loss = 0.12263480\n",
      "Iteration 5, loss = 0.11061077\n",
      "Iteration 6, loss = 0.10305921\n",
      "Iteration 7, loss = 0.09770082\n",
      "Iteration 8, loss = 0.09333071\n",
      "Iteration 9, loss = 0.08909739\n",
      "Iteration 10, loss = 0.08707868\n",
      "Iteration 1, loss = 0.33041193\n",
      "Iteration 2, loss = 0.17399560\n",
      "Iteration 3, loss = 0.14429026\n",
      "Iteration 4, loss = 0.12604185\n",
      "Iteration 5, loss = 0.11450893\n",
      "Iteration 6, loss = 0.10691431\n",
      "Iteration 7, loss = 0.09997780\n",
      "Iteration 8, loss = 0.09515561\n",
      "Iteration 9, loss = 0.09154212\n",
      "Iteration 10, loss = 0.08872786\n",
      "Iteration 1, loss = 0.32565014\n",
      "Iteration 2, loss = 0.17308850\n",
      "Iteration 3, loss = 0.13977370\n",
      "Iteration 4, loss = 0.12101251\n",
      "Iteration 5, loss = 0.11011633\n",
      "Iteration 6, loss = 0.10172117\n",
      "Iteration 7, loss = 0.09610960\n",
      "Iteration 8, loss = 0.09152365\n",
      "Iteration 9, loss = 0.08826406\n",
      "Iteration 10, loss = 0.08494918\n",
      "Iteration 1, loss = 0.29506389\n",
      "Iteration 2, loss = 0.13183500\n",
      "Iteration 3, loss = 0.09719699\n",
      "Iteration 4, loss = 0.07892097\n",
      "Iteration 5, loss = 0.06479694\n",
      "Iteration 6, loss = 0.05513066\n",
      "Iteration 7, loss = 0.04824435\n",
      "Iteration 8, loss = 0.04313394\n",
      "Iteration 9, loss = 0.03800487\n",
      "Iteration 10, loss = 0.03420222\n",
      "Iteration 1, loss = 0.29803052\n",
      "Iteration 2, loss = 0.13659130\n",
      "Iteration 3, loss = 0.10373084\n",
      "Iteration 4, loss = 0.08280233\n",
      "Iteration 5, loss = 0.06962321\n",
      "Iteration 6, loss = 0.05968517\n",
      "Iteration 7, loss = 0.05172204\n",
      "Iteration 8, loss = 0.04527722\n",
      "Iteration 9, loss = 0.04006127\n",
      "Iteration 10, loss = 0.03624973\n",
      "Iteration 1, loss = 0.29314425\n",
      "Iteration 2, loss = 0.13566467\n",
      "Iteration 3, loss = 0.09934579\n",
      "Iteration 4, loss = 0.07841785\n",
      "Iteration 5, loss = 0.06605170\n",
      "Iteration 6, loss = 0.05505128\n",
      "Iteration 7, loss = 0.04789957\n",
      "Iteration 8, loss = 0.04135617\n",
      "Iteration 9, loss = 0.03697052\n",
      "Iteration 10, loss = 0.03309285\n",
      "Iteration 1, loss = 0.29155400\n",
      "Iteration 2, loss = 0.12737423\n",
      "Iteration 3, loss = 0.09213668\n",
      "Iteration 4, loss = 0.07330185\n",
      "Iteration 5, loss = 0.05840143\n",
      "Iteration 6, loss = 0.04824502\n",
      "Iteration 7, loss = 0.04122594\n",
      "Iteration 8, loss = 0.03527789\n",
      "Iteration 9, loss = 0.02975104\n",
      "Iteration 10, loss = 0.02537422\n",
      "Iteration 1, loss = 0.29462037\n",
      "Iteration 2, loss = 0.13244832\n",
      "Iteration 3, loss = 0.09886417\n",
      "Iteration 4, loss = 0.07707905\n",
      "Iteration 5, loss = 0.06322731\n",
      "Iteration 6, loss = 0.05288853\n",
      "Iteration 7, loss = 0.04449314\n",
      "Iteration 8, loss = 0.03735601\n",
      "Iteration 9, loss = 0.03142793\n",
      "Iteration 10, loss = 0.02733370\n",
      "Iteration 1, loss = 0.28980615\n",
      "Iteration 2, loss = 0.13125853\n",
      "Iteration 3, loss = 0.09404097\n",
      "Iteration 4, loss = 0.07291184\n",
      "Iteration 5, loss = 0.05977777\n",
      "Iteration 6, loss = 0.04829043\n",
      "Iteration 7, loss = 0.04096340\n",
      "Iteration 8, loss = 0.03387250\n",
      "Iteration 9, loss = 0.02931666\n",
      "Iteration 10, loss = 0.02473635\n",
      "Iteration 1, loss = 0.24919265\n",
      "Iteration 2, loss = 0.11119727\n",
      "Iteration 3, loss = 0.08101890\n",
      "Iteration 4, loss = 0.06582402\n",
      "Iteration 5, loss = 0.05551116\n",
      "Iteration 6, loss = 0.04768295\n",
      "Iteration 7, loss = 0.04212951\n",
      "Iteration 8, loss = 0.03863674\n",
      "Iteration 9, loss = 0.03435591\n",
      "Iteration 10, loss = 0.03156028\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=MLPClassifier(activation='relu', alpha=0.0001, batch_size='auto', beta_1=0.9,\n",
       "       beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
       "       hidden_layer_sizes=(50,), learning_rate='constant',\n",
       "       learning_rate_init=0.1, max_iter=10, momentum=0.9,\n",
       "       nesterovs_momentum=True, power_t=0.5, random_state=42, shuffle=True,\n",
       "       solver='sgd', tol=0.0001, validation_fraction=0.1, verbose=10,\n",
       "       warm_start=False),\n",
       "       fit_params={}, iid=True, n_jobs=1,\n",
       "       param_grid={'alpha': [0.1, 0.01, 0.001], 'activation': ['logistic', 'relu'], 'random_state': [42]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, scoring=None, verbose=0)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "param_grid = { 'alpha' : [0.1,0.01,0.001], 'activation' : ['logistic', 'relu'], 'random_state':[42]}\n",
    "\n",
    "#cross validate\n",
    "clf = GridSearchCV(MLP, param_grid)\n",
    "clf.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best parameters:{'alpha': 0.01, 'activation': 'relu', 'random_state': 42}\n",
      "Best score:0.976947902259\n"
     ]
    }
   ],
   "source": [
    "print \"{}:{}\".format(\"Best parameters\", clf.best_params_)\n",
    "print \"{}{}\".format(\"Best score:\",clf.best_score_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### Conclusion:\n",
    "Though the \"optimal\" parameters identified still performed worse then the parameters provided in Part 1 (likely due to differences in other parameters like SGD v Adam as solver), the specific parameters that performed the best do make some intuitive since. A modest L2 regularization penalty of .01 seems to best optimize the variance-bias trade-off. The Recified Linear Unit Function (relu) as the activation function also performed better then a logistic sigmoid activation function (logistic). This could be because the sigmoid function saturates quicker (i.e. there is a comparatively narrow interval of inputs for which the sigmoid's derivative is sufficiently nonzero) then the RELU function, preventing it from learning long enough to capture the more subtle differences between the hand-written numbers. RELU functions are also better suited for supervised neural nets, while logistic functions have a tendency to get stuck in local maxima."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 3 - Display Misclassifications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample Missclassifications:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVUAAAEKCAYAAACi1MYMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW5//EPICooKuJuAqIJjwLKJmoUFTSgEhOBn7tE\ngRglIi4EvS7XXQkao8YdN+JyBcWLBjfQKIpcVzZ3HzEG9wVRcUVR/P3RfaprZnpmumeql+n5vl8v\nXxanqqvPzJk+/Zw6W4uffvoJERFJRstSZ0BEpJKoUhURSZAqVRGRBKlSFRFJkCpVEZEEqVIVEUnQ\naoW6sZmtBrwNLHL3wTlcPws4xN0/ree6Y4HTgQ/SSV+6++5ZrpsNHOHub1dLPwsYA7wL/ETqd/AR\nMMbdF9f7g9Werw7AUnev84vKzNYBbgS2BloAt7j7RQ1932IrYLnuBZwPtAJWAae5+0NZrivXcp0G\nbJX+ZwugM/CYuw9p6HsXUwHL9UBSn9efgE+A0e7+RpbryrJc09cuBd6JJf3V3afUdn3BKlVgKPA8\n0MfMzN29nusH5njfnYET3X1qPdfVNQB3qrsfF/6RrqhvB/rmmIdsWtTznsF5wDvufoCZtQVeNrPH\n3f2ZRrx3MSVerukvmv8B+rn7a2a2LTDHzH7m7l9Xu7wsy9XdD4i97/bANOCYRrxvsRWiXDcCrga2\nc/f3zWwMcCWwd5bLy7JczawLsMzde+d640JWqscAU4DFwInAaAAzGwWMA34g9c01Ajg3/ZrZZjYY\n2Bfo4+5HZbnvzsDaZnYyqW+sk9z9pSzXtcgjr48AE9L5mw18ChhwDXAr8HegO9A6fe1J7r7KzIaR\niq6+BubFb2hmC4E/uPuCeLq7H29m4dtxM2B1YHkeeS21QpRra+BP7v5a+t+vkCq/DUj9buPKslxj\n51sDNwPHu/v7eeS11BIvV3f/2Mw2dvcf05Fwp/Q9sinXct0ZWGVmjwIdgLuAC9x9VW2ZK8gzVTPr\nCuwI3AHcAgw3s/Zm1gOYCAxy957ADFLNvFHpl/Z39/fcfVK2CjUd2b1K6ofqDdwEPJhOb2heWwF/\nAB6NJX/q7t3d/SrgUmCeu/cFegMbAuPS38I3AkPT596K39fde9X2wUsX8K3AC8BjQH1RQVkoVLm6\n+zJ3nxZLOi+V7G9VvzaPvBa9XNOOBN5z9xkNzXuxFapcAdIVah9Szec/Alc0Mq/FLtfVgIeAQcCu\nwF7AsXXlsVCR6mjgfndfDswzsyXptBXAzPAN7u6XV3tdnd9W7v4NsE/s39PM7AxSzYDH88jfwWbW\nL/1+rYH5QPyP4onY8b5AXzM7Mv3vNUk1G/oBL8SaSZOAC3LNgLv/3syOBqYDZwLn5JH/UilIuQbp\nD8ylpP5w92hA/kpersAJpCrWpqSg5eru84FN08/NHzCzzu7+RR75K1m5uvsNsX9+YWaXAGOB6r+L\nSOKVajpqPBz41szeJPWLaEeqeXFRtWvXBDrl8PwmXN8R+J27XxlLbgGszDObVZ7RZPFV7LglcEDI\nY/r5H8CeVP2j+jGXNzazQcCL7v6Bu39jZlOAYblnvTQKWa7p16xHqmn1E7Cju3/egGyWrFzT9+gJ\ntHL3J+q9uEwU+PO6KbBt6HB091lm9gWpDr2FeWSzlJ/X4cDz7v5iOqne+qYQzf/hpHrVNnX3Ld29\nM6lf4tpAe2BPM9s4fe1o4ML08Y+kvoXq8jVwXrojgPTznDbAswn/DHGzSD1TwszWAO4l1Rv5BNAt\n3akCqWdNuTiQVGQa7ncgVZsy5apg5WpmqwMPA28CezWwQs1X0uUKsDtNoyzjCvl5XRO4w8y2BDCz\nAaRGd7ya8M8Ql3S5dgfOMbOWZtaGVNO/zk7yQlSqo4G/xRPSzYrLgd8AJwGz0g+GBwFHpy+bDsw1\ns65mdrSZXVf9xu6+jFQldJ2ZvUhqqMYQd/8hwfxX7xE8Hmibfr9FpHpIL3L3T4BDgdvNbB6ph/AR\nM1toZtl6DMcB66Xv9yzwnLv/PcH8F0rByhXYn9Tzrx2B+enf3QIz65Zg/gtdrgC/BJYkmOdiKOTn\n9T/AKGC6mS0AzgD2dfcVCea/0OV6DqmOsHC/ue5+U10ZalGpS/9ZLePepGlTuVamSirXSp5RVZnf\nFqJyrUwVU64VG6mKiJRCJUeqIiJFp0pVRCRBhZymGqdnDBn5TMdrClS2KSrXypR3uSpSFRFJkCpV\nEZEEqVIVEUmQKlURkQSpUhURSVCxev9FRPJy2mmnAfDAAw9EaX369AHgsMMOA2D77bePzq2zzjqU\nA0WqIiIJKtY01ZKNeXvssceq/B/gnHNqrgc9e/ZsAPr371/oLGk8Y2VSuSbs22+/BWDx4sz+flOm\npPbbmzlzZpVrAHbddVcALrwwtTrh+uuvn0Q2NE5VRKSUVKmKiCSo4juqQrM/W5M/bsCAAQBo1S6R\n8tCmTRsAzCxK22yzzQB47bXUxruTJ0+Ozo0dOxaAs88+u0g5zE6RqohIgio+Us3VWWedVeosNAuf\nf57afuqtt1I7BL///vvRuf/85z8APPXUUzndK3Quvvfee3nlYZdddgFg//33B6BXr17Rud133z2v\ne0nhhY4nqBmF/ulPf4qOR44cCcDmm29elHzVRpGqiEiCVKmKiCRIzf+0xx9/vNRZqFhz5syJjk8+\n+WQAnn/+eQC+++67GtfHOwtbtMh9mGCu1/7f//0fAE8++SQAq62W+Rj069cPgOuvvz5K22qrrXLO\ngzTe66+/DsDEiRMBuPPOO2tcs9566wFwyimnRGlHHHFEEXJXP0WqIiIJqtgZVWEoVRgqla9sHVcJ\nDdVodjNvevbsGR2/8MILVc6FiANghx12AOBXv/pVjXtst912AGy77bY1zoUOrs6dO+eS36hz7OKL\nLwaqdowtW7YMgP322y9Ku/vuu3O5bbMr13zNnTs3Op4+fXqV/4fZUJDpxHziiSdq3GObbbYB4Oab\nbwagb9++SWezOs2oEhEppYqNVOt6vpZtfn98bYBchEi2AdFrs4toQnQB8Nvf/hbIDGfaeOONo3Od\nOnVKOm85iZfhueeeC8Dee+8dpcVXSapDsyvXusSHyoXfb/w5dfXPZ3xY2xtvvAHAuuuuC8CoUaOi\nc+PHjwdg7bXXbkz28qFIVUSklFSpiogkqKKa//FOqbqa89l+5tBEqW+NgNqE2T1Q7/KBza6Z+M9/\n/jM6Dp1WpWrqx33yyScA7LXXXlHawoULARg+fHiUdsstt+Ryu2ZXrnUJi0lD5nca/9yFef3h8U/o\nsAJ4+eWXAfjXv/4FwKpVq6JzOZZFktT8FxEppYqIVHOJMuNDpPLtXMq20HUQJg3E54zXc39FNCW2\ndOlSAH79618DVYd5hRWRHn300SgtrIxUD5VrzIgRI6LjZ599FoBXX301SqseqX711VfRufbt2wOw\nxRZbAJmJIpApi9AaXGONNWq8Z4cOHQDYcMMNG/MjBIpURURKqSIi1fAsNd9hUXHaTqXBmlykOmTI\nEABmzJgBQNeuXaNzYfhUx44d872tyrUeYVowZKaidunSpcZ13bt3r/Lvzz77LDpu164dAIMHDwYy\nUTBkntkOHToUqPqcthEUqYqIlJIqVRGRBDXp5n9j5/dnU4Tfh5qJRfTjjz8C8Pvf/z5KCztyduvW\nDYD7778/OteIoV4q1xLbbbfdAHjuuecAuOeee6Jz8WFzeVLzX0SklJr0eqqN6Ziq755F6LCSAqoe\noU6dOjU6Fzo7LrvsMqA8JiJI4/3yl78EMqthXXHFFdG5gQMHAtCyZeHjSEWqIiIJatKRai5CxBkf\nnF/XJIFwTpFq0xOmnUJmE7jwvDS+1mpYpzPHQf3SRMVXFwvDssLEgEJSpCoikiBVqiIiCWrSzf+6\n5tjnMr8/22OAQnR+SWE9+OCDAAwbNixKW7FiBZAZbnfbbbdF59TsL7zPP/88Oo5vmVPI95k3b16V\nc+uss0503KpVq4LloTpFqiIiCWrSg/8bqq6tVoqwBoAGiTfCDz/8EB2HjftCiyO+3fXhhx8OwIUX\nXghU3balQFSuZDoBw7Y0AA8//HAyOYoJEWrYngcyQ6nC5zus8QCNWgdAg/9FREqpST9TzUeuz0o1\nlKo8LVmyBIBTTjklSrvzzjuBzNTiY489Njp3ySWXANC6desi5bB5W7lyJQD/+Mc/AHj33XcTvzfA\nXXfdBWQ2AwzPzuPCuqoJbSmfN0WqIiIJUqUqIpKgsmz+h6Z6fMhT2A4l3+Z5rgtYxzfuk/Lx9NNP\nA3DggQcCVZuVPXr0AGDSpEkA9O7dOzq32mpl+addsW666SYAJk+eDMBGG20UnXvvvfcA2HzzzWt9\nfbwDMiw8PXHiRADcPTq3ePHiKq+LdzqHjsvwGGj11VfP86dIhiJVEZEEleXXea7ro1aPWuMPpnPZ\najqPbaWliKZNmxYdh1Wmvv/+ewBOOOGE6Nypp54KJLbBmzTC8uXLq/z7448/jo7D53LfffeN0l58\n8UUA5s+fD1Td+O+RRx4BMh2Q2YZAhoH9Rx99dJQ2bty4Buc/SYpURUQSpEpVRCRBZTmjKondUetS\nxJ1Ts9HMm1pcdNFFAPzXf/1XjXOHHHIIALfffnuNc2Ff+PiuqCUYn9qsy3XOnDkADB8+HMg+TjVe\n19Q1q7H69b/4xS+itH322QeA008/HSjPmXKKVEVEElSWkWqQb8dTNiEaLaMhU806oqkuPi88zOOO\nz+E/44wzgMyQujA8B2Ds2LEAPPTQQzXOrb/++o3JVkOoXIFFixYBcNVVV9V5XSjjjz76CICOHTvW\nuCYsNN6rV68orU2bNg3JVmMoUhURKaWyjlSziT9nDdFrts36yigyrU4RTcxRRx0VHd9www0AbL/9\n9lFamMN/9913A1XXRV26dCmQWQ/gvPPOi84Vc/3MNJVrZVKkKiJSSqpURUQS1OSa/xVAzUTgnXfe\nAaB79+5R2pdffpm6YR1Db+JzykOzPz7LqoRUrpVJzX8RkVIqy7n/UvnC4Pz99tuvxrlskeraa68N\nwIQJE6JzhdxQTqShFKmKiCRIz1SLT8/eKpPKtTLpmaqISCmpUhURSZAqVRGRBKlSFRFJULE6qkRE\nmoWCjVM1s9WAt4FF7j44h+tnAYe4+6f1XPdz4Gpgc6AVcJK7P5TlutnAEe7+drX0s4AxwLukejhX\nAz4Cxrj74ur3yZWZdQCWunvO0b+ZTQfedffjGvq+xaZyrfPapcA7saS/uvuUhr53MRWqXGPX7wDM\nATbL9ppyLVczmwZslf5nC6Az8Ji7D6ntNYVs/g8Fngf6mJnlcP3AHO97L3Cvu/cGjgCmmFm2Zd7r\nCsGnuntvd+/j7j2AGUDNJeXz06Ke96zCzE4Gdmnke5aCyjULM+sCLEu/f/ivSVSoaYUq11CBXQ3U\ntR1DWZarux8QyhP4I/AZcExdrynkjKpjgCnAYuBEYDSAmY0CxgE/AJ8AI4Bz06+ZbWaDgX2BPu5+\nVPyGZtYDaO/u1wG4+yIz6wesyvL++YwvewSYkH6P2cCngAHXALcCfwe6k/qjeIRUFLXKzIYB5wNf\nA/Oq5XUh8Ad3X1D9zcxsADAIuBZon0c+y4HKNXu57gysMrNHgQ7AXcAF7p7tZyhHiZdr+vUtgNuA\nU4GZdbx/uZZrON8auBk43t3frytzBalUzawrsCMwBFgAPGZmpwIdgYlAT3d/38yOA05z91FmNgLo\n7+6fAZNquXUX4C0z+xupKG8lcLa7v9qIvLYC/gA8Gkv+1N27p8/fCMxz95Fm1hL4BzDOzG4BbgR2\ncnc3s1Pi93X3XmRhZpsBlwJ7kf7DbSpUrrWXK6nP0kPAeKAt8ACwHLi8oT9DsRSwXCFVAT/t7g+n\nK9jG5rXY5RocCbzn7jPqy2OhItXRwP3uvhyYZ2ZL0mkrgJmhpnf36n9w9f3SW5P60F3k7n82s77A\ng2bW3d0/zCN/B6cjoRbpe84H4t+yT8SO9wX6mtmR6X+vSarZ0A94wd09nT4JuKCuN00/t5oCnODu\nH+XWyiorKtdauPsNsX9+YWaXAGNpApUqBSpXM/sNsKO7D2pk/kpWrjEnkKpY65V4pWpmbYHDgW/N\n7E1Sv4h2pJoXF1W7dk2gU+wHrc/7wGfufh+Auz+Xfo8eQD4fvqn1dA59FTtuCRwQ8mhm66TT96Tq\nH9WPObzv9sAWwCXpb+1NgJZmtma2plM5UbnWzcyGA8+7+4vppBakIu6yVuByHQlsbmYLyPxOZ5vZ\nyNqa2bUoWbmm79ETaOXuT9R7MYXpqBpOqldtU3ff0t07k+o9W5vU88M9zSzsKzsauDB9/CN1P8gG\neBJYkf4GxMy2BrYEXkj4Z4ibReqZEma2BqkOlTGkvh27mdm26etG1Hcjd3/a3TulH3z3IvVM9Y5y\nr1DTVK516w6cY2YtzawNcCwwNdEcF0bBytXd93f3brG/d0g9MsinQs1X0uUKsDtVHzfUqRCV6mjg\nb/GEdLPicuA3wEnArPSD4UHA0enLpgNzzayrmR1tZtdVv7G7f0/qWeR4M3sRuBMY6e4fJJj/6j2C\nxwNt0++3iFQP6UXu/glwKHC7mc0DOsVfZGYLzax3gvkqNZUrdZbrOaQ6TML95rr7TQnmv1AKVq5Z\n/ETyC88U4/P6S2BJrhmq2MH/Vsu4N2naVK6VqZLKtZKnqVbmt4WoXCtTxZRrxUaqIiKlUMmRqohI\n0RVrjyqFwxlaIb4yqVwrk1b+FxEpJVWqIiIJUqUqIpKgYj1TLZlLL70UgLlz50Zp//u//1uq7IhI\nhVOkKiKSoIqPVMPaD7mvASGVLrRapk+fDsCCBZmp6C1apDp7R44cGaUdfvjhRcydNHWKVEVEEqRK\nVUQkQRXf/A923XXXUmdBSuCpp54C4IEHHojSJk1KLVS/bNkyAOJTtUPzf/ny5VHa0KFDAWjXrl1h\nMysVQZGqiEiCmk2kKpXvnXcyu0Nfc801AEycOBHIRKDZtG3bNjred999Adhpp52iNEWokg9FqiIi\nCVKkKk3efffdB8Bpp50Wpb388sv1vu6oo1K72Jx55plR2qabbppw7iTYe++9o+Pnnnuuyv+33HLL\nkuSpEBSpiogkSJWqiEiCKr75f/fddwNVm4ZSGW677TYAzj//fABef/31Gtesu+66AJhZlPbf//3f\nQKZTSgorlNNjjz0WpX333XcAzJw5E4Bjjjmm6PkqFEWqIiIJqthINczrbtOmDQCHHXZYKbMjCbn+\n+uuj46OPTu2WHIZL7bzzztG5E044AYDtttsOgC5duhQriwK8+uqr0fGRRx4JZKJTgE022QSAffbZ\np7gZKwJFqiIiCVKlKiKSoIpt/v/lL38BYLfddgNggw02KGV2pJGuvfZaAE488cQa5371q18BMGHC\nhCgtlLsU18cffwzAsGHDorTQ7I9/Bq+++moAOnfunHgeFi9eDMAll1wSpa1atQqAc889N0rbeOON\nE39vUKQqIpKoiopUv/766xrH/fr1K1V2pIEef/zx6DhEFk8++SQA33//fXQuDIm64447gEynpJTO\n2WefDcBrr71W49z9998fHe+www6JvN8///nPGsdTp04F4Ntvv61x/cCBA6Pj/fffP5E8VKdIVUQk\nQRUVqca/HcP2KfFnO1J6P/zwAwAffPABAOedd1507oYbbgCyr2+aTa9evWpcL6URosMpU6bUONet\nWzcAtt566wbde86cOdHxXXfdVeX/4e+oPi1bpuLHnj17NigP+VCkKiKSIFWqIiIJalGkplNR3iTe\n1A9NjnjzMgidWGFdAIC99toLgA033LCQWQSovT3bNOVVtmEh6S222KL2G+bY/A/XhZlUF1xwQXRu\n9913zydbSWjW5Tpo0CAAHn74YQA233zz6FxY3q+uZRXjzfhwfOeddwKZ9QEAnn/++XyyFQkzKsM6\nBHnIu1wVqYqIJKgiItUwzzhEpwDz5s0DoHfv3jWunz9/PlB1WEfYfiMsXFxAzTqiCQPBQ1T5+eef\nR+euuuqq1A1jf5Nh2Mu4ceMAWLBgQXTu8ssvBzKDvTt06BCd+8UvfgFkItYxY8ZE5372s5/lk+Vc\nNbtyffvtt6PjHj16AJnyHDJkSHRu7NixALz//vtRWhgit2jRIgDefffd6Fx8W5zqwjY3nTp1AjId\n0vF7BWF9AcgsWr7++uvX/UPVpEhVRKSUVKmKiCSoIsapPvHEE0Du4xXD44L49WEZuVmzZgFw/PHH\nR+c0jzw5a6yxBpCZKRU6COP69+8fHZ9++ulApnkZ3+X00EMPBTIzquKdks888wwATz/9NAAPPPBA\ndC50fGg/qsaJjx+NP8YB+Ne//hUdh5lUK1euzOm+YWZcWLYxvofYnnvuWeXa9dZbr9b7xD+3DWj2\nN5giVRGRBFVEpBpmUm2zzTZRWpi9Eb5Nw6pVkIlGsw3Xueeee4CqEZQi1eRNnDgRyAzBiXv00Udz\nukeIUkIrI/wfMotZh47HF198MTo3YMAAILOtyvDhw/PKu6QsWbKk1nNfffVVna8NQ+p22WUXoOpO\nqyGtrhWsDj74YABWrFhR49xGG20EwBlnnFFnHgpFkaqISIIqIlINz1TjWziEyDRsChePSsO6jp98\n8kmU1rFjRyAzFEvrrxbWwoULgboH9zfGH//4RwDat28PZJ67QmarnSOOOAKAPfbYIzq32WabFSQ/\nlWj8+PHR8TfffANknqXGN1oMRo0aFR13794dyH+yTRg2NWPGjFqvOfXUU6u8R7EpUhURSZAqVRGR\nBFXEjKq+ffsCmZlSkGlWhp8vdEpAZnfHwYMHR2lhGEeYp1xAzW7mTTYHHXQQkFnCDeCQQw4BGjQ/\nu15ffvlldBwWLn/ppZcAuPjii6Nz2bZryZHKlcyMuTB0LglLly6NjsMaA9VnTwGcdNJJQObRX6tW\nrZJ4e82oEhEppYroqAqdSvGoOwypCh1V8RWsQkT7yiuvRGmhoyp0Xqmjqvjuu+++gt07vkXLF198\nUbD3ae6SjFCDsMYD1IxQV1999eg4bK+TUITaYIpURUQSVBGR6q233gpUXTUnRKpt27atcX2YJDB0\n6NAoLbxWEWpxrLXWWkDV1kWIIM8666woLTz/DuUYX4kqWLZsGZAZ1hMXVksKQ7ji2rVrBzTqOaoU\nQVhhKpuw2hVAnz59ipGdeilSFRFJkCpVEZEEVcSQqoaaNGlSdDx69Ggg04mVbXHrhGjoDZnhTGF4\nE2SGPWXbTuXnP/85ADvuuGONe4WVqOILHUeZS98rPnNr2223BeDKK6+skYdGULkm7NhjjwUyi5fH\nhZXMHnrooSitdevWhciGhlSJiJRSRXRUNVTXrl2j4zDkqqF7k0t+wrzsyZMnR2lhPdRsA7vDFhvx\nrTayRaHVhXVYw3YskOm8Ch1VUl5Ch2V8TdYgfD7Dpp0Fik4bRZGqiEiCVKmKiCSoWXdUlYg6NGoR\nOqrOPvvsKC2+DQpU3QIldCqGhajDegKQ2Ulz3XXXBYrS1Fe5JuTGG28EMmOU40KHctj9uAjUUSUi\nUkqKVItPEU1lUrk2Qnw2XJgZFbZJis98DKuatWxZtHhQkaqISCk16yFVIlIewjqskIlQg169ekXH\nRYxQG6z8cygi0oSoUhURSZCa/yJSlsKOrNmGVpUzRaoiIgnSkKri09CbyqRyrUwaUiUiUkqqVEVE\nEqRKVUQkQapURUQSVKyOKhGRZqFg41TNbDXgbWCRuw/O4fpZwCHu/mk91+0FnA+0AlYBp7n7Q1mu\nmw0c4e5vV0s/CxgDvEuqh3M14CNgjLsvzuVnqyVfHYCl7l5n9G9m6wA3AluT6lm8xd0vauj7Flsh\nytXM9gQuJtPj3BboAvRx94XVri3Xcl0TuAroS6pcn0m/93d1va5cqFxrvS7vci1k838o8DzQx8Io\n3roNrO+CdIX0P8Dv3b03MBK4w8zWynJ5XSH4VHfv7e593L0HMAO4PYc81qVFPe8ZnAe84+7bAjsA\nfzKzmrvZla/Ey9XdH3H3Xuky6Q28CEyo/sFLK9dyPR1o5e7bAduRqkBObeR7F5PKNbu8y7WQM6qO\nAaYAi4ETgdEAZjYKGAf8AHwCjADOTb9mtpkNBvYl9W12VLV7tgb+5O5hxYVXSP1yNgC+rnZtPuPL\nHgEmpPM3G/gUMOAa4Fbg70D39Ps/Apzk7qvMbBipqPlrYF78hma2EPiDuy+Ip7v78WYWvsw2A1YH\nlueR11IrRLlGzGw40Ak4qJZLyrJcgceBJQDu/lP6uq40HSrXhMq1IJWqmXUFdgSGAAuAx8zsVKAj\nMBHo6e7vm9lxpJrvo8xsBNDf3T8DJmW7r7svA6bFks5LJftbjchrK+APwKOx5E/dvXv6/I3APHcf\nma4M/wGMM7NbSDXjd3J3N7NTquW1F7VIF/CtwP8D7ga8ofkvpkKVa+z+rUl9WA5y91WNzGtRy9Xd\no13qzKwTcALQJOZXqlyTLddCRaqjgfvdfTkwz8yWpNNWADPd/f10hi+v9rqcvq3Sv9hLgb2APRqQ\nv4PNrF/6/VoD84H4t+wTseN9gb5mFn6Ra5JqNvQDXnD3UCFOAi7INQPu/nszOxqYDpwJnNOAn6PY\nClquwP7AG+7+VAPzV/JyNbM+pMr0cnd/sEE/RfGpXOuRT7kmXqmaWVvgcOBbM3uT1C+iHanmxUXV\nrl0T6BT7QXO5/3rAXaR+UTu6++cNyOZUdz+ujvNfxY5bAgeEPKaf6wLsSdU/qh9zeWMzGwS86O4f\nuPs3ZjYFGJZ71kuj0OWadhAwud6raleyck3f42DgSlIdGXfk+rpSUrnWL99yLURH1XBSvWqbuvuW\n7t4Z2ApYG2gP7GlmG6evHQ1cmD7+kdS3UK3MbHXgYeBNYK8GVqj5mkXqmRJmtgZwL6neyCeAbma2\nbfq6ETne70BSkWm434FUbcqUq4KVa8xupJ6BFUOi5Wpm+5N6ljeoqVSoaSrXOjSkXAtRqY4G/hZP\nSDcrLgd+A5wEzEo/8B0EHJ2+bDow18y6mtnRZnZdlnvvD/Qm9fxnvpktNLMFZtYtwfxX7xE8Hmhr\nZi8Ci0hrKvOrAAAI+UlEQVT1kF7k7p8AhwK3m9k8Ug/hI+m89c5y/3HAeun7PQs85+5/TzD/hVLI\ncsXMNgDWCk3NAih0uU5I//+G2N/lFcn+CAWhciXZcq3Ywf9Wy7g3adpUrpWpksq1kqepVua3hahc\nK1PFlGvFRqoiIqVQyZGqiEjRFWuPKoXDGVohvjKpXCuTVv4XESklVaoiIglSpSoikiBVqiIiCVKl\nKiKSoGL1/pelK67IzDb76qvUmgxvvvkmAC1aZDr9xo8fD0CXLl2KmDsRaYoUqYqIJEiVqohIgoo1\nTbXkA4nnz58fHV922WUATJ06NUpbtar2Bck7duwIwMyZM6M0y2kbn6w0SLwyqVwrkwb/i4iUUsVG\nqitXrgRg5MiRANx3333RuS+//LJB9zzssMOi41tuuaWhWVNEU5lUrpVJkaqISClV1JCq+HPTK6+8\nEoApU6YAEI/I48Ol8jFwYL1bnUuOzj77bADOOSez32Hnzp0BOOCAA6K0/v37A7DbbrsBsNZaaxUn\ngyINpEhVRCRBqlRFRBJUUR1VI0aMiI5vvfXWqhloRPN/iy22AOCZZ56J0jbYYIP8M5h++4a+sEw1\nqGzD45njjqu583C2slp33XUBWG21mk+sNt10UwAGDBiQVx7+/Oc/A5khc43U7Mr1ww8/jI5Dee63\n334A9O3bt0DZKjp1VImIlJIi1Rz89a9/BWDcuHENz1xGs4toslmyZAkAffr0idLuvvvuGtfNnj0b\nyKzNcO+999bMQLpscy3XDz74AIAdd9wRgIceeijHXNep2ZVrmzZtouMVK1YAmZbErrvuGp0LHZBD\nhgyJ0gYNGgTAGmuskUBWC0qRqohIKalSFRFJUEU1/0NTEeCYY44B4PXXX09lIM/m/y677BId33HH\nHUCmQ6SRml0zsS7Dhg2LjqdPn97ozORihx12AKBbt24ATJ48OYnbNrty3WeffaLjjTfeGIBFixYB\n8Pzzz9f52tDxGDq22rZtW+OabbbZBoDly5dHaeFzGR4bhfsUkJr/IiKlVFEzquJDasLwp/bt2+d1\nj/DgfMKECVFaQhGqZBGPVAvppptuio6fe+45AA4//PCivHelincsrrnmmgB89913ADzyyCPRufBZ\njLckw2LwDV1DI3wm462bnXbaqUH3SpoiVRGRBFXUM9W4pUuXArDJJpukMlDPM9UePXoAmTnpv/vd\n7wqVtWb37K0ujz76aHS8xx57NDoz1X3//fcA9OvXL0oLw7n+/e9/A9CuXbsk3krlmofwnPTJJ58E\nwN1zet0ll1wCwDvvvAPA2LFjo3OXX355klkM9ExVRKSUVKmKiCSoojqqpk2bFh2HWVC5Gj16NFDQ\nZr9kUYgmf9yxxx4LwLx586K0G2+8EUis2S8NEIZChWFZ8eFZ1YXZWgCTJk2qcm677bYrQO4aR5Gq\niEiCKqKj6tprrwUyA/6zZiBLR9W7774bpW222WYFyl0N6tAogmXLlgHQq1cvILPSGMD9998PJB6p\nqlwLJD50KwzBCxtvhskGkBnWlTB1VImIlJIqVRGRBDXpjqqwMO748eOBuuf0b7/99tHxgw8+COQ/\n20qajrD3VXjEc91110Xn1EHVNITZWdWX8QQYPHgwULAmf6MoUhURSVCTi1Tvu+++6DgMr1i5cmW9\nrzvppJOi4w4dOiSfMSkrs2bNAmCdddYBqi6GLU1D6FCMd1T9/Oc/B2DUqFElyVMuFKmKiCSorCPV\ne+65Jzo+7bTTAHj77bejtG+//bbW16699toAPPvss0BmCIZUrnhEE9bRnTp1KgAbbrhhSfIkdQtz\n+bOtVvXRRx/VSAtbs7z22mtA1S1dwrDIeFopKFIVEUmQKlURkQSVdfM/Plsil6XB4jsz/uUvfwHq\nbvaHXTUBXnnlFQDeeOMNILOwLlRd4FjK15gxY6Lj0AQMW6ZIeerSpQuQWaIRMp/BbB3Qc+bMqfL/\nuPBZD0Mlhw4dGp3r1KkTAL/97W+jtGxbuCRBkaqISILKcu5/6KA69NBDo7QwELgu8Uh14sSJQPYt\nFp5++mkArr/++igtRKpBGIoDmb3m4wsdN4LmiCds8eLFQGaeP8DJJ58MwJlnnlmsbKhcE7L11lsD\nmdbpEUccEZ3baqutAPjss8+AqitYzZgxo9Z7hrUgBg4cmNP1MZr7LyJSSmX5TDWskRiGRUFukWr8\nmhNPPLHKuXy3qP7iiy+i4/AtJ+Xp0ksvBeCbb76J0uKtHCl/L730UnT81ltvAbDeeusBcMIJJ0Tn\nevbsWes9rr766lrPhZZoy5aFjyMVqYqIJEiVqohIgsqy+b/lllsCsNFGG0VpxW6Cx5sZvXv3Lup7\nS36uueYaINPBAZl94aVpmDx5cnQcOp9GjBgB1N3kz1XXrl0bfY9cKVIVEUlQWUaqwemnnx4dh61S\n4p0RP/zwQ6PuH19Xs3rnVej8gMzKOFJe5s6dW+Xf559/fnS81lpr1fv60HkRBobn+jpJzsyZM4FM\nawMyQyMPOuigkuSpsRSpiogkSJWqiEiCyrr5f8ghh9Q4vvnmm6O0Dz/8EICPP/4YgMsuuyyn++68\n885AZlsVqDomVpqG6dOnV/n3gAEDar02LBUHcMYZZwCw3377AVoWspTCzMV4U3/IkCEA9O/fvxRZ\najRFqiIiCSrLuf8VTnPEExI2f/v3v/8NVF3VrFWrVgBcccUVAFx88cXRuXPPPRfIbMkRrm0klWtl\n0tx/EZFSKutnqiK5aN26NQA33HBDlDZt2jQgsybnnXfeGZ3bddddi5g7aW4UqYqIJEiVqohIgtRR\nVXzq0EjIXXfdBcABBxwAVJ0VF4ZNhSUgwzJyBaRyrUzqqBIRKSVFqsWniKYyqVwrkyJVEZFSUqUq\nIpIgVaoiIglSpSoikqBidVSJiDQLilRFRBKkSlVEJEGqVEVEEqRKVUQkQapURUQSpEpVRCRBqlRF\nRBKkSlVEJEGqVEVEEqRKVUQkQapURUQSpEpVRCRBqlRFRBKkSlVEJEGqVEVEEqRKVUQkQapURUQS\npEpVRCRBqlRFRBKkSlVEJEGqVEVEEqRKVUQkQf8f/CsktndOBoYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xcf5dda0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#NOTE - We used the parameters from Part 1 to investigate misclassified digits as they performed better then Part 2 Parameters.\n",
    "\n",
    "misclass_idx = np.where(y_test != pred_y_test)\n",
    "\n",
    "misclass_act = y_test[misclass_idx]\n",
    "misclass_pred = pred_y_test[misclass_idx]\n",
    "misclass_img = X_test[misclass_idx]*255\n",
    "misclass_img = misclass_img.astype(int)\n",
    "\n",
    "misclass_list = zip(misclass_act,misclass_pred,misclass_img)\n",
    "\n",
    "print \"Sample Missclassifications:\"\n",
    "for index, missclass in enumerate(misclass_list[:9]):\n",
    "    plt.subplot(3, 3, index+1)\n",
    "    plt.axis('off')\n",
    "    plt.imshow(missclass[2].reshape((28,28)), cmap=plt.cm.gray_r, interpolation='nearest')\n",
    "    plt.title('Act: %i | Pred: %i' % (missclass[0], missclass[1]))\n",
    "    \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Question 2: Regression Trees (10 points)\n",
    "\n",
    "In this question, we will be exploring the application of regression tree (RT) using sklearn package in Python. You will be using the same Hitters.csv dataset (available on Canvas) used in HW2 Q5 to predict a baseball player’s Salary using all the 16 performance variables. Use a random state of 42 and a test size of 1/3 to split the data into training and test.\n",
    "\n",
    "1. Build a regression using [DecisionTreeRegressor](http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html) with max_depth = 5. Report the mean squared errors on both training and test datasets. (4)\n",
    "2. Repeat Part-1 with max_depth = 2. (4)\n",
    "3. Briefly explain what you observe from these MSE values obtained by using maximum tree depths 5 and 2? Which tree is better and why? (2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: pylab import has clobbered these variables: ['mat', 'clf']\n",
      "`%matplotlib` prevents importing * from pylab and numpy\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn import datasets\n",
    "import pandas as pd\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.metrics import mean_squared_error\n",
    "%pylab inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hitters = pd.read_csv(\"hitters.csv\")\n",
    "hitters_target = hitters['Salary']\n",
    "hitters_data = hitters\n",
    "del hitters_data['Salary']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_train, x_test, y_train, y_test = train_test_split(hitters_data, hitters_target, test_size=0.333, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data MSE: 18749.4789728\n",
      "training data RMSE: 136.928736841\n",
      "test data MSE: 225103.898055\n",
      "test data RMSE: 474.451154551\n"
     ]
    }
   ],
   "source": [
    "DTR = DecisionTreeRegressor(max_depth=5)\n",
    "model = DTR.fit(x_train, y_train)\n",
    "#training data\n",
    "pred_y_train = model.predict(x_train)\n",
    "train_MSE = mean_squared_error(y_train, pred_y_train)\n",
    "train_RMSE = np.sqrt(train_MSE)\n",
    "print \"training data MSE:\", train_MSE\n",
    "print \"training data RMSE:\", train_RMSE\n",
    "#test data\n",
    "pred_y_test = model.predict(x_test)\n",
    "test_MSE = mean_squared_error(y_test, pred_y_test)\n",
    "test_RMSE = np.sqrt(test_MSE)\n",
    "print \"test data MSE:\", test_MSE\n",
    "print \"test data RMSE:\", test_RMSE\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the errors on the both the training and test data have gone down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data MSE: 69909.5877084\n",
      "training data RMSE: 264.404212728\n",
      "test data MSE: 166165.705731\n",
      "test data RMSE: 407.634279387\n"
     ]
    }
   ],
   "source": [
    "DTR = DecisionTreeRegressor(max_depth=2)\n",
    "model = DTR.fit(x_train, y_train)\n",
    "#training data\n",
    "pred_y_train = model.predict(x_train)\n",
    "train_MSE = mean_squared_error(y_train, pred_y_train)\n",
    "train_RMSE = np.sqrt(train_MSE)\n",
    "print \"training data MSE:\", train_MSE\n",
    "print \"training data RMSE:\", train_RMSE\n",
    "#test data\n",
    "pred_y_test = model.predict(x_test)\n",
    "test_MSE = mean_squared_error(y_test, pred_y_test)\n",
    "test_RMSE = np.sqrt(test_MSE)\n",
    "print \"test data MSE:\", test_MSE\n",
    "print \"test data RMSE:\", test_RMSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3\n",
    "\n",
    "The second tree with the max depth of 5 is better but it is possible that it's overfitting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Question 3 - Support Vector Regression vs. Linear Regression (10pts)\n",
    "Download datasets 'summer_gpa_test.csv' and 'summer_gpa_train.csv' from Canvas. With this toy dataset, we want to predict GPA in Summer 2016 for students using 5 different features. So, target variable will be 'GPA_summer2016' in this problem.\n",
    "\n",
    "1. Fit a support vector regression using the default options on [sklearn's SVR](http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html) on training data. Note that the default kernel is \"rbf\".  Report the root MSE for both train and test data. (2)\n",
    "\n",
    "2. Fit SVR with 'linear' kernel and Linear Regression (for other options, use default parameter settings). Report RMSE of prediction on train and test data for the two methods. (3)\n",
    "\n",
    "3. Now, compare the results of three different methods, then provide a possible reason for SVR with RBF kernel not working well on test set. (1)\n",
    "\n",
    "4.  Provide simple residual plots on Train and Test set for all three methods. Specifically, submit a scatter plot wherein y-axis shows the residuals and x-axis the predicted values. What can you learn about the effect of outliers on different models from this problem? (4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.svm import SVR\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "d_train = pd.read_csv('summer_gpa_train.csv',index_col='student')\n",
    "d_test = pd.read_csv('summer_gpa_test.csv',index_col='student')\n",
    "\n",
    "y_train = d_train['GPA_summer2016']\n",
    "y_test = d_test['GPA_summer2016']\n",
    "x_train = d_train.drop('GPA_summer2016',axis=1)\n",
    "x_test = d_test.drop('GPA_summer2016',axis=1)\n",
    "svr = SVR()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1\n",
    "RBF Kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data RMSE: 0.174703118961\n",
      "test data RMSE: 0.549667753896\n"
     ]
    }
   ],
   "source": [
    "model = svr.fit(x_train, y_train)\n",
    "#training data\n",
    "pred_y_train = model.predict(x_train)\n",
    "train_MSE = mean_squared_error(y_train, pred_y_train)\n",
    "train_RMSE = np.sqrt(train_MSE)\n",
    "print \"training data RMSE:\", train_RMSE\n",
    "#test data\n",
    "pred_y_test = model.predict(x_test)\n",
    "test_MSE = mean_squared_error(y_test, pred_y_test)\n",
    "test_RMSE = np.sqrt(test_MSE)\n",
    "print \"test data RMSE:\", test_RMSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2\n",
    "Linear Kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data RMSE: 0.477967720085\n",
      "test data RMSE: 0.231968041522\n"
     ]
    }
   ],
   "source": [
    "svr = SVR(kernel='linear')\n",
    "model = svr.fit(x_train, y_train)\n",
    "#training data\n",
    "pred_y_train = model.predict(x_train)\n",
    "train_MSE = mean_squared_error(y_train, pred_y_train)\n",
    "train_RMSE = np.sqrt(train_MSE)\n",
    "print \"training data RMSE:\", train_RMSE\n",
    "#test data\n",
    "pred_y_test = model.predict(x_test)\n",
    "test_MSE = mean_squared_error(y_test, pred_y_test)\n",
    "test_RMSE = np.sqrt(test_MSE)\n",
    "print \"test data RMSE:\", test_RMSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training data RMSE: 0.458060629343\n",
      "test data RMSE: 0.260057137391\n"
     ]
    }
   ],
   "source": [
    "LR = LinearRegression()\n",
    "model = LR.fit(x_train, y_train)\n",
    "#training data\n",
    "pred_y_train = model.predict(x_train)\n",
    "train_MSE = mean_squared_error(y_train, pred_y_train)\n",
    "train_RMSE = np.sqrt(train_MSE)\n",
    "print \"training data RMSE:\", train_RMSE\n",
    "#test data\n",
    "pred_y_test = model.predict(x_test)\n",
    "test_MSE = mean_squared_error(y_test, pred_y_test)\n",
    "test_RMSE = np.sqrt(test_MSE)\n",
    "print \"test data RMSE:\", test_RMSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RBF test RMSE:0.549667753896\n",
      "Linear test RMSE:0.231968041522\n",
      "Linear Regression test RMSE:0.260057137391\n"
     ]
    }
   ],
   "source": [
    "svr_rbf = SVR(kernel='rbf')\n",
    "svr_lin = SVR(kernel='linear')\n",
    "LinReg = LinearRegression()\n",
    "y_rbf = svr_rbf.fit(x_train, y_train).predict(x_test)\n",
    "y_lin = svr_lin.fit(x_train, y_train).predict(x_test)\n",
    "y_LinReg = LinReg.fit(x_train, y_train).predict(x_test)\n",
    "rbf_test_mse = mean_squared_error(y_test, y_rbf)\n",
    "lin_test_mse = mean_squared_error(y_test, y_lin)\n",
    "LinReg_test_mse = mean_squared_error(y_test, y_LinReg)\n",
    "rbf_test_rmse = np.sqrt(rbf_test_mse)\n",
    "lin_test_rmse = np.sqrt(lin_test_mse)\n",
    "LinReg_test_rmse = np.sqrt(LinReg_test_mse)\n",
    "print 'RBF test RMSE:{}'.format(rbf_test_rmse)\n",
    "print 'Linear test RMSE:{}'.format(lin_test_rmse)\n",
    "print 'Linear Regression test RMSE:{}'.format(LinReg_test_rmse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEZCAYAAACAZ8KHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VOX1+PHPJATCEjYbUKQEInLcq6Bi1ZZqLWrVYrV+\nLVatVqPgUndR61rqhhatWqOCYKlaqXXBpa5152ddQaW1RxHEBRQkhEUI2eb3x3MnuZnMnlmT8369\nfMncmbn3zJ3Jc+6z3kAwGMQYY4xJVFGuAzDGGFNYLHEYY4xJiiUOY4wxSbHEYYwxJimWOIwxxiTF\nEocxxpikdMt1ACZ1IrIe2BEoB6ao6v/FeO3uwEmqOjnJY9wKrFLV33co2BSJyGHAj1X17AjPfQCc\nrqqvpLjvF4FbVfXhOK/7ADgd+Bh4UFX3jfHa4cCNqvqLCM9tFXq/iFwBbKGqv00y5ruAalVdICIz\ngL+p6gvJ7CNdROQnwAzga+CHqrrZ95wAfwSGAgGgBrhUVeeLyCygXlUnhe3vSO81u4nIp8Am7z+A\nHkATcIGqPpPRD+ZiOQ/YSVVPjPO6ZuA7qlqT6ZjyiSWOwhYEUNV3gKhJw7MTsHXGI0ozVX0ceDzX\ncQCo6gogatLwDAdGdeD98fwEuMPbX1UH99VRvwTuUtVrIjz3EHCJqj4GICI/AJ4QkRHAn4HnReQs\nf7IBqoDbvH8HgWNUdUHoSS+xzAaGpP+jRJTIJLcuORHOEkcWiMg44AbgS6AS2AicoKoqIrOBgd72\nJ4DLgeuBHwLFwALgt6q6wfvjuwVoBt7Ga2r09n+bqu4sIr2BW4F9gAZgHlANXAX0FZG7VfUk70r+\nd0CJF88FqvpvESkDZgK7ACtwV3mrwj5PAFgGHK6q73rb/ga8pKp3+l5XAbwKfAhUAOOAbYDrgF7e\n57hKVZ8UkcHAHGAL7+1PquoVIvJr4BeqepiI7ADcDfQE1NtH6DiLVLUs/LGI9PI+/7beeV6PK5A+\njvF9bQ/MinUcEdnOO089cFfUM4G7cFfgQ0TkKWAS8BrwX+/znwA8F4oT2EFEXgYGeN/zaar6rYgs\nBY70ndulwJHAEbhC8z7vvFyPV2MSkcNxv50iYB1wnqq+5dVshgNbeTGsBI5W1a9EZDJwKrAZqANO\nVdX/hZ2LbsB04MdAI/AGcK732Q4HNopIP1WdEnYatwT6hB6o6qsi8n9Ak6q+IyIK/AK4z3dux3j7\nDAmE7XMEsJoIRGQTcBNwKFAGXAgcBeyM+7s7TFU3eX9D03DfbT1wmao+433OW4EDcDWolUCtt+++\nwJ9wF18lwL9wfy/NoRgj/H7/qaqXR4q1M7A+juzZDbhBVb8H3APc63uup6rurKoXAxcBDaq6u6ru\nhiu8rxOREuDvwDmqOgZ4EffjDwld+UwFeqiqeMfcG5eULgde9ZLGSOBq4GBvX6cCD4tIT+D3wEZV\n3R5Xi5HwD6KqQVwBfiKAiAzA/cHdH+FzD8Ulh+1wBdQs4FhV3R2YAFSLyFDc1eYn3vYfAtt6Scz/\n2e4F7lTVXXF/yBURPn/444OBNaq6txfD28AZEeL0uy+B45wPPKaqewCH4JpqmoGTvc9xsPe6rX2f\nf0VYnNsAP1fVXXB/i5fGCkpVLwWW4xLfm6HtXrNQtbevXYErgHkiEiq498Ulou1xheGpIlKEK2gP\nVNWxuKQXqTZ0KS7p7Oz9douBaap6I/AYcFOEpAGuae82EflCROaKyOnA26q63nv+du9chVQBc1S1\nzrftPhF5V0Q+E5HPcb/Fw6Kcnh7Al965rMYl8N8COwD9gQkiMhB4EDjTO08nAPd6Set0YCSwHTAe\nGObb901e7HsAo3FNw+eGHT/89zvS9/vtdCxxZM97qvr/vH/PAnb1ClxwV6Uhh+J+5AtEZAGucN0e\nd+VUr6ovAajqA7ir53A/xhXqqGqDqu4XoQ/gJ7grwn95x7gPdzW5rff+Od77vwEeifJ5ZgNHeVdq\nE4HHfYWCXwPwb+/f38cVQo96x/0nrkazC/A0cKSIPIlLZBf59+f90e8C/NWL7f8B/4kSWwtVfQj4\ni4icISI3Az/CdyUczjvOzgkc5xHgQhF5CFcTiNZX4f/84R72tY3Pxn0viQi/Et8feF5Vl3kxv4i7\nah7jPf+Sqn7r/XsBMNBLcn8HXvf6sdbh/W7CHAzc4b0e3FX5wRFe14aqzsX9xo7H1Th/A/xHREIF\n8lxcjWuEiBTjCvHbw3ZzjKqOBn6AqxF9qKqfxjhsqK/qE+ADVf3Ku8hZiqttjgU+VtW3vRj/i/vb\n2w/3u79fVZtUdSNeTchzKC7ZLgDeAfbA1T78Yv5+OxtLHNnT6Pt3Ee6Pv8l7vMH3XDFwlqru5tU4\n9sRVuYO0LzCaaK8R31WtiAz1CkO/YuBfqjrad5y9gUURjtNIBKr6GfAu7grwBNwVXiSbfYVOMfDf\nCMd9xvtjHgHcibvCf0tE9vLtJxgjtiBtf8vdfZ9/Mq5A/BZXGPyN9ufRL3TuYp4DVX0Sl2jnArsC\ni7z2+3D+zx/O//0FcEkmFIP/+N2JLfR78ivGNatAawdzm32r6vG4QvFjYAqRLxLCywj/fiMS51pV\nrVfVF1T1Sq9muwjXPIXXtzEbOMmL4X1V/SRsV6E4l+ES0HUiskeMQ/v7SxoiPB/tPHXDNZtG+86L\ngaN8v9m9gDP9O0ng99upWOLInt1EJHSVcgowX1XXRXjdM8AZIlLiNSfcDVwLfAAEROQgABH5Ga4K\nHu554NciEhCRHsA/cFXnRloLoBeA8V4TByLyU+A9XHX/aeAk7/0DcDWeaGbiCpyeqvp6lNf4/xj/\njWuC+oF33F1xhdYQEbkWuFxVH1M3guo/+DqZVXUN7mrvZO+9o3E1A3DNLyVevwO4GkDIeGC2qs72\njnUYriCIKM5xWojIfcAvVfXvuGaOtcB3cefZX7DGSlI/E5F+3hX3KbgaGLg+pd294+yFq6WFhO8f\n3Pf5E3EjuhCR/XFNhG9EO7CIbCEinwGrVfUWXJPULhFe+gwwSUS6eb/H04BnY3wmcLWdKhFp+R68\ni5dBuIuNkDuBo3EXHn+OtUPv93UPrhkqVf8GRokbYYiI7IirzbyE+5zHi0gPESn14gp5Bq9pyvub\neoyw5s54v9/OxhJH9nwFXC0i7wM/A47ztoe3zU8FPsU1KYRqAOepaiOu4/APIvKu9++VEY5zFe5q\n6z1cAfiEqj4KvI67GHzIq6KfAjzgVb+vwus8BK7EFU4f4jrW34/xmR7DXV3NjPGals/nNX0dCdwg\nIguBvwC/UtXPgZtxzXfvi8jbwBJc7cDvGGCiiLyH69j/r7ffdbjO0KdF5A3aXsnfiCv43gWe887J\nyPDYEjlOmKnAr7zz929cs9MruAKjWUT+jUsasUbd/Bd4EvddrcF1doNLxmd7MZ+E65cJeRSYKyIH\n0Dqq7kNcgf6I9/u6Bjg0VlOJqq72PsML3vm+1jtWuD/gfrsLvc/WDQgNjY742VS1Ftd8drKILBE3\nnPlZXN/IS77XLcUNPtjRq8H5Rdr3xUCFiJwS4blY5zl0nlbjau+3eefpXtwglcW4JPYO7m/uRdzv\nL+S3QC/vcyzEfV/Two7r//2+ReTfb6cRsGXVM0/cqKdbvY47Y4wpaDkbjisiY4HrVHW/sO1n45oJ\nQlfTp2qMoZPGGGOyKyeJQ0QuwDXVbIjw9BjgOPVN/Cl0qvoykduPjTGm4OSqj2Mx8PMoz40BLhaR\nV0XkoizGZIwxJgE5SRyq+ghRhnniOpQm4cZW7+uN+DHGGJMn8nHJkT+Fhql6k2l2o3WYYkTBYDAY\nCMQa9WiMMSaClArOXCeONkF7a8Is8sbjb8IN6Ys0m7XtTgIBVq3K/0ma5eVlFmcaFUKchRAjWJzp\nVkhxpiLXiSMIICITgd6qOlNELsZNyKnDzW5+OofxGWOMCZOzxOEtI7C39++/+bbfR9t1YowxxuQR\nmzlujDEmKZY4jDHGJMUShzHGmKRY4jDGGJMUSxzGGGOSkuvhuMYYk7fuvfce3n77TRobGykuLua0\n086irKyMs846jQcfnNfyusbGRiZOPIJ77rmfQw45gDFjxlBf30hjYyPBYJArr7yaLbfcKupxjjrq\nZ9x//0OUlLTeaqWpqYmzzz6NxsZGbrjhT/TpE/XGlVlnicMY0yls3gyzZpWwcWOAo49uYOjQjt0y\n4tNPlzJ//itUV88CYPHij7n66iuYPft+hg4dysKF77LrrqMBmD//FcaM2YPevfvQr19/5syZ0zIB\ncN68h3nggXs5++wLYhyt/QTuVatWsWnTJmbOnNOhz5EJljiMMQWvsRGOP74nL77oirSHH+7G/fdv\noqIi9eTRp08fvv76a554Yh577bU3I0duy4wZrhA/9NDDeeqpJ1oSx5NPPsaJJ1Z572x7zK++WkFZ\nWd822xYseIfq6lvp3r07hx12OBBk2rSrWbFiOVtssQWXXHIlf/zjtXzxxWfceOO1nH/+xSl/jkyw\nPg5jTMF7882ilqQB8PHHxdx/f8xbo8f1ne+Uc/310/ngg/c49dQTOfbYo5g//xUAxo3bj/feW0B9\nfT2rV39DTU0N22+/IwDr1q3j+OOP5ze/OZajjvoZDQ0N/OpXv263/4aGem677S4OPNCt43rEEUdx\n2213MXjwVjzxxKOcd97FDB9emXdJA6zGYYzpBHr0gOLiIE1NrU0+xVHvLJ+YL7/8gl69enPxxZcD\noPo/zjvvTEaP3oOysjJ+8IMf8corL/LVVys45JCftbyvb99+zJkzh5Ur13H11VdSUlJCaWlpu/0P\nG1bR8u+SkpKWxLPzzrvw9ttvss8++Xt3VqtxGGMK3ujRzfziFw0UFbnCdvfdG5k0qb5D+1y8+GOm\nT59GY6O7A8TQoUMpKyujuNgVm4ceOoHnn3+G1157hQMPPNj3ThdDIBDgggsu4eWXX+D1119rt/9A\noLX4bWhoYPFid6PT995byIgR27g95emtva3GYYwpeIEA3HLLZn7+8wbWri3ioIMa6dmzY/scN24/\nPvvsU04++Xh69epFMNjM6aefTa9evQGoqBjOpk11jBhR2bLNi6blXz169GDKlMu45porGT16d3r0\naF/zAOjevTsPPTSXzz//jC233IrJk89k1aqV5OvtIgL5mtGSFCyUJYwtzvQphDgLIUawONOtgOJM\nKTNZU5UxxpikWOIwxhiTFEscxhhjkmKJwxhjTFIscRhjjEmKJQ5jjDFJsXkcxhgTRaZWx12w4B0e\nffQhrrrqGi699EL+8Idpufh4KbPEYUyG1NTAGWfARx/1oqKimWnT6hgwINdRdWKbN9Nz1gzY+C11\nRx9DcOh3O7S7TK+OG5rcV2hJAyxxGJMxU6aUMm8eQDELF7qFk2bMqMtpTJ1WYyN9j59IjxefB6D0\n4QdZe/9DNFdUxHljdJlcHddvwoQDmTfvGc4881S23XYUS5Z8wsaNG5k69ToGD96Shx6ay3PPPUMg\nEOCAA8Zz5JFHs2TJJ9x22000Nzezdm0t5513MTvttDNHHnkow4dXMnz4CM4885yUP3s8ljiMyZBl\ny4piPjbpU/Lmv1uSBkC3jz+i9P45bLz4spT3GVod9x//mMvs2TPo2bMnVVWTGTduf8aN248ZM26n\nvr6e9evXRVwdd82ataxfv45x4/bn178+KcaRWidv77DDTvz2t+dx11238/zzz7DPPj/kX/96jurq\nuwkGg5xzzunsuef3Wbp0CWeccQ6Vldvw3HNP889/PsZOO+3MqlUrueeev1FWVpby506EJQ5jMqSi\normlphF6bDKjuUcpweJiAk1NrRs7uDxuplfHjWTUKAFg0KDBrFlTw5Iln/DVVys466zJBINBNmxY\nzxdffEZ5eTn33DOT0tJSvv12A717u7sD9u8/IONJA2xUlTFpV1MDVVWlLFlSxNChsNNOTUyY0MC0\nadZMlSlNo8dQ94ujCRa5Iq1+9z3ZNOn0Du0z06vjhr/eabt01LBhFVRWbsMtt9zBrbfeycEHH0Zl\n5UhuvvlGTj75VC655AoqK0e2vjtLayJajcOYNHN9G603ERozpjmjfRs1Ne6Yy5YVdd1O+ECADbdU\ns/nnRxJYu5b6gw6ho8vjZm913ID3EdqX+iNHbsvo0XswefJJNDQ0sMMOOzJo0GAOPPBgLr10Cn37\n9qO8fBBr19a2O3Ym2eq4WVRAK2ZanB0wfnyvNk1Uu+7axLPPbszY8aqq2iaqCRMakk5U+Xouw1mc\n6WWr4xqTJ8L7MjLdt2Gd8CbbrKnKmDQL9WUsW1bEqFHFTJ2a2b4N64Q32ZazxCEiY4HrVHW/sO2H\nAZcBDcBsVZ2Zi/iMSdWAAa3zNVyTRWaP509UoT4OYzIpJ4lDRC4AjgM2hG3vBkwHxgCbgPkiMk9V\nM/ynZ0zh8icqY7IhV42hi4GfR9i+PfCxqq5T1QbgNeCHWY3MGGNMTDlJHKr6CNAY4am+wFrf4/VA\nv6wEZYwxJiH51jm+Dpc8QsqA2iivbaO8PPOzJdPB4kyvQoizEGIEizOSN998kwceeIDp06e3bDvv\nvPO4/vrr6dYtcvG5//77s/XWWxMIBGhqamLTpk1MnTqVHXfcMVthZ1yuE0f4GOIPgZEi0h/YiGum\nuiGRHRXImGmLM40KIc7wGPN1sl4hnEuIF+dmevacAXxLXd0xBIMdWx0XoLZ2I5s3N7Y55kUXXcma\nNZuivqe5GWbNmkVtret3evPNf3PjjTcxbdpNHY4n3VJNwrlOHEEAEZkI9FbVmSJyLvAsLqnMVNUV\nuQzQ5Fa+FrSp8s8qtxVz06mRvn0n0qOHtzpu6YOsXfsQzc2pr44bzVFH/Yz773+IG264hpKSElas\nWEFNzWp+97sr2HZbAYL4J1Z/9dUK+vZ1DSkLFrzDjBnVFBcXs/XWQ7nggktobGzkD3+4gtWrv6G8\nfBDvvbeARx99Ku1xp1POEoeqLgP29v79N9/2J4EncxWXyS+draC1yXqZUVLy75akAdCt20eUls5h\n48bUV8eNrrWhZMsth3DBBZfw+OOPMm/eI5x//kUAnHTSSaxf/y2rV3/D2LF7c/rpZwMwbdrVVFfP\non///syceQf//Ofj1NXVMWTI1kydeh2fffYpxx13dAZiTq9c1ziMiamzFbQ2WS8zmptLCQaLCQR8\nq+PSsdVxE+FfzfaDD95r2R5qqrrrrttZsWI5AwYMYM2aNaxevZrLL7+IYDBIfX09e+wxltraWvba\n6/sADBs2nP79879KXdh/habTy/byHZk2bVodEyY0sOuutmJuOjU1jaGu7miCQW913Po92bSpY6vj\nhrRfz6/1caSFCf3vqaqazKpVK3n44Qfp378/gwYN5rrr/sitt97JccedyOjRu1NZuQ0ffPA+4JZy\nb12wMH9ZjcPktc42K9om62VKgA0bqtm8+UgCgbXU1x8CdGx13JC3336DqqrjAQgGocl/z48osbT8\nKxDgoosu44wzTmHcuP0466xzOf/8swgGm+nduw+XXvp7dtxxZ6655krOOOMUBg/eku7de6Ql7kyy\n1XGzqHOMXMkfhRBnIcQIFme6JRPnokXvs2nTRvbYYy+++OJzzj//tzzwwCMZjtBJdXVcq3EYY0wO\nDRmyNVde+TtmzZpBU1MT5503JdchxWWJwxhPZxv6awrDwIFbcMstd+Q6jKRY4jDG09mG/hqTKTaq\nyhhPZxv6a0ym2F+GMZ7ONvTXmEyxpipjPJ1t6K8xmWKJwxiPzbEwJjHWVGWMMSYpljiMMcYkxRKH\nMcaYpFjiMMYYkxRLHMYYY5JiicMYY0xSLHEYY4xJiiUOY4wxSbHEYYwxJimWOIwxxiTFEocxxpik\nWOIwxhiTFEscxhhjkmKJwxhjTFIscRiTRTU1UFVVyvjxvaiqKmXNmlxHlLpATQ1lVSfQf/yPKKv6\nNYE1NbkOyWSJ3Y/DmCypqYH99+/N8uXueq3Q72veZ8q5lM57GICShe8CAdbPuCenMZnssMRhTJZM\nmVLakjRC/Pc1DwRq6NPnXIqLP6WpqYING24CaLctGByY1bijKV72aczHpvOyxGFMimpqXDLw32p2\nwIDor/cniRD/fc379DmX0lLvCr7EXcED7batX39Puj5ChzRVVHg1jdDj4bkLxmRV1hOHiASA24Hv\nAXXAyaq6xPf82cDJwEpv06mq+nG24zQmnilTSpk3rwRIrNmpoqK55XUAQ4a0va95cfGnbV4f/jja\ntlzZMO0mIEDxsk9pqhjOhmnTcx2SyZJc1DgOB3qo6t4iMhaY7m0LGQMcp6oLchCbMQkLr0FEqlH4\nhZJEeA0lVHOZPLmSAw/0XcE3DQeCXk3Dvy27otWsggMGWp9GF5WLxLEv8DSAqr4hIruHPT8GuFhE\ntgKeVNXrsh2gMYkIr0H4m50iGTAgco0kVHN5+eU7qa4uQmQJX389gr/+9VauuqqOoUMDXh/HcDZs\nyP5VfbI1q0QFamroM+Vcr8ZSwYZpNxEckB/9Nya2XCSOvsBa3+NGESlS1dBf3d+APwPrgEdF5Keq\n+s9sB2kyK1JHcL50+iYqUg0iFaGaSm3tQCZOnNvmuW+/bWBGjq/qk61ZJcpGZRWuXCSOdUCZ77E/\naQD8SVXXAYjIk8BuQNzEUV5eFu8lecHiDDkZaO30LS0tAebGfEckuTyf5eXw6KOhR8VASZTXxY5x\n1ChYuDDyc8uXl1BeHnm/6RYtzvD4Ro0qTs95X/55m4elyz+nNIH92t9Q7uUiccwHDgX+ISJ7AR+E\nnhCRvsAiEdkO2ATsD9ydyE5XrVqfgVDTq7y8zOL09O+/mBJfedjQsJja2uSOWQjnM5EYp06FzZtd\nH8LKlYE2Q3aHDGlg1arMz/OIFac/voqKZqZOrWPVqo4fs2zIUEp5q+Vx3ZDvsj7OuSqE7xwKK85U\n5CJxPAL8RETme49PFJGJQG9VnSkiFwMv4UZc/UtVn85BjCZDWpuolrTZnotO32yJN2zX3/exZg1c\neGFph5u/0ila30yiojVL2qiswhUIBoO5jiEdgoWS3bt6nGVlJ7TMSwBoaupPQ8P+bNgwPek+jkI4\nn+XlZRx+eENL5zLAhAkNeTdbvLy8jG++WRa13ynZOSt+4d95Xd0RKc9FKYTvHAoqzkAq77MJgCar\nwuchNDdX5s2EtkzJVOdyukWagBj6bjoysiqR+SmmsOTnL9h0Wk1NFWGPh+cmkCwKH6Ybb9hursQq\n4DuS/Lrid97ZWY3DZJVbfym38xKyLV3DdjM9hLmpqSLqZMNk56z4dcXvvLOzxGGyKhgc2OmbpsIN\nDNbwAOdSzKc0UcEGbiJI4gV+KGGUlLxAcXEtkJl1q2IV8B1Jfl3xO+/sLHGYjLBZwa06OtHN3/fg\nl+6+glABH/ru+i07ouW7GzBgYN516JvcscRhMsJmBbdKdfnxUAHe/Q9PwU7tn89UX4F9dyYe6xw3\nGdGV7tUQ7054TRVhncMJLj8eKsCL/rOpzfbm5v7U1R2R9r6CQKCGsrITXKJ6AOjvtnfm786kxmoc\nJiO60r0a4l2hpzrRraXAngQEoXmnntSPPDilOS+JaGkS2wn3XxCY2Lm/u2isqTU2SxwmZbEmhXWl\nWcHxaleJLj8efj7/upWXfGuBiVA/4eCMNhm1m2OzU0/qJxzcqb+7aKy5LjZLHJ1ELlabjTUpLFf3\nasjFeUhX7WrKlFJembeO2zmNyoVLWTR4S3Y7+FCKVyxPKPl29Co5fDhu/cjMJqqUrF5NWdUpGa8J\ndKWm1lRY4ugkYs36zZRMzIjuaMGfjfMQXjO48ZKbGJqG2tWyZUXczmn8kr+7DV9DXfcjqH32pYTe\n39Gr5IKYb3HaaVmpCXSlptZUWOLoJHKxrENHJoVFk2jBHwjUACfTv//iNgkmG+ehfU1rSFrumVFR\n0UzlwqVttiVzpdvRq+SCmG+xNPXzk4yu1NSaCkscnUSsWb+Zkq4Z0X6JFvx9+pwLPExJSdsEk43z\nkKm1p6ZNq2PlWxWwvHWp8WSudLvEVfKIEfBWaucnGXZb3NgscXQSuWhmGDiwhgceSG9/QqIFf7QE\nk43zkImaFrjlywe+eCN1FzYnfaUbCNTAnAaavupPYCnUz9mHDVe0f2/B33mxupq6zY1WE8ixhBKH\niHQHtlPV90XkGNxd+aar6oqMRmcSlotmhkz0JyRa8EdLMNk4D5moaYUkcqUbqfB338UT7sbMo4Bx\nPQiub58QctEXllYDrSaQDxKtcdwL/E9EegJXAXOAvwDjMxWYyX+Z6E9ItODfsOEmSktLaGhYnPWO\n3I7e2KijIhX+iX4XtsS5SYdEG2dHqOrlwJHATFWdCiR4GxfT2YRmGOfyLn6ueWUutbUvsX79PYXV\n3NJBkQr/RJcuz/clzkO/rf79f0RZ2a+9QRAm3yRa4+gmIt8BDgeOEJEtgV6ZC8vks/BF9/x38TOx\ndeROeiGRmuncuY/fxJfvQ24z2ZRW8P07eSTRxHED8AbwmKouEpGPgMsyF5bJhdAEMpZ/TtmQoVEn\nV6V6Fz9bxqFjd9JrOX81i2m8ZmuCY7agKTCyZQmSRL6DXA25TfS7z2RTWsH37+SRhBKHqt4P3O/b\ntL2qNmUmJJMr/glkpbxFaHJV+B9901+3oqT1FtoJN3fYMg7JD+X111BuXfkb9l3u1fR+AHUTxrY7\nf/l6VZ3od5/J4dTWv5M+MROHiCzFLXUW6TlUtTIjUZmciDaBLPyPvm7SIdTdfUTSzR22jEPyQ3n9\nNZTuLGvzXKTzF35V3f2lF6m/Y7+c1e5Ciaz7H56CibgFG2ujf/eZbErLxVynzipejeNH2QjC5Ido\nE8jaFfj/XUHt+pfStv/ObPVqqKpq7dO45JLkhvL6ayRLGcGexJ78Fn4VXTSwltJ5j5Cr2l2yK+5m\nsikt3/t3CknMxKGqywBEpAfwU6APEACKgRHA5ZkO0GRPaJmF0uWfUzfkuy2Tq9JV4HfFZRxOO42U\n+zSgbQ1lEtVsPaSZsYOWtDl//qbEwK1fw76+HXgD33JVu8unFXcLYkmVApFo5/jDuFFUI4FXgR8C\nr2cqKJPKxwV3AAAa9klEQVQboclnpeVlrF+1vmV7ugr8SJPbInWaEqTTdKKHLa2U9PIkbScbljFo\n2ixqw0ZhtTQlDgC+huZ1JRCAoucaYLJ7Ta5qdwWx4q5JWqKJQ4BtgT8Bs4DzgX9kKqiuJt9HG2Vy\n3Z5InaZASp3oiXQMZ7vzOGxppbh9GjU1MHXqtxx77G+prFzCVlsNY+bM6DEGamooefkF9+B24Ego\nogGAxkFbExw+OKe1O2se6pwSTRxfq2pQRP4H7KKqc7zmK5MGXXm0UXgTymcvL6N+c6DNLbYTbWZJ\nZLhltucJVFeXsXlzQ8J9GlOmlPLLX57MQQc96G15h5KS6DH2mXIuxbW17kHYUJXg2MEJL8merEQv\ndqx5qHNKNHH8R0RuBaqB+0RkCFAS5z0mQV15tFF4/8m7tdsQAHbibd9rhsfcR8vIne5Ptdkeabhl\ntucJlJY+lFSfxrJlRVRWhi0dHiPGNr+VpcCerQ8zOWqoK1/smMQTx2Rgb1X9r4hcDhwAHJO5sLqW\nrjjaKMTff/KvT7dhcm014Abf7NRzCSPHD4vbzBI+kz0kUsGZ7/MEKiqaWbp0BHvu6Rs9FSPGNr+d\nSdC49dYExw7OeLNQV77YMYknjn0AROSHwFrgISB/GuELXFccbRTi7z+5o6qUWm8E0kTmMmF8Q0JX\n6+1nsvekvv7giAVnsm3uyfSJREpKoYmSiS41Mm1aHVdddQv9+gWprFzClltWUF8fPcZ2v51B0wnW\nJvan6W9uYtRIAlOnJdy31pUvdkziieMq379LgF1wo6teSfaAIhLAdeN9D6gDTlbVJb7nD8MtZ9IA\nzFbVmckeo9DYTWOcVJcrbzdyp/7gqO3qyba5R2p+WrbsnnYd2Js33xQxKZWWuv0kutTIgAFw8829\nCARuaElY3bufEzVhdeS3429uYuG79NncmPC+uvLFjkl8yZH9/I9FZARwU4rHPBzooap7i8hYYLq3\nDRHp5j0eA2wC5ovIPFVdleKxTAFJdbnyTI7cCa/NLF78GQce2Js//jFyB3a0pBQ+DHfNJ2soq5oc\ntXM5G+sqdaS5yS522utIDa7QpHQHQFVdKiLbpXjMfYGnvf28ISK7+57bHvhYVdcBiMhruDkjD6V4\nLNMF+GsR6R7aHF6bWbSokuXLk+vAhvZLjVxdczqli6J3LmdjXSVrbkqvjtTgCk2idwCcTeuaVQFc\nAb8oxWP2xfWThDSKSJGqNkd4bj3QL8XjmC4o3mifZJc1D9VmFi/+jEWLKpk82XXeJ9OBDe2b4cYs\nWQLLW58vXvZpzBngmRgh5W9uKhk1kg1Tp3V4n+lYNr5QdaUBA4nWOF7y/TsIPAg8n+Ix1wFlvseh\npBF6rq/vuTKgNpGdlpeXxX9RHrA406tdnMs/b/OwdPnnlPpec8YZMG+e+/fChcX06FHC3LmxjlAG\nq+9i3cGnUfnFUu5gMpOoZurUagYMgMrKpXz3uyMoLa2mtDTyOSsvL6O8HB59NLSlGI4eCR+0Xu2X\njBrJdy6/EEJXrIcBTw6FvbcCYu8/ZeVl8GhrZf47adhl8uc3eXn72xw1Eha2/U7zNtYOirc67jDv\nny9GeHpL4LMUjjkfOBT4h4jsBXzge+5DYKSI9Ac24pqpbkhkp6t8S2Tkq/LyMoszjSLFWTZkqLck\nvFM35Lttlk/56KNeuKXWQo+bWLVqY8zjlFWdwt5fuAJ9T95i6yHNDHp4FgMGuHEb69ZBUe0n9Pt6\nF4r619BcO5C1Wz5Jc7/KqOcyMHUafTY3tnYuT51Gv6OPaJ0cVQsNpw+i9tl/+d6V/HeS6KiwUJwd\nnVmfyvlNRj7/Nv3facmokXwzdRrBPI01JNXEFq/G8TKuhlEKDMYtmdaEW7PqE9xSJMl6BPiJiMz3\nHp8oIhOB3qo6U0TOBZ7FNYnNVNUVKRzDdFHxRvsku6w5tG9yGDtoSbv1ovp9fSjd9v0SgCK+pN9r\nh7Cm34dR9xmpczkTfQ7hnezdur1JMDg4alLoaKd8Kue3s/B/p+XlZXmfNDoi3uq4IwBE5AHgz6r6\nqvd4D+DCVA6oqkFall5r8ZHv+SeBJ1PZt2mV7+tfxROoqaHPOafT/fX5BIGG7+/Dhpv/HPczxBvt\nk8iQ3/B2+r9uFb9AL+pfE/NxIjIxxDW8U71bty+BL6MmhY52yqc6pNoUlkT7OLYPJQ0AVX2rA6Oq\nTBYU+pIQfaacS+lTrdcPxU89Cd17dPgzxBryG2qmWbNmGRMnbsOkSdUsXDiQSQdXc/eE2AV6c+1A\niviyzWMGJxdbJoa4ho8K84uUFOLNrI/X+Z3qkGpTWBJNHF+IyO+BuUARcCy+WoLJP4U+wiNSvOn8\nDJEKwGHDXDPNqFEwatQ7HBB8jucn/oQ/f34b61+4J+b+1m75JP1eO6RNH0c+2LDhJrq9/Sbdun/p\neiWHtT4XaaRWvDkxHblnuuk8Ek0cxwK/Bx7A9Xk8D5yQoZhMGhT6GP3w+N224R3eb6gJb83Ly5hY\nuw3XDjifW6YcQfn6Gro3NLjePM8WlWs4mr+zdU0z7m4C0TX3q2zt00iyppFJweBAgmcOhoVfQn+g\n2ruZ0sjIS7LEm1mf7D3TTeeU6MzxNcCZGY7FpFGhLwmxYdpNUF/v6+PYNy2fIdSENwoYxTscevvj\n9P7lpsgv9hbCGbPFEjZ0+MjtZWvOQ0sSrgUmQv2E1G+mFK3zO9v3OTG5FW847ruqOlpEmmmdAAhu\nxFNQVYujvNXkWK6XhOhoQRIcMJD1f/lb2o8X3txVWtm2maV5bRHBr/tS/G5tyxCOQOXwhONIRraa\nfdJ5ERGt8zsbS6SY/BFvVNVo7/9WHzVJyXZBkujxwpvANi8rpdeerTWO5g+2onbQfPo8cC7FwzNb\nW8tWs0/oIiKUXPsVH5FyrSBa53c2lkgx+SPRJUe2AfYC7gfuAEYD56jqaxmMzRSwbBckiR4v/Oq7\nrvIsur92TJtO7WC/xGtriTY3RRoeXVFRmtU5D5lM5pm8z0miCn0IeiFJtHN8NnArMAE36e9c4EZc\nMjGmnWwXJAkfL9j2QXNZBWuGtXZq19TAlKrE+x0SbW6KNDx62rR7gOhzHtJdEGYymefDvcULfQh6\nIUk0cZSq6oMiMhO4T1VfFRG7dayJKtsFSaLHi1e4JNvvkGhzU6Th0fHmlPRftQ/dLvkSlkLJpHfh\nwo4VhOlK5tESWq77NAp9CHohSTRxNInIkbg1pi4TkcNxS48YE1GyN0zK1vHiFS7J9juEjzJauTLA\n+PG9WmoQ5eVue7LDo/v0ObdlCRP2BIJQfP2nsd7STngB/+2NV8DQjifzfL2yT2UIuo0GS02iieMU\n4BzgNFVdISK/BE7OXFjGZEakwsVfeNx6ayWHHXYntd7tV+P1O/hHGa1cGWD58iKWL4dly2pZuXIS\no0Yto6xsqCu044xs8veXzJ27jFGjfE9WJj+PJVMFfL5e2acyesxGg6Um0XkcH4jIVGAHESkGLlbV\npfHeZ0y+iVS4+AuPffd9l8cfhzPPfCDuWkuBQA3Dhp3Lgw+6q9Uf//hOli93VYzbbz+Nfff9BwCl\npW/B0PiFtr+ZbMGCbRg16p2W5xrrt056dFemCvh8nVyayhB0Gw2WmkRHVR0NXAr0BPYGXheR81X1\n3kwGZ/JTIY9eiVS4hBcWY8cu4dlnYy8FHqjx+iC85qSSkne58MJuHH743wGSukNgqKbx7LOtf46T\nJlXTr1+QH//4E9estN30pJtQMlXAF/rkUr98GA1WiBJtqpqCSxivqOpKEdkNt+yIJY4uKFtt3NlK\nUKkUHn2mnOs6rn38ySKZOwT6axohtbUDueOO+xgzJvUJgZkq4HM9uTSd8mE0WCFKuHNcVdeLuNtv\neP0cXWehfdNGttq4s5WgUik8ipd9CktxHdeetWuHt/x70qRqgkGXTGpqhrP77tHvRxbeAd+zZ5Dx\n4xsTXpI8WgdvrAI+PCkzaybQ9QZKZnsQR2eRaOL4j4icAZSIyK7AacDCzIVl8lm22rizlaBSKTya\nKircENkgUOn6ILbc8kYmTGjg5ZeLqa0dyMSJ7p6pEyY0xBzSGz4ya/z4xoSWHgkV/t0nvUDRQe4O\ny4l28LZLypNL4LaZcY9pDCSeOPoAWwObcMuEvgCcl6mgTH7LVht3vnbCgncOLgxQfH3rOejXbyAz\nZtSxZg1ceKEbHTVqVDFTp8ZOAtHWf4o3K72l8L+k7f4S6eBtl4SX2lgXk7hEE0cFcKKqXpzJYExh\nyFYbd7oSVCb6SmKdA//EPneP7Nj7ijYRMN5kxJbCP6zJLJE+mnbL1o8YEfc9xoQkmjiagWUiorha\nBwCqun9GojKG9CWofJ2wFk+8yYgthf8kIAhNo/vTMGz/hPpowpNyaXW1Tek1CUs0caR0f3Fj8kG+\nTliLJ9q9L0LaFP4PDGfD6OkE1ydWkwpPyqUDy2DV+nSEbbqARCcAvpzpQIzJlHztK4nXhxGt7yOk\nMw2LNYUl0RqHMSnJh7WA8nXCWrw+jFiLIMaTD+c9WYUYc1dlicNkVD6sBZSvV+aZvJFTPpz3ZBVi\nzF2V3dnPZJStBRRdeJ+F/3FNDVRVlTJ+fC+qqkpZsya5fRfieS/EmLsqq3GYjLK1gKKL1YfR0fuR\nF+J5L8SYuypLHCajbC2g6GL1YXS0GasQz3shxtxVWeIwaZOvd4YrRPGG4sZTiGswFWLMXZUlDpM2\nhTrRLh/FG4prTC5Z4jBpU6gT7fJRR4biGpNpNqrKpE1TRUXY4+G5CSSKjo5U6ioxGRNP1mscIlKK\nuwHUIGAd8GtVXR32mpuBfYDQGggTVNXWQ8hz+TrRLqSjI5UyIR9jMiaeXDRVTQbeV9Xfe7ekvQw4\nO+w1Y4ADVbUm69GZlOXrRLuQTE64S1U+xmRMPLn4le4LPO39+yngAP+TIhIAtgXuEpHXROTELMdn\nOqlYE+5yJR9jMiaeQDAYzNjOReQ3wDm4+6QBBICvgDNUVb0ksUxVh/ne0wf4LTAdVyN6EXcvkEUx\nDpW5D2E6jZoamDzZ3bNoxAioroaBOV4KKR9jMl1KIKU3ZTJxRCIiDwHXqurbItIXeE1Vd/E9XwT0\nUtUN3uPrcU1b98XYbXBVASwJ7W7qY3GmSyHEWQgxgsWZbgUUZ0qJIxdNVfOBn3r//inwatjzo4D5\nIhIQkRJc09a7GGOMyQu56ByvBv4iIq8Cm4FjAETkHOBjVX1CROYAbwD1wF9U9cMcxGmMMSaCrCcO\nVd0E/F+E7Tf5/v1H4I/ZjMsYY0xibOyfMcaYpFjiMMYYkxRLHMYYY5JiicOYNLA1p0xXYqvjGpMG\ntuaU6UqsxmFMGtiaU6YrsV+3MWlga06ZrsSaqoxJA7tjn+lKLHEYkwYduWNfTY3rI/EnnQED0hyg\nMWlkicOYHLOOdVNorI/DmByzjnVTaOwXakyOWce6KTTWVGUKQqR+gPLyXEeVHtaxbgqNJQ5TECL1\nAzz6aC4jSp+OdKwbkwvWVGUKgvUDGJM/7K/PFATrBzAmf1hTlSkIkfsBSnIbVBbYHA+TjyxxmILQ\nVfsBbI6HyUfWVGVMmHxaIt36dkw+shqHMWHy6Sq/oqK5JYbQY2NyzRKHMWHy6Srf5niYfGSJw5gw\n+XSV31X7dkx+s8RhTBi7yndsRJeJxhKHMWHsKt/Jp74ek19siIYxJqJ86usx+cV+CcaYiGy2vonG\nmqqMMRFZX4+JxhKHMSYi6+sx0VhTlTHGmKTkrMYhIj8HfqGqv4rwXBVwCtAAXK2qT2Y7PmOMMZHl\npMYhIjcDVwOBCM8NBs4Evg8cBFwrIp1/GVRjjCkQuWqqmg9MjvLcnsBrqtqoquuAj4FdshaZMcaY\nmDLaVCUivwHOAYK42kUQOFFVHxSRcVHe1hdY63u8AeiXyTiNMcYkLqOJQ1VnAbOSfNs6XPIIKQNq\n472pvLwsycPkhsWZXoUQZyHECBZnuhVKnKnIx+G4bwJ/EJHuQE9gO2BRvDetWrU+03F1WHl5mcWZ\nRoUQZyHECBZnuhVSnKnIm8QhIucAH6vqEyJyC/AarnnrElWtz210xhhjQnKWOFT1ZeBl3+ObfP++\nG7g7F3EZY4yJzSYAGmOMSYolDmOMMUmxxGGMMSYpljiMMcYkxRKHMcaYpFjiMMYYkxRLHMYYY5Ji\nicMYY0xSLHEYY4xJiiUOY4wxSbHEYYwxJimWOIwxxiTFEocxxpikWOIwxhiTFEscxhhjkmKJwxhj\nTFIscRhjjEmKJQ5jjDFJscRhTBrU1EBVVSnjx/eiqqqUNWtyHZExmZOze44b05lMmVLKvHklACxc\nWAzAjBl1uQzJmIyxGocxabBsWVHMx8Z0JvbrNiYNKiqaYz42pjOxpipj0mDaNNcstWxZERUVzS2P\njemMLHEYkwYDBlifhuk6rKnKGGNMUixxGGOMSYolDmOMMUmxxGGMMSYpljiMMcYkJWejqkTk58Av\nVPVXEZ67GdgHWO9tmqCq68NfZ4wxJvtykji8xDAeWBjlJWOAA1W1JntRGWOMSUSumqrmA5MjPSEi\nAWBb4C4ReU1ETsxqZMYYY2LKaI1DRH4DnAMEgYD3/xNV9UERGRflbb2BW4DpXnwvishbqrook7Ea\nY4xJTCAYDObkwF7iOFVVjwnbXgT0UtUN3uPrgfdV9b4chGmMMSZMPo6qGgXMF5GAiJQA+wLv5jgm\nY4wxnrxZq0pEzgE+VtUnRGQO8AZQD/xFVT/MbXTGGGNCctZUZYwxpjDlY1OVMcaYPGaJwxhjTFIs\ncRhjjElK3nSOx+NNDLwd+B5QB5ysqkt8z58NnAys9DadqqofZz3Q1njGAtep6n5h2w8DLgMagNmq\nOjMX8fniiRZnXpxPEekGzAKGA92Bq1X1cd/zeXE+E4gzX85nETADEKAZmKSq//U9ny/nM16ceXE+\nvVgGAW8DB6jqR77teXEuffFEizPpc1kwiQM4HOihqnt7hd10b1vIGOA4VV2Qk+h8ROQC4DhgQ9j2\nbri4xwCbcMOO56nqquxHGT1OT76cz2OBb1T1eBEZgFum5nHIu/MZNU5PvpzPw4Cgqu7rzaW6Bu/v\nKM/OZ9Q4PXlxPr1zdgewMcL2fDmXUeP0JH0uC6mpal/gaQBVfQPYPez5McDFIvKqiFyU7eDCLAZ+\nHmH79rghx+tUtQF4DfhhViNrK1qckD/n8++4qzZwv9cG33P5dD5jxQl5cj5VdR5wivdwOLDG93Te\nnM84cUKenE/gRqAaWB62PW/OpSdanJDCuSykxNEXWOt73OhVZ0P+BkwC9gP2FZGfZjM4P1V9BGiM\n8FT4Z1gP9MtKUBHEiBPy5Hyq6kZV/VZEyoAHgd/5ns6b8xknTsiT8wmgqs0icg/wJ8C/IkPenE+I\nGSfkwfkUkROAlar6HG5JJb+8OZdx4oQUzmUhJY51QJnvcZGqNvse/0lVa1S1EXgS2C2r0SVmHe4H\nFVIG1OYolnjy5nyKyHeBF3CTQef6nsqr8xkjTsij8wmgqifgVmmYKSI9vc15dT4hapyQH+fzROAn\nIvIisCswx+tHgPw6l7HihBTOZSH1ccwHDgX+ISJ7AR+EnhCRvsAiEdkO1564P3B3TqJsKzy7fwiM\nFJH+uLbGHwI3ZD2q9trEmU/nU0QGA88Ap6vqi2FP5835jBVnnp3PY4GhqnodbpBJE67zGfLrfEaN\nM1/Op6q2LNTqFcqnqmqogzlvzmWsOFM9l4WUOB7BZc353uMTRWQi0FtVZ4rIxcBLuB/Zv1T16RzF\n6RcECIvzXOBZXGE9U1VX5DJAT6Q48+V8Xgz0By4Tkcu9WGeQf+czXpz5cj4fBmaLyMu4v/+zgSNE\nJN/OZ7w48+V8hnSpv3VbcsQYY0xSCqmPwxhjTB6wxGGMMSYpljiMMcYkxRKHMcaYpFjiMMYYkxRL\nHMYYY5JiicMYHxGZLSLHi8hWIvJEnNe+kOS+x3kTsDoSX4WILO3IPozpqEKaAGhM1niTtQ6N87If\npbDrdEycsslXJqcscZiC5i25fRVuNdrvAm/g7i0wBLea8je4pRQOwi35MA4oBu5R1T95+5gOHIJb\nObQYeFFEKoCXVHWEiAwDZgODgG+BKu8YiMjrqvp9ETnIi6MbsBSoUtU1IjIet7z2JkAjxL8zcL+q\n7uw9PsTb/5G41Ux3BAZ77z0i7L2zgRdVdY73uFlVi0SkN/Bn773FwPWqOtc71l3etjrgRFX9JPmz\nbro6a6oyncEewGRV3Q7oCZzubR8FHKOq43GFcVBVdwfGAoeLyD4iciTu5mDbA0cBI337DV3Z3w48\n6BXuVwG/U9WzALyk8R3gWmC8qo7BLTMxTUS6A/cAR6jqHrjk0YaqfoBb6XkHb9NE4F5gb2Czqu4D\nbAv0AuKtWhqK91Lgbe+Y44BLRWQEcA5wo6ruCdwK7BVnf8ZEZDUO0xm8oqqLvX//FZckHsEtJf25\nt/0A4Hsi8mPvcW9gZ9xV+cPeSsvfiMg/I+x/HPBLAFV9CnjK2x4qqMcCw3A1lQDugmy1t/8vfXdb\n+wvw+wj7/yvwSxG5Ftf89RtVrReR1SJyGrAdLqH1SfB8HAD0FJGTvMc9gR2AJ4DbReRg79//SHB/\nxrRhNQ7TGfjvKVLke+y/wi8GLlTV3VR1N9zV9j24wt//dxDp/iRtbsokItuHPV8MvKqqo71974Gr\nvQS952LtG9z9EI7CNZc97SWNn+HuQbEBd0vaV2m/2nIwtE1ESsLiOdb3Wff29vswbsnsN3CLBt4Z\nJR5jYrLEYTqDfb1RUEXA8UCo1uAvaF8AThGRbiLSB7dM/57A88BRItLdu+XrQRH2/zJejUNEfkJr\ngdvkHfMN4Psisq23/Qpcf8r7QLnXtwCuGaodryP+c9wKu/d6m38MzPX6L1biluUuDnvrN7gaE7S9\nreoLwGlevFt5cQwTkQeAsao6A3e3wny8Z40pAJY4TGewApgDLMIVwKH7CfhHH90BfAQsAN4E7lbV\nV1T1MVxiWAQ8Cvwnwv7PBH4hIgtwSaHK2/4Y8B7utqa/Af4uIu/hbpZznndjnInAvSLyNq7JKJq/\nAt9R1Ze8xzOAY0TkHVyT0uvAiLD3VAM/EpGFwPe98wCuH6aniHyAS4znq+pS3H27L/H2eQOuz8OY\npNmy6qageaOqrlDV/XMdizFdhdU4jDHGJMVqHMYYY5JiNQ5jjDFJscRhjDEmKZY4jDHGJMUShzHG\nmKRY4jDGGJMUSxzGGGOS8v8Bpb9mJznmPHoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x16fd8160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "resid_rbf = y_rbf - y_test\n",
    "resid_lin = y_lin - y_test\n",
    "resid_linreg = y_LinReg - y_test\n",
    "\n",
    "plt.figure()\n",
    "plt.scatter(y_rbf, resid_rbf, color='blue', label='SVR rbf')\n",
    "plt.scatter(y_lin, resid_lin, color='red', label='SVR linear')\n",
    "plt.scatter(y_LinReg, resid_linreg, color='yellow', label = 'LinReg')\n",
    "plt.xlabel('predicted values')\n",
    "plt.ylabel('residuals')\n",
    "plt.title('predicted v residual distributions of SVR models')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see from this plot that RBF seems to be the least robust to outliers as its residuals are the most stretched out vertically, meaning RBf makes worse errors on outliers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 4: Bayesian Networks (10 pts)\n",
    "#### Part (a) (5 points)\n",
    "Consider the following Bayesian network of binary (True/False) variables.\n",
    "\n",
    "<img src=\"hw4q4pic1.png\">\n",
    "\n",
    "This is equivalent to saying that X and Y are conditionally independent given C, or P(X,Y|C) = P(X|C)P(Y|C). This happens to be the assumption used by the Naive Bayes classifier.\n",
    "\n",
    "The exact probabilities are given:  \n",
    "P(X = True | C = True) = .75  \n",
    "P(X = True | C = False) = .5  \n",
    "P(Y = True | C = True) = .25  \n",
    "P(Y = True | C = False) = .5  \n",
    "P(C = True) = .5\n",
    "\n",
    "Find P(C = True | X = True, Y = True). The easiest method is to use Bayes rule, along with the conditional independence equation given above.\n",
    "\n",
    "#### (b) (5 points)\n",
    "A new feature Z is added, and based on prior knowledge, we believe that one of the two networks given below properly captures the dependencies among the variables. Our goal is to determine P(C|X,Y,Z). For each of these two different networks:\n",
    "\n",
    "<img src=\"hw4q4pic2.png\">\n",
    "\n",
    "will P(C|X,Y,Z) be the same as P(C|X,Y) (the inference from part a)? Or will it be different? Give a separate answer for each network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part A\n",
    "By Baye's Rule we have:  \n",
    "$P(C=T|X=T,Y=T) = \\frac{P(X=T,Y=T | C=T)* P(C=T)}{P(X=T,Y=T)} $\n",
    "\n",
    "By the conditional indepedence equation we have:  \n",
    "$P(C=T|X=T,Y=T) = \\frac{P(X=T|C=T) * P(Y=T|C=T) * P(C=T)}{P(X=T,Y=T)} $\n",
    "\n",
    "Next, use law of total probability to calculate joint distribution of P(X=T, Y=T):\n",
    "$P(X=T,Y=T) = P(C=T) * P(X=T | C=T) * P(Y=T | C=T)) + (P(C=F) * P(X=T | C=F) * P (Y=T | C=F) $  \n",
    "$P(X=T,Y=T) = .5 * .75 * .25 + .5 * .5 * .5 = .21875 $\n",
    "\n",
    "Finally, plug in given values and calulation of P(X=T,Y=T) into the denominator:  \n",
    "$P(C=T|X=T,Y=T) = \\frac{.75*.25*.5}{.21875}$  \n",
    "$P(C=T|X=T,Y=T) = .429$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part B\n",
    "\n",
    "For the first network it will be the *same* since C is indepedent from Z due to the fact Z is a child of X and Y. So in this network due to conditionally independent relationships you can simplify $P(C | X,Y,Z)$ to $P(C | X,Y)$\n",
    "\n",
    "In the second network you can no longer make this simplification since now Y is conditionlly dependent upon the new feature Z. Therefore, you will get a *different* answer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 5: Multiclass Classification (5 points)\n",
    "\n",
    "One way of using a binary classifier for addressing a multiclass classification problem is to use a One-vs.-All (or One-vs.-Rest) strategy.\n",
    "\n",
    "#### 1) Briefly describe the  One-vs.-All method.  (2pts)\n",
    "\n",
    "If you have 3 classes, you would make a new training set and fit a classifier where you compare one class as the positive class (1) and all of the other classes as the negative class (0). You do this for each class and when you want to make a new prediction, you would run the 3 classifiers and choose the class for which confidence is mazimized. \n",
    "\n",
    "#### 2) What are two disadvantages of using the One-vs.-All method in situations where the number of classes $N$ is very large? (3pts)\n",
    "\n",
    "It would be computationally expensive because you have to run all the classifiers each to you want to make a new prediction. Also, the One-vs.-All method is more sensitive to outliers than a multinomial model. Another thing that One-vs.-All is bad at dealing with, where one-vs-one method is better, is that it's not  good at discerning classes when there are pair specific features to consider or there are unbalanced number of instances per class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}