Indian-Liver-Patient-Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Indian-Liver-Patient-Classification\n",
    "______\n",
    "\n",
    "Dataset : https://archive.ics.uci.edu/ml/datasets/ILPD+(Indian+Liver+Patient+Dataset)\n",
    "\n",
    "This data set contains 10 variables that are age, gender, total Bilirubin, direct Bilirubin, total proteins, albumin, A/G ratio, SGPT, SGOT and Alkphos.\n",
    "\n",
    "#### Data Set Information:\n",
    "\n",
    "This data set contains 416 liver patient records and 167 non liver patient records.The data set was collected from north east of Andhra Pradesh, India. Selector is a class label used to divide into groups(liver patient or not). This data set contains 441 male patient records and 142 female patient records. \n",
    "\n",
    "Any patient whose age exceeded 89 is listed as being of age \"90\".\n",
    "\n",
    "\n",
    "#### Attribute Information:\n",
    "\n",
    "1. Age\tAge of the patient \n",
    "2. Gender\tGender of the patient \n",
    "3. TB\tTotal Bilirubin \n",
    "4. DB\tDirect Bilirubin \n",
    "5. Alkphos Alkaline Phosphotase \n",
    "6. Sgpt Alamine Aminotransferase \n",
    "7. Sgot Aspartate Aminotransferase \n",
    "8. TP\tTotal Protiens \n",
    "9. ALB\tAlbumin \n",
    "10. A/G Ratio\tAlbumin and Globulin Ratio \n",
    "11. Selector field used to split the data into two sets (labeled by the experts) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from sklearn.tree import DecisionTreeClassifier, export_graphviz\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn import cross_validation, metrics\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC\n",
    "from time import time\n",
    "from sklearn import preprocessing\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.metrics import roc_auc_score , classification_report\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.metrics import precision_score, recall_score, accuracy_score, classification_report\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# read .csv from provided dataset\n",
    "csv_filename=\"Indian Liver Patient Dataset (ILPD).csv\"\n",
    "\n",
    "# df=pd.read_csv(csv_filename,index_col=0)\n",
    "df=pd.read_csv(csv_filename, \n",
    "              names=[\"Age\", \"Gender\" , \"TB\" , \"DB\" , \"Alkphos\" , \"Sgpt\",\n",
    "                     \"Sgot\" , \"TP\" , \"ALB\" , \"A/G\", \"Selector\" ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>TB</th>\n",
       "      <th>DB</th>\n",
       "      <th>Alkphos</th>\n",
       "      <th>Sgpt</th>\n",
       "      <th>Sgot</th>\n",
       "      <th>TP</th>\n",
       "      <th>ALB</th>\n",
       "      <th>A/G</th>\n",
       "      <th>Selector</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>65</td>\n",
       "      <td>Female</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.1</td>\n",
       "      <td>187</td>\n",
       "      <td>16</td>\n",
       "      <td>18</td>\n",
       "      <td>6.8</td>\n",
       "      <td>3.3</td>\n",
       "      <td>0.90</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>62</td>\n",
       "      <td>Male</td>\n",
       "      <td>10.9</td>\n",
       "      <td>5.5</td>\n",
       "      <td>699</td>\n",
       "      <td>64</td>\n",
       "      <td>100</td>\n",
       "      <td>7.5</td>\n",
       "      <td>3.2</td>\n",
       "      <td>0.74</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>62</td>\n",
       "      <td>Male</td>\n",
       "      <td>7.3</td>\n",
       "      <td>4.1</td>\n",
       "      <td>490</td>\n",
       "      <td>60</td>\n",
       "      <td>68</td>\n",
       "      <td>7.0</td>\n",
       "      <td>3.3</td>\n",
       "      <td>0.89</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>58</td>\n",
       "      <td>Male</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>182</td>\n",
       "      <td>14</td>\n",
       "      <td>20</td>\n",
       "      <td>6.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>72</td>\n",
       "      <td>Male</td>\n",
       "      <td>3.9</td>\n",
       "      <td>2.0</td>\n",
       "      <td>195</td>\n",
       "      <td>27</td>\n",
       "      <td>59</td>\n",
       "      <td>7.3</td>\n",
       "      <td>2.4</td>\n",
       "      <td>0.40</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Age  Gender    TB   DB  Alkphos  Sgpt  Sgot   TP  ALB   A/G  Selector\n",
       "0   65  Female   0.7  0.1      187    16    18  6.8  3.3  0.90         1\n",
       "1   62    Male  10.9  5.5      699    64   100  7.5  3.2  0.74         1\n",
       "2   62    Male   7.3  4.1      490    60    68  7.0  3.3  0.89         1\n",
       "3   58    Male   1.0  0.4      182    14    20  6.8  3.4  1.00         1\n",
       "4   72    Male   3.9  2.0      195    27    59  7.3  2.4  0.40         1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>TB</th>\n",
       "      <th>DB</th>\n",
       "      <th>Alkphos</th>\n",
       "      <th>Sgpt</th>\n",
       "      <th>Sgot</th>\n",
       "      <th>TP</th>\n",
       "      <th>ALB</th>\n",
       "      <th>A/G</th>\n",
       "      <th>Selector</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>578</th>\n",
       "      <td>60</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.1</td>\n",
       "      <td>500</td>\n",
       "      <td>20</td>\n",
       "      <td>34</td>\n",
       "      <td>5.9</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.37</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>579</th>\n",
       "      <td>40</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.1</td>\n",
       "      <td>98</td>\n",
       "      <td>35</td>\n",
       "      <td>31</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>580</th>\n",
       "      <td>52</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.2</td>\n",
       "      <td>245</td>\n",
       "      <td>48</td>\n",
       "      <td>49</td>\n",
       "      <td>6.4</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>581</th>\n",
       "      <td>31</td>\n",
       "      <td>Male</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.5</td>\n",
       "      <td>184</td>\n",
       "      <td>29</td>\n",
       "      <td>32</td>\n",
       "      <td>6.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>582</th>\n",
       "      <td>38</td>\n",
       "      <td>Male</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>216</td>\n",
       "      <td>21</td>\n",
       "      <td>24</td>\n",
       "      <td>7.3</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.50</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Age Gender   TB   DB  Alkphos  Sgpt  Sgot   TP  ALB   A/G  Selector\n",
       "578   60   Male  0.5  0.1      500    20    34  5.9  1.6  0.37         2\n",
       "579   40   Male  0.6  0.1       98    35    31  6.0  3.2  1.10         1\n",
       "580   52   Male  0.8  0.2      245    48    49  6.4  3.2  1.00         1\n",
       "581   31   Male  1.3  0.5      184    29    32  6.8  3.4  1.00         1\n",
       "582   38   Male  1.0  0.3      216    21    24  7.3  4.4  1.50         2"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Convert Gender,Selector to numbericals\n",
    "le = preprocessing.LabelEncoder()\n",
    "df['Gender'] = le.fit_transform(df.Gender)\n",
    " \n",
    "df['Selector'] = le.fit_transform(df.Selector)\n",
    "#Get binarized gender columns\n",
    "#df['Gender'] = pd.get_dummies(df.Gender)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>TB</th>\n",
       "      <th>DB</th>\n",
       "      <th>Alkphos</th>\n",
       "      <th>Sgpt</th>\n",
       "      <th>Sgot</th>\n",
       "      <th>TP</th>\n",
       "      <th>ALB</th>\n",
       "      <th>A/G</th>\n",
       "      <th>Selector</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>578</th>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.1</td>\n",
       "      <td>500</td>\n",
       "      <td>20</td>\n",
       "      <td>34</td>\n",
       "      <td>5.9</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0.37</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>579</th>\n",
       "      <td>40</td>\n",
       "      <td>1</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.1</td>\n",
       "      <td>98</td>\n",
       "      <td>35</td>\n",
       "      <td>31</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.10</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>580</th>\n",
       "      <td>52</td>\n",
       "      <td>1</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.2</td>\n",
       "      <td>245</td>\n",
       "      <td>48</td>\n",
       "      <td>49</td>\n",
       "      <td>6.4</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>581</th>\n",
       "      <td>31</td>\n",
       "      <td>1</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.5</td>\n",
       "      <td>184</td>\n",
       "      <td>29</td>\n",
       "      <td>32</td>\n",
       "      <td>6.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>582</th>\n",
       "      <td>38</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>216</td>\n",
       "      <td>21</td>\n",
       "      <td>24</td>\n",
       "      <td>7.3</td>\n",
       "      <td>4.4</td>\n",
       "      <td>1.50</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Age  Gender   TB   DB  Alkphos  Sgpt  Sgot   TP  ALB   A/G  Selector\n",
       "578   60       1  0.5  0.1      500    20    34  5.9  1.6  0.37         1\n",
       "579   40       1  0.6  0.1       98    35    31  6.0  3.2  1.10         0\n",
       "580   52       1  0.8  0.2      245    48    49  6.4  3.2  1.00         0\n",
       "581   31       1  1.3  0.5      184    29    32  6.8  3.4  1.00         0\n",
       "582   38       1  1.0  0.3      216    21    24  7.3  4.4  1.50         1"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "features=(list(df.columns[:-1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X = df[features]\n",
    "y = df['Selector']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>TB</th>\n",
       "      <th>DB</th>\n",
       "      <th>Alkphos</th>\n",
       "      <th>Sgpt</th>\n",
       "      <th>Sgot</th>\n",
       "      <th>TP</th>\n",
       "      <th>ALB</th>\n",
       "      <th>A/G</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.1</td>\n",
       "      <td>187</td>\n",
       "      <td>16</td>\n",
       "      <td>18</td>\n",
       "      <td>6.8</td>\n",
       "      <td>3.3</td>\n",
       "      <td>0.90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>62</td>\n",
       "      <td>1</td>\n",
       "      <td>10.9</td>\n",
       "      <td>5.5</td>\n",
       "      <td>699</td>\n",
       "      <td>64</td>\n",
       "      <td>100</td>\n",
       "      <td>7.5</td>\n",
       "      <td>3.2</td>\n",
       "      <td>0.74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>62</td>\n",
       "      <td>1</td>\n",
       "      <td>7.3</td>\n",
       "      <td>4.1</td>\n",
       "      <td>490</td>\n",
       "      <td>60</td>\n",
       "      <td>68</td>\n",
       "      <td>7.0</td>\n",
       "      <td>3.3</td>\n",
       "      <td>0.89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>58</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>182</td>\n",
       "      <td>14</td>\n",
       "      <td>20</td>\n",
       "      <td>6.8</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>72</td>\n",
       "      <td>1</td>\n",
       "      <td>3.9</td>\n",
       "      <td>2.0</td>\n",
       "      <td>195</td>\n",
       "      <td>27</td>\n",
       "      <td>59</td>\n",
       "      <td>7.3</td>\n",
       "      <td>2.4</td>\n",
       "      <td>0.40</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Age  Gender    TB   DB  Alkphos  Sgpt  Sgot   TP  ALB   A/G\n",
       "0   65       0   0.7  0.1      187    16    18  6.8  3.3  0.90\n",
       "1   62       1  10.9  5.5      699    64   100  7.5  3.2  0.74\n",
       "2   62       1   7.3  4.1      490    60    68  7.0  3.3  0.89\n",
       "3   58       1   1.0  0.4      182    14    20  6.8  3.4  1.00\n",
       "4   72       1   3.9  2.0      195    27    59  7.3  2.4  0.40"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\nfrom sklearn.preprocessing import StandardScaler\\n# Scaling the features using StandardScaler:\\nX_scaler = StandardScaler()\\ny_scaler = StandardScaler()\\nX = X_scaler.fit_transform(X)\\ny = y_scaler.fit_transform(y)\\nX = X_scaler.transform(X)\\ny = y_scaler.transform(y)\\n'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "# Scaling the features using StandardScaler:\n",
    "X_scaler = StandardScaler()\n",
    "y_scaler = StandardScaler()\n",
    "X = X_scaler.fit_transform(X)\n",
    "y = y_scaler.fit_transform(y)\n",
    "X = X_scaler.transform(X)\n",
    "y = y_scaler.transform(y)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import Imputer\n",
    "X = Imputer().fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# split dataset to 60% training and 40% testing\n",
    "X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.4, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(349, 10) (349,)\n"
     ]
    }
   ],
   "source": [
    "print (X_train.shape, y_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature importances with forests of trees\n",
    "\n",
    "This examples shows the use of forests of trees to evaluate the importance of features on an artificial classification task. The red bars are the feature importances of the forest, along with their inter-trees variability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature ranking:\n",
      "1. feature 4 - Alkphos (0.119578) \n",
      "2. feature 6 - Sgot (0.118929) \n",
      "3. feature 5 - Sgpt (0.115570) \n",
      "4. feature 0 - Age (0.115264) \n",
      "5. feature 2 - TB (0.111479) \n",
      "6. feature 3 - DB (0.102925) \n",
      "7. feature 8 - ALB (0.099643) \n",
      "8. feature 9 - A/G (0.097362) \n",
      "9. feature 7 - TP (0.093450) \n",
      "10. feature 1 - Gender (0.025800) \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABHUAAANDCAYAAAAjFiMTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3XuY1nWd//HXDJADiHgYBYsYkZIoC2WINc0fJmwYlbVl\n5KyE1K7akpliB83S0PJwRY1ReCizbLXR3Nq1g60l2pnLjCG3ZGIti0nM2e5KrcZRhPv3x8TkyGAM\nQjMfeDyu674u+Mz3/t7ve778Yc++h5pqtVoNAAAAAEWpHegBAAAAAOg/UQcAAACgQKIOAAAAQIFE\nHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcA2OmtXbs2tbW1+dzn\nPjfQowAAbDeiDgDsZK655prU1tb2+Xrve9+7wz7361//ehYvXrzD9v901dTUDPQIT8vll1+ea665\nZqDHAAAGkaEDPQAAsP3V1NTkggsuyAEHHNBr/eCDD95hn3nzzTfnsssuy3nnnbfDPmNbNTQ05JFH\nHsmwYcMGepRtdtlll2XffffNiSeeONCjAACDhKgDADupY445JlOnTv27fV61Wt0h++3s7MyIESOe\n9n6e8YxnbIdp/v4eeeSRDB8+fKDHAAAGIZdfAcAu7Nprr820adMyYsSI7LPPPmlqasp9993Xa5vv\nfe97mTt3bhoaGlJXV5fx48dn0aJF6erq6tnmzW9+cy677LIk6bnUa8iQIUmSb33rW6mtrc13vvOd\nXvvt6z43CxYsyKhRo3Lvvfdmzpw52WOPPTJv3ryen99xxx055phjsueee2bkyJE56qij8oMf/OBv\nfs+n+qxf//rXedWrXpVRo0Zl3LhxPd/jJz/5SWbOnJndd989BxxwQFpaWnrtc9Nlbt/97ndzyimn\npL6+PqNHj86JJ56YBx98cLMZLrvsshx88MGpq6vLs571rJx66ql56KGHem1z1FFH5UUvelFaW1vz\n//7f/8vIkSPz3ve+NxMmTMjdd9/d87usra3N0UcfnST5wx/+kHe+85150YtelFGjRmX06NGZM2dO\n/ud//qfXvr/97W+ntrY2N954Yz70oQ/l2c9+doYPH55Zs2blF7/4xWbz3nHHHZkzZ0723nvv7L77\n7pkyZUqWLl3aa5s1a9bkuOOOyz777JPhw4fnxS9+cb7yla/02ubxxx/P4sWLc9BBB2X48OGpr6/P\nkUcemeXLl/+twwYA/A3O1AGAndRDDz2U3/3ud73W9tlnn54/f+hDH8q5556b448/PieddFJ++9vf\nZunSpZkxY0ZWrVqVPfbYI0ly44035pFHHsnChQuzzz775Ic//GE+/vGPZ926dbnhhhuSJG9961tz\n//3359Zbb811113X66ydmpqarb6fTU1NTR5//PHMnj07Rx55ZD7ykY/0nKVz2223Zc6cOZk2bVo+\n8IEPpLa2Np/5zGdy9NFH53vf+16mTZvWr99PTU1NNm7cmFe84hWZMWNGPvzhD+e6667L29/+9owc\nOTLnnHNO5s2bl9e//vW54oorcuKJJ+bwww9PQ0NDr/2ceuqp2WuvvbJ48eKsWbMml112Wdrb23P7\n7bf3bPOBD3wg559/fl7+8pdn4cKFPdv96Ec/yve///2eAFZTU5NKpZI5c+bk+OOPz/z58zNmzJi8\n7GUvy6mnnppRo0blfe97X6rVasaMGZMkuffee/PlL385b3jDGzJhwoR0dHTkyiuvzFFHHZXVq1dn\n7Nixvea9+OKLM2TIkLzrXe/KQw89lEsuuSTz5s3LihUrerb55je/mVe/+tV55jOfmdNPPz1jx45N\nW1tbvva1r+W0005Lktx999156UtfmnHjxuXss8/OyJEj84UvfCGvfe1r86UvfSmvec1rkiTnnXde\nLr744px88sl58YtfnIcffjg/+tGP0trampkzZ/brmAEAT1IFAHYqn/3sZ6s1NTWbvWpra3u2Wbt2\nbXXo0KHViy++uNd777777uqwYcOqF110Uc9aV1fXZp9x8cUXV4cMGVL99a9/3bN26qmn9vqMTb71\nrW9Va2trq9/+9rd7rf/qV7+q1tTUVK+55pqetQULFlRra2ur55xzzmb7Oeigg6pz5szptdbV1VU9\n8MADq7Nnz97Sr+NvftYll1zSs/bggw9WR4wYUR0yZEj1xhtv7Flfs2ZNtaamprp48eKetU2/5+nT\np1cff/zxnvUPf/jD1dra2upXvvKVarVarf72t7+t7rbbbtVXvOIVvWZatmxZtba2tvrZz362Z+2o\no46q1tbWVj/1qU9t9h0OPvjg6ste9rLN1h977LHN1tauXVutq6urfvCDH+xZ+9a3vlWtqampvuAF\nL+g179KlS6u1tbXVu+++u1qtVqsbNmyoTpgwoXrggQdWH3744c32vcnMmTOrhxxySHX9+vW91o84\n4ojqpEmTev5+yCGHVF/96ldvcT8AwLZz+RUA7IRqampy+eWX59Zbb+15ffOb3+z5+Re/+MVUq9W8\n4Q1vyO9+97ue13777ZfnPve5vc4y2W233Xr+3NnZmd/97nd5yUteko0bN2bVqlU7ZP63vvWtvf7+\n4x//OPfcc0+ampp6zfvHP/4xM2fO3OzSrv74l3/5l54/jx49OpMmTcrIkSNz3HHH9awfdNBB2XPP\nPXPvvfdu9v6TTz6550ybJPm3f/u3DBkyJDfffHOS7rNe1q9fn9NPP73X+0466aSMGjUqX/va13qt\n77bbblmwYMFWz//Emz9v3Lgxv//97zNixIhMmjQpra2tm23/lre8pde8Rx55ZKrVas93W7VqVX71\nq1/l9NNPz6hRo/r8zD/84Q+5/fbb84Y3vKHnjLBNr5e//OW555578pvf/CZJsueee+buu+/Oz3/+\n863+TgDA1nH5FQDspF784hdv8UbJP//5z7Nx48Y85znP2exnNTU1vW4q/Otf/zrvf//785WvfCV/\n+MMfem335HvCbA9Dhw7NuHHjeq3dc889SZL58+f3+Z7a2to89NBDGT16dL8+q66urtclaUl32Hny\n529af+L3T7p/B0/+HY4cOTL7779/fvWrXyVJ2tvbk3SHoScaNmxYDjzwwKxdu7bX+rOe9awMHbr1\n/4lWrVZz6aWX5vLLL88vf/nLbNiwoWe2+vr6zbZ/9rOf3evve+21V5L0fLdf/OIXqampyQte8IIt\nfubPf/7zVKvVvP/978/73ve+zX5eU1OT//u//8v++++f888/P6997Wtz0EEH5eCDD84xxxyTN73p\nTXnhC1+41d8RAOibqAMAu6CNGzemtrY2//3f/53a2s1P3N199917tps1a1YefPDBnH322T1nsaxb\nty4nnnhiNm7c+Dc/a0v309kUH57siWcGPXHeJPnIRz6SKVOm9Pm+TTP3xxPPWNma9eoOesLXE/X3\nSVeb7o30r//6r/ngBz+YvffeO7W1tXnHO97R5/HZHt9t037f+c53Zvbs2X1usyl2HXnkkfnFL36R\nm266Kd/4xjfy6U9/Os3Nzbnyyivzlre8Zas/EwDYnKgDALugiRMnplqt5oADDujzbJ1NfvKTn+Se\ne+7Jv//7v+eEE07oWb/11ls323ZL8WavvfZKtVrd7IlQm85k2dp5k2TUqFE9T30aDKrVau65557M\nmDGjZ+3Pf/5zfvOb3+SVr3xlkvTcWHnNmjU54IADerZbv359fvnLX+Yf//Eft+qztvT7/eIXv5ij\njz46n/zkJ3utP/jgg9l3333783WS/PXfxk9/+tMt/q4PPPDAJN1nG23N8dhzzz1z4okn5sQTT0xn\nZ2eOPPLIfOADHxB1AOBpck8dANgFve51r0ttbW0WL17c589///vfJ/nrWR1PPuPj0ksv3SwyjBw5\nMkny8MMP91pvaGjIkCFDNrvvzWWXXbbVT8VqbGzMxIkTs2TJkvz5z3/e7OeVSmWr9rMjfPKTn8zj\njz/e8/fLLrssGzZsyJw5c5Iks2bNyrBhwzZ7HPhVV12Vhx9+OK961au26nNGjhzZ56PShwwZstlZ\nNjfeeGPWrVvX36+SJJk6dWomTJiQSy+9dIuX1+2777456qijcuWVV+aBBx7Y7OdPPB6b/i1tMmLE\niDznOc/Jo48+uk3zAQB/5UwdANgJ/a1LaQ488MB88IMfzHvf+9788pe/zGtf+9qMGjUq9957b/7r\nv/4rp5xyShYtWpTnPe95mThxYs4888zcd9992WOPPfLFL36xz7jQ2NiYarWat7/97Zk9e3aGDBmS\nN77xjdljjz3yhje8oSdqTJw4MV/96lfz29/+dqu/T01NTa666qrMmTMnL3jBC/LmN785z3rWs7Ju\n3brcfvvtGT16dG666ab+/ZK2k8ceeywzZ87M3Llz87Of/SyXX355jjzyyJ5YU19fn7PPPjvnn39+\njjnmmBx77LE9202fPr3XGVBPpbGxMVdccUU+9KEP5TnPeU7222+/vOxlL8urXvWqXHDBBXnLW96S\nww8/PD/5yU9y3XXX9Zzd1F+bbrJ97LHH5pBDDsmb3/zm7L///vnZz36W1atX5+tf/3qSZNmyZTny\nyCPzwhe+MCeddFIOPPDAdHR0ZMWKFVm3bl3PTbSf//zn56ijjkpjY2P23nvv3HnnnfmP//iPnkej\nAwDbTtQBgJ3Q1pwB8573vCeTJk1Kc3Nzzj///CTdN9HdFB6S7psWf/WrX81pp52Wiy++OHV1dXnd\n616Xt73tbZvd2+Z1r3tdTjvttFx//fW57rrrUq1W88Y3vjFJ8vGPfzyPP/54rrzyyuy222554xvf\nmCVLluTggw/e6tlnzJiRFStW5IILLsiyZcvypz/9KWPHjs0//MM/5JRTTtmm38mWPmtL2z55vaam\nJp/4xCdy3XXX5bzzzsv69etzwgkn5GMf+1iv7c4777zst99++cQnPpFFixZl7733zlvf+tZ86EMf\n2uweN1ua6dxzz017e3s+/OEP549//GNmzJiRl73sZXnve9+bzs7OfP7zn88XvvCFNDY25uabb85Z\nZ53V57xb831f/vKX5/bbb8/ixYvz0Y9+NBs3bszEiRNz8skn92wzefLk/OhHP8rixYtzzTXX9Dw9\n7dBDD825557bs9073vGOfPnLX843v/nNPProo2loaMiFF16Yd77znX3OAgBsvZrq3+OOfwAAO5lr\nrrkmb3nLW3LnnXdu8SljAAA70jbdU2fZsmWZMGFChg8fnsMOOyx33nnnFrddvXp1jjvuuEyYMCG1\ntbWbXU++yf333583velNqa+vz4gRIzJlypS0trZuy3gAAAAAO71+R50bbrghZ555ZhYvXpxVq1Zl\nypQpmT179hZvUNjZ2ZmJEyfmkksuyf7779/nNg8++GCOOOKI7LbbbrnlllvS1taWj3zkI9lrr736\nOx4AwN+NE54BgIHU73vqNDc355RTTsn8+fOTJFdccUW+9rWv5eqrr8673/3uzbafNm1apk2blqT7\n2v2+XHzxxRk/fnyuuuqqnrVNj/8EABistvbpXQAAO0K/ztRZv359Vq5cmZkzZ/as1dTUZNasWVmx\nYsU2D/GVr3wl06ZNy9y5czNmzJhMnTq1V+ABABhsTjzxxGzYsMH9dACAAdOvM3UqlUo2bNiQMWPG\n9FofM2ZM1qxZs81D3Hvvvbn88stz5pln5pxzzskPf/jDnHbaadltt93ypje9qc85brnllhxwwAEZ\nPnz4Nn8uAAAAwGDyyCOP5Fe/+lVmz56d+vr6p9x2UDzSfOPGjZk+fXouuOCCJMmUKVPy05/+NFdc\ncUWfUeeWW27JvHnz/t5jAgAAAPxdXHvttTnhhBOecpt+RZ36+voMGTIkHR0dvdY7OjoyduzY/k/4\nF/vvv38mT57ca23y5Mn50pe+1Of2BxxwQJLuL/jk9/H3d8YZZ6S5uXmgxyCOxWDhOAwejsXg4VgM\nDo7D4OFYDB6OxeDgOAwejsXAa2try7x583rax1PpV9QZNmxYGhsbs3z58hx77LFJup/6sHz58px2\n2mnbNGySHHHEEZtdvrVmzZot3ix50yVXkydPdh37IDB69GjHYZBwLAYHx2HwcCwGD8dicHAcBg/H\nYvBwLAYHx2HwcCwGj6253Uy/L79atGhRFixYkMbGxkyfPj3Nzc3p7OzMggULkiTz58/PuHHjcuGF\nFybpvrny6tWrU61W89hjj2XdunW56667svvuu2fixIlJukvgEUcckYsuuihz587NHXfckauuuiqf\n+tSn+jseAAAAwC6h31Fn7ty5qVQqOffcc9PR0ZFDDjkkt9xyS/bdd98kyX333ZehQ/+62/vvvz+H\nHnpozyM/lyxZkiVLlmTGjBm57bbbknQ/9vw///M/c9ZZZ+WCCy7IhAkT8rGPfSzHH3/89viOAAAA\nADudbbpR8sKFC7Nw4cI+f7Yp1GzS0NCQjRs3/s19zpkzJ3PmzNmWcQAAAAB2ObUDPQDla2pqGugR\n+AvHYnBwHAYPx2LwcCwGB8dh8HAsBg/HYnBwHAYPx6IsNdVqtTrQQ/RXa2trGhsbs3LlSjdwAgAA\nAHYa/WkeztQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok\n6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok\n6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok\n6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok\n6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok\n6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok\n6gAAAAAUSNQBAAAAKJCoAwAAAFCgoQM9AOVpael+JUlXV7J2bdLQkNTVda81NXW/AAAAgB1H1KHf\nnhhtWluTxsbuyDN16sDOBQAAALsSl18BAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECB\nRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQoKEDPQCw7Vpaul9J0tWV\nrF2bNDQkdXXda01N3S8AAAB2PqIOFOyJ0aa1NWls7I48U6cO7FwAAADseC6/AgAAACiQqAMAAABQ\nIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKNHSgBwAAdj4tLd2vJOnqStauTRoa\nkrq67rWmpu4XAADbTtQBALa7J0ab1taksbE78kydOrBzAQDsTFx+BQAAAFAgUQcAAACgQKIOAAAA\nQIHcUwfgaXJDWAAAYCCIOgBPkxvCAgAAA8HlVwAAAAAFEnUAAAAACiTqAAAAABRI1AEAAAAokKgD\nAAAAUKBtijrLli3LhAkTMnz48Bx22GG58847t7jt6tWrc9xxx2XChAmpra3N0qVLn3LfF198cWpr\na7No0aJtGQ0AAABgl9DvqHPDDTfkzDPPzOLFi7Nq1apMmTIls2fPTqVS6XP7zs7OTJw4MZdcckn2\n33//p9z3nXfemU9+8pOZMmVKf8cCAAAA2KX0O+o0NzfnlFNOyfz58/O85z0vV1xxRUaMGJGrr766\nz+2nTZuWSy65JHPnzs0znvGMLe73T3/6U+bNm5errroqe+65Z3/HAgAAANil9CvqrF+/PitXrszM\nmTN71mpqajJr1qysWLHiaQ3ytre9La9+9atz9NFHP639AAAAAOwKhvZn40qlkg0bNmTMmDG91seM\nGZM1a9Zs8xDXX399fvzjH+dHP/rRNu8DAAAAYFfSr6izI9x33305/fTTc+utt2bYsGH9eu8ZZ5yR\n0aNH91prampKU1PT9hwRAAAAYLtraWlJS0tLr7WHHnpoq9/fr6hTX1+fIUOGpKOjo9d6R0dHxo4d\n259d9Vi5cmV++9vfZurUqalWq0mSDRs25Dvf+U4+8YlP5NFHH01NTU2f721ubs7UqVO36XMBAAAA\nBlJfJ6a0tramsbFxq97fr3vqDBs2LI2NjVm+fHnPWrVazfLly3P44Yf3Z1c9Zs2alZ/85Cf58Y9/\nnLvuuit33XVXpk2blnnz5uWuu+7aYtABAAAA2JX1+/KrRYsWZcGCBWlsbMz06dPT3Nyczs7OLFiw\nIEkyf/78jBs3LhdeeGGS7psrr169OtVqNY899ljWrVuXu+66K7vvvnsmTpyYkSNH5vnPf36vzxg5\ncmT22WefTJ48+el/QwAAAICdUL+jzty5c1OpVHLuueemo6MjhxxySG655Zbsu+++SbrvkTN06F93\ne//99+fQQw/tOeNmyZIlWbJkSWbMmJHbbrutz89wdg4AAADAU9umGyUvXLgwCxcu7PNnTw41DQ0N\n2bhxY7/2v6XYAwAAAEC3ft1TBwAAAIDBQdQBAAAAKNA2XX4FAINRS0v3K0m6upK1a5OGhqSurnut\nqan7BQAAOwNRB4CdxhOjTWtr0tjYHXmmTh3YuQAAYEdw+RUAAABAgUQdAAAAgAKJOgAAAAAFEnUA\nAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAog4AAABAgUQdAAAAgAKJOgAAAAAFEnUA\nAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAog4AAABAgUQdAAAAgAKJOgAAAAAFEnUA\nAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAQwd6AAAAdoyWlu5XknR1JWvXJg0NSV1d\n91pTU/cLACiTqAMAsJN6YrRpbU0aG7sjz9SpAzsXALB9uPwKAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6\nAAAAAAUSdQAAAAAKNHSgBwAAgJ1dS0v3K0m6upK1a5OGhqSurnutqan7BQD9IeoAAMAO9sRo09qa\nNDZ2R56pUwd2LgDK5vIrAAAAgAJtU9RZtmxZJkyYkOHDh+ewww7LnXfeucVtV69eneOOOy4TJkxI\nbW1tli5dutk2F110UaZPn5499tgjY8aMyT/90z/lf//3f7dlNAAAAIBdQr+jzg033JAzzzwzixcv\nzqpVqzJlypTMnj07lUqlz+07OzszceLEXHLJJdl///373Oa73/1u3v72t+eOO+7IrbfemvXr1+fl\nL395Hnnkkf6OBwAAALBL6Pc9dZqbm3PKKadk/vz5SZIrrrgiX/va13L11Vfn3e9+92bbT5s2LdOm\nTUuSvOc97+lznzfffHOvv3/2s5/Nfvvtl5UrV+alL31pf0cEAAAA2On160yd9evXZ+XKlZk5c2bP\nWk1NTWbNmpUVK1Zst6EefPDB1NTUZO+9995u+wQAAADYmfQr6lQqlWzYsCFjxozptT5mzJg88MAD\n22WgarWa008/PS996Uvz/Oc/f7vsEwAAAGBnM+geab5w4cKsXr063//+9//mtmeccUZGjx7da62p\nqSlNm54XCQAAADBItbS0pKWlpdfaQw89tNXv71fUqa+vz5AhQ9LR0dFrvaOjI2PHju3Prvp06qmn\n5uabb853v/vdLd5U+Ymam5szderUp/25JWhvb9/izagHUlvb8CST09bWlmRw3ti6vr4+48ePH+gx\nAAAAoJe+TkxpbW1NY2PjVr2/X1Fn2LBhaWxszPLly3Pssccm6b5cavny5TnttNP6s6vNnHrqqbnp\nppvy7W9/2/8Af5L29vZMnjQpnV1dAz1KHw5N0pp5805Ismqgh+nTiLq6tK1Z498VAAAAO5V+X361\naNGiLFiwII2NjZk+fXqam5vT2dmZBQsWJEnmz5+fcePG5cILL0zSfXPl1atXp1qt5rHHHsu6dety\n1113Zffdd8/EiROTdF9y1dLSki9/+csZOXJkz5lAo0ePTl1d3Xb6quWqVCrp7OrKtUkmD/QwT9KW\nZF4yKGdL/jJfV1cqlYqoAwAAwE6l31Fn7ty5qVQqOffcc9PR0ZFDDjkkt9xyS/bdd98kyX333Zeh\nQ/+62/vvvz+HHnpoampqkiRLlizJkiVLMmPGjNx2221Juh+LXlNTk6OOOqrXZ33mM5/peXQ63dFk\nsF5sNphn254G62VwyeC/FM5lcAAAANvXNt0oeeHChVm4cGGfP9sUajZpaGjIxo0bn3J/f+vnMBgM\n7svgksF+KZzL4AAAALavQff0KxisBvNlcMngvhTOZXAAAADbn6gD/TTYLzUb7PMBAAyklpbuV5J0\ndSVr1yYNDcmmW3k2NXW/AEog6gAAALuMJ0ab1taksbE78kz1/4oBBaod6AEAAAAA6D9RBwAAAKBA\nog4AAABAgUQdAAAAgAKJOgAAAAAFEnUAAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBA\nog4AAABAgUQdAAAAgAINHegBALZFe3t7KpXKQI+xmba24Ukmp62tLckjAz1On+rr6zN+/PiBHgMA\nAHiaRB1flRw0AAAgAElEQVSgOO3t7Zk8aVI6u7oGepQ+HJqkNfPmnZBk1UAP06cRdXVpW7Nmu4Ud\ngW3biGsAADxdog5QnEqlks6urlybZPJAD/MkbUnmJYNytuQv83V1pVKpbJegILBtu+0d1wAA2PWI\nOkCxJieZOtBDbMFgnm17Eti2zfaOawAA7JpEHQCetsEcsQbzbAAA8HR4+hUAAABAgUQdAAAAgAKJ\nOgAAAAAFEnUAAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAQwd6AAAAAHYtLS3dryTp\n6krWrk0aGpK6uu61pqbuF/DURB0AAAD+rp4YbVpbk8bG7sgzderAzgWlcfkVAAAAQIFEHQAAAIAC\niToAAAAABRJ1AAAAAArkRskAsBNob29PpVIZ6DH61NY2PMnktLW1JXlkoMfZTH19fcaPHz/QYwAA\n9JuoAwCFa29vz+RJk9LZ1TXQo2zBoUlaM2/eCUlWDfQwmxlRV5e2NWuEHQCgOKIOABSuUqmks6sr\n1yaZPNDD9KEtybxkUM7XlmReV1cqlYqoAwAUR9QBgJ3E5CRTB3qIpzDY5wMAKI0bJQMAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACDR3oAQAAdibt7e2pVCoDPcZm2tqGJ5mc\ntra2JI8M9Dh9qq+vz/jx4wd6DAAohqgDALCdtLe3Z/KkSens6hroUfpwaJLWzJt3QpJVAz1Mn0bU\n1aVtzRphBwC2kqgDALCdVCqVdHZ15dokkwd6mCdpSzIvGZSzJX+Zr6srlUpF1AGArSTqAABsZ5OT\nTB3oIbZgMM8GAPSPGyUDAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAm1T\n1Fm2bFkmTJiQ4cOH57DDDsudd965xW1Xr16d4447LhMmTEhtbW2WLl36tPcJAAAAsKvrd9S54YYb\ncuaZZ2bx4sVZtWpVpkyZktmzZ6dSqfS5fWdnZyZOnJhLLrkk+++//3bZJwAAAMCurt9Rp7m5Oaec\nckrmz5+f5z3vebniiisyYsSIXH311X1uP23atFxyySWZO3dunvGMZ2yXfQIAAADs6voVddavX5+V\nK1dm5syZPWs1NTWZNWtWVqxYsU0D7Ih9AgAAAOzs+hV1KpVKNmzYkDFjxvRaHzNmTB544IFtGmBH\n7BMAAABgZzd0oAd4Os4444yMHj2611pTU1OampoGaCIAAACArdPS0pKWlpZeaw899NBWv79fUae+\nvj5DhgxJR0dHr/WOjo6MHTu2P7vaLvtsbm7O1KlTt+lzAQDYebW3tw/ah260tQ1PMjltbW1JHhno\ncTZTX1+f8ePHD/QYALuEvk5MaW1tTWNj41a9v19RZ9iwYWlsbMzy5ctz7LHHJkmq1WqWL1+e0047\nrT+72qH7BABg19Xe3p7Jkyals6troEfZgkOTtGbevBOSrBroYTYzoq4ubWvWCDsABej35VeLFi3K\nggUL0tjYmOnTp6e5uTmdnZ1ZsGBBkmT+/PkZN25cLrzwwiTdN0JevXp1qtVqHnvssaxbty533XVX\ndt9990ycOHGr9gkAAFurUqmks6sr1yaZPNDD9KEtybxkUM7XlmReV1cqlYqoA1CAfkeduXPnplKp\n5Nxzz01HR0cOOeSQ3HLLLdl3332TJPfdd1+GDv3rbu+///4ceuihqampSZIsWbIkS5YsyYwZM3Lb\nbbdt1T4BAKC/JicZzBfqD/b5ABj8tulGyQsXLszChQv7/NmmULNJQ0NDNm7c+LT2CQAAAEBv/Xqk\nOQAAAACDg6gDAAAAUCBRBwAAAKBAog4AAABAgUQdAAAAgAKJOgAAAAAF2qZHmgMAAGyN9vb2VCqV\ngR6jT21tw5NMTltbW5JHBnqczdTX12f8+PEDPQYwiIk6AADADtHe3p7Jkyals6troEfZgkOTtGbe\nvBOSrBroYTYzoq4ubWvWCDvAFok6AADADlGpVNLZ1ZVrk0we6GH60JZkXjIo52tLMq+rK5VKRdQB\ntkjUAQAAdqjJSaYO9BBPYbDPB7AlbpQMAAAAUCBRBwAAAKBAog4AAABAgUQdAAAAgAKJOgAAAAAF\nEnUAAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAog4AAABAgUQdAAAAgAKJOgAAAAAF\nEnUAAAAACiTqAAAAABRo6EAPAAAAwI7X3t6eSqUy0GNspq1teJLJaWtrS/LIQI/Tp/r6+owfP36g\nx4DNiDoAAAA7ufb29kyeNCmdXV0DPUofDk3SmnnzTkiyaqCH6dOIurq0rVkj7DDoiDoAAAA7uUql\nks6urlybZPJAD/MkbUnmJYNytuQv83V1pVKpiDoMOqIOAADALmJykqkDPcQWDObZYLByo2QAAACA\nAjlTBwrWkuPTkqYkSVd2y0FZk7NyUeryaJKkKS1pyvUDOSIAAAA7iKgDBWvK9aINAADALsrlVwAA\nAAAFcqYOADsNlyQCALArEXUA2Gm4JBEAgF2Jy68AAAAACiTqAAAAABTI5Vf0m3tWAAAAwMATdeg3\n96wAAACAgefyKwAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACg\nQKIOAAAAQIFEHQAAAIACiToAAAAABRo60AMAlK4lx6clTUmSruyWg7ImZ+Wi1OXRJElTWtKU6wdy\nRAAAYCck6gA8TU25XrQBAAD+7lx+BQAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1\nAAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1\nAAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1\nAAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1\nAAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQNsUdZYtW5YJEyZk+PDhOeyww3LnnXc+\n5fY33nhjJk+enOHDh2fKlCn5+te/3uvnf/7zn3Pqqafm2c9+dkaMGJEXvOAFufLKK7dlNAAAAIBd\nQr+jzg033JAzzzwzixcvzqpVqzJlypTMnj07lUqlz+1/8IMf5J//+Z9z0kkn5cc//nFe85rX5LWv\nfW1Wr17ds80ZZ5yRb3zjG/n85z+fn/3sZzn99NNz6qmn5qtf/eq2fzMAAACAnVi/o05zc3NOOeWU\nzJ8/P8973vNyxRVXZMSIEbn66qv73H7p0qV5xStekUWLFmXSpEk5//zzM3Xq1HziE5/o2WbFihU5\n8cQTc+SRR2b8+PE56aSTMmXKlPzwhz/c9m8GAAAAsBPrV9RZv359Vq5cmZkzZ/as1dTUZNasWVmx\nYkWf71mxYkVmzZrVa2327Nm9tj/88MPz5S9/Offff3+S5Pbbb88999yT2bNn92c8AAAAgF3G0P5s\nXKlUsmHDhowZM6bX+pgxY7JmzZo+3/PAAw/0uf0DDzzQ8/ePf/zjOfnkkzNu3LgMHTo0Q4YMyac+\n9akcccQR/RkPAAAAYJfRr6izoyxdujR33HFHvvrVr2b8+PH5zne+k4ULF+aZz3xmjj766IEeDwAA\nAGDQ6VfUqa+vz5AhQ9LR0dFrvaOjI2PHju3zPWPHjn3K7bu6unLOOefkpptuyjHHHJMkOfjgg7Nq\n1aosWbLkKaPOGWeckdGjR/daa2pqSlNTU3++FgAAAMDfXUtLS1paWnqtPfTQQ1v9/n5FnWHDhqWx\nsTHLly/PsccemySpVqtZvnx5TjvttD7f85KXvGSzn3/zm9/MS17ykiTd9+lZv359ampqer1vyJAh\n2bhx41PO09zcnKlTp/bnKwAAAAAMCn2dmNLa2prGxsaten+/L79atGhRFixYkMbGxkyfPj3Nzc3p\n7OzMggULkiTz58/PuHHjcuGFFyZJ3vGOd+Soo47KRz/60bzyla9MS0tLVq5cmU996lNJklGjRmXG\njBl517velbq6ujQ0NORb3/pWPve5z+XSSy/t73gAwCDQkuPTku7/QOnKbjkoa3JWLkpdHk2SNKUl\nTbl+IEcEAChev6PO3LlzU6lUcu6556ajoyOHHHJIbrnlluy7775Jkvvuuy9Dh/51ty95yUvy+c9/\nPuecc07OOeecPPe5z81NN92U5z//+T3b3HDDDTn77LMzb968/P73v09DQ0MuuuiinHzyydvhKwIA\nf29NuV60AQDYwbbpRskLFy7MwoUL+/zZbbfdttna61//+rz+9a/f4v7222+/fPrTn96WUQAAAAB2\nSbUDPQAAAAAA/SfqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAog4AAABAgbbpkeYAAMDWa8nx\naUlTkqQru+WgrMlZuSh1eTRJ0pSWNOX6gRwRgAKJOgAAsIM15XrRBoDtzuVXAAAAAAUSdQAAAAAK\nJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoEAeaQ4AAOwyWnJ8WtKUJOnKbjkoa3JWLkpdHk2S\nNKXF4+eBYog6AADALqMp14s2wE7D5VcAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAg\nUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRo60AMAAACwa2nJ8WlJU5KkK7vloKzJWbkodXk0\nSdKUljTl+oEcEYog6gAAAPB31ZTrRRvYDlx+BQAAAFAgUQcAAACgQC6/AgDYSblnBQDs3EQdAICd\nlHtWAMDOzeVXAAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECB\nRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECB\nRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECB\nRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECB\nRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECB\nRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAA/P/27j7Iyvq+//9ruVEBK1GhuxGJIS3lJo6LrEXW\nm4yJmA0agZSEEWqQNHVs0YDChNqgJGjiTaaVyqDSaGkmoyC1EyWxoYiro0aI4nIzcaRE4y9fRd01\n26lYgxqE/f3hT5r9uqEuyc+zH8/jMXON8XM+1/F9zSUBn3OdswUSdQAAAAAKJOoAAAAAFEjUAQAA\nACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAA\nACiQqAMAAABQoIOKOjfddFOGDRuWfv36Zfz48dm0adMB9991110ZNWpU+vXrl/r6+qxdu/Zde7Zv\n357JkyfnQx/6UA4//PCcfPLJ2blz58GMBwAAAPCB1+2os3r16syfPz+LFy/Oli1bUl9fn6amprS3\nt3e5f8OGDZkxY0YuvPDCbN26NZMnT86UKVPy1FNP7d/z85//PKeffnpGjx6dhx9+OD/96U9z5ZVX\n5rDDDjv4KwMAAAD4AOt21FmyZEkuuuiizJw5MyNHjszy5cvTv3//rFixosv9S5cuzcSJEzNv3ryM\nGDEiV111VcaOHZtly5bt33PFFVfknHPOybXXXpsTTjghw4YNy2c/+9kMGjTo4K8MAAAA4AOsW1Fn\nz549aWlpyZlnnrl/raamJhMmTMjGjRu7PGfjxo2ZMGFCp7Wmpqb9+zs6OvJv//ZvGT58eD7zmc+k\ntrY248ePz5o1a7p7LQAAAABVo1tRp729PXv37k1tbW2n9dra2rS2tnZ5Tmtr6wH3v/zyy3nttddy\n/fXX5+yzz8769evzuc99Ln/2Z3+WRx55pDvjAQAAAFSNPpUeYN++fUmSKVOmZM6cOUmSE044IRs2\nbMjy5ctz+umnV3I8AAAAgB6pW1Fn0KBB6d27d9ra2jqtt7W1pa6urstz6urqDrh/0KBB6dOnT0aN\nGtVpz6hRo/Loo48ecJ7LLrssAwcO7LQ2ffr0TJ8+/T1dDwAAAEClrFq1KqtWreq0tmvXrvd8frei\nTt++fdPQ0JDm5uZMmjQpydvfidPc3Lz/KZv/W2Nj47teX79+fRobG/e/55/+6Z9mx44dnc772c9+\nluOOO+6A8yxZsiRjx47tziUAAAAA9AhdPZiyefPmNDQ0vKfzu/3xq3nz5mXWrFlpaGjIuHHjsmTJ\nkuzevTuzZs1KksycOTPHHntsrrnmmiTJ3Llzc8YZZ+SGG27IOeeck1WrVqWlpSW33nrr/vf86le/\nmvPOOy+nn356PvnJT2bt2rW5995789BDD3V3PAAAAICq0O2oM23atLS3t2fRokVpa2vLmDFjsm7d\nugwePDhJsnPnzvTp8z9v29jYmJUrV2bhwoVZuHBhhg8fnjVr1mT06NH790yZMiXLly/PNddck7lz\n52bEiBH5/ve/v/9pHgAAAAA6O6gvSp49e3Zmz57d5WsPPPDAu9amTp2aqVOnHvA9Z82atf9pHwAA\nAAAOrFs/0hwAAACAnkHUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAok6AAAAAAUS\ndQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAAoECiDgAAAECBRB0AAACAAvWp9AAAAABA\nZaxa9faRJG+8kfyf/5Mcd1xy2GFvr02f/vZBzyTqAAAAQJX6zWizeXPS0PB25Bk7trJz8d74+BUA\nAABAgUQdAAAAgAKJOgAAAAAFEnUAAAAACiTqAAAAABRI1AEAAAAo0EFFnZtuuinDhg1Lv379Mn78\n+GzatOmA+++6666MGjUq/fr1S319fdauXftb9/7VX/1VevXqlaVLlx7MaAAAAABVodtRZ/Xq1Zk/\nf34WL16cLVu2pL6+Pk1NTWlvb+9y/4YNGzJjxoxceOGF2bp1ayZPnpwpU6bkqaeeetfeu+++O489\n9liGDBnS/SsBAAAAqCLdjjpLlizJRRddlJkzZ2bkyJFZvnx5+vfvnxUrVnS5f+nSpZk4cWLmzZuX\nESNG5KqrrsrYsWOzbNmyTvteeOGFzJ07NytXrkyfPn0O7moAAAAAqkS3os6ePXvS0tKSM888c/9a\nTU1NJkyYkI0bN3Z5zsaNGzNhwoROa01NTZ32d3R0ZObMmVmwYEFGjRrVnZEAAAAAqlK3ok57e3v2\n7t2b2traTuu1tbVpbW3t8pzW1tb/df91112XQw45JJdcckl3xgEAAACoWhX/nFNLS0uWLl2aLVu2\nVHoUAAAAgGJ0K+oMGjQovXv3TltbW6f1tra21NXVdXlOXV3dAff/+Mc/zi9/+csMHTp0/+t79+7N\nvHnz8g//8A959tlnf+s8l112WQYOHNhpbfr06Zk+fXp3LgsAAADgfbdq1aqsWrWq09quXbve8/nd\nijp9+/ZNQ0NDmpubM2nSpCRvfx9Oc3Nz5syZ0+U5jY2N73p9/fr1aWxsTJLMnDkzZ511VqdzPv3p\nT2fmzJn50pe+dMB5lixZkrFjx3bnEgAAAAB6hK4eTNm8eXMaGhre0/nd/vjVvHnzMmvWrDQ0NGTc\nuHFZsmRJdu/enVmzZiV5O9Ice+yxueaaa5Ikc+fOzRlnnJEbbrgh55xzTlatWpWWlpbceuutSZIj\njzwyRx55ZKd/Rt++fVNXV5fhw4d3dzwAAACAqtDtqDNt2rS0t7dn0aJFaWtry5gxY7Ju3boMHjw4\nSbJz585OP5K8sbExK1euzMKFC7Nw4cIMHz48a9asyejRo3/rP6OmpuYgLgUAAACgehzUFyXPnj07\ns2fP7vK1Bx544F1rU6dOzdSpU9/z+x/oe3QAAAAA6OaPNAcAAACgZxB1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQB\nAAAAKJCoAwAAAFAgUQcAAACgQH0qPQAAAABUi+eeey7t7e2VHqNL27f3SzIq27dvT/J6pcd5l0GD\nBuUjH/lIpcfoUUQdAAAAeB8899xzGTViRHa/8UalR/ktTkyyOeef/+dJtlR6mHfpf9hh2b5jh7Dz\nG0QdAAAAeB+0t7dn9xtv5PYkoyo9TBe2Jzk/6ZHzbU9y/htvpL29XdT5DaIOAAAAvI9GJRlb6SEO\noKfPx//wRckAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFE\nHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAgUQcAAACgQKIOAAAAQIFE\nHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKNBBRZ2bbropw4YNS79+/TJ+/Phs2rTp\ngPvvuuuujBo1Kv369Ut9fX3Wrl27/7W33norf/M3f5MTTjghhx9+eIYMGZILLrggL7300sGMBgAA\nAFAVuh11Vq9enfnz52fx4sXZsmVL6uvr09TUlPb29i73b9iwITNmzMiFF16YrVu3ZvLkyZkyZUqe\neuqpJMnu3buzdevWfP3rX8+WLVty9913Z8eOHZk8efLvdmUAAAAAH2DdjjpLlizJRRddlJkzZ2bk\nyJFZvnx5+vfvnxUrVnS5f+nSpZk4cWLmzZuXESNG5KqrrsrYsWOzbNmyJMkRRxyRdevWZerUqRk+\nfHjGjRuXZcuWpaWlJTt37vzdrg4AAADgA6pbUWfPnj1paWnJmWeeuX+tpqYmEyZMyMaNG7s8Z+PG\njZkwYUKntaampt+6P0leeeWV1NTU5EMf+lB3xgMAAACoGt2KOu3t7dm7d29qa2s7rdfW1qa1tbXL\nc1pbW7u1/80338zll1+eGTNm5PDDD+/OeAAAAABVo0f99Ku33norX/jCF1JTU5Obb7650uMAAAAA\n9Fh9urN50KBB6d27d9ra2jqtt7W1pa6urstz6urq3tP+d4LO888/nwceeOA9PaVz2WWXZeDAgZ3W\npk+fnunTp7+XywEAAAComFWrVmXVqlWd1nbt2vWez+9W1Onbt28aGhrS3NycSZMmJUk6OjrS3Nyc\nOXPmdHlOY2Pju15fv359Ghsb9//9O0Hn2WefzYMPPpgjjzzyPc2zZMmSjB07tjuXAAAAANAjdPVg\nyubNm9PQ0PCezu9W1EmSefPmZdasWWloaMi4ceOyZMmS7N69O7NmzUqSzJw5M8cee2yuueaaJMnc\nuXNzxhln5IYbbsg555yTVatWpaWlJbfeemuSt4PO1KlTs3Xr1tx7773Zs2fP/id7jjrqqPTt27e7\nI8IJN2oAABECSURBVAIAAAB84HU76kybNi3t7e1ZtGhR2traMmbMmKxbty6DBw9OkuzcuTN9+vzP\n2zY2NmblypVZuHBhFi5cmOHDh2fNmjUZPXp0kuSFF17IvffemyQZM2ZMkref/qmpqcmDDz6YT3zi\nE7/zRQIAAAB80HQ76iTJ7NmzM3v27C5fe+CBB961NnXq1EydOrXL/ccdd1z27t17MGMAAAAAVK0e\n9dOvAAAAAHhvRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAA\noECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAA\noECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAA\noECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAA\noECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAA\noECiDgAAAECBRB0AAACAAok6AAAAAAUSdQAAAAAKJOoAAAAAFEjUAQAAACiQqAMAAABQIFEHAAAA\noECiDgAAAECB+lR6AAAAAKAyVuW8rMr0JMkbOTR/kh25PNfmsLyZJJmeVZmeOys5Igcg6gAAAECV\nmp47RZuC+fgVAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAg\nUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAg\nUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAg\nUQcAAACgQKIOAAAAQIFEHQAAAIACiToAAAAABRJ1AAAAAAok6gAAAAAUSNQBAAAAKJCoAwAAAFAg\nUQcAAACgQKIOAAAAQIFEHQAAAIACiTr8zlZVegD2cy96Bveh53Aveg73omdwH3oO96LncC96Bveh\n53AvynJQUeemm27KsGHD0q9fv4wfPz6bNm064P677roro0aNSr9+/VJfX5+1a9e+a8+iRYtyzDHH\npH///jnrrLPyzDPPHMxoVIBf9D2He9EzuA89h3vRc7gXPYP70HO4Fz2He9EzuA89h3tRlm5HndWr\nV2f+/PlZvHhxtmzZkvr6+jQ1NaW9vb3L/Rs2bMiMGTNy4YUXZuvWrZk8eXKmTJmSp556av+e66+/\nPsuWLct3vvOdPP744xkwYECampry61//+uCvDAAAAOADrNtRZ8mSJbnooosyc+bMjBw5MsuXL0//\n/v2zYsWKLvcvXbo0EydOzLx58zJixIhcddVVGTt2bJYtW7Z/z4033pgrr7wyn/3sZ3P88cfne9/7\nXl588cXcc889B39lAAAAAB9g3Yo6e/bsSUtLS84888z9azU1NZkwYUI2btzY5TkbN27MhAkTOq01\nNTXt3//ss8+mtbW103seccQROfnkk3/rewIAAABUuz7d2dze3p69e/emtra203ptbW127NjR5Tmt\nra1d7m9tbU2StLW1paam5oB7/m+vv/56kmT79u3dGb9Y71znj5L0xCvemeSOSg/xW/w//99ffx//\nrvT0+5D03Hvx+7wPv/k+PfVe9NT7kLgXPUW13YfEvegpeup9SPye3VNU26+JxL3oKXrqfUj8/1NP\n8fv+NdGTvXON77SPA+rohhdffLGjpqam4yc/+Umn9QULFnSMHz++y3MOOeSQjjvvvLPT2s0339xR\nV1fX0dHR0bFhw4aOXr16dbS2tnbaM23atI7zzjuvy/e8/fbbO5I4HA6Hw+FwOBwOh8PhcHwgj9tv\nv/1/7TTdelJn0KBB6d27d9ra2jqtt7W1pa6urstz6urqDri/rq4uHR0daWtr6/S0TltbW0488cQu\n37OpqSm33357PvrRj6Zfv37duQQAAACAHuv111/PL37xizQ1Nf2ve7sVdfr27ZuGhoY0Nzdn0qRJ\nSZKOjo40Nzdnzpw5XZ7T2Nj4rtfXr1+fxsbGJMmwYcNSV1eX5ubmnHDCCUmSV199NY899lguvvji\nLt9z0KBB+fM///PujA4AAABQhFNPPfU97ev9jW984xvdeeMjjjgiixYtytChQ3PooYfmiiuuyLZt\n23LbbbdlwIABmTlzZjZt2rT/i4+HDBmSK664IgMGDMhRRx2VZcuW5a677so//dM/ZfDgwUmSvXv3\n5tprr83o0aPz61//OnPmzMmbb76ZpUuXpnfv3t27cgAAAIAq0K0ndZJk2rRpaW9vz6JFi9LW1pYx\nY8Zk3bp1+wPNzp0706fP/7xtY2NjVq5cmYULF2bhwoUZPnx41qxZk9GjR+/fs2DBguzevTsXXXRR\nXnnllZx++ulZu3ZtDjnkkN/DJQIAAAB88NR0dHR0VHoIAAAAALqnV6UHAAAAAKD7RB1+Z9ddd116\n9eqVefPmVXqUqvTiiy/mi1/8YgYNGpT+/funvr4+mzdvrvRYVWfx4sXp1atXp+M3P2bK++umm27K\nsGHD0q9fv4wfPz6bNm2q9EhV59prr824ceNyxBFHpLa2Np/73Ofys5/9rNJjVaXly5envr4+AwcO\nzMCBA3PKKafk3//93ys9VtXZt29frrzyynzsYx9L//7988d//Mf55je/WemxqtZrr72WSy+9NB/9\n6EfTv3//nHbaaXniiScqPVbVGTZs2Lv+/NSrV6985StfqfRoVeeRRx7JpEmTMmTIkPTq1Ss/+MEP\nKj0S75Gow+9k06ZN+c53vpP6+vpKj1KVXnnllZx66qk59NBDs27dumzfvj1///d/nyOPPLLSo1Wl\n448/Pm1tbWltbU1ra2t+/OMfV3qkqrR69erMnz8/ixcvzpYtW1JfX5+mpqa0t7dXerSq8sgjj+Qr\nX/lKHnvssdx///3Zs2dPPv3pT+f111+v9GhVZ+jQobn++uuzefPmtLS05FOf+lQmT56c7du3V3q0\nqnLdddflH//xH3PzzTfnP/7jP/Ltb3873/72t7Ns2bJKj1aVvvzlL6e5uTl33HFHnnzyyZx11lmZ\nMGFCXnrppUqPVlWeeOKJ/X9uam1tzfr161NTU5Np06ZVerSq86tf/SpjxozJzTffnJqamkqPQzf4\nTh0O2muvvZaGhobccsstufrqq3PiiSfmhhtuqPRYVeXyyy/Pxo0b89BDD1V6lKq3ePHirFmzxlNS\nPcD48eNz8skn58Ybb0ySdHR0ZOjQoZkzZ04WLFhQ4emqV3t7e/7wD/8wDz/8cE477bRKj1P1jj76\n6Pzd3/1dvvSlL1V6lKpx7rnnpq6uLrfeeuv+tc9//vPp379/vve971Vwsurzxhtv5A/+4A/ywx/+\nMJ/5zGf2r5900kk5++yzc9VVV1Vwuup26aWX5kc/+pEnOyusV69eueeeezJp0qRKj8J74EkdDtrF\nF1+cc889N5/61KcqPUrV+uEPf5iTTjop06ZNS21tbcaOHZvbbrut0mNVraeffjpDhgzJH/3RH+X8\n88/P888/X+mRqs6ePXvS0tKSM888c/9aTU1NJkyYkI0bN1ZwMl555ZXU1NTkqKOOqvQoVW3fvn25\n8847s3v37jQ2NlZ6nKpyyimnpLm5OU8//XSSZNu2bXn00Udz9tlnV3iy6vPWW29l7969OfTQQzut\n9+vXz1O2FbRnz57ccccd+fKXv1zpUaAo3f6R5pAkd955Z7Zu3eqzxxX27LPP5pZbbsn8+fOzcOHC\nPP7445kzZ04OPfTQfPGLX6z0eFVl/Pjx+e53v5sRI0bkpZdeyje+8Y184hOfyJNPPpkBAwZUeryq\n0d7enr1796a2trbTem1tbXbs2FGhqejo6Mill16a0047zXdNVciTTz6ZxsbG/U8o3H333Rk5cmSl\nx6oql19+eV599dWMHDkyvXv3zr59+/Ktb30r5513XqVHqzqHH354Ghsbc/XVV2fkyJGpra3NypUr\ns3HjxgwfPrzS41Wtu+++O7t27coFF1xQ6VGgKKIO3bZz585ceumluf/++9O3b99Kj1PV9u3bl3Hj\nxuXqq69OktTX1+fJJ5/M8uXLRZ33WVNT0/7/ffzxx2fcuHE57rjj8i//8i8+3kDVmz17dp566qk8\n+uijlR6lao0cOTLbtm3Lrl278q//+q+ZOXNmHn74YWHnfbR69eqsXLkyd955Z0aPHp2tW7dm7ty5\nOeaYY/yeXQG33357/uIv/iJDhgxJnz59Mnbs2MyYMSMtLS2VHq1qrVixIhMnTkxdXV2lR4GiiDp0\nW0tLS375y19m7Nixeecrmfbu3ZuHH344y5Yty5tvvunLtd4nH/7whzNq1KhOa6NGjcr3v//9Ck3E\nOwYOHJg/+ZM/yTPPPFPpUarKoEGD0rt377S1tXVab2tr84fECrnkkkvyox/9KI888kg+/OEPV3qc\nqtWnT5987GMfS5KceOKJefzxx3PjjTfmlltuqfBk1WPBggX52te+li984QtJko9//OP5xS9+kWuv\nvVbUqYBhw4blwQcfzOuvv55XX301tbW1Oe+88/b/OuH99dxzz+X+++/PPffcU+lRoDi+U4dumzBh\nQn76059m69at2bZtW7Zt25aTTjop559/frZt2ybovI9OPfXUd32kZMeOHTnuuOMqNBHveO211/Lz\nn//cf8S+z/r27ZuGhoY0NzfvX+vo6Ehzc3NOOeWUCk5WnS655JKsWbMmDz74YD7ykY9Uehx+w759\n+/Lmm29Weoyqsnv37nf9GalXr17Zt29fhSYieft7dGpra/Nf//VfWbduXaZMmVLpkarSihUrUltb\n6zum4CB4UoduGzBgwLu+E2HAgAE5+uij3/XUCP//uuyyy3Lqqafm2muvzbRp0/LYY4/ltttu6/ST\nNXh/fPWrX825556b4447Li+88EK+/vWvp0+fPpk+fXqlR6s68+bNy6xZs9LQ0JBx48ZlyZIl2b17\nd2bNmlXp0arK7Nmzs2rVqvzgBz/IgAED9j89NXDgwBx22GEVnq66fO1rX8vEiRPzkY98JP/93/+d\nO+64Iw899FDuu+++So9WVc4999x861vfytChQ/Pxj388mzdvzpIlS/KXf/mXlR6tKt13333p6OjI\niBEj8vTTT2fBggUZPXq03ysqoKOjI9/97ncza9as9OrlmYNK+dWvfpVnnnlm/ycxnn322Wzbti1H\nHXVUhg4dWuHpOBBRh98LT+dUxkknnZS77747l19+ea6++uoMGzYsN954oy9drICdO3dmxowZ+c//\n/M8MHjw4p512Wn7yk5/k6KOPrvRoVWfatGlpb2/PokWL0tbWljFjxmTdunUZPHhwpUerKsuXL09N\nTU3OOOOMTuv//M//nJkzZ1ZmqCr18ssv54ILLshLL72UgQMH5oQTTsh9993np1e+z5YtW5Yrr7wy\nF198cV5++eUcc8wx+eu//utceeWVlR6tKu3atSt/+7d/mxdeeCFHHXVUPv/5z+eb3/xmevfuXenR\nqs7999+f559/3ncQVtgTTzyRT37yk6mpqUlNTU3mz5+fJLnggguyYsWKCk/HgdR0vJPiAAAAACiG\n59sAAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUCBRBwAAAKBAog4AAABAgUQdAAAAgAKJOgAAAAAF\nEnUAAAAACiTqAAAAABRI1AEAAAAokKgDAAAAUKD/F4nN4pGa3qczAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa30a7f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.ensemble import ExtraTreesClassifier\n",
    "\n",
    "# Build a classification task using 3 informative features\n",
    "\n",
    "# Build a forest and compute the feature importances\n",
    "forest = ExtraTreesClassifier(n_estimators=250,\n",
    "                              random_state=0)\n",
    "\n",
    "forest.fit(X, y)\n",
    "importances = forest.feature_importances_\n",
    "std = np.std([tree.feature_importances_ for tree in forest.estimators_],\n",
    "             axis=0)\n",
    "indices = np.argsort(importances)[::-1]\n",
    "\n",
    "# Print the feature ranking\n",
    "print(\"Feature ranking:\")\n",
    "\n",
    "for f in range(X.shape[1]):\n",
    "    print(\"%d. feature %d - %s (%f) \" % (f + 1, indices[f], features[indices[f]], importances[indices[f]]))\n",
    "\n",
    "# Plot the feature importances of the forest\n",
    "plt.figure(num=None, figsize=(14, 10), dpi=80, facecolor='w', edgecolor='k')\n",
    "plt.title(\"Feature importances\")\n",
    "plt.bar(range(X.shape[1]), importances[indices],\n",
    "       color=\"r\", yerr=std[indices], align=\"center\")\n",
    "plt.xticks(range(X.shape[1]), indices)\n",
    "plt.xlim([-1, X.shape[1]])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.11957832,  0.11892912,  0.11556982,  0.11526371,  0.11147937])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "importances[indices[:5]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. feature 4 - Alkphos (0.119578)\n",
      "2. feature 6 - Sgot (0.118929)\n",
      "3. feature 5 - Sgpt (0.115570)\n",
      "4. feature 0 - Age (0.115264)\n",
      "5. feature 2 - TB (0.111479)\n"
     ]
    }
   ],
   "source": [
    "for f in range(5):\n",
    "    print(\"%d. feature %d - %s (%f)\" % (f + 1, indices[f], features[indices[f]] ,importances[indices[f]]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "best_features = []\n",
    "for i in indices[:5]:\n",
    "    best_features.append(features[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqQAAAINCAYAAADoVW2PAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3XtcVVX+//H3OYByUfGCgYkSminWCHGMER3ThAm1Usti\nonGQ+o3Zl3E0cbpSllZeHjnRMGp00amZJnKcapwshzHsLl9zAM1CKUsrNU6dSptCE2H9/vDrqRN4\nOYgujr6ejwePZO211/7sswTf7avDGGMEAAAAWOK0XQAAAABObwRSAAAAWEUgBQAAgFUEUgAAAFhF\nIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgBoIR999JGcTqf+/Oc/2y4FAAIKgRRA\nszzxxBNyOp1Nft1+++0nbLurVq3SrFmzTtj4x8vhcNgu4bg89NBDeuKJJ2yXAeA0E2y7AACBy+Fw\n6J577tFZZ53l037eeeedsG2++OKLWrx4se66664Tto3miouL0969exUSEmK7lGZbvHixunbtqokT\nJ9ouBcBphEAK4LiMHDlSycnJJ217xpgTMm5tba3Cw8OPe5w2bdq0QDUn3969exUWFma7DACnKU7Z\nAzjhnnzySQ0cOFDh4eHq0qWLsrKytGPHDp8+b7zxhjIzMxUXF6fQ0FD17NlTeXl52rdvn7fPtdde\nq8WLF0uS9/KAoKAgSdIrr7wip9Op1157zWfcpq7rzMnJUfv27fXhhx9q9OjR6tChgyZMmOBdvm7d\nOo0cOVIdO3ZURESEhg8frrVr1x51P4+0rU8++USXXnqp2rdvr9jYWO9+bNq0SWlpaWrXrp3OOuss\nFRcX+4x56NKI119/XZMnT1ZUVJQiIyM1ceJE7d69u1ENixcv1nnnnafQ0FB1795dU6ZM0Z49e3z6\nDB8+XAMGDFBFRYUuvPBCRURE6Pbbb1d8fLzeffdd72fpdDo1YsQISdJXX32l3/3udxowYIDat2+v\nyMhIjR49Wm+//bbP2K+++qqcTqeWL1+u++67Tz169FBYWJjS09P1wQcfNKp33bp1Gj16tDp37qx2\n7dopMTFRhYWFPn2qq6t15ZVXqkuXLgoLC9MFF1yg559/3qfPgQMHNGvWLJ1zzjkKCwtTVFSUhg4d\nqtLS0qNNG4BWgCOkAI7Lnj179MUXX/i0denSxfvn++67TzNnztTVV1+tSZMm6fPPP1dhYaGGDRum\nyspKdejQQZK0fPly7d27V7m5uerSpYveeust/fGPf9TOnTu1bNkySdINN9ygXbt26aWXXtJf//pX\nn6OlDofjmK/fdDgcOnDggDIyMjR06FD9/ve/9x4dXbNmjUaPHq2BAwfq7rvvltPp1J/+9CeNGDFC\nb7zxhgYOHOjX5+NwONTQ0KBRo0Zp2LBhuv/++/XXv/5Vv/3tbxUREaH8/HxNmDBB48ePV1FRkSZO\nnKjBgwcrLi7OZ5wpU6aoU6dOmjVrlqqrq7V48WJ9/PHHevnll7197r77bs2ePVsXX3yxcnNzvf3+\n85//6M033/SGd4fDIY/Ho9GjR+vqq69Wdna2oqOjddFFF2nKlClq37697rjjDhljFB0dLUn68MMP\n9c9//lNXXXWV4uPj5Xa79fDDD2v48OGqqqpSTEyMT73z5s1TUFCQbrrpJu3Zs0fz58/XhAkTVFZW\n5u2zevVqXXbZZTrzzDN14403KiYmRps3b9YLL7ygqVOnSpLeffdd/exnP1NsbKxuu+02RURE6G9/\n+5vGjRunZ599VmPHjpUk3XXXXZo3b56uv/56XXDBBfr666/1n//8RxUVFUpLS/NrzgBYYACgGR5/\n/HHjcDgafTmdTm+fjz76yAQHB5t58+b5rPvuu++akJAQM3fuXG/bvn37Gm1j3rx5JigoyHzyySfe\ntilTpvhs45BXXnnFOJ1O8+qrr/q0b9++3TgcDvPEE09423JycozT6TT5+fmNxjnnnHPM6NGjfdr2\n7dtnevXqZTIyMg73cRx1W/Pnz/e27d6924SHh5ugoCCzfPlyb3t1dbVxOBxm1qxZ3rZDn3NKSoo5\ncOCAt/3+++83TqfTPP/888YYYz7//HPTtm1bM2rUKJ+aFi1aZJxOp3n88ce9bcOHDzdOp9M8+uij\njfbhvPPOMxdddFGj9v379zdq++ijj0xoaKi59957vW2vvPKKcTgc5txzz/Wpt7Cw0DidTvPuu+8a\nY4ypr6838fHxplevXubrr79uNPYhaWlpJikpydTV1fm0DxkyxPTt29f7fVJSkrnssssOOw6A1o1T\n9gCazeFw6KGHHtJLL73k/Vq9erV3+TPPPCNjjK666ip98cUX3q8zzjhDffr08Tm617ZtW++fa2tr\n9cUXXyg1NVUNDQ2qrKw8IfXfcMMNPt9v2LBB77//vrKysnzq/e9//6u0tLRGlwP44//9v//n/XNk\nZKT69u2riIgIXXnlld72c845Rx07dtSHH37YaP3rr7/ee4RTkv7nf/5HQUFBevHFFyUdPNpYV1en\nG2+80We9SZMmqX379nrhhRd82tu2baucnJxjrv+HN2o1NDToyy+/VHh4uPr27auKiopG/a+77jqf\neocOHSpjjHffKisrtX37dt14441q3759k9v86quv9PLLL+uqq67yHok/9HXxxRfr/fff16effipJ\n6tixo959911t3br1mPcJQOvBKXsAx+WCCy447E1NW7duVUNDg84+++xGyxwOh88NQJ988onuvPNO\nPf/88/rqq698+v34GsiWEBwcrNjYWJ+2999/X5KUnZ3d5DpOp1N79uxRZGSkX9sKDQ31uYxBOhhK\nf7z9Q+0/3H/p4Gfw488wIiJC3bp10/bt2yVJH3/8saSDofaHQkJC1KtXL3300Uc+7d27d1dw8LH/\nE2CM0YMPPqiHHnpI27ZtU319vbe2qKioRv179Ojh832nTp0kybtvH3zwgRwOh84999zDbnPr1q0y\nxujOO+/UHXfc0Wi5w+HQZ599pm7dumn27NkaN26czjnnHJ133nkaOXKkfvWrX+knP/nJMe8jAHsI\npABOmIaGBjmdTv3rX/+S09n4hEy7du28/dLT07V7927ddttt3qOHO3fu1MSJE9XQ0HDUbR3u+tFD\nwenHfnhE9of1StLvf/97JSYmNrneoZr98cMjhcfSbk7QkwR+yN876g9dC/zrX/9a9957rzp37iyn\n06lp06Y1OT8tsW+Hxv3d736njIyMJvscCupDhw7VBx98oBUrVujf//63lixZooKCAj388MO67rrr\njnmbAOwgkAI4YXr37i1jjM4666wmj5IesmnTJr3//vv6y1/+ol/+8pfe9pdeeqlR38MFz06dOskY\n0+jO80NHEI+1Xklq37699+7y1sAYo/fff1/Dhg3ztn377bf69NNPdckll0iS9yao6upqn+fC1tXV\nadu2bfr5z39+TNs63Of7zDPPaMSIEXrkkUd82nfv3q2uXbv6szuSvv+78c477xz2s+7Vq5ekg0d5\nj2U+OnbsqIkTJ2rixImqra3V0KFDdffddxNIgQDANaQATpgrrrhCTqfzsG9W+vLLLyV9fzTtx0fa\nHnzwwUYBKSIiQpL09ddf+7THxcUpKCio0XWeixcvPua7710ul3r37q0FCxbo22+/bbTc4/Ec0zgn\nwiOPPKIDBw54v1+8eLHq6+s1evRoSVJ6erpCQkIaPTLpscce09dff61LL730mLYTERHR5OOkgoKC\nGh3dXL58uXbu3OnvrkiSkpOTFR8frwcffPCwl2R07dpVw4cP18MPP6yamppGy384H4f+Lh0SHh6u\ns88+W999912z6gNwcnGEFECzHe30a69evXTvvffq9ttv17Zt2zRu3Djv8z//8Y9/aPLkycrLy1O/\nfv3Uu3dvzZgxQzt27FCHDh30zDPPNBmMXC6XjDH67W9/q4yMDAUFBekXv/iFOnTooKuuusobyHr3\n7q2VK1fq888/P+b9cTgceuyxxzR69Gide+65uvbaa9W9e3ft3LlTL7/8siIjI7VixQr/PqQWsn//\nfqWlpSkzM1NbtmzRQw89pKFDh3qDZlRUlG677TbNnj1bI0eO1JgxY7z9UlJSfI48H4nL5VJRUZHu\nu+8+nX322TrjjDN00UUX6dJLL9U999yj6667ToMHD9amTZv017/+1XtU2V+HbogbM2aMkpKSdO21\n16pbt27asmWLqqqqtGrVKknSokWLNHToUP3kJz/RpEmT1KtXL7ndbpWVlWnnzp3eG9769++v4cOH\ny+VyqXPnzlq/fr3+/ve/ex8fBaCVs3JvP4CA9/jjjxun02nKy8uP2ve5554zF154oWnfvr1p3769\n6d+/v5k6dap5//33vX22bNliLr74YtOhQwdzxhlnmBtuuMFs2rTJOJ1On8co1dfXm2nTppno6GgT\nFBTk8wgoj8djrrrqKtOuXTvTpUsXk5uba6qqqhqNkZOTYzp06HDYejdu3GiuvPJK07VrVxMWFmbi\n4+PN1VdfbV5++eUj7uf27duPeVvDhw83AwYMaNQeHx9vxowZ4/3+0Of8+uuvmxtuuMF06dLFdOjQ\nwWRnZ5uvvvqq0fqLFy82/fv3N23btjXdunUzU6ZMMXv27DmmbRtjjNvtNpdddpmJjIw0TqfT+wio\n7777ztx0002me/fuJiIiwlx44YVm3bp15qKLLjIjRozwrn/o8VvPPPPMUT8bY4xZu3atycjIMJGR\nkaZ9+/YmKSnJLF682KfPtm3bTE5OjjnzzDNN27ZtTY8ePcyYMWPMs88+6+0zZ84cM2jQINO5c2cT\nERFh+vfvb+bNm+fz6CkArZfDmJNw9TwAoFmeeOIJXXfddVq/fv1JfUUrAJxMzbqGdNGiRYqPj1dY\nWJgGDRqk9evXH7ZvVVWVrrzySsXHx8vpdDa6vumQXbt26Ve/+pWioqIUHh6uxMTEJp9tBwAAgFOL\n34F02bJlmjFjhmbNmqXKykolJiYqIyPjsBf719bWqnfv3po/f766devWZJ/du3dryJAhatu2rUpK\nSrR582b9/ve/9z63DgBOZ5zIAnCq8/umpoKCAk2ePNn74OiioiK98MILWrp0qW6++eZG/QcOHOh9\n9/Mtt9zS5Jjz5s1Tz5499dhjj3nbfvweZwA4XR3rUwIAIFD5dYS0rq5O5eXlSktL87Y5HA6lp6er\nrKys2UU8//zzGjhwoDIzMxUdHa3k5GSfcAoAp6uJEyeqvr6e60cBnNL8OkLq8XhUX1+v6Ohon/bo\n6GhVV1c3u4gPP/xQDz30kGbMmKH8/Hy99dZbmjp1qtq2batf/epXTdZRUlKis846y++3jQAAAODE\n27t3r7Zv366MjIwmXzH8Q63iOaQNDQ1KSUnRPffcI0lKTEzUO++8o6KioiYDaUlJiSZMmHCyywQA\nAICfnnzyyaM+C9mvQBoVFaWgoCC53W6fdrfbrZiYGP8r/D/dunVTQkKCT1tCQoKeffbZJvsfei3e\nk08+2Wg9tA7Tp09XQUGB7TLQBOam9WJuWjfmp/ViblqnzZs3a8KECT6vMz4cvwJpSEiIXC6XSktL\nNWbMGEkH7/4sLS09rrdhDBkypNEp/+rq6sPe2HToNH1CQgLXVbVSkZGRzE0rxdy0XsxN68b8tF7M\nTet2LJdX+n3KPi8vTzk5OXK5XEpJSVFBQYFqa2uVk5MjScrOzlZsbKzmzJkj6eCNUFVVVTLGaP/+\n/dq5c6c2btyodu3aeV85N336dA0ZMkRz585VZmam1q1bp8cee0yPPvqov+UBAAAgwPgdSDMzM+Xx\neDRz5ky53W4lJSWppKREXbt2lSTt2LFDwcHfD7tr1y6df/753seWLFiwQAsWLNCwYcO0Zs0aSQcf\nDfXcc8/p1ltv1T333KP4+Hj94Q9/0NVXX90S+wgAAIBWrFk3NeXm5io3N7fJZYdC5iFxcXFqaGg4\n6pijR4/W6NGjm1MOAAAAAlizXh0KHE1WVpbtEnAYzE3rxdy0bsxP68XcBD6HCcB30lVUVMjlcqm8\nvJyLmAEAAFohf/IaR0gBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABg\nFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAA\nAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgB\nAABgFYEUAAAAVhFIAQAAYFWw7QJwaiguPvglSfv2SR99JMXFSaGhB9uysg5+AQAA/BiBFC3ih4Gz\nokJyuQ4G1ORku3UBAIDWj1P2AAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAq3js\nE3CK4xmxAIDWjkAKnOJ4RiwAoLXjlD0AAACsIpACAADAKgIpAAAArOIaUgAAEFC4WfPUQyAFAAAB\nhZs1Tz2csgcAAIBVHCEFAEs47QgABxFIAcASTjsCwEHNOmW/aNEixcfHKywsTIMGDdL69esP27eq\nqkpXXnml4uPj5XQ6VVhYeMSx582bJ6fTqby8vOaUBgAAgADjdyBdtmyZZsyYoVmzZqmyslKJiYnK\nyMiQx+Npsn9tba169+6t+fPnq1u3bkcce/369XrkkUeUmJjob1kAAAAIUH4H0oKCAk2ePFnZ2dnq\n16+fioqKFB4erqVLlzbZf+DAgZo/f74yMzPVpk2bw477zTffaMKECXrsscfUsWNHf8sCAABAgPIr\nkNbV1am8vFxpaWneNofDofT0dJWVlR1XIb/5zW902WWXacSIEcc1DgAAAAKLXzc1eTwe1dfXKzo6\n2qc9Ojpa1dXVzS7i6aef1oYNG/Sf//yn2WMAAAAgMFm/y37Hjh268cYb9dJLLykkJMR2OQAAADjJ\n/AqkUVFRCgoKktvt9ml3u92KiYlpVgHl5eX6/PPPlZycLGOMJKm+vl6vvfaaFi5cqO+++04Oh6PJ\ndadPn67IyEiftqysLGXx4D4AAICTpri4WMWHHqz8f/bs2XPM6/sVSENCQuRyuVRaWqoxY8ZIkowx\nKi0t1dSpU/0Zyis9PV2bNm3yacvJyVFCQoJuvfXWw4ZR6eANVsk8sA8AAMCqpg4IVlRUyOVyHdP6\nfp+yz8vLU05Ojlwul1JSUlRQUKDa2lrl5ORIkrKzsxUbG6s5c+ZIOngjVFVVlYwx2r9/v3bu3KmN\nGzeqXbt26t27tyIiItS/f3+fbURERKhLly5KSEjwtzwAAAAEGL8DaWZmpjwej2bOnCm3262kpCSV\nlJSoa9eukg5eExoc/P2wu3bt0vnnn+890rlgwQItWLBAw4YN05o1a5rcxpGOigIAAODU0qybmnJz\nc5Wbm9vksh+HzLi4ODU0NPg1/uGCKgAAJ0tx8cEvSdq3T/roIykuTgoNPdj2w1e/Ajg+1u+yBwCg\nNfph4KyokFyugwGVWxeAltesd9kDAAAALYVACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAA\nqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAA\nALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAK\nAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsI\npAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACw\nikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrmhVIFy1apPj4eIWFhWnQoEFav379YftW\nVVXpyiuvVHx8vJxOpwoLCxv1mTt3rlJSUtShQwdFR0fr8ssv13vvvdec0gAAABBg/A6ky5Yt04wZ\nMzRr1ixVVlYqMTFRGRkZ8ng8Tfavra1V7969NX/+fHXr1q3JPq+//rp++9vfat26dXrppZdUV1en\niy++WHv37vW3PAAAAASYYH9XKCgo0OTJk5WdnS1JKioq0gsvvKClS5fq5ptvbtR/4MCBGjhwoCTp\nlltuaXLMF1980ef7xx9/XGeccYbKy8v1s5/9zN8SAQAAEED8OkJaV1en8vJypaWledscDofS09NV\nVlbWYkXt3r1bDodDnTt3brExAQAA0Dr5FUg9Ho/q6+sVHR3t0x4dHa2ampoWKcgYoxtvvFE/+9nP\n1L9//xYZEwAAAK2X36fsT7Tc3FxVVVXpzTfftF0KAAAATgK/AmlUVJSCgoLkdrt92t1ut2JiYo67\nmClTpujFF1/U66+/ftgboH5o+vTpioyM9GnLyspSVlbWcdfSWn388ceHvYGstdi8OUxSgjZv3iyp\n9d6YFhUVpZ49e9ouAwCAgFdcXKzi4mKftj179hzz+n4F0pCQELlcLpWWlmrMmDGSDp5iLy0t1dSp\nU/0ZqpEpU6ZoxYoVevXVV485JBQUFCg5Ofm4thtIPv74YyX07avafftsl3IU50uq0IQJv5RUabuY\nwwoPDdXm6mpCKQAAx6mpA4IVFRVyuVzHtL7fp+zz8vKUk5Mjl8ullJQUFRQUqLa2Vjk5OZKk7Oxs\nxcbGas6cOZIO3ghVVVUlY4z279+vnTt3auPGjWrXrp169+4t6eBp+uLiYv3zn/9URESE9whsZGSk\nQkND/S3xlOXxeFS7b5+elJRgu5gj2CxpgtSq69wsacK+ffJ4PC0aSFv7EWyOXgMAWiO/A2lmZqY8\nHo9mzpwpt9utpKQklZSUqGvXrpKkHTt2KDj4+2F37dql888/Xw6HQ5K0YMECLViwQMOGDdOaNWsk\nHXx0lMPh0PDhw3229ac//cn7eCl8L0FSIBwXDpQ6W0pgHMHm6DUAoPVp1k1Nubm5ys3NbXLZoZB5\nSFxcnBoaGo443tGWA4EgEI5gn85HrwEArVeru8seCHSBcGQ4EGoEAJw+mvUuewAAAKClEEgBAABg\nFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVj30CcFrgLVotp6XfpNXa50YKnPnhLWcIVARSAKc8\n3qLVslryTVqBMTdSoMwPbzlDoCKQAjjl8RatltPSb9IKhLmRAmN+eMsZAhmBFMBpIxDeUBUINZ4I\ngbLfgVInEGi4qQkAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABW8RxS\nAABwWK391a681vXUQCAFAABNCoxXu/Ja11MBgRQAADQpEF7tymtdTw0EUgAAcESB8MrUQKgRh8dN\nTQAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACw\nikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAA\nAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCqWYF00aJFio+P\nV1hYmAYNGqT169cftm9VVZWuvPJKxcfHy+l0qrCw8LjHBAAAwKnD70C6bNkyzZgxQ7NmzVJlZaUS\nExOVkZEhj8fTZP/a2lr17t1b8+fPV7du3VpkTAAAAJw6/A6kBQUFmjx5srKzs9WvXz8VFRUpPDxc\nS5cubbL/wIEDNX/+fGVmZqpNmzYtMiYAAABOHX4F0rq6OpWXlystLc3b5nA4lJ6errKysmYVcCLG\nBAAAQODwK5B6PB7V19crOjrapz06Olo1NTXNKuBEjAkAAIDAwV32AAAAsCrYn85RUVEKCgqS2+32\naXe73YqJiWlWAccz5vTp0xUZGenTlpWVpaysrGbVAgAAAP8VFxeruLjYp23Pnj3HvL5fgTQkJEQu\nl0ulpaUaM2aMJMkYo9LSUk2dOtWfoVpkzIKCAiUnJzdruwAAAGgZTR0QrKiokMvlOqb1/QqkkpSX\nl6ecnBy5XC6lpKSooKBAtbW1ysnJkSRlZ2crNjZWc+bMkXTwpqWqqioZY7R//37t3LlTGzduVLt2\n7dS7d+9jGhMAAACnLr8DaWZmpjwej2bOnCm3262kpCSVlJSoa9eukqQdO3YoOPj7YXft2qXzzz9f\nDodDkrRgwQItWLBAw4YN05o1a45pTAAAAJy6/A6kkpSbm6vc3Nwmlx0KmYfExcWpoaHhuMYEAADA\nqYu77AEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgB\nAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYFWw\n7QIAnFjFulrFypIk7VNbnaNq3aq5CtV3kqQsFStLT9ssEQBwmiOQAqe4LD1N4ASagf+ZA04eAikA\nAE3gf+aAk4drSAEAAGAVR0jRIji1BQAAmotAihbBqS0AANBcnLIHAACAVQRSAAAAWEUgBQAAgFUE\nUgAAAFhFIAUAAIBV3GUPAJbwuDQAOIhACgCW8Lg0ADiIU/YAAACwikAKAAAAqwikAAAAsIpACgAA\nAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQA\nAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCqWYF00aJFio+PV1hYmAYNGqT169cfsf/y5cuV\nkJCgsLAwJSYmatWqVT7Lv/32W02ZMkU9evRQeHi4zj33XD388MPNKQ0AAAABxu9AumzZMs2YMUOz\nZs1SZWWlEhMTlZGRIY/H02T/tWvX6pprrtGkSZO0YcMGjR07VuPGjVNVVZW3z/Tp0/Xvf/9bTz31\nlLZs2aIbb7xRU6ZM0cqVK5u/ZwAAAAgIfgfSgoICTZ48WdnZ2erXr5+KiooUHh6upUuXNtm/sLBQ\no0aNUl4MVO7oAAAgAElEQVRenvr27avZs2crOTlZCxcu9PYpKyvTxIkTNXToUPXs2VOTJk1SYmKi\n3nrrrebvGQAAAAKCX4G0rq5O5eXlSktL87Y5HA6lp6errKysyXXKysqUnp7u05aRkeHTf/Dgwfrn\nP/+pXbt2SZJefvllvf/++8rIyPCnPAAAAASgYH86ezwe1dfXKzo62qc9Ojpa1dXVTa5TU1PTZP+a\nmhrv93/84x91/fXXKzY2VsHBwQoKCtKjjz6qIUOG+FMeAAAAApBfgfREKSws1Lp167Ry5Ur17NlT\nr732mnJzc3XmmWdqxIgRtssDAADACeRXII2KilJQUJDcbrdPu9vtVkxMTJPrxMTEHLH/vn37lJ+f\nrxUrVmjkyJGSpPPOO0+VlZVasGDBEQPp9OnTFRkZ6dOWlZWlrKwsf3YLAAAAx6G4uFjFxcU+bXv2\n7Dnm9f0KpCEhIXK5XCotLdWYMWMkScYYlZaWaurUqU2uk5qa2mj56tWrlZqaKungdal1dXVyOBw+\n6wUFBamhoeGI9RQUFCg5OdmfXQAAAAGuWFerWAcPPu1TW52jat2quQrVd5KkLBUrS0/bLPG009QB\nwYqKCrlcrmNa3+9T9nl5ecrJyZHL5VJKSooKCgpUW1urnJwcSVJ2drZiY2M1Z84cSdK0adM0fPhw\nPfDAA7rkkktUXFys8vJyPfroo5Kk9u3ba9iwYbrpppsUGhqquLg4vfLKK/rzn/+sBx980N/yAADA\nKS5LTxM4TzF+B9LMzEx5PB7NnDlTbrdbSUlJKikpUdeuXSVJO3bsUHDw98OmpqbqqaeeUn5+vvLz\n89WnTx+tWLFC/fv39/ZZtmyZbrvtNk2YMEFffvml4uLiNHfuXF1//fUtsIsAAABozZp1U1Nubq5y\nc3ObXLZmzZpGbePHj9f48eMPO94ZZ5yhJUuWNKcUAAAABDjeZQ8AAACrCKQAAACwikAKAAAAqwik\nAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCK\nQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAA\nqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAA\nALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAK\nAAAAqwikAAAAsIpACgAAAKsIpAAAALCKQAoAAACrCKQAAACwikAKAAAAqwikAAAAsIpACgAAAKsI\npAAAALCqWYF00aJFio+PV1hYmAYNGqT169cfsf/y5cuVkJCgsLAwJSYmatWqVY36bN68WWPHjlXH\njh3Vrl07/fSnP9WOHTuaUx4AAAACiN+BdNmyZZoxY4ZmzZqlyspKJSYmKiMjQx6Pp8n+a9eu1TXX\nXKNJkyZpw4YNGjt2rMaNG6eqqipvnw8++EBDhw5V//799dprr2nTpk268847FRoa2vw9AwAAQEDw\nO5AWFBRo8uTJys7OVr9+/VRUVKTw8HAtXbq0yf6FhYUaNWqU8vLy1LdvX82ePVvJyclauHCht88d\nd9yhSy65RHPnztWAAQMUHx+vSy+9VFFRUc3fMwAAAAQEvwJpXV2dysvLlZaW5m1zOBxKT09XWVlZ\nk+uUlZUpPT3dpy0jI8Pb3xijF154QX369NHIkSMVHR2tQYMGacWKFf7uCwAAAAKQX4HU4/Govr5e\n0dHRPu3R0dGqqalpcp2ampoj9v/ss8/0zTffaP78+Ro9erRWr16tyy+/XFdccYVef/11f8oDAABA\nAAq2XUBDQ4Mkady4cZo6daokacCAAVq7dq2Kioo0dOjQw647ffp0RUZG+rRlZWUpKyvrxBUMAAAA\nH8XFxSouLvZp27NnzzGv71cgjYqKUlBQkNxut0+72+1WTExMk+vExMQcsX9UVJSCg4OVkJDg0ych\nIUFvvvnmEespKChQcnKyP7sAAACAFtbUAcGKigq5XK5jWt+vU/YhISFyuVwqLS31thljVFpaqsGD\nBze5Tmpqqk9/SVq9erVSU1O9Y15wwQWqrq726fPee+8pLi7On/IAAAAQgPw+ZZ+Xl6ecnBy5XC6l\npKSooKBAtbW1ysnJkSRlZ2crNjZWc+bMkSRNmzZNw4cP1wMPPKBLLrlExcXFKi8v16OPPuod86ab\nbtLVV1+toUOH6qKLLtKqVau0cuVKvfrqqy2zlwAAAGi1/A6kmZmZ8ng8mjlzptxut5KSklRSUqKu\nXbtKknbs2KHg4O+HTU1N1VNPPaX8/Hzl5+erT58+WrFihfr37+/tM27cOBUVFWnOnDmaNm2a+vbt\nq2effdZ7FBUAAACnrmbd1JSbm6vc3Nwml61Zs6ZR2/jx4zV+/PgjjpmTk+M9ygoAAIDTB++yBwAA\ngFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIA\nAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUg\nBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBV\nBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAA\nWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUA\nAIBVBFIAAABYRSAFAACAVc0KpIsWLVJ8fLzCwsI0aNAgrV+//oj9ly9froSEBIWFhSkxMVGrVq06\nbN8bbrhBTqdThYWFzSkNAAAAAcbvQLps2TLNmDFDs2bNUmVlpRITE5WRkSGPx9Nk/7Vr1+qaa67R\npEmTtGHDBo0dO1bjxo1TVVVVo77PPfec1q1bp+7du/u/JwAAAAhIfgfSgoICTZ48WdnZ2erXr5+K\niooUHh6upUuXNtm/sLBQo0aNUl5envr27avZs2crOTlZCxcu9Om3c+dOTZs2TU899ZSCg4ObtzcA\nAAAIOH4F0rq6OpWXlystLc3b5nA4lJ6errKysibXKSsrU3p6uk9bRkaGT39jjLKzs3XzzTcrISHB\nn5IAAAAQ4PwKpB6PR/X19YqOjvZpj46OVk1NTZPr1NTUHLX/vHnz1KZNG02ZMsWfcgAAAHAKsH5u\nvLy8XIWFhaqsrPR73enTpysyMtKnLSsrS1lZWS1VHgAAAI6iuLhYxcXFPm179uw55vX9CqRRUVEK\nCgqS2+32aXe73YqJiWlynZiYmCP2f+ONN/T555+rR48e3uX19fXKy8vTgw8+qA8//PCw9RQUFCg5\nOdmfXQAAAEALa+qAYEVFhVwu1zGt79cp+5CQELlcLpWWlnrbjDEqLS3V4MGDm1wnNTXVp78krV69\nWqmpqZKk7Oxsvf3229q4caP368wzz9TNN9+skpISf8oDAABAAPL7lH1eXp5ycnLkcrmUkpKigoIC\n1dbWKicnR9LBgBkbG6s5c+ZIkqZNm6bhw4frgQce0CWXXKLi4mKVl5fr0UcflSR16tRJnTp18tlG\nSEiIYmJi1KdPn+PcPQAAALR2fgfSzMxMeTwezZw5U263W0lJSSopKVHXrl0lSTt27PB5bFNqaqqe\neuop5efnKz8/X3369NGKFSvUv3//w27D4XA0Y1cAAAAQiJp1U1Nubq5yc3ObXLZmzZpGbePHj9f4\n8eOPefwjXTcKAACAUwvvsgcAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUA\nAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRS\nAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhF\nIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACA\nVQRSAAAAWEUgBQAAgFUEUgAAAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFUEUgAA\nAFhFIAUAAIBVBFIAAABYRSAFAACAVQRSAAAAWEUgBQAAgFXNCqSLFi1SfHy8wsLCNGjQIK1fv/6I\n/ZcvX66EhASFhYUpMTFRq1at8i47cOCAbrnlFg0YMEDt2rVT9+7dNXHiRH366afNKQ0AAAABxu9A\numzZMs2YMUOzZs1SZWWlEhMTlZGRIY/H02T/tWvX6pprrtGkSZO0YcMGjR07VuPGjVNVVZUkqba2\nVhs2bNBdd92lyspKPffcc6qurtbYsWOPb88AAAAQEPwOpAUFBZo8ebKys7PVr18/FRUVKTw8XEuX\nLm2yf2FhoUaNGqW8vDz17dtXs2fPVnJyshYuXChJ6tChg0pKSjR+/Hj16dNHKSkpWrhwocrLy7Vj\nx47j2zsAAAC0en4F0rq6OpWXlystLc3b5nA4lJ6errKysibXKSsrU3p6uk9bRkbGYftL0u7du+Vw\nONSxY0d/ygMAAEAA8iuQejwe1dfXKzo62qc9OjpaNTU1Ta5TU1PjV//vvvtOt956q6655hq1a9fO\nn/IAAAAQgIJtF/BDBw4c0FVXXSWHw6HFixcftf/06dMVGRnp05aVlaWsrKwTVSIAAAB+pLi4WMXF\nxT5te/bsOeb1/QqkUVFRCgoKktvt9ml3u92KiYlpcp2YmJhj6n8ojH7yySdas2bNMR0dLSgoUHJy\nsj+7AAAAgBbW1AHBiooKuVyuY1rfr1P2ISEhcrlcKi0t9bYZY1RaWqrBgwc3uU5qaqpPf0lavXq1\nUlNTvd8fCqMffvihSktL1alTJ3/KAgAAQADz+5R9Xl6ecnJy5HK5lJKSooKCAtXW1ionJ0eSlJ2d\nrdjYWM2ZM0eSNG3aNA0fPlwPPPCALrnkEhUXF6u8vFyPPvqopINhdPz48dqwYYNWrlypuro67xHV\nzp07KyQkpIV2FQAAAK2R34E0MzNTHo9HM2fOlNvtVlJSkkpKStS1a1dJ0o4dOxQc/P2wqampeuqp\np5Sfn6/8/Hz16dNHK1asUP/+/SVJO3fu1MqVKyVJSUlJkg4edXU4HHr55Zd14YUXHvdOAgAAoPVq\n1k1Nubm5ys3NbXLZmjVrGrWNHz9e48ePb7J/XFyc6uvrm1MGAAAATgG8yx4AAABWEUgBAABgFYEU\nAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYR\nSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABg\nFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAA\nAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgB\nAABgFYEUAAAAVhFIAQAAYBWBFAAAAFYRSAEAAGAVgRQAAABWEUgBAABgFYEUAAAAVhFIAQAAYBWB\nFCdEse0CcFjMTevF3LRuzE/rxdwEvmYF0kWLFik+Pl5hYWEaNGiQ1q9ff8T+y5cvV0JCgsLCwpSY\nmKhVq1Y16jNz5kydeeaZCg8P189//nNt3bq1OaWhleCXQ+vF3LRezE3rxvy0XsxN4PM7kC5btkwz\nZszQrFmzVFlZqcTERGVkZMjj8TTZf+3atbrmmms0adIkbdiwQWPHjtW4ceNUVVXl7TN//nwtXLhQ\njzzyiN566y1FREQoIyND+/fvb/6eAQAAICD4HUgLCgo0efJkZWdnq1+/fioqKlJ4eLiWLl3aZP/C\nwkKNGjVKeXl56tu3r2bPnq3k5GQtXLjQ2+cPf/iD7rzzTl166aU677zz9Oc//1m7du3SP/7xj+bv\nGQAAAAKCX4G0rq5O5eXlSktL87Y5HA6lp6errKysyXXKysqUnp7u05aRkeHt/+GHH6qmpsZnzA4d\nOuinP/3pYccEAADAqSPYn84ej0f19fWKjo72aY+OjlZ1dXWT69TU1DTZv6amRpLkdrvlcDiO2OfH\n9u7dK0navHmzP+UHvEP7+6Kk1r7nOyT91XYRR7Dt//7bkn+HAmV+mJvWq7XPjdTy8xMocyO1/vnh\nZ6f1OhFzEwgO7e+h3HYkfgXS1mL79u2SpAkTJtgtxJI7bRdwjAJhdk7E36FAmB/mpvUKhLmRWn5+\nAmFupMCYH352Wq/TNbds375dQ4YMOWIfvwJpVFSUgoKC5Ha7fdrdbrdiYmKaXCcmJuaI/WNiYmSM\nkdvt9jlK6na7df755zc5ZkZGhp588kmdddZZCgsL82cXAAAAcBLs3btX27dvV0ZGxlH7+hVIQ0JC\n5HK5VFpaqjFjxkiSjDEqLS3V1KlTm1wnNTW10fLVq1crNTVVkhQfH6+YmBiVlpZqwIABkqSvv/5a\n69at029+85smx4yKitIvf/lLf0oHAADASXa0I6OHBN199913+zNwhw4dNHPmTPXo0UNt27bVHXfc\noY0bN+qxxx5TRESEsrOztX79eu9NSt27d9cdd9yhiIgIde7cWQsXLtTy5cu1ZMkSde3aVZJUX1+v\nuXPnqn///tq/f7+mTp2q7777ToWFhQoKCvJvzwEAABBQ/L6GNDMzUx6PRzNnzpTb7VZSUpJKSkq8\n4XLHjh0KDv5+2NTUVD311FPKz89Xfn6++vTpoxUrVqh///7ePjfffLNqa2s1efJk7d69W0OHDtWq\nVavUpk2bFthFAAAAtGYOY4yxXQQAAABOX7zLHgAAAFYRSOH16quvyul06uuvv5YkPfHEE+rUqdNx\njdkSYwAATl9Op1NBQUFyOp2NvoKCgjR79mx99NFHPu1t27ZVnz59dN9999kuH8eIQHoa+t///V8F\nBwfrsssua7TM4XAc8fvmaIkx0JjH49H//M//KC4uTqGhoerWrZtGjRrVom84u/baa3XFFVe02Hin\ni5MxNz/+H0g0z5F+H6J1qKmp0aeffqqamho9+OCDioyMlNvt9rb/7ne/k3Tw35o1a9aopqZGW7du\n1ezZs3XffffpT3/6k+U9wLEIyAfj4/gsWbJEU6dO1ZIlS1RTU3PYZ8iidbviiit04MAB/eUvf1F8\nfLzcbrdKS0v1xRdf2C7ttHcy5sYYI4fDIW4DOD78Pmz9zjjjDO+fIyMj5XA4vDdS/5AxRp07d/b2\nz8rK0tKlS1VRUaFrr732pNWLZjI4rXzzzTemffv25r333jNZWVlm7ty53mWvvPKKcTqdZs+ePcYY\nYx5//HHTqVMn7/LPPvvMDBw40FxxxRVm//795pVXXjEOh8O88MILZsCAASY0NNQMGjTIvPPOO951\nDo1RUlJiEhISTLt27czIkSNNTU2Nt09DQ4OZNWuWiY2NNW3btjVJSUnmX//6l3f5/v37zW9+8xvT\nrVs3Exoaas466ywzb968E/kxtXq7d+82DofDvPbaa4fts2XLFjNkyBATGhpqzj33XPPSSy8Zh8Nh\nVqxY4e2zadMmM2LECBMWFma6dOlirr/+evPtt98aY4y5++67jcPhME6n0/vfV1999YTvW6BribnZ\nvn27cTgc5umnnzaDBw82oaGh5rzzzvN+/oeW/3Burr322pOyf6eSI/0+NMaYFStWmD59+pjQ0FAz\nYsQI88QTTxiHw+H9HWmMMa+//roZOnSoCQsLMz179jRTp071/gyh5f3436VDDv1MbNy40du2fv16\n07lzZ/Pkk0+ezBLRTATS08ySJUtMSkqKMcaYlStXmrPPPtu77EiB9OOPPzb9+vUz1113nWloaPD2\ndzgc5txzzzWlpaXmnXfeMZdddpnp1auXOXDggHeMNm3amIsvvthUVFSYyspK079/fzNhwgTvdh94\n4AHTsWNH87e//c2899575pZbbjFt2rQxW7duNcYYc//995u4uDjz5ptvmo8//ti8+eab5umnnz7x\nH1YrduDAAdO+fXuTl5dnvvvuu0bL6+vrTd++fc3IkSPNpk2bzJtvvml++tOfGqfT6Q093377rTnz\nzDPNVVddZaqqqszLL79sevXq5Q0233zzjfnFL35hRo8ebT777DPjdrtNXV3dSd3PQNQSc3PoH9ee\nPXua5557zmzZssVMmjTJdOjQwXz55Zemvr7ePPvss8bpdJqtW7cat9ttvv7665O9qwHvSL8Pt23b\nZtq0aWNuueUW895775lly5aZ2NhYn9+RW7duNe3atTOFhYXmgw8+MGVlZcblcpnrrrvOyv6cDo4W\nSCMiIky7du1MmzZtjNPpNDfccIOFKtEcBNLTzJAhQ8wf//hHY8zBfzjPOOMM71GXwwXS6upq07Nn\nTzN9+nSfsQ4F0uXLl3vbvvzySxMeHu5te/zxx43T6TTbtm3z9lm8eLHp1q2b9/vu3bs3OuKZkpJi\npkyZYowxZurUqSY9Pb2FPoFTx7PPPmu6dOliwsLCzJAhQ8ztt99u3n77bWOMMatWrTJt2rQxn332\nmbf/j4/CPfLII6ZLly5m79693j4vvviiCQoK8q6Xk5NjLr/88pO4V6eG452bQ/+43n///d4+Bw4c\nMD169PC2/fjnFf470u/DW265xQwYMMCn/x133OHzmf/6179uFHhef/11ExQU1OT/jOD4HS2QvvDC\nC+aDDz4wW7ZsMX//+99NTEyMufXWWy1UCn9xU9NppLq6Wm+99ZauvvpqSVJQUJAyMzO1ZMmSw65T\nW1uroUOHavz48XrggQcaLXc4HBo0aJD3+06dOqlv377avHmzty08PFxnnXWW9/tu3brps88+kyT9\n97//1a5duzR48GCfcYcMGeIdIycnR5WVlerbt6+mTZum1atX+7/zp6DLL79cu3bt0vPPP69Ro0bp\n1Vdflcvl0hNPPKH33ntPPXr08LnOKiUlxWf9LVu2KDExUaGhod62IUOGqKGhQdXV1SdtP05Fxzs3\nh/zwZysoKEgDBw70+dlC8x3t9+F7772nCy64wGedH8/Txo0b9fjjj6t9+/ber5EjR0qStm3bdhL2\nAj8WGxurXr16qW/fvho/frxuvPFGPfDAA9q/f7/t0nAUBNLTyJIlS1RfX69u3bopJCREISEhKioq\n0jPPPKP//ve/Ta7Ttm1b/fznP9fKlSu1a9euZm03JCTE53t/b8Q4//zztX37dt17773at2+fMjMz\nlZmZ2axaTjVt2rRRWlqa8vPz9cYbb2jixIm66667bJcFMTet3ZF+Hx7rkwu++eYbTZ48WW+//bY2\nbtyojRs36u23/3979/fS1B/GAfzxjHDZNoy8yIUyFXNmRooWQglNRG/0RvYXhOJFiDey8CYI60oi\n9ELMKzEQg3AYIggGwgYiiraJOFeIyhDZZIJiCZ7e34u+HpyNnN/8evzxfsG52PkxnnM+nMMz9nme\n45PFxUXJycn5n8+ADovX0UVRFNnb22NCeg4wIb0kVFWVvr4+efPmjfbg3F+sVqv09/fHPc5gMEhf\nX58UFxfLkydPZG1tLWY7AJmYmNA+R6NRWVxcjHk17J+YzWaxWq3i9Xpj1nu93pjvMJlM4nQ6pbu7\nWwYGBuTjx4+yubmZ6OlfGvn5+bKzsyN2u11WVlYkHA5r2yYnJ3/b98uXL/L9+3dtncfjEYPBIHl5\neSLyK6lSVfV0gr/gjjM2+w7eW6qqyvT0tHZf7L9ameNzfIk8D/Py8mRqairmuMPjVFxcLPPz85KV\nlSXZ2dkxy8FXaNPpACCRSETW19clFArJyMiIdHR0iMPhEJPJpHd4dBSdpwzQKRkcHITRaIxb+OBy\nuVBaWqrNCY1X1LS3twen0wm73a5VyO/vX1hYiLGxMfj9ftTW1sJms2nFL/Hm+7jdbiiKon1++/Yt\nUlNTMTAwgEAgAJfLheTkZHz79g3Ar6Kn/v5+LCwsIBAI4OnTp7BarSd/kc6RjY0NOBwOvH//Hj6f\nD0tLS/jw4QNu3ryJ+vp6qKoKu92O6upq+Hw+eDwelJWVQVEUDA0NAQB2dnZw69YtOJ1OzM3N4fPn\nz8jJyYkpyHj9+jVsNhsCgQAikQiLmhJwEmOzPx/OZrNpRU0NDQ2wWCzY2NgAAIRCIRgMBvT29iIc\nDmN7e1vP0z5XjnoePnjwIG5RU0ZGBhRF0Y7z+Xy4du0anj17htnZWQSDQbjdbm3+O528P80hVRRF\nW65cuYLMzEw0NjYiEonoECkdFxPSS6KmpgY1NTVxt01OTkJRFHR2dv6x7ZOqqqirq0NBQQHC4bBW\nVDE8PIy7d+/CaDSirKwMfr9fOyaRhPTnz594+fIlMjIykJycjKKiIoyOjmrbe3p6UFRUBLPZjNTU\nVFRWVmJ2dvZErst5tbu7i9bWVpSUlOD69eswmUzIz8/Hixcv8OPHDwBAIBDA48ePYTQacefOHQwP\nDyMpKSnm2s7NzaGiogIpKSlIS0tDY2NjTMuacDiMqqoqmM1mtn1K0EmMzcG2Tw8fPvyt7dO+trY2\npKenw2AwsO3TMSTyPPT7/fj06RNu376Nq1evwuFwoLu7G4qixBQsTU1NoaqqChaLBWazGffv3/+t\nfRQRHS0JYFdl+m/Gx8fF4XBINBoVi8Widzh0BK/XK+Xl5fL161fJysrSOxw64PDYLC8vS3Z2tszM\nzMi9e/f0Do/+9erVK3n37p0sLy/rHQrRhcNJLvRX+Hvm7HK73WIymSQ3N1eCwaA0NzfLo0ePmIye\nATcbmX8AAACvSURBVImMDe8t/XV1dUlpaancuHFDPB6PtLe3S1NTk95hEV1ITEjpr/A99WfX1taW\nuFwuWV1dlbS0NKmsrJT29na9wyJJbGx4b+kvGAxKW1ubRKNRyczMlJaWFnn+/LneYRFdSPzLnoiI\niIh0xbZPRERERKQrJqREREREpCsmpERERESkKyakRERERKQrJqREREREpCsmpERERESkKyakRERE\nRKQrJqREREREpKt/AIcc9WUgvZTcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x9d6f898>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the top 5 feature importances of the forest\n",
    "plt.figure(num=None, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')\n",
    "plt.title(\"Feature importances\")\n",
    "plt.bar(range(5), importances[indices][:5], \n",
    "       color=\"r\",  yerr=std[indices][:5], align=\"center\")\n",
    "plt.xticks(range(5), best_features)\n",
    "plt.xlim([-1, 5])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree accuracy and time elapsed caculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DecisionTree\n",
      "Acurracy:  0.675213675214\n",
      "time elapsed:  0.01000070571899414\n"
     ]
    }
   ],
   "source": [
    "t0=time()\n",
    "print (\"DecisionTree\")\n",
    "\n",
    "dt = DecisionTreeClassifier(min_samples_split=20,random_state=99)\n",
    "# dt = DecisionTreeClassifier(min_samples_split=20,max_depth=5,random_state=99)\n",
    "\n",
    "clf_dt=dt.fit(X_train,y_train)\n",
    "\n",
    "print (\"Acurracy: \", clf_dt.score(X_test,y_test))\n",
    "t1=time()\n",
    "print (\"time elapsed: \", t1-t0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cross validation for DT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross result========\n",
      "[ 0.64102564  0.67179487  0.60103627]\n",
      "0.63795226075\n",
      "time elapsed:  0.029001712799072266\n"
     ]
    }
   ],
   "source": [
    "tt0=time()\n",
    "print (\"cross result========\")\n",
    "scores = cross_validation.cross_val_score(dt, X, y, cv=3)\n",
    "print (scores)\n",
    "print (scores.mean())\n",
    "tt1=time()\n",
    "print (\"time elapsed: \", tt1-tt0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Tuning our hyperparameters using GridSearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 27 candidates, totalling 81 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:    7.3s\n",
      "[Parallel(n_jobs=-1)]: Done  81 out of  81 | elapsed:    7.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score: 0.466\n",
      "Best parameters set:\n",
      "\tclf__max_depth: 50\n",
      "\tclf__min_samples_leaf: 2\n",
      "\tclf__min_samples_split: 2\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.73      0.81      0.77       167\n",
      "          1       0.36      0.27      0.31        67\n",
      "\n",
      "avg / total       0.63      0.65      0.64       234\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "\n",
    "pipeline = Pipeline([\n",
    "    ('clf', DecisionTreeClassifier(criterion='entropy'))\n",
    "])\n",
    "\n",
    "parameters = {\n",
    "    'clf__max_depth': (5, 25 , 50),\n",
    "    'clf__min_samples_split': (2, 5, 10),\n",
    "    'clf__min_samples_leaf': (1, 2, 3)\n",
    "}\n",
    "\n",
    "grid_search = GridSearchCV(pipeline, parameters, n_jobs=-1, verbose=1, scoring='f1')\n",
    "grid_search.fit(X_train, y_train)\n",
    "\n",
    "print ('Best score: %0.3f' % grid_search.best_score_)\n",
    "print ('Best parameters set:')\n",
    "\n",
    "best_parameters = grid_search.best_estimator_.get_params()\n",
    "for param_name in sorted(parameters.keys()):\n",
    "    print ('\\t%s: %r' % (param_name, best_parameters[param_name]))\n",
    "\n",
    "predictions = grid_search.predict(X_test)\n",
    "\n",
    "print (classification_report(y_test, predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Random Forest accuracy and time elapsed caculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RandomForest\n",
      "Acurracy:  0.717948717949\n",
      "time elapsed:  0.5140295028686523\n"
     ]
    }
   ],
   "source": [
    "t2=time()\n",
    "print (\"RandomForest\")\n",
    "rf = RandomForestClassifier(n_estimators=100,n_jobs=-1)\n",
    "clf_rf = rf.fit(X_train,y_train)\n",
    "print (\"Acurracy: \", clf_rf.score(X_test,y_test))\n",
    "t3=time()\n",
    "print (\"time elapsed: \", t3-t2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cross validation for RF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross result========\n",
      "[ 0.7025641   0.68717949  0.72020725]\n",
      "0.703316947877\n",
      "time elapsed:  1.4800846576690674\n"
     ]
    }
   ],
   "source": [
    "tt2=time()\n",
    "print (\"cross result========\")\n",
    "scores = cross_validation.cross_val_score(rf, X, y, cv=3)\n",
    "print (scores)\n",
    "print (scores.mean())\n",
    "tt3=time()\n",
    "print (\"time elapsed: \", tt3-tt2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tuning Models using GridSearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 108 candidates, totalling 324 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done  34 tasks      | elapsed:    8.0s\n",
      "[Parallel(n_jobs=-1)]: Done 184 tasks      | elapsed:   14.7s\n",
      "[Parallel(n_jobs=-1)]: Done 324 out of 324 | elapsed:   22.7s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score: 0.734\n",
      "Best parameters set:\n",
      "\tclf__max_depth: 50\n",
      "\tclf__min_samples_leaf: 1\n",
      "\tclf__min_samples_split: 2\n",
      "\tclf__n_estimators: 25\n",
      "Accuracy: 0.67094017094\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.73      0.85      0.79       167\n",
      "          1       0.38      0.22      0.28        67\n",
      "\n",
      "avg / total       0.63      0.67      0.64       234\n",
      "\n"
     ]
    }
   ],
   "source": [
    "\n",
    "pipeline2 = Pipeline([\n",
    "('clf', RandomForestClassifier(criterion='entropy'))\n",
    "])\n",
    "\n",
    "parameters = {\n",
    "    'clf__n_estimators': (5, 25, 50, 100),\n",
    "    'clf__max_depth': (5, 25 , 50),\n",
    "    'clf__min_samples_split': (2, 5, 10),\n",
    "    'clf__min_samples_leaf': (1, 2, 3)\n",
    "}\n",
    "\n",
    "grid_search = GridSearchCV(pipeline2, parameters, n_jobs=-1, verbose=1, scoring='accuracy', cv=3)\n",
    "\n",
    "grid_search.fit(X_train, y_train)\n",
    "\n",
    "print ('Best score: %0.3f' % grid_search.best_score_)\n",
    "\n",
    "print ('Best parameters set:')\n",
    "best_parameters = grid_search.best_estimator_.get_params()\n",
    "\n",
    "for param_name in sorted(parameters.keys()):\n",
    "    print ('\\t%s: %r' % (param_name, best_parameters[param_name]))\n",
    "\n",
    "predictions = grid_search.predict(X_test)\n",
    "print ('Accuracy:', accuracy_score(y_test, predictions))\n",
    "print (classification_report(y_test, predictions))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Naive Bayes accuracy and time elapsed caculation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NaiveBayes\n",
      "Acurracy:  0.713675213675\n",
      "time elapsed:  0.07000398635864258\n"
     ]
    }
   ],
   "source": [
    "t4=time()\n",
    "print (\"NaiveBayes\")\n",
    "nb = BernoulliNB()\n",
    "clf_nb=nb.fit(X_train,y_train)\n",
    "print (\"Acurracy: \", clf_nb.score(X_test,y_test))\n",
    "t5=time()\n",
    "print (\"time elapsed: \", t5-t4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cross-validation for NB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross result========\n",
      "[ 0.71282051  0.71282051  0.71502591]\n",
      "0.713555644126\n",
      "time elapsed:  0.029001712799072266\n"
     ]
    }
   ],
   "source": [
    "tt4=time()\n",
    "print (\"cross result========\")\n",
    "scores = cross_validation.cross_val_score(nb, X,y, cv=3)\n",
    "print (scores)\n",
    "print (scores.mean())\n",
    "tt5=time()\n",
    "print (\"time elapsed: \", tt5-tt4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KNN accuracy and time elapsed caculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNN\n",
      "Acurracy:  0.679487179487\n",
      "time elapsed:  0.011000633239746094\n"
     ]
    }
   ],
   "source": [
    "t6=time()\n",
    "print (\"KNN\")\n",
    "# knn = KNeighborsClassifier(n_neighbors=3)\n",
    "knn = KNeighborsClassifier(n_neighbors=3)\n",
    "clf_knn=knn.fit(X_train, y_train)\n",
    "print (\"Acurracy: \", clf_knn.score(X_test,y_test) )\n",
    "t7=time()\n",
    "print (\"time elapsed: \", t7-t6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cross validation for KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross result========\n",
      "[ 0.66101695  0.64957265  0.63793103  0.57758621  0.72413793]\n",
      "0.650048954228\n",
      "time elapsed:  0.03300189971923828\n"
     ]
    }
   ],
   "source": [
    "tt6=time()\n",
    "print (\"cross result========\")\n",
    "scores = cross_validation.cross_val_score(knn, X,y, cv=5)\n",
    "print (scores)\n",
    "print (scores.mean())\n",
    "tt7=time()\n",
    "print (\"time elapsed: \", tt7-tt6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fine tuning the model using GridSearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 2 candidates, totalling 6 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Done   6 out of   6 | elapsed:    4.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score: 0.685\n",
      "Best parameters set:\n",
      "\tn_neighbors: 1\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.75      0.74      0.74       167\n",
      "          1       0.37      0.39      0.38        67\n",
      "\n",
      "avg / total       0.64      0.64      0.64       234\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn import grid_search\n",
    "\n",
    "knn = KNeighborsClassifier()\n",
    "\n",
    "parameters = {'n_neighbors':[1,10]}\n",
    "\n",
    "grid = grid_search.GridSearchCV(knn, parameters, n_jobs=-1, verbose=1, scoring='accuracy')\n",
    "\n",
    "\n",
    "grid.fit(X_train, y_train)\n",
    "\n",
    "print ('Best score: %0.3f' % grid.best_score_)\n",
    "\n",
    "print ('Best parameters set:')\n",
    "best_parameters = grid.best_estimator_.get_params()\n",
    "\n",
    "for param_name in sorted(parameters.keys()):\n",
    "    print ('\\t%s: %r' % (param_name, best_parameters[param_name]))\n",
    "    \n",
    "predictions = grid.predict(X_test)\n",
    "print( classification_report(y_test, predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM accuracy and time elapsed caculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM\n",
      "Acurracy:  0.717948717949\n",
      "time elapsed:  0.14200806617736816\n"
     ]
    }
   ],
   "source": [
    "t7=time()\n",
    "print (\"SVM\")\n",
    "\n",
    "svc = SVC()\n",
    "clf_svc=svc.fit(X_train, y_train)\n",
    "print (\"Acurracy: \", clf_svc.score(X_test,y_test) )\n",
    "t8=time()\n",
    "print (\"time elapsed: \", t8-t7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cross validation for SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross result========\n",
      "[ 0.71186441  0.70940171  0.71551724  0.71551724  0.71551724]\n",
      "0.713563568064\n",
      "time elapsed:  47.82873582839966\n"
     ]
    }
   ],
   "source": [
    "tt7=time()\n",
    "print (\"cross result========\")\n",
    "scores = cross_validation.cross_val_score(svc,X,y, cv=5)\n",
    "print (scores)\n",
    "print (scores.mean())\n",
    "tt8=time()\n",
    "print (\"time elapsed: \", tt7-tt6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM with RBF kernel accuracy and time elapsed caculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM\n",
      "Acurracy:  0.717948717949\n",
      "time elapsed:  0.03200173377990723\n"
     ]
    }
   ],
   "source": [
    "t7=time()\n",
    "print (\"SVM\")\n",
    "\n",
    "svc_r = SVC(kernel='rbf')\n",
    "clf_svc=svc.fit(X_train, y_train)\n",
    "print (\"Acurracy: \", clf_svc.score(X_test,y_test) )\n",
    "t8=time()\n",
    "print (\"time elapsed: \", t8-t7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cross validation for SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cross result========\n",
      "[ 0.71186441  0.70940171  0.71551724  0.71551724  0.71551724]\n",
      "0.713563568064\n",
      "time elapsed:  35752.70882368088\n"
     ]
    }
   ],
   "source": [
    "tt7=time()\n",
    "print (\"cross result========\")\n",
    "scores = cross_validation.cross_val_score(svc_r,X,y, cv=5)\n",
    "print (scores)\n",
    "print (scores.mean())\n",
    "tt8=time()\n",
    "print (\"time elapsed: \", tt7-tt6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}