Mercedes-Benz Greener Manufacturing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project 1 - Mercedes-Benz Greener Manufacturing\n",
    "Since the first automobile, the Benz Patent Motor Car in 1886, Mercedes-Benz has stood for important automotive innovations. These include the passenger safety cell with the crumple zone, the airbag, and intelligent assistance systems. Mercedes-Benz applies for nearly 2000 patents per year, making the brand the European leader among premium carmakers. Mercedes-Benz cars are leaders in the premium car industry. With a huge selection of features and options, customers can choose the customized Mercedes-Benz of their dreams.\n",
    "\n",
    "To ensure the safety and reliability of every unique car configuration before they hit the road, Daimler’s engineers have developed a robust testing system. As one of the world’s biggest manufacturers of premium cars, safety and efficiency are paramount on Daimler’s production lines. However, optimizing the speed of their testing system for many possible feature combinations is complex and time-consuming without a powerful algorithmic approach.\n",
    "\n",
    "You are required to reduce the time that cars spend on the test bench. Others will work with a dataset representing different permutations of features in a Mercedes-Benz car to predict the time it takes to pass testing. Optimal algorithms will contribute to faster testing, resulting in lower carbon dioxide emissions without reducing Daimler’s standards.\n",
    "\n",
    "#### Following actions should be performed:\n",
    "* If for any column(s), the variance is equal to zero, then you need to remove those variable(s).\n",
    "* Check for null and unique values for test and train sets\n",
    "* Apply label encoder.\n",
    "* Perform dimensionality reduction.\n",
    "* Predict your test_df values using xgboost\n",
    "\n",
    "**We are required to reduce the time that cars spend on the test bench.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "train=pd.read_csv('train.csv')\n",
    "test=pd.read_csv('test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of train data is (4209, 378)\n",
      "Shape of test data is (4209, 377)\n"
     ]
    }
   ],
   "source": [
    "print(\"Shape of train data is {}\".format(train.shape))\n",
    "print(\"Shape of test data is {}\".format(test.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Our traget variable is : y \n"
     ]
    }
   ],
   "source": [
    "for i in train.columns:\n",
    "    if i not in test.columns:\n",
    "        print(\"Our traget variable is : {} \".format(i))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets explore type our train dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4209 entries, 0 to 4208\n",
      "Columns: 378 entries, ID to X385\n",
      "dtypes: float64(1), int64(369), object(8)\n",
      "memory usage: 12.1+ MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are total 378 features out of which below are diferent types of data we are with:\n",
    "* float64 : 1\n",
    "* int64 : 369\n",
    "* object : 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Below are the total columns we are provided with :\n",
      "['ID', 'y', 'X0', 'X1', 'X2', 'X3', 'X4', 'X5', 'X6', 'X8', 'X10', 'X11', 'X12', 'X13', 'X14', 'X15', 'X16', 'X17', 'X18', 'X19', 'X20', 'X21', 'X22', 'X23', 'X24', 'X26', 'X27', 'X28', 'X29', 'X30', 'X31', 'X32', 'X33', 'X34', 'X35', 'X36', 'X37', 'X38', 'X39', 'X40', 'X41', 'X42', 'X43', 'X44', 'X45', 'X46', 'X47', 'X48', 'X49', 'X50', 'X51', 'X52', 'X53', 'X54', 'X55', 'X56', 'X57', 'X58', 'X59', 'X60', 'X61', 'X62', 'X63', 'X64', 'X65', 'X66', 'X67', 'X68', 'X69', 'X70', 'X71', 'X73', 'X74', 'X75', 'X76', 'X77', 'X78', 'X79', 'X80', 'X81', 'X82', 'X83', 'X84', 'X85', 'X86', 'X87', 'X88', 'X89', 'X90', 'X91', 'X92', 'X93', 'X94', 'X95', 'X96', 'X97', 'X98', 'X99', 'X100', 'X101', 'X102', 'X103', 'X104', 'X105', 'X106', 'X107', 'X108', 'X109', 'X110', 'X111', 'X112', 'X113', 'X114', 'X115', 'X116', 'X117', 'X118', 'X119', 'X120', 'X122', 'X123', 'X124', 'X125', 'X126', 'X127', 'X128', 'X129', 'X130', 'X131', 'X132', 'X133', 'X134', 'X135', 'X136', 'X137', 'X138', 'X139', 'X140', 'X141', 'X142', 'X143', 'X144', 'X145', 'X146', 'X147', 'X148', 'X150', 'X151', 'X152', 'X153', 'X154', 'X155', 'X156', 'X157', 'X158', 'X159', 'X160', 'X161', 'X162', 'X163', 'X164', 'X165', 'X166', 'X167', 'X168', 'X169', 'X170', 'X171', 'X172', 'X173', 'X174', 'X175', 'X176', 'X177', 'X178', 'X179', 'X180', 'X181', 'X182', 'X183', 'X184', 'X185', 'X186', 'X187', 'X189', 'X190', 'X191', 'X192', 'X194', 'X195', 'X196', 'X197', 'X198', 'X199', 'X200', 'X201', 'X202', 'X203', 'X204', 'X205', 'X206', 'X207', 'X208', 'X209', 'X210', 'X211', 'X212', 'X213', 'X214', 'X215', 'X216', 'X217', 'X218', 'X219', 'X220', 'X221', 'X222', 'X223', 'X224', 'X225', 'X226', 'X227', 'X228', 'X229', 'X230', 'X231', 'X232', 'X233', 'X234', 'X235', 'X236', 'X237', 'X238', 'X239', 'X240', 'X241', 'X242', 'X243', 'X244', 'X245', 'X246', 'X247', 'X248', 'X249', 'X250', 'X251', 'X252', 'X253', 'X254', 'X255', 'X256', 'X257', 'X258', 'X259', 'X260', 'X261', 'X262', 'X263', 'X264', 'X265', 'X266', 'X267', 'X268', 'X269', 'X270', 'X271', 'X272', 'X273', 'X274', 'X275', 'X276', 'X277', 'X278', 'X279', 'X280', 'X281', 'X282', 'X283', 'X284', 'X285', 'X286', 'X287', 'X288', 'X289', 'X290', 'X291', 'X292', 'X293', 'X294', 'X295', 'X296', 'X297', 'X298', 'X299', 'X300', 'X301', 'X302', 'X304', 'X305', 'X306', 'X307', 'X308', 'X309', 'X310', 'X311', 'X312', 'X313', 'X314', 'X315', 'X316', 'X317', 'X318', 'X319', 'X320', 'X321', 'X322', 'X323', 'X324', 'X325', 'X326', 'X327', 'X328', 'X329', 'X330', 'X331', 'X332', 'X333', 'X334', 'X335', 'X336', 'X337', 'X338', 'X339', 'X340', 'X341', 'X342', 'X343', 'X344', 'X345', 'X346', 'X347', 'X348', 'X349', 'X350', 'X351', 'X352', 'X353', 'X354', 'X355', 'X356', 'X357', 'X358', 'X359', 'X360', 'X361', 'X362', 'X363', 'X364', 'X365', 'X366', 'X367', 'X368', 'X369', 'X370', 'X371', 'X372', 'X373', 'X374', 'X375', 'X376', 'X377', 'X378', 'X379', 'X380', 'X382', 'X383', 'X384', 'X385']\n"
     ]
    }
   ],
   "source": [
    "print(\"Below are the total columns we are provided with :\")\n",
    "print(list(train.columns))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of unique values in ID column to identify what it means : 4209\n"
     ]
    }
   ],
   "source": [
    "print('Number of unique values in ID column to identify what it means :',train.ID.nunique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interpretation : *From above we can clearly interpret that ID column is nothing just a unique ID provided to different columns. So we can drop that for our further analysis.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.drop('ID',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### If for any column(s), the variance is equal to zero, then we need to remove those variable(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>variance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>X268</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X297</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X290</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X289</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X330</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X347</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X235</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X293</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X233</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X107</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X11</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X93</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X257</th>\n",
       "      <td>0.000238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X207</th>\n",
       "      <td>0.000238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X280</th>\n",
       "      <td>0.000238</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      variance\n",
       "X268  0.000000\n",
       "X297  0.000000\n",
       "X290  0.000000\n",
       "X289  0.000000\n",
       "X330  0.000000\n",
       "X347  0.000000\n",
       "X235  0.000000\n",
       "X293  0.000000\n",
       "X233  0.000000\n",
       "X107  0.000000\n",
       "X11   0.000000\n",
       "X93   0.000000\n",
       "X257  0.000238\n",
       "X207  0.000238\n",
       "X280  0.000238"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "var_df=pd.DataFrame(np.var(train,0),columns=['variance'])\n",
    "var_df.sort_values(by='variance').head(15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Below are columns with variance 0.\n",
      "['X11', 'X93', 'X107', 'X233', 'X235', 'X268', 'X289', 'X290', 'X293', 'X297', 'X330', 'X347']\n"
     ]
    }
   ],
   "source": [
    "print('Below are columns with variance 0.')\n",
    "col=list((var_df[var_df['variance']==0]).index)\n",
    "print(col)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X11 int64\n",
      "X93 int64\n",
      "X107 int64\n",
      "X233 int64\n",
      "X235 int64\n",
      "X268 int64\n",
      "X289 int64\n",
      "X290 int64\n",
      "X293 int64\n",
      "X297 int64\n",
      "X330 int64\n",
      "X347 int64\n"
     ]
    }
   ],
   "source": [
    "for i in col:\n",
    "    print(i, train[i].dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interpretation : *Above variance 0 shows that all the values in column is same and there is no variablity in data.* Lets drop above 12 columns with variance 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.drop(col,axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4209, 365)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check for null and unique values for test and train sets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Lets check out if there are any Missing value in our train dataset**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of missing values : \n",
      " 0    365\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of missing values : \\n {}\".format(train.isna().sum().value_counts()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interpretation : *From above we can interpret that there is no missing observation is our train datset.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets explore different variable type and perform label encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['X0', 'X1', 'X2', 'X3', 'X4', 'X5', 'X6', 'X8']\n"
     ]
    }
   ],
   "source": [
    "object_col=[]\n",
    "for i in train.columns:\n",
    "    a= train[i].dtype\n",
    "    if a =='object':\n",
    "        object_col.append(i)\n",
    "print(object_col)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X0 47\n",
      "X1 27\n",
      "X2 44\n",
      "X3 7\n",
      "X4 4\n",
      "X5 29\n",
      "X6 12\n",
      "X8 25\n"
     ]
    }
   ],
   "source": [
    "for i in object_col:\n",
    "    print(i, train[i].nunique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "LE=LabelEncoder()\n",
    "for i in object_col:\n",
    "    train[i]=LE.fit_transform(train[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4209 entries, 0 to 4208\n",
      "Columns: 365 entries, y to X385\n",
      "dtypes: float64(1), int64(364)\n",
      "memory usage: 11.7 MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "      <th>X0</th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "      <th>X3</th>\n",
       "      <th>X4</th>\n",
       "      <th>X5</th>\n",
       "      <th>X6</th>\n",
       "      <th>X8</th>\n",
       "      <th>X10</th>\n",
       "      <th>...</th>\n",
       "      <th>X375</th>\n",
       "      <th>X376</th>\n",
       "      <th>X377</th>\n",
       "      <th>X378</th>\n",
       "      <th>X379</th>\n",
       "      <th>X380</th>\n",
       "      <th>X382</th>\n",
       "      <th>X383</th>\n",
       "      <th>X384</th>\n",
       "      <th>X385</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>130.81</td>\n",
       "      <td>32</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>24</td>\n",
       "      <td>9</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>88.53</td>\n",
       "      <td>32</td>\n",
       "      <td>21</td>\n",
       "      <td>19</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>28</td>\n",
       "      <td>11</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>76.26</td>\n",
       "      <td>20</td>\n",
       "      <td>24</td>\n",
       "      <td>34</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "      <td>9</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>80.62</td>\n",
       "      <td>20</td>\n",
       "      <td>21</td>\n",
       "      <td>34</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>27</td>\n",
       "      <td>11</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>78.02</td>\n",
       "      <td>20</td>\n",
       "      <td>23</td>\n",
       "      <td>34</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>12</td>\n",
       "      <td>3</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4204</th>\n",
       "      <td>107.39</td>\n",
       "      <td>8</td>\n",
       "      <td>20</td>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4205</th>\n",
       "      <td>108.77</td>\n",
       "      <td>31</td>\n",
       "      <td>16</td>\n",
       "      <td>40</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4206</th>\n",
       "      <td>109.22</td>\n",
       "      <td>8</td>\n",
       "      <td>23</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4207</th>\n",
       "      <td>87.48</td>\n",
       "      <td>9</td>\n",
       "      <td>19</td>\n",
       "      <td>25</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4208</th>\n",
       "      <td>110.85</td>\n",
       "      <td>46</td>\n",
       "      <td>19</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4209 rows × 365 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           y  X0  X1  X2  X3  X4  X5  X6  X8  X10  ...  X375  X376  X377  \\\n",
       "0     130.81  32  23  17   0   3  24   9  14    0  ...     0     0     1   \n",
       "1      88.53  32  21  19   4   3  28  11  14    0  ...     1     0     0   \n",
       "2      76.26  20  24  34   2   3  27   9  23    0  ...     0     0     0   \n",
       "3      80.62  20  21  34   5   3  27  11   4    0  ...     0     0     0   \n",
       "4      78.02  20  23  34   5   3  12   3  13    0  ...     0     0     0   \n",
       "...      ...  ..  ..  ..  ..  ..  ..  ..  ..  ...  ...   ...   ...   ...   \n",
       "4204  107.39   8  20  16   2   3   0   3  16    0  ...     1     0     0   \n",
       "4205  108.77  31  16  40   3   3   0   7   7    0  ...     0     1     0   \n",
       "4206  109.22   8  23  38   0   3   0   6   4    0  ...     0     0     1   \n",
       "4207   87.48   9  19  25   5   3   0  11  20    0  ...     0     0     0   \n",
       "4208  110.85  46  19   3   2   3   0   6  22    0  ...     1     0     0   \n",
       "\n",
       "      X378  X379  X380  X382  X383  X384  X385  \n",
       "0        0     0     0     0     0     0     0  \n",
       "1        0     0     0     0     0     0     0  \n",
       "2        0     0     0     1     0     0     0  \n",
       "3        0     0     0     0     0     0     0  \n",
       "4        0     0     0     0     0     0     0  \n",
       "...    ...   ...   ...   ...   ...   ...   ...  \n",
       "4204     0     0     0     0     0     0     0  \n",
       "4205     0     0     0     0     0     0     0  \n",
       "4206     0     0     0     0     0     0     0  \n",
       "4207     0     0     0     0     0     0     0  \n",
       "4208     0     0     0     0     0     0     0  \n",
       "\n",
       "[4209 rows x 365 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train1=pd.DataFrame(train,columns=train.columns)\n",
    "train1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Defining dependent(Y) and independent(X) variables.**\n",
    "* independent(X) : X_data\n",
    "* dependent(Y) : Y_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_data=train1.drop('y',axis=1)\n",
    "Y_data=train1['y']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4209, 364)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "PCA_=PCA(random_state=0)\n",
    "Principal_comp=PCA_.fit(X_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAGFCAYAAAALqAHuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA9+0lEQVR4nO3dd5xcdb3/8ddnW3bTewwppCfSwRBKgBCqFAW91IsCys+IoqIXENB7FfV61QsKelUQ6RaKWAhIJxsQCCUBEkqyIQ2SkGQ3vWzf/fz+ON9JJpPd7Gyys2d35v18POaxc77ne858zpnZ+cz3e8rX3B0RERHp3PLiDkBERET2nhK6iIhIFlBCFxERyQJK6CIiIllACV1ERCQLKKGLiIhkASV0wcxuMDM3s/ebmf9+mH9DO4fWamZ2k5kta4fXuTTsEzezRjPbZGbzzOwWMxu9h+ucZmZnt3GorXn9U8zsm2nWnZm0/amPoRmIbZmZ3dTKZRLvUfe2jmc3rznbzO5pr9drJoYbzGxtnDFIPJTQJaEaGGlmE5MLzexwYESYL7s6ATga+DfgTuAUYJ6ZnbYH65oGnN12obXaKcA3W1G/FDiqiUd5m0e2Z/5JFE9l3IG0szuAU+MOQtpfQdwBSIexDXgDuACYnVR+ATAD+ERbvIiZFbt7Nv04eN3dt4bnz5rZbcBjwJ/NbIS7b4oxtkxb7+6vxB1Ec9y9AqiIO472YmaFQKO7rwBWxB2PtD+10CXZA8B5ZmYA4e95oXwXZnasmT1vZpVmts7Mfm9mPZLmJ7o8J4Uu2irgmjDvIDN71Mw2mtlWM3vNzE5OWravmd1uZmvMrNrMXjazI1Jev7eZ/Tksv8rMvttMnMPN7AEzWx9ifcrMxqfUud7MFoXXWmNmT5rZx1q7A929BvgG0Bu4MGn9V5nZ66Frfk3Y9jFJ82cS/Wi6JKnr+tIw72IzezHEv8HMSpvoSdk/xLzezLaZ2XwzuyKlzlmhS7jazFab2f+GJEA4nHIVsG/S69/T2u1Peb3HzKzMzEpS9kO1mR2QeF0zW2tmk83sjTDvLTM7poV1H2Vm08P7vi0sc1FKnZ263M1sRJg+z8x+F96LFWb2AzPLS1n2ADP7p5ltCY+/pH4eQp2XQszzzezTaeyTe8zs9SbKrwifzR5J+6nZz0uoM9PMHrboUM1iol60fSyly93MupnZr8N7UWlmS83sN2bWM2V9bmZXmtn/mFmFmZWHel1S6u1rZveH963SokNN/540vzh8tpabWY2ZzTWz01vaN7L31EKXZH8DbgWOAf4FHAsMCOU3Jlc0s8nAs8A/gHOAfsBPgT5hOtn9wG+BHwAbzWwC8BJQBlwOrAMmAsPCuruEdfcm+gFQDnyFqAU81t1Xh/XeDRwPfAtYDVwNjAbqk+LsC7wYXuNyou7X68K6xrl7lZldDHwHuBZ4N2zLCUC31uy8BHefb2YrgCOB20LxUODXwAdAzxDLy2F7NgFfBf4KLAF+FJZZHP6OAO4L00VEPxT+ZWb7u/uSUOdRYD7wOaAGGB9eJ7EfziN6H34XtnU08BOiH/VXE3XTjg3b/ZmwWEutWzOz1O8Qd/eG8PxLwDvhdb5pZh8H/hv4vru/k7RMV+CPod4qoh8WT6S816n2JfoM3UaUyCYDd5tZo7vf30Lc/0u0r88BTgS+R/S+PxQ2akxY92yi/VlA9J48amaT3N3Dj5SngLXAvwMlwC1A97DNzXkQeNzMRrr70qTy84HH3X1LmG7p85Iwmei9vJbos91Uj1BXIB/4LtF7Oiw8/wu7ds1fRdQj9zngIKL35IOwzzCzgcCs8FpXA8uBA8I6Ex4GJgHfJ/rMngdMN7OJ7v7WbvaN7C131yPHH8ANwNrw/BHgN+H5b4F/hOdrgRuSlvkXUJqynhMABw4I05eG6StT6t1P1CVY0kw8lwG1wNiksgKiL4cbw/T+Yd3nJ9XpDqwHliWV/YgomfdNKutD9MV3RZj+NfDXVu6zxLZ1b2b+LOCJZublEyWALcDFSeWzgXtaeN28sC8WAN8LZf1DLAc2s4wRfSnfnVL+RaAK6Bemb0redy3EMTO8ZupjWUq9C4FGosT5GvAykJ/y2XPg35t4H3+aVLYMuGk321dA9GNlRnPvEdEPIwfuS1n+LeCBpOk/EP3YLEoqGws0AGeE6a8CdcDQpDqTw/qbfQ9DnGuB65LKhoR9dE4rPy8zw/s3qLn/593EkIh1eFK5Ay+k1P0H8ErS9E+IDs8NbmbdJ4b1TEkpfwH4S2v+x/Ro/UNd7pLqAeCc0Eo+hya6282sK9HJRg+ZWUHiQdQSrmPX4+3/TJk+AXjQ3auaieEkYA6wNGndAM8TteQBDg9/H0ks5NGx7GeaWNczwOakdW0J60+s6y3g9ND1OsnM8puJqzVspwmzI83sGTNbR9SDUEmUuMa1uCKzj5vZ381sDVFSqSNqgSeWXU/UUrrNzM4Prahk44Dh7Pp+zQCKiVpYe2IG0fuQ/PhUcgWPWst/JfoM7A9c4jta8Mn+nrRM4n2c1NwLm1kfM/uVmX1AtD/qiE4qbHF/Ak+nTL9H1CJOOCnE05i0r5YS/ahIfGYmAXM8Ol6diPslWjgh0N3riXq8zk8qPpcoSW7/P2nF52WOu6/Z3WuG9X3ezN40s61E++rFMCt1fS3tmxOAJ919VTMvdRJRb9lLKZ+159ix7yRDlNAl1XSiL44fE3U5P9pEnT5ErYbfsuPLtI6oq7eQnbvfAFK/cPoRda02pz9Rd3VdyuMLSev+GLDFdz3BLvULtT/Rl2fquqYmresuom7o84BXgTVm9t97mdiHELbbzIYTfVEa8GWi1tHhIdbi3a0kHFN9OsT6H0SHQQ4H5iaWdfdGojPUV4dtWW1m/zKzQ5P2AcDj7LwPEl2+qe9Xuja4++yUx9tN1Lsf6AI86+5NXRq5tYkfd+XA4N289j1E7+uNRNt+ONG273Z/BhtTpmtTlutP1IWd+pkZxc6fv6aSdzpn+D8AHGJmiWR6PjA9sQ9a+XlJJ5l/huiQzSyiHw9HsuOwSur6NqZMp+6bdP53P8au++4G9vxzJmnSMXTZibtvM7PHiI5L/8XdtzVRbSNRt9oNREki1Uepq02ZXsfuv6zXE3U/f6WJeTXh72qgh+161nxq63Q90Y+UH7GrLbA9Id4M3Gxmw4CLiH7QrGDHMfC0hWPFQ4m+QAE+SXQc86zE/gytlr5prO6osK6T3X1B0mv0Sq4U5v2bRSe5HQv8DPinRdeErw/VpgFvNvEaS5soaxPhxKubiXpBPm1mp7r7UynVuptZSUpSH0gzicPMioEziQ6Z3JZU3lYNlPVELfQ7mpiXONlsNTChifmpn7+mPE+UiM83s/uIEuxPkua35vOSzvjX5wKvuvtXEwVmNiWN5ZqSzv/uSuK9/DJnKaFLU24lalE1mcxC0n8FGO/uP9yD9T9HdDb9d5toYSfmnwJ86O7NtXgSZwqfRXSiERadzXwysDn1tYB3d9PFv527Lwd+amZfAPZLZ2OShUMVvyL60ZM4XFFCdIy0Pqnqeez6/5faGkosCzt+yGBmRxMdD57TRPx1wAwz+wXwZ6ITC8uIvmRHuPvvdxN+U6+/t24h6s2ZCvwGuMPMDvBdL+f7TIg3+X28vZl1diHqXUzeJz2AT5NegmvJc0SHB+a4e3Prex24yMyGJrrdw4miLSZ0d28ws78QtcyriT4rTyZVSffzkq4SkvZVcFFTFdPwHPANMxvUTFf/c0Qn1m1N/gEq7UMJXXbh7jOJTrjZnW8Dz5lZI9FZrVuIjtOeAXzX3RfuZtkfEH0hvmBmPyf61X8osM7d7yLqHrwcmGnR3cGWEHX1TQJWu/vN7v6umU0Hbg2twFVEZ8Sn3kTkF0Rn7M4ws/8jSmyDgCnAi+5+v5n9jqhl8QrRyXJTiU6CuraFfQBwuEWX43UlOhb9ZaIzsM9NSloziJLa3WZ2J1GyuJpduzcXAKea2alhnywNMW0Ffm9m/0vUWr8hbAcQXQJIdELbg2Ff9Qmxz3X39aHOVcAfwr56gih5jyJqSZ3j7pXh9QdZdLncO0QnVi3bzbb3NbMjmyh/1923mNmZRCemne7uG83s62G9vwzlCVXAj0Mi/yjsm6JQbxfuvsmiS7++Z2abiZLfdUTvXc+mlmmlG4hO4Punmd1F1CofQvQj457w/3E38J+hzg1ESfNH7GjBt+RB4GtEPWH/cPfapHnpfl7S9QzwG4su63wVOJ3o5LU9cTNwMdFVFj8mOnfj40A3d//f8FpPAc+Y2c+Irh7oCRwCFLv79Xv4upKOuM/K0yP+By2cFRvq7HSWeyg7gqhlsZnopJ73iBJorzD/Upo5E5zokpjHiX4IbCH6ojkxaX4voi/05UTJZwXRyUSTk+r0IWoFbyPqwvweTZypDexD9AW8hqilsozoMqn9k+J8iSipVwLzgMta2B+JbUs8tgBvh5hHN1H/80Rn6VcRJekjSDlzmyjBPkuUmBy4NJR/kigRVoXYTif6wfVwmD+Q6MzsJUQtvtVEx62Hp8RwGtHVCdvCe/YW0WVkBWF+cdhP5bR8tvbMlO1PfhxD1D38EXB7ynJnhDqfSv7sER0meCu8P3OB41KWS91XY4hag9uAD4l+YN5A0ueY5s9yPzNl3fcAs1PKJhD9UF0f9vsiorPok89qP4jorP0aol6Qs0njSoWwrIW4HTh1Dz8v2z8Du/t/JvpxcFN4XzcTnaR4ROq+CNNfa+m7gegH64PABqL/l7nABUnzuxD9aF9E9L+7muh74ow4vt9y6WHhDRARaXehdfs1d+/fUl0R2T2d5S4iIpIFlNBFRESygLrcRUREsoBa6CIiIllACV1ERCQLdOrr0Pv37+8jRoyIOwwREZF2M2fOnLXuPiC1vFMn9BEjRjB79uy4wxAREWk3YVCiXajLXUREJAsooYuIiGQBJXQREZEsoIQuIiKSBZTQRUREsoASuoiISBZQQhcREckCSugiIiJZQAldREQkCyihi4iIZAEldBERkSyghC4iIpIFlNBFRESygBK6iIhIFlBCFxERyQJK6CIiIllACV1ERCQLKKGLiIhkASV0ERGRLFAQdwAiIiIC7k5Do1PfuONvfUPjLmXNUUIXEZFOyd2pbWikriFKfLUNjdQ3OPUNUXl9Y+OO56l1GqPl6sJ0XWMjdfWN1Dd6UnkjdY2eVN6YVN9pCOvYkXAT6951OqrTSEPS/NYk63QooYuISLMaG6OEWNvQSG19lNASf2vqE899p7LkOonlahsaqat3ahsattdvbp07pndeb23DznXrGvYuAaajMN8ozM+jIC/6W5ifR0FSWX4oz8+z7dPFhXkU5O2YX5Bv5OflUbjTtO2ok59YNtRJmi4I9ZOnP/OzpmNVQhcR6WDcnZr6KInV1DVQU99IdcrfmvoGquuS/tY1UF3fSE0oq01KgDXbk+Puk2lUnlQvtCDbUpQAjaL8PIoK8inKN4oKokSZ/LdrUUGYNooK8inMN7ok5ufnUViQF9aRtyPp5kcJMTXpFhbkUZi3o6wwP0qmRQUhqYZ4ChLL5UV18vMMM2vT7c8kJXQRkRYkEmx1XQNVdQ1U1UZ/q+saqKpt3PF8e1moV9ewPcFWp/ytqWukOvXv9mTduFfxFuRFSbIoKQEWFSQSYZS8CvPz6FFckJQUd/ztUrAjURbl529fZvs6UuoWJiXWop0S7a718vM6T4LsbJTQRaTTSk601XWNTSTb5MSaSMaNTc6vTkrKVbU7r6+6vgHfg4ZqUUEeJYX5dCnIozjlb5fCPHqVFNKlMI/igny6FObRJelvcWK6qWUTz1OWLS7M297SlNyjhC4iGVNb30hlbT3bahuorNnxt2qnxJmUPJPKUpPzTgm2bkci3ptEW1KYT0lRlCRLiqLpnsWFFIfnifnF4XlxYd4uZYnlind6HiXaPLVGpR0poYsI0HTy3VZTz7aaeiprG9hWW09lTQNba+qbrhemE3W31dS3+qSlovyo5bpLoizMZ0CPwqTEGSXMlhLr9uSaMl/dvpKNlNBFskRjo7Olpp5NlXVsrKplY2Udm6rq2FhVx6bKaHpjVV0or2VTVR3banYk6tqG9I/blhTm061LPt26FNC1qIBuRfn0Kilkn17FdC0qoHuXfLp2icq7FhXQrUt+KC+guDCfrkW7JuDiAnUVi+wNJXSRDqa+oXF7Ik4k3+3Jefvf2qT50fSmqjp2d0Jyt6J8enctomdJIb1LChnVvzvdi0PSDcm3W5cCuhUV0LVLfvQ3UdZlR70StXBFOiQldJF2Ut/QyOrN1azcUMXKjVWs3FDFR5uqWLmxmvXbaqLkXFnHlpr6ZtdhBj2LC+ndNUrKPUsKGda3K71LorJeJYX07lq0fToqK6JXSSFFBWr9imQzJXSRNlJV28DKjZWs2FDFRxurWbmxcqfkvXpz9S4t6P7di9indwkDexQzbmAPenUtpHdJUVIy3jlB9yguVOtYRJqkhC6SBndnQ2UdH22sYkVSkl65sTIk7yrWb6vdaZn8PGNwr2L26V3CkaP6MaRPCUN6lzCkTwn79I6eFxfmx7RFIpJtlNBFgo2VtbxfvnVHq3p70q7io41VVNY27FS/pDB/e5I+cGivKFmHhD2kdwmDeharNS0i7UYJXXLWluo6Xl+2npcXrWPWknW8t2rzTtc09+1WxD69ixk9oBvHjR2wPVEPDS3sPl0LO9VtIUUkuymhS86oqm1gzgcbmLVkLS8vXse8FZtoaHSKCvL4xPA+fOukcRw4tBfDQsLuWqR/DxHpPPSNJVmrtr6RuSs28vKidby8eC1vfriR2oZG8vOMg4f24itTRnP06H4ctm8fHcsWkU5PCV2yRn1DI+9+tJmXF0cJfPayDVTVNWAG++/Tk0snj+Co0f04fERfunfRR19Esou+1aTTamx0ytZs4eXF65i1eC2vLl3PluroGu5xg7pz/uHDOGp0P44c2Y9eXQtjjlZEJLMymtDNbBmwBWgA6t19opn1BR4ERgDLgPPcfYNFZxf9EjgdqAQudfc3MhmfdC7uzpK123h58TpeWRydyJa4VGxEv66cedA+HD26H0eO6seAHl1ijlZEpH21Rwt9qruvTZq+DnjO3X9qZteF6WuB04Cx4XEEcGv4Kzls+fpKZi1Zx6zQjb5mcw0A+/QqZur4gRw9uh9Hje7HPr1LYo5URCRecXS5nwUcH57fC8wkSuhnAfe5uwOvmFlvMxvs7qtiiFFi9MG6bfy2dDEvL1nL8vVVQHRHtaNG948S+Kh+7Nuvqy4ZExFJkumE7sDTZubA79z9dmBQUpJeDQwKz4cAy5OWXRHKlNBzSE19A9Pum8PyDZUcO7Y//++YURw1uh9jB3ZXAhcR2Y1MJ/Rj3H2lmQ0EnjGzBckz3d1Dsk+bmU0DpgEMHz687SKVDuFXz71P2Zot3HXpRE6YMKjlBUREBICMDr/k7ivD33Lg78AkYI2ZDQYIf8tD9ZXAsKTFh4ay1HXe7u4T3X3igAEDMhm+tLO5yzdy68zFnPuJoUrmIiKtlLGEbmbdzKxH4jlwCvAOMB24JFS7BHgkPJ8OXGyRI4FNOn6eO6rrGrjqL3MZ1LOY/zxzv7jDERHpdDLZ5T4I+Hs47lkA/NndnzSz14GHzOwy4APgvFD/caJL1hYRXbb2hQzGJh3Mzc8uZFH5Vu794iR6leiacRGR1spYQnf3JcDBTZSvA05sotyBKzIVj3Rccz7YwO9fWMKFk4YxZZwOo4iI7ImMHkMXaUl1XQPX/GUug3uV8N0z1NUuIrKndOtXidWNT5WxZO02/vT/jtD91UVE9oJa6BKb15au566XlvL5I/dl8pj+cYcjItKpKaFLLCpr67nm4bkM7VPCdadNiDscEZFOT32cEov/fbKMD9ZV8sC0I+mmrnYRkb2mFrq0u1mL13HPy8u49OgRHDmqX9zhiIhkBSV0aVfbaqKu9hH9uvLtT46POxwRkayhvk5pVz95Yj4rN1bx0JePomuRPn4iIm1FLXRpNy++v5Y/vvIhl00eyeEj+sYdjohIVlFCl3axpbqOa/86j1EDunH1qepqFxFpa+rzlHbxP4/PZ9WmKh7+ytEUF+bHHY6ISNZRC10ybmZZOfe/tpxpx43msOF94g5HRCQrKaFLRm2qquO6v77N2IHd+eZJY+MOR0Qka6nLXTLqR4+9R8XWGn73+U+oq11EJIPUQpeMeW7+Gh6es4KvTBnNwcN6xx2OiEhWU0KXjNhYWcv1f3ubCR/rwddPHBN3OCIiWU9d7pIRP3j0PdZvq+WuSw+nS4G62kVEMk0tdGlzT727mr+/uZIrpo7hgCG94g5HRCQnKKFLm1q/rZbv/v1t9hvckyumqqtdRKS9qMtd2tT3p7/Lpqo6/nDZERQV6PeiiEh70TeutJnH317Fo3M/4soTx/LxwT3jDkdEJKcooUubWLu1hv/8xzscOKQXl08ZHXc4IiI5Rwld9pq781//eIet1fX8/LyDKcjXx0pEpL3pm1f22qPzVvHEO6v51snjGDeoR9zhiIjkJCV02SvlW6r53iPvcMiw3nzp2JFxhyMikrOU0GWPuTvf/fs7VNY2cNO56moXEYmTvoFlj/3jrZU8894arjllPGMGdo87HBGRnKaELntkzeZqvv/Iu0zctw9fPEZd7SIicVNCl1Zzd67/29vUNjRy47kHk59ncYckIpLzlNCl1R6es4IZC8q59pMTGNm/W9zhiIgISujSSqs2VfHDR99j0si+XHLUiLjDERGRQAld0ubuXPvXt2lw56ZzDiZPXe0iIh2GErqk7YHXl/PCwgquP20Cw/t1jTscERFJooQuaVmxoZL/fuw9jh7dj4uO2DfucEREJIUSurSosdH59sPzAPjZvx2krnYRkQ5ICV1a9KfXPuTlxev47hn7MayvutpFRDqijCd0M8s3szfN7LEwfY+ZLTWzt8LjkFBuZvYrM1tkZvPM7LBMxyYt+3BdJT95fD7Hju3PhZOGxR2OiIg0o6AdXuNKYD7QM6nsGnd/OKXeacDY8DgCuDX8lZg0NjrXPDyXfDN+9m8HYaaudhGRjiqjLXQzGwqcAdyRRvWzgPs88grQ28wGZzI+2b2/v7mSV5eu57/O3I99epfEHY6IiOxGprvcbwG+DTSmlP84dKvfbGZdQtkQYHlSnRWhTGJQW9/Izc8u5MAhvTh34tC4wxERkRZkLKGb2ZlAubvPSZl1PTABOBzoC1zbyvVOM7PZZja7oqKibYKVXTz4+oes2FDF1aeOV1e7iEgnkMkW+mTg02a2DHgAOMHM/ujuq0K3eg1wNzAp1F8JJJ91NTSU7cTdb3f3ie4+ccCAARkMP3dV1TbwqxmLmDSyL8eN7R93OCIikoaMJXR3v97dh7r7COACYIa7fy5xXNyiZt/ZwDthkenAxeFs9yOBTe6+KlPxSfPunbWMii01XKPWuYhIp9EeZ7mn+pOZDQAMeAu4PJQ/DpwOLAIqgS/EEFvO21xdx60zFzN1/AAOH9E37nBERCRN7ZLQ3X0mMDM8P6GZOg5c0R7xSPPueGEJm6rquOqU8XGHIiIiraA7xcl2a7fWcMeLSznjoMEcMKRX3OGIiEgrKKHLdrfOXEx1XQP/cfK4uEMREZFWUkIXAD7aWMUfXvmAcz4xlNEDuscdjoiItJISugDwfzPeB4dvnDg27lBERGQPKKELS9du46HZK/j3I4YztI9GUxMR6YyU0IWbn1lIUX4eV0wdE3coIiKyh5TQc9x7H21m+tyP+OIxIxjQo0vLC4iISIekhJ7jfvFMGT2LC5h27Oi4QxERkb2ghJ7D5nywgWfnl/PlKaPp1bUw7nBERGQvKKHnKHfnxqcW0L97EV+YPCLucEREZC8poeeolxat45Ul6/na1DF0LYrjlv4iItKWlNBzUKJ1PqR3CRceMTzucEREpA0ooeegp99bw9wVm7jypLF0KciPOxwREWkDSug5pqHR+fnTZYwa0I3PHjok7nBERKSNKKHnmOlzV7JwzVauOnk8Bfl6+0VEsoW+0XNIbX0jNz/zPvvv05PTDvhY3OGIiEgbUkLPIQ/NXs6H6yu5+tTx5OVZ3OGIiEgbUkLPEdV1Dfzqufc5fEQfjh83IO5wRESkjSmh54j7Zi2jfEsN15w6ATO1zkVEso0Seg7YUl3Hb2cuZsq4AUwa2TfucEREJAOU0HPAHf9aysbKOq4+ZXzcoYiISIYooWe59dtqueNfSzj9wI9x4NBecYcjIiIZooSe5W6duYiqugb+4+RxcYciIiIZpISexVZtquLeWR/wmUOHMmZgj7jDERGRDFJCz2L/N2MR7s43TxobdygiIpJhSuhZatnabTz0+nIunDScYX27xh2OiIhkmBJ6lrrl2YUU5Btfmzom7lBERKQdKKFnoQWrN/PI3I+49OiRDOxZHHc4IiLSDpTQs9DPn15I96ICLp8yKu5QRESknSihZ5k3P9zAM++tYdpxo+jdtSjucEREpJ0ooWeZm54uo1+3Ir5wzMi4QxERkXakhJ5FXlq0lpcWreOrU8fQvUtB3OGIiEg7UkLPEu7OjU+VMbhXMRcdMTzucEREpJ0poWeJZ+eX89byjVx54liKC/PjDkdERNqZEnoWaGx0bnqqjJH9u/FvnxgadzgiIhIDJfQs8Oi8jyhbs4VvnTyOwny9pSIiuSjj3/5mlm9mb5rZY2F6pJm9amaLzOxBMysK5V3C9KIwf0SmY8sGdQ2N/OKZhUz4WA/OPHBw3OGIiEhM2qM5dyUwP2n6Z8DN7j4G2ABcFsovAzaE8ptDPWnBw3NW8MG6Sq45dTx5eRZ3OCIiEpOMJnQzGwqcAdwRpg04AXg4VLkXODs8PytME+afGOpLM6rrGvjls+9z2PDenDBhYNzhiIhIjFq8WDkk5QuAY4F9gCrgHeCfwBPu3ribxW8Bvg0kBuPuB2x09/owvQIYEp4PAZYDuHu9mW0K9de2Yntyyh9f+YDVm6u5+fxD0G8fEZHcttsWupndDdwF1BJ1gV8IfBV4Fvgk8KKZHdfMsmcC5e4+py0DNrNpZjbbzGZXVFS05ao7la019fx25mKOHdufo0b3izscERGJWUst9J+7+ztNlL8D/C2c0NbcXUwmA582s9OBYqAn8Eugt5kVhFb6UGBlqL8SGAasMLMCoBewLnWl7n47cDvAxIkTvYX4s9ZdLy5l/bZarj5lfNyhiIhIB7DbFnpTydzMRpvZgWF+rbsvambZ6919qLuPIOqyn+HuFwGlwDmh2iXAI+H59DBNmD/D3XM2Ye/Ohm21/P6FJZy6/yAOHtY77nBERKQDaNUNv83sO8AYoNHMurj75/fgNa8FHjCz/wbeBO4M5XcCfzCzRcB6oh8B0oTbXljM1tp6rlLrXEREgt0mdDP7BvAbd28IRQe7+/lh3rx0X8TdZwIzw/MlwKQm6lQD56a7zly1ZnM19768jM8cMoRxg3q0vICIiOSEli5bWwc8aWafDtNPm9mTZvY08FRmQ5Om/HrGIuobnG+eNC7uUEREpANp6Rj6n4BPAQeZ2XRgDvBZ4Fx3v6Yd4pMkH66r5P7XPuSCScMY3q9r3OGIiEgHks6NZUYDDwHTgCuIzlQvyWRQ0rRbnltIfp7x9RPGxh2KiIh0MC0dQ78HqAO6Aivd/UtmdijwezN73d1/2A4xCrBwzRb+/uZKph07ikE9i+MOR0REOpiWznI/1N0PBjCzNwHc/U3gU2Z2VqaDkx1+8fRCuhUVcPmU0XGHIiIiHVBLCf1JM3sKKAT+nDzD3R9pehFpa/NXbebJd1dz5Ylj6dOtKO5wRESkA9ptQnf3a82sJ9Do7lvbKSZJ8bvnF9OtKJ8vTh4ZdygiItJBtXQv988BW5tL5uGuccdkJDIBYPn6Sh6dt4oLJw2nV9fCuMMREZEOqqUu937Am2Y2h+iStQqi+7KPAaYQjYR2XUYjzHF3/GsJeQaXHavWuYiINK+lLvdfmtmvicYwnwwcRDR86nzg8+7+YeZDzF3rttbw4OzlnH3IEAb30pWCIiLSvBbv5R5u+/pMeEg7uvflZdTUN/LlKaPiDkVERDq4dG4sIzHYVlPPvbM+4OSPD2LMQN2zXUREdk8JvYO6/7UP2VRVx+XH67pzERFpmRJ6B1Rb38idLy7liJF9OWx4n7jDERGRTiCthG5mg8zsTjN7IkzvZ2aXZTa03PXIWytZtamar6h1LiIiaUq3hX4P0XCp+4TphcA3MxBPzmtsdH73whI+PrgnU8YNiDscERHpJNJN6P3d/SGgEcDd64GGjEWVw56dv4ZF5Vu5fMoozCzucEREpJNIN6FvM7N+gAOY2ZHApoxFlaPcnVufX8ywviWcceDguMMREZFOpMXr0IP/AKYDo83sJWAAcE7GospRry1dz5sfbuRHZ+1PQb7OVxQRkfSlldDd/Q0zmwKMBwwoc/e6jEaWg257fjH9uhVx7sRhcYciIiKdTLpnuV8BdHf3d939HaC7mX01s6HllvmrNlNaVsGlR4+guDA/7nBERKSTSbdf90vuvjEx4e4bgC9lJKIclRgi9eKjRsQdioiIdELpJvR8Szrl2szygaLMhJR7NESqiIjsrXRPinsSeNDMfhemvxzKpA1oiFQREdlb6Sb0a4mS+FfC9DPAHRmJKMdoiFQREWkL6Z7l3gjcGh7ShjREqoiItIW0ErqZTQZuAPYNyxjg7q4stBc0RKqIiLSVdLvc7wS+BcxBt3xtMxoiVURE2kq6CX2Tuz+R0UhyjIZIFRGRtpRuQi81sxuBvwE1iUJ3fyMjUeWAxBCpP/nsgXGHIiIiWSDdhH5E+DsxqcyBE9o2nNygIVJFRKStpXuW+9RMB5JLEkOk/vKCQzREqoiItIl0W+iY2RnA/kBxoszdf5iJoLKZhkgVEZFMSHdwltuA84GvE12ydi7RJWzSSokhUqcdO0pDpIqISJtJN6Mc7e4XAxvc/QfAUcC4zIWVvTREqoiIZEK6Cb0q/K00s32AOkD9xa2kIVJFRCRT0j2G/piZ9QZuBN4gOsNd93JvJQ2RKiIimZJWC93df+TuG939r0THzie4+3/tbhkzKzaz18xsrpm9a2Y/COX3mNlSM3srPA4J5WZmvzKzRWY2z8wO28tt61A0RKqIiGTSblvoZnaCu88ws882MQ93/9tuFq8BTnD3rWZWCLxoZom7zV3j7g+n1D8NGBseRxANBHMEWUJDpIqISCa11OU+BZgBfKqJeU5057gmubsDW8NkYXj4bl7rLOC+sNwrZtbbzAa7+6oWYuzwNESqiIhk2m4Turt/38zygCfc/aHWrtzM8okGdBkD/MbdXzWzrwA/NrPvAc8B17l7DTAEWJ60+IpQtiplndOAaQDDhw9vbUix0BCpIiKSaS0eQw9joX97T1bu7g3ufggwFJhkZgcA1wMTgMOBvsC1rVzn7e4+0d0nDhjQ8W+bqiFSRUSkPaR72dqzZna1mQ0zs76JR7ov4u4bgVLgk+6+yiM1wN3ApFBtJZB8cfbQUNapaYhUERFpD+km9POBK4AXiLrQ5wCzd7eAmQ0Il7phZiXAycACMxscygw4G3gnLDIduDic7X4k0ZCtnfr4uYZIFRGR9pLu4Cx7cmr2YODecBw9D3jI3R8zsxlmNoDoFrJvAZeH+o8DpwOLgErgC3vwmh2KhkgVEZH20prBWQ4A9mPnwVnua66+u88DDm2ivMkhV8PZ7VekG09HpyFSRUSkPaWV0M3s+8DxRAn9caJrxl8Emk3ouU5DpIqISHtK9xj6OcCJwGp3/wJwMNArY1F1choiVURE2lvag7OEy9fqzawnUM7OZ6RLEg2RKiIi7S3dY+izwxnrvyc6w30rMCtTQXV2GiJVRETaW0v3cv8N8Gd3/2oous3MngR6hpPeJEViiNSrTh6nIVJFRKTdtNRCXwjcFK4dfwi4393fzHxYnZeGSBURkTjs9gCvu//S3Y8iGqRlHXCXmS0ws++b2bh2ibAT0RCpIiISl3THQ//A3X/m7ocCFxLd4W1+JgPrjDREqoiIxCWthG5mBWb2KTP7E/AEUAbsMkZ6LtMQqSIiEqeWToo7mahFfjrwGvAAMM3dt7VDbJ2KhkgVEZE4tXRS3PXAn4Gr3H1DO8TTKWmIVBERidtuE3pz912XnWmIVBERiZtuY7aXNESqiIh0BEroeykxROpX1DoXEZEYKaHvBQ2RKiIiHYUS+l5IDJF6+ZRRGiJVRERipYS+hzREqoiIdCRK6HtIQ6SKiEhHoky0hzREqoiIdCRK6HsgMUTqpUeP0BCpIiLSISih7wENkSoiIh2NEnoraYhUERHpiJTQW0lDpIqISEekhN4KDY3O9LkfcdoBgzVEqoiIdChK6K0wd8VGNlTWcdJ+g+IORUREZCdK6K0wc0E5eQbHje0fdygiIiI7UUJvhdKyCg4b3ofeXYviDkVERGQnSuhpKt9SzdsrNzF1wsC4QxEREdmFEnqani+rAOD48RpVTUREOh4l9DSVlpUzqGcX9hvcM+5QREREdqGEnoa6hkb+tXAtU8cP1DCpIiLSISmhp2HOBxvYUlPP8eN1/FxERDomJfQ0lJaVU5hvTB7TL+5QREREmqSEnoaZCyo4fERfehTr3u0iItIxKaG3YOXGKsrWbGGquttFRKQDU0JvwcyycgCmTtDlaiIi0nFlLKGbWbGZvWZmc83sXTP7QSgfaWavmtkiM3vQzIpCeZcwvSjMH5Gp2FqjdEEFQ/uUMHpA97hDERERaVYmW+g1wAnufjBwCPBJMzsS+Blws7uPATYAl4X6lwEbQvnNoV6sauobeGmRLlcTEZGOL2MJ3SNbw2RheDhwAvBwKL8XODs8PytME+afaDFn0deWrqeqrkHd7SIi0uFl9Bi6meWb2VtAOfAMsBjY6O71ocoKYEh4PgRYDhDmbwJ2uU7MzKaZ2Wwzm11RUZHJ8CldUEFRQR5HjdLoaiIi0rFlNKG7e4O7HwIMBSYBE9pgnbe7+0R3nzhgQGZbzjPLyjlqVD9KivIz+joiIiJ7q13Ocnf3jUApcBTQ28wKwqyhwMrwfCUwDCDM7wWsa4/4mrJs7TaWrN3GVA3GIiIinUAmz3IfYGa9w/MS4GRgPlFiPydUuwR4JDyfHqYJ82e4u2cqvpaUhsvVTpgwKK4QRERE0lbQcpU9Nhi418zyiX44POTuj5nZe8ADZvbfwJvAnaH+ncAfzGwRsB64IIOxtai0rIJRA7oxvF/XOMMQERFJS8YSurvPAw5tonwJ0fH01PJq4NxMxdMalbX1vLJkHZ8/ct+4QxEREUmL7hTXhFmL11Fb36jbvYqISKehhN6E0rJyuhblc/jIPnGHIiIikhYl9BTuTumCCiaP6U+XAl2uJiIinYMSeopF5VtZubFK3e0iItKpKKGnSFyudryuPxcRkU5ECT1F6YIKJnysB/v0Lok7FBERkbQpoSfZUl3H68vWc7y620VEpJNRQk/y0qK11De6bvcqIiKdjhJ6ktIFFfQoLuAT++pyNRER6VyU0AN3p7SsnOPGDaAgX7tFREQ6F2Wu4N2PNlO+pUaXq4mISKekhB7MDJerTRmn4+ciItL5KKEHpWUVHDS0FwN6dIk7FBERkVZTQgc2bKvlzQ836HI1ERHptJTQgRfer6DR0eVqIiLSaSmhAzPLKujbrYiDhvaOOxQREZE9kvMJvaHReX5hBVPGDSA/z+IOR0REZI/kfEKft2Ij67fVajAWERHp1HI+oZeWVZBncNxYJXQREem8cj6hzywr59DhfejTrSjuUERERPZYTif0ii01zFuxiRMm6HI1ERHp3HI6oT+/sAJAx89FRKTTy+mEXlpWzsAeXdhvcM+4QxEREdkrOZvQ6xoaeWFhBVPHD8RMl6uJiEjnlrMJ/Y0PNrClup6pE9TdLiIinV/OJvTSsgoK8ozJY/rHHYqIiMhey9mEPrOsnMNH9KVHcWHcoYiIiOy1nEzoH22sYsHqLepuFxGRrJGTCX1mWXS52lQNlyoiIlkiJxN6aVk5Q3qXMGZg97hDERERaRM5l9Br6ht4adFapk4YoMvVREQka+RcQn996QYqaxvU3S4iIlkl5xJ6aVk5RQV5HD1al6uJiEj2yMmEftSofpQU5ccdioiISJvJqYT+wbptLKnYxlQNxiIiIlkmpxJ64nK143X8XEREskzGErqZDTOzUjN7z8zeNbMrQ/kNZrbSzN4Kj9OTlrnezBaZWZmZndrWMc1YUM6o/t0Y0b9bW69aREQkVgUZXHc9cJW7v2FmPYA5ZvZMmHezu9+UXNnM9gMuAPYH9gGeNbNx7t7QFsFU1TYwa8k6PnfEvm2xOhERkQ4lYy10d1/l7m+E51uA+cCQ3SxyFvCAu9e4+1JgETCpreKZtWQttfWNut2riIhkpXY5hm5mI4BDgVdD0dfMbJ6Z3WVmfULZEGB50mIraOIHgJlNM7PZZja7oqIi7RhKF1RQUpjPpJF992gbREREOrKMJ3Qz6w78Ffimu28GbgVGA4cAq4Cft2Z97n67u09094kDBqTX2nZ3SsvKmTymP10KdLmaiIhkn4wmdDMrJErmf3L3vwG4+xp3b3D3RuD37OhWXwkMS1p8aCjba4srtrJiQ5W620VEJGtl8ix3A+4E5rv7L5LKBydV+wzwTng+HbjAzLqY2UhgLPBaW8RSukCXq4mISHbL5Fnuk4HPA2+b2Vuh7DvAhWZ2CODAMuDLAO7+rpk9BLxHdIb8FW11hntpWTnjB/VgSO+StlidiIhIh5OxhO7uLwJNDWf2+G6W+THw47aMY0t1Ha8vW89lx4xqy9WKiIh0KFl/p7iXFq2jrsF1u1cREclqWZ/QZ5aV06O4gMP27dNyZRERkU4qqxN64nK148YOoDA/qzdVRERyXFZnufmrtrBmcw3Hq7tdRESyXFYn9NKycgCmKKGLiEiWy+6EvqCcA4f0YmCP4rhDERERyaisTegbK2t548MNOrtdRERyQtYm9BfeX0ujw/ETdHc4ERHJflmb0GcuKKdP10IOHto77lBEREQyLisTemOjM3NhBVPGDSA/r6mb1YmIiGSXrEzo81ZuYv22Wqaqu11ERHJEVib00gXl5BkcN1YnxImISG7IyoQ+s6ycQ4f3oU+3orhDERERaRdZl9ArttQwd8UmXa4mIiI5JesS+gsLKwA4fryOn4uISO7IuoReWlbOwB5d2H+fnnGHIiIi0m6yKqHXNzTywsIKjh8/ADNdriYiIrkjqxL6Gx9uZHN1PVPV3S4iIjkmqxJ6aVk5BXnG5LH94w5FRESkXWVXQl9QzsQRfehZXBh3KCIiIu0qaxL6qk1VLFi9Rd3tIiKSk7Imoc8siy5X0+1eRUQkF2VNQi9dUM6Q3iWMHdg97lBERETaXVYk9Jr6Bl5atJapE3S5moiI5KasSOizl21gW22Djp+LiEjOyoqEXrqgnKKCPI4a3S/uUERERGKRHQm9rJwjR/Wja1FB3KGIiIjEotMn9A/XVbK4YptGVxMRkZzW6RP6zIXlADp+LiIiOa3TJ/QZC8oZ2b8bI/p3izsUERGR2HTqhN7oMGvxOo5Xd7uIiOS4Tp3Qt9XUU1PfqO52ERHJeZ06oW+prqOkMJ9JI/vGHYqIiEisOnlCr2fymH4UF+bHHYqIiEisOnVCr21o5Hh1t4uIiHTuhA4aXU1ERAQymNDNbJiZlZrZe2b2rpldGcr7mtkzZvZ++NsnlJuZ/crMFpnZPDM7rKXX6FlcyJDeJZnaBBERkU4jky30euAqd98POBK4wsz2A64DnnP3scBzYRrgNGBseEwDbm3pBfbt1zUTcYuIiHQ6GUvo7r7K3d8Iz7cA84EhwFnAvaHavcDZ4flZwH0eeQXobWaDMxWfiIhINmmXY+hmNgI4FHgVGOTuq8Ks1cCg8HwIsDxpsRWhTERERFqQ8YRuZt2BvwLfdPfNyfPc3QFv5fqmmdlsM5tdUVHRhpGKiIh0XhlN6GZWSJTM/+TufwvFaxJd6eFveShfCQxLWnxoKNuJu9/u7hPdfeKAAbrlq4iICGT2LHcD7gTmu/svkmZNBy4Jzy8BHkkqvzic7X4ksCmpa15ERER2oyCD654MfB5428zeCmXfAX4KPGRmlwEfAOeFeY8DpwOLgErgCxmMTUREJKtkLKG7+4uANTP7xCbqO3BFpuIRERHJZp3+TnEiIiKihC4iIpIVlNBFRESygBK6iIhIFlBCFxERyQJK6CIiIlnAoqvFOicz2wKUxR1HO+oPrI07iHaSS9sKubW9ubStkFvbm0vbCvFt777uvsutUjN5Y5n2UObuE+MOor2Y2exc2d5c2lbIre3NpW2F3NreXNpW6Hjbqy53ERGRLKCELiIikgU6e0K/Pe4A2lkubW8ubSvk1vbm0rZCbm1vLm0rdLDt7dQnxYmIiEiks7fQRUREhE6c0M3sk2ZWZmaLzOy6uOPJFDMbZmalZvaemb1rZlfGHVN7MLN8M3vTzB6LO5ZMMrPeZvawmS0ws/lmdlTcMWWSmX0rfI7fMbP7zaw47pjakpndZWblZvZOUllfM3vGzN4Pf/vEGWNbaWZbbwyf5Xlm9ncz6x1jiG2qqe1NmneVmbmZ9Y8jtoROmdDNLB/4DXAasB9woZntF29UGVMPXOXu+wFHAldk8bYmuxKYH3cQ7eCXwJPuPgE4mCzeZjMbAnwDmOjuBwD5wAXxRtXm7gE+mVJ2HfCcu48FngvT2eAedt3WZ4AD3P0gYCFwfXsHlUH3sOv2YmbDgFOAD9s7oFSdMqEDk4BF7r7E3WuBB4CzYo4pI9x9lbu/EZ5vIfrCHxJvVJllZkOBM4A74o4lk8ysF3AccCeAu9e6+8ZYg8q8AqDEzAqArsBHMcfTptz9BWB9SvFZwL3h+b3A2e0ZU6Y0ta3u/rS714fJV4Ch7R5YhjTz3gLcDHwbiP2EtM6a0IcAy5OmV5DlSQ7AzEYAhwKvxhxKpt1C9A/SGHMcmTYSqADuDocX7jCzbnEHlSnuvhK4iaglswrY5O5PxxtVuxjk7qvC89XAoDiDaUdfBJ6IO4hMMrOzgJXuPjfuWKDzJvScY2bdgb8C33T3zXHHkylmdiZQ7u5z4o6lHRQAhwG3uvuhwDaypzt2F+HY8VlEP2T2AbqZ2efijap9eXRZUewtuUwzs+8SHS78U9yxZIqZdQW+A3wv7lgSOmtCXwkMS5oeGsqykpkVEiXzP7n73+KOJ8MmA582s2VEh1JOMLM/xhtSxqwAVrh7osflYaIEn61OApa6e4W71wF/A46OOab2sMbMBgOEv+Uxx5NRZnYpcCZwkWf3ddGjiX6czg3fV0OBN8zsY3EF1FkT+uvAWDMbaWZFRCfWTI85powwMyM6xjrf3X8RdzyZ5u7Xu/tQdx9B9L7OcPesbMW5+2pguZmND0UnAu/FGFKmfQgcaWZdw+f6RLL4JMAk04FLwvNLgEdijCWjzOyTRIfLPu3ulXHHk0nu/ra7D3T3EeH7agVwWPi/jkWnTOjhpIuvAU8RfSE85O7vxhtVxkwGPk/UUn0rPE6POyhpM18H/mRm84BDgP+JN5zMCT0RDwNvAG8Tff90qDtt7S0zux+YBYw3sxVmdhnwU+BkM3ufqJfip3HG2Faa2dZfAz2AZ8J31W2xBtmGmtneDkV3ihMREckCnbKFLiIiIjtTQhcREckCSugiIiJZQAldREQkCyihi4iIZAEldJE2FEZc+nnS9NVmdkMbrfseMzunLdbVwuucG0Z+K21i3jgzezyMHPaGmT1kZp36VqZmdnaODHgkWU4JXaRt1QCfjXsYxVRhMJR0XQZ8yd2npqyjGPgn0a1qx7r7YcBvgQFtF2ksziYatVGkU1NCF2lb9UQ3S/lW6ozUFraZbQ1/jzez583sETNbYmY/NbOLzOw1M3vbzEYnreYkM5ttZgvDfe8TY8ffaGavh3Gov5y03n+Z2XSauAOdmV0Y1v+Omf0slH0POAa408xuTFnk34FZ7v5oosDdZ7r7O2ZWbGZ3h/W9aWZTw/ouNbN/hHHAl5nZ18zsP0KdV8ysb6g308x+GW5G8o6ZTQrlfcPy80L9g0L5DRaNTz0z7LNvJG3X58K+e8vMfmfRcMuY2VYz+7GZzQ3rGmRmRwOfBm4M9Ueb2TfM7L3wmg+k86aLdARK6CJt7zfARRYNj5qug4HLgY8T3RlwnLtPIhpC9utJ9UYQDR98BnBbaDVfRjRy2eHA4cCXzGxkqH8YcKW7j0t+MTPbB/gZcALRHeoON7Oz3f2HwGyi+3BfkxLjAUBzg+ZcQTT2yIHAhcC9IbbEcp8Nsf0YqAyD0cwCLk5aR1d3PwT4KnBXKPsB8GYYX/s7wH1J9ScAp4b98X0zKzSzjwPnA5PDuhqAi0L9bsAr7n4w8AJRL8TLRLdmvcbdD3H3xUQD5BwaXvPyZrZXpMNRQhdpY2E0vPuAb7RUN8nr7r7K3WuAxUBiWNG3iZJ4wkPu3uju7wNLiJLaKcDFZvYW0dC6/YCxof5r7r60idc7HJgZBkpJjIp1XCviTXUM8EcAd18AfAAkfkSUuvsWd68ANgGJFn7qtt0fln8B6GlmvcN6/xDKZwD9zKxnqP9Pd69x97VEA54MIro//CeA18P+OBEYFerXAo+F53NSXjvZPKLb8X6OqMdFpFNozXE1EUnfLUT3LL87qaye8CPazPKAoqR5NUnPG5OmG9n5/zT1Xs0OGPB1d38qeYaZHU80JGtbeReYsgfL7c22pbvehrAuA+519+ubqF+XNPpXon5TziD6cfMp4LtmdmD40SPSoamFLpIB7r4eeIioOzxhGVHrEaLjtoV7sOpzzSwvHFcfBZQRDVL0FYuG2U2cid6thfW8Bkwxs/7hGPOFwPMtLPNn4GgzOyNRYGbHmdkBwL8IXdtmNg4YHmJrjfPD8scQHULYlLLe44G1oQekOc8B55jZwLBMXzPbt4XX3UI0oEjih9Ywdy8FrgV6Ad1buR0isVALXSRzfk40KmDC74FHzGwu8CR71nr+kCgZ9wQud/dqM7uDqPv4DTMzoILozO1mufsqM7sOKCVq1f7T3Xc7rKe7V4UT8W4xs1uAOqLu6SuJzna/1czeJuqJuNTda6Jw0lZtZm8S/dD5Yii7AbjLotHoKtkxDGlzMb5nZv8JPB2Scx3R8f0PdrPYA8Dvw4l1FxCdENiLaL/8yt03tmYjROKi0dZEJHZmNhO42t1nxx2LSGelLncREZEsoBa6iIhIFlALXUREJAsooYuIiGQBJXQREZEsoIQuIiKSBZTQRUREsoASuoiISBb4/ygoaEsV1TY5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8,6))\n",
    "plt.plot(np.cumsum(PCA_.explained_variance_))\n",
    "plt.title(\"Mercedes Dataset Explained variance\",fontdict = {'fontsize':15})\n",
    "plt.xlabel(\"Number of Components\")\n",
    "plt.ylabel(\"Variance (%)\")\n",
    "plt.xlim(0,15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9401733147573702"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PCA_=PCA(n_components=5)\n",
    "Principal_data=PCA_.fit_transform(X_data)\n",
    "PCA_.explained_variance_ratio_.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([38.33478209, 21.38803259, 13.2618659 , 11.82664248,  9.20600842])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PCA_.explained_variance_ratio_*100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interpretation : *The first component is the most important one, followed by the second, then the third, and so on. In our case 5 components expains 94.01% variance with 5.99% loss of Information.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "Principal_data=pd.DataFrame(Principal_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Our next aim is model fitting. We will fit XGboost regressor algorithm here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,Y_train,Y_test=train_test_split(X_data,Y_data,test_size=0.25,random_state=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1053, 364)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find out best parameters for fitting lets apply GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best CV Score 0.5564287019742021\n",
      "Model Parameters {'learning_rate': 0.1, 'max_depth': 3, 'n_estimators': 50, 'random_state': 0}\n",
      "Best Estimator XGBRegressor(base_score=0.5, booster=None, colsample_bylevel=1,\n",
      "             colsample_bynode=1, colsample_bytree=1, gamma=0, gpu_id=-1,\n",
      "             importance_type='gain', interaction_constraints=None,\n",
      "             learning_rate=0.1, max_delta_step=0, max_depth=3,\n",
      "             min_child_weight=1, missing=nan, monotone_constraints=None,\n",
      "             n_estimators=50, n_jobs=0, num_parallel_tree=1, random_state=0,\n",
      "             reg_alpha=0, reg_lambda=1, scale_pos_weight=1, subsample=1,\n",
      "             tree_method=None, validate_parameters=False, verbosity=None)\n"
     ]
    }
   ],
   "source": [
    "from xgboost import XGBRegressor\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "XGboost=XGBRegressor()\n",
    "xgbparams={'max_depth': [3,5,10,15], 'learning_rate' : [0.1], 'n_estimators' : [50,100,200,300],'random_state' :[0]}\n",
    "grid=zip([XGboost],[xgbparams])\n",
    "best_model=None\n",
    "for model,param in grid:\n",
    "    temp=GridSearchCV(model,param_grid=param,cv=3,n_jobs=1)\n",
    "    temp.fit(X_train,Y_train)\n",
    "    if best_model is None:\n",
    "        best_model=temp\n",
    "    else:\n",
    "        if temp.best_score_ > best_model.best_score_:\n",
    "            best_model = temp\n",
    "print (\"Best CV Score\",best_model.best_score_)\n",
    "print (\"Model Parameters\",best_model.best_params_)\n",
    "print(\"Best Estimator\",best_model.best_estimator_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=best_model.best_estimator_\n",
    "model.fit(X_train,Y_train)\n",
    "pred=model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5847471533820565"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import r2_score,mean_absolute_error,mean_squared_error\n",
    "r2_score(Y_test,pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5858725179405241\n",
      "0.5847471533820565\n"
     ]
    }
   ],
   "source": [
    "print(model.score(X_train,Y_train))\n",
    "print(model.score(X_test,Y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_data_copy=X_data.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets apply cleaning on test data and then find prediction for that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    364\n",
       "dtype: int64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.drop('ID',axis=1,inplace=True)\n",
    "test.drop(col,axis=1,inplace=True)\n",
    "test.isna().sum().value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in object_col:\n",
    "    test[i]=LE.fit_transform(test[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "test1=pd.DataFrame(test,columns=test.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 77.21249 ,  94.13634 ,  77.83281 , ...,  93.775505, 111.64696 ,\n",
       "        94.01045 ], dtype=float32)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(test1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Here we will try alternative dimension reduction techniques if any works to increase model accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import FastICA\n",
    "ICA=FastICA(n_components=5,random_state=0)\n",
    "New_df=ICA.fit_transform(X_data_copy)\n",
    "New_df=pd.DataFrame(New_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train,X_test,Y_train,Y_test=train_test_split(New_df,Y_data,test_size=0.25,random_state=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_2=best_model.best_estimator_\n",
    "model_2.fit(X_train,Y_train)\n",
    "pred=model_2.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.2849866924276957"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(Y_test,pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.366993910138537\n",
      "0.2849866924276957\n"
     ]
    }
   ],
   "source": [
    "print(model_2.score(X_train,Y_train))\n",
    "print(model_2.score(X_test,Y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import FactorAnalysis\n",
    "FA=FactorAnalysis(n_components=5,random_state=0)\n",
    "\n",
    "New_df_2=FA.fit_transform(X_data_copy)\n",
    "New_df_2=pd.DataFrame(New_df_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train,X_test,Y_train,Y_test=train_test_split(New_df,Y_data,test_size=0.25,random_state=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_3=best_model.best_estimator_\n",
    "model_3.fit(X_train,Y_train)\n",
    "pred=model_3.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.2849866924276957"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(Y_test,pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.366993910138537\n",
      "0.2849866924276957\n"
     ]
    }
   ],
   "source": [
    "print(model_3.score(X_train,Y_train))\n",
    "print(model_3.score(X_test,Y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interpretation : *dimension reduction techniques does'nt help in increasing model accuracy. Its making model overfitted which can't be a ideal model for prediction.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion :\n",
    "**After applying Xgboost regressor algorithm i.e. represented as \"model\".**<br>\n",
    "**Therefore, our Ideal model for prediction is \"model\". where we get accuracy of 60% and it can be used for test prediction.** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "                                                                                       -Rajeev Vhanhuve"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}