European Soccer Regression Analysis using scikit-learn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:2.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Regresssion with scikit-learn<br><br> using Soccer Dataset\n",
    "\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will again be using the open dataset from the popular site <a href=\"https://www.kaggle.com\">Kaggle</a> that we used in Week 1 for our example. \n",
    "\n",
    "Recall that this <a href=\"https://www.kaggle.com/hugomathien/soccer\">European Soccer Database</a> has more than 25,000 matches and more than 10,000 players for European professional soccer seasons from 2008 to 2016. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Import Libraries<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "import pandas as pd \n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from math import sqrt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Read Data from the Database into pandas\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create your connection.\n",
    "cnx = sqlite3.connect('database.sqlite')\n",
    "df = pd.read_sql_query(\"SELECT * FROM Player_Attributes\", cnx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>player_fifa_api_id</th>\n",
       "      <th>player_api_id</th>\n",
       "      <th>date</th>\n",
       "      <th>overall_rating</th>\n",
       "      <th>potential</th>\n",
       "      <th>preferred_foot</th>\n",
       "      <th>attacking_work_rate</th>\n",
       "      <th>defensive_work_rate</th>\n",
       "      <th>crossing</th>\n",
       "      <th>...</th>\n",
       "      <th>vision</th>\n",
       "      <th>penalties</th>\n",
       "      <th>marking</th>\n",
       "      <th>standing_tackle</th>\n",
       "      <th>sliding_tackle</th>\n",
       "      <th>gk_diving</th>\n",
       "      <th>gk_handling</th>\n",
       "      <th>gk_kicking</th>\n",
       "      <th>gk_positioning</th>\n",
       "      <th>gk_reflexes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>218353</td>\n",
       "      <td>505942</td>\n",
       "      <td>2016-02-18 00:00:00</td>\n",
       "      <td>67.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>right</td>\n",
       "      <td>medium</td>\n",
       "      <td>medium</td>\n",
       "      <td>49.0</td>\n",
       "      <td>...</td>\n",
       "      <td>54.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>218353</td>\n",
       "      <td>505942</td>\n",
       "      <td>2015-11-19 00:00:00</td>\n",
       "      <td>67.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>right</td>\n",
       "      <td>medium</td>\n",
       "      <td>medium</td>\n",
       "      <td>49.0</td>\n",
       "      <td>...</td>\n",
       "      <td>54.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>218353</td>\n",
       "      <td>505942</td>\n",
       "      <td>2015-09-21 00:00:00</td>\n",
       "      <td>62.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>right</td>\n",
       "      <td>medium</td>\n",
       "      <td>medium</td>\n",
       "      <td>49.0</td>\n",
       "      <td>...</td>\n",
       "      <td>54.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>218353</td>\n",
       "      <td>505942</td>\n",
       "      <td>2015-03-20 00:00:00</td>\n",
       "      <td>61.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>right</td>\n",
       "      <td>medium</td>\n",
       "      <td>medium</td>\n",
       "      <td>48.0</td>\n",
       "      <td>...</td>\n",
       "      <td>53.0</td>\n",
       "      <td>47.0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>218353</td>\n",
       "      <td>505942</td>\n",
       "      <td>2007-02-22 00:00:00</td>\n",
       "      <td>61.0</td>\n",
       "      <td>65.0</td>\n",
       "      <td>right</td>\n",
       "      <td>medium</td>\n",
       "      <td>medium</td>\n",
       "      <td>48.0</td>\n",
       "      <td>...</td>\n",
       "      <td>53.0</td>\n",
       "      <td>47.0</td>\n",
       "      <td>62.0</td>\n",
       "      <td>63.0</td>\n",
       "      <td>66.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 42 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  player_fifa_api_id  player_api_id                 date  overall_rating  \\\n",
       "0   1              218353         505942  2016-02-18 00:00:00            67.0   \n",
       "1   2              218353         505942  2015-11-19 00:00:00            67.0   \n",
       "2   3              218353         505942  2015-09-21 00:00:00            62.0   \n",
       "3   4              218353         505942  2015-03-20 00:00:00            61.0   \n",
       "4   5              218353         505942  2007-02-22 00:00:00            61.0   \n",
       "\n",
       "   potential preferred_foot attacking_work_rate defensive_work_rate  crossing  \\\n",
       "0       71.0          right              medium              medium      49.0   \n",
       "1       71.0          right              medium              medium      49.0   \n",
       "2       66.0          right              medium              medium      49.0   \n",
       "3       65.0          right              medium              medium      48.0   \n",
       "4       65.0          right              medium              medium      48.0   \n",
       "\n",
       "      ...       vision  penalties  marking  standing_tackle  sliding_tackle  \\\n",
       "0     ...         54.0       48.0     65.0             69.0            69.0   \n",
       "1     ...         54.0       48.0     65.0             69.0            69.0   \n",
       "2     ...         54.0       48.0     65.0             66.0            69.0   \n",
       "3     ...         53.0       47.0     62.0             63.0            66.0   \n",
       "4     ...         53.0       47.0     62.0             63.0            66.0   \n",
       "\n",
       "   gk_diving  gk_handling  gk_kicking  gk_positioning  gk_reflexes  \n",
       "0        6.0         11.0        10.0             8.0          8.0  \n",
       "1        6.0         11.0        10.0             8.0          8.0  \n",
       "2        6.0         11.0        10.0             8.0          8.0  \n",
       "3        5.0         10.0         9.0             7.0          7.0  \n",
       "4        5.0         10.0         9.0             7.0          7.0  \n",
       "\n",
       "[5 rows x 42 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(183978, 42)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['id', 'player_fifa_api_id', 'player_api_id', 'date', 'overall_rating',\n",
       "       'potential', 'preferred_foot', 'attacking_work_rate',\n",
       "       'defensive_work_rate', 'crossing', 'finishing', 'heading_accuracy',\n",
       "       'short_passing', 'volleys', 'dribbling', 'curve', 'free_kick_accuracy',\n",
       "       'long_passing', 'ball_control', 'acceleration', 'sprint_speed',\n",
       "       'agility', 'reactions', 'balance', 'shot_power', 'jumping', 'stamina',\n",
       "       'strength', 'long_shots', 'aggression', 'interceptions', 'positioning',\n",
       "       'vision', 'penalties', 'marking', 'standing_tackle', 'sliding_tackle',\n",
       "       'gk_diving', 'gk_handling', 'gk_kicking', 'gk_positioning',\n",
       "       'gk_reflexes'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Declare the Columns You Want to Use as Features\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "features = [\n",
    "       'potential', 'crossing', 'finishing', 'heading_accuracy',\n",
    "       'short_passing', 'volleys', 'dribbling', 'curve', 'free_kick_accuracy',\n",
    "       'long_passing', 'ball_control', 'acceleration', 'sprint_speed',\n",
    "       'agility', 'reactions', 'balance', 'shot_power', 'jumping', 'stamina',\n",
    "       'strength', 'long_shots', 'aggression', 'interceptions', 'positioning',\n",
    "       'vision', 'penalties', 'marking', 'standing_tackle', 'sliding_tackle',\n",
    "       'gk_diving', 'gk_handling', 'gk_kicking', 'gk_positioning',\n",
    "       'gk_reflexes']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Specify the Prediction Target\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "target = ['overall_rating']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Clean the Data<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = df.dropna()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Extract Features and Target ('overall_rating') Values into Separate Dataframes\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = df[features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y = df[target]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us look at a typical row from our features: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "potential             66.0\n",
       "crossing              49.0\n",
       "finishing             44.0\n",
       "heading_accuracy      71.0\n",
       "short_passing         61.0\n",
       "volleys               44.0\n",
       "dribbling             51.0\n",
       "curve                 45.0\n",
       "free_kick_accuracy    39.0\n",
       "long_passing          64.0\n",
       "ball_control          49.0\n",
       "acceleration          60.0\n",
       "sprint_speed          64.0\n",
       "agility               59.0\n",
       "reactions             47.0\n",
       "balance               65.0\n",
       "shot_power            55.0\n",
       "jumping               58.0\n",
       "stamina               54.0\n",
       "strength              76.0\n",
       "long_shots            35.0\n",
       "aggression            63.0\n",
       "interceptions         41.0\n",
       "positioning           45.0\n",
       "vision                54.0\n",
       "penalties             48.0\n",
       "marking               65.0\n",
       "standing_tackle       66.0\n",
       "sliding_tackle        69.0\n",
       "gk_diving              6.0\n",
       "gk_handling           11.0\n",
       "gk_kicking            10.0\n",
       "gk_positioning         8.0\n",
       "gk_reflexes            8.0\n",
       "Name: 2, dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.iloc[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us also display our target values: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>overall_rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>62.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>61.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>61.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>74.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>74.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>74.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>71.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>71.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>71.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>69.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>68.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183933</th>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183934</th>\n",
       "      <td>75.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183935</th>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183936</th>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183937</th>\n",
       "      <td>63.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183938</th>\n",
       "      <td>63.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183939</th>\n",
       "      <td>63.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183940</th>\n",
       "      <td>63.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183941</th>\n",
       "      <td>63.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183942</th>\n",
       "      <td>66.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183943</th>\n",
       "      <td>66.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183944</th>\n",
       "      <td>66.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183945</th>\n",
       "      <td>66.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183946</th>\n",
       "      <td>66.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183947</th>\n",
       "      <td>68.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183948</th>\n",
       "      <td>68.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183949</th>\n",
       "      <td>68.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183950</th>\n",
       "      <td>68.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183951</th>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183952</th>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183968</th>\n",
       "      <td>78.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183969</th>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183970</th>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183971</th>\n",
       "      <td>81.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183972</th>\n",
       "      <td>83.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183973</th>\n",
       "      <td>83.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183974</th>\n",
       "      <td>78.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183975</th>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183976</th>\n",
       "      <td>78.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183977</th>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>180354 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        overall_rating\n",
       "0                 67.0\n",
       "1                 67.0\n",
       "2                 62.0\n",
       "3                 61.0\n",
       "4                 61.0\n",
       "5                 74.0\n",
       "6                 74.0\n",
       "7                 73.0\n",
       "8                 73.0\n",
       "9                 73.0\n",
       "10                73.0\n",
       "11                74.0\n",
       "12                73.0\n",
       "13                71.0\n",
       "14                71.0\n",
       "15                71.0\n",
       "16                70.0\n",
       "17                70.0\n",
       "18                70.0\n",
       "19                70.0\n",
       "20                70.0\n",
       "21                70.0\n",
       "22                69.0\n",
       "23                69.0\n",
       "24                69.0\n",
       "25                69.0\n",
       "26                69.0\n",
       "27                69.0\n",
       "28                69.0\n",
       "29                68.0\n",
       "...                ...\n",
       "183933            76.0\n",
       "183934            75.0\n",
       "183935            77.0\n",
       "183936            77.0\n",
       "183937            63.0\n",
       "183938            63.0\n",
       "183939            63.0\n",
       "183940            63.0\n",
       "183941            63.0\n",
       "183942            66.0\n",
       "183943            66.0\n",
       "183944            66.0\n",
       "183945            66.0\n",
       "183946            66.0\n",
       "183947            68.0\n",
       "183948            68.0\n",
       "183949            68.0\n",
       "183950            68.0\n",
       "183951            67.0\n",
       "183952            67.0\n",
       "183968            78.0\n",
       "183969            81.0\n",
       "183970            81.0\n",
       "183971            81.0\n",
       "183972            83.0\n",
       "183973            83.0\n",
       "183974            78.0\n",
       "183975            77.0\n",
       "183976            78.0\n",
       "183977            80.0\n",
       "\n",
       "[180354 rows x 1 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Split the Dataset into Training and Test Datasets\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=324)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "(1) Linear Regression: Fit a model to the training set\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regressor = LinearRegression()\n",
    "regressor.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Perform Prediction using Linear Regression Model\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 66.51284879],\n",
       "       [ 79.77234615],\n",
       "       [ 66.57371825],\n",
       "       ..., \n",
       "       [ 69.23780133],\n",
       "       [ 64.58351696],\n",
       "       [ 73.6881185 ]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_prediction = regressor.predict(X_test)\n",
    "y_prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "What is the mean of the expected target value in test set ?\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>overall_rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>59517.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>68.635818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>7.041297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>33.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>64.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>69.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>73.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>94.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       overall_rating\n",
       "count    59517.000000\n",
       "mean        68.635818\n",
       "std          7.041297\n",
       "min         33.000000\n",
       "25%         64.000000\n",
       "50%         69.000000\n",
       "75%         73.000000\n",
       "max         94.000000"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Evaluate Linear Regression Accuracy using Root Mean Square Error\n",
    "\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "RMSE = sqrt(mean_squared_error(y_true = y_test, y_pred = y_prediction))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.805303046855208\n"
     ]
    }
   ],
   "source": [
    "print(RMSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "(2) Decision Tree Regressor: Fit a new regression model to the training set\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(criterion='mse', max_depth=20, max_features=None,\n",
       "           max_leaf_nodes=None, min_impurity_split=1e-07,\n",
       "           min_samples_leaf=1, min_samples_split=2,\n",
       "           min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "           splitter='best')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regressor = DecisionTreeRegressor(max_depth=20)\n",
    "regressor.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Perform Prediction using Decision Tree Regressor\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 62.        ,  84.        ,  62.38666667, ...,  71.        ,\n",
       "        62.        ,  73.        ])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_prediction = regressor.predict(X_test)\n",
    "y_prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "For comparision: What is the mean of the expected target value in test set ?\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>overall_rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>59517.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>68.635818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>7.041297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>33.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>64.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>69.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>73.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>94.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       overall_rating\n",
       "count    59517.000000\n",
       "mean        68.635818\n",
       "std          7.041297\n",
       "min         33.000000\n",
       "25%         64.000000\n",
       "50%         69.000000\n",
       "75%         73.000000\n",
       "max         94.000000"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Evaluate Decision Tree Regression Accuracy using Root Mean Square Error\n",
    "\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "RMSE = sqrt(mean_squared_error(y_true = y_test, y_pred = y_prediction))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.4490015098935145\n"
     ]
    }
   ],
   "source": [
    "print(RMSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}