MLB.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Schools.ipynb",
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyNllqn2+NDQA0vzNdWduCAq",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/bdg221/ml_project/blob/main/MLB.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **MLB Position Player Career of 7 Years**"
      ],
      "metadata": {
        "id": "XMUFeKa65TX8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Project Background and Description**"
      ],
      "metadata": {
        "id": "EdBdJJQUvN-1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Background\n",
        "\n",
        "After 3 years of a rookie contract, the salary for the player is determined by arbitration for the next 3 years. After the first 6 years of the players career, the player is considered a free agent and can get a new contract, including changing teams. Usually, the players subsequent contract includes an enormous salary increase (at least 2x the league minimum.)\n",
        "\n",
        "The goal of the model is to predict if a player's career will be last at least 7 years based off 2 years of stats. With this information, an MLB team could offer the player a subsequent contract early, locking in the player for a lower salary than if they waited until the player became a free agent.\n",
        "\n",
        "### Example:\n",
        "A player completes two years in the majors and is given a contract above the league minimum. If the player has a career of at least 7 years, then the team would save money in the last few years of the contract by locking the player into an early contract.  "
      ],
      "metadata": {
        "id": "6oc0YbkN_p_9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Project Description\n",
        "When a minor league player makes it to the major leagues, the MLB team has the rookie player under contract for 6 years. The first 3 years of the rookie contract is considered pre-arbitration where the MLB team determines the player's salary (usually league minimum which is $700,000 in 2022.) After 3 years in the majors, the player reaches an arbitration level where the team and the player submit salaries and a third party chooses one. Recently, the arbitration has been won by the players more than the teams. After the 6 years of the rookie contract, if an agreement is not met prior, the player is considered a free agent and can sign with any team for any amount.\n",
        "\n",
        "Citation: https://franchisesports.co.uk/mlb-contract-extension-rules-explained/\n",
        "\n",
        "In 2019 close to two thirds of MLB players had less than 3 years of experience which meant that the majority made close to the league minimum. Only a little more than one third of all MLB players make meaningifully above the minimum salary. Therefore, knowing if a player will have a longer career and being able to sign a player earlier, saves the MLB team money.\n",
        "\n",
        "Citation: https://www.thescore.com/mlb/news/2241177/manfreds-letter-ought-to-be-a-call-for-players-to-dig-in"
      ],
      "metadata": {
        "id": "TacdwHXC5SHz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Performance Metric\n",
        "\n",
        "Unfortunately, it is difficult to find median values for the first contracts rookies receive from MLB clubs. However, the current median salary for an MLB player is \\$1.2 million and the current league minimum is \\$700,000.\n",
        "\n",
        "If a team is able to offer a contract for multiple years at or below the current median, the team would save significant money in the later years of the contract when the model is correct in the prediction.\n",
        "\n",
        "Obviously, if the team is incorrect and gives the player a contract but his career is less than 7 years the team would not only be paying a higher salary than the player is worth, but potentially be paying when the player is no longer playing.\n",
        "\n",
        "If the player is predicted to have a shorter career and the prediction is incorrect, this could have 2 results. If the team wishes to keep the player on the team, the subsequent contract will be much higher and for potentially later years in the players career when performance decline is more expected and injuries are more common. However, if a team decides NOT to keep the player, the team would save money since it paid the minimum amount during the first 6 years.\n",
        "\n",
        "Please note that this model and metric are for average players and not outliers and superstars. Since we are referring to average players, the teams would be able to offer the player closer to the league median (still above league minimum.)\n",
        "\n",
        "While a 51% success rate would be successful, to further mitigate against losses, I would recommend at 70% success rate to ensure financial success."
      ],
      "metadata": {
        "id": "d3rKLqG-HwO7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#tables and visualizations\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "import datetime as dt\n",
        "\n",
        "#machine learning\n",
        "from sklearn.model_selection import train_test_split, GridSearchCV\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.pipeline import Pipeline \n",
        "from sklearn.compose import ColumnTransformer, make_column_selector\n",
        "from sklearn.impute import SimpleImputer\n",
        "from sklearn.preprocessing import OneHotEncoder, LabelBinarizer, StandardScaler\n",
        "from sklearn import config_context\n",
        "from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay"
      ],
      "metadata": {
        "id": "kYqt0dqLw9cN"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Load Data**\n",
        "\n",
        "We will need to load data from 2 CSV files. \n",
        "\n",
        "The PLAYERS csv file includes the players information including the date they debuted and the date of their last game.\n",
        "\n",
        "The BATTING csv file includes the batting stats for players which will be used for feature engineering."
      ],
      "metadata": {
        "id": "yIqIlVdGwiun"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "# Read in Players so we can calculate the lengths of careers\n",
        "players = pd.read_csv('https://raw.githubusercontent.com/chadwickbureau/baseballdatabank/master/core/People.csv')\n",
        "players = players.drop(columns=['birthYear','birthMonth', 'birthDay','birthCountry','birthState','birthCity','deathYear','deathMonth','deathDay','deathCity','deathState','deathCountry','nameGiven','weight','height','bats','throws', 'retroID', 'bbrefID'])\n",
        "display(players.head())\n",
        "players.info()\n",
        "\n",
        "# Read in Batting statistics \n",
        "batting = pd.read_csv('https://raw.githubusercontent.com/chadwickbureau/baseballdatabank/master/core/Batting.csv')\n",
        "#batting = pd.read_csv('https://raw.githubusercontent.com/bdg221/ml_project/main/Batting.csv')\n",
        "batting = batting.drop(columns=['teamID','stint','lgID','CS','IBB','HBP','SH','SF','GIDP'])\n",
        "display(batting.head())\n",
        "batting.info()"
      ],
      "metadata": {
        "id": "xrJI72t3FH4r",
        "outputId": "19b629f7-7f44-4eb5-c49b-15b15250d5ea",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 951
        }
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "    playerID nameFirst nameLast       debut   finalGame\n",
              "0  aardsda01     David  Aardsma  2004-04-06  2015-08-23\n",
              "1  aaronha01      Hank    Aaron  1954-04-13  1976-10-03\n",
              "2  aaronto01    Tommie    Aaron  1962-04-10  1971-09-26\n",
              "3   aasedo01       Don     Aase  1977-07-26  1990-10-03\n",
              "4   abadan01      Andy     Abad  2001-09-10  2006-04-13"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9ed783ca-d275-418a-9b42-63cb728fd2d4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>playerID</th>\n",
              "      <th>nameFirst</th>\n",
              "      <th>nameLast</th>\n",
              "      <th>debut</th>\n",
              "      <th>finalGame</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>aardsda01</td>\n",
              "      <td>David</td>\n",
              "      <td>Aardsma</td>\n",
              "      <td>2004-04-06</td>\n",
              "      <td>2015-08-23</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>aaronha01</td>\n",
              "      <td>Hank</td>\n",
              "      <td>Aaron</td>\n",
              "      <td>1954-04-13</td>\n",
              "      <td>1976-10-03</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>aaronto01</td>\n",
              "      <td>Tommie</td>\n",
              "      <td>Aaron</td>\n",
              "      <td>1962-04-10</td>\n",
              "      <td>1971-09-26</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>aasedo01</td>\n",
              "      <td>Don</td>\n",
              "      <td>Aase</td>\n",
              "      <td>1977-07-26</td>\n",
              "      <td>1990-10-03</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>abadan01</td>\n",
              "      <td>Andy</td>\n",
              "      <td>Abad</td>\n",
              "      <td>2001-09-10</td>\n",
              "      <td>2006-04-13</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9ed783ca-d275-418a-9b42-63cb728fd2d4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-9ed783ca-d275-418a-9b42-63cb728fd2d4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-9ed783ca-d275-418a-9b42-63cb728fd2d4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 20514 entries, 0 to 20513\n",
            "Data columns (total 5 columns):\n",
            " #   Column     Non-Null Count  Dtype \n",
            "---  ------     --------------  ----- \n",
            " 0   playerID   20514 non-null  object\n",
            " 1   nameFirst  20477 non-null  object\n",
            " 2   nameLast   20514 non-null  object\n",
            " 3   debut      20304 non-null  object\n",
            " 4   finalGame  20304 non-null  object\n",
            "dtypes: object(5)\n",
            "memory usage: 801.5+ KB\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "    playerID  yearID   G   AB   R   H  2B  3B  HR   RBI   SB  BB   SO\n",
              "0  abercda01    1871   1    4   0   0   0   0   0   0.0  0.0   0  0.0\n",
              "1   addybo01    1871  25  118  30  32   6   0   0  13.0  8.0   4  0.0\n",
              "2  allisar01    1871  29  137  28  40   4   5   0  19.0  3.0   2  5.0\n",
              "3  allisdo01    1871  27  133  28  44  10   2   2  27.0  1.0   0  2.0\n",
              "4  ansonca01    1871  25  120  29  39  11   3   0  16.0  6.0   2  1.0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-096b4cda-2663-41b6-acf2-b2430cd3193c\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>playerID</th>\n",
              "      <th>yearID</th>\n",
              "      <th>G</th>\n",
              "      <th>AB</th>\n",
              "      <th>R</th>\n",
              "      <th>H</th>\n",
              "      <th>2B</th>\n",
              "      <th>3B</th>\n",
              "      <th>HR</th>\n",
              "      <th>RBI</th>\n",
              "      <th>SB</th>\n",
              "      <th>BB</th>\n",
              "      <th>SO</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>abercda01</td>\n",
              "      <td>1871</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>addybo01</td>\n",
              "      <td>1871</td>\n",
              "      <td>25</td>\n",
              "      <td>118</td>\n",
              "      <td>30</td>\n",
              "      <td>32</td>\n",
              "      <td>6</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>13.0</td>\n",
              "      <td>8.0</td>\n",
              "      <td>4</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>allisar01</td>\n",
              "      <td>1871</td>\n",
              "      <td>29</td>\n",
              "      <td>137</td>\n",
              "      <td>28</td>\n",
              "      <td>40</td>\n",
              "      <td>4</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>19.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>2</td>\n",
              "      <td>5.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>allisdo01</td>\n",
              "      <td>1871</td>\n",
              "      <td>27</td>\n",
              "      <td>133</td>\n",
              "      <td>28</td>\n",
              "      <td>44</td>\n",
              "      <td>10</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "      <td>27.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ansonca01</td>\n",
              "      <td>1871</td>\n",
              "      <td>25</td>\n",
              "      <td>120</td>\n",
              "      <td>29</td>\n",
              "      <td>39</td>\n",
              "      <td>11</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>16.0</td>\n",
              "      <td>6.0</td>\n",
              "      <td>2</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-096b4cda-2663-41b6-acf2-b2430cd3193c')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-096b4cda-2663-41b6-acf2-b2430cd3193c button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-096b4cda-2663-41b6-acf2-b2430cd3193c');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 110495 entries, 0 to 110494\n",
            "Data columns (total 13 columns):\n",
            " #   Column    Non-Null Count   Dtype  \n",
            "---  ------    --------------   -----  \n",
            " 0   playerID  110495 non-null  object \n",
            " 1   yearID    110495 non-null  int64  \n",
            " 2   G         110495 non-null  int64  \n",
            " 3   AB        110495 non-null  int64  \n",
            " 4   R         110495 non-null  int64  \n",
            " 5   H         110495 non-null  int64  \n",
            " 6   2B        110495 non-null  int64  \n",
            " 7   3B        110495 non-null  int64  \n",
            " 8   HR        110495 non-null  int64  \n",
            " 9   RBI       109739 non-null  float64\n",
            " 10  SB        108127 non-null  float64\n",
            " 11  BB        110495 non-null  int64  \n",
            " 12  SO        108395 non-null  float64\n",
            "dtypes: float64(3), int64(9), object(1)\n",
            "memory usage: 11.0+ MB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Feature Engineering\n",
        "\n",
        "1. Only take players from 1990 - 2015.\n",
        "2. Only count a row if the player has > 0 ABs.\n",
        "3. Find the first 2 years (may be more than 2 rows) for a unique player and sum the values of the rows\n",
        "\n",
        "*The rookie year does not count for a player if he has fewer than 130 at bats\n",
        "\n",
        "\n",
        "\n",
        "RUNS - add the 2 years' worth of runs and divide by 2 years' worth of games to give average runs per game over 2 years\n",
        "\n",
        "HITS - average hits per game over 2 years\n",
        "\n",
        "DOUBLES - average doubles per game over 2 years\n",
        "\n",
        "TRIPLES - average triples per game over 2 years\n",
        "\n",
        "HOMERUNS - average homeruns per game over 2 years\n",
        "\n",
        "AVERAGE - total hits over 2 years divided by total at bats over 2 years\n",
        "\n",
        "RBI - average RBIs per game over 2 years\n",
        "\n",
        "WALKS - average walks per game over 2 years\n",
        "\n",
        "STRIKEOUTS - average strikeouts per game over 2 years\n",
        "\n",
        "STOLEN BASES - average stolen bases per game over 2 years\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "wo3mIb7Sb4iZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Change debut and finalGame to datetime objects for calculations\n",
        "players['finalGame'] = pd.to_datetime(players['finalGame'])\n",
        "players['debut'] = pd.to_datetime(players['debut'])\n",
        "\n",
        "# Remove players that fall outside the 1990-2015 range\n",
        "players = players[players['finalGame'].dt.year > 1991]\n",
        "players = players[players['debut'].dt.year < 2015]\n",
        "\n",
        "#Calculate the length of the careers in years\n",
        "players['length'] = (players['finalGame'] - players['debut'])/np.timedelta64(1,'Y')\n",
        "\n",
        "#Set a column called result with the answer to our prediction\n",
        "players['result'] = np.select([(players['length'] > 6), (players['length'] < 7)], [1, 0])\n",
        "players['name'] = players['nameFirst']+' '+players['nameLast']\n",
        "players = players.drop(columns=['nameFirst', 'nameLast','debut','finalGame','length'])\n",
        "display(players.head())\n",
        "players.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 380
        },
        "id": "EudRbHY-bMSK",
        "outputId": "edba936d-577f-4345-ddc0-23669cb0df04"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "     playerID  result           name\n",
              "0   aardsda01       1  David Aardsma\n",
              "4    abadan01       0      Andy Abad\n",
              "5    abadfe01       1  Fernando Abad\n",
              "14  abbotje01       0    Jeff Abbott\n",
              "15  abbotji01       1     Jim Abbott"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-ffa3dd48-63ea-4f57-9f79-22528121a7ab\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>playerID</th>\n",
              "      <th>result</th>\n",
              "      <th>name</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>aardsda01</td>\n",
              "      <td>1</td>\n",
              "      <td>David Aardsma</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>abadan01</td>\n",
              "      <td>0</td>\n",
              "      <td>Andy Abad</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>abadfe01</td>\n",
              "      <td>1</td>\n",
              "      <td>Fernando Abad</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>abbotje01</td>\n",
              "      <td>0</td>\n",
              "      <td>Jeff Abbott</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>abbotji01</td>\n",
              "      <td>1</td>\n",
              "      <td>Jim Abbott</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ffa3dd48-63ea-4f57-9f79-22528121a7ab')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-ffa3dd48-63ea-4f57-9f79-22528121a7ab button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-ffa3dd48-63ea-4f57-9f79-22528121a7ab');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Int64Index: 5663 entries, 0 to 20508\n",
            "Data columns (total 3 columns):\n",
            " #   Column    Non-Null Count  Dtype \n",
            "---  ------    --------------  ----- \n",
            " 0   playerID  5663 non-null   object\n",
            " 1   result    5663 non-null   int64 \n",
            " 2   name      5663 non-null   object\n",
            "dtypes: int64(1), object(2)\n",
            "memory usage: 177.0+ KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "\n",
        "# This query will return only stats for the years 1990 through 2015.\n",
        "# Also the rows will only be included if ABs is greater than 0\n",
        "batting = batting.query(\"yearID > 1989 and yearID < 2016 and AB > 130\")\n",
        "aggregations = {\n",
        "    'G':'sum',\n",
        "    'AB':'sum',\n",
        "    'R':'sum',\n",
        "    'H':'sum',\n",
        "    '2B':'sum',\n",
        "    '3B':'sum',\n",
        "    'HR':'sum',\n",
        "    'RBI':'sum',\n",
        "    'SB':'sum',\n",
        "    'BB':'sum',\n",
        "    'SO':'sum',\n",
        "}\n",
        "\n",
        "batting = batting.groupby('playerID').apply(lambda x:x.sort_values('yearID').head(2).agg(aggregations))\n",
        "batting = batting.query('AB > 200')\n",
        "\n",
        "\n",
        "finalset = pd.merge(players, batting, on='playerID')\n",
        "display(finalset.head())"
      ],
      "metadata": {
        "id": "82z9BqmD-g5b",
        "outputId": "6973d6ae-a7ce-4166-f1e8-151a3a72c266",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "    playerID  result                name      G     AB      R      H    2B  \\\n",
              "0  abbotje01       0         Jeff Abbott  169.0  459.0   64.0  127.0  29.0   \n",
              "1  abbotku01       1         Kurt Abbott  221.0  765.0  101.0  193.0  35.0   \n",
              "2  abercre01       0  Reggie Abercrombie  111.0  255.0   39.0   54.0  12.0   \n",
              "3  abernbr01       0     Brent Abernathy  196.0  767.0   89.0  194.0  35.0   \n",
              "4  abnersh01       0         Shawn Abner  188.0  392.0   38.0  103.0  19.0   \n",
              "\n",
              "     3B    HR   RBI    SB    BB     SO  \n",
              "0   2.0  15.0  70.0   5.0  30.0   66.0  \n",
              "1  10.0  26.0  93.0   7.0  52.0  208.0  \n",
              "2   2.0   5.0  24.0   6.0  18.0   78.0  \n",
              "3   5.0   7.0  73.0  18.0  52.0   81.0  \n",
              "4   1.0   2.0  31.0   3.0  21.0   63.0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6b5ab380-a32f-48a9-b43e-7502dc9f77b9\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>playerID</th>\n",
              "      <th>result</th>\n",
              "      <th>name</th>\n",
              "      <th>G</th>\n",
              "      <th>AB</th>\n",
              "      <th>R</th>\n",
              "      <th>H</th>\n",
              "      <th>2B</th>\n",
              "      <th>3B</th>\n",
              "      <th>HR</th>\n",
              "      <th>RBI</th>\n",
              "      <th>SB</th>\n",
              "      <th>BB</th>\n",
              "      <th>SO</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>abbotje01</td>\n",
              "      <td>0</td>\n",
              "      <td>Jeff Abbott</td>\n",
              "      <td>169.0</td>\n",
              "      <td>459.0</td>\n",
              "      <td>64.0</td>\n",
              "      <td>127.0</td>\n",
              "      <td>29.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>15.0</td>\n",
              "      <td>70.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>30.0</td>\n",
              "      <td>66.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>abbotku01</td>\n",
              "      <td>1</td>\n",
              "      <td>Kurt Abbott</td>\n",
              "      <td>221.0</td>\n",
              "      <td>765.0</td>\n",
              "      <td>101.0</td>\n",
              "      <td>193.0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>10.0</td>\n",
              "      <td>26.0</td>\n",
              "      <td>93.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>52.0</td>\n",
              "      <td>208.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>abercre01</td>\n",
              "      <td>0</td>\n",
              "      <td>Reggie Abercrombie</td>\n",
              "      <td>111.0</td>\n",
              "      <td>255.0</td>\n",
              "      <td>39.0</td>\n",
              "      <td>54.0</td>\n",
              "      <td>12.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>24.0</td>\n",
              "      <td>6.0</td>\n",
              "      <td>18.0</td>\n",
              "      <td>78.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>abernbr01</td>\n",
              "      <td>0</td>\n",
              "      <td>Brent Abernathy</td>\n",
              "      <td>196.0</td>\n",
              "      <td>767.0</td>\n",
              "      <td>89.0</td>\n",
              "      <td>194.0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>73.0</td>\n",
              "      <td>18.0</td>\n",
              "      <td>52.0</td>\n",
              "      <td>81.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>abnersh01</td>\n",
              "      <td>0</td>\n",
              "      <td>Shawn Abner</td>\n",
              "      <td>188.0</td>\n",
              "      <td>392.0</td>\n",
              "      <td>38.0</td>\n",
              "      <td>103.0</td>\n",
              "      <td>19.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>31.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>21.0</td>\n",
              "      <td>63.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6b5ab380-a32f-48a9-b43e-7502dc9f77b9')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6b5ab380-a32f-48a9-b43e-7502dc9f77b9 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6b5ab380-a32f-48a9-b43e-7502dc9f77b9');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "group by playerID filter based on start date + 2 years, operatoin is \"group by with summarize\" \n",
        "\n"
      ],
      "metadata": {
        "id": "iLhXtvxHltEj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "finalset['AVG'] = finalset['H'] / finalset['AB']\n",
        "finalset['R'] = finalset['R'] / finalset['G']\n",
        "finalset['H'] = finalset['H'] / finalset['G']\n",
        "finalset['2B'] = finalset['2B'] / finalset['G']\n",
        "finalset['3B'] = finalset['3B'] / finalset['G']\n",
        "finalset['HR'] = finalset['HR'] / finalset['G']\n",
        "finalset['RBI'] = finalset['RBI'] / finalset['G']\n",
        "finalset['SB'] = finalset['SB'] / finalset['G']\n",
        "finalset['BB'] = finalset['BB'] / finalset['G']\n",
        "finalset['SO'] = finalset['SO'] / finalset['G']\n",
        "backup = finalset\n",
        "finalset = finalset.drop(columns=['playerID','name','G','AB'])\n",
        "display(finalset.head())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "cSW3xJfj5-ZD",
        "outputId": "5b1b870e-c9b4-4514-9d7d-e9a75cdbb3d5"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "   result         R         H        2B        3B        HR       RBI  \\\n",
              "0       0  0.378698  0.751479  0.171598  0.011834  0.088757  0.414201   \n",
              "1       1  0.457014  0.873303  0.158371  0.045249  0.117647  0.420814   \n",
              "2       0  0.351351  0.486486  0.108108  0.018018  0.045045  0.216216   \n",
              "3       0  0.454082  0.989796  0.178571  0.025510  0.035714  0.372449   \n",
              "4       0  0.202128  0.547872  0.101064  0.005319  0.010638  0.164894   \n",
              "\n",
              "         SB        BB        SO       AVG  \n",
              "0  0.029586  0.177515  0.390533  0.276688  \n",
              "1  0.031674  0.235294  0.941176  0.252288  \n",
              "2  0.054054  0.162162  0.702703  0.211765  \n",
              "3  0.091837  0.265306  0.413265  0.252934  \n",
              "4  0.015957  0.111702  0.335106  0.262755  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-ea76ac23-c93b-4baa-a697-11d8864d69a4\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>result</th>\n",
              "      <th>R</th>\n",
              "      <th>H</th>\n",
              "      <th>2B</th>\n",
              "      <th>3B</th>\n",
              "      <th>HR</th>\n",
              "      <th>RBI</th>\n",
              "      <th>SB</th>\n",
              "      <th>BB</th>\n",
              "      <th>SO</th>\n",
              "      <th>AVG</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0.378698</td>\n",
              "      <td>0.751479</td>\n",
              "      <td>0.171598</td>\n",
              "      <td>0.011834</td>\n",
              "      <td>0.088757</td>\n",
              "      <td>0.414201</td>\n",
              "      <td>0.029586</td>\n",
              "      <td>0.177515</td>\n",
              "      <td>0.390533</td>\n",
              "      <td>0.276688</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0.457014</td>\n",
              "      <td>0.873303</td>\n",
              "      <td>0.158371</td>\n",
              "      <td>0.045249</td>\n",
              "      <td>0.117647</td>\n",
              "      <td>0.420814</td>\n",
              "      <td>0.031674</td>\n",
              "      <td>0.235294</td>\n",
              "      <td>0.941176</td>\n",
              "      <td>0.252288</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0.351351</td>\n",
              "      <td>0.486486</td>\n",
              "      <td>0.108108</td>\n",
              "      <td>0.018018</td>\n",
              "      <td>0.045045</td>\n",
              "      <td>0.216216</td>\n",
              "      <td>0.054054</td>\n",
              "      <td>0.162162</td>\n",
              "      <td>0.702703</td>\n",
              "      <td>0.211765</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0.454082</td>\n",
              "      <td>0.989796</td>\n",
              "      <td>0.178571</td>\n",
              "      <td>0.025510</td>\n",
              "      <td>0.035714</td>\n",
              "      <td>0.372449</td>\n",
              "      <td>0.091837</td>\n",
              "      <td>0.265306</td>\n",
              "      <td>0.413265</td>\n",
              "      <td>0.252934</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>0.202128</td>\n",
              "      <td>0.547872</td>\n",
              "      <td>0.101064</td>\n",
              "      <td>0.005319</td>\n",
              "      <td>0.010638</td>\n",
              "      <td>0.164894</td>\n",
              "      <td>0.015957</td>\n",
              "      <td>0.111702</td>\n",
              "      <td>0.335106</td>\n",
              "      <td>0.262755</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ea76ac23-c93b-4baa-a697-11d8864d69a4')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-ea76ac23-c93b-4baa-a697-11d8864d69a4 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-ea76ac23-c93b-4baa-a697-11d8864d69a4');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Split Data\n",
        "\n",
        "Splitting the data "
      ],
      "metadata": {
        "id": "i6niB0FJwCWy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class_column = 'result'\n",
        "random_seed = 86753\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(finalset.drop(columns=class_column), finalset[class_column],\n",
        "                                                   test_size=0.25, random_state=random_seed, stratify=finalset[class_column])"
      ],
      "metadata": {
        "id": "CjA-g5s1wSvR"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Pipeline\n",
        "\n",
        "The values have already been cleaned. However, I will try using a num_pipe with a StandardScaler following by using a LogisticRegression as the first test."
      ],
      "metadata": {
        "id": "HJ7ijfP_teVp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "\n",
        "#generate the whole modeling pipeline with preprocessing\n",
        "pipe = Pipeline(steps=[('mdl', LogisticRegression(penalty='elasticnet', solver='saga', tol=0.01, l1_ratio=0.1))])\n",
        "\n",
        "pipe.fit(X_train, y_train)\n",
        "pipe['mdl']\n",
        "prediction = pipe.predict(X_test)\n",
        "from sklearn.metrics import classification_report\n",
        "print(classification_report(y_test, prediction))"
      ],
      "metadata": {
        "id": "Uixoz8WAulh1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "999993d2-a05d-42f5-8e89-90806d4baafb"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      0.01      0.03        76\n",
            "           1       0.81      1.00      0.89       319\n",
            "\n",
            "    accuracy                           0.81       395\n",
            "   macro avg       0.90      0.51      0.46       395\n",
            "weighted avg       0.85      0.81      0.73       395\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "trying a different model to see difference in results\n"
      ],
      "metadata": {
        "id": "1pcW4yx9kbQy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pipe = Pipeline(steps=[('mdl', RandomForestClassifier(n_estimators=250, criterion=\"gini\", max_depth=10, min_samples_split=2, min_samples_leaf=2, min_weight_fraction_leaf=0, max_features=\"auto\", max_leaf_nodes=None))])\n",
        "pipe.fit(X_train, y_train)\n",
        "pipe['mdl']\n",
        "prediction = pipe.predict(X_test)\n",
        "from sklearn.metrics import classification_report\n",
        "print(classification_report(y_test, prediction))\n"
      ],
      "metadata": {
        "id": "EJGbUIwokkeT",
        "outputId": "dd97d72f-6969-4b2c-a65e-f22a9841a817",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.36      0.05      0.09        76\n",
            "           1       0.81      0.98      0.89       319\n",
            "\n",
            "    accuracy                           0.80       395\n",
            "   macro avg       0.59      0.52      0.49       395\n",
            "weighted avg       0.73      0.80      0.73       395\n",
            "\n"
          ]
        }
      ]
    }
  ]
}