notes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basketball Statistics Prediction\n",
    "\n",
    "This projects aims at predicting basketball games scores and statistics, using French championship (LNB Jeep Elite) data. This notebook will be used as a raw note book, with planning, daily updates, notes etc.\n",
    "\n",
    "## Planning\n",
    "\n",
    "| Phase | Description                                                    |\n",
    "|-------|----------------------------------------------------------------|\n",
    "| 1     | Research on the data source, tools and methods                 |\n",
    "| 2     | Data retrieval                                                 |\n",
    "| 3     | Exploratory data analysis                                      |\n",
    "| 4     | Feature engineering                                            |\n",
    "| 5     | Model for game score prediction                                |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Phase 1 - Research of datasources, tools and methods\n",
    "\n",
    "**05/04 - 3:27pm**  \n",
    "Installation of a conda environment with Jupyter and pandas. Git repository creation .\n",
    "\n",
    "*Data-source(s) and data retrieval*:\n",
    "- It seems Jeep Elite data is not opened. \n",
    "- We will probably need to do web-scrapping with **Selenium**. Selenium with Python documentation here: https://selenium-python.readthedocs.io/.\n",
    "- List of all games here: https://www.lnb.fr/fr/pro-a/calendrier-proa-25.html.  \n",
    "- Season statistics per player avilable here : https://www.lnb.fr/fr/pro-a/statistiques-joueurs-39.html (both average and absolute values).\n",
    "- Game statistics here: https://www.lnb.fr/fr/proa/match/chalons-reims-pau-lacq-orthez-201615620.html#stats (available through list of games).\n",
    "- Players list here: https://www.lnb.fr/fr/pro-a/joueurs-proa-26.html\n",
    "- To get injuries: outliers in minutes played for a given player ?\n",
    "- Basketball refernce seems to have ProA statistics (https://www.basketball-reference.com/international/france-lnb-pro-a/) and a Python API (https://sportsreference.readthedocs.io/en/latest/)\n",
    "\n",
    "Features ideas:\n",
    "- 3 categories of features: historical matches, player performance indicators, and opposition information.\n",
    "- Team defense/offense averages.\n",
    "- Home/Away split.\n",
    "- Average physical measures of the players in each position (size/weight).\n",
    "- Injuries.\n",
    "- Offensive KPIs:\n",
    "    - Points, \n",
    "    - Assists,\n",
    "    - 3PM and 3PA (attempts to see tendency to play beyond the 3pts line),\n",
    "    - 2PM and 2PA (attempts to see tendency to play close to the paint),\n",
    "    - FTM and FTA (attempts to see tendency to play close to the paint and draw fouls),\n",
    "    - Offensive rebounds,\n",
    "    - Turnovers.\n",
    "- Defensive KPIs:\n",
    "    - Blocks,\n",
    "    - Steals,\n",
    "    - Defensive rebounds,\n",
    "    - Opponents turnovers,\n",
    "    - Opponents FG%.\n",
    "- New players added to roster or coach change.\n",
    "- Four Factors Rating (https://medium.com/@patrickoxford/an-introduction-to-four-factors-rating-3e9ee475ceed).\n",
    "- Lagged KPIs over past few games.\n",
    "- Team budget/salary mass (accessible here: [https://www.lnb.fr/fr/article/budgets-et-masses-salariales-des-clubs-de-jeep-elite-et-de-pro-b-14077.html](https://www.lnb.fr/fr/article/budgets-et-masses-salariales-des-clubs-de-jeep-elite-et-de-pro-b-14077.html)).\n",
    "- Arena attendance.\n",
    "   \n",
    "\n",
    "Model:\n",
    "- Basic model using Random Forest, with few teams statistics as features, applied to college basketball: https://towardsdatascience.com/predict-college-basketball-scores-in-30-lines-of-python-148f6bd71894\n",
    "- Research paper on basketball game prediction. Importance of features selection and their relationships rather than model selection. Limit to the prediction task itself. https://arxiv.org/abs/1310.3607v1\n",
    "- NBA prediction using mutliple models and comparing them. Importance of injuries? Using past seasons data? http://dionny.github.io/NBAPredictions/website/index.html#conclusions\n",
    "- Baseline model could be to select the best ranked team as winner and average points scored on the season.\n",
    "- Model to predict March Madness outcome with more subtle features such as experience and leadership: https://harvardsportsanalysis.wordpress.com/2012/03/14/survival-of-the-fittest-a-new-model-for-ncaa-tournament-prediction/\n",
    "\n",
    "Other links:\n",
    "- Onine forum about Basketball statistics: http://apbr.org/metrics/viewforum.php?f=2&sid=ddc8d5a6aece569fcc353bcac518233e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**08/04 - 10:37pm**  \n",
    "\n",
    "Red https://towardsdatascience.com/predict-college-basketball-scores-in-30-lines-of-python-148f6bd71894 article. Key takeawys:\n",
    "- Usage of Basketball reference Python API.  \n",
    "- Target variables: home/away scores.  \n",
    "- Straight forward implementation of random forests using scikit-learn.  \n",
    "- No feature engineering.  \n",
    "- Could be a good model to start.  \n",
    "  \n",
    "Red https://arxiv.org/abs/1310.3607v1. Key takeaways:\n",
    "- All features are standardize by number of possessions in the game. However the number of possessions is usually not directly computed live, so it is estimated. As a possession end either when a shot is taken, a rebound is taken, or if there is a turnover, it is usually estimated using those stats. The way they are weighted varies. This article does a nice recap: https://fansided.com/2015/12/21/nylon-calculus-101-possessions/. Here are two formulas used:\n",
    "    - In the paper -> $Possessions=0.96*(FGA−OR−TO+(0.475∗FTA))$\n",
    "    - On NBA.com -> $Possessions=(FGA+0.44*FTA–ORB+TOV)/2$    \n",
    "This however raises the question on if the computation should be different for FIBA basketball.\n",
    "- \"Four Factors\" are four stats that are said to be the most relevant in a team success in the litterature. They are: *Effective field goal percentage*, *Turnover percentage*, *Offensive Rebound Percentage* and *Free throw rate*. They should be considered as based features probably.\n",
    "- Usage of offensive and defensive ratings, but adjusted with the opponents ratings, as putting good offensive number should be more relevant aginst a good defensive rather than a poor defensive team. Adjusted offensive and defensive ratings should also be taken into account more.\n",
    "- A model to estmate win ratio: https://en.wikipedia.org/wiki/Pythagorean_expectation#Use_in_basketball\n",
    "- They estimated which were the most relevant features to make a prediction (measure accuracy among all models): \"We found the combi-nations of location and adjusted offensive and defensive efficiencies, and locationand Four Factors to work best\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**06/10 - 04:24pm** \n",
    "\n",
    "Very interesting article about how to quantify a coach impact's on a team win/loss ratio: [https://towardsdatascience.com/quantifying-the-contribution-of-nba-coaches-using-fixed-effects-56f77f22153a](https://towardsdatascience.com/quantifying-the-contribution-of-nba-coaches-using-fixed-effects-56f77f22153a). ALthough it is not of direct interest for this project, it first, shows the importance of the coach, which may be interesting to quantify somehow in our features, and secondly, talks about a statistic to quantify player's quality: the **Value Over Replacement Player (VORP)**. This page ([https://www.basketball-reference.com/about/bpm2.html](https://www.basketball-reference.com/about/bpm2.html)) gives a more in-depth look at how it is computed and how it can be used.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Phase 2 - Data retrieval\n",
    "\n",
    "**06/05 - 10:32pm**\n",
    "\n",
    "Installation of Selenium to pull data from [lnb.fr](lnb.fr).\n",
    "First pulled game data, with the following features :\n",
    "- date, \n",
    "- home team name, \n",
    "- home team score,\n",
    "- away team name,\n",
    "- away team score,\n",
    "- each quarters home team scores,\n",
    "- each quarters away team scores,\n",
    "- each of the eventual overtime home team scores,\n",
    "- each of the eventual overtime away team scores,\n",
    "- if the game is on TV\n",
    "\n",
    "This should allow to build a first model. Script available in notebook `data_retrieval.ipynb`.\n",
    "\n",
    "**05/09 - 10:00pm**  \n",
    "Pulled aditional game statistics for ech game. Statistics pulled:\n",
    "- minutes,\n",
    "- total assists, \n",
    "- total defensive rebounds,\n",
    "- total offensive rebounds,\n",
    "- total 2 points shots attempts, \n",
    "- total 2 points shots made, \n",
    "- total 3 points shots attempts, \n",
    "- total 3 points shots made, \n",
    "- total free throws attempts, \n",
    "- total free throws made, \n",
    "- total blocks,\n",
    "- total steals,\n",
    "- total turnovers,\n",
    "- total personal fouls,\n",
    "- total personal fouls drawn, \n",
    "\n",
    "and this for both the home and away team, of each game pulled in the step before."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Phase 3 - Data preparation\n",
    "\n",
    "**30/10 - 06:00pm**\n",
    "First clean of the data:\n",
    "- Removing all star game teams,\n",
    "- Adding the current w/l ratio of both teams at each game,\n",
    "\n",
    "To add more statistics, a good start is to add possessions. The formula to compute possessions seems to still be discussed. Red a couple more articles on the subjet:\n",
    "- [https://fansided.com/2015/12/21/nylon-calculus-101-possessions/](https://fansided.com/2015/12/21/nylon-calculus-101-possessions/)\n",
    "- [https://cbbstatshelp.com/efficiency/possessions/](https://cbbstatshelp.com/efficiency/possessions/)\n",
    "- [https://www.nbastuffer.com/analytics101/possession/](https://www.nbastuffer.com/analytics101/possession/)  \n",
    "Although the NBA formula is pretty clear ($(FGA + 0.44*FTA – ORB + TOV)$), it acocunts for all ways a possessions ends (FGA, TOV, FTA) minus when it is not ending (ORB). \n",
    "\n",
    "The mutiplier is here to balance the fact that free-throws are either shot alone (\"and one\"), by pairs (most of the cases), or by three. Thus 44% of the FA end the possession on average. It is not sure this also applies to FIBA basketball. If all FTAs are from 2 point shooting foul possessions, the coefficient should be 0.5, 0.333 for all 3 point shooting foul possessions, and 0 for all and-1 possessions. For the sake of simplicty, we will keep the NBA measure, as we don't have access to the French championship share of shooting foul types.\n",
    "\n",
    "Possession count is usefull to adjust stats per possession, but it also interesting to compute pace. See [this article about pace](https://medium.com/her-hoop-stats/what-does-that-stat-mean-pace-per-100-possession-ratings-explained-406d2b75e4cf). Basically it adjusts the number of possessions a team has in **40** minutes (48 if it was NBA instead of FIBA rules). \n",
    "\n",
    "**24/11 - 11:17pm**  \n",
    "Adding budgets and salary mass of all teams. It will be intersting to see which one contributes the most to a team success. Adding two seasons in case we retrieve more date afterward. Official source here:\n",
    "- [https://s.lnb.fr/file/202009/budgets-masses-salariales-previsionnels-actualises-2020-2021.pdf](https://s.lnb.fr/file/202009/budgets-masses-salariales-previsionnels-actualises-2020-2021.pdf)\n",
    "- [https://s.lnb.fr/file/201910/budgets-masses-salariales-2018-2019.pdf](https://s.lnb.fr/file/201910/budgets-masses-salariales-2018-2019.pdf)  \n",
    "I compiled them in `data/budgets.csv` and used the team names already retrieved in the games statistics to be able to join the game stats table and this one.\n",
    "\n",
    "**27/11 - 2:55pm**  \n",
    "Added a winning percentage instead of a win/loss count, as the win/loss count depends on the number of game played, while the winning percentage is a lot more agnostic of the advencement of the season."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ideas: \n",
    "- To add more data (games), we could add past season, but it would require to modify season depending stats, such as the W/L ratio.\n",
    "- Use LIME framework to explain why a team won."
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}