statistics/4-Law-of-Large-Numbers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To open this notebook in Google Colab and start coding, click on the Colab icon below.\n",
    "\n",
    "<table style=\"border:2px solid orange\" align=\"left\">\n",
    "  <td style=\"border:2px solid orange \">\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/neuefische/ds-welcome-package/blob/main/statistics/4-Law-of-Large-Numbers.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Introduction to the Law of large numbers\n",
    "\n",
    "     \n",
    "In this notebook you will find the statistics basics that we require for the bootcamp.\n",
    "If you still have problems with one or the other term, we strongly recommend you to work on it a bit more. \n",
    "\n",
    "<img src=\"https://i.redd.it/e23ufi7yv7361.jpg\"\n",
    "     alt=\"Alice through the looking glass\"\n",
    "     style=\"float: left; margin-right: 10px; height: 300px\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Law of large numbers\n",
    "\n",
    "     \n",
    "In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value and will tend to become closer to the expected value as more trials are performed.\n",
    "\n",
    "The LLN is important because it guarantees stable long-term results for the averages of some random events. For example, while a casino may lose money in a single spin of the roulette wheel, its earnings will tend towards a predictable percentage over a large number of spins. Any winning streak by a player will eventually be overcome by the parameters of the game. It is important to remember that the law only applies (as the name indicates) when a large number of observations is considered. There is no principle that a small number of observations will coincide with the expected value or that a streak of one value will immediately be \"balanced\" by the others.\n",
    "\n",
    "$$ \\overline X_{n}= \\frac {1}{n} (X_{1}+\\cdots +X_{n})$$\n",
    "converges to the expected value\n",
    "\n",
    "$$ \\overline X_{n}\\,\\to \\,\\mu \\qquad \\textrm {for} \\qquad n\\to \\infty .$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Central limit theorem\n",
    "\n",
    "In probability theory, the central limit theorem (CLT) establishes that, in many situations, when independent random variables are added, their properly normalized sum tends toward a normal distribution (informally a bell curve) even if the original variables themselves are not normally distributed. The theorem is a key concept in probability theory because it implies that probabilistic and statistical methods that work for normal distributions can be applicable to many problems involving other types of distributions.\n",
    "\n",
    "If $X_{1},X_{2},...,X_{n}$ are random samples each of size $ n $ taken from a population with overall mean $\\mu$ and finite variance $ \\sigma ^{2}$ and if $\\bar {X}$ is the sample mean, the limiting form of the distribution of $ Z=\\left({\\frac {{\\bar {X}}_{n}-\\mu }{\\sigma /\\surd n}}\\right)$ as $ n\\to \\infty $, is the standard normal distribution.\n",
    "\n",
    "For example, suppose that a sample is obtained containing many observations, each observation being randomly generated in a way that does not depend on the values of the other observations, and that the arithmetic mean of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the probability distribution of the average will closely approximate a normal distribution. A simple example of this is that if one flips a coin many times, the probability of getting a given number of heads will approach a normal distribution, with the mean equal to half the total number of flips. At the limit of an infinite number of flips, it will equal a normal distribution.\n",
    "\n",
    "The central limit theorem has several variants. In its common form, the random variables must be identically distributed. In variants, convergence of the mean to the normal distribution also occurs for non-identical distributions or for non-independent observations, if they comply with certain conditions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "## weak law of large number\n",
    "# Step 1\n",
    "# create population with a gamma distribution\n",
    "shape, scale = 2., 2.  # mean=4, std=2*sqrt(2)\n",
    "s = np.random.gamma(shape, scale, 1000000)\n",
    "\n",
    "# Step 2\n",
    "samplemeanlist = [] # list of sample mean\n",
    "l = [] # list of smaple size, for x-axis of box plots\n",
    "numberofsample = 50 # number of sample in each sample size\n",
    "    \n",
    "# set sample size (i) between 100 to 8100, step by 500\n",
    "for i in range(100,8101,500):\n",
    "    # set x-axis\n",
    "    l.append(i)\n",
    "    # list of mean of each sample\n",
    "    ml = []\n",
    "    # sample 50 time.\n",
    "    for n in range(0,numberofsample):\n",
    "        # random pick from population with sample size = i\n",
    "        rs = random.choices(s, k=i)\n",
    "        # calculate the mean of each sample and save it in list of mean\n",
    "        ml.append(sum(rs)/i)  \n",
    "    \n",
    "    # save the 50 sample mean in samplemeanlist for box plots\n",
    "    samplemeanlist.append(ml)\n",
    "   \n",
    "# Step 3\n",
    "# set figure size\n",
    "boxplots = plt.figure(figsize=(20,10))\n",
    "# plot box plots of each sample mean\n",
    "plt.boxplot(samplemeanlist,labels = l)\n",
    "# show plot.\n",
    "boxplots.show()\n",
    "\n",
    "print(\"sample with 100 sample size,\" + \\\n",
    "      \"mean:\" + str(np.mean(samplemeanlist[0])) + \\\n",
    "      \", standard deviation: \"+ str(np.std(samplemeanlist[0])))\n",
    "print(\"sample with 8100 sample size,\" + \\\n",
    "      \"mean:\" + str(np.mean(samplemeanlist[16])) + \\\n",
    "      \", standard deviation: \"+ str(np.std(samplemeanlist[16])))\n",
    "\n",
    "# last hist plot\n",
    "histplot = plt.figure(figsize=(20,10))\n",
    "plt.hist(samplemeanlist[0], 10, density=True)\n",
    "plt.hist(samplemeanlist[16], 10, density=True)\n",
    "histplot.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}