07_ModellAccuracy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generalisierung funktionaler Zusammenhänge\n",
    "\n",
    "In den letzten Kapiteln haben wir Modelle kennen gelernt, die für quantitative und kategoriale Prognoseprobleme verwendet werden können. Ziel der Modellierung ist immer den tatsächlichen Zusammenhang zwischen den unabhängigen Variablen $x_1, ..., x_p$ und einer abhängigen Variablen $y$ möglichst gut abzubilden. Bisher haben wir immer den gesamten Datensatz verwendet, um die Modelle zu trainieren und haben verschiedene Kennzahlen kennen gelernt, die verwendet werden können, die Güte der Modelle für die Trainingsdaten zu bestimmen. Leider ist es jedoch so, dass eine hohe Modellgüte für die Trainingsdaten nicht unbedingt auch eine hohe Modellgüte für neue Testdaten zur Folge haben muss. Gelingt dies jedoch nicht, kann nicht davon ausgegangen werden, dass ein Modell den allgemein bestehenden Zusammenhang zwischen unabhängigen und abhängigen Variablen abbildet. Entsprechend ist es wichtig ein Modell immer für neue, nicht beim Trainieren des Modells verwendete, Daten zu evaluieren. Daten sind jedoch nicht in beliebiger Menge verfügbar, weshalb normalerweise ein Teil der Trainingsdaten nicht für die Schätzung der Modellparameter, sondern zur Evaluation der out-of-sample Performance verwendet wird. Im Folgenden verwenden wir die Termini Trainings- und Testfehler um die Güte eines Modells für Trainings- und Testdaten zu bezeichnen. \n",
    "\n",
    "## Einfacher Split der Daten\n",
    "\n",
    "Die einfachste Möglichkeit den Testfehler zu schätzen, besteht darin den gesamten Datensatz zufällig aufzuteilen und einen Teil zum Trainieren des Modells und zur Ermittlung des Trainingsfehlers zu bestimmen, während der Testfehler mit dem geschätzten Modell für die Testdaten ermittelt wird. In der nächsten Zelle kehren wir zum Advertising Datensatz aus dem Kapitel der linearen Regression zurück und teilen die Daten zufällig in Trainings- und Testdaten auf. Zunächst machen wir dies in gleichen Anteilen. Der Trainingsdatensatz wird zur Schätung der Parameter des linearen Regressionsmodells verwendet, im Anschluss berechnen wir den mittleren quadratischen Fehler (MSE) der Trainings- und Testdaten. Um ein Gefühl für die Schwankung bei der Schätzung des Testfehlers zu bekommen, führen wir diesen Vorgang $100$ Mal durch. Die untere Grafik zeigt die Histogramme der MSE des Trainings- und des Testdatensatzes. Zum einen fällt auf, dass es im Testdatensatz im Mittelwert zu einem etwas höheren MSE kommt als im Trainingsdatensatz (vertikale Linie). Zudem erkennen wir auch, dass die Variation des MSE im Testdatensatz relativ hoch ist. Dies ist dann problematisch, wenn wir uns vorstellen, dass wir unsere Beurteilung des Models nur auf eine zufällige Aufteilung der Daten stützen würden. Mit dem unten vorgestellten Vorgehen des mehrfachen Aufteilens haben wir dieses Problem nicht, jedoch ist das gezeigt Vorgehen aus einem anderen Grund kritisch zu sehen. In unserem Beispiel teilen wir die Daten gleichmäßig auf, was dazu führt, dass nur die Hälfte der Daten zum Trainieren des Modells verwendet werden kann. Dies erhöht die Varianz der Prognosen aus dem Modell. Zudem benötigen komplexere Modelle mehr Daten, um komplexere Zusammenhänge lernen zu können. Reduzieren wir den Anteil der Testdaten, erhöht sich hierdurch zwar auch die Schwankung des Testfehlers, jedoch kann der hierdurch entstehende  Nachteil durch das wie unten vorgestellte Verfahren mit mehrfachen Ziehungen reduziert werden. Ledglich ein kleines Problem bleibt dennoch bestehen. Wenn wir die Daten jedes Mal zufällig aufteilen, wird der geschätzte durchschnittliche Testfehler variieren. Daher wird oft die Kreuzvalidierung, und hierbei vor allem die k-fold Kreuzvalidierung dem hier gezeigten Vorgehen vorgezogen.\n",
    "\n",
    "```{note}\n",
    "Hinweis: In Python existieren mehrere Pakete, die zum Teil für den gleichen Zweck, wie das Schätzen statistischer Modelle verwendet werden können. Beispielsweise können verschiedene Regressionsmodelle sowohl mit dem statsmodels wie auc mit dem scikit-learn Paket geschätzt werden. Während das statsmodels Paket eher einen traditionellen statistischen Hintergrund hat und entsprechend viele Kennzahlen für die statistische Inferenz ausgegeben werden, sind im scikit-learn Paket oftmals nützliche Routinen aus dem Machine Learning Bereich verfügbar. Wir nützen immer Pakete, die für die vorgestellten Methoden am besten geeignet sind.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAHDCAYAAAAjsZnKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABEYElEQVR4nO3de1wWZf7/8fct6I2h4IljIpqa50OREmaKSiKRiZWmuYlW1paWLt8s2S2PbVhrZaVptQl2cC1NqdXSyAOuechDbGppaiBUgmkKgooG8/ujn/d2x8FB7psb8PV8POaxzTXXzHxm9q7rfjNzz1gMwzAEAAAAAAAuqY6rCwAAAAAAoKYgRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDtdDGjRtlsVi0cePGCq+bkZEhi8WipKQkh9d1uZKSkmSxWJSRkeHqUgAAQCnGjBmjli1buroMoEoQolFjXAxSFotFmzdvLrHcMAwFBQXJYrHotttus1uWn5+vadOmqXPnzvL09FTTpk3VvXt3TZw4UT/99JOt3/Tp0237KG3Kzs522PG89tpr1Sqo1lZLlizR3LlzXV0GAKAK8F2h4r755htNnz7dpX+ofvbZZ5WcnOyy/QMV5e7qAoCK8vDw0JIlS9S7d2+79tTUVP3www+yWq127RcuXFCfPn20f/9+xcbG6tFHH1V+fr727dunJUuWaOjQoQoMDLRbZ8GCBWrQoEGJfTdq1Mhhx/Haa6+pWbNmGjNmjMO2eVGfPn109uxZ1atXr8LrBgcH6+zZs6pbt67D63KFJUuWaO/evZo0aZKrSwEAVBG+K5j3zTffaMaMGQoPD3fZleRnn31Wd911l2JiYlyyf6CiCNGocW699VYtW7ZMr7zyitzd//cRXrJkiUJCQnT8+HG7/snJyfrqq6/03nvv6Z577rFbdu7cOZ0/f77EPu666y41a9bMOQdwGQoKCuTp6Wm6f506deTh4XFZ+7JYLJe9LgAA1cGV+F0BQNXhdm7UOCNHjtSJEyeUkpJiazt//ryWL19eYuCTpMOHD0uSbrrpphLLPDw85OXl5bxiy9CyZUvt27dPqampttu/wsPDJf3vVrTU1FQ98sgj8vX1VfPmzSVJR44c0SOPPKJ27dqpfv36atq0qYYNG1biFqzSfhMdHh6uzp0765tvvlG/fv101VVX6eqrr9bzzz9vt25pv4keM2aMGjRooB9//FExMTFq0KCBfHx89Pjjj6uoqMhu/RMnTujee++Vl5eXGjVqpNjYWP33v/81/Tvrffv2qX///qpfv76aN2+uZ555RsXFxSX6ffTRR4qOjlZgYKCsVqtat26tWbNm2dUTHh6u1atX68iRI7bzfPGv7OfPn9fUqVMVEhIib29veXp66uabb9aGDRsuWSMAoHqr7d8VJOnUqVOaNGmSgoKCZLVa1aZNGz333HMlxsylS5cqJCREDRs2lJeXl7p06aKXX35Z0m/fOYYNGyZJ6tevn20/l3qmSnJysjp37iwPDw917txZK1euLLXfnDlz1KtXLzVt2lT169dXSEiIli9fbtfHYrGooKBAixcvtu3/4pV3s997gKrGlWjUOC1btlRYWJj+9a9/KSoqSpL06aefKjc3VyNGjNArr7xi1z84OFiS9Pbbb+upp56SxWK55D5++eWXEm3u7u4Ou0Vr7ty5evTRR9WgQQP97W9/kyT5+fnZ9XnkkUfk4+OjqVOnqqCgQJK0Y8cObdmyRSNGjFDz5s2VkZGhBQsWKDw8XN98842uuuqqcvd78uRJDRo0SHfccYeGDx+u5cuX68knn1SXLl1s57IsRUVFioyMVGhoqObMmaPPP/9cL7zwglq3bq2HH35YklRcXKzBgwfryy+/1MMPP6z27dvro48+UmxsrKnzkp2drX79+unXX3/VlClT5OnpqTfeeEP169cv0TcpKUkNGjRQXFycGjRooPXr12vq1KnKy8vTP/7xD0nS3/72N+Xm5uqHH37QSy+9JEm2W+/y8vL0z3/+UyNHjtS4ceN0+vRpvfXWW4qMjNSXX36p7t27m6oZAFD91PbvCmfOnFHfvn31448/6qGHHlKLFi20ZcsWxcfH6+jRo7ZngaSkpGjkyJEaMGCAnnvuOUnSt99+qy+++EITJ05Unz599Nhjj+mVV17RX//6V3Xo0EGSbP9bms8++0x33nmnOnbsqISEBJ04cUJjx461/cH/915++WXdfvvtGjVqlM6fP6+lS5dq2LBhWrVqlaKjoyVJ77zzjh544AH17NlTDz74oCSpdevWkir/vQdwGgOoIRITEw1Jxo4dO4x58+YZDRs2NM6cOWMYhmEMGzbM6Nevn2EYhhEcHGxER0fb1jtz5ozRrl07Q5IRHBxsjBkzxnjrrbeMnJycEvuYNm2aIanUqV27dg49nk6dOhl9+/Yt8zh79+5t/Prrr3bLLh7v723dutWQZLz99tu2tg0bNhiSjA0bNtja+vbtW6JfYWGh4e/vb9x55522tvT0dEOSkZiYaGuLjY01JBkzZ8602/d1111nhISE2OY//PBDQ5Ixd+5cW1tRUZHRv3//EtsszaRJkwxJxvbt221tx44dM7y9vQ1JRnp6ernn4qGHHjKuuuoq49y5c7a26OhoIzg4uETfX3/91SgsLLRrO3nypOHn52fcd9995dYJAKierpTvCrNmzTI8PT2N7777zq59ypQphpubm5GZmWkYhmFMnDjR8PLyKvF94veWLVtW4jtDebp3724EBAQYp06dsrV99tlntnP3e38cq8+fP2907tzZ6N+/v127p6enERsbW2JfZr/3AFWN27lRIw0fPlxnz57VqlWrdPr0aa1atarU27MkqX79+tq+fbsmT54s6bcrmPfff78CAgL06KOPqrCwsMQ6H374oVJSUuymxMREpx7TH40bN05ubm52bb+/InvhwgWdOHFCbdq0UaNGjbR79+5LbrNBgwb605/+ZJuvV6+eevbsqe+//95UTX/+85/t5m+++Wa7ddesWaO6detq3LhxtrY6depo/Pjxprb/ySef6MYbb1TPnj1tbT4+Pho1alSJvr8/F6dPn9bx48d1880368yZM9q/f/8l9+Xm5mZ78FpxcbF++eUX/frrr7rhhhtMnUsAQPVWm78rLFu2TDfffLMaN26s48eP26aIiAgVFRVp06ZNkn57yFlBQYHdbe2VcfToUaWlpSk2Nlbe3t629ltuuUUdO3Ys0f/3Y/XJkyeVm5urm2++2fQ4W9nvPYCzcDs3aiQfHx9FRERoyZIlOnPmjIqKinTXXXeV2d/b21vPP/+8nn/+eR05ckTr1q3TnDlzNG/ePHl7e+uZZ56x69+nT58KPyzkl19+sXvwSP369e0GmIpq1apVibazZ88qISFBiYmJ+vHHH2UYhm1Zbm7uJbfZvHnzEreoNW7cWF9//fUl1/Xw8JCPj0+JdU+ePGmbP3LkiAICAkrcXtWmTZtLbv/i+qGhoSXa27VrV6Jt3759euqpp7R+/Xrl5eXZLTNzLiRp8eLFeuGFF7R//35duHDB1l7auQcA1Cy1+bvCwYMH9fXXX5cYly86duyYpN9+GvbBBx8oKipKV199tQYOHKjhw4dr0KBBFd6n9Ns4LUlt27Ytsaxdu3Ylgu2qVav0zDPPKC0tze4PEWZul5cq/70HcBZCNGqse+65R+PGjVN2draioqJM/wYpODhY9913n4YOHaprrrlG7733XomB8XLccccdSk1Ntc3HxsZW6t2Opf0O+NFHH1ViYqImTZqksLAweXt7y2KxaMSIEaU+fOuP/nhl+6LfD0oVXdcVTp06pb59+8rLy0szZ85U69at5eHhod27d+vJJ580dS7effddjRkzRjExMZo8ebJ8fX3l5uamhIQE2wNmAAA1W239rlBcXKxbbrlFTzzxRKnLr732WkmSr6+v0tLStHbtWn366af69NNPlZiYqNGjR2vx4sWXdQxm/ec//9Htt9+uPn366LXXXlNAQIDq1q2rxMRELVmyxNQ2Kvu9B3AWQjRqrKFDh+qhhx7Stm3b9P7771d4/caNG6t169bau3evQ+p54YUX7K7K/vF9kn9k9q+wv7d8+XLFxsbqhRdesLWdO3dOp06dqvC2nCE4OFgbNmzQmTNn7K5GHzp0yPT6Bw8eLNF+4MABu/mNGzfqxIkTWrFihfr06WNrT09PL7FuWed5+fLluuaaa7RixQq7PtOmTTNVKwCg+qut3xVat26t/Px8RUREXHKf9erV0+DBgzV48GAVFxfrkUce0euvv66nn35abdq0qdD3kYsPYDMzVn/44Yfy8PDQ2rVr7d7LXdot7+WN1dX5ew+uXPwmGjVWgwYNtGDBAk2fPl2DBw8us99///vfEu+DlH67Jembb74p9VbhyxESEqKIiAjbVNpvg37P09OzwoOAm5tbiavGr776aonXTLlKZGSkLly4oDfffNPWVlxcrPnz55ta/9Zbb9W2bdv05Zdf2tp+/vlnvffee3b9Ll4V//25OH/+vF577bUS2/T09Cz1lq/StrF9+3Zt3brVVK0AgOqvtn5XGD58uLZu3aq1a9eWWHbq1Cn9+uuvkn577eTv1alTR127dpUk2+3Vnp6etvUuJSAgQN27d9fixYvtxtaUlBR98803dn3d3NxksVjsvqNkZGQoOTnZ9HFW9+89uHJxJRo1mplXJ6WkpGjatGm6/fbbdeONN6pBgwb6/vvvtWjRIhUWFmr69Okl1lm+fLntVUi/d8stt5R4FdXlCgkJ0YIFC/TMM8+oTZs28vX1Vf/+/ctd57bbbtM777wjb29vdezYUVu3btXnn3+upk2bOqSmyoqJiVHPnj31f//3fzp06JDat2+vjz/+2PYakEv9tfuJJ57QO++8o0GDBmnixIm2V1wFBwfb/W67V69eaty4sWJjY/XYY4/JYrHonXfeKfW29JCQEL3//vuKi4tTjx491KBBAw0ePFi33XabVqxYoaFDhyo6Olrp6elauHChOnbsqPz8fMeeGACAy9TG7wqTJ0/Wxx9/rNtuu01jxoxRSEiICgoKtGfPHi1fvlwZGRlq1qyZHnjgAf3yyy/q37+/mjdvriNHjujVV19V9+7dba+x6t69u9zc3PTcc88pNzdXVqtV/fv3l6+vb6k1JSQkKDo6Wr1799Z9992nX375Ra+++qo6depkN35GR0frxRdf1KBBg3TPPffo2LFjmj9/vtq0aVPiWSwhISH6/PPP9eKLLyowMFCtWrVSaGhotf/egyuY6x4MDlTM719bUZ4/vrbi+++/N6ZOnWrceOONhq+vr+Hu7m74+PgY0dHRxvr16+3WLe+1FarA6x/MyM7ONqKjo42GDRsakmyvsCjvOE+ePGmMHTvWaNasmdGgQQMjMjLS2L9/vxEcHGz3aoiyXnHVqVOnEtuMjY21eyVFWa+48vT0LLHuxfP1ez///LNxzz33GA0bNjS8vb2NMWPGGF988YUhyVi6dOklz8vXX39t9O3b1/Dw8DCuvvpqY9asWcZbb71V4hVXX3zxhXHjjTca9evXNwIDA40nnnjCWLt2bYnjzs/PN+655x6jUaNGdq/fKC4uNp599lkjODjYsFqtxnXXXWesWrWqxPkAANQcV8p3BcMwjNOnTxvx8fFGmzZtjHr16hnNmjUzevXqZcyZM8c4f/68YRiGsXz5cmPgwIGGr6+vUa9ePaNFixbGQw89ZBw9etRuP2+++aZxzTXXGG5ubqaO4cMPPzQ6dOhgWK1Wo2PHjsaKFStKHT/feusto23btobVajXat29vJCYmlvrdYf/+/UafPn2M+vXrG5Js32nMfu8BqprFMEw8UQgAKiE5OVlDhw7V5s2bddNNN7m6HAAAAOCyEaIBONTZs2ftnixeVFSkgQMHaufOncrOzi71qeMAAABATcFvogE41KOPPqqzZ88qLCxMhYWFWrFihbZs2aJnn32WAA0AAIAajyvRABxqyZIleuGFF3To0CGdO3dObdq00cMPP6wJEya4ujQAAACg0gjRAAAAAACYxHuiAQAAAAAwiRANAAAAAIBJ1e7BYsXFxfrpp5/UsGFDWSwWV5cDAIAMw9Dp06cVGBioOnX4+7MjMN4DAKqTioz11S5E//TTTwoKCnJ1GQAAlJCVlaXmzZu7uoxagfEeAFAdmRnrq12IbtiwoaTfivfy8nJxNcCV6cz5X9Xz7+skSV/+bYCuqlft/lMBVKm8vDwFBQXZxihUHuM9riSMq0D1V5Gxvtr9G3zxli4vLy8GVcBF3M//qjrWqyT99u8igz3wG247dhzGe1xJGFeBmsPMWM8PuwAAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATKpwiN60aZMGDx6swMBAWSwWJScnl+jz7bff6vbbb5e3t7c8PT3Vo0cPZWZmOqJeAAAAAABcpsIhuqCgQN26ddP8+fNLXX748GH17t1b7du318aNG/X111/r6aefloeHR6WLBQAAAADAldwrukJUVJSioqLKXP63v/1Nt956q55//nlbW+vWrS+vOgAAAAAAqhGH/ia6uLhYq1ev1rXXXqvIyEj5+voqNDS01Fu+AQAAAACoaRwaoo8dO6b8/HzNnj1bgwYN0meffaahQ4fqjjvuUGpqaqnrFBYWKi8vz24CAAAAAKA6qvDt3OUpLi6WJA0ZMkR/+ctfJEndu3fXli1btHDhQvXt27fEOgkJCZoxY4YjywDwOy2nrK7U+h2nrnVQJfYyZkc7ZbsAgJqtsuOWMzF2AZAcfCW6WbNmcnd3V8eOHe3aO3ToUObTuePj45Wbm2ubsrKyHFkSAAAAAAAO49Ar0fXq1VOPHj104MABu/bvvvtOwcHBpa5jtVpltVodWQYAAAAAAE5R4RCdn5+vQ4cO2ebT09OVlpamJk2aqEWLFpo8ebLuvvtu9enTR/369dOaNWv073//Wxs3bnRk3QAAAAAAVLkKh+idO3eqX79+tvm4uDhJUmxsrJKSkjR06FAtXLhQCQkJeuyxx9SuXTt9+OGH6t27t+OqBgAAAADABSocosPDw2UYRrl97rvvPt13332XXRQAAAAAANWRQx8sBgAAAABAbUaIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATCJEAwAAAABgEiEaAIBaKiEhQT169FDDhg3l6+urmJgYHThwwK7PuXPnNH78eDVt2lQNGjTQnXfeqZycnHK3axiGpk6dqoCAANWvX18RERE6ePCgMw8FAIBqgxANAEAtlZqaqvHjx2vbtm1KSUnRhQsXNHDgQBUUFNj6/OUvf9G///1vLVu2TKmpqfrpp590xx13lLvd559/Xq+88ooWLlyo7du3y9PTU5GRkTp37pyzDwkAAJdzd3UBAADAOdasWWM3n5SUJF9fX+3atUt9+vRRbm6u3nrrLS1ZskT9+/eXJCUmJqpDhw7atm2bbrzxxhLbNAxDc+fO1VNPPaUhQ4ZIkt5++235+fkpOTlZI0aMcP6BAQDgQlyJBgDgCpGbmytJatKkiSRp165dunDhgiIiImx92rdvrxYtWmjr1q2lbiM9PV3Z2dl263h7eys0NLTMdSSpsLBQeXl5dhMAADURIRoAgCtAcXGxJk2apJtuukmdO3eWJGVnZ6tevXpq1KiRXV8/Pz9lZ2eXup2L7X5+fqbXkX77fba3t7dtCgoKqsTRAADgOoRoAACuAOPHj9fevXu1dOlSl+w/Pj5eubm5tikrK8sldQAAUFmEaAAAarkJEyZo1apV2rBhg5o3b25r9/f31/nz53Xq1Cm7/jk5OfL39y91Wxfb//gE7/LWkSSr1SovLy+7CQCAmogQDQBALWUYhiZMmKCVK1dq/fr1atWqld3ykJAQ1a1bV+vWrbO1HThwQJmZmQoLCyt1m61atZK/v7/dOnl5edq+fXuZ6wAAUJsQogEAqKXGjx+vd999V0uWLFHDhg2VnZ2t7OxsnT17VtJvDwS7//77FRcXpw0bNmjXrl0aO3aswsLC7J7M3b59e61cuVKSZLFYNGnSJD3zzDP6+OOPtWfPHo0ePVqBgYGKiYlxxWECAFCleMUVAAC11IIFCyRJ4eHhdu2JiYkaM2aMJOmll15SnTp1dOedd6qwsFCRkZF67bXX7PofOHDA9mRvSXriiSdUUFCgBx98UKdOnVLv3r21Zs0aeXh4OPV4AACoDgjRAFyi5ZTVri6hTBmzo11dAuAQhmFcso+Hh4fmz5+v+fPnm96OxWLRzJkzNXPmzErXCABATcPt3AAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATCJEAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCRCNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYVOEQvWnTJg0ePFiBgYGyWCxKTk4us++f//xnWSwWzZ07txIlAgAAAABQPVQ4RBcUFKhbt26aP39+uf1Wrlypbdu2KTAw8LKLAwAAAACgOnGv6ApRUVGKiooqt8+PP/6oRx99VGvXrlV0dPRlFwcAAAAAQHVS4RB9KcXFxbr33ns1efJkderU6ZL9CwsLVVhYaJvPy8tzdEkAAAAAADiEwx8s9txzz8nd3V2PPfaYqf4JCQny9va2TUFBQY4uCQAAAAAAh3BoiN61a5defvllJSUlyWKxmFonPj5eubm5tikrK8uRJQEAAAAA4DAODdH/+c9/dOzYMbVo0ULu7u5yd3fXkSNH9H//939q2bJlqetYrVZ5eXnZTQAAAAAAVEcO/U30vffeq4iICLu2yMhI3XvvvRo7dqwjdwUAAAAAQJWrcIjOz8/XoUOHbPPp6elKS0tTkyZN1KJFCzVt2tSuf926deXv76927dpVvloAAAAAAFyowiF6586d6tevn20+Li5OkhQbG6ukpCSHFQYAAAAAQHVT4RAdHh4uwzBM98/IyKjoLgAAAAAAqJYc/p5oAAAAAFWn5ZTVri6hTBmzo11dAuBwDn9PNAAAAAAAtRUhGgAAAAAAkwjRAAAAAACYRIgGAKAW27RpkwYPHqzAwEBZLBYlJyfbLbdYLKVO//jHP8rc5vTp00v0b9++vZOPBACA6oEQDQBALVZQUKBu3bpp/vz5pS4/evSo3bRo0SJZLBbdeeed5W63U6dOdutt3rzZGeUDAFDt8HRuAABqsaioKEVFRZW53N/f327+o48+Ur9+/XTNNdeUu113d/cS6wIAcCXgSjQAAJAk5eTkaPXq1br//vsv2ffgwYMKDAzUNddco1GjRikzM7MKKgQAwPW4Eg0AACRJixcvVsOGDXXHHXeU2y80NFRJSUlq166djh49qhkzZujmm2/W3r171bBhw1LXKSwsVGFhoW0+Ly/PobUDAFBVCNEAAECStGjRIo0aNUoeHh7l9vv97eFdu3ZVaGiogoOD9cEHH5R5FTshIUEzZsxwaL0AALgCt3MDAAD95z//0YEDB/TAAw9UeN1GjRrp2muv1aFDh8rsEx8fr9zcXNuUlZVVmXIBAHAZQjQAANBbb72lkJAQdevWrcLr5ufn6/DhwwoICCizj9VqlZeXl90EAEBNRIgGAKAWy8/PV1pamtLS0iRJ6enpSktLs3sQWF5enpYtW1bmVegBAwZo3rx5tvnHH39cqampysjI0JYtWzR06FC5ublp5MiRTj0WAACqA34TDQBALbZz507169fPNh8XFydJio2NVVJSkiRp6dKlMgyjzBB8+PBhHT9+3Db/ww8/aOTIkTpx4oR8fHzUu3dvbdu2TT4+Ps47EAAAqglCNAAAtVh4eLgMwyi3z4MPPqgHH3ywzOUZGRl280uXLnVEaQAA1Ejczg0AAAAAgElciQaAP2g5ZbWrSyhVxuxoV5cAAABwxeNKNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCpwiF606ZNGjx4sAIDA2WxWJScnGxbduHCBT355JPq0qWLPD09FRgYqNGjR+unn35yZM0AAAAAALhEhUN0QUGBunXrpvnz55dYdubMGe3evVtPP/20du/erRUrVujAgQO6/fbbHVIsAAAAAACu5F7RFaKiohQVFVXqMm9vb6WkpNi1zZs3Tz179lRmZqZatGhxeVUCAAAAAFANVDhEV1Rubq4sFosaNWpU6vLCwkIVFhba5vPy8pxdEgAAAAAAl8WpIfrcuXN68sknNXLkSHl5eZXaJyEhQTNmzHBmGYDTtZyy2tUlAAAAAKgCTns694ULFzR8+HAZhqEFCxaU2S8+Pl65ubm2KSsry1klAQAAAABQKU65En0xQB85ckTr168v8yq0JFmtVlmtVmeUAQAAAACAQzk8RF8M0AcPHtSGDRvUtGlTR+8CAAAAAACXqHCIzs/P16FDh2zz6enpSktLU5MmTRQQEKC77rpLu3fv1qpVq1RUVKTs7GxJUpMmTVSvXj3HVQ4AAAAAQBWrcIjeuXOn+vXrZ5uPi4uTJMXGxmr69On6+OOPJUndu3e3W2/Dhg0KDw+//EoBAAAAAHCxCofo8PBwGYZR5vLylgEAAAAAUJM57encAAAAAADUNoRoAAAAAABMIkQDAFCLbdq0SYMHD1ZgYKAsFouSk5Ptlo8ZM0YWi8VuGjRo0CW3O3/+fLVs2VIeHh4KDQ3Vl19+6aQjAACgeiFEAwBQixUUFKhbt26aP39+mX0GDRqko0eP2qZ//etf5W7z/fffV1xcnKZNm6bdu3erW7duioyM1LFjxxxdPgAA1Y7D3xMNAACqj6ioKEVFRZXbx2q1yt/f3/Q2X3zxRY0bN05jx46VJC1cuFCrV6/WokWLNGXKlErVCwBAdceVaAAArnAbN26Ur6+v2rVrp4cfflgnTpwos+/58+e1a9cuRURE2Nrq1KmjiIgIbd26tcz1CgsLlZeXZzcBAFATEaIBALiCDRo0SG+//bbWrVun5557TqmpqYqKilJRUVGp/Y8fP66ioiL5+fnZtfv5+Sk7O7vM/SQkJMjb29s2BQUFOfQ4AACoKtzODQDAFWzEiBG2f+7SpYu6du2q1q1ba+PGjRowYIDD9hMfH6+4uDjbfF5eHkEaAFAjcSUaAADYXHPNNWrWrJkOHTpU6vJmzZrJzc1NOTk5du05OTnl/q7aarXKy8vLbgIAoCYiRAMAAJsffvhBJ06cUEBAQKnL69Wrp5CQEK1bt87WVlxcrHXr1iksLKyqygQAwGUI0QAA1GL5+flKS0tTWlqaJCk9PV1paWnKzMxUfn6+Jk+erG3btikjI0Pr1q3TkCFD1KZNG0VGRtq2MWDAAM2bN882HxcXpzfffFOLFy/Wt99+q4cfflgFBQW2p3UDAFCb8ZtoAABqsZ07d6pfv362+Yu/S46NjdWCBQv09ddfa/HixTp16pQCAwM1cOBAzZo1S1ar1bbO4cOHdfz4cdv83XffrZ9//llTp05Vdna2unfvrjVr1pR42BgAALURIRoAgFosPDxchmGUuXzt2rWX3EZGRkaJtgkTJmjChAmVKQ0AgBqJ27kBAAAAADCJEA0AAAAAgEmEaAAAAAAATOI30ShVyymrXV0CAAAAAFQ7XIkGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJN4TzQAAMAVqOWU1a4uAQBqJK5EAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCRCNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkyocojdt2qTBgwcrMDBQFotFycnJdssNw9DUqVMVEBCg+vXrKyIiQgcPHnRUvQAAAAAAuEyFQ3RBQYG6deum+fPnl7r8+eef1yuvvKKFCxdq+/bt8vT0VGRkpM6dO1fpYgEAAAAAcCX3iq4QFRWlqKioUpcZhqG5c+fqqaee0pAhQyRJb7/9tvz8/JScnKwRI0ZUrloAAAAAAFzIob+JTk9PV3Z2tiIiImxt3t7eCg0N1datW0tdp7CwUHl5eXYTAAAAAADVkUNDdHZ2tiTJz8/Prt3Pz8+27I8SEhLk7e1tm4KCghxZEgAAAAAADuPyp3PHx8crNzfXNmVlZbm6JAAAAAAASuXQEO3v7y9JysnJsWvPycmxLfsjq9UqLy8vuwkAAAAAgOrIoSG6VatW8vf317p162xteXl52r59u8LCwhy5KwAAAAAAqlyFn86dn5+vQ4cO2ebT09OVlpamJk2aqEWLFpo0aZKeeeYZtW3bVq1atdLTTz+twMBAxcTEOLJuAAAAAACqXIVD9M6dO9WvXz/bfFxcnCQpNjZWSUlJeuKJJ1RQUKAHH3xQp06dUu/evbVmzRp5eHg4rmoAAAAAAFygwiE6PDxchmGUudxisWjmzJmaOXNmpQoDAAAAAKC6cfnTuQEAgPNs2rRJgwcPVmBgoCwWi5KTk23LLly4oCeffFJdunSRp6enAgMDNXr0aP3000/lbnP69OmyWCx2U/v27Z18JAAAVA+EaAAAarGCggJ169ZN8+fPL7HszJkz2r17t55++mnt3r1bK1as0IEDB3T77bdfcrudOnXS0aNHbdPmzZudUT4AANVOhW/nBgAANUdUVJSioqJKXebt7a2UlBS7tnnz5qlnz57KzMxUixYtytyuu7t7ma+vBACgNuNKNAAAsMnNzZXFYlGjRo3K7Xfw4EEFBgbqmmuu0ahRo5SZmVlu/8LCQuXl5dlNAADURIRoAAAgSTp37pyefPJJjRw5Ul5eXmX2Cw0NVVJSktasWaMFCxYoPT1dN998s06fPl3mOgkJCfL29rZNQUFBzjgEAACcjhANAAB04cIFDR8+XIZhaMGCBeX2jYqK0rBhw9S1a1dFRkbqk08+0alTp/TBBx+UuU58fLxyc3NtU1ZWlqMPAQCAKsFvogEAuMJdDNBHjhzR+vXry70KXZpGjRrp2muv1aFDh8rsY7VaZbVaK1sqAAAux5VoAACuYBcD9MGDB/X555+radOmFd5Gfn6+Dh8+rICAACdUCABA9UKIBgCgFsvPz1daWprS0tIkSenp6UpLS1NmZqYuXLigu+66Szt37tR7772noqIiZWdnKzs7W+fPn7dtY8CAAZo3b55t/vHHH1dqaqoyMjK0ZcsWDR06VG5ubho5cmRVHx4AAFWO27kBAKjFdu7cqX79+tnm4+LiJEmxsbGaPn26Pv74Y0lS9+7d7dbbsGGDwsPDJUmHDx/W8ePHbct++OEHjRw5UidOnJCPj4969+6tbdu2ycfHx7kHAwBANUCIBgCgFgsPD5dhGGUuL2/ZRRkZGXbzS5curWxZAADUWNzODQAAAACASYRoAAAAAABM4nZuAAAAAE7RcspqV5dQqozZ0a4uATUYV6IBAAAAADCJEA0AAAAAgEmEaAAAAAAATCJEAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCRCNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMcHqKLior09NNPq1WrVqpfv75at26tWbNmyTAMR+8KAAAAAIAq5e7oDT733HNasGCBFi9erE6dOmnnzp0aO3asvL299dhjjzl6dwAAAAAAVBmHh+gtW7ZoyJAhio6OliS1bNlS//rXv/Tll186elcAAAAAAFQph9/O3atXL61bt07fffedJOm///2vNm/erKioqFL7FxYWKi8vz24CAAAAAKA6cviV6ClTpigvL0/t27eXm5ubioqK9Pe//12jRo0qtX9CQoJmzJjh6DIAoNZpOWW1q0soU8bsaFeXAAAAUCUcfiX6gw8+0HvvvaclS5Zo9+7dWrx4sebMmaPFixeX2j8+Pl65ubm2KSsry9ElAQAAAADgEA6/Ej158mRNmTJFI0aMkCR16dJFR44cUUJCgmJjY0v0t1qtslqtji4DAAAAAACHc/iV6DNnzqhOHfvNurm5qbi42NG7AgAAAACgSjk8RA8ePFh///vftXr1amVkZGjlypV68cUXNXToUEfvCgAAXMKmTZs0ePBgBQYGymKxKDk52W65YRiaOnWqAgICVL9+fUVEROjgwYOX3O78+fPVsmVLeXh4KDQ0lLdwAACuGA4P0a+++qruuusuPfLII+rQoYMef/xxPfTQQ5o1a5ajdwUAAC6hoKBA3bp10/z580td/vzzz+uVV17RwoULtX37dnl6eioyMlLnzp0rc5vvv/++4uLiNG3aNO3evVvdunVTZGSkjh075qzDAACg2nD4b6IbNmyouXPnau7cuY7eNAAAqKCoqKgyXzNpGIbmzp2rp556SkOGDJEkvf322/Lz81NycrLt+SZ/9OKLL2rcuHEaO3asJGnhwoVavXq1Fi1apClTpjjnQAAAqCYcfiUaAADUDOnp6crOzlZERIStzdvbW6Ghodq6dWup65w/f167du2yW6dOnTqKiIgocx0AAGoTh1+JBgAANUN2drYkyc/Pz67dz8/PtuyPjh8/rqKiolLX2b9/f5n7KiwsVGFhoW0+Ly/vcssGAMClCNEAAMDpEhISNGPGDFeXAQCSpJZTVru6hFJlzI52dQkwgdu5AQC4Qvn7+0uScnJy7NpzcnJsy/6oWbNmcnNzq9A6khQfH6/c3FzblJWVVcnqAQBwDUI0AABXqFatWsnf31/r1q2zteXl5Wn79u0KCwsrdZ169eopJCTEbp3i4mKtW7euzHUkyWq1ysvLy24CAKAm4nZuAABqsfz8fB06dMg2n56errS0NDVp0kQtWrTQpEmT9Mwzz6ht27Zq1aqVnn76aQUGBiomJsa2zoABAzR06FBNmDBBkhQXF6fY2FjdcMMN6tmzp+bOnauCggLb07oBAKjNCNEAANRiO3fuVL9+/WzzcXFxkqTY2FglJSXpiSeeUEFBgR588EGdOnVKvXv31po1a+Th4WFb5/Dhwzp+/Lht/u6779bPP/+sqVOnKjs7W927d9eaNWtKPGwMAIDaiBANAEAtFh4eLsMwylxusVg0c+ZMzZw5s8w+GRkZJdomTJhguzINAMCVhN9EAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCRCNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAExyd3UBV7KWU1a7ugQAAAAAQAVwJRoAAAAAAJMI0QAAAAAAmMTt3AAAAE7CT7dqF0f8/9lx6loHVALAlbgSDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATCJEAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCSnhOgff/xRf/rTn9S0aVPVr19fXbp00c6dO52xKwAAAAAAqoy7ozd48uRJ3XTTTerXr58+/fRT+fj46ODBg2rcuLGjdwUAAAAAQJVyeIh+7rnnFBQUpMTERFtbq1atHL0bAAAAAACqnMNv5/744491ww03aNiwYfL19dV1112nN99809G7AQAAAACgyjk8RH///fdasGCB2rZtq7Vr1+rhhx/WY489psWLF5fav7CwUHl5eXYTAAAAAADVkcNv5y4uLtYNN9ygZ599VpJ03XXXae/evVq4cKFiY2NL9E9ISNCMGTMcXQYAoAq1nLLa1SWUKmN2tKtLAAAAtYzDr0QHBASoY8eOdm0dOnRQZmZmqf3j4+OVm5trm7KyshxdEgAAAAAADuHwEH3TTTfpwIEDdm3fffedgoODS+1vtVrl5eVlNwEAgKrRsmVLWSyWEtP48eNL7Z+UlFSir4eHRxVXDQCA6zj8du6//OUv6tWrl5599lkNHz5cX375pd544w298cYbjt4VAACopB07dqioqMg2v3fvXt1yyy0aNmxYmet4eXnZ/cHcYrE4tUYAAKoTh4foHj16aOXKlYqPj9fMmTPVqlUrzZ07V6NGjXL0rgAAQCX5+PjYzc+ePVutW7dW3759y1zHYrHI39/f2aUBAFAtOTxES9Jtt92m2267zRmbBgAATnL+/Hm9++67iouLK/fqcn5+voKDg1VcXKzrr79ezz77rDp16lTutgsLC1VYWGib520cAICayuG/iQYAADVTcnKyTp06pTFjxpTZp127dlq0aJE++ugjvfvuuyouLlavXr30ww8/lLvthIQEeXt726agoCAHVw8AQNUgRAMAAEnSW2+9paioKAUGBpbZJywsTKNHj1b37t3Vt29frVixQj4+Pnr99dfL3TZv4wAA1BZOuZ0bAADULEeOHNHnn3+uFStWVGi9unXr6rrrrtOhQ4fK7We1WmW1WitTIgAA1QJXogEAgBITE+Xr66vo6OgKrVdUVKQ9e/YoICDASZUBAFC9EKIBALjCFRcXKzExUbGxsXJ3t79JbfTo0YqPj7fNz5w5U5999pm+//577d69W3/605905MgRPfDAA1VdNgAALsHt3AAAXOE+//xzZWZm6r777iuxLDMzU3Xq/O9v7idPntS4ceOUnZ2txo0bKyQkRFu2bFHHjh2rsmQAAFyGEA0AwBVu4MCBMgyj1GUbN260m3/ppZf00ksvVUFVAABUT9zODQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATCJEAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCRCNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJOcHqJnz54ti8WiSZMmOXtXAAAAAAA4lVND9I4dO/T666+ra9euztwNAAAAAABVwmkhOj8/X6NGjdKbb76pxo0bO2s3AAAAAABUGaeF6PHjxys6OloRERHO2gUAAAAAAFXK3RkbXbp0qXbv3q0dO3Zcsm9hYaEKCwtt83l5ec4oCQAAAACASnP4leisrCxNnDhR7733njw8PC7ZPyEhQd7e3rYpKCjI0SUBAIAyTJ8+XRaLxW5q3759uessW7ZM7du3l4eHh7p06aJPPvmkiqoFAMD1HB6id+3apWPHjun666+Xu7u73N3dlZqaqldeeUXu7u4qKiqy6x8fH6/c3FzblJWV5eiSAABAOTp16qSjR4/aps2bN5fZd8uWLRo5cqTuv/9+ffXVV4qJiVFMTIz27t1bhRUDAOA6Dr+de8CAAdqzZ49d29ixY9W+fXs9+eSTcnNzs1tmtVpltVodXQYAADDJ3d1d/v7+pvq+/PLLGjRokCZPnixJmjVrllJSUjRv3jwtXLjQmWUCAFAtODxEN2zYUJ07d7Zr8/T0VNOmTUu0AwAA1zt48KACAwPl4eGhsLAwJSQkqEWLFqX23bp1q+Li4uzaIiMjlZycXO4+eAYKAKC2cOp7ogEAQPUWGhqqpKQkrVmzRgsWLFB6erpuvvlmnT59utT+2dnZ8vPzs2vz8/NTdnZ2ufvhGSgAgNrCKU/n/qONGzdWxW4AAEAFRUVF2f65a9euCg0NVXBwsD744APdf//9DttPfHy83RXsvLw8gjQAoEaqkhANAABqhkaNGunaa6/VoUOHSl3u7++vnJwcu7acnJxL/qaaZ6AAAGoLbucGAAA2+fn5Onz4sAICAkpdHhYWpnXr1tm1paSkKCwsrCrKAwDA5QjRAABcwR5//HGlpqYqIyNDW7Zs0dChQ+Xm5qaRI0dKkkaPHq34+Hhb/4kTJ2rNmjV64YUXtH//fk2fPl07d+7UhAkTXHUIAABUKW7nBgDgCvbDDz9o5MiROnHihHx8fNS7d29t27ZNPj4+kqTMzEzVqfO/v7n36tVLS5Ys0VNPPaW//vWvatu2rZKTk3kDBwDgikGIBgDgCrZ06dJyl5f2cNBhw4Zp2LBhTqoIAIDqjdu5AQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASe6uLgAAAKCyWk5Z7eoSAABXCK5EAwAAAABgEiEaAAAAAACTCNEAAAAAAJhEiAYAAAAAwCRCNAAAAAAAJhGiAQAAAAAwiRANAAAAAIBJhGgAAAAAAEwiRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmOTwEJ2QkKAePXqoYcOG8vX1VUxMjA4cOODo3QAAAAAAUOUcHqJTU1M1fvx4bdu2TSkpKbpw4YIGDhyogoICR+8KAAAAAIAq5fAQvWbNGo0ZM0adOnVSt27dlJSUpMzMTO3atcvRuwIAAJV0OXeQJSUlyWKx2E0eHh5VVDEAAK7l7uwd5ObmSpKaNGlS6vLCwkIVFhba5vPy8pxdEgAA+P8u3kHWo0cP/frrr/rrX/+qgQMH6ptvvpGnp2eZ63l5edmFbYvFUhXlAgDgck4N0cXFxZo0aZJuuukmde7cudQ+CQkJmjFjhjPLAAAAZVizZo3dfFJSknx9fbVr1y716dOnzPUsFov8/f2dXR4AANWOU5/OPX78eO3du1dLly4ts098fLxyc3NtU1ZWljNLAgAA5bjUHWQX5efnKzg4WEFBQRoyZIj27dtXFeUBAOByTrsSPWHCBK1atUqbNm1S8+bNy+xntVpltVqdVQYAADDJzB1kktSuXTstWrRIXbt2VW5urubMmaNevXpp3759ZY75/HwLAFBbODxEG4ahRx99VCtXrtTGjRvVqlUrR+8CAAA4wcU7yDZv3lxuv7CwMIWFhdnme/XqpQ4dOuj111/XrFmzSl2Hn28BAGoLh9/OPX78eL377rtasmSJGjZsqOzsbGVnZ+vs2bOO3hUAAHCQi3eQbdiwodw7yEpTt25dXXfddTp06FCZffj5FgCgtnB4iF6wYIFyc3MVHh6ugIAA2/T+++87elcAAKCSDMPQhAkTtHLlSq1fv/6y7iArKirSnj17FBAQUGYfq9UqLy8vuwkAgJrIKbdzAwCAmmH8+PFasmSJPvroI9sdZJLk7e2t+vXrS5JGjx6tq6++WgkJCZKkmTNn6sYbb1SbNm106tQp/eMf/9CRI0f0wAMPuOw4AACoKk5/TzQAAKi+FixYIEkKDw+3a09MTNSYMWMkSZmZmapT5383r508eVLjxo1Tdna2GjdurJCQEG3ZskUdO3asqrIBAHAZQjQAAFcwM3eQbdy40W7+pZde0ksvveSkigAAqN6c+p5oAAAAAABqE0I0AAAAAAAmcTs3AAAAAFQDLaesdnUJZcqYHe3qEqoNrkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATCJEAwAAAABgEiEaAAAAAACTCNEAAAAAAJjk7uoCqkLLKatdXQIAAAAAoBbgSjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGDSFfF0bgAAAADA5auubzzKmB1d5fvkSjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADAJEI0AAAAAAAmEaIBAAAAADCJEA0AAAAAgEmEaAAAAAAATHJaiJ4/f75atmwpDw8PhYaG6ssvv3TWrgAAQCVVdNxetmyZ2rdvLw8PD3Xp0kWffPJJFVUKAIBrOSVEv//++4qLi9O0adO0e/dudevWTZGRkTp27JgzdgcAACqhouP2li1bNHLkSN1///366quvFBMTo5iYGO3du7eKKwcAoOo5JUS/+OKLGjdunMaOHauOHTtq4cKFuuqqq7Ro0SJn7A4AAFRCRcftl19+WYMGDdLkyZPVoUMHzZo1S9dff73mzZtXxZUDAFD13B29wfPnz2vXrl2Kj4+3tdWpU0cRERHaunVrif6FhYUqLCy0zefm5kqS8vLyHFZTceEZh20LAFBzOGosubgdwzAcsr3qpKLjtiRt3bpVcXFxdm2RkZFKTk4ucz/OHu8Z6wHgyuSKsd7hIfr48eMqKiqSn5+fXbufn5/2799fon9CQoJmzJhRoj0oKMjRpQEArjDecx27vdOnT8vb29uxG3Wxio7bkpSdnV1q/+zs7DL3w3gPAHAGV4z1Dg/RFRUfH2/31+zi4mL98ssvatq0qSwWiwsrq13y8vIUFBSkrKwseXl5ubqcKwbnvepxzl2jtp93wzB0+vRpBQYGurqUGqs6jve1/XPrbJy/yuH8VQ7nr3I4fyVVZKx3eIhu1qyZ3NzclJOTY9eek5Mjf3//Ev2tVqusVqtdW6NGjRxdFv4/Ly8v/kVxAc571eOcu0ZtPu+17Qr0RRUdtyXJ39+/Qv2l6j3e1+bPbVXg/FUO569yOH+Vw/mzZ3asd/iDxerVq6eQkBCtW7fO1lZcXKx169YpLCzM0bsDAACVcDnjdlhYmF1/SUpJSWGcBwBcEZxyO3dcXJxiY2N1ww03qGfPnpo7d64KCgo0duxYZ+wOAABUwqXG7dGjR+vqq69WQkKCJGnixInq27evXnjhBUVHR2vp0qXauXOn3njjDVceBgAAVcIpIfruu+/Wzz//rKlTpyo7O1vdu3fXmjVrSjyEBFXHarVq2rRpJW6lg3Nx3qse59w1OO8126XG7czMTNWp87+b13r16qUlS5boqaee0l//+le1bdtWycnJ6ty5s6sO4bLwua0czl/lcP4qh/NXOZy/yrEYtfF9HQAAAAAAOIHDfxMNAAAAAEBtRYgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQXQskJCSoR48eatiwoXx9fRUTE6MDBw5ccr1ly5apffv28vDwUJcuXfTJJ59UQbW1x+Wc96SkJFksFrvJw8Ojiiqu+RYsWKCuXbvKy8tLXl5eCgsL06efflruOnzOK6+i553POaoDxsbKYYyrHMarymHccazZs2fLYrFo0qRJ5fbjM2geIboWSE1N1fjx47Vt2zalpKTowoULGjhwoAoKCspcZ8uWLRo5cqTuv/9+ffXVV4qJiVFMTIz27t1bhZXXbJdz3iXJy8tLR48etU1HjhypooprvubNm2v27NnatWuXdu7cqf79+2vIkCHat29fqf35nDtGRc+7xOccrsfYWDmMcZXDeFU5jDuOs2PHDr3++uvq2rVruf34DFaQgVrn2LFjhiQjNTW1zD7Dhw83oqOj7dpCQ0ONhx56yNnl1VpmzntiYqLh7e1ddUVdARo3bmz885//LHUZn3PnKe+88zlHdcTYWDmMcZXHeFU5jDsVd/r0aaNt27ZGSkqK0bdvX2PixIll9uUzWDFcia6FcnNzJUlNmjQps8/WrVsVERFh1xYZGamtW7c6tbbazMx5l6T8/HwFBwcrKCjokn9VRdmKioq0dOlSFRQUKCwsrNQ+fM4dz8x5l/ico/phbKwcxrjLx3hVOYw7l2/8+PGKjo4u8dkqDZ/BinF3dQFwrOLiYk2aNEk33XSTOnfuXGa/7Oxs+fn52bX5+fkpOzvb2SXWSmbPe7t27bRo0SJ17dpVubm5mjNnjnr16qV9+/apefPmVVhxzbVnzx6FhYXp3LlzatCggVauXKmOHTuW2pfPueNU5LzzOUd1w9hYOYxxl4fxqnIYdypn6dKl2r17t3bs2GGqP5/BiiFE1zLjx4/X3r17tXnzZleXckUxe97DwsLs/oraq1cvdejQQa+//rpmzZrl7DJrhXbt2iktLU25ublavny5YmNjlZqaWubACseoyHnnc47qhrGxchjjLg/jVeUw7ly+rKwsTZw4USkpKTxgzUkI0bXIhAkTtGrVKm3atOmSf3Xz9/dXTk6OXVtOTo78/f2dWWKtVJHz/kd169bVddddp0OHDjmputqnXr16atOmjSQpJCREO3bs0Msvv6zXX3+9RF8+545TkfP+R3zO4UqMjZXDGHf5GK8qh3Hn8u3atUvHjh3T9ddfb2srKirSpk2bNG/ePBUWFsrNzc1uHT6DFcNvomsBwzA0YcIErVy5UuvXr1erVq0uuU5YWJjWrVtn15aSklLub01g73LO+x8VFRVpz549CggIcEKFV4bi4mIVFhaWuozPufOUd97/iM85XIGxsXIY4xyP8apyGHfMGzBggPbs2aO0tDTbdMMNN2jUqFFKS0srEaAlPoMV5trnmsERHn74YcPb29vYuHGjcfToUdt05swZW597773XmDJlim3+iy++MNzd3Y05c+YY3377rTFt2jSjbt26xp49e1xxCDXS5Zz3GTNmGGvXrjUOHz5s7Nq1yxgxYoTh4eFh7Nu3zxWHUONMmTLFSE1NNdLT042vv/7amDJlimGxWIzPPvvMMAw+585S0fPO5xzVAWNj5TDGVQ7jVeUw7jjeH5/OzWewcgjRtYCkUqfExERbn759+xqxsbF2633wwQfGtddea9SrV8/o1KmTsXr16qotvIa7nPM+adIko0WLFka9evUMPz8/49ZbbzV2795d9cXXUPfdd58RHBxs1KtXz/Dx8TEGDBhgG1ANg8+5s1T0vPM5R3XA2Fg5jHGVw3hVOYw7jvfHEM1nsHIshmEYVXfdGwAAAACAmovfRAMAAAAAYBIhGgAAAAAAkwjRAAAAAACYRIgGAAAAAMAkQjQAAAAAACYRogEAAAAAMIkQDQAAAACASYRoAAAAAABMIkQDAAAAAGASIRoAAAAAAJMI0QAAAAAAmESIBgAAAADApP8H+CmVPuaSaSMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1200x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "\n",
    "advertising_df = pd.read_csv(\"../Daten/Advertising.csv\", index_col= \"Unnamed: 0\")\n",
    "\n",
    "draws = 100\n",
    "mse_train_results, mse_test_results = [], []\n",
    "for _ in range(draws):\n",
    "\n",
    "    train_df = advertising_df.sample(frac = 0.50, replace=False)\n",
    "    test_df = advertising_df.loc[[idx for idx in advertising_df.index if not(idx in train_df.index)], :]\n",
    "\n",
    "    X_train, X_test = train_df.drop([\"sales\"], axis = 1), test_df.drop([\"sales\"], axis = 1)\n",
    "    y_train, y_test = train_df.sales, test_df.sales\n",
    "\n",
    "    linear_regression = LinearRegression()\n",
    "    linear_regression.fit(X_train, y_train)\n",
    "    y_train_hat, y_test_hat = linear_regression.predict(X_train), linear_regression.predict(X_test)\n",
    "\n",
    "    mse_train, mse_test = mean_squared_error(y_train, y_train_hat), mean_squared_error(y_test, y_test_hat)\n",
    "    mse_train_results.append(mse_train)\n",
    "    mse_test_results.append(mse_test)\n",
    "\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize = (12, 5))\n",
    "axs[0].hist(mse_train_results)\n",
    "axs[0].axvline(np.mean(mse_train_results))\n",
    "axs[0].set_title(\"MSE - training data\")\n",
    "\n",
    "axs[1].hist(mse_test_results)\n",
    "axs[1].axvline(np.mean(mse_test_results))\n",
    "axs[1].set_title(\"MSE - test data\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K-fold Kreuzvalidierung\n",
    "\n",
    "Bei der K-fold Kreuzvalidierung wird der Datensatz in $k$ gleich große Teile aufgeteilt. Anschließend wird immer einer der Teile als Testdatensatz verwendet, während die restlichen Daten zum Trainieren des Modells verwendet werden. Dieser Vorgang wird $k$ Mal durch Iterieren über alle Teile (folds) durchgeführt. Die untere Grafik visualisiert diese Form der Datenaufteilung."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAHHCAYAAACRAnNyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA0GElEQVR4nO3deXQUZaLG4bcTSCcsWVjDEoJAQLYgi0BkHFTCEpFFZ5SreEFQcRwQIqAjxxFwjcoM4sLFGXWAOSIqKjgqspOgQFAIjCA7ogQhYRGSECAhne/+wdBDmwCp0J2uJL/nnD7SVdVVb1IH+rXqqyqHMcYIAADAhgL8HQAAAOBSKCoAAMC2KCoAAMC2KCoAAMC2KCoAAMC2KCoAAMC2KCoAAMC2KCoAAMC2KCoAAMC2KCoAyr05c+bI4XDoxx9/vOKyTZs21X333efzTAC8g6IC4KpcKAkbN270mJ6VlaWuXbsqODhYS5YsueTnins98cQTZRUfgM1V8XcAABVPdna2+vTpo++++04LFy5Uv379LrnsM888o2uuucZjWrt27XwdEUA5QVEB4FU5OTnq27evtmzZok8++UQJCQmXXT4hIUFdunQpo3QAyhtO/QDwmlOnTqlfv35KS0vTxx9/rP79+1/1OletWqUbb7xR1atXV3h4uAYNGqQdO3Zc8XPGGD333HNq3LixqlWrpptvvlnff//9VecBULY4ogLAK3Jzc5WQkKBvv/1WH330kW677bYSfS4rK0vHjh3zmFanTh1J0ooVK5SQkKBmzZpp6tSpOnPmjF5//XX16NFDaWlpatq06SXXO3nyZD333HO69dZbdeuttyotLU19+vRRfn5+qX9GAGWPogLAK4YPH65Dhw5pwYIFGjhwYIk/Fx8fX2SaMUaS9Nhjj6lWrVpav369atWqJUkaPHiwOnbsqClTpmju3LnFrvPo0aN6+eWX1b9/f3322WdyOBySpCeffFIvvPCC1R8NgB9RVAB4RWZmpoKDgxUVFWXpczNnzlTLli2LTD98+LC2bNmixx9/3F1SJCk2Nla9e/fW4sWLL7nOFStWKD8/X4888oi7pEhSYmIiRQUoZygqALzib3/7m8aPH69+/frpq6++UqtWreRyuXT06FGP5WrVqqWgoCD3+65duxY7mPann36SJLVq1arIvNatW2vp0qXKzc1V9erVL/nZmJgYj+l169ZVRESE9R8OgN8wmBaAV7Rp00aLFy/WmTNn1Lt3b6Wnpys9PV0NGjTweK1bt87fUQGUIxxRAeA1Xbt21aJFi9S/f3/17t1by5cv1/Llyz2W6dChQ4nWFR0dLUnatWtXkXk7d+5UnTp1ij2acvFn9+zZo2bNmrmnHz16VCdOnCjR9gHYA0dUAHhVr169NH/+fO3du1eDBw9W165dFR8f736V9NRLgwYNdN1112nu3Lk6efKke/q2bdu0bNky3XrrrZf8bHx8vKpWrarXX3/dPTBXkmbMmFHaHwuAn1BUAHjd7bffrrfeektpaWkaOHCgzp49W6r1TJs2TcePH1dcXJz+8pe/6Nlnn9Utt9yisLAwTZ069ZKfq1u3riZOnKgvvvhCt912m2bOnKkHHnhAc+bMcV/6DKB8oKgA8IkRI0boL3/5i1JSUnTnnXeqoKDA8jri4+O1ZMkS1a5dW5MnT9Zf/vIXde/eXWvXri1y2/1fe+655/T0009r8+bNeuyxx7Rv3z4tW7bskqeLANiTw1x8XBQAAMBGOKICAABsi6ICAABsi6ICAABsi6ICAABsi6ICAABsi6ICAABsq1zfQr+wsFCHDh1SzZo1PZ6QCgAA7MsYo5ycHDVs2FABAZc/ZlKui8qhQ4csP1IeAADYQ3p6uho3bnzZZcp1UalZs6ak8z9oaGion9MAAICSyM7OVlRUlPt7/HLKdVG5cLonNDSUogIAQDlTkmEbDKYFAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2Va7vo+Jtia+ckCSdzfdzEABAsSYeGCNJCjJnPabzEBXfcwSHSJIaT3+3TLdLUbkIBQUA7M35q4KCsmPOnvHLdjn1AwAAbMs2ReXFF1+Uw+FQYmKiv6MAAACbsEVR+fbbb/W3v/1NsbGx/o4CAABsxO9F5dSpUxo6dKjeeustRURE+DsOAACwEb8XldGjR6t///6Kj4+/4rJ5eXnKzs72eAEAgIrLr1f9vP/++0pLS9O3335bouWTkpL09NNP+zgVAACwC78dUUlPT9e4ceM0b948BQcHl+gzkyZNUlZWlvuVnp7u45QAAMCf/HZEZdOmTTpy5Ig6derknuZyubRmzRq98cYbysvLU2BgoMdnnE6nnE5nWUcFAAB+4rei0qtXL23dutVj2ogRI3TttdfqT3/6U5GSAgAAKh+/FZWaNWuqXbt2HtOqV6+u2rVrF5kOAAAqJ79f9QMAAHAptnrWT3Jysl+3Hxx0/r888wcA7CnPcf7iCx5KWPYuPJSwrNmqqPjbjEe54RwA2Ns8fwdAGePUDwAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsC2KCgAAsK0q/g5gJ4mvnJAknc33cxAA5crEA2Pcfw4yZz3mOco6TAU3Lep1SVK+I9hzhoPftK8FB53/74xHI8p0uxSVi1BQAJSG81flBL6THxDi7wiVlr++Izn1AwAAbMuvRWXWrFmKjY1VaGioQkNDFRcXpy+//NKfkQAAgI34tag0btxYL774ojZt2qSNGzfqlltu0aBBg/T999/7MxYAALAJv45RGTBggMf7559/XrNmzVJqaqratm3rp1QAAMAubDOY1uVyacGCBcrNzVVcXFyxy+Tl5SkvL8/9Pjs7u6ziAQAAP7B06ufcuXMaOXKk9u/f77UAW7duVY0aNeR0OvWHP/xBCxcuVJs2bYpdNikpSWFhYe5XVFSU13IAAAD7sVRUqlatqo8//tirAVq1aqUtW7Zow4YNevjhhzV8+HBt37692GUnTZqkrKws9ys9Pd2rWQAAgL1YHkw7ePBgLVq0yGsBgoKC1KJFC3Xu3FlJSUnq0KGDXn311WKXdTqd7iuELrwAAEDFZXmMSkxMjJ555hmtXbtWnTt3VvXq1T3mjx079qoCFRYWeoxDAQAAlZflovLOO+8oPDxcmzZt0qZNmzzmORwOS0Vl0qRJSkhIUJMmTZSTk6P33ntPycnJWrp0qdVYAACgArJcVLw5kPbIkSMaNmyYDh8+rLCwMMXGxmrp0qXq3bu317YBAADKr1Jfnpyfn6/9+/erefPmqlKldKt55513Srt5n7jwwCWe+QPAiryLHpDHQwl9K6jwjCQeSugPF74jy5rDGGOsfOD06dN65JFHNHfuXEnS7t271axZMz3yyCNq1KiRnnjiCZ8ELU52drbCwsKUlZXFwFoAAMoJK9/flq/6mTRpkv79738rOTlZwcH/bbTx8fH64IMPrKcFAAC4BMvnbBYtWqQPPvhA3bt3l+OiQ21t27bVvn37vBoOAABUbpaPqBw9elT16tUrMj03N9ejuAAAAFwty0WlS5cu+uKLL9zvL5STt99++5LP6AEAACgNy6d+XnjhBSUkJGj79u0qKCjQq6++qu3bt2vdunVKSUnxRUYAAFBJWT6i8pvf/EZbtmxRQUGB2rdvr2XLlqlevXpav369Onfu7IuMAACgkrJ8ebKdcHkyAADlj5Xv71Ldqc3lcmnhwoXasWOHJKlNmzYaNGhQqW/8BgAAUBzLzeL777/XwIEDlZGRoVatWkmSXnrpJdWtW1efffaZ2rVr5/WQAACgcrI8RuWBBx5Q27ZtdfDgQaWlpSktLU3p6emKjY3VqFGjfJERAABUUpaPqGzZskUbN25URESEe1pERISef/55XX/99V4NBwAAKjfLR1RatmypzMzMItOPHDmiFi1aeCUUAACAVMKikp2d7X4lJSVp7Nix+uijj3Tw4EEdPHhQH330kRITE/XSSy/5Oi8AAKhESnR5ckBAgMft8S985MK0i9+7XC5f5CwWlycDAFD+eP3y5NWrV3slGAAAgBUlKio9e/b0dQ4AAIAiSnWHtrNnz+q7777TkSNHVFhY6DFv4MCBXgkGAABguagsWbJEw4YN07Fjx4rMK+sxKgAAoGKzfHnyI488ojvvvFOHDx9WYWGhx4uSAgAAvMlyUcnMzNT48eNVv359X+QBAABws1xUfv/73ys5OdkHUQAAADyV6D4qFzt9+rTuvPNO1a1bV+3bt1fVqlU95o8dO9arAS+H+6gAAFD+eP0+KhebP3++li1bpuDgYCUnJ3vcCM7hcJRpUQEAABWb5aLy5JNP6umnn9YTTzyhgADLZ44AAABKzHLTyM/P15AhQygpAADA5yy3jeHDh+uDDz7wRRYAAAAPlk/9uFwuvfzyy1q6dKliY2OLDKadPn2618IBAIDKzXJR2bp1qzp27ChJ2rZtm8e8iwfWAgAAXC3LRYUnKQMAgLLCiFgAAGBblo+o3HzzzZc9xbNq1aqrCgQAAHCB5aJy3XXXebw/d+6ctmzZom3btmn48OHeygUAAGC9qLzyyivFTp86dapOnTp11YEAAAAu8NoYlXvvvVf/+Mc/vLU6AAAA7xWV9evXKzg42FurAwAAsH7q54477vB4b4zR4cOHtXHjRj311FNeCwYAAGC5qISFhXm8DwgIUKtWrfTMM8+oT58+XgsGAABguajMnj3bFzkAAACKsFxULsjPz9eRI0dUWFjoMb1JkyZXHQoAAEAqRVHZvXu37r//fq1bt85jujFGDodDLpfLa+EAAEDlZrmojBgxQlWqVNHnn3+uBg0a8CBCAADgM5aLypYtW7Rp0yZde+21vsgDAADgZvk+Km3atNGxY8d8kQUAAMCD5aLy0ksv6fHHH1dycrKOHz+u7OxsjxcAAIC3OIwxxsoHAgLOd5tfj03xx2Da7OxshYWFKSsrS6GhoWW2XQAAUHpWvr8tj1FZvXp1qYMBAABYYbmo9OzZ0xc5AAAAivDaQwkBAAC8jaICAABsi6ICAABsi6ICAABsi6ICAABsy3JRyczM1P/+7/+qYcOGqlKligIDAz1eAAAA3mL58uT77rtPBw4c0FNPPcVDCQEAgE9ZLipff/21vvrqK1133XU+iAMAAPBflk/9REVFyeJd9wEAAErFclGZMWOGnnjiCf34448+iAMAAPBflk/9DBkyRKdPn1bz5s1VrVo1Va1a1WP+L7/84rVwAACgcrNcVGbMmOGDGAAAAEVZLirDhw/3RQ4AAIAiLBcVSXK5XFq0aJF27NghSWrbtq0GDhzIfVQAAIBXWS4qe/fu1a233qqff/5ZrVq1kiQlJSUpKipKX3zxhZo3b+71kAAAoHKyfNXP2LFj1bx5c6WnpystLU1paWk6cOCArrnmGo0dO9YXGQEAQCVl+YhKSkqKUlNTVatWLfe02rVr68UXX1SPHj28Gg4AAFRulo+oOJ1O5eTkFJl+6tQpBQUFeSUUAACAVIqictttt2nUqFHasGGDjDEyxig1NVV/+MMfNHDgQF9kBAAAlZTlovLaa6+pefPmiouLU3BwsIKDg9WjRw+1aNFCr776qi8yAgCASsryGJXw8HB9+umn2rNnj3bu3ClJat26tVq0aOH1cAAAoHIr1X1UJCkmJkYxMTHezAIAAOChREVl/PjxevbZZ1W9enWNHz/+sstOnz7dK8H8IfGVE5Kks/l+DgJ4ycQDYyRJQeasx3SHP8JUYNOiXnf/Od8R7DnTwW8bFUPwf66XmfFoRJlut0RFZfPmzTp37pz7zxUVBQUVjfNXBQW+kR8Q4u8IgM/56zuyREVl9erVxf4ZAADAlyxf9TNy5Mhi76OSm5urkSNHWlpXUlKSrr/+etWsWVP16tXT4MGDtWvXLquRAABABWW5qMydO1dnzpwpMv3MmTP65z//aWldKSkpGj16tFJTU7V8+XKdO3dOffr0UW5urtVYAACgAirxVT/Z2dnuG7zl5OQoOPi/A8ZcLpcWL16sevXqWdr4kiVLPN7PmTNH9erV06ZNm/Tb3/7W0roAAEDFU+KiEh4eLofDIYfDoZYtWxaZ73A49PTTT19VmKysLEnyeI7QxfLy8pSXl+d+n52dfVXbAwAA9lbiorJ69WoZY3TLLbfo448/9igTQUFBio6OVsOGDUsdpLCwUImJierRo4fatWtX7DJJSUlXXYYAAED5UeKi0rNnT0nS/v37FRUVpYAAy8NbLmv06NHatm2bvv7660suM2nSJI/7uGRnZysqKsqrOQAAgH1YvjNtdHS0JOn06dM6cOCA8vM9L6yOjY21HGLMmDH6/PPPtWbNGjVu3PiSyzmdTjmdTsvrBwAA5ZPlonL06FGNGDFCX375ZbHzXS5XiddljNEjjzyihQsXKjk5Wddcc43VOAAAoAKzfP4mMTFRJ0+e1IYNGxQSEqIlS5Zo7ty5iomJ0b/+9S9L6xo9erTeffddvffee6pZs6YyMjKUkZFR7OXPAACg8rF8RGXVqlX69NNP1aVLFwUEBCg6Olq9e/dWaGiokpKS1L9//xKva9asWZKkm266yWP67Nmzdd9991mNBgAAKhjLRSU3N9d9v5SIiAgdPXpULVu2VPv27ZWWlmZpXcYYq5v3qQsPXOKZP6go8v7zgDweSuhbQYX/PQrMQwlRUV34jixrlotKq1attGvXLjVt2lQdOnTQ3/72NzVt2lRvvvmmGjRo4IuMZaasnwgJ+N48fweoFF7zdwCgArNcVMaNG6fDhw9LkqZMmaJ+/fpp3rx5CgoK0pw5c7ydDwAAVGIOc5XnX06fPq2dO3eqSZMmqlOnjrdylUh2drbCwsKUlZWl0NDQMt02AAAoHSvf35aPqPxatWrV1KlTp6tdDQAAQBElKioX3w32SqZPn17qMAAAABcrUVHZvHmzx/u0tDQVFBSoVatWkqTdu3crMDBQnTt39n5CAABQaZWoqKxevdr95+nTp6tmzZqaO3euIiLOXyVz4sQJjRgxQjfeeKNvUgIAgErJ8mDaRo0aadmyZWrbtq3H9G3btqlPnz46dOiQVwNeDoNpAQAof6x8f1u+hX52draOHj1aZPrRo0eVk5NjdXUAAACXZLmo3H777RoxYoQ++eQTHTx4UAcPHtTHH3+s+++/X3fccYcvMgIAgErK8uXJb775piZOnKh77rlH586dO7+SKlV0//33a9q0aV4PCAAAKq9S3/AtNzdX+/btkyQ1b95c1atX92qwkmCMCgAA5U+Z3PCtevXqio2NLe3HAQAArqhEReWOO+7QnDlzFBoaesVxKJ988olXggEAAJSoqISFhcnxn0eVh4WF+TQQAADABVf9UEJ/YowKAADlj0/vowIAAFBWSnTqp2PHju5TP1eSlpZ2VYEAAAAuKFFRGTx4sI9jAAAAFMUYFQAAUKYYowIAACoEyzd8c7lceuWVV/Thhx/qwIEDys/P95j/yy+/eC0cAACo3CwfUXn66ac1ffp0DRkyRFlZWRo/frzuuOMOBQQEaOrUqT6ICAAAKivLRWXevHl66623NGHCBFWpUkV333233n77bU2ePFmpqam+yAgAACopy0UlIyND7du3lyTVqFFDWVlZkqTbbrtNX3zxhXfTAQCASs1yUWncuLEOHz4s6fxTk5ctWyZJ+vbbb+V0Or2bDgAAVGqWi8rtt9+ulStXSpIeeeQRPfXUU4qJidGwYcM0cuRIrwcEAACV11XfRyU1NVXr1q1TTEyMBgwY4K1cJcJ9VAAAKH+sfH9bvjz57NmzCg4Odr/v3r27unfvbj0lAADAFVg+9VOvXj0NHz5cy5cvV2FhoS8yAQAASCpFUZk7d65Onz6tQYMGqVGjRkpMTNTGjRt9kQ0AAFRypRpMu2DBAmVmZuqFF17Q9u3b1b17d7Vs2VLPPPOMLzICAIBKyisPJdy+fbuGDh2q7777Ti6Xyxu5SoTBtAAAlD9l8lDCs2fP6sMPP9TgwYPVqVMn/fLLL3rsscdKuzoAAIAiLF/1s3TpUr333ntatGiRqlSpot///vdatmyZfvvb3/oiHwAAqMQsF5Xbb79dt912m/75z3/q1ltvVdWqVX2RCwAAwHpRyczMVM2aNX2RBQAAwIPlMSqUFAAAUFZKPZgWAADA1ygqAADAtigqAADAtq66qGRnZ2vRokXasWOHN/IAAAC4WS4qd911l9544w1J0pkzZ9SlSxfdddddio2N1ccff+z1gAAAoPKyXFTWrFmjG2+8UZK0cOFCGWN08uRJvfbaa3ruuee8HhAAAFRelotKVlaWatWqJUlasmSJfve736latWrq37+/9uzZ4/WAAACg8rJcVKKiorR+/Xrl5uZqyZIl6tOnjyTpxIkTCg4O9npAAABQeVm+M21iYqKGDh2qGjVqKDo6WjfddJOk86eE2rdv7+18AACgErNcVP74xz+qa9euSk9PV+/evRUQcP6gTLNmzRijAgAAvMphjDFXswKXy6WtW7cqOjpaERER3spVItnZ2QoLC1NWVpZCQ0PLdNsAAKB0rHx/Wx6jkpiYqHfeeUfS+ZLSs2dPderUSVFRUUpOTi5VYAAAgOJYLiofffSROnToIEn67LPPtH//fu3cuVOPPvqonnzySa8HBAAAlZflonLs2DFFRkZKkhYvXqw777xTLVu21MiRI7V161avBwQAAJWX5aJSv359bd++XS6XS0uWLFHv3r0lSadPn1ZgYKDXAwIAgMrL8lU/I0aM0F133aUGDRrI4XAoPj5ekrRhwwZde+21Xg8IAAAqL8tFZerUqWrXrp3S09N15513yul0SpICAwP1xBNPeD0gAACovK768mR/4vJkAADKH59enixJKSkpGjBggFq0aKEWLVpo4MCB+uqrr0oVFgAA4FIsF5V3331X8fHxqlatmsaOHauxY8cqJCREvXr10nvvveeLjAAAoJKyfOqndevWGjVqlB599FGP6dOnT9dbb72lHTt2eDXg5XDqBwCA8senp35++OEHDRgwoMj0gQMHav/+/VZXBwAAcEmWi0pUVJRWrlxZZPqKFSsUFRXllVAAAABSKS5PnjBhgsaOHastW7bohhtukCStXbtWc+bM0auvvur1gAAAoPKyXFQefvhhRUZG6q9//as+/PBDSefHrXzwwQcaNGiQ1wMCAIDKy1JRKSgo0AsvvKCRI0fq66+/9lUmAAAASRbHqFSpUkUvv/yyCgoKfJUHAADAzfJg2l69eiklJcUXWQAAADxYHqOSkJCgJ554Qlu3blXnzp1VvXp1j/kDBw70WjgAAFC5Wb7hW0DApQ/COBwOuVyuqw5VUtzwDQCA8sfK97flIyqFhYWlDgYAAGBFqR5KCAAAUBZKfERl1apVGjNmjFJTU4scpsnKytINN9ygWbNm6be//a3XQ5aVxFdOSJLO5vs5SAU38cAYSVKQOesx3eGPMBXctKjXJUn5jmDPGQ5+2wCsCQ46/98Zj0aU6XZLfERlxowZevDBB4s9lxQWFqaHHnpIr7zyilfDlbWz+ZSUsuA0Z+U0Z+WQPF7wvvyAEOUHhJwvJhe/AMAif31Hlrio/Pvf/1a/fv0uOb9Pnz7atGmTV0IBAABIFopKZmamqlatesn5VapU0dGjRy1tfM2aNRowYIAaNmwoh8OhRYsWWfo8AACo2EpcVBo1aqRt27Zdcv53332nBg0aWNp4bm6uOnTooJkzZ1r6HAAAqBxKPJj21ltv1VNPPaV+/fopONhzYN6ZM2c0ZcoU3XbbbZY2npCQoISEBEufAQAAlUeJi8qf//xnffLJJ2rZsqXGjBmjVq1aSZJ27typmTNnyuVy6cknn/RZUEnKy8tTXl6e+312drZPtwcAAPyrxEWlfv36WrdunR5++GFNmjRJF25o63A41LdvX82cOVP169f3WVBJSkpK0tNPP+3TbQAAAPuwdGfa6OhoLV68WCdOnNDevXtljFFMTIwiIsrmmupJkyZp/Pjx7vfZ2dmKiooqk20DAICyZ/kW+pIUERGh66+/3ttZrsjpdMrpdJb5dgEAgH9wC30AAGBbpTqi4i2nTp3S3r173e/379+vLVu2qFatWmrSpIkfkwEAADvwa1HZuHGjbr75Zvf7C+NPhg8frjlz5vgpFQAAsAu/FpWbbrrJffWQHVx44BLP+/GtvP88II+HEvpeUOEZSTyUEMDVu/AdWdYcxk5NwaLs7GyFhYUpKyur2IclAgAA+7Hy/c1gWgAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFsUFQAAYFtV/B3ATg6Ov1eSZM6e8XOSim1a1OuSpHxHsOcMh8MPaQAAJREcdP6/Mx6NKNPtUlQuQkEpG/kBIf6OAACw6Gy+f7bLqR8AAGBbtigqM2fOVNOmTRUcHKxu3brpm2++8XckAABgA34vKh988IHGjx+vKVOmKC0tTR06dFDfvn115MgRf0cDAAB+5veiMn36dD344IMaMWKE2rRpozfffFPVqlXTP/7xD39HAwAAfubXopKfn69NmzYpPj7ePS0gIEDx8fFav359keXz8vKUnZ3t8QIAABWXX4vKsWPH5HK5VL9+fY/p9evXV0ZGRpHlk5KSFBYW5n5FRUWVVVQAAOAHfj/1Y8WkSZOUlZXlfqWnp/s7EgAA8CG/3kelTp06CgwMVGZmpsf0zMxMRUZGFlne6XTK6XSWVTwAAOBnfj2iEhQUpM6dO2vlypXuaYWFhVq5cqXi4uL8mAwAANiB3+9MO378eA0fPlxdunRR165dNWPGDOXm5mrEiBH+jgYAAPzM70VlyJAhOnr0qCZPnqyMjAxdd911WrJkSZEBtgAAoPLxe1GRpDFjxmjMmDH+jiFH8Pln0PDMH98KKjz/++WhhABQflx4KGFZs0VRsYvG09/1d4RK4TV/BwAAlBvl6vJkAABQuVBUAACAbVFUAACAbVFUAACAbVFUAACAbVFUAACAbVFUAACAbVFUAACAbZXrG74ZYyRJ2dnZfk4CAABK6sL39oXv8csp10UlJydHkhQVFeXnJAAAwKqcnByFhYVddhmHKUmdsanCwkIdOnRINWvWlMPLz4nJzs5WVFSU0tPTFRoa6tV1o2ywD8s/9mHFwH4s/7y9D40xysnJUcOGDRUQcPlRKOX6iEpAQIAaN27s022EhobyF6ucYx+Wf+zDioH9WP55cx9e6UjKBQymBQAAtkVRAQAAtkVRuQSn06kpU6bI6XT6OwpKiX1Y/rEPKwb2Y/nnz31YrgfTAgCAio0jKgAAwLYoKgAAwLYoKgAAwLYoKgAAwLYoKsWYOXOmmjZtquDgYHXr1k3ffPONvyPhEqZOnSqHw+Hxuvbaa93zz549q9GjR6t27dqqUaOGfve73ykzM9OPiSFJa9as0YABA9SwYUM5HA4tWrTIY74xRpMnT1aDBg0UEhKi+Ph47dmzx2OZX375RUOHDlVoaKjCw8N1//3369SpU2X4U1RuV9qH9913X5G/m/369fNYhn3oX0lJSbr++utVs2ZN1atXT4MHD9auXbs8linJv6EHDhxQ//79Va1aNdWrV0+PPfaYCgoKvJaTovIrH3zwgcaPH68pU6YoLS1NHTp0UN++fXXkyBF/R8MltG3bVocPH3a/vv76a/e8Rx99VJ999pkWLFiglJQUHTp0SHfccYcf00KScnNz1aFDB82cObPY+S+//LJee+01vfnmm9qwYYOqV6+uvn376uzZs+5lhg4dqu+//17Lly/X559/rjVr1mjUqFFl9SNUelfah5LUr18/j7+b8+fP95jPPvSvlJQUjR49WqmpqVq+fLnOnTunPn36KDc3173Mlf4Ndblc6t+/v/Lz87Vu3TrNnTtXc+bM0eTJk70X1MBD165dzejRo93vXS6XadiwoUlKSvJjKlzKlClTTIcOHYqdd/LkSVO1alWzYMEC97QdO3YYSWb9+vVllBBXIsksXLjQ/b6wsNBERkaaadOmuaedPHnSOJ1OM3/+fGOMMdu3bzeSzLfffute5ssvvzQOh8P8/PPPZZYd5/16HxpjzPDhw82gQYMu+Rn2of0cOXLESDIpKSnGmJL9G7p48WITEBBgMjIy3MvMmjXLhIaGmry8PK/k4ojKRfLz87Vp0ybFx8e7pwUEBCg+Pl7r16/3YzJczp49e9SwYUM1a9ZMQ4cO1YEDByRJmzZt0rlz5zz257XXXqsmTZqwP21s//79ysjI8NhvYWFh6tatm3u/rV+/XuHh4erSpYt7mfj4eAUEBGjDhg1lnhnFS05OVr169dSqVSs9/PDDOn78uHse+9B+srKyJEm1atWSVLJ/Q9evX6/27durfv367mX69u2r7Oxsff/9917JRVG5yLFjx+RyuTx+4ZJUv359ZWRk+CkVLqdbt26aM2eOlixZolmzZmn//v268cYblZOTo4yMDAUFBSk8PNzjM+xPe7uwby739zAjI0P16tXzmF+lShXVqlWLfWsT/fr10z//+U+tXLlSL730klJSUpSQkCCXyyWJfWg3hYWFSkxMVI8ePdSuXTtJKtG/oRkZGcX+Xb0wzxvK9dOTgYSEBPefY2Nj1a1bN0VHR+vDDz9USEiIH5MBldv//M//uP/cvn17xcbGqnnz5kpOTlavXr38mAzFGT16tLZt2+Yxxs8uOKJykTp16igwMLDIiObMzExFRkb6KRWsCA8PV8uWLbV3715FRkYqPz9fJ0+e9FiG/WlvF/bN5f4eRkZGFhngXlBQoF9++YV9a1PNmjVTnTp1tHfvXknsQzsZM2aMPv/8c61evVqNGzd2Ty/Jv6GRkZHF/l29MM8bKCoXCQoKUufOnbVy5Ur3tMLCQq1cuVJxcXF+TIaSOnXqlPbt26cGDRqoc+fOqlq1qsf+3LVrlw4cOMD+tLFrrrlGkZGRHvstOztbGzZscO+3uLg4nTx5Ups2bXIvs2rVKhUWFqpbt25lnhlXdvDgQR0/flwNGjSQxD60A2OMxowZo4ULF2rVqlW65pprPOaX5N/QuLg4bd261aN0Ll++XKGhoWrTpo3XguIi77//vnE6nWbOnDlm+/btZtSoUSY8PNxjRDPsY8KECSY5Odns37/frF271sTHx5s6deqYI0eOGGOM+cMf/mCaNGliVq1aZTZu3Gji4uJMXFycn1MjJyfHbN682WzevNlIMtOnTzebN282P/30kzHGmBdffNGEh4ebTz/91Hz33Xdm0KBB5pprrjFnzpxxr6Nfv36mY8eOZsOGDebrr782MTEx5u677/bXj1TpXG4f5uTkmIkTJ5r169eb/fv3mxUrVphOnTqZmJgYc/bsWfc62If+9fDDD5uwsDCTnJxsDh8+7H6dPn3avcyV/g0tKCgw7dq1M3369DFbtmwxS5YsMXXr1jWTJk3yWk6KSjFef/1106RJExMUFGS6du1qUlNT/R0JlzBkyBDToEEDExQUZBo1amSGDBli9u7d655/5swZ88c//tFERESYatWqmdtvv90cPnzYj4lhjDGrV682koq8hg8fbow5f4nyU089ZerXr2+cTqfp1auX2bVrl8c6jh8/bu6++25To0YNExoaakaMGGFycnL88NNUTpfbh6dPnzZ9+vQxdevWNVWrVjXR0dHmwQcfLPI/fOxD/ypu/0kys2fPdi9Tkn9Df/zxR5OQkGBCQkJMnTp1zIQJE8y5c+e8ltPxn7AAAAC2wxgVAABgWxQVAABgWxQVAABgWxQVAABgWxQVAABgWxQVAABgWxQVAABgWxQVoJxp2rSpZsyY4e8YXpOcnCyHw1HkeSIVyU033aTExER/xwDKJYoKYBPp6ekaOXKkGjZsqKCgIEVHR2vcuHE6fvy4v6N5TXFf2DfccIMOHz6ssLAw/4QCYGsUFcAGfvjhB3Xp0kV79uzR/PnztXfvXr355pvuB2L+8ssvfsvmcrlUWFjos/UHBQUpMjJSDofDZ9uoiHy9XwC7oKgANjB69GgFBQVp2bJl6tmzp5o0aaKEhAStWLFCP//8s5588kmP5XNycnT33XerevXqatSokWbOnOmeZ4zR1KlT1aRJEzmdTjVs2FBjx451z8/Ly9PEiRPVqFEjVa9eXd26dVNycrJ7/pw5cxQeHq5//etfatOmjZxOp95++20FBwcXOT0zbtw43XLLLZKk48eP6+6771ajRo1UrVo1tW/fXvPnz3cve9999yklJUWvvvqqHA6HHA6Hfvzxx2JP/Xz88cdq27atnE6nmjZtqr/+9a8e223atKleeOEFjRw5UjVr1lSTJk3097///bK/45tuukljx47V448/rlq1aikyMlJTp051z//xxx/lcDi0ZcsW97STJ0/K4XC4fz8Xsi5dulQdO3ZUSEiIbrnlFh05ckRffvmlWrdurdDQUN1zzz06ffq0x/YLCgo0ZswYhYWFqU6dOnrqqad08RNMSrNfDhw4cNmfGagQvPbUIAClcvz4ceNwOMwLL7xQ7PwHH3zQREREmMLCQmOMMdHR0aZmzZomKSnJ7Nq1y7z22msmMDDQLFu2zBhjzIIFC0xoaKhZvHix+emnn8yGDRvM3//+d/f6HnjgAXPDDTeYNWvWmL1795pp06YZp9Npdu/ebYwxZvbs2aZq1armhhtuMGvXrjU7d+40p06dMvXr1zdvv/22ez0FBQUe0w4ePGimTZtmNm/ebPbt2+fOtWHDBmOMMSdPnjRxcXHmwQcfdD+ltaCgwP1wuxMnThhjjNm4caMJCAgwzzzzjNm1a5eZPXu2CQkJ8XhQWnR0tKlVq5aZOXOm2bNnj0lKSjIBAQFm586dl/w99+zZ04SGhpqpU6ea3bt3m7lz5xqHw+H+ve3fv99IMps3b3Z/5sSJE0aSWb16tTHmvw/i6969u/n6669NWlqaadGihenZs6fp06ePSUtLM2vWrDG1a9c2L774ose2a9SoYcaNG2d27txp3n33XVOtWrWr3i+5ubmX/HmBioKiAvhZamqqkWQWLlxY7Pzp06cbSSYzM9MYc/5Lul+/fh7LDBkyxCQkJBhjjPnrX/9qWrZsafLz84us66effjKBgYHm559/9pjeq1cv92PZZ8+ebSSZLVu2eCwzbtw4c8stt7jfL1261DidTnfBKE7//v3NhAkT3O979uxpxo0b57HMr4vKPffcY3r37u2xzGOPPWbatGnjfh8dHW3uvfde9/vCwkJTr149M2vWrEtm6dmzp/nNb37jMe366683f/rTn4wx1orKihUr3MskJSUZSWbfvn3uaQ899JDp27evx7Zbt27tLpvGGPOnP/3JtG7d2hhzdfsFqOg49QPYhLHwIPO4uLgi73fs2CFJuvPOO3XmzBk1a9ZMDz74oBYuXKiCggJJ0tatW+VyudSyZUvVqFHD/UpJSdG+ffvc6wsKClJsbKzHNoYOHark5GQdOnRIkjRv3jz1799f4eHhks6PmXj22WfVvn171apVSzVq1NDSpUstn57YsWOHevTo4TGtR48e2rNnj1wul3vaxfkcDociIyN15MiRy6771z9TgwYNrviZK62nfv36qlatmpo1a+Yx7dfr7d69u8c4nLi4OPfPdDX7Bajoqvg7AFDZtWjRQg6HQzt27NDtt99eZP6OHTsUERGhunXrlmh9UVFR2rVrl1asWKHly5frj3/8o6ZNm6aUlBSdOnVKgYGB2rRpkwIDAz0+V6NGDfefQ0JCigxuvf7669W8eXO9//77evjhh7Vw4ULNmTPHPX/atGl69dVXNWPGDLVv317Vq1dXYmKi8vPzLfw2Sq5q1aoe7x0OxxUHl17uMwEB5/+/7eLCeO7cuSuux+FwlCrLxa5mvwAVHUUF8LPatWurd+/e+r//+z89+uijCgkJcc/LyMjQvHnzNGzYMI8vqNTUVI91pKamqnXr1u73ISEhGjBggAYMGKDRo0fr2muv1datW9WxY0e5XC4dOXJEN954o+WsQ4cO1bx589S4cWMFBASof//+7nlr167VoEGDdO+990qSCgsLtXv3brVp08a9TFBQkMdRkeK0bt1aa9eu9Zi2du1atWzZssiXuDddKIKHDx9Wx44dJcljYO3V2rBhg8f71NRUxcTEKDAw8Kr3C1CRceoHsIE33nhDeXl56tu3r9asWaP09HQtWbJEvXv3VqNGjfT88897LL927Vq9/PLL2r17t2bOnKkFCxZo3Lhxks5fHfLOO+9o27Zt+uGHH/Tuu+8qJCRE0dHRatmypYYOHaphw4bpk08+0f79+/XNN98oKSlJX3zxxRVzDh06VGlpaXr++ef1+9//Xk6n0z0vJiZGy5cv17p167Rjxw499NBDyszM9Ph806ZNtWHDBv344486duxYsUcdJkyYoJUrV+rZZ5/V7t27NXfuXL3xxhuaOHFiaX61JRYSEqLu3bvrxRdf1I4dO5SSkqI///nPXlv/gQMHNH78eO3atUvz58/X66+/7t5nV7tfgIqMogLYQExMjDZu3KhmzZrprrvuUvPmzTVq1CjdfPPNWr9+vWrVquWx/IQJE7Rx40Z17NhRzz33nKZPn66+fftKksLDw/XWW2+pR48eio2N1YoVK/TZZ5+pdu3akqTZs2dr2LBhmjBhglq1aqXBgwfr22+/VZMmTa6Ys0WLFuratau+++47DR061GPen//8Z3Xq1El9+/bVTTfdpMjISA0ePNhjmYkTJyowMFBt2rRR3bp1ix2/0qlTJ3344Yd6//331a5dO02ePFnPPPOM7rvvPgu/0dL5xz/+oYKCAnXu3FmJiYl67rnnvLbuYcOG6cyZM+ratatGjx6tcePGadSoUe75V7NfgIrMYayM4AMAAChDHFEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC2RVEBAAC29f/yXbVLiCk8UAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "import numpy as np\n",
    "import matplotlib.pylab as plt\n",
    "import pandas as pd\n",
    "\n",
    "cmap_cv = plt.cm.coolwarm\n",
    "n_splits = 5\n",
    "cv = KFold(n_splits)\n",
    "\n",
    "\n",
    "advertising_df = pd.read_csv(\"../Daten/Advertising.csv\", index_col= \"Unnamed: 0\")\n",
    "X = advertising_df.drop([\"sales\"], axis = 1)\n",
    "y = advertising_df.sales\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "for ii, (tr, tt) in enumerate(cv.split(X, y)):\n",
    "    # Fill in indices with the training/test groups\n",
    "        indices = np.array([np.nan] * len(X))\n",
    "        indices[tt] = 1\n",
    "        indices[tr] = 0\n",
    "\n",
    "        # Visualize the results\n",
    "        ax.scatter(\n",
    "            range(len(indices)),\n",
    "            [ii + 0.5] * len(indices),\n",
    "            c=indices,\n",
    "            marker=\"_\",\n",
    "            lw=10,\n",
    "            cmap=cmap_cv,\n",
    "            vmin=-0.2,\n",
    "            vmax=1.2,\n",
    "        )\n",
    "\n",
    "\n",
    "ax.set_xticks(np.arange(0, 201, 50))\n",
    "ax.set_yticks(np.arange(n_splits)+0.5)\n",
    "ax.set_yticklabels([i for i in range(n_splits)])\n",
    "\n",
    "ax.set_xlabel(\"Observation number\")\n",
    "ax.set_ylabel(\"Cross validation number\")\n",
    "ax.set_title(\"K-Fold\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nach jeder Iteration haben wir einen Schätzer eines Testfehlers, beispielsweise für ein Regressionsproblem den $MSE_k = \\frac{1}{n_k} \\sum_{i \\in k_{\\text{fold}}} \\left(y_i - \\hat{y}_i\\right)^2 $. Als finaler Schätzer des Testfehlers können wir den durchschnittlichen Testfehler über alle k-folds verwenden:\n",
    "\n",
    "$$\n",
    "CV_{(k)} = \\frac{1}{k} \\sum_k MSE_k\n",
    "$$\n",
    "\n",
    "In der nächsten Zelle wird die K-fold Kreuzvalidierung für der Advertising Datensatz mit $k=5$ durchgeführt. Da die Anzahl der Beobachtungen der Testdaten geringer ist, als bei der oberen Sampling-Methode schwanken die einzelnen Schätzungen $MSE_k$ der Testdaten stärker. Jedoch sehen wir, dass der eigentliche Schätzer $CV_{(k)}$ einen sehr ähnlichen Wert wie im oberen Beispiel annimmt. Zudem können mehr Daten zum Trainieren des Modells verwendet werden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9UAAAHDCAYAAAAqWjmwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA4cklEQVR4nO3de3hV1Z0/4G+4JEFuUpFwMQUFBREVCwNFRdRGUSkdnGm16ihSpV7AwTJtharEW0UdodgWpfWCTqeMVKpMH6VaRNGfSouC1CtWRMSxBsQLIGgiyf794WNqSMBztrkR3vd5zsPDOmudvdbeJ2ftz9n77J2TJEkSAAAAQNaaNXQHAAAAYFclVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFTDbmDx4sWRk5MTixcvzrrtmjVrIicnJ+68885a71dad955Z+Tk5MSaNWsauisAQA3OPvvs6NGjR0N3A+qFUM0u67NglZOTE0888US155MkicLCwsjJyYlvfvObVZ778MMPo7i4OPr16xetW7eOvfbaK/r37x8TJkyIv//975X1rrjiispl1PQoKSmptfHcfPPNjSq4NlVz5syJGTNmNHQ3AKgH9hWy99JLL8UVV1zRoF9cX3vttTF//vwGWz5kq0VDdwC+rPz8/JgzZ04ceeSRVcofe+yx+L//+7/Iy8urUv7JJ5/EUUcdFStXrozRo0fHRRddFB9++GG8+OKLMWfOnDj55JOja9euVdrccsst0aZNm2rL3nPPPWttHDfffHN07Ngxzj777Fp7zc8cddRR8dFHH0Vubm7Wbbt37x4fffRRtGzZstb71RDmzJkTL7zwQlx88cUN3RUA6ol9hcy99NJLceWVV8bRRx/dYEear7322vj2t78do0aNapDlQ7aEanZ5J510Utxzzz3x85//PFq0+Mdbes6cOTFgwIDYsGFDlfrz58+PZ599Nn7729/G6aefXuW5jz/+OMrKyqot49vf/nZ07NixbgaQwpYtW6J169YZ12/WrFnk5+enWlZOTk7qtgDQGOyO+wpA/XH6N7u80047Ld59991YuHBhZVlZWVnMmzev2kQYEfHaa69FRMQRRxxR7bn8/Pxo165d3XV2B3r06BEvvvhiPPbYY5Wnix199NER8Y9T1x577LG48MILo1OnTrHPPvtERMQbb7wRF154YfTu3TtatWoVe+21V3znO9+pdspWTb+pPvroo6Nfv37x0ksvxTHHHBN77LFHdOvWLW644YYqbWv6TfXZZ58dbdq0ibfeeitGjRoVbdq0ib333jt++MMfRnl5eZX27777bpx55pnRrl272HPPPWP06NHx17/+NePfab/44otx7LHHRqtWrWKfffaJa665JioqKqrV+9///d8YMWJEdO3aNfLy8qJnz55x9dVXV+nP0UcfHQ888EC88cYblev5s2/hy8rKYsqUKTFgwIBo3759tG7dOoYOHRqPPvroF/YRgMatqe8rRER88MEHcfHFF0dhYWHk5eVFr1694vrrr682Z959990xYMCAaNu2bbRr1y4OPvjguOmmmyLi032O73znOxERccwxx1Qu54uuyTJ//vzo169f5OfnR79+/eK+++6rsd6NN94Yhx9+eOy1117RqlWrGDBgQMybN69KnZycnNiyZUvcddddlcv/7Mh8pvs9UN8cqWaX16NHjxgyZEj8z//8T5x44okREfHHP/4xNm7cGN/97nfj5z//eZX63bt3j4iI//qv/4rLLrsscnJyvnAZ7733XrWyFi1a1NopXTNmzIiLLroo2rRpE5deemlERBQUFFSpc+GFF8bee+8dU6ZMiS1btkRExNNPPx1PPfVUfPe734199tkn1qxZE7fcckscffTR8dJLL8Uee+yx0+W+//77ccIJJ8S//Mu/xCmnnBLz5s2LSy65JA4++ODKdbkj5eXlMXz48Bg8eHDceOON8fDDD8e0adOiZ8+eccEFF0REREVFRYwcOTKWLl0aF1xwQfTp0yf+93//N0aPHp3ReikpKYljjjkmtm3bFpMmTYrWrVvHr3/962jVqlW1unfeeWe0adMmJk6cGG3atIlHHnkkpkyZEps2bYr//M//jIiISy+9NDZu3Bj/93//Fz/72c8iIipP1du0aVPcdtttcdppp8XYsWNj8+bNcfvtt8fw4cNj6dKl0b9//4z6DEDj09T3FbZu3RrDhg2Lt956K84777z46le/Gk899VRMnjw53n777cpriSxcuDBOO+20+MY3vhHXX399RES8/PLL8eSTT8aECRPiqKOOin//93+Pn//85/GTn/wkDjzwwIiIyn9r8qc//Sn+9V//Nfr27RtTp06Nd999N8aMGVN5AODzbrrppvjWt74VZ5xxRpSVlcXdd98d3/nOd+L++++PESNGRETEb37zmzj33HNj0KBB8f3vfz8iInr27BkRX36/B+pMAruo2bNnJxGRPP3008kvf/nLpG3btsnWrVuTJEmS73znO8kxxxyTJEmSdO/ePRkxYkRlu61btya9e/dOIiLp3r17cvbZZye33357sm7dumrLKC4uTiKixkfv3r1rdTwHHXRQMmzYsB2O88gjj0y2bdtW5bnPxvt5S5YsSSIi+a//+q/KskcffTSJiOTRRx+tLBs2bFi1eqWlpUnnzp2Tf/3Xf60se/3115OISGbPnl1ZNnr06CQikquuuqrKsg877LBkwIABlf///e9/n0REMmPGjMqy8vLy5Nhjj632mjW5+OKLk4hI/vKXv1SWrV+/Pmnfvn0SEcnrr7++03Vx3nnnJXvssUfy8ccfV5aNGDEi6d69e7W627ZtS0pLS6uUvf/++0lBQUHyve99b6f9BKBx2l32Fa6++uqkdevWyd/+9rcq5ZMmTUqaN2+erF27NkmSJJkwYULSrl27avsTn3fPPfdU22fYmf79+yddunRJPvjgg8qyP/3pT5Xr7vO2n6vLysqSfv36Jccee2yV8tatWyejR4+utqxM93ugvjn9mybhlFNOiY8++ijuv//+2Lx5c9x///01ns4VEdGqVav4y1/+Ej/60Y8i4tMjnOecc0506dIlLrrooigtLa3W5ve//30sXLiwymP27Nl1OqbtjR07Npo3b16l7PNHbD/55JN49913o1evXrHnnnvG8uXLv/A127RpE//2b/9W+f/c3NwYNGhQrF69OqM+nX/++VX+P3To0CptH3zwwWjZsmWMHTu2sqxZs2Yxbty4jF5/wYIF8fWvfz0GDRpUWbb33nvHGWecUa3u59fF5s2bY8OGDTF06NDYunVrrFy58guX1bx588oLuVVUVMR7770X27Zti4EDB2a0LgFo3JryvsI999wTQ4cOjQ4dOsSGDRsqH0VFRVFeXh6PP/54RHx60bQtW7ZUOQ3+y3j77bdjxYoVMXr06Gjfvn1l+XHHHRd9+/atVv/zc/X7778fGzdujKFDh2Y8z37Z/R6oK07/pknYe++9o6ioKObMmRNbt26N8vLy+Pa3v73D+u3bt48bbrghbrjhhnjjjTdi0aJFceONN8Yvf/nLaN++fVxzzTVV6h911FFZX3zkvffeq3Ihk1atWlWZcLK17777Viv76KOPYurUqTF79ux46623IkmSyuc2btz4ha+5zz77VDulrUOHDvHcc899Ydv8/PzYe++9q7V9//33K///xhtvRJcuXaqdjtWrV68vfP3P2g8ePLhaee/evauVvfjii3HZZZfFI488Eps2baryXCbrIiLirrvuimnTpsXKlSvjk08+qSyvad0DsGtpyvsKr776ajz33HPV5uXPrF+/PiI+/SnZ7373uzjxxBOjW7ducfzxx8cpp5wSJ5xwQtbLjPh0no6I2H///as917t372pB9/77749rrrkmVqxYUeWLiUxOr4/48vs9UFeEapqM008/PcaOHRslJSVx4oknZvwbpu7du8f3vve9OPnkk2O//faL3/72t9UmyjT+5V/+JR577LHK/48ePfpL3Vuypt8RX3TRRTF79uy4+OKLY8iQIdG+ffvIycmJ7373uzVezGt72x/5/sznJ6ls2zaEDz74IIYNGxbt2rWLq666Knr27Bn5+fmxfPnyuOSSSzJaF//93/8dZ599dowaNSp+9KMfRadOnaJ58+YxderUygvWALBra6r7ChUVFXHcccfFj3/84xqfP+CAAyIiolOnTrFixYp46KGH4o9//GP88Y9/jNmzZ8dZZ50Vd911V6oxZOr//b//F9/61rfiqKOOiptvvjm6dOkSLVu2jNmzZ8ecOXMyeo0vu98DdUWopsk4+eST47zzzos///nPMXfu3Kzbd+jQIXr27BkvvPBCrfRn2rRpVY7abn8/y+1l+i3t582bNy9Gjx4d06ZNqyz7+OOP44MPPsj6tepC9+7d49FHH42tW7dWOVq9atWqjNu/+uqr1cpfeeWVKv9fvHhxvPvuu3HvvffGUUcdVVn++uuvV2u7o/U8b9682G+//eLee++tUqe4uDijvgLQ+DXVfYWePXvGhx9+GEVFRV+4zNzc3Bg5cmSMHDkyKioq4sILL4xf/epXcfnll0evXr2y2h/57IJumczVv//97yM/Pz8eeuihKvcFr+kU+Z3N1Y15v4fdl99U02S0adMmbrnllrjiiiti5MiRO6z317/+tdr9KCM+PYXppZdeqvHU4jQGDBgQRUVFlY+aflv0ea1bt856UmjevHm1o8q/+MUvqt3WqqEMHz48Pvnkk7j11lsryyoqKmLmzJkZtT/ppJPiz3/+cyxdurSy7J133onf/va3Vep9dtT88+uirKwsbr755mqv2bp16xpPEavpNf7yl7/EkiVLMuorAI1fU91XOOWUU2LJkiXx0EMPVXvugw8+iG3btkXEp7e5/LxmzZrFIYccEhFReTp269atK9t9kS5dukT//v3jrrvuqjK3Lly4MF566aUqdZs3bx45OTlV9lHWrFkT8+fPz3icjX2/h92XI9U0KZncqmnhwoVRXFwc3/rWt+LrX/96tGnTJlavXh133HFHlJaWxhVXXFGtzbx58ypvvfR5xx13XLVbX6U1YMCAuOWWW+Kaa66JXr16RadOneLYY4/daZtvfvOb8Zvf/Cbat28fffv2jSVLlsTDDz8ce+21V6306csaNWpUDBo0KP7jP/4jVq1aFX369Ik//OEPlbcd+aJvw3/84x/Hb37zmzjhhBNiwoQJlbfU6t69e5XffR9++OHRoUOHGD16dPz7v/975OTkxG9+85saT2MfMGBAzJ07NyZOnBj/9E//FG3atImRI0fGN7/5zbj33nvj5JNPjhEjRsTrr78es2bNir59+8aHH35YuysGgAbTFPcVfvSjH8Uf/vCH+OY3vxlnn312DBgwILZs2RLPP/98zJs3L9asWRMdO3aMc889N95777049thjY5999ok33ngjfvGLX0T//v0rb5vVv3//aN68eVx//fWxcePGyMvLi2OPPTY6depUY5+mTp0aI0aMiCOPPDK+973vxXvvvRe/+MUv4qCDDqoyf44YMSKmT58eJ5xwQpx++umxfv36mDlzZvTq1avatVwGDBgQDz/8cEyfPj26du0a++67bwwePLjR7/ewG2u4C4/Dl/P522TszPa3yVi9enUyZcqU5Otf/3rSqVOnpEWLFsnee++djBgxInnkkUeqtN3ZbTIii9tNZKKkpCQZMWJE0rZt2yQiKm+ZsbNxvv/++8mYMWOSjh07Jm3atEmGDx+erFy5MunevXuVW1Hs6JZaBx10ULXXHD16dJVbYOzollqtW7eu1vaz9fV577zzTnL66acnbdu2Tdq3b5+cffbZyZNPPplERHL33Xd/4Xp57rnnkmHDhiX5+flJt27dkquvvjq5/fbbq91S68knn0y+/vWvJ61atUq6du2a/PjHP04eeuihauP+8MMPk9NPPz3Zc889q9zuo6KiIrn22muT7t27J3l5eclhhx2W3H///dXWBwC7jt1lXyFJkmTz5s3J5MmTk169eiW5ublJx44dk8MPPzy58cYbk7KysiRJkmTevHnJ8ccfn3Tq1CnJzc1NvvrVrybnnXde8vbbb1dZzq233prst99+SfPmzTMaw+9///vkwAMPTPLy8pK+ffsm9957b43z5+23357sv//+SV5eXtKnT59k9uzZNe47rFy5MjnqqKOSVq1aJRFRuU+T6X4P1LecJMngikQAtWj+/Plx8sknxxNPPBFHHHFEQ3cHAABSE6qBOvXRRx9VuXJ5eXl5HH/88fHMM89ESUlJjVc1BwCAXYXfVAN16qKLLoqPPvoohgwZEqWlpXHvvffGU089Fddee61ADQDALs+RaqBOzZkzJ6ZNmxarVq2Kjz/+OHr16hUXXHBBjB8/vqG7BgAAX5pQDQAAACm5TzUAAACkJFQDAABASrvEhcoqKiri73//e7Rt2zZycnIaujsA7OaSJInNmzdH165do1kz30/XBnM9AI1NpvP9LhGq//73v0dhYWFDdwMAqnjzzTdjn332aehuNAnmegAaqy+a73eJUN22bduI+HQw7dq1a+DeALVha9m2GPTTRRERsfTSb8QeubvExxFERMSmTZuisLCwcn7iyzPX01iZr2D3lel8v0t8Knx2Gli7du1MtNBEtCjbFs3y9oiIT/+27aSwK3Kacu0x19NYma+AL5rv/RAMAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSyjpUP/744zFy5Mjo2rVr5OTkxPz587+wzeLFi+NrX/ta5OXlRa9eveLOO+9M0VUAoD6Y6wEgc1mH6i1btsShhx4aM2fOzKj+66+/HiNGjIhjjjkmVqxYERdffHGce+658dBDD2XdWQCg7pnrASBzLbJtcOKJJ8aJJ56Ycf1Zs2bFvvvuG9OmTYuIiAMPPDCeeOKJ+NnPfhbDhw/PdvEAQB0z1wNA5ur8N9VLliyJoqKiKmXDhw+PJUuW1PWiAYB6YK4HYHeW9ZHqbJWUlERBQUGVsoKCgti0aVN89NFH0apVq2ptSktLo7S0tPL/mzZtqutuAgApmesB2J3VeahOY+rUqXHllVfW2ev3mPRAnb32l7HmuhEN3QV2A43x/d93it9dptGYPzMa4/ssonGvs93N7jrXRzTe92FjXmeNxa4yXzXW9xhNS2P+zKjvv4E6P/27c+fOsW7duipl69ati3bt2tX4zXVExOTJk2Pjxo2VjzfffLOuuwkApGSuB2B3VudHqocMGRILFiyoUrZw4cIYMmTIDtvk5eVFXl5eXXcNAKgF5noAdmdZH6n+8MMPY8WKFbFixYqI+PQ2GitWrIi1a9dGxKffPJ911lmV9c8///xYvXp1/PjHP46VK1fGzTffHL/73e/iBz/4Qe2MAACoVeZ6AMhc1qH6mWeeicMOOywOO+ywiIiYOHFiHHbYYTFlypSIiHj77bcrJ92IiH333TceeOCBWLhwYRx66KExbdq0uO2229xiAwAaKXM9AGQu69O/jz766EiSZIfP33nnnTW2efbZZ7NdFADQAMz1AJC5Or9QGQAAADRVQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASqlC9cyZM6NHjx6Rn58fgwcPjqVLl+60/owZM6J3797RqlWrKCwsjB/84Afx8ccfp+owAFD3zPUAkJmsQ/XcuXNj4sSJUVxcHMuXL49DDz00hg8fHuvXr6+x/pw5c2LSpElRXFwcL7/8ctx+++0xd+7c+MlPfvKlOw8A1D5zPQBkLutQPX369Bg7dmyMGTMm+vbtG7NmzYo99tgj7rjjjhrrP/XUU3HEEUfE6aefHj169Ijjjz8+TjvttC/8xhsAaBjmegDIXFahuqysLJYtWxZFRUX/eIFmzaKoqCiWLFlSY5vDDz88li1bVjmxrl69OhYsWBAnnXTSDpdTWloamzZtqvIAAOqeuR4AstMim8obNmyI8vLyKCgoqFJeUFAQK1eurLHN6aefHhs2bIgjjzwykiSJbdu2xfnnn7/TU8KmTp0aV155ZTZdAwBqgbkeALJT51f/Xrx4cVx77bVx8803x/Lly+Pee++NBx54IK6++uodtpk8eXJs3Lix8vHmm2/WdTcBgJTM9QDszrI6Ut2xY8do3rx5rFu3rkr5unXronPnzjW2ufzyy+PMM8+Mc889NyIiDj744NiyZUt8//vfj0svvTSaNaue6/Py8iIvLy+brgEAtcBcDwDZyepIdW5ubgwYMCAWLVpUWVZRURGLFi2KIUOG1Nhm69at1SbT5s2bR0REkiTZ9hcAqEPmegDITlZHqiMiJk6cGKNHj46BAwfGoEGDYsaMGbFly5YYM2ZMREScddZZ0a1bt5g6dWpERIwcOTKmT58ehx12WAwePDhWrVoVl19+eYwcObJywgUAGg9zPQBkLutQfeqpp8Y777wTU6ZMiZKSkujfv388+OCDlRc0Wbt2bZVvqy+77LLIycmJyy67LN56663Ye++9Y+TIkfHTn/609kYBANQacz0AZC7rUB0RMX78+Bg/fnyNzy1evLjqAlq0iOLi4iguLk6zKACgAZjrASAzdX71bwAAAGiqhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABIKVWonjlzZvTo0SPy8/Nj8ODBsXTp0p3W/+CDD2LcuHHRpUuXyMvLiwMOOCAWLFiQqsMAQN0z1wNAZlpk22Du3LkxceLEmDVrVgwePDhmzJgRw4cPj1deeSU6depUrX5ZWVkcd9xx0alTp5g3b15069Yt3njjjdhzzz1ro/8AQC0z1wNA5rIO1dOnT4+xY8fGmDFjIiJi1qxZ8cADD8Qdd9wRkyZNqlb/jjvuiPfeey+eeuqpaNmyZURE9OjR48v1GgCoM+Z6AMhcVqd/l5WVxbJly6KoqOgfL9CsWRQVFcWSJUtqbPOHP/whhgwZEuPGjYuCgoLo169fXHvttVFeXr7D5ZSWlsamTZuqPACAumeuB4DsZBWqN2zYEOXl5VFQUFClvKCgIEpKSmpss3r16pg3b16Ul5fHggUL4vLLL49p06bFNddcs8PlTJ06Ndq3b1/5KCwszKabAEBK5noAyE6dX/27oqIiOnXqFL/+9a9jwIABceqpp8all14as2bN2mGbyZMnx8aNGysfb775Zl13EwBIyVwPwO4sq99Ud+zYMZo3bx7r1q2rUr5u3bro3LlzjW26dOkSLVu2jObNm1eWHXjggVFSUhJlZWWRm5tbrU1eXl7k5eVl0zUAoBaY6wEgO1kdqc7NzY0BAwbEokWLKssqKipi0aJFMWTIkBrbHHHEEbFq1aqoqKioLPvb3/4WXbp0qXGSBQAajrkeALKT9enfEydOjFtvvTXuuuuuePnll+OCCy6ILVu2VF4h9KyzzorJkydX1r/gggvivffeiwkTJsTf/va3eOCBB+Laa6+NcePG1d4oAIBaY64HgMxlfUutU089Nd55552YMmVKlJSURP/+/ePBBx+svKDJ2rVro1mzf2T1wsLCeOihh+IHP/hBHHLIIdGtW7eYMGFCXHLJJbU3CgCg1pjrASBzWYfqiIjx48fH+PHja3xu8eLF1cqGDBkSf/7zn9MsCgBoAOZ6AMhMnV/9GwAAAJoqoRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSShWqZ86cGT169Ij8/PwYPHhwLF26NKN2d999d+Tk5MSoUaPSLBYAqCfmegDITNaheu7cuTFx4sQoLi6O5cuXx6GHHhrDhw+P9evX77TdmjVr4oc//GEMHTo0dWcBgLpnrgeAzGUdqqdPnx5jx46NMWPGRN++fWPWrFmxxx57xB133LHDNuXl5XHGGWfElVdeGfvtt9+X6jAAULfM9QCQuaxCdVlZWSxbtiyKior+8QLNmkVRUVEsWbJkh+2uuuqq6NSpU5xzzjnpewoA1DlzPQBkp0U2lTds2BDl5eVRUFBQpbygoCBWrlxZY5snnngibr/99lixYkXGyyktLY3S0tLK/2/atCmbbgIAKZnrASA7dXr1782bN8eZZ54Zt956a3Ts2DHjdlOnTo327dtXPgoLC+uwlwBAWuZ6AHZ3WR2p7tixYzRv3jzWrVtXpXzdunXRuXPnavVfe+21WLNmTYwcObKyrKKi4tMFt2gRr7zySvTs2bNau8mTJ8fEiRMr/79p0yaTLQDUA3M9AGQnq1Cdm5sbAwYMiEWLFlXeKqOioiIWLVoU48ePr1a/T58+8fzzz1cpu+yyy2Lz5s1x00037XDyzMvLi7y8vGy6BgDUAnM9AGQnq1AdETFx4sQYPXp0DBw4MAYNGhQzZsyILVu2xJgxYyIi4qyzzopu3brF1KlTIz8/P/r161el/Z577hkRUa0cAGgczPUAkLmsQ/Wpp54a77zzTkyZMiVKSkqif//+8eCDD1Ze0GTt2rXRrFmd/lQbAKhD5noAyFzWoToiYvz48TWeAhYRsXjx4p22vfPOO9MsEgCoR+Z6AMiMr5kBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAICWhGgAAAFISqgEAACAloRoAAABSEqoBAAAgJaEaAAAAUhKqAQAAIKVUoXrmzJnRo0ePyM/Pj8GDB8fSpUt3WPfWW2+NoUOHRocOHaJDhw5RVFS00/oAQMMz1wNAZrIO1XPnzo2JEydGcXFxLF++PA499NAYPnx4rF+/vsb6ixcvjtNOOy0effTRWLJkSRQWFsbxxx8fb7311pfuPABQ+8z1AJC5rEP19OnTY+zYsTFmzJjo27dvzJo1K/bYY4+44447aqz/29/+Ni688MLo379/9OnTJ2677baoqKiIRYsWfenOAwC1z1wPAJnLKlSXlZXFsmXLoqio6B8v0KxZFBUVxZIlSzJ6ja1bt8Ynn3wSX/nKV3ZYp7S0NDZt2lTlAQDUPXM9AGQnq1C9YcOGKC8vj4KCgirlBQUFUVJSktFrXHLJJdG1a9cqk/X2pk6dGu3bt698FBYWZtNNACAlcz0AZKder/593XXXxd133x333Xdf5Ofn77De5MmTY+PGjZWPN998sx57CQCkZa4HYHfTIpvKHTt2jObNm8e6deuqlK9bty46d+6807Y33nhjXHfddfHwww/HIYccstO6eXl5kZeXl03XAIBaYK4HgOxkdaQ6Nzc3BgwYUOXCI59diGTIkCE7bHfDDTfE1VdfHQ8++GAMHDgwfW8BgDplrgeA7GR1pDoiYuLEiTF69OgYOHBgDBo0KGbMmBFbtmyJMWPGRETEWWedFd26dYupU6dGRMT1118fU6ZMiTlz5kSPHj0qf4/Vpk2baNOmTS0OBQCoDeZ6AMhc1qH61FNPjXfeeSemTJkSJSUl0b9//3jwwQcrL2iydu3aaNbsHwfAb7nlligrK4tvf/vbVV6nuLg4rrjiii/XewCg1pnrASBzWYfqiIjx48fH+PHja3xu8eLFVf6/Zs2aNIsAABqQuR4AMlOvV/8GAACApkSoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJSEagAAAEhJqAYAAICUhGoAAABISagGAACAlIRqAAAASEmoBgAAgJRSheqZM2dGjx49Ij8/PwYPHhxLly7daf177rkn+vTpE/n5+XHwwQfHggULUnUWAKgf5noAyEzWoXru3LkxceLEKC4ujuXLl8ehhx4aw4cPj/Xr19dY/6mnnorTTjstzjnnnHj22Wdj1KhRMWrUqHjhhRe+dOcBgNpnrgeAzGUdqqdPnx5jx46NMWPGRN++fWPWrFmxxx57xB133FFj/ZtuuilOOOGE+NGPfhQHHnhgXH311fG1r30tfvnLX37pzgMAtc9cDwCZa5FN5bKysli2bFlMnjy5sqxZs2ZRVFQUS5YsqbHNkiVLYuLEiVXKhg8fHvPnz9/hckpLS6O0tLTy/xs3boyIiE2bNmXT3R2qKN1aK69T22prfLAzjfX9T/Ya82dGY32f1dY6++x1kiSplddrTMz1da+x/u025nVGdhrre4ympTF/ZtT3fJ9VqN6wYUOUl5dHQUFBlfKCgoJYuXJljW1KSkpqrF9SUrLD5UydOjWuvPLKauWFhYXZdHeX035GQ/cA2JX4zMheba+zzZs3R/v27Wv3RRuYub7u+dulrnmPsbur7/k+q1BdXyZPnlzlG+8PPvggunfvHmvXrm1yOy+NxaZNm6KwsDDefPPNaNeuXUN3p8mynuuedVz3rONPv7HevHlzdO3ataG7ssvafq6vqKiI9957L/baa6/IycnZadum8h40jsbFOBoX42hcdtdxZDrfZxWqO3bsGM2bN49169ZVKV+3bl107ty5xjadO3fOqn5ERF5eXuTl5VUrb9++/S69EXcF7dq1s47rgfVc96zjure7r+Om+iVvQ871e+65Z1Z9bSrvQeNoXIyjcTGOxmV3HEcm831WFyrLzc2NAQMGxKJFiyrLKioqYtGiRTFkyJAa2wwZMqRK/YiIhQsX7rA+ANBwzPUAkJ2sT/+eOHFijB49OgYOHBiDBg2KGTNmxJYtW2LMmDEREXHWWWdFt27dYurUqRERMWHChBg2bFhMmzYtRowYEXfffXc888wz8etf/7p2RwIA1ApzPQBkLutQfeqpp8Y777wTU6ZMiZKSkujfv388+OCDlRcoWbt2bTRr9o8D4IcffnjMmTMnLrvssvjJT34S+++/f8yfPz/69euX8TLz8vKiuLi4xlPCqR3Wcf2wnuuedVz3rOOmryHm+mw0lfegcTQuxtG4GEfjYhw7l5M0xfuBAAAAQD3I6jfVAAAAwD8I1QAAAJCSUA0AAAApCdUAAACQUoOH6qlTp8Y//dM/Rdu2baNTp04xatSoeOWVV3ba5tZbb42hQ4dGhw4dokOHDlFUVBRLly6tpx7vmtKs58+7++67IycnJ0aNGlV3ndzFpV3HH3zwQYwbNy66dOkSeXl5ccABB8SCBQvqoce7nrTreMaMGdG7d+9o1apVFBYWxg9+8IP4+OOP66HHu55bbrklDjnkkGjXrl20a9cuhgwZEn/84x932uaee+6JPn36RH5+fhx88MHev3wpjz/+eIwcOTK6du0aOTk5MX/+/J3WX7x4ceTk5FR7lJSU1E+HdyDt51Vj+3tKM44777yz2vbIz8+vpx7XrKl8tmU7jsa4LWpy3XXXRU5OTlx88cU7rdcYt8nnZTKOxrhNrrjiimp96tOnz07bNMZtke04anNbNHiofuyxx2LcuHHx5z//ORYuXBiffPJJHH/88bFly5Ydtlm8eHGcdtpp8eijj8aSJUuisLAwjj/++Hjrrbfqsee7ljTr+TNr1qyJH/7whzF06NB66OmuK806Lisri+OOOy7WrFkT8+bNi1deeSVuvfXW6NatWz32fNeRZh3PmTMnJk2aFMXFxfHyyy/H7bffHnPnzo2f/OQn9djzXcc+++wT1113XSxbtiyeeeaZOPbYY+Of//mf48UXX6yx/lNPPRWnnXZanHPOOfHss8/GqFGjYtSoUfHCCy/Uc89pKrZs2RKHHnpozJw5M6t2r7zySrz99tuVj06dOtVRDzOT5vOqMf49pd1/aNeuXZXt8cYbb9RTj2vWVD7bsh1HROPbFtt7+umn41e/+lUccsghO63XWLfJZzIdR0Tj3CYHHXRQlT498cQTO6zbmLdFNuOIqMVtkTQy69evTyIieeyxxzJus23btqRt27bJXXfdVYc9a1oyXc/btm1LDj/88OS2225LRo8enfzzP/9z/XSwCchkHd9yyy3Jfvvtl5SVldVjz5qOTNbxuHHjkmOPPbZK2cSJE5MjjjiirrvXZHTo0CG57bbbanzulFNOSUaMGFGlbPDgwcl5551XH12jiYuI5L777ttpnUcffTSJiOT999+vlz6llcnn1a7w95TJOGbPnp20b9++/jqVUlP5bNvZOBr7tti8eXOy//77JwsXLkyGDRuWTJgwYYd1G/M2yWYcjXGbFBcXJ4ceemjG9Rvrtsh2HLW5LRr8SPX2Nm7cGBERX/nKVzJus3Xr1vjkk0+yarO7y3Q9X3XVVdGpU6c455xz6qNbTUom6/gPf/hDDBkyJMaNGxcFBQXRr1+/uPbaa6O8vLy+urlLy2QdH3744bFs2bLKn4isXr06FixYECeddFK99HFXVl5eHnfffXds2bIlhgwZUmOdJUuWRFFRUZWy4cOHx5IlS+qji1Cpf//+0aVLlzjuuOPiySefbOjuVJPJ59Wu8PeU6f7Dhx9+GN27d4/CwsIvPJJa35rKZ1sm44ho3Nti3LhxMWLEiGrruiaNeZtkM46IxrlNXn311ejatWvst99+ccYZZ8TatWt3WLcxb4tsxhFRe9uiRapWdaSioiIuvvjiOOKII6Jfv34Zt7vkkkuia9euGb+Rd3eZrucnnngibr/99lixYkX9da6JyHQdr169Oh555JE444wzYsGCBbFq1aq48MIL45NPPoni4uJ67PGuJ9N1fPrpp8eGDRviyCOPjCRJYtu2bXH++ec7/Xsnnn/++RgyZEh8/PHH0aZNm7jvvvuib9++NdYtKSmJgoKCKmUFBQUN/ntWdh9dunSJWbNmxcCBA6O0tDRuu+22OProo+Mvf/lLfO1rX2vo7kVE5p9Xjf3vKdNx9O7dO+6444445JBDYuPGjXHjjTfG4YcfHi+++GLss88+9djjqprKZ1s242is2yLi0+v1LF++PJ5++umM6jfWbZLtOBrjNhk8eHDceeed0bt373j77bfjyiuvjKFDh8YLL7wQbdu2rVa/sW6LbMdRq9uiVo5315Lzzz8/6d69e/Lmm29m3Gbq1KlJhw4dkr/+9a912LOmJZP1vGnTpqRHjx7JggULKsuc/p25TN/L+++/f1JYWJhs27atsmzatGlJ586d67qLu7xM1/Gjjz6aFBQUJLfeemvy3HPPJffee29SWFiYXHXVVfXU011PaWlp8uqrrybPPPNMMmnSpKRjx47Jiy++WGPdli1bJnPmzKlSNnPmzKRTp0710VWauMjg9O+aHHXUUcm//du/1X6HUsr086qx/z2l2U9LkiQpKytLevbsmVx22WV11LPMNJXPtmzGsb3Gsi3Wrl2bdOrUqcr++xedNt0Yt0macWyvsWyTz3v//feTdu3a7fBnBY1xW9Tki8axvS+zLRrNkerx48fH/fffH48//njG3wzceOONcd1118XDDz+c0UUByHw9v/baa7FmzZoYOXJkZVlFRUVERLRo0SJeeeWV6NmzZ533d1eUzXu5S5cu0bJly2jevHll2YEHHhglJSVRVlYWubm5dd3dXVI26/jyyy+PM888M84999yIiDj44INjy5Yt8f3vfz8uvfTSaNas0f0KpsHl5uZGr169IiJiwIAB8fTTT8dNN90Uv/rVr6rV7dy5c6xbt65K2bp166Jz58710leoyaBBg77w4jT1JZvPq8b895RmP+0zLVu2jMMOOyxWrVpVR73LTFP5bMtmHNtrLNti2bJlsX79+ipnk5SXl8fjjz8ev/zlL6O0tLTKvlFE49wmacaxvcayTT5vzz33jAMOOGCHfWqM26ImXzSO7X2ZbdHge5NJksT48ePjvvvui0ceeST23XffjNrdcMMNcfXVV8eDDz4YAwcOrONe7vqyXc99+vSJ559/PlasWFH5+Na3vhXHHHNMrFixIgoLC+up57uONO/lI444IlatWlX5hUVExN/+9rfo0qWLQF2DNOt469at1YLzZxNckiR10s+mpqKiIkpLS2t8bsiQIbFo0aIqZQsXLtzp7/ugrq1YsSK6dOnSoH1I83nVGP+e0u6nfV55eXk8//zzDb5NttdUPtt2No7tNZZt8Y1vfKPafubAgQPjjDPOiBUrVtQYRBvjNkkzju01lm3yeR9++GG89tprO+xTY9wWNfmicWzvS22LrI9t17ILLrggad++fbJ48eLk7bffrnxs3bq1ss6ZZ56ZTJo0qfL/1113XZKbm5vMmzevSpvNmzc3xBB2CWnW8/ac/r1zadbx2rVrk7Zt2ybjx49PXnnlleT+++9POnXqlFxzzTUNMYRGL806Li4uTtq2bZv8z//8T7J69erkT3/6U9KzZ8/klFNOaYghNHqTJk1KHnvsseT1119PnnvuuWTSpElJTk5O8qc//SlJkurr98knn0xatGiR3HjjjcnLL7+cFBcXJy1btkyef/75hhoCu7jNmzcnzz77bPLss88mEZFMnz49efbZZ5M33ngjSZJP36NnnnlmZf2f/exnyfz585NXX301ef7555MJEyYkzZo1Sx5++OGGGkKSJOk+rxrj31OacVx55ZXJQw89lLz22mvJsmXLku9+97tJfn5+xqco14Wm8tmW7Tga47bYke1Pm95Vtsn2vmgcjXGb/Md//EeyePHi5PXXX0+efPLJpKioKOnYsWOyfv36JEl2nW2R7Thqc1s0eKiOiBofs2fPrqwzbNiwZPTo0ZX/7969e41tiouL673/u4o063l7QvXOpV3HTz31VDJ48OAkLy8v2W+//ZKf/vSnVX5jzT+kWceffPJJcsUVVyQ9e/ZM8vPzk8LCwuTCCy9s9LffaSjf+973ku7duye5ubnJ3nvvnXzjG9+o3FlLkprfw7/73e+SAw44IMnNzU0OOuig5IEHHqjnXtOUfHaLrO0fn73vRo8enQwbNqyy/vXXX1/59/2Vr3wlOfroo5NHHnmkYTr/OWnnhMb295RmHBdffHHy1a9+NcnNzU0KCgqSk046KVm+fHn9d/5zmspnW7bjaIzbYke2D6O7yjbZ3heNozFuk1NPPTXp0qVLkpubm3Tr1i059dRTk1WrVlU+v6tsi2zHUZvbIidJnP8IAAAAaTT4b6oBAABgVyVUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASkI1AAAApCRUAwAAQEpCNQAAAKQkVAMAAEBKQjUAAACkJFQDAABASv8feQ7zvryDqrEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1200x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "import numpy as np\n",
    "import matplotlib.pylab as plt\n",
    "import pandas as pd\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "\n",
    "advertising_df = pd.read_csv(\"../Daten/Advertising.csv\", index_col= \"Unnamed: 0\")\n",
    "advertising_df.reset_index(drop = True, inplace = True)\n",
    "X = advertising_df.drop([\"sales\"], axis = 1)\n",
    "y = advertising_df.sales\n",
    "\n",
    "n_splits = 5\n",
    "cv = KFold(n_splits)\n",
    "\n",
    "mse_train_results, mse_test_results = [], []\n",
    "for ii, (train_idx, test_idx) in enumerate(cv.split(X, y)):\n",
    "    X_train, X_test = X.loc[train_idx, :], X.loc[test_idx, :]\n",
    "    y_train, y_test = y.loc[train_idx], y.loc[test_idx]\n",
    "    \n",
    "    linear_regression = LinearRegression()\n",
    "    linear_regression.fit(X_train, y_train)\n",
    "    y_train_hat, y_test_hat = linear_regression.predict(X_train), linear_regression.predict(X_test)\n",
    "    \n",
    "    mse_train, mse_test = mean_squared_error(y_train, y_train_hat), mean_squared_error(y_test, y_test_hat)\n",
    "    mse_train_results.append(mse_train)\n",
    "    mse_test_results.append(mse_test)\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize = (12, 5))\n",
    "axs[0].hist(mse_train_results)\n",
    "axs[0].axvline(np.mean(mse_train_results))\n",
    "axs[0].set_title(\"MSE - training data\")\n",
    "\n",
    "axs[1].hist(mse_test_results)\n",
    "axs[1].axvline(np.mean(mse_test_results))\n",
    "axs[1].set_title(\"MSE - test data\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Außderdem sollte man bei der Interpretion der Histogramme in der obigen Zelle vorsichtig sein. Es ist gerade der Vorteil der Kreuzvalidierung, dass wir die Evaluation des Modells nicht anhand der einzelnen Testfehler durchführen würden, sondern anhand deren Durchschnitt. Würden wir dieses Vorgehen mehrfach durchführen, wäre die Schwankung der geschätzten Werte für $CV_{(k)}$ deutlich geringer als bei der Methodes des einfachen Aufteilens der Daten. Um dies zu demonstrieren, verzichten wir für einen Moment auf die Reproduzierbarkeit des Wertes für $CV_{(k)}$ und führen die Kreuzvalidierung mit $k=5$ mehrfach durch indem wir die Teile der Daten doch zufällig aufteilen. Im Anschluss betrachten wir das Histogramm für alle Realisierungen von $CV_{(k)}$ über alle K-fold Durchläufe. Hierbei wird deutlich, dass die Beurteilung eines Modells anhand der K-Fold Validierung deutlich weniger mit unterschiedlichen (zufälligen) Aufteilungen von Trainings- und Testdaten variiert. Genau hierin besteht der Vorteil, da die Evaluation der Generalisierungsfähigkeit eines Modells stabiler mit Blick auf das Ziehen zufälliger Trainings- und Testdaten ist. \n",
    "\n",
    "Bei der hier vorgestellten Art der Kreuzvalidierung wird davon ausgegangen, dass die Beobachtungen der Daten keine Abhängigkeiten untereinander aufweisen. Diese Annahme ist verletzt, wenn Beobachtungen geographische, zeitliche oder andere Arten von Abhängigkeiten aufweisen. Beispielsweise kann es bei Zeitreihendaten problematisch sein, wenn durch die Art der fold-Bildung zukünftige Daten für die Prognose vergangenere Daten verwendet werden könnnen. Hierdurch können systematische Fehleinschätungen der Modellgüte entstehen, weshalb derartige Zusammenhänge bei Bedarf angepasst werden. Dies muss jedoch je nach Problemstellung spezifisch durchgeführt werden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAHDCAYAAAAjsZnKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABCd0lEQVR4nO3de5xNdd//8fc2Y/aMMTMYh5mpwaByTCVcKIeaK7mETqSIdEWFxFyJ6c6xGB2uUmiUW6gc4krqh0gOUQ5h6Hg1Dg0mhS4xk5GNme/vj277sps9Zm3W3nuY1/PxWI/a33X6fmbZe633Xmuv5TDGGAEAAAAAgGKVCXYHAAAAAAC4WBCiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogF4VbNmTT3wwAPnNW/btm3Vtm1bW/tzoRwOh0aPHh3sbgAAAC/WrFkjh8OhNWvWBLsrQLEI0Qio3bt36+GHH1atWrUUHh6u6OhotWrVSq+88op+//13ZWRkyOFw6Omnny5yGTt37pTD4VBKSkqR05z5IC5qmDdvnj/KC6j169dr9OjROnr0aLC7ckn77rvvNHr0aO3ZsyfYXQGAUoFjBfsE6lhh/PjxWrRokV/XcS5Lly7li3IEVGiwO4DSY8mSJerataucTqd69eqlhg0b6uTJk/rss880dOhQffvtt3rjjTdUt25dzZ07V88++6zX5cyZM0eS1LNnz2LXOWjQIDVt2rRQe4sWLS6smBJg/fr1GjNmjB544AFVqFDB9uVnZmaqTJnz+57t448/trk3wfPdd99pzJgxatu2rWrWrBns7gDAJY1jBXv5+1jhjPHjx+vuu+/W7bff7rd1nMvSpUs1ZcoUgjQChhCNgMjKylL37t1Vo0YNrVq1SvHx8e5xAwYM0K5du7RkyRJJUo8ePTRixAht3LhRf/nLXwota+7cuapbt66uu+66Ytd744036u677/aprwUFBTp58qTCw8MLjcvLy1NkZKRPy/uz48ePq1y5che0DF+dq6aiOJ3O815fWFjYec8LACidOFb4r2AcKwCwjsu5ERDPP/+8jh07punTp3vsFM+oU6eOHn/8cUl/7Bil/36LfLatW7cqMzPTPY0dHA6HBg4cqNmzZ6tBgwZyOp1atmyZZs6cKYfDoU8//VT9+/dX1apVdfnll7vne+2119zTJyQkaMCAAYUul2rbtq0aNmyorVu3qnXr1ipXrpyeeuqpC+7z6NGjNXToUElSUlKS+9KzM5ccF1WTJL344otq2bKlYmNjFRERoSZNmuhf//pXoXX8+TfRZ/4en3/+uVJSUlSlShVFRkbqjjvu0C+//FKo7rN/E33mkrn58+dr3LhxuvzyyxUeHq6bb75Zu3btKrTuKVOmqFatWoqIiFCzZs20bt06y7+zdrlcGjJkiKpUqaKoqCh17txZP/74Y6Hp9u7dq/79++uqq65SRESEYmNj1bVrV4/LtmfOnKmuXbtKktq1a+f+O5/5vdYHH3ygjh07KiEhQU6nU7Vr19Yzzzyj/Pz8YvsJAPDEsUJgjxUk6Z133lGTJk0UERGhSpUqqXv37srOzvZYzs6dO3XXXXcpLi5O4eHhuvzyy9W9e3fl5OS4/zZ5eXmaNWuWex3F3VPlxx9/1O23367IyEhVrVpVQ4YMkcvlKjTdunXr1LVrV1WvXl1Op1OJiYkaMmSIfv/9d/c0DzzwgKZMmeLuy5nhDKvHPYAvOBONgPh//+//qVatWmrZsmWx0yYlJally5aaP3++Xn75ZYWEhLjHndlZ3nfffZbW+9tvv+k///lPofbY2FiPD9hVq1Zp/vz5GjhwoCpXrqyaNWtq+/btkqT+/furSpUqGjlypPLy8iT9sWMaM2aMkpOT9eijjyozM1Pp6enavHmzPv/8c5UtW9a97MOHD6tDhw7q3r27evbsqWrVqlnq+7nceeed2rFjh+bOnauXX35ZlStXliRVqVLlnDVJ0iuvvKLOnTurR48eOnnypObNm6euXbtq8eLF6tixY7Hrfuyxx1SxYkWNGjVKe/bs0cSJEzVw4EC9++67xc47YcIElSlTRk888YRycnL0/PPPq0ePHtq0aZN7mvT0dA0cOFA33nijhgwZoj179uj2229XxYoVPQ5MivLQQw/pnXfe0X333aeWLVtq1apVXuvavHmz1q9fr+7du+vyyy/Xnj17lJ6errZt2+q7775TuXLl1Lp1aw0aNEivvvqqnnrqKdWrV0+S3P+dOXOmypcvr5SUFJUvX16rVq3SyJEjlZubqxdeeKHYvgIA/otjhcAeK4wbN04jRoxQt27d9NBDD+mXX37RpEmT1Lp1a23btk0VKlTQyZMn1b59e7lcLj322GOKi4vT/v37tXjxYh09elQxMTF6++239dBDD6lZs2bq16+fJKl27dpF9uv333/XzTffrH379mnQoEFKSEjQ22+/rVWrVhWadsGCBTp+/LgeffRRxcbG6osvvtCkSZP0448/asGCBZKkhx9+WD/99JNWrFiht99+u9AyLvS4B/DKAH6Wk5NjJJkuXbpYnmfKlClGklm+fLm7LT8/31x22WWmRYsWxc6/evVqI6nI4eeff3ZPK8mUKVPGfPvttx7LmDFjhpFkbrjhBnP69Gl3+6FDh0xYWJi55ZZbTH5+vrt98uTJRpJ588033W1t2rQxkszUqVMt127VCy+8YCSZrKysQuOKqskYY44fP+7x+uTJk6Zhw4bmpptu8mivUaOG6d27t/v1mb9HcnKyKSgocLcPGTLEhISEmKNHj7rb2rRpY9q0aeN+fWZ71KtXz7hcLnf7K6+8YiSZr7/+2hhjjMvlMrGxsaZp06bm1KlT7ulmzpxpJHks05vt27cbSaZ///4e7ffdd5+RZEaNGlXk38EYYzZs2GAkmbfeesvdtmDBAiPJrF69utD03pbx8MMPm3LlypkTJ06cs68AgP/iWCGwxwp79uwxISEhZty4cR7tX3/9tQkNDXW3b9u2zUgyCxYsOOd6IiMjPY4ZzmXixIlGkpk/f767LS8vz9SpU6fQ/tbbfjYtLc04HA6zd+9ed9uAAQNMUbHG6nEP4Asu54bf5ebmSpKioqIsz3PPPfeobNmyHpdpffrpp9q/f79Pl2eNHDlSK1asKDRUqlTJY7o2bdqofv36XpfRt29fj2+4P/nkE508eVKDBw/2uPFW3759FR0d7f691hlOp1N9+vSx3Ge7FFVTRESE+/+PHDminJwc3XjjjcrIyLC03H79+nl8M3/jjTcqPz9fe/fuLXbePn36ePxe+sYbb5Qk/fDDD5KkLVu26PDhw+rbt69CQ/97oUyPHj1UsWLFYpe/dOlSSX/cJOZsgwcPLjTt2X+HU6dO6fDhw6pTp44qVKhg+W9x9jLOnMm48cYbdfz4cX3//feWlgEA4Fgh0McKCxcuVEFBgbp166b//Oc/7iEuLk5XXHGFVq9eLUmKiYmRJC1fvlzHjx+3Zd1Lly5VfHy8x+/Qy5Ur5z6Lfbaz97N5eXn6z3/+o5YtW8oYo23btlla34Ue9wDecDk3/C46OlrSHyHDqtjYWLVv317vv/++pk6dqvDwcM2ZM0ehoaHq1q2b5eU0atRIycnJxU6XlJRkedyZsHjVVVd5tIeFhalWrVqFwuRll11m6UZbv//+u/v3RWfExcUVO19Riqpp8eLFevbZZ7V9+3aP3x+dHYzPpXr16h6vz4TbI0eOXPC8Z/52derU8ZguNDTU0p2x9+7dqzJlyhS6jOzP20r64++dlpamGTNmaP/+/TLGuMf9eTsU5dtvv9XTTz+tVatWuQ8AfV0GAIBjhUAfK+zcuVPGGF1xxRVex5+51DwpKUkpKSl66aWXNHv2bN14443q3Lmzevbs6Q7Yvtq7d6/q1KlT6LjD27563759GjlypD788MNCxxlW97MXetwDeEOIht9FR0crISFB33zzjU/z9ezZU4sXL9bixYvVuXNnvffee7rllls8fvdrl7O/pfRl3IUu+2zvvvtuoW+hzw52dqx33bp16ty5s1q3bq3XXntN8fHxKlu2rGbMmOH15izenP1Nu699vZB57fbYY49pxowZGjx4sFq0aKGYmBg5HA51795dBQUFxc5/9OhRtWnTRtHR0Ro7dqxq166t8PBwZWRkaNiwYZaWAQD4A8cKgT1WKCgokMPh0EcffeR131y+fHn3///zn//UAw88oA8++EAff/yxBg0apLS0NG3cuNHSvUrOV35+vv7617/q119/1bBhw1S3bl1FRkZq//79euCBByztZ+047gG8IUQjIG677Ta98cYb2rBhg+XnLnbu3FlRUVGaM2eOypYtqyNHjth6p83zVaNGDUl/PEe5Vq1a7vaTJ08qKyvL0rfZ3rRv314rVqywPP35fIP63nvvKTw8XMuXL/d4hNWMGTN8XpY/nPnb7tq1S+3atXO3nz59Wnv27NHVV19d7PwFBQXavXu3xzfamZmZhab917/+pd69e+uf//ynu+3EiROF7ppa1N95zZo1Onz4sBYuXKjWrVu727Oyss7ZRwCAdxwrFM+uY4XatWvLGKOkpCRdeeWVxS6nUaNGatSokZ5++mmtX79erVq10tSpU93P6fblmKRGjRr65ptvZIzxmO/P++qvv/5aO3bs0KxZs9SrVy93u7f6i1p/ST/uwcWL30QjIJ588klFRkbqoYce0sGDBwuN3717t1555RWPtoiICN1xxx1aunSp0tPTFRkZqS5dugSqy0VKTk5WWFiYXn31VY9vf6dPn66cnJzzvtNjfHy8kpOTPYZzOfMMyj+HvnMJCQmRw+HweATTnj17tGjRovPpsu2uv/56xcbGatq0aTp9+rS7ffbs2ZYuF+/QoYMk6dVXX/VonzhxYqFpQ0JCCn17P2nSpEKPpyrq73zmm/uzl3Hy5Em99tprxfYTAFAYxwrFs+tY4c4771RISIjGjBlTaF9ojNHhw4cl/fFb9bP3x9IfgbpMmTIel0ZHRkZaPh7529/+pp9++snjMVPHjx/XG2+84TGdt/2sMabQv4Fz1VnSj3tw8eJMNAKidu3amjNnju655x7Vq1dPvXr1UsOGDXXy5EmtX79eCxYs8PpMwZ49e+qtt97S8uXL1aNHD/eHpFXr1q3TiRMnCrVfffXVxZ7VLEqVKlWUmpqqMWPG6NZbb1Xnzp2VmZmp1157TU2bNlXPnj3Pa7m+atKkiSTpf/7nf9S9e3eVLVtWnTp1OuffqGPHjnrppZd066236r777tOhQ4c0ZcoU1alTR1999VVA+n0uYWFhGj16tB577DHddNNN6tatm/bs2aOZM2eqdu3axX7Tfc011+jee+/Va6+9ppycHLVs2VIrV670+izq2267TW+//bZiYmJUv359bdiwQZ988oliY2MLLTMkJETPPfeccnJy5HQ6ddNNN6lly5aqWLGievfurUGDBsnhcOjtt98OyqXpAHAp4FjBfkUdK9SuXVvPPvusUlNT3Y+SjIqKUlZWlt5//33169dPTzzxhFatWqWBAweqa9euuvLKK3X69Gm9/fbbCgkJ0V133eWxnk8++UQvvfSSEhISlJSUpObNm3vtU9++fTV58mT16tVLW7duVXx8vN5++22VK1fOY7q6deuqdu3aeuKJJ7R//35FR0frvffe8/ql+pk6Bw0apPbt2yskJETdu3cv8cc9uIgF+nbgKN127Nhh+vbta2rWrGnCwsJMVFSUadWqlZk0aZLXRwKdPn3axMfHG0lm6dKlltdT3GMrzn7UkSQzYMCAQss489iKzZs3e13H5MmTTd26dU3ZsmVNtWrVzKOPPmqOHDniMU2bNm1MgwYNLPfbV88884y57LLLTJkyZTweYVFUTcYYM336dHPFFVcYp9Np6tata2bMmGFGjRpV6NEQRT3i6s9/jzN/67MfSVHUI67+/IiMrKwsI8nMmDHDo/3VV181NWrUME6n0zRr1sx8/vnnpkmTJubWW28t9m/y+++/m0GDBpnY2FgTGRlpOnXqZLKzswtt9yNHjpg+ffqYypUrm/Lly5v27dub77//vlDdxhgzbdo0U6tWLRMSEuJR6+eff27+8pe/mIiICJOQkGCefPJJs3z58iIfiQUAKB7HCvYq6ljBGGPee+89c8MNN5jIyEgTGRlp6tatawYMGGAyMzONMcb88MMP5sEHHzS1a9c24eHhplKlSqZdu3bmk08+8VjH999/b1q3bm0iIiKMpGIfd7V3717TuXNnU65cOVO5cmXz+OOPm2XLlhXaf3733XcmOTnZlC9f3lSuXNn07dvXfPnll4WOHU6fPm0ee+wxU6VKFeNwODyOaawe9wC+cBjDaRMAJVtBQYGqVKmiO++8U9OmTQt2dwAAAFCK8ZtoACXKiRMnCl0S/dZbb+nXX39V27Ztg9MpAAAA4P9wJhpAibJmzRoNGTJEXbt2VWxsrDIyMjR9+nTVq1dPW7dutfQcTQAAAMBfuLEYgBKlZs2aSkxM1Kuvvqpff/1VlSpVUq9evTRhwgQCNAAAAIKOM9EAAAAAAFjEb6IBAAAAALCIEA0AAAAAgEUl7jfRBQUF+umnnxQVFSWHwxHs7gAAIGOMfvvtNyUkJKhMGb5/tgP7ewBASeLLvr7EheiffvpJiYmJwe4GAACFZGdn6/LLLw92Ny4J7O8BACWRlX19iQvRUVFRkv7ofHR0dJB7A+BicvzkaTUbt1KS9MX/3KxyYSXuIw4XqdzcXCUmJrr3Ubhw7O9hFz77AdjBl319ifuUOXNJV3R0NDtVAD4JPXlaZZzlJP3xGcKBFOzGZcf2YX8Pu/DZD8BOVvb1/LALAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAin0P02rVr1alTJyUkJMjhcGjRokXucadOndKwYcPUqFEjRUZGKiEhQb169dJPP/1kZ58BAAAAAAgKn0N0Xl6eGjdurClTphQad/z4cWVkZGjEiBHKyMjQwoULlZmZqc6dO9vSWQAAAAAAginU1xk6dOigDh06eB0XExOjFStWeLRNnjxZzZo10759+1S9evXz6yUAAAAAACWA338TnZOTI4fDoQoVKvh7VQAAAAAA+JXPZ6J9ceLECQ0bNkz33nuvoqOjvU7jcrnkcrncr3Nzc/3ZJQAAAAAAzpvfQvSpU6fUrVs3GWOUnp5e5HRpaWkaM2aMv7oBAD6rOXxJsLvg1Z4JHYPdBQCXED7rAOD8+OVy7jMBeu/evVqxYkWRZ6ElKTU1VTk5Oe4hOzvbH10CAAAAAOCC2X4m+kyA3rlzp1avXq3Y2NhzTu90OuV0Ou3uBgAAAAAAtvM5RB87dky7du1yv87KytL27dtVqVIlxcfH6+6771ZGRoYWL16s/Px8HThwQJJUqVIlhYWF2ddzAAAAAAACzOcQvWXLFrVr1879OiUlRZLUu3dvjR49Wh9++KEk6ZprrvGYb/Xq1Wrbtu359xQAAAAAgCDzOUS3bdtWxpgix59rHAAAAAAAFzO/PycaAAAAAIBLBSEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAHhYu3atOnXqpISEBDkcDi1atMg97tSpUxo2bJgaNWqkyMhIJSQkqFevXvrpp5+C12EAAAKIEA0AADzk5eWpcePGmjJlSqFxx48fV0ZGhkaMGKGMjAwtXLhQmZmZ6ty5cxB6CgBA4IUGuwMAAKBk6dChgzp06OB1XExMjFasWOHRNnnyZDVr1kz79u1T9erVA9FFAACChjPRAADgguTk5MjhcKhChQrB7goAAH7HmWgAAHDeTpw4oWHDhunee+9VdHR0kdO5XC65XC7369zc3EB0DwAA23EmGgAAnJdTp06pW7duMsYoPT39nNOmpaUpJibGPSQmJgaolwAA2IsQDQAAfHYmQO/du1crVqw451loSUpNTVVOTo57yM7ODlBPAQCwF5dzAwiKmsOX+HX59Ucu9+vygdLsTIDeuXOnVq9erdjY2GLncTqdcjqdAegdAAD+RYgGAAAejh07pl27drlfZ2Vlafv27apUqZLi4+N19913KyMjQ4sXL1Z+fr4OHDggSapUqZLCwsKC1W0AAAKCEA0AADxs2bJF7dq1c79OSUmRJPXu3VujR4/Whx9+KEm65pprPOZbvXq12rZtG6huAgAQFIRoAADgoW3btjLGFDn+XOMAALjUcWMxAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAIkI0AAAAAAAWEaIBAAAAALCIEA0AAAAAgEWEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABb5HKLXrl2rTp06KSEhQQ6HQ4sWLfIYb4zRyJEjFR8fr4iICCUnJ2vnzp129RcAAAAAgKDxOUTn5eWpcePGmjJlitfxzz//vF599VVNnTpVmzZtUmRkpNq3b68TJ05ccGcBAAAAAAimUF9n6NChgzp06OB1nDFGEydO1NNPP60uXbpIkt566y1Vq1ZNixYtUvfu3S+stwAAAAAABJGtv4nOysrSgQMHlJyc7G6LiYlR8+bNtWHDBjtXBQAAAABAwPl8JvpcDhw4IEmqVq2aR3u1atXc4/7M5XLJ5XK5X+fm5trZJQAAAAAAbBP0u3OnpaUpJibGPSQmJga7SwAAAAAAeGVriI6Li5MkHTx40KP94MGD7nF/lpqaqpycHPeQnZ1tZ5cAAAAAALCNrSE6KSlJcXFxWrlypbstNzdXmzZtUosWLbzO43Q6FR0d7TEAAAAAAFAS+fyb6GPHjmnXrl3u11lZWdq+fbsqVaqk6tWra/DgwXr22Wd1xRVXKCkpSSNGjFBCQoJuv/12O/sNAAAAAEDA+Ryit2zZonbt2rlfp6SkSJJ69+6tmTNn6sknn1ReXp769euno0eP6oYbbtCyZcsUHh5uX68BAAAAAAgCn0N027ZtZYwpcrzD4dDYsWM1duzYC+oYAAAAAAAlTdDvzg0AAAAAwMWCEA0AAAAAgEWEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAADAw9q1a9WpUyclJCTI4XBo0aJFHuONMRo5cqTi4+MVERGh5ORk7dy5MzidBQAgwAjRAADAQ15enho3bqwpU6Z4Hf/888/r1Vdf1dSpU7Vp0yZFRkaqffv2OnHiRIB7CgBA4IUGuwMAAKBk6dChgzp06OB1nDFGEydO1NNPP60uXbpIkt566y1Vq1ZNixYtUvfu3QPZVQAAAo4z0QAAwLKsrCwdOHBAycnJ7raYmBg1b95cGzZsKHI+l8ul3NxcjwEAgIsRIRoAAFh24MABSVK1atU82qtVq+Ye501aWppiYmLcQ2Jiol/7CQCAvxCiAQCA36WmpionJ8c9ZGdnB7tLAACcF0I0AACwLC4uTpJ08OBBj/aDBw+6x3njdDoVHR3tMQAAcDEiRAMAAMuSkpIUFxenlStXuttyc3O1adMmtWjRIog9AwAgMLg7NwAA8HDs2DHt2rXL/TorK0vbt29XpUqVVL16dQ0ePFjPPvusrrjiCiUlJWnEiBFKSEjQ7bffHrxOAwAQIIRoAADgYcuWLWrXrp37dUpKiiSpd+/emjlzpp588knl5eWpX79+Onr0qG644QYtW7ZM4eHhweoyAAABQ4gGAAAe2rZtK2NMkeMdDofGjh2rsWPHBrBXAACUDPwmGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAIkI0AAAAAAAWEaIBAAAAALCIEA0AAAAAgEWEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwKDTYHQAAAADOqDl8yXnPW3/kcht7UtieCR39unwAFwfORAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAi20N0fn6+RowYoaSkJEVERKh27dp65plnZIyxe1UAAAAAAASU7Xfnfu6555Senq5Zs2apQYMG2rJli/r06aOYmBgNGjTI7tUBAAAAABAwtofo9evXq0uXLurY8Y9HANSsWVNz587VF198YfeqAAAAAAAIKNsv527ZsqVWrlypHTt2SJK+/PJLffbZZ+rQoYPdqwIAAAAAIKBsPxM9fPhw5ebmqm7dugoJCVF+fr7GjRunHj16eJ3e5XLJ5XK5X+fm5trdJQAAAAAAbGH7mej58+dr9uzZmjNnjjIyMjRr1iy9+OKLmjVrltfp09LSFBMT4x4SExPt7hIAAAAAALawPUQPHTpUw4cPV/fu3dWoUSPdf//9GjJkiNLS0rxOn5qaqpycHPeQnZ1td5cAAAAAALCF7ZdzHz9+XGXKeGbzkJAQFRQUeJ3e6XTK6XTa3Q0AAAAAAGxne4ju1KmTxo0bp+rVq6tBgwbatm2bXnrpJT344IN2rwoAAAAAgICyPURPmjRJI0aMUP/+/XXo0CElJCTo4Ycf1siRI+1eFQAAAAAAAWV7iI6KitLEiRM1ceJEuxcNAAAAAEBQ2X5jMQAAAAAALlWEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAB8kp+frxEjRigpKUkRERGqXbu2nnnmGRljgt01AAD8LjTYHQAAABeX5557Tunp6Zo1a5YaNGigLVu2qE+fPoqJidGgQYOC3T0AAPyKEA0AAHyyfv16denSRR07dpQk1axZU3PnztUXX3wR5J4BAOB/XM4NAAB80rJlS61cuVI7duyQJH355Zf67LPP1KFDhyD3DAAA/+NMNAAA8Mnw4cOVm5urunXrKiQkRPn5+Ro3bpx69OhR5Dwul0sul8v9Ojc3NxBdBQDAdpyJBgAAPpk/f75mz56tOXPmKCMjQ7NmzdKLL76oWbNmFTlPWlqaYmJi3ENiYmIAewwAgH0I0QAAwCdDhw7V8OHD1b17dzVq1Ej333+/hgwZorS0tCLnSU1NVU5OjnvIzs4OYI8BALAPl3MDAACfHD9+XGXKeH4PHxISooKCgiLncTqdcjqd/u4aAAB+R4gGAAA+6dSpk8aNG6fq1aurQYMG2rZtm1566SU9+OCDwe4aAAB+R4gGAAA+mTRpkkaMGKH+/fvr0KFDSkhI0MMPP6yRI0cGu2sAAPgdIRoAAPgkKipKEydO1MSJE4PdFQAAAo4biwEAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAIkI0AAAAAAAWEaIBAAAAALCIEA0AAAAAgEWEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwKDTYHQAAALhU1Ry+JNhdAADYjDPRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWOSXEL1//3717NlTsbGxioiIUKNGjbRlyxZ/rAoAAAAAgIAJtXuBR44cUatWrdSuXTt99NFHqlKlinbu3KmKFSvavSoAAAAAAALK9hD93HPPKTExUTNmzHC3JSUl2b0aAAAAAAACzvbLuT/88ENdf/316tq1q6pWraprr71W06ZNK3J6l8ul3NxcjwEAAAAAgJLI9jPRP/zwg9LT05WSkqKnnnpKmzdv1qBBgxQWFqbevXsXmj4tLU1jxoyxuxsAcMmpOXxJsLtw0dkzoWOwuwAAAC4xtp+JLigo0HXXXafx48fr2muvVb9+/dS3b19NnTrV6/SpqanKyclxD9nZ2XZ3CQAAAAAAW9geouPj41W/fn2Ptnr16mnfvn1ep3c6nYqOjvYYAAAAAAAoiWwP0a1atVJmZqZH244dO1SjRg27VwUAAAAAQEDZHqKHDBmijRs3avz48dq1a5fmzJmjN954QwMGDLB7VQAAAAAABJTtIbpp06Z6//33NXfuXDVs2FDPPPOMJk6cqB49eti9KgAAAAAAAsr2u3NL0m233abbbrvNH4sGAAAAACBobD8TDQAAAADApYoQDQAAAACARYRoAAAAAAAsIkQDAACf7d+/Xz179lRsbKwiIiLUqFEjbdmyJdjdAgDA7/xyYzEAAHDpOnLkiFq1aqV27drpo48+UpUqVbRz505VrFgx2F0DAMDvCNEAAMAnzz33nBITEzVjxgx3W1JSUhB7BABA4HA5NwAA8MmHH36o66+/Xl27dlXVqlV17bXXatq0aeecx+VyKTc312MAAOBixJloAADgkx9++EHp6elKSUnRU089pc2bN2vQoEEKCwtT7969vc6TlpamMWPG+K1PNYcv8duyAQA4G2eiAQCATwoKCnTddddp/Pjxuvbaa9WvXz/17dtXU6dOLXKe1NRU5eTkuIfs7OwA9hgAAPsQogEAgE/i4+NVv359j7Z69epp3759Rc7jdDoVHR3tMQAAcDEiRAMAAJ+0atVKmZmZHm07duxQjRo1gtQjAAAChxANAAB8MmTIEG3cuFHjx4/Xrl27NGfOHL3xxhsaMGBAsLsGAIDfEaIBAIBPmjZtqvfff19z585Vw4YN9cwzz2jixInq0aNHsLsGAIDfcXduAADgs9tuu0233XZbsLsBAEDAcSYaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYFBrsDgCXgprDlwS7CwAAAAACgDPRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAotBgdwAAAAC4GNQcviTYXbio7JnQMdhdAPyCM9EAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAs8nuInjBhghwOhwYPHuzvVQEAAAAA4Fd+DdGbN2/W66+/rquvvtqfqwEAAAAAICD8FqKPHTumHj16aNq0aapYsaK/VgMAAAAAQMD4LUQPGDBAHTt2VHJy8jmnc7lcys3N9RgAAAAAACiJQv2x0Hnz5ikjI0ObN28udtq0tDSNGTPGH90AAAAAAMBWtp+Jzs7O1uOPP67Zs2crPDy82OlTU1OVk5PjHrKzs+3uEgAAAAAAtrD9TPTWrVt16NAhXXfdde62/Px8rV27VpMnT5bL5VJISIh7nNPplNPptLsbAAAAAADYzvYQffPNN+vrr7/2aOvTp4/q1q2rYcOGeQRoAAAAAAAuJraH6KioKDVs2NCjLTIyUrGxsYXaAQAAAAC4mPj1OdEAAODSN2HCBDkcDg0ePDjYXQEAwO/8cnfuP1uzZk0gVgMAAAJs8+bNev3113X11VcHuysAAAQEZ6IBAMB5OXbsmHr06KFp06apYsWKwe4OAAABQYgGAADnZcCAAerYsaOSk5OLndblcik3N9djAADgYhSQy7kBAMClZd68ecrIyNDmzZstTZ+WlqYxY8b4uVcAAPgfZ6IBAIBPsrOz9fjjj2v27NkKDw+3NE9qaqpycnLcQ3Z2tp97CQCAf3AmGgAA+GTr1q06dOiQrrvuOndbfn6+1q5dq8mTJ8vlcikkJMRjHqfTKafTGeiuAgBgO0I0AADwyc0336yvv/7ao61Pnz6qW7euhg0bVihAAwBwKSFEAwAAn0RFRalhw4YebZGRkYqNjS3UDgDApYbfRAMAAAAAYBFnogEAwAVbs2ZNsLsAAEBAcCYaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsCg12BwKh5vAlwe6CV3smdAx2FwAAAAAAPuBMNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAi0rFI64AAAAAAOePxwb/F2eiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAIkI0AAAAAAAWEaIBAAAAALCIEA0AAAAAgEW2h+i0tDQ1bdpUUVFRqlq1qm6//XZlZmbavRoAAAAAAALO9hD96aefasCAAdq4caNWrFihU6dO6ZZbblFeXp7dqwIAAAAAIKBC7V7gsmXLPF7PnDlTVatW1datW9W6dWu7VwcAAAAAQMDYHqL/LCcnR5JUqVIlr+NdLpdcLpf7dW5urr+7BAAAAADAefFriC4oKNDgwYPVqlUrNWzY0Os0aWlpGjNmjD+7gUtIzeFLgt0FACj10tLStHDhQn3//feKiIhQy5Yt9dxzz+mqq64KdtcAAPA7v96de8CAAfrmm280b968IqdJTU1VTk6Oe8jOzvZnlwAAwAXi/icAgNLMb2eiBw4cqMWLF2vt2rW6/PLLi5zO6XTK6XT6qxsAAMBm3P8EAFCa2R6ijTF67LHH9P7772vNmjVKSkqyexUAAKAEKe7+JwAAXEpsD9EDBgzQnDlz9MEHHygqKkoHDhyQJMXExCgiIsLu1QEAgCCycv8TiRuJAgAuHbaH6PT0dElS27ZtPdpnzJihBx54wO7VAQCAIDpz/5PPPvvsnNNxI1Gg9CnJN4TdM6FjsLuAi5hfLucGAACXPqv3P5H+uJFoSkqK+3Vubq4SExP93UUAAGzn9+dEAwCAS8v53P+EG4kCAC4VhGgAAOAT7n8CACjN/PqcaAAAcOlJT09XTk6O2rZtq/j4ePfw7rvvBrtrAAD4HWeiAQCAT7j/CQCgNONMNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAIkI0AAAAAAAWEaIBAAAAALCIEA0AAAAAgEWEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAiwjRAAAAAABYRIgGAAAAAMAiQjQAAAAAABYRogEAAAAAsIgQDQAAAACARYRoAAAAAAAsIkQDAAAAAGARIRoAAAAAAIsI0QAAAAAAWESIBgAAAADAIkI0AAAAAAAWEaIBAAAAALCIEA0AAAAAgEWEaAAAAAAALCJEAwAAAABgESEaAAAAAACL/Baip0yZopo1ayo8PFzNmzfXF1984a9VAQCAIGBfDwAojfwSot99912lpKRo1KhRysjIUOPGjdW+fXsdOnTIH6sDAAABxr4eAFBa+SVEv/TSS+rbt6/69Omj+vXra+rUqSpXrpzefPNNf6wOAAAEGPt6AEBpFWr3Ak+ePKmtW7cqNTXV3VamTBklJydrw4YNhaZ3uVxyuVzu1zk5OZKk3Nxc2/pU4Dpu27LsZGeNpUVJ3ZYASia7PmfPLMcYY8vyLna+7usl/+/v2T8A8AXH4b4rqZ+zwdjX2x6i//Of/yg/P1/VqlXzaK9WrZq+//77QtOnpaVpzJgxhdoTExPt7lqJEzMx2D0AgEub3Z+zv/32m2JiYuxd6EXI1329VLr39wBKHo7DLx3B2NfbHqJ9lZqaqpSUFPfrgoIC/frrr4qNjZXD4Qhiz6zLzc1VYmKisrOzFR0dHezu+E1pqVMqPbVS56WntNQa6DqNMfrtt9+UkJDg93VdqnzZ35eWf8d/VhrrLo01S9RN3Ze+i7FmX/b1tofoypUrKyQkRAcPHvRoP3jwoOLi4gpN73Q65XQ6PdoqVKhgd7cCIjo6+qL5R3IhSkudUumplTovPaWl1kDWyRno//J1Xy+d3/6+tPw7/rPSWHdprFmi7tKmNNZ9sdVsdV9v+43FwsLC1KRJE61cudLdVlBQoJUrV6pFixZ2rw4AAAQY+3oAQGnml8u5U1JS1Lt3b11//fVq1qyZJk6cqLy8PPXp08cfqwMAAAHGvh4AUFr5JUTfc889+uWXXzRy5EgdOHBA11xzjZYtW1boBiSXCqfTqVGjRhW6TO1SU1rqlEpPrdR56SkttZaWOksyf+7rS+v2LY11l8aaJeqm7kvfpV6zw/C8DgAAAAAALLH9N9EAAAAAAFyqCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFpXqEJ2WlqamTZsqKipKVatW1e23367MzMxzztO2bVs5HI5CQ8eOHb1O/8gjj8jhcGjixIke7TVr1iy0jAkTJthVWiH+qvWBBx4oNP7WW2/1WM6vv/6qHj16KDo6WhUqVNDf//53HTt27JKrM5Db1J//dv/973+rc+fOiomJUWRkpJo2bap9+/a5x584cUIDBgxQbGysypcvr7vuuksHDx70S51ScGv1tpxHHnnkoqrT23iHw6EXXnjBPU0g36PBrjXQn72l3flsa0maOHGirrrqKkVERCgxMVFDhgzRiRMn3ONHjx5daDvWrVvXn6X45HzqPnXqlMaOHavatWsrPDxcjRs31rJlywpNN2XKFNWsWVPh4eFq3ry5vvjiC3+V4TN/1V3St3d6erquvvpqRUdHKzo6Wi1atNBHH310znkWLFigunXrKjw8XI0aNdLSpUs9xhtjNHLkSMXHxysiIkLJycnauXOnP8vwmT/qtnK8FUy+1vztt9/qrrvucu97/pwHzijJ72vJP3WX9Pd1sUwp1r59ezNjxgzzzTffmO3bt5u//e1vpnr16ubYsWNFznP48GHz888/u4dvvvnGhISEmBkzZhSaduHChaZx48YmISHBvPzyyx7jatSoYcaOHeuxrHOt90L5q9bevXubW2+91WO6X3/91WM5t956q2ncuLHZuHGjWbdunalTp4659957L7k6A7lN/VXnrl27TKVKlczQoUNNRkaG2bVrl/nggw/MwYMH3dM88sgjJjEx0axcudJs2bLF/OUvfzEtW7b0S53BrrVNmzamb9++HsvKycm5qOo8e/zPP/9s3nzzTeNwOMzu3bvd0wTyPRrsWgP92Vvanc+2nj17tnE6nWb27NkmKyvLLF++3MTHx5shQ4a4pxk1apRp0KCBx3b85ZdfAlGSJedT95NPPmkSEhLMkiVLzO7du81rr71mwsPDTUZGhnuaefPmmbCwMPPmm2+ab7/91vTt29dUqFDB43MrmPxVd0nf3h9++KFZsmSJ2bFjh8nMzDRPPfWUKVu2rPnmm2+8Tv/555+bkJAQ8/zzz5vvvvvOPP3006Zs2bLm66+/dk8zYcIEExMTYxYtWmS+/PJL07lzZ5OUlGR+//33QJVVLH/UbeV4K5h8rfmLL74wTzzxhJk7d66Ji4srlAeMKfnva2P8U3dJf18Xp1SH6D87dOiQkWQ+/fRTy/O8/PLLJioqqtAO4scffzSXXXaZ+eabb0yNGjW8hmhv/6ACxa5ae/fubbp06VLkPN99952RZDZv3uxu++ijj4zD4TD79+8/r777IlB1GhPcbWpXnffcc4/p2bNnkfMcPXrUlC1b1ixYsMDd9u9//9tIMhs2bDi/zvsoULUa80eIfvzxx8+3qxfEzs+js3Xp0sXcdNNN7tfBfo8aE7hajQn+Z29pZ2VbDxgwoNB2S0lJMa1atXK/HjVqlGncuLG/umk7K3XHx8ebyZMne7TdeeedpkePHu7XzZo1MwMGDHC/zs/PNwkJCSYtLc3+TtvArrovtu1tjDEVK1Y0//u//+t1XLdu3UzHjh092po3b24efvhhY4wxBQUFJi4uzrzwwgvu8UePHjVOp9PMnTvXf522wYXUbYy1462S5lw1n62o/c/F9r4+40Lrvhjf12cr1Zdz/1lOTo4kqVKlSpbnmT59urp3767IyEh3W0FBge6//34NHTpUDRo0KHLeCRMmKDY2Vtdee61eeOEFnT59+vw77yO7apWkNWvWqGrVqrrqqqv06KOP6vDhw+5xGzZsUIUKFXT99de725KTk1WmTBlt2rTpAqsoXqDqPCNY29SOOgsKCrRkyRJdeeWVat++vapWrarmzZtr0aJF7nm2bt2qU6dOKTk52d1Wt25dVa9eXRs2bLCnmGIEqtYzZs+ercqVK6thw4ZKTU3V8ePHbamjOHb+2z3j4MGDWrJkif7+97+724L9HpUCV+sZwfzsLe2sbOuWLVtq69at7ssZf/jhBy1dulR/+9vfPKbbuXOnEhISVKtWLfXo0cPjpxgljZW6XS6XwsPDPdoiIiL02WefSZJOnjyprVu3enz+lilTRsnJyQH7/PWVHXWfcbFs7/z8fM2bN095eXlq0aKF12k2bNjgsR0lqX379u7tmJWVpQMHDnhMExMTo+bNm5fYbW1H3WdYOd4qCazUXJyL8X1tR91nXCzva6+CneJLivz8fNOxY0ePb7qLs2nTJiPJbNq0yaN9/Pjx5q9//aspKCgwxnj/Buaf//ynWb16tfnyyy9Nenq6qVChgselav5kZ61z5841H3zwgfnqq6/M+++/b+rVq2eaNm1qTp8+bYwxZty4cebKK68stLwqVaqY11577cIKKUYg6zQmeNvUrjp//vlnI8mUK1fOvPTSS2bbtm0mLS3NOBwOs2bNGmPMH5dZhoWFFVpe06ZNzZNPPnnhxRQjkLUaY8zrr79uli1bZr766ivzzjvvmMsuu8zccccdttbkjZ3/ds/23HPPmYoVK3pcDhjM96gxga3VmOB+9pZ2vmzrV155xZQtW9aEhoYaSeaRRx7xGL906VIzf/588+WXX5ply5aZFi1amOrVq5vc3Fx/df+8Wa373nvvNfXr1zc7duww+fn55uOPPzYRERHuz9z9+/cbSWb9+vUe8w0dOtQ0a9bMb/0/X3bVbczFsb2/+uorExkZaUJCQkxMTIxZsmRJkdOWLVvWzJkzx6NtypQppmrVqsaYPy57lmR++uknj2m6du1qunXrZn/nL4CddRtj7Xgr2Hyp+Wze8sDF9L62s25jLo739bkQov/PI488YmrUqGGys7Mtz9OvXz/TqFEjj7YtW7aYatWqeVwGaeXywenTp5vQ0FBz4sQJn/p9Puyq1Zvdu3cbSeaTTz4xxgT3AD2QdXoTqG1qV51nPsj//FvYTp06me7duxtjgh+iA1mrNytXrjSSzK5du3zruI/89W/3qquuMgMHDvRoC3aIDmSt3gTys7e0s7qtV69ebapVq2amTZtmvvrqK7Nw4UKTmJhoxo4dW+Q8R44cMdHR0ZYuLQw0q3UfOnTIdOnSxZQpU8aEhISYK6+80vTv39+Eh4cbYy6ug21j7Kvbm5K4vV0ul9m5c6fZsmWLGT58uKlcubL59ttvvU57KYVoO+v2xsrxVqD5UvPZLvYQbWfd3pTE9/W5cDm3pIEDB2rx4sVavXq1Lr/8ckvz5OXlad68eYUuFVy3bp0OHTqk6tWrKzQ0VKGhodq7d6/+8Y9/qGbNmkUur3nz5jp9+rT27NlzAZUUz85avalVq5YqV66sXbt2SZLi4uJ06NAhj2lOnz6tX3/9VXFxcb4XYFGg6/QmENvUzjorV66s0NBQ1a9f36O9Xr167str4uLidPLkSR09etRjmoMHD/p1e0qBr9Wb5s2bS9I5t/uF8te/3XXr1ikzM1MPPfSQR3uw3qNS4Gv1JlCfvaWdL9t6xIgRuv/++/XQQw+pUaNGuuOOOzR+/HilpaWpoKDA6zwVKlTQlVde6df35vnwpe4qVapo0aJFysvL0969e/X999+rfPnyqlWrlqQ/PrdCQkIKPQ0hEJ+/vrKzbm9K4vYOCwtTnTp11KRJE6Wlpalx48Z65ZVXvE4bFxd3zu145r8Xw7a2s25vrBxvBZovNRfnYnpf21m3NyXxfX0upTpEG2M0cOBAvf/++1q1apWSkpIsz7tgwQK5XC717NnTo/3+++/XV199pe3bt7uHhIQEDR06VMuXLy9yedu3b1eZMmVUtWrV867nXPxRqzc//vijDh8+rPj4eElSixYtdPToUW3dutU9zapVq1RQUOAOJHYKVp3e+HOb+qPOsLAwNW3atNDjSHbs2KEaNWpIkpo0aaKyZctq5cqV7vGZmZnat2/fBf8upijBqtWb7du3S9I5t/v58ve/3enTp6tJkyZq3LixR3ug36NS8Gr1xt+fvaXd+Wzr48ePq0wZz8OTkJAQ9/K8OXbsmHbv3u2X9+b5uJB/4+Hh4brssst0+vRpvffee+rSpYukPz63mjRp4vH5W1BQoJUrV/rt89dX/qjbm5K2vb0pKCiQy+XyOq5FixYe21GSVqxY4d6OSUlJiouL85gmNzdXmzZtKjHbuigXUrc3Vo63gu1cNRfnYnhfF+VC6vbmYnhfewjSGfAS4dFHHzUxMTFmzZo1HrdXP378uHua+++/3wwfPrzQvDfccIO55557LK3nz5cxrF+/3rz88stm+/btZvfu3eadd94xVapUMb169brgmorij1p/++0388QTT5gNGzaYrKws88knn5jrrrvOXHHFFR6XRt56663m2muvNZs2bTKfffaZueKKK/z2+Jxg1Rnobeqvf7sLFy40ZcuWNW+88YbZuXOnmTRpkgkJCTHr1q1zT/PII4+Y6tWrm1WrVpktW7aYFi1amBYtWthf5P8JVq27du0yY8eONVu2bDFZWVnmgw8+MLVq1TKtW7e+qOo0xpicnBxTrlw5k56e7nV8IN+jxgSv1mB89pZ257OtR40aZaKioszcuXPNDz/8YD7++GNTu3Ztj8tY//GPf5g1a9aYrKws8/nnn5vk5GRTuXJlc+jQoYDWV5TzqXvjxo3mvffeM7t37zZr1641N910k0lKSjJHjhxxTzNv3jzjdDrNzJkzzXfffWf69etnKlSoYA4cOBDI8orkr7pL+vYePny4+fTTT01WVpb56quvzPDhw43D4TAff/yxMaZwzZ9//rkJDQ01L774ovn3v/9tRo0a5fURVxUqVHD/PrhLly4l7hFXdtdt9bgymHyt2eVymW3btplt27aZ+Ph488QTT5ht27aZnTt3uqcp6e9rY/xTd0l/XxenVIdoSV6Hs5872qZNG9O7d2+P+b7//nsjyf0Ppzh/DtFbt241zZs3NzExMSY8PNzUq1fPjB8/3q8fEP6o9fjx4+aWW24xVapUMWXLljU1atQwffv2LfSmP3z4sLn33ntN+fLlTXR0tOnTp4/57bff/FFm0OoM9Db157/d6dOnmzp16pjw8HDTuHFjs2jRIo/xv//+u+nfv7+pWLGiKVeunLnjjjvMzz//bGd5HoJV6759+0zr1q1NpUqVjNPpNHXq1DFDhw7123Oi/Vnn66+/biIiIszRo0e9jg/ke9SY4NUajM/e0u58tvWpU6fM6NGjTe3atU14eLhJTEw0/fv39whV99xzj4mPjzdhYWHmsssuM/fcc4/f71Xgi/Ope82aNaZevXrG6XSa2NhYc//993t9zNykSZNM9erVTVhYmGnWrJnZuHFjACqyxl91l/Tt/eCDD5oaNWqYsLAwU6VKFXPzzTd7fE55+zybP3++ufLKK01YWJhp0KBBoZs0FRQUmBEjRphq1aoZp9Npbr75ZpOZmRmIciyzu26rx5XB5GvNWVlZXt8Tbdq08VhuSX5fG+Ofukv6+7o4DmOKuDYKAAAAAAB4KNW/iQYAAAAAwBeEaAAAAAAALCJEAwAAAABgESEaAAAAAACLCNEAAAAAAFhEiAYAAAAAwCJCNAAAAAAAFhGiAQAAAACwiBANAAAAAIBFhGgAAAAAACwiRAMAAAAAYBEhGgAAAAAAi/4/9HanDFPpGjQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1200x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "import numpy as np\n",
    "import matplotlib.pylab as plt\n",
    "import pandas as pd\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "\n",
    "advertising_df = pd.read_csv(\"../Daten/Advertising.csv\", index_col= \"Unnamed: 0\")\n",
    "advertising_df.reset_index(drop = True, inplace = True)\n",
    "X = advertising_df.drop([\"sales\"], axis = 1)\n",
    "y = advertising_df.sales\n",
    "\n",
    "n_splits = 5\n",
    "n_fold_runs = 50\n",
    "\n",
    "mse_train_cv, mse_test_cv = [], []\n",
    "for fold_run in range(n_fold_runs):\n",
    "    cv = KFold(n_splits, shuffle = True)\n",
    "    mse_train_results, mse_test_results = [], []\n",
    "    for ii, (train_idx, test_idx) in enumerate(cv.split(X, y)):\n",
    "        X_train, X_test = X.loc[train_idx, :], X.loc[test_idx, :]\n",
    "        y_train, y_test = y.loc[train_idx], y.loc[test_idx]\n",
    "        \n",
    "        linear_regression = LinearRegression()\n",
    "        linear_regression.fit(X_train, y_train)\n",
    "        y_train_hat, y_test_hat = linear_regression.predict(X_train), linear_regression.predict(X_test)\n",
    "        \n",
    "        mse_train, mse_test = mean_squared_error(y_train, y_train_hat), mean_squared_error(y_test, y_test_hat)\n",
    "        mse_train_results.append(mse_train)\n",
    "        mse_test_results.append(mse_test)\n",
    "    mse_train_cv.append(np.mean(mse_train_results))\n",
    "    mse_test_cv.append(np.mean(mse_test_results))\n",
    "\n",
    "fig, axs = plt.subplots(1, 2, figsize = (12, 5))\n",
    "axs[0].hist(mse_train_cv)\n",
    "axs[0].axvline(np.mean(mse_train_cv))\n",
    "axs[0].set_title(\"CV Error - training data\")\n",
    "\n",
    "axs[1].hist(mse_test_cv)\n",
    "axs[1].axvline(np.mean(mse_test_cv))\n",
    "axs[1].set_title(\"CV Error - test data\")\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}