exercise-dropout-and-batch-normalization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b6c1f292",
   "metadata": {
    "papermill": {
     "duration": 0.015861,
     "end_time": "2021-12-29T13:00:10.807249",
     "exception": false,
     "start_time": "2021-12-29T13:00:10.791388",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**This notebook is an exercise in the [Intro to Deep Learning](https://www.kaggle.com/learn/intro-to-deep-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/ryanholbrook/dropout-and-batch-normalization).**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6679183b",
   "metadata": {
    "papermill": {
     "duration": 0.01443,
     "end_time": "2021-12-29T13:00:10.837108",
     "exception": false,
     "start_time": "2021-12-29T13:00:10.822678",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Introduction #\n",
    "\n",
    "In this exercise, you'll add dropout to the *Spotify* model from Exercise 4 and see how batch normalization can let you successfully train models on difficult datasets.\n",
    "\n",
    "Run the next cell to get started!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d1f8936f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:10.879786Z",
     "iopub.status.busy": "2021-12-29T13:00:10.879003Z",
     "iopub.status.idle": "2021-12-29T13:00:10.913439Z",
     "shell.execute_reply": "2021-12-29T13:00:10.912817Z",
     "shell.execute_reply.started": "2021-12-29T12:49:13.780249Z"
    },
    "papermill": {
     "duration": 0.061163,
     "end_time": "2021-12-29T13:00:10.913580",
     "exception": false,
     "start_time": "2021-12-29T13:00:10.852417",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Setup plotting\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('seaborn-whitegrid')\n",
    "# Set Matplotlib defaults\n",
    "plt.rc('figure', autolayout=True)\n",
    "plt.rc('axes', labelweight='bold', labelsize='large',\n",
    "       titleweight='bold', titlesize=18, titlepad=10)\n",
    "plt.rc('animation', html='html5')\n",
    "\n",
    "# Setup feedback system\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.deep_learning_intro.ex5 import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e8617d",
   "metadata": {
    "papermill": {
     "duration": 0.01464,
     "end_time": "2021-12-29T13:00:10.943316",
     "exception": false,
     "start_time": "2021-12-29T13:00:10.928676",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "First load the *Spotify* dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "94f5e0b9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:10.982385Z",
     "iopub.status.busy": "2021-12-29T13:00:10.981744Z",
     "iopub.status.idle": "2021-12-29T13:00:17.182621Z",
     "shell.execute_reply": "2021-12-29T13:00:17.182173Z",
     "shell.execute_reply.started": "2021-12-29T12:49:13.834846Z"
    },
    "papermill": {
     "duration": 6.224642,
     "end_time": "2021-12-29T13:00:17.182751",
     "exception": false,
     "start_time": "2021-12-29T13:00:10.958109",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: [18]\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n",
    "from sklearn.compose import make_column_transformer\n",
    "from sklearn.model_selection import GroupShuffleSplit\n",
    "\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras import callbacks\n",
    "\n",
    "spotify = pd.read_csv('../input/dl-course-data/spotify.csv')\n",
    "\n",
    "X = spotify.copy().dropna()\n",
    "y = X.pop('track_popularity')\n",
    "artists = X['track_artist']\n",
    "\n",
    "features_num = ['danceability', 'energy', 'key', 'loudness', 'mode',\n",
    "                'speechiness', 'acousticness', 'instrumentalness',\n",
    "                'liveness', 'valence', 'tempo', 'duration_ms']\n",
    "features_cat = ['playlist_genre']\n",
    "\n",
    "preprocessor = make_column_transformer(\n",
    "    (StandardScaler(), features_num),\n",
    "    (OneHotEncoder(), features_cat),\n",
    ")\n",
    "\n",
    "def group_split(X, y, group, train_size=0.75):\n",
    "    splitter = GroupShuffleSplit(train_size=train_size)\n",
    "    train, test = next(splitter.split(X, y, groups=group))\n",
    "    return (X.iloc[train], X.iloc[test], y.iloc[train], y.iloc[test])\n",
    "\n",
    "X_train, X_valid, y_train, y_valid = group_split(X, y, artists)\n",
    "\n",
    "X_train = preprocessor.fit_transform(X_train)\n",
    "X_valid = preprocessor.transform(X_valid)\n",
    "y_train = y_train / 100\n",
    "y_valid = y_valid / 100\n",
    "\n",
    "input_shape = [X_train.shape[1]]\n",
    "print(\"Input shape: {}\".format(input_shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65ea27b3",
   "metadata": {
    "papermill": {
     "duration": 0.014943,
     "end_time": "2021-12-29T13:00:17.213276",
     "exception": false,
     "start_time": "2021-12-29T13:00:17.198333",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 1) Add Dropout to Spotify Model\n",
    "\n",
    "Here is the last model from Exercise 4. Add two dropout layers, one after the `Dense` layer with 128 units, and one after the `Dense` layer with 64 units. Set the dropout rate on both to `0.3`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "52a06822",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:17.248382Z",
     "iopub.status.busy": "2021-12-29T13:00:17.247843Z",
     "iopub.status.idle": "2021-12-29T13:00:19.637820Z",
     "shell.execute_reply": "2021-12-29T13:00:19.637377Z",
     "shell.execute_reply.started": "2021-12-29T12:50:30.395943Z"
    },
    "lines_to_next_cell": 2,
    "papermill": {
     "duration": 2.409602,
     "end_time": "2021-12-29T13:00:19.637935",
     "exception": false,
     "start_time": "2021-12-29T13:00:17.228333",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-12-29 13:00:17.330791: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:17.421580: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:17.422318: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:17.423544: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2021-12-29 13:00:17.424740: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:17.425451: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:17.426051: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:19.256827: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:19.257720: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:19.258371: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2021-12-29 13:00:19.258937: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 15403 MB memory:  -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"1_Q1\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# YOUR CODE HERE: Add two 30% dropout layers, one after 128 and one after 64\n",
    "model = keras.Sequential([\n",
    "    layers.Dense(128, activation='relu', input_shape=input_shape),\n",
    "    layers.Dropout(rate=0.3),\n",
    "    layers.Dense(64, activation='relu'),\n",
    "    layers.Dropout(rate=0.3),\n",
    "    layers.Dense(1)\n",
    "])\n",
    "\n",
    "# Check your answer\n",
    "q_1.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "039882f6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:19.674472Z",
     "iopub.status.busy": "2021-12-29T13:00:19.673663Z",
     "iopub.status.idle": "2021-12-29T13:00:19.676476Z",
     "shell.execute_reply": "2021-12-29T13:00:19.676013Z",
     "shell.execute_reply.started": "2021-12-29T12:50:37.624228Z"
    },
    "papermill": {
     "duration": 0.021869,
     "end_time": "2021-12-29T13:00:19.676583",
     "exception": false,
     "start_time": "2021-12-29T13:00:19.654714",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#q_1.hint()\n",
    "#q_1.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c696ae7f",
   "metadata": {
    "papermill": {
     "duration": 0.016724,
     "end_time": "2021-12-29T13:00:19.709600",
     "exception": false,
     "start_time": "2021-12-29T13:00:19.692876",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now run this next cell to train the model see the effect of adding dropout."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fac0b083",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:19.749570Z",
     "iopub.status.busy": "2021-12-29T13:00:19.747871Z",
     "iopub.status.idle": "2021-12-29T13:00:27.435696Z",
     "shell.execute_reply": "2021-12-29T13:00:27.436154Z",
     "shell.execute_reply.started": "2021-12-29T12:50:40.180612Z"
    },
    "papermill": {
     "duration": 7.709219,
     "end_time": "2021-12-29T13:00:27.436305",
     "exception": false,
     "start_time": "2021-12-29T13:00:19.727086",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-12-29 13:00:19.822612: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Validation Loss: 0.1906\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4mUlEQVR4nO3deXxU9f3v8deZfclGtgloCCJBIwRxBUSIDWKEmKKA9qe0LpWflZ9WXOr666VeWvUC2gdc24JcFGhRawUkQvSHNS5BwYWgRiQiUQIhkAGyJ5PJrPePQyIxOwkkc/J5Ph7zmMnMOWe+89XwzneZ71cJBoNBhBBCiH5G19cFEEIIIdoiASWEEKJfkoASQgjRL0lACSGE6JckoIQQQvRLhr4uAEB+fn5fF0EIIUQfuuSSS1o91y8CCtouXHcUFhaSkpLSS6XRHqmf9knddEzqp31SN+3rTt2010iRLj4hhBD9kgSUEEKIfkkCSgghRL8kASWEEKJfkoASQgjRL0lACSGE6JckoIQQQvRLElBCCCH6JQkoIYQIcRdddFFfF+G00ERAHap08ft/H6HW7e3rogghhOglmgio75y15B9u4Ptj9X1dFCGE6DPBYJBFixZx3XXXkZWVxVtvvQXA0aNHmTNnDjNmzOC6665j586d+P1+HnvsseZj16xZ07eFb0O/WYuvJ+wm9WPUN/r6uCRCiIFsQ/4h/rWzpFevedOlicy65OwuHfvOO+/w7bffkp2dTWVlJbNnz+bSSy9ly5YtXHnllcybNw+/309DQwOFhYU4nU62bNkCQE1NTa+WuzdoogVlN0tACSFEfn4+mZmZ6PV6YmNjueyyy/j6669JTU1l48aNPP/883z33XeEhYWRmJhISUkJf/zjH8nLyyMsLKyvi9+KJlpQNpMeAJfH38clEUIMZLMuObvLrZ0z6bLLLmPdunV8+OGHPPbYY9xxxx1cf/31ZGdn89FHH/HPf/6Tt99+m2eeeaavi9qCtlpQHmlBCSEGrksvvZS3334bv99PRUUFO3fuZMyYMZSWlhIbG8tNN93EjTfeyDfffENFRQXBYJCMjAzuv/9+9uzZ09fFb0VTLSjp4hNCDGRTp07liy++YMaMGSiKwsMPP0xcXBxvvPEGL774IgaDAZvNxqJFizh69CiPP/44gUAAgAcffLCPS9+aRgKqaQxKuviEEAPPF198AYCiKDz66KM8+uijLV6/4YYbuOGGG1qd98Ybb5yR8p0qTXTx6XUKZoOCS7r4hBBCMzQRUABWg456mSQhhBCaoZmAshgUXDIGJYQQmqGZgLIZddTJGJQQQmiGZgLKYpQxKCGE0BLNBJSMQQkhhLZoJqAsRp2MQQkhhIZ0KaDy8vLIyMhg6tSprFy5stXrr776KllZWcyYMYObb76ZoqIiAD7++GNmzpxJVlYWM2fOZMeOHb1b+pPYDIp8UVcIIbqgo/2jDh06xHXXXXcGS9O+Tr+o6/f7WbhwIatXr8bhcDB79mzS09MZMWJE8zFZWVncfPPNAOTm5vLMM8/w4osvMmjQIJYvX47D4eC7777jzjvvZNu2baflg1iM0sUnhBBa0mlAFRQUkJSURGJiIgCZmZnk5ua2CKiTV8FtaGhAURQALrjggubnk5OTaWxsxOPxYDKZeu0DNLEadDJJQgjRt758Fb5Y17vXvOiXMPbmDg959tlnGTx4MHPmzAHg+eefR6/X8+mnn1JTU4PP52P+/PlcffXV3XrrxsZGnnzySXbv3o1er+exxx5j/Pjx7Nu3j8cffxyv10sgEOD5558nPj6e+++/n7KyMgKBADNmzCAlJeWUPzZ0IaCcTicJCQnNPzscDgoKClod9/LLL7N69Wq8Xi9r165t9frWrVu54IIL2g2nwsLC7pS7FYPix+sPUrB7D0a90qNraZHb7e5xHWuV1E3HpH7a99O6iTx8mEhX726cWn34MNXmjus/JSWFF198kYsvvhiATZs28Yc//IFx48Zhs9moqanhkUceYciQISiKQiAQaPe/qdPppLGxkcLCQjZt2kR1dTWLFy/m0KFDPPTQQ/ztb39jzZo1XH311aSlpeH1eqmqqiI3NxeTycSiRYsAqKio6Pm/6z06+yRz5sxhzpw5bN68meXLlzcXEmDfvn08++yzvPTSS+2e39OkDd9TDcDQ4SOIsvV+Cy3UFRYW9riOtUrqpmNSP+1rVTcpKcBDvfoedmBIJ8ekpKTwl7/8hejoaCorK4mLi2PChAk888wzfP755+h0uubn4+Li0Ol07f43DQ8Px2w2N1/zV7/6FSkpKaSkpLB69WrMZjPp6emsWLECnU7HNddcw7Bhw4iMjOTll19my5Yt/OxnPyM6OrrL/9/k5+e3+XynkyQcDgdlZWXNPzudThwOR7vHZ2Zm8u677zb/XFZWxr333suiRYsYOnRolwp7KqxGtdVUJxMlhBAD0LXXXsvWrVt56623mD59Ops3b6aiooKNGzeSnZ1NbGwsjY2NvfJeWVlZLF++HIvFwl133cWOHTs455xz2LhxIyNHjmTp0qW89tprPX6fTgMqNTWV4uJiSkpK8Hg85OTkkJ6e3uKY4uLi5scffPABSUlJgLqF8F133cVDDz3EJZdc0uPCdsRqVD+KbFoohBiIpk+fzltvvcXWrVu59tprqa2tJSYmBqPRyCeffEJpaWm3r3nppZeyefNmAPbv38+RI0cYPnw4JSUlJCYmcuuttzJlyhT27t2L0+nEarUyY8YM7rzzTr7//vsef6ZOu/gMBgMLFixg7ty5+P1+Zs2aRXJyMsuWLWP06NFMmTKFdevWsWPHDgwGAxEREc3de+vWrePgwYP89a9/5a9//SsAL730EjExMT0u+E9ZDGpAyVRzIcRAlJycTH19PfHx8cTHx5OVlcW8efPIyspi9OjRDB8+vNvXvOWWW3jyySfJyspCr9fzzDPPYDKZePvtt8nOzsZgMBAbG8tvfvMbvv76axYvXoxOp8NgMHD77bf3+DMpwWAw2OOr9FB+fn6PW1j/en8Xj2w9wstzxzFxRGwvlUw7ZByhfVI3HZP6aZ/UTfu6UzftZYBmVpJo6uKTMSghhNAGTeyoCyePQUlACSFEZ/bu3csjjzzS4jmTycTrr7/eRyVqTTsBZVBn8cm270II0bnzzjuP7Ozsvi5GhzTTxWeRFpQQQmiKdgLK0PQ9KGlBCSGEFmgmoHSKgs2kly03hBBCIzQTUAA2k0FWNBdCCI3QVEDZzXoZgxJCCI3QVEDZTAaZxSeEEBqhqYAKM+tlqSMhhNAITQWUzWSQLj4hhNAITQWU3ayXSRJCCKERmgoom8kg08yFEEIjNBVQdpNeFosVQgiN0FZAmQ24PH76wQ4iQgghekhzAeULBPH4A31dFCGEED2kqYCymfQAuOS7UEIIEfI0FVB2k7p7SL1MNRdCiJCnrYAynwgoaUEJIUTI01RA2cxqF5+0oIQQIvRpKqCauvhkDEoIIUKfpgKqaZKEtKCEECL0aSqgfhyDkoASQohQ16WAysvLIyMjg6lTp7Jy5cpWr7/66qtkZWUxY8YMbr75ZoqKippfe+GFF5g6dSoZGRls27at90reBnvzGJR08QkhRKgzdHaA3+9n4cKFrF69GofDwezZs0lPT2fEiBHNx2RlZXHzzTcDkJubyzPPPMOLL75IUVEROTk55OTk4HQ6ueOOO9i6dSt6vf60fJgfx6CkBSWEEKGu0xZUQUEBSUlJJCYmYjKZyMzMJDc3t8UxYWFhzY8bGhpQFAVQwyozMxOTyURiYiJJSUkUFBT08kf4kdUoLSghhNCKTltQTqeThISE5p8dDkebIfPyyy+zevVqvF4va9eubT73wgsvbHGu0+nsjXK3SadTsJn00oISQggN6DSgumrOnDnMmTOHzZs3s3z5chYtWtSt8wsLC3v0/m63m8LCQsx6OOQ83uPraU1T/YjWpG46JvXTPqmb9vVG3XQaUA6Hg7KysuafnU4nDoej3eMzMzN58sknu31uSkpKV8vcpsLCQlJSUoi0lWGyhff4elrTVD+iNambjkn9tE/qpn3dqZv8/Pw2n+90DCo1NZXi4mJKSkrweDzk5OSQnp7e4pji4uLmxx988AFJSUkApKenk5OTg8fjoaSkhOLiYsaMGdOlAp8q2fZdCCG0odMWlMFgYMGCBcydOxe/38+sWbNITk5m2bJljB49milTprBu3Tp27NiBwWAgIiKiuXsvOTmZadOmMX36dPR6PQsWLDhtM/ia2M16WYtPCCE0oEtjUGlpaaSlpbV4bv78+c2Pf//737d77rx585g3b94pFq/77GYDFfWeM/Z+QgghTg9NrSQB6nehZCUJIYQIfZoLKJtJj0u+ByWEECFPcwFlN0sLSgghtEBzAdXUggoGg31dFCGEED2guYCymw34AkEafYG+LooQQoge0F5AndgTSsahhBAitGkuoGyyJ5QQQmiC5gKqecsNaUEJIURI015Andi0sE5aUEIIEdI0GFBNLSgJKCGECGWaCyjbiUkSsh6fEEKENs0F1I9jUNKCEkKIUKa5gLKZm1pQElBCCBHKNBdQYU3TzGUWnxBChDTNBZTFoEdRwCUtKCGECGmaCyidTsFm1EsLSgghQpzmAgrU1SRkkoQQQoQ2TQZUmNlAnUwzF0KIkKbJgLKZ9DIGJYQQIU6TAWU3GaiXLj4hhAhpmgwom1m2fRdCiFCnyYCymwyyWKwQQoQ4bQaUWY9LJkkIIURI02RA2WQMSgghQl6XAiovL4+MjAymTp3KypUrW72+evVqpk+fTlZWFrfddhulpaXNry1evJjMzEymTZvGn/70J4LBYO+Vvh32E2NQZ+K9hBBCnB6dBpTf72fhwoWsWrWKnJwctmzZQlFRUYtjUlJS2LBhA5s3byYjI4MlS5YAsGvXLnbt2sWbb77Jli1b+Prrr/nss89Ozyc5ic1kwB8I0ugLnPb3EkIIcXp0GlAFBQUkJSWRmJiIyWQiMzOT3NzcFseMHz8eq9UKwNixYykrKwNAURQ8Hg9er7f5PjY29jR8jJaaF4yViRJCCBGyDJ0d4HQ6SUhIaP7Z4XBQUFDQ7vHr169n8uTJAFx00UWMGzeOK6+8kmAwyC9/+UvOPffcNs8rLCzsbtlbcLvdzdeoLq8FoGDPXhLCjT26rlacXD+iJambjkn9tE/qpn29UTedBlR3ZGdns3v3btatWwfAgQMH+P777/nwww8B+PWvf83OnTu59NJLW52bkpLSo/cuLCxsvsZ+3xH4+BiDhw7j/ISIHl1XK06uH9GS1E3HpH7aJ3XTvu7UTX5+fpvPd9rF53A4mrvsQG1RORyOVsdt376dFStWsHz5ckwmEwD//ve/ufDCC7Hb7djtdiZNmsQXX3zRpQL3hGz7LoQQoa/TgEpNTaW4uJiSkhI8Hg85OTmkp6e3OGbPnj0sWLCA5cuXExMT0/z8kCFD+Pzzz/H5fHi9Xj7//PN2u/h6k13GoIQQIuR12sVnMBhYsGABc+fOxe/3M2vWLJKTk1m2bBmjR49mypQpLF68GJfLxfz58wEYPHgwK1asICMjg08++YSsrCwURWHSpEmtwu10sJvUjyVbbgghROjq0hhUWloaaWlpLZ5rCiOANWvWtHmeXq9n4cKFp166U2Q3SxefEEKEOs2uJAHSghJCiFCmyYBqbkHJiuZCCBGyNBlQVqMeRZFJEkIIEco0GVCKoqibFsoYlBBChCxNBhSc2PZdxqCEECJkaTag7GaDjEEJIUQI02xA2Ux6GYMSQogQptmAspsNElBCCBHCtBtQJnXTQiGEEKFJswFlM8u270IIEco0G1B2kx6XTDMXQoiQpd2AkjEoIYQIadoNKJPaxRcMBvu6KEIIIU6BZgPKZtYTCEKjL9DXRRFCCHEKNBtQTXtCSTefEEKEJu0GVPOuujJRQgghQpF2A8rUtOWGtKCEECIUaTagbGbZtFAIIUKZZgOquQUlXXxCCBGSNBtQNpkkIYQQIU2zARXWNElC1uMTQoiQpNmAspnVLj4ZgxJCiNCk2YD68XtQ0oISQohQpNmAshh16BRpQQkhRKjqUkDl5eWRkZHB1KlTWblyZavXV69ezfTp08nKyuK2226jtLS0+bXDhw/z61//mmnTpjF9+nQOHTrUe6XvgKIo2E0G6mSShBBChCRDZwf4/X4WLlzI6tWrcTgczJ49m/T0dEaMGNF8TEpKChs2bMBqtfLKK6+wZMkSli5dCsCjjz7K3XffzcSJE6mvr0enO3ONNptZttwQQohQ1WlaFBQUkJSURGJiIiaTiczMTHJzc1scM378eKxWKwBjx46lrKwMgKKiInw+HxMnTgTAbrc3H3cmNK1oLoQQIvR02oJyOp0kJCQ0/+xwOCgoKGj3+PXr1zN58mQAiouLiYiI4N577+XQoUNMmDCB3/3ud+j1+lbnFRYWnkr5m7nd7lbX0AW8HK2o7vG1taCt+hEqqZuOSf20T+qmfb1RN50GVHdkZ2eze/du1q1bB4DP52Pnzp1s2rSJwYMH88ADD7Bx40ZuvPHGVuempKT06L0LCwtbXSM6r6pXrq0FbdWPUEnddEzqp31SN+3rTt3k5+e3+XynAeVwOJq77EBtUTkcjlbHbd++nRUrVrBu3TpMJhMACQkJpKSkkJiYCMCUKVP46quvulTg3hBmNnC01n3G3k8IIUTv6XQMKjU1leLiYkpKSvB4POTk5JCent7imD179rBgwQKWL19OTExMi3NramqoqKgA4NNPP20xueJ0s5lkkoQQQoSqTltQBoOBBQsWMHfuXPx+P7NmzSI5OZlly5YxevRopkyZwuLFi3G5XMyfPx+AwYMHs2LFCvR6PY8++ii33XYbAKNGjWqze+90kUkSQggRuro0BpWWlkZaWlqL55rCCGDNmjXtnjtx4kQ2b958aqXrIZlmLoQQoUuzK0mAOgZV7/ERDAb7uihCCCG6SdMBZTMZCATB7Q30dVGEEEJ0k6YDym6Wbd+FECJUaTqgmjYtlHEoIYQIPZoOqKZt32XBWCGECD3aDqgTu+rKlhtCCBF6NB5QTWNQ0sUnhBChRtMB9eMYlLSghBAi1Gg6oJq3fZcWlBBChBxtB1RTF5+0oIQQIuRoPKCaWlASUEIIEWo0HVBmgw6dIt+DEkKIUKTpgFIURVY0F0KIEKXpgAJ1RXMZgxJCiNCj+YCymw0yi08IIUKQ9gPKZJDvQQkhRAjSfEDZTHppQQkhRAjSfEDZzQZZi08IIULQgAioeplmLoQQIUf7AWWSWXxCCBGKNB9QNpMBl4xBCSFEyNF8QNnNeuo9PoLBYF8XRQghRDcMgIAyEAxCg1daUUIIEUq0H1CmphXNJaCEECKUdCmg8vLyyMjIYOrUqaxcubLV66tXr2b69OlkZWVx2223UVpa2uL1uro6Jk+ezMKFC3un1N3QvGmhTDUXQoiQ0mlA+f1+Fi5cyKpVq8jJyWHLli0UFRW1OCYlJYUNGzawefNmMjIyWLJkSYvXly5dymWXXda7Je+iH/eEkhaUEEKEkk4DqqCggKSkJBITEzGZTGRmZpKbm9vimPHjx2O1WgEYO3YsZWVlza/t3r2b8vJyJk6c2MtF7xqbSfaEEkKIUGTo7ACn00lCQkLzzw6Hg4KCgnaPX79+PZMnTwYgEAiwaNEilixZwvbt2zt8n8LCwq6WuU1ut7vNaxw/6gbg26L9hDU4e/Qeoay9+hFSN52R+mmf1E37eqNuOg2o7sjOzmb37t2sW7cOgFdeeYXJkye3CLj2pKSk9Oi9CwsL27yGJa4e3j6MPjyOlJShPXqPUNZe/Qipm85I/bRP6qZ93amb/Pz8Np/vNKAcDkeLLjun04nD4Wh13Pbt21mxYgXr1q3DZDIB8MUXX5Cfn8+rr75KfX09Xq8Xm83G7373uy4VujcMi7HhiDDz8ffHuWXcwA0oIYQINZ0GVGpqKsXFxZSUlOBwOMjJyeG5555rccyePXtYsGABq1atIiYmpvn5k4/buHEju3fvPqPhBOquupOS43i30Ik/EESvU87o+wshhDg1nU6SMBgMLFiwgLlz5zJ9+nSmTZtGcnIyy5Yta54ssXjxYlwuF/Pnz2fGjBncfffdp73g3TEpOZYql5fdpdV9XRQhhBBd1KUxqLS0NNLS0lo8N3/+/ObHa9as6fQaM2fOZObMmd0rXS+ZOCIWgI+KjnNhYlSflEEIIUT3aH4lCYDYMDOjhkSQ992xvi6KEEKILhoQAQVwZXIsuw5WytYbQggRIgZMQE1OjsPrD/Lp/vK+LooQQoguGDABdUnSICxGHXnfHe/rogghhOiCARNQFqOey8+JYds+GYcSQohQMGACCmBycizfH6vncFVDXxdFCCFEJwZUQE1KjgPgo33SzSeEEP3dgAqokY4w4sPNbCuSgBJCiP5uQAWUoihcmRzLR/uOEQgE+7o4QgghOjCgAgrUZY8qXV6+OVzT10URQgjRgQEXUE3LHm0rktl8QgjRnw24gIoPt5AyOIJt8n0oIYTo1wZcQIHazbfzQAUu2QZeCCH6rQEbUOqyRxV9XRQhhBDtGJABddmwaMwGnXTzCSFEPzYgA0pd9ihalj0SQoh+bEAGFKjdfPuO1lFW7e7rogghhGjDAA4oddkjaUUJIUT/NGAD6vyEcGLDzGyTdfmEEKJfGrABpSgKk5Jj+bjouCx7JIQQ/dCADShQx6HK6z3sOSLLHgkhRH8zoAPqyhPLHm3cVdrHJRFCCPFTAzqg4iMs/MdliazdUcw3h6v7ujhCCCFOoo2A8rgIP/gu+Lu/dNHj01IYZDPy+Mav8ctYlBBC9BtdCqi8vDwyMjKYOnUqK1eubPX66tWrmT59OllZWdx2222UlqpdZoWFhfziF78gMzOTrKws3nrrrd4tfZPyfZy94/fw1avdPjXSZuR/XXcBBYeq+ceO4t4vmxBCiFPSaUD5/X4WLlzIqlWryMnJYcuWLRQVFbU4JiUlhQ0bNrB582YyMjJYsmQJABaLhUWLFpGTk8OqVat4+umnqak5DRMSEsbQMOg8+OjPp9SK+vmFQ5iUHMuSrXs5Ut3Q++UTQgjRbZ0GVEFBAUlJSSQmJmIymcjMzCQ3N7fFMePHj8dqtQIwduxYysrKADjnnHMYNmwYAA6Hg+joaCoqTsMCrYpC+QV3QMUP8M3GUzhd4anrU/EHg/wh+5veL58QQohuM3R2gNPpJCEhoflnh8NBQUFBu8evX7+eyZMnt3q+oKAAr9fL0KFD2zyvsLCwK+VtlzvmcmIjz0V59yl+MIwCpfvDazenRrF6l5MXt+7kiqH2HpWnv3G73T2uY62SuumY1E/7pG7a1xt102lAdUd2dja7d+9m3bp1LZ4/evQoDz/8MIsWLUKnazs4UlJSevTehYWFWK5+AjbcSUpwH1xwfbev8cTIADsOf8SqXdX84qqxhJl7tXr6VGFhYY/rWKukbjom9dM+qZv2dadu8vPz23y+02aGw+Fo7rIDtUXlcDhaHbd9+3ZWrFjB8uXLMZlMzc/X1dXxm9/8hgceeICxY8d2qbCnbNQNEDMC8p6FYPdn5Bn1Op6emUpZjZtnt+49DQUUQgjRVZ0GVGpqKsXFxZSUlODxeMjJySE9Pb3FMXv27GHBggUsX76cmJiY5uc9Hg/33HMPM2bM4Nprr+390v+UTg+THgLn1/Dd/5zSJS4eOohfjkti7Y5iviqp6t3yCSGE6LJOA8pgMLBgwQLmzp3L9OnTmTZtGsnJySxbtqx5ssTixYtxuVzMnz+fGTNmcPfddwPw9ttvs3PnTt544w1mzJjBjBkzTn9/beqNEDUUPlx8Sq0ogIevPY+4MDOPb/wanz/QywUUQgjRFV0aZElLSyMtLa3Fc/Pnz29+vGbNmjbPawqlM0pvhCsfhC33w/fvwYgp3b5EhMXI//75KOa9vIvn3yvigakje7+cQgghOqSNlSR+auwtEHEW5C055VbUtaMTmHnRWSzL3cfKvO97uYBCCCE6o82AMphh4nw4uAMOfHxKl1AUhcWzx3DdmME8/da3rNr2Qy8XUgghREe0GVAAF98K9nh1LOoUGfQ6lv5iLJmpg/lTTqGElBBCnEHaDSijFa74Lez/EEo+O+XLGPQ6lv7HWKanJvCnnEJe+mh/LxZSCCFEe7QbUACX/hqs0epYVA8Y9TqW/cdFTBudwMIte1jzsYSUEEKcbtoOKHMYTPgv2PcO7M/r0aWMeh3/9+aLyBjl4MnNe/i7rHwuhBCnlbYDCuDy30BMMvxzDhz+okeXMup1PH/zxVxzgYMF2d+watsPBE9xlqAQQoiOaT+gLBFwazZYo+AfM+Foz74obDLo+MstFzNttDom9eC/vsLl6f4WH0IIITqm/YACiDxLDSmDGf4+A8p79r0mk0HHX2+5mIemjmTTl6XM/Nt29h+v76XCCiGEgIESUADRw9WQCvjUkKoq6dHldDqF305JZu0dl+OscfPz5z/inW/KOj9RCCFElwycgAKIOw9+9Qa4a9SQqnX2+JKTR8ax+bdXck6cnbv+kc+i//lW1u8TQoheMLACCmDwhTDndagtg3/cAK6e7/B79iAbr989gVvGDWX5B99z60ufcbyusRcKK4QQA9fACyiAoePg5legvAjWzQJ3dY8vaTboefqGVJbMHkP+gUquXbqNDfmHZJafEEKcooEZUADDr4Kb1kJZAay4EvZv65XL3nhpIpvumcjZg6w89PpX3LhiB3sO1/TKtYUQYiAZuAEFcN40uP0t0Blg7XXw9mPgcfX4simDI9g47woWzxrDD8frue75bTz55jdUN3h7odBCCDEwDOyAArW77+6P4PK74NPl8MIkKPm8x5fV6RRuuiyR9x+6il+OT+LvO4qZ8twHvL6zhEBAuv2EEKIzElAAJjtMX6JOQ/c1wkvXwLv/W33cQ5E2IwtnjObNe69kaLSNh9cXcMPy7eQUHMErs/2EEKJdElAnG34VzNsOY+fAR3+GlT+DIwW9cunRZ0Wy/u4rWDJ7DBX1jdzzyi4mL36fv75fRLnM+BNCiFYkoH7KEgEz/gK3/Atc5fD/fgYfLgF/z5cz0ukUbrw0kQ8euopVt17KuXFhLNm6lwn/5z0efv0rdpf2fDahEEJohaGvC9BvjcyA/9oBbz0M7/8J9r4FN6xQv+x7qkp3Qe5C9Mf3cfWdW7l67jj2OWtZu6OYDfmlvJ5/iEuSBjFtdALp58czPC6s9z6PEEKEGGlBdcQWDbNfhBvXQGUxrJgE2/8CgW6OHR37Dl77ldoaKysAdxX86zbweUh2hPOn61P55Ikp/D4zhTq3jz/lFJL+3If87NkP+OOWPWwvOo7HJ+NVQoiBRVpQXTHqBkiaCJvnwzv/rbamZvwVos/p+LyqEvjw/8CXr4DRBmmPwYR74PtceP12eOf3MF3dkj7SamTupOHMnTSckgoX7+89Sm7hUf6x4wAvfrSfcLOBySPj+OX4JMYPj0ZRlNP/uYUQog9JQHVVWDz8xyvw1T/h7Udg+UQYc5O6jYcpTL2Zw9QZgaZw+P49+HwVEIRxd8Okh8Aeq15r1A3qNvSf/A0SL4fU2S3eKjHaxq0ThnHrhGHUN/r4uOg47317lHf2OMn5+ggXnh3J3Wnncs2oBPQ6CSohhDZJQHWHosDYm+GcSZDzO9izCRrrINDGF3AVHVx4C1z1GEQltn596kJ1TOrN+8AxGuLPb/Mt7WYD14xK4JpRCTz5cz/r8w/x/7b9wLyXd3FOrJ3/nDScmRefhcWo793PKoQQfaxLAZWXl8dTTz1FIBDgxhtv5K677mrx+urVq3n99dfR6/VER0fz9NNPc9ZZZwHwxhtvsHz5cgDmzZvHDTfc0MsfoQ9Eng23/PPHn32N4KmHxlrw1KmhFe6AQcPav4beCDeuhhcmw79+Bf/5HpjDO3xbi1HPL8cncfPlQ/mf3WWs+PB7nnjja/787++4Y+IwpqTEMyIuDINehhaFEKGv04Dy+/0sXLiQ1atX43A4mD17Nunp6YwYMaL5mJSUFDZs2IDVauWVV15hyZIlLF26lKqqKv7yl7+wYcMGFEVh5syZpKenExkZeVo/1BlnMKs3W3T3zosYArNehH9cr7akZr+kttI6odcpZI4ZzPTUBHZ8X86KvB9YsnUvS7buxWTQcX5COKOGRDJqSASjhkRwfkLEqX0uIYToQ50GVEFBAUlJSSQmqt1UmZmZ5Obmtgio8ePHNz8eO3Ysb775JgAfffQREydOJCoqCoCJEyeybds2rrvuut78DKFteBqk/x5yF0LiOBh/d5dPVRSFK0bEcsWIWIqP1/NlSRXfHK7mm8M15BQc5tXPDgKgUyDWZmBYXBVDoiwMibIyJMrKWSfuk2Js0kUohOh3Og0op9NJQkJC888Oh4OCgvZXV1i/fj2TJ09u91yns+1NAgsLC7tc6La43e4eX6PPxFzL2UPeI2zrExzwRtMQm3pKlznPAucNV5g5PJJgMIKj9T6Kyj38UNlIaVUj5Q0udhyv4Xi9D/9JywEadHB+rIXRDvWWEm/BZhw43YQh/f/OGSD10z6pm/b1Rt306iSJ7Oxsdu/ezbp167p9bkpKSo/eu7CwsMfX6FPnrIOVVzHssz/A7Vsg5tweX/IC4KoTj0+uH38gyLHaRkqrGiitamB3aTWf7q/g9W+q+efXVeh1CqOHRHD5OdGMPiuSs6KsDI6y4gg3a3J8K+T/3znNpH7aJ3XTvu7UTX5+fpvPdxpQDoeDsrKy5p+dTicOh6PVcdu3b2fFihWsW7cOk8nUfO5nn33W4tzLL7+8SwUecKyD4BfrYO3PYdXV6pT2pAmn5a30OoWESAsJkRYuSRrEzy8cAkB9o49dByv5bH8Fn+6vYO2OAy2+IKxTwBFhYXCkhcFRVobF2Lh46CAuGjqIaLvptJRVCDFwdRpQqampFBcXU1JSgsPhICcnh+eee67FMXv27GHBggWsWrWKmJiY5uevvPJK/vznP1Ndra4x99FHH/Hggw/28kfQkIRUmPsuvHwj/P3ncP3yVt+ROp3sZgOTkuOYlBwHgNvr52CFi8NVDRyucnOk+sf7b0qr2bq7DN+JrUOGx9q5OGkQFw8dxCVJg0iOD0Mn39ESQvRApwFlMBhYsGABc+fOxe/3M2vWLJKTk1m2bBmjR49mypQpLF68GJfLxfz58wEYPHgwK1asICoqiv/6r/9i9mz1H9l77rmnecKEaEfMuWpI/XMObLhTXWJp0kNdmt3X2yxGPSMd4Yx0tD39vcHjp+BQFbsOVpF/oJL3vz3K+vxDAJgNOiKsRsLNBsIsBsItBsLMBsLMRiKsBs6KspIYbWNotI3EaBthZvlKnhCipS79q5CWlkZaWlqL55rCCGDNmjXtnjt79uzmgBJdZIuGWzdB9r3w3h+hcj9ct1T97lQ/YjXpGTc8hnHD1VZzMBjkQLmL/AOV7HXWUuv2Uuv2Udfoo87to7zORa3bR5XLQ73H3+Ja0XYTidE2EgepswsTIk90JUZaGRxpISbMLKtmCDHAyJ+t/ZXBDDNXquv9fbhIXdfvpr+rSyv1U4qiMCzWzrBYe4fHBYNBqhu8HKxwUVLRwMEKFwcrXByqdPF1aTXv7HG2WhzXoFNwRFiItpuIshmJtBqJshmJspqItBqJtBmJCzerEzoiLYRb+leYCyG6TwKqP1MU+NkT6ooUb94HL06FS26Hsy6BwReC0dr+uY216np/Bz+BQ58xtLYavkhQ97syR6irVlhO3A8apl7TOugMfSyFKJuJKJuJMWdHtXo9GAxSUe/hSLWbsmp1zKvpcYXLQ5XLy6HKBqpcHqobvASCrd8j3GxgSJSVwSe+9xUfbj4RbiaibWrIDbKrj0Gd2ej1B/AHgvj8QbwB9bECRFiN8j0xIfqABFQoGHuLurzSm7+FrU+oz+kMEH8BnH2pGi4JqVDxgxpIB3dA2dcQDKhrAjpGAQaoOQRHa9TwaqyBwE82YYwZAWdfpl7v7EvVNQL7oFtRURRiwszEhJkZfVbHq44EAkHqPD6q6r0crXVzuNrN4aoGjlQ1UHpiQsdXJVVUutpYL7Hp/YAgP3T4PlajXm2x2UxEWY0MsquPh0RaOGuQlSGR6peeEyItGDU4FV+IviABFSrOmQzzv4JaJ5TmQ+lO9f7r9bDzpR+PM9rUcJn8MAwdrwaOOZyDP/1OQjAIPje4q+HYXvV6h/KhKBe+elU9xmBRrzF6FqRknbEWVnfodAoRFiMRFiNDY2ztHuf1B6hyealyeaio91Dp8lLp8lDp8nCw1EmCIw6jXodBp2DQ6zDqFfQ6hWAQqhvU8ypd3uZr7C2rbb7OyRQFHOEWBkdZiLAYsZv12EwG7CY9NrM6UcRq1KNTIBCEIGqLMRAMEgyqzw2OtHDBkAiGx9o1+b0zIbpKAirUhDvg/OnqDdTNE8uLwPm12lWXMKZrrR5FUbsIjVYIT1CXXAI1uKpL4NBO9fbd/6gtty0PQvJUNazOm6ZuK9JVvkb1WsXbYH+e2rqzRUNkotoyPPkWPkRt2TUvvFv742NPvRqY56R1e1ajUa8jLtxMXLi51WuFhV5SUkZ263pNGjx+Dlc3nJiKr7baDlc1cKRa7YIsrfLjavRR7/FT3+hrnpbfFU3rKqYkRHDBEPVmM+k5WtOIs8aNs6YRZ62boyce1zX6MOoVNWj1Okx6BYNOh9Ggw2LQEW03Nd9iwkxE283E2E3En6gX2WNM9DcSUKFOp4O4keqtNygKRA1Vb6NnQsZTcPgL2L0Bdm9UN2s02tWQGjFF3QfLYAa9SW1xGU7cN9bBgY/UQDr4KfgaAEUdO0u9UW25VR+C/dug9rDaHdlVsefB5f8JF96s7sHVnmBQbR3ue0cNXUWvdnnqdOq9ogNFT3StB8Kvh8FdDPeTWE16zo0L49y4DspxEo8vgMvjIxhUq1pRFHQn3QMcqmxgz+Ea9hypYc/hGt7ZU8ZrO0vavF6M3UR8hAVHhJmkGJs6fuYP4A0E8foC+AIBXA1+yjw+dh2sotLlwd9GSEbZjGoYDo5QbwkRJDvCmsfefP4A5fUejtU2/nirawTAdKLFaTToMOp1mPQ6DHoFBYVAi9ZhkMCJ+2ibiRHxYSRG22R2pmiXBJTomKLAWRert6l/hIPb4evXYU827F7f+fnxo+CS29QuyqQr2u4m9Pug9ogaIjWH1cAzhZ2YzBGmTuQwhanjbns2wacvwFu/UxfYHTtHDaumpaG8brWl9t1W2LcVqtQFc7FEnuhP86thGAxAQH3sCPrhy/+rBm/iZTD0CrWsZ1/a8USUkwX8cHwflBXAka/UVu2gYTDkYhhykTq+p9NhMugwGTpedaPpu2fXjx0CdU6Cx76ltqSI6kPf4A/qUBJGYx06lqihqZgs7XdrtlnMQJAat5fyerWrs6LeQ1m1m2/Laik8UsO/PjtAlPcoQ3VHOUfnJNlay2FvGN95YjkYjKc0GIuH3huXNBl0DI+1MyI+rPmWEGE5KbwVFFDvFfD4A2p3a/2PXbQV9Wq3a6MvwNBoG+fE2hkeZ+ecWDtDIq2tvjBe1+ijtLKBkhMzR8tqGokPNzM8zs7w2DDOGmSV0OwnlGAw2PU+h9MkPz+fSy65pEfXkDWxOtbr9ePzQNUBdRzL5wF/44+PfW61JXL25RAW13vv2SQYVLsMP3sBvtmkbhg54mrQGeGHD9TWmsEKw6+CkddA8jVq92E79u3aRrLpmDq55MB2cH4DBNXrOS4Aa7QacJYI9d4cqd7rdODco4aS8xvwutQL6s0QPVytn6bnzBFq63HIRTBkrDpW6HOr3Z8n33vdaqge3wvHvoPG6h8LagpXA7bpmooeYkdCwmh1Qkv0cLWMpvCWwW4KU//QcFdBfTm4jkP98R/v646q37Wr2E+w6gCK39N+1aPgtTkIRg3FED0M9AYCfh9Bv4+Az0sw4CXY9LPBSsASRcAyiKAl6sRtEAFLJFX1jZQfK6O64iiuqmN468vRN1YTSR0KQY4HIzlOpHp/4vGxYCR1QSsWxYMF9WbXeYk1B4gx+zHoYG+tmYPeCI4GB+HCgtmgU4Mqysqx2kYOVbpajRvqdUqLVqXJoHDxIDfjwspJMR/DYDRTb4ql3hRLnSkOlyGCIOrYYHXlcYYnDiHMbMBuNhBu0hNm8BGhc2MwmgmaI9vsjtbpFCwGHRajHotRf3oD0e8FV7n6/5w5/Ix96b+7a/G1lQHSghKnxmCC2OS+eW9FUVs6iZfBNU9B/hrYtRZ0erjolzAyA4Zd2eXWj88aCymT1C5NgIZKdYr+gY/V4HFXqy07d7U6+7EpIEANnoQx6vT/hDFqCMWOBL1BbRke36t2kZbugsO74JPlbe/AfDJ7PMSdpy5zFXeeer2489WxwmAAKvarY45lu8G5Gw7sUFu1HdaZXg23tpgjYFASxKegnJ+pfvdu0Dnqffhg9n25neRYE1QdQKk8gKmyWA3fkk8gGECv06utW71Rvdfp1fdzO+FYJTRUwE9CL/6nZTCFERgURaMxEn8ggNFdisldjtJemU8WABpOPNYBJ4YZvXobNYYYyhuicNZHEjCFYYgOx3J2BLbwSMIjIomMHESYzYbr2H4aDheiHP+OsNofMNe6oLbtt/MG9RwjkqPBKILosO9uwK64CacBG24Myo/d1a6gmSPB6BO3GI6g3tcFrYQrDURQT7jiIkpxEaVzE6lzEaY0YtaDSa9gOnFv1HFifFEhYLDjN4UTMNrxG+34jWH4DWH4DRbMnkrM7mOYGo5hcB1FV++E+uMoqAEc1JsJ2mLxW2PxWmNpNEfTaIoGRcHkb8AQcGH0udD76tF7Xei89QQBv96CV2fBo5hpVMy4FQsNQROGkVMYMekXnf83OkUSUCK0hTvgqkfVW2+xDlJDbmRG26/7veCuUVuNYQlqS6oteoM6xd8xSg1OUFtKx/aqE0EMlhObXVpaPtZ38Gup6CF2hHobddLu1K4KtYu0sa715JLGOvX9bDFgjz3p/sRjo6XD6vDZ4iEp5dQXLw4GwdugtuAaKtWbolfr2TpI/fK5wYwOaPEnRSCgHlt/VG3l1R9TP5PRqtaT0aaW3XBiso+iqMfVOaG2DGOdk5jaMmLqnIysc4LnB6ith/LaVmOedsAePkT9o2vkhBN/GCTjH3QuAb8Xpa4MpdaJUu/EUOtkcH0Zg2ud1NXVYoqIx2uw0aiz4dTZaFCsuLAS8DViayjD6naS0lDGxe69WN3HUGj53n5FT6M+DLc+jAadnQasuP1Q5Q3S2BDEF1BbrgEUdASxKuWEcwg7ajBG0IBZUb8y4gvqOEYUR4Pq7VgwlaNEUaWLwhxoZJCvmlhPDTFV1cQq3xOjfEEMNYBCPWZqsFAftODCQn3QTAMWgoAFD1alFiserDRiUTw4aOSb6loJKCH6Fb0R7DGdH9cWg1mdjNHbbNHd39H5TFEUMNnUW8SQrp+n06n1bI+B+C52TztGdX5MMKj+oeCpB0+t2q0aMUTtHv0J/YkbscPbvNShE91YZqBL02T8XjVAG+uau4z1Rhs2RaGt0cRgMEhdow9nTSNHa9wcr/dQAxz7yeQafcCL3u/GpbPi9oHbG6DB68ft9eP3+jH5Ahj0Cj6jnmqTAY9RT7VJT6lRj9WkJxAM0ugN0OhTz2n0BXB7/bi9AfQ6hUirkQiruoJLpNWIYjFgsRqZYDu9uxhIQAkhBhZFUVteRsup/6FxqvTGDsdDf0pRFMItRsItRkbEd22mqJbItwCFEEL0SxJQQggh+iUJKCGEEP2SBJQQQoh+SQJKCCFEvyQBJYQQol+SgBJCCNEvSUAJIYTolySghBBC9Ev9ZjVzIYQQA1dbq5n3i4ASQgghfkq6+IQQQvRLElBCCCH6JQkoIYQQ/ZImttvIy8vjqaeeIhAIcOONN3LXXXf1dZH61OOPP84HH3xATEwMW7ZsAaCqqooHHniA0tJSzjrrLJYuXUpkZGQfl/TMO3LkCI888gjl5eUoisJNN93EbbfdJvUDNDY2MmfOHDweD36/n4yMDO677z5KSkp48MEHqaqqYtSoUSxevBiT6fTuA9Rf+f1+Zs2ahcPh4IUXXpC6OUl6ejp2ux2dToder2fjxo09/r0K+RaU3+9n4cKFrFq1ipycHLZs2UJRUVFfF6tPzZw5k1WrVrV4buXKlUyYMIF33nmHCRMmsHLlyj4qXd/S6/U89thjvPXWW7z22mu88sorFBUVSf0AJpOJtWvX8uabb7Jp0ya2bdvGl19+ybPPPsvtt9/Ov//9byIiIli/fn1fF7XP/P3vf+fcc89t/lnqpqW1a9eSnZ3Nxo0bgZ7/uxPyAVVQUEBSUhKJiYmYTCYyMzPJzc3t62L1qcsuu6zVXym5ublcf/31AFx//fW8++67fVCyvhcfH8+oUequq2FhYQwfPhyn0yn1g7o5nt1uB8Dn8+Hz+VAUhU8++YSMjAwAbrjhhgH7+1VWVsYHH3zA7NmzAXW3W6mbjvX09yrkA8rpdJKQkND8s8PhwOl09mGJ+qfy8nLi4+MBiIuLo7y8vI9L1PcOHTpEYWEhF154odTPCX6/nxkzZnDFFVdwxRVXkJiYSEREBAaDOhqQkJAwYH+/nn76aR5++GF0OvWfzcrKSqmbn7jzzjuZOXMmr732GtDzf3c0MQYlukdRFBRF6eti9Kn6+nruu+8+nnjiCcLCWm6lPZDrR6/Xk52dTU1NDffccw8//PBDXxepX3j//feJjo5m9OjRfPrpp31dnH7p1VdfxeFwUF5ezh133MHw4cNbvH4qv1chH1AOh4OysrLmn51OJw6How9L1D/FxMRw9OhR4uPjOXr0KNHR0X1dpD7j9Xq57777yMrK4pprrgGkfn4qIiKCcePG8eWXX1JTU4PP58NgMFBWVjYgf7927drFe++9R15eHo2NjdTV1fHUU09J3Zyk6bPHxMQwdepUCgoKevx7FfJdfKmpqRQXF1NSUoLH4yEnJ4f09PS+Lla/k56ezqZNmwDYtGkTU6ZM6dsC9ZFgMMh///d/M3z4cO64447m56V+oKKigpqaGgDcbjfbt2/n3HPPZdy4cWzduhWAN954Y0D+fj300EPk5eXx3nvv8ec//5nx48fz3HPPSd2c4HK5qKura3788ccfk5yc3OPfK00sdfThhx/y9NNPN08BnTdvXl8XqU89+OCDfPbZZ1RWVhITE8Nvf/tbrr76au6//36OHDnCkCFDWLp0KVFRUX1d1DNu586dzJkzh5EjRzaPJTz44IOMGTNmwNfPt99+y2OPPYbf7ycYDHLttddy7733UlJSwgMPPEB1dTUpKSk8++yzA3YqNcCnn37KSy+91DzNXOoGSkpKuOeeewB1HPO6665j3rx5VFZW9uj3ShMBJYQQQntCvotPCCGENklACSGE6JckoIQQQvRLElBCCCH6JQkoIYQQ/ZIElBBCiH5JAkoIIUS/9P8BWqOhwOBWidkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.compile(\n",
    "    optimizer='adam',\n",
    "    loss='mae',\n",
    ")\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data=(X_valid, y_valid),\n",
    "    batch_size=512,\n",
    "    epochs=50,\n",
    "    verbose=0,\n",
    ")\n",
    "history_df = pd.DataFrame(history.history)\n",
    "history_df.loc[:, ['loss', 'val_loss']].plot()\n",
    "print(\"Minimum Validation Loss: {:0.4f}\".format(history_df['val_loss'].min()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87a5201b",
   "metadata": {
    "papermill": {
     "duration": 0.017712,
     "end_time": "2021-12-29T13:00:27.472120",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.454408",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 2) Evaluate Dropout\n",
    "\n",
    "Recall from Exercise 4 that this model tended to overfit the data around epoch 5. Did adding dropout seem to help prevent overfitting this time?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9edd8b78",
   "metadata": {
    "papermill": {
     "duration": 0.017542,
     "end_time": "2021-12-29T13:00:27.507178",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.489636",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "It did improve it a a bit. The validation error is a lot less. But I would argue that we are still overfitting a bit 🙁."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5f9e46a4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:27.548119Z",
     "iopub.status.busy": "2021-12-29T13:00:27.546294Z",
     "iopub.status.idle": "2021-12-29T13:00:27.550993Z",
     "shell.execute_reply": "2021-12-29T13:00:27.551413Z",
     "shell.execute_reply.started": "2021-12-29T12:53:07.180421Z"
    },
    "papermill": {
     "duration": 0.026863,
     "end_time": "2021-12-29T13:00:27.551535",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.524672",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"2_Q2\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct:</span> \n",
       "\n",
       "\n",
       "From the learning curves, you can see that the validation loss remains near a constant minimum even though the training loss continues to decrease. So we can see that adding dropout did prevent overfitting this time. Moreover, by making it harder for the network to fit spurious patterns, dropout may have encouraged the network to seek out more of the true patterns, possibly improving the validation loss some as well).\n"
      ],
      "text/plain": [
       "Correct: \n",
       "\n",
       "\n",
       "From the learning curves, you can see that the validation loss remains near a constant minimum even though the training loss continues to decrease. So we can see that adding dropout did prevent overfitting this time. Moreover, by making it harder for the network to fit spurious patterns, dropout may have encouraged the network to seek out more of the true patterns, possibly improving the validation loss some as well)."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# View the solution (Run this cell to receive credit!)\n",
    "q_2.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b12703c6",
   "metadata": {
    "papermill": {
     "duration": 0.018203,
     "end_time": "2021-12-29T13:00:27.588030",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.569827",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now, we'll switch topics to explore how batch normalization can fix problems in training.\n",
    "\n",
    "Load the *Concrete* dataset. We won't do any standardization this time. This will make the effect of batch normalization much more apparent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d6174906",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:27.630373Z",
     "iopub.status.busy": "2021-12-29T13:00:27.629849Z",
     "iopub.status.idle": "2021-12-29T13:00:27.645807Z",
     "shell.execute_reply": "2021-12-29T13:00:27.645376Z",
     "shell.execute_reply.started": "2021-12-29T12:53:44.680808Z"
    },
    "papermill": {
     "duration": 0.039477,
     "end_time": "2021-12-29T13:00:27.645910",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.606433",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "concrete = pd.read_csv('../input/dl-course-data/concrete.csv')\n",
    "df = concrete.copy()\n",
    "\n",
    "df_train = df.sample(frac=0.7, random_state=0)\n",
    "df_valid = df.drop(df_train.index)\n",
    "\n",
    "X_train = df_train.drop('CompressiveStrength', axis=1)\n",
    "X_valid = df_valid.drop('CompressiveStrength', axis=1)\n",
    "y_train = df_train['CompressiveStrength']\n",
    "y_valid = df_valid['CompressiveStrength']\n",
    "\n",
    "input_shape = [X_train.shape[1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c47b62bf",
   "metadata": {
    "papermill": {
     "duration": 0.018145,
     "end_time": "2021-12-29T13:00:27.682792",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.664647",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Run the following cell to train the network on the unstandardized *Concrete* data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a3266a25",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:27.730754Z",
     "iopub.status.busy": "2021-12-29T13:00:27.729358Z",
     "iopub.status.idle": "2021-12-29T13:00:32.700320Z",
     "shell.execute_reply": "2021-12-29T13:00:32.700948Z",
     "shell.execute_reply.started": "2021-12-29T12:53:49.684368Z"
    },
    "papermill": {
     "duration": 5.000018,
     "end_time": "2021-12-29T13:00:32.701127",
     "exception": false,
     "start_time": "2021-12-29T13:00:27.701109",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Validation Loss: nan\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZzUlEQVR4nO3de1BU9x338c8qhdxU8MJiE6JNSzoyIYmjyeBIxuniLgaComI7SaNCkiHB6EhNc9PRsXamxpSON3KReEkm1ieVPkFHiMUISRRDm5b6BE23VjtxhESWkatahYLn+aPjTnaWRJOzwM/1/fqLc/a3u9/zGyfv7LKzOCzLsgQAgGEGDfQAAAD0hkABAIxEoAAARiJQAAAjESgAgJEiBnqAq1FbWzvQIwAA+tCECROCzl0TgZJ6H/5a5fV6NW7cuIEewxjsRyD2Ixh7Eijc9uPrXoTwFh8AwEgECgBgJAIFADASgQIAGIlAAQCMRKAAAEYiUAAAIxEoAICRCBQAXOPGjx8/0CP0CQIFADDSNfNVRwCAb2ZZll5++WUdPHhQDodD+fn5Sk9PV1NTk37xi1/o3Llz6unp0cqVKzV+/HgtW7ZMR48elcPh0OzZs5WTkzPQlxCAQAFAiPzf2gbt/Ft9SB/zpxPjNXvCbVe1dt++ffrnP/+p3bt3q7W1VdnZ2Zo4caLKysqUkpKi/Px89fT06MKFC/J6vfL5fCorK5MkdXR0hHTuUOAtPgAIE7W1tcrIyNDgwYM1cuRI3XfffTpy5IiSkpL07rvvauPGjfrXv/6lW265RfHx8aqvr9evf/1rHThwQLfccstAjx+EV1AAECKzJ9x21a92+tN9992n7du366OPPtILL7yg3NxcZWVlaffu3aqurtY777yjvXv3avXq1QM9agBeQQFAmJg4caL27t2rnp4etbS06G9/+5vuvvtuffHFFxo5cqR++tOfas6cOfrss8/U0tIiy7KUlpamgoIC/eMf/xjo8YPwCgoAwoTb7dbhw4c1Y8YMORwOPfvssxo1apRKS0u1ZcsWRURE6KabbtKaNWvU1NSkF198UZcuXZIkLVmyZICnD0agAOAad/jwYUmSw+HQ888/r+effz7g9pkzZ2rmzJlB9ystLe2X+b4r3uIDABiJQAEAjESgAABGIlAAACMRKACAkQgUAMBIBAoAYCQCBQDXmW/6+1ENDQ166KGH+nGar0egAABG4pskACBU/t//kQ5vD+1jjn9Uuvfhb1xSWFio0aNH6+c//7kkaePGjRo8eLD+8pe/qKOjQ93d3Vq8eLGmTp36rZ66s7NTK1eu1NGjRzV48GC98MILSk5O1vHjx/Xiiy/qv//9ry5duqSNGzcqNjZWBQUFamxs1KVLl7RgwQKlp6d/58uWCBQAXPPS09P1m9/8xh+ovXv3asuWLZo3b55uueUWtbS06Gc/+5lSU1PlcDiu+nF///vfS5L27Nmjf//733r88cdVUVGhd955R/PmzdP06dPV1dWlS5cu6aOPPlJsbKyKi4slSWfPnrV9XQQKAELl3oev+GqnLyQmJqq5uVk+n0+tra0aOnSoRo4cqdWrV+uvf/2rBg0aJJ/PpzNnzmjUqFFX/bi1tbV69NFHJUk//OEP9f3vf1+ff/657r33Xr3++utqbGyUx+PR2LFjdeedd2rNmjX67W9/q5/85CeaOHGi7evid1AAEAamTZumiooKvffee0pPT9eePXvU0tKid999V7t379bIkSPV2dkZkufKzMzUa6+9phtuuEF5eXmqqanRD37wA7377ru68847tW7dOhUVFdl+npAE6sCBA0pLS5Pb7fa/vPuqrq4uFRQUyO12a86cOWpoaAi4/csvv9T48eO1ZcuWUIwDANed9PR0vffee6qoqNC0adN09uxZjRgxQt/73vf05z//WV988cW3fsyJEydqz549kqTPP/9cp0+f1h133KH6+nrFx8dr3rx5Sk1N1bFjx+Tz+XTjjTdqxowZevzxx0Py96Vsv8XX09OjVatWadu2bXI6ncrOzpbL5dKPfvQj/5qSkhINHTpU77//vsrLy1VYWKh169b5b3/ppZf0wAMP2B0FAK5bCQkJOn/+vGJjYxUbG6vMzEzl5+crMzNTd911l+64445v/ZiPPPKIVq5cqczMTA0ePFirV69WZGSk9u7dq927dysiIkIjR47Uk08+qSNHjujll1/WoEGDFBERoZUrV9q+JtuBqqur05gxYxQfHy9JysjIUGVlZUCgqqqqtHDhQklSWlqaVq1aJcuy5HA4tH//ft1666266aab7I4CANe1y692JGn48OH6wx/+0Ou6y38/qje33XabysrKJElRUVG9/hn4vLw85eXlBZx74IEHQv5Cw3agfD6f4uLi/MdOp1N1dXVBa0aPHv2/J4yI0JAhQ9Ta2qqoqCi98cYb2rp1q7Zu3fqNz+P1eu2OaoyLFy+G1fXYxX4EYj+CsSeBrpf9GNBP8RUVFWn+/Pm6+eabr7h23Lhx/TBR//B6vWF1PXaxH4HYj2DsSaBQ7MexY8f03HPPBZyLjIxUSUmJrcf9Lmpra3s9bztQTqdTjY2N/mOfzyen0xm05vTp04qLi1N3d7fOnj2rmJgYffrpp6qoqFBhYaE6Ojo0aNAgRUVF+T/WCADoGz/+8Y+1e/fugR7jG9kOVFJSkk6ePKn6+no5nU6Vl5frd7/7XcAal8ul0tJSjR8/XhUVFUpOTpbD4dCOHTv8azZu3KibbrqJOAEAJIUgUBEREVqxYoWeeOIJ9fT0aPbs2UpISND69et11113KTU1VdnZ2Xr22Wfldrs1bNgwrV27NhSzAwDCWEh+BzVlyhRNmTIl4NzixYv9P0dFRWnDhg3f+BiLFi0KxSgAgDDBN0kAAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASCEJ1IEDB5SWlia3263i4uKg27u6ulRQUCC32605c+aooaFBknTo0CHNmjVLmZmZmjVrlmpqakIxDgAgDNgOVE9Pj1atWqXNmzervLxcZWVlOnHiRMCakpISDR06VO+//75ycnJUWFgoSYqJidFrr72mPXv26KWXXtJzzz1ndxwAQJiwHai6ujqNGTNG8fHxioyMVEZGhiorKwPWVFVVaebMmZKktLQ01dTUyLIsJSYmyul0SpISEhLU2dmprq4uuyMBAMJAhN0H8Pl8iouL8x87nU7V1dUFrRk9evT/njAiQkOGDFFra6uGDx/uX1NRUaHExERFRkb2+jxer9fuqMa4ePFiWF2PXexHIPYjGHsS6HrZD9uBCoXjx4+rsLBQW7du/do148aN68eJ+pbX6w2r67GL/QjEfgRjTwKF237U1tb2et72W3xOp1ONjY3+Y5/P53/b7qtrTp8+LUnq7u7W2bNnFRMTI0lqbGzUwoULtWbNGt1+++12xwEAhAnbgUpKStLJkydVX1+vrq4ulZeXy+VyBaxxuVwqLS2V9L+38pKTk+VwONTR0aG8vDw988wzmjBhgt1RAABhxHagIiIitGLFCj3xxBNKT0/Xgw8+qISEBK1fv97/YYns7Gy1tbXJ7XZr27Zt+uUvfylJ2r59u06dOqVXXnlFM2bM0IwZM9Tc3Gx3JABAGAjJ76CmTJmiKVOmBJxbvHix/+eoqCht2LAh6H4LFizQggULQjECACDM8E0SAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMFJIAnXgwAGlpaXJ7XaruLg46Pauri4VFBTI7XZrzpw5amho8N+2adMmud1upaWl6eDBg6EYBwAQBmwHqqenR6tWrdLmzZtVXl6usrIynThxImBNSUmJhg4dqvfff185OTkqLCyUJJ04cULl5eUqLy/X5s2b9atf/Uo9PT12RwIAhAHbgaqrq9OYMWMUHx+vyMhIZWRkqLKyMmBNVVWVZs6cKUlKS0tTTU2NLMtSZWWlMjIyFBkZqfj4eI0ZM0Z1dXV2RwIAhIEIuw/g8/kUFxfnP3Y6nUGR8fl8Gj169P+eMCJCQ4YMUWtrq3w+n+65556A+/p8vl6fx+v12h3VGBcvXgyr67GL/QjEfgRjTwJdL/thO1D9Zdy4cQM9Qsh4vd6wuh672I9A7Ecw9iRQuO1HbW1tr+dtv8XndDrV2NjoP/b5fHI6nUFrTp8+LUnq7u7W2bNnFRMTc1X3BQBcn2wHKikpSSdPnlR9fb26urpUXl4ul8sVsMblcqm0tFSSVFFRoeTkZDkcDrlcLpWXl6urq0v19fU6efKk7r77brsjAQDCgO23+CIiIrRixQo98cQT6unp0ezZs5WQkKD169frrrvuUmpqqrKzs/Xss8/K7XZr2LBhWrt2rSQpISFBDz74oNLT0zV48GCtWLFCgwcPtn1RAIBrX0h+BzVlyhRNmTIl4NzixYv9P0dFRWnDhg293jc/P1/5+fmhGAMAEEb4JgkAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASAQKAGAkAgUAMBKBAgAYyVag2tralJubK4/Ho9zcXLW3t/e6rrS0VB6PRx6PR6WlpZKkCxcuKC8vT9OmTVNGRoYKCwvtjAIACDO2AlVcXKxJkyZp3759mjRpkoqLi4PWtLW1qaioSDt37lRJSYmKior8IXvsscf0pz/9SaWlpfr73/+ujz76yM44AIAwYitQlZWVysrKkiRlZWVp//79QWuqq6s1efJkRUdHa9iwYZo8ebIOHjyoG2+8UcnJyZKkyMhIJSYmyufz2RkHABBGIuzcubm5WbGxsZKkUaNGqbm5OWiNz+dTXFyc/9jpdAaFqKOjQx988IHmz5//tc/l9XrtjGqUixcvhtX12MV+BGI/grEnga6X/bhioHJycnTmzJmg8wUFBQHHDodDDofjWw/Q3d2tJUuWaO7cuYqPj//adePGjfvWj20qr9cbVtdjF/sRiP0Ixp4ECrf9qK2t7fX8FQP15ptvfu1tI0aMUFNTk2JjY9XU1KThw4cHrXE6nfrkk0/8xz6fT/fff7//ePny5Ro7dqxycnKuNAoA4Dpi63dQLpdLu3btkiTt2rVLqampQWtSUlJUXV2t9vZ2tbe3q7q6WikpKZKktWvX6ty5c1q6dKmdMQAAYchWoPLy8nTo0CF5PB59/PHHysvLkyQdOXJEy5YtkyRFR0drwYIFys7OVnZ2tp5++mlFR0ersbFRr7/+uk6cOKGZM2dqxowZKikpsX9FAICwYOtDEjExMXrrrbeCziclJSkpKcl/fDlOXxUXF6djx47ZeXoAQBjjmyQAAEYiUAAAIxEoAICRCBQAwEgECgBgJAIFADASgQIAGIlAAQCMRKAAAEYiUAAAIxEoAICRCBQAwEgECgBgJAIFADASgQIAGIlAAQCMRKAAAEYiUAAAIxEoAICRCBQAwEgECgBgJAIFADASgQIAGIlAAQCMRKAAAEYiUAAAIxEoAICRCBQAwEgECgBgJAIFADASgQIAGIlAAQCMRKAAAEYiUAAAIxEoAICRCBQAwEgECgBgJFuBamtrU25urjwej3Jzc9Xe3t7rutLSUnk8Hnk8HpWWlgbd/tRTT+mhhx6yMwoAIMzYClRxcbEmTZqkffv2adKkSSouLg5a09bWpqKiIu3cuVMlJSUqKioKCNm+fft088032xkDABCGbAWqsrJSWVlZkqSsrCzt378/aE11dbUmT56s6OhoDRs2TJMnT9bBgwclSefPn9e2bduUn59vZwwAQBiKsHPn5uZmxcbGSpJGjRql5ubmoDU+n09xcXH+Y6fTKZ/PJ0lav369HnvsMd1www1XfC6v12tnVKNcvHgxrK7HLvYjEPsRjD0JdL3sxxUDlZOTozNnzgSdLygoCDh2OBxyOBxX/cRer1enTp3S0qVL1dDQcMX148aNu+rHNp3X6w2r67GL/QjEfgRjTwKF237U1tb2ev6KgXrzzTe/9rYRI0aoqalJsbGxampq0vDhw4PWOJ1OffLJJ/5jn8+n+++/X4cPH9bRo0flcrnU3d2tlpYWzZ07V2+//fZVXA4AINzZ+h2Uy+XSrl27JEm7du1Sampq0JqUlBRVV1ervb1d7e3tqq6uVkpKih555BFVV1erqqpKO3bs0NixY4kTAMDPVqDy8vJ06NAheTweffzxx8rLy5MkHTlyRMuWLZMkRUdHa8GCBcrOzlZ2draefvppRUdH2x4cABDebH1IIiYmRm+99VbQ+aSkJCUlJfmPL8fp69x2220qKyuzMwoAIMzwTRIAACMRKACAkQgUAMBIBAoAYCQCBQAwEoECABiJQAEAjESgAABGIlAAACMRKACAkQgUAMBIBAoAYCQCBQAwEoECABiJQAEAjESgAABGIlAAACMRKACAkQgUAMBIBAoAYCQCBQAwEoECABiJQAEAjESgAABGIlAAACMRKACAkQgUAMBIBAoAYCSHZVnWQA9xJbW1tQM9AgCgD02YMCHo3DURKADA9Ye3+AAARiJQAAAjESgAgJEIVB9oa2tTbm6uPB6PcnNz1d7e3uu60tJSeTweeTwelZaWBt3+1FNP6aGHHurrcfuFnT25cOGC8vLyNG3aNGVkZKiwsLA/Rw+pAwcOKC0tTW63W8XFxUG3d3V1qaCgQG63W3PmzFFDQ4P/tk2bNsntdistLU0HDx7sz7H7zHfdj0OHDmnWrFnKzMzUrFmzVFNT09+j9xk7/0Yk6csvv9T48eO1ZcuW/hq571gIuTVr1libNm2yLMuyNm3aZL388stBa1pbWy2Xy2W1trZabW1tlsvlstra2vy3V1RUWEuWLLEyMjL6be6+ZGdP/vOf/1g1NTWWZVlWZ2en9fDDD1sffvhhv84fCt3d3VZqaqp16tQpq7Oz08rMzLSOHz8esGb79u3W8uXLLcuyrLKyMmvx4sWWZVnW8ePHrczMTKuzs9M6deqUlZqaanV3d/f3JYSUnf347LPPrMbGRsuyLOvYsWNWSkpKv87eV+zsyWWLFi2yFi1aZG3evLm/xu4zvILqA5WVlcrKypIkZWVlaf/+/UFrqqurNXnyZEVHR2vYsGGaPHmy//+Kz58/r23btik/P78/x+5TdvbkxhtvVHJysiQpMjJSiYmJ8vl8/Tl+SNTV1WnMmDGKj49XZGSkMjIyVFlZGbCmqqpKM2fOlCSlpaWppqZGlmWpsrJSGRkZioyMVHx8vMaMGaO6urqBuIyQsbMfiYmJcjqdkqSEhAR1dnaqq6ur368h1OzsiSTt379ft956qxISEvp99r5AoPpAc3OzYmNjJUmjRo1Sc3Nz0Bqfz6e4uDj/sdPp9P9Hd/369Xrsscd0ww039M/A/cDunlzW0dGhDz74QJMmTerbgfvA1Vyfz+fT6NGjJUkREREaMmSIWltbr+q+1xo7+/FVFRUVSkxMVGRkZN8P3cfs7Mn58+f1xhtvaOHChf06c1+KGOgBrlU5OTk6c+ZM0PmCgoKAY4fDIYfDcdWP6/V6derUKS1dujTovWXT9dWeXNbd3a0lS5Zo7ty5io+P/65jIowcP35chYWF2rp160CPMuCKioo0f/583XzzzQM9SsgQqO/ozTff/NrbRowYoaamJsXGxqqpqUnDhw8PWuN0OvXJJ5/4j30+n+6//34dPnxYR48elcvlUnd3t1paWjR37ly9/fbbfXEZIdVXe3LZ8uXLNXbsWOXk5IRy7H7jdDrV2NjoP/b5fP63qb665vTp04qLi1N3d7fOnj2rmJiYq7rvtcbOfkhSY2OjFi5cqDVr1uj222/v19n7ip09+fTTT1VRUaHCwkJ1dHRo0KBBioqK0qOPPtrflxEyvMXXB1wul3bt2iVJ2rVrl1JTU4PWpKSkqLq6Wu3t7Wpvb1d1dbVSUlL0yCOPqLq6WlVVVdqxY4fGjh17TcTpSuzsiSStXbtW586d09KlS/tz7JBKSkrSyZMnVV9fr66uLpWXl8vlcgWscblc/k8vVlRUKDk5WQ6HQy6XS+Xl5erq6lJ9fb1Onjypu+++eyAuI2Ts7EdHR4fy8vL0zDPP9PoVOdcqO3uyY8cOVVVVqaqqSvPnz9eTTz55TcdJEp/i6wstLS3WvHnzLLfbbc2fP99qbW21LMuy6urqrKVLl/rXlZSUWFOnTrWmTp1q/fGPfwx6nPr6+rD5FJ+dPTl9+rR15513WtOmTbOmT59uTZ8+3dq5c+dAXIZtH374oeXxeKzU1FTr1VdftSzLstatW2ft37/fsizLunjxorVo0SJr6tSp1uzZs61Tp0757/vqq69aqamplsfjuSY/xdib77ofr7zyinXPPff4/z1Mnz7dOnPmzIBdRyjZ+Tdy2YYNG8LiU3x8Fx8AwEi8xQcAMBKBAgAYiUABAIxEoAAARiJQAAAjESgAgJEIFADASP8fKaugYaDoLMAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(512, activation='relu', input_shape=input_shape),\n",
    "    layers.Dense(512, activation='relu'),    \n",
    "    layers.Dense(512, activation='relu'),\n",
    "    layers.Dense(1),\n",
    "])\n",
    "model.compile(\n",
    "    optimizer='sgd', # SGD is more sensitive to differences of scale\n",
    "    loss='mae',\n",
    "    metrics=['mae'],\n",
    ")\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data=(X_valid, y_valid),\n",
    "    batch_size=64,\n",
    "    epochs=100,\n",
    "    verbose=0,\n",
    ")\n",
    "\n",
    "history_df = pd.DataFrame(history.history)\n",
    "history_df.loc[0:, ['loss', 'val_loss']].plot()\n",
    "print((\"Minimum Validation Loss: {:0.4f}\").format(history_df['val_loss'].min()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e135866",
   "metadata": {
    "papermill": {
     "duration": 0.020027,
     "end_time": "2021-12-29T13:00:32.741392",
     "exception": false,
     "start_time": "2021-12-29T13:00:32.721365",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Did you end up with a blank graph? Trying to train this network on this dataset will usually fail. Even when it does converge (due to a lucky weight initialization), it tends to converge to a very large number.\n",
    "\n",
    "# 3) Add Batch Normalization Layers\n",
    "\n",
    "Batch normalization can help correct problems like this.\n",
    "\n",
    "Add four `BatchNormalization` layers, one before each of the dense layers. (Remember to move the `input_shape` argument to the new first layer.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a748d669",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:32.792851Z",
     "iopub.status.busy": "2021-12-29T13:00:32.792047Z",
     "iopub.status.idle": "2021-12-29T13:00:32.800284Z",
     "shell.execute_reply": "2021-12-29T13:00:32.800687Z",
     "shell.execute_reply.started": "2021-12-29T12:57:18.757411Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 0.040097,
     "end_time": "2021-12-29T13:00:32.800809",
     "exception": false,
     "start_time": "2021-12-29T13:00:32.760712",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"3_Q3\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# YOUR CODE HERE: Add a BatchNormalization layer before each Dense layer\n",
    "model = keras.Sequential([\n",
    "    layers.BatchNormalization(),\n",
    "    layers.Dense(512, activation='relu', input_shape=input_shape),\n",
    "    layers.BatchNormalization(),\n",
    "    layers.Dense(512, activation='relu'),\n",
    "    layers.BatchNormalization(),\n",
    "    layers.Dense(512, activation='relu'),\n",
    "    layers.BatchNormalization(),\n",
    "    layers.Dense(1),\n",
    "])\n",
    "\n",
    "# Check your answer\n",
    "q_3.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "018fefd0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:32.846592Z",
     "iopub.status.busy": "2021-12-29T13:00:32.844978Z",
     "iopub.status.idle": "2021-12-29T13:00:32.847216Z",
     "shell.execute_reply": "2021-12-29T13:00:32.847618Z",
     "shell.execute_reply.started": "2021-12-29T12:57:20.999989Z"
    },
    "papermill": {
     "duration": 0.0257,
     "end_time": "2021-12-29T13:00:32.847733",
     "exception": false,
     "start_time": "2021-12-29T13:00:32.822033",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#q_3.hint()\n",
    "#q_3.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c393690a",
   "metadata": {
    "papermill": {
     "duration": 0.01981,
     "end_time": "2021-12-29T13:00:32.888235",
     "exception": false,
     "start_time": "2021-12-29T13:00:32.868425",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Run the next cell to see if batch normalization will let us train the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8ebaa748",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:32.936379Z",
     "iopub.status.busy": "2021-12-29T13:00:32.935570Z",
     "iopub.status.idle": "2021-12-29T13:00:41.265111Z",
     "shell.execute_reply": "2021-12-29T13:00:41.265660Z",
     "shell.execute_reply.started": "2021-12-29T12:57:28.244660Z"
    },
    "papermill": {
     "duration": 8.357592,
     "end_time": "2021-12-29T13:00:41.265823",
     "exception": false,
     "start_time": "2021-12-29T13:00:32.908231",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Validation Loss: 3.8497\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAABJIUlEQVR4nO3dd3hUVfrA8e+dmTQySSBlElIgDUjoCAihiHSkSFFEFF0QF0VXLGsB/Omqu4ptFcWKZUVFsSEoRRBEeg0gLZRAgARIQhJIrzPz++NAICSkh5mQ9/M8PnFuZu68c7zOm3POe87VrFarFSGEEMLO6GwdgBBCCFEWSVBCCCHskiQoIYQQdkkSlBBCCLskCUoIIYRdMtTlyaOjo+vy9EIIIa4TnTt3LnWsThPU1d60qmJiYoiMjKyFaK5f0kblk/apmLRR+aR9yleT9rlaZ0aG+IQQQtglSVBCCCHskiQoIYQQdkkSlBBCCLskCUoIIYRdkgQlhBDCLkmCEkIIYZckQQkhhLBLkqCEEKKe69Spk61DqBN2n6COJGUyY+UZdhxPs3UoQgghrqEKtzrKz8/n7rvvpqCgALPZzODBg5k2bRrTp09n27ZtuLm5AfDqq6/WyTYgHi4OJGYWcsfHm3m0f0se7huGQW/3eVUIIa45q9XK66+/zvr169E0jalTpzJ06FCSk5N5/PHHycrKwmw288ILL9CpUyeeffZZ9u3bh6Zp3HbbbUycONHWH6GEChOUo6Mj8+bNw9XVlcLCQu666y5uuukmAJ5++mmGDBlSpwGa3J15b0Qg3xws5O1Vh9kQe5a3x3UksEmjOn1fIYSoqp+iE/h+R3ytnvOOLkHc1jmwUs9duXIlBw8eZPHixZw7d47bb7+dLl26sGTJEnr16sXUqVMxm83k5uYSExNDUlISS5YsASAjI6NW464NFXZFNE3D1dUVgKKiIoqKitA0rc4Du5yro463xnVk9riOxJzJZNi7G1gdk3RNYxBCCHsXHR3NsGHD0Ov1eHt707VrV/bu3Uu7du1YuHAhc+bM4fDhwxiNRoKCgoiPj+ff//4369atw2g02jr8Uiq1m7nZbGbMmDGcPHmSu+66iw4dOvDtt9/y9ttv8/777xMVFcWTTz6Jo6NjnQY7qlMAnZo15qH5O5k8bwcP9AnlyUGtcJAhPyGEHbitc2ClezvXUteuXfn6669Zu3Yt06dPZ9KkSYwaNYrFixezYcMGFixYwPLly5k1a5atQy1Bs1qt1so+OSMjg4cffpjnnnuOxo0b4+PjQ2FhIc899xxBQUH84x//KPH86OhoGjWq+VBcXl4ezs7OxY8LzBbmbk9j6aEM2picmdnHhGejOr9ziF27so1ESdI+FZM2Kp89t8+dd97JggUL2Lx5MytWrOC5554jKyuLJ598ktdff53CwkK8vLzQ6/UsXbqUM2fOcMcdd2AwGGjUqBEnTpzg7bffZvbs2dWOoSbtk5OTU/P7Qbm7u9OtWzfWr1/P5MmTATVHNWbMGD7//PMyX1MbhRNl3Wfk/bYwaPcpZizcyxMrkvjk3i60D2xc4/eqr+ReNeWT9qmYtFH57Ll9dDodkZGRREREcPbsWaZPn46macycOZMePXrw888/88YbbxQnpNdee43s7GxmzJiBxWIBYObMmTX6fHVxP6gKE1RaWhoGgwF3d3fy8vLYtGkTf//730lOTsZkMmG1Wlm1ahUtWrSoVmA1MbJjAC193bh/3g7GfrSZ129vz8iOAdc8DiGEsKVdu3YBqmbgmWee4Zlnninx+9GjRzN69OhSr/v555+vSXzVVWGCSk5OZvr06ZjNZqxWK0OGDKFv377ce++9nDt3DqvVSkREBC+++OK1iLeUyKbu/PKPnkz9eiePLtjNkaQs/jmo5TUv5BBCCFG7KkxQERERLFq0qNTxL7/8si7iqRYvoxNf39+N5xfv4701sZzNzOeVMe3Q6yRJCSFEfVV/KguykuHYWohbCwZnaDMKmkWBTg+Ao0HHrDHtMLk58e4fsWTkFTL7zo44GfS2jVsIIUS12H+CSjtGyIp74PwR9di5MRTlw/ZPwOgHbUZD1MPQOAhN03hiUCvcXRz4z9IYsubt4KMJnXF1sv+PKYQQoiT7/+bWdOS7h+Dc5W4IvRn82kNRHhz+DfYthB2fQ/QX0OcpiPoHGJy4v3co7i4OTP9pD/d9sZ15992Is4P0pIQQoj6x/xWuTYI5HfUS9Hoc/DupIT1HV2h7G9w5Hx7ZAS0GwOqX4MMecHQNoLYHeXtcR7YdT+Oh+TspNFts/EGEEEJUhf0nqIo0bgbjvoa7fwKrBb4aBb//C8xFjOwYwL9HtuWPg8n88/u/MFsqvSZZCCGEjdX/BHVRiwEwdTN0ngQbZ8OXt0JmIhO6N+fpIa345a/TPL94H1XYOEMIIa5L5d0/KiEhgeHDh1/DaK7u+klQAA7OMGI2jP4YTu+Cj3rB8Y08dHM4D/YJY/7Wk3y6Ps7WUQohhKgE+y+SqI4Od0LTDvDdBDXkd9unPDPkVk6kZjNreQxt/N3pEe5t6yiFENeb3d/Crq9r95ydJkDH8eU+5c0336Rp06bcfffdAMyZMwe9Xs/WrVvJyMigqKiIRx99lAEDBlTprfPz83nhhRfYt28fer2e6dOn0717d44cOcKMGTMoLCzEYrEwZ84c8vLymDJlComJiVgsFh566CGGDh1a7Y8N12uCAjBFwuTf4Ztx8P3f0Ia9yRtjJxKbnMXD3+zk10d6yT2lhBDXhaFDh/LKK68UJ6jly5fz2Wefce+992I0GklLS2PcuHH079+/SrvszJ8/H4Bff/2Vo0ePMnnyZFasWMGCBQu49957ufXWWykoKMBisbB69WpMJhNz584FIDMzs8af6/pNUACNPOHexfDjJFj6T4yZSXw84RFGvr+ZB7+O5scHe0j5uRCi9nQcX2Fvpy60bt2a1NRUkpKSOHfuHO7u7nh7ezNr1iy2b9+OTqcjKSmJlJQUfHx8Kn3e6OhoJkyYAEBYWBj+/v7ExcXRsWNHPvroIxITExk0aBDBwcE0b96c+fPn88Ybb9C3b1+6dOlS4891fc1BlcWxEYybr7rJ614ndPndfDTCm32nMnh+8T5bRyeEELViyJAhrFixgmXLljF06FB+/fVX0tLSWLhwIYsXL8bb25v8/Pxaea8RI0bw4Ycf4uzszJQpU9i8eTMBAQEsXLiQli1bMnv2bN57770av8/1n6AA9Aa49T0YPhtO7aTnylv5JPIvvt8Rz6oDcmdeIUT9N3ToUJYtW8aKFSsYMmQImZmZeHl54eDgwJYtWzh16lSVz9mlSxd+/fVXAOLi4jhz5gyhoaHEx8cTFBTEvffeS//+/Tl06BBpaWm4uLgwcuRIJk+ezIEDB2r8ma7vIb7LaRp0mQTh/eGXaQw89hrz3brzz4VP0DW4Hx6NHGwdoRBCVFuLFi3Izs7GZDJhMpkYMWIEU6dOZcSIEbRt25bQ0NAqn/Ouu+7ihRdeYMSIEej1embNmoWjoyPLly9n8eLFGAwGvL29eeCBB1i2bBmvvvoqOp0Og8HACy+8UOPPVKU76lZVdHR0mXdJrKpav1GY1Qob34FV/+KtorGcav8I/72jQ+2d3wbs+WZq9kDap2LSRuWT9ilfTW9YWFauaBhDfFfSNOj5KLQby2OGn0jY9Tt/HJShPiGEsCcNZ4jvSpoGw9+GU7v44Nz7TPgphM5P3IqHiwz1CSGuf4cOHeLpp58ucczR0ZEffvjBRhGV1nATFICTG7o75tHkk37MzH+b2Stb8q+R7WwdlRBC1LlWrVqxePFiW4dRroY5xHc5v7bohr5Ob91eHHd8yInUbFtHJIQQAklQyg1/Iz9kANN0P/LJkg22jkYIIQSSoBRNw2nEmzjpoXvsW+yOP2/riIQQosGTBHWRZwjmno8zXL+FJT/Pl9tyCCGEjUmCuozTTY+T4RLE+JQ5/HkgwdbhCCFEgyYJ6nIOzjQa9V/CdGc49strcgdeIYSwIUlQVzC0GsyZpgO4K+871u7cb+twhBCiwZIEVQaf0a/iqBWRvuYdW4cihBANliSoMhhMLTjhO4ABWb+yN/aErcMRQogGSRLUVfgNnYGblkvcculFCSGELVSYoPLz87n99tu59dZbGTZsGO+++y4A8fHxjB07loEDB/LYY49RUFBQ58FeS42a38BRjyh6pvzAqeQUW4cjhBANToUJytHRkXnz5vHLL7+waNEi1q9fz+7du3nzzTeZOHEiv//+O+7u7vz444/XIt5rymPQM3hpGez99X1bhyKEEA1OhQlK0zRcXV0BKCoqoqioCE3T2LJlC4MHDwZg9OjRrF69um4jtQHvNn056tKO9ie/JCM7x9bhCCFEg1Kp3czNZjNjxozh5MmT3HXXXQQFBeHu7o7BoF7u5+dHUlLZ91OKiYmpcZB5eXm1cp7qSG81ge67n+GXb96kRa+xNomhMmzZRvWBtE/FpI3KJ+1Tvrpon0olKL1ez+LFi8nIyODhhx/m2LFjlX6D2rgDpU3vZBkRwfF9H9P+9Hc0b/Usmk5vmzgqIHf7LJ+0T8Wkjcon7VO+mt5RtyxVquJzd3enW7du7N69m4yMDIqKigBITEzE19e3WoHZPU0jse3fCbYmELd5ka2jEUKIBqPCBJWWlkZGRgagunCbNm0iLCyMbt26sWLFCgB+/vln+vXrV7eR2lDrgRM5Y/VE2zzH1qEIIUSDUeEQX3JyMtOnT8dsNmO1WhkyZAh9+/YlPDycxx9/nNmzZxMZGcnYsfY7P1NT7q6NWOUzjjEpH1J4MhqHZp1tHZIQQlz3KkxQERERLFq0qNTxoKCg67K0/Gq8+/ydjB+/IGflG/jdv8DW4QghxHVPdpKopB6tQ1ikG4QpYQWcO27rcIQQ4ronCaqSDHodqW3vw2zVyF//nq3DEUKI654kqCoY2L0Tv1h6oP/rK8hJs3U4QghxXZMEVQVt/N1Z4TEWgzkPdn5p63CEEOK6JgmqCjRNo1OXXmw2t6Zo6ydgMds6JCGEuG5JgqqiWzv6M888CENmAhxeYetwhBDiuiUJqooCGrtw0udmUnXesO1jW4cjhBDXLUlQ1dAnsilfFPSDY3/C2UO2DkcIIa5LkqCqoW8rE98U9cWsc4Dtn9o6HCGEuC5JgqqGG5o1psjFm11u/WD3N5CXYeuQhBDiuiMJqhoMeh03tfRhTlZfKMiCPd/ZOiQhhLjuSIKqpr6tfFib3Ywcnw6w/TNbhyOEENcdSVDV1KelD5oGO4x94WwMZJyxdUhCCHFdkQRVTV5GJzoENmbxuWB14ORmm8YjhBDXG0lQNdAvwsQvSZ5YDY3g5BZbhyOEENcVSVA10C/CRKHVwNnG7aUHJYQQtUwSVA20buqOj5sT2y2tIGmflJsLIUQtkgRVAzqdxs0tfViUFgRWCyRss3VIQghx3ZAEVUNRYV5szAvFqullHkoIIWqRJKga6hrsSQ7OpLpFwAmZhxJCiNoiCaqGApu40NTDmT26SDi1A4oKbB2SEEJcFyRB1ZCmaXQN9mRFVggU5cGZv2wdkhBCXBckQdWCriGerM4KVQ+k3FwIIWqFJKhacGOwJyl4kOnaXBKUEELUEklQtaCFyYiHiwMHHdqqSj6LxdYhCSFEvScJqhbodBpdg5uwJjcUctMg9YitQxJCiHqvwgR15swZ7rnnHoYOHcqwYcOYN28eAHPmzKF3796MHDmSkSNHsnbt2joP1p51DfZkWUaweiDDfEIIUWOGip6g1+uZPn06bdq0ISsri9tuu42ePXsCMHHiRCZPnlznQdYHXUM8mWX1I9/JE6f47dB5oq1DEkKIeq3CBGUymTCZTAAYjUZCQ0NJSkqq88Dqm7b+Hjg76Il3akF4opSaCyFETVWYoC6XkJBATEwMHTp0YOfOncyfP59FixbRtm1bpk+fjoeHR6nXxMTE1DjIvLy8WjlPXWvl5ci2LH/CMn/h0L6/sOodr9l715c2shVpn4pJG5VP2qd8ddE+lU5Q2dnZTJs2jZkzZ2I0Ghk/fjwPPfQQmqbxzjvv8OqrrzJr1qxSr4uMjKxxkDExMbVynrrWJ0HPxj+bcZeDmQhPC/hfu5jrSxvZirRPxaSNyiftU76atE90dHSZxytVxVdYWMi0adMYMWIEgwYNAsDb2xu9Xo9Op2Ps2LHs3bu3WoFdT24M9mSPJVg9kB0lhBCiRipMUFarlWeffZbQ0FAmTZpUfDw5Obn431etWkWLFi3qJsJ6pFOzxpzWfMnTGyVBCSFEDVU4xBcdHc3ixYtp2bIlI0eOBOCJJ55gyZIlHDx4EICAgABeeumluo20HnB1MtDC5EZcbjiRkqCEEKJGKkxQXbp04dChQ6WO9+nTp04Cqu9a+7uz82AzIhNXgLkQ9A62DkkIIeol2UmilrVu6s7WvCAw50PKYVuHI4QQ9ZYkqFrWxt+D/dZg9UCG+YQQotokQdWy1k3dibM2pVDvIglKCCFqQBJULfNo5EDTxq6cdAyTBCWEEDUgCaoOtPF3Z09Rc0jcK7feEEKIapIEVQda+7uzKScACrIg7ZitwxFCiHpJElQdaN3UnX2WEPXgzG6bxiKEEPWVJKg60NrfnSPWAMyag8xDCSFENUmCqgMBjV1wdXHhjLMUSgghRHVJgqoDmqapYT5riEpQVqutQxJCiHpHElQdae3vzqbsAMg7D+dP2jocIYSodyRB1ZE2/u7sKwpUD84etG0wQghRD0mCqiOt/d2JtfqrB5KghBCiyiRB1ZEwHyN5BneyHLzgrGwaK4QQVSUJqo446HW08nXjhC5QelBCCFENkqDqUOum7uwv8MOaclgq+YQQoookQdWh1v7u7C1oipafAZmJtg5HCCHqFUlQdaiVnxux1gD1QIb5hBCiSiRB1aFwk5FYy4UEJXfXFUKIKpEEVYe8XB0pdPEmV2+UHpQQQlSRJKg6pGkaLXzdOKFrJqXmQghRRZKg6li4yUhMUVPpQQkhRBVJgqpjYT5G9hf4QU4KZKfaOhwhhKg3JEHVsXCT8VIlX8oh2wYjhBD1iCSoOlYiQZ2VBCWEEJUlCaqO+Xu4kGYwUaBzlgQlhBBVIAmqjul0GqEmN07pA2WITwghqqDCBHXmzBnuuecehg4dyrBhw5g3bx4A58+fZ9KkSQwaNIhJkyaRnp5e58HWVy1MbhwyB0gPSgghqqDCBKXX65k+fTrLli3ju+++45tvviE2Npa5c+cSFRXFypUriYqKYu7cudci3nop3GRkT74vZJyCvAxbhyOEEPVChQnKZDLRpk0bAIxGI6GhoSQlJbF69WpGjRoFwKhRo1i1alWdBlqfhfkYOVpcyXfEtsEIIUQ9YajKkxMSEoiJiaFDhw6kpqZiMpkA8PHxITW17DU+MTExNQ4yLy+vVs5jK7qsAo5cSFCn96whPbNRrb9HfW+juibtUzFpo/JJ+5SvLtqn0gkqOzubadOmMXPmTIxGY4nfaZqGpmllvi4yMrJmEaKSXG2cx1bCzRYe+eUkZs2Av0MG/nXwWep7G9U1aZ+KSRuVT9qnfDVpn+jo6DKPV6qKr7CwkGnTpjFixAgGDRoEgJeXF8nJyQAkJyfj6elZrcAaAge9jiBvdxINgbInnxBCVFKFCcpqtfLss88SGhrKpEmTio/369ePRYsWAbBo0SL69+9fZ0FeD1qYjBy2BkDSfluHIoQQ9UKFCSo6OprFixezZcsWRo4cyciRI1m7di1Tpkxh48aNDBo0iE2bNjFlypRrEW+9FW4ysj0vCNJPQk6arcMRQgi7V+EcVJcuXTh0qOz1OxfXRImKhZuM/GRprh4k7oXQPrYNSAgh7JzsJHGNhPkY2W8JVg8S99g0FiGEqA8kQV0jYT5G0jQPshxNcEYSlBBCVEQS1DXi4qgnoLELcQ5hcOYvW4cjhBB2TxLUNRRuMrLX3BxSj0BBjq3DEUIIuyYJ6hoK8XZlc04gWC1Sbi6EEBWQBHUNhXi7srMgSD1IlGE+IYQojySoayjYy5VTeFPk6CHzUEIIUQFJUNdQiLcroJHqFiGVfEIIUQFJUNeQf2MXHPU6jjuGQ/IBMBfaOiQhhLBbkqCuIb1OI8jThX3mYDAXyB12hRCiHJKgrrEQb1e25F64eaHMQwkhxFVJgrrGgr1c2XCuMVaHRrLlkRBClEMS1DUW7O1KbhEUereWQgkhhCiHJKhrTFXyoSr5EveCxWLjiIQQwj5JgrrGgi8kqBOOYVCQCefibByREELYJ0lQ11hTd2ecDLpLt96QQgkhhCiTJKhrTKfTaO7ViO05fqAzqGE+IYQQpUiCsoFgL1di0wrBu5UkKCGEuApJUDYQ4u3KydQcLH5tJUEJIcRVSIKygWBvVwrMFtLdIyArEbLO2jokIYSwO5KgbCDYS1XyxTuFqwNJ0osSQogrSYKygYtroWIszdQBGeYTQohSJEHZgK+7Ey4Oeg5lOIJ7oCQoIYQogyQoG9A0VWoel5IFfu0kQQkhRBkkQdlIiLcrx1NzVIJKOQyFubYOSQgh7IokKBsJ9nYlPi0Hs6ktWC3qBoZCCCGKVZigZsyYQVRUFMOHDy8+NmfOHHr37s3IkSMZOXIka9eurdMgr0ch3q4UWayccWmhDsgwnxBClGCo6AljxoxhwoQJPPPMMyWOT5w4kcmTJ9dZYNe7i5V8Rwo9CXRylwQlhBBXqLAH1bVrVzw8PK5FLA3KxbVQcSm54Cs7SgghxJUq7EFdzfz581m0aBFt27Zl+vTpV01iMTEx1Q7uory8vFo5jz2xWq0YHXXsOBzPKKcAGsct4dCB/aBVb1rwemyj2iTtUzFpo/JJ+5SvLtqnWglq/PjxPPTQQ2iaxjvvvMOrr77KrFmzynxuZGRkjQIEleRq4zz2pqXfOVILdXh27ANHfiDS1xm8wqp1ruu1jWqLtE/FpI3KJ+1Tvpq0T3R0dJnHq/Xnure3N3q9Hp1Ox9ixY9m7V4anqiPcZOTo2QtroUCG+YQQ4jLVSlDJycnF/75q1SpatGhRawE1JOEmIylZBZw3hsq9oYQQ4goVDvE98cQTbNu2jXPnznHTTTfxyCOPsG3bNg4ePAhAQEAAL730Up0Hej0KNxkBiE0roovcG0oIIUqoMEG99dZbpY6NHTu2ToJpaMJ93ACITc6ii19biFtv44iEEMJ+yE4SNhTQxAUng47Y5CwI6AKZpyFpv63DEkIIuyAJyob0Oo1QHyOxZ7Og7W2gd4ToebYOSwgh7IIkKBsLNxlVD8rVCyJvhT0LoCDH1mEJIYTNSYKysTAfV06dzyW3wAydJ0JeOhxYbOuwhBDC5iRB2Vi4yYjVCsdSsiC4F3iFQ/T/bB2WEELYnCQoGysuNU/OAk1Tvaj4rZAkt98QQjRskqBsLMTbFZ0GR5Oz1IEOd6liiZ1SLCGEaNgkQdmYk0FPM89GqpIPLhVL/PWt3GVXCNGgSYKyA8WVfBddLJbYv8hWIQkhhM1JgrIDYSYjcSnZFJkt6kBwL2jcDGJ+tW1gQghhQ5Kg7EC4j5FCs5WTaRfWP2ma2lkiaZ9tAxNCCBuSBGUHwi6v5LvItw2cPwF5GTaKSgghbEsSlB24WGp+9Gz2pYMX7xEle/MJIRooSVB2wN3ZAZOb0xU9qLbqpwzzCSEaKElQdiLcZLxUag7g7g8uTeQeUUKIBksSlJ0INxk5mpyF1WpVBzRN9aJkiE8I0UBJgrITrZu6k5VfVHIeyrctJB8Ai9l2gQkhhI1IgrITN4Z4ArA1LvXSQb+2UJgDaXE2ikoIIWxHEpSdCPF2xcfNia3H0i4dLC6UqME8VF66lKoLIeolSVB2QtM0uoV4sjUu9dI8lE8EaPqazUN9fy98ORIunlMIIeoJSVB2pHuoF0kZ+ZxIvbCjhIMzeLeAxGqWmpuL4ORWOL0Tjq6uvUCFEOIakARlR7qHljEP5du2+muhzh6Eogs7oq9/u4bRCSHEtSUJyo6E+RjxNjqWnIfyawvp8ZB7ruonPL1T/ex6P5zYoHpTQghRT0iCsiOapnFjiCdb48oqlKjGPNTpXeDkAQNeBBdP2PBW7QQqhBDXgCQoO9MtxItT53OJv7izeU0S1Kmd4N8RnIzQfSoc/q3681lCCHGNSYKyM92K56Eu9KLc/KCRV9W3PCrKV0kt4Ab1+Ma/g6MRNshclBCifqgwQc2YMYOoqCiGDx9efOz8+fNMmjSJQYMGMWnSJNLT0+s0yIakpcmNxo0c2HLsQqFE8ZZHVez5JO0DSyH4d1KPXZpA18mwfyGkHavdoIUQog5UmKDGjBnDp59+WuLY3LlziYqKYuXKlURFRTF37tw6C7Ch0ek0bgz2LF3JlxyjysYr69SFAgn/Gy4d6/p3sFrg8MraCVYIIepQhQmqa9eueHh4lDi2evVqRo0aBcCoUaNYtWpVnQTXUHUL9SI+LZfT5y+UiPu1haI8SNhW+ZOc3gWuPuAReOmYRyA4N1bl50IIYeeqNQeVmpqKyWQCwMfHh9TU1ApeIaqi25X78oX0UYnli+Gw7GnISbv6iy86tVMN72napWOaBqZISVBCiHrBUNMTaJqGdvmX4BViYmJq+hbk5eXVynnqDYsVo6OOFTuPEeGcCYB+8Lf47PuExts/wbL7W5LbP8z5sJHFL7m8jbTCHFqlHCLF1IOUK9rNz8EP9/jVHD5woGTyus41uGuoGqSNyiftU766aJ9qJSgvLy+Sk5MxmUwkJyfj6el51edGRkZWO7iLYmJiauU89Un3sGwOnc0u+bk79YCkJ9Evf4amO2bR1DEbBr4EmlayjU5sAqsFnw6D8Wl5RbtlRMHRn4kM8lQVgg1EQ7yGqkraqHzSPuWrSftER0eXebxaQ3z9+vVj0aJFACxatIj+/ftXKyhxdd1CvIhLySYpI6/kL3zbwL2LVcHDpndh0UNgLiz5nOICiU6lT+wToX4my1+CQgj7VmGCeuKJJ7jzzjuJi4vjpptu4ocffmDKlCls3LiRQYMGsWnTJqZMmXItYm1QLq6HKi43v5xOD0PfgL7Pwl/fwIK70YouS2Snd4F7IBhNpV9ruvAXjsxDCSHsXIVDfG+9Vfb2OPPmzav1YMQlrZu64+ZkYGtcGiM7BpR+gqZBn6fB1RuW/pPgpCNg+lr1sE7vhIAyek+gKvtcPKUHJYSwe7KThJ0y6HV0CW5Sdg/qcl3ugwk/YSg4D3P7wvq31ELcsob3QCr5hBD1hiQoO9Yt1ItjZ7NJzswr/4lh/Tg2+GsI7QOrX1THLl+geyWfCEg+KDcxFELYNUlQdqx7qBcA2+IqXvdkdvaEu76HIa9B6M0Q2PXqTzZFQn46ZJ6ppUiFEKL2SYKyY2393XF11Fc8zHeRpkH3B1WVn5Px6s+TSj4hRD0gCcqOGfQ6Ogd7lryBYW0oruQ7VLvnFUKIWiQJys51D/XkSHIWKVn5tXdSV291C4+z0oMSQtgvSVB2rltI5eehqsQnUhVK2LOjayBmia2jEELYiCQoO9c+0AMXBz1bKzsPVVmmCFVqXp1KvsI82PIRpMTWbkxXWjETlj0p1YZCNFCSoOycQ/F6qNruQUVAfgZknC7793npqgdTkF3yeEosfDYAfnsGPhsICWXvoVVpFnPZCSgrGZIPqErD8ydr9h5CiHpJElQ90C3Ek0NJmaRlF9TeSYsLJS6bh7Ja4eRWtb/ffyPgq1Hq57KnVMXfX9/B3D6QngAj3gFnd5g3QiWy6jj0G7wRDuv/W/p3x9df+vf4rdU7vxCiXpMEVQ/0DPcGYNneWly35HMhQV2chzp/Ej7tD58PggOLod1YuONLaDkYor+AD7rDz1PArx08uAE6T4T7VkCTYJg/Fvb/XPn3NhfBqhfh23GQmwa755fuRcWtB0c39c/JLbXwgYUQ9U2N7wcl6l7HoMZ0CPRg7rpj3Nk1CIO+Fv6ucPVS+/KdjYG4dfDDRJU4hr8N7e64tI6q9UgY8ir89S3oHaHLZNBfuGzc/GDSUvjmTvjxPvBqoe7+ezlzEez9QQ0nOriAQyOV8I6vhxv+poYaV8xQJe+miEuvi1sHwT2hKF96UEI0UNKDqgc0TWPqzeGcTMth+b7E2juxTwQcXApfjoJG3jBljdrb78pFvq7e0OMR6PbApeR0kUsTGP8tOLnDymdL94Q2vweLHoTlT8Mvj8BPkyFhO4z6EG59F9qMVs87eFm1XnoCpB2FkJugWXdI2q/mxIQQDYokqHpiUGtfQn1c+fDPo1hrq6rN1Bpyz0HLIXD/KvAKq955GnnCzdPh2J9w5PdLx9OOwZ+zoNUweOoYPLYPHt6mfna8Sz3HvSkEdCmZoOIuzD+F3ARB3QArJOyoXmxCiHpLElQ9odNpPNgnjANnMlh7+GztnLTX43D75zDua1XwUBNdJoNnmOpFmQtVT2rJ46BzgGFvqiHFxkHg0wqMPiVfGzFM3cMqPUE9Pr5e3RLE1AYCu4Cmk2E+IRogSVD1yKiOATT1cObDP4/Wzgndm0Lb20BXC5eBwREG/RtSDqs5pr8WqB7VgH+Bu3/5r40Yrn4eWq4SW9w6CO6l4nJyU/e4kkIJIRocSVD1iKNBx/29Q9kal0b0iXO2Dqe0VkMhuLca1lsxUw3PdZlc8et8WqoCi4NL4FwcpMer4b2LgrqrIT5zUd3FXlOFebDtEyiqxaUAQjRwkqDqmTu7BtG4kQNv/X6IvEKzrcMpSdNg0H8gJw3yM9Vaqcr2ziKGwfENEPOrehzS59LvmnWHwmxI2lf7MZenqAD+fBUSK/G+BxarXS+qUm4vhCiXJKh6xtXJwD8HtmRjbCq3f7SJE6nZFb/oWvLvqMrSR75/aTFwZUQMB0uRWrRr9APvFpd+F9RN/bzW81B7v1e9wc8Gwt4fy3/uyc3q54FFdR6WEA2FJKh66J6oYD69twvxabkMf3cDv+2zsxsPdn8QOoyr2msCOoPRV5WTh9ykemMXNQ4C94BrOw9ltcLm91UpftMOqjx+5f9dfZjxYmyxq6QkXohaIgmqnhrQ2pel03oRajLy4Nc7+V90GhZLPd5UVadTc1hQcv7poqBu17YHdXS12guw56Nw7y/Q9e+waQ4suKv0Wq/cc2rBc4vBYC5QxR61JTsVTu2svfMJUY9IgqrHAps04ocHohh/YzO+33eex7/fTX6Rnc1LVUWne6BJCLQYWPp3zbpDxinVU9n1NSycAp8OgJ/uV/NEe39U817VkV3GTvGb3lNDjW1vVxWKw96Em2fCkRUqcV0ufpv62eMf4B4I+xdVL44rFeXD16Phi2GqCKMqrFbY84OaD6xI6lH4qDeOGSeqF6cQdUQSVD3naNDxyui2TLrBk8W7T/O3z7eRnlto67CqJ7AzPLpbbaF0pYvzUJ8PhsUPw9E/wOCselV/vqqG4H6YVPX3jP4C3ghVPy9K3AfH1kC3KSo5XdT5b+rnwWUlz3FyM+gMasFxm1Gq95V7vuqxXGnVi3DmLyjMqXrvMWk/LLy/7I14r7T2dUjcg8eJWuz5CVELJEFdBzRN4452jZk9riPRJ84x9qNNxKXYWfFETfm1hz7PwOBZMHUTPHkEJi6Bx/bCs4lq0XHs71W7R5XFDBtmq4XAvz56KUltfl/tGdj5ioTn5qeS0KGlJY+f3AJNO4JjI2g9qurDfPHbS8+vHfoNtrwPHe8GTQ9xayt/PoAjK9XPvT+qz3k1qUdVMQhgPLX+6s8TwgYkQV1HRnUKYN59N5Kcmc+IORtYXondz+vNvJVOB31nQtRDauHu5UUUDs7QbaratWLHZ5U/56Hlat3VqI+gxSCVpNa9oTa37TRBbeF0pYihateLi/fRKspXc0TNuqvHgV3AI6jy1XyJe2HecNUz/HY8nD2szr1oKvi2g2FvQcANcKyKCSp2lWqPrMSSty650oa31CbAPR/DOf0opMWV/L3Fom6/suPzqr2/qJ8K8+zqBqGSoK4zPcK8WTqtN2EmI1Pn7+SlXw9QUGQp9bz4tBz+9vk2er32B8mZpec3iswWNh1NwVxfEpibL7S+FXbNL32TxavZ/B40bqZ207jjK5Wk/viPKnfvPrXs17Qapn4eujDMd3o3mPOhWZR6rGlqB/jYSgzz5Z6H7+5RG+7ePFPtQfhBd5WsivJh7P9U8g3pA6d3Vr46MPe86pF1e0DdrmTP92U/79xxteNH50nq9ilQuud3dLW6HcqSx2HZ0+X3xuxJXrpaOP3pADVnWZeyU9VQbH1XkA3vdVXD5XaSpCRBXYcCGrvwwwNRTOwRzOcb4xj49lre/v0wx85mYbZY+WJjHINnr2PH8TRSswt46oc9pTag/dcv+7nrk61M+XIH2fl2vIPD5br+HfLTVQ+oIgnRau6o21S1Q7uDs0pS7e6A7g+BZ2jZr/NppX53cR7q4vqniz0oUDu0WwrLH+azWODnB9WuGWPnwc3PwLRd0GUSZCbC8LcurQUL7QNWC5zYVPIcJ7fC7Halez3H/gSrGSJHqKR94BcozC0dw4a31fBmz2ngGUKeR9ilxHvRtk/A1aTaZNvHqooxP0v9zmqFrLNln9tWMk6rXfP/G6EWTp/ZA+vfqtsv3IX3w8d91HyfnXyxV8v2zyD9JOz7Sf13twM1uh9Uv379cHV1RafTodfrWbhwYW3FJWrI0aDjhVvb0DPcm/9tjOPdP47wzuoj+Lg5cTYznz4tfXhlTDv+iEniucX7mbfpOBN7hgDw1ZYTzN96kl7h3qw5lMwdH2/ms791xc/D2cafqgLNuqsNZrd/qu41dfkw4JU2v6duEXLDPZeOOTjDbRX8j6lpqhx+68eQl6GKF7zC1S1JLgrorIb51r2hKv4aN4PGzaFJc/XvDi5qaO3wcrjldWh2oQDE6APD/gtDXit5W5PAG1VByLG10OqWS8c3vK1uNLn5fVVleNGR38HZQ82XFeWpHtDh3y7d2gTgfLzqbXb+W/FeiVn+vXA++LWq/GvkqXpYR1bCTU9Bv2fVbvfLnlK9PL2jqqosygPnxhD1MNw4BVwal99+lZGTpgo3OtypFn5Xxa+Pqfm69neonmHyAVVUk7ADgrrWPLYrndmjCnaahMDql9QNQG+do66l+qQgBza9q3rrDi5q0+egruDfyaZh1fiGhfPmzcPTs4yxemEXBrb2ZWBrX5Iy8vj1r9NsOprK8PZNGd0pAE3TmNC9OWsOneWV5QeJCvMmNTufF3/ZT78IE5/c24V1R87yj/k7GfX+RqbfEkFmfhFJ6Xmcyyngzq7NaBfoYeuPeImmwY33q+Go+G3qi99cCNFfYDq6E7weVTdFPH9SbU0U9ZDajLaqIoapBBf7uxpKixhaOo4+T8PGd1UiM+eX/L3RD7KTVQn7jVNKn//Ke245OKvke3mhRFqcSjqORpWA+s5UScVqVfNPYf3UeYJ7q/fb833JBLXuDfWz52PFhzIDbsI7Zp5KSh3uVH9RazrVqwPoej80DlZfZI081ed2D1Cb+655WZXmd39QndOxUVVa9JLEffDd3So5ntgAU9ZVfruslCNqGUCf6dB3hjrmFQ5Ln1Q33KyLBLXpXTWMOuVP9YfRH/9Wt5kZ97XajLm+2PE5ZJ+Fm2eoUYKPequbmD6wTv2xYyNyR90Gwtfdmft7h3J/75JDV5qm8dpt7Rkyex3/+GYnKVn5BHu78s6dHdHrNPq2MvHj1B5M/mI7j323GwCdpnpoi3ad4ov7bqRrsB39gdLuDvj9X7D9E7UuasVMSDmEp6aHQ9+oL26HC1+e3R6s3nsEdYNGXrDxHXXL+ovzT5e74V71j8WiktG5E3D+hPriPXcC9A5q38LyenmXC+kDq1+ErGQwmtSXoU6vhgfn3wbR/4Pe/1RFF1mJaj4N1HPa3a4SZU6a6u38/hzsnKeGNxsHFb9FnmekSmYHl6p5tF1fqWR8+W70LQaofy7Xfaqag1n7Oqx9TfVWxi8oWaJ/JYtFFZs4e6gYDE6q4vCXR9Sxno+q9t33o+oNVcbWj1TPrutlGxQ7u0PkcDVsNWSWep/acv4k7Fuo/tBxaQw3Pam+3Bc+AB/1VD2piGElX3NxDk+nr704ymO1QswvkHTg0h2tnd3V1mIXb0xakKPaOuQmaH7hWr79c/jfLbD4HzDqg+r9IVcLapygJk+ejKZpjBs3jnHjSm9vExMTU9O3IC8vr1bOcz2raRtN6+7Jv1YnYnTUMaNnExLiSpZrvzvUj1MZhXg10tPYWc+5XDMzVp5hwqdbeLGfHx2autT0I9Qa32ZD8Nz7Pez9gQJjIEm9Xue8sSW+p5bT5MhPOOSlkN5sEKdPZ8Lp6rVZU1N3Gh9X5eaxhT4UVtj27uDQDkztwHTh0LH4Sr+fs9aMEODUxgVk+veixY55ZAf04VRRAEG+N+K08QNimwzE69B8TMBhazPMF2JyNnYhxPIeib+/i0vqPjxO/k5a+O0kNZsAl8Wdl1/AOd8oPI78RtJvb9M09xwnfAeTU6nryhE6/B8ebh3w3/4y6fPGc7r7i2V+ERuyz+C/9SVcz+4qPlbk7IUhL5Uc7/Yk9JiF2bkJIY2Xo1vxPMd0rbDqy0l2gC4/nRY7vyaj2SDOxKcClxZfu3r2olneDyT88SmZQf0q8VnAkJOMU0Yc2X7dio9d+f+Y7863aQLEeg2kqPh4OI4DPiNg8/M4L7iLc2GjSerwCM7nj+B+ciXu8aspbNSUk33fw+LgWqlYqsuQnUjT6Ncwntlc6ncFxkBOdXuBPO+2NDm0AL/sZI4Hv0hu8edwx7P9VHz/eg/LoeXk+HQkq2lPMoP6UtTIt8z3q4vvac1ag9uzJiUl4evrS2pqKpMmTeK5556ja9dL3ejo6Gg6d+5c4yBjYmKIjKzCxqMNUG200eLdpwj1NlZ62C45M4+7P9nKybQcXr+9PQadjkNJmcQmZ9Ir3Ie7ujUr9/VxKdmsP3KWG5o1oW1A9YcRsvOLeOTbXbTyc+OZIRFq+OuHiao6r9sDYHC61D5FBWoRbmDXssvIKytmiRqKcvVRa7Iq2xOqLosZXguBNiPVvMCSx+G+FWro78gq1Ysa/bFay1WYCw9cNhxotcL73dS9urDCgBdVD+WKmGNiYog0JMD829WwlUcAPLSl6p9t4zvw+/NqSHDom5deb7XC7m9g+TPqcf/nVG/p3AnVG/EIVL3Aiz2vo3/AV6PV2reoh8p/zw1vw6oX1Bo53zal2+6t1qpcf/y35Z8n97yaH9z6sZpfGztPLb7miv/HctLg7TZq3dvoD0ufp6gA1vxHDfPqHdUwr8EFwvrC4RWq8GX8d+X3MrOS1RBy2jFIO6qKU/r9X8V3vrZY1HKLVS+o4pr+/1K9SnOBujYS98Iv09QcYu8nYOeX4N1SrSu8nNWqCnMOL4fDKyHlkLoubv8MWg4u9bY1+Q66Wq6oUQ/K11dlUi8vLwYOHMiePXtKJChRv4zsGFCl55vcnFkwpTsTPtvGowt2A2r4z9PViWV7E3F20DHmhsASrzmTnss3W0+yYn8ih5NUNZhBp/HPQa144KZQdLqqfRnmF5mZ8tUONsam8sfBZFo3dWdEh5CSX9CXMziW+T9XlYX1U184zbrXfXIC1RMJ7qUq9OK3q4XLF3fXCO+vNrVd96b6Iuv9z5Kv1TRVDPH7v2DUh9B+7NXfJ7g3OLhCQaZKMNX5bD0fhZxUlagKstVwaFaSWhR8eic076niaNK8/POE9YPQvmq+rNPdKpmlHYOtc9Udmns8qv57mgvVsZA+pZMTqLZrfwds+QCyU0oWtFyUlw7R81QlXl66en7KEfh12oW1bSWvY7Z/pnb46PFI2bEbHGHgSxDWX83/hfRWw31Obqo4ZfFD6tyjPiy7jbPOwgdRkJOiHrv6qDVK8Vth4lLwDCn7fQty4Ocp6rY1oX1hxGxoEqx+p3cAR1eVJKduUH8oXJyLvK2M9YOaBsE91T+D/qPa48f74JtxMPBF6DGtzq/9aieonJwcLBYLRqORnJwcNm7cyEMPVfBXjrjueBmd+O6B7myKTSWwiQvhJiM6TeNvn2/jmZ/24N/Yhe6hXgCsP3KWad/uIj23kK7Bnjw/vDU9wr2YszqW1347yMbYFN66owMm99IVUIVmC0eSsmju1QhXJ3XZFpktTPt2FxtjU3n9tvYs2H6SGQv30i7Ag2Dvuh0+wbGR+mu8cfm9xFoV2ufSLhYj37/05aBpqorulwtfluGl9zIs6PIgycG3E9i07OGZYg7O0HKQ6pW1r+KO9Jcb8KKaA9zxuZr3MJpUufrgV9TcX2XnYAa8AHP7wG8zVY9m/0K1s4alUBW6jPlEbeuUeVp9GV9Nh/GqoGHfT6pXfVHSATVf+dd36p5jYf1U7E3bq4T68U1q38e//XrpNdmpar6rxSDwbV1+/KF91D+X63S36r2seVnN7/V/vvTrlj8F+RlwzyLV83P2uLCoewTMuxUmLS197WUmwbd3qrm9wa+opQFXSyDOHjD6I1WRmnZUJdCKeLeA+35TC7d/f1613a3v1u683hWqPcQXHx/Pww8/DIDZbGb48OFMnVpycaMM8V079tZG6TmFjPlwIylZBfw0tQcr9ify35WHCDcZ+XBCZ8J8jMXPtVqtfLc9nhd+3Y+DXkeHwMaE+bgSZjKSnW9m87FUdhxPI6fAjJNBx82tfBjarinrDqfw084EXhjRmok9Qzh1Ppeh76wnyNOFn6b2wMlw6UuwMu2TlV/EhiMpBHm60Ma/biuX8ovMJeKrlOSD8EE3cPHE+vh+lh48z/J9iYzpFEC/cHe02e3UIuOnjpZIAFn5Rdz3v+1EnzzHL//oedXPVtxG2Smqoqsq9/O6msK8mpdc/3S/WtvmaIQu96lkfGqnSsj5maqH5ugKD28rv+Lvo96AFXo/qdavHd8ISXtB76QKSbrer5LB5XZ/o3b16PccMV5DiMzerBZzF2TBxGWXlghUldUKSx5TQ7L9n4deT1xKJheHj/v9nyrxv9zp3fDlrWpx9z2LVM/KalE7oiyYoHpct31aujijNlmtque15mW46/viEYm6GOKr0RxUdd+0quzty9ce2WMbnUzNYfQHG8nKLyK/yMLw9k157bb2xT2gK8UmZ/LR2mMcSc7iaHIWWRcWCLf0NdI91IsOgY3Zk6C+lJMzVen24wNa8uiASzc3/P1AEn//cgf3RjVnxi2ROOg1DHod+/YfIDAknPM5hZzPLSSnQMWUX2ghKSOP1QeT2XI0lQKzBQe9xuxxnRjWvvbLhM0WK88v3sfi3adZ+FAPWvpWoTrKaoVP+5PabAiPJfRh/ZEUnB105BVauDHEk1ltzxDmAbQdU/yS9NxCJv5vG3sS0mnkqKelrxs/PBBV5lCqPV5DgOqxxPyi5oJcmlw6nnVWDZMdWgbD31bJqzybP4AVF8rPDS6q7Dx8oNrv0NWr7NdYrWpnhf2LyHMPVttBhdyk1q/VNIGbi2DRgyr5Rv0DBv5bLTR/v5vqcf59jRqWu1JCNHw1SvWwLmf0hbu+u3Zrl9ITwM2/+I8CSVDiquy1jXaePMejC3YxsUcI9/UMRqvkmLXVaiU5Mx+DTsPLWHIIwWKxsvPkOZIz87mlrV+pc7706wE+33hpdwWdpr5nyrvQQ7xdGdjal5ta+PDO6sNEnzjHrDHtGNe15DBKYnoeG2JT2HDkLNuPn6NnuBf/GdUOR0PJv9wPJmZwJj2PPi18ipNBkdnCkz/8xaLdp3E06Ij0c+OnqT0w6Eu+NiUrHy9Xx1KfK7/IzPt/xPLR2mM4Oeh4clArxnUN4ocd8byz+ggpWQXc1NKHgZEm+kaYMDoZuOezbRxMzGDO+BvIyCvk6R/38ObYDtze+Yo5FS5dQ3sSzpOUkc/A1hUMB9oDq1UtxjW1rng+pDBPbYzrE6luQllegcLlcs/D3D4U5ufiMOx1VYJfW3MvFgv8Nl3t0tFhvPo8e3+AKWtUjFeTHKOKLTSd+kfvoHYOuXxJwDUmCUpclbTRJQVFFhbvPkVqdgGFRRYKzBZSU1MID/KniasDHi4ONHI04GTQ4WjQ4eHiQGCTSwtLcwvMPPh1NGsPn+XpIa0I9Tay6WgKm46mEpusCju8XB1pE+DBusNn6RXuzUf3dMboZMBqtfLVlhP8Z0kMBWYLEX5uPNKvBf0jTTy6YBcr9ifx1OBWNPNsxCPf7uKZIRFMvflSVdYXG+N44dcD3BjiyXPDWhdXVO4/nc4/v/+Lg4mZjOroz8xhkZjcLg2dZeUX8en6Y/y86xQnUnMAMDoZKDBb+GjCDfSL8MVisXLbR5s4mZrDH0/ejIdLyb/Od+3Zz5KTGp9vjMNqhUf7t+CxAS0q/UdFbSg0W1i4M4H/bTzO33oEM/7GazjHV56CHGIOxxLZtn3tn/vyITNQw30D/lX771PHJEGJq5I2Kl9V26egyMLj3+9m6R61I3wjRz1dgz3pGe5Fr3AfIvzc0Ok0ftgRz/SFe2nd1J054zvxxopDLN17hr6tfLilXVM+XnuUo2ezcXMykJlfxPPDW3NfrxCsVitTv97JH4eSWTatF+EmN+auO8oryw5yY7AnR89mkZZTwJhOgQQ0ceGDNbE0cXXk1THt6B959Z6N1WrlWEo2aw4mszv+PONvbEbP8EtVa/tOpXPrexu4p3tzXhzZtvg1G2JTeOq7nSRmFTGhezNyCyz8tDOBe6Oa88KINuh0Guk5hXy99QRb49K4pa0fozoG4OJYOwtOzRYrv/x1itmrjnAiNQd3ZwPZBWbmTbqRXi1KVt1tOZaKg16jc/PSywSiT6Sx8kBSiS3x/D2caRfoQWRTdxo5Vr9wuc7/H9v5pdpkePTH9W+rJOywzFyI65WjQce7d3ZieLumeLs50SGwcalhPICxXYLwMjry0Pyd3Pzmn+h1GtNviWBKb1Uyf9sNgSzbe4avNp/g9i6B3NFF7dygaRr/HtWWLW+v5ckf9tAvwsRbvx9mWPumzB7XkdxCMx+sOcrnG+IoMFu4tYM/L97ahiau5Q9LaZpGmI+xRBHK5doGeDChe3O+2nICk7szMWcy2BqXxtnMfALcHfhuSne6hXphtVrxMjoyd90x0rIL1JKC7SfJKTDj7+HMjMNnee23g9zZtRkTujcr0QOtjHPZBayKSWLfqXT2nc4g5kwGOQVmWjd159N7u9At1JPbP9zMQ/OjWfyPXoR4u2KxWJnzRyxvrzoMwJ1dg5gxNBIPFwdyC8y8seIQ/9sUh0GnYbgwL2KxWsm/sJu/ToMwHyOdmzehS7AnXZo3oblXo0r1ENNzCykwl/5bPrfAzM+7TpFXaGZC9+ZlXiOVdnH3kQrkFZr5YE0sZ9LzeGpwqzKrXi+yWKz8e+kBzpzPo1+EGvr1cXMqPs/x1GwaORho5lXNranqmPSgrhPSRuWr6/aJPnGOOX8c4eG+4VXa+mnx7lPFa8hGdwrgjdvbl5iTik/LIeFcLlFhV5nEr4b0nEL6v/UnKVkF+Lk70y3Uk6hQLyJcMunYruQ6oo/WHuXV5Qcx6DRu7eDP328KJcLPjW1xaXyx6Tgr9idisUKPMC/GdglkSJumV+1VFZotrDmYzE87E/jjYDKFZiuujnpa+7vTxt+DHmFeDIj0LZ6zi0/L4db3NtDE1ZGvJ3fjhV/2s/JAEmNuCMDb6MSn64/hbXTigT5hfL3lBHEp2dzTvTnTb4koLsSxWq0kZeRfSITp7ElIJ/rEueK7Tjdy1OPj5oS30QlvoyOuTgacHfQ4G/QUWSzEJmdxJDmLs5n5OBs0+rQy0T/Sl/aBHvyy+zTfbDvJ+Rx1rnYBHrw7vhMhF5Y4WK1W1h9J4bf9iZzPKSA9t5CM3CKcDDqCvV0J8XYlsIkLCedyiTmjkrRO03j99vZ0atakjBaEjbEpzPx5LydSc3DQazg76Jl+SwTjuzYrVfhitVp5fvF+vtpyAi9XR1KzC9A0iPRzJyOvkFPnc7Fa1TrEN8a2Z3SnkvOSK/cn8sGfR/E2OhFuMhLm44q3mxMWixWzxYpBr9G7hQ8OeimSkC/fCkgblc9e28dqtfLirwdw0GtMvyUSfRUXKlfXmfRc8gstJXoQV2uj6BNp+Hm4ENC49HZWp87n8uOOBH7cGU98Wi6ujnpa+LoR0MSFwCYuuDjoOXY2myPJWRw7m0V+kQVvoyMjOwYwulMArZu6l7s4e+uxVO7+dCuaBhYrPDs0kkkXim32JJzn6R/3cDAxk8AmLrx+W3t6hJexCPcKFouV2LNZbD+exrGz2aRk5XM2M5+UrHyy883kF5nJK7SgAaEmIy1Mqke699gpdiYWkpih7p+m02BQaz/u6xVCWnYB0xfuoaDIwgsj2mCxWvl8YxyHk7Jwczbg6+6Mh4sD7s4GcgrMxKVkF1eigrpFTmRTdw4mZpCUkcfzw1szoXvz4v82hxIz+XjtURbuOkWItysvj2qLn4czz/68j83HUunSvAkzhkZwQ7Mmxa95Y8VB3l9zlAf6hDJ9SAQHzmSwOiaZrXGpeLo6EertSqiPKwu2xbP5WCr/NyyS+3uHYrZYmb3qMHP+iCXU2xWDXiMuJZvCMnqQn/2tS/GQsyQocVXSRuWT9qlYTdrIYrGy7XgaS/ac5nhKDgnncjh1PpdCs5WAxi608DUS7mMkKsyLm1pe+qu7Mn7YEc+Hfx7lP6PalkpAhWYLfx46S1SYF8arLF+oLTExMURERLD/dAa748/Tp6UPQZ6XhsbOpOfy2ILdbI1LAyCyqTv39wpheIemZa55y8ovIuFcDk3dXfBopApW0nMKefz73fxxMJlRHf1pH9iYn3YmsP90Bg56jQf7hPFw33CcHdT5rFYrP0Yn8MqyGM7lFNI2wJ2/RQWTnJnPGysOMf7GZrwyum25w5h5hWYe/243y/clcn+vEI6ezWLNobPc0SWQl0a2xdlBT5HZwsm0HM7lFGLQaeh1Gi6O+hJDyZKgxFVJG5VP2qditd1GFouVArOl+Mu0vqtM+5gtVpbsOY2PmxNRoV7VqoC0WKy8vyaWt1YdxmpVQ4djbgjg1g7+pZZcXJSdX8TPu04xb9NxjlyoNB3RwZ/Z4zpWqld+cX3e/K0ncdBr/GtEG+7u1qxK8UuRhBCi3tDpNJyv1W0l7IRep1V5T8sr6XQaj/RvwZC2fgC0qMRiblcnAxO6N+fubs3YfDSVvafSua9XSKWHjPU6jf+MakunZk1oYTLSIahxTT5CrZEEJYQQdqgyielKmqbRI9y7UnNxZb22rAXctlSDmkghhBCi7kiCEkIIYZckQQkhhLBLkqCEEELYJUlQQggh7JIkKCGEEHZJEpQQQgi7JAlKCCGEXZIEJYQQwi7V+V58QgghREWu+WaxQgghRHXJEJ8QQgi7JAlKCCGEXZIEJYQQwi7ZfYJat24dgwcPZuDAgcydO9fW4djcmTNnuOeeexg6dCjDhg1j3rx5AJw/f55JkyYxaNAgJk2aRHp6uo0jtS2z2cyoUaN44IEHAIiPj2fs2LEMHDiQxx57jIKCAhtHaFsZGRlMmzaNIUOGcMstt7Br1y65hi7zxRdfMGzYMIYPH84TTzxBfn5+g7+GZsyYQVRUFMOHDy8+drVrxmq18p///IeBAwcyYsQI9u/fX633tOsEZTabeemll/j0009ZunQpS5YsITY21tZh2ZRer2f69OksW7aM7777jm+++YbY2Fjmzp1LVFQUK1euJCoqqsEn8y+//JKwsLDix2+++SYTJ07k999/x93dnR9//NGG0dneyy+/TO/evfntt99YvHgxYWFhcg1dkJSUxJdffslPP/3EkiVLMJvNLF26tMFfQ2PGjOHTTz8tcexq18y6des4fvw4K1eu5N///jcvvPBCtd7TrhPUnj17aN68OUFBQTg6OjJs2DBWr15t67BsymQy0aZNGwCMRiOhoaEkJSWxevVqRo0aBcCoUaNYtWqVDaO0rcTERP78809uv/12QP01t2XLFgYPHgzA6NGjG/R1lJmZyfbt24vbx9HREXd3d7mGLmM2m8nLy6OoqIi8vDx8fHwa/DXUtWtXPDw8Shy72jVz8bimaXTs2JGMjAySk5Or/J52naCSkpLw8/Mrfuzr60tSUpINI7IvCQkJxMTE0KFDB1JTUzGZTAD4+PiQmppq4+hs55VXXuGpp55Cp1OX97lz53B3d8dgUDeQ9vPza9DXUUJCAp6ensyYMYNRo0bx7LPPkpOTI9fQBb6+vtx333307duXXr16YTQaadOmjVxDZbjaNXPld3d128uuE5S4uuzsbKZNm8bMmTMxGo0lfqdpGpqm2Sgy21qzZg2enp60bdvW1qHYraKiIg4cOMD48eNZtGgRLi4upYbzGvI1lJ6ezurVq1m9ejXr168nNzeX9evX2zosu1cX14yhVs9Wy3x9fUlMTCx+nJSUhK+vrw0jsg+FhYVMmzaNESNGMGjQIAC8vLxITk7GZDKRnJyMp6enjaO0jZ07d/LHH3+wbt068vPzycrK4uWXXyYjI4OioiIMBgOJiYkN+jry8/PDz8+PDh06ADBkyBDmzp0r19AFmzZtIjAwsPjzDxo0iJ07d8o1VIarXTNXfndXt73sugfVrl07jh8/Tnx8PAUFBSxdupR+/frZOiybslqtPPvss4SGhjJp0qTi4/369WPRokUALFq0iP79+9soQtv65z//ybp16/jjjz9466236N69O//973/p1q0bK1asAODnn39u0NeRj48Pfn5+HDt2DIDNmzcTFhYm19AF/v7+/PXXX+Tm5mK1Wtm8eTPh4eFyDZXhatfMxeNWq5Xdu3fj5uZWPBRYFXa/1dHatWt55ZVXMJvN3HbbbUydOtXWIdnUjh07uPvuu2nZsmXxHMsTTzxB+/bteeyxxzhz5gz+/v7Mnj2bxo0b2zZYG9u6dSuff/45H3/8MfHx8Tz++OOkp6cTGRnJm2++iaOjo61DtJmYmBieffZZCgsLCQoKYtasWVgsFrmGLnj33XdZtmwZBoOByMhIXn75ZZKSkhr0NfTEE0+wbds2zp07h5eXF4888ggDBgwo85qxWq289NJLrF+/HhcXF1555RXatWtX5fe0+wQlhBCiYbLrIT4hhBANlyQoIYQQdkkSlBBCCLskCUoIIYRdkgQlhBDCLkmCEkIIYZckQQkhhLBL/w93g4dd4X+iNAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.compile(\n",
    "    optimizer='sgd',\n",
    "    loss='mae',\n",
    "    metrics=['mae'],\n",
    ")\n",
    "EPOCHS = 100\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data=(X_valid, y_valid),\n",
    "    batch_size=64,\n",
    "    epochs=EPOCHS,\n",
    "    verbose=0,\n",
    ")\n",
    "\n",
    "history_df = pd.DataFrame(history.history)\n",
    "history_df.loc[0:, ['loss', 'val_loss']].plot()\n",
    "print((\"Minimum Validation Loss: {:0.4f}\").format(history_df['val_loss'].min()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d99303e4",
   "metadata": {
    "papermill": {
     "duration": 0.021272,
     "end_time": "2021-12-29T13:00:41.308837",
     "exception": false,
     "start_time": "2021-12-29T13:00:41.287565",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 4) Evaluate Batch Normalization\n",
    "\n",
    "Did adding batch normalization help?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4957131",
   "metadata": {
    "papermill": {
     "duration": 0.021521,
     "end_time": "2021-12-29T13:00:41.351883",
     "exception": false,
     "start_time": "2021-12-29T13:00:41.330362",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Oh, a lot! Last time it didn't even work. Now we have a resonable model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "7d524782",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-12-29T13:00:41.400389Z",
     "iopub.status.busy": "2021-12-29T13:00:41.399572Z",
     "iopub.status.idle": "2021-12-29T13:00:41.403680Z",
     "shell.execute_reply": "2021-12-29T13:00:41.404119Z",
     "shell.execute_reply.started": "2021-12-29T12:59:04.650332Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 0.030499,
     "end_time": "2021-12-29T13:00:41.404247",
     "exception": false,
     "start_time": "2021-12-29T13:00:41.373748",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"4_Q4\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct:</span> \n",
       "\n",
       "\n",
       "You can see that adding batch normalization was a big improvement on the first attempt! By adaptively scaling the data as it passes through the network, batch normalization can let you train models on difficult datasets.\n"
      ],
      "text/plain": [
       "Correct: \n",
       "\n",
       "\n",
       "You can see that adding batch normalization was a big improvement on the first attempt! By adaptively scaling the data as it passes through the network, batch normalization can let you train models on difficult datasets."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# View the solution (Run this cell to receive credit!)\n",
    "q_4.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "348e99b7",
   "metadata": {
    "papermill": {
     "duration": 0.021863,
     "end_time": "2021-12-29T13:00:41.448058",
     "exception": false,
     "start_time": "2021-12-29T13:00:41.426195",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Keep Going #\n",
    "\n",
    "[**Create neural networks**](https://www.kaggle.com/ryanholbrook/binary-classification) for binary classification."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "216b6bfe",
   "metadata": {
    "papermill": {
     "duration": 0.022671,
     "end_time": "2021-12-29T13:00:41.493154",
     "exception": false,
     "start_time": "2021-12-29T13:00:41.470483",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "*Have questions or comments? Visit the [course discussion forum](https://www.kaggle.com/learn/intro-to-deep-learning/discussion) to chat with other learners.*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 42.334091,
   "end_time": "2021-12-29T13:00:45.024475",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2021-12-29T13:00:02.690384",
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}