03 Neural Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dfdf7397-1670-4a8c-849a-f5d2bfdaa3d4",
   "metadata": {},
   "source": [
    "# Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2743a5e-bd1f-414e-8d99-0afccb0d9b32",
   "metadata": {},
   "source": [
    "### Introduction "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98c602f3-8ce1-467f-84a9-cb1beea74b28",
   "metadata": {},
   "source": [
    "This notebook contains our neural network analyses that were performed on the data downloaded from [UCI machine learning repository](http://archive.ics.uci.edu/ml/datasets/Smartphone-Based+Recognition+of+Human+Activities+and+Postural+Transitions)\n",
    "\n",
    "We will be using the train/test data provided in order to use neural network models to identify which activity is being performed based off of phone sensor data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e421b300-3934-4a2f-bdb8-b8e88b5fe7a1",
   "metadata": {},
   "source": [
    "First, we'll start off by importing our required libraries for this analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "6c476da3-4c50-4055-b410-32154bfea7a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing basic libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# Specific neural network models & layer types\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout\n",
    "from tensorflow.keras.layers import Embedding, LSTM, GRU, BatchNormalization, TimeDistributed\n",
    "from keras.layers.convolutional import Conv1D, MaxPooling1D\n",
    "from tensorflow.keras.utils import timeseries_dataset_from_array\n",
    "\n",
    "# Data reformatting\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "# Ignore futurewarnings\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6c9e02b-e045-4080-9c83-3547c9d999b0",
   "metadata": {},
   "source": [
    "The way this data is read is explained in `01 Basic_EDA.ipynb` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "095ce252-f7e5-4450-9ec4-66a420bca8a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# reading in the data\n",
    "columns = pd.read_csv('data/features2.txt', header = None)\n",
    "\n",
    "X_train_orig = pd.read_csv('data/Train/X_train.txt', names = columns[0], sep=' ')\n",
    "X_test_orig = pd.read_csv('data/Test/X_test.txt', names = columns[0], sep=' ')\n",
    "\n",
    "y_train_orig = pd.read_csv('data/Train/y_train.txt', sep=' ', header = None)\n",
    "y_test_orig = pd.read_csv('data/Test/y_test.txt', sep=' ', header = None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37b94a28-0614-4be4-ad16-64ddf021b806",
   "metadata": {},
   "source": [
    "For our recurrent neural network model, we'll need to reformat our data into arrays in order to create the input window with the `.timeseries_dataset_from_array` method. We do this by using the pandas `.values` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "5a120a5f-0d58-467e-8f10-b293aaa3291f",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X_train_orig.values\n",
    "X_test = X_test_orig.values"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6b28e10-d360-47cf-bfb5-6227a7d5a1d3",
   "metadata": {},
   "source": [
    "We will also one hot encode our target data because we have a total of 12 targets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "a2294165-2657-4dd0-84a6-72279c2d0f5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# One hot encoding target data\n",
    "ohe = OneHotEncoder()\n",
    "y_train = ohe.fit_transform(y_train_orig).toarray()\n",
    "y_test = ohe.fit_transform(y_test_orig).toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94ab6d53-b7d8-4f7c-9820-bce835c1f929",
   "metadata": {},
   "source": [
    "Now, we'll also prepare some train and validation sets, with the validation sets being 10% of the total train data. This is to refine our models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "9cb664f7-2199-45f8-bfbf-1d750fb86cac",
   "metadata": {},
   "outputs": [],
   "source": [
    "# split train and validation set for OHE data\n",
    "X_train2, X_validation, y_train2, y_validation = train_test_split(X_train, y_train, test_size=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ce01e3-6c01-48d5-89df-5b50784f69d7",
   "metadata": {},
   "source": [
    "### PLAN\n",
    "\n",
    "For neural networks, our goal is to first try a simple fully connected NN with a few layers, first ignoring the time component and previous activity. \n",
    "\n",
    "Then second, we will create a recurrent neural network by feeding in 20 time-steps and perform some sequential classification, predicting the current/next movement."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd699dee-f220-4125-87b7-1d80f0a2ecbe",
   "metadata": {},
   "source": [
    "### Building a Neural Network Using Keras\n",
    "\n",
    "The process of building a network using Keras can generally be broken down into four separate steps:\n",
    "1. **Build the model**: This is the step where we will declare the structure of the network — primarily the types and sizes of the hidden layers.\n",
    "2. **Compile the model**: This step allows us to customize some of the settings that will be used for training.\n",
    "3. **Train the model**\n",
    "4. **Evaluate the model and generate predictions**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85e65fbf-376f-49ba-9ea5-9b9794937a82",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "### Fully Connected Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7666378e-63a7-4bba-8fed-57043088ae1a",
   "metadata": {},
   "source": [
    "For our fully connect neural networks, we'll be using the following parameters:\n",
    "\n",
    "- softmax output layer: ensures all probabilities add up to 1\n",
    "- adam optimizer: best adaptive optimizer and is good with datasets with large number of features\n",
    "- sparse categorical crossentropy loss function: obtains cross entropy between labels and predictions\n",
    "- sparse categorical accuracy metric: for multivariate classification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f6a0ed5-95a1-46e5-a401-61aad9400cdb",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### 3 Layer NN"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab28d023-f963-4c1f-ba48-d71fbb93bb86",
   "metadata": {},
   "source": [
    "We will start building our model by instantiating the sequential mode, adding layers, then compiling the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "7ed19462-64aa-4d56-a18b-8b93b4fec305",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new sequential model\n",
    "model = keras.Sequential()\n",
    "\n",
    "# Declare the hidden layers\n",
    "# We'll be using the RELU activation function for our layers for the unbound benefit\n",
    "# Dense layers will be used to create a fully connected neural network model\n",
    "\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "\n",
    "\n",
    "# Declare the output layer\n",
    "# Our output layer has 12 units for the 12 different activities to be classified\n",
    "# There are 6 main actions, and 6 transition actions\n",
    "# Transition actions are when the participant transitions to the next position\n",
    "model.add(layers.Dense(13, activation=\"softmax\"))\n",
    "\n",
    "# Compile the model\n",
    "model.compile(\n",
    "    # Optimizer\n",
    "    optimizer=keras.optimizers.Adam(),  \n",
    "    # Loss function to minimize\n",
    "    loss=keras.losses.SparseCategoricalCrossentropy(),\n",
    "    # Metric used to evaluate model\n",
    "    metrics=[keras.metrics.SparseCategoricalAccuracy()]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1664444b-1c42-473f-b507-40cb81cb1d11",
   "metadata": {},
   "source": [
    "Now we'll fit our model with the train data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "f6a2f020-f94b-46d5-913f-69c5c60bb297",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Fit the model\n",
    "history = model.fit(X_train_orig, y_train_orig, epochs=100, verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf6539ea-ef0d-41a7-bb97-71b7afa51943",
   "metadata": {},
   "source": [
    "Then we will test the accuracy of our model and compare it to the train accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "75c9f6a9-8b00-402a-bcc4-f81d201a532d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Accuracy: 0.9946\n",
      "Test Accuracy: 0.9194\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model using the test data and generate predictions\n",
    "train_accuracy = history.history[\"sparse_categorical_accuracy\"][-1]\n",
    "result = model.evaluate(X_test_orig, y_test_orig, verbose=0)\n",
    "\n",
    "print(f\"Train Accuracy: {train_accuracy:.4f}\")\n",
    "print(f\"Test Accuracy: {result[1]:.4f}\")\n",
    "\n",
    "predictions = model.predict(X_test_orig)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12d24536-3512-4e26-aa5b-f8ea6df2a240",
   "metadata": {},
   "source": [
    "With a train accuracy of 99.46% and a test accuracy of 91.94%, this is a very good start. The high train accuracy scores along with the test accuracy not being too far behind indicate that the model isn't overfit and is performing very well.\n",
    "\n",
    "Let's try taking a look at a neural network model with 5 layers."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad147883-aa5f-4598-8b79-63eb17ab44d8",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### 5 Layer NN "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a763bcc-2afe-4a23-a21c-c8443e4b64c3",
   "metadata": {},
   "source": [
    "We will follow the same steps as above to build, compile and train the neural network model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "d8dfd0cf-dcd2-41de-be29-39a71bde3597",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a new sequential model\n",
    "model = keras.Sequential()\n",
    "\n",
    "# Declare the hidden layers\n",
    "# We'll be using the RELU activation function for our layers for the unbound benefit\n",
    "# Dense layers will be used to create a fully connected neural network model\n",
    "\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "model.add(layers.Dense(40, activation=\"relu\"))\n",
    "\n",
    "# Declare the output layer\n",
    "# Our output layer has 12 units for the 12 different activities to be classified\n",
    "# There are 6 main actions, and 6 transition actions\n",
    "# Transition actions are when the participant transitions to the next position\n",
    "model.add(layers.Dense(13, activation=\"softmax\"))\n",
    "\n",
    "# Compile the model\n",
    "model.compile(\n",
    "    # Optimizer\n",
    "    optimizer=keras.optimizers.Adam(),  \n",
    "    # Loss function to minimize\n",
    "    loss=keras.losses.SparseCategoricalCrossentropy(),\n",
    "    # Metric used to evaluate model\n",
    "    metrics=[keras.metrics.SparseCategoricalAccuracy()]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89f480bb-c8ed-44f7-9b3a-59f6b4acc942",
   "metadata": {},
   "source": [
    "Now we will fit the model on our train data, with the same number of epochs as our 3-layer model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "49f09df9-d4a6-4590-bbbc-da260c24aebc",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "243/243 [==============================] - 1s 1ms/step - loss: 0.8520 - sparse_categorical_accuracy: 0.6710\n",
      "Epoch 2/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.2736 - sparse_categorical_accuracy: 0.8926\n",
      "Epoch 3/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1765 - sparse_categorical_accuracy: 0.9279\n",
      "Epoch 4/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1439 - sparse_categorical_accuracy: 0.9426\n",
      "Epoch 5/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1358 - sparse_categorical_accuracy: 0.9441\n",
      "Epoch 6/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1118 - sparse_categorical_accuracy: 0.9552\n",
      "Epoch 7/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1084 - sparse_categorical_accuracy: 0.9571\n",
      "Epoch 8/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1115 - sparse_categorical_accuracy: 0.9548\n",
      "Epoch 9/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1087 - sparse_categorical_accuracy: 0.9556\n",
      "Epoch 10/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1041 - sparse_categorical_accuracy: 0.9598\n",
      "Epoch 11/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.1021 - sparse_categorical_accuracy: 0.9579\n",
      "Epoch 12/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0875 - sparse_categorical_accuracy: 0.9634\n",
      "Epoch 13/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0756 - sparse_categorical_accuracy: 0.9695\n",
      "Epoch 14/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0809 - sparse_categorical_accuracy: 0.9679\n",
      "Epoch 15/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0816 - sparse_categorical_accuracy: 0.9661\n",
      "Epoch 16/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0853 - sparse_categorical_accuracy: 0.9652\n",
      "Epoch 17/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0687 - sparse_categorical_accuracy: 0.9706\n",
      "Epoch 18/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0984 - sparse_categorical_accuracy: 0.9607\n",
      "Epoch 19/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0608 - sparse_categorical_accuracy: 0.9736\n",
      "Epoch 20/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0818 - sparse_categorical_accuracy: 0.9670\n",
      "Epoch 21/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0696 - sparse_categorical_accuracy: 0.9714\n",
      "Epoch 22/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0704 - sparse_categorical_accuracy: 0.9735\n",
      "Epoch 23/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0672 - sparse_categorical_accuracy: 0.9741\n",
      "Epoch 24/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0698 - sparse_categorical_accuracy: 0.9721\n",
      "Epoch 25/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0682 - sparse_categorical_accuracy: 0.9724\n",
      "Epoch 26/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0987 - sparse_categorical_accuracy: 0.9668\n",
      "Epoch 27/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0495 - sparse_categorical_accuracy: 0.9802\n",
      "Epoch 28/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0538 - sparse_categorical_accuracy: 0.9794\n",
      "Epoch 29/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0630 - sparse_categorical_accuracy: 0.9762\n",
      "Epoch 30/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0574 - sparse_categorical_accuracy: 0.9772\n",
      "Epoch 31/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0601 - sparse_categorical_accuracy: 0.9762\n",
      "Epoch 32/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0622 - sparse_categorical_accuracy: 0.9741\n",
      "Epoch 33/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0439 - sparse_categorical_accuracy: 0.9821\n",
      "Epoch 34/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0560 - sparse_categorical_accuracy: 0.9777\n",
      "Epoch 35/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0518 - sparse_categorical_accuracy: 0.9786\n",
      "Epoch 36/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0459 - sparse_categorical_accuracy: 0.9830\n",
      "Epoch 37/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0443 - sparse_categorical_accuracy: 0.9839\n",
      "Epoch 38/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0633 - sparse_categorical_accuracy: 0.9761\n",
      "Epoch 39/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0476 - sparse_categorical_accuracy: 0.9818\n",
      "Epoch 40/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0483 - sparse_categorical_accuracy: 0.9824\n",
      "Epoch 41/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0437 - sparse_categorical_accuracy: 0.9824\n",
      "Epoch 42/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0499 - sparse_categorical_accuracy: 0.9795\n",
      "Epoch 43/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0428 - sparse_categorical_accuracy: 0.9836\n",
      "Epoch 44/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0381 - sparse_categorical_accuracy: 0.9835\n",
      "Epoch 45/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0511 - sparse_categorical_accuracy: 0.9800\n",
      "Epoch 46/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0467 - sparse_categorical_accuracy: 0.9818\n",
      "Epoch 47/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0337 - sparse_categorical_accuracy: 0.9873\n",
      "Epoch 48/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0516 - sparse_categorical_accuracy: 0.9815\n",
      "Epoch 49/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0347 - sparse_categorical_accuracy: 0.9865\n",
      "Epoch 50/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0385 - sparse_categorical_accuracy: 0.9845\n",
      "Epoch 51/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0518 - sparse_categorical_accuracy: 0.9782\n",
      "Epoch 52/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0294 - sparse_categorical_accuracy: 0.9897\n",
      "Epoch 53/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0299 - sparse_categorical_accuracy: 0.9880\n",
      "Epoch 54/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9790\n",
      "Epoch 55/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0301 - sparse_categorical_accuracy: 0.9896\n",
      "Epoch 56/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0248 - sparse_categorical_accuracy: 0.9897\n",
      "Epoch 57/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0284 - sparse_categorical_accuracy: 0.9892\n",
      "Epoch 58/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0449 - sparse_categorical_accuracy: 0.9842\n",
      "Epoch 59/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0276 - sparse_categorical_accuracy: 0.9905\n",
      "Epoch 60/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0409 - sparse_categorical_accuracy: 0.9825\n",
      "Epoch 61/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0360 - sparse_categorical_accuracy: 0.9861\n",
      "Epoch 62/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0494 - sparse_categorical_accuracy: 0.9808\n",
      "Epoch 63/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0303 - sparse_categorical_accuracy: 0.9893\n",
      "Epoch 64/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0313 - sparse_categorical_accuracy: 0.9887\n",
      "Epoch 65/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0433 - sparse_categorical_accuracy: 0.9848\n",
      "Epoch 66/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0292 - sparse_categorical_accuracy: 0.9887\n",
      "Epoch 67/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0358 - sparse_categorical_accuracy: 0.9870\n",
      "Epoch 68/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0303 - sparse_categorical_accuracy: 0.9885\n",
      "Epoch 69/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0306 - sparse_categorical_accuracy: 0.9871\n",
      "Epoch 70/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0224 - sparse_categorical_accuracy: 0.9906\n",
      "Epoch 71/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0334 - sparse_categorical_accuracy: 0.9884\n",
      "Epoch 72/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0269 - sparse_categorical_accuracy: 0.9894\n",
      "Epoch 73/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0359 - sparse_categorical_accuracy: 0.9879\n",
      "Epoch 74/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0318 - sparse_categorical_accuracy: 0.9884\n",
      "Epoch 75/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0192 - sparse_categorical_accuracy: 0.9920\n",
      "Epoch 76/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0198 - sparse_categorical_accuracy: 0.9923\n",
      "Epoch 77/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0178 - sparse_categorical_accuracy: 0.9929\n",
      "Epoch 78/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0521 - sparse_categorical_accuracy: 0.9853\n",
      "Epoch 79/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0232 - sparse_categorical_accuracy: 0.9918\n",
      "Epoch 80/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0153 - sparse_categorical_accuracy: 0.9948\n",
      "Epoch 81/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0244 - sparse_categorical_accuracy: 0.9907\n",
      "Epoch 82/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0331 - sparse_categorical_accuracy: 0.9888\n",
      "Epoch 83/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0168 - sparse_categorical_accuracy: 0.9934\n",
      "Epoch 84/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0193 - sparse_categorical_accuracy: 0.9934\n",
      "Epoch 85/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0306 - sparse_categorical_accuracy: 0.9883\n",
      "Epoch 86/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0268 - sparse_categorical_accuracy: 0.9909\n",
      "Epoch 87/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0187 - sparse_categorical_accuracy: 0.9937\n",
      "Epoch 88/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0239 - sparse_categorical_accuracy: 0.9911\n",
      "Epoch 89/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0387 - sparse_categorical_accuracy: 0.9861\n",
      "Epoch 90/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0131 - sparse_categorical_accuracy: 0.9951\n",
      "Epoch 91/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0195 - sparse_categorical_accuracy: 0.9927\n",
      "Epoch 92/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0133 - sparse_categorical_accuracy: 0.9954\n",
      "Epoch 93/100\n",
      "243/243 [==============================] - 0s 2ms/step - loss: 0.0210 - sparse_categorical_accuracy: 0.9919\n",
      "Epoch 94/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0254 - sparse_categorical_accuracy: 0.9928\n",
      "Epoch 95/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0231 - sparse_categorical_accuracy: 0.9914\n",
      "Epoch 96/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0088 - sparse_categorical_accuracy: 0.9970\n",
      "Epoch 97/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0417 - sparse_categorical_accuracy: 0.9853\n",
      "Epoch 98/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0305 - sparse_categorical_accuracy: 0.9883\n",
      "Epoch 99/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0169 - sparse_categorical_accuracy: 0.9943\n",
      "Epoch 100/100\n",
      "243/243 [==============================] - 0s 1ms/step - loss: 0.0292 - sparse_categorical_accuracy: 0.9898\n"
     ]
    }
   ],
   "source": [
    "# Fit the model\n",
    "history = model.fit(X_train_orig, y_train_orig, epochs=100, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f486fc64-c5ca-4d54-b773-712507507238",
   "metadata": {},
   "source": [
    "Now we can evaluate our model on the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "55a251fb-83dc-4f1a-8f4e-df5d14ff7957",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Accuracy: 0.9898\n",
      "Test Accuracy: 0.9405\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the model using the test data and generate predictions\n",
    "train_accuracy = history.history[\"sparse_categorical_accuracy\"][-1]\n",
    "result = model.evaluate(X_test_orig, y_test_orig, verbose=0)\n",
    "\n",
    "print(f\"Train Accuracy: {train_accuracy:.4f}\")\n",
    "print(f\"Test Accuracy: {result[1]:.4f}\")\n",
    "\n",
    "predictions = model.predict(X_test_orig)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c797a1d0-806c-482c-853a-7c53270720f4",
   "metadata": {},
   "source": [
    "Look at that! Although our train accuracy is slightly lower, we have a smaller gap between the train and test accuracys when compared to the 3-layer model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20938def-227e-4675-a3a4-a6769a3d0a2a",
   "metadata": {},
   "source": [
    "| Model | Train Accuracy | Test Accuracy |\n",
    "| --- | --- | --- |\n",
    "| 3-Layer | 99.46% | 91.94% |\n",
    "| 5-Layer | 98.98% | 94.05% |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1f14f41-d6d5-427e-93bd-1daf06f02dba",
   "metadata": {},
   "source": [
    "### Recurrent Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47fb3e0a-af05-43fe-a30d-771abaa6da3d",
   "metadata": {},
   "source": [
    "Let's now take a look at how well a recurrent neural network (RNN) does with this data. \n",
    "\n",
    "In this analysis, we'll be building a RNN model that has a window size of 20, and the model will evaluate all activities at once to make a prediction.\n",
    "The window size allows the prediction to be made on the 20 records, performing a one shot prediction.\n",
    "\n",
    "To create these windows, we'll use the `.timeseries_dataset_from_array` function from the tensorflow, keras library. This function requres an array input in order to easily split the data into a format compatible with the RNN.\n",
    "\n",
    "- The sample_length = window size\n",
    "- The stride is the period between successive output sequences\n",
    "- Batch size is set to a number that would include all records of our data for the train and validation sets\n",
    "- Shuffle = False because we don't want to reorganize our timeseries data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "99248549-d50d-4d9e-b176-d52e28181d36",
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_length = 20\n",
    "stride = 5\n",
    "\n",
    "X_train3 = tf.keras.preprocessing.timeseries_dataset_from_array(\n",
    "  X_train2, None, sequence_length=sample_length, sequence_stride=stride, shuffle=False, batch_size = 540)\n",
    "y_train3 = tf.keras.preprocessing.timeseries_dataset_from_array(\n",
    "  y_train2, None, sequence_length=sample_length, sequence_stride=stride, shuffle=False, batch_size = 540)\n",
    "X_val = tf.keras.preprocessing.timeseries_dataset_from_array(\n",
    "  X_validation, None, sequence_length=sample_length, sequence_stride=stride, shuffle=False, batch_size = 60)\n",
    "y_val = tf.keras.preprocessing.timeseries_dataset_from_array(\n",
    "  y_validation, None, sequence_length=sample_length, sequence_stride=stride, shuffle=False, batch_size = 60)\n",
    "\n",
    "# create datasets for the training RNN input and target\n",
    "for batch in zip(X_train3, y_train3):\n",
    "    inputs, targets = batch\n",
    "    break\n",
    "    \n",
    "# create datasets for the validation RNN input and target\n",
    "for batch in zip(X_val, y_val):\n",
    "    input_val, target_val = batch\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "e80def19-19af-41b3-a75c-a8646d9caac1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of the X_train data is: (6990, 561)\n",
      "The shape of the y_train data is: (6990, 12)\n",
      "The shape of the X_validation data is: (777, 561)\n",
      "The shape of the y_validation data is: (777, 12)\n"
     ]
    }
   ],
   "source": [
    "print(f\"The shape of the X_train data is: {X_train2.shape}\")\n",
    "print(f\"The shape of the y_train data is: {y_train2.shape}\")\n",
    "print(f\"The shape of the X_validation data is: {X_validation.shape}\")\n",
    "print(f\"The shape of the y_validation data is: {y_validation.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "4331d584-f4d9-433d-9599-8d1d6d41a832",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of the inputs data is: (540, 20, 561)\n",
      "The shape of the targets data is: (540, 20, 12)\n",
      "The shape of the validation input data is: (60, 20, 561)\n",
      "The shape of the validation targets data is: (60, 20, 12)\n"
     ]
    }
   ],
   "source": [
    "print(f\"The shape of the inputs data is: {inputs.shape}\")\n",
    "print(f\"The shape of the targets data is: {targets.shape}\")\n",
    "print(f\"The shape of the validation input data is: {input_val.shape}\")\n",
    "print(f\"The shape of the validation targets data is: {target_val.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ee4b9eb-0f93-490d-9e62-8e6e807cd345",
   "metadata": {},
   "source": [
    "Now that our data is split up properly, we can see that our shapes are looking good. There are the proper number of features and targets, and the window size is also looking good. Let's move on to building the RNN."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fdcea1f-a863-4904-a9e8-b665b7604a6a",
   "metadata": {},
   "source": [
    "Here are some of the reasons as to why we built the model the way we did.\n",
    "\n",
    "- GRU is like LSTM, but with less features with a forget gate and performs similar to the LSTM.\n",
    "- Number of Nodes is arbitrary\n",
    "- activation function of `relu` seemed like a safe and very common choice\n",
    "- return_sequences is required for our RNN to obtain a sequence as the output instead of a single vector\n",
    "- softmax output to ensure all output values add up to 1\n",
    "- dropout layers help with overfitting\n",
    "- batch normalization standardizes inputs and accelerates training time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "ab013a61-668e-46df-b5ea-3dbb05db8060",
   "metadata": {},
   "outputs": [],
   "source": [
    "# instantiate the model\n",
    "RNN = Sequential()\n",
    "RNN.add(GRU(128, activation='relu', return_sequences=True))\n",
    "\n",
    "RNN.add(GRU(64, activation='relu', return_sequences=True))\n",
    "RNN.add(BatchNormalization())\n",
    "RNN.add(Dropout(0.15))\n",
    "\n",
    "RNN.add(GRU(32, activation='relu', return_sequences=True))\n",
    "RNN.add(BatchNormalization())\n",
    "RNN.add(Dropout(0.15))\n",
    "\n",
    "RNN.add(Dense(16, activation='relu'))\n",
    "RNN.add(BatchNormalization())\n",
    "RNN.add(Dropout(0.15))\n",
    "\n",
    "RNN.add(Dense(12, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2baa0201-ea76-4605-9a8f-25587772109a",
   "metadata": {},
   "source": [
    "Now that the model is built, let's compile our model and fit it on our reformatted data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "5832a98b-a1fb-41b3-817f-ea117f3ed540",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rv_npeAPBKZ9"
   },
   "outputs": [],
   "source": [
    "# Compile model\n",
    "RNN.compile(\n",
    "    loss='categorical_crossentropy',\n",
    "    optimizer=keras.optimizers.Adam(lr=0.001),\n",
    "    metrics=['accuracy']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "04ed2525-0bc5-4af3-bf6e-486c4516b843",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "history = RNN.fit(inputs, targets,\n",
    "        epochs=20,\n",
    "        validation_data=(input_val, target_val),\n",
    "        verbose = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "436cbb06-9543-443c-8b82-b6ec4cf4a541",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ef534fd-f430-4013-8a5b-1fb8fad6390f",
   "metadata": {},
   "source": [
    "#### RNN Model Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86869cda-ab62-408e-ac7f-a75b07252fcc",
   "metadata": {},
   "source": [
    "Everything is now complete, let's take a look at our model performance by plotting our train and validation loss and accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "3a20607b-e4d4-4683-8387-2af18ffb7e74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3gAAAFNCAYAAABSRs15AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAACC6ElEQVR4nO3dd3xV9f3H8dcnOyQhYY8QhsiUEYaI4gJcOBEtddbRarVaq7VUbftzt7VqtXWvausWFXGhqKg4UdkbBNkrgCQESELG9/fHucGQ3ISMu3Lzfj4e93HvPffc7/ncm5v7PZ/7XeacQ0RERERERBq/mHAHICIiIiIiIoGhBE9ERERERCRKKMETERERERGJEkrwREREREREooQSPBERERERkSihBE9ERERERCRKKMGTiGJm75nZRREQx61m9nwQyr3YzL6ocH+XmR1Um33rcayIeC9FRCS8IqU+UN0qEhpK8KTBfF+k5ZcyMyuocP/8upTlnBvjnPtfsGJtKDPLNLMSM+vu57E3zOzeupTnnEt1zv0QgLiqVJrBfi99x3RmNixYxxARaapUt+57rEnUrWb2XzO7M9DlStOkBE8azPdFmuqcSwXWAqdV2PZC+X5mFhe+KAPDObcBmAZcWHG7mbUETgYitgINJDMzvPfgRyCkv2RGw+dIRORAVLc2vbpVJFCU4EnQmNmxZrbezG4ws83AM2bWwszeMbOtZrbDd7tThed8ama/8t2+2My+MLN7ffuuMrMxNRzvRjNbaWb5ZrbYzM6s8FiNZZlZNzOb7nvuh0DrGl7a/6hUCQHnAIuccwtqisNPzM7MDvbdbmVmb5nZTjP7Fuhead9/m9k63+OzzOwo3/aTgD8BP/f9sjvPz3sZY2Z/MbM1ZpZjZs+aWbrvsa6+OC4ys7Vmts3M/lzD6wc4CugI/A44x8wSKsSZbGb/9B0rz/e+J/seO9LMvjKzXN9rubhyrL77lbvbODO7ysy+B76v6f3wPRZrZn+q8HeYZWZZZvawmf2z0vv6tplde4DXKyISEVS3RnXdWt3ruczMVpjZj77X0tG33czsft+x88xsvpn18z12su99yjezDWb2h/ocWxonJXgSbO2BlkAX4HK8z9wzvvudgQLgoRqefxiwDK9SuBv4j5lZNfuuxEs80oHbgOfNrEMty3oRmOV77A5qbpV6A2htZkdW2HYh8Gwt46jOw0Ah0AG41Hep6DsgG+/9fBF41cySnHPvA38DXvH9sjvQT9kX+y4jgYOAVKq+70cCvYDRwM1m1qeGWC8C3gZe8d0/tcJj9wJDgCN8sf4RKDOzzsB7wINAG99rmVvDMSobi/c37Ou77/f98D32e+BcvF9+m+O9l3vwTiDONbMYADNr7Xu9L9UhDhGRcFPdGp11axVmNgr4OzDe9xrWAC/7Hj4BOBroCWQAPwe2+x77D/Br51wa0A/4uC7HlUbOOaeLLgG7AKuB43y3jwX2Akk17J8N7Khw/1PgV77bFwMrKjzWDHBA+1rGMhc440Bl4VWGJUBKhcdfBJ6voeyngCd8t3v4XmfbWsbxRYXHHHAwEAsUA70rPPa3ivv6KXcHMNB3+9bK8VZ6L6cBv6nwWC/f8eKArr44OlV4/FvgnGqO2wzYCYz13X8ceNN3OwbvxGKgn+fdBLxRTZn7Yq3hfRp1gL93xfdjWfl77me/JcDxvttXA1NC/X+iiy666FKXC6pbaxNHY69b/wvc6Wf7f4C7K9xP9R2jKzAKWA4MB2IqPW8t8Gugebg/v7qE/qIWPAm2rc65wvI7ZtbMzB73dWfYCXwGZJhZbDXP31x+wzm3x3cz1d+OZvYLM5trXve/XLxfrCp2B6murI54FeHuCvuuOcDr+h8w3tdidCHwvnMup5Zx+NMGr0JYV10MZna9mS3xdcPIxfsV80DllutYqbw1vuO1q7Btc4Xbe6jmfQbOxKu0p/juvwCMMbM2vniS8H5prSyrmu21VfG9OdD7UdOx/gdc4Lt9AfBcA2ISEQkH1a3RV7fW6hjOuV14rXSZzrmP8VoMHwa2mNkTZtbct+tZeL1Y1pjXTfbwOh5XGjEleBJsrtL96/F+4TrMOdccr2sBQHVdQ2rFzLoAT+K1yLRyzmUAC2tZ7iaghZmlVNjWuaYnOOc+x/uCPQMvSXi2gXFsxUuasvzF4BsTcANeF40WvnLzKpRb+X2ubCNe152KZZcAWw7wPH8uwqug1po3/uNVIB6vS+Q2vK4wVWZCw6tg/W0H2I33y2+59n722fcaa/F+1HSs54EzzGwg0AeYXM1+IiKRSnVr9NWttTqG7/1sBWwAcM494JwbAhyC11Vzgm/7d865M4C2ePXcxADGJBFOCZ6EWhpeF75c82bHuiVA5abgfRFvBTCzS/B+3Tsg59waYCZwm5kl+Pr/n1aLpz4L/AOv3/vbDYnDOVcKTAJu9f0S25f9xyqk4VUaW4E4M7sZb2xZuS1A1/KxZX68BFxn3oD3VH4aV1BSi9e5j5ll4o0jOBWvC1A2MBDvfbjIOVcGPA3cZ2YdzZvs5HAzS8Rr6TvOzMabWZx5A9+zfUXPBcb5XvvBwC8PEMqB3o+ngDvMrId5BphZKwDn3Hq8MRfPAa875wrq8h6IiEQg1a3+Y2gUdWsFsWaWVOGSgNet9RIzy/bVpX8DvnHOrTazQ83sMDOLx/uhtBAo9b3f55tZunOuGG9YRWk9Y5JGSAmehNq/gGS8lp4ZwPuBKNQ5txj4J/A13hdyf+DLOhRxHt5A8R/xKsZna94dfPt0xvsyLwpAHFfjtYxtxuuL/0yFx6biTVCyHK+rRiH7dzl51Xe93cxm+yn7abyE5jNgle/5v61lXBVdCMx1zn3gnNtcfgEeAAaYN3vXH4AFeEnUj3gVdYxzbi1ed5Hrfdvn4iWHAPfjjbXYgtdF5wVqdqD34z68Xys/wKvY/oP3uSv3P7y/jbpnikg0+BeqW6vTGOrWcjfiJerll4+dc9OA/wNex2sV7Y43uyh4yeiTeOMG1+C1fpavGXghsNrXZfcKfhqaIE2AOXeg1mcRkehiZkfjddXs6mt1FBEREYkKasETkSbF15Xld8BTSu5EREQk2ijBE5Emw7z1h3Lx1hL6V1iDEREREQkCddEUERERERGJEmrBExERERERiRJK8ERERERERKJEXLgDqKvWrVu7rl27hjsMEREJgVmzZm1zzrUJdxyNhepIEZGmoab6sdEleF27dmXmzJnhDkNERELAzNaEO4bGRHWkiEjTUFP9qC6aIiIiIiIiUUIJnoiIiIiISJQIWoJnZk+bWY6ZLazmcTOzB8xshZnNN7PBwYpFRERERESkKQjmGLz/Ag8Bz1bz+Bigh+9yGPCo77rOiouLWb9+PYWFhfV5uviRlJREp06diI+PD3coIiIiItJI6Lw8sOpzTh60BM8595mZda1hlzOAZ5230voMM8swsw7OuU11Pdb69etJS0uja9eumFl9QxYf5xzbt29n/fr1dOvWLdzhiIiIiEgjofPywKnvOXk4x+BlAusq3F/v21aFmV1uZjPNbObWrVurPF5YWEirVq30IQoQM6NVq1b65UVERERE6kTn5YFT33PycCZ4/v7qzt+OzrknnHNDnXND27TxvxySPkSBpfdTREREROpD55GBU5/3MpwJ3nogq8L9TsDGMMXSILm5uTzyyCN1ft7JJ59Mbm5ujfvcfPPNfPTRR/WMTESkdibP2cCIuz6m243vMuKuj5k8Z0O4Q5JQmz8R7u8Ht2Z41/MnhjsiEZE603l5eBc6fwu42sxexptcJa8+4+8iQfkH6Te/+c1+20tLS4mNja32eVOmTDlg2bfffnuD4xOR6DV5zgbumbqMjbkFdMxIZsKJvRg7yG9v9xrLuGnSAgqKSwHYkFvATZMWANSrrIbGI2EwfyK8fQ0UF3j389Z59wEGjA9fXCIidaTz8iAmeGb2EnAs0NrM1gO3APEAzrnHgCnAycAKYA9wSbBiqSzQJyA33ngjK1euJDs7m/j4eFJTU+nQoQNz585l8eLFjB07lnXr1lFYWMjvfvc7Lr/8cgC6du3KzJkz2bVrF2PGjOHII4/kq6++IjMzkzfffJPk5GQuvvhiTj31VM4++2y6du3KRRddxNtvv01xcTGvvvoqvXv3ZuvWrZx33nls376dQw89lPfff59Zs2bRunXrQL1lIhJgwUrMbpw0n11FxRzftz17S8ooKXMUl5b5Lo6S0jL2lpZRUvrT9tveXrSvjHIFxaXc9vYizCDGjNgYI2a/20ZMjBFrRkyMt/2rldt4fPoPFJWU7YunvoliNDOzLLwZptsDZcATzrl/V9rnWOBNYJVv0yTnXPDOLKbd/lNyV664wNuuBE9Egkjn5YEXzFk0zz3A4w64KljHr04gf6kud9ddd7Fw4ULmzp3Lp59+yimnnMLChQv3zXbz9NNP07JlSwoKCjj00EM566yzaNWq1X5lfP/997z00ks8+eSTjB8/ntdff50LLrigyrFat27N7NmzeeSRR7j33nt56qmnuO222xg1ahQ33XQT77//Pk888US9XoeIHFgoW8wKi0vZml/E1l1FbM0vYpvvuvzyybIcikv3H7pcWFzGXyYv4i+TFzXwlcKOPcX87uW5DSqjoLiUe6YuU4K3vxLgeufcbDNLA2aZ2YfOucWV9vvcOXdqSCLKW1+37SIiAaDz8uAIZxfNoLjt7UUs3riz2sfnrM1lb2nZftsKikv542vzeenbtX6f07djc2457ZBaxzBs2LD9pjJ94IEHeOONNwBYt24d33//fZUPUrdu3cjOzgZgyJAhrF692m/Z48aN27fPpEmTAPjiiy/2lX/SSSfRokWLWscqEgzR2k2vuorIOcepAzvuayUrrtRCVlzhdkmZ4853F/ttMbtpkvc9VJ7Q5ReWVInBDFo2S6BNWmKV5K6iv53Zn/hYIz42hvjYGOJijQTftbfNu46LieHiZ74lJ7+oShntmify4mXDKStzlDkoLXOUObfv2ruNd7vMcd5T3/iNZWNugd/tTZVvOMIm3+18M1uCN4t05QQvdNI7ed0y/W0XEaknnZeH57w86hK8A6n8ITrQ9vpISUnZd/vTTz/lo48+4uuvv6ZZs2Yce+yxfqc6TUxM3Hc7NjaWggL/J0Tl+8XGxlJS4p38eY2hIpEhEsdz1bacsjLHj3v27msp++l6L1vzi5iyYNO+7oflCopLuW7iPK6bOK/OcVVWUFyGA/p0aM7RPRJpk+a7pP50u2VKAvGx3vxYI+76mA1+kqfMjGTOO6xzrY/7p5P77Pc3A0iOj+WmMX3o3ia11uVkZiT7jadjRnKty2hqfOvFDgL8ZceHm9k8vAnI/uCca3izbHVG37z/GLxy3Y4O2iFFRHReHhxRl+AdKKOv6YTolV8fXq9jpqWlkZ+f7/exvLw8WrRoQbNmzVi6dCkzZsyo1zFqcuSRRzJx4kRuuOEGPvjgA3bs2BHwY4jU1j/eX+q3deqOdxbTKtVLThLiYkiocB1fft+3LT7WeGf+pmoTxTOyO1Ja5ijxXUpLHSVlZfvul/haykpKHR8u2cyD01bsNy5swmvzeHveRjKaJeyXzG3fvZfSsqpfzIlxMbRJS6yS3FX0hxN6EudrLfupdcxIiPNaySq2pl37yhy27dpbpYzMjGQm1uF7aMKJvfwmZhNO7FXrMuCnxLuhyXSg4mkqzCwVeB241jlX+Sfu2UAX59wuMzsZmAz0qKacy4HLATp3rn1iv5/ycXbTbve6ZTbPhGYtYe4L0KYXjPhd/coVkSZN5+XhOS+PugTvQIJxAtKqVStGjBhBv379SE5Opl27dvseO+mkk3jssccYMGAAvXr1Yvjw4Q2K359bbrmFc889l1deeYVjjjmGDh06kJaWFvDjSPSrbUtXUUkpa7fvYdW23azevptV2366bNlZtasfwPbde7nwP982KL6C4lKufWUu174yt0HlFJc6pi3NoWN6Em3SEumQnsSATum09rWU/XTtdYVMTYzDzGqsiK4e5ffc26+/nNI3ohKz8rIa2o02kPFEOzOLx0vuXnDOTar8eMWEzzk3xcweMbPWzrltfvZ9AngCYOjQofX/6XjA+P0nVCkthjd+DR/eDEX5MPLPXh9hEZEA0Xl5cFgkNCPWxdChQ93MmTP327ZkyRL69OlT6zKibXxQUVERsbGxxMXF8fXXX3PllVcyd+7cBpdb1/dVwicYE3+A13J10eFdaJ+evF8itzG3gIoNXS1TEujWOoWurVL4cPFmdvoZO9YmNZFHLhhMcUkZRaVl7C3xxqRVvC4q8car7S0p4/6Pllcb6zWjDiYuNobYGCM+1oj1tZDFxhjxMd728rFmv3lhtt8yDFh11ykNfo+S42P5+7j+Wk4gSMxslnNuaLjjCCTzVq39H/Cjc+7aavZpD2xxzjkzGwa8hteiV2Ol7a+ObJCyUnjnWpj9LBx2BZz4d4gJ5xK6IhLpdF4e+PNyf+9pTfVjk2vBg8D8Uh1J1q5dy/jx4ykrKyMhIYEnn3wy3CFJCNU05u30gR3ZU1zKrsISdhUVs7OwxHfbu84vKiG/sJhdhSW8+O3aKl0ri0rKeOJzb5b2tKQ4urVOYUiXFpw1uBPdWqd4SV3rFNKT4yvE09pvEvTnU/pwaNeWtX5dE2euq7a17Pcn1P6XvUCOC4u0FjNptEYAFwILzGyub9ufgM6wbymhs4ErzawEKADOOVByFxQxsXDaA5CQBjMehr27vPsx1a8lJSJSF9FWH0bCeXmTTPCiTY8ePZgzZ064w5A6qs8vVruLSsjJLyJnZyFbdxWRs7OI+z5c7nfM23WvzOW6iXOpzSlhs4RY9uwt9fuYAd/95ThapSRgteieFWnjuQLd/SPaKiIJPefcF3j/WjXt8xDwUGgiOgAzOPGvkJgG0++Col0w7kmISwh3ZCIiEScSzsuV4ImEgb9Wtxten8/iTXn0aJtGToW1znLyC33XRdUmYf444JqRB5OaFEdqYjxpSXGkJsWRlhjn2xZHWlI8KQmxxMXGVDu+rGNGMq1TE6seoAaRNJ5L48JEAsAMRt4EianwwV+geA+MfxbiNUOqiEikUYInEkK7ikpYvHEnt7y10H93yM9W7buflhhHm+beFPn9O2XQ1jdN/k/X3gQhpz74ORtzq07xW9eujJE4A2KgWsvU6iYSIEf8FhJS4Z3r4Pmz4byXvZY9ERGJGErwROqotl0rd+zey6KNO1m4MY+FG/JYvHEnP2zbXWPZBkyfMJI2aYkkJ9RujMsfT+wdcTMyikgUG3qJl9RNuhyePQPOf81bUkFERCKCEjyROqhuQpOdBXvp1LIZCzfsZOGGPBZt3Llfd8dOLZLp1zGdcYMzOaRjOjdNWsDmnVVb3TpmJNO5VbM6xaSJP0Qk5PqfDfHN4NWL4b+nwoVvQFq7Az5NRESCT3Mdh0FqaioAGzdu5Oyzz/a7z7HHHsuBprr+17/+xZ49e/bdP/nkk8nNzQ1YnFLV3VP9L+J981uLufS/M7n/o+Ws2LqLwV1acNOY3rzwq8OYe/PxfHHDKB67cAhXj+rByN5tuXFMb5Lj92+ha+jEH1/eOIpVd53ClzeOUpLWlMyfCPf3g1szvOv5E8MdkTQVvU+G8yfCjlXwzBjIXRfuiERE6iwaz8vVghdGHTt25LXXXqv38//1r39xwQUX0KyZ1+IzZcqUQIUmQGFxKcu35LN4404Wb9rJkk07/Y51K/fqFYfTp0NzUhMP/G+l7pAhNn8iTLsd8tZDeicYffP+Czo3VvMnwtvXQLGvtThvnXcfouP1SeQ76Fi4cDK88DN4+iS46C1o1T3cUYmI1Fk0nZc3zQQvwCd7N9xwA126dOE3v/kNALfeeitmxmeffcaOHTsoLi7mzjvv5IwzztjveatXr+bUU09l4cKFFBQUcMkll7B48WL69OlDQcFP3fuuvPJKvvvuOwoKCjj77LO57bbbeOCBB9i4cSMjR46kdevWfPLJJ3Tt2pWZM2fSunVr7rvvPp5++mkAfvWrX3HttdeyevVqxowZw5FHHslXX31FZmYmb775JsnJTWMWtJrGzm3bVcSSTTv3JXOLN+5k5dZd+xbzTk2Mo0+HNFISYtntZybLzIzkOq3xBuoOGTLRnARNu/2n11WuuMDb3thfmzQenQ+Di9+B5870krwL34D2/cIdlYg0FjovD/h5edNL8IJwsnfOOedw7bXX7vsgTZw4kffff5/rrruO5s2bs23bNoYPH87pp59e7Tpijz76KM2aNWP+/PnMnz+fwYMH73vsr3/9Ky1btqS0tJTRo0czf/58rrnmGu677z4++eQTWrduvV9Zs2bN4plnnuGbb77BOcdhhx3GMcccQ4sWLfj+++956aWXePLJJxk/fjyvv/46F1xwQb1ed2Pib+zcH16dx+PTV7J9915y8ov27dsxPYm+HZszpl97+nRoTt+Ozclq0YyYGKtSDoR/pkmpRmkJ5CyC9/4YvUlQ3vq6bRcJlg4D4JL3vElXnjoOktJg19boajEXkcDTeXlQzsujL8F770bYvKD6x9d/B6VF+28rLoA3r4ZZ//P/nPb9Ycxd1RY5aNAgcnJy2LhxI1u3bqVFixZ06NCB6667js8++4yYmBg2bNjAli1baN++vd8yPvvsM665xvtADxgwgAEDBux7bOLEiTzxxBOUlJSwadMmFi9evN/jlX3xxReceeaZpKSkADBu3Dg+//xzTj/9dLp160Z2djYAQ4YMYfXq1dWWEy2cc9z57pIqY+dKyhzf5+zi9OyO9PUlcn3aN6dFSvWL96prZQTL3+z9f6//DtbPhI1zvLW6qtOYk6C89V6CSjWr2Kf5/54RqUltZwiuVpue3jIKU/8Eu6KwxVxE6k7n5WE5L4++BO9AKn+IDrS9ls4++2xee+01Nm/ezDnnnMMLL7zA1q1bmTVrFvHx8XTt2pXCwurHbwF+f0VYtWoV9957L9999x0tWrTg4osvPmA5zlVz0gckJv60YHVsbOx+Tc7R5vst+bw5dyNvztvAtl3+/76lZY77xmfXqVx1rQyh6rptlBTBpvkVErrvvBNJgJh46DAQBl8EnYZ6izLnb6pathksex96nRTa19QQRfnwxb/g64fAOeh5EvwwHUoq/R/v+RHmvgQDz/Fep8gBVDdDMFC377sZj1Dlh4doaTEXkcDTefm+24E8L4++BK+GjB7wZpnL8zPTV3oWXPJuvQ97zjnncNlll7Ft2zamT5/OxIkTadu2LfHx8XzyySesWbOmxucfffTRvPDCC4wcOZKFCxcyf/58AHbu3ElKSgrp6els2bKF9957j2OPPRaAtLQ08vPzqzQFH3300Vx88cXceOONOOd44403eO655+r92hqTjbkFvD1vI5PnbmTJpp3EGIw4uDW7CkvYsae4yv4dM5rG+MNGyV+3jclXwsd/g/wNULrX256e5SVyw6+ETodC+wEQn/RTOa5s/3IA4hKhWRt46ecw5GI44a+QmBqyl1ZnpSUw5zn45G+wOwf6nQ3H3QIZnasmwYdfBYvfgslXwIqP4NT7ICk93K9AItw9U5f5nSH4nqnL6pbgqduwiFSk8/KwnJdHX4J3IKNvrnqyF5/sbW+AQw45hPz8fDIzM+nQoQPnn38+p512GkOHDiU7O5vevXvX+Pwrr7ySSy65hAEDBpCdnc2wYcMAGDhwIIMGDeKQQw7hoIMOYsSIEfuec/nllzNmzBg6dOjAJ598sm/74MGDufjii/eV8atf/YpBgwZFbXfM3D17mbJgM2/O3cC3q3/EOcjOyuCW0/py6oCOtElL1Ni5xsjfBCJlJZC/8adkLnMoNO9QcznlrQaVWwL7nuElTF/+22sFO/Nxb7KISLPiI/jg/yBnMWQNh3Nf8hLacgPGV20ZGXY5fH4ffPp3WP8tjHsqMl+bRIyNuf5/Na5ue7XSO1VzstapHlGJSNTTeXlQWE3NhpFo6NChrvI6FEuWLKFPnz61LyRap0wPsDq/r0Hib1zIiYe056MlW3hz7kamL8+huNTRvU0KY7MzOT27I11apdSqHHW1jGC3ZuB/jJnBrbmBO86ar+CNX3vfB0deB8fcCHHVj8MMmS2Lve6lK6dBi65w/O3Q5/S6dblc9x28/kvvhPuYG+Go6yE2yL/rBfj71cxmOeeGHnhPAf91ZG2MuOtjNvhJ5jIzkvnyxlG1L6hyyztATByMfVT1rEgTofPywPP3ntZUPza9Fjzw/4u3RCR/40KunziPP8bMY2+po13zRC4+oitnZGdySMfm1c6GBBo71+iktIbdW6tuD3RLQJcj4IovYepN8Pk/4fsPYdyT0LbmX/eCZlcOfPJXmP0sJKZ53UeHXeZ1K62rrEPhii9gyh/g07/Byo/hrCe9rp3BEM1LUkS5CSf2Ckwvh8ot5nFJXjfpHscHMFoRiSo6Lw+4ppngSaNx9/tLq4wLKXWOxNhY/nfpoQzr1pLYGE0iEXVy18LePYCxXyteALpt+JXUHM54GHqdDG9dA48fDcfdCoddATExgT8eVP3F8tgbvZlAv7gfSgq9bpbH3ADN6ra+YhVJzWHcE9B9NLx7PTx6JJx2P/Q7KzCvo6IPb4neJSmiXPmPX3e/v5SNeYWkJsZy59j+9ftRrOLJ2uaF8NgImPEYjLwpgBGLiEh1lOBJxNmaX8Qny3KYtmQLG/P8z0xUsLeUw7u3CnFkESyaujcUF8ArF0BMLBx/B3z7eOheV+9TvLF9b13jtegtf8/rWhboVkN/LV1vXg046H0qHHcbtD44sMcc+HPIGgaTLoPXLoUV02DMP7xWwvpyzhsbuGwKLHvPGx/pjybYaBTKezmMe+RLYmMsMD0e2vfzPtMzHoXDf6MJf0REQiBqEjznXI3d86RuQjk20znHkk35TFuyhWlLc5i3PhfnoEN6Es0SYtmzt7TKczT7ZQXR1C3OOXjnOm8JhHNf9pYwGPHb0MaQ2tabyGT2s/D+TfDIEXDKvdD/Z4FbcsDfBDI4SGkD57wQmGP407Kbtxj19Lvh83th7ddw1lOQOaT2ZZTshTVfegnd8ve81lbwJrxJTIeivKrP0QQbjUp2Vgte/HYNxaVlxMcGoAX7mD/C0nfg2yfg6AkNL09EIp7OywOnPufkUZHgJSUlsX37dlq1aqUPUwA459i+fTtJSUkH3rmeCotL+XrldqYt3cLHS3L2tdQNzMrg98f1ZFSftvTt0Jw3527U7JcH4i9ZaKzd4r55HOa9BCP/HN716cxgyEXQ7Sh44wqv1WvZFDjlPm9Wy9q2lpaVQd5a2Locti2Hbctg2/f+ZxkE2L0teK+pXGw8jPozHHQsTLoc/nOC93437wgf3+n/dRXsgO8/8t6DFR9B0U5vbNVBI+GoP0DPE73F1f1NsBGsbrUSNNmdM3j6y1Us25xPv8wAtLh1GOit2fj1w16354a0GotIxNN5eeDU95w8KhK8Tp06sX79erZu9TMhg9RLUlISnTo17Ff3yrNW/vrobiTExfLRkhy+XLGNguJSmiXEclSP1lx7XE+O7d2Gtmn7f4DLuwhp9stqFOVXnyw0tm5xq7+AqX+CXqd4SUMkaHmQ1+L15b+9JRVWTPMWWS9fgLW8tbR0L3TI/imB2+q73v69N56uXLNW0LoXxKdA8e6qxwtlS1fXEXDlF/D2tTDtNrAYbzIM8F7XW7/1JpzJ3+TNNOpKIaUtHDLWG6vY7RhIaLZ/mdUtSdHYfmho4gZlZQAwd11uYBI8gKP/CE+Ngu+e8marFZGopfPywKrPOXlULJMgkceb/XI+BcVlVR7LzEhmdJ+2jO7TjsO6tSQpPjYMEUaB1V94C3+Xd5GrrFlr+OPK0MZUX3nr4fFjILkFXPaxNzFIpNk0H54c6a3FVyODFl2gdc+qlxTfuNHqWrpOeyD0yZBzcHc3r5XOn7Z9odcYL6nrODh4k85UQ8sk1E0g6kjnHEPv/IiRvdty788GBigy4PmzYOMcuHYBJFRdykZERGpPyyRISBUWl3Lr24v8Jndt0xL54oaRarJviOICmHYHzHjEWx/tmBvgqwcqddM02LPNa5054U5ITA1TsLVQPqlKSRGc82JkJncAHQZAWdXxoPuc/bTXOtequ5es1SSSWrrMoCC3ugfhN1+HMhqJAGZGdlYGc9flBrbgY26A/xwPM5+GI0I8tlZEpAlRgicBs37HHp6fsZZXvltL7p5iv/tszS9SctcQ62d5i3Jv/x4OvQyOv837JbzVwZWm3L8Jti6Brx7y1j4b+6jXJS/SOAfv/N77Vf+cl6BNz3BHVLP0Tv67xKZn1X3ZgUha96fa16XJUZqq7KwMPl6Ww87CYponxQem0Kxh3tjPLx+Aob+s2sVXREQCIrR9bSTqOOf4asU2fv3cTI6++xOe+Gwlh3VrRevUBL/7a/bLeirZ602A8Z/jvRavCyd7MzuWd3MaMB6uWwi35nrXg873Wu4umeK10Pz3FJj6Zz8zN4bZd0/BvBfhmBuh98nhjubARt9ctXUuGiYRidbXJfWW3TkD52D+Oj+zojbEMTfA7hyY/b/AlisiIvuoBU/qZXdRCW/M2cCzX69m+ZZdtGgWz6+P6c4Fw7uQmZHsG4On2S8DYssir9Vu8wIYeB6Muav2a0l1OQKu+BI+/D/4+iFv0owzH4PMwcGNuTbWfAXv3+jNrnfMDeGOpnYiqWtlIEXr65J6G9ApA4C563ZwZI/WgSu4yxHQ9Sj44l8w5BKID95szSIiTZUSPKmT1dt28+zXa3h11jryC0vol9mce84ewGkDO+43WYpmvwyAstKfZm9MzvDGp/U+pe7lJKbCqfd7z33zt/DUcXD0H7yZKuP8t7QGXd4GmPgLbwzhuCdCPnFHg0RS18pAitbXJfWSnhxP9zYpgR+HB966eP87DeY8B8MuC3z5IiJNnBI82U/lpQ0mnNiL0wd2ZPr3W3n2q9V8unwrsWac3L8DFx3RhcGdW1Q7pm7soEwldPW1faXXarf+O+h7Bpxy/08zMNbXwcd5E2a8dwNM/4e3UPWZj0O7voGJubaKC2HihV530YveqX1rpIiEVHZWC6Yvzwn8gsVdj4LOh8MX98PgX0BcYuDKFhERJXjyk8rdKjfkFjDhtXnc+e5itu3aS5u0RK4Z1YPzD+tM2+ZNuFvN/InB68pWVuaNS/vwZu+k56z/eJN3BOrkKjkDxj0OfU71Zth84hhvkesjfgsxIViuwjmYcj1smAU/fx7a9g7+MUWkXrI7Z/D67PWs31FAVssATohi5rXiPXcmzH0Bhl4auLJFREQJnvzknqnL9hszB1Bc6sgrKObf52Qzpl8HEuLC0JUumAlVfWKpuH5Z+ULXUPeYKr+uw6+CZVNg1Wdw8PFw+oPQvENg4y/X5zTIGg7vXAsf3eIdd+yjXuIVzPd65tMw53k4eoIXg4hErIoLngc0wQM4aCR0OhQ+vw+yLwhfd3ERkSikBE/22Zjrf4bFklLHGdlh6moZyIQqED66repMlMUF8NY13sLjKW18l9b7Xye3hNgK/27+Xtf7N0JsorfY9eBfBK7VrjqpbbxWtPkTYcoEePgwb3tZ8U8xBfK9XjvD6x7a4wRvGQcRiWi92qeRGBfD3HW5nDawY2ALN/MmV3rhbJj/svedJyIiAaEETwD4ZGkOMTFGaZmr8li9lzaobcubc1CYC/lbYNfm/a9nPe0/oZp2e2gTvC2LYNZ/Yed6/4+XFHhj2vZsB+dvMWyDZi29ZK9Za6+lrMRPQt2sJQy5KJCR18wMBv4cuh4JDw6GksL9Hw/Ue71zI7xyIWRkwbgnQ9MdVEQaJD42hv6Z6cGZaAW8ccEdB8Hn//RmCI7VKYmISCDo27SJW/fjHm57ezEfLdlCm7QE8vaUsLe0bN/j9V7awF8L1Zu/gSXvQmpryN8Mu7b4krktUFpUtYz4ZtWv25a3DlZMg27HBO+koLgAFr0BM5+B9d96rWvxzaB4T9V907O89efKyrxkdffWCpdtvkuF+/6SO/Del3BIz4QSP38D8N7rha97XTrT69GSW1LkzZi5dzdc9JY3DlBEGoXsrAyem7GG4tIy4mMD3EW/vBXvpXNgwauQfW5gyxcRaaKU4DVRhcWlPDZ9JY9+upLYGOPGMb25dEQ3pizYFJilDabdXjU5Ky2GJZMhKQPS2kNqO+hyuHddfj+tPaS2h9S2kJgG/+rvJRhVGDw/Dpq1gj6nQ79x0GVEYFqGcpbCrGdg3ktQmAetesAJf4Xs82DFR/snrrD/gtAxMV4rXLOW0KaGxPj+fv5fV3qnhsdfX+mdqn+vX/NNgpDeGTofBlmHQefh0Lbvgd/zKRO82UDHPwtt+wQ8bBEJnoFZGTz1xSqWbsqnf6cgzHjb8yRo3x8+u8frKaDWfRGRBlOC1wR9tHgLt7+zmLU/7uHUAR348yl96JDudcMMyNIGZaXVJAoABjeuqX1Zo2/2n1Cd/E9vev1Fk2D+K15CltoO+o71kr1Ow+q2tlpxASx+0+uGufZriE3wEsehl3iJY/l4uEAtCF3d6ypPFMOhuphOud9LVtfOgHUzYNXn3q/tAInNodNQb8rzrMO82wkpFbrn+j4HPcd4yz2ISKOSvW+ilR3BSfDM4Og/ekunLJwEA34W+GOIiDQxSvCakDXbd3Pb24v5eGkOB7dN5cVfHcYRB7cO7EH27oHXf1n943VtoTpQQtXnVK/r3/KpXrI3+3/w7ePQPBMOOdNL9joO9k4i/I0J7DDQS+rmvuh1rWzZHY6/w2utS6nmvQnEgtCBShQD6UAxZQ6Gw3/jjZnMXQNrv/GS4XXfeIux48Bivfc+fyOUlfxU9qpPvfdfC2mLNCqdWiTTOjWBOetyufDwIB2k96leb4DP7vG+s9WKJyLSIOZc1Uk1ItnQoUPdzJkzwx1Go1Kwt5RHP13BY5/9QHyMce1xPbl4RNfAj6fYtRVe+jlsnAMDfg6LJ1dtDTrtgeCe5Bfle5OdLJzkdacsK4YWXaFNb/jhk/3HmVkMuDKIifem7B9yMXQ7OvizV0ajglyvG+bar+Grh/yPqSwfpyhSB2Y2yzk3NNxxNBbBqCN/9b/v+GHbbj6+/tiAlrufhZPgtUvg7Ge8JE9ERGpUU/2oFrwo5pzjg8VbuP3txWzILeCM7I786eQ+tAvGIuXbvofnz4JdOd7U+71Pge6jQt9ClZj2UwtbwQ5Y+q534rD8/ar7ujKvm+fVs7wlA6T+kjOgx/He5fP7/O+TV80MpCIS0bKzMvhoSQ55e4pJbxYfnIP0PQNa9/Ja8fqOrVsXexER2Y8SvCgxec6G/SZHufiIrnyxYhvTl2+lV7s0Xr58OMMPahWcg6+d4c2CZrFw8TveOCwITFfGhkhuAYMu8C63ZgB+WqsLdyq5C7TqJmsJ5wQyIlJv2VktAJi3Ppejewbp+zImFo6eAJN+BUvfgb6nB+c4IiJNQFB/IjOzk8xsmZmtMLMb/TyebmZvm9k8M1tkZpcEM55oNXnOBm6atIANuQU4YENuAX+dsoQZK7fxf6f25Z1rjgxecrdoMvzvdG8h7199+FNyF2mqSy6UdATe6Ju97rgVhXsCGRGptwFZ6ZgRvPXwyvUbB60Ohul3e2N9RUSkXoKW4JlZLPAwMAboC5xrZn0r7XYVsNg5NxA4FvinmSUEK6Zodc/UZRQUV11cu0VKIr88slvgx9qBV/l+9RC8ejF0zIZffggtDwr8cQJFSUfoDBjvjbVMzwLMuw722EsRCZrmSfF0b5Ma/AQvJhaO+gNsWeCNpRYRkXoJZhfNYcAK59wPAGb2MnAGsLjCPg5IMzMDUoEfgZLKBUnNNub6XzR7y87C4BywrBTev8mbrbLP6TDuiarJU6SJxFkro1m4u+eKSEBlZ2Xw8dIcnHNYMCei6v8zmH4XTP8H9BqjSa9EROohmAleJlBxIM564LBK+zwEvAVsBNKAnzvnyoIYU1TqkJ7ExryqyVzHjCAkXXv3wKTLvDESw6+CE+5sPIPhlXSIiNRLdlYGr81az7ofC+jcqlnwDhQb57XivXU1fP8h9DwheMcSEYlSwTwz9/ezW+VO9ScCc4GOQDbwkJk1r1KQ2eVmNtPMZm7dujXQcTZ6XfxUtsnxsUw4sVdgD7R7G/zvNG9mypP+ASf9rfEkdyIiUm/lC57PWbcj+AcbeA6kd/Za8TQWT0SkzoJ5dr4eyKpwvxNeS11FlwCTnGcFsAroXbkg59wTzrmhzrmhbdpoxsOKJs1ez9c//MhxfdqSmZGMAZkZyfx9XH/GDsoM3IG2r4SnjoMtC+Hnz8HwKwJXtohIE2JmWWb2iZkt8U0w9js/+5iZPeCbpGy+mQ0OR6zlerdPIyk+Jvjj8ABi4+Go38OGmbDy4+AfT0QkygSzi+Z3QA8z6wZsAM4Bzqu0z1pgNPC5mbUDegE/BDGmqLJ8Sz5/fmMhh3VryWMXDCEuGJOpAKz7Fl78uTcW4qJ3IOvQ4BxHRKRpKAGud87NNrM0YJaZfeicqzhGfQzQw3c5DHiUqsMcQiYuNob+memhSfAAss+Dz+71ZtTsPkpj8URE6iBoLXjOuRLgamAqsASY6JxbZGZXmFl5888dwBFmtgCYBtzgnNsWrJiiya6iEq54fhYpiXE8eO6g4CV3i9/yumUmZ3gzZSq5ExFpEOfcJufcbN/tfLw6snKXizOAZ309XGYAGWbWIcSh7ic7K4NFG3eytyQEQ+XjEqHbUbBuBtzWAu7vB/MnBv+4IiJRIKgLnTvnpgBTKm17rMLtjYBGUNeRc46bJi1g9bbdvPCr4bRtnhS4wudP/GmmyaR0KMyFTofCuS9DSuvAHUdERDCzrsAg4JtKD/mbqCwT2BSayKrKzmrBk5+vYsmmnQz0jckLmvkTvXVWAXCQtw7evsa7q8myRERqpBkyGqHnZ6zh7Xkbuf6EXhzePYALmM+f6FWgeesA5yV3FguDL1ZyJyISYGaWCrwOXOuc21n5YT9P8TvjSKgmIsvunAGEYMFz8H5oLKm0BFBxgbddRERqpASvkZm/Ppc73lnCyF5tuPKY7oEtfNrtXgVakSv11iQSEZGAMbN4vOTuBefcJD+71GaiMiB0E5F1TE+iTVpiaBK8vPV12y4iIvsowWtE8vYU85sXZtMmLZH7xmcTExPgQeeqUEVEgs68lcL/Ayxxzt1XzW5vAb/wzaY5HMhzzoWteyaAmZGdlRGaBC+9k//tSVVWUhIRkUqU4DUSZWWO30+cy5adhTx8/mBapCQE/iDVVajVbRcRkfoYAVwIjDKzub7LyZUmIZuCN6v0CuBJ4DdhinU/2VkZrNq2m9w9e4N7oNE3Q3zy/tssFgrzvNk1RUSkWkGdZEUC5/HPfmDa0hxuPa3vvgVnA270zfDGr8FVmCEtPtnbLiIiAeGc+wL/Y+wq7uOAq0ITUe0N8tU/c9flcmyvtsE7UPlEKuWTfqV3gpF/gZXT4OM7vHrqmD8G7/giIo2YErxG4JsftnPvB8s4pX8HLjqia/AO1PIgr9JMyvB+JU3v5CV3mrFMRESA/p3SMQtBggde3VO5/hnwM4iJhU/+CmUlcOxNWiNPRKQSJXgRbmt+Eb99aQ5dWjbjrrP6Y8GsyGY8ConN4bqFkJgWvOOIiEijlJYUT4+2qaFb8LyymFg442Gvu+b0f0BZKYz6i5I8EZEKlOBFsNIyx+9ensPOwmKe/eUw0pLig3ewvA2weDIcdoWSOxERqVZ2VgYfLt6Ccy64PzpWJyYWTn8QYmLg83u92Z5H36IkT0TER5OsRLB/fbScr1Zu544z+tG7fZBnDvvuKa975rDLgnscERFp1LKzWrBjTzFrtu8JXxAxMXDqv2HopfDF/fDh/4Hzu0ygiEiToxa8CPXJshwe/HgF44d24mdDsw78hIbYuwdmPQO9ToYWXYN7LBERadSyK0y00rV1SvgCiYmBU+7zumt+9SCUlcGJf1VLnog0eWrBi0Abcgu47pW59G6fxu1n9Av+Aee/AgU7YHhEzMItIiIRrGe7VJLjY8M3Dq8iMzj5Hm94wYyH4f0b1ZInIk2eWvAizN6SMq56YTYlpY5HLxhCUnxscA/onDe5SvsB0OWI4B5LREQavbjYGPp3SmdOJCR44CV5J93lteTNeNibXXPMPV4Ln4hIE6QEL8L8/b0lzF2Xy6PnD6ZbKLq+rPwYti2DsY+pW4uIiNTKoKwMnvlyNUUlpSTGBfmHyNow87pnxsTCVw94s2uecp+SPBFpkpTgRZApCzbxzJeruWREV8b07xCag854FFLaQr9xoTmeiIg0etlZGewtLWPxxp0M6twi3OF4zOD42yEmDr64z5td89R/K8kTkSZHCV6YTZ6zgXumLmNjbgEAXVomc9OYPqE5+LbvYcWHcOyfIC4xNMcUEZFGL7tzBuBNtBIxCR54Sd7om72WvM/u8VryTn/Quy8i0kQowQujyXM2cNOkBRQUl+7btnlnEVMWbGLsoMzgB/DNYxCb4E0zLSIiUksd0pNp1zwxMiZaqczMW/w8Jg4+/buX5I19REmeiDQZSvDC6J6py/ZL7gCKSsq4Z+qy4Cd4BTtg7ovQfzyktgnusUREJOpkZ2VEZoJX7tgbvYlXPrkTfvwB8jdC3gZI7+S18g0YH+4IRUSCQh3Tw6i8W2ZttwfU7GeheA8MvyL4xxIRkagzMCuDNdv38OPuveEOpXrHTIC+Z8L6byFvPeAgbx28fQ3Mnxju6EREgkIJXhh1zEiu0/aAKS2Bb56ArkdB+/7BPZaIiESl8gXP50VyKx7AhplVtxUXwLTbQx+LiEgIKMELo98f36PKtuT4WCac2Cu4B176Nuxcr4XNRUSk3gZ0ysCMyO6mCb6WuzpsFxFp5JTghVFSvDcEslVKAgZkZiTz93H9gz/+bsaj0KIr9DwxuMcREZGolZoYR8+2aZGf4KV3qtt2EZFGTpOshNHzM9bQqUUy0yeMJDYmRIuMr58F676Bk+7SjGIiItIg2VkZTF28GeccZiGqx+pq9M3emLviCuPbY+K97SIiUUgteGGyIiefr3/YznmHdQ5dcgfwzaOQkAbZ54fumCIiEpWyO2eQu6eY1dv3hDuU6g0YD6c9AOlZgEFcMrgy6JAd7shERIJCCV6YPD9jLQmxMYwfmhW6g+7cCIvegMEXQlLz0B1XRESiUvlEK3PX7QhvIAcyYDxctxBuzYVr53t14FtXe2vkiYhEGSV4YbBnbwmvz1rPyf3b0zo1MXQH/u4przIbdnnojikiIlGrZ7s0miXEMndtbrhDqb3Utt4whXXfwLdPhjsaEZGAU4IXBm/O3Uh+UQkXDO8SuoMWF8DMZ6D3KdCyW+iOKyIiUSs2xuifmR75E61UNuDncPDxMO022LE63NGIiASUErwQc87x3Ndr6N0+jSFdWoTuwPMnQsGPMPzK0B1TRESiXnbnDBZv2klhcSPq7mgGp/0LLAbe/h04F+6IREQCRgleiM1Zl8viTTu58PAuoZtxzDlvaYT2/aHLiNAcU0REmoRBWRkUlzoWb9oZ7lDqJr0THH8b/PApzHk+3NGIiASMErwQe/7rNaQmxjE2O8hr3VX0w6ewdYm3sHmkTmMtIiKNUnaW1xulUY3DKzfkUu+Hz6l/hp2bwh2NiEhAKMELoR937+WdBZsYNziTlMQQLkE441FIaQP9zgrdMUVEpElon55E++ZJjW8cHkBMDJz+IJQWwbvXq6umiEQFJXgh9OrMdewtKQvt5CrbVsD3U+HQX0FcCGfsFBGRJiM7K6NxJngArbrDyD/Bsne9pYRERBo5JXghUlbmeOGbtQzr1pKe7dJCd+BvHoPYBBh6aeiOKSIiTUp25wzW/riH7buKwh1K/Qy/CjoOgikTYPf2cEcjItIgSvBC5LPvt7L2xz1cGMrWu4IdMPcF6P8zb90fERGRIChf8Hze+tywxlFvsXFwxsNQmAvv3xjuaEREGkQJXog8P2MNrVMTOfGQ9qE76OznoHgPHHZF6I4pIiJNTv/MdGKskU60Uq7dIXDU9bBgIiyfGu5oRETqTQleCKzfsYdpS3M459AsEuJC9JaXlsC3T0CXI6HDgNAcU0REmqSUxDh6tktjTmMdh1fuqD9Amz7wznVQ2MiWfRAR8VGCFwIvfbsWA849rHPoDrr0Hchbp4XNRUQkJAZ1zmDeulzKyhrxTJRxCV5XzfxN8OHN4Y5GRKRelOAFWVFJKa98t45RvduRmZEcugPPeBQyukCvMaE7poiINFnZWRnsLCxh1fbd4Q6lYToN8daNnfUMrPo83NGIiNSZErwge3/hZrbt2suFh4dwcpUNs2HdDG/sXUxs6I4rIiJNVqNe8LyykX+GFt3grd/C3j3hjkZEpE6U4AXZCzPW0qVVM446uHXoDvrNY5CQBoMuCN0xRUSkSTu4bSopCbGNdz28ihKaeQug71gFn/w13NGIiNSJErwgWrp5J9+u/pHzD+tMTIyF5qA7N8HCSV5yl9Q8NMcUEZEmLzbGGNCpES94Xlm3o2DIJTDjEVg/K9zRiIjUmhK8IHp+xhoS4mL42ZCs4B9s/kS4vx/c1xvKiiGjU/CPKSIiUkF25wyWbNpJYXFpuEMJjONvg9T28OZVULI33NGIiNSKErwg2VVUwhuzN3DagI60SEkI7sHmT4S3r/FmzSz38Z3edhERkRDJzsqgpMyxaGNeuEMJjKR0OO1fsHUJfP7PcEcjIlIrSvCC5I05G9i9t5QLhodgaYRpt0Nxwf7bigu87SIiIiEyKCsDgDnRMNFKuZ4nQv/x8Pm9sGVRuKMRETmguHAHEI2cczz/9Rr6ZTYn21fZBVXe+rptFxERCYK2zZPomJ4UPePwyp10F6z8GF46F1wp5G2A9E4w+mYYMD7c0YmI7CeoLXhmdpKZLTOzFWZ2YzX7HGtmc81skZlND2Y8oTJzzQ6WbcnnwuFdMAvB5Crp1Yy3q267iIhIkGR3jqKJVsqltIJ+Z0LuGt+Pp84bFvH2NRoOISIRJ2gJnpnFAg8DY4C+wLlm1rfSPhnAI8DpzrlDgJ8FK55Qeu7rNaQlxXHawI6hOeCo/wMqJZLxyd4viyIiIiGUnZXB+h0FbNtVFO5QAmvZ+1W3aTiEiESgYLbgDQNWOOd+cM7tBV4Gzqi0z3nAJOfcWgDnXE4Q4wmJrflFvLdwE2cP6USzhBD1gI1LABw0awUYpGfBaQ+o24iIiIRcVC14XpGGQ4hIIxHMDCQTqDCtI+uBwyrt0xOIN7NPgTTg3865Z4MYU9BNnLmO4lLHBcO7hOaAZWUw/R5o1QOu+gZiYkNzXBERqTczexo4FchxzvXz8/ixwJvAKt+mSc65RtFUtHrbLgB+9exMMjOSmXBiL8YOygxzVAGQ3mn/2aorbhcRiSDBbMHzN/jMVbofBwwBTgFOBP7PzHpWKcjscjObaWYzt27dGvhIA6S0zPHiN2s5onsrurdJDc1Bl70LOYvg6AlK7kREGo//AicdYJ/PnXPZvkujSO4mz9nALW8t3nd/Q24BN01awOQ5G8IYVYCMvtkb/lBRXJKGQ4hIxDlggmdmp5pZfRLB9UDFFb47ARv97PO+c263c24b8BkwsHJBzrknnHNDnXND27RpU49QQuOTpTlsyC3gwlC13jkH0/8BLQ+CfmeF5pgiItJgzrnPgB/DHUeg3TN1GQWVFjkvKC7lnqnLwhRRAA0Y7w1/SM/C+w3bIKML9Ds73JGJiOynNonbOcD3Zna3mfWpQ9nfAT3MrJuZJfjKeavSPm8CR5lZnJk1w+vCuaQOx4goz3+zhnbNEzmub7vQHHD5+7B5ARz1B4jVihciIlHmcDObZ2bvmdkh4Q6mNjbmFtRpe6MzYDxctxBuzYXT/g3blsGsp8MdlYjIfg6Y4DnnLgAGASuBZ8zsa1+XybQDPK8EuBqYipe0TXTOLTKzK8zsCt8+S4D3gfnAt8BTzrmFDXpFYbJm+26mL9/KOYd2Jj42BOvHl7feZXTRZCoiItFnNtDFOTcQeBCYXN2OkTSMoWNGcp22N2qDfwEHjYQPboYda8IdjYjIPrXKRJxzO4HX8WbC7ACcCcw2s98e4HlTnHM9nXPdnXN/9W17zDn3WIV97nHO9XXO9XPO/au+LyTcXvxmLTFmnDusc2gOuOIj2DgHjroeYuNDc0wREQkJ59xO59wu3+0peBOSta5m34gZxjDhxF4kx+8/HjwhLoYJJ/YKU0RBZAanP+hdv/Vb74dXEZEIUJsxeKeZ2RvAx0A8MMw5NwZvrNwfghxfo1BYXMrEmes4vk872qcnBf+AzsGnd3njAAaeG/zjiYhISJlZezMz3+1hePX19vBGdWBjB2Xy93H9ycxI9kapGfRulxods2j6k5EFJ9wBq6bDrP+GOxoREaB2yyT8DLjfNyB8H+fcHjO7NDhhNS5TFmxix55iLjw8RJOr/PAJbJgJp9znWwNPREQaEzN7CTgWaG1m64Fb8H5ExdfL5WzgSjMrAQqAc5xrHE1EYwdl7kvo/vH+Uh6fvpJ1P+4hq2WzMEcWJEMugUVvwAf/Bwcf5yV9IiJhVJsumrfgjY8DwMySzawrgHNuWpDialSem7GGg9qkcET3VsE/mHPw6T8grSMMuiD4xxMRkYBzzp3rnOvgnIt3znVyzv2n4hAG59xDzrlDnHMDnXPDnXNfhTvm+vjF4V0wM/731epwhxI8ZnD6Q+DK4O1r1FVTRMKuNgneq0BZhfulvm0CLNyQx5y1uZx/mFeJBd3qz2HdDDjyOohLDP7xRERE6qlDejIn9+/AK9+tY1dRSbjDCZ4WXeD422DlxzDnuXBHIyJNXG0SvDjn3N7yO77b6hfo88I3a0iKj+HswZ1Cc8Dpd0Nqe2/2LhERkQh36Yiu5BeV8NrMdeEOJbiG/hK6HgVT/wx5UbCwu4g0WrVJ8Laa2enld8zsDGBb8EJqHCbP2cDhf5/GS9+uI8aMT5blBP+ga77yWvBG/A7iQzCZi4iISAMN6tyCQZ0z+O9Xqykri+LuizExcPoDUFYCb/9OXTVFJGxqk+BdAfzJzNaa2TrgBuDXwQ0rsk2es4GbJi1gU14hAHv2lnLTpAVMnhPkX+ym3w0pbWDIxcE9joiISABdOqIbq7fvCc2PoeHU8iA47lZY8SHMfTHc0YhIE1Wbhc5XOueGA32Bvs65I5xzK4IfWuS6Z+oyCopL99tWUFzKPVOXBe+g6771Zs884hpIiNKZyEREJCqd1K897Zsn8fSXq8IdSvAdehl0PgLevwl2bgx3NCLSBNVqoXMzOwX4DXCdmd1sZjcHN6zItjG3oE7bA2L63dCsFQzVyhQiIpHEzFLMLMZ3u6eZnW5m8eGOK5LEx8bwiyO68OWK7SzdvDPc4QRXTAyc8RCU7oW3r1VXTREJudosdP4Y8HPgt4DhrYsXogXfIlPHjOQ6bW+wDbO87h6HXw2JqcE5hoiI1NdnQJKZZQLTgEuA/4Y1ogh07qGdSYqP4ZkvVoc7lOBr1R1G/x98PxXmvxLuaESkialNC94RzrlfADucc7cBhwNNehXPCSf2IjZm/yURkuNjmXBir+AccPo9kNwChl0WnPJFRKQhzDm3BxgHPOicOxNvWINU0CIlgXGDO/HG3A1s31UU7nCC77ArIOsweO8GyN8c7mhEpAmpTYJX6LveY2YdgWKgW/BCinxjB2XSJjWBxLgYDMjMSObv4/ozdlBm4A+2aR4sfw+GXwWJaYEvX0REGsrM7HDgfOBd37a4MMYTsS45oit7S8p48Zu14Q4l+GJi4YyHoaQQ3vm9umqKSMjUpgJ628wygHuA2YADngxmUJEur6CYzTuLuP74nvx2dI/gHmz63ZCYDoddHtzjiIhIfV0L3AS84ZxbZGYHAZ+EN6TI1KNdGkf3bMOzM9bw62O6kxBXq6kAGq/WPWDkn+HD/4OFr0P/s8MdkYg0ATV+s/oGjU9zzuU6517HG3vX2znXpCdZmbN2BwBDurQI7oE2L4Sl78DwKyEpPbjHEhGRenHOTXfOne6c+4ev3tzmnLsm3HFFqktHdGVrfhHvLmgiM0wefhV0OhSm/AF2RfkyESISEWpM8JxzZcA/K9wvcs7lBT2qCDd7zQ5iDAZmZQT3QJ/dAwlpMPyK4B5HRETqzcxeNLPmZpYCLAaWmdmEcMcVqY7u0YbubVL4zxercE2h22J5V829e+BdddUUkeCrTd+ID8zsLDOzA+/aNMxau4M+HZqTkhjEIRY5S2Dxm3DYr70JVkREJFL1dc7tBMYCU4DOwIVhjSiCxcQYl4zoxsINO5m5Zke4wwmNNr1g5E2w5G1Y9Ea4oxGRKFebBO/3wKtAkZntNLN8M4vyRWyqV1Jaxty1ucHvnvnZvRDfzOvaISIikSzet+7dWOBN51wx3nh1qca4wZmkJ8fz9BdNYOHzcof/FjoO9rpq7t4W7mhEJIodMMFzzqU552KccwnOuea++81DEVwkWrYln917S4Ob4G1d7g3GHnYZNGsZvOOIiEggPA6sBlKAz8ysC9BkfwitjWYJcZw7rDNTF21m3Y97wh1OaMTGwdhHoCjfS/JERIKkNgudH+3vEorgItFsX3eSwZ2DmOB9/k+IT/YWNhcRkYjmnHvAOZfpnDvZedYAI8MdV6T7xeFdMDOe/Xp1uEMJnbZ94JgbvG6adx8Et2bA/f1g/sRwRyYiUaQ2g8gqDhRPAoYBs4BRQYkows1as4O2aYl0apEcnANsXwkLJsLw30Bqm+AcQ0REAsbM0oFbgPIfP6cDtwNNflKymnTMSGZMv/a8/N06rj2uZ3DHtUeS5pmAwZ7t3v28dfC2b9LVAePDFpaIRI/adNE8rcLleKAfsCX4oUWmWWt3MKRLC4I258zn90FsAhyhGbZFRBqJp4F8YLzvshN4JqwRNRKXHtmN/MISXp+9PtyhhM4nf6XKEM3iAph2e1jCEZHoU58VRtfjJXlNTs7OQtb9WBC88Xc7VsO8l2DIJZDWLjjHEBGRQOvunLvFOfeD73IbcFC4g2oMBnduQXZWBs98uZqysiYyL01eNclsddtFROqoNmPwHjSzB3yXh4DPgXnBDy3yzPYtcD44WAne5/dBTByM+F1wyhcRkWAoMLMjy++Y2QigIIzxNCqXjOjKqm27+XR5E1kEPL2T/+3JGVojT0QCojYteDPxxtzNAr4GbnDOXRDUqCLUrDU7SIiL4ZCOQZhENHctzH0RBv8CmncIfPkiIhIsVwAPm9lqM1sNPAT8OrwhNR4n9+9Au+aJPP3F6nCHEhqjb/YmUqvIYqBgB7x2KRRqAlYRaZjajGh+DSh0zpUCmFmsmTVzzjWReY1/MnttLgMy00mMiw1cofMnev3u89Z591sdHLiyRUQk6Jxz84CBZtbcd3+nmV0LzA9rYI1EfGwMvzi8K/dMXcayzfn0ap8W7pCCq3wilWm3e90y0zvBqP+Dnevh47/Cprnws/9Ch4HhjFJEGrHatOBNAyr+1JQMfBSccCJXUUkpC9bnBXb83fyJ3sxZ5ckdwLRbNV2yiEgj5Jzb6Zwrb375fViDaWTOG9aZxLgYnvmyiSx8PmA8XLcQbs31rgf+HI66Hi5+F4oL4anj4Lun1GVTROqlNgleknNuV/kd3+1mwQspMi3csJO9pWWBHX837XZv5qyKNJOWiEg0CNJUy9GpRUoC4wZ3YtKcDWzfVRTucMKny+FwxRfQ7Rh493p49WIo1GobIlI3tUnwdpvZ4PI7ZjaEJjh4PCgLnGsmLRGRaKWmlzq6dERX9paU8dK3a8MdSniltILzJsJxt8GSt+Hxo2HjnHBHJSKNSG0SvGuBV83sczP7HHgFuDqoUUWgWWt20KVVM9qkJQau0Opm0qpuu4iIRAwzyzeznX4u+UDHcMfX2PRol8ZRPVrz7Ndr2FtSFu5wwismBo68Fi6ZAqXF8J8T4Jsn1GVTRGqlNgudfwf0Bq4EfgP0cc7NCnZgkcQ55y1wHsjWO/Bm0oqtlDDGJ3vbRUQkojnn0pxzzf1c0pxztZnETCq59Mhu5OQXMWXBpnCHEhk6D/e6bB40Et6bABMvhILccEclIhGuNuvgXQWkOOcWOucWAKlm9pvghxY51u8oYGt+UeDXvxswHg4503fHID0LTnvgpxm2REREmpBjerThoDYpPP3lKpxaqzzNWsK5L8Pxd8Cy97wumxua1O/sIlJHtemieZlzLrf8jnNuB3BZ0CKKQLN84+8COoNmufgkSG4Bt+zwZtJSciciIk1UTIxxyYhuzF+ft6/uFbwumyOugUveB1cG/zkRZjyqLpsi4ldtErwYM9s3G5iZxQIJwQsp8sxas4PUxDh6tgvC2jw5S6BtXzBNuCYiInLW4EyaJ8XxdFNZMqEusg6FX38GPY6H92+EJ46F+/rCrRlwfz8tsyQiQO0SvKnARDMbbWajgJeA94IbVmSZtWYH2VkZxMYEOAlzzpfg9QlsuSIiIo1Us4Q4zj2sM+8v3Mz6HXvCHU7kadYSznkR+v/MWxR95wbAeWvqvn2NkjwRqVWCdwPeYudXAlcB89l/4fOotquohKWbdwZ+/B14yyEU7VSCJyIiUsEvDu+KmfHc12vCHUpkMoO1M6pu11q6IkLtZtEsA2YAPwBDgdHAkiDHFTHmrculzAVp/F2O721se0jgyxYREWmkMjOS6Z/ZnCc++4FuN77LiLs+ZvKcDeEOK7JUu5buOijQ+EWRpqzaBM/MeprZzWa2BHgIWAfgnBvpnHsoVAGG26w1OzCD7KyMwBees9i7bts78GWLiIg0UpPnbGDJpnwc3orxG3ILuGnSAiV5FdW0Zu6/s+HLB6C4MGThiEjkqKkFbylea91pzrkjnXMPAqWhCStyzFqzg55t00hPjg984TlLIK2jN4umiIiIAHDP1GUUVVrsvKC4lHumLgtTRBFo9M3e2rkVla+l22kofPh/8NBQmPsSlDW50zeRJq2mBO8sYDPwiZk9aWajgSY11WNZmWP22h3BGX8HXguext+JiIjsZ2NuQZ22N0kDxntr56Znsd9aukddDxe8Dr94C5q1gslXwGNHwfcfalkFkSYirroHnHNvAG+YWQowFrgOaGdmjwJvOOc+CE2I4bNi6y7yC0uCM/6utAS2LoNuRwe+bBERkUasY0YyG/wkcx0zmswcb7UzYHz16+cedAxc9gksfsObeOWFs6HrUXD8bZA5JLRxikhI1WaSld3OuRecc6cCnYC5wI3BDiwSBHWB8x2roLQI2mmCFRERkYomnNiL5PjY/bYZ8LvRB4cnoMYqJgb6nQVXfQdj7vGGhjw5Cl69GLavDHd0IhIktVkmYR/n3I/Oucedc6OCFVAkmbVmBy1TEujaqlngC983wYq6aIqIiFQ0dlAmfx/Xn8yMZAxonZqAA2atyQ1zZI1UXAIcdjlcMweOuQGWfwAPD4N3/wC7tnpr593fTwumi0SJartoCsxes4PBnVtgFoShhzlLAIPWvQJftoiISCM3dlAmYwdl7rt/z9SlPPzJSkb0aM3pAzuGMbJGLKk5jPwTDP0lTP8HzHwaZj8LrhTKSrx9yhdMh+q7f4pIRKtTC15T8uPuvfywbXdwumeC14LXshskBKF1UEREJMpce1xPBnfO4E+TFrB2+55wh9O4pbWDU++Dq74Fi/kpuSunBdNFGrWgJnhmdpKZLTOzFWZW7bg9MzvUzErN7OxgxlMXs4M5/g5gy2Jo2zc4ZYuIiESZ+NgY/n3OIMzgty/Pobi07MBPkpq1PhhKqlkrr7qF1EUk4gUtwTOzWOBhYAzQFzjXzKpkNL79/gFMDVYs9TFr7Q7iYowBndIDX3hxIfy4UgmeiEgTZWZPm1mOmS2s5nEzswd8P5DON7PBoY4xEmW1bMY/zhrAvHW53PuB1sQLiOoWTE/P9L9dRCJeMFvwhgErnHM/OOf2Ai8DZ/jZ77fA60BOEGOps1lrdnBIZjpJlWbxCohty8GVaYIVEZGm67/ASTU8Pgbo4btcDjwagpgahZP7d+DcYZ15fPoPfLZ8a7jDafz8LZgO0LK71s0TaaSCmeBlAusq3F/v27aPmWUCZwKPBTGOOisuLWPeulyGdA7W+Lsl3rVa8EREmiTn3GfAjzXscgbwrPPMADLMrENooot8N5/al57tUvn9xHlszS8KdziNm78F03ucCKumw5QJSvJEGqFgJnj+pp6s/C3xL+AG51xpjQWZXW5mM81s5tatwf+1bvHGnRSVlAV3gpWYeGjVPTjli4hIY3fAH0mbsuSEWB48dzD5hcX8fuJcysqUhDTIgPFw3UK4Nde7Pu8VOPxq+O5JeO8GJXkijUwwE7z1QFaF+52AjZX2GQq8bGargbOBR8xsbOWCnHNPOOeGOueGtmnTJkjh/qR8gfPBXTKCc4CcxdC6J8TGB6d8ERFp7GrzI6m3Y4h/BI0Uvdqn8X+n9uXz77fx5Oc/hDuc6GIGJ9wJw6+Cbx+H929SkifSiARzHbzvgB5m1g3YAJwDnFdxB+dct/LbZvZf4B3n3OQgxlQrs9buIDMjmQ7pfvqkB0LOEug8PDhli4hINKjNj6SA9yMo8ATA0KFDm9RZ+PmHdebLFdu4Z+oyDjuoFdlZGeEOKXqYwYl/BRzMeMR3/2/etYhEtKC14DnnSoCr8WbHXAJMdM4tMrMrzOyKYB03EGav2cHgYHXPLNzpLSKqCVZERKR6bwG/8M2mORzIc85tCndQkcbMuGvcANo1T+Kal+aQX1gc7pCiS3lSd9gVXpL3wV/UkifSCASzBQ/n3BRgSqVtfidUcc5dHMxYamtjbgGb8goZ0jkjOAfYutS71gQrIiJNlpm9BBwLtDaz9cAtQDzsqyenACcDK4A9wCXhiTTypTeL59/nZPPzJ2bwpzcW8sA52ZhamQLHDE66y5v9++uHvPvH36GWPJEIFtQErzGavbZ8gfOWwTlAzmLvWi14IiJNlnPu3AM87oCrQhROoze0a0uuO64H936wnKMObs34Q7MO/CSpPTMYc7eX5H31IGBw/O1K8kQilBK8Smat2UFyfCy9O6QF5wBbFkN8CqR3Dk75IiIiTdCVxx7Mlyu2c8tbixjcJYOD2wapHm+qzODke70uml89ABYDx92qJE8kAgVzFs1GafaaHQzMSic+NkhvTc5iaNsbYvTWi4iIBEpsjPGvc7JJTojl6hfnUFhc4wpMUh/lSd6QS+DLf8G02zUmTyQCKcuooGBvKYs27gze+nfgzaCp8XciIiIB1655Evf+bABLN+fz9ylLwh1OdIqJgVPugyEXwxf3wcd3KskTiTDqolnB/PW5lJS54CV4u7bCnm1K8ERERIJkVO92XDqiG09/uYoRB7fmhEPahzuk6BMTA6fc743J+/xeiImFkX8Kd1Qi4qMEr4JZvglWBmUFKcHTBCsiIiJBd8OYXnyzajt/fH0+/TulB29d26YsJgZO/beX5E3/hzcm79gbwx2ViKAumvuZvWYH3duk0CIlITgHyPF1F1ELnoiISNAkxsXy4LmD2FtSxu9enktpmboQBkVMDJz2IGSfD5/+HT79R7gjEhHUgrePc45Za3ZwfN92wTtIziJIbgmpbYN3DBEREeGgNqnccUY/rn91Hle9MIsFG3ayMbeAjhnJTDixF2MHZYY7xOgQEwOnP+iNw/v0b95s4RtnQd56SO8Eo2+GAePDHaVIk6IEz2fVtt3s2FMc/AlW2h2iKYVFRERC4KwhnXjp2zW8v2jLvm0bcgu4adICACV5gRITC2c8BD+ugiWTf9qetw7evsa7rSRPJGTURdNn1pryBc6DlOA555tBU+PvREREQmVDbmGVbQXFpdwzdVkYooliMbGwc13V7cUF3nIKIhIySvB8Zq/dQXpyPAe1Tg3OAfLWwd5dSvBERERCaHNe1QQPYGNuQYgjaQLyNlSzfX1o4xBp4pTg+cxas4PBnTOIiQlS90lNsCIiIhJyHTP8z6BZ3XZpgPRO1Tzg4NkzYPWXIQ1HpKlSggfkFRSzfMuu4I6/27LIu27TO3jHEBERkf1MOLEXyfGxVbafNrBDGKKJcqNvhvhKiXNcMvQ725t85b8nw9NjYOXHWhxdJIg0yQowx7f+3eBgT7DSvBMkZwTvGCIiIrKf8olU7pm6jI25BbRrnoTheOrzVRzcNo2zh1TX6iR1Vj6RyrTbq86iWVwAs5+FL/4Fz50JmUPh6AnQ80RNPicSYErw8Na/i40xBnbKCN5BNMGKiIhIWIwdlLnfjJn5hcVc8fws/vDqPDblFnD1qIMxJRmBMWC8/xkz45PhsF/DkIth7ovwxX3w0s+hfX8v0et9mrfkgog0mP6TgFlrd9CnQxopiUHKd0tLYNsyJXgiIiIRIC0pnmcuHsaZgzL554fL+dMbCykpLQt3WE1DXCIMvQR+OxvGPuq17E38BTx6BCx4DcpKwx2hSKPX5BO8ktIy5q7NZUjnIHbP/PEHKN2rCVZEREQiREJcDPeNH8hVI7vz0rdrufy5WezZWxLusJqO2HjIPg+u+hbO+o+37fVfwkOHwpwXYO5LcH8/uDXDu54/MazhijQmTT7BW7Yln917S4M8/s43wYpa8ERERCKGmTHhxN7cObYfny7L4ZwnZrBtV1G4w2paYmKh/9lw5Vcw/jlISIE3fwOTr/SWmML9tGC6kjyRWmnyCd5s3wLng4PZgpezBCwG2vQK3jFERESkXi4Y3oXHLxzK8i35jHvkK1Zt2x3ukJqemBjoezr8+jNo1hqoNMumFkwXqbUmn+DNWrODtmmJdGoRxPVwchZDy4OqTh0sIiIiEeH4vu146bLh7CoqYdwjXzLbN8O2hJgZ7Nnu/7G8dd5aelpiQaRGSvDW7mBIlxbBnT1LM2iKiIhEvEGdWzDpyiNonhzPuU/M4INFm8MdUtNU7YLp5q2l99hRMOt/sHdPSMMSaSyadIKXs7OQdT8WBHeB8+ICb5IVTbAiIiIS8bq2TuH1K4+gd4fmXPH8LJ77enW4Q2p6/C2YHp8Mpz8Epz0AOG9M3n194IO/wI+rwhKmSKRq0gne7FAscL51GbgyteCJiIg0Eq1TE3npssMY1bst//fmIu56byllZeoWGDIDxnuJXHoWYN71aQ/A4AtgyEVwxRdwyXvQfSR8/Qg8MAhePAdWTIMyLXch0qQXOp+1ZgcJcTEc0rF58A6Ss8S7VgueiIhIo9EsIY7HLhjCLW8t4rHpK9mUV8BRB7fm/o++Z2NuAR0zkplwYq/9FlCXAKpuwXTwxul1OcK77NwIM5+GWf+F59+DVj1g2GUw8FxIau7NvDntdshb73X9HH1z9eWKRIkmn+ANyEwnMS42eAfJWQyxCdCye/COISIiIgEXFxvDnWP70TEjmXumLuPteRspb8jbkFvATZMWACjJC6fmHWHUX+DoCbBoMnz7BLz3Ry+p63QorP0aSgq9fcuXWwAleRLVmmwXzcLiUhZu2Bnc8XfgteC17gWxTTqXFhERaZTMjKtGHkyLZvFU7qVZUFzKPVOXhScw2V9cIgz8OVw2DS77GHqfCj988lNyV07LLUgT0GQTvEUb89hbWhbc8XegGTRFRESiQO6eYr/bN+YWhDgSOaDMITDucaCaGdLz1oc0HJFQa7IJ3qxQLHBemAc71yvBExERaeQ6Zvhfy7Zd86QQRyK1Vu1yCw6eGAnfPQUFuaGMSCQkmnSC16VVM9qkJQbvIJpgRUREJCpMOLEXyfFVx+zn7ili0uz1OC2+HXn8LbcQlwz9x0PpXnj3eri3J7z2S1j5MZSVhidOkQBrkgPDnHPMXpvLUQe3Du6BchZ71+2U4ImIiDRm5ROp3DN12b5ZNC8+oitTF23m9xPnMWXBJv52Zn/aqkUvcpRPpOJvFk3nYNM8mPM8LHgVFr4GzTtB9rmQfR60PCi8sYs0QJNM8NbvKGBrflFoxt8lpPrWcREREZHGbOygzCozZl56ZDee+XIV90xdxvH3f8atp/dlbHYmZtWM/5LQqm65BTPomO1dTrgTlk3xkr3P7oXP7oEuR8Kg86HvGZCQouUWpFFpkgle+fi7kMyg2baP9yUiIiIiUSc2xvjVUQcxsndb/vjafK57ZR7vzt/M387sp9a8xiI+CfqN8y55G2DeSzD3BZh8JUyZAB0GwvqZUFrk7a/lFiTCNckxeLPW7CA1MY6e7dKCdxDnYMsiTbAiIiLSBHRvk8rEXx/OX07pw+ffb+X4+z9j8pwNGpvX2KRnwtF/gN/Ohkveg75jYc1XPyV35YoLYNptYQlR5ECaVII3ec4GRtz1Mc/NWENxaRlvz9sYvIPtyoGCHzXBioiISBNR3po35XdH0b1NCte+MpfLn5tFTn7hgZ8skcUMuhwBYx+ufp+89fDYkfD6ZfD5P2Hpu7B9Zc2TtcyfCPf3g1szvOv5EwMeukiT6aI5ec4Gbpq0gIJi75+uqKSMmyYtAKjSnz4gyidYUYInIiLSpHRvk8qrVxzBf774gXs/WM4J93/GbacfwukDO2psXmOU3snrlllZQhqktvNa+BZUSNTikqB1T2jTG9r2hjZ9vOt138I713qtf6CunhI0TSbBu2fqsn3JXbmC4lLumbosSAmelkgQERFpqmJjjMuP7s6o3u2Y8No8fvfyXN6dv4m/ntk/uEs0SeCNvtlLxIorLGofnwyn3vdTYla4E7Yt987/ti71LpUTPwyo1GW3uMCbvEUJngRQk0nwNuYW1Gl7g+UshmatIbVNcMoXERGRiHdw21Reu+IInvr8B/754XKOv386pw3owMdLc9iYW0jHjGQmnNgrOD82S2DUtNxCuaTm0Gmod6mocCdsXeYlfG9d7b/8vPXBiVuarCaT4HXMSGaDn2SuY0ayn70DoHwGTREREWnSYmOMXx/TndF92nLpf7/juRlr9z22IbcguENGJDCqW27hQJKaQ9ah3mX6P/x39Uzv1PD4RCpoMpOsTDixF8nxsfttS46PZcKJvQJ/sLIyX4Kn7pkiIiLiObhtGiVlVWfVLB8yIlFu9M1e187KBpwT+lgkqjWZBG/soEz+Pq4/mRnJGJCZkczfx/UPzq9leWuheDe0U4InIiIiP9mU639GzQ25BcEbNiKRYcB4OO0BSM8CDJpnQko7mPWMumlKQDWZLprgJXkh6f6gCVZERETEj+qGjAAcc88nnDW4E1ce250urVJCHJmEROWuntu+hydHwcvnw6Xv+2/hE6mjJtOCF1LlSyS06R3eOERERCSiVDdk5OZT+3DusM5MmrOBkfd+ynWvzOX7LflhilJCpnUPGPckbJoHb/8OXNUuvCJ11aRa8EImZ4nX/J7UPNyRiIiISAQp70l0z9RlbMwtqDKL5tUjD+bJz3/ghW/WMnnuBk46pD1XjTyYfpnp4QxbgqnXSTDyz/DJndAhGw7/TbgjkkZOCV4wbFmsGTRFRKRaZnYS8G8gFnjKOXdXpcePBd4EVvk2TXLO3R7KGCV4ahoy0rZ5En8+pS9XHnswz3y5iv9+uZr3Fm5mZK82XD2qB0O6tAhxtBISR10Pm+fBB3/x5nA46NhwRySNWFC7aJrZSWa2zMxWmNmNfh4/38zm+y5fmdnAYMYTEqXF3kKXGn8nIiJ+mFks8DAwBugLnGtm/iqNz51z2b6LkrsmpmVKAtef0IsvbxrFhBN7MXddLmc9+hXnPTmDr1ZuwznH5DkbGHHXx3S78V1G3PUxk+dsCHfYUl8xMTD2UWjdE169BHasDndE0ogFrQWvQgV2PLAe+M7M3nLOLa6w2yrgGOfcDjMbAzwBHBasmEJi+0ooK1aCJyIi1RkGrHDO/QBgZi8DZwCLa3yWNEnNk+K5auTBXHxEV176di2Pf/YD5z35DV1aJrMpr4i9pWWA1tOLColpcM4L8ORIePkC+OVUSNBkO1J3wWzB21eBOef2AuUV2D7Oua+cczt8d2cAjX+lx/IJVtRFU0RE/MsEKq52vN63rbLDzWyemb1nZoeEJjSJVCmJcfzqqIP4/I8juWNsP9bvKNyX3JXTenpRoFV3OOtp2LIQ3rxak65IvQQzwattBVbul8B7QYwnNHKWgMV4TewiIiJVmZ9tlc/iZgNdnHMDgQeBydUWZna5mc00s5lbt24NXJQSkZLiY7lweBfKqjnx11p6UaDHcXDcLbBoEnz573BHI41QMBO82lRg3o5mI/ESvBuqebzxVF45i6Fld4hPCnckIiISmdYDWRXudwI2VtzBObfTObfLd3sKEG9mrf0V5px7wjk31Dk3tE2bNsGKWSJMx4zq10u7/e3FrN2+J4TRSMCNuBYOORM+uhW+/yjc0UgjE8wE74AVGICZDQCeAs5wzm33V1CjqrxyNIOmiIjU6Dugh5l1M7ME4BzgrYo7mFl7MzPf7WF49bXfOlKaJn/r6SXGxTC4cwbPfr2aY+/9hF8/N5NvftiOUze/xscMzngY2h0Cr1/qzfEgUkvBXCZhXwUGbMCrwM6ruIOZdQYmARc655YHMZbQ2LsHflwFA34e7khERCRCOedKzOxqYCreMglPO+cWmdkVvscfA84GrjSzEqAAOMfpLF0qqGk9vc15hTz79Wpe/HYtUxdtoV9mc355ZDdO6d+RhLigTqAugZSQ4k268sSx8PL58KsPvYlYRA7AgllfmNnJwL/4qQL7a8UKzMyeAs4C1vieUuKcG1pTmUOHDnUzZ84MWswNsnGO9084/lnoe8YBdxcRkZqZ2awD1Qvyk4iuIyXkCvaWMmnOep7+YhUrt+6mbVoivzi8C+cd1oWWKQnhDk9q64dP4bkzodfJMP45b0kFafJqqh+DutC5b9zAlErbHqtw+1fAr4IZQ0jlLPGutUSCiIiIhFlyQiznH9aFcw/tzGffb+U/X6zi3g+W8+DHKxg3OJNLR3SjR7s0Js/Z4LclUCLEQcfCCXfC1D/B5/+EYyaEOyL/5k+EabdD3npI7wSjb4YB48MdVZMU1ASvyclZDLGJ0KJbuCMRERERASAmxji2V1uO7dWW5VvyeebLVUyavYGXvl1Hr/ZprNq6W+vpRbrhv4FN8+CTv0L7ftBrTLgj2t/8ifD2NVDsm8U1b513H5TkhYHaeANpy2Jo0xNilTeLiIhI5OnZLo2/jxvAVzeO4g8n9OT7LflaT68xMIPT/g0dBsCky2FrhE1d8dFtPyV35YoL4P2bYOcmrecXYkrwAilnCbTVWrQiIiIS2VqlJnL1qB7VnndvyC1gz96S0AYlNYtPhp+/ALEJ8PJ5UJgX3nic8+afePd62Lne/z57tsF9veGeg+HZM+CDv8C8V7xGkdJi/8+ZPxHu7we3ZnjX8ycG7SVEKzU1BUrBDsjfqCUSREREpNHomJHMhmoWRx9yx0eM7tOW0wZ25JiebUiqtCyDhEFGFoz/H/zvdPjvqVDwI+RtCO2Yt93bYcFEmPM8bFkIcUkQ3wyK/ay9mNIWjroetiyAzQvgm8ehdK/3WGyCd97crj+07+91Pd22AqbeGL1dPUM0TlEJXqDkLPWuNcGKiIiINBITTuzFTZMWUFBcum9bcnwMvzyyG7kFxUxZsJl35m8iLTGOEw5pz2kDOzDi4NbEx6oTWNh0PRL6j4f5L/20LdiJUGkJrPwY5jwHy96DsmLoOBhOuQ/6nQXff7D/GDzwWhxP/Ov+8ZQWw7bvvWSvPOlb/h7Mfb76YxcXeElRY0/wQjhOUQleoOQs9q7VgiciIiKNRE3r6QHcetohfLVyO2/P28j7izbz+uz1tGgWz5j+HThtQEeGdWtJbIyF8yU0TWu+qLqtuACm/hm6j4aUVoE5zvaVXkvdvJcgfxM0awXDLodB53uLsJcrT1AO1DoVGw/t+noXfOtGOwf5m73WwBfO9h9HXjVdQBuTadWMUwxC8qoEL1ByFkNic+8DLSIiItJIjB2UWe2MmXGxMRzdsw1H92zDnWf247Pl23h73kbemL2BF79ZS9u0RE4Z0IHTBnZkUFYGb87dqCUXQqG6hGd3DtxzECS3hDa9oHVP71J+Oz2r6jp6lbsNHvNHsFgvsVv7FVgMHHw8jLkbep4EcdWsoThgfP0SFTNo3sG7pGd5LVuVJaZCSRHEJda9/HApLoD1M2HNl96lur9ZEJJXJXiBkrPEa70z/YolIiIi0ScxLpbj+7bj+L7t2LO3hI+X5vD2vI288M1anvlyNS2axZNfWEJJmTdzi5ZcCKL0Tv4ToWat4cjrYNty77LkbW+cXrm4ZGh9MLTu5SV9u7fC7GehpNB7PG8dvPVb73bL7jD6Fhh4rpd8hcLom6t29bRYKMqHJ0bCmY9Ch4GhiaWuivJh3Tew5itY/SVsmOV1ZcW88YUJqbB3V9XnBaFxSAleIDjnteD1PSPckYiIiIgEXbOEOE4d0JFTB3RkZ2ExHy7awp/eWLAvuStXUFzK3VOXKsELNH+JUHwynPT3qq1ou7d5yd7WZT8lfuu+hYWvVV9+Slv47azQN1xU19Uzsbn3ep8cBUdP8CZuiY0PXVz+JkfpcTys+drXQveVt06hK/US0o6DYPiV0GUEdB4OyRlVx+CB9zcbfXPAw1WCFwi7tnizaGqCFREREWlimifFc9aQTvzh1Xl+H9+YW8hvX5rDcX3acmzPtqQ3C+GJebSq7Zg3gJTW3qXLEftv37sb/pYJ+FkrY/fW8PVKq66rZ9YMeO+P8OnfYem7cOZj+48DDBZ/k6NMupx971tsInQaCkf93nuPOw3zupRWVpe/WQMpwQsETbAiIiIiTVx1Sy40S4jl65Xe2L3YGGNY15Yc17cdx/VpS5dWKWGINErUd8xbuYSU6rt6RuKcEs1awllPQZ/T4Z3r4PFjYORNcMTvIDZIKU1xIbx/Y9XJUXCQmA7nvgSZQyA+qXblNfRvVktK8AJhS3mCpxY8ERERaZr8L7kQy9/O7M/pAzsyd30uHy3ewkdLtnDHO4u5453F9Gibui/Zy85qsW9GzslzNmiyllCorqtnELoNBkzf072Wsnd/77WGLX0Xxj7qjSkMhPIF3Oe+AAteg8Jc//sV7YSuIwJzzABTghcIOUu8vsoprcMdiYiIiEhYHGjJhcGdWzC4cwv+eFJv1m7fw0dLvGTvyc9+4NFPV9IqJYFRvduSlhTHi9+upbC4DNBkLUEVwm6DAZXSGsY/Cwtfh3evh8eOgtH/B8N/AzGx9Sszf4tvAfcXYOsSbwH33qfCqulel9XKIrGV08ec89PvNoINHTrUzZw5M9xh7O+JkZCYBhe9Fe5IRESiipnNcs4NDXccjUVE1pEiB5BXUMz05Vv5aPEWPlmWQ35hid/9MjOS+fLGUSGOTiJe/hZ451pYNgWyhsPYR6BV99o9t2QvLH8f5r7oLdbuSqHToZB9HhwyrubJUU57IKyJcE31o1rwGqqsDLYuhcEXhTsSERERkUYnPTme0wd25PSBHSkuLaPHn9/zu9+G3AJuf3sxgzpnMLhLCzqmJ2FankrS2sE5L8L8V7xJWB4dAcffBodeVnXNv3Kb5ntdMOdP9JaRSG0PR1wN2edX7erZCFs5leA1VO4aKN6jCVZEREREGig+NobMaiZrSYiN4YVv1vD0l6sAaJuW6CV7nVswqHMLBnRKJyl+/+55GsvXRJjBwHOg29Hw1jVeovfdf7xxcvmbvaTsyOugdK/XBXPLAohNgF4ne0ld91E1T9QSoslRAkUJXkPlaIIVERERkUCpbrKWv4/rzykDOrB0Uz6z1+5gztodzFmXy9RFWwCIizH6dGjO4M4ZDOrcgm27ivjnB8so0Fi+pqN5Rzj/Va9L5exnf9qet86blAWgQzacfC/0O8ubmTMKKcFrqH0JXu/wxiEiIiISBQ40WUv/Tun075TORUd0BWDbriLmrs31JX25vDprPf/7eo3fsguKS7n7fS28HtXMYOUn/h9LbQe/nh7aeMJACV5D5SyBjM7eJCsiIiIi0mBjB2XWOglrnZroLbXQtx0AJaVlLN+yi5Mf+Nzv/hvzCjni79M4qE0qB7VJ4aDWKRzUJpXubVPp0DyJmJiq4/rU1bORyVvvf/uunNDGESZK8BoqZ4m6Z4qIiIhEiLjYGPp2bF7tWL60pDiGdWvJD9t2M2n2BnYV/TRrZ1J8DN1ae4lfd1/it/bH3Tzy6Uot29CYNKYF3INACV5DlOyFbcuh54nhjkREREREKqhuLN8dZ/Tbl5g559iaX8TKrbv5YdsuVuZ41wvW5/Hegk2UVbOaWEFxKX+bsoTTBnbctzi7RJDGuIB7ACnBa4gfV0JZiVrwRERERCLMgcbyAZgZbZsn0bZ5Eod3b7Xf84tKSlmzfQ8n3P+Z3/Jz8ovod8tU+nZsTr+OzTmkYzqHZDanR9s0EuKqmZ5fQqMRLm0QSErwGmLLIu9aSySIiIiIRJy6jOWrLDEulp7t0qrt6pnRLJ6x2Zks2pjHaxUmdkmIjaFn+1QO6ZBOv8zmHJKZTp/2zUlOiNVYvlBqZEsbBJISvIbIWQIWC617hjsSEREREQmC6rp63nraIfuSs7Iyx+rtu1m4cSeLNuaxaMNOPli8mVdmeuPAYgzapCaybddeSp3X73NDbgE3vj6f0rIyzhqSVee4lCxKdcy5ajoXR6ihQ4e6mTNnhjsMb+X7t66BkgJIz2pSzb4iIqFiZrOcc0PDHUdjETF1pEiUqU8y5ZxjY14hizbksXDjTp6YvpLCkjK/+6YlxtEiJYEWzeJ91wlkNIunZbMEMlISaNls/8c+/34rN7+5yO9agUrymoaa6ke14NXH/InewM0SX3N93jrvPijJExEREYky9enqaWZkZiSTmZHMCYe058Fp31e771lDOpG7Zy8/7inmx917WZGzi9w9xfvN8HkgBcWl3PLWQuJijQ7pyXRIT6JtWiJxsTWPB1RLYPRRglcfH/xl/1l5wLs/7XYleCIiIiJSRcdqxvJlZiRz6+mH+H3O3pIycvfsZYcv8Su//ac3FvjdP6+ghKtfnLPvfoxBm7RE2qcn0zE9ifbpSXRIT9p3f/76XO6eukxLQEQZJXi1VVoMS96Cb56AXVv871PdoooiIiIi0qRVN5Zvwom9qn1OQlzMvlk+K3r4kxV+k8UO6Un895JhbMwrYHNeIZvyCtmcV8CmvEK+z9nFZ8u3sntvaZXnVVRQXMrNby6kzDnaN0+inS8pbJZQc9qglsDIoQTvQHblwKz/wsynIX8TtOgKSRlQmFt13yayeKKIiIiI1E1tlm2oreqSxRtO6k2v9mn0ap9W7XPzC4v3JX+/ePpbv/vsLCzh9xPn7bctLSmODulJtGvuawWskPwt3rSThz5eEZCWQCWKDacErzobZnmtdYsmQele6D4aTvs3HHw8LHytSS+eKCIiIiJ115BlGyqXA/VLFtOS4klLiqdHDUtAdExP4oXLhrM5r5DNOwvYnFfE5rwCNu8sZHNeIcu35JOTX0RNczUWFJfypzcWsHxLPq1SE2mdmkDr1ERa+a5bNEuoskj85Dkb9ktc1WW0fpTgVVRSBIsmw7dPwIaZkJAKQy6GYZdD6x4/7dfEF08UERERkfAKRLJYXUvgH0/qTbfWKXRrnVLtc0tKy9i6q4hNeYWMe+Qrv/vs2VvK45/9QGlZ1UzQDFo2+ynpa5WayMdLt+wXC3iJ4t1Tl6olsA6aVoI3f6L/pGznJq8L5qz/wu4caHUwjLkbBp4LSc39l9WEF08UERERkcavIS2BcbExvtk6k6ttCczMSObzP44kr6CY7buL2LZrL9t37WXbriK27ypi2+693vWuvSxYn8vuIv/jAzfmFjL4jg9pm5ZIG9+lbVrSvvtt0xJp2zyJNmmJpCbGBbwlMFDJYqiSzqaT4JUvbVBcYWmDN6+Gbx6HTXOhrBR6nACHXQ4HjYKYmqeUFRERERFp7ILZEjjhxF7ExJi3fl9KAge3rbmcEXd97DdRTEuKY0y/9uTkF5GTX8TKnF1s3VVEcWnVlsFmCbEUlZRVaTX0lpFYBEBKYhypvktKYiypSd7t5PhYzILTbTSU3U+bToI37faqSxuUFnlj7Yb/Bob9CloeFJ7YREREREQaqUBNIFNdonjHGf2qlOWcI3dPMTn5RWzNLyInv3Df7f98scpv+XkFxVz7ytxqjx9jlZO/OJZs2klRpQXqC4pL+cvkBSzdnE9cjBEbY8THGrExMX7vx8Uad7yz2G/303umLlOCV281LWFw0t9CF4eIiIiISJQJREtgXRJFs59aBivPGvr+ws1+WwLbN0/ixcsOY3dRKflFxewuKmVXUTG7ikrZXVTCrsISdhV5l92+68rJXbldRaU8/eUqSkrL8DPEsNY2+omzoZpOgpfeyeuW6W+7iIiIiIiEXTC7jN44pjcHtUmtU1nVdRvNzEjmyxtHAVBW5ih1jtIyR3Gp1z20pGz/+z977Gty8ouqlNMxI7mOr+7Ams5As9E3e0sZVKSlDUREREREosrYQZn8fVx/MjOSMbxk7O/j+td7zcHk+Nj9tlVeoD4mxoiPjSEpPpa0pHgyfLODtmueRKcWzejSKoU/ndzngOUEStNpwdPSBiIiIiIiTUIkrDkYjHJqw1xNKxRGoKFDh7qZM2eGOwwREQkBM5vlnBsa7jgaC9WRIiJNQ031Y9PpoikiIiIiIhLllOCJiIiIiIhECSV4IiIiIiIiUUIJnoiISIiZ2UlmtszMVpjZjX4eNzN7wPf4fDMbHI44RUSk8VGCJyIiEkJmFgs8DIwB+gLnmlnfSruNAXr4LpcDj4Y0SBERabSU4ImIiITWMGCFc+4H59xe4GXgjEr7nAE86zwzgAwz6xDqQEVEpPFRgiciIhJamcC6CvfX+7bVdR8AzOxyM5tpZjO3bt0a0EBFRKTxUYInIiISWuZnW+VFaWuzj7fRuSecc0Odc0PbtGnT4OBERKRxiwt3AHU1a9asbWa2poHFtAa2BSKeAJYVreUEsiyVE7qyVE7oylI5NesSgDIizXogq8L9TsDGeuxTRYTVkZFWTiDLUjmhK0vlhK4slRO6sgJRTvX1o3OuyV2AmZFWVrSWE4kxRWs5kRhTtJYTiTFFaznReMH7cfUHoBuQAMwDDqm0zynAe3gtecOBb0MYX0R9BvR/2/jKicSYorWcSIwpWsuJ1Jj8XRpdC56IiEhj5pwrMbOrgalALPC0c26RmV3he/wxYApwMrAC2ANcEq54RUSkcVGCJyIiEmLOuSl4SVzFbY9VuO2Aq0Idl4iINH5NdZKVJyKwrGgtJ5BlqZzQlaVyQleWypFIE2mfAf3fNr5yAlmWygldWSondGUFtY40Xz9QERERERERaeSaagueiIiIiIhI1GlSCZ6ZPW1mOWa2sIHlZJnZJ2a2xMwWmdnv6llOkpl9a2bzfOXc1sC4Ys1sjpm908ByVpvZAjOba2YzG1BOhpm9ZmZLfe/V4fUoo5cvjvLLTjO7tp7xXOd7nxea2UtmllSfcnxl/c5XzqK6xOPvM2hmLc3sQzP73nfdop7l/MwXT5mZDW1APPf4/mbzzewNM8toQFl3+MqZa2YfmFnH+pRT4bE/mJkzs9b1jOdWM9tQ4fN0cn3jMbPfmtky33t+94HKqSGmVyrEs9rM5taznGwzm1H+f2tmw+pZzkAz+9r3HfC2mTWvRTl+vxPr89mW8FEdWasyAlI/+sqKyjqyvvWj77lRWUdGWv1YQ0xhqyNVPwa4fgzmFJ2RdgGOBgYDCxtYTgdgsO92GrAc6FuPcgxI9d2OB74Bhjcgrt8DLwLvNPD1rQZaB+D9/h/wK9/tBCCjgeXFApuBLvV4biawCkj23Z8IXFzPOPoBC4FmeBMVfQT0qO9nELgbuNF3+0bgH/Uspw/QC/gUGNqAeE4A4ny3/1GbeGooq3mF29cAj9WnHN/2LLxZB9fU5vNZTTy3An+o49/bXzkjfX/3RN/9tvUtq9Lj/wRurmdMHwBjfLdPBj6tZznfAcf4bl8K3FGLcvx+J9bns61L+C4H+nzWoZyorSMJUP3oKyvq6kgaUD9W9xmsz/dINeWErY6sppyw1Y81xHQrYaojD/T9g+rHOl2aVAuec+4z4McAlLPJOTfbdzsfWIL35VjXcpxzbpfvbrzvUq9BkWbWCW/dpKfq8/xA8/2qcTTwHwDn3F7nXG4Dix0NrHTO1XcR3zgg2czi8CqfAy4aXI0+wAzn3B7nXAkwHTizNk+s5jN4Bl5Fj+96bH3Kcc4tcc4tq00cByjnA9/rApiBt8ByfcvaWeFuCrX4fNfwf3o/8MfalHGAcuqkmnKuBO5yzhX59slpaExmZsB44KV6luOA8l8T06nF57uacnoBn/lufwicVYtyqvtOrPNnW8JHdWToRHEdWe/6EaK3joy0+vEAZdVJoOpI1Y+BrR+bVIIXDGbWFRiE98tifZ4f62tyzgE+dM7VqxzgX3j/3GX1fH5FDvjAzGaZ2eX1LOMgYCvwjK9LzFNmltLAuM6hFv/c/jjnNgD3AmuBTUCec+6DesaxEDjazFqZWTO8X4Oy6lkWQDvn3CZfnJuAtg0oK9AuxVtsud7M7K9mtg44H7i5nmWcDmxwzs1rSCw+V/u6xTzdgC4RPYGjzOwbM5tuZocGIK6jgC3Oue/r+fxrgXt87/W9wE31LGchcLrv9s+o42e70ndiJH+2JQSisI4MRP0I0VtHBrp+hMj+HmlQHRmB9SNEZh2p+rGOlOA1gJmlAq8D11b6JabWnHOlzrlsvF+AhplZv3rEcSqQ45ybVZ8Y/BjhnBsMjAGuMrOj61FGHF7T9qPOuUHAbrwm6HoxswS8f6pX6/n8Fni/lnQDOgIpZnZBfcpyzi3B65bxIfA+MA8oqfFJjZCZ/Rnvdb3QkHKcc392zmX5yrm6HnE0A/5MPSu/Sh4FugPZeCcx/6xnOXFAC2A4MAGY6PuFsSHOpZ4nZz5XAtf53uvr8LUM1MOleP/3s/C6k+yt7RMD8Z0o0SNK68hA1I8QpXVkU6kfITB1ZITVjxC5daTqxzpSgldPZhaP94d6wTk3qaHl+bpmfAqcVI+njwBON7PVwMvAKDN7vgGxbPRd5wBvAAccjOrHemB9hV9bX8OrzOprDDDbObelns8/DljlnNvqnCsGJgFH1DcY59x/nHODnXNH4zXh1/dXJYAtZtYBwHddq+5+wWRmFwGnAuc75wK1lsqL1KI7gx/d8U465vk+452A2WbWvq4FOee2+E4Yy4Anqd9nG7zP9yRfF7Jv8VoFajWw3R9fl6hxwCv1LQO4CO9zDd5JXr1em3NuqXPuBOfcELwKdWVtnlfNd2LEfbYlNKK1jgxQ/QhRXEcGuH6ECPweCUIdGfb6ESKzjlT9WD9K8OrB9yvEf4Alzrn7GlBOG/PNvmRmyXhfsEvrWo5z7ibnXCfnXFe8LhofO+fq1TplZilmllZ+G28wcZ1nVHPObQbWmVkv36bRwOL6xOTT0F9v1gLDzayZ7+83Gq8fdL2YWVvfdWe8L56GxPYW3pcPvus3G1BWg5nZScANwOnOuT0NLKtHhbunU7/P9wLnXFvnXFffZ3w93oDlzfWIp0OFu2dSj8+2z2RglK/MnngTJGyrZ1ng+993zq1vQBkbgWN8t0dRz5OqCp/tGOAvwGO1eE5134kR9dmW0IjWOjJQ9aMvpqitIwNcP0KEfY8Eqo6MtPrRF1Mk1pGqH+vDBXDGlki/4H3JbAKK8f4JflnPco7E64c/H5jru5xcj3IGAHN85SykFrMD1aLMY2nYDGEH4XWpmAcsAv7cgLKygZm+1zcZaFHPcpoB24H0Br43t+F9gS4EnsM3u1M9y/ocrzKeB4xuyGcQaAVMw/vCmQa0rGc5Z/puFwFbgKn1LGcFsK7CZ/uAM3vVUNbrvvd7PvA2kFmfcio9vprazaLpL57ngAW+eN4COtSznATged9rmw2Mqu975Nv+X+CKBn6OjgRm+T6T3wBD6lnO7/Bm+VoO3AVYLcrx+51Yn8+2LuG7HOh/rw7lRGUdSQDrR1952URhHUk968fqPoP1+R6pppyw1ZHVlBO2+rGGmMJWR1b32lD9WK+L+Q4uIiIiIiIijZy6aIqIiIiIiEQJJXgiIiIiIiJRQgmeiIiIiIhIlFCCJyIiIiIiEiWU4ImIiIiIiEQJJXgiIWBmpWY2t8LlxgCW3dXM6rtWjYiISNiofhQJvLhwByDSRBQ457LDHYSIiEiEUf0oEmBqwRMJIzNbbWb/MLNvfZeDfdu7mNk0M5vvu+7s297OzN4ws3m+yxG+omLN7EkzW2RmH5hZsm//a8xssa+cl8P0MkVEROpE9aNI/SnBEwmN5EpdUH5e4bGdzrlhwEPAv3zbHgKedc4NAF4AHvBtfwCY7pwbCAwGFvm29wAeds4dAuQCZ/m23wgM8pVzRXBemoiISL2pfhQJMHPOhTsGkahnZrucc6l+tq8GRjnnfjCzeGCzc66VmW0DOjjnin3bNznnWpvZVqCTc66oQhldgQ+dcz18928A4p1zd5rZ+8AuYDIw2Tm3K8gvVUREpNZUP4oEnlrwRMLPVXO7un38Kapwu5SfxteeAjwMDAFmmZnG3YqISGOh+lGkHpTgiYTfzytcf+27/RVwju/2+cAXvtvTgCsBzCzWzJpXV6iZxQBZzrlPgD8CGUCVX0lFREQilOpHkXrQrxUioZFsZnMr3H/fOVc+FXSimX2D94PLub5t1wBPm9kEYCtwiW/774AnzOyXeL9EXglsquaYscDzZpYOGHC/cy43QK9HREQkEFQ/igSYxuCJhJFvjMFQ59y2cMciIiISKVQ/itSfumiKiIiIiIhECbXgiYiIiIiIRAm14ImIiIiIiEQJJXgiIiIiIiJRQgmeiIiIiIhIlFCCJyIiIiIiEiWU4ImIiIiIiEQJJXgiIiIiIiJR4v8BIacado2+YrYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1080x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "epochs = range(1, 21)\n",
    "\n",
    "plt.figure(figsize=(15, 5))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.plot(epochs, history.history[\"accuracy\"], label=\"training\", marker=\"o\")\n",
    "plt.plot(epochs, history.history[\"val_accuracy\"], label=\"validation\", marker=\"o\")\n",
    "plt.title('Train and Validation Accuracy')\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.xticks(epochs)\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.legend()\n",
    "\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(epochs, history.history[\"loss\"], label=\"training\", marker=\"o\")\n",
    "plt.plot(epochs, history.history[\"val_loss\"], label=\"validation\", marker=\"o\")\n",
    "plt.title('Train and Validation Loss')\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.xticks(epochs)\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.legend()\n",
    "plt.savefig('score.png', transparent=True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92937f2b-7231-4ce4-82f0-661082f857c9",
   "metadata": {},
   "source": [
    "This model is looking really good, performance seems to be doing really well. \n",
    "\n",
    "The trends for the train and validation accuracies are both uptrending, and do not hover around a certain level, which means there's no indication of overfitting.\n",
    "\n",
    "The trends for the train and validation losses both show that they're approaching zero over time, and again, there's no hovering around a constant value, meaning there's no indication of overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "8ce877b7-afd8-4f6d-acf8-eacf6062ca66",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "EFtVheveBKZ-",
    "outputId": "c0fa989e-b6eb-4292-8e28-34bf91962cbd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_9\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "gru_12 (GRU)                 (None, 20, 128)           265344    \n",
      "_________________________________________________________________\n",
      "gru_13 (GRU)                 (None, 20, 64)            37248     \n",
      "_________________________________________________________________\n",
      "batch_normalization_12 (Batc (None, 20, 64)            256       \n",
      "_________________________________________________________________\n",
      "dropout_12 (Dropout)         (None, 20, 64)            0         \n",
      "_________________________________________________________________\n",
      "gru_14 (GRU)                 (None, 20, 32)            9408      \n",
      "_________________________________________________________________\n",
      "batch_normalization_13 (Batc (None, 20, 32)            128       \n",
      "_________________________________________________________________\n",
      "dropout_13 (Dropout)         (None, 20, 32)            0         \n",
      "_________________________________________________________________\n",
      "dense_34 (Dense)             (None, 20, 16)            528       \n",
      "_________________________________________________________________\n",
      "batch_normalization_14 (Batc (None, 20, 16)            64        \n",
      "_________________________________________________________________\n",
      "dropout_14 (Dropout)         (None, 20, 16)            0         \n",
      "_________________________________________________________________\n",
      "dense_35 (Dense)             (None, 20, 12)            204       \n",
      "=================================================================\n",
      "Total params: 313,180\n",
      "Trainable params: 312,956\n",
      "Non-trainable params: 224\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Display its summary\n",
    "RNN.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d0a338f-cbe9-4348-b0f8-70df1c243852",
   "metadata": {},
   "source": [
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "capstone",
   "language": "python",
   "name": "capstone"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}