exercise_06_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Programming assignment 3: Optimization - Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, f1_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Your task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "In this notebook code skeleton for performing logistic regression with gradient descent is given. \n",
    "Your task is to complete the functions where required. \n",
    "You are only allowed to use built-in Python functions, as well as any `numpy` functions. No other libraries / imports are allowed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "For numerical reasons, we actually minimize the following loss function\n",
    "\n",
    "$$\\mathcal{L}(\\mathbf{w}) = \\frac{1}{N} NLL(\\mathbf{w}) +  \\frac{1}{2}\\lambda ||\\mathbf{w}||^2_2$$\n",
    "\n",
    "where $NLL(\\mathbf{w})$ is the negative log-likelihood function, as defined in the lecture (see  Eq. 33)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Exporting the results to PDF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "Once you complete the assignments, export the entire notebook as PDF and attach it to your homework solutions. \n",
    "The best way of doing that is\n",
    "1. Run all the cells of the notebook.\n",
    "2. Export/download the notebook as PDF (File -> Download as -> PDF via LaTeX (.pdf)).\n",
    "3. Concatenate your solutions for other tasks with the output of Step 2. On a Linux machine you can simply use `pdfunite`, there are similar tools for other platforms too. You can only upload a single PDF file to Moodle.\n",
    "\n",
    "Make sure you are using `nbconvert` Version 5.5 or later by running `jupyter nbconvert --version`. Older versions clip lines that exceed page width, which makes your code harder to grade."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Load and preprocess the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "In this assignment we will work with the UCI ML Breast Cancer Wisconsin (Diagnostic) dataset https://goo.gl/U2Uwz2.\n",
    "\n",
    "Features are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass. They describe characteristics of the cell nuclei present in the image. There are 212 malignant examples and 357 benign examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "X, y = load_breast_cancer(return_X_y=True)\n",
    "\n",
    "# Add a vector of ones to the data matrix to absorb the bias term\n",
    "X = np.hstack([np.ones([X.shape[0], 1]), X])\n",
    "\n",
    "# Set the random seed so that we have reproducible experiments\n",
    "np.random.seed(123)\n",
    "\n",
    "# Split into train and test\n",
    "test_size = 0.3\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=test_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Task 1: Implement the sigmoid function "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def sigmoid(t):\n",
    "    \"\"\"\n",
    "    Applies the sigmoid function elementwise to the input data.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    t : array, arbitrary shape\n",
    "        Input data.\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "    t_sigmoid : array, arbitrary shape.\n",
    "        Data after applying the sigmoid function.\n",
    "    \"\"\"\n",
    "    # TODO\n",
    "    t_sigmoid = np.ones(t.size)/(np.ones(t.size)+np.exp(-t))\n",
    "    return t_sigmoid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Task 2: Implement the negative log likelihood"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "As defined in Eq. 33"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def negative_log_likelihood(X, y, w):\n",
    "    \"\"\"\n",
    "    Negative Log Likelihood of the Logistic Regression.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array, shape [N, D]\n",
    "        (Augmented) feature matrix.\n",
    "    y : array, shape [N]\n",
    "        Classification targets.\n",
    "    w : array, shape [D]\n",
    "        Regression coefficients (w[0] is the bias term).\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "    nll : float\n",
    "        The negative log likelihood.\n",
    "    \"\"\"\n",
    "    # TODO\n",
    "    N = X.shape[0]\n",
    "    D = X.shape[1]\n",
    "    nll = -np.sum(y*np.log(sigmoid(X.dot(w)))+(np.ones(N)-y)*(np.log(np.ones(N)-sigmoid(X.dot(w)))))\n",
    "    return nll "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "### Computing the loss function $\\mathcal{L}(\\mathbf{w})$ (nothing to do here)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def compute_loss(X, y, w, lmbda):\n",
    "    \"\"\"\n",
    "    Negative Log Likelihood of the Logistic Regression.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array, shape [N, D]\n",
    "        (Augmented) feature matrix.\n",
    "    y : array, shape [N]\n",
    "        Classification targets.\n",
    "    w : array, shape [D]\n",
    "        Regression coefficients (w[0] is the bias term).\n",
    "    lmbda : float\n",
    "        L2 regularization strength.\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "    loss : float\n",
    "        Loss of the regularized logistic regression model.\n",
    "    \"\"\"\n",
    "    # The bias term w[0] is not regularized by convention\n",
    "    return negative_log_likelihood(X, y, w) / len(y) + lmbda * 0.5 * np.linalg.norm(w[1:])**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Task 3: Implement the gradient $\\nabla_{\\mathbf{w}}\\mathcal{L}(\\mathbf{w})$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "Make sure that you compute the gradient of the loss function $\\mathcal{L}(\\mathbf{w})$ (not simply the NLL!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def get_gradient(X, y, w, mini_batch_indices, lmbda):\n",
    "    \"\"\"\n",
    "    Calculates the gradient (full or mini-batch) of the negative log likelilhood w.r.t. w.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array, shape [N, D]\n",
    "        (Augmented) feature matrix.\n",
    "    y : array, shape [N]\n",
    "        Classification targets.\n",
    "    w : array, shape [D]\n",
    "        Regression coefficients (w[0] is the bias term).\n",
    "    mini_batch_indices: array, shape [mini_batch_size]\n",
    "        The indices of the data points to be included in the (stochastic) calculation of the gradient.\n",
    "        This includes the full batch gradient as well, if mini_batch_indices = np.arange(n_train).\n",
    "    lmbda: float\n",
    "        Regularization strentgh. lmbda = 0 means having no regularization.\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "    dw : array, shape [D]\n",
    "        Gradient w.r.t. w.\n",
    "    \"\"\"\n",
    "    # TODO\n",
    "    X_batch = X[mini_batch_indices,:]\n",
    "    Y_batch = y[mini_batch_indices]\n",
    "    dw = (-np.transpose(X_batch).dot(Y_batch)+ np.transpose(X_batch).dot(sigmoid(X_batch.dot(w))))/len(mini_batch_indices)+lmbda*w\n",
    "    return dw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "### Train the logistic regression model (nothing to do here)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def logistic_regression(X, y, num_steps, learning_rate, mini_batch_size, lmbda, verbose):\n",
    "    \"\"\"\n",
    "    Performs logistic regression with (stochastic) gradient descent.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array, shape [N, D]\n",
    "        (Augmented) feature matrix.\n",
    "    y : array, shape [N]\n",
    "        Classification targets.\n",
    "    num_steps : int\n",
    "        Number of steps of gradient descent to perform.\n",
    "    learning_rate: float\n",
    "        The learning rate to use when updating the parameters w.\n",
    "    mini_batch_size: int\n",
    "        The number of examples in each mini-batch.\n",
    "        If mini_batch_size=n_train we perform full batch gradient descent. \n",
    "    lmbda: float\n",
    "        Regularization strentgh. lmbda = 0 means having no regularization.\n",
    "    verbose : bool\n",
    "        Whether to print the loss during optimization.\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "    w : array, shape [D]\n",
    "        Optimal regression coefficients (w[0] is the bias term).\n",
    "    trace: list\n",
    "        Trace of the loss function after each step of gradient descent.\n",
    "    \"\"\"\n",
    "    \n",
    "    trace = [] # saves the value of loss every 50 iterations to be able to plot it later\n",
    "    n_train = X.shape[0] # number of training instances\n",
    "    \n",
    "    w = np.zeros(X.shape[1]) # initialize the parameters to zeros\n",
    "    \n",
    "    # run gradient descent for a given number of steps\n",
    "    for step in range(num_steps):\n",
    "        permuted_idx = np.random.permutation(n_train) # shuffle the data\n",
    "        \n",
    "        # go over each mini-batch and update the paramters\n",
    "        # if mini_batch_size = n_train we perform full batch GD and this loop runs only once\n",
    "        for idx in range(0, n_train, mini_batch_size):\n",
    "            # get the random indices to be included in the mini batch\n",
    "            mini_batch_indices = permuted_idx[idx:idx+mini_batch_size]\n",
    "            gradient = get_gradient(X, y, w, mini_batch_indices, lmbda)\n",
    "\n",
    "            # update the parameters\n",
    "            w = w - learning_rate * gradient\n",
    "        \n",
    "        # calculate and save the current loss value every 50 iterations\n",
    "        if step % 50 == 0:\n",
    "            loss = compute_loss(X, y, w, lmbda)\n",
    "            trace.append(loss)\n",
    "            # print loss to monitor the progress\n",
    "            if verbose:\n",
    "                print('Step {0}, loss = {1:.4f}'.format(step, loss))\n",
    "    return w, trace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Task 4: Implement the function to obtain the predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "def predict(X, w):\n",
    "    \"\"\"\n",
    "    Parameters\n",
    "    ----------\n",
    "    X : array, shape [N_test, D]\n",
    "        (Augmented) feature matrix.\n",
    "    w : array, shape [D]\n",
    "        Regression coefficients (w[0] is the bias term).\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "    y_pred : array, shape [N_test]\n",
    "        A binary array of predictions.\n",
    "    \"\"\"\n",
    "    # TODO\n",
    "    y_pred = (sigmoid(X.dot(w))>0.5).astype(np.int)\n",
    "    return y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "### Full batch gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# Change this to True if you want to see loss values over iterations.\n",
    "verbose = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "n_train = X_train.shape[0]\n",
    "w_full, trace_full = logistic_regression(X_train, \n",
    "                                         y_train, \n",
    "                                         num_steps=8000, \n",
    "                                         learning_rate=1e-5, \n",
    "                                         mini_batch_size=n_train, \n",
    "                                         lmbda=0.1,\n",
    "                                         verbose=verbose)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "n_train = X_train.shape[0]\n",
    "w_minibatch, trace_minibatch = logistic_regression(X_train, \n",
    "                                                   y_train, \n",
    "                                                   num_steps=8000, \n",
    "                                                   learning_rate=1e-5, \n",
    "                                                   mini_batch_size=50, \n",
    "                                                   lmbda=0.1,\n",
    "                                                   verbose=verbose)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "Our reference solution produces, but don't worry if yours is not exactly the same. \n",
    "\n",
    "    Full batch: accuracy: 0.9240, f1_score: 0.9384\n",
    "    Mini-batch: accuracy: 0.9415, f1_score: 0.9533"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Full batch: accuracy: 0.9240, f1_score: 0.9384\n",
      "Mini-batch: accuracy: 0.9415, f1_score: 0.9533\n"
     ]
    }
   ],
   "source": [
    "y_pred_full = predict(X_test, w_full)\n",
    "y_pred_minibatch = predict(X_test, w_minibatch)\n",
    "print('Full batch: accuracy: {:.4f}, f1_score: {:.4f}'\n",
    "      .format(accuracy_score(y_test, y_pred_full), f1_score(y_test, y_pred_full)))\n",
    "print('Mini-batch: accuracy: {:.4f}, f1_score: {:.4f}'\n",
    "      .format(accuracy_score(y_test, y_pred_minibatch), f1_score(y_test, y_pred_minibatch)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "hidden": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3sAAAJNCAYAAACImWznAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABjkElEQVR4nO3dd3xb5dn/8e8leSXOXoQkhCQQCJBNyt6jbGgpZXQ8jFJK+1C6W+iiDx0/2tJCKau0jJZSoKyWWfamjIQMIAkkISFk78TblnT//riPbNmRHTvR0bHlz/v18kuWdHR0+1hO9NV139cx55wAAAAAAIUlFvUAAAAAAAC5R9gDAAAAgAJE2AMAAACAAkTYAwAAAIACRNgDAAAAgAJE2AMAAACAAlQU9QB2xKBBg9yoUaOiHgYAAAAARGLGjBnrnHODs93XpcPeqFGjNH369KiHAQAAAACRMLOPWruPaZwAAAAAUIAIewAAAABQgAh7AAAAAFCAuvSaPQAAAACdR0NDg5YtW6ba2tqoh1JwysrKNGLECBUXF7f7MYQ9AAAAADmxbNky9e7dW6NGjZKZRT2cguGc0/r167Vs2TKNHj263Y9jGicAAACAnKitrdXAgQMJejlmZho4cGCHK6aEPQAAAAA5Q9ALx/YcV8IeAAAAgIIRj8c1efLkxq8lS5a0uu0dd9yhSy65RJL0s5/9TFdfffVW25x33nm6//772/38S5Ys0T/+8Y9tbjdq1CitW7eu3fvdHqzZAwAAAFAwevTooVmzZkX2/Omw97nPfS6yMaRR2QMAAABQ0DKraNOnT9cRRxzRocc/88wzmjZtmvbYYw89+uijknyoO/TQQzV16lRNnTpVr732miTpsssu08svv6zJkyfrmmuuUTKZ1He/+12NHz9eEydO1B//+MfG/f7xj3/U1KlTNWHCBM2fPz83P2wGKnsAAAAACkZNTY0mT54sSRo9erQeeuihHd7nkiVL9Oabb2rRokU68sgjtXDhQg0ZMkRPP/20ysrKtGDBAp1zzjmaPn26rrrqKl199dWNofCmm27SkiVLNGvWLBUVFWnDhg2N+x00aJDefvtt3Xjjjbr66qv1l7/8ZYfHmomwBwAAACDn/u+R9zR3xZac7nPvYX10xSn7tLlNGNM4zzzzTMViMY0dO1ZjxozR/PnzNXr0aF1yySWaNWuW4vG4Pvjgg6yPfeaZZ3TxxRerqMhHrwEDBjTed/rpp0uS9t13Xz344IM5HbNE2AMAAABQ4IqKipRKpSRpu0743rITppnpmmuu0U477aTZs2crlUqprKysw/stLS2V5JvKJBKJDj9+Wwh7AAAAAHJuWxW4fBo1apRmzJihE044QQ888ECHH3/ffffp3HPP1eLFi/Xhhx9qzz331ObNmzVixAjFYjH99a9/VTKZlCT17t1bFRUVjY899thj9ac//UlHHnlk4zTOzOpemGjQAgAAAKCgXXHFFfrGN76hadOmKR6Pd/jxI0eO1H777acTTjhBN998s8rKyvS1r31Nf/3rXzVp0iTNnz9f5eXlkqSJEycqHo9r0qRJuuaaa3ThhRdq5MiRmjhxoiZNmtSu0zLkijnn8vZkuTZt2jQ3ffr0qIcBAAAAQNK8efO01157RT2MgpXt+JrZDOfctGzbU9kDAAAAgAJE2AMAAACAAkTYAwAAAIACRNgDAAAAgAJE2AMAAACAAkTYAwAAAIACRNjLtTf+JF3/iahHAQAAAHRLZqYvfOELjdcTiYQGDx6sk08+WZL08MMP66qrrmpzHytWrNAZZ5yR9b4jjjhCHTn926xZs/T4449vc7tevXq1e5/tRdjLteoN0roPpC58/kIAAACgqyovL9e7776rmpoaSdLTTz+t4cOHN95/6qmn6rLLLmtzH8OGDdP999+fk/G0N+yFgbCXa7G4v3SpaMcBAAAAdFMnnniiHnvsMUnS3XffrXPOOafxvjvuuEOXXHKJJOm8887TpZdeqoMOOkhjxoxpDHhLlizR+PHjW93/nXfeqcmTJ2v8+PF68803JUlvvvmmDjzwQE2ZMkUHHXSQ3n//fdXX1+unP/2p7r33Xk2ePFn33nuvKisrdf7552vChAmaOHGiHnjggcb9/uhHP9KkSZN0wAEHaPXq1Tt8HAh7uWbmL1PJaMcBAAAAdFNnn3227rnnHtXW1mrOnDnaf//9W9125cqVeuWVV/Too49us+KXVl1drVmzZunGG2/UBRdcIEkaN26cXn75Zc2cOVNXXnmlfvjDH6qkpERXXnmlzjrrLM2aNUtnnXWWfv7zn6tv37565513NGfOHB111FGSpKqqKh1wwAGaPXu2DjvsMP35z3/e4eNQtMN7QHOWruwR9gAAANCNPXGZtOqd3O5z6ATphLbX20nSxIkTtWTJEt1999068cQT29z2U5/6lGKxmPbee+92V9PSlcLDDjtMW7Zs0aZNm1RRUaFzzz1XCxYskJmpoaEh62OfeeYZ3XPPPY3X+/fvL0kqKSlpXFe477776umnn27XWNpCZS/X0tM4qewBAAAAkTn11FP13e9+t9kUzmxKS0sbv3dZ+m6cf/75mjx5crPQaOnZfBnXf/KTn+jII4/Uu+++q0ceeUS1tbUdGm9xcXHjfuPxuBKJRIcenw2VvVyjsgcAAAC0qwIXpgsuuED9+vXThAkT9MILL2z3fm6//fatbrv33nt15JFH6pVXXlHfvn3Vt29fbd68ubERzB133NG4be/evVVRUdF4/dhjj9UNN9yga6+9VpK0cePGxuperlHZyzUatAAAAACRGzFihC699NJQ9l1WVqYpU6bo4osv1q233ipJ+v73v6/LL79cU6ZMaVaVO/LIIzV37tzGBi0//vGPtXHjRo0fP16TJk3S888/H8oYJcmylSq7imnTprmOnOMiL974k/TE96XvfSiVD4x6NAAAAEDezJs3T3vttVfUwyhY2Y6vmc1wzk3Ltj2VvVyz4JAyjRMAAABAhAh7uUaDFgAAAACdAGEv12jQAgAAAKATIOzlGg1aAAAA0I115Z4gndn2HFfCXq6l1+wxjRMAAADdTFlZmdavX0/gyzHnnNavX6+ysrIOPY7z7OWaUdkDAABA9zRixAgtW7ZMa9eujXooBaesrEwjRozo0GMIe7lGgxYAAAB0U8XFxRo9enTUw0CAaZy5xqkXAAAAAHQChL1co0ELAAAAgE6AsJdrNGgBAAAA0AkQ9nKN8+wBAAAA6AQIe7nW2KCFaZwAAAAAokPYyzUqewAAAAA6AcJersXS3Tip7AEAAACIDmEv12jQAgAAAKATIOzlGtM4AQAAAHQChL1ca2zQQtgDAAAAEB3CXq5R2QMAAADQCRD2ci1d2XMu2nEAAAAA6NYIe7lm5i+ZxgkAAAAgQoS9XGMaJwAAAIBOgLCXazRoAQAAANAJEPZyjcoeAAAAgE6AsJdrjQ1aUtGOAwAAAEC3RtjLNQsOaYqwBwAAACA6hL1cS4c9pnECAAAAiBBhL9do0AIAAACgEyDs5RoNWgAAAAB0AoS9XKNBCwAAAIBOgLCXa40NWqjsAQAAAIgOYS/XjMoeAAAAgOgR9nKNBi0AAAAAOgHCXq5x6gUAAAAAnQBhL9do0AIAAACgEyDs5RoNWgAAAAB0AoS9XOM8ewAAAAA6AcJerjU2aGEaJwAAAIDoEPZyjcoeAAAAgE6AsJdrZv6SBi0AAAAAIkTYyzUz36SFBi0AAAAAIkTYC4PFmcYJAAAAIFKEvTDE4lT2AAAAAESKsBcGi7NmDwAAAECkCHthsBhhDwAAAECkCHthiNGgBQAAAEC0CHthoEELAAAAgIgR9sJAgxYAAAAAESPshYHKHgAAAICI5SXsmdltZrbGzN5t5f7Pm9kcM3vHzF4zs0n5GFdoaNACAAAAIGL5quzdIen4Nu5fLOlw59wEST+XdEs+BhWaWFxKEfYAAAAARKcoH0/inHvJzEa1cf9rGVdflzQi9EGFyWJM4wQAAAAQqc64Zu9Lkp6IehA7hAYtAAAAACKWl8pee5nZkfJh75A2trlI0kWSNHLkyDyNrINo0AIAAAAgYp2msmdmEyX9RdJpzrn1rW3nnLvFOTfNOTdt8ODB+RtgR9CgBQAAAEDEOkXYM7ORkh6U9EXn3AdRj2eHMY0TAAAAQMTyMo3TzO6WdISkQWa2TNIVkoolyTl3s6SfShoo6UYzk6SEc25aPsYWCotT2QMAAAAQqXx14zxnG/dfKOnCfIwlL2IxKnsAAAAAItUppnEWHBq0AAAAAIgYYS8MNGgBAAAAEDHCXhho0AIAAAAgYoS9MNCgBQAAAEDECHthoLIHAAAAIGKEvTBYjAYtAAAAACJF2AsDDVoAAAAARIywFwamcQIAAACIGGEvDJxnDwAAAEDECHthiMWlFNM4AQAAAESHsBcG1uwBAAAAiBhhLwx04wQAAAAQMcJeGGjQAgAAACBihL0w0KAFAAAAQMQIe2GgsgcAAAAgYoS9MFhcci7qUQAAAADoxgh7YaBBCwAAAICIEfbCEIsxjRMAAABApAh7YaBBCwAAAICIEfbCQIMWAAAAABEj7IXB4pJLRT0KAAAAAN0YYS8MFiPsAQAAAIgUYS8MTOMEAAAAEDHCXhg49QIAAACAiBH2wkBlDwAAAEDECHthoEELAAAAgIgR9sLANE4AAAAAESPshSEW95cpqnsAAAAAokHYC4MFYY/qHgAAAICIEPbCEAsOK01aAAAAAESEsBeGxsoe0zgBAAAARIOwFwYLDivTOAEAAABEhLAXhsYGLYQ9AAAAANEg7IWBaZwAAAAAIkbYCwOVPQAAAAARI+yFoXHNHpU9AAAAANEg7IWBBi0AAAAAIkbYCwPTOAEAAABEjLAXhsYGLYQ9AAAAANEg7IWByh4AAACAiBH2wtBY2XPRjgMAAABAt0XYC4OZv2QaJwAAAICIEPbCwDROAAAAABEj7IWBBi0AAAAAIkbYCwOVPQAAAAARI+yFofGk6qloxwEAAACg2yLshaFxGidhDwAAAEA0CHthiAWHlWmcAAAAACJC2AsDDVoAAAAARIywFwYatAAAAACIGGEvDDRoAQAAABAxwl4YmMYJAAAAIGKEvTA0TuOksgcAAAAgGoS9MFDZAwAAABAxwl4YOPUCAAAAgIgR9sJAgxYAAAAAESPshYFpnAAAAAAiRtgLA+fZAwAAABAxwl4YGit7TOMEAAAAEA3CXhio7AEAAACIGGEvDGb+ksoeAAAAgIgQ9sJAgxYAAAAAESPshYFpnAAAAAAiRtgLA5U9AAAAABEj7IWByh4AAACAiBH2wmDBYXUu2nEAAAAA6LYIe2FoDHtU9gAAAABEg7AXBqZxAgAAAIgYYS8MNGgBAAAAEDHCXhio7AEAAACIGGEvDI1r9lLRjgMAAABAt0XYC0PjNE7CHgAAAIBoEPbCwDROAAAAABEj7IXBTJLRoAUAAABAZAh7YYnFqewBAAAAiAxhLywWY80eAAAAgMgQ9sJicaZxAgAAAIhMXsKemd1mZmvM7N1W7jczu87MFprZHDObmo9xhSoWl1JU9gAAAABEI1+VvTskHd/G/SdIGht8XSTppjyMKVxU9gAAAABEKC9hzzn3kqQNbWxymqS/Oe91Sf3MbOd8jC00sRgNWgAAAABEprOs2Rsu6eOM68uC27ouGrQAAAAAiFBnCXvtZmYXmdl0M5u+du3aqIfTOqZxAgAAAIhQZwl7yyXtknF9RHDbVpxztzjnpjnnpg0ePDgvg9sunGcPAAAAQIQ6S9h7WNL/BF05D5C02Tm3MupB7RCLM40TAAAAQGSK8vEkZna3pCMkDTKzZZKukFQsSc65myU9LulESQslVUs6Px/jChUNWgAAAABEKC9hzzl3zjbud5L+Nx9jyRsatAAAAACIUGeZxll4aNACAAAAIEKEvbDQoAUAAABAhAh7YaGyBwAAACBChL2wxOJSijV7AAAAAKJB2AuLGQ1aAAAAAESGsBcWpnECAAAAiBBhLyw0aAEAAAAQIcJeWKjsAQAAAIgQYS8sVPYAAAAARIiwFxaLSc5FPQoAAAAA3RRhLywWYxonAAAAgMgQ9sLCNE4AAAAAESLshYUGLQAAAAAiRNgLSyzOSdUBAAAARIawFxaLMY0TAAAAQGQIe2ExKnsAAAAAokPYC0uMyh4AAACA6BD2wkKDFgAAAAARIuyFhQYtAAAAACJE2AsLDVoAAAAARIiwF5aMaZzPz1+jJ95ZGfGAAAAAAHQnRVEPoGDF4lLKT+P800uLVF2f1AkTdo54UAAAAAC6C8JeWCzWWNmrqE0omXIRDwgAAABAd0LYC0tGg5bKuoTiMYt4QAAAAAC6E9bshSWjQUtFbUL1CTpzAgAAAMgfKnthyWjQUlmbUBGVPQAAAAB5RGUvLEGDltqGpOqTKTUkqewBAAAAyB/CXliCyl5lXUKSmMYJAAAAIK8Ie2Exk1xKFbVB2KOyBwAAACCPCHthicWlVFKVQdhrSDqlOP0CAAAAgDwh7IUlmMZZUdvQeFNDiuoeAAAAgPwg7IUlqOxVBGv2JNbtAQAAAMgfwl5YLC7JqaKmqbJH2AMAAACQL4S9sJg/tFW1dY030aQFAAAAQL4Q9sIS84e2sqa+8aaGBA1aAAAAAOQHYS8sFpckVddlVvaSUY0GAAAAQDdD2AtLzIe9zMpeHWv2AAAAAOQJYS8sjZW9prBHgxYAAAAA+ULYC0tjg5aMNXtJ1uwBAAAAyA/CXliCaZw1dfUqjpskKnsAAAAA8oewF5agslddW6+B5aWSaNACAAAAIH8Ie2EJKnu19Q0aUF4iicoeAAAAgPwh7IUlaNBSVVengb2CsMeaPQAAAAB5QtgLSzCNs7aOyh4AAACA/CPshSWYxmlKNa3ZI+wBAAAAyBPCXliCaZxxpZqmcSZo0AIAAAAgPwh7YYk1hb30NE7OswcAAAAgXwh7YQnW7MXkmtbsJZnGCQAAACA/CHthaQx7TZW9OtbsAQAAAMgTwl5YMqZx9ikrVkk8RoMWAAAAAHlD2AtL0KAlppR6lRWpOG5qYBonAAAAgDwh7IUlo7LXu6xIJUVU9gAAAADkD2EvLBmnXigvIewBAAAAyC/CXljMJEnlJTHFY+bDHtM4AQAAAOQJYS8swTTO8mIf+orjhD0AAAAA+UPYC0swjbNXiQ97dOMEAAAAkE+EvbCkK3tB2CtlzR4AAACAPCLshSVd2QumcdKgBQAAAEA+EfbCYv7Q9iz2l8XxGOfZAwAAAJA3hL2wxPyhLS/2V+nGCQAAACCfCHthCaZx9iyiQQsAAACA/CPshSThfMjryZo9AAAAABEg7IWkOuEve6SncXKePQAAAAB5RNgLSVW9D3ZU9gAAAABEgbAXkqoGf9nDL92jQQsAAACAvCLshaSqwUmSemQ0aGmgsgcAAAAgTwh7IakMpnH2KPLXi6nsAQAAAMgjwl5I0pW9siDslcRjakg6pVIuwlEBAAAA6C4IeyGpCCp7ZRlr9iRR3QMAAACQF4S9kFTWpSt7TWv2JKmBsAcAAAAgDwh7IUlX9orNXzZW9mjSAgAAACAPCHshqaj3lT1zLcIelT0AAAAAedDhsGdm5WYWD2MwhWRLXRDqXFJS0zROKnsAAAAA8mGbYc/MYmb2OTN7zMzWSJovaaWZzTWz35rZ7uEPs+upaAx7wXTOItbsAQAAAMif9lT2npe0m6TLJQ11zu3inBsi6RBJr0v6tZl9IcQxdklbaoNQl2pe2aujsgcAAAAgD4rasc0xzrmGljc65zZIekDSA2ZWnPORdXFb6ptP4yylQQsAAACAPNpm2EsHPTN7WdKLkl6R9KpzrqLlNmiyubGyRzdOAAAAAPnXkQYte0n6oaTHJW0wsxlmdk17H2xmx5vZ+2a20Mwuy3L/SDN73sxmmtkcMzuxA2PrdFo2aCluPM+ei2pIAAAAALqRdoc959wgSeMknS/pBUlTJF3anscG3TtvkHSCpL0lnWNme7fY7MeS/umcmyLpbEk3tndsnY1zTlvqEsGVlqdeSEY1LAAAAADdSHvW7EmSzOz7kg6QdKCkAZLekvTfdj58P0kLnXMfBvu6R9JpkuZmbOMk9Qm+7ytpRXvH1tnUJVJqSEqp4phiKU69AAAAACD/2h32JF0VXP5H0h8lveicq27nY4dL+jjj+jJJ+7fY5meSnjKzr0sql3RMB8bWqVTU+qqes1jTefaK6MYJAAAAIH86smbvQEnfllQh6WZJm8xsRg7Hco6kO5xzIySdKOlOM9tqfGZ2kZlNN7Ppa9euzeHT505legqnxbc69QJr9gAAAADkQ0fC3kZJG4KvTZLikia387HLJe2ScX1EcFumL0n6pyQ55/4rqUzSoJY7cs7d4pyb5pybNnjw4A4MP38qan1zUheLb71mj8oeAAAAgDzoyDTO+fLr6lKSZku6TtLL7XzsW5LGmtlo+ZB3tqTPtdhmqaSjJd1hZnvJh73OWbrbhsradGUvliXs0aAFAAAAQPg6EvZ+JR/uXss8x157OOcSZnaJpCflK4K3OefeM7MrJU13zj0s6TuS/mxm35IPlec557rknMctQdgzizVN42zsxkllDwAAAED4thn2zMyc9+NtbdPWfpxzj8ufoy/ztp9mfD9X0sHbHnLn17hmLxbPOM+eSWLNHgAAAID8aM+avefN7OtmNjLzRjMrMbOjzOyvks4NZ3hdU3rNnsWKtmrQQjdOAAAAAPnQnmmcx0u6QNLdwZq7TfLr6eKSnpJ0rXNuZmgj7ILSa/Yso7JnZiqJx2jQAgAAACAvthn2nHO1km6UdKOZFct3yKxxzm0KeWxdVkVdQqVFMb9mzzWFu5Iiwh4AAACA/OjIqRfknGtwzq10zm0ys3FhDaqrq6hNqHdZsV+zl2oKd8VxUwMNWgAAAADkQbvCnpl9z8z+G5wSIW25mV0c0ri6tIraBvUuKwpOvdB0qgUqewAAAADypb2Vvd0lfVMZ570LTr9wSghj6vIq6xI+7MXijQ1apCDsUdkDAAAAkAftDXvPSTpEUn36BjMbpAI5VUKuVdQm1Ks0S2WPBi0AAAAA8qRdYc85d2+w7SIze8vMfinpIEnvhzm4rqqyNqjsWbxZg5biOJU9AAAAAPnR7gYtzrnfShop6Qr50y58V1JFSOPq0ipqG9SrtHiraZylrNkDAAAAkCftOc9eI+dcjaTHgy+Z2RG5H1LXV1GXvbJHgxYAAAAA+dKhUy+05Jx7IUfjKBiplMto0BKjQQsAAACASOxQ2MPWqhuSck5ZT71QHI9xnj0AAAAAeUHYy7GK2gZJ8mv2Wk7jpBsnAAAAgDwh7OVYZW1Cklo/zx5hDwAAAEAetDvsmdlnzax38P2PzexBM5sa3tC6pi1B2OuVrUFLPKY6wh4AAACAPOhIZe8nzrkKMztE0jGSbpV0UzjD6roq63zY69NKZY81ewAAAADyoSNhL51aTpJ0i3PuMUkluR9S19Z8zZ5tfeoFwh4AAACAPOhI2FtuZn+SdJakx82stIOP7xaardmzeLNunDRoAQAAAJAvHQlrZ0p6UtJxzrlNkvpL+l4Yg+rKKjLX7LWYxlnMNE4AAAAAedKRsHeSpKedcwvM7MeSbpS0LpxhdV0VwZq9XiXZK3sNSadUykU1PAAAAADdRFEHtv2Jc+6+jAYtv5Vv0LJ/KCProk6dtLP2GtpbsZgFlb3ma/YkqT6ZUlksHtUQAQAAAHQDNGjJsd2H9NYJE3b2VyzWrEFLaUbYAwAAAIAw0aAlTBZrNo2zOO4PVwNNWgAAAACEbEcatAwQDVraluU8exKVPQAAAADha3fYc85VS1ok6Tgzu0TSEOfcU6GNrBBkadAiidMvAAAAAAhdu8OemX1D0l2ShgRffzezr4c1sILQWmWPsAcAAAAgZB3pxvklSfs756okycx+Lem/kv4YxsAKgsUk13SahfSaPaZxAgAAAAhbR9bsmZo6cir43nI7nALTYhpnKZU9AAAAAHnSkcre7ZLeMLOHguufknRrzkdUSGIxpnECAAAAiERHGrT8XtL5kjYEX+eHNaiC0bJBC904AQAAAORJRyp7cs69Lent9HUz+7eka3M8psLRokFL43n2CHsAAAAAQrajJ0VnzV5bLCa5pmDHqRcAAAAA5MuOhj237U26MYs3D3vBNM46wh4AAACAkG1zGqeZVSh7qDNJPXI+okLSYhon3TgBAAAA5Ms2w55zrnc+BlKQLNasQUvTmj0KogAAAADCtaPTONGWFpW9plMvJFt7BAAAAADkBGEvTC0btHDqBQAAAAB5QtgLU8vz7NGNEwAAAECeEPbCFIv7y5QPd8Vxf6aKetbsAQAAAAgZYS9MFoS9oLpnZiqJx6jsAQAAAAgdYS9MseDwtmjSQtgDAAAAEDbCXpgsOLwZTVqK46b6JN04AQAAAISLsBemFtM4JV/Za0iwZg8AAABAuAh7YWps0NJiGienXgAAAAAQMsJemBorexnn2qNBCwAAAIA8IOyFKUtlrzgeUx1hDwAAAEDICHthMn9evczKXmlRTA1M4wQAAAAQMsJemFpp0MI0TgAAAABhI+yFiQYtAAAAACJC2AtTlspeMQ1aAAAAAOQBYS9M2Sp7cdbsAQAAAAgfYS9MFhxe13QSddbsAQAAAMgHwl6YGsNe8zV7nHoBAAAAQNgIe2FqZRonDVoAAAAAhI2wF6ZWTr3Amj0AAAAAYSPsham1yh7TOAEAAACEjLAXpsY1e03hjgYtAAAAAPKBsBemxmmcTeGuOB5TIuWUSrlWHgQAAAAAO46wF6ZYcHhTzdfsSaJJCwAAAIBQEfbClKVBSylhDwAAAEAeEPbClK1BSzrssW4PAAAAQIgIe2HK0qClOO5v4/QLAAAAAMJE2AtTtvPsxansAQAAAAgfYS9MjdM4m596QSLsAQAAAAgXYS9M2Sp7QdirI+wBAAAACBFhL0zZTr3Amj0AAAAAeUDYC1OWBi1M4wQAAACQD4S9MLUxjZPz7AEAAAAIE2EvTFnOs1dMN04AAAAAeUDYC1NjZS9jGidr9gAAAADkAWEvTFkqe3TjBAAAAJAPhL0wmfnLjMpeKQ1aAAAAAOQBYS9MWRq0NK7ZYxonAAAAgBAR9sLUxjTOBip7AAAAAEJE2AsTp14AAAAAEBHCXpjSJ1XPrOxx6gUAAAAAeUDYC1N6GqdzjTcVx33TFsIeAAAAgDDlLeyZ2fFm9r6ZLTSzy1rZ5kwzm2tm75nZP/I1ttCkK3sZ0zjNTCXxmOqTrpUHAQAAAMCOK8rHk5hZXNINko6VtEzSW2b2sHNubsY2YyVdLulg59xGMxuSj7GFKkuDFsmv26OyBwAAACBM+ars7SdpoXPuQ+dcvaR7JJ3WYpsvS7rBObdRkpxza/I0tvBkadAiBWEvmczyAAAAAADIjXyFveGSPs64viy4LdMekvYws1fN7HUzOz5PYwtPlgYtkl+3R2UPAAAAQJjyMo2znYokjZV0hKQRkl4yswnOuU2ZG5nZRZIukqSRI0fmeYgd1NigpXmwKymKqYE1ewAAAABClK/K3nJJu2RcHxHclmmZpIedcw3OucWSPpAPf804525xzk1zzk0bPHhwaAPOCWsl7MVZswcAAAAgXPkKe29JGmtmo82sRNLZkh5usc2/5Kt6MrNB8tM6P8zT+MLRaoOWuOoIewAAAABClJew55xLSLpE0pOS5kn6p3PuPTO70sxODTZ7UtJ6M5sr6XlJ33POrc/H+EJjJsm2btASN9UnCXsAAAAAwpO3NXvOucclPd7itp9mfO8kfTv4KhwWy3rqhQYqewAAAABClLeTqndbsXjWBi1U9gAAAACEibAXNotnmcZJgxYAAAAA4SLshS0Wl1LNg10xYQ8AAABAyAh7YctW2SuKqYFpnAAAAABCRNgLm1nWBi2cegEAAABAmAh7YcvSoKWUBi0AAAAAQkbYC1uWaZys2QMAAAAQNsJe2GLxradxxlmzBwAAACBchL2wWSvn2aOyBwAAACBEhL2wWSxrg5ZEyimVchENCgAAAEChI+yFLRbbqrJXHPeHnSYtAAAAAMJC2AtblgYtpUWEPQAAAADhIuyFLVuDlnTYY90eAAAAgJAQ9sLWyqkXJMIeAAAAgPAQ9sJmMSnVohtnEPY4/QIAAACAsBD2wpalQQvTOAEAAACEjbAXtizTONNhr46wBwAAACAkhL2wZWvQwqkXAAAAAISMsBe2Nip7DVT2AAAAAISEsBc2a2PNHpU9AAAAACEh7IUtFm+1GycNWgAAAACEhbAXNotxnj0AAAAAeUfYC1u2Bi1M4wQAAAAQMsJe2LI0aCnlPHsAAAAAQkbYCxsNWgAAAABEgLAXtizTOFmzBwAAACBshL2wWbzVyl4DlT0AAAAAISHshS0W27pBC5U9AAAAACEj7IUtS4OW4rhJIuwBAAAACA9hL2xZGrSYmUriMdUxjRMAAABASAh7YcvSoEXy6/bqGgh7AAAAAMJB2AtblmmcktS3R7G21DREMCAAAAAA3QFhL2yxuJTauoI3qFeJ1lXVRzAgAAAAAN0BYS9sFsta2RvYq1QbquoiGBAAAACA7oCwF7YsDVokaWB5idZXUtkDAAAAEA7CXthaadAyoJcPe865CAYFAAAAoNAR9sLWSoOWQeWlqk+mVFGXiGBQAAAAAAodYS9srTRoGdirRJKYygkAAAAgFIS9sLVS2RvYq1SStL6SJi0AAAAAco+wFzazVhu0SNI6KnsAAAAAQkDYC1srDVoGpSt7nH4BAAAAQAgIe2FrZRrngHLW7AEAAAAID2EvbK1U9kqKYupTVsSaPQAAAAChIOyFzeKSnJTlfHqDepVqXRWVPQAAAAC5R9gLmwWHOFuTll4lVPYAAAAAhIKwF7ZYcIizTOUcWF7Kmj0AAAAAoSDshc3i/jLrufZKtIFpnAAAAABCQNgLWywIe9kqe71KtaG6XsnU1uv5AAAAAGBHEPbC1kZlb1CvEjknbaymugcAAAAgtwh7YWurQUt5cGJ11u0BAAAAyDHCXtgap3FuHfaaTqxOR04AAAAAuUXYC1tjZS/7NE5JnGsPAAAAQM4R9sK2jQYtEpU9AAAAALlH2AtbGw1a+vUoVsxYswcAAAAg9wh7YWujQUssZhpQXqr1VVT2AAAAAOQWYS9sbUzjlPy6vXVU9gAAAADkGGEvbI3TOLeu7EnSwF4lrNkDAAAAkHOEvbBto7I3sLxU6+nGCQAAACDHCHthM/OXWRq0SOnKHmEPAAAAQG4R9sK2jWmcg3qVqrIuodqG7GEQAAAAALYHYS9s25zG6U+svoGpnAAAAAByiLAXtjbOsydlnlidsAcAAAAgdwh7YWus7LXejVOS1nGuPQAAAAA5RNgLW+NJ1Vs5z145lT0AAAAAuUfYC1tj2Gu7sse59gAAAADkEmEvbNto0NKzJK7Sohjn2gMAAACQU4S9sG2jQYuZaVCvUq2jsgcAAAAghwh7YdtGZU/ixOoAAAAAco+wF7ZtNGiR/Ln21tONEwAAAEAOEfbC1jiN07W6ycBepVT2AAAAAOQUYS9sseAQt2Map2sjEAIAAABARxD2wraNBi2SP9defTKlirpEngYFAAAAoNAR9sLWzgYtEidWBwAAAJA7hL2wtadBS69SSZxYHQAAAEDuEPbC1jiNM9XqJgPLg8oeJ1YHAAAAkCOEvbA1TuNsPewNaqzsEfYAAAAA5AZhL2ztmMY5IF3ZYxonAAAAgBzJW9gzs+PN7H0zW2hml7Wx3WfMzJnZtHyNLVTtaNBSUhRTn7IipnECAAAAyJm8hD0zi0u6QdIJkvaWdI6Z7Z1lu96SviHpjXyMKy/aUdmT/FTOdVT2AAAAAORIvip7+0la6Jz70DlXL+keSadl2e7nkn4tqTZP4wpfOxq0SE0nVgcAAACAXMhX2Bsu6eOM68uC2xqZ2VRJuzjnHsvTmPKjHdM4Jb9ub30VlT0AAAAAudEpGrSYWUzS7yV9px3bXmRm081s+tq1a8Mf3I5qd2WvlMoeAAAAgJzJV9hbLmmXjOsjgtvSeksaL+kFM1si6QBJD2dr0uKcu8U5N805N23w4MEhDjlH2lnZG1Reog3V9UqmXB4GBQAAAKDQ5SvsvSVprJmNNrMSSWdLejh9p3Nus3NukHNulHNulKTXJZ3qnJuep/GFx8xfbqNBy8BepXJO2lhNdQ8AAADAjstL2HPOJSRdIulJSfMk/dM5956ZXWlmp+ZjDJHpQIMWiROrAwAAAMiNonw9kXPucUmPt7jtp61se0Q+xpQX7ZzGObC8VFL6xOq9Qx4UAAAAgELXKRq0FLTGyl7bYW9IHx/23l9dEfaIAAAAAHQDhL2wNVb22p7GOWZQuabt2l83PL9QFbUNeRgYAAAAgEJG2AubBYd4G5U9M9MVp+yj9VX1uv65hXkYGAAAAIBCRtgLW2PYa7uyJ0kTRvTVGVNH6LZXF2vxuqqQBwYAAACgkBH2wmbmA982GrSkfe/4PVUSj+mXj80LeWAAAAAAChlhLx8svs1pnGlDepfpkqPG6pl5q/XygrUhDwwAAABAoSLs5UMs3u7KniRdcMgojRzQU1c+MleJ5LanfwIAAABAS4S9fLBYu9bspZUWxfWjk/bSgjWVuuuNpSEODAAAAEChIuzlg8U7FPYk6ZN776SDdhuo3z/9gdZV1oU0MAAAAACFirCXD7H2N2hJMzP936n7qKYhqe/8c7ZSKRfS4AAAAAAUIsJePnSgQUumsTv11k9O2ksvfrBWt726OISBAQAAAChUhL186GCDlkxfOGBXHbv3Tvr1f+br3eWbczwwAAAAAIWKsJcPFtuuyp7kp3P+5jMTNbC8VJfePVNVdYkcDw4AAABAISLs5cN2NGjJ1L+8RNeePVmL11fpZw+/l8OBAQAAAChUhL18iMWl1I6dL++AMQN1yZG7674Zy/TvWctzNDAAAAAAhYqwlw87MI0z0zeOHqt9d+2vHz74Duv3AAAAALSJsJcPO9CgJVNRPKYbPjdVfXsU64I73tLyTTU5GBwAAACAQkTYy4ccVfYkaWjfMt1+/n6qaUjqvNve1ObqhpzsFwAAAEBhIezlww42aGlpz6G99acv7qsl66t00Z3TVZfITZAEAAAAUDgIe/mQo2mcmQ7abZCu/uwkvbF4g7533xylUi6n+wcAAADQtRVFPYBuIceVvbTTJg/X8k01+s1/3tfg3qX68Ul7ycxy/jwAAAAAuh7CXj7EYjmv7KV99fDdtGZLnW59ZbESyZSuOGUfxWIEPgAAAKC7I+zlg8VCqexJkpnpilP2VklRTLe89KFqGpL6f6dPVJzABwAAAHRrhL18sHjOunFm3b2ZLj9hnHoUx/WHZxeotiGl3505ScVxlmQCAAAA3RVhLx9CaNDSkpnpW8fuoR4lcV31xHzVNiT1x89NUWlRPNTnBQAAANA5UfrJh5AatGRz8eG76f9O3UdPzV2tL/7lTa2vrMvL8wIAAADoXAh7+ZCHyl6mcw8apevOmaLZyzbp1Otf1dwVW/L23AAAAAA6B8JePpjlrbKXduqkYbrv4gOVTDl95qbX9MQ7K/P6/AAAAACiRdjLh1iRlKzP+9NOHNFPD19ysMbt3FtfvettXfvMB5x8HQAAAOgmCHv5MGRvafW7Um3+p1MO6VOmu798gD4zdYSufWaBzr39Ta3eUpv3cQAAAADIL8JePow7yVf2Fj4TydOXFcd19Wcn6lefnqC3lmzQ8de+pCffWxXJWAAAAADkB2EvH3bZX+o5SJr/aGRDMDN9bv+RevTrh2p4/x76yp0zdPmD76i6PhHZmAAAAACEh7CXD7G4tOcJ0gdPSYloT4Ww+5BeevCrB+viw3fTPW8t1cnXvaLXP1wf6ZgAAAAA5B5hL1/GnSzVV0hLXo56JCopiumyE8bprgv3V0MqpbNveV3fu2+2Nlblv4kMAAAAgHAQ9vJlzBFScbk0L7qpnC0dtNsgPfXNw/XVI3bTQzOX6+jfv6j7ZyyTc3TsBAAAALo6wl6+FJdJY4+R3n9cSuX3nHtt6VES1w+OH6dHLz1Eowb21Hfvm62zbnldc5ZtinpoAAAAAHYAYS+fxp0sVa6Wls+IeiRbGTe0j+6/+CD96tMTtHBNpU69/lV9856ZWraxOuqhAQAAANgOhL18GnusP8H6/EeiHklWsZjv2Pni947Q/x65m554d5WO+t2L+n+Pz9PmmoaohwcAAACgAwh7+dSjvzTqUL9urxOvi+tdVqzvHTdOz3/3CJ0ycZhueflDHfLr5/T7pz/QpmqauAAAAABdAWEv38adJG1YJK37IOqRbNOwfj30uzMn6bGvH6pDdh+k655doEN+/bx+++R8baBzJwAAANCpEfbybdxJ/jLCE6x31N7D+uimL+yr/3zzUB2+52Dd+MIiHfLr5/R/j7ynpetZ0wcAAAB0RtaV2+xPmzbNTZ8+PephdNyfj/LTOC96PuqRbJcFqyt04wuL9MjsFUo5p2P33kkXHjpG03btLzOLengAAABAt2FmM5xz07LdR2UvCuNOkla8LW1eHvVItsvYnXrrmrMm65UfHKWLD99NbyzeoM/e/F+ddsOr+udbH6u6PhH1EAEAAIBuj7AXhXEn+8v3H492HDtoaN8yff/4cfrvZUfrF58ar+r6pL7/wBzt/8tn9ZN/vat5K7dEPUQAAACg22IaZ1Ru2N9357zgP1GPJGecc3pryUbd/eZSPfbOStUnUpq0Sz99ZupwnTxxmAaUl0Q9RAAAAKCgtDWNk7AXlZd/Jz17pXTpLGnA6KhHk3Mbq+r1wNvLdP+MZZq/qkJFMdMRew7R6VOH66hxQ1RWHI96iAAAAECXR9jrjDZ9LF07QTricumIH0Q9mlDNXbFFD81cpn/PWqE1FXXqVVqkY/YaohMn7KzD9hhM8AMAAAC2E2Gvs7rjZGnzMunSmVI36GKZTDm9unCdHp2zQk++t1qbaxrUq7RIR+81RMfvM1SH7jFYvUqLoh4mAAAA0GUQ9jqrmXdJ//6adMFT0sj9ox5NXjUkU3pt0Xo9Pmelnpy7SpuqG1QSj+mA3Qbq2L2G6Oi9dtKwfj2iHiYAAADQqRH2Oqu6Cum3Y6XJ50gnXxP1aCKTSKY0/aONenbeaj0zb40Wr6uSJI0d0kuHjB2kQ3YfpP3HDKTqBwAAALRA2OvMHrhQWvC09N0PpKLSqEfTKSxaW6ln563WywvW6c3FG1SXSKkoZpo6sr8O3n2QDhk7SJNG9FVRnDOHAAAAoHsj7HVmC5+R/v4Z6cw7pb1PjXo0nU5tQ1IzPtqoVxau0ysL1undFZvlnNS7rEgHjhmog3YbqGmjBmjc0N6EPwAAAHQ7hL3OLJmQrtlbGj5NOucfUY+m09tQVa/XFq3TqwvX6eUF67RsY40kqbwkrikj+2vqrv01bdf+mjKyn3qXFUc8WgAAACBcbYU9FkFFLV4kTfis9MafpKr1UvnAqEfUqQ0oL9HJE4fp5InD5JzT8k01mvHRRk1fslHTP9qo659boJSTYibtObSPpu3aX9NG9dekEf2068Cesm7Q9RQAAACQqOx1DqvelW4+WDrxamm/L0c9mi6torZBsz7epOlLNmrGRxs1c+lGVdUnJfmpn/sM66MJw/tq/PC+mjC8r0YNLFcsRgAEAABA18Q0zq7gpoOlojLpy89K1RukJa/4ry3LpdOul3r0j3qEXVIimdL8VRV6d/lmvbN8s95dvlnzVlWoPpGSJPUuLdLeGQFwz6G9NWZwuUqLONE7AAAAOj/CXlfw2h+lp34sDdlbWjPX31bcU0rUSlO+KJ16XbTjKyANyZQ+WN0UAN9ZvkXzVm5pDIBFMdPoQeXaY2hv7blTb+0ZXI4c0JMqIAAAADoVwl5XULlGuuMkqffO0uhDpVGHScOmSM9d6YPgeY9Low6OepQFqyGZ0qK1lfpgdaXeX7VF76+q1AerK7R0Q3XjNj2K4xq7Uy/tPriXxgwu126De2nM4F7adWBPlRVTCQQAAED+Efa6svoq6cYDpHip9NVXORdfnlXVJbRgTaU+WFWh+asq9P7qLVq0pkqrttQ2bmMmjejfQ2MG+RA4ZnAv7TaoXCMH9tTOfXsoTjUQAAAAIaEbZ1dWUi6ddI1012ekl38vHXl51CPqVspLizR5l36avEu/ZrdX1SW0eF2VFq2t1Idrq/Thuip9uLZSby3ZoOqgIYwkFcdNI/r31C4DemrXAT01ckBPjRwYXA7oqfJS/gQBAAAQDt5pdgVjj/GnZ3jl99L406XBe0Y9om6vvLRI44OmLpmcc1q1pVaL11Zp6YZqfbShWkvXV2vphmrNWrpRW2oTzbYfWF6iYf16aOe+ZRrWr4eG9+uhYf16aFi/Mg3v10ODepWyThAAAADbhWmcXUXlWun6ab6By3mPSbFY1CPCdthc3aCPNgRBcH21lm+q0Yrga/nGmsbTRKQVx01D+5ZpWN/MINhDO/cr05DepdqpT5kG9CwhEAIAAHRTTOMsBL0GS8f9Uvr3/0oz/ybte17UI8J26NuzWBN79tPEEf22us85py21icbwt2JzbdP3m2r0xuINWrWlVslU8w9oimKmwb1LNaR3qQb3LtNOfUo1JH0ZfD+kT6kGlpeyfhAAAKAbIex1JZM/L82+R3ryR9LQidLwqVGPCDlkZurbo1h9exRrr537ZN0mkUxpTUWdVm6u0ZotdVpTUafVW2q1psJ/v2xjtd5eulEbquq3emw8ZhpYXqJBvUo1sFdwWV6iAb1KNKjc3zYwuG1grxL1LOGfBwAAgK6MaZxdzZaV0m2f9F06z/+PNHiPqEeETqg+kdLayjqtSQfB4HL1llqtr6zX+qp6ra+q0/rK+mYNZTL1KI77AFjuQ+CA8hL161Gsfj2L1a9nifr1LFb/niXqG9zWv2eJepbEZUb1EAAAIF849UKhWb9Iuu14KV4sXfCk1G+XqEeUW85J8x6Rdj9GKukZ9WgKXnV9Qusr67UhCIDrKut9IKys04aqeq2ravp+U3WDahqyh0NJKonH1LdncfNQ2KNY/cubh8K+PYrVp6xYvcuK1KeHvyyOsw4VAACgowh7hWjVO9LtJ0m9hkgX/EcqHxT1iHLnvYek+86TDv2udPRPoh4NWqhtSGpLTYM2VjdoU3W9NtUEl9X+ts016e/95eYa/31tQ6rN/fYojqt3WVFGACxWn7Kixst0KGwZEnuXFatXSZHKS+MqIjACAIBuhrBXqD76r3Tnp/1UznMflcqyr/PaIWs/kB74krTLftJJv8v9/ltKpaSbDpLWzpNK+0rfekcq67vtx6HTq21IalN1gzYFYbCiNqGK2gZtqfHfb6lN3+a/31KbUEVNcFnboLpE22FRkkqKYupV6oNfeUmRepUWqWdpkXoF18tL07fF/XbZbistIjwCAIAug26chWrXA6Wz7pTuPlu6eqzUf1Tzrz2OkwaM2f79z3tEeuhiKVkvrZojjdhPmnRWbsbemrkP+aB38DelV6+V3vyzdNh3w31O5EVZcVxD+8Y1tG/Zdj2+LpH0QbCmeSisqG1QZV1SVXUJVdUn/GVdUpV1CVXXJ7S5pkErNtWoqi6hyjp/f6qdn3Glw2OP4rh6lMT9Zcb3PUviKsv8Prhstn1Jy8f4/ZWVxFQSj7HGEQAAhIbKXiFY8or0/hPSxiXSxo+kjYul+kpJJu11inTQpdIun2j//lJJ6blf+JO4D99XOuN2H/pWzZG+8pI0cLdwfo5UUrrxAMni0ldfk/5xprTibemb77J2rzNaM1/qOdCfFqQLcc6pLpFqDH5VdUlV1Scyrge31SVUWZ9QdV1SNQ1J1dT7y+r6hGoaUqqtT6q6IaGa+pRq6hOqbkiqo/+cxmPmg1+LkFhWHFNpUdNlaVFMZcX+srQ4prKiuEq3sU2z+zK2KS0iYAIAUEiYxtndOCdtXibNuF1661apdpO0ywHSJy6UUg3Shg99k5cNH0o1G6R+u/oAN2A3Xwmcfqu06Dlp6rnSib+Vikr9/m46WOq/q/Slp/1tuTbnPunBC6XP3iHt82k/TfX246Xjr5IO+Grunw/br3Kt9IdJvoL8lZekOJME0iGytiEdCpsCYsvL6vqk364+2K4hqZr6RON9dQ0p1SWSjfurS6Qav69tSLa7MtmadOgrK247NBbHYyopCr6C74vjppJ4vPH7zO2yb9/0fbN9ZNwX4/yP4alaJ825V9r/YikWj3o0AIAQEPa6s7pKadZd0n+vlzYt9bdZXOo30ge7Hv2lTR/58Fezwd8fL/Ehr+WJ2+c/Jt3zOemAr0nH/7/cjjOVlG7Y3z/3xa9IsWCt1O0nShsWS9+YFU7AxPZ54gfSGzf770/4rbT/RdGOp5tJJFOqTaRU1yII1gW3tXpfIqnadJBsdtnivuAxDUmn+mAfDcmU6oPLxI6mzRaK49Y8MMZjLUJkUzhM314Uj6k4ZiqKm0bWL9L68rGKx/06y5K4qSgeU1HcVBzzjy+KB5exmIqL0o/125TEYyqKNW1TnLFt4/0t9hePWdeokP7ncun1G6Wz7pL2Ojnq0QAAQsCave6stJe0/1ekaV/yUyJ7DpT67iIVlWy9bfUGX+0rH+QrNi2NO0na7yv+jcPow6U9j+/YWOqrpQ9f8FMyRx8uZb5Reud+af0C6cy/NQU9STr0O9LfT5dm3711+EQ0Nn7kK8ZTz/VTh5//hTT+9OwdYesqpQVP+enE8eK8D7VQFcVj6hX36wmjkEw5H/6CAFifEQab3+ZUn0yqPuEab88MjXWJrW+rT6bDpVN9oilw1idSqmhIaH1G4GxIpjS5YbYuSfxMv9MXdGvqFCWS/rnyoTE8BgGxKLheUtQ8PMZjpqJY+jK9XdP1ZvfHW9zWuG2scRu/z1jGPpoeG4/5IBuPmUpcrQ6e8XcVS9r00k1a2OOg1sfQ7HoQhDOud4lgCwDYCpU9dExDrfSXY6Qty6XJn5MqV/uvitVSXYU0ZC+/zm/4vtLwqb5St+Apad7D0sJnpYZqv59d9peO+ok0+lApmZBu2E8q7iF95eXmYc856c9HSjWbpEumd3y64Oq50vTbpBHTpIlnNQ+YXVEq5afZvveQ7446ZK/8j+Ghi/3zXzpTqt0i3XywNPnz0qnXNd8uUSfd9Vlp8Yu+4c6x/5f/sRaqVFKa/6i029H+A53uyjnp9hOkpf/1sxS+MUcq6yPnnJIp1xgIG5JOiWRKDangsvE2p4ZUSg2JpvCYSAaPCbZNh8dEY8B0akgkVFazWhuLhwT7y3hsKmMfSadkyj8umfLPl0ilGsfW/DIV3J/eduvtOuqM+Iu6uvhPejE5UYfH5+jIut9psdt5uw51zNQYQuMx89fjMcXMFI9JcTPFgmAYi5nilt7Oh9BYcN1vp2A/McVNjdul9920XbCfePP9xWPyj8143uzbZdlf+rHBftLbpcfetF0w9ljL/TU9Pv1zxCz95fcXC45Py/u6TDUYQJdDZQ+5U1wmffZ26Y6TfHWn905Sr52kwXtKxT2l1e9JL18tueCTdYv573sN9eFw3Mm+GvTib6S/niyNOUIaNlXasMhPM4q1aHVv5s+3d+/nfcCY+FkfMD56VfrwRal2s7THJ/0J2Et7Nz1uw4fSC1dJc/7p9/HWn6WZf5dOvlYatHueDlaObVwi/fsSacnLPkTfdrz0uX9KI/fPvv1H//XVtBFZ//a3z+q50ux7pIMvlfoM81/pau++5/mAL/kw8sCFPugNn+Y7q445XNrtqNyNpSPqKqRYkf9AYUelklKywf8tROW1P0rPXCGNP0M649boxhG1Ja/4oDflC/7v+/UbpSMukwUBoyjuu8Dm3KPfkt6+w//9jT029/vPIjPAtgyJ6SDZGBqD66P/9RvV1O+uPp/8s1L3H6w7J72rBZNPDR7bPISm95kOnU3PkWr2nD6ESqmM8aRSTknXdNl4W8o13y74PpVSUJ1NbrVdMthHMmO/jbennFJOzbZJ5nhKcT6kw3KzkJgRKDPv8wExMzhmPC7WPETGWwmbrd3nb2/x/EGIzXpfeiwtnr+t+1p7nmz3mfnnzfw5rcVxslYu09s0bh9T42OkLNvEsj+HKbgt4/GtjQPoKqjsYfukUul/mbe+r75KWjlHWj7DN4cZe5yv9GUGuYYaHxZf+b1UvV4aOtE3+si2v/S59+orpd47+/26pFRU5t+812yU4qU+OI47UVo5W3r7b1Ks2K8lO+hSX1l8+mdSosaHx0O+2f41gM75dY3L35ZWzPS39Rvpp8P228VfhnGOw7RUSppxm/TUT314Pu4X/me983RpywrpzL/602ykbV4u/ecH/tQZkp9CeeyVO3YajrS7z5GWvOrXUPYc4G+r3Sz9cZpv3nPBU/53+Mil/ndw3K+kfc/31dnqDdJXX5V6DdnxcXTEklek+8734/rkL6UJZ2x/hbd2i/T3z/iGRV94QNpp79yOtT1WveuPZ8+BUsVK6TO3+p9pe21Z4cPLLvtLB32980y3TTZIy97yP+fgPbNvc8fJ0roF0jdm+/OBLn7Jf59+bYZh4TP+NVDUw3/o8uVnpUFjw3u+tPT/1e197a6YJd1yuHT8r6UDLvZ/A4uek749r+C6G28VCl1T2My8PZVSlvCYbbtsIdMpGTw+lRFyUy54PqfG65nfJ52TCwJqyqUfE+ynjfvS4b75vhQ8Jvt9Tbf7nzXlst237bGlH5dqx9i6s6YA2EbozAiL7dmm+WXm9tkfb23uL7PaLJnasU36uZT5nE1hPP0Y/33TdmrxuHTTrViLbbf+Gf1zWMttt7qt+TFsGl/zbTN/zpbbaqtjJUlbH1tled7MMRXHY9plQOf7N5QGLei86iqkmXf56Zw77dP6dnMflu47Txo2xVeIxhzhz/sXK5I+fsNPaZv/qG9CEyuW9j3Xh7o+GVOWKlZLT14uvfuADz7Dpko9+kll/fxlcU8fQuurpIYqf7lpqQ+X1ev9PuLBWsdkffPxDR7nxzTmSGnUwc2rjJJ/o+ZSHeuGV7Hav7GcdZevZI45Ujr1jz5gSr4j5l1nSKvekU67QZrwWenNP0nP/0pKJaTDvifJSS9f47uw7n+xP2dhayepd85XSV79gw8UB1wsfeLLTRWspW9It33ST79tee7DmXdJ//6a9KmbpHUfSK9c44//0T/x96+e6wPKrgdLn79/6wrutjTUSnVb/NTQviPa94bXOd+Y6Okr/O+7tJcP66MOlU68WhoyrmNjqKv0b/KXT/evmVRC+vx90i77dWw/OyJRJ91ypFS11gfnez4vrXvfn6qk74iO72/NPOnvZ/ip2KkG/6HLaTdIO0/M/djbI/2aX/CUtOh5qW6zVNpXuvAZafAezbdd8qp0x4lN3XrXzJNuPFA6+BvhTRmu2SjdeJD/cOesv0u3HSf1GOADX2t/V7nQUOM/aEkl/N9Pe6rKj3xDmn2v9J35/t+3Ja/4GRmn3eAroeiYhhr/ASMVnWaahch0uNxGqGwWMF3GY1OSk78/83bXLECr8bpTi21STY9zrvmYpKbnSI8h63Nk3NfWNunncK7lc269n6b729qm+c+R/tmyPWfLnyPzOLS2TXocTmo+rlTrz+Hkj7lrDPnNx9V4nNT0WVR3MLh3qd760TFRD2MrnSLsmdnxkv4gKS7pL865q1rc/21JF0pKSFor6QLn3Edt7ZOw182kUm2HBOf8G76yvlLf4a1vt+Bp6eXfS5Wr/Ju32s1N007T4qVSSbnUe6gPhcOn+OrkkH18wKxaI236WNq81E8Z/eg1/5Wo9ffvtI8fb91mv/+6Cr/f3jv7N+V9R/iKYPkg/wYiXaWMF/tP5Bc+7UOc5KfJHnG5nybZ8o1GXYV/w7/4Ran/aH+Oxd2P9d1UB4z222xZ6c+bOOsuXyXZ+1T/pn7nSdKQvf1zzn/Mh7zl04NKyjgfMPuO9IFt/Bn+jeL6hb6qV1K+9e/mtk/6kJio8dW8k69pPt63bpUe+7Z07M/9NNCWEnXS6nd9IFs+U1o5S6pY5UNeZrgeeZD0yV9II/Zt/XdcVyH9+3+luf/2lc3TbvRjfvuv0jP/56vEB3zNNwDq0a/1/aTVV/n1h0tfl864TRo2Wfrbp3xIOutOP404G+f8a2zTR/6Dg7oKae/Ttv4wIK1yrf9AItkgnfAbP00601M/kV67zk8f3OM430X35kP9VN0v/qtjIXrJK767blGZDxAbF0uPfceP95Bv+0Cfjw64tZul9/7lpwcvfc3f1muonx6568HSUz/24erCZ5tX7P56qv97/+acpum5D3zZV7S/MXvrY5cLD35Feuc+H+6GTfHH8G+n+bWT59wdzqkNkgnpn1/051KV8+tjT7uh7dBRu0X63Thp/Kf9tpJ/Ld54gD9WF72Q+3EWsnmP+N/9TvtIJ/y6abo6gEbObR1QpRYBWv7tVmOYTIfIjOuZYTLzcZnbNob9jA8Itg616edt/piWYb7lttnGlPmY0qKYTpiwfWufwxR52DOzuKQPJB0raZmktySd45ybm7HNkZLecM5Vm9lXJR3hnDurrf0S9pATqZRUX+E/uS3u6b+257xxDbXSsjd9RWLFTP+mqrSPf6Na2luS+Slzmz/20wC3LN+6Qij5sLjLAdLuR/sQsdP4tt/EJ+p8sFn6unTcL6W9Ts3+RnDFLF/1W/q6D6Hp5yrrJ1Wv8x1YD/q6NOlzfprXouelp38qrZrjz8G4YZGviO335ezjWP62b96z96l+amHLN77OSf/8H+n9x6Xzn/DHeUUwNXb52369Z6rBb9tzkH8z3W+XjGPYx4eu/97gw/Y+p0tH/7Qp1DrnA9XK2dJzP/fB9Jif+Wm8mcejap30zM+kmXf6qXj7fMp3Fh15QPbj1lAj/eNM/8b+9D83TZmsXOOn0q6dL53+J2l8ML1z6es++C970582pL6y+f567SQdfYU06Zym36tz0nsPSo99N9jefCXylOua2uUvedUH7n3Pk065tml/M+7wVZzj/p904Ney/25aevcB32in/yg/HbXfSH979Qbfqn/OPdLgvfwb2zGHt2+fHZFK+tfX7Lt9RT5RKw0c65so7Xm8f82nfxdLX/fTNXc9yI81Xuxvu+04Py33oEua9rt+kXT9J/w5RU/8TW7HPO9Rv3b48B9IR/6w6fa3/uJD8iHf8q+3bdnwof+gpz3TZZ2THr7Er0c88Wpf0X3x19s+9+ibf5Ye/6705ef8h1Tbuj3Xlr7h/33b59NduxLmnPTSb6XnfyntNMF/QFi1Vpr8Bf9vTxgfKETJOV897ixTuTu7ugo/BX7gWP//YpjTx4Ft6Axh70BJP3POHRdcv1ySnHNZT9ZmZlMkXe+cO7it/RL20KWlUn66aEOtr4Y11Po3vf133b4pYc61f3rjxiU+FK2a47/f6xQfElsGtFRKevd+6dkrfXi9+NXsp+1I27LSh5nWwmnNRl+J2vxx022lfX2lbNgU/4n5sCn+zXBrP0tdhW9Q8toffQVs79N8hW3VO36NqCSVD/YVuNGHtT7WVe/4Tq1z7vNhf9CevolQ3xF+um5Rqb987TofTD59szTp7Ob7qN0s/eNsP/2174imn6uklzTiE75C2m9k01d9pa/OLZ/uf87jf+2nmD72bb+udNhUPxVWkh66yP+OJn9BOupHPtxY3J+HMrMDp3N+it+i56SvvNh6h9YtK324/vBFP9135IHS2f/I/gblg6d8gNm8VBr7Sb/mc1udX1MpP6X6vYf8m8UxR/jnyBzrmnnSrH/4xkmVq/wHDRPO8MF3+L6t/87T04Q/caHvQnvn6f7YfHPO1lXmhy/1IfLrbzdNeW6vVe/6Kk6fYb6y2GeYv71qvXTj/r4yf+GzW/8NPPJNacbtvoo2+fPZf46ajdKTP/IV9p3G+yDfVnVa8n93L/9OOuz7/jWQSjVV+b7wgLTbkVs/xjm/xjleLF30YvOxpCt++3xa+tQNTbfXVfjKe2kvaep52/dhV/q537zFf2Dgkr4Kf/Lvo+kavKPqq/2HaO89KE0403cbTjb48Pf6Tb4ifth3feguhHPAVm+Q7j9fWjPf/7uwrddmd5dMSHef5f/ddSn/4eW+50kH/u/2TamPwsrZPqwe9v2On0oLnU5nCHtnSDreOXdhcP2LkvZ3zl3SyvbXS1rlnPtFW/sl7AF5kkz4T3xz0YFy5Rw/FW7oRB94Bozp+Bo+yYeXF37l13MO3E0aOqFpeupO+7S/82ZdpQ8oM+7wISybU6+Xpn4x+30NNX6qYdVa/+Z25AH+zXxrb5hTKemdf/q1hJWrfDBM1vupugdd2vS4RL2v4rzye1+BTSWkC57Mvkawco1fr9ajnz+HpVzTIoqqtb5yWrHCX7e4D1inXNf277Oh1ofCl37nw/CUL0qHf98H6ViRbxZk5t8cvvNP/zvdtNRXS13S/0yxIt+NdcQ030V25Wx/29hP+uC8x/Htf6P81I99wN/3PP+7OvZKvz6vpc3LpOum+DfoJ/7WB/ZYvPUgWbHaj332PdLqd5rfN3SCbzC16p2mMJ1tbXGiXrrzU37q8/B9fZVvz5OaXtfzHvHhuWqdfx198KSforz/xdJRP85++ozXb/aNlvY9z3cRTo+/rkL6y7G+Oc9Fz2/deCld9TzlOr92uaVHvunD8LeDKe8z/+6neVetafqZT/5Dx9/sJ+r8hxYz/y7teaIPy89e6cd70KV+DXGUjWE2LJae/on/4OYTX/LjaW069eblforzytnSMVf4U8dkvn7WL/LB/YMn/PE/7lf+tZzLKmaiXlrzXvBvSciVtvWL/OyFjR/5v++ajX62wt6nhfu82aycI73w//xzb8/pktr7oeeOcK6pCdkpf/D9A179g/9gVPLr58ed7Nfv9+gf7lgkf2qqzcukoePb/5iVc6S/nRo0uCvx09BbW46ALqFLhT0z+4KkSyQd7pyry3L/RZIukqSRI0fu+9FHbS7rA4D227LSvzlN1vk3W8m6trtB7oi6St/IZuVs6ZM/b736sfQNHxTGny4d+u3W97fgGenRb/rw2fhmx/wU2GFTgrWn+/o38x150129wVcz3vxz0zTbtHQItZhvIDTxLGncST5cLX3dd8dc/JKvKA6d4KcIj/+M1Gtw+58/LZX0FcwFT/qmKN98p/VzDD7xA+mNmzNusKZKbfqrqMQ3c9qwyH8yP3xfX2Hc53RfKV7wpK9wfvyGD69H/9Sv8WxNQ62v2r12na+WD9rTf8q/6Dlp7r/8z3/q9b6KXbvZrx2dfquvYh/zMz++TR/76vDGJX6687iTpTP/tnXFfcOHvlFP752lz//Tv0FPf7jx4EW+8ved+VtXPSUfXG8+xP8uVs72gWKX/X1g2bLcH7uKVdK0C/zP3J41rRWrpXu/4KcuH/Z9/8FFLOYrok//VJr1d1/ZbpxiHrw+Y3Fp0B7+zfLA3Tr+Jr1qnZ9eveQV/3rrO8JPy97zhKbZEXWV/gOT1673r9dRB/sGQC2nU6dS/gOJmXf6D5DixdJn/uL31ZqFz0j/+aFvkrTbUX4qdbbGTzUb/TT1Ve/6479mrq86T/68X++Z+cFQXYX/MOO/N/oPaHoOkiae6WcdDJ3gt0mlfOOw9x+T3v+Pf/yU//Hbtef3lWnxy/53F4v70x8N3F265xzfCfeY//MfqORjKm6ywVexX/qtv55K+MB38rXbnh6ZSvkK7HM/97/vCZ+VJp/jP/wLY+wv/dZ/QJLZhEzyf7//vcGHwIYqSeabXY0+TNp5sv8ZG6qCRnDV/m9+4O7+a8Do7asQf/Ckn81QGfzNHvvzbZ97ddU70l9P8dXIs++S/v11af0Cv3Z79KEdH0NnsOlj6YP/+K/qDX5N/vjTw1lH3Ul1hrDXrmmcZnaMpD/KB70129ovlT0AyJP1i3yISNY1nWswlfCBY+/T2l6/lExs/9TATLVb/FSzfU6Xpny+9e0aanylrnazrzAm633lKdngx5+s998n6nzImHj21t0+09Jv1Ece1L4KdDLhw90r1/iGQ/ESv87v4G9sXaFZ+rpfb7l2ftNtxeU+CIz4hF+n11r1ddHz0t9Pb2ouFS/1VYSqtb5ydeJvWx/jrZ/0Ibbfrr5CuvdpTW+Ka7f4tb1v/skHjeFT/e85lfC/d5fyP0dRD//mtKjMB/raTX4K8j6f2vr5lrzqq5TrP1RT1dk17VfyYx/xCf8G3SX9+tz6Kj/1OVEnyZqCopkPvGvmNh2zEdP8a3TLMn/MdzvK3/bmX/wb4Yln+ypdn2HSshnSE98PplNP9VXIOff6kF3a14emA77qXxvbkmzwU2Bf+JUPGnuf6o9R1Tr/u6ha619DaT0H+VO2rH7Pd3nuNVSadJYPwu8/4c8JW7vZdw2ecIa08Fl/e6rBrxscOt7fVrXGV+lHHex/Zytn+d/J+NN9g6yhE/yHMLF4UxXeuaA7RvC7fOc+X40duLt0zj1N658baqR/fc0HqKn/418jVet9Nblilf8wpOcAv5Z7wBjfaKy1UOWcX6u+cpZfN56s88d8xDSpz3D/uFXvSP/6qr+c8Fm/JjVdce45wE+Pbu18lotf9h8orHjbV0EHjPFv+JP1vqHapLP9PtIfpGxa6sc/ZC9/jEcd6j/Ma28onH2vn2I/8Szp03/K/rhEnQ/ji1/2fxvL3sy+Pr8Z83/3w/f1TdZ2P9o3iGtN7Wb/QcOsv/tGa7se5F+H/Xf1f4e7HpT9cave9UGvqEw6/zF/vKrW+fXgmz6WvvhQ6+fubU0q6T8cWPSc39+eJ+b+dFQ1m6SP3wz+Ta8L/j2v97NaFjzdNCtjwBj/Id669/0Hbod/309bz0Xoa6j1szTWL5SOvHzH95djnSHsFck3aDla0nL5Bi2fc869l7HNFEn3y1cAF7Rnv4Q9AECn5Jxfy9l7aNvnuEzU+emfPfr78NWjf/vfeC6f4atzNZuCzsKb/Bv1o3/a1HQnmzXz/BunSWe3Xk1YMVN69uc+rMSKmr4sFoTnWj/2RK0f82nXN1We2iuVlNa+798opr/Wvu/DZEm5n+JcUt50ypt0SHTOf7gw6hD/Zn3YFP+YdMVr7r98h9cty3ywOOHXW09/bjmdetShPtjsdUr7p4BnqlrvG7nMf9SvRS0f7ENQ+WDfHTod1Hrt5H+/iXpfPZ55l680uqQk8w2ZDv5W82m01Rt8U6VZd/mQu9vRwVTZY5qmCa6Y6SuC79y/dVMoyf/eWnadlvy+Pnv71uvEUyn/87x89bZ/9tI+PmSU9Gpa71xU6l+LK2f711B6DLGipuDTa6ifFr34Rf9znHxtU0MqyYe/B7/iq8+TzglmWGSE/o9e9cGuzwg/HXrimf5NffUGH1Rn/cO/HvyT+w+m+o30v5cVs/zrQ/K/o50n+9dy7SapZnOw/tukAaP83++AMf7nfO4Xfqr+Fx5se/16poYaP424uMx/MFHcw1fV6iv9zIL1H/oAse4D39yrcpV/3NAJ/vczcDc/wyT9tWGxn8VRsdJPGT/8B/54L3nVh+ZNS33jqsO+5z8QUBDyN3zoTx8UL5HOe6z5hxkVq6TbT/DB73/+5bt7b1rqp4duXuYrkuWDg68hUvlA/7c6/1H/YUT6dyz5D552P8aHrD2P9+GrdpMParWb/b8b5YP9OXez/XvnnP+gZ8VM6cMX/NeKt7O/fi3mG9rtebz/mxg01r925/1beuHX0tp5fu381P/xH070H+Vfq8U9/PNUb/Af8mxa4mf29B3uw/OAMU0fzq2ZJ834q29cVrPRfzjy1dc63VrdyMNeMIgTJV0rf+qF25xzvzSzKyVNd849bGbPSJogaWXwkKXOuVPb2idhDwCAApJK5uZT+HR33r67tF2Rra/2p3dpq4oStnR1Ypf9/JvVHVFX4asPFat8gEyl/KVLNYWtdMWvfLCveLZVdf/gSf9mt/fO/hj1HuofV73BB5UNH/qK6qalUkN1RhW93j/H0Ik+SA2bHHSWjvvq0vLp0rLpvuI3bKqfRlw+cOvnT9T5gPXfG4JAnKG0r3Tot/za19YC+obFkpwPhJnhLN2obMnLfhrw6rl++mP6vLvpc6luXOx/xk1L/fXBe0kX/Kfj02Xbyzkfchc+47+Wvr71zy356c+funnrtbV1lX5984zbs++/11Dp/MezV603L/OBb9PS9o+3pLe0xyf99P3djvaB9b2H/Ict6XXibYmX+PBYXOYDXl1l8GFFkE0s7qudux3pP5DpPbR5A7Xinq3PfkilpLkPSS/+pvnsifRxqK/y69GziRX7Y1xU4kNnrNh/EDH1XL8ufnv6DISsU4S9MBD2AAAAClx62nh6Kqqcn4qYr9NEJBO+Eth7WPsrernQUOurZtXrm75SST9duq0K9JJXfJg28+Fe5oP2uJPb7lK8aan09p2+4tZ3hN+27y6+wp6ellwZXPbZ2QewbBWudHfmD1/wx6usrw/QZX3976xqnf+Qo3K131dDta8Ml/YOqvrlPmyNOmT7upNncs4ftw2LfcBPf5WUB5W+4Kv3UB9418zzU8TXzPOP2+dTvrJcPmjHxhEywh4AAAAAFKC2wl7nq0MCAAAAAHYYYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAACpA556Iew3Yzs7WSPop6HFkMkrQu6kF0Yxz/aHH8o8OxjxbHP1oc/2hx/KPDsY9WZzj+uzrnBme7o0uHvc7KzKY756ZFPY7uiuMfLY5/dDj20eL4R4vjHy2Of3Q49tHq7MefaZwAAAAAUIAIewAAAABQgAh74bgl6gF0cxz/aHH8o8OxjxbHP1oc/2hx/KPDsY9Wpz7+rNkDAAAAgAJEZQ8AAAAAChBhL8fM7Hgze9/MFprZZVGPp5CZ2S5m9ryZzTWz98zsG8HtA8zsaTNbEFz2j3qshczM4mY208weDa6PNrM3gr+Be82sJOoxFioz62dm95vZfDObZ2YH8vrPDzP7VvDvzrtmdreZlfHaD4+Z3WZma8zs3Yzbsr7Wzbsu+D3MMbOp0Y28MLRy/H8b/Nszx8weMrN+GfddHhz/983suEgGXUCyHf+M+75jZs7MBgXXef3nWGvH38y+HvwNvGdmv8m4vVO9/gl7OWRmcUk3SDpB0t6SzjGzvaMdVUFLSPqOc25vSQdI+t/geF8m6Vnn3FhJzwbXEZ5vSJqXcf3Xkq5xzu0uaaOkL0Uyqu7hD5L+45wbJ2mS/O+B13/IzGy4pEslTXPOjZcUl3S2eO2H6Q5Jx7e4rbXX+gmSxgZfF0m6KU9jLGR3aOvj/7Sk8c65iZI+kHS5JAX/D58taZ/gMTcG74+w/e7Q1sdfZraLpE9KWppxM6//3LtDLY6/mR0p6TRJk5xz+0i6Ori9073+CXu5tZ+khc65D51z9ZLukX8hIATOuZXOubeD7yvk3+gOlz/mfw02+6ukT0UywG7AzEZIOknSX4LrJukoSfcHm3D8Q2JmfSUdJulWSXLO1TvnNonXf74USephZkWSekpaKV77oXHOvSRpQ4ubW3utnybpb857XVI/M9s5LwMtUNmOv3PuKedcIrj6uqQRwfenSbrHOVfnnFssaaH8+yNsp1Ze/5J0jaTvS8pswMHrP8daOf5flXSVc64u2GZNcHune/0T9nJruKSPM64vC25DyMxslKQpkt6QtJNzbmVw1ypJO0U1rm7gWvn/aFLB9YGSNmW8AeBvIDyjJa2VdHswjfYvZlYuXv+hc84tl/8Ud6l8yNssaYZ47edba691/i/OvwskPRF8z/HPAzM7TdJy59zsFndx/PNjD0mHBlP3XzSzTwS3d7rjT9hDl2dmvSQ9IOmbzrktmfc5326WlrMhMLOTJa1xzs2IeizdVJGkqZJucs5NkVSlFlM2ef2HI1gbdpp84B4mqVxZplghf3itR8fMfiS/rOKuqMfSXZhZT0k/lPTTqMfSjRVJGiC/jOh7kv4ZzG7qdAh7ubVc0i4Z10cEtyEkZlYsH/Tucs49GNy8Oj1lIbhc09rjsUMOlnSqmS2Rn7J8lPwasn7B1DaJv4EwLZO0zDn3RnD9fvnwx+s/fMdIWuycW+uca5D0oPzfA6/9/Grttc7/xXliZudJOlnS513Tubw4/uHbTf7DptnB/8EjJL1tZkPF8c+XZZIeDKbLvik/w2mQOuHxJ+zl1luSxgYd2UrkF2g+HPGYClbwCcqtkuY5536fcdfDks4Nvj9X0r/zPbbuwDl3uXNuhHNulPxr/Tnn3OclPS/pjGAzjn9InHOrJH1sZnsGNx0taa54/efDUkkHmFnP4N+h9LHntZ9frb3WH5b0P0FXwgMkbc6Y7okcMbPj5afxn+qcq86462FJZ5tZqZmNlm8U8mYUYyxUzrl3nHNDnHOjgv+Dl0maGvy/wOs/P/4l6UhJMrM9JJVIWqdO+Pov2vYmaC/nXMLMLpH0pHx3ttucc+9FPKxCdrCkL0p6x8xmBbf9UNJV8uX0L0n6SNKZ0Qyv2/qBpHvM7BeSZipoIIJQfF3SXcGHSx9KOl/+Qzxe/yFyzr1hZvdLelt++tpMSbdIeky89kNhZndLOkLSIDNbJukKtf5v/eOSTpRvjFAt/3eBHdDK8b9cUqmkp4PZa6875y52zr1nZv+U/wAkIel/nXPJaEZeGLIdf+dca/++8PrPsVZe/7dJui04HUO9pHOD6nane/1bU9UdAAAAAFAomMYJAAAAAAWIsAcAAAAABYiwBwAAAAAFiLAHAAAAAAWIsAcAAAAABYiwBwDocsysMrgcZWafy/G+f9ji+mu53H+LfVvwM5zXyv2jzKzGzGYFXzdn3Levmb1jZgvN7LrgnH8AADQi7AEAurJRkjoU9sxsW+eYbRb2nHMHdXBMHXGzpEMkjTSzW81seJZtFjnnJgdfF2fcfpOkL8uftHespONDHCcAoAsi7AEAurKrJB0aVL2+ZWZxM/utmb1lZnPM7CuSZGZHmNnLZvaw/MluZWb/MrMZZvaemV0U3HaVpB7B/u4KbktXES3Y97tBRe2sjH2/YGb3m9l8M7srXWUzs6vMbG4wlquzjP9rks6RdIGky51zy9vzQ5vZzpL6OOdeD07k+zdJn9rOYwgAKFDb+nQTAIDO7DJJ33XOnSxJQWjb7Jz7hJmVSnrVzJ4Ktp0qabxzbnFw/QLn3AYz6yHpLTN7wDl3mZld4pybnOW5Tpc0WdIkSYOCx7wU3DdF0j6SVkh6VdLBZjZP0qcljXPOOTPrl2Wf10u6W9IYSb80syuccytabDPazGZK2iLpx865lyUNl7QsY5tlwW0AADQi7AEACsknJU00szOC633lpzjWS3ozI+hJ0qVm9ung+12C7da3se9DJN3tnEtKWm1mL0r6hHwIe9M5t0ySzGyW/PTS1yXVSrrVzB6V9GiWfX5N0q6SipxzV2a5f6Wkkc659Wa2r6R/mdk+bR0AAADSCHsAgEJikr7unHuy2Y1mR0iqanH9GEkHOueqzewFSWU78Lx1Gd8n5cNbwsz2k3S0pDMkXSLpqMwHBVMwl0i6I9tOnXN16X0752aY2SJJe0haLmlExqYjgtsAAGjEmj0AQFdWIal3xvUnJX3VzIolycz2MLPyLI/rK2ljEPTGSTog476G9ONbeFnSWcG6wMGSDpP0ZmsDM7Nekvo65x6X9C356Z8dYmaDzSwefD9Gvvr4oXNupaQtZnZAsD7wfyT9u6P7BwAUNip7AICubI6kpJnNlq+O/UF+CuXbQQhaq+yNS/4j6eJgXd378lMu026RNMfM3nbOfT7j9ockHShptiQn6fvOuVVBWMymt6R/m1mZfMXx29vx8x0m6Uoza5CUknSxc25DcN/X5H/mHpKeCL4AAGhkfgYJAAAAAKCQMI0TAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAAChBhDwAAAAAKEGEPAAAAAAoQYQ8AAAAACtD/B4rOM+Fru5oeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1080x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=[15, 10])\n",
    "plt.plot(trace_full, label='Full batch')\n",
    "plt.plot(trace_minibatch, label='Mini-batch')\n",
    "plt.xlabel('Iterations * 50')\n",
    "plt.ylabel('Loss $\\mathcal{L}(\\mathbf{w})$')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}