Pre_trained_models_with_Resnet_18_Review_PyTorch.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MtL5zAlkHc7s"
      },
      "source": [
        "<a href=\"http://cocl.us/pytorch_link_top\">\n",
        "    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/Pytochtop.png\" width=\"750\" alt=\"IBM Product \" />\n",
        "</a> \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "37lBAlOaHc7z"
      },
      "source": [
        "<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/cc-logo-square.png\" width=\"200\" alt=\"cognitiveclass.ai logo\" />\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XRZDpeEzHc71"
      },
      "source": [
        "<h1><h1>Pre-trained-Models with PyTorch </h1>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pI0ITQeWHc74"
      },
      "source": [
        "In this lab, you will use pre-trained models to classify between the negative and positive samples; you will be provided with the dataset object. The particular pre-trained model will be resnet18; you will have three questions: \n",
        "\n",
        "<ul>\n",
        "<li>change the output layer</li>\n",
        "<li> train the model</li> \n",
        "<li>  identify  several  misclassified samples</li> \n",
        " </ul>\n",
        "You will take several screenshots of your work and share your notebook. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IvGwmi85Hc79"
      },
      "source": [
        "<h2>Table of Contents</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BiaIhBRhHc7_"
      },
      "source": [
        "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
        "\n",
        "<ul>\n",
        "    <li><a href=\"#download_data\"> Download Data</a></li>\n",
        "    <li><a href=\"#auxiliary\"> Imports and Auxiliary Functions </a></li>\n",
        "    <li><a href=\"#data_class\"> Dataset Class</a></li>\n",
        "    <li><a href=\"#Question_1\">Question 1</a></li>\n",
        "    <li><a href=\"#Question_2\">Question 2</a></li>\n",
        "    <li><a href=\"#Question_3\">Question 3</a></li>\n",
        "</ul>\n",
        "<p>Estimated Time Needed: <strong>120 min</strong></p>\n",
        " </div>\n",
        "<hr>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IMzT-omnHc8D"
      },
      "source": [
        "<h2 id=\"download_data\">Download Data</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GrR18FdTHc8G"
      },
      "source": [
        "Download the dataset and unzip the files in your data directory, unlike the other labs, all the data will be deleted after you close  the lab, this may take some time:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "r4VQfwn0Hc8N",
        "outputId": "4e64df08-4f19-4861-cc1b-5db90c2fdaf7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "--2020-11-10 19:01:32--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Positive_tensors.zip\n",
            "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
            "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2598656062 (2.4G) [application/zip]\n",
            "Saving to: ‘Positive_tensors.zip’\n",
            "\n",
            "Positive_tensors.zi 100%[===================>]   2.42G  42.2MB/s    in 61s     \n",
            "\n",
            "2020-11-10 19:02:33 (40.6 MB/s) - ‘Positive_tensors.zip’ saved [2598656062/2598656062]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "!wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Positive_tensors.zip "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iOeQuqZsHc8d"
      },
      "outputs": [],
      "source": [
        "!unzip -q Positive_tensors.zip"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N5LxY24mHc8r",
        "outputId": "32f190e3-bdcd-4a96-e8d8-8c52a47ac107"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "--2020-11-10 19:07:14--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Negative_tensors.zip\n",
            "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
            "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2111408108 (2.0G) [application/zip]\n",
            "Saving to: ‘Negative_tensors.zip’\n",
            "\n",
            "Negative_tensors.zi 100%[===================>]   1.97G  12.9MB/s    in 52s     \n",
            "\n",
            "2020-11-10 19:08:06 (38.6 MB/s) - ‘Negative_tensors.zip’ saved [2111408108/2111408108]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "! wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0321EN/data/images/Negative_tensors.zip\n",
        "!unzip -q Negative_tensors.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ww9NhktsHc83"
      },
      "source": [
        "We will install torchvision:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lOQaLQ7KHc85",
        "outputId": "645613a5-7149-45c2-ace4-716158b68f75"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: torchvision in /usr/local/lib/python3.6/dist-packages (0.8.1+cu101)\n",
            "Requirement already satisfied: torch==1.7.0 in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.7.0+cu101)\n",
            "Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision) (7.0.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.18.5)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch==1.7.0->torchvision) (0.16.0)\n",
            "Requirement already satisfied: dataclasses in /usr/local/lib/python3.6/dist-packages (from torch==1.7.0->torchvision) (0.7)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.6/dist-packages (from torch==1.7.0->torchvision) (3.7.4.3)\n"
          ]
        }
      ],
      "source": [
        "!pip install torchvision"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "njYIc_BOHc9D"
      },
      "source": [
        "<h2 id=\"auxiliary\">Imports and Auxiliary Functions</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gINkrOaIHc9H"
      },
      "source": [
        "The following are the libraries we are going to use for this lab. The <code>torch.manual_seed()</code> is for forcing the random function to give the same number every time we try to recompile it.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zyPTwhVSHc9J",
        "outputId": "8c2f2942-3c6c-4bdd-a871-9346b3c6e50e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7f06dd8cd678>"
            ]
          },
          "execution_count": 5,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# These are the libraries will be used for this lab.\n",
        "import torchvision.models as models\n",
        "from PIL import Image\n",
        "import pandas\n",
        "from torchvision import transforms\n",
        "import torch.nn as nn\n",
        "import time\n",
        "import torch \n",
        "import matplotlib.pylab as plt\n",
        "import numpy as np\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "import h5py\n",
        "import os\n",
        "import glob\n",
        "torch.manual_seed(0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XXQl9bBJHc9W"
      },
      "outputs": [],
      "source": [
        "from matplotlib.pyplot import imshow\n",
        "import matplotlib.pylab as plt\n",
        "from PIL import Image\n",
        "import pandas as pd\n",
        "import os"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N_hqOeYPHc_G"
      },
      "source": [
        "<!--Empty Space for separating topics-->\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JCtjMt0YHc_H"
      },
      "source": [
        "<h2 id=\"data_class\">Dataset Class</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ZN8RztvHc_K"
      },
      "source": [
        " This dataset class is essentially the same dataset you build in the previous section, but to speed things up, we are going to use tensors instead of jpeg images. Therefor for each iteration, you will skip the reshape step, conversion step to tensors and normalization step.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MiZ_-a6cHc_N",
        "outputId": "81e24565-f4ac-4954-a007-44a8c0595d99"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "done\n"
          ]
        }
      ],
      "source": [
        "# Create your own dataset object\n",
        "\n",
        "class Dataset(Dataset):\n",
        "\n",
        "    # Constructor\n",
        "    def __init__(self,transform=None,train=True):\n",
        "        directory=\"/content\"\n",
        "        positive=\"Positive_tensors\"\n",
        "        negative='Negative_tensors'\n",
        "\n",
        "        positive_file_path=os.path.join(directory,positive)\n",
        "        negative_file_path=os.path.join(directory,negative)\n",
        "        positive_files=[os.path.join(positive_file_path,file) for file in os.listdir(positive_file_path) if file.endswith(\".pt\")]\n",
        "        negative_files=[os.path.join(negative_file_path,file) for file in os.listdir(negative_file_path) if file.endswith(\".pt\")]\n",
        "        number_of_samples=len(positive_files)+len(negative_files)\n",
        "        self.all_files=[None]*number_of_samples\n",
        "        self.all_files[::2]=positive_files\n",
        "        self.all_files[1::2]=negative_files \n",
        "        # The transform is goint to be used on image\n",
        "        self.transform = transform\n",
        "        #torch.LongTensor\n",
        "        self.Y=torch.zeros([number_of_samples]).type(torch.LongTensor)\n",
        "        self.Y[::2]=1\n",
        "        self.Y[1::2]=0\n",
        "        \n",
        "        if train:\n",
        "            self.all_files=self.all_files[0:30000]\n",
        "            self.Y=self.Y[0:30000]\n",
        "            self.len=len(self.all_files)\n",
        "        else:\n",
        "            self.all_files=self.all_files[30000:]\n",
        "            self.Y=self.Y[30000:]\n",
        "            self.len=len(self.all_files)     \n",
        "       \n",
        "    # Get the length\n",
        "    def __len__(self):\n",
        "        return self.len\n",
        "    \n",
        "    # Getter\n",
        "    def __getitem__(self, idx):\n",
        "               \n",
        "        image=torch.load(self.all_files[idx])\n",
        "        y=self.Y[idx]\n",
        "                  \n",
        "        # If there is any transform method, apply it onto the image\n",
        "        if self.transform:\n",
        "            image = self.transform(image)\n",
        "\n",
        "        return image, y, idx\n",
        "    \n",
        "print(\"done\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "76DtVGZwMg5s",
        "outputId": "2c674b68-3a13-42e5-f2f3-59eaf6af171a"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'/content'"
            ]
          },
          "execution_count": 8,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "pwd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dOAIySirHc_X"
      },
      "source": [
        "We create two dataset objects, one for the training data and one for the validation data.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bA6I7-HhHc_Z",
        "outputId": "b408e764-1b8d-4be7-e615-e856471dd059"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "done\n"
          ]
        }
      ],
      "source": [
        "train_dataset = Dataset(train=True)\n",
        "validation_dataset = Dataset(train=False)\n",
        "print(\"done\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nAF89M0pHc_i"
      },
      "source": [
        "<h2 id=\"Question_1\">Question 1</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ckDLEE-XHc_k"
      },
      "source": [
        "<b>Prepare a pre-trained resnet18 model :</b>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "veuJhnoxHc_l"
      },
      "source": [
        "<b>Step 1</b>: Load the pre-trained model <code>resnet18</code> Set the parameter <code>pretrained</code> to true:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86,
          "referenced_widgets": [
            "d9e5d78073e048ff8f68f02073013ee0",
            "e59f53c8a47d45919630bbe1418f1700",
            "2d88c58def74401e9acea356747e0211",
            "dc41bb498d4f49b29f6f717171e27b2c",
            "8f84cdb058ae40d682504dc0da1f893d",
            "03694ed5947943938e79f9970d69e2d6",
            "8924865c73824d55b7be06ee745752af",
            "a00647490ff9463f982832b129370373"
          ]
        },
        "id": "N0auqGlDHc_m",
        "outputId": "c621ac6b-94ee-4648-efa2-5d11f4dda147"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Downloading: \"https://download.pytorch.org/models/resnet18-5c106cde.pth\" to /root/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d9e5d78073e048ff8f68f02073013ee0",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, max=46827520.0), HTML(value='')))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        }
      ],
      "source": [
        "# Step 1: Load the pre-trained model resnet18\n",
        "\n",
        "# Type your code here\n",
        "model = models.resnet18(pretrained=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5lCrxqn-Hc_y"
      },
      "source": [
        "<b>Step 2</b>: Set the attribute <code>requires_grad</code> to <code>False</code>. As a result, the parameters will not be affected by training.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ouGFIATgHc_2"
      },
      "outputs": [],
      "source": [
        "# Step 2: Set the parameter cannot be trained for the pre-trained model\n",
        "\n",
        "\n",
        "# Type your code here\n",
        "for param in model.parameters():\n",
        "    param.requires_grad = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4l4AtkD5Hc_-"
      },
      "source": [
        "<code>resnet18</code> is used to classify 1000 different objects; as a result, the last layer has 1000 outputs.  The 512 inputs come from the fact that the previously hidden layer has 512 outputs. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fZ1ojAscHc__"
      },
      "source": [
        "<b>Step 3</b>: Replace the output layer <code>model.fc</code> of the neural network with a <code>nn.Linear</code> object, to classify 2 different classes. For the parameters <code>in_features </code> remember the last hidden layer has 512 neurons.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "s_dvKHUtHdAE"
      },
      "outputs": [],
      "source": [
        "model.fc = nn.Linear(512, 2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cO9F3BRHHdAN"
      },
      "source": [
        "Print out the model in order to show whether you get the correct answer.<br> <b>(Your peer reviewer is going to mark based on what you print here.)</b>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fikAQuCXHdAP",
        "outputId": "a545d16e-8fb6-4007-9c43-1c033c081839"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "ResNet(\n",
            "  (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
            "  (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "  (relu): ReLU(inplace=True)\n",
            "  (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
            "  (layer1): Sequential(\n",
            "    (0): BasicBlock(\n",
            "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    )\n",
            "    (1): BasicBlock(\n",
            "      (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    )\n",
            "  )\n",
            "  (layer2): Sequential(\n",
            "    (0): BasicBlock(\n",
            "      (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (downsample): Sequential(\n",
            "        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
            "        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      )\n",
            "    )\n",
            "    (1): BasicBlock(\n",
            "      (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    )\n",
            "  )\n",
            "  (layer3): Sequential(\n",
            "    (0): BasicBlock(\n",
            "      (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (downsample): Sequential(\n",
            "        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
            "        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      )\n",
            "    )\n",
            "    (1): BasicBlock(\n",
            "      (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    )\n",
            "  )\n",
            "  (layer4): Sequential(\n",
            "    (0): BasicBlock(\n",
            "      (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (downsample): Sequential(\n",
            "        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
            "        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      )\n",
            "    )\n",
            "    (1): BasicBlock(\n",
            "      (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "      (relu): ReLU(inplace=True)\n",
            "      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
            "      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    )\n",
            "  )\n",
            "  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))\n",
            "  (fc): Linear(in_features=512, out_features=2, bias=True)\n",
            ")\n"
          ]
        }
      ],
      "source": [
        "print(model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2Q6SZ-QvHdAW"
      },
      "source": [
        "<h2 id=\"Question_2\">Question 2: Train the Model</h2>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6DhGvJ3uHdAX"
      },
      "source": [
        "In this question you will train your, model:\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZKZ4DKBwHdAY"
      },
      "source": [
        "<b>Step 1</b>: Create a cross entropy criterion function \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bCa7Si2QHdAa"
      },
      "outputs": [],
      "source": [
        "# Step 1: Create the loss function\n",
        "\n",
        "# Type your code here\n",
        "criterion = nn.CrossEntropyLoss()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WCH2Bx3dHdAj"
      },
      "source": [
        "<b>Step 2</b>: Create a training loader and validation loader object, the batch size should have 100 samples each.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3syspa3cHdAk"
      },
      "outputs": [],
      "source": [
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=100)\n",
        "validation_loader = torch.utils.data.DataLoader(dataset=validation_dataset, batch_size=100)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GnzF6fSqHdAs"
      },
      "source": [
        "<b>Step 3</b>: Use the following optimizer to minimize the loss \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QnFkbCUPHdAt"
      },
      "outputs": [],
      "source": [
        "optimizer = torch.optim.Adam([parameters  for parameters in model.parameters() if parameters.requires_grad],lr=0.001)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UMm5n5Q1HdA0"
      },
      "source": [
        "<!--Empty Space for separating topics-->\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LBHetyuqHdA1"
      },
      "source": [
        "**Complete the following code to calculate  the accuracy on the validation data for one epoch; this should take about 45 minutes. Make sure you calculate the accuracy on the validation data.**\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rQo6L3HgHdA4"
      },
      "outputs": [],
      "source": [
        "n_epochs=1\n",
        "loss_list=[]\n",
        "accuracy_list=[]\n",
        "correct=0\n",
        "N_test=len(validation_dataset)\n",
        "N_train=len(train_dataset)\n",
        "start_time = time.time()\n",
        "#n_epochs\n",
        "\n",
        "Loss=0\n",
        "start_time = time.time()\n",
        "for epoch in range(n_epochs):\n",
        "    loss_sublist = []\n",
        "    for x, y, idx in train_loader:\n",
        "\n",
        "        model.train() \n",
        "        #clear gradient \n",
        "        optimizer.zero_grad()\n",
        "        #make a prediction \n",
        "        z=model(x)\n",
        "        # calculate loss \n",
        "        loss=criterion(z,y)\n",
        "        loss_sublist.append(loss.data.item())\n",
        "        # calculate gradients of parameters \n",
        "        loss.backward()\n",
        "        # update parameters \n",
        "        optimizer.step()\n",
        "        loss_list.append(np.mean(loss_sublist))\n",
        "        \n",
        "    correct=0\n",
        "    for x_test, y_test, idx in validation_loader:\n",
        "        # set model to eval \n",
        "        model.eval()\n",
        "        #make a prediction \n",
        "        z = model(x_test)\n",
        "        #find max \n",
        "        _,yhat = torch.max(z.data, 1)\n",
        "        \n",
        "        #Calculate misclassified  samples in mini-batch \n",
        "        #hint +=(yhat==y_test).sum().item()\n",
        "        correct +=(yhat==y_test).sum().item()\n",
        "   \n",
        "    accuracy=correct/N_test\n",
        "    accuracy_list.append(accuracy)\n",
        "    "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zUia7Y8xHdA-"
      },
      "source": [
        "<b>Print out the Accuracy and plot the loss stored in the list <code>loss_list</code> for every iteration and take a screen shot.</b>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cOQ68apuHdA-",
        "outputId": "755bd7ba-a660-4b1a-843c-6ddd192f3c89"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.995"
            ]
          },
          "execution_count": 34,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "accuracy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        },
        "id": "RmZM3TlVHdBH",
        "outputId": "c42e0adc-816b-404b-8d18-1cbcfcc5e3a2"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEICAYAAABI7RO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5b348c93sgeyQBK2hCXsIohAAHHBBUXQWrBqRW211Vbb/rhdrLel11/9tert1dZeW6/WXlqtVltFUSsuFa27KEvYDWuAAAlbIEAIIdvM9/fHOQmTyWQyiRmSMN/36zWvnPOc55x5Tgbmm2c5zyOqijHGGBMuT0cXwBhjTNdigcMYY0yrWOAwxhjTKhY4jDHGtIoFDmOMMa1igcMYY0yrRDRwiMgMEdksIoUiMi/I8QQRWeAeXyYig9z0y0RkpYisd39e4ndOvIjMF5EtIrJJRK6J5D0YY4xpLDZSFxaRGOAx4DKgGFghIotUdYNfttuAw6o6VETmAA8C1wMHgatUdY+IjAYWA9nuOXcDB1R1uIh4gJ4tlSUzM1MHDRrUXrdmjDFRYeXKlQdVNSswPWKBA5gEFKrqdgAReR6YBfgHjlnAL9zthcCjIiKqutovTwGQJCIJqloN3AqMBFBVH06QCWnQoEHk5+d/wdsxxpjoIiI7g6VHsqkqG9jtt1/MyVpDkzyqWgccBTIC8lwDrFLVahFJd9PuE5FVIvKiiPRu/6IbY4xpTqfuHBeRM3Gar+5wk2KBHOBTVR0PfAY81My5t4tIvojkl5aWnpLyGmNMNIhk4CgB+vvt57hpQfOISCyQBhxy93OAV4CbVXWbm/8QUAm87O6/CIwP9uaqOl9V81Q1LyurSROdMcaYNopk4FgBDBORXBGJB+YAiwLyLAJucbevBd5TVXWbpN4A5qnqkvrM6szI+BpwkZs0jcZ9JsYYYyIsYoHD7bOYizMiaiPwgqoWiMi9IvJlN9sTQIaIFAJ3AvVDducCQ4F7RGSN++rlHvsp8AsRWQd8HfhxpO7BGGNMUxIN06rn5eWpjaoyxpjWEZGVqpoXmN6pO8eNMcZ0PhY4WmHt7iOsLz7a0cUwxpgOFckHAE87sx5z+umLHriyg0tijDEdx2ocxhhjWsUChzHGmFaxwGGMMaZVLHAYY4xpFQscxhhjWsUChzHGmFaxwGGMMaZVLHAYY4xpFQscxhhjWsUCRxtEw8SQxhjTHAscbVDns8BhjIleFjjaoNbr6+giGGNMh7HA0QY1dRY4jDHRywJHG9RYjcMYE8UiGjhEZIaIbBaRQhGZF+R4gogscI8vE5FBbvplIrJSRNa7Py8Jcu4iEfk8kuVvTq3X+jiMMdErYoFDRGKAx4CZwCjgBhEZFZDtNuCwqg4FHgYedNMPAlep6hjgFuCZgGt/BaiIVNlbYk1VxphoFskaxySgUFW3q2oN8DwwKyDPLOBpd3shME1ERFVXq+oeN70ASBKRBAAR6Q7cCdwfwbKHZJ3jxphoFsnAkQ3s9tsvdtOC5lHVOuAokBGQ5xpglapWu/v3Ab8FKtu7wOGyGocxJpp16s5xETkTp/nqDnf/bGCIqr4Sxrm3i0i+iOSXlpa2a7msc9wYE80iGThKgP5++zluWtA8IhILpAGH3P0c4BXgZlXd5uafAuSJSBHwCTBcRD4I9uaqOl9V81Q1Lysrq11uqF6t1TiMMVEskoFjBTBMRHJFJB6YAywKyLMIp/Mb4FrgPVVVEUkH3gDmqeqS+syq+riq9lPVQcD5wBZVvSiC9xCU1TiMMdEsYoHD7bOYCywGNgIvqGqBiNwrIl92sz0BZIhIIU6Hd/2Q3bnAUOAeEVnjvnpFqqytZZ3jxphoFhvJi6vqm8CbAWn3+G1XAdcFOe9+Whg1papFwOh2KWgr1dTZcxzGmOjVqTvHOytrqjLGRDMLHGHy+c2Ia53jxphoZoEjTD6/NTisxmGMiWYWOMLkvwSHdY4bY6KZBY4wNapxWFOVMSaKWeAIkzVVGWOMwwJHmLyNOsdtOK4xJnpZ4AiTfx9HjdfbcQUxxpgOZoEjTI2G49pCTsaYKGaBI0zWOW6MMQ4LHGHyWue4McYAFjjCpv7PcViNwxgTxSxwhMl/VJXVOIwx0cwCR5j8+zjsyXFjTDSzwBEm/6Yqm1bdGBPNLHCEyZqqjDHGYYEjTI2aqqxz3BgTxSIaOERkhohsFpFCEZkX5HiCiCxwjy8TkUFu+mUislJE1rs/L3HTk0XkDRHZJCIFIvJAJMvvzz9wVNbak+PGmOgVscAhIjHAY8BMYBRwg4iMCsh2G3BYVYcCDwMPuukHgatUdQxwC/CM3zkPqepIYBxwnojMjNQ9+KtvqRKBY1W1p+ItjTGmU4pkjWMSUKiq21W1BngemBWQZxbwtLu9EJgmIqKqq1V1j5teACSJSIKqVqrq+wDuNVcBORG8hwb1fRzpSXGUn6g7FW9pjDGdUiQDRzaw22+/2E0LmkdV64CjQEZAnmuAVapa7Z8oIunAVcC77VjmZtU3VaUnx1uNwxgT1Tp157iInInTfHVHQHos8BzwiKpub+bc20UkX0TyS0tLv3BZfG5/eGpSHNV1PqrrrJ/DGBOdIhk4SoD+fvs5blrQPG4wSAMOufs5wCvAzaq6LeC8+cBWVf1dc2+uqvNVNU9V87Kysr7QjYBfjSMpDoBjVdZcZYyJTpEMHCuAYSKSKyLxwBxgUUCeRTid3wDXAu+pqrrNUG8A81R1if8JInI/ToD5YQTL3sTJpioncJSfsOYqY0x0iljgcPss5gKLgY3AC6paICL3isiX3WxPABkiUgjcCdQP2Z0LDAXuEZE17quXWwu5G2eU1io3/VuRugd/VuMwxhhHbCQvrqpvAm8GpN3jt10FXBfkvPuB+5u5rLRnGcNVPxw3LTkegHLrIDfGRKlO3TnemfgPxwWrcRhjopcFjjAF9nHYkFxjTLSywBGm+uG4JzvHrcZhjIlOFjjCVF/jSEmMs2lHjDFRzQJHmOrXHI/xCCkJsZRbH4cxJkpZ4AiTuoHDI0JKYpyNqjLGRC0LHGGq7+OIESHVJjo0xkQxCxxhqm+qEoGUxFjr4zDGRC0LHGFSvz6O1MQ46+MwxkQtCxxhql9m3CNCqtU4jDFRzAJHmHwNNQ7cPg4LHMaY6GSBI0y+hj4OISUxlorqOnw+beEsY4w5/VjgCFNDjUOcPg6fwvEa6+cwxkQfCxxh8u/jSEl0JhW2iQ6NMdHIAkeYfH7DcVPdGXLtIUBjTDSywBGm+v6MGI/VOIwx0c0CR5jq+8E9bh8H2PKxxpjoFNHAISIzRGSziBSKyLwgxxNEZIF7fJmIDHLTLxORlSKy3v15id85E9z0QhF5REROyYqA9U1VHg9W4zDGRLWIBQ4RiQEeA2birBF+g4iMCsh2G3BYVYcCDwMPuukHgatUdQxwC/CM3zmPA98GhrmvGZG6B38+v0kOrY/DGBPNIlnjmAQUqup2Va0BngdmBeSZBTztbi8EpomIqOpqVd3jphcASW7tpC+QqqpL1ZkD5K/A7AjeQ4OGPg4bVWWMiXKRDBzZwG6//WI3LWgeVa0DjgIZAXmuAVaparWbv7iFa0aE16+PIyE2hoRYj/VxGGOiUmxHFyAUETkTp/lqehvOvR24HWDAgAFfuCzq18cBuGtyWI3DGBN9IlnjKAH6++3nuGlB84hILJAGHHL3c4BXgJtVdZtf/pwWrgmAqs5X1TxVzcvKyvqCtwJe38k+DoDUpFjr4zDGRKVIBo4VwDARyRWReGAOsCggzyKczm+Aa4H3VFVFJB14A5inqkvqM6vqXqBcRM5xR1PdDLwawXtoUD8cN8bjBo7EOOvjMMZEpYgFDrfPYi6wGNgIvKCqBSJyr4h82c32BJAhIoXAnUD9kN25wFDgHhFZ4756uce+B/wZKAS2Af+M1D34839yHJwhudbHYYyJRhHt41DVN4E3A9Lu8duuAq4Lct79wP3NXDMfGN2+JW2ZL7CpKjGOPUdOnOpiGGNMh7Mnx8PU0FTlBg5n+VhrqjLGRB8LHGHyBmuqss5xY0wUssARJlXFI85CTuAMx62q9VFbP9+6McZECQscYfL6tKF/A2y+KmNM9LLAESafgsfjHzic+aqOWXOVMSbKWOAIk89tqqpnNQ5jTLSywBEmn08bRlTBycDxlyVF/Gbxpo4qljHGnHKdeq6qzsSrjfs46hdzemmVM+filWP6MapfaoeUzRhjTiWrcbRgw55yig4eR/XkUFw4GTjq/eGDwlNcMmOM6RhW42jBFY98DMDNUwY2zFMFJ5uq6q3cefiUlssYYzqK1TjCFDgct7tf4EiI9XCooqZh6nVjjDmdWeAIU+Bw3LgYD0lxMQCMyU6jxuvjWLWNsDLGnP4scITJ52s8HBdONleNzk4D4FBFzakuljHGnHIWOMLk08bDcSFY4Kg+5eUyxphTzQJHmLyqDfNU1UtJjCMuRhjZJwWAg1bjMMZEAQscYVKl0agqcGocvVMT6ZWSAMBBq3EYY6JAWIFDRH4gIqnieEJEVonI9EgXrjPxBunjuGnyAL570RB6dIsHrI/DGBMdwq1x3Kqq5cB0oAfwdeCBlk4SkRkisllECkVkXpDjCSKywD2+TEQGuekZIvK+iFSIyKMB59wgIutFZJ2IvCUimWHewxcS+OQ4wIzRfblp8kDiYjykJ8dx6LjVOIwxp79wA0f9N+YVwDOqWuCXFvwEkRjgMWAmMAq4QURGBWS7DTisqkOBh4EH3fQq4OfAXQHXjAV+D1ysqmcB63DWJ4+4EzXeRsNxA2V0i7cahzEmKoQbOFaKyNs4gWOxiKQALa1gNAkoVNXtqloDPA/MCsgzC3ja3V4ITBMRUdXjqvoJTgDxJ+6rmzg91anAnjDv4Qs5Xl3XpKnKX0b3BOvjMMZEhXADx23APGCiqlYCccA3WzgnG9jtt1/spgXNo6p1wFEgo7kLqmot8F1gPU7AGAU8EeY9fCGVNd4mTVX+MrvHW+AwxkSFcAPHFGCzqh4Rka8B/xfnS/6UEpE4nMAxDuiH01T1s2by3i4i+SKSX1pa+oXf+3hNXcjA0S8tiZIjJ2zaEWPMaS/cwPE4UCkiY4EfA9uAv7ZwTgnQ328/x00Lmsftv0gDDoW45tkAqrpNnW/oF4Bzg2VU1fmqmqeqeVlZWS0UtWWV1d4mw3H9DcxIpqrWx4FjVuswxpzewg0cde4X9SzgUVV9DEhp4ZwVwDARyRWReGAOsCggzyLgFnf7WuA9Df0newkwSkTqI8FlwMYw7+ELaamPY0BGNwB2Hqo8FcUxxpgOE+606sdE5Gc4w3AvEBEPTj9Hs1S1TkTmAouBGOBJVS0QkXuBfFVdhNM/8YyIFAJlOMEFABEpwun8jheR2cB0Vd0gIr8EPhKRWmAn8I3wb7ftjtfUhRxVNbBnMgA7Dx1nUm7PU1EkY4zpEOEGjuuBG3Ge59gnIgOA37R0kqq+CbwZkHaP33YVcF0z5w5qJv2PwB/DLHe78Skh+ziyeyQR4xF2lVmNwxhzegurqUpV9wF/A9JE5EtAlaq21Mdx2gnVVBUX46FfeiJF1lRljDnNhTvlyFeB5Ti1g68Cy0Tk2kgWrDMKVeMAGNizG7sOHT9FpTHGmI4RblPV3TjPcBwAcDun/4Xz0F7UaClw9O+ZxNsF5aeoNMYY0zHCHVXlqQ8arkOtOLfLChzgFWo4LkBWSiJllTXUeVt6qN4YY7qucGscb4nIYuA5d/96Ajq9T0eBA4PjYloKHAmoQtnxGnqlJkawZMYY03HC7Rz/d2A+cJb7mq+qP41kwToDX0DkGJLVPWT+rO7OuhwHjlXzlyU7uOp/PolY2YwxpqOEW+NAVV8CXopgWTodX0CNY1S/1JD5s9wFnUorqinYU876kqNU1tSRHB/2r9kYYzq9kN9oInIMCPYktwCqqqG/Sbu4wBrHGX1D3279SoClx6opP1ELQMnhEwzr3dJD9sYY03WEDByqGtXfeK1tqsrsfjJwHHUDR/ERCxzGmNPLaT8y6osIbKqKjw3960qKjyElIdapcVTVAU6NwxhjTifW+B6C140cgzO78f1pw8I6JyslgdIKv6aqIxY4jDGnF6txhFD/HMfXpwxk9rjANaiCy0xJaNTHUWw1DmPMacYCRwj1TVUtPTHuLyslgf3lVRyrdpqqNu8rZ7dNfGiMOY1Y4AihvqmqhQfGG+mdktgoUGzZX8FlD39IdZ23vYtnjDEdwgJHCPVNVaHW4QiU3SOpoaYy++x+xMd6qKr1UbDH5rAyxpweLHCE0Jamquz0pIbtK8/qxyc/uRiAVTsPt2vZjDGmo1jgCMGrrW+qyulxMnCkJsbSKzWR7PQkVu860t7FM8aYDhHRwCEiM0Rks4gUisi8IMcTRGSBe3yZiAxy0zNE5H0RqRCRRwPOiReR+SKyRUQ2icg1kSq/r6GPo201jrRkZ3Xd8QN7sHLn4Saz7RpjTFcUscAhIjHAY8BMYBRwg4iMCsh2G3BYVYcCDwMPuulVwM+Bu4Jc+m7ggKoOd6/7YQSKD5ycHbc1gSM9OY7k+BgAUhOdwDEptyf7yqvYcdAWeTLGdH2RrHFMAgpVdbuq1gDPA7MC8swCnna3FwLTRERU9biqfoITQALdCvwXgKr6VPVgZIrv11TVit+SiDTUOtKSnMAxdVgmAB9tKW3fAhpjTAeIZODIBnb77Re7aUHzqGodcBTIaO6CIpLubt4nIqtE5EUR6d1+RW7Mp61vqgJnZFWMRxpqHgMzujEwI5mPtzoxrryqlj98UMgD/9zUMOTXGGO6iq7WOR4L5ACfqup44DPgoWAZReR2EckXkfzS0rb9pa9tDBwjeqeQnZ6E+J03dVgWn20/hNenvJhfzK/f2swfP9zG+pKjbSqbMcZ0lEgGjhKgv99+jpsWNI+IxAJpOMvSNucQUAm87O6/CIwPllFV56tqnqrmZWVltb70QP0KsK0NHD+8dDgLvzulUdro7FQqa7yUHD7BwYrqhvRPt0Wspc0YYyIikoFjBTBMRHJFJB6YAywKyLMIuMXdvhZ4T0MMPXKPvQZc5CZNAza0Z6H91TdVxbTyt5QUH0OvlMZLx+ZmOlOybz9YweHjNfRKSWBE7xQ+2xYqThpjTOcTsdlxVbVOROYCi4EY4ElVLRCRe4F8VV0EPAE8IyKFQBlOcAFARIqAVCBeRGYD01V1A/BT95zfAaXANyN1D/WBQ1pZ4wgmN7MbADsOHudwZQ09kuM5d2gGzy3fZasEGmO6lIh+W6nqm8CbAWn3+G1XAdc1c+6gZtJ3AlPbr5TN87WxqSqYzO7xpCTEOoHjeC09usVx1dh+/GVJEQ+/s4W7rwwcqWyMMZ1TV+scP6Xa2lQVjIiQm9WtUY1j/IAe3DBpAE98ssNm0DXGdBkWOEJoz6YqcJqrtpe6gaNbPADfu2gIPoXX1u1pl/cwxphIs8ARQluf42hObmY3So6c4GBFDT3c6Uj690xm/IB0Fq2xwGGM6RoscIRwcnbc9rne0F7dG7Z7JMc3bH95bD827TvGlv3H2ueNjDEmgixwhFA/yWFMO9U4hvdOadj2DxxXntUPj8CiNXuo9fp4f/MBmxDRGNNpWeAIob7G0V59HIMyujVs9+x2MnBkpSRw7pBMFq3dw7NLd/LNv6zgU3u+o9OrqvWydrdNl2+ijwWOEHxtWI8jlPjYk7/udLePo96ss/uxq6yS/35nCwDvbNjfPm8apX74/OqG32UkVNV6ufWpFcx6bAk/e3k9x9015o2JBhY4Qjg5HLedIocf/xoHwNXjshk3IJ1jVXUkx8fw7qb91lzVRtV1Xv6xZg+PvLuVF/N3t3xCGE7UeNlWWgHAql2HueShD/h02yGmj+rNc8t3cclvP+AvS3Y0NG8aczqzx5VDaO+mKoBJg3qyvKisYThuvdgYD4/eOJ4Fy3fRo1s8v3xtA1sPVDTqFzHh2V56ct2Tf1+4jvKqOkb1TaWwtIILh2UxICO54bjXpwgtryv/+IfbeOTdrVx+Zm827j2GxyP8/VuTOXdoJit3lvHgW5v55WsbWL6jjF9fexYpiXEhr2dMV2aBI4STKwC23zX/dEsea3cfaVjkyV92ehJ3Th9ByZET/PK1DXy0pdQCRxvUj057be75/OGDQu57/eR0Zj27xfOP753HgIxknl++i98s3owI/OTykXx1Yv/mLsma3UdIT47j460HqazxsuD2c5g82FkBYMLAnrxwxxT+/PF2fvXmRtYVH+UHlw7j2vE5LQYkY7oia6oKIRJNVWlJcUwdHnq23uz0JAZndmNJoc2c2xab9x0j1iOM6JPC7+eM4+px2dw8ZSCv/p/z8PqUm55YyjOfFfEfr6xnSK/uDM7szk9eWsd5D7zHV/6whBM13ibX3LS3nGkje/PBv1/ES9+d0hA0/H3rgsG8+J0p9OgWx08WruPfnlsd9FoAFdV13P/6Bj63afVNF2Q1jhB8bVg6tr2cPyyThSuLqanzNepUN8E98u5W8gb14NwhmWzed4whWd0bfm8PX392Q76/3jqJm59czs9fLWBYr+785RsTSYyL4ffvbmXVzsMs2XaQn760jq/m9WdFURmzx2WTmhjLgWPVnNE3hV4piU1mPvY3YWBPXpt7Pn/6eDv/9c9NbCut4Kqx/fjGuYPwiPDUp0UcPVHLvqMn+MeaPTyzdCf/efUYrhmf3a5NosZEkgWOEOpX5+uI/8/nD83kr5/tZNWuw5wT5K/baNTcLMLbSyv473e2kJ4cxz++dx5ri48yZUjw39nY/un8684L2VV2nFF900hyV2m887LhADz2fiG/WbyZRWudJ/kf/2Abs8f1A2Bkn9Swyiki3D51CMN6pXDf6xv4zeLNvF2wj75pSbxVsI9Yj1DnU2af3Y8Dx6q568W1/NebGxnVL5UrxvTllVUlXDMhm+snDmj178iYU8ECRwgawVFVLTlnSAYxHmFJ4UELHMCRyhom3P8vvn7OQH7x5TMbHVu0dg8iUF3r44pHPqayxssNk5rvr8hKSSArJSHose9dNARV5fOScu66fAT3vr6BF/KLARjZt3X9TReP7MXFI3vxzob93LlgDWuLj/KDacO49bxc3t6wjyvG9CUh1sNLq4pZtfMIn20/xM9eXk+MR1heVMbHWw9y3tBMLhnZi96pzddyjDnVLHCE0JFNVamJcYzNSePjrQf58fQRp/z9O5vlO8rw+pSnPi1if3kVXzqrH3uPnuC8oZn8Y3UJk3N78qNLh/Ptv+bzlXHZnDsks03vIyLMvWRYw/5fvjGRx94vpOjQcTK7Bw82LblsVG+W330pG/YeZVz/Hng8wnV5JwPb9RMHcP3EAdTU+Xht7R4m5fbk2aU7eW75Ll5ft5f4WA9zJvbnjguHsGDFbkb1TWHG6L5tKks9n0+t4960mQWOELzt/ABga50/LItH39vK2wX7uGhEL15dU8JVY/uRGBfTMQU6hXYdqqRfeiIeETweYfmOMsB5UHLp9kP88/N9gFMb9PqUu68cxeTBGSz9j2kkxLbf7yfGI3x/2rCWM7YgKT6GCQN7hswTH+vhmgk5APzsijP4yYyR7DhYwROf7OC55bt4dunOhj9mRvZJoWe3eKad0ZtbpgwkNsjc/3/+eDvbSo8zb+ZI0pKcUXzri4/y7wvXUnz4BLPH9eNr5wwMuwnOmHoWOELQdp4dt7Wmj+rNY+8XcvszKxmdncrnJeXUepUbJ3e9tu+K6jq6J4T3z23HweNc/NAHxHoEEfjPq8ewoqiMSbk9+f2ccdR6fXy4uZQYj3D3K+u5enw2l43qDXBaraQY4xGG9krhv75yFndMHcJDb2/mnMEZHKqoYX3JUfYcOcF9r2/gnQ37uH/2GLonxJKVkkCMR6iq9fK7f22lorqOj7aU8quvjKG61ssPF6whLSmOS0b24oX8Yp5duou8gT24cfIAZozuc1r9/kzkSCSfThaRGcDvcZaO/bOqPhBwPAH4KzABOARcr6pFIpIBLAQmAk+p6twg114EDFbV0S2VIy8vT/Pz81td/ldWF/OjBWv54K6LGJTZreUTIuBYVS0/WrCWf210piCZNrIXT3xjYkTfc+XOMo5U1jLtDOfLeNO+cg5V1HDe0LY1/7z1+T6+8+xK7po+nJsmD2zy8GOgRWv38P3nVpOdnkT3hFh2lVVS4/XxvYuGNGm2i+YmF1Vl4cpi/uOV9dR6nf/HgzO7cef04RyrquNnL6/nJzNG8NzyXewuOwHAiN4pPHPbJHqlJnL4eA0vrSrmb8t2sePgcXokx/HTGSOZObovacn2AKMBEVmpqnmB6RH780JEYoDHgMuAYmCFiCxy1w2vdxtwWFWHisgc4EHgeqAK+Dkw2n0FXvsrQEWkyl7P245Lx7ZVSmIcv7p6ND27xVF+oo73Nx9otzXKV+86THpyfMN66PXuebWAgj3lfOPcQeT0SOLBtzZR61W+mpfDr68d2+r3+aSwFICH3t7Co+8X8vdvn8OB8mo27Stn+qg+jOrXuKlk6/5jxHiE9+66kLLjNdz2VD4Deibz9SkDm1w7WoMGOP0x1+X1Z0xOGgUl5RyrquWJJTuY+/fVAGR2T+COqUP45rm5vLPRmcJm5ui+DcOUe3SL51sXDObW83JZXlTGA//cxLyX1/P/FhVwwbAsVJUz+6VSVefjm+cNom9aUkferulEIlkvnQQUqup2ABF5HpgF+AeOWcAv3O2FwKMiIqp6HPhERIYGXlREugN3ArcDL0Su+H6THHbwYxS9UhP59bVj+bTwIG8V7GNxwT6uHpfDwYpqvvV0PjdOGhDyqefmXP2HTwEoeuDKhrQ6r4+t+yvI6ZHEU58WAU7nbp/URJ5ZupPR2Wn0SI5n6/5jPLN0J5eM7M1/Xj06ZL/Lxr3HyBvYg7suH8EPnl/N9f/7WcNfyL/711bum3Um107oz8JVxcw+ux9b9h9jYEYyCbEx9E1L4s0fXNDqe4smI/ukNvRT3HTOQNYVH2Hp9jJG9U0lxiMkxcfw5bH9mj3f4xHOGZzBy989l9W7D/P88t0sLypDFd7ddIBYj/DMZzv5t2lDue38XFZkUmoAABVSSURBVIoOVrJgxW4m5fbg0jN6B+1fMae3SAaObMB/hrliYHJzeVS1TkSOAhlAqEem7wN+C4RcpFtEbscJLgwY0LY+gY7u4wh0zuAMRvRO4X/eLeSqs/rx+AfbWLP7CGt2HyElMZaZY8IfaXOsqrZhe+3uI4ztnw44/Qs1Xh8/nj6c5PhYyk/Ucu2EHCqq63h1TQn3vFoAOAMGRvRJ5aVVxRw9UcPjX5tAnN8XSH0TktenbNhTzpxJ/TlncAYPfOUsfvrSOn48fTiXntGbn760jp+/WsBLq0pYs/sIG/YcZet+m6OrreJiPEwY2LPFjvhgPB5pdK6qcqLWy6GKGu5/YwO/fmszTy0p4lhVHSdqvTy5ZAf90hI5Kyed71w0hLPdf0Pm9Nel/lQQkbOBIar6Skt5VXW+quapal5WVugpPprTGZqq/Hk8wo8uG872g8d5cWUxzy7dyeyz+zGqbyr3v7GRqlpneotar4+v/vEz/ufdrTy7dCfPLt3J0craRtfa5jcR4J8+3t6wvWmfM8/TiN6pXH5mH67L64+IkJIYx8+/NIqv5uUwObcnPbvF8/dvTea+2aP518YD/GjBmoYHJl/M3834+9/h462l7DhYwYlaL6P7pQHOsw3L776U6ycOIKN7Ao/dNJ4Lh2exZvcR+qUl8tzy3Ww/eJzhvbtjOpaIkBwfS/+eyfzv1/N4+tZJTMztycwxfVgy7xL++LUJjM5OI39nGVf/YQnzXlpH2fEam9U5CkSyxlEC+Lef5LhpwfIUi0gskIbTSd6cKUCeiBThlL2XiHygqhe1V6H9dZamKn+XntGLlIRYfvv2ZqrrfHzzvFyO19Rx45+W8fUnlvGrq8dwuLKW5UVlLC8qazjvpVXFvHDHlIZaQeEBp4voyrP68vq6vew89AlVtV52lVUS4xGG9Go6GOC6vP5cl9efOq+P49Ve0pLj+Po5AzlRU8ev3tyEAnMvHsqfP97Bkcpabn1qBdNH9QFgdHZa0PtJiI3hf78+gXc27GfqsCxue3oF+TsPN5vfdJwLh2dxod88a9npScwY3YdjVbU88u5WnlxSxMurS/AIXH5mH/KLDjM5tydfndifybk9T/mUKqt2HWb1riNMGZzRpB/NfDGRDBwrgGEikosTIOYANwbkWQTcAnwGXAu8pyH+XFHVx4HHAURkEPB6pIKG+35A56lxgDP9+pQhGby9YT+Z3eMZk52GxyM8dN1Y7nt9A1c88jGj+qUhArdPHcw5uRkcPVHLDxes4aG3N/OzmWcATuCIixEevOYsNu4pZ195FbmZ3dh6oILk+JiQz0LExnhISz4ZTW+fOoSaOh8P/2srb6zbC8C8mSN5ZVUJb6zfy5fO6huyBpEYF8NVbhv8C3dMYcPecs60/+hdRkpiHHdfOYrr8vrzt6U7Ka2o5tU1exjbP523Cvbx8uoSpgzO4D+vHs3grMjVJAsPVFB44Bj//Hwfy7aXceBYVcNzL1eN7cdPLh9B/57JoS9iwhKxwOH2WcwFFuMMx31SVQtE5F4gX1UXAU8Az4hIIVCGE1wAcGsVqUC8iMwGpgeMyIo4r6/zBQ6AC4Zl8vaG/Vw4vFfDqKJrJ+Rw0YgsvvV0Pmt2H2FSbs+GIAGwvKiM//1wO0Myu9OjWzzPLt3JgJ7JdE+I5bV/O58Yj5AQ6+HF/GIyuoceLhvM3EuGcdPkgfzytQKW7yjjpskDuGZ8Dh9tKWX2uPAn8PN4xGobXdTw3in8cpYzCHJ/eRW9UhKorPHy8qpifr14MzN+/zG3TBnIty8YTOGBCjK6JzC8d/dm/20sWLGLuBgP5w7JxKeKT5VeKYmNJv186/O97DlSRa3Xx2/f2UJNnY8Yj3D5mb3J6ZHM1yYP5MWVu5n/0XbeWLeHs3LSyewezxl9U7lx8oCGkWJen7KvvIrsdGe/us7brg+Snm4i+hxHZ9HW5zie/GQH976+gbX3TO9U49p3l1Vy6X9/yB+/NoGLR/ZqdGzHweNc9T+f8OPpw/nmebkN6VW1Xm7801JW7Tq5RvaVY/ry2E3j2718qmozvZpGDpRX8cBbm/jH6hL8F0kc2SeFW84dRFb3BPYePcHuwycoPlxJenI8f1+2q8l1slIS+M6FQ/hwSyn7jp5g64EK6r/CLhnZi29fMJislHiG9mo8uGLPkRP8bdlOVu48zJHKWrYeqMAjzv+B8QN78Ob6vSzdXsbU4Vlkpyfy/IrdDM7sxncuHMLEQT2p8/nITk9umBQzWjT3HIcFjhD+/PF27n9jI+t+MT3owksdqarW2+wQ2OPVdSTFxTR5xsHrUxYX7CMpPgafTxnRJ4WcHlZ1N6fOjoPHeSF/NyP7pHCsqo6/LdvFxr3lDcfjYz30Skmg+PAJxmSncef04ew9UoVHnLnjFq0tYen2MmI9wrDeKWSnJ/K9i4cSI9IwMjAcu8sqeeKTHby8qpjyqjq6xcdw7YQcXl+3l0PHa7hqbD+2l1ZQsKe80Xnd4mMY2z+djO4JzD67H1OHZzUaTfhF+HzKrU+v4GBFNbPGZjMxtydnuU3RHcUCRxsCx/yPtvGrNzfx+S8vD3u6DGNM+FSV9SVH8Sn0S0sks3sCHo+wYU85/dITSU+Ob5L//c0HSE2MI29Q64ccB/L6lIMV1aQnx5EQG0NlTR1b9ldwdv90fD5l5a7DFB08TmyMUHL4BPvLq1mz+wh7j1ZxsKKahFgPY3PSuX5if8b2T2dwZrc2f9EvXFnMXS+upW9aInuPVgHQJzWRi0dmcc7gDKYMzqBXaiIlR05w8Fg1Z+WkRbxmf8qfHD8dnJwdt2PLYczpSkQ4K6dpTaG5UVAiwiUje7fb+8d4pNGU9cnxsQ3Po3g8wsRBPZkYJEDVen28u3E/K4oO89GWUn784loAhvfuzncuHMJVY/uFXRNRVf7fogKeXbqTsf3TeeW753LoeA0fby1lccE+Xl+7l+eWO4/E5WZ2Y8+RE1TX+TijbyrfPG8QX+6AiU+txhHCHz4o5NdvbWbTfTOiYkZaY0zr+XzKiqIyCksr+OunO9m8/xjZ6UlMP7M3103oT25mNxLjPE1qB/XfvX/4YBu/WbyZGycP4K7pI+gZMJdbnddHwZ5ylu04xPIdZaQmxjF+YA+e+cx5r4xu8cw6O5u8QT2YOjyL37y1iX+scabnnzOxPxeN6NXmNYWsqaoNgaN+Nbgt98+05VuNMS2qb0r7y5Iilu8oo7rOeYp4VN9Uvj01lyvG9OWllSUcrqzh78t2UVFdx9ETtVw5pi+P3jiuVU1Pqspn2w7x5JIdfLTlIDX1TywDF4/I4vM95RyprOGzn01r81oy1lTVBieH43ZwQYwxXUJ9U9olI3tztLKWF1fupqK6jjfW7eVHC9Zy9yufU1njzPAwJKsbEwdlMTo7jW+el9vq/goR4dyhmZw7NJM6r49Ptx1iXfERhvVO4fIz+1Dr9fF5ydE2B41QLHCEUP/keEcsHWuM6drSkuP41gWDAfj+JcP4cGspf1u6i6nDM5lxZh96dItvtxFZsTEepg7PYqrfk/1xMR7GDejRLtdv8n4Rueppor5z3J5JMMZ8ER6PcPGIXlw8olfLmbsAa7gPwedTa6YyxpgAFjhC8KlaM5UxxgSwwBGCT62ZyhhjAlngCMGn1lRljDGBLHCE4PMpMVbjMMaYRixwhODTzjelujHGdDQLHCH4VLG4YYwxjVngCMFGVRljTFMRDRwiMkNENotIoYjMC3I8QUQWuMeXucvBIiIZIvK+iFSIyKN++ZNF5A0R2SQiBSLyQCTL73SOW+Awxhh/EQscIhIDPAbMBEYBN4jIqIBstwGHVXUo8DDwoJteBfwcuCvIpR9S1ZHAOOA8EZkZifIDeH02HNcYYwJFssYxCShU1e2qWgM8D8wKyDMLeNrdXghMExFR1eOq+glOAGmgqpWq+r67XQOsAnIidQOqSjtNJWOMMaeNSH4tZgO7/faL3bSgeVS1DjgKZIRzcRFJB64C3v3CJW2GNVUZY0xTXfLvaRGJBZ4DHlHV7c3kuV1E8kUkv7S0tE3v4/XZcFxjjAkUycBRAvT3289x04LmcYNBGnAojGvPB7aq6u+ay6Cq81U1T1XzsrKymssWkqri6ZKh1RhjIieSX4srgGEikisi8cAcYFFAnkXALe72tcB72sKShCJyP06A+WE7l7cJa6oyxpimIrYeh6rWichcYDEQAzypqgUici+Qr6qLgCeAZ0SkECjDCS4AiEgRkArEi8hsYDpQDtwNbAJWuSOeHlXVP0fiHrz25LgxxjQR0YWcVPVN4M2AtHv8tquA65o5d1Azlz1l3+Q2yaExxjRlLfghqDVVGWNMExY4QvD6LHAYY0wgCxwhOAs5dXQpjDGmc7HAEYLaJIfGGNOEBY4QbD0OY4xpygJHCE4fR0eXwhhjOhcLHCH4VPFY5DDGmEYscISg1lRljDFNWOAIwZqqjDGmKQscIdhcVcYY05QFjhCsqcoYY5qywBGC16ZVN8aYJuxrMQRrqjLGmKYscIRgDwAaY0xTFjhC8NmoKmOMacICRwg+m6vKGGOasMARgjM7rgUOY4zxF9HAISIzRGSziBSKyLwgxxNEZIF7fJmIDHLTM0TkfRGpEJFHA86ZICLr3XMekQh+s1tTlTHGNBWxwCEiMcBjwExgFHCDiIwKyHYbcFhVhwIPAw+66VXAz4G7glz6ceDbwDD3NaP9S++wpipjjGkqkjWOSUChqm5X1RrgeWBWQJ5ZwNPu9kJgmoiIqh5X1U9wAkgDEekLpKrqUlVV4K/A7EjdwAXDshg/oEekLm+MMV1SbASvnQ3s9tsvBiY3l0dV60TkKJABHAxxzeKAa2a3S2mDuOeqwAqSMcaY07ZzXERuF5F8EckvLS3t6OIYY8xpI5KBowTo77ef46YFzSMisUAacKiFa+a0cE0AVHW+quapal5WVlYri26MMaY5kQwcK4BhIpIrIvHAHGBRQJ5FwC3u9rXAe27fRVCquhcoF5Fz3NFUNwOvtn/RjTHGNCdifRxun8VcYDEQAzypqgUici+Qr6qLgCeAZ0SkECjDCS4AiEgRkArEi8hsYLqqbgC+BzwFJAH/dF/GGGNOEQnxB/5pIy8vT/Pz8zu6GMYY06WIyEpVzQtMP207x40xxkSGBQ5jjDGtYoHDGGNMq0RFH4eIlAI723h6Js0/kNjV2L10TnYvnc/pch/wxe5loKo2eZ4hKgLHFyEi+cE6h7oiu5fOye6l8zld7gMicy/WVGWMMaZVLHAYY4xpFQscLZvf0QVoR3YvnZPdS+dzutwHROBerI/DGGNMq1iNwxhjTKtY4GhGS8vednYiUuQusbtGRPLdtJ4i8o6IbHV/dspVqkTkSRE5ICKf+6UFLbs4HnE/p3UiMr7jSt5UM/fyCxEpcT+bNSJyhd+xn7n3sllELu+YUgcnIv3dJZ03iEiBiPzATe9yn02Ie+lyn42IJIrIchFZ697LL930XHdJ7kJ3ie54Nz3okt2toqr2CnjhTMq4DRgMxANrgVEdXa5W3kMRkBmQ9mtgnrs9D3iwo8vZTNmnAuOBz1sqO3AFzkSXApwDLOvo8odxL78A7gqSd5T7by0ByHX/DcZ09D34la8vMN7dTgG2uGXucp9NiHvpcp+N+/vt7m7HAcvc3/cLwBw3/Y/Ad93t7wF/dLfnAAta+55W4wgunGVvuyL/pXqfJoLL7n4RqvoRzmzJ/por+yzgr+pYCqS7Swx3Cs3cS3NmAc+rarWq7gAKcf4tdgqquldVV7nbx4CNOCtwdrnPJsS9NKfTfjbu77fC3Y1zXwpcgrMkNzT9XJos2d2a97TAEVywZW8jtkRthCjwtoisFJHb3bTe6qxpArAP6N0xRWuT5sreVT+ruW7zzZN+TYZd5l7c5o1xOH/ddunPJuBeoAt+NiISIyJrgAPAOzg1oiOqWudm8S9voyW7gfolu8NmgeP0db6qjgdmAv9HRKb6H1Snntolh9R15bK7HgeGAGcDe4HfdmxxWkdEugMvAT9U1XL/Y13tswlyL13ys1FVr6qejbMq6iRgZCTfzwJHcOEse9upqWqJ+/MA8ArOP6b99U0F7s8DHVfCVmuu7F3us1LV/e5/dB/wJ042eXT6exGROJwv2r+p6stucpf8bILdS1f+bABU9QjwPjAFp2mwfrE+//K2dsnuJixwBBfOsredloh0E5GU+m1gOvA5jZfqvYWutexuc2VfBNzsjuA5Bzjq12zSKQW081+N89mAcy9z3FEvucAwYPmpLl9z3HbwJ4CNqvrffoe63GfT3L10xc9GRLJEJN3dTgIuw+mzeR9nSW5o+rmEvWR3UB09IqCzvnBGhGzBaSu8u6PL08qyD8YZAbIWKKgvP0475rvAVuBfQM+OLmsz5X8Op5mgFqdt9rbmyo4zouQx93NaD+R1dPnDuJdn3LKuc/8T9/XLf7d7L5uBmR1d/oB7OR+nGWodsMZ9XdEVP5sQ99LlPhvgLGC1W+bPgXvc9ME4wa0QeBFIcNMT3f1C9/jg1r6nPTlujDGmVaypyhhjTKtY4DDGGNMqFjiMMca0igUOY4wxrWKBwxhjTKtY4DCmFUTkU/fnIBG5sZ2v/R/B3suYzsaG4xrTBiJyEc4sql9qxTmxenLuoGDHK1S1e3uUz5hIshqHMa0gIvWzkD4AXOCu2fAjd5K534jICneCvDvc/BeJyMcisgjY4Kb9w518sqB+AkoReQBIcq/3N//3cp+8/o2IfC7OGivX+137AxFZKCKbRORvrZ3l1Ji2iG05izEmiHn41TjcAHBUVSeKSAKwRETedvOOB0arMx03wK2qWuZOD7FCRF5S1XkiMledieoCfQVn0r2xQKZ7zkfusXHAmcAeYAlwHvBJ+9+uMSdZjcOY9jEdZ16mNTjTc2fgzGcEsNwvaAB8X0TWAktxJpsbRmjnA8+pM/nefuBDYKLftYvVmZRvDTCoXe7GmBCsxmFM+xDg31R1caNEpy/keMD+pcAUVa0UkQ9w5g5qq2q/bS/2f9qcAlbjMKZtjuEsOVpvMfBdd6puRGS4OzNxoDTgsBs0RuIs8Vmvtv78AB8D17v9KFk4y9F2iplZTXSyv06MaZt1gNdtcnoK+D1OM9Eqt4O6lOBL874FfEdENuLMsrrU79h8YJ2IrFLVm/zSX8FZX2EtzoyuP1HVfW7gMeaUs+G4xhhjWsWaqowxxrSKBQ5jjDGtYoHDGGNMq1jgMMYY0yoWOIwxxrSKBQ5jjDGtYoHDGGNMq1jgMMYY0yr/Hwl+u0Ok/J96AAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "plt.plot(loss_list)\n",
        "plt.xlabel(\"iteration\")\n",
        "plt.ylabel(\"loss\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GfKyHa66HdBR"
      },
      "source": [
        "<h2 id=\"Question_3\">Question 3:Find the misclassified samples</h2> \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dZz2w2OtHdBS"
      },
      "source": [
        "<b>Identify the first four misclassified samples using the validation data:</b>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4YcVyiyMZUBF"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Sth1-PODHdBT",
        "outputId": "f4b71e0a-58e5-40ed-f1e4-82428e6ee660"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "sample 280 yhat: tensor([0]) y:tensor([1])\n",
            "sample 424 yhat: tensor([0]) y:tensor([1])\n",
            "sample 516 yhat: tensor([0]) y:tensor([1])\n",
            "sample 638 yhat: tensor([0]) y:tensor([1])\n"
          ]
        }
      ],
      "source": [
        "validation_loader = torch.utils.data.DataLoader(dataset=validation_dataset, batch_size=1)\n",
        "\n",
        "# Plot the misclassified samples\n",
        "count = 0\n",
        "for (x_test, y_test, idx) in validation_loader:\n",
        "    z = model(x_test.reshape(-1,3,224,224))\n",
        "    _,yhat = torch.max(z.data, 1)\n",
        "    if yhat != y_test:\n",
        "        print(\"sample {} yhat: {} y:{}\".format(tf.reshape(idx, []), yhat, y_test))\n",
        "        count += 1\n",
        "    if count >= 4:\n",
        "        break    "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hH75BTupHdBZ"
      },
      "source": [
        "<a href=\"https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/share-notebooks.html\"> CLICK HERE </a> Click here to see how to share your notebook.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ybzHLlj1HdBa"
      },
      "source": [
        "<h2>About the Authors:</h2> \n",
        "\n",
        "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KHKvoYYVHdBe"
      },
      "source": [
        "## Change Log\n",
        "\n",
        "| Date (YYYY-MM-DD) | Version | Changed By | Change Description                                          |\n",
        "| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n",
        "| 2020-09-21        | 2.0     | Shubham    | Migrated Lab to Markdown and added to course repo in GitLab |\n",
        "\n",
        "<hr>\n",
        "\n",
        "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0XfY5Fr5HdBg"
      },
      "source": [
        "Copyright © 2018 <a href=\"cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu\">cognitiveclass.ai</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "Pre_trained_models_with_Resnet_18_Review_PyTorch.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.7",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.9"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "03694ed5947943938e79f9970d69e2d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": "initial"
          }
        },
        "2d88c58def74401e9acea356747e0211": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8924865c73824d55b7be06ee745752af",
            "placeholder": "​",
            "style": "IPY_MODEL_a00647490ff9463f982832b129370373",
            "value": " 44.7M/44.7M [1:02:25&lt;00:00, 12.5kB/s]"
          }
        },
        "8924865c73824d55b7be06ee745752af": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8f84cdb058ae40d682504dc0da1f893d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a00647490ff9463f982832b129370373": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d9e5d78073e048ff8f68f02073013ee0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e59f53c8a47d45919630bbe1418f1700",
              "IPY_MODEL_2d88c58def74401e9acea356747e0211"
            ],
            "layout": "IPY_MODEL_dc41bb498d4f49b29f6f717171e27b2c"
          }
        },
        "dc41bb498d4f49b29f6f717171e27b2c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e59f53c8a47d45919630bbe1418f1700": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "100%",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8f84cdb058ae40d682504dc0da1f893d",
            "max": 46827520,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_03694ed5947943938e79f9970d69e2d6",
            "value": 46827520
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}