notebooks/03_sgd_from_scratch.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D7ExutUSGoY-"
      },
      "source": [
        "# บทที่ 3 - Stochastic Gradient Descent ตั้งแต่เริ่มต้น"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GtlT-MCeHftk"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ai-builders/curriculum/blob/main/notebooks/03_sgd_from_scratch.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jUUXviwGayJP"
      },
      "source": [
        "ในบทเรียนนี้ เราจะทำการสร้างวิธีที่โมเดลของเราเรียนรู้ในบทเรียนที่แล้วๆมา เรียกว่า stochastic gradient descent ขึ้นมาเองตั้งแต่ต้นโดยใช้เพียงแค่ Pytorch สำหรับ linear algebra และการทำ partial derivatives เท่านั้น ด้วยตัวอย่างการจำแนกรูปภาพตัวเลข 3 และ 7 ออกจากกัน\n",
        "\n",
        "บทเรียนแปล-สรุปมาจาก [04_mnist_basics.ipynb](https://github.com/fastai/fastbook/blob/master/04_mnist_basics.ipynb) ของ [fastai](https://course.fast.ai/) ผู้ที่สนใจสามารถไปติดตามบทเรียนต้นทางได้ที่ [course.fast.ai](https://course.fast.ai/)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6IlUoU8YGlQ7",
        "outputId": "96ea0400-f259-4d81-dcbf-4a7665998010"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[K     |████████████████████████████████| 720 kB 12.8 MB/s \n",
            "\u001b[K     |████████████████████████████████| 188 kB 49.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 1.2 MB 45.6 MB/s \n",
            "\u001b[K     |████████████████████████████████| 60 kB 5.6 MB/s \n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "#ติดตั้ง fastai\n",
        "!pip install -q fastbook\n",
        "import fastbook"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "-2VyXV7kpoju"
      },
      "outputs": [],
      "source": [
        "from fastai.vision.all import *\n",
        "from fastbook import *\n",
        "import torch\n",
        "\n",
        "matplotlib.rc('image', cmap='Greys')\n",
        "\n",
        "#fix plot_function as new pytorch requires steps argument for torch.linspace\n",
        "def plot_function(f, tx=None, ty=None, title=None, min=-2, max=2, figsize=(6,4)):\n",
        "    x = torch.linspace(min, max, steps=100)\n",
        "    fig,ax = plt.subplots(figsize=figsize)\n",
        "    ax.plot(x,f(x))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gBWvqPiOpoju"
      },
      "source": [
        "# เทรนโมเดลจำแนกรูปเลข 3 และเลข 7 จาก [ชุดข้อมูล MNIST](http://yann.lecun.com/exdb/mnist/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_5WWrXnD6Vpj"
      },
      "source": [
        "## โหลดข้อมูลรูปเลข 3 และเลข 7"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 37
        },
        "id": "wkh3fHu5pojw",
        "outputId": "c0fe5213-0052-462a-b8ee-84882e0c73cd"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      <progress value='3219456' class='' max='3214948' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      100.14% [3219456/3214948 00:00<00:00]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "path = untar_data(URLs.MNIST_SAMPLE)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z6OjxHEPpojw",
        "outputId": "b1c65a6c-315f-4233-fe92-215e817ecdb3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#3) [Path('/root/.fastai/data/mnist_sample/valid'),Path('/root/.fastai/data/mnist_sample/labels.csv'),Path('/root/.fastai/data/mnist_sample/train')]"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "#แบ่งเป็น train, validation, test \n",
        "path.ls()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mMmCdDscpojw",
        "outputId": "fc98608c-0131-4be1-d56e-922e924a03af"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#2) [Path('/root/.fastai/data/mnist_sample/train/7'),Path('/root/.fastai/data/mnist_sample/train/3')]"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "#ในแต่ละ set จะมีเลข 3 และ 7\n",
        "(path/'train').ls()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EilsFTaWpojx",
        "outputId": "97aecb3b-fce5-4f7c-9b62-e6d43253e616"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#6131) [Path('/root/.fastai/data/mnist_sample/train/3/10.png'),Path('/root/.fastai/data/mnist_sample/train/3/10000.png'),Path('/root/.fastai/data/mnist_sample/train/3/10011.png'),Path('/root/.fastai/data/mnist_sample/train/3/10031.png'),Path('/root/.fastai/data/mnist_sample/train/3/10034.png'),Path('/root/.fastai/data/mnist_sample/train/3/10042.png'),Path('/root/.fastai/data/mnist_sample/train/3/10052.png'),Path('/root/.fastai/data/mnist_sample/train/3/1007.png'),Path('/root/.fastai/data/mnist_sample/train/3/10074.png'),Path('/root/.fastai/data/mnist_sample/train/3/10091.png')...]"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "#ในละ folder 3 และ 7 จะเป็นไฟล์รูป\n",
        "threes = (path/'train'/'3').ls().sorted()\n",
        "sevens = (path/'train'/'7').ls().sorted()\n",
        "threes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2nKh8v3OBG5G",
        "outputId": "9442d7c1-6125-4d88-c81a-e7e87d8b49ce"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(6131, 6265)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "#จำนวนรูปในแต่ละ class\n",
        "len(threes), len(sevens)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 45
        },
        "id": "jJEUNNQKpojx",
        "outputId": "16277235-20fb-47c8-d1f4-bb6bbb0483a9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7F883E8AFED0>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAA3UlEQVR4nGNgGNxA1XnOxX//vxlgkVIqe//379+///4+EkCX4lv//e+/v38vnf33968wumTd379/rzUZ8lf9e1XChC4Zf39RAAMDA+eFf2txuIg18htuObW/f9/y45CM/Pt3iwYOuaxvv7Zz4pBjmPvvoxwSF9XNrxhZ/XFpZOCf9+tjMk5Zhsl/d+AyloHhI4MsTo22T//uF8cu5Xbo/79/8eiiXGVlyqyOk3/9/fe7nRld0urv3/uH/v79+/d1GKZ5In///v339+/zDdiClbVu9sF/X2cr4XQoVQEA4o1d+YAEyFQAAAAASUVORK5CYII=\n"
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "im3_path = threes[6000]\n",
        "im3 = Image.open(im3_path)\n",
        "im3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GBYX4Mf0BTb6",
        "outputId": "b77ba285-8176-455e-bd86-6d16b75f9acd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "#รูปขนาด 28 x 28 pixels \n",
        "array(im3).shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RZphr7qzpojx",
        "outputId": "ba2bf4b9-1cdd-4583-8734-abf34a064f96"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0, 0, 0, 0, 0, 0],\n",
              "       [0, 0, 0, 0, 0, 0],\n",
              "       [0, 0, 0, 0, 0, 0],\n",
              "       [0, 0, 0, 0, 0, 0],\n",
              "       [0, 0, 0, 0, 0, 0],\n",
              "       [0, 0, 0, 0, 0, 0]], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "#เปลี่ยนจาก numpy array\n",
        "array(im3)[4:10,4:10]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "o52Dig-Vpojx",
        "outputId": "9295e887-7371-4494-fca8-829823e96a64"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0, 0, 0, 0, 0, 0],\n",
              "        [0, 0, 0, 0, 0, 0],\n",
              "        [0, 0, 0, 0, 0, 0],\n",
              "        [0, 0, 0, 0, 0, 0],\n",
              "        [0, 0, 0, 0, 0, 0],\n",
              "        [0, 0, 0, 0, 0, 0]], dtype=torch.uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "#เป็น torch tensor\n",
        "tensor(im3)[4:10,4:10]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 394
        },
        "id": "0yZCaBRNpojy",
        "outputId": "9e3f04a1-6be4-48db-ac47-3ee450e0c536"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x7f883e8af710>"
            ],
            "text/html": [
              "<style type=\"text/css\">\n",
              "#T_e84c1_row0_col0, #T_e84c1_row0_col1, #T_e84c1_row0_col2, #T_e84c1_row0_col3, #T_e84c1_row0_col4, #T_e84c1_row0_col5, #T_e84c1_row0_col6, #T_e84c1_row0_col7, #T_e84c1_row0_col8, #T_e84c1_row0_col9, #T_e84c1_row0_col10, #T_e84c1_row0_col11, #T_e84c1_row0_col12, #T_e84c1_row0_col13, #T_e84c1_row0_col14, #T_e84c1_row0_col15, #T_e84c1_row0_col16, #T_e84c1_row0_col17, #T_e84c1_row1_col0, #T_e84c1_row1_col1, #T_e84c1_row1_col2, #T_e84c1_row1_col3, #T_e84c1_row1_col4, #T_e84c1_row1_col5, #T_e84c1_row1_col6, #T_e84c1_row1_col7, #T_e84c1_row1_col8, #T_e84c1_row1_col9, #T_e84c1_row2_col0, #T_e84c1_row2_col1, #T_e84c1_row2_col2, #T_e84c1_row2_col3, #T_e84c1_row2_col4, #T_e84c1_row2_col5, #T_e84c1_row2_col6, #T_e84c1_row2_col7, #T_e84c1_row3_col0, #T_e84c1_row3_col1, #T_e84c1_row3_col2, #T_e84c1_row3_col3, #T_e84c1_row3_col4, #T_e84c1_row3_col5, #T_e84c1_row4_col0, #T_e84c1_row4_col1, #T_e84c1_row4_col2, #T_e84c1_row4_col3, #T_e84c1_row4_col4, #T_e84c1_row4_col5, #T_e84c1_row5_col0, #T_e84c1_row5_col1, #T_e84c1_row5_col2, #T_e84c1_row5_col3, #T_e84c1_row5_col4, #T_e84c1_row5_col5, #T_e84c1_row5_col10, #T_e84c1_row5_col11, #T_e84c1_row5_col12, #T_e84c1_row5_col17, #T_e84c1_row6_col0, #T_e84c1_row6_col1, #T_e84c1_row6_col2, #T_e84c1_row6_col3, #T_e84c1_row6_col4, #T_e84c1_row6_col5, #T_e84c1_row6_col6, #T_e84c1_row6_col7, #T_e84c1_row6_col8, #T_e84c1_row6_col9, #T_e84c1_row6_col10, #T_e84c1_row6_col16, #T_e84c1_row6_col17, #T_e84c1_row7_col0, #T_e84c1_row7_col1, #T_e84c1_row7_col2, #T_e84c1_row7_col3, #T_e84c1_row7_col4, #T_e84c1_row7_col5, #T_e84c1_row7_col6, #T_e84c1_row7_col7, #T_e84c1_row7_col8, #T_e84c1_row7_col9, #T_e84c1_row7_col16, #T_e84c1_row7_col17, #T_e84c1_row8_col0, #T_e84c1_row8_col1, #T_e84c1_row8_col2, #T_e84c1_row8_col3, #T_e84c1_row8_col4, #T_e84c1_row8_col5, #T_e84c1_row8_col6, #T_e84c1_row8_col7, #T_e84c1_row8_col8, #T_e84c1_row8_col9, #T_e84c1_row8_col15, #T_e84c1_row8_col16, #T_e84c1_row8_col17, #T_e84c1_row9_col0, #T_e84c1_row9_col1, #T_e84c1_row9_col2, #T_e84c1_row9_col3, #T_e84c1_row9_col4, #T_e84c1_row9_col5, #T_e84c1_row9_col6, #T_e84c1_row9_col7, #T_e84c1_row9_col8, #T_e84c1_row9_col14, #T_e84c1_row9_col15, #T_e84c1_row9_col16, #T_e84c1_row9_col17, #T_e84c1_row10_col0, #T_e84c1_row10_col1, #T_e84c1_row10_col2, #T_e84c1_row10_col3, #T_e84c1_row10_col4, #T_e84c1_row10_col5, #T_e84c1_row10_col6, #T_e84c1_row10_col7, #T_e84c1_row10_col8, #T_e84c1_row10_col13, #T_e84c1_row10_col14, #T_e84c1_row10_col15, #T_e84c1_row10_col16, #T_e84c1_row10_col17 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #ffffff;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row1_col10, #T_e84c1_row7_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #ececec;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row1_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #d6d6d6;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row1_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #767676;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row1_col13 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #363636;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row1_col14, #T_e84c1_row1_col15, #T_e84c1_row2_col11, #T_e84c1_row2_col12, #T_e84c1_row2_col13, #T_e84c1_row2_col16, #T_e84c1_row3_col9, #T_e84c1_row3_col10, #T_e84c1_row3_col11, #T_e84c1_row3_col12, #T_e84c1_row3_col16, #T_e84c1_row3_col17, #T_e84c1_row4_col6, #T_e84c1_row4_col7, #T_e84c1_row4_col8, #T_e84c1_row4_col9, #T_e84c1_row6_col14, #T_e84c1_row7_col12, #T_e84c1_row7_col13, #T_e84c1_row8_col11, #T_e84c1_row8_col12, #T_e84c1_row10_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #000000;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row1_col16, #T_e84c1_row6_col13 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #080808;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row1_col17, #T_e84c1_row4_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #e4e4e4;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row2_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #eeeeee;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row2_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #a0a0a0;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row2_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #111111;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row2_col14, #T_e84c1_row2_col15, #T_e84c1_row3_col15, #T_e84c1_row4_col15, #T_e84c1_row5_col15 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #010101;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row2_col17 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #1f1f1f;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row3_col6 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f2f2f2;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row3_col7 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #616161;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row3_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #070707;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row3_col13 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #353535;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row3_col14 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #3d3d3d;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row4_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #303030;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row4_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #929292;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row4_col13, #T_e84c1_row7_col15 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f8f8f8;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row4_col14 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #9d9d9d;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row4_col16 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #161616;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row4_col17 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #a3a3a3;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row5_col6 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #505050;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row5_col7 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #232323;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row5_col8 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #6f6f6f;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row5_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #cbcbcb;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row5_col13, #T_e84c1_row9_col13 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #fbfbfb;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row5_col14 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #393939;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row5_col16 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #636363;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row6_col11, #T_e84c1_row7_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #fdfdfd;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row6_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #c2c2c2;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row6_col15 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #656565;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row7_col14 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #141414;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row8_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #c3c3c3;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row8_col13 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #5c5c5c;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row8_col14 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #eaeaea;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row9_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #afafaf;\n",
              "  color: #000000;\n",
              "}\n",
              "#T_e84c1_row9_col10 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #090909;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row9_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #030303;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row9_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #585858;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row10_col9 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #757575;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row10_col11 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #0e0e0e;\n",
              "  color: #f1f1f1;\n",
              "}\n",
              "#T_e84c1_row10_col12 {\n",
              "  font-size: 6pt;\n",
              "  background-color: #f1f1f1;\n",
              "  color: #000000;\n",
              "}\n",
              "</style>\n",
              "<table id=\"T_e84c1_\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr>\n",
              "      <th class=\"blank level0\" >&nbsp;</th>\n",
              "      <th class=\"col_heading level0 col0\" >0</th>\n",
              "      <th class=\"col_heading level0 col1\" >1</th>\n",
              "      <th class=\"col_heading level0 col2\" >2</th>\n",
              "      <th class=\"col_heading level0 col3\" >3</th>\n",
              "      <th class=\"col_heading level0 col4\" >4</th>\n",
              "      <th class=\"col_heading level0 col5\" >5</th>\n",
              "      <th class=\"col_heading level0 col6\" >6</th>\n",
              "      <th class=\"col_heading level0 col7\" >7</th>\n",
              "      <th class=\"col_heading level0 col8\" >8</th>\n",
              "      <th class=\"col_heading level0 col9\" >9</th>\n",
              "      <th class=\"col_heading level0 col10\" >10</th>\n",
              "      <th class=\"col_heading level0 col11\" >11</th>\n",
              "      <th class=\"col_heading level0 col12\" >12</th>\n",
              "      <th class=\"col_heading level0 col13\" >13</th>\n",
              "      <th class=\"col_heading level0 col14\" >14</th>\n",
              "      <th class=\"col_heading level0 col15\" >15</th>\n",
              "      <th class=\"col_heading level0 col16\" >16</th>\n",
              "      <th class=\"col_heading level0 col17\" >17</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
              "      <td id=\"T_e84c1_row0_col0\" class=\"data row0 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col1\" class=\"data row0 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col2\" class=\"data row0 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col3\" class=\"data row0 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col4\" class=\"data row0 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col5\" class=\"data row0 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col6\" class=\"data row0 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col7\" class=\"data row0 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col8\" class=\"data row0 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col9\" class=\"data row0 col9\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col10\" class=\"data row0 col10\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col11\" class=\"data row0 col11\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col12\" class=\"data row0 col12\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col13\" class=\"data row0 col13\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col14\" class=\"data row0 col14\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col15\" class=\"data row0 col15\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col16\" class=\"data row0 col16\" >0</td>\n",
              "      <td id=\"T_e84c1_row0_col17\" class=\"data row0 col17\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
              "      <td id=\"T_e84c1_row1_col0\" class=\"data row1 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col1\" class=\"data row1 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col2\" class=\"data row1 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col3\" class=\"data row1 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col4\" class=\"data row1 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col5\" class=\"data row1 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col6\" class=\"data row1 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col7\" class=\"data row1 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col8\" class=\"data row1 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col9\" class=\"data row1 col9\" >0</td>\n",
              "      <td id=\"T_e84c1_row1_col10\" class=\"data row1 col10\" >37</td>\n",
              "      <td id=\"T_e84c1_row1_col11\" class=\"data row1 col11\" >67</td>\n",
              "      <td id=\"T_e84c1_row1_col12\" class=\"data row1 col12\" >156</td>\n",
              "      <td id=\"T_e84c1_row1_col13\" class=\"data row1 col13\" >209</td>\n",
              "      <td id=\"T_e84c1_row1_col14\" class=\"data row1 col14\" >254</td>\n",
              "      <td id=\"T_e84c1_row1_col15\" class=\"data row1 col15\" >255</td>\n",
              "      <td id=\"T_e84c1_row1_col16\" class=\"data row1 col16\" >246</td>\n",
              "      <td id=\"T_e84c1_row1_col17\" class=\"data row1 col17\" >48</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
              "      <td id=\"T_e84c1_row2_col0\" class=\"data row2 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col1\" class=\"data row2 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col2\" class=\"data row2 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col3\" class=\"data row2 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col4\" class=\"data row2 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col5\" class=\"data row2 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col6\" class=\"data row2 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col7\" class=\"data row2 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row2_col8\" class=\"data row2 col8\" >34</td>\n",
              "      <td id=\"T_e84c1_row2_col9\" class=\"data row2 col9\" >118</td>\n",
              "      <td id=\"T_e84c1_row2_col10\" class=\"data row2 col10\" >239</td>\n",
              "      <td id=\"T_e84c1_row2_col11\" class=\"data row2 col11\" >253</td>\n",
              "      <td id=\"T_e84c1_row2_col12\" class=\"data row2 col12\" >253</td>\n",
              "      <td id=\"T_e84c1_row2_col13\" class=\"data row2 col13\" >253</td>\n",
              "      <td id=\"T_e84c1_row2_col14\" class=\"data row2 col14\" >253</td>\n",
              "      <td id=\"T_e84c1_row2_col15\" class=\"data row2 col15\" >254</td>\n",
              "      <td id=\"T_e84c1_row2_col16\" class=\"data row2 col16\" >253</td>\n",
              "      <td id=\"T_e84c1_row2_col17\" class=\"data row2 col17\" >226</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
              "      <td id=\"T_e84c1_row3_col0\" class=\"data row3 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row3_col1\" class=\"data row3 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row3_col2\" class=\"data row3 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row3_col3\" class=\"data row3 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row3_col4\" class=\"data row3 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row3_col5\" class=\"data row3 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row3_col6\" class=\"data row3 col6\" >14</td>\n",
              "      <td id=\"T_e84c1_row3_col7\" class=\"data row3 col7\" >175</td>\n",
              "      <td id=\"T_e84c1_row3_col8\" class=\"data row3 col8\" >247</td>\n",
              "      <td id=\"T_e84c1_row3_col9\" class=\"data row3 col9\" >253</td>\n",
              "      <td id=\"T_e84c1_row3_col10\" class=\"data row3 col10\" >254</td>\n",
              "      <td id=\"T_e84c1_row3_col11\" class=\"data row3 col11\" >253</td>\n",
              "      <td id=\"T_e84c1_row3_col12\" class=\"data row3 col12\" >253</td>\n",
              "      <td id=\"T_e84c1_row3_col13\" class=\"data row3 col13\" >210</td>\n",
              "      <td id=\"T_e84c1_row3_col14\" class=\"data row3 col14\" >205</td>\n",
              "      <td id=\"T_e84c1_row3_col15\" class=\"data row3 col15\" >254</td>\n",
              "      <td id=\"T_e84c1_row3_col16\" class=\"data row3 col16\" >253</td>\n",
              "      <td id=\"T_e84c1_row3_col17\" class=\"data row3 col17\" >253</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
              "      <td id=\"T_e84c1_row4_col0\" class=\"data row4 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row4_col1\" class=\"data row4 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row4_col2\" class=\"data row4 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row4_col3\" class=\"data row4 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row4_col4\" class=\"data row4 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row4_col5\" class=\"data row4 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row4_col6\" class=\"data row4 col6\" >126</td>\n",
              "      <td id=\"T_e84c1_row4_col7\" class=\"data row4 col7\" >253</td>\n",
              "      <td id=\"T_e84c1_row4_col8\" class=\"data row4 col8\" >253</td>\n",
              "      <td id=\"T_e84c1_row4_col9\" class=\"data row4 col9\" >253</td>\n",
              "      <td id=\"T_e84c1_row4_col10\" class=\"data row4 col10\" >214</td>\n",
              "      <td id=\"T_e84c1_row4_col11\" class=\"data row4 col11\" >130</td>\n",
              "      <td id=\"T_e84c1_row4_col12\" class=\"data row4 col12\" >49</td>\n",
              "      <td id=\"T_e84c1_row4_col13\" class=\"data row4 col13\" >15</td>\n",
              "      <td id=\"T_e84c1_row4_col14\" class=\"data row4 col14\" >122</td>\n",
              "      <td id=\"T_e84c1_row4_col15\" class=\"data row4 col15\" >254</td>\n",
              "      <td id=\"T_e84c1_row4_col16\" class=\"data row4 col16\" >234</td>\n",
              "      <td id=\"T_e84c1_row4_col17\" class=\"data row4 col17\" >116</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
              "      <td id=\"T_e84c1_row5_col0\" class=\"data row5 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col1\" class=\"data row5 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col2\" class=\"data row5 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col3\" class=\"data row5 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col4\" class=\"data row5 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col5\" class=\"data row5 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col6\" class=\"data row5 col6\" >95</td>\n",
              "      <td id=\"T_e84c1_row5_col7\" class=\"data row5 col7\" >223</td>\n",
              "      <td id=\"T_e84c1_row5_col8\" class=\"data row5 col8\" >162</td>\n",
              "      <td id=\"T_e84c1_row5_col9\" class=\"data row5 col9\" >80</td>\n",
              "      <td id=\"T_e84c1_row5_col10\" class=\"data row5 col10\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col11\" class=\"data row5 col11\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col12\" class=\"data row5 col12\" >0</td>\n",
              "      <td id=\"T_e84c1_row5_col13\" class=\"data row5 col13\" >9</td>\n",
              "      <td id=\"T_e84c1_row5_col14\" class=\"data row5 col14\" >208</td>\n",
              "      <td id=\"T_e84c1_row5_col15\" class=\"data row5 col15\" >254</td>\n",
              "      <td id=\"T_e84c1_row5_col16\" class=\"data row5 col16\" >173</td>\n",
              "      <td id=\"T_e84c1_row5_col17\" class=\"data row5 col17\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
              "      <td id=\"T_e84c1_row6_col0\" class=\"data row6 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col1\" class=\"data row6 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col2\" class=\"data row6 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col3\" class=\"data row6 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col4\" class=\"data row6 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col5\" class=\"data row6 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col6\" class=\"data row6 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col7\" class=\"data row6 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col8\" class=\"data row6 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col9\" class=\"data row6 col9\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col10\" class=\"data row6 col10\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col11\" class=\"data row6 col11\" >5</td>\n",
              "      <td id=\"T_e84c1_row6_col12\" class=\"data row6 col12\" >89</td>\n",
              "      <td id=\"T_e84c1_row6_col13\" class=\"data row6 col13\" >246</td>\n",
              "      <td id=\"T_e84c1_row6_col14\" class=\"data row6 col14\" >254</td>\n",
              "      <td id=\"T_e84c1_row6_col15\" class=\"data row6 col15\" >173</td>\n",
              "      <td id=\"T_e84c1_row6_col16\" class=\"data row6 col16\" >0</td>\n",
              "      <td id=\"T_e84c1_row6_col17\" class=\"data row6 col17\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
              "      <td id=\"T_e84c1_row7_col0\" class=\"data row7 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col1\" class=\"data row7 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col2\" class=\"data row7 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col3\" class=\"data row7 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col4\" class=\"data row7 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col5\" class=\"data row7 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col6\" class=\"data row7 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col7\" class=\"data row7 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col8\" class=\"data row7 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col9\" class=\"data row7 col9\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col10\" class=\"data row7 col10\" >5</td>\n",
              "      <td id=\"T_e84c1_row7_col11\" class=\"data row7 col11\" >38</td>\n",
              "      <td id=\"T_e84c1_row7_col12\" class=\"data row7 col12\" >253</td>\n",
              "      <td id=\"T_e84c1_row7_col13\" class=\"data row7 col13\" >253</td>\n",
              "      <td id=\"T_e84c1_row7_col14\" class=\"data row7 col14\" >237</td>\n",
              "      <td id=\"T_e84c1_row7_col15\" class=\"data row7 col15\" >15</td>\n",
              "      <td id=\"T_e84c1_row7_col16\" class=\"data row7 col16\" >0</td>\n",
              "      <td id=\"T_e84c1_row7_col17\" class=\"data row7 col17\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
              "      <td id=\"T_e84c1_row8_col0\" class=\"data row8 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col1\" class=\"data row8 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col2\" class=\"data row8 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col3\" class=\"data row8 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col4\" class=\"data row8 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col5\" class=\"data row8 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col6\" class=\"data row8 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col7\" class=\"data row8 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col8\" class=\"data row8 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col9\" class=\"data row8 col9\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col10\" class=\"data row8 col10\" >89</td>\n",
              "      <td id=\"T_e84c1_row8_col11\" class=\"data row8 col11\" >253</td>\n",
              "      <td id=\"T_e84c1_row8_col12\" class=\"data row8 col12\" >253</td>\n",
              "      <td id=\"T_e84c1_row8_col13\" class=\"data row8 col13\" >180</td>\n",
              "      <td id=\"T_e84c1_row8_col14\" class=\"data row8 col14\" >40</td>\n",
              "      <td id=\"T_e84c1_row8_col15\" class=\"data row8 col15\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col16\" class=\"data row8 col16\" >0</td>\n",
              "      <td id=\"T_e84c1_row8_col17\" class=\"data row8 col17\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
              "      <td id=\"T_e84c1_row9_col0\" class=\"data row9 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col1\" class=\"data row9 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col2\" class=\"data row9 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col3\" class=\"data row9 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col4\" class=\"data row9 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col5\" class=\"data row9 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col6\" class=\"data row9 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col7\" class=\"data row9 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col8\" class=\"data row9 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col9\" class=\"data row9 col9\" >106</td>\n",
              "      <td id=\"T_e84c1_row9_col10\" class=\"data row9 col10\" >246</td>\n",
              "      <td id=\"T_e84c1_row9_col11\" class=\"data row9 col11\" >250</td>\n",
              "      <td id=\"T_e84c1_row9_col12\" class=\"data row9 col12\" >183</td>\n",
              "      <td id=\"T_e84c1_row9_col13\" class=\"data row9 col13\" >9</td>\n",
              "      <td id=\"T_e84c1_row9_col14\" class=\"data row9 col14\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col15\" class=\"data row9 col15\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col16\" class=\"data row9 col16\" >0</td>\n",
              "      <td id=\"T_e84c1_row9_col17\" class=\"data row9 col17\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e84c1_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
              "      <td id=\"T_e84c1_row10_col0\" class=\"data row10 col0\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col1\" class=\"data row10 col1\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col2\" class=\"data row10 col2\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col3\" class=\"data row10 col3\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col4\" class=\"data row10 col4\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col5\" class=\"data row10 col5\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col6\" class=\"data row10 col6\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col7\" class=\"data row10 col7\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col8\" class=\"data row10 col8\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col9\" class=\"data row10 col9\" >157</td>\n",
              "      <td id=\"T_e84c1_row10_col10\" class=\"data row10 col10\" >254</td>\n",
              "      <td id=\"T_e84c1_row10_col11\" class=\"data row10 col11\" >241</td>\n",
              "      <td id=\"T_e84c1_row10_col12\" class=\"data row10 col12\" >30</td>\n",
              "      <td id=\"T_e84c1_row10_col13\" class=\"data row10 col13\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col14\" class=\"data row10 col14\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col15\" class=\"data row10 col15\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col16\" class=\"data row10 col16\" >0</td>\n",
              "      <td id=\"T_e84c1_row10_col17\" class=\"data row10 col17\" >0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "im3_t = tensor(im3)\n",
        "df = pd.DataFrame(im3_t[4:15,4:22])\n",
        "df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jMdNhmiXpojy"
      },
      "source": [
        "## วิธีที่ง่ายที่สุด: ดูว่ามี pixel เหมือนกันแค่ไหน"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MqYq_GbFT7XA"
      },
      "source": [
        "สมมุติเราไม่ทำ ML อะไรเลย แต่เราพยามสร้างกฎแบบ Rule-based Systems โดยบอกว่า \"รูปที่มี pixels ใกล้เคียงกับ pixels เฉลี่ยของเลข 3 และ 7 มากกว่า ให้ถือว่าเป็นเลขนั้น\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JDsXZWi8pojy",
        "outputId": "34c39b1d-84e9-43ed-8364-e30f8b4969a3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(6131, 6265)"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "seven_tensors = [tensor(Image.open(o)) for o in sevens]\n",
        "three_tensors = [tensor(Image.open(o)) for o in threes]\n",
        "len(three_tensors),len(seven_tensors)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "id": "9h_vpbbvpojy",
        "outputId": "3873c893-9aad-43cb-be95-7ba2aa239f63"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJHElEQVR4nO2bXXMSZxuAL1jYXYQsiCYxIWIIjImJ0TaVTu2H41FnnGmPPOtMf0NP+i/6H9oDx+OOOtMjW6cdGz/Sk1ZqxkRDhCTQEAjfsLDsvgeWbbMmxgrEzDtcR5ln+bi5eJ5n7/t+iM0wDPr8g/1tB3DY6Aux0BdioS/EQl+IBcc+1/+fb0G23Qb7M8RCX4iFvhALfSEW+kIs9IVY6Aux0BdiYb/ErGMMw6DVatFsNqlUKmiaRrPZpNls0mg0Xnq8LMtIkoSu6+i6jtPpxOFw4HQ6EQQBWZZxOHoXds+FaJpGvV4nHo/zww8/kMlkSKVSxONxnjx5wr/7MTabjffee4+pqSmq1SqqqjIyMsKxY8eIRCIMDw8zOzuLz+frWbxdF6LrujkDisUilUqFbDbLn3/+yeLiIoVCgfX1dTKZDJVKBVEUEUWRer1OrVZjZWUFh8NhCqlUKmxublIulzl+/DgTExM9FWLbp2P2n2sZVVXZ3t5mdXWV77//nlQqxaNHj8jlcqTTaQzDwDAMZFnG7XYzODhIIBDgyZMnPH/+HLvdjt1uN2eOzWbDZrPh8/nwer1cv36daDT6hh93B7vWMh3PEE3TqNVq1Go1UqkUlUqFVCrFysoKT58+pVQqIQgCExMTRKNRnE4nkiQhiiIulwtFUfB6vczOzpLJZFheXubZs2eUy2VqtZr5Po1GA1VV0XW905BfScdCVFUlFovx22+/8c0331CpVGg2m7RaLRqNBoFAgGg0ysWLF7l69Soejwe3220+3263Y7PZ0HUdwzC4ceMG165dIxaLkUwmOw3vP9OxEMMwaDQaVKtVSqUS1WoVXdex2+2IokggEGBubo5z587h8/lwOp04nU7z+e0l8e+lJAgCNtvOGe33+wkGg8iy3GnIr6QrQqrVKrVaDVVV0TQNAFEUOXr0KHNzc3zxxRf4fD4URXnpg7ZpjzudTux2+0vXpqenmZmZQVGUTkN+JR0LcTqdhEIhRFEkn8/TbDaBF0Lcbjdzc3N4vV5EUdxTBrzYizRNI5/Pk8/nzRzFZrMhCALDw8OEw2FcLlenIb+SjoVIksTp06cJh8N88MEH5nh7KQiCgCiK+75Oo9GgVCqxtrZGMpmkWq0CmM+PRCJEo9Ed+08v6FhI+1sXBGHH3tC+Zp3+e7G+vs7PP//M77//TqlUQtM0BEEgHA4zPj7OzMwMJ06ceOk9uk1XErP2bHidmbAXd+7c4auvvkLTNHRdx+FwIIoiFy9eJBqN8u6773LixIluhPtKep66WzEMA13XqdfrlMtlM2FbWFgwN2S73U4kEiESifDhhx8SjUZ7vpm2OXAhuq6jaRrZbJbHjx/z448/cvPmTbLZrHm7djgcXLhwgY8//phPP/2UkydPHlh8B1LtGoZBsVhkdXXVrGWSySSrq6ssLS2Ry+Wo1+vAi3zD7/dz+vRpzp8/z8DAQK9D3MGBlf/JZJLvvvuO5eVlHjx4gKqqO1LzNkNDQ0xPT3PhwgUmJyd7fpu1ciBLRtd1CoUCjx49Yn19HVVVzXzFSiaTIRaLcevWLeLxOMeOHcPj8TA2NobX62VwcLCnkg5syWxtbfHgwQM0TTM31t3IZDJkMhnW19dxu90oioLH4+HKlSucP3+eS5cuIcvyK5O8Tuh6+f/SC/y9ZDY2Nrh9+zblcplCoUCxWCSXy5mPi8fjLC0tmT0UURRxOp1mv2RqaorR0VE++ugjzpw5w9mzZ/F6vQiC8Nq5joVdjfZciJVarWbKWFtbM8d/+uknfvnlF1ZXV0mn07s+t91Ri0QifP3110xOTiJJEoIgvEkovemH/FecTieKoiDL8o7O19DQEJcuXWJtbY10Ok0mkyGXy3H//n3i8TjwYrYlEgmq1SrPnz9ncHCQ48ePv6mQXTlwIQ6HA4fDgcvlwuv1muPDw8NMT09TrVbNW/TKygrZbNYUArC5uUkulyMejxMKhTh69Gh34+vqq3VAuxBsd9VlWSYYDBKPx/nrr79IJBJsb28DL2bKxsYG8XicU6dOdTWOQ3Mu0y4EJUkye63BYJDZ2VmmpqZ2pO6GYZgzR1XVrsZxaITsRbtwtI55vd6eVL+HXgjAbndCt9uN3+/v6oYKh2gPsVIsFtne3mZhYYGHDx/uyFlsNhunTp0iEol01HLYjUMppF0MJpNJEokEiURiR2YrCAJDQ0P4/f6uH2seOiHVapVKpcK9e/e4c+cOf/zxh3lEATA5Ocn4+DiBQABZlt80S92TQyOk/YHr9TrZbJZYLMb8/DypVMq8ZrfbCQaDRCIRvF4vDoej6zXNoRFSKBRIp9Pcvn2bu3fv8vjxY5LJpNknGRgYwO12c/XqVS5fvszo6Oiu5zed8laFtCthwzDI5/M8ffqUhw8fcuPGDbO3Ci820SNHjjA4OMi5c+cIhUI9kQFvUYiqqqiqysbGBktLS9y9e5f5+XkSiQTNZtNcJi6XC1mW+fLLL/nkk08Ih8M9kwFdFvLvb9yaULXH2383Gg0KhQLxeJz5+XkWFha4d++e+fj2me/AwACKonD27FneeecdPB5Pz2RAF4Vomka1WqVcLrO2toaiKIyMjJiH3pVKha2tLUqlEpubmywvL7O4uGjWKfl8HvgnM41EIoTDYa5cuUI0GiUUCqEoSk9/PQRdFNJqtSiVSmxtbRGLxRgdHUWSJPMXRLlcjuXlZba2tsxl0u6tqqq645RPFEXGx8cJh8NEo1FmZmbMhlGv6ZqQfD7Pt99+SzKZ5P79++Zhd6vVotVqmecwqqqaM6ZSqZgb5/DwMGNjY7z//vtmk/nkyZMoioIkSV3PN/aia0IajQaJRIJnz56xuLj4Wj9ssdlsSJKEJEmMjY0RiUQ4c+YM0WiUiYkJ/H5/t8J7bbomxOPxcPnyZTweD7/++uu+QmRZ5siRI3z++ed89tlnhEIhRkZGcLlcB7Y8dqNrQhwOB8FgkHQ6TSAQMI8l98LlcuHz+ZicnGR2dpahoaEdHbS3RdeazK1WyzxvKRaL+7/x3w0ht9ttdsm6XcrvF8KugwfddT9E9P+j6nXoC7HQF2KhL8TCfrfd3lVRh5T+DLHQF2KhL8RCX4iFvhALfSEW/gcMlBno19ugeQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "show_image(three_tensors[1]);"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D5B7mZNspojy",
        "outputId": "0d8770e5-0ccc-4af4-8781-f871c04e7b14"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([6131, 28, 28]), torch.Size([6265, 28, 28]))"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "#เรามัดเลข 3 และ 7 ทั้งหมดรวมกันตามคลาสแล้วหารมันด้วย 255 เพื่อให้ได้ค่าระหว่าง 0 และ 1\n",
        "stacked_sevens = torch.stack(seven_tensors).float()/255\n",
        "stacked_threes = torch.stack(three_tensors).float()/255\n",
        "stacked_threes.shape, stacked_sevens.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K4mmFyngpojz",
        "outputId": "03cb8688-0ce1-44a6-c239-25654368389a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "len(stacked_threes.shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Jck_enHBpojz",
        "outputId": "7c732263-92aa-40d7-caec-2c98b91026aa"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "stacked_threes.ndim"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 103
        },
        "id": "cxcx0MIlg-fI",
        "outputId": "b7af8b59-816c-4937-fe79-d9b1514d5a3d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f883d9cc910>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAIEElEQVR4nO2bS08TbRuArx6ntJQONBQoYIFgEPCQYBQ0Rl2YuDFGoy504Q/xJ7j1B7hwR+LGjboQSZRo8IQBLIdWC5aDI1Baep525luYzgulKF++TjHv1yvpZmae9u41z9z3c2gNqqpS5R+MBx3A30ZVSBFVIUVUhRRRFVKE+Q/n/80lyFDqYLWHFFEVUkRVSBFVIUVUhRRRFVLEn8pu2VEUhXw+Tz6fR5ZlMpkMqVQKs9mMxWLBbDZjMpm06wVBwGKxYDCUrJJlp+JCstksW1tbrK6uMjs7y/v373nz5g0+nw+v16u9Cpw5c4bm5maMRmNFpFRMiCzLpNNp1tfX+fbtG9++fWN+fp5AIEAoFCKbzZJMJolEIqysrABgNBrp7u6mvr4eq9WK2ax/uBUTEo1GmZiY4MWLFzx69IhUKkUymSSXy5HP51laWuL9+/cYjUaMxl+pzWAwIIoiDoeDlpYWamtrdY+zYkIsFgv19fXYbDaSySTpdJp0Oq2dz+fzJdvNzc3x7t07hoaGMJlMWp7RDVVVf/cqG7Isq4lEQh0eHlZbWlpUp9Op8muu9NuXy+VS29vb1QcPHqh+v1+Nx+PlCqnkd65Y2TUajZjNZlpaWjh9+jRtbW2YTCYtUZrNZux2+667n0ql2Nzc5MePH0iShCzL+sap67tv/yCjEavVSldXF3fv3uXixYsIgqAJqK2tpampCYfDsaNdNpslHo8TDAYZHx8nHo/rG6eu716C2tpaenp6aG9vx+VyIQgCACaTCUEQdoxBtuPxeDhy5Ag1NTW6xldxIU6nk76+PoaGhvD5fLhcLgCsVit1dXVYLJZdbQwGA8ePH+fs2bO6V5qKD8wKOcPtdjM0NIQoilq1kSRpR+U5CCoupIDX6+XWrVuMjIwQi8UIhUKEQqE9ry8M+fXmwCZ3NTU1tLe3c/78eW7cuMG5c+dwu90lc4SqqgQCAfx+P6lUSte4DqyHOBwOHA4HTU1NDAwM4Ha7CYVCLC4u7vrSqqoyPj7O1tYWXq8XURR1i+vAhKTTaeLxOIlEgmg0SiAQYHV1lUQisetag8GA2+2mpaUFq9Wqa1wHJmRrawu/38/y8jLhcJhPnz7x/ft31D32mj0eDz09PbqX3YoJyeVyyLLM2toac3NzzM/PMzMzQywWIxqNMjc3t6cMgKamJrq7u7HZbLrGWVEh8XicsbEx7t+/jyRJ/PjxA0VRUBTlt20NBgM+n4/Ozk5tIKcXFasy+XyeWCyGJEmsra2RSCRQFOW3vaKAqqrMz8/z5cuXf0+VyeVybGxsIEkSP3/+RJblP/aM7czNzWGz2Whra9NGt3pQMSGCINDR0cGFCxcIh8P4/X4+fvy4r0cGIJlMEovFdB+cVUyIzWbDZrPR39/PlStXEASBycnJffUUVVVJJBLE43FyuZyucVa87IqiyODgIKIo0tDQQDabJZvNaudVVUVRFMbHx5menkaW5YoM2QtUXIjdbsdut+N0OmlsbCSTyZDJZLTzBSHpdJpgMKhtWVSKAxuY1dTU4PP5Sk7aFEXh4sWLKIrC69evCQaDrK6u4nA42NjYIJ1OY7FY9lw7+V84MCGCIOw5plBVlRMnThCLxVhYWCAYDLK5ucni4iKxWIxMJoPJZNJFyF+3lVl4ZKampnj8+DEzMzPAP2uyem9YHVgP2YuCkEAgwOjoqHa8sAXxfydkdXWVYDDI/Py8dsxgMDAwMMCxY8doa2vDZrPp8riADkIK+xvb7+J/c0clSWJsbIzv37/vOO71eunv70cUxZLrruWibEIURdGG569evaKhoYGOjg5EUcTtdv+xfS6XI5fLMT09zdOnT3f1kK6uLk6ePIndbi9XyCUpW1JVVRVZlvn58yfPnj1jdHSUUChEJBLZcxK3fccsl8uRyWRYXFzk8+fPrK+vA79kmEwmmpqaaGxs1LV3QBl7yNbWFs+fP2dycpKXL1/idrtZWlri1KlTXL9+HavVitVqxWKxIAgC6XSaVCqlbXr7/X4+fPjAq1evtJkwwODgIL29vfT19emaOwqUTUgqlWJiYoJAIEA4HCYSiZBOp3E6nUiSpK2hOhwOzGYz2WyWSCRCNBolEonw9u1bRkZGCIVC2nzFaDTS2dnJsWPHaGho0MqunpRNiNPp5OrVq3z69InFxUXW1tZYWFjgyZMnzM7OarmksDYaDof5+vUr8XicaDTK8vIy6+vr2r5MQ0MDoihy6dIlLl++TH19/Y69YL0omxCz2UxrayvxeJy2tjby+TwrKyuEw2FCoRC1tbV4PB6am5s5dOgQX79+xe/3k8lkdkzu4FfecLlc+Hw+Dh8+jMfjqYgMAMMfVqz2/dNuRVHIZrMsLS3x8OFDVFXFarXi9/sZHh7WNrutVqv2G5FkMrkr4dpsNgRB4N69e9y8eROPx4PdbsdgMJRbSMk3K1sPMRqN2Gw26urqaG1tRRAEvF4vsixjt9u1JJnJZDQRpWaxdrsdURQ5ceIEXV1d5Qpv35R9YCaKIrdv3wZ+df3CD+YKQjY3N5EkCb/fz9TUlNauUH3u3LnDtWvXOHr0aLlD2xdlF2KxWBBFUZuTNDc3MzAwoPUGSZJwuVwoioIkSVq7worakSNH6O3txel0lju0fVG2HFKy8bYBV+FzCo9KJpPZsdNfyBGiKFJTU1OJElsyh+gq5C+n+n+Z/VAVUkRVSBFVIUVUhRRRFVLEnwZmlfmTyl9EtYcUURVSRFVIEVUhRVSFFFEVUsR/AP0FXN1zCRLUAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "#เลข 3 ที่ index 0\n",
        "show_image(stacked_threes[0,:,:])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "id": "2Es3hPcXpojz",
        "outputId": "492d2c79-b892-48a9-f20d-8c4eba02024e"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJtUlEQVR4nO1b2XLiWhJM7QsChDG22x3h//+qfnKzWVhoX5HmoaNqDufK9jRge2aCiiCEAS0nVUtWlqz0fY+r/dvU776A/za7AiLZFRDJroBIdgVEMv2D7/+fS5Ay9OHVQyS7AiLZFRDJroBI9lFS/RQ7tV1QlME8eFH7dEDkxdPf4udDAMmLVxQFfd8Pfn5JuyggQ4vs+/7o1XUdfy6+l01RFCiKAlX9E9WqqvJn9JJ/fwm7CCDy4ruu423XdTgcDjgcDqiqCm3boq5rtG2LpmlwOBzQti3vr6oqVFWFaZrQNA2WZUHXdViWBU3ToOs6NE3j3xFQZOcCcxYgQ15AIHRdh7Zt0bYtqqpCXdcoigJlWaIoCtR1jTzPUdc1yrLk4+i6DlVVMRqNYFnW0dY0Tdi2DcMwYBgGNE1jEGRgTrWTARHBkD2haRpUVYU8z1EUBcIwRJIk2Gw2CMMQu90OaZoiiiKUZYk8z3E4HNB1HUzThGEY8DwPo9EIi8UCs9kMj4+PmM1mmM/ncF0X4/EYlmUdeY4YYqeCc7aHDAFSliWqqkKSJMjzHLvdDmEYYrPZII5jBEGALMsQRRGyLEOWZRw6dOdnsxk8z0PXdSiKAoqioKoq6LqOw+EAXdfR9z17CYXPUOL9dEDkXCHmiKqqEMcx0jTFdrtFGIZ4fn7Gfr/HZrNBmqYIggBJkiCKIlRVhbIs0bYtDocDn8NxHFiWhYeHB9ze3iIMQ9zc3CDLMtze3qJtW0wmEyiKAsdxjpLvOXZ2UhXzBy1K3LZty+GlaRpM08RoNIKiKNA0jQFpmgZt27Knkaf0fY+6rlHXNYdhlmX8N53DNE2+ji/3EAJiKHfQhdZ1zVVEVVVYlsVxb9s270P7U/WhvNM0DZqmga7rnJgp76iqivl8Dk3TMB6Poes6uq7jkKEbcAowJ4eMaHRiimNd19kTyHNs24Zt27xQApRAITAJkKIokOc5NE07SpbyfqKHXsIuRszoonVdh+M4nOw8z4Pv++wB8j60QAJgv98jjmOuTMRZdP3PpYqe+R4Q31Jl6MQiGADQdR2Tp6ZpOESImYpGn2dZBl3XUVUViqJgPkILo5xD5zEM44ikib87x04CRD45ubVpmnyRXdfBcZyjaiTfTSJvTdPAsiyYpvmPUAFwlBNM02SCRpxlCLxT7SwPES9AVVVehBgKsnuLpZq25BVhGGK/32O/3yNNU+R5zhVLVVUYhnHEXonWEykb6nG+HBDxLhIQcqKTgaBS3Pc9V4+Xlxes12usViu8vLwgiiLs93tesKZpsG0bk8kEvu9jNBrBdV0GhRL6uXYyIDIQiqKg67pBUESeIvKJJEmYwa7Xa2y3WwRBgN1uhyzLkOc5JpMJl2rXdeF5HsbjMRzHOQpRsRv+FkBEUAgEIlJvtfvU4NHdXy6XWK1W7BUESBiGHIau60LTNDiOg8lkwpTecRw4jnPkHd+WQwgA8b28JRAOhwN3tHEcI4oiDo3lcsmhst1usdlsUBQFqqri5EkVi7jNUHW5VIU5GZD/BBSRxRZFgSRJsN1usVqt8OvXL6zXazw/P+P3799YLpeI4xhJkvAxPc+D53kA/lQxSqhi6y+LRnQt59jFRGbxQihcyDuKouCmbrPZYLPZIAgCLJdLBEGANE1R1zULRCLPAHBUiaiBpLZAZqvnMtazc8iQZirS67qukWUZ4jjGer1mQJbLJZbLJZIkQZIkXIYVRTnyAjpWVVUsFbiue9QMUh/zrSHznhFIIg+h5EpyoO/7WCwWGI1GmEwmDKSmaZxEKUwo7KIoQhAE3NRR9yxrr8A3UnfRZJFZJmZEvy3Lgud5uLu7Y3WNTNRLAcAwDHRdhzzPoaoqXl9foWkabm5uoOs6bNvm4367h7w3SlAUhXOB67po2xY/fvxghlkUBbIs49AiI/BkQbrve65UqqoiiiJomgbXdTnviJ5C1/C3dramKr8XL0am2/P5HI7jwHVdFn1kSk+9DVH3OI4ZOGK1qqpiv9/DsixUVcVeJHriqXaWHjK0FUsxjRNc14VhGDBNE3VdYz6fs2fIgJCC9vr6ijiOmXiRStY0DVet0WiEsixhmib3O8SWvyyHvFVVxO+o+pD7ip0pdbhDs5yu61igtm2bQ4tAAoC6rqFpGqv1ojInMmU69t8Cc5aHyG39EDAU3zRzeav5I05BJZd6HgLGMAxOvqJsQFLkECf58hwytDBxS6CQejaUa2QPIbNtm0uvyErl38tgnGt/BYjsGVQd3tM236PW4gLp+LRYyhUkWFOYDc13L8FQyU7OISIAlBxliVAeWL8Finx8keWSQCQn7KF9ZftS1V2Me3FoTQuStVZRURMBot9TcozjmGn+arXCbrfjkSclTmK7NBCn7ve9pwM+DRBZ6xABaZrmKBcQ42zbli9cFKMVRWFQReEoTVNW37MsQ1EURyFD9F7WU7+NqYqA0BCpbVu+8Kqq+Ht5VkPAkFG1oMZts9mwRvL6+ookSVAUBatjJBT5vo/pdMojTzr2uU3eSUlVBoW8g2a0RVFwDiAKL48OyGhARSradrvFbrdDEAQcKsRGiehRBSJuQ99dwlP+ChBZFBJPTqSqLEsEQcC0m7xIfIaDRo9Ex8uyRJqmSNOU5YA8z1GWJfMQz/MwnU5xf3+Pu7s73N3dYTKZwPM82LY9mEc+HRARGPmkYnIkgTiKIvYcAk0UpEV5kYbYSZLw4xF93zMpcxwHnudhMplgMpnAcRxmwEO66qn214CIYFBiI+2TdFAAR3khDEPUdY04jo9yDok8IuOkhOn7PsbjMR4fH+H7Pp6ennB/f4+npydMp1PMZjMGhfb58pAZAoUSpvwiLxCbsd1ux5WE8g6Vb3J3Yqeu68L3ffi+j/l8jsVigdvb26MwuVQiPRkQOinxCHERNL60bRsAMJ1Ooes69vs9DMNAkiQwDIPlRNI66O7SvGU+n2M8HuP+/h6z2Qw/f/7EYrHAzc0NC88iGEPc5ssAkcERgSHdAwCr5VmWQVGOH4Ui8IjIkUfNZjOMRiPMZjP4vs9PDj08PGA8HnPeoFmMqKxdcgyhfNADDH451NOQuEP6Z9M0XCnSNOVHrUg9pypDi6PRpLilZ0rkofZQiT0BjMEdzgJkiLVSmaXRgchPaEu5gzQTEotN02SSRS+x2x16NvUMr7gcIPzlAFED/tn9fjQ7kZO03JO81aOcGSKDO19ktiv/LbblQ9uPjvfW9q3zXtLO8pCP7BIaxScu/vIe8uEZP/FOfpZd/4FIsisgkn0UMv97Pn+mXT1Esisgkl0BkewKiGRXQCS7AiLZvwBtCZqwAvXF1QAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "#เลข 3 โดยเฉลี่ย\n",
        "mean3 = stacked_threes.mean(0)\n",
        "show_image(mean3);"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "id": "ambkeHzzpojz",
        "outputId": "e21cad11-daa9-459a-ebd7-a5a7169567c4"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAI6klEQVR4nO1baVPiTBc9ZF/IhoOWOjUf5v//KgelNEZICCErvB+eutemjaNC8N04VanG7H1y99uOdrsdzniF8u9+gf80nAmRcCZEwpkQCWdCJGgfHP9fdkGjvp1nCZFwJkTCmRAJZ0IknAmRcCZEwpkQCWdCJJwJkfBRpHoQPlNjkc8ZjXoDxzf47HmHYhBCxMnR74/Gz0IkYDQa8d/y2Hf+ITiKEHmS2+2Wx91uxxv9TcfFY+L1BHGy9FtRFIxGo96RjsvXH4KDCBEnIk6aJt51HbbbLbquQ9d1aNuWR9pP58n3kUET1zQNqqrCMAyoqgrTNKGqKjRNg6IoexvhEGK+TIj44kQCEdA0DZqmQVmWaJoGm80GdV1jvV6jrmvkeY6qqrDZbNA0Deq6Rtu2TFSfJKmqCkVR4LouTNPExcUFXNdFFEVwHAe+78M0TViWxQSNRiOoqordbvdlUr5EiCwZ9KVJAoqiQNM0WK/XKMsSWZahKAokSYLNZoMsy1CWJRPVNA1fS6SKKgaACRmPxzBNE9PpFJ7n4devX/B9H5qmYbfbMRHb7RaKohxExpcI6VMPmgxNcLVaoSgKxHGMNE0xn8+RZRniOEae51gsFlitVlgulywpoiqJ6kQbffUgCOB5Hn7//o0wDJGmKS4vLwEAQRBA0/6ZCqnYyQnpI0ckhlSFJCHLMqRpymOe50iSBOv1GqvVCm3boq7rvfuJIHLatkVVVRiNRmiaBlEUQVEU5HkO13VZ0kjCjsWXVUYkous6NE2DqqpQliXyPEeapojjGMvlEnEcI8sy3N/fY7VaIUkS1HWNsiyZAEVR2DCS1xCfQXamrmvoug7TNFHXNS4vL2EYBvI8h23bLK3vGeeTEPI3kIskEdd1HbquwzAM+L4PVVUBgL+6fA55ETKyy+US6/UaWZZhvV7zMwDskSkSSb9Fd/1VfIqQjxgXyaAJmqYJ27aZhPF4jDAMoaoqu03HcfhcXdehqip7qiRJkGUZ7u7u8PT0xAabDCdBdrnHkPFpQshIyQQoisISsd1uYZomuq5DEARQFAV1XcO2bei6ziqg6zosy4Jt23BdF5ZlwbIslpDNZoOyLHl/nucoioLVgQik0bIsjk2+jRCRiD5CDMMAALiuC1VV0XUdTNOEoiioqgphGLKtoNjB8zy4rssTo3sSIZ7nYTabIc9zrNdrVFWFtm1hWRYcx4Ft27Btm4kjCRNV5tu8jKizAKDrOn89ALBtm41j13WoqgqapsE0Tbiuy5LhOA50XedYYrfb7UWmwD/qRt5IVVU4jgPXdTEejxEEAUuIaJiPwacJER+kKAoHQPTypNuqqrL6ULS42+1YVSzLYsmwLIuJJZUiIk3TZEKqquKYxHEcOI4Dz/M4WqUoVbQjJydEJIaCHjHxAv6RFABMhghR98muEJF0Pbnbuq6RZRkHcuRlDMPAeDyG7/sIw5CjV13X3+Qxh+JglQFeJYV0V9d1Jqxt2z2dFr0P6TtNgK6h2KYsS6RpisVigcViwUEYqVwQBIii6A0hx7rcLxMiehvRsJLEkOEkkogQ2i8TQRCDvDzP8fz8jMfHRzw/PyNNU9R1zaG77/sIggCu68K27T3SAey938mTO3oQPVj0OuRxSBrETJU2MVUX70PGl/Kh+XyOp6cnzGYzpGmKpmmgaRo8z4PneQjDkCWGpGMoHBypytICvNoSsh80igUdoL+EUBQF0jTF/f097u7uEMcx4jhG0zRQVRVhGGIymfBIrvZYFZFxVOjeV84Tv5ZMmLyfJKNtW2w2G86QHx4eMJ/PkaYpezFys7TJrnYoUg42qrItod8A9ryGDLmMQMlekiT48+cPZrMZ5vM5FosFmqZhb3J1dYXLy0tMp1OMx2OOP0TJG4KUoyVEtCVkYGkTiZOLS2RI67pm6Xh4eEAcx3h4eECe5+i6DoZhIIoihGGIi4sLNqhiZErvMgSOtiF9Ve/tdvsm/wH2SaGSY1EUeHl5wWw2w2w2w8vLC6uK53m4vb3Fz58/cX19jZubGwRBwEmhHIx9u9v9G+RIVm5NyG6RCktUR0mSBEmSYLFYoCxLqKoK27bx48cPXFxcYDKZYDKZwHEcmKb5xn70kfBtucxnH0xSIhdtKHCjuuv9/T3HHGVZQtM0RFGEIAhwfX2N29tb3NzcIIoi2LbNkXBfyn+sCg1iQ+S/33sZUWWoClYUBZbLJfI8R5Zl7GZ938d0OkUURZhMJvB9H7Zt73mX98g4BkdLSB8phPdUheqvaZri6ekJSZJwi4LynNvbW0ynU5YQMqaidPTZjW/Ldv8GedLv7SPVIemgEiGRQV5FzGajKILneRyIDW1EZQza7O7zLMCrVxGN6HK5xOPjI6vLdruF4zgIggC2bePq6gpXV1fchxErY3LkKz7/WJx8OURfy4J6Mnmec08HANdIKGchcvq8CtAfFB6LQSVEVg+5b0M9mcVigefnZywWC2w2G4xGI1aJyWSCKIrY1ZLdoJrr0KG6jJOtD/lbD2ez2aAoCpRlia7rOF/RdZ2Lz5TeExGniEr7cJL1IWJoTipSFAVWqxWyLMPLywvyPOe2AnkWUUJEdZELQKfE4CrTJx3Ua6mqipO5uq65qKxpGtsPalGIlfTvIgMYgJC+tSLUZyXpoJ7ver1mQ0pVNbHOats2wjCE7/uwLOvDmOMUGNyG9NkOMqo00nIHsTBMTScav8uIyhhsSdV7RpQWxtBGrQbTNN8siKEikOhqRYP6X6EyIvoW1IgrgyiUp8YUqYSqqiwdhmFwi2KIPstXMWhy99451AR3XZcnKa8CIBsitiffa1mI49A4SRwC7Lc7adLU6gReWw+apnGb0zAM3khy/lbzOAUpow++8KdWnvTZEHHJFdkSMqxt2+6pEEkRkUNumEj5qDJ2IDG9Fw1eMaNqmdinESdMC+xEQoDXxXWiIZXvcYp0/808hpAQPrknYhX391XPel+qJ4EbqiImPqZ355CE8EXvlADeO953ft/EB5aKgwj5v8P530MknAmRcCZEwpkQCWdCJJwJkfAv6ObhbeIGuNEAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "#เลข 7 โดยเฉลี่ย\n",
        "mean7 = stacked_sevens.mean(0)\n",
        "show_image(mean7);"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "id": "aKESuqYHpojz",
        "outputId": "2ef7a17e-7e6e-4844-b49a-c094483c5679"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 72x72 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAEQAAABECAYAAAA4E5OyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAJUElEQVR4nO2by08bVxuHn/HM2OP7BbsmYJNQKCQUCGkuyratmi7aRbfZt1L/ga66Sfbtv9FVF11E6aKLKG2RqBIlaUpSih0IoSF2YkrAl7l5xl1830w/GxJKGBP0yY+EkOacmfPOb95zed9zLLRaLXr8g+91G3DY6AnSQU+QDnqCdNATpANpl/L/5ylI2Oliz0M66AnSQU+QDnqCdNATpIOeIB3sNu16hm3bWJaFaZo0Gg0Mw0BV1RfW9/l8+Hw+UqkUiqIgiiI+X/e/34EJYhgG1WqVxcVFrl69ysLCAtevX39h/VgsRiwW48svv+Tdd98lGo0SCAS6bmfXBFFVlSdPniAIAqIoUq/XefbsGXfu3KFQKLC4uMj6+vpL769Wq9y6dYu+vj6GhoZIJBJdF0bYJR/yyivV3377jc8//xxJkkin0zx//pwnT57w/Plz1tfXsSwLy7JebJggIAgCwWCQQCDAxx9/zNtvv83FixfJ5/OvalZbEztd9NxDTNNEVVXK5TKlUgmAjY0N6vU6m5ubaJqGaZr/sUjY0SaXVquFqqrous6jR48IBAJsbm6SzWaRJKkrY4rngjQaDW7evMns7Cy1Wo1Go4GqqjieuNcMXavVwrIs7ty5w9LSEh9++CHZbJZYLNaVruOZIM7s8fjxY2ZnZykUCqiqSrPZxLZtt14qlSKfz5PJZHjjjTe2PadYLLKyssLW1lbbLNRsNjEMA13XMU1zz8L+WzwTRFVVbty4wezsLF999RWapu04RkxPT/PJJ59w9uxZJicnt5VfuXKFb7/9llu3brGysuJet23b7Y6GYRx+QVqtFoZhYJomhmFsE0OSJBRFIZfLMTk5STabbXN5VVWp1WqUy2XW1tbQNK3tflmWURSFvr4+otEooih6ZXq7nV4+zBFkp68XCoXIZrNMTExw+vRpAoEAfr/fLS+VSszPz3P79m3u3r2LYRht9weDQZLJJLlcjlQqteuA/Kp4JojP5yORSBCJRJAkyZ0yE4kEw8PDDA4OMjo6yvnz5/H7/QiCgGmaaJpGtVplfn6e69evs7y8jGma7rgjiiKSJHHy5EnGx8dJJpNdEwM8FEQURdLpNOl0GkmSCAaDZLNZTp06xYULFzhx4gQTExPIskwgEHAHyKWlJX755ReuXbvGlStX0HWdZrPpPtfv96MoCu+99x4XLlzYcSD2Es8EkSSJTCbD6dOn+fTTTxFFkUQiwdDQEDMzM6RSKWRZdvu+rutuN5mbm6NQKKDruusZ8XiccDjM5OQkg4ODnDlzhv7+fmRZ9srkHfF8pWoYBrVaDZ/PhyiKrkd0uvnq6io//fQT33//Pd988822cWd8fJzR0VE++ugjzp07x8jICPF4fK/mvIyDWamKooiiKAiC4IqyU5+vVqvcu3ePUqnUJkYmkyGdTvPBBx/wzjvvMD09TX9//4EEdtAlQUKh0K71KpUKc3NzLC0ttV0fHh7m5MmTXLx4kZmZGSRJ6toUuxOvNUHUarW2dRVd1914Z6fybnPoMma6rlOv19F1HcuyDlwQ8dKlSy8rf2nhflBVlXq9jm3bPHr0yPWGVqtFo9EAoFarYVkWzWbT/e+MSx5weaeLr00QQRBIJpM8ffqU+fl5bNum2WyiqiqVSoWVlRV+/fVXNzgUBIFWq0UwGPRq6j18goTDYSKRCOFwGFEUKZfLbeOGruuoqsrjx49ZXl5maWmJ/v5+r3KsOwpyYDnVToLBIMFgkJmZGaLRKLZtc/fuXQAsy6JWq7nBHkAymSQejzM0NEQkEkGWZSTJe/Nfm4c4+Hw+IpEImUyGY8eOceTIESKRCK1Wa9tMY5ompmny4MEDMpkMsiy3rX73yOHyEAfHUxRF4ejRo8zNzREOhwHcQdeyLDRNQ9M0vvvuOxRFYWBgAEmSCIVCbVHzfnntHuIgCAKKopDJZDhx4gTHjx9ncnKSTCaDoihomuZm0GzbJhaLUalUiEQi2LaNoih77UKH00Mc/H4/fr+fcDhMLpfj6NGjnDp1ilgsht/v56+//mJ9fR3btmm1Wty4cYM///yTaDTK1NQUyWQSRVH2bUfXtiH2i5NsevbsGZVKhZ9//plCocAPP/xAsVgkFosRDAaZmpoin89z6dKlvW5PHExw5xWOxwQCAY4cOYKiKAwNDbGwsECxWGRra4utrS2q1SqpVIovvvjCk3YPrSAOTrSczWaRZZlMJtPV9g69IM5WaDgcRhCEfxVJ74dDL4iz5bm2tsbq6qq7G9gtPBWkc4B+1WTw/+7ymaZJs9mkXC5z//59NjY2trXh5ZamZ4Lous7Tp09pNpvouk40GiWdTrsGO67vBHGw87Zms9nENE1WV1dZXV1lcXGRUqlEsVjk4cOHLC8vu3VFUeTMmTOMjIy4i7n94okgzpcslUrutkI2myUSibhBmLO8dl74RckfVVVRVZU//viDmzdv8vvvv7OwsEC5XGZra8ut5/P5kGWZfD7PyMiIZynGfQtiGAabm5sUCgW+/vprd082FArR19dHOBwmnU6TzWY5fvw4Dx484N69e+5SvBNHsFKpRKVSoVqt0mg02uoKgsDZs2d58803+eyzz3jrrbdIJBL7fRXAA0Fs26Zer7O2tsaPP/5IrVZD13W3PB6Pk8vlGB8fZ2Njg8XFRa5du0aj0aBer++pLafbybLMsWPHGB0dZWRkhP7+/v2+xj9t7Helats2hmHw8OFDLl++TLFY5Pbt2207b4qiEAqFiEaj1Go1NjY2sCyr7VTANsP+e2DG+QuFQgQCAd5//32Gh4c5f/48uVyOsbGxVx0/urNS9fl8KIpCLBYjn89jmiYLCwvu0tuyLOr1unuk6qUWCgKSJCFJEn6/3824OymCcDjM2NgY09PTbjfxOifiWSxTr9e5f/8+a2trzM3Nsby8zNWrVzFNs60LvYhIJEI0GuXcuXOMjY0xMTHB8PAw8XicUCjkzlTOGTO/34/P59vPSaLuxjKSJDEwMEAwGETTNCRJYnBwEFVV0TRt1+x5KpUilUoxOjrKzMwMU1NTDAwMEA6HD2yTCjz0EGd94RxsMU2TarXqlu2GMzUrikIgEHBThM551S6wo4cc2vD/AOj9Xubf0BOkg54gHfQE6aAnSAc9QTrYbWHWveN+h5Seh3TQE6SDniAd9ATpoCdIBz1BOvgb74E4FYpj4GsAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "#เลข 3 อันที่ 125\n",
        "a_3 = stacked_threes[125]\n",
        "show_image(a_3);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k4Rm7EZjU-Gb"
      },
      "source": [
        "เราจะเห็นได้ว่าหากเราคำนวน mean absolute error และ mean squared error ของแต่ละ pixel ระหว่าง \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\" และ \"เลข 7 โดยเฉลี่ย\" เราจะเห็นกว่าค่าของ \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\" มีค่าน้อยกว่า และในกรณีนี้ระบบ (ที่ไม่ใช่ ML) ของเราจะทำนายถูกว่า \"เลข 3 อันที่ 125\" คือเลข 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JbfdH2zWpoj0",
        "outputId": "a2d7d370-8bae-45ab-fa02-e10e6ad40d2d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.1259), tensor(0.2290))"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "#ความห่างระหว่า \"เลข 3 อันที่ 125\" กับ \"เลข 3 โดยเฉลี่ย\"\n",
        "dist_3_abs = (a_3 - mean3).abs().mean()\n",
        "dist_3_sqr = ((a_3 - mean3)**2).mean().sqrt()\n",
        "dist_3_abs,dist_3_sqr"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s5xmB8f1poj0",
        "outputId": "fd56a041-cae9-4195-be48-44491f38be2e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.1836), tensor(0.3390))"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "#ความห่างระหว่า \"เลข 3 อันที่ 125\" กับ \"เลข 7 โดยเฉลี่ย\"\n",
        "dist_7_abs = (a_3 - mean7).abs().mean()\n",
        "dist_7_sqr = ((a_3 - mean7)**2).mean().sqrt()\n",
        "dist_7_abs,dist_7_sqr"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lDlxHuO8Vazc",
        "outputId": "11eda802-037f-4abb-8308-f5045f90d468"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(0.1836), tensor(0.3390))"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "#ใช้ function ของ pytorch คิดก็ได้\n",
        "F.l1_loss(a_3.float(), mean7), F.mse_loss(a_3, mean7).sqrt()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Lx6GAdSVgD2"
      },
      "source": [
        "เรื่องน่าคิดถึงความแตกต่างระหว่าง mean squared error และ mean absolute error"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IcAtO7IwDzUp",
        "outputId": "a4f00685-f56e-4a9b-ca7b-051ba45ff758"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1.1666666269302368, 1.1902379989624023)"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "a = torch.tensor([1, 2, 3]).float()\n",
        "b = torch.tensor([2.,3.,4.5])\n",
        "c = torch.tensor([2.,3.,40.])\n",
        "\n",
        "#mse และ mae ไม่ต่างกันเท่าไหร่สำหรับ a และ b\n",
        "(a-b).abs().mean().item(), \\\n",
        "((a-b)**2).mean().sqrt().item(),"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y5zmXGk2poj0",
        "outputId": "5457536b-f88b-4077-f57f-8b3aa5021cdf"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(13.0, 21.3775577545166)"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "#mse และ mae ต่างกันเกือบเท่าตัวสำหรับ a และ c\n",
        "(a-c).abs().mean().item(),\\\n",
        "((a-c)**2).mean().sqrt().item()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ErTcHuGkpoj4"
      },
      "source": [
        "## Stochastic Gradient Descent (SGD)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XTAhc8bJWi6y"
      },
      "source": [
        "หากยังจำได้จากบทที่ 1 เราเรียนรู้ว่าเราจะคำนวณ `Gradients` จาก `Loss` แล้วให้ Optimizer ทำหน้าที่ update `Weights` ในบทเรียนนี้เราจะมาเรียนรู้ขั้นตอนเหล่านี้กัน"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V9fzeVckWhP9"
      },
      "source": [
        "<img src=https://github.com/ai-builders/curriculum/raw/main/images/training_loop50.png width=\"700px\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OISepsl-poj5"
      },
      "source": [
        "### คำนวณ Gradients เพื่อทำ Backpropagation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_9WrHtiu0ScN"
      },
      "source": [
        "หากคุณยังไม่เคยเรียนเกี่ยวกับ [partial derivative](https://en.wikipedia.org/wiki/Partial_derivative) และ [chain rule](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/a/chain-rule-review) ในชั้นเรียนมัธยมปลาย คุณอาจจะไม่จำเป็นต้องเข้าใจเนื้อหาส่วนนี้ทั้งหมดก็ได้ ใจความสำคัญคือเราสามารถปรับแต่ง `Weights` ได้ด้วย `Gradients` ที่ถูกคำนวณมาจาก `Loss` เพื่อให้ได้ `Loss` ที่น้อยที่สุดเท่าที่จะทำได้ใน iteration ต่อๆไป"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tP7xMjbi6er4"
      },
      "source": [
        "ตัวอย่างการทำ Backpropagation อย่างง่ายจาก [cs231n](https://cs231n.github.io/optimization-2/#backprop)\n",
        "\n",
        "Independent variables:\n",
        "(เราอาจจะคิดว่า `x, y, z` เป็น `Inputs` หรือ `Weights` สำหรับโมเดลก็ได้)\n",
        "\\begin{align}\n",
        "x & = -2 \\\\\n",
        "y & = 5 \\\\\n",
        "z & = -4 \\\\\n",
        "\\end{align}\n",
        "\n",
        "Dependent variables: \n",
        "(`q` และ `f` คือฟังชั่นอะไรบางอย่าง เช่น `Loss Function` ของโมเดลก็ได้)\n",
        "\n",
        "ถ้าแทนค่า `x, y, z` เข้าไปในฟังชั่น `q, f` จะได้\n",
        "\n",
        "\\begin{align}\n",
        "q & = x+y = -2+5 = 3\\\\\n",
        "f & = q*z = 3*-4 = -12\n",
        "\\end{align}\n",
        "\n",
        "การทำแบบนี้เปรียบเสมือนการเปลี่ยน `Inputs` เป็น `Predictions` ด้วย `Weights` เรียกว่า `Forward Pass`\n",
        "\n",
        "หลังจากนั้น เราสามารถคำนวณหา `Gradients` ซึ่งโดยทั่วไปแล้วหมายถึงค่าอัตราการเปลี่ยนแปลงของฟังชั่นท้ายสุด (ในที่นี้คือ `f`) เทียบกับตัวแปรแรกสุด (ในที่นี้คือ `x, y, z`) ได้แก่ $\\frac{df}{dx}$, $\\frac{df}{dy}$, $\\frac{df}{dz}$ เราสามารถหาสิ่งนี้ด้วยการหา [partial derivative](https://en.wikipedia.org/wiki/Partial_derivative) และ [chain rule](https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/a/chain-rule-review) เรียกว่า `Backward Pass`\n",
        "\n",
        "\\begin{align}\n",
        "\\frac{df}{dq} & = z = -4\\\\\n",
        "\\frac{dq}{dx} & = 1\\\\\n",
        "\\frac{df}{dx} & = \\frac{df}{dq} * \\frac{dq}{dx}\\\\\n",
        "& = -4*1\\\\\n",
        "& = -4\\\\\n",
        "\\end{align}\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VXdDIlJl6jYC",
        "outputId": "9770d555-8e05-4b73-bf31-48d1d6b782c7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(-4.0, -4.0)"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "# set some inputs\n",
        "x = -2; y = 5; z = -4\n",
        "\n",
        "# perform the forward pass\n",
        "q = x + y # q becomes 3\n",
        "f = q * z # f becomes -12\n",
        "\n",
        "# perform the backward pass (backpropagation) in reverse order:\n",
        "# first backprop through f = q * z\n",
        "dfdz = q # df/dz = q, so gradient on z becomes 3\n",
        "dfdq = z # df/dq = z, so gradient on q becomes -4\n",
        "# now backprop through q = x + y\n",
        "dfdx = 1.0 * dfdq # dq/dx = 1. And the multiplication here is the chain rule!\n",
        "dfdy = 1.0 * dfdq # dq/dy = 1\n",
        "\n",
        "dfdx, dfdy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kt9JIcVZ3CuU"
      },
      "source": [
        "Pytorch สามารถทำ `Backward Pass` ให้เราโดยอัตโนมัติด้วยฟังชั่น Autograd โดยที่เราไม่ต้องคิด partial derivative เอง ผลข้างเคียงอีกอย่างคือเราสามารถใช้ Pytorch ช่วยทำการบ้านวิชาแคลคูลัสเวลาเราหา derivative ที่ยากเกินไปไม่ออกได้อีกด้วย"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SWg90ZHApoj6",
        "outputId": "d7fbd5b9-1170-424e-e049-26bb3ca5a1a1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 3.,  4., 10.], requires_grad=True)"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "xt = torch.tensor([3.,4.,10.]).requires_grad_()\n",
        "xt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WpM1Q5nApoj6",
        "outputId": "826205b5-d40f-4b04-986d-7c0beb6a0265"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(125., grad_fn=<SumBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "#ฟังชั่น f(x) = (x1^2 + x2^2 +...+xn^2)\n",
        "def f(x): return (x**2).sum()\n",
        "\n",
        "yt = f(xt) #ใส่ 3, 4, 10 เข้าไปได้ 3^2+4^2+10^2 = 125\n",
        "yt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "POJB2ewmpoj6",
        "outputId": "83cd31b2-2071-4a9e-af97-f3b301b2c807"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 6.,  8., 20.])"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "#ถ้าคิดด้วยมือ df(x)/dx = 2x \n",
        "#สำหรับ x1=3, x2=4, x3=10 ก็จะเป็น \n",
        "#df(x1)/dx1 = 6, df(x2)/dx2 = 8, df(x3)/dx3 = 20\n",
        "\n",
        "#ใช้ autograd หา df(x1)/dx1, df(x2)/dx2, df(x3)/dx3\n",
        "yt.backward()\n",
        "xt.grad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mHHqwvz2poj_"
      },
      "source": [
        "## สร้าง Loss Function สำหรับจำแนกรูปเลข 3 และเลข 7"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U0EQ6WD4zEf0"
      },
      "source": [
        "#### สร้าง X และ y"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t2aoyhac4ZrI"
      },
      "source": [
        "จัดการ `Inputs` คือรูปตัวเลข 28x28 pixels และ `Labels` คือ `1 ถ้าเป็นเลข 3` และ `0 ถ้าเป็นเลข 7`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "-4LTBmPcpoj_"
      },
      "outputs": [],
      "source": [
        "train_x = torch.cat([stacked_threes, stacked_sevens]).view(-1, 28*28)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q5bU0JpipokA",
        "outputId": "798e931e-2e7b-41bf-e94c-231da4a25c83"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([12396, 784]), torch.Size([12396, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "train_y = tensor([1]*len(threes) + [0]*len(sevens)).unsqueeze(1)\n",
        "train_x.shape,train_y.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rkavLIe1IcmD",
        "outputId": "c7cabbb3-e47f-4403-b5d7-b09f38036698"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(1, 4), (2, 5), (3, 6)]"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "#เราสามารถนำ iterables สองอันมาต่อกันแบบนี้ได้ด้วย zip\n",
        "a = [1,2,3]\n",
        "b = [4,5,6]\n",
        "list(zip(a,b))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "08BIemoApokA",
        "outputId": "a209e1e9-e5ec-4c78-c2ed-3e9842c8974c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([784]), torch.Size([1]))"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "dset = list(zip(train_x,train_y))\n",
        "example = dset[0]\n",
        "\n",
        "#คู่ Inputs　และ Labels\n",
        "example[0].shape, example[1].shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "Y6ImID5ApokA"
      },
      "outputs": [],
      "source": [
        "#สร้าง validation set ในแบบเดียวกัน\n",
        "valid_3_tens = torch.stack([tensor(Image.open(o)) \n",
        "                            for o in (path/'valid'/'3').ls()])\n",
        "valid_3_tens = valid_3_tens.float()/255\n",
        "valid_7_tens = torch.stack([tensor(Image.open(o)) \n",
        "                            for o in (path/'valid'/'7').ls()])\n",
        "valid_7_tens = valid_7_tens.float()/255\n",
        "\n",
        "valid_x = torch.cat([valid_3_tens, valid_7_tens]).view(-1, 28*28)\n",
        "valid_y = tensor([1]*len(valid_3_tens) + [0]*len(valid_7_tens)).unsqueeze(1)\n",
        "valid_dset = list(zip(valid_x,valid_y))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RR1v1Z2tzHmo"
      },
      "source": [
        "#### Initiate `Weights`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7slIAUaOI0_d"
      },
      "source": [
        "สมมุติว่าเราจะใช้ architecture สุดเรียบง่าย แค่คูณค่า pixels ของรูป `Inputs` ด้วย `W` และบวกด้วย `b`\n",
        "\n",
        "$$prediction = \\Sigma(xW^T) + b$$\n",
        "\n",
        "เราสามารถเริ่มตั้ง `Weights` เป็นการ random จาก standard normal distribution"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "WQ9vpYHIpokA"
      },
      "outputs": [],
      "source": [
        "def init_params(size, std=1.0): return (torch.randn(size)*std).requires_grad_()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "itXv1TZ9pokA",
        "outputId": "cc10560d-27d6-4ff4-b29e-99fca3878da9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([784, 1])"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "#Inputs มี dimension (batch_size, 28*28)\n",
        "#เพราะงั้นถ้าเราจะคูณรายตัว (element-wise multiplication) ด้วย W^T, W ต้องมี dimension (28*28,1)\n",
        "weights = init_params((28*28,1))\n",
        "weights.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7Ef5fK0KpokA",
        "outputId": "087d1c9a-adf1-4cf2-a3ea-b24fea8066e0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([0.3472], requires_grad=True)"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ],
      "source": [
        "#bias มีแค่ (1) dimension แล้วจะถูก broadcast ไปทุก dimension ของ batch size เอง\n",
        "# (1) -> (batch_size,1)\n",
        "bias = init_params(1)\n",
        "bias"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pnMvUz7kFoIU",
        "outputId": "07d34405-6a39-4b0f-b374-56fe7f0a7272"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([784, 1]),\n",
              " torch.Size([1]),\n",
              " torch.Size([12396, 784]),\n",
              " torch.Size([784]))"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ],
      "source": [
        "weights.shape, bias.shape, train_x.shape, train_x[7].shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KXK2LAZqzLYk"
      },
      "source": [
        "#### Forward Pass"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S9md33qTEu6n",
        "outputId": "1469e6f5-bd86-45e3-ed3d-831e52ded4ab"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[-15.2588]], grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ],
      "source": [
        "#คำนวณ forward pass สำหรับตัวอย่าง 7\n",
        "(train_x[7]*weights.T).sum(1)[:,None] + bias"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "22qYPEQDEwjC",
        "outputId": "528c7509-1eea-49cf-d5b6-181fd62059ca"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ -6.2330],\n",
              "        [-10.6388],\n",
              "        [-20.8865],\n",
              "        ...,\n",
              "        [-15.9176],\n",
              "        [ -1.6866],\n",
              "        [-11.3568]], grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ],
      "source": [
        "#คำนวณ forward pass สำหรับทุกตัวอย่าง\n",
        "(train_x*weights.T).sum(1)[:,None] + bias"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I65On42BpokB",
        "outputId": "d77c1723-af03-485b-b036-c98e1b0f50af"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ -6.2330],\n",
              "        [-10.6388],\n",
              "        [-20.8865],\n",
              "        ...,\n",
              "        [-15.9176],\n",
              "        [ -1.6866],\n",
              "        [-11.3568]], grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "#เขียนเป็นฟังชั่น; @ คือ matrix multiplication ใน pytorch\n",
        "def linear1(xb): return xb@weights + bias\n",
        "preds = linear1(train_x)\n",
        "preds"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XJKeTmrPzOr3"
      },
      "source": [
        "#### Metric ใช้ Accuracy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SLnbggPopokB",
        "outputId": "478badc3-ae34-444e-c7ea-ff028b18cb23"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[False],\n",
              "        [False],\n",
              "        [False],\n",
              "        ...,\n",
              "        [ True],\n",
              "        [ True],\n",
              "        [ True]])"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ],
      "source": [
        "#ถ้าสมมุติว่า Predictions >0 ให้ทายเป็น 1 (เลข 3)\n",
        "corrects = (preds>0.0).float() == train_y\n",
        "corrects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5DVZlAVzJqbV",
        "outputId": "299bf0b0-09cf-4822-a55a-9a43537c0888"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Counter({0.0: 10788, 1.0: 1608})"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ],
      "source": [
        "from collections import Counter\n",
        "Counter((preds>0.0).float().numpy()[:,0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CiSVOFonpokB",
        "outputId": "4f584989-ecc1-47ef-a7e1-b95075af4f18"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.5379961133003235"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ],
      "source": [
        "#จะเห็นได้ว่ามันถูกเครื่องๆ (เพราะเราเดาสุ่มด้วย weights ที่สุ่มมา)\n",
        "corrects.float().mean().item()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "id": "RwkizTBqpokC"
      },
      "outputs": [],
      "source": [
        "#ถ้าเราเปลี่ยน weights เล็กน้อย\n",
        "with torch.no_grad():\n",
        "    weights *= 5.0001\n",
        "    bias+=10"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d5Kv_vFrpokC",
        "outputId": "351fac1c-2a3a-485c-f48e-af448f3141c3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.548402726650238"
            ]
          },
          "metadata": {},
          "execution_count": 47
        }
      ],
      "source": [
        "#ความแม่นยำก็เปลี่ยนนิดนึง\n",
        "preds = linear1(train_x)\n",
        "((preds>0.0).float() == train_y).float().mean().item()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PTOZYvLRy-Vz"
      },
      "source": [
        "#### Loss Function อย่างง่าย"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "id": "w7wVT8TjpokC"
      },
      "outputs": [],
      "source": [
        "trgts  = tensor([1,0,1])\n",
        "prds   = tensor([0.9, 0.4, 0.2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "id": "9HgV5HQppokC"
      },
      "outputs": [],
      "source": [
        "#loss function ที่จะเป็นจะเป็น Predictions ของ class ที่ผิด\n",
        "#เพราะงั้นยิ่งทาย class ที่ผิดแบบมั่นใจมาก Loss ก็จะยิ่งสูง\n",
        "def mnist_loss(predictions, targets):\n",
        "    return torch.where(targets==1, 1-predictions, predictions).mean()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5xGnIzeHpokC",
        "outputId": "29656655-4d49-491a-abfa-b4f3c87a63c7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([0.1000, 0.4000, 0.8000])"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ],
      "source": [
        "#if target == 1, loss is how far it is from 1\n",
        "#if target == 0, loss is how far it is from 0\n",
        "torch.where(trgts==1, 1-prds, prds)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dhQH45dYpokC",
        "outputId": "2a3c5978-ecea-493d-cb92-02b89b64c610"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(0.4333)"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ],
      "source": [
        "mnist_loss(prds,trgts)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uiUjD7aJpokD",
        "outputId": "c2fabcb4-0df8-4d71-9196-d110404ea9bb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(0.2333)"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ],
      "source": [
        "#ถ้าทายแม่นขึ้น loss ก็ลดลง\n",
        "mnist_loss(tensor([0.9, 0.4, 0.8]), trgts)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dqsrmegupokD"
      },
      "source": [
        "### Activation Function - Sigmoid"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-0DXOc8w0YeU"
      },
      "source": [
        "ปัญหาอย่างนึงของเราตอนนี้คือโมเดลเราให้ค่า `Predictions` เป็นจำนวนจริง แต่ `Loss Function` ของเราต้องการค่าระหว่าง 0 ถึง 1 เราจึงใช้ `Activation Function` ที่ชื่อ Sigmoid เพื่อเปลี่ยนให้ตัวเลขอยู่ระหว่าง 0 ถึง 1\n",
        "\n",
        "`Activation Function` นั้นมีมากมายหลายประเภท ที่นิยมใช้ เช่น [ReLU](https://en.wikipedia.org/wiki/Rectifier_(neural_networks))(x) - ฟังชั่นที่บอกว่าถ้าน้อยกว่า 0 ให้เป็น นอกนั้นเป็น x, [tanh](https://mathworld.wolfram.com/HyperbolicTangent.html)(x) - ฟังชั่นที่เปลี่ยนค่าจำนวนจริงให้อยู่ระหว่าง -1 และ 1 เป็นต้น อ่านเพิ่มเกี่ยวกับ [Activation Functions](https://en.wikipedia.org/wiki/Activation_function)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {
        "id": "sQIUTk5ZpokD"
      },
      "outputs": [],
      "source": [
        "def sigmoid(x): return 1/(1+torch.exp(-x))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "TY9tdedYpokD",
        "outputId": "f1b91b5a-9267-4d78-93e5-6fa068f979f2"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV5b328e+PAAkkzAlTBgISZJQpgnVoHfAccICqVUGrpfqKWqeqPa0ee/StrXayrVqpU6UqteDYyhGsU51RIchMBMIcpgxAQgIJSfbv/SOxb8QoAXay9t65P9fF5R4We9+Sve/rybOetZa5OyIiEv1aBR1ARETCQ4UuIhIjVOgiIjFChS4iEiNU6CIiMaJ1UG+cnJzsmZmZQb29iEhUWrRoUZG7pzT0XGCFnpmZSU5OTlBvLyISlcxs01c9pykXEZEYcchCN7MZZlZgZiu+4nkzswfNLM/MlpnZqPDHFBGRQ2nMCP1JYPzXPD8ByKr7Mw14+OhjiYjI4Tpkobv7e8Cur9lkEvC01/oY6GxmvcIVUEREGiccc+ipwJZ69/PrHhMRkWbUrDtFzWyameWYWU5hYWFzvrWISMwLR6FvBdLr3U+re+xL3P0xd8929+yUlAaXUYqIyBEKxzr0OcD1ZjYbGAuUuPv2MLyuiEhUC4WcovJKdpZUsrO0goK9tf89Y1B3jkvrHPb3O2Shm9ks4FQg2czygbuANgDu/ggwDzgLyAP2Ad8Pe0oRkQhUUVVD/u59bNm9n/zd+9m6ez/b9uxne8l+tu2pYGdpBdWhL19zIqVDfDCF7u5TDvG8A9eFLZGISASprK5hY9E+1heWsaG4nI1F5Wws2semXeXsLK38wrZt41rRs1MCvTsnMLZvV3p2SqBnpwR6dKz9071DPMlJ8bRt3TS7LwM79F9EJJIcqA6RV1DGmp17Wb1zL2t37iWvoIzNu/ZRf5Cd0iGezG7tOSUrhYyu7cno2p70ru1I69KelKR4WrWywP4fVOgi0uKUV1azanspK7aWsGJrKSu3lbCusIyqmtrmbt3K6JucyJDenZg4IpVjUhI5JiWJzOREkuIjtzYjN5mISBiEQs66wjIWbdrN4s17WJq/hzU79/571J2cFM+Q3h059djuDOrVgUG9OpLZLbHJpkWakgpdRGJKVU2I5VtLWLBhF5+sL+bTzXso2V8FQOf2bRie1pn/HNKT49I6MSy1E907JgScOHxU6CIS1dyd1Tv38sHaIj7MK2LBhl2UH6gBoF9KIhOG9mR0ny6M7tOFvsmJmAU3x93UVOgiEnXKKqt5f00h76wu5N01heworQCgX3Ii541K5Rv9khnTtyspHeIDTtq8VOgiEhV2llbw+sodvJFbwMfrijlQE6JDQmtOyUrm1AHdOTkrmd6d2wUdM1AqdBGJWDtKKpi7fDuvLt/Oos27cYe+yYlMPSmTMwZ2Z3SfLrSOi76dl01FhS4iEaVkfxWvLt/Oy0u28fGGYtxhYM8O3DxuABOG9iSrR4egI0YsFbqIBK4m5HyYV8QLi/J5beUOKqtD9E1O5KYzsjh3eG+OSUkKOmJUUKGLSGB2llbw3MItzF64ha179tOpXRsuPj6dC0alcVxap5hekdIUVOgi0qzcnYUbd/Pk/A28tnInNSHnpP7duP2sgZw5uAfxreOCjhi1VOgi0iwOVIf436XbmPHhBlZuK6VjQmuuPLkvU8Zk0Dc5Meh4MUGFLiJNqqyymtkLNvPEBxvYXlJBVvck7j1vGN8e2Zv2bVVB4aR/TRFpEiX7q3hq/kae+GADJfurOKFfV+49fxinDkjR3HgTUaGLSFiVVlTxxPsbmPHhBvZWVDNuUHeuO60/IzO6BB0t5qnQRSQs9h+o4emPNvLwu+vYs6+K/xzSgxtOz2Joaqego7UYKnQROSo1IeelT/P53etr2FFawbcGpPCj/ziWYWkq8uamQheRIzY/r4ifz80ld3spw9M788DkEYzt1y3oWC2WCl1EDlv+7n3cMzeXV1fsILVzOx6cMpJzj+ulnZ0BU6GLSKNVVtfw2Lvrmf5OHgC3njmAq77Zj4Q2OhgoEqjQRaRRFmzYxe0vLWNdYTkThvbkp+cMJrWFn6420qjQReRrlVZU8ct5ucxasIW0Lu34y/eP57RjuwcdSxqgQheRr/T26gJuf3E5BXsruPqb/bhpXJaO7oxg+smIyJfsraji7v9dxfOL8snqnsSjl53E8PTOQceSQ1Chi8gXLNy4i5ufXcK2Pfv5wanHcNO4LJ0BMUqo0EUEgKqaEA+8uZY/vZNHWpf2PH/NiYzuo8P1o4kKXUTYumc/N85azKJNu7lwdBp3TRxCUrzqIdroJybSwr25aic/emEpVdUhHpg8gkkjUoOOJEdIhS7SQlXXhLjv9TU88u46BvfqyPRLR+lCE1FOhS7SAhWXVXLj7MV8mFfMlDEZ3HXuYB3tGQNU6CItzPL8Eq6emUNR+QF+853juCg7PehIEiatGrORmY03s9VmlmdmtzXwfIaZvW1mi81smZmdFf6oInK0Xlm2jQsfnY+Z8eI1J6rMY8whR+hmFgdMB84E8oGFZjbH3VfV2+ynwHPu/rCZDQbmAZlNkFdEjkAo5Nz/1loefGst2X268Mhlo0lOig86loRZY6ZcxgB57r4ewMxmA5OA+oXuQMe6252AbeEMKSJHrqKqhlufX8rcZdu5cHQavzhvqA4UilGNKfRUYEu9+/nA2IO2+b/A62Z2A5AIjAtLOhE5KrvKDzDt6RxyNu3mtgkDufqb/XTO8hjWqDn0RpgCPOnuacBZwEwz+9Jrm9k0M8sxs5zCwsIwvbWINGRTcTkXPDyfZVtLmH7JKK751jEq8xjXmELfCtTfc5JW91h9VwLPAbj7R0ACkHzwC7n7Y+6e7e7ZKSkpR5ZYRA5pxdYSLnh4Pnv2HWDWVWM5+7heQUeSZtCYQl8IZJlZXzNrC0wG5hy0zWbgDAAzG0RtoWsILhKA+XlFTH7sY+Jbx/HCtScyuk/XoCNJMzlkobt7NXA98BqQS+1qlpVmdreZTazb7FbgKjNbCswCprq7N1VoEWnYP1dsZ+pfFtK7cwIvXnsix6QkBR1JmlGjDixy93nULkWs/9id9W6vAk4KbzQRORwvLsrnv15Yyoj0zsyYejyd27cNOpI0Mx0pKhIDZn68if/5xwpO6t+Nxy/P1lWFWij91EWi3OPvreeeebmcMbA70y8dpXOytGAqdJEo9si76/jVq59x9rBe3D95BG3iwrUSWaKRCl0kSv3pnTx+88/VnDu8N3+4aDitVeYtnj4BIlHo8zKfqDKXevQpEIkyf35/Pb/552omjejN71XmUo8+CSJRZOZHG/nF3FzOGtaT312oMpcv0qdBJEo8l7OF/3l5JeMGdef+i0eqzOVL9IkQiQLzlm/ntheXcUpWMg9dMoq2rfXVlS/Tp0Ikwr2/tpCbZi9mVEYXHr1stNaZy1dSoYtEsEWbdjPt6UX0796BJ6YeryNA5Wup0EUi1Nqde7niyYX06BjP01eMoVO7NkFHkginQheJQNtL9vO9GQto27oVM68cS0oHXf9TDk2FLhJhSvZVMXXGQkorqnny+8eT3rV90JEkSqjQRSJIRVUNV83MYUNROY9dPpohvTsFHUmiiPawiESIUMj50fNLWbBhF3+cMpITj/nSVRxFvpZG6CIR4jevreaVZdu5bcJAzh3eO+g4EoVU6CIR4JlPNvHIu+u4dGwGV3+zX9BxJEqp0EUC9u6aQu58eSWnD+zOzyYOwcyCjiRRSoUuEqC1O/dy/TOfMqBHBx6covOzyNHRp0ckIMVllVzx1EIS2sbxxPeySYrXGgU5Oip0kQBUVtdw9cxFFJRW8vjl2fTu3C7oSBIDNCQQaWbuzh1/X0HOpt08dMlIRqR3DjqSxAiN0EWa2RMfbOCFRfncdEYW5xyn5YkSPip0kWb0zuoC7p2Xy4ShPbnpjKyg40iMUaGLNJP1hWXcMGsxx/bsyO8uGk6rVlqeKOGlQhdpBnsrqrjq6RzaxLXi8ctH67zm0iT0qRJpYqGQc/OzS9lYvI+/XjmWtC46e6I0DY3QRZrYg/9ay5u5O/np2YP4xjHdgo4jMUyFLtKE3li1k/vfXMsFo9KYemJm0HEkxqnQRZrI+sIybnl2CcNSO3HPeUN1jhZpcip0kSZQXlnNNX9dROs44+HvjiKhTVzQkaQFaFShm9l4M1ttZnlmdttXbHORma0ys5Vm9rfwxhSJHu7OT15cRl5BGQ9OGamdoNJsDrnKxczigOnAmUA+sNDM5rj7qnrbZAG3Aye5+24z695UgUUi3YwPN/LKsu38ePyxnJKVEnQcaUEaM0IfA+S5+3p3PwDMBiYdtM1VwHR33w3g7gXhjSkSHRZu3MUv5+XyH4N7cO23jgk6jrQwjSn0VGBLvfv5dY/VNwAYYGYfmtnHZjY+XAFFokXh3kque+ZT0rq0476LhmsnqDS7cB1Y1BrIAk4F0oD3zGyYu++pv5GZTQOmAWRkZITprUWCV10T4oZZn1JaUcVTV4yhY0KboCNJC9SYEfpWIL3e/bS6x+rLB+a4e5W7bwDWUFvwX+Duj7l7trtnp6RoblFix32vr+Hj9bu459vDGNSrY9BxpIVqTKEvBLLMrK+ZtQUmA3MO2uYf1I7OMbNkaqdg1ocxp0jEeit3J4+8u44pY9K5YHRa0HGkBTtkobt7NXA98BqQCzzn7ivN7G4zm1i32WtAsZmtAt4G/svdi5sqtEik2LJrH7c8t5QhvTty17lDgo4jLVyj5tDdfR4w76DH7qx324Fb6v6ItAiV1TVc97dPCbnzp0t18JAET2dbFDlC987NZVl+CY98dzR9uiUGHUdEh/6LHIm5y7bz1EebuPLkvowf2jPoOCKACl3ksG0sKucnLy5jRHpnfjJ+YNBxRP5NhS5yGCqqaufN41oZD10ykrat9RWSyKE5dJHDcM/cXFZuK+XPl2frpFsScTS8EGmkV5ZtY+bHm7jqlL6MG9wj6DgiX6JCF2mEjUXl3PbickZmdObHmjeXCKVCFzmEyuoarp9VO2/+xykjaROnr41EJs2hixzCvXNzWbG1lMc1by4RTkMNka/xzxX/f735mZo3lwinQhf5Clt27eO/XljG8LROWm8uUUGFLtKAA9Uhrp+1GICHLhml9eYSFTSHLtKA3772GUu37OHhS0eR3lXz5hIdNOwQOchbuTt5/P0NXHZCHyYM6xV0HJFGU6GL1LO9ZD+3Pr+Uwb06csfZg4KOI3JYVOgidaprQtw4azFV1SEeumSkzm8uUUdz6CJ17n9zLQs37uaBySPol5IUdByRw6YRugjw/tpCpr+Tx0XZaUwakRp0HJEjokKXFq+gtIKbn11C/5QkfjZxaNBxRI6YplykRasJOT98dgllldX87aoTaNdW8+YSvVTo0qI99K885q8r5jffOY4BPToEHUfkqGjKRVqsj9YV88Bba/j2iN5cODot6DgiR02FLi1S4d5Kbpy9mMxuifzivGGYWdCRRI6aplykxQmFnFueW0Lp/iqevmIMSfH6Gkhs0AhdWpw/vZPH+2uLuOvcIQzq1THoOCJho0KXFuXj9cX8/o01nDu8N1PGpAcdRySsVOjSYhSVVXLjrMX06ZbIvecN1by5xBwVurQINSHn5meXULK/iumXjKJDQpugI4mEnfYGSYvwp7dr581/ef4wBvfWvLnEJo3QJebNX1fEH96sXW8++XjNm0vsUqFLTCsoreDGWUvol5LEPVpvLjFOUy4Ss6praq8LWl5ZzayrxpKo9eYS4xo1Qjez8Wa22szyzOy2r9nuAjNzM8sOX0SRI/O7N9awYMMu7j1/KFk6T4u0AIcsdDOLA6YDE4DBwBQzG9zAdh2Am4BPwh1S5HC9uWonD7+zjiljMjhvpM7TIi1DY0boY4A8d1/v7geA2cCkBrb7OfBroCKM+UQO2+bifdz83BKGpnbkrnO/NPYQiVmNKfRUYEu9+/l1j/2bmY0C0t19bhiziRy2iqoarvnrIlqZ8fClo3VdUGlRjnovkZm1An4PTG3EttOAaQAZGRlH+9YiX+Du/M8/VrBqeyl/mXo86V3bBx1JpFk1ZoS+Fai/eDet7rHPdQCGAu+Y2UbgBGBOQztG3f0xd8929+yUlJQjTy3SgNkLt/D8onxuOL0/pw3sHnQckWbXmEJfCGSZWV8zawtMBuZ8/qS7l7h7srtnunsm8DEw0d1zmiSxSAOWbNnDXS+v5JsDUvjhuAFBxxEJxCEL3d2rgeuB14Bc4Dl3X2lmd5vZxKYOKHIoRWWVXPvXRXTvGM8DF48grpUOHpKWqVFz6O4+D5h30GN3fsW2px59LJHGqa4JccPfFrOr/AAvXnsiXRLbBh1JJDA6dE6i2q//+RkfrS/mvguHMzS1U9BxRAKlc7lI1PrH4q08/v4Gpp6YyXd0kWcRFbpEpxVbS/jJi8sY27crd5w9KOg4IhFBhS5Rp7iskqtnLqJbYlumXzqKNnH6GIuA5tAlyhyoDnHtXz+lqKySF645keSk+KAjiUQMFbpEDXfnrjkrWLBxFw9MHsGwNO0EFalPv6tK1Hhq/kZmLdjCdacdw6QRqYf+CyItjApdosIHa4v4+dxcxg3qwa1nHht0HJGIpEKXiJdXUMa1zyyif0oS908eQSsdCSrSIBW6RLRd5Qe48qmFxLduxRNTs0nSZeREvpK+HRKxKqtruGbmIraXVDB72gmkddHpcEW+jkboEpHcndtfXM6Cjbu478LhjMroEnQkkYinQpeI9Ic31/LS4q3ceuYAJg7vHXQckaigQpeI81zOFh58ay0XZ6dz/en9g44jEjVU6BJRPlhbxH+/tJxTspL5xXlDMdOKFpHGUqFLxFixtYSrZ+bQv3sSf9I5WkQOm74xEhE2FZcz9S8L6Ny+LU9dMYYOCW2CjiQSdbRsUQJXVFbJ92YsoDrkzL5iDD06JgQdSSQqaYQugdpbUcX3/7KQHaUVPPG94+nfPSnoSCJRS4UugamoquH/PJVD7vZSHr50NKP7aK25yNHQlIsEoqomxHXPfMqCjbu4/+IRnDawe9CRRKKeRujS7GpCzo+eX8pbnxVw96ShOhWuSJio0KVZhULO7S8t4+Ul2/jx+GO57IQ+QUcSiRkqdGk2tVccWslzOfnceEYWPzhVR4GKhJMKXZqFu3PP3FxmfryJq7/Zj5vHZQUdSSTmqNClyX1e5n/+YANTT8zktgkDdUi/SBPQKhdpUu7Oz1/JZcaHtWV+17mDVeYiTUSFLk3G3fnZ/67iyfkb+f5Jmdx5jspcpCmp0KVJ1IScO/6+nNkLt3DlyX356dmDVOYiTUyFLmFXVRPiR88v5eUl27jh9P7ccuYAlblIM1ChS1hVVNVww6zFvLFqJz8ZP5BrTz0m6EgiLYYKXcKmZH8VVz2dw8KNu7h70hAu/0Zm0JFEWpRGLVs0s/FmttrM8szstgaev8XMVpnZMjN7y8x0+F8LU1BawcWPfsTizbt5YPJIlblIAA5Z6GYWB0wHJgCDgSlmNvigzRYD2e5+HPAC8JtwB5XIlVdQxgWPzGfzrn3MmHq8LuosEpDGjNDHAHnuvt7dDwCzgUn1N3D3t919X93dj4G08MaUSPXJ+mIueHg++w/UMOuqEzglKyXoSCItVmMKPRXYUu9+ft1jX+VK4NWjCSXR4eUlW7nsiQV0S2rL339wEsPTOwcdSaRFC+tOUTP7LpANfOsrnp8GTAPIyMgI51tLMwqFnPvfXMOD/8pjbN+uPHrZaDq3bxt0LJEWrzGFvhVIr3c/re6xLzCzccAdwLfcvbKhF3L3x4DHALKzs/2w00rg9h2o5pZnl/LPlTu4cHQavzhvKPGt44KOJSI0rtAXAllm1pfaIp8MXFJ/AzMbCTwKjHf3grCnlIiwZdc+rp65iM92lPLTswdx5cl9dcCQSAQ5ZKG7e7WZXQ+8BsQBM9x9pZndDeS4+xzgt0AS8HzdF3yzu09swtzSzN5bU8iNsxdTE3KemHo8px2rS8aJRJpGzaG7+zxg3kGP3Vnv9rgw55IIEQo5D7+7jvteX82xPTrwyHdHk5mcGHQsEWmAjhSVr1RcVsmtzy/lndWFTBrRm1+eP4z2bfWREYlU+nZKgxZs2MUNsz5l974qfj5pCN89oY/my0UinApdvqC6JsRDb+fx4Ftr6dMtkRlTj2dI705BxxKRRlChy79tLCrnh88uYcmWPZw3MpW7Jw2hQ0KboGOJSCOp0AV3528LNnPP3FxatzL+OGUk5+p8LCJRR4XewuXv3sdtLy7ng7wiTurfjd9+Zzi9O7cLOpaIHAEVegtVE3Ke+WQTv371MwDuOW8ol4zJ0I5PkSimQm+BcreXcvtLy1myZQ8n90/ml+cPI71r+6BjichRUqG3IGWV1Tz41lpmfLCBju3a8IeLh/PtEakalYvECBV6C+DuvLxkG/fOy6VgbyUXZ6dz24SBdEnUGRJFYokKPcYt2rSbe+au4tPNexie1onHLs9mhM5bLhKTVOgxanPxPn792mfMXbadlA7x/PqCYVw4Op1WrTS9IhKrVOgxZmdpBX/811pmL9hCm7hW3HRGFtO+2Y/EeP2oRWKdvuUxomBvBY+/t56nP9pETciZPCadG07PokfHhKCjiUgzUaFHuR0lFTz63jr+9slmqmpCfHtEKj8cN4CMblqGKNLSqNCj1Nqde3nsvfX8Y8lWQg7nj0zlB6f1p6/OVS7SYqnQo4i782FeMX/5cANvfVZAQptWTBmTwVWn9NOBQSKiQo8GZZXV/H3xVp6ev5G1BWV0S2zLTWdkcfk3+tAtKT7oeCISIVToEcrdWbmtlGc+2cycJVspP1DDkN4due/C4ZxzXC8S2sQFHVFEIowKPcIUlVXyj8VbeWFRPp/t2EtCm1acc1xvLhmbwcj0zjpMX0S+kgo9ApRVVvPGqh28vGQb768toibkDE/rxN2ThjBpRCqd2ukiEyJyaCr0gOytqOJfnxXw6vIdvLOmgIqqEKmd23HVKf04f1QqA3p0CDqiiEQZFXoz2lFSwVuf7eTNVTv5cF0xB6pDdO8Qz0XZ6Uwc3ptRGV10aL6IHDEVehOqqgnx6abdvLOmkHdXF7JqeykAGV3bc9kJfZgwtKdKXETCRoUeRqGQk7ujlI/WFTN/XTGfrC+m/EANrVsZo/t04cfjj+XMQT3o3z1JOzdFJOxU6EehoqqGFVtLyNm0m5yNu1iwYRelFdUA9EtO5PxRaZzUvxsn9k+mY4J2bIpI01KhN1Io5GwsLmdZfglLtuxhyZY9rNpWyoGaEAB9kxM5a1gvxvbryti+3XShZRFpdir0BlRU1bB2Zxm520tZtb2UVdtKWbmthPIDNQC0axPHsLROfP+kTEb36cKoPl1I1hGbIhKwFl3o5ZXVrC8sJ69wL+sKylmzcy9rC8rYVFxOyGu3ad82joE9O/Cd0WkMSe3EsNROZHVPonVcq2DDi4gcJKYL3d3Zva+K/N372FS8j8279rG5eB8bisvZWFROwd7Kf28b18ro0609A3t24NzhvRnYswODenWkT9f2WoUiIlEhagvd3dlbWU1BaQU7SirZUVrBjpL9bN1TwfaS/Wzbs5/83fvZVzdN8rnkpHgyu7XnWwNSyExO5JiURPp3TyKjayJtW2vULSLRK+oK/dmFm5n+9joK9lZQURX60vPdEtvSu3M7MrslcnL/FNK6tCO1SzsyurYno2t7XYpNRGJWo9rNzMYDDwBxwJ/d/VcHPR8PPA2MBoqBi919Y3ij1uqWGM/w9M706BBP947xdO+QQM9OCfTqlECPjgk6C6GItFiHLHQziwOmA2cC+cBCM5vj7qvqbXYlsNvd+5vZZODXwMVNEXjc4B6MG9yjKV5aRCSqNWbSeAyQ5+7r3f0AMBuYdNA2k4Cn6m6/AJxhOhRSRKRZNabQU4Et9e7n1z3W4DbuXg2UAN3CEVBERBqnWZd1mNk0M8sxs5zCwsLmfGsRkZjXmELfCqTXu59W91iD25hZa6ATtTtHv8DdH3P3bHfPTklJObLEIiLSoMYU+kIgy8z6mllbYDIw56Bt5gDfq7v9HeBf7u7hiykiIodyyFUu7l5tZtcDr1G7bHGGu680s7uBHHefAzwBzDSzPGAXtaUvIiLNqFHr0N19HjDvoMfurHe7ArgwvNFERORw6Fh3EZEYYUFNdZtZIbDpCP96MlAUxjjholyHR7kOX6RmU67DczS5+rh7g6tKAiv0o2FmOe6eHXSOgynX4VGuwxep2ZTr8DRVLk25iIjECBW6iEiMiNZCfyzoAF9BuQ6Pch2+SM2mXIenSXJF5Ry6iIh8WbSO0EVE5CAqdBGRGBH1hW5mt5qZm1ly0FkAzOznZrbMzJaY2etm1jvoTABm9lsz+6wu29/NrHPQmQDM7EIzW2lmITMLfHmZmY03s9VmlmdmtwWdB8DMZphZgZmtCDpLfWaWbmZvm9mqup/hTUFnAjCzBDNbYGZL63L9LOhM9ZlZnJktNrNXwv3aUV3oZpYO/AewOegs9fzW3Y9z9xHAK8Cdh/oLzeQNYKi7HwesAW4POM/nVgDnA+8FHaTe1bkmAIOBKWY2ONhUADwJjA86RAOqgVvdfTBwAnBdhPx7VQKnu/twYAQw3sxOCDhTfTcBuU3xwlFd6MAfgB8DEbNn191L691NJEKyufvrdRcfAfiY2tMgB87dc919ddA56jTm6lzNzt3fo/akdxHF3be7+6d1t/dSW1IHX/ym2Xmtsrq7ber+RMT30MzSgLOBPzfF60dtoZvZJGCruy8NOsvBzOweM9sCXErkjNDruwJ4NegQEagxV+eSBphZJjAS+CTYJLXqpjWWAAXAG+4eEbmA+6kdhIaa4sUbdbbFoJjZm0DPBp66A/hvaqdbmt3X5XL3l939DuAOM7sduB64KxJy1W1zB7W/Kj/THJkam0uil5klAS8CPzzoN9TAuHsNMKJuX9HfzWyouwe6D8LMzgEK3H2RmZ3aFO8R0YXu7uMaetzMhgF9gaV116JOAz41szHuviOoXA14htrTDjdLoR8ql5lNBc4BzhUpRp4AAAE4SURBVGjOC5Acxr9X0BpzdS6px8zaUFvmz7j7S0HnOZi77zGzt6ndBxH0TuWTgIlmdhaQAHQ0s7+6+3fD9QZROeXi7svdvbu7Z7p7JrW/Go9qjjI/FDPLqnd3EvBZUFnqM7Px1P6qN9Hd9wWdJ0I15upcUsdqR1NPALnu/vug83zOzFI+X8VlZu2AM4mA76G73+7uaXWdNZnaK7uFrcwhSgs9wv3KzFaY2TJqp4QiYikX8BDQAXijbknlI0EHAjCz88wsH/gGMNfMXgsqS91O48+vzpULPOfuK4PK8zkzmwV8BBxrZvlmdmXQmeqcBFwGnF73mVpSN/oMWi/g7brv4EJq59DDvkQwEunQfxGRGKERuohIjFChi4jECBW6iEiMUKGLiMQIFbqISIxQoYuIxAgVuohIjPh/yzfYtX4+p3UAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "plot_function(torch.sigmoid, title='Sigmoid', min=-4, max=4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "id": "htbxxyxlpokD"
      },
      "outputs": [],
      "source": [
        "def mnist_loss(predictions, targets):\n",
        "    predictions = predictions.sigmoid() #ใส่ sigmoid ไป; pytorch tensor มี built-in function ให้แล้ว\n",
        "    return torch.where(targets==1, 1-predictions, predictions).mean()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UUxkL3_2pokD"
      },
      "source": [
        "### SGD and Mini-Batches"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rxCnTrNRInAz"
      },
      "source": [
        "หลายครั้งข้อมูลทั้งชุดของเราใหญ่เกินไปที่จะใส่เข้าไปใน memory ของเครื่อง เราจึงต้องใส่เข้าไปทีละ batch ด้วย `DataLoader`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2-twSiAspokD",
        "outputId": "c3b8269f-b27a-4fb0-bda5-cbe2e18f9deb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[tensor([ 3, 12,  8, 10,  2]),\n",
              " tensor([ 9,  4,  7, 14,  5]),\n",
              " tensor([ 1, 13,  0,  6, 11])]"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ],
      "source": [
        "#ทดลองใช้ Dataloader สุ่มตัวเลขจาก 0-14; batch ละ 5 ตัวอย่าง; ให้สุ่มด้วย\n",
        "coll = range(15)\n",
        "dl = DataLoader(coll, batch_size=5, shuffle=True)\n",
        "list(dl)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7W43iIU_pokE",
        "outputId": "1b46408a-2fdc-40f7-a20d-fa1ab4c988a8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#26) [(0, 'a'),(1, 'b'),(2, 'c'),(3, 'd'),(4, 'e'),(5, 'f'),(6, 'g'),(7, 'h'),(8, 'i'),(9, 'j')...]"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ],
      "source": [
        "#สมมุติเรามี DataSet　แบบนี้\n",
        "ds = L(enumerate(string.ascii_lowercase))\n",
        "ds"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GOccHc_rpokE",
        "outputId": "1d364e87-dedd-455f-8e95-96271d3e9f2e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(tensor([17, 18, 10, 22,  8, 14]), ('r', 's', 'k', 'w', 'i', 'o')),\n",
              " (tensor([20, 15,  9, 13, 21, 12]), ('u', 'p', 'j', 'n', 'v', 'm')),\n",
              " (tensor([ 7, 25,  6,  5, 11, 23]), ('h', 'z', 'g', 'f', 'l', 'x')),\n",
              " (tensor([ 1,  3,  0, 24, 19, 16]), ('b', 'd', 'a', 'y', 't', 'q')),\n",
              " (tensor([2, 4]), ('c', 'e'))]"
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ],
      "source": [
        "#ก็ใส่เข้าไปใน dataloader ได้เช่นกัน\n",
        "dl = DataLoader(ds, batch_size=6, shuffle=True)\n",
        "list(dl)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvWlUYU8pokE"
      },
      "source": [
        "## Putting It All Together"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fjUk_EOS0deu"
      },
      "source": [
        "#### Initialize weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 59,
      "metadata": {
        "id": "lK9dlPP4pokE"
      },
      "outputs": [],
      "source": [
        "weights = init_params((28*28,1))\n",
        "bias = init_params(1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L4ejZTDi0fds"
      },
      "source": [
        "#### Initialize dataloader"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XesYQeEUpokE",
        "outputId": "b0490d69-191f-49a7-c487-eb96763a85a3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([256, 784]), torch.Size([256, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ],
      "source": [
        "#สร้าง dataloader สำหรับ train\n",
        "dl = DataLoader(dset, batch_size=256, shuffle=True)\n",
        "xb,yb = first(dl)\n",
        "xb.shape,yb.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "id": "hlw0qizVpokE"
      },
      "outputs": [],
      "source": [
        "#เราจะไม่ shuffle validation set\n",
        "valid_dl = DataLoader(valid_dset, batch_size=256, shuffle=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "opSgyb70pokF",
        "outputId": "3f99d56f-92b8-4b35-f887-72298850466e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 784])"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ],
      "source": [
        "#(batch size, 28*28)\n",
        "batch = train_x[:4]\n",
        "batch.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d_OZXa2l0iwq"
      },
      "source": [
        "#### Forward pass"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IQ5eVzE5pokF",
        "outputId": "8c2e6586-aa3f-4a20-ef4c-7d0a4b8b0d1b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([256, 1])"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ],
      "source": [
        "def linear1(xb): return xb@weights + bias\n",
        "preds = linear1(xb)\n",
        "preds.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P0dawUHM0kpf"
      },
      "source": [
        "#### Calculate loss"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 64,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_zK9HJE7pokF",
        "outputId": "5dc8119e-fac6-4fb6-d34f-67632d490ec2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(0.7690, grad_fn=<MeanBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 64
        }
      ],
      "source": [
        "def mnist_loss(predictions, targets):\n",
        "    predictions = predictions.sigmoid()\n",
        "    return torch.where(targets==1, 1-predictions, predictions).mean()\n",
        "loss = mnist_loss(preds, xb)\n",
        "loss"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tMIHxJcw0mW5"
      },
      "source": [
        "#### Backward pass - get gradients and update weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 65,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lMdSQrorpokF",
        "outputId": "65a29105-64da-4008-a8a3-37b98344972e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([784, 1]), tensor(0.0049), tensor([0.0425]))"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ],
      "source": [
        "loss.backward()\n",
        "weights.grad.shape,weights.grad.mean(), bias.grad"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "id": "K4x5HLp4pokF"
      },
      "outputs": [],
      "source": [
        "#คำนวณ gradient ด้วย .backward()\n",
        "def calc_grad(xb, yb, model):\n",
        "    preds = model(xb)\n",
        "    loss = mnist_loss(preds, yb)\n",
        "    loss.backward()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 67,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CZzqYNtgpokF",
        "outputId": "8c50d09c-6e74-4943-ce7a-a9ec8dee08f7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(-0.0007), tensor([0.0070]))"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ],
      "source": [
        "calc_grad(batch, train_y[:4], linear1)\n",
        "weights.grad.mean(),bias.grad"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R5yY7Az-pokF",
        "outputId": "9596cc88-13f5-4f32-c0d4-41a9c2de62c4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor(-0.0064), tensor([-0.0285]))"
            ]
          },
          "metadata": {},
          "execution_count": 68
        }
      ],
      "source": [
        "calc_grad(batch, train_y[:4], linear1)\n",
        "weights.grad.mean(),bias.grad"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 69,
      "metadata": {
        "id": "R-Bzu26wpokG"
      },
      "outputs": [],
      "source": [
        "#เราจำเป็นต้อง reset gradients เพื่อไม่ให้มันบวกเพิ่มขึ้นเรือ่ยๆ\n",
        "weights.grad.zero_();\n",
        "bias.grad.zero_();"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {
        "id": "gKkqYbaqpokG"
      },
      "outputs": [],
      "source": [
        "#เทรน 1 epoch\n",
        "def train_epoch(model, lr, params):\n",
        "    #โยนทุก batch ให้โมเดลที่ละ batch จนหมด\n",
        "    for xb,yb in dl:\n",
        "        #คำนวน loss และ gradients\n",
        "        calc_grad(xb, yb, model)\n",
        "        #เปลี่ยน weights ด้วย gradient * learning rate (lr)\n",
        "        for p in params:\n",
        "            p.data -= p.grad*lr\n",
        "            p.grad.zero_()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 71,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2XWh1J1ppokG",
        "outputId": "cfe8a98d-d53f-45e8-fed6-bb81899f0325"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(0.5469)"
            ]
          },
          "metadata": {},
          "execution_count": 71
        }
      ],
      "source": [
        "#คิด accuracy\n",
        "((preds>0.0).float() == yb).float().mean()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4ZfrHoey0w2z"
      },
      "source": [
        "###  Calculate metric (accuracy in this case)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {
        "id": "5QPI55L4pokG"
      },
      "outputs": [],
      "source": [
        "def batch_accuracy(xb, yb):\n",
        "    preds = xb.sigmoid()\n",
        "    correct = (preds>0.5) == yb\n",
        "    return correct.float().mean()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 73,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8UVCx6RApokG",
        "outputId": "fcdbc646-2421-4dfe-bb67-6d51e28d62ef"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(0.5000)"
            ]
          },
          "metadata": {},
          "execution_count": 73
        }
      ],
      "source": [
        "batch_accuracy(linear1(batch), train_y[:4])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 74,
      "metadata": {
        "id": "GbJQiOgIpokG"
      },
      "outputs": [],
      "source": [
        "#ทำการ validate ด้วย validation set\n",
        "def validate_epoch(model):\n",
        "    accs = [batch_accuracy(model(xb), yb) for xb,yb in valid_dl]\n",
        "    return round(torch.stack(accs).mean().item(), 4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 75,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "txrLkdJQpokH",
        "outputId": "eeafe072-31d3-438f-e99e-f2377b03eac0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.5484"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ],
      "source": [
        "validate_epoch(linear1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 76,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HRIapIkfpokH",
        "outputId": "8537b6d8-70fc-4d5b-cca9-40ab62cf8d2e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9253"
            ]
          },
          "metadata": {},
          "execution_count": 76
        }
      ],
      "source": [
        "#เทรนไป 1 epoch; accuracy เพิ่มเกือบเท่าตัว!\n",
        "lr = 1.\n",
        "\n",
        "params = weights, bias\n",
        "train_epoch(linear1, lr, params)\n",
        "validate_epoch(linear1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 77,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hvBUTwULpokH",
        "outputId": "79511794-ad36-4c6b-a2a9-02df71aa8e64"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9554 0.9608 0.9632 0.9657 0.9681 0.9696 0.9716 0.973 0.974 0.974 0.975 0.9775 0.9745 0.976 0.9779 0.9779 0.9784 0.9784 0.9784 0.9789 "
          ]
        }
      ],
      "source": [
        "#เทรนไป 20 epoch; ทายถูกเกือบหมด\n",
        "for i in range(20):\n",
        "    train_epoch(linear1, lr, params)\n",
        "    print(validate_epoch(linear1), end=' ')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uVt65caepokH"
      },
      "source": [
        "### Optimizer as a class"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gyvY33oqMdZk"
      },
      "source": [
        "หลายครั้งเรารวบรวมหน้าที่การ update weights มาเป็น class ชื่อ optimizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 78,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ETZ0sdlXpokH",
        "outputId": "acb76f13-b382-48dd-d2a4-f19d8b93b984"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Linear(in_features=784, out_features=1, bias=True)"
            ]
          },
          "metadata": {},
          "execution_count": 78
        }
      ],
      "source": [
        "#nn.Linear ก็คือฟังชั่นที่ x*W.T + b เราเขียนเมื่อกี้เลย\n",
        "linear_model = nn.Linear(28*28,1, bias=True)\n",
        "linear_model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 79,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Op_9VxfipokH",
        "outputId": "52bc6520-0ddf-4954-9f5e-083a5ab08813"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([1, 784]), torch.Size([1]))"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ],
      "source": [
        "w,b = linear_model.parameters()\n",
        "w.shape,b.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 80,
      "metadata": {
        "id": "HDPzEqO-pokI"
      },
      "outputs": [],
      "source": [
        "#optimizer　แบบง่ายที่สุด\n",
        "class BasicOptim:\n",
        "    def __init__(self,params,lr): \n",
        "        self.params,self.lr = list(params),lr\n",
        "\n",
        "    #step ทำการ update weights\n",
        "    def step(self, *args, **kwargs):\n",
        "        for p in self.params: p.data -= p.grad.data * self.lr\n",
        "\n",
        "    #zero grad เพื่อทำการรีเซต gradients\n",
        "    def zero_grad(self, *args, **kwargs):\n",
        "        for p in self.params: p.grad = None"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 81,
      "metadata": {
        "id": "UIAhC389pokI"
      },
      "outputs": [],
      "source": [
        "opt = BasicOptim(linear_model.parameters(), lr)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 82,
      "metadata": {
        "id": "zNokDJCupokI"
      },
      "outputs": [],
      "source": [
        "def train_epoch(model):\n",
        "    for xb,yb in dl:\n",
        "        calc_grad(xb, yb, model)\n",
        "        #เปลี่ยนโค้ดเป็น optimizer แทน\n",
        "        opt.step()\n",
        "        opt.zero_grad()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 83,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pUr7KQecpokI",
        "outputId": "03eae252-fa47-49ef-dae7-7cf2500497e5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.3567"
            ]
          },
          "metadata": {},
          "execution_count": 83
        }
      ],
      "source": [
        "validate_epoch(linear_model)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 84,
      "metadata": {
        "id": "wQptVRJfpokI"
      },
      "outputs": [],
      "source": [
        "def train_model(model, epochs):\n",
        "    for i in range(epochs):\n",
        "        train_epoch(model)\n",
        "        print(validate_epoch(model), end=' ')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 85,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kRbgjPURpokI",
        "outputId": "8a680b82-56a8-4e0a-8444-e5dde4b97520"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9706 0.975 0.975 0.9779 0.9779 0.9784 0.9794 0.9794 0.9799 0.9804 0.9804 0.9804 0.9804 0.9814 0.9819 0.9823 0.9823 0.9819 0.9823 0.9828 "
          ]
        }
      ],
      "source": [
        "#ได้ผลดีเหมือนเดิม\n",
        "train_model(linear_model, 20)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rBjvWLYwNII8"
      },
      "source": [
        "## ทำงานง่ายขึ้นด้วย PyTorch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "60qaqoMbNMA4"
      },
      "source": [
        "Pytorch ได้ทำการเปลี่ยนฟังชั่นและ class ต่างๆที่เราใช้งานเป็นฟังชั่นพื้นฐานของ package ให้เราเรียกใช้ได้"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 86,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DY1kWv7lpokI",
        "outputId": "21c7a564-f0be-4d80-e07c-ff0fd34667e1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.9726 0.9755 0.976 0.976 0.977 0.9764 0.9789 0.9799 0.9789 0.9789 0.9799 0.9799 0.9794 0.9803 0.9803 0.9808 0.9803 0.9809 0.9823 0.9813 "
          ]
        }
      ],
      "source": [
        "linear_model = nn.Linear(28*28,1,bias=False) #เหมือนกับ init_weights และการคูณ matrix ที่เราเพิ่งทำไป\n",
        "opt = SGD(linear_model.parameters(), lr) #เหมือนกับ BasicOptim ที่เราเพิ่งทำไป\n",
        "\n",
        "#เทรนโมเดลได้ผลดีเหมือนเดิม\n",
        "train_model(linear_model, 20)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ctjZxC3Nxi8"
      },
      "source": [
        "## ใช้ fastai ให้สะดวกยิ่งขึ้น"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 87,
      "metadata": {
        "id": "jtzVcFkxpokJ"
      },
      "outputs": [],
      "source": [
        "dls = DataLoaders(dl, valid_dl)\n",
        "#รวมทุกอย่างเข้าด้วยกันด้วย class Learner\n",
        "learn = Learner(dls, nn.Linear(28*28,1), opt_func=SGD,\n",
        "                loss_func=mnist_loss, metrics=batch_accuracy)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 88,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 676
        },
        "id": "0WcORvumpokJ",
        "outputId": "0a15116b-a4ca-4557-e01e-0be547013147"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>batch_accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>0.059673</td>\n",
              "      <td>0.041307</td>\n",
              "      <td>0.972031</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.040368</td>\n",
              "      <td>0.034897</td>\n",
              "      <td>0.975466</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.032661</td>\n",
              "      <td>0.032365</td>\n",
              "      <td>0.974975</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.028252</td>\n",
              "      <td>0.029463</td>\n",
              "      <td>0.977429</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.025178</td>\n",
              "      <td>0.028448</td>\n",
              "      <td>0.977920</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.023642</td>\n",
              "      <td>0.027670</td>\n",
              "      <td>0.977920</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.022653</td>\n",
              "      <td>0.026340</td>\n",
              "      <td>0.978410</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.021488</td>\n",
              "      <td>0.025264</td>\n",
              "      <td>0.979882</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.020988</td>\n",
              "      <td>0.024670</td>\n",
              "      <td>0.979392</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.020430</td>\n",
              "      <td>0.024237</td>\n",
              "      <td>0.979392</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.019526</td>\n",
              "      <td>0.023921</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.019002</td>\n",
              "      <td>0.023656</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.018814</td>\n",
              "      <td>0.023699</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>0.018070</td>\n",
              "      <td>0.022686</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>14</td>\n",
              "      <td>0.017431</td>\n",
              "      <td>0.022708</td>\n",
              "      <td>0.981845</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>0.017356</td>\n",
              "      <td>0.022409</td>\n",
              "      <td>0.981845</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>16</td>\n",
              "      <td>0.017296</td>\n",
              "      <td>0.021927</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>17</td>\n",
              "      <td>0.016958</td>\n",
              "      <td>0.021855</td>\n",
              "      <td>0.981845</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>18</td>\n",
              "      <td>0.017005</td>\n",
              "      <td>0.021677</td>\n",
              "      <td>0.981845</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>19</td>\n",
              "      <td>0.016577</td>\n",
              "      <td>0.021702</td>\n",
              "      <td>0.981845</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "learn.fit(20, lr=lr)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H_UoKZ24pokJ"
      },
      "source": [
        "## สร้าง Architecture ที่ Deep ขึ้น"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 89,
      "metadata": {
        "id": "4s1EGNszpokK"
      },
      "outputs": [],
      "source": [
        "#แทนที่จะทำ matrix multipilcation + bias ครั้งเดียว เราใส่ ReLU activation ไปเพิ่ม\n",
        "def simple_net(xb): \n",
        "    res = xb@w1 + b1\n",
        "    res = res.max(tensor(0.0)) #นี่คือ relu\n",
        "    res = res@w2 + b2 #แล้วทำ matrix multiplication + bias อีกรอบ\n",
        "    return res"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 90,
      "metadata": {
        "id": "bKDW0LWlpokK"
      },
      "outputs": [],
      "source": [
        "w1 = init_params((28*28,30))\n",
        "b1 = init_params(30)\n",
        "w2 = init_params((30,1))\n",
        "b2 = init_params(1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 91,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "Fza6OpHHpokK",
        "outputId": "b23d6134-c2bc-4d4f-d92c-af0c8a2c14df"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3/8ddHVlmUJRHZF0EBRSCkoKJ1q4pUxa6C2KrVUhFqrd2oXrU/7GbtdceF2/KwLbs7tSjiSq2FEkJYZQkgS0AS9n1J8vn9MYd7x5iQSZjJmZm8n4/HPDJzlpn3HMJnTr7nzOeYuyMiIunrpLADiIhIYqnQi4ikORV6EZE0p0IvIpLmVOhFRNJc3bADlCcjI8M7deoUdgwRkZSxYMGCbe6eWd68pCz0nTp1IicnJ+wYIiIpw8zWVzRPQzciImlOhV5EJM2p0IuIpDkVehGRNKdCLyKS5iot9GbW3szeN7PlZrbMzH5UzjJmZk+aWb6ZLTazrKh5N5vZ6uB2c7zfgIiIHF8sp1cWAz9x91wzawosMLPZ7r48apmrgW7BbQDwLDDAzFoADwLZgAfrznD3nXF9FyIiUqFK9+jdfYu75wb39wKfAG3LLDYE+KtHzAWamVlr4CpgtrvvCIr7bGBQXN+BiEga+M+6Hfzpn2tJROv4Ko3Rm1knoC8wr8ystsDGqMebgmkVTS/vuUeYWY6Z5RQVFVUllohISivce4hRk3OZNG8DB4+WxP35Yy70ZtYEeBm42933xDuIu49392x3z87MLPdbvCIiaae4pJQfTl7I3kNHefamLBrVj3/DgpgKvZnVI1LkJ7n7K+UsUgC0j3rcLphW0XQREQH++PYq5q3bwW+u70X3009JyGvEctaNAX8GPnH3RytYbAbw3eDsm/OA3e6+BZgFXGlmzc2sOXBlME1EpNabvXwrz324hmH9O/CNfu0S9jqx/I0wEPgOsMTM8oJp9wIdANz9OWAmMBjIBw4AtwbzdpjZQ8D8YL2x7r4jfvFFRFLT+u37uWd6Hue0PYUHr+2Z0NeqtNC7+0eAVbKMA6MqmDcBmFCtdCIiaejQ0RJGTszFgGeH96NhvToJfb2kbFMsIpLOHnx9Gcu37GHCLdm0b9Eo4a+nFggiIjVoes5GpuVsZPSlXbmse6saeU0VehGRGrJs827uf20pA7u25MdXnFljr6tCLyJSA3YfPMqdk3Jp3qg+TwztS52TjnvoM640Ri8ikmDuzk9fXETBzoNM+8F5ZDRpUKOvrz16EZEEe37OWmYv38q9g3vQr2OLGn99FXoRkQSau3Y7f3hrBV/t1ZpbB3YKJYMKvYhIghTuOcToyQvplNGYh795LpFGAzVPY/QiIglQXFLK6CkL2X+4mMnfH0CTBuGVWxV6EZEEeGTWSv6zbgeP3dCbM1s1DTWLhm5EROJs1rLPeH7OWoYP6MDX+iauWVmsVOhFROLo0237+en0RZzb7lQeSHCzslip0IuIxMmhoyWMnJRLnTrGM8OzaFA3sc3KYqUxehGROLn/taWs+GwPE275Eu2aJ75ZWay0Ry8iEgfT5m/gxQWb+OGlXbn0rNPCjvM5KvQiIidoacFu7n99GRd1y+BHX6m5ZmWxqnToxswmANcAhe5+TjnzfwYMj3q+HkBmcHWpT4G9QAlQ7O7Z8QouIpIMjjUra9m4Po/f0KdGm5XFKpY9+heAQRXNdPdH3L2Pu/cBfgl8WOZygZcG81XkRSStlJY6P5m+iC27DzJueBYta7hZWawqLfTuPgeI9Tqvw4ApJ5RIRCRFPD9nLe98spX7Bvcgq0PzsONUKG5j9GbWiMie/8tRkx1428wWmNmIStYfYWY5ZpZTVFQUr1giIgnx8ZptPDJrBdec25qbL+gUdpzjiufB2GuBf5UZtrnQ3bOAq4FRZvblilZ29/Hunu3u2ZmZmXGMJSISX1v3HOKuKQvpnNGYh78RXrOyWMWz0A+lzLCNuxcEPwuBV4H+cXw9EZEad7SklNGTczlwpITnbupH4xCblcUqLoXezE4FLgZej5rW2MyaHrsPXAksjcfriYiE5Q9vrWD+pzv53dd70S3kZmWxiuX0yinAJUCGmW0CHgTqAbj7c8FiXwPedvf9Uau2Al4N/qSpC0x297fiF11EpGa9tXQL//PPdXznvI4M6dM27Dgxq7TQu/uwGJZ5gchpmNHT1gK9qxtMRCSZrC3ax09fXEzv9s34r2t6hB2nSvTNWBGRShw8UsKdk3Kpl2TNymKV/EcRRERC5O7c99oSVm7dywu39qdts5PDjlRl2qMXETmOqfM38kpuAXdd1o2Lz0zNU79V6EVEKrC0YDcPzog0K7vr8m5hx6k2FXoRkXLsPnCUOyYuIKNxfZ4Y2jcpm5XFSmP0IiJllJY690zPY+ueQ0z/wfm0aFw/7EgnRHv0IiJlPPvhGt5dUch/fbUnfZO4WVmsVOhFRKL8K38b//32Sq7t3Ybvnt8x7DhxoUIvIhL4bHekWVmXzCb8/uu9kr5ZWaw0Ri8iwv81Kzt4tISpw7NSollZrNLnnYiInIDfv7mCnPU7eXJY35RpVhYrDd2ISK03c8kW/vzROm65oBPX9W4Tdpy4U6EXkVptTdE+fv7SYvp2aMa9g1OrWVmsVOhFpNY6cKSYkRMXUL/uSYy7MYv6ddOzJGqMXkRqJXfnvleXsrpwH3/9Xn/apGCzslil58eXiEglJs3bwKsLC7j78jO5qFtqNiuLVaWF3swmmFmhmZV7GUAzu8TMdptZXnB7IGreIDNbaWb5ZjYmnsFFRKpr8aZdjP37ci4+M5MfXtY17DgJF8se/QvAoEqW+ae79wluYwHMrA4wDrga6AkMM7OeJxJWRORE7dx/hJETc8ls2oDHb+jDSSncrCxWlRZ6d58D7KjGc/cH8t19rbsfAaYCQ6rxPCIicVFa6vx4eh6Few8xbngWzVO8WVms4jVGf76ZLTKzN83s7GBaW2Bj1DKbgmnlMrMRZpZjZjlFRUVxiiUi8n+efj+fD1YW8cA1PenTvlnYcWpMPAp9LtDR3XsDTwGvVedJ3H28u2e7e3ZmZnofGBGRmvfR6m089s4qru/ThpvOS49mZbE64ULv7nvcfV9wfyZQz8wygAKgfdSi7YJpIiI1avOug9w1dSFdM5vw2zRqVharEy70Zna6BVvNzPoHz7kdmA90M7POZlYfGArMONHXExGpiiPFpYyanMvhoyU8951+NKpf+74+VOk7NrMpwCVAhpltAh4E6gG4+3PAN4GRZlYMHASGursDxWY2GpgF1AEmuPuyhLwLEZEK/HbmJyzcsItxN2ZxRmaTsOOEotJC7+7DKpn/NPB0BfNmAjOrF01E5MS8sXgzL3z8KbcO7MRXz20ddpzQ6JuxIpKW8gv38YuXFpPVoRm/vDo9m5XFSoVeRNLO/sORZmUN6tVh3PD0bVYWq9p3VEJE0pq7c++rS8gv2sffvjeA1qemb7OyWNXujzkRSTsT567n9bzN3POVM7mwW0bYcZKCCr2IpI28jbsY+8ZyLjkrk1GXpn+zslip0ItIWti5/wijJuVyWtOGtaZZWaw0Ri8iKa+01Ll7Wh5Few/z0sjzadaodjQri5X26EUk5T31Xj4fririgWt7cm672tOsLFYq9CKS0uasKuLxd1fx9b5tGT6gQ9hxkpIKvYikrIJdB/nR1IWceVpTfvO12tesLFYq9CKSko4UlzJqUi5HS5xnb8ri5Pp1wo6UtHQwVkRS0m/+sZy8jbt4ZngWXWpps7JYaY9eRFLOjEWb+cu/13P7hZ0Z3Kv2NiuLlQq9iKSU1Vv3MublxWR3bM4vru4edpyUoEIvIilj/+FiRk7KpVH9SLOyenVUwmJR6VYyswlmVmhmSyuYP9zMFpvZEjP72Mx6R837NJieZ2Y58QwuIrWLuzPmlSWsLdrHk0P70uqUhmFHShmxfBy+AAw6zvx1wMXu3gt4CBhfZv6l7t7H3bOrF1FEBP767/X8fdFmfnLlWVzQVc3KqiKWK0zNMbNOx5n/cdTDuUQuAi4iEje5G3by638s5/LupzHy4jPCjpNy4j3AdRvwZtRjB942swVmNuJ4K5rZCDPLMbOcoqKiOMcSkVS1Y/8RRk/KpdUpDXn022pWVh1xO4/ezC4lUugvjJp8obsXmNlpwGwzW+Huc8pb393HEwz7ZGdne7xyiUjqKil1fjR1Idv2H+GVkRdwaqN6YUdKSXHZozezc4E/AUPcffux6e5eEPwsBF4F+sfj9USkdnjy3dX8c/U2/t91Z3NO21PDjpOyTrjQm1kH4BXgO+6+Kmp6YzNreuw+cCVQ7pk7IiJlfbCykCffW803stox9Evtw46T0iodujGzKcAlQIaZbQIeBOoBuPtzwANAS+CZoKFQcXCGTSvg1WBaXWCyu7+VgPcgImlm084D3D0tj7NaNeXX15+jZmUnKJazboZVMv924PZypq8Fen9xDRGRih0uLmHU5IWUlDjP3tRPzcriQE3NRCSp/PqNT1i0cRfP3ZRF54zGYcdJC/r+sIgkjdfzCvjb3PV8/6LODDpHzcriRYVeRJLCqq17GfPyEr7UqTk/H6RmZfGkQi8iodt3uJiRExfQuEFdnr5RzcriTVtTRELl7vzi5cWs27afp4apWVkiqNCLSKhe+PhT/rF4Cz+7qjvnn9Ey7DhpSYVeREKzYP1OfvOPT/hKj1bccXGXsOOkLRV6EQnF9n2HGT05lzbNTua/v91bX4pKIJ1HLyI1LtKsLI/tx5qVnaxmZYmkPXoRqXFPvLOKj/K38dAQNSurCSr0IlKj3l9RyJPv5fOtfu244Usdwo5TK6jQi0iN2bgj0qysR+tTeOj6c8KOU2uo0ItIjYg0K8ultNR5dngWDeupWVlN0cFYEakRY/++nMWbdvP8d/rRSc3KapT26EUk4V5duIlJ8zbwgy934aqzTw87Tq2jQi8iCbXys73c+8pS+nduwc+uOivsOLVSTIXezCaYWaGZlXspQIt40szyzWyxmWVFzbvZzFYHt5vjFVxEkt/eQ0cZOXEBTRrW5elhfamrZmWhiHWrvwAMOs78q4FuwW0E8CyAmbUgcunBAUQuDP6gmTWvblgRSR3HmpWt33GAp4f15TQ1KwtNTIXe3ecAO46zyBDgrx4xF2hmZq2Bq4DZ7r7D3XcCszn+B4aIpIkJ//qUmUs+4+dXncWALmpWFqZ4/R3VFtgY9XhTMK2i6V9gZiPMLMfMcoqKiuIUS0TCsGD9Dn438xOu7NmKEV9Ws7KwJc2AmbuPd/dsd8/OzMwMO46IVNO2fYe5c1IubZufzCPfUrOyZBCvQl8AtI963C6YVtF0EUlDkWZlC9l14CjPDu+nZmVJIl6Ffgbw3eDsm/OA3e6+BZgFXGlmzYODsFcG00QkDT02exX/yt/OQ9efQ882p4QdRwIxfTPWzKYAlwAZZraJyJk09QDc/TlgJjAYyAcOALcG83aY2UPA/OCpxrr78Q7qikiKem/FVp5+P5+hX2rPt7PbV76C1JiYCr27D6tkvgOjKpg3AZhQ9Wgikio27jjA3VPzOLvNKfzqurPDjiNlJM3BWBFJTYeOljBy0gIAnh3eT83KkpCamonICRn7xnKWFuzhf76bTYeWjcKOI+XQHr2IVNsruZuYPG8Dd1x8Blf0bBV2HKmACr2IVMuKz/Zw76tLGNC5BT+98syw48hxqNCLSJXtOXSUkRNzOaVhPZ66Uc3Kkp3G6EWkStydn7+4mA07DjDl++dxWlM1K0t2+hgWkSr580freGvZZ4wZ1J3+nVuEHUdioEIvIjGb/+kOfvfmCgadfTq3X9Q57DgSIxV6EYlJ0d7DjJqUS/vmJ/OHb52rZmUpRGP0IlKp4pJS7pqykD2HjvKX7/XnlIZqVpZKVOhFpFKPzl7Fv9du54/f6k2P1mpWlmo0dCMixzV7+Vae+WANw/q355v92oUdR6pBhV5EKrRh+wHumZ7HOW1P4cFr1awsVanQi0i5jjUrM9SsLNVpjF5EyvWrGctYtnkPf745m/Yt1KwslWmPXkS+4MWcjUydv5E7LzmDy3uoWVmqi6nQm9kgM1tpZvlmNqac+Y+ZWV5wW2Vmu6LmlUTNmxHP8CISf8s37+G/XlvK+V1acs8ValaWDiodujGzOsA44ApgEzDfzGa4+/Jjy7j7j6OW/yHQN+opDrp7n/hFFpFE2XPoKHdOWkCzRvV4cpialaWLWP4V+wP57r7W3Y8AU4Ehx1l+GDAlHuFEpOa4Oz97cRGbdh5k3I1ZZDZtEHYkiZNYCn1bYGPU403BtC8ws45AZ+C9qMkNzSzHzOaa2fUVvYiZjQiWyykqKoohlojE0//8cy2zlm1lzNXdye6kZmXpJN5/lw0FXnL3kqhpHd09G7gReNzMzihvRXcf7+7Z7p6dmZkZ51gicjzz1m7n4bdWMrjX6dx2oZqVpZtYCn0B0D7qcbtgWnmGUmbYxt0Lgp9rgQ/4/Pi9iISscO8hRk9ZSMcWjXj4G2pWlo5iKfTzgW5m1tnM6hMp5l84e8bMugPNgX9HTWtuZg2C+xnAQGB52XVFJBzFJaX8cPJC9h46yjM3ZdFUzcrSUqVn3bh7sZmNBmYBdYAJ7r7MzMYCOe5+rOgPBaa6u0et3gN43sxKiXyo/D76bB0RCdcf317FvHU7ePTbvel+upqVpauYvhnr7jOBmWWmPVDm8a/KWe9joNcJ5BORBHl72Wc89+EabhzQga9nqVlZOtNJsiK10Prt+/nJi4vo1fZUHrimZ9hxJMFU6EVqmUNHSxg5MZeTzHhmeJaaldUCamomUss88PpSlm/Zw4Rb1KysttAevUgtMn3+RqbnbGL0pV25rLualdUWKvQitcSyzbu5//WlDOzakh+rWVmtokIvUgvsPniUOyfl0rxRfZ4Y2pc6J+lLUbWJxuhF0py789MXF1Gw8yDTfnAeGU3UrKy20R69SJp7fs5aZi/fyr2De9Cvo5qV1UYq9CJpbO7a7fzhrRV89dzW3DqwU9hxJCQq9CJpqnDPIUZPXkinjMZqVlbLaYxeJA0Vl5QyespC9h8uZtLtA2jSQP/VazP964ukoUdmreQ/63bw+A19OOv0pmHHkZBp6EYkzby19DOen7OWm87rwPV9y70YnNQyKvQiaeTTbfv52YuL6N3uVO5XszIJqNCLpImDR0q4Y+IC6tQxxg3PokFdNSuTiJgKvZkNMrOVZpZvZmPKmX+LmRWZWV5wuz1q3s1mtjq43RzP8CIS4e7c//pSVm7dy2M39KFdczUrk/9T6cFYM6sDjAOuADYB881sRjlXiprm7qPLrNsCeBDIBhxYEKy7My7pRQSAafM38tKCTdx1WVcuPeu0sONIkollj74/kO/ua939CDAVGBLj818FzHb3HUFxnw0Mql5UESnP0oLdPDBjGRd1y+BHX1GzMvmiWAp9W2Bj1ONNwbSyvmFmi83sJTNrX8V1MbMRZpZjZjlFRUUxxBKR3QeOcsfEBbRsXJ/Hb+ijZmVSrngdjP070MndzyWy1/6Xqj6Bu49392x3z87MzIxTLJH0VVrq3DM9j617DjFueBYt1axMKhBLoS8A2kc9bhdM+1/uvt3dDwcP/wT0i3VdEameZz9cw7srCrlvcA+yOjQPO44ksVgK/Xygm5l1NrP6wFBgRvQCZtY66uF1wCfB/VnAlWbW3MyaA1cG00TkBHy8Zhv//fZKru3dhpsv6BR2HElylZ514+7FZjaaSIGuA0xw92VmNhbIcfcZwF1mdh1QDOwAbgnW3WFmDxH5sAAY6+47EvA+RGqNz3Yf4q4pC+mc0Zjffb2XmpVJpczdw87wBdnZ2Z6TkxN2DJGkc7SklGHj57J8yx5eHzWQbq3Ux0YizGyBu2eXN09NzURSyMNvriBn/U6eGNpHRV5iphYIIinizSVb+NNH6/ju+R0Z0kfNyiR2KvQiKWBt0T5+9tJierdvxn1f7RF2HEkxKvQiSe7gkRJGTsylXh3jGTUrk2rQGL1IEnN37nttCasK9/LCrf1p2+zksCNJCtIevUgSm/KfjbySW8Bdl3Xj4jP1jXGpHhV6kSS1eNMufhU0K7vr8m5hx5EUpkIvkoR2HTjCyIm5ZDSpzxND+6pZmZwQjdGLJJnSUufH0/Io3HuIF++4gBaN64cdSVKc9uhFkswzH+Tz/soi7r+mJ33aNws7jqQBFXqRJPKv/G08OnsV1/Vuw3fO6xh2HEkTKvQiSeJYs7IumU3UrEziSmP0IkngaEkpoybncvBoCdNuyqJxA/3XlPjRb5NIEvjdzBUsWL+Tp4b1petpalYm8aWhG5GQvbF4MxP+tY5bLujEtb3bhB1H0pAKvUiI8gv38YuXFtO3QzPuHaxmZZIYMRV6MxtkZivNLN/MxpQz/x4zW25mi83sXTPrGDWvxMzygtuMsuuK1FYHjhRz56QFNKhXh3E3ZlG/rva7JDEqHaM3szrAOOAKYBMw38xmuPvyqMUWAtnufsDMRgJ/AG4I5h109z5xzi2S0tyde19ZwurCffz1e/1po2ZlkkCx7EL0B/Ldfa27HwGmAkOiF3D39939QPBwLtAuvjFF0svEeRt4LW8zd19+Jhd1U7MySaxYCn1bYGPU403BtIrcBrwZ9bihmeWY2Vwzu76ilcxsRLBcTlFRUQyxRFLToo27eOjvy7nkrEx+eFnXsONILRDX0yvN7CYgG7g4anJHdy8wsy7Ae2a2xN3XlF3X3ccD4yFycfB45hJJFjv3H+HOSblkNm3AY9/uw0lqViY1IJY9+gKgfdTjdsG0zzGzrwD3Ade5++Fj0929IPi5FvgA6HsCeUVSVmmp8+PpeRTtPcwzw7NormZlUkNiKfTzgW5m1tnM6gNDgc+dPWNmfYHniRT5wqjpzc2sQXA/AxgIRB/EFak1nn4/nw9WFvHAtT3prWZlUoMqHbpx92IzGw3MAuoAE9x9mZmNBXLcfQbwCNAEeDHoz7HB3a8DegDPm1kpkQ+V35c5W0ekVvjn6iIee2cVX+vbluEDOoQdR2oZc0++4fDs7GzPyckJO4ZIXGzedZBrnvqIjCb1eW3UQBrVV+cRiT8zW+Du2eXN0zc0RBLoSHGkWdmR4lKevamfiryEQr91Ign025mfsHDDLsbdmMUZmU3CjiO1lPboRRJkxqLNvPDxp3xvYGe+em7rsONILaZCL5IA+YV7GfPyYvp1bM4vB3cPO47Ucir0InG2/3AxIyfmcnLQrKxeHf03k3BpjF4kjtydX76yhDVF+/jbbQM4/dSGYUcS0R69SDz9be56ZizazD1XnMnArhlhxxEBVOhF4mbhhp089MZyLut+GndeomZlkjxU6EXiYMf+I4yalEurUxry6Ld7q1mZJBWN0YucoJJS5+5peWzbd4SXR15As0ZqVibJRYVe5AQ99d5q5qwq4rdf60WvdqeGHUfkCzR0I3ICPlxVxBPvrubrWW0Z1r995SuIhECFXqSaNu86yN1TF3JWq6b85vpeBJ1bRZKOCr1INRwpLuXOSbkcLXGeGZ7FyfXrhB1JpEIaoxepht/8Yzl5G3fx3E1ZdFGzMkly2qMXqaLX8wr4y7/Xc/uFnRl0jpqVSfKLqdCb2SAzW2lm+WY2ppz5DcxsWjB/npl1ipr3y2D6SjO7Kn7RRWreW0u38MtXlvClTs35xdVqViapodKhGzOrA4wDrgA2AfPNbEaZSwLeBux0965mNhR4GLjBzHoSucbs2UAb4B0zO9PdS+L9RkQSqXDvIR58fRlvLv2Ms9ucwtNqViYpJJYx+v5AvruvBTCzqcAQPn+R7yHAr4L7LwFPW+QUhCHAVHc/DKwzs/zg+f4dn/ifd+1TH3HoqD5DJP627D7EkZJSfj7oLL5/URcVeUkpsRT6tsDGqMebgAEVLRNcTHw30DKYPrfMum3LexEzGwGMAOjQoXoXTz4jszFHSkqrta7I8fRp34wfXHwGXU/TgVdJPUlz1o27jwfGQ+Ti4NV5jseH9o1rJhGRdBDL358FQPRX/toF08pdxszqAqcC22NcV0REEiiWQj8f6GZmnc2sPpGDqzPKLDMDuDm4/03gPXf3YPrQ4KyczkA34D/xiS4iIrGodOgmGHMfDcwC6gAT3H2ZmY0Fctx9BvBn4G/BwdYdRD4MCJabTuTAbTEwSmfciIjULIvseCeX7Oxsz8nJCTuGiEjKMLMF7p5d3jydIyYikuZU6EVE0pwKvYhImlOhFxFJc0l5MNbMioD11Vw9A9gWxzjxolxVo1xVo1xVk465Orp7ZnkzkrLQnwgzy6noyHOYlKtqlKtqlKtqalsuDd2IiKQ5FXoRkTSXjoV+fNgBKqBcVaNcVaNcVVOrcqXdGL2IiHxeOu7Ri4hIFBV6EZE0l/KF3sweMbMVZrbYzF41s2YVLHfcC5wnINe3zGyZmZWaWYWnS5nZp2a2xMzyzCzhndyqkKumt1cLM5ttZquDn80rWK4k2FZ5Zla2XXY88xz3/Qett6cF8+eZWadEZalirlvMrChqG91eA5kmmFmhmS2tYL6Z2ZNB5sVmlpXoTDHmusTMdkdtqwdqKFd7M3vfzJYH/xd/VM4y8d1m7p7SN+BKoG5w/2Hg4XKWqQOsAboA9YFFQM8E5+oBnAV8AGQfZ7lPgYwa3F6V5gppe/0BGBPcH1Pev2Mwb18NbKNK3z9wJ/BccH8oMC1Jct0CPF1Tv0/Ba34ZyAKWVjB/MPAmYMB5wLwkyXUJ8EZNbqvgdVsDWcH9psCqcv4d47rNUn6P3t3fdvfi4OFcIlexKut/L3Du7keAYxc4T2SuT9x9ZSJfozpizFXj2yt4/r8E9/8CXJ/g1zueWN5/dN6XgMvNzJIgV41z9zlErkNRkSHAXz1iLtDMzFonQa5QuPsWd88N7u8FPuGL19KO6zZL+UJfxveIfAqWVd4Fzsu9SHkIHHjbzBYEF0hPBmFsr1buviW4/xnQqoLlGppZjpnNNbNEfRjE8v7/d5lgR2M30DJBeaqSC+AbwZ/7L5lZ+3Lm17Rk/v93vpktMrM3zezsmn7xYMivLzCvzKy4brOkuTj48ZjZO8Dp5cy6z91fDyKcOkgAAAJNSURBVJa5j8hVrCYlU64YXOjuBWZ2GjDbzFYEeyJh54q74+WKfuDubmYVnffbMdheXYD3zGyJu6+Jd9YU9ndgirsfNrMfEPmr47KQMyWrXCK/T/vMbDDwGpHLndYIM2sCvAzc7e57EvlaKVHo3f0rx5tvZrcA1wCXezDAVUZCLlJeWa4Yn6Mg+FloZq8S+fP8hAp9HHLV+PYys61m1trdtwR/ohZW8BzHttdaM/uAyN5QvAt9LO//2DKbzKwucCqwPc45qpzL3aMz/InIsY+wJeT36URFF1d3n2lmz5hZhrsnvNmZmdUjUuQnufsr5SwS122W8kM3ZjYI+DlwnbsfqGCxWC5wXuPMrLGZNT12n8iB5XLPEKhhYWyv6AvM3wx84S8PM2tuZg2C+xnAQCLXI463WN5/dN5vAu9VsJNRo7nKjONeR2T8N2wzgO8GZ5KcB+yOGqYLjZmdfuy4ipn1J1IPE/1hTfCafwY+cfdHK1gsvtuspo84x/sG5BMZy8oLbsfOhGgDzIxabjCRo9triAxhJDrX14iMqx0GtgKzyuYicvbEouC2LFlyhbS9WgLvAquBd4AWwfRs4E/B/QuAJcH2WgLclsA8X3j/wFgiOxQADYEXg9+//wBdEr2NYsz1u+B3aRHwPtC9BjJNAbYAR4PfrduAO4A7gvkGjAsyL+E4Z6HVcK7RUdtqLnBBDeW6kMixucVRdWtwIreZWiCIiKS5lB+6ERGR41OhFxFJcyr0IiJpToVeRCTNqdCLiKQ5FXoRkTSnQi8ikub+PzGlRgmpqkeaAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "plot_function(F.relu)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 92,
      "metadata": {
        "id": "QyuXxFphpokK"
      },
      "outputs": [],
      "source": [
        "#แน่นอนว่าอ่านง่ายกว่าถ้าใช้ pytorch แทนที่จะเขียนฟังชั่นเอง\n",
        "simple_net = nn.Sequential(\n",
        "    nn.Linear(28*28,30),\n",
        "    nn.ReLU(),\n",
        "    nn.Linear(30,1)\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 93,
      "metadata": {
        "id": "94urzFOSpokK"
      },
      "outputs": [],
      "source": [
        "learn = Learner(dls, simple_net, opt_func=SGD,\n",
        "                loss_func=mnist_loss, metrics=batch_accuracy)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 94,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "Z8WM5ImkpokK",
        "outputId": "cb7de009-0755-421c-c896-4ecbeee52390"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>batch_accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>0.253839</td>\n",
              "      <td>0.101416</td>\n",
              "      <td>0.962709</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.118263</td>\n",
              "      <td>0.055467</td>\n",
              "      <td>0.969578</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.071047</td>\n",
              "      <td>0.044955</td>\n",
              "      <td>0.970559</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.050435</td>\n",
              "      <td>0.039946</td>\n",
              "      <td>0.972031</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.039966</td>\n",
              "      <td>0.036746</td>\n",
              "      <td>0.972522</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.034656</td>\n",
              "      <td>0.034883</td>\n",
              "      <td>0.973994</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.030598</td>\n",
              "      <td>0.033072</td>\n",
              "      <td>0.974485</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.028436</td>\n",
              "      <td>0.031563</td>\n",
              "      <td>0.974485</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.026895</td>\n",
              "      <td>0.030731</td>\n",
              "      <td>0.975957</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.025961</td>\n",
              "      <td>0.029774</td>\n",
              "      <td>0.977429</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.025033</td>\n",
              "      <td>0.028958</td>\n",
              "      <td>0.977429</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.023533</td>\n",
              "      <td>0.028051</td>\n",
              "      <td>0.977429</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.022895</td>\n",
              "      <td>0.027639</td>\n",
              "      <td>0.978410</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>0.021776</td>\n",
              "      <td>0.026808</td>\n",
              "      <td>0.978410</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>14</td>\n",
              "      <td>0.021131</td>\n",
              "      <td>0.026761</td>\n",
              "      <td>0.977429</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>0.020641</td>\n",
              "      <td>0.026007</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>16</td>\n",
              "      <td>0.020077</td>\n",
              "      <td>0.025396</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>17</td>\n",
              "      <td>0.019857</td>\n",
              "      <td>0.024965</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>18</td>\n",
              "      <td>0.019774</td>\n",
              "      <td>0.024951</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>19</td>\n",
              "      <td>0.018943</td>\n",
              "      <td>0.024718</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>20</td>\n",
              "      <td>0.018828</td>\n",
              "      <td>0.024210</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>21</td>\n",
              "      <td>0.018704</td>\n",
              "      <td>0.023884</td>\n",
              "      <td>0.978901</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>22</td>\n",
              "      <td>0.018680</td>\n",
              "      <td>0.023585</td>\n",
              "      <td>0.979392</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>23</td>\n",
              "      <td>0.018192</td>\n",
              "      <td>0.023434</td>\n",
              "      <td>0.979882</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>24</td>\n",
              "      <td>0.017612</td>\n",
              "      <td>0.022855</td>\n",
              "      <td>0.979882</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>25</td>\n",
              "      <td>0.017698</td>\n",
              "      <td>0.022698</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>26</td>\n",
              "      <td>0.017734</td>\n",
              "      <td>0.022616</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>27</td>\n",
              "      <td>0.017673</td>\n",
              "      <td>0.022282</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>28</td>\n",
              "      <td>0.016994</td>\n",
              "      <td>0.022069</td>\n",
              "      <td>0.980864</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>29</td>\n",
              "      <td>0.017185</td>\n",
              "      <td>0.021872</td>\n",
              "      <td>0.980864</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>30</td>\n",
              "      <td>0.016830</td>\n",
              "      <td>0.021661</td>\n",
              "      <td>0.980864</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>31</td>\n",
              "      <td>0.016727</td>\n",
              "      <td>0.021472</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>32</td>\n",
              "      <td>0.016233</td>\n",
              "      <td>0.021404</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>33</td>\n",
              "      <td>0.015987</td>\n",
              "      <td>0.021101</td>\n",
              "      <td>0.980373</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>34</td>\n",
              "      <td>0.015671</td>\n",
              "      <td>0.020923</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>35</td>\n",
              "      <td>0.015617</td>\n",
              "      <td>0.020793</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>36</td>\n",
              "      <td>0.015403</td>\n",
              "      <td>0.020697</td>\n",
              "      <td>0.981845</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>37</td>\n",
              "      <td>0.015225</td>\n",
              "      <td>0.020656</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>38</td>\n",
              "      <td>0.015505</td>\n",
              "      <td>0.020305</td>\n",
              "      <td>0.982336</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>39</td>\n",
              "      <td>0.015315</td>\n",
              "      <td>0.020342</td>\n",
              "      <td>0.981354</td>\n",
              "      <td>00:00</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "learn.fit(40, 0.1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 95,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "Q0d__m-FpokL",
        "outputId": "b39fc03d-2bb8-4a3e-8036-e57a88e15c62"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD4CAYAAAAHHSreAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV1bn/8c/XMMskg8wQlDEiMkScBVErapVBa6HVqj97USvXa3uxSr21t9zrxatYrUNVqlSpcy0i9YqggKB1DPM8g5AECEgSIAlkeH5/nB08ZiCHcOAccp7363Ve7r3W2vs8e7ecJ2vtYcnMcM4558KdFOsAnHPOxR9PDs4558rx5OCcc64cTw7OOefK8eTgnHOunFqxDiAaWrRoYcnJybEOwznnTigLFizYZWYtK6qrEckhOTmZtLS0WIfhnHMnFElbKqvzYSXnnHPleHJwzjlXjicH55xz5USUHCQNkbRG0npJ91dQ30nSbElLJX0sqX1Y3SOSVkhaJelJhTSQ9H+SVgd1D4e1v0VSlqTFwefn0TlU55xzkaoyOUhKAp4BrgRSgFGSUso0mwhMMbPewHhgQrDt+cAFQG+gF3A2MLB0GzPrAfQFLpB0Zdj+3jSzPsHnhWofnXPOuWqJpOcwAFhvZhvN7CDwBjC0TJsUYE6wPDes3oB6QB2gLlAb2GFmeWY2FyDY50KgPc455+JCJMmhHbA1bH1bUBZuCTAiWB4ONJLU3Mw+J5QsMoPPTDNbFb6hpKbANcDssOLrgiGqtyV1iPhonHPORUW0LkiPBQZKWkRo2CgdKJbUBehJqFfQDhgs6aLSjSTVAl4HnjSzjUHxP4DkYIjqQ+Dlir5Q0mhJaZLSsrKyonQYzjkXP77YuJsFW76NyXdHkhzSgfC/3tsHZYeYWYaZjTCzvsADQVk2oV7EF2a2z8z2ATOA88I2nQSsM7Mnwva128wOBKsvAP0rCsrMJplZqpmltmxZ4QN+zjl3wlr0zR5+9uJX/PSFL1mVmXvcvz+S5PA10FVSZ0l1gJHA9PAGklpIKt3XOGBysPwNoR5FLUm1CfUqVgXb/DfQBLinzL7ahK1eW9reOecSRdbeA9z5ykJObVyXJvVrc/tfF5CTV3hcY6gyOZhZETAGmEnoh/otM1shabyka4Nmg4A1ktYCrYCHgvK3gQ3AMkLXJZaY2T+CW10fIHQhe2GZW1bvDm5vXQLcDdwSheN0zrkTQlFxCWNeW8ievIM8f1N//vTT/mTm5PNvby6ipOT4zdypmjBNaGpqqvm7lZxzNcF/v7eSFz7dxOM/PovhfUM3cf71iy38dtpy7r60K7+6vFvUvkvSAjNLrajOn5B2zrljZPX2XJ6es449+w9G1H76kgxe+HQTt5yffCgxANx4Tkeu79+eJ2ev46OVO45VuN/jycE556KsoLCYRz5YzQ+f/JSJs9Zy6R/mMW1ROocbqVm9PZf73l7K2cmn8Juren6vThL/PawXvdo15pdvLmbTrv3H+hA8OTjnXDR9tn4XQ56Yz58+3sDQPu14Y/S5dGzWgHveXMzNf/mard/mldsmJ7+Q2/+6gEb1avHMT/pRp1b5n+Z6tZN47sb+1EoSo6eksf9A0TE9Dk8OzjkXBXv2H2Ts35bwkxe+xIBXf34Oj91wFuee1py/33k+v7/2DBZs/pbLH5/HpPkbKCouAaCkxLjnjUVkZOfz7I39OLVxvUq/o/0pDXhqVD82ZO3j128vPWxP5GjViMl+nHMuVsyMdxdnMP69leTmF/KLQadz96VdqVc76VCbpJPEzecnc3lKKx58dwX/8/5qpi3K4OHrzmT2qp3MXZPFfw09g/6dmlX5fRd2bcGvh/Tg4RmrOeuTJoy++PRjclyeHJxzNdqXG3ezZsdebjq3E5KOaNuSEuPZeRsOO8b/ze48vtr8LX06NGXCiDPp2aZxpW3bNq3Pn3/Wn5krtvPguysY9sw/KTG4rl97bjy3U8Rx3X7xaSzZms3DM1bTq20Tzu/S4oiOKxKeHJxzNdb6nfu47eU09h0oIukk8dNzIv8BBnhqznoe/2gtrRvXI+mkihNLnVon8ftrz+DGcztV2iacJIb0asP5XVowceYaMrLzeWh4ryNKXJJ49EdnsSFrH0vTc45JcvDnHJxzNdLegkKGPfNPsvMK6daqEWlbvuXN28+jX8dTItp+zuod3PZyGsP7tOOxG8464l7H8VBQWPy94asj5c85OOcSipkx9m9L2Lw7j6d/0o9nb+xHmyb1ufOVBWTtPVDl9pt37eeeNxbTs3VjHhp+ZlwmBuCoEkNVPDk452qcZ+dtYOaKHYy7sgfnnd6cpg3q8NyN/cnJL+Su1xZSGNwpVJG8g0Xc8coCJPH8Tf2pX+fY/QDHM7/m4Jw7pvIOFvHcvI2s2V79N4vWqZXELecn079T1UNC89dmMXHmGq45qy23Xdj5UHlK28Y8PKI397y5mAnvr+bBa8pOaBnqcdz/92Ws2bGXl24dQIdmDaod84nOk4Nz7piZtzaLB95ZxrY9+XRr1ZCTqjk8s3PvAd5bmsGN53Ti3iHdaVyvdoXttn6bx91vLKLrqY343+vKDwcN69uOJduymfzPTZzVoQlD+3x/3rLJ/9zM9CUZ3HtFdwZ2S+ypADw5OOeibte+A/zXeyt5d3EGp7c8mbduP48Bnau+h78y+w4U8disNbz02WZmrdzO+KG9uOKM1t9rU1BYzB2vLKC4xHj+pv40qFPxz9tvrurJivRc7vv7Urq1anTo1tMvNu7mf95fxRVntOIXg47NswMnEr9byTkXNWbG3xZs46H/W0X+wWJ+ccnp3DnodOrWis64/eKt2YybuoxVmblccUYrfn9tL1o3qYeZ8e9/W8LUhelMviWVwT1aHXY/O/cWcM1Tn1K3VhLTx1xAfmEx1zz1KY3r1+bduy6gUSU9k5rmcHcreXJwzkXFxqx9PPDOcj7fuJuzk09hwogz6XJqo6h/T2FxCS9+uonHP1xL7aSTuG9Id0oMfjd9Bfdc1pV7LovsldYLtuxh5KTPOf/0FmTnF7J+x17eHXPBMYk5XnlycM4d1lebvuX5eRvILyyu1vZmsOCbPdStdRK/uaonP07twEkRPBB2NLbs3s8D7yzn0/W7ALi0x6n8+WepR/S9r3yxhf+YthyA527sx5BebarYomY5XHLwaw7OJbCc/EIenrGa17/6hlaN69LxKO7OGdanLWN/0P2wL46Lpk7NT+avtw3gnUXpzFubxfihvY44If30nI58u/8gTerXTrjEUJWIeg6ShgB/BJKAF8zs4TL1nQjNG90S+Ba40cy2BXWPAFcTeqbiQ+DfzMwk9QdeAuoD74eVNwPeBJKBzcANZrbncPF5z8G5I2NmvL9sO//5jxXs3neA2y7szC8v71bpRVxXMx3VE9KSkoBngCsJzfk8SlLZG4QnAlPMrDcwHpgQbHs+cAHQG+gFnA0MDLZ5FvgXoGvwGRKU3w/MNrOuwOxg3TkXJenZ+fz85TTuem0hrRrXZfqYC3ng6hRPDO57Ivl/wwBgvZltBJD0BjAUWBnWJgX4VbA8F5gWLBtQD6gDCKgN7JDUBmhsZl8E+5wCDANmBPseFGz/MvAxcN+RH5pzLlxxifHyZ5uZOGsNZvAfV/fklvOTqZXkL0pw5UWSHNoBW8PWtwHnlGmzBBhBaOhpONBIUnMz+1zSXCCTUHJ42sxWSUoN9hO+z9KnUVqZWWawvB2o8J40SaOB0QAdO3aM4DCcq7l25BbwvzNWs27nvkrbZOcfZOu3+Qzq3pL/GtoroZ/+dVWLVj9yLPC0pFuA+UA6UCypC9ATKJ0p+0NJFwH5kew0uAZR4UURM5sETILQNYejC9+5E1NJifHqV9/wyIzVHCwu4bzTm1f6FHKrxnW594oeXNO7Tdy+SM7Fj0iSQzrQIWy9fVB2iJllEOo5IKkhcJ2ZZUv6F+ALM9sX1M0AzgP+yncJo+w+d0hqY2aZwfDTziM/LOdqvrU79jJu6jIWbNnDBV2a89CwM0lucXKsw3I1RCSDjV8DXSV1llQHGAlMD28gqYWk0n2NI3TnEsA3wEBJtSTVJnQxelUwbJQr6VyF/oT5GfBusM104OZg+eawcuccoddE/GHWGq5+8hM2Zu3jsR+dxSu3neOJwUVVlT0HMyuSNAaYSehW1slmtkLSeCDNzKYTuoA8IRgCmg/cFWz+NjAYWEbo4vQHZvaPoO4XfHcr64zgA/Aw8Jak24AtwA1He5DO1RRfbNzNb6YuY+Ou/Yzo244Hru5J84Z1Yx2Wq4H8CWnnTgA5eYX8z/ureDNtKx2bNeCh4b24qGtivzXUHT1/Qtq5E5SZ8d7STH7/jxXsySvk9oGncc+l3RJ2Ahp3/HhycC5ObduTx2+nLWfumizOat+EKf/vHFLaNo51WC5BeHJwLs4UFZfw0mebeWzWWiR48Icp3Hx+MknH+EV2zoXz5OBcHFmensO4qctYlp7DpT1OZfywXrRrWj/WYbkE5MnB1SgrM3L5lylpFFTz1dOxtifvIM1OrsszP+nHVWe29ofVXMx4cnA1ysMfrGbfgSKuOevEfP1ys5PrctsFnWnSIDFmInPxy5ODqzG+2Lib+Wuz+M1VPRh9sc8B7NzR8NcxuhrBzHjkg9W0blyPn52XHOtwnDvheXJwNcKc1TtZ+E02d1/alXq1/RkA546WDyu548rMuP/vy3hnUfph2910Xid++8Oyc0pVrKTEeHTmGpKbN+BHqe2r3sA5VyVPDu64en7+Rt5M28rQPm1pW8ktmpuy9vPip5s4vWVDfnJO1XN1/GNpBqu37+WPI/tQ2yeucS4qPDm44+bTdbt45IPVXH1mG574cZ9Kb9MsLjFufelrfjd9OT3aNKJfx1Mq3WdhcQl/+HAtPds05prebY9V6M4lHP8zyx0X2/bk8a+vL6TLqQ155Preh71/P+kk8eTIPrRuUo9fvLKQrL0HKm37VtpWtuzO494runGSP0HsXNR4cnDHXEFhMXe8soCiYuP5m1I5uW7VHdamDerw/I2pZOcfZMxrCyksLqlwv0/OXkdqp1O4pPupxyJ05xKWJwd3TJkZ/zFtOcvTc3n8x33ofAQT0qS0bcyEEWfy5aZveXjG6nL1L3+2mR25B7j3iu7+JLFzUebXHNwx9eqX3/D2gm3cfWlXLktpdcTbD+/bniVbc3jx0030bt+EoX3aAZBbUMiz8zYwsFtLzjmtebTDdi7hRdRzkDRE0hpJ6yXdX0F9J0mzJS2V9LGk9kH5JZIWh30KJA0L6j4JK8+QNC0oHyQpJ6zuwWgesDt+FmzZw+//sYJLurfknku7Vns/D1zdkwHJzbjv70tZlZkLwAvzN5KdV8i9V3SPVrjOuTBVJgdJScAzwJVACjBKUtkb0CcCU8ysNzAemABgZnPNrI+Z9SE0XWgeMCuouyis7nNgatj+PimtM7PxR3eILhZ27i3gF68uoG3T+jzx475HdbG4dtJJPP3TvjSpX5s7XlnAxqx9vPDpJq7u3YZe7ZpEMWrnXKlIhpUGAOvNbCOApDeAocDKsDYpwK+C5bnAtAr2cz0ww8zywgslNSaUOG49stDdsbT/QBF5B6v3ZlMzY8xri8jNL+KlWwdE5SVypzaqx59+2p+Rkz7n2qf/yYGiEn51ebej3q9zrmKRJId2wNaw9W3AOWXaLAFGAH8EhgONJDU3s91hbUYCf6hg/8OA2WaWG1Z2nqQlQAYw1sxWlN1I0mhgNEDHjlU/KOUi983uPK54Yj75R/na6z+O7EPPNtGbuax/p1P43TVn8B/TlvPj1A6c3rJh1PbtnPu+aF2QHgs8LekWYD6QDhz6ZZHUBjgTmFnBtqOAF8LWFwKdzGyfpKsI9ULKDVib2SRgEkBqaqpF5zAcwOMfrcUwfn/tGdUeDurUrAEXd2sZ5cjgp+d0JLn5yfTr1DTq+3bOfSeS5JAOdAhbbx+UHWJmGYR6DkhqCFxnZtlhTW4A3jGzwvDtJLUgNGw1PGxfuWHL70v6k6QWZrYrskNyR2P19lymLU5n9MWncfP5ybEOpxxJXNi1RazDcK7Gi+Rupa+BrpI6S6pDaHhoengDSS0kle5rHDC5zD5GAa9XsO/rgffMrCBsX60V3LQuaUAQ4+4KtnXHwGOz1tKwbi3uHOjzITiXyKpMDmZWBIwhNCS0CnjLzFZIGi/p2qDZIGCNpLVAK+Ch0u0lJRPqecyrYPcjKZ80rgeWB9ccngRGmpkPGx0HC7/Zw4crd3D7xafRtEGdWIfjnIsh1YTf3dTUVEtLS4t1GCc0M+Mnf/6SdTv3Mu/eSyJ6xYVz7sQmaYGZpVZU56/PcAB8un4Xn2/czV2XdPHE4Jzz5OBCvYZHZ66hXdP6Ec2f4Jyr+Tw5OGau2M7SbTncc1lX6tbyKTadc54cEl5xiTFx1lpOb3kyw/u2i3U4zrk44ckhwU1duI31O/cx9gfdqeVTbDrnAv5rkMAOFBXzxEfr6N2+CUN6tY51OM65OOLJIYG99uU3pGfn+2Q5zrlyPDkkqP0Hinhm7nrOPa0ZF3bx11E4577Pk0OC+ss/N7Fr30F+PaSH9xqcc+V4ckhA2XkHeX7+Ri7r2Yp+HU+JdTjOuTjkySEBPTtvA/sOFDH2Cp8sxzlXMU8OCWZHbgEvf7aZoWe1pUfr6E3E45yrWTw5JJin5qyjqNj4pU+x6Zw7DE8OCWTL7v288dVWRg7oQKfmJ8c6HOdcHPPkkEAe/3AttZLE3YPLzbrqnHPf48khQazensu7SzK45fzOnNq4XqzDcc7FuYiSg6QhktZIWi/p/grqO0maLWmppI8ltQ/KL5G0OOxTIGlYUPeSpE1hdX2Cckl6MviupZL6RfOAE9XEmaHpP+8YeFqsQ3HOnQCqTA6SkoBngCuBFGCUpJQyzSYCU8ysNzAemABgZnPNrI+Z9QEGA3nArLDt7i2tN7PFQdmVQNfgMxp4ttpH5wBYsGUPH63y6T+dc5GLpOcwAFhvZhvN7CDwBjC0TJsUYE6wPLeCegjNDT3DzPKq+L6hhBKNmdkXQFNJbSKI01UgNJHPalo0rMOtF3SOdTjOuRNEJMmhHbA1bH1bUBZuCTAiWB4ONJLUvEybkcDrZcoeCoaOHpdU9wi+D0mjJaVJSsvKyorgMBLTp+t38cXGbxnj0386545AtC5IjwUGSloEDATSgeLSyuAv/zOBmWHbjAN6AGcDzYD7juQLzWySmaWaWWrLli2PMvyaKXz6z1E+/adz7ghEkhzSgQ5h6+2DskPMLMPMRphZX+CBoCw7rMkNwDtmVhi2TWYwdHQA+Auh4auIvs9F5oPlPv2nc656IkkOXwNdJXWWVIfQ8ND08AaSWkgq3dc4YHKZfYyizJBS6XUEhV4JOgxYHlRNB34W3LV0LpBjZplHcEwOKCouYeKsNXQ5tSEj+rWPdTjOuRNMlYPQZlYkaQyhIaEkYLKZrZA0Hkgzs+nAIGCCJAPmA3eVbi8pmVBPYF6ZXb8qqSUgYDFwR1D+PnAVsJ7Q3U23VvfgarItu/czZ/VOSqzi+k279rEhaz/P3diPpJP8ldzOuSMjs0p+XU4gqamplpaWFuswjovC4hImzd/Ik7PXcaCo5LBtUzudwt/uOM/na3DOVUjSAjNLrajOb185gSz8Zg+/mbqM1dv3ctWZrbl/SE+aNKhdafuGdWt5YnDOVYsnhxPA3oJCJs5cw5QvttC6cT3+/LNULk9pFeuwnHM1mCeHODdrxXYefHcFO/YWcPN5yYy9ojsN/XkF59wx5r8ycWrfgSLu/dsSZizfTo/WjXj2xn709Sk9nXPHiSeHOPXU7HV8sGI7vx7SnX+56DRqJ/kLdJ1zx48nhzi0PaeAlz7bzPC+7fjFoC6xDsc5l4D8z9E49NScdZSY8cvLfCpP51xseHKIM1t27+fNr7cyakBHOjRrEOtwnHMJypNDnPnDh2upnXQSYwb7cJJzLnY8OcSRVZm5TF+Swa0XJHNqI5/K0zkXO54c4shjs9bQqG4tbr/49FiH4pxLcJ4c4sSCLd/y0aqd3DHo9MO+EsM5544HTw5xwMx45IM1tGhYl1vOT451OM4558khHsxft4svN33L3Zd2oUEdf/TEORd7nhxiLDSV52o6NKvPyLN9Kk/nXHzw5BBjM5ZvZ3l6Lr+8rBt1avn/HM65+BDRr5GkIZLWSFov6f4K6jtJmi1pqaSPJbUPyi+RtDjsUyBpWFD3arDP5ZImS6odlA+SlBO2zYPRPOB4UjqVZ7dWDRnap12sw3HOuUOqTA6SkoBngCuBFGCUpJQyzSYCU8ysNzAemABgZnPNrI+Z9QEGE5r2c1awzatAD+BMoD7w87D9fVK6nZmNr/bRxbmpC9PZmLWff/9Bd5/K0zkXVyLpOQwA1pvZRjM7CLwBDC3TJgWYEyzPraAe4HpghpnlAZjZ+xYAvgLaV+cATlTf7M7jiY/W0qdDU37gE/c45+JMJMmhHbA1bH1bUBZuCTAiWB4ONJLUvEybkcDrZXceDCfdBHwQVnyepCWSZkg6o6KgJI2WlCYpLSsrK4LDiA9FxSU8P28DP3hiHrkFRfz2hz19Kk/nXNyJ1n2TY4GnJd0CzAfSgeLSSkltCA0fzaxg2z8B883sk2B9IdDJzPZJugqYBnQtu5GZTQImAaSmplqUjuOYWrI1m3FTl7EyM5fLU1oxfugZtGlSP9ZhOedcOZEkh3SgQ9h6+6DsEDPLIOg5SGoIXGdm2WFNbgDeMbPC8O0k/Q5oCdwetq/csOX3Jf1JUgsz2xXZIcWf/QeKeGzWWl76bBMtG9XluRv7M6RX61iH5ZxzlYokOXwNdJXUmVBSGAn8JLyBpBbAt2ZWAowDJpfZx6igPHybnwNXAJcG25WWtwZ2mJlJGkBo6Gv3ER1VHJmzege/nbaCjJx8bjynE/cO6U7jev56DOdcfKsyOZhZkaQxhIaEkoDJZrZC0nggzcymA4OACZKM0LDSXaXbS0om1POYV2bXzwFbgM+DMfepwZ1J1wN3SioC8oGRwUXrE4qZMW7qMt74eivdWjXk7TvOo3+nZrEOyznnIqIT8He3nNTUVEtLS4t1GN/zybosbnrxK269IJlxV/b0B9ycc3FH0gIzS62ozn+xjoHQKzHW0K5pfe6/socnBufcCcd/tY6BD5ZvZ+m2HO65rCt1ayXFOhznnDtinhyirPSVGF1ObciIfgn1XJ9zrgbx5BBlUxelsyFrP2N/0M1fieGcO2F5coiiA0XF/PGjdfRu34QrzvDnGJxzJy5PDlH02pffkJ6dz71XdPdXYjjnTmieHKJk/4Einp6znvNOa86FXVrEOhznnDsqnhyiZPKnm9i9/yD3DvFeg3PuxOfJIQqy8w4yaf5GLk9pRb+Op8Q6HOecO2qeHKLg2Xkb2HewiLE/6B7rUJxzLio8ORylHbkFvPTPzQzr047urRvFOhznnIsKTw5H6cnZ6yguMX55WbdYh+Kcc1HjyeEobNm9nze/3sqoAR3p2LxBrMNxzrmo8eRQTWbGQ/+3ilpJ4l8Hd4l1OM45F1WeHKrpxU83MWvlDn55WTdObVwv1uE451xUeXKohs827GLCjNVc2as1oy8+LdbhOOdc1EWUHCQNkbRG0npJ91dQ30nSbElLJX0sqX1QfomkxWGfAknDgrrOkr4M9vmmpDpBed1gfX1Qnxy9wz16Gdn5jHltEZ1bnMyjPzrLH3hzztVIVSYHSUnAM8CVQAowSlJKmWYTgSlm1hsYD0wAMLO5ZtbHzPoAg4E8YFawzf8Cj5tZF2APcFtQfhuwJyh/PGgXFwoKi7nzlQUcLCrh+Zv607BuJFNwO+fciSeSnsMAYL2ZbTSzg8AbwNAybVKAOcHy3ArqITQ39Awzy1Poz+3BwNtB3cvAsGB5aLBOUH+p4uTP8/+cvoIl23J47IazOL1lw1iH45xzx0wkyaEdsDVsfVtQFm4JMCJYHg40ktS8TJuRwOvBcnMg28yKKtjnoe8L6nOC9t8jabSkNElpWVlZERzG0Xn9q2944+utjLmki7+O2zlX40XrgvRYYKCkRcBAIB0oLq2U1AY4E5gZpe/DzCaZWaqZpbZs2TJau63Qom/28Lt3V3Bxt5b88nJ/2M05V/NFMmieDnQIW28flB1iZhkEPQdJDYHrzCw7rMkNwDtmVhis7waaSqoV9A7C91n6fdsk1QKaBO1jImvvAe58ZSGtmtTlyZF9fHY351xCiKTn8DXQNbi7qA6h4aHp4Q0ktZBUuq9xwOQy+xjFd0NKmJkRujZxfVB0M/BusDw9WCeonxO0P+4Ki0sY89pC9uQd5Lkb+9O0QZ1YhOGcc8ddlckh+Mt+DKEhoVXAW2a2QtJ4SdcGzQYBayStBVoBD5VuH9yK2gGYV2bX9wG/krSe0DWFF4PyF4HmQfmvgHK3zh4vf/18C19u+paHrzuTM9o2iVUYzjl33EV0L6aZvQ+8X6bswbDlt/nuzqOy226m/AVszGwjoTuhypYXAD+KJK5jbUVGLq0b12N43/axDsU5544rf0L6MLbn5tOmqb8awzmXeDw5HEZmdgFtmnhycM4lHk8OlTAzMnLyadOkfqxDcc65486TQyWy8wopKCzxnoNzLiF5cqhERk4+AG2bes/BOZd4PDlUIjO7AMB7Ds65hOTJoRKZ3nNwziUwTw6VyMwpoNZJokXDurEOxTnnjjtPDpXIzCmgVeN6/i4l51xC8uRQiYzsfNr6A3DOuQTlyaESmTkFtPZnHJxzCcqTQwVKSoztOQW09TuVnHMJypNDBXbvP8jBYn8AzjmXuDw5VKD0NtY2fhurcy5BeXKoQEbwAFxbv+bgnEtQnhwqsP1Qz8GHlZxziSmi5CBpiKQ1ktZLKjczm6ROkmZLWirpY0ntw+o6SpolaZWklcHMcEj6RNLi4JMhaVpQPkhSTljdg2W/71jLzCmgTq2TaH6yTwvqnEtMVc4EJykJeAa4HNgGfC1pupmtDGs2EZhiZi9LGgxMAG4K6qYAD5nZh5IaAiUAZnZR2Hf8ne/mkAb4xMx+eBTHdVQyckLzOEj+AJxzLjFF0nMYAKw3swT/P70AAAyDSURBVI1mdhB4Axhapk0KMCdYnltaLykFqGVmHwKY2T4zywvfUFJjYDAwrdpHEWWZ2fl+p5JzLqFFkhzaAVvD1rdRfk7oJcCIYHk40EhSc6AbkC1pqqRFkh4NeiLhhgGzzSw3rOw8SUskzZB0RsRHEyWZOQU+yY9zLqFF64L0WGCgpEXAQCAdKCY0bHVRUH82cBpwS5ltRwGvh60vBDqZ2VnAU1TSo5A0WlKapLSsrKwoHQYUlxjbc316UOdcYoskOaQDHcLW2wdlh5hZhpmNMLO+wANBWTahXsbiYEiqiNAPfb/S7SS1IDRs9X9h+8o1s33B8vtA7aDd95jZJDNLNbPUli1bRna0Ecjae4DiEvNnHJxzCS2S5PA10FVSZ0l1gJHA9PAGklpIKt3XOGBy2LZNJZX+eg8Gwi9kXw+8Z2YFYftqreBKsKQBQYy7j+ywqu/QDHDec3DOJbAqk0PwF/8YYCawCnjLzFZIGi/p2qDZIGCNpLVAK+ChYNtiQkNKsyUtAwT8OWz3I/n+kBKEEsZySUuAJ4GRZmbVPL4jtj2ndAY47zk45xJXlbeywqHhnffLlD0Ytvw28HYl234I9K6kblAFZU8DT0cS17GQkV06A5z3HJxzicufkC4jM6eA+rWTaFK/dqxDcc65mPHkUEZmTj5tmvoDcM65xObJoYyM7AJ/4Z5zLuF5cigjMyef1n6nknMuwXlyCFNYXMLOvQf8NlbnXMLz5BBmR24BZj7Jj3POeXII890zDt5zcM4lNk8OYTKC5NDWew7OuQTnySFMZvAAnPccnHOJzpNDmMycAhrVrUWjev4AnHMusXlyCJORne/zRjvnHJ4cvscn+XHOuRBPDmEyc3x6UOecA08OhxwoKmbXvoPec3DOOTw5HLIj5wCAX3Nwzjk8ORzy3Qxw3nNwzjlPDoHMIDl4z8E55yJMDpKGSFojab2k+yuo7yRptqSlkj6W1D6srqOkWZJWSVopKTkof0nSJkmLg0+foFySngy+a6mkftE51MPLyA6ejvaeg3POVZ0cJCUBzwBXAinAKEkpZZpNBKaYWW9gPDAhrG4K8KiZ9QQGADvD6u41sz7BZ3FQdiXQNfiMBp498sM6cpk5+TRtUJv6dZKOx9c551xci6TnMABYb2Ybzewg8AYwtEybFGBOsDy3tD5IIrWCeaQxs31mllfF9w0llGjMzL4AmkpqE9nhVF9mtj/j4JxzpSJJDu2ArWHr24KycEuAEcHycKCRpOZANyBb0lRJiyQ9GvRESj0UDB09LqnuEXwfkkZLSpOUlpWVFcFhHF5GToHP4+Ccc4FoXZAeCwyUtAgYCKQDxUAt4KKg/mzgNOCWYJtxQI+gvBlw35F8oZlNMrNUM0tt2bLlUR/Adp8BzjnnDokkOaQDHcLW2wdlh5hZhpmNMLO+wANBWTahv/oXB0NSRcA0oF9QnxkMHR0A/kJo+Cqi74u2/IPF7Mkr9Fd1O+dcIJLk8DXQVVJnSXWAkcD08AaSWkgq3dc4YHLYtk0llf5pPxhYGWzTJvivgGHA8qDNdOBnwV1L5wI5ZpZZraOL0KHbWL3n4JxzQGjY57DMrEjSGGAmkARMNrMVksYDaWY2HRgETJBkwHzgrmDbYkljgdlBElgA/DnY9atB0hCwGLgjKH8fuApYD+QBt0blSA8j89AMcN5zcM45iCA5AJjZ+4R+tMPLHgxbfht4u5JtPwR6V1A+uJL2RpBcjpeMYJKftv4AnHPOAf6ENPBdz8EvSDvnXIgnB0LXHFo0rEPdWv4AnHPOgScHIPTqDL/e4Jxz3/HkAGzPKfA7lZxzLownB0Kv6/bk4Jxz30n45LDvQBF7C4po4w/AOefcIQmfHDKz/QE455wrK+GTQ0ZwG6u/OsM5576T8MnBew7OOVdewieHjJwCJGjV2JODc86VSvjkkJmdz6mN6lI7KeFPhXPOHZLwv4jbc/0BOOecKyvhk0NGdr6/cM8558pI6ORgZmTmFNC6sfccnHMuXEInh9z8IvIOFnvPwTnnykjo5JBxaAY47zk451y4iJKDpCGS1khaL+n+Cuo7SZotaamkjyW1D6vrKGmWpFWSVkpKDspfDfa5XNJkSbWD8kGSciQtDj4Plv2+aDk0Paj3HJxz7nuqTA6SkoBngCuBFGCUpJQyzSYCU8ysNzAemBBWNwV41Mx6AgOAnUH5q0AP4EygPvDzsG0+MbM+wWf8kR9WZBrXq80VZ7SiwykNjtVXOOfcCSmSaUIHAOvNbCOApDeAocDKsDYpwK+C5bnAtKBtClArmCoUM9tXukEw9ShBu6+AQ72N4yU1uRmpyc2O99c651zci2RYqR2wNWx9W1AWbgkwIlgeDjSS1BzoBmRLmippkaRHg57IIcFw0k3AB2HF50laImmGpDMqCkrSaElpktKysrIiOAznnHORitYF6bHAQEmLgIFAOlBMqGdyUVB/NnAacEuZbf8EzDezT4L1hUAnMzsLeIqgF1KWmU0ys1QzS23ZsmWUDsM55xxElhzSgQ5h6+2DskPMLMPMRphZX+CBoCybUC9jsZltNLMiQj/0/Uq3k/Q7oCXfDUlhZrmlw0/B0FNtSS2qc3DOOeeqJ5Lk8DXQVVJnSXWAkcD08AaSWkgq3dc4YHLYtk0llf5pP5jgWoWknwNXAKPMrCRsX60lKVgeEMS4uzoH55xzrnqqTA7BX/xjgJnAKuAtM1shabyka4Nmg4A1ktYCrYCHgm2LCQ0pzZa0DBDw52Cb54K2n5e5ZfV6YLmkJcCTwEgzs6M/VOecc5FSTfjdTU1NtbS0tFiH4ZxzJxRJC8wstaK6hH5C2jnnXMU8OTjnnCunRgwrScoCtlRz8xbAriiGE00eW/XEc2wQ3/F5bNVzosbWycwqfBagRiSHoyEprbIxt1jz2KonnmOD+I7PY6uemhibDys555wrx5ODc865cjw5wKRYB3AYHlv1xHNsEN/xeWzVU+NiS/hrDs4558rznoNzzrlyPDk455wrJ6GTQ1XTn8aSpM2SlgXvnYrpu0GCaVx3SloeVtZM0oeS1gX/PSWOYvtPSelhU81eFaPYOkiaG0yPu0LSvwXlMT93h4kt5udOUj1JXwVzuqyQ9PugvLOkL4N/r28GLwKNl9hekrQp7Lz1Od6xhcWYFMyf816wXr3zZmYJ+QGSgA2E5pioQ2jCopRYxxUW32agRazjCGK5mNCr1peHlT0C3B8s3w/8bxzF9p/A2Dg4b22AfsFyI2AtoVkTY37uDhNbzM8doRd0NgyWawNfAucCbxF6ESeEXtx5ZxzF9hJwfaz/PxfE9SvgNeC9YL1a5y2Rew6Hpj81s4NA6fSnrgwzmw98W6Z4KPBysPwyMOy4BhWoJLa4YGaZZrYwWN5L6K3G7YiDc3eY2GLOQkqnFK4dfIzQK//fDspjdd4qiy0uSGoPXA28EKyLap63RE4OkUx/GksGzJK0QNLoWAdTgVZmlhksbyf0+vV4MkbS0mDYKSZDXuEkJQN9Cf2lGVfnrkxsEAfnLhgaWQzsBD4k1MvPttAUAhDDf69lYzOz0vP2UHDeHpdUNxaxAU8AvwZK58hpTjXPWyInh3h3oZn1A64E7pJ0cawDqoyF+qtx89cT8CxwOtAHyAQei2UwkhoCfwfuMbPc8LpYn7sKYouLc2dmxWbWh9DMkwOAHrGIoyJlY5PUi9AkZz0ITYfcDLjveMcl6YfATjNbEI39JXJyqHL601gys/TgvzuBdwj9A4knOyS1AQj+uzPG8RxiZjuCf8AlhCaXitm5k1Sb0I/vq2Y2NSiOi3NXUWzxdO6CeLKBucB5hGaVrBVUxfzfa1hsQ4JhOjOzA8BfiM15uwC4VtJmQsPkg4E/Us3zlsjJocrpT2NF0smSGpUuAz8Alh9+q+NuOnBzsHwz8G4MY/me0h/ewHBidO6C8d4XgVVm9oewqpifu8pii4dzJ6mlpKbBcn3gckLXROYSmikSYnfeKoptdViyF6Ex/eN+3sxsnJm1N7NkQr9nc8zsp1T3vMX6ynosP8BVhO7S2AA8EOt4wuI6jdDdU0uAFbGODXid0BBDIaExy9sIjWXOBtYBHwHN4ii2vwLLgKWEfojbxCi2CwkNGS0FFgefq+Lh3B0mtpifO6A3sCiIYTnwYFB+GvAVsB74G1A3jmKbE5y35cArBHc0xepDaOrm0ruVqnXe/PUZzjnnyknkYSXnnHOV8OTgnHOuHE8OzjnnyvHk4JxzrhxPDs4558rx5OCcc64cTw7OOefK+f/lmSSjovFjeQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "#ดู accuracy เพิ่มขึ้นเรื่อยๆ\n",
        "plt.plot(L(learn.recorder.values).itemgot(2));"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 96,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-pnmg5QqpokL",
        "outputId": "f6df8472-bbd5-4b94-c0a0-91cc4a363bbb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.981354296207428"
            ]
          },
          "metadata": {},
          "execution_count": 96
        }
      ],
      "source": [
        "learn.recorder.values[-1][2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fQQNpNZ-pokL"
      },
      "source": [
        "## ไปให้ Deep กว่านั้นอีก"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jq3YclVaOjCk"
      },
      "source": [
        "แทนที่เราจะใช้ Architecture ง่ายๆที่เราคิดขึ้นเอง เรามาลองใช้ Architecture ที่ซับซ้อนขึ้นอย่าง `resnet18` กัน"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 97,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        },
        "id": "LB3qg01jpokL",
        "outputId": "d2515ddb-c29c-43f5-9612-f04e927985e5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/fastai/vision/learner.py:265: UserWarning: `cnn_learner` has been renamed to `vision_learner` -- please update your code\n",
            "  warn(\"`cnn_learner` has been renamed to `vision_learner` -- please update your code\")\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* Turns off some styling */\n",
              "    progress {\n",
              "        /* gets rid of default border in Firefox and Opera. */\n",
              "        border: none;\n",
              "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
              "        background-size: auto;\n",
              "    }\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #F44336;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>0.085800</td>\n",
              "      <td>0.008205</td>\n",
              "      <td>0.997547</td>\n",
              "      <td>00:23</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "dls = ImageDataLoaders.from_folder(path)\n",
        "learn = cnn_learner(dls, resnet18, pretrained=False,\n",
        "                    loss_func=F.cross_entropy, metrics=accuracy)\n",
        "\n",
        "#เทรนแค่ epoch เดียวก็ดีกว่าเทรน 20-40 epoch เมื่อกี้แล้ว\n",
        "learn.fit_one_cycle(1, 0.1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uxfYQcZfJEsN"
      },
      "source": [
        "# Checkpoint ท้ายบท"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wB1TBtie2VPG"
      },
      "source": [
        "### ☑️ ดูวิดีโอ [3Blue1Brown](https://www.youtube.com/watch?v=IHZwWFHWa-w) เกี่ยวกับ SGD (มีซับไทย)\n",
        "\n",
        "วิดีโอนี้จะอธิบายการทำงานของ SGD ใน Neural Networks ที่เราเรียนกันในบทที่ 1 และ 3 ด้วยกราฟฟิคที่สวยงาม ช่วยเพิ่มความเข้าใจได้มากยิ่งขึ้น\n",
        "\n",
        "Youtube Link: [https://www.youtube.com/watch?v=IHZwWFHWa-w](https://www.youtube.com/watch?v=IHZwWFHWa-w)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4a0-VMv6Pau1"
      },
      "source": [
        "## คำถามชวนคิดเกี่ยวกับบทเรียน"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t_NIkQBMPgEh"
      },
      "source": [
        "1. เราได้เรียนรู้การทำงานเกือบทุกส่วนของ ML Models ในบทเรียนนี้ คุณคิดว่าสำหรับโครงงานของคุณ การปรับแต่งส่วนไหนของโมเดลจะทำให้ได้ผลดีเพิ่มขึ้นมากที่สุด\n",
        "\n",
        "2. คุณคิดว่าการ initialize weights มีผลต่อคุณภาพของโมเดลที่ถูกเทรนหรือไม่ หากไม่ใช้ weights แบบสุ่ม เราควรใช้ weights อะไรในการเริ่มเทรนโมเดล\n",
        "\n",
        "3. Loss กับ Metric ต่างกันอย่างไร ทำไมคุณคิดว่าเราถึงเทรนโมเดลเพื่อให้ได้ Loss ที่ดีที่สุด แต่กลับวัดผลมันด้วย Metric\n",
        "\n",
        "4. คุณคิดว่า `Activation Function` มีความสำคัญอย่างไรกับประสิทธิภาพของโมเดล ML เราควรเลือกใช้อันไหนในกรณีไหน\n",
        "\n",
        "5. คุณคิดว่าทำไม Architecture อย่าง resnet18 ถึงทำได้ดีกว่า Architecture ที่เราสร้างขึ้นมาเองง่ายๆ ทั้งที่เทรนแค่ 1 epoch เทียบกับที่เราเทรนถึง 20-40 epochs\n",
        "\n",
        "6. คุณคิดว่ามีการ update weights ที่ดีกว่าเอา gradients มาลบไปทีละ iteration อย่างที่เราทำหรือไม่ คิดว่าวิธีไหนบ้าง หรือเป็นไปได้ไหมที่เราจะเทรนโมเดลโดยไม่ใช้ backpropagation กล่าวคือ update weights โดยไม่ใช้ gradients\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HeM7xTrJPc8w"
      },
      "source": [
        "## สิ่งที่ควรเตรียมพร้อมสำหรับทำโครงงาน"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oPvpAnmLJGgU"
      },
      "source": [
        "### ☑️ ทบทวนระบบการให้คะแนนโครงงานให้เรียบร้อย\n",
        "\n",
        "AI Builders จะออกใบประกาศนียบัตรจบการศึกษาให้กับผู้เข้าร่วมโครงการที่ส่งโครงงานได้คะแนนอย่างน้อย 70 จาก 100 คะแนนตามเกณฑ์ต่อไปนี้เท่านั้น\n",
        "\n",
        "1. problem statement; เหตุผลในการแก้ปัญหาเชิงธุรกิจ/ชีวิตประจำวันด้วย machine learning - 15 คะแนน\n",
        "2. metrics and baselines; การให้เหตุผลเชื่อมโยงการแก้ปัญหากับตัวชี้วัดที่เลือก / การวัดผลเทียบกับวิธีแก้ปัญหาในปัจจุบัน - 15 คะแนน\n",
        "3. data collection and cleaning; การเก็บและทำความสะอาดข้อมูล - 15 คะแนน\n",
        "\n",
        "**วันนี้เราจะคิดถึง 2 ข้อนี้เป็นพิเศษ**\n",
        "\n",
        "**4. exploratory data analysis; การทำความเข้าใจข้อมูล - 20 คะแนน**\n",
        "\n",
        "**5. modeling, validation and error analysis; การทำโมเดล, ทดสอบโมเดล และวิเคราะห์ข้อผิดพลาดของโมเดล - 20 คะแนน**\n",
        "\n",
        "6. deployment; การนำโมเดลไปใช้แก้ปัญหาจริง - 15 คะแนน"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9u8rUq1uPSuE"
      },
      "source": [
        "### ☑️ เข้าใจชุดข้อมูล\n",
        "\n",
        "คุณรู้เกี่ยวกับ labels ที่คุณกำลังทำนายแค่ไหน \n",
        "* ถ้าเป็น classification มันมี labels กี่ประเภท (class) แต่ละตัวอย่างเป็นได้มากกว่าหนึ่งประเภท (multi-label) หรือได้แค่ประเภทเดียว (multi-class) \n",
        "* ถ้าเป็น regression แล้ว labels ของคุณกระจายตัวยังไง\n",
        "* คุณมีตัวอย่างพอในแต่ละ class ให้โมเดลเรียนรู้หรือเปล่า\n",
        "\n",
        "features ของคุณหน้าตาเป็นอย่างไร\n",
        "\n",
        "* ถ้าเป็นรูปภาพ เป็นรูปจากมุมมองเดียวหรือหลายมุมมอง ขนาดรูป-ความละเอียดเท่ากันไหม\n",
        "* ถ้าเป็นข้อความ การกระจายตัวของคำเป็นยังไง ทั้ง word count และ tfidf; คำที่เกิดบ่อยๆในแต่ละ labels คืออะไร\n",
        "* ถ้าเป็นข้อมูลตาราง การกระจายตัวของ feature ที่เป็นตัวเลข (numerical) และประเภท (categorical) เป็นอย่างไรบ้าง ลองหา correlation ระหว่าง features ด้วยกัน และ feature กับ labels ดูว่ามีอะไรน่าสนใจไหม\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xcPUWlA9JGio"
      },
      "source": [
        "### ☑️ เทรนโมเดลแรกให้จบจนเห็นผลลัพธ์เร็วที่สุดเท่าที่จะทำได้\n",
        "\n",
        "แน่นอนว่าการทำความสะอาดข้อมูลและการทำความเข้าใจข้อมูลเป็นสิ่งสำคัญ (คะแนนรวม 15+20=35) แต่วิธีที่ดีที่สุดที่จะรู้ว่าโมเดลของเราทำได้ดีแค่ไหนคือการลองเทรนไปเลยทั้งๆที่ไม่ต้องทำอะไรกับข้อมูลมาก หลังจากนั้นเราอาจจะมาไล่ดูข้อผิดพลาดของโมเดลเพื่อให้มีไอเดียในการจัดการข้อมูลและโมเดลต่อไปได้ \n",
        "\n",
        "คุณอาจจะคิดว่าการสร้าง ML model มีขั้นตอนแบบนี้:\n",
        "\n",
        "<img src=https://github.com/ai-builders/curriculum/raw/main/images/ml_what_you_think_it_is.png width=\"500px\">\n",
        "\n",
        "แต่จริงๆแล้วมันคือแบบนี้:\n",
        "\n",
        "<img src=https://github.com/ai-builders/curriculum/raw/main/images/ml_what_it_really_is.png width=\"500px\">\n",
        "\n",
        "เพราะฉะนั้นอย่ากลัวที่จะลองทำให้ครบขั้นตอนก่อนแล้วจึงย้อนกลับไปกลับมาเพื่อทำใหม่ให้ดียิ่งขึ้น"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "03_sgd_from_scratch.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "jupytext": {
      "split_at_heading": true
    },
    "kernelspec": {
      "display_name": "conda_amazonei_pytorch_latest_p36",
      "language": "python",
      "name": "conda_amazonei_pytorch_latest_p36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}