ACY_Workshop (Alphabets).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ezSK-7-ftRX4"
   },
   "source": [
    "## ACY Deep Learning Training Ground #2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wEzTwxcWtWfy"
   },
   "source": [
    "Welcome to Deep Learning Training Ground! Here we are going to train a simple Deep Neural Network which can recognize Myanmar Handwritten Alphabets. First, make sure that your Colab Runtime is using GPU. In order to do so:\n",
    "\n",
    "\n",
    "* Go to Edit > Notebook Settings\n",
    "* Select **GPU** in Hardware Accelerator dropdown\n",
    "* Click on \"Save\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1O77mmR6tudr"
   },
   "source": [
    "### Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RmkLU1nktNBT"
   },
   "source": [
    "Install necessary libraries.\n",
    "\n",
    "\n",
    "*   **Augmentor** is a data agumentation library. As a matter of fact, we don't need to perform prelimenary data augmentation since we can implement data augmentation when the data batches are loaded for training. The purpose of having a separate data augmentation step is for demonstrative purpose.\n",
    "*   **fastai** is the deep learning wrapper library which makes training neural networks extremely easy. **fastai** is a good place to start getting familiar with Deep Learning.\n",
    "* **torch** == `PyTorch` is a deep learning framework by Facebook. It is one of the most popular deep learning frameworks out there. Tensorflow is another extremely popular choice. **fastai** uses `PyTorch` as a backend therefore, we have to install `PyTorch`.\n",
    "\n",
    "* Additionally, we also install `unzip` software to unzip files on Linux.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hZLZWG71tJiW"
   },
   "outputs": [],
   "source": [
    "%%sh\n",
    "\n",
    "pip install -q Augmentor\n",
    "pip install -q fastai\n",
    "pip install -q \"torch==1.4\" \"torchvision==0.5.0\"\n",
    "sudo apt-get -q install unzip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jXPKiQC7tzEZ"
   },
   "source": [
    "We have Myanmar Handwritten Alphabets by drawing them in MSPaint with different brushes. First, just download that zip and check out the images in there. We are going to train a neural network which can recognize Myanmar Hand written Alphabets from this dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aLnm_ZbC3HQc"
   },
   "source": [
    "Download the zip file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://github.com/ArkAung/dl_acy_workshop/raw/master/datasets/alphabets/alphabets.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RqVofcS-NXhp"
   },
   "outputs": [],
   "source": [
    "!unzip -qq alphabets.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZtJPqliUw0eM"
   },
   "source": [
    "## Augment Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "V2uZWUs4_bX3"
   },
   "source": [
    "As you are aware of the fact that there only are a few images for each alphabet, and deep learning is such a data hungry technique, we are going to **augment** the data. (Data augmentation = data ကိုတိုးပွားလာအောင်လုပ်သည်)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LX6cMvl2_zoK"
   },
   "source": [
    "Using `Augmentor` library, we are going to create different variants of our original data by applying transformations such as:\n",
    "\n",
    "\n",
    "* Random rotations\n",
    "* Random distortions\n",
    "* Random shearing\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/mdbloice/AugmentorFiles/master/UsageGuide/rotate_aug_b.png\" alt=\"Rotation Example\" width=\"100\"/>\n",
    "<img src=\"https://cloud.githubusercontent.com/assets/16042756/23697283/802698a6-03e7-11e7-94b7-f0b61977ef33.gif\" alt=\"Random Distortion\" width=\"100\"/>\n",
    "<img src=\"https://raw.githubusercontent.com/mdbloice/AugmentorFiles/master/UsageGuide/shear_x_aug.png\" alt=\"Shearing along x-axis\" width=\"100\"/>\n",
    "<img src=\"https://raw.githubusercontent.com/mdbloice/AugmentorFiles/master/UsageGuide/shear_y_aug.png\" alt=\"Shearing along y-axis\" width=\"100\"/>\n",
    "\n",
    "Ref: [Augmentor Repo](https://github.com/mdbloice/Augmentor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A3yxYdYKr8p1"
   },
   "outputs": [],
   "source": [
    "import Augmentor\n",
    "p = Augmentor.Pipeline(\"alphabets\") # Since our images are in \"alphabets\" directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "da2rHRKesY19"
   },
   "outputs": [],
   "source": [
    "p.resize(probability=1, width=100, height=100)\n",
    "p.rotate(probability=0.7, max_left_rotation=10, max_right_rotation=10)\n",
    "p.random_distortion(probability=0.9, grid_width=3, grid_height=3, magnitude=8)\n",
    "p.shear(probability=0.3, max_shear_left=5, max_shear_right=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dU0kNFrisb6m"
   },
   "outputs": [],
   "source": [
    "p.process()\n",
    "p.sample(33000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "toGh5nVBAK_u"
   },
   "source": [
    "Augmented images are saved in a folder `output`. We are going to move this to a directory called `train`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3hpqqmCLVecD"
   },
   "outputs": [],
   "source": [
    "!mv alphabets/output train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aD7u5Kub9fq3"
   },
   "source": [
    "## Prepare data for training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ysQdmABIUD7-"
   },
   "outputs": [],
   "source": [
    "from fastai.vision import * "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TW7EoJWABfHO"
   },
   "source": [
    "We have folders containing image files of each alphabet in a parent folder called \"train\". In oroder to train a neural network, we have to prepare those images in a format which the neural network can consume."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jHXII64PUPJg"
   },
   "outputs": [],
   "source": [
    "path=\"train\"\n",
    "np.random.seed(42)\n",
    "data = ImageDataBunch.from_folder(path, train=path,valid_pct=0.2,\n",
    "                                  ds_tfms=get_transforms(do_flip=False, max_rotate=2.0,\n",
    "                                                         max_zoom=1, max_warp=0.0,\n",
    "                                                         p_affine=0.0),\n",
    "                                  size=100, num_workers=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sahi7RrB9k_2"
   },
   "source": [
    "## Visualize Training Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9gkcUQqnbEtA"
   },
   "outputs": [],
   "source": [
    "data.show_batch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gLu3ED_T9rdD"
   },
   "source": [
    "## Create Deep Neural Network to Train on Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Wi652pcjCDf3"
   },
   "source": [
    "We are going to use a neural network architecture called [AlexNet](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf).\n",
    "We are going to measure how well our neural network is doing by two metrics, Error Rate and Accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-zMno3-BV7QW"
   },
   "outputs": [],
   "source": [
    "from fastai.metrics import error_rate\n",
    "learn = cnn_learner(data, models.alexnet, metrics=[error_rate, accuracy],\n",
    "                    model_dir=\"/content/model/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qpJTnVq29vb0"
   },
   "source": [
    "## Start Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MPJcfVblCaTE"
   },
   "source": [
    "Since we are using Google CoLab with GPU, we can specify 'cuda' here. If you only have a CPU available (for example, on your machine), you would have to specify cpu here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Z2xPv19JDWVE"
   },
   "source": [
    "Then, start training the neural network for 4 epochs using [Cyclical Learning Rates](https://arxiv.org/pdf/1506.01186.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OxnteXEXYLr-"
   },
   "outputs": [],
   "source": [
    "defaults.device = torch.device('cuda')\n",
    "learn.fit_one_cycle(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "snNRv_Z69yjD"
   },
   "source": [
    "## Understanding the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "V1TP62Seaxqp"
   },
   "outputs": [],
   "source": [
    "interp = ClassificationInterpretation.from_learner(learn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wBJOxeUxa8G0"
   },
   "outputs": [],
   "source": [
    "interp.plot_top_losses(9, figsize=(6,6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WR6ri06Abmpc"
   },
   "outputs": [],
   "source": [
    "interp.plot_confusion_matrix()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "u_z5RqRh96S_"
   },
   "source": [
    "## Export the Trained Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-r2aQIl4cuPm"
   },
   "outputs": [],
   "source": [
    "learn.export()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9w3KGkt0xoq9"
   },
   "source": [
    "## Loading saved model to make prediction on a single image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1yyUM5HqndFQ"
   },
   "outputs": [],
   "source": [
    "from fastai.vision import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UWdU9wFHh-KF"
   },
   "outputs": [],
   "source": [
    "learn = load_learner('train', 'export.pkl')\n",
    "im = open_image('/content/alphabets/ခ/Annotation 2020-06-27 205404.png')\n",
    "preds_class, preds_idx, preds_output = learn.predict(im)\n",
    "class_idx = preds_class.data.item()\n",
    "mapping = {v:k for k,v in learn.data.c2i.items()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wxxu4Tjaq9kd"
   },
   "outputs": [],
   "source": [
    "im.show()\n",
    "print(\"Result is: {}\".format(mapping[class_idx]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Jc-NicBCwqlN"
   },
   "source": [
    "## Setting up a webserver to predict hand written alphabets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wT8FJmdr3RiE"
   },
   "source": [
    "Install necessary libraries for Flask server"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A1PtN41arGgk"
   },
   "outputs": [],
   "source": [
    "%%sh\n",
    "\n",
    "pip install -q flask-ngrok\n",
    "pip install -q flask==0.12.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4oZc8LaUyY1H"
   },
   "source": [
    "Prepare source for webserver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "B5Fn8G0xX-Oq"
   },
   "outputs": [],
   "source": [
    "%%sh\n",
    "git clone https://github.com/ArkAung/dl_acy_workshop.git\n",
    "mv dl_acy_workshop/webserver.py .\n",
    "mv dl_acy_workshop/static .\n",
    "mv dl_acy_workshop/templates ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XlWNPvwMYmAA"
   },
   "source": [
    "## Run the Webserver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7QBQTDJcYH8R"
   },
   "outputs": [],
   "source": [
    "!python webserver.py --type \"alphabets\""
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "ACY Workshop.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}