KPSo_et_al_2021_OpenCV_Notebook.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "KPSo_et_al_2021-OpenCV_Notebook.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mstapelberg/KPSo_et_al_2021MRL/blob/main/KPSo_et_al_2021_OpenCV_Notebook.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FKlDyEz2QuAI"
      },
      "source": [
        "# OpenCV Super Resolution Implementations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6qZz9gLKQ51U"
      },
      "source": [
        "Two standard algorithms (EDSR and FSRCNN) for implementing super resolution using deep neural networks were used in our work. \n",
        "\n",
        "The following Goolge Colab Notebook contains a reproduceable coding software that can be run on any computer with an internet browser.\n",
        "\n",
        "Further information on the implementation of Super Resolution using the python interfaced OpenCV package can be found here in the OpenCV documentation - https://docs.opencv.org/master/d5/d29/tutorial_dnn_superres_upscale_image_single.html \n",
        "\n",
        "This notebook will output two upscaled images based on an image taken from TEM analysis discussed in the body of our published work. These images are included in Figure **WIP - ADD FIGURE NUMBER ONCE PUBLISHED**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7EYVHGhWHR6P"
      },
      "source": [
        "# Load in Needed Libraries, Images, and Models"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gUhutQhXgAGF",
        "outputId": "f5114654-d075-4873-91ec-f3f0d59ff947"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "92ZQA8Y9Ezkw",
        "outputId": "ac5c5186-4707-4040-8019-ffee713ba899"
      },
      "source": [
        "# install default opencv in collab and update to latest opencv version\n",
        "!pip uninstall --yes opencv-python\n",
        "!pip uninstall --yes opencv-contrib-python\n",
        "!pip3 install opencv-contrib-python"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[33mWARNING: Skipping opencv-python as it is not installed.\u001b[0m\n",
            "Uninstalling opencv-contrib-python-4.5.2.54:\n",
            "  Successfully uninstalled opencv-contrib-python-4.5.2.54\n",
            "Collecting opencv-contrib-python\n",
            "  Using cached https://files.pythonhosted.org/packages/27/f5/21227eb87cd5990a0f4235d8264db7d8f62f6a9ccbf2cd2c367f7d903cd2/opencv_contrib_python-4.5.2.54-cp37-cp37m-manylinux2014_x86_64.whl\n",
            "Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/dist-packages (from opencv-contrib-python) (1.19.5)\n",
            "Installing collected packages: opencv-contrib-python\n",
            "Successfully installed opencv-contrib-python-4.5.2.54\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s81Udvz8D1PU",
        "outputId": "8be547ff-47d3-4394-e46c-4fa1daf43c0a"
      },
      "source": [
        "#import libraries needed and checks that OpenCV >= 4.3 to use SR Modules\n",
        "import cv2\n",
        "import matplotlib.pyplot as plt\n",
        "print(cv2.__version__)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4.5.2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kfyK_cN8Gow7"
      },
      "source": [
        "#Download the images used in the paper\n",
        "\n",
        "#Input the name of the picture and load inn image\n",
        "image_path = \"/content/drive/MyDrive/SR_Models/1.png\"\n",
        "image_name = cv2.imread(image_path)\n",
        "\n",
        "#Download the EDSR and FSRCNN models\n",
        "# do once GITHUB repository set up\n",
        "\n",
        "#set the paths for the models\n",
        "edsr_path = \"/content/drive/MyDrive/SR_Models/EDSR_x4.pb\"\n",
        "fsrcnn_path = \"/content/drive/MyDrive/SR_Models/FSRCNN_x2.pb\""
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LCD9gV4XHH6u"
      },
      "source": [
        "# *EDSR Model Application*\n",
        "Based on the model from the following paper - https://arxiv.org/pdf/1707.02921.pdf \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lb81lfc6EVQc",
        "outputId": "61acb4ba-7df8-407c-abca-6d3a45c41f51"
      },
      "source": [
        "#Creates the modules object and reads in the model\n",
        "sr_edsr = cv2.dnn_superres.DnnSuperResImpl_create()\n",
        "sr_edsr.readModel(edsr_path)\n",
        "\n",
        "#Set the model to EDSR and use a 4x upscaling \n",
        "sr_edsr.setModel(\"edsr\",4)\n",
        "\n",
        "#Upscale the Image\n",
        "edsr_up_image = sr_edsr.upsample(image_name)\n",
        "\n",
        "#Save the image for viewing later\n",
        "cv2.imwrite(\"edsr_up_image.png\",edsr_up_image)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EnyDlwk5IF-6"
      },
      "source": [
        "# FSRCNN Model Application \n",
        "Based on the Model from the following paper - https://arxiv.org/abs/1608.00367 "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tS_3nqbJIFC5",
        "outputId": "c24a7649-f3cf-4220-f510-65cd0b37aa56"
      },
      "source": [
        "#Creates the modules object and reads in the model\n",
        "sr_fsrcnn = cv2.dnn_superres.DnnSuperResImpl_create()\n",
        "sr_fsrcnn.readModel(fsrcnn_path)\n",
        "\n",
        "#Set the model to FSRCNN and use a 2x upscaling \n",
        "sr_fsrcnn.setModel(\"fsrcnn\",2)\n",
        "\n",
        "#Upscale the Image\n",
        "fsrcnn_up_image = sr_fsrcnn.upsample(image_name)\n",
        "\n",
        "#Save the image for viewing later\n",
        "cv2.imwrite(\"fsrcnn_up_image.png\",fsrcnn_up_image)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    }
  ]
}