T5_Model_Demo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "T5 Model Demo",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n-wFJmDJ1jzs",
        "colab_type": "text"
      },
      "source": [
        "# T5 Model Demo\n",
        "Author: Prakhar Mishra\n",
        "\n",
        "Connect me on [LinkedIn](https://www.linkedin.com/in/prakhar21) \\\\\n",
        "See more of my work at [GitHub](https://www.github.com/prakhar21) \\\\\n",
        "Read by [Blogs](https://www.prakhartechviz.blogspot.com)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yfcFfjXUj_kC",
        "colab_type": "code",
        "outputId": "a5a67f7a-a37a-4dd1-b3ca-589c0332bea8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 416
        }
      },
      "source": [
        "!pip install transformers --upgrade\n",
        "#transformers >= 2.8.0"
      ],
      "execution_count": 115,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already up-to-date: transformers in /usr/local/lib/python3.6/dist-packages (2.8.0)\n",
            "Requirement already satisfied, skipping upgrade: sacremoses in /usr/local/lib/python3.6/dist-packages (from transformers) (0.0.41)\n",
            "Requirement already satisfied, skipping upgrade: boto3 in /usr/local/lib/python3.6/dist-packages (from transformers) (1.12.40)\n",
            "Requirement already satisfied, skipping upgrade: tqdm>=4.27 in /usr/local/lib/python3.6/dist-packages (from transformers) (4.38.0)\n",
            "Requirement already satisfied, skipping upgrade: filelock in /usr/local/lib/python3.6/dist-packages (from transformers) (3.0.12)\n",
            "Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.6/dist-packages (from transformers) (2.21.0)\n",
            "Requirement already satisfied, skipping upgrade: regex!=2019.12.17 in /usr/local/lib/python3.6/dist-packages (from transformers) (2019.12.20)\n",
            "Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from transformers) (1.18.2)\n",
            "Requirement already satisfied, skipping upgrade: sentencepiece in /usr/local/lib/python3.6/dist-packages (from transformers) (0.1.85)\n",
            "Requirement already satisfied, skipping upgrade: dataclasses; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from transformers) (0.7)\n",
            "Requirement already satisfied, skipping upgrade: tokenizers==0.5.2 in /usr/local/lib/python3.6/dist-packages (from transformers) (0.5.2)\n",
            "Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers) (1.12.0)\n",
            "Requirement already satisfied, skipping upgrade: joblib in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers) (0.14.1)\n",
            "Requirement already satisfied, skipping upgrade: click in /usr/local/lib/python3.6/dist-packages (from sacremoses->transformers) (7.1.1)\n",
            "Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.6/dist-packages (from boto3->transformers) (0.9.5)\n",
            "Requirement already satisfied, skipping upgrade: botocore<1.16.0,>=1.15.40 in /usr/local/lib/python3.6/dist-packages (from boto3->transformers) (1.15.40)\n",
            "Requirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in /usr/local/lib/python3.6/dist-packages (from boto3->transformers) (0.3.3)\n",
            "Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (1.24.3)\n",
            "Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (2.8)\n",
            "Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->transformers) (2020.4.5.1)\n",
            "Requirement already satisfied, skipping upgrade: docutils<0.16,>=0.10 in /usr/local/lib/python3.6/dist-packages (from botocore<1.16.0,>=1.15.40->boto3->transformers) (0.15.2)\n",
            "Requirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.6/dist-packages (from botocore<1.16.0,>=1.15.40->boto3->transformers) (2.8.1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4JoyxwyzskuP",
        "colab_type": "text"
      },
      "source": [
        "# Using Main Class"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HeYHsEhuCkqE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from transformers import T5Tokenizer, T5ForConditionalGeneration\n",
        "import time\n",
        "\n",
        "model = T5ForConditionalGeneration.from_pretrained('t5-small')  #tiny-base\n",
        "tokenizer = T5Tokenizer.from_pretrained('t5-small')  #tiny-base\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "def summarization_infer(text, max=50):\n",
        "  preprocess_text = text.replace(\"\\n\", \" \").strip()\n",
        "  t5_prepared_Text = \"summarize: \"+preprocess_text\n",
        "  tokenized_text = tokenizer.encode(t5_prepared_Text, return_tensors=\"pt\")\n",
        "\n",
        "  summary_ids = model.generate(tokenized_text, min_length=30, max_length=max, top_k=100, top_p=0.8) #top-k top-p sampling strategy\n",
        "  output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)\n",
        "  end_time = time.time()\n",
        "  print (f'Time taken : {end_time-start_time}')\n",
        "  return output\n",
        "\n",
        "def translation_infer(text, max=50):\n",
        "  preprocess_text = text.replace(\"\\n\", \" \").strip()\n",
        "  t5_prepared_Text = \"translate English to German: \"+preprocess_text\n",
        "  tokenized_text = tokenizer.encode(t5_prepared_Text, return_tensors=\"pt\")\n",
        "\n",
        "  translation_ids = model.generate(tokenized_text, min_length=10, max_length=50, early_stopping=True, num_beams=2)\n",
        "  output = tokenizer.decode(translation_ids[0], skip_special_tokens=True)\n",
        "  end_time = time.time()\n",
        "  print (f'Time taken : {end_time-start_time}')\n",
        "  return output\n",
        "\n",
        "def grammatical_acceptibility_infer(text):\n",
        "  preprocess_text = text.replace(\"\\n\", \" \").strip()\n",
        "  t5_prepared_Text = \"cola sentence: \"+preprocess_text\n",
        "  tokenized_text = tokenizer.encode(t5_prepared_Text, return_tensors=\"pt\")\n",
        "\n",
        "  grammar_ids = model.generate(tokenized_text, min_length=1, max_length=3)\n",
        "  output = tokenizer.decode(grammar_ids[0], skip_special_tokens=True)\n",
        "  end_time = time.time()\n",
        "  print (f'Time taken : {end_time-start_time}')\n",
        "  return output"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NE8F2GoWsuZ3",
        "colab_type": "text"
      },
      "source": [
        "# Using PyTorch Pipelines\n",
        "Newly introduced in transformers v2.3.0, pipelines provides a high-level, easy to use, API for doing inference over a variety of downstream-tasks. Read [more practical documentation](https://www.kaggle.com/funtowiczmo/hugging-face-transformers-how-to-use-pipelines)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "arzvSJ87kWsG",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "17e70727-8131-47c7-b25e-ac7ea9412564"
      },
      "source": [
        "from transformers import pipeline\n",
        "\n",
        "summarization_pipeline = pipeline(task='summarization', model=\"t5-small\") \n",
        "output = summarization_pipeline(text, min_length=30, max_length=50, top_k=100, top_p=0.8)\n",
        "print (output)"
      ],
      "execution_count": 147,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[{'summary_text': \"india's novel coronavirus count has increased to 21,700, according to the latest ministry of health and family welfare data . the death toll due to the virus is 686 after 34 more patients succumbed to the highly\"}]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ksJD3i3w6Ee4",
        "colab_type": "text"
      },
      "source": [
        "# Making Flask API"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HOkFL8pp6GCB",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        },
        "outputId": "2160c980-1b92-4314-ee18-e968cb5fcc9c"
      },
      "source": [
        "from flask import Flask, request\n",
        "\n",
        "app = Flask(__name__)\n",
        "\n",
        "@app.route('/infer', methods=['POST'])\n",
        "def infer():\n",
        "  args = request.args['task']\n",
        "  text = request.args['text']\n",
        "  if args=='summarize':\n",
        "    return summarization_infer(text)\n",
        "  elif args=='translation':\n",
        "    return translation_infer(text)\n",
        "  else:\n",
        "    return grammatical_acceptibility_infer(text)\n",
        "\n",
        "if __name__=='__main__':\n",
        "  app.run(host='0.0.0.0', port=5555, debug=False, threaded=True)\n",
        "    "
      ],
      "execution_count": 150,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " * Serving Flask app \"__main__\" (lazy loading)\n",
            " * Environment: production\n",
            "\u001b[31m   WARNING: This is a development server. Do not use it in a production deployment.\u001b[0m\n",
            "\u001b[2m   Use a production WSGI server instead.\u001b[0m\n",
            " * Debug mode: off\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            " * Running on http://0.0.0.0:5555/ (Press CTRL+C to quit)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w5OWx66m8dyY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "text =\"\"\"\n",
        "With 1,229 fresh cases in the last 24 hours, India's novel coronavirus count has increased to 21,700, according to the latest Ministry of Health and Family Welfare data. Along with that, the death toll due to the virus has increased to 686 after 34 more patients succumbed to the highly contagious disease since yesterday, it said. So far, India has 16,689 active cases. There are also 77 foreign nationals who are affected by the virus, the ministry said. Apart from that, there are at least 4,324 patients who have been discharged or cured from the highly contagious disease and one has migrated from the country. Speaking at the press briefing today, Lav Agarwal, Joint Secretary, Health Ministry, said, \"As on today, we have 12 districts that did not have a fresh case in the last 28 days or more. There are now 78 districts (23 States/UTs) that has not reported any fresh cases during the last 14 Days.\" However, he also said that the increase in the number of coronavirus cases in the country is \"more or less linear, not exponential.\" According to the Thursday morning data of health ministry, 4,257 Covid-19 patients have been cured so far, bring the recovery rate to is 19.89% as of now, said Aggrawal. He also added, \"We have been able to cut virus transmission, minimise spread of COVID-19 in 30 days of lockdown.\"\n",
        "\"\"\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YOQ-SwN08w3n",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        },
        "outputId": "83d5fc39-7553-427f-db8f-3fdf048f0b78"
      },
      "source": [
        "summarization_infer(text)"
      ],
      "execution_count": 161,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Time taken : 158.3986692428589\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"india's novel coronavirus count has increased to 21,700, according to the latest ministry of health and family welfare data. the death toll due to the virus has increased to 686 after 34 more patients succumbed to\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OMZsSvnT9sT5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}