Hw4_part_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Hw4 part 2.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOtBkrZ2qa6uDJlst1kcqrI",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/lnpetrova/comp_ling/blob/master/Hw4_part_2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ii1DtbNgfucC",
        "colab_type": "code",
        "outputId": "06a176e0-8b6d-4df0-9940-36a3583b24eb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 793
        }
      },
      "source": [
        "!pip install deeppavlov"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: deeppavlov in /usr/local/lib/python3.6/dist-packages (0.7.1)\n",
            "Requirement already satisfied: scikit-learn==0.21.2 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.21.2)\n",
            "Requirement already satisfied: rusenttokenize==0.0.5 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.0.5)\n",
            "Requirement already satisfied: uvicorn==0.9.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.9.0)\n",
            "Requirement already satisfied: pymorphy2==0.8 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.8)\n",
            "Requirement already satisfied: nltk==3.2.5 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (3.2.5)\n",
            "Requirement already satisfied: fuzzywuzzy==0.17.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.17.0)\n",
            "Requirement already satisfied: overrides==1.9 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (1.9)\n",
            "Requirement already satisfied: pymorphy2-dicts-ru in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (2.4.404381.4453942)\n",
            "Requirement already satisfied: numpy==1.16.4 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (1.16.4)\n",
            "Requirement already satisfied: pytelegrambotapi==3.6.6 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (3.6.6)\n",
            "Requirement already satisfied: tqdm==4.32.2 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (4.32.2)\n",
            "Requirement already satisfied: fastapi==0.38.1 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.38.1)\n",
            "Requirement already satisfied: h5py==2.9.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (2.9.0)\n",
            "Requirement already satisfied: Cython==0.29.12 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.29.12)\n",
            "Requirement already satisfied: pandas==0.24.2 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (0.24.2)\n",
            "Requirement already satisfied: requests==2.22.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (2.22.0)\n",
            "Requirement already satisfied: pyopenssl==19.0.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (19.0.0)\n",
            "Requirement already satisfied: scipy==1.3.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (1.3.0)\n",
            "Requirement already satisfied: keras==2.2.4 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (2.2.4)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn==0.21.2->deeppavlov) (0.14.1)\n",
            "Requirement already satisfied: websockets==8.* in /usr/local/lib/python3.6/dist-packages (from uvicorn==0.9.0->deeppavlov) (8.1)\n",
            "Requirement already satisfied: uvloop==0.*; sys_platform != \"win32\" and sys_platform != \"cygwin\" and platform_python_implementation != \"pypy\" in /usr/local/lib/python3.6/dist-packages (from uvicorn==0.9.0->deeppavlov) (0.14.0)\n",
            "Requirement already satisfied: h11==0.8.* in /usr/local/lib/python3.6/dist-packages (from uvicorn==0.9.0->deeppavlov) (0.8.1)\n",
            "Requirement already satisfied: httptools==0.0.13; sys_platform != \"win32\" and sys_platform != \"cygwin\" and platform_python_implementation != \"pypy\" in /usr/local/lib/python3.6/dist-packages (from uvicorn==0.9.0->deeppavlov) (0.0.13)\n",
            "Requirement already satisfied: click==7.* in /usr/local/lib/python3.6/dist-packages (from uvicorn==0.9.0->deeppavlov) (7.0)\n",
            "Requirement already satisfied: dawg-python>=0.7 in /usr/local/lib/python3.6/dist-packages (from pymorphy2==0.8->deeppavlov) (0.7.2)\n",
            "Requirement already satisfied: pymorphy2-dicts<3.0,>=2.4 in /usr/local/lib/python3.6/dist-packages (from pymorphy2==0.8->deeppavlov) (2.4.393442.3710985)\n",
            "Requirement already satisfied: docopt>=0.6 in /usr/local/lib/python3.6/dist-packages (from pymorphy2==0.8->deeppavlov) (0.6.2)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from nltk==3.2.5->deeppavlov) (1.12.0)\n",
            "Requirement already satisfied: pydantic<=0.32.2,>=0.32.2 in /usr/local/lib/python3.6/dist-packages (from fastapi==0.38.1->deeppavlov) (0.32.2)\n",
            "Requirement already satisfied: starlette<=0.12.8,>=0.11.1 in /usr/local/lib/python3.6/dist-packages (from fastapi==0.38.1->deeppavlov) (0.12.8)\n",
            "Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas==0.24.2->deeppavlov) (2018.9)\n",
            "Requirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python3.6/dist-packages (from pandas==0.24.2->deeppavlov) (2.6.1)\n",
            "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (2.8)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (2019.11.28)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (1.24.3)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (3.0.4)\n",
            "Requirement already satisfied: cryptography>=2.3 in /usr/local/lib/python3.6/dist-packages (from pyopenssl==19.0.0->deeppavlov) (2.8)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from keras==2.2.4->deeppavlov) (3.13)\n",
            "Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from keras==2.2.4->deeppavlov) (1.0.8)\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from keras==2.2.4->deeppavlov) (1.1.0)\n",
            "Requirement already satisfied: dataclasses>=0.6; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from pydantic<=0.32.2,>=0.32.2->fastapi==0.38.1->deeppavlov) (0.7)\n",
            "Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.6/dist-packages (from cryptography>=2.3->pyopenssl==19.0.0->deeppavlov) (1.13.2)\n",
            "Requirement already satisfied: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.3->pyopenssl==19.0.0->deeppavlov) (2.19)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "meEQr7n7f-Sx",
        "colab_type": "code",
        "outputId": "5e116b08-577b-4be2-d2f9-dd1ddc0f1d6d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 370
        }
      },
      "source": [
        "!pip install tensorflow==1.15.0"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: tensorflow==1.15.0 in /usr/local/lib/python3.6/dist-packages (1.15.0)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.0)\n",
            "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.0.8)\n",
            "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (3.10.0)\n",
            "Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.16.4)\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.1.0)\n",
            "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.33.6)\n",
            "Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.1)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.11.2)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (3.1.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.1.0)\n",
            "Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.15.0)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.9.0)\n",
            "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.12.0)\n",
            "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.1.8)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.8.1)\n",
            "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.2.2)\n",
            "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==1.15.0) (2.9.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tensorflow==1.15.0) (42.0.2)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (0.16.0)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (3.1.1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SUKtEKOcgBjH",
        "colab_type": "code",
        "outputId": "1494b39e-b85e-4e8d-d86e-4b6546c93903",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 151
        }
      },
      "source": [
        "!apt-get --yes install git"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Reading package lists... Done\n",
            "Building dependency tree       \n",
            "Reading state information... Done\n",
            "git is already the newest version (1:2.17.1-1ubuntu0.5).\n",
            "The following package was automatically installed and is no longer required:\n",
            "  libnvidia-common-430\n",
            "Use 'apt autoremove' to remove it.\n",
            "0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G4biLNp9gUnL",
        "colab_type": "code",
        "outputId": "939af98a-3e17-4a58-a57f-be5da8e70f17",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "!pip install git+https://github.com/deepmipt/bert.git@feat/multi_gpu"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting git+https://github.com/deepmipt/bert.git@feat/multi_gpu\n",
            "  Cloning https://github.com/deepmipt/bert.git (to revision feat/multi_gpu) to /tmp/pip-req-build-6b4zgfyg\n",
            "  Running command git clone -q https://github.com/deepmipt/bert.git /tmp/pip-req-build-6b4zgfyg\n",
            "  Running command git checkout -b feat/multi_gpu --track origin/feat/multi_gpu\n",
            "  Switched to a new branch 'feat/multi_gpu'\n",
            "  Branch 'feat/multi_gpu' set up to track remote branch 'feat/multi_gpu' from 'origin'.\n",
            "Requirement already satisfied (use --upgrade to upgrade): bert-dp==1.0 from git+https://github.com/deepmipt/bert.git@feat/multi_gpu in /usr/local/lib/python3.6/dist-packages\n",
            "Building wheels for collected packages: bert-dp\n",
            "  Building wheel for bert-dp (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for bert-dp: filename=bert_dp-1.0-cp36-none-any.whl size=23580 sha256=f3d3863015100d7485bfd6e04177a7503691abdf04cbfa54fca62919797517a6\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-eptlaqij/wheels/1e/41/94/886107eaf932532594886fd8bfc9cb9d4db632e94add49d326\n",
            "Successfully built bert-dp\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2vdpXZqAgXaG",
        "colab_type": "code",
        "outputId": "44c48408-b2f7-4d8b-ff4d-8c70e640f585",
        "colab": {
          "resources": {
            "http://localhost:8080/nbextensions/google.colab/files.js": {
              "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7Ci8vIE1heCBhbW91bnQgb2YgdGltZSB0byBibG9jayB3YWl0aW5nIGZvciB0aGUgdXNlci4KY29uc3QgRklMRV9DSEFOR0VfVElNRU9VVF9NUyA9IDMwICogMTAwMDsKCmZ1bmN0aW9uIF91cGxvYWRGaWxlcyhpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IHN0ZXBzID0gdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKTsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIC8vIENhY2hlIHN0ZXBzIG9uIHRoZSBvdXRwdXRFbGVtZW50IHRvIG1ha2UgaXQgYXZhaWxhYmxlIGZvciB0aGUgbmV4dCBjYWxsCiAgLy8gdG8gdXBsb2FkRmlsZXNDb250aW51ZSBmcm9tIFB5dGhvbi4KICBvdXRwdXRFbGVtZW50LnN0ZXBzID0gc3RlcHM7CgogIHJldHVybiBfdXBsb2FkRmlsZXNDb250aW51ZShvdXRwdXRJZCk7Cn0KCi8vIFRoaXMgaXMgcm91Z2hseSBhbiBhc3luYyBnZW5lcmF0b3IgKG5vdCBzdXBwb3J0ZWQgaW4gdGhlIGJyb3dzZXIgeWV0KSwKLy8gd2hlcmUgdGhlcmUgYXJlIG11bHRpcGxlIGFzeW5jaHJvbm91cyBzdGVwcyBhbmQgdGhlIFB5dGhvbiBzaWRlIGlzIGdvaW5nCi8vIHRvIHBvbGwgZm9yIGNvbXBsZXRpb24gb2YgZWFjaCBzdGVwLgovLyBUaGlzIHVzZXMgYSBQcm9taXNlIHRvIGJsb2NrIHRoZSBweXRob24gc2lkZSBvbiBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcCwKLy8gdGhlbiBwYXNzZXMgdGhlIHJlc3VsdCBvZiB0aGUgcHJldmlvdXMgc3RlcCBhcyB0aGUgaW5wdXQgdG8gdGhlIG5leHQgc3RlcC4KZnVuY3Rpb24gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpIHsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIGNvbnN0IHN0ZXBzID0gb3V0cHV0RWxlbWVudC5zdGVwczsKCiAgY29uc3QgbmV4dCA9IHN0ZXBzLm5leHQob3V0cHV0RWxlbWVudC5sYXN0UHJvbWlzZVZhbHVlKTsKICByZXR1cm4gUHJvbWlzZS5yZXNvbHZlKG5leHQudmFsdWUucHJvbWlzZSkudGhlbigodmFsdWUpID0+IHsKICAgIC8vIENhY2hlIHRoZSBsYXN0IHByb21pc2UgdmFsdWUgdG8gbWFrZSBpdCBhdmFpbGFibGUgdG8gdGhlIG5leHQKICAgIC8vIHN0ZXAgb2YgdGhlIGdlbmVyYXRvci4KICAgIG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSA9IHZhbHVlOwogICAgcmV0dXJuIG5leHQudmFsdWUucmVzcG9uc2U7CiAgfSk7Cn0KCi8qKgogKiBHZW5lcmF0b3IgZnVuY3Rpb24gd2hpY2ggaXMgY2FsbGVkIGJldHdlZW4gZWFjaCBhc3luYyBzdGVwIG9mIHRoZSB1cGxvYWQKICogcHJvY2Vzcy4KICogQHBhcmFtIHtzdHJpbmd9IGlucHV0SWQgRWxlbWVudCBJRCBvZiB0aGUgaW5wdXQgZmlsZSBwaWNrZXIgZWxlbWVudC4KICogQHBhcmFtIHtzdHJpbmd9IG91dHB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIG91dHB1dCBkaXNwbGF5LgogKiBAcmV0dXJuIHshSXRlcmFibGU8IU9iamVjdD59IEl0ZXJhYmxlIG9mIG5leHQgc3RlcHMuCiAqLwpmdW5jdGlvbiogdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKSB7CiAgY29uc3QgaW5wdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaW5wdXRJZCk7CiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gZmFsc2U7CgogIGNvbnN0IG91dHB1dEVsZW1lbnQgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZChvdXRwdXRJZCk7CiAgb3V0cHV0RWxlbWVudC5pbm5lckhUTUwgPSAnJzsKCiAgY29uc3QgcGlja2VkUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBpbnB1dEVsZW1lbnQuYWRkRXZlbnRMaXN0ZW5lcignY2hhbmdlJywgKGUpID0+IHsKICAgICAgcmVzb2x2ZShlLnRhcmdldC5maWxlcyk7CiAgICB9KTsKICB9KTsKCiAgY29uc3QgY2FuY2VsID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnYnV0dG9uJyk7CiAgaW5wdXRFbGVtZW50LnBhcmVudEVsZW1lbnQuYXBwZW5kQ2hpbGQoY2FuY2VsKTsKICBjYW5jZWwudGV4dENvbnRlbnQgPSAnQ2FuY2VsIHVwbG9hZCc7CiAgY29uc3QgY2FuY2VsUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBjYW5jZWwub25jbGljayA9ICgpID0+IHsKICAgICAgcmVzb2x2ZShudWxsKTsKICAgIH07CiAgfSk7CgogIC8vIENhbmNlbCB1cGxvYWQgaWYgdXNlciBoYXNuJ3QgcGlja2VkIGFueXRoaW5nIGluIHRpbWVvdXQuCiAgY29uc3QgdGltZW91dFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgc2V0VGltZW91dCgoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9LCBGSUxFX0NIQU5HRV9USU1FT1VUX01TKTsKICB9KTsKCiAgLy8gV2FpdCBmb3IgdGhlIHVzZXIgdG8gcGljayB0aGUgZmlsZXMuCiAgY29uc3QgZmlsZXMgPSB5aWVsZCB7CiAgICBwcm9taXNlOiBQcm9taXNlLnJhY2UoW3BpY2tlZFByb21pc2UsIHRpbWVvdXRQcm9taXNlLCBjYW5jZWxQcm9taXNlXSksCiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdzdGFydGluZycsCiAgICB9CiAgfTsKCiAgaWYgKCFmaWxlcykgewogICAgcmV0dXJuIHsKICAgICAgcmVzcG9uc2U6IHsKICAgICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICAgIH0KICAgIH07CiAgfQoKICBjYW5jZWwucmVtb3ZlKCk7CgogIC8vIERpc2FibGUgdGhlIGlucHV0IGVsZW1lbnQgc2luY2UgZnVydGhlciBwaWNrcyBhcmUgbm90IGFsbG93ZWQuCiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gdHJ1ZTsKCiAgZm9yIChjb25zdCBmaWxlIG9mIGZpbGVzKSB7CiAgICBjb25zdCBsaSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xpJyk7CiAgICBsaS5hcHBlbmQoc3BhbihmaWxlLm5hbWUsIHtmb250V2VpZ2h0OiAnYm9sZCd9KSk7CiAgICBsaS5hcHBlbmQoc3BhbigKICAgICAgICBgKCR7ZmlsZS50eXBlIHx8ICduL2EnfSkgLSAke2ZpbGUuc2l6ZX0gYnl0ZXMsIGAgKwogICAgICAgIGBsYXN0IG1vZGlmaWVkOiAkewogICAgICAgICAgICBmaWxlLmxhc3RNb2RpZmllZERhdGUgPyBmaWxlLmxhc3RNb2RpZmllZERhdGUudG9Mb2NhbGVEYXRlU3RyaW5nKCkgOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnbi9hJ30gLSBgKSk7CiAgICBjb25zdCBwZXJjZW50ID0gc3BhbignMCUgZG9uZScpOwogICAgbGkuYXBwZW5kQ2hpbGQocGVyY2VudCk7CgogICAgb3V0cHV0RWxlbWVudC5hcHBlbmRDaGlsZChsaSk7CgogICAgY29uc3QgZmlsZURhdGFQcm9taXNlID0gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgY29uc3QgcmVhZGVyID0gbmV3IEZpbGVSZWFkZXIoKTsKICAgICAgcmVhZGVyLm9ubG9hZCA9IChlKSA9PiB7CiAgICAgICAgcmVzb2x2ZShlLnRhcmdldC5yZXN1bHQpOwogICAgICB9OwogICAgICByZWFkZXIucmVhZEFzQXJyYXlCdWZmZXIoZmlsZSk7CiAgICB9KTsKICAgIC8vIFdhaXQgZm9yIHRoZSBkYXRhIHRvIGJlIHJlYWR5LgogICAgbGV0IGZpbGVEYXRhID0geWllbGQgewogICAgICBwcm9taXNlOiBmaWxlRGF0YVByb21pc2UsCiAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgYWN0aW9uOiAnY29udGludWUnLAogICAgICB9CiAgICB9OwoKICAgIC8vIFVzZSBhIGNodW5rZWQgc2VuZGluZyB0byBhdm9pZCBtZXNzYWdlIHNpemUgbGltaXRzLiBTZWUgYi82MjExNTY2MC4KICAgIGxldCBwb3NpdGlvbiA9IDA7CiAgICB3aGlsZSAocG9zaXRpb24gPCBmaWxlRGF0YS5ieXRlTGVuZ3RoKSB7CiAgICAgIGNvbnN0IGxlbmd0aCA9IE1hdGgubWluKGZpbGVEYXRhLmJ5dGVMZW5ndGggLSBwb3NpdGlvbiwgTUFYX1BBWUxPQURfU0laRSk7CiAgICAgIGNvbnN0IGNodW5rID0gbmV3IFVpbnQ4QXJyYXkoZmlsZURhdGEsIHBvc2l0aW9uLCBsZW5ndGgpOwogICAgICBwb3NpdGlvbiArPSBsZW5ndGg7CgogICAgICBjb25zdCBiYXNlNjQgPSBidG9hKFN0cmluZy5mcm9tQ2hhckNvZGUuYXBwbHkobnVsbCwgY2h1bmspKTsKICAgICAgeWllbGQgewogICAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgICBhY3Rpb246ICdhcHBlbmQnLAogICAgICAgICAgZmlsZTogZmlsZS5uYW1lLAogICAgICAgICAgZGF0YTogYmFzZTY0LAogICAgICAgIH0sCiAgICAgIH07CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPQogICAgICAgICAgYCR7TWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCl9JSBkb25lYDsKICAgIH0KICB9CgogIC8vIEFsbCBkb25lLgogIHlpZWxkIHsKICAgIHJlc3BvbnNlOiB7CiAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgIH0KICB9Owp9CgpzY29wZS5nb29nbGUgPSBzY29wZS5nb29nbGUgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYiA9IHNjb3BlLmdvb2dsZS5jb2xhYiB8fCB7fTsKc2NvcGUuZ29vZ2xlLmNvbGFiLl9maWxlcyA9IHsKICBfdXBsb2FkRmlsZXMsCiAgX3VwbG9hZEZpbGVzQ29udGludWUsCn07Cn0pKHNlbGYpOwo=",
              "ok": true,
              "headers": [
                [
                  "content-type",
                  "application/javascript"
                ]
              ],
              "status": 200,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 142
        }
      },
      "source": [
        "from google.colab import files\n",
        "uploaded = files.upload()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-b103cb56-85ac-4813-9a1d-da7a074a4ac5\" name=\"files[]\" multiple disabled />\n",
              "     <output id=\"result-b103cb56-85ac-4813-9a1d-da7a074a4ac5\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Saving train.txt to train (1).txt\n",
            "Saving TEST.txt to TEST (1).txt\n",
            "Saving valid.txt to valid (1).txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4RnioJD-2x2B",
        "colab_type": "code",
        "outputId": "0fff4c72-972b-4bbb-b6b1-86bb1115d943",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 202
        }
      },
      "source": [
        "!wget https://raw.githubusercontent.com/mannefedov/compling_nlp_hse_course/master/data/pristavki.csv"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-01-21 16:52:17--  https://raw.githubusercontent.com/mannefedov/compling_nlp_hse_course/master/data/pristavki.csv\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2939556 (2.8M) [text/plain]\n",
            "Saving to: ‘pristavki.csv.1’\n",
            "\n",
            "\rpristavki.csv.1       0%[                    ]       0  --.-KB/s               \rpristavki.csv.1     100%[===================>]   2.80M  --.-KB/s    in 0.06s   \n",
            "\n",
            "2020-01-21 16:52:17 (50.2 MB/s) - ‘pristavki.csv.1’ saved [2939556/2939556]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B-GHt2SU271D",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9mBOTDOC28A-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data = pd.read_csv('pristavki.csv', header=None, names=['text'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5h0GYvde6TWt",
        "colab_type": "code",
        "outputId": "1aa08784-ee34-4ff1-c883-0b69fba10d6c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        }
      },
      "source": [
        "!ls *.txt"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'TEST (1).txt'\t'train (1).txt'  'valid (1).txt'\n",
            " TEST.txt\t train.txt\t  valid.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hNnp8Wqu6VgX",
        "colab_type": "code",
        "outputId": "8aaae848-0419-4933-8672-6c59021daba2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 185
        }
      },
      "source": [
        "!head train.txt"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Рассмотрю O\n",
            "вариант O\n",
            "обмена O\n",
            "на O\n",
            "Batllefield B-PRS\n",
            "1 I-PRS\n",
            "или O\n",
            "еще O\n",
            "что-нибудь O\n",
            "из O\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-bPz9a9j28FB",
        "colab_type": "code",
        "outputId": "e12ef2dc-1536-4c4d-daa1-b86ad4e09afe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "import json\n",
        "from deeppavlov import configs, build_model, train_model\n",
        "\n",
        "with configs.ner.ner_ontonotes_bert_mult.open(encoding='utf8') as f:\n",
        "    ner_config = json.load(f)\n",
        "\n",
        "ner_config['dataset_reader']['data_path'] = './'  # directory with train.txt, valid.txt and test.txt files\n",
        "ner_config['metadata']['variables']['NER_PATH'] = './'\n",
        "ner_config['metadata']['download'] = [ner_config['metadata']['download'][-1]]  # do not download the pretrained ontonotes model\n",
        "\n",
        "ner_model = train_model(ner_config, download=True)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 16:52:24.513 INFO in 'deeppavlov.download'['download'] at line 117: Skipped http://files.deeppavlov.ai/deeppavlov_data/bert/multi_cased_L-12_H-768_A-12.zip download because of matching hashes\n",
            "2020-01-21 16:52:24.515 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip 'предпочтении \\n', splitted as ['предпочтении']\n",
            "2020-01-21 16:52:24.517 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip 'предпочтении \\n', splitted as ['предпочтении']\n",
            "2020-01-21 16:52:24.518 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip ':O\\n', splitted as [':O']\n",
            "2020-01-21 16:52:24.521 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip '/\\n', splitted as ['/']\n",
            "2020-01-21 16:52:24.522 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip '5 \\n', splitted as ['5']\n",
            "2020-01-21 16:52:24.523 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip 'макс \\n', splitted as ['макс']\n",
            "2020-01-21 16:52:24.524 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip 'пейн\\n', splitted as ['пейн']\n",
            "2020-01-21 16:52:24.527 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip ', \\n', splitted as [',']\n",
            "2020-01-21 16:52:24.528 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip '.O\\n', splitted as ['.O']\n",
            "2020-01-21 16:52:24.529 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip ':O\\n', splitted as [':O']\n",
            "2020-01-21 16:52:24.530 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip '/\\n', splitted as ['/']\n",
            "2020-01-21 16:52:24.531 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip '5 \\n', splitted as ['5']\n",
            "2020-01-21 16:52:24.532 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip 'макс \\n', splitted as ['макс']\n",
            "2020-01-21 16:52:24.534 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip 'пейн\\n', splitted as ['пейн']\n",
            "2020-01-21 16:52:24.537 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip ', \\n', splitted as [',']\n",
            "2020-01-21 16:52:24.538 WARNING in 'deeppavlov.dataset_readers.conll2003_reader'['conll2003_reader'] at line 96: Skip '.O\\n', splitted as ['.O']\n",
            "2020-01-21 16:52:24.539 INFO in 'deeppavlov.core.trainers.fit_trainer'['fit_trainer'] at line 68: NNTrainer got additional init parameters ['pytest_max_batches', 'pytest_batch_size'] that will be ignored:\n",
            "2020-01-21 16:52:24.953 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /content/tag.dict]\n",
            "2020-01-21 16:52:24.957 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 101: [saving vocabulary to /content/tag.dict]\n",
            "2020-01-21 16:52:54.774 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from /content/model]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Restoring parameters from /content/model\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 16:53:01.991 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 61 tokens with 9 phrases; found: 13 phrases; correct: 0.\n",
            "\n",
            "precision:  46.15%; recall:  66.67%; FB1:  54.55\n",
            "\n",
            "\tPRS: precision:  46.15%; recall:  66.67%; F1:  54.55 13\n",
            "\n",
            "\n",
            "2020-01-21 16:53:01.992 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 198: Initial best ner_f1 of 54.5455\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 54.5455, \"ner_token_f1\": 88.8889}, \"time_spent\": \"0:00:03\", \"epochs_done\": 0, \"batches_seen\": 0, \"train_examples_seen\": 0, \"impatience\": 0, \"patience_limit\": 100}}\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 16:59:20.787 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 256 tokens with 70 phrases; found: 70 phrases; correct: 0.\n",
            "\n",
            "precision:  100.00%; recall:  100.00%; FB1:  100.00\n",
            "\n",
            "\t: precision:  0.00%; recall:  0.00%; F1:  0.00 0\n",
            "\n",
            "\tPRS: precision:  100.00%; recall:  100.00%; F1:  100.00 70\n",
            "\n",
            "\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "{\"train\": {\"eval_examples_count\": 11, \"metrics\": {\"ner_f1\": 100.0, \"ner_token_f1\": 100.0}, \"time_spent\": \"0:06:22\", \"epochs_done\": 19, \"batches_seen\": 20, \"train_examples_seen\": 220, \"head_learning_rate\": 0.009999999776482582, \"bert_learning_rate\": 1.9999999552965164e-05, \"loss\": 1.1115442879498005}}\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 16:59:22.546 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 61 tokens with 9 phrases; found: 13 phrases; correct: 0.\n",
            "\n",
            "precision:  46.15%; recall:  66.67%; FB1:  54.55\n",
            "\n",
            "\tPRS: precision:  46.15%; recall:  66.67%; F1:  54.55 13\n",
            "\n",
            "\n",
            "2020-01-21 16:59:22.548 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 211: Did not improve on the ner_f1 of 54.5455\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 54.5455, \"ner_token_f1\": 88.8889}, \"time_spent\": \"0:06:23\", \"epochs_done\": 19, \"batches_seen\": 20, \"train_examples_seen\": 220, \"impatience\": 1, \"patience_limit\": 100}}\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 17:02:22.605 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /content/tag.dict]\n",
            "2020-01-21 17:02:51.751 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from /content/model]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Restoring parameters from /content/model\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 17:03:23.205 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 61 tokens with 9 phrases; found: 13 phrases; correct: 0.\n",
            "\n",
            "precision:  46.15%; recall:  66.67%; FB1:  54.55\n",
            "\n",
            "\tPRS: precision:  46.15%; recall:  66.67%; F1:  54.55 13\n",
            "\n",
            "\n",
            "2020-01-21 17:03:23.207 WARNING in 'deeppavlov.core.trainers.fit_trainer'['fit_trainer'] at line 214: Got empty data iterable for scoring\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 54.5455, \"ner_token_f1\": 88.8889}, \"time_spent\": \"0:00:03\"}}\n",
            "{\"test\": {\"eval_examples_count\": 0, \"metrics\": null, \"time_spent\": \"0:00:00\"}}\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2020-01-21 17:03:24.251 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /content/tag.dict]\n",
            "2020-01-21 17:03:52.344 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from /content/model]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Restoring parameters from /content/model\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "VKEWNLwhiK-H",
        "outputId": "c0880188-80a4-40df-9da1-3fa42e5a1bfe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 390
        }
      },
      "source": [
        "marked = []\n",
        "\n",
        "for text in data.text.values[:1000]:\n",
        "    # BERT имеет лимит на длину текста в 512 слов, возьмем даже еще меньше\n",
        "    if len(text.split()) > 100:\n",
        "        continue\n",
        "    pred = ner_model([text])\n",
        "    sent, tags = pred[0][0], pred[1][0]\n",
        "    \n",
        "    # достанем только тексты с сущностями\n",
        "    if len(set(tags[0])) > 1:\n",
        "        marked.append(list(zip(sent,tags)))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-21-cca4dc8c5f62>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m100\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m         \u001b[0;32mcontinue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m     \u001b[0mpred\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mner_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m     \u001b[0msent\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtags\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpred\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpred\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/deeppavlov/core/common/chainer.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args)\u001b[0m\n\u001b[1;32m    205\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    206\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 207\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_compute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparam_names\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0min_x\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpipe\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtargets\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mout_params\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    208\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    209\u001b[0m     \u001b[0;34m@\u001b[0m\u001b[0mstaticmethod\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/deeppavlov/core/common/chainer.py\u001b[0m in \u001b[0;36m_compute\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n\u001b[1;32m    228\u001b[0m                 \u001b[0mres\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcomponent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0min_keys\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    229\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 230\u001b[0;31m                 \u001b[0mres\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcomponent\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    231\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mout_params\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    232\u001b[0m                 \u001b[0mmem\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mout_params\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mres\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, input_ids, input_masks, y_masks)\u001b[0m\n\u001b[1;32m    621\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreturn_probas\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    622\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0muse_crf\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 623\u001b[0;31m                 \u001b[0mpred\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_decode_crf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfeed_dict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    624\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    625\u001b[0m                 \u001b[0mpred\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mseq_lengths\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msess\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0my_predictions\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mseq_lengths\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfeed_dict\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_backend.py\u001b[0m in \u001b[0;36m_wrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m     26\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_wrapped\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     27\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_default\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 28\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     29\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     30\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0m_wrapped\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py\u001b[0m in \u001b[0;36m_decode_crf\u001b[0;34m(self, feed_dict)\u001b[0m\n\u001b[1;32m    585\u001b[0m                                                                  \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0my_masks_ph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    586\u001b[0m                                                                  self.seq_lengths],\n\u001b[0;32m--> 587\u001b[0;31m                                                                 feed_dict=feed_dict)\n\u001b[0m\u001b[1;32m    588\u001b[0m         \u001b[0;31m# iterate over the sentences because no batching in viterbi_decode\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    589\u001b[0m         \u001b[0my_pred\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m    954\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    955\u001b[0m       result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[0;32m--> 956\u001b[0;31m                          run_metadata_ptr)\n\u001b[0m\u001b[1;32m    957\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    958\u001b[0m         \u001b[0mproto_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\u001b[0m in \u001b[0;36m_run\u001b[0;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1178\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1179\u001b[0m       results = self._do_run(handle, final_targets, final_fetches,\n\u001b[0;32m-> 1180\u001b[0;31m                              feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[1;32m   1181\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1182\u001b[0m       \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\u001b[0m in \u001b[0;36m_do_run\u001b[0;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[1;32m   1357\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1358\u001b[0m       return self._do_call(_run_fn, feeds, fetches, targets, options,\n\u001b[0;32m-> 1359\u001b[0;31m                            run_metadata)\n\u001b[0m\u001b[1;32m   1360\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1361\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\u001b[0m in \u001b[0;36m_do_call\u001b[0;34m(self, fn, *args)\u001b[0m\n\u001b[1;32m   1363\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_do_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1364\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1365\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1366\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1367\u001b[0m       \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcompat\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[0;34m(feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[1;32m   1348\u001b[0m       \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_extend_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1349\u001b[0m       return self._call_tf_sessionrun(options, feed_dict, fetch_list,\n\u001b[0;32m-> 1350\u001b[0;31m                                       target_list, run_metadata)\n\u001b[0m\u001b[1;32m   1351\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1352\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_prun_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py\u001b[0m in \u001b[0;36m_call_tf_sessionrun\u001b[0;34m(self, options, feed_dict, fetch_list, target_list, run_metadata)\u001b[0m\n\u001b[1;32m   1441\u001b[0m     return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,\n\u001b[1;32m   1442\u001b[0m                                             \u001b[0mfetch_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1443\u001b[0;31m                                             run_metadata)\n\u001b[0m\u001b[1;32m   1444\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1445\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_call_tf_sessionprun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "24t7n4HMEFqw",
        "colab_type": "code",
        "outputId": "e1b4b7ce-7bb8-4c1e-eebe-3caca9c3a7b9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "marked"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[('Battlefield', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Granturismo', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('и', 'O'),\n",
              "  ('Звезды', 'O'),\n",
              "  ('Playstation', 'B-PRS'),\n",
              "  ('Битва', 'I-PRS'),\n",
              "  ('сильнейших', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('За', 'O'),\n",
              "  ('все', 'O'),\n",
              "  ('1500', 'O'),\n",
              "  ('руб', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Sony', 'B-PRS'),\n",
              "  ('Playstation', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('Slim', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('лицензионная', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('никогда', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('прошивалась', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Все', 'O'),\n",
              "  ('провода', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('комплекте', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('hdmi', 'B-PRS'),\n",
              "  ('кабель', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('большого', 'O'),\n",
              "  ('ТВ', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('тормозит', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('перегревается', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Купили', 'O'),\n",
              "  ('Пс4', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('эту', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('играем', 'O'),\n",
              "  ('почти', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Kaspersky', 'B-PRS'),\n",
              "  ('Anti', 'B-PRS'),\n",
              "  ('-', 'B-PRS'),\n",
              "  ('Virus', 'B-PRS'),\n",
              "  ('–', 'O'),\n",
              "  ('это', 'O'),\n",
              "  ('решение', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('базовой', 'O'),\n",
              "  ('защиты', 'O'),\n",
              "  ('компьютера', 'O'),\n",
              "  ('от', 'O'),\n",
              "  ('основных', 'O'),\n",
              "  ('видов', 'O'),\n",
              "  ('интернет', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('угроз', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('снижающее', 'O'),\n",
              "  ('его', 'O'),\n",
              "  ('производительность', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('Надежная', 'O'),\n",
              "  ('защита', 'O'),\n",
              "  ('от', 'O'),\n",
              "  ('вредоносных', 'O'),\n",
              "  ('программ', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('Высокая', 'O'),\n",
              "  ('скорость', 'O'),\n",
              "  ('работы', 'O'),\n",
              "  ('без', 'O'),\n",
              "  ('снижения', 'O'),\n",
              "  ('производительности', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('Удобное', 'O'),\n",
              "  ('управление', 'O'),\n",
              "  ('защитой', 'O'),\n",
              "  ('из', 'O'),\n",
              "  ('любой', 'O'),\n",
              "  ('точки', 'O')],\n",
              " [('Wi', 'B-PRS'),\n",
              "  ('-', 'B-PRS'),\n",
              "  ('Fi', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('камеры', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('Задняя', 'O'),\n",
              "  ('сенсорная', 'O'),\n",
              "  ('панель', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('Передний', 'O'),\n",
              "  ('сенсорный', 'O'),\n",
              "  ('экран', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('Зарядное', 'O'),\n",
              "  ('устройство', 'O'),\n",
              "  ('от', 'O'),\n",
              "  ('сети', 'O'),\n",
              "  ('и', 'O'),\n",
              "  ('USB', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('ПО', 'O'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('61', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Ps', 'B-PRS'),\n",
              "  ('move', 'I-PRS'),\n",
              "  ('с', 'O'),\n",
              "  ('камерой', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Продаю', 'O'),\n",
              "  ('из', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('за', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('надобности', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Работает', 'O'),\n",
              "  ('отлично', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Торга', 'O'),\n",
              "  ('нет', 'O')],\n",
              " [('Assassins', 'B-PRS'),\n",
              "  ('creed', 'B-PRS'),\n",
              "  ('unity', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('800', 'I-PRS'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Bloodborn', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('800', 'I-PRS'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Watch', 'B-PRS'),\n",
              "  ('dogs', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('800', 'I-PRS'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Wolfenstein', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('800', 'I-PRS'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('MGS', 'B-PRS'),\n",
              "  ('5', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1500', 'I-PRS'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('The', 'B-PRS'),\n",
              "  ('last', 'B-PRS'),\n",
              "  ('of', 'B-PRS'),\n",
              "  ('us', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1500', 'O'),\n",
              "  ('₽', 'O')],\n",
              " [('Fifa', 'B-PRS'),\n",
              "  ('400', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Destiny', 'B-PRS'),\n",
              "  ('600', 'I-PRS')],\n",
              " [('Uncharted', 'B-PRS'),\n",
              "  ('3', 'O'),\n",
              "  ('коллекционка', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1000', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Ufc', 'B-PRS'),\n",
              "  ('undisputed', 'B-PRS'),\n",
              "  ('2009', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1200', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Fight', 'B-PRS'),\n",
              "  ('night', 'B-PRS'),\n",
              "  ('round', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('700', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Fight', 'B-PRS'),\n",
              "  ('night', 'B-PRS'),\n",
              "  ('champion', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Michael', 'B-PRS'),\n",
              "  ('jackson', 'B-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('400', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Max', 'B-PRS'),\n",
              "  ('.', 'I-PRS'),\n",
              "  ('Payne', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Дурная', 'O'),\n",
              "  ('репутация', 'O'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('600', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Smack', 'B-PRS'),\n",
              "  ('down', 'B-PRS'),\n",
              "  ('vs', 'I-PRS'),\n",
              "  ('raw', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Рассмотрю', 'O'),\n",
              "  ('варианты', 'O'),\n",
              "  ('обмена', 'O')],\n",
              " [('watch', 'B-PRS'),\n",
              "  ('dogs', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('2400', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('far', 'B-PRS'),\n",
              "  ('cry', 'B-PRS'),\n",
              "  ('primal', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1000', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('при', 'O'),\n",
              "  ('покупке', 'O'),\n",
              "  ('двух', 'O'),\n",
              "  ('дисков', 'O'),\n",
              "  ('скидка', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('звоните', 'O')],\n",
              " [('Gran', 'B-PRS'),\n",
              "  ('Turismo', 'B-PRS'),\n",
              "  ('6', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Fight', 'B-PRS'),\n",
              "  ('Night', 'B-PRS'),\n",
              "  ('champion', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'I-PRS')],\n",
              " [('Ps3', 'B-PRS'),\n",
              "  ('в', 'O'),\n",
              "  ('хорошем', 'O'),\n",
              "  ('состоянии', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('джойстика', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('5', 'O'),\n",
              "  ('игр', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('диске', 'O'),\n",
              "  ('и', 'O'),\n",
              "  ('20', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('приставке', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Джойстик', 'B-PRS'), ('и', 'O'), ('провода', 'O')],\n",
              " [('COLLAPSE', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('WARCRAFT', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('STALKER', 'B-PRS'),\n",
              "  ('(', 'O'),\n",
              "  ('ИСТОРИЯ', 'B-PRS'),\n",
              "  ('ПРИБОЯ', 'I-PRS'),\n",
              "  (')', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('WARRIOR', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('MАФИЯ', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('ALONE', 'B-PRS'),\n",
              "  ('DARK', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('MERCENARIES2', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('SNIPER', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('ТАМПЛИЕРЫ', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('КВЕЙК4', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('DEWIL', 'B-PRS'),\n",
              "  ('MAY', 'B-PRS'),\n",
              "  ('CRY', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('WORMS4', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('HOSPITAL', 'B-PRS'),\n",
              "  ('TYCOON', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('FLASHPOINT', 'B-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('200', 'O'),\n",
              "  ('рулей', 'O'),\n",
              "  ('за', 'O'),\n",
              "  ('весь', 'O'),\n",
              "  ('комплект', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('или', 'O'),\n",
              "  ('50', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  ('за', 'O'),\n",
              "  ('диск', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Sony', 'B-PRS'),\n",
              "  ('Playstaition', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('обем', 'O'),\n",
              "  ('памяти', 'O'),\n",
              "  (':', 'O'),\n",
              "  ('250гб', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('В', 'O'),\n",
              "  ('отличном', 'O'),\n",
              "  ('состоянии', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('провода', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('геймпад', 'B-PRS'),\n",
              "  ('2шт', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Не', 'O'),\n",
              "  ('прошивалась', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('имеет', 'O'),\n",
              "  ('доступ', 'O'),\n",
              "  ('к', 'O'),\n",
              "  ('интернету', 'O'),\n",
              "  ('!', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Лучшее', 'O'),\n",
              "  ('предложение', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('соотношении', 'O'),\n",
              "  ('цена', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('качество', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Игры', 'O'),\n",
              "  (':', 'O'),\n",
              "  ('Mortal', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Dirt', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('FIFA', 'B-PRS'),\n",
              "  ('13', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('FIFA', 'B-PRS'),\n",
              "  ('16', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Fifa', 'B-PRS'),\n",
              "  ('STREET', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('TENNIS', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('GTA', 'B-PRS'),\n",
              "  ('5', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('UFS', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Gold', 'B-PRS'),\n",
              "  ('of', 'B-PRS'),\n",
              "  ('Wars', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Grand', 'B-PRS'),\n",
              "  ('Turismo', 'B-PRS'),\n",
              "  ('6', 'I-PRS'),\n",
              "  ('.', 'O')],\n",
              " [('Анчартед', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('2000', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('анчартид', 'B-PRS'),\n",
              "  ('коллекция', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1500', 'O')],\n",
              " [('NHL', 'B-PRS'),\n",
              "  ('15', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('FIFA', 'B-PRS'),\n",
              "  ('15', 'I-PRS')],\n",
              " [('Джойстики', 'B-PRS'),\n",
              "  ('беспроводные', 'O'),\n",
              "  ('PS3', 'B-PRS'),\n",
              "  ('800р', 'I-PRS'),\n",
              "  ('за', 'O'),\n",
              "  ('штуку', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Джойстики', 'B-PRS'),\n",
              "  ('беспроводные', 'O'),\n",
              "  ('XBOX360', 'B-PRS'),\n",
              "  ('1100р', 'I-PRS'),\n",
              "  ('за', 'O'),\n",
              "  ('штуку', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Игры', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('пиратский', 'O'),\n",
              "  ('XBOX360', 'B-PRS'),\n",
              "  ('200р', 'I-PRS'),\n",
              "  ('за', 'O'),\n",
              "  ('все', 'O')],\n",
              " [('For', 'B-PRS'),\n",
              "  ('Honor', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('000', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('NBA', 'B-PRS'),\n",
              "  ('2K15', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('800', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('(', 'O'),\n",
              "  ('состояние', 'O'),\n",
              "  ('диска', 'O'),\n",
              "  ('отличное', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('коробки', 'O'),\n",
              "  ('удов', 'O'),\n",
              "  ('.', 'O'),\n",
              "  (')', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Uncharted', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('A', 'B-PRS'),\n",
              "  (\"Thief's\", 'B-PRS'),\n",
              "  ('End', 'B-PRS'),\n",
              "  ('(', 'O'),\n",
              "  ('PS4', 'B-PRS'),\n",
              "  (')', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('English', 'O'),\n",
              "  ('version', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1500', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('(', 'O'),\n",
              "  ('обратите', 'O'),\n",
              "  ('внимание', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('игре', 'O'),\n",
              "  ('нет', 'O'),\n",
              "  ('русского', 'O'),\n",
              "  ('языка', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('состояние', 'O'),\n",
              "  ('идеальное', 'O'),\n",
              "  (')', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Продажа', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('возможен', 'O'),\n",
              "  ('обмен', 'O'),\n",
              "  ('(', 'O'),\n",
              "  ('предложения', 'O'),\n",
              "  ('пишите', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('личку', 'O'),\n",
              "  (')', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('King', 'B-PRS'),\n",
              "  ('Kong', 'B-PRS'),\n",
              "  ('лицензионный', 'O'),\n",
              "  ('диск', 'O'),\n",
              "  ('x', 'B-PRS'),\n",
              "  ('-', 'B-PRS'),\n",
              "  ('box', 'B-PRS'),\n",
              "  ('360', 'I-PRS')],\n",
              " [('АКЦИЯ', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('АКЦИЯ', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('АКЦИЯ', 'B-PRS'),\n",
              "  ('!', 'I-PRS'),\n",
              "  ('!', 'I-PRS'),\n",
              "  ('!', 'O'),\n",
              "  ('РАСПРОДАЖА', 'B-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('ПРОДАЁМ', 'B-PRS'),\n",
              "  ('Новые', 'O'),\n",
              "  ('игровые', 'O'),\n",
              "  ('приставки', 'O'),\n",
              "  ('Sony', 'B-PRS'),\n",
              "  ('PS4', 'B-PRS'),\n",
              "  ('Slim', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('В', 'O'),\n",
              "  ('УПАКОВКЕ', 'B-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('Гарантия', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1', 'O'),\n",
              "  ('год', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('В', 'O'),\n",
              "  ('наличии', 'O'),\n",
              "  ('и', 'O'),\n",
              "  ('под', 'O'),\n",
              "  ('заказ', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Sony', 'B-PRS'),\n",
              "  ('PlayStation', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('Slim', 'I-PRS'),\n",
              "  ('500GB', 'I-PRS'),\n",
              "  ('21', 'I-PRS'),\n",
              "  ('000', 'O'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Sony', 'B-PRS'),\n",
              "  ('PlayStation', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('Slim', 'I-PRS'),\n",
              "  ('1Tb', 'I-PRS'),\n",
              "  ('21', 'I-PRS'),\n",
              "  ('500', 'I-PRS'),\n",
              "  ('₽', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Комплектация', 'O'),\n",
              "  (':', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Система', 'O'),\n",
              "  ('PlayStation', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('черного', 'O'),\n",
              "  ('цвета', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Беспроводной', 'O'),\n",
              "  ('контроллер', 'O'),\n",
              "  ('DUALSHOCK', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Шнур', 'B-PRS'),\n",
              "  ('питания', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Кабель', 'B-PRS'),\n",
              "  ('HDMI', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Кабель', 'B-PRS'),\n",
              "  ('USB', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Viber', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('WhatsApp', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('возможен', 'O'),\n",
              "  ('приём', 'O'),\n",
              "  ('заказов', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('PS4', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Playstation', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('PS4', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('console', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('плейстейшн', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('игры', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('Playstation', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('games', 'O'),\n",
              "  ('PS4', 'B-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Playstation', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('.', 'O')],\n",
              " [('star', 'B-PRS'),\n",
              "  ('wars', 'I-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('Battlefront', 'B-PRS'),\n",
              "  ('+', 'I-PRS'),\n",
              "  ('For', 'B-PRS'),\n",
              "  ('honor', 'B-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('геймпад', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('встроенная', 'O'),\n",
              "  ('память', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1', 'O'),\n",
              "  ('Тб', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('HDMI', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('кабель', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('USB', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('кабель', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('гарнитура', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('документация', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('кабель', 'O'),\n",
              "  ('питания', 'O'),\n",
              "  ('полный', 'O'),\n",
              "  ('комплект', 'O'),\n",
              "  ('и', 'O'),\n",
              "  ('все', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('идеальном', 'O'),\n",
              "  ('состоянии', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('Упаковка', 'O'),\n",
              "  ('коробка', 'O'),\n",
              "  ('и', 'O'),\n",
              "  ('чек', 'O'),\n",
              "  ('есть', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('На', 'O'),\n",
              "  ('гарантии', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('Куплена', 'O'),\n",
              "  ('консоль', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('ноябре', 'O'),\n",
              "  ('2016', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('28', 'O'),\n",
              "  ('999', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('+', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('игры', 'O'),\n",
              "  ('(', 'O'),\n",
              "  ('3', 'O'),\n",
              "  ('000', 'O'),\n",
              "  ('р', 'O'),\n",
              "  (')', 'O'),\n",
              "  ('+', 'O'),\n",
              "  ('подписка', 'O'),\n",
              "  ('PS', 'O'),\n",
              "  ('+', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('3', 'O'),\n",
              "  ('месяца', 'O'),\n",
              "  ('вперед', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('(', 'O'),\n",
              "  ('В', 'O'),\n",
              "  ('PS', 'O'),\n",
              "  ('Store', 'O'),\n",
              "  ('эти', 'O'),\n",
              "  ('игры', 'O'),\n",
              "  ('стоят', 'O'),\n",
              "  ('по', 'O'),\n",
              "  ('4', 'O'),\n",
              "  ('000', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('каждая', 'O'),\n",
              "  (')', 'O')],\n",
              " [('xbox', 'B-PRS'), ('one', 'I-PRS'), ('игры', 'O'), ('обмен', 'O')],\n",
              " [('Toefl', 'B-PRS'),\n",
              "  ('Test', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('Рекордный', 'O'),\n",
              "  ('Результат', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('50', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  (';', 'O'),\n",
              "  ('English', 'B-PRS'),\n",
              "  ('Platinum', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('50', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  (';', 'O'),\n",
              "  ('Англо', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('Русская', 'B-PRS'),\n",
              "  ('и', 'O'),\n",
              "  ('Русско', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('Английская', 'B-PRS'),\n",
              "  ('программа', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('тренажер', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('пополнения', 'O'),\n",
              "  ('лексического', 'O'),\n",
              "  ('запаса', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('50', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  (';', 'O'),\n",
              "  ('Учим', 'O'),\n",
              "  ('язык', 'O'),\n",
              "  ('English', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('4', 'O'),\n",
              "  ('CD', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('150', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  (';', 'O'),\n",
              "  ('English', 'B-PRS'),\n",
              "  ('(', 'O'),\n",
              "  ('Компьютерный', 'O'),\n",
              "  ('курс', 'O'),\n",
              "  ('+', 'O'),\n",
              "  ('аудиоразговорники', 'O'),\n",
              "  ('Doki', 'B-PRS'),\n",
              "  (')', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('уникальные', 'O'),\n",
              "  ('интерактивные', 'O'),\n",
              "  ('мультфильмы', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('изучения', 'O'),\n",
              "  ('английского', 'O'),\n",
              "  ('языка', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('диска', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('100', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('За', 'O'),\n",
              "  ('все', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('9', 'O'),\n",
              "  ('дисков', 'O'),\n",
              "  ('(', 'O'),\n",
              "  ('вместе', 'O'),\n",
              "  (')', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('250', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Doom', 'B-PRS'),\n",
              "  ('1500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Call', 'B-PRS'),\n",
              "  ('of', 'B-PRS'),\n",
              "  ('Duty', 'B-PRS'),\n",
              "  ('1500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Assassins', 'B-PRS'),\n",
              "  ('creed', 'B-PRS'),\n",
              "  ('черный', 'O'),\n",
              "  ('флаг', 'B-PRS'),\n",
              "  ('700', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('God', 'B-PRS'),\n",
              "  ('of', 'B-PRS'),\n",
              "  ('War', 'B-PRS'),\n",
              "  ('700', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Killzone', 'B-PRS'),\n",
              "  ('700', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('The', 'B-PRS'),\n",
              "  ('Crew', 'B-PRS'),\n",
              "  ('600', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Garden', 'B-PRS'),\n",
              "  ('Warfare', 'B-PRS'),\n",
              "  ('400', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Обмен', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('интересует', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Far', 'B-PRS'),\n",
              "  ('Cry', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('Специальное', 'O'),\n",
              "  ('издание', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Диск', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('отличном', 'O'),\n",
              "  ('состоянии', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Любые', 'O'),\n",
              "  ('проверки', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Могу', 'O'),\n",
              "  ('рассмотреть', 'O'),\n",
              "  ('обмен', 'O')],\n",
              " [('Watch', 'B-PRS'),\n",
              "  ('dogs', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('2700', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Far', 'B-PRS'),\n",
              "  ('cry', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1400', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Homefront', 'B-PRS'),\n",
              "  ('the', 'I-PRS'),\n",
              "  ('revolution', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('2300', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Destiny', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Good', 'B-PRS'),\n",
              "  ('of', 'B-PRS'),\n",
              "  ('war', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1500', 'I-PRS')],\n",
              " [('Fifa', 'B-PRS'),\n",
              "  ('08', 'I-PRS'),\n",
              "  ('лицензионный', 'O'),\n",
              "  ('диск', 'O'),\n",
              "  ('x', 'B-PRS'),\n",
              "  ('-', 'B-PRS'),\n",
              "  ('box', 'B-PRS'),\n",
              "  ('360', 'I-PRS')],\n",
              " [('СИСТЕМНЫЕ', 'B-PRS'),\n",
              "  ('ТРЕБОВАНИЯ', 'I-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('ПРОЦЕССОР', 'B-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('intel', 'B-PRS'),\n",
              "  ('core', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('duo', 'I-PRS'),\n",
              "  ('1', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('8ггц', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('amd', 'B-PRS'),\n",
              "  ('athlon', 'B-PRS'),\n",
              "  ('x2', 'B-PRS'),\n",
              "  ('64', 'I-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('ггц', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('ОПЕРАЦИОННАЯ', 'B-PRS'),\n",
              "  ('СИСТЕМА', 'I-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('ОПЕРАТИВНАЯ', 'B-PRS'),\n",
              "  ('ПАМЯТЬ', 'I-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('гб', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('DVD', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('ROM', 'B-PRS'),\n",
              "  ('ПРИВОД', 'I-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('4', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('х', 'O'),\n",
              "  ('скоростной', 'O'),\n",
              "  ('dvd', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('rоm', 'B-PRS'),\n",
              "  ('привод', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('СВОБОДНОЕ', 'B-PRS'),\n",
              "  ('МЕСТО', 'I-PRS'),\n",
              "  ('НА', 'O'),\n",
              "  ('ЖЕСТКОМ', 'B-PRS'),\n",
              "  ('ДИСКЕ', 'I-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('32', 'I-PRS'),\n",
              "  ('гб', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('ВИДЕОКАРТА', 'B-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('nvidia', 'B-PRS'),\n",
              "  ('7900', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('ati', 'B-PRS'),\n",
              "  ('x1900', 'I-PRS'),\n",
              "  ('c', 'O'),\n",
              "  ('256', 'O'),\n",
              "  ('мб', 'O'),\n",
              "  ('видеопамяти', 'O')],\n",
              " [('Nyko', 'B-PRS'),\n",
              "  ('Zoom', 'I-PRS'),\n",
              "  ('уменьшает', 'O'),\n",
              "  ('пространство', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('необходимое', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('ваших', 'O'),\n",
              "  ('любимых', 'O'),\n",
              "  ('игр', 'O'),\n",
              "  ('снижает', 'O'),\n",
              "  ('диапазон', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('необходимый', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('игры', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('Kinect', 'B-PRS'),\n",
              "  ('на', 'O'),\n",
              "  ('40', 'O'),\n",
              "  ('%', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('что', 'O'),\n",
              "  ('позволяет', 'O'),\n",
              "  ('играть', 'O'),\n",
              "  ('ближе', 'O'),\n",
              "  ('к', 'O'),\n",
              "  ('телевизору', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('также', 'O'),\n",
              "  ('позволяет', 'O'),\n",
              "  ('двум', 'O'),\n",
              "  ('людям', 'O'),\n",
              "  ('играть', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('области', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('которая', 'O'),\n",
              "  ('раньше', 'O'),\n",
              "  ('подходила', 'O'),\n",
              "  ('только', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('одного', 'O'),\n",
              "  ('человека', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('что', 'O'),\n",
              "  ('делает', 'O'),\n",
              "  ('Nyko', 'B-PRS'),\n",
              "  ('просто', 'O'),\n",
              "  ('идеальным', 'O'),\n",
              "  ('дополнением', 'O'),\n",
              "  ('к', 'O'),\n",
              "  ('Kinect', 'B-PRS'),\n",
              "  ('.', 'O')],\n",
              " [('dishonored2', 'B-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('hitman', 'B-PRS'),\n",
              "  ('полный', 'O'),\n",
              "  ('сезон', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('всё', 'O'),\n",
              "  ('вместе', 'O'),\n",
              "  ('3500', 'O'),\n",
              "  ('торг', 'O'),\n",
              "  ('уместен', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('фото', 'O'),\n",
              "  ('с', 'O'),\n",
              "  ('интернета', 'O'),\n",
              "  ('камера', 'O'),\n",
              "  ('сломана', 'O'),\n",
              "  ('звоните', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('телефон', 'O')],\n",
              " [('Dualshock', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('Для', 'O'),\n",
              "  ('ps3', 'B-PRS')],\n",
              " [('Hd', 'B-PRS'),\n",
              "  ('-', 'B-PRS'),\n",
              "  ('mi', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('5', 'O'),\n",
              "  ('игр', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('джойстик', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Все', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('хорошем', 'O'),\n",
              "  ('состоянии', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('PlayStation', 'B-PRS'),\n",
              "  ('3CECH', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('3008B', 'B-PRS'),\n",
              "  ('320Gb', 'I-PRS'),\n",
              "  ('PS', 'O'),\n",
              "  ('Eye', 'B-PRS'),\n",
              "  ('+', 'I-PRS'),\n",
              "  ('PS', 'I-PRS'),\n",
              "  ('Move', 'I-PRS')],\n",
              " [('Dark', 'B-PRS'),\n",
              "  ('Soul', 'B-PRS'),\n",
              "  ('II', 'I-PRS'),\n",
              "  ('коллекционное', 'O'),\n",
              "  ('издание', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('1750', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Ледниковый', 'B-PRS'),\n",
              "  ('период', 'I-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1000', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Мадагаскар', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1000', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Праздник', 'O'),\n",
              "  ('спорта', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('600', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Dualshock', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('sixaxis', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('800', 'O'),\n",
              "  ('р', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Кружка', 'O'),\n",
              "  ('геймера', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('350', 'O')],\n",
              " [('Ben10', 'B-PRS'),\n",
              "  ('(', 'O'),\n",
              "  ('Ben', 'B-PRS'),\n",
              "  ('10', 'I-PRS'),\n",
              "  (')', 'O'),\n",
              "  ('Omniverse', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('лицензионный', 'O'),\n",
              "  ('диск', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('Playstation', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('(', 'O'),\n",
              "  ('PS3', 'O'),\n",
              "  (')', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('идеальное', 'O'),\n",
              "  ('состояние', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('английском', 'O'),\n",
              "  ('языке', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('для', 'O'),\n",
              "  ('1', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('2', 'O'),\n",
              "  ('игроков', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Стоимость', 'O'),\n",
              "  ('обмена', 'O'),\n",
              "  ('зависит', 'O'),\n",
              "  ('от', 'O'),\n",
              "  ('номинала', 'O'),\n",
              "  ('предлагаемой', 'O'),\n",
              "  ('игры', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Sega', 'B-PRS'),\n",
              "  ('mega', 'B-PRS'),\n",
              "  ('drive', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  (',', 'O'),\n",
              "  ('сега', 'O'),\n",
              "  ('мега', 'B-PRS'),\n",
              "  ('драйв', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('Два', 'O'),\n",
              "  ('джойстика', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('имеются', 'O'),\n",
              "  ('немного', 'O'),\n",
              "  ('дисков', 'O'),\n",
              "  ('игровых', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('Полностью', 'O'),\n",
              "  ('рабочая', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('Ребёнок', 'O'),\n",
              "  ('вырос', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Uncharted', 'B-PRS'),\n",
              "  ('Золотая', 'O'),\n",
              "  ('Бездна', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('600р', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Ninja', 'B-PRS'),\n",
              "  ('Gaiden', 'B-PRS'),\n",
              "  ('Sigma', 'B-PRS'),\n",
              "  ('Plus', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('600р', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Ultimate', 'B-PRS'),\n",
              "  ('Marvel', 'B-PRS'),\n",
              "  ('vs', 'B-PRS'),\n",
              "  ('Capcom', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('600р', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Карта', 'O'),\n",
              "  ('памяти', 'O'),\n",
              "  ('8гб', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('700р', 'O')],\n",
              " [('1', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('400', 'I-PRS'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('500', 'O')],\n",
              " [('X', 'B-PRS'),\n",
              "  ('-', 'B-PRS'),\n",
              "  ('box', 'B-PRS'),\n",
              "  ('360', 'I-PRS'),\n",
              "  ('Slim', 'O'),\n",
              "  ('250', 'O'),\n",
              "  ('Gb', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('матовый', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('Не', 'O'),\n",
              "  ('прошитый', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('состояние', 'O'),\n",
              "  ('4', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('5', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('В', 'O'),\n",
              "  ('комплекте', 'O'),\n",
              "  ('1', 'O'),\n",
              "  ('контроллер', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('3', 'O'),\n",
              "  ('игры', 'O'),\n",
              "  ('на', 'O'),\n",
              "  ('дисках', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Так', 'O'),\n",
              "  ('же', 'O'),\n",
              "  ('есть', 'O'),\n",
              "  ('ассортимент', 'O'),\n",
              "  ('игр', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('лицензии', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('300', 'O'),\n",
              "  ('-', 'O'),\n",
              "  ('400', 'O'),\n",
              "  ('рублей', 'O'),\n",
              "  ('за', 'O'),\n",
              "  ('штуку', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('подробности', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('личку', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Mafia', 'B-PRS'),\n",
              "  ('3', 'I-PRS'),\n",
              "  ('PS', 'O'),\n",
              "  ('4', 'O'),\n",
              "  ('лицензия', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('диск', 'O'),\n",
              "  ('из', 'O'),\n",
              "  ('бруталити', 'O'),\n",
              "  ('4', 'O'),\n",
              "  ('категория', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('брал', 'O'),\n",
              "  ('за', 'O'),\n",
              "  ('3000', 'O'),\n",
              "  (',', 'O'),\n",
              "  ('обмен', 'O'),\n",
              "  ('не', 'O'),\n",
              "  ('интересен', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Gta', 'B-PRS'),\n",
              "  ('5', 'I-PRS'),\n",
              "  ('лицензия', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('подарок', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('Watch', 'B-PRS'),\n",
              "  ('dogs', 'B-PRS'),\n",
              "  ('2', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('2000', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Watch', 'B-PRS'),\n",
              "  ('Dogs', 'B-PRS'),\n",
              "  ('-', 'I-PRS'),\n",
              "  ('1000', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Средиземье', 'B-PRS'),\n",
              "  (':', 'O'),\n",
              "  ('Тени', 'B-PRS'),\n",
              "  ('Мордора', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('1200', 'O'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Battlefield', 'B-PRS'),\n",
              "  ('4', 'I-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('900', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'B-PRS'),\n",
              "  ('Destiny', 'B-PRS'),\n",
              "  ('-', 'I-PRS'),\n",
              "  ('800', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('NFS', 'B-PRS'),\n",
              "  ('Rivals', 'B-PRS'),\n",
              "  ('-', 'O'),\n",
              "  ('700', 'I-PRS'),\n",
              "  ('.', 'O'),\n",
              "  ('/', 'O'),\n",
              "  ('\\n', 'O'),\n",
              "  ('Рассмотрю', 'O'),\n",
              "  ('любые', 'O'),\n",
              "  ('варианты', 'O'),\n",
              "  ('обмена', 'O'),\n",
              "  ('.', 'O')],\n",
              " [('NHL', 'B-PRS'),\n",
              "  ('15', 'O'),\n",
              "  ('в', 'O'),\n",
              "  ('отличном', 'O'),\n",
              "  ('состоянии', 'O')]]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EU_BN2WfH663",
        "colab_type": "text"
      },
      "source": [
        "Данный метод оказался не таким эффективным при работе с неструктурированными данными. Скорее всего это обусловлено тем, что тренировочных данных довольно мало. Вместе с играми разметились названия приставок, видеокарт, \"жесткий диск\" и т.д. Сами названия игр иногда неправильно размечаются, часть названия размечается как версия (например, ('Ледниковый', 'B-PRS'),  ('период', 'I-PRS'),('3', 'I-PRS')). Иногда название игры не размечается полностью (например,('Assassins', 'B-PRS'), ('creed', 'B-PRS'), ('черный', 'O'), ('флаг', 'B-PRS'), ('700', 'I-PRS')). Качество в данном случае очень страдает, а скорость всего процесса поиска именованных сущностей зависит от того, есть ли размеченные данные или нет. "
      ]
    }
  ]
}