Practical_8_Word_sense_disambiguation.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "orig_nbformat": 2,
    "deepnote_notebook_id": "3d376988-2d5a-459c-bc34-7ae1d95b544c",
    "deepnote_execution_queue": [],
    "colab": {
      "name": "Practical_8_Word_sense_disambiguation.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00000-7e6a854b-617d-47ff-93b5-410e732a2227",
        "id": "nDuowGrBK2Lk"
      },
      "source": [
        "# Word sense disambiguation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00002-efb73a09-fe0b-4811-9219-da93beeca52f",
        "id": "2A-ssek_K2Ln"
      },
      "source": [
        "## Lexical ambiguity"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00003-e3a2bb0d-3e10-4abe-9f06-26f66c551089",
        "id": "d2syYXJpK2Lo"
      },
      "source": [
        "Let's refresh our knowlegde of the concept of **ambiguity**. \n",
        "Ambiguity can be\n",
        "- semantic (category  - e.g. noun or verb?)\n",
        "- syntactic (when a sentence has more than one phrase structure tree that can be assigned to it)\n",
        "\n",
        "![picture](https://www.thoughtco.com/thmb/3DSAgVoW-eEc3f3pwxPnjYT_zWw=/1500x1000/filters:fill(auto,1)/ambiguity-language-1692388_FINAL-dd68c7d1dd374507aa633d27539f0e62.png)\n",
        "(Image source: https://www.thoughtco.com/ambiguity-language-1692388)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00004-fc50c9bd-ccd5-4a6b-964b-0700d05ce2c1",
        "id": "NYAe8t-QK2Lp"
      },
      "source": [
        "![picture](https://i.pinimg.com/564x/af/77/fe/af77fe576a152e4bcbf9e24762a70886.jpg)\n",
        "(Image source: http://criticalthinkingexamples.blogspot.com/2011/03/semantic-ambiguity-ii.html)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00005-028e7f8c-ce9e-4d44-82ef-8e71098e84aa",
        "id": "wSRqmp38K2Lp"
      },
      "source": [
        "![picture] (http://1.bp.blogspot.com/-nbpn8LI40ko/Tocbqxw4DTI/AAAAAAAAAhY/aYP-9EVfYkE/s400/Ambiguity.jpg)\n",
        "(Image source: http://criticalthinkingexamples.blogspot.com/2011/10/semantic-ambiguity-iii.html)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00006-de6f3c4b-a553-47a0-90b2-54775730c38d",
        "id": "AU8ey5utK2Lq"
      },
      "source": [
        "![picture](https://jennykellerford.files.wordpress.com/2011/10/students-cook-serve-grandparents.jpg)\n",
        "(Image source: https://jennykellerford.wordpress.com/2011/10/02/students-cook-and-serve-grandparents-and-other-misplaced-modifiers)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00007-7ebf177a-83cc-4135-a5b1-31b1c42a568c",
        "id": "ZiVk1cRBK2Lq"
      },
      "source": [
        "CURIOSITY MINUTE: Have you ever heard about **Oxford comma**? It may help you avoid ambiguity when you write in English!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00008-e51a716c-0444-4033-b25f-3daa191c49d9",
        "id": "pHhebq2IK2Lr"
      },
      "source": [
        "![picture] (https://mereinkling.files.wordpress.com/2019/09/commas.png)\n",
        "(Image source: https://mereinkling.net/2019/09/17/peculiarities-of-punctuation/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00009-ed5096ec-358a-4e3a-bc2e-8d22b3040db6",
        "id": "w6XCytsEK2Lr"
      },
      "source": [
        "In this notebook we will focus only on **semantic ambiguity**.\n",
        "It can be of two types:\n",
        "\n",
        "- **polysemy** (different meanings of a word are somewhat related)\n",
        "\n",
        "(Crane: a) a bird with a long neck b) a type of construction equipment which looks like it has a long neck)\n",
        "\n",
        "- **homonymy** (different meanings of a word are not related)\n",
        "\n",
        "(peer: a) person belonging to the same group in age and status b) look searchingly)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00010-8d1d18fb-2cf9-4024-9803-d92fc64d2b25",
        "id": "8PoNQbpHK2Ls"
      },
      "source": [
        "In dictionary, polysemous items are usually found in a single dictionary entry listing all meanings of the words while homonyms are all represented by several separate entries."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00011-33470fe3-ab4f-4f12-8d4d-86aafafaf167",
        "id": "fFd1PT0JK2Ls"
      },
      "source": [
        "**Glosses** - definitions for each sense of a word in a dictionary."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00012-fff5fb01-eb8b-41c4-ae8c-a4edbed5a03f",
        "id": "yU8vqZzUK2Ls"
      },
      "source": [
        "## Relations between word senses"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00013-e546242c-2889-4adb-a233-34dcb5a6e415",
        "id": "m7vfeOZVK2Ls"
      },
      "source": [
        "**Synonyms** (words with (nearly) identical senses): couch/sofa\n",
        "\n",
        "Note: it is a relationship between word senses and NOT between words.\n",
        "Why?\n",
        "\n",
        "Example:\n",
        "- \"big plane and small plane\": in this context, \"big\" can be replaced by \"large\" because these two words both have a sense of \"significant size\", and in this sense you can use either \"big\" or \"large\" to describe a plane.\n",
        "- \"she is my big sister\" - here \"big\" is used in the sense \"older\" and cannot be replace by \"large\" because \"large\" does not have such a sense. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00014-39f8887f-c4a2-40b6-b110-a415343efb7b",
        "id": "GYPs3gqEK2Lt"
      },
      "source": [
        "**Antonyms** (words with opposite meaning: binary opposition/opposite ends of a scale) (Long/short, Black/white)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00015-050a93e9-1217-4939-a29f-71f9a806b822",
        "id": "WlTS1PoSK2Lt"
      },
      "source": [
        "**Hyponyms (subordinate)** (one word is a specific case (subclass) of the other) (car/vehicle, mango/fruit).\n",
        "\n",
        "**Hypernym (superordinate)** (one word denotes a class to which the other word belongs) (animal is a hypernym of dog, fruit is a hypernym of mango, and animal is a hypernym of dog). "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00016-139484f4-5a6d-40ac-ae1f-6718b0724e5f",
        "id": "eDlLa698K2Lt"
      },
      "source": [
        "**Meronymy** - part-whole relationship (foot/leg)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00016-f4650e32-d6de-4cb3-8bb1-852518bedc06",
        "id": "U0ZVC_BoK2Lu"
      },
      "source": [
        "**Metonymy** - using one aspect of a concept or entity to refer to other aspects of the entity or to the entity itself (\"I love Charles Dickens\" (meaning \"I like books by Charles Dickens\"))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00017-71872c65-4bcd-47bc-9f8d-ce4b18e0efdb",
        "id": "PQVZiq1XK2Lu"
      },
      "source": [
        "## WordNet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00018-1b25eb0e-35c8-4a3d-9381-c5748124162a",
        "id": "2sRKpsfxK2Lu"
      },
      "source": [
        "Enough linguistics! Let's move on to NLP."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00019-d150f573-581a-4ec2-9979-d73dc1bf8c1d",
        "id": "lVi3a3vkK2Lu"
      },
      "source": [
        "**WordNet** (https://wordnet.princeton.edu/) is the most commonly used **resource for sense relations** in English and many other languages.\n",
        "It is similar to a traditional **thesaurus** (dictionary of synonyms) but has a richer structure.\n",
        "The set of near-synonyms for a WordNet sense is called a **synset (for synonym\n",
        "set)**. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00020-ac30024a-3842-4afc-8cc4-0603e2e48001",
        "id": "rjuOpcoBK2Lv"
      },
      "source": [
        "Let's play with it a little. Enter your favorite word (do you have one?) and check out its entries in WordNet. http://wordnetweb.princeton.edu/perl/webwn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00022-5b280b5e-a27e-492e-b039-b74616782bd7",
        "id": "DNgPkFsVK2Lv"
      },
      "source": [
        "WordNet is included in NLTK. Below you will find many code examples to analyse word relations. Feel free to replace example words with your own to check your intuitions and have some fun. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "16d64022",
        "execution_millis": 2661,
        "cell_id": "00023-e81a9711-6755-4518-ae95-332ba1da9ebd",
        "execution_start": 1604411358918,
        "id": "FiFMc0_KK2Lw"
      },
      "source": [
        "!pip install nltk\n",
        "import nltk\n",
        "nltk.download('wordnet')\n",
        "from nltk.corpus import wordnet as wn"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00024-f77de7d4-96a6-42d4-a25d-5bcecfab2723",
        "id": "VZ0AQzQ7K2Lw"
      },
      "source": [
        "Some words have many senses:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "13ac2074",
        "execution_millis": 25,
        "execution_start": 1604329026440,
        "cell_id": "00025-0c02c83d-8b66-45d4-bf08-6416a30575b5",
        "id": "Tz4tvzbeK2Lx"
      },
      "source": [
        "wn.synsets(\"bank\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00026-b8ca901d-0ac7-4207-9ad7-0485038427d1",
        "id": "8uIa8AkCK2Lx"
      },
      "source": [
        "Some words only have one sense:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "859cbd2a",
        "execution_millis": 1,
        "cell_id": "00027-2ba5d384-dcb2-4602-81dd-880abf7040ab",
        "execution_start": 1604410542062,
        "id": "ZSjQHmGPK2Lx"
      },
      "source": [
        "wn.synsets('siege')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00028-51fe4c84-d53f-45c5-8750-e52e1152e4a7",
        "id": "1ygdo_YlK2Ly"
      },
      "source": [
        "You can restrict search by POS:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "321eb845",
        "execution_millis": 1,
        "execution_start": 1604329046330,
        "cell_id": "00029-e251e39c-26e1-4351-9d99-6b3882421b20",
        "id": "nHxTWCCjK2Ly"
      },
      "source": [
        "wn.synsets('dog', pos='v')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00030-04e28a8b-7c83-4beb-8427-e7eec20725c9",
        "id": "3p3P_p_GK2Ly"
      },
      "source": [
        "**Lemmas** are words in a synset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "3bee328f",
        "execution_millis": 7,
        "execution_start": 1604329084518,
        "cell_id": "00031-0a1fa9f5-ad7c-4e8c-9227-fed56df640f3",
        "id": "gNii0DtAK2Lz"
      },
      "source": [
        "wn.synset('car.n.01').lemmas() "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00032-f70be64f-16e2-4242-a4ec-58960879164d",
        "id": "NJQim5LhK2Lz"
      },
      "source": [
        "We can get a **list of lemmas** for a word sense:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "887dc89",
        "execution_millis": 7,
        "cell_id": "00033-7950acf0-a7e8-445e-aaa1-160567f79977",
        "execution_start": 1604410729071,
        "id": "fPf_UtgyK2L0"
      },
      "source": [
        "wn.synset('car.n.01').lemma_names()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00034-37970fc0-7a70-4b44-bd76-0db41f254481",
        "id": "-4sq1Qw2K2L0"
      },
      "source": [
        "or for all senses of a word at once:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "5cfbfb0f",
        "execution_millis": 5,
        "execution_start": 1604329099781,
        "cell_id": "00035-dc1c6e6d-86c2-4050-9228-4e59c3aec237",
        "id": "PqXTARwIK2L1"
      },
      "source": [
        "for synset in wn.synsets('car'):\n",
        "    print(synset.lemma_names())"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00036-62efe0d4-c695-45d2-a38b-3173dd0a3dee",
        "id": "aEuJI3-wK2L1"
      },
      "source": [
        "NOTE: To explore WordNet in languages other than English, we can use [ISO-639 language codes](http://www.loc.gov/standards/iso639-2/php/code_list.php). \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "e36cdd1b",
        "execution_millis": 1047,
        "cell_id": "00037-a3ea25f0-cc63-4d01-863e-b92282ce4eb4",
        "execution_start": 1604410745006,
        "id": "M7x7l9MgK2L1"
      },
      "source": [
        "nltk.download('omw')\n",
        "sorted(wn.langs())"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00037-bef14e71-b649-48a1-95ca-b5682cc30bed",
        "id": "bHZk-OqAK2L2"
      },
      "source": [
        "Let's try French (and then some other languages if you want):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "f4bf7f3c",
        "execution_millis": 6,
        "cell_id": "00038-d9bbe56c-5087-48fd-837f-de153b8d9b34",
        "execution_start": 1604410780975,
        "id": "0pcfMAdCK2L2"
      },
      "source": [
        "wn.synset('dog.n.01').lemma_names('fra')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00039-34099d07-6988-4f1a-a94c-39ad5d06e99b",
        "id": "WZGj_7JyK2L2"
      },
      "source": [
        "We can get a sense **definition**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "893ac0f0",
        "execution_millis": 3,
        "cell_id": "00040-f23cc4fe-53d6-4a8b-95d8-e964fc62a765",
        "execution_start": 1604410846513,
        "id": "YSMXFofiK2L3"
      },
      "source": [
        "wn.synset('moose.n.01').definition() "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00041-80e6f341-e59f-4424-a1b3-fe8790e2385b",
        "id": "4VnM4W9vK2L3"
      },
      "source": [
        "and **examples**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "2c5afcd4",
        "execution_millis": 3,
        "cell_id": "00042-9eb03064-4e95-470f-98b1-089c1e9cb3a6",
        "execution_start": 1604413676619,
        "id": "kL96uZJUK2L4"
      },
      "source": [
        "wn.synset('deer.n.01').examples()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00043-265000a7-d260-465c-8d6e-ea45b64a415f",
        "id": "Og6oXttfK2L4"
      },
      "source": [
        "**CODE IT (1):** define a function that take a word as an input and returns a definiton and examples for each of its word senses. Try it on a word with many senses and/or with part of speech ambiguity (to make it more interesting)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "3c9a664",
        "execution_millis": 3,
        "cell_id": "00045-39e6dd1a-3e96-40d2-8949-11782919375a",
        "execution_start": 1604410932737,
        "id": "1Afb2mLnK2L4"
      },
      "source": [
        "# your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00046-da30b07a-d233-45a0-8373-4526ccac86dd",
        "id": "Xp6Q4IGGK2L4"
      },
      "source": [
        "### WordNet hierarchy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00047-6bd7f249-fa33-4bad-b2d1-62f0e8450951",
        "id": "h7NHb3FDK2L5"
      },
      "source": [
        "There is a certain hierarchy on conepts in WordNet. Some concepts are very general, such as Entity, State, Event — these are called **unique beginners** or **root synsets**. \n",
        "\n",
        "![picture](https://www.researchgate.net/profile/Bastian_Entrup/publication/321671739/figure/tbl3/AS:668725552361484@1536448004654/List-of-25-unique-beginners-in-the-noun-set-of-WordNet.png)\n",
        "(Source: https://www.researchgate.net/figure/List-of-25-unique-beginners-in-the-noun-set-of-WordNet_tbl3_321671739)\n",
        "\n",
        "Others, such as gas guzzler and hatchback, are much more specific. A small portion of a concept hierarchy is illustrated below:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00048-32010c52-de1d-4013-90c4-e959ce402351",
        "id": "2HfjdJGmK2L5"
      },
      "source": [
        "![picture](https://www.nltk.org/images/wordnet-hierarchy.png)\n",
        "(Image (and text above) source: https://www.nltk.org/book/ch02.html)\n",
        "*Fragment of WordNet Concept Hierarchy: nodes correspond to synsets; edges indicate the hypernym/hyponym relation, i.e. the relation between superordinate and subordinate concepts*.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00049-34828e98-2f26-47c2-9b60-1def471c5be2",
        "id": "wLe3NrubK2L5"
      },
      "source": [
        "Let's use WordNet to navigate between concepts:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "1b5adb6c",
        "execution_start": 1603991484681,
        "execution_millis": 3,
        "cell_id": "00050-b8b0912c-7fc2-43e8-8fb9-519c916538e4",
        "id": "PQQd-baNK2L5"
      },
      "source": [
        "wn.synsets('car')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "f7b70bb4",
        "execution_millis": 2,
        "execution_start": 1603991578395,
        "cell_id": "00051-afad52e0-8349-456a-91c7-03be6a542216",
        "id": "HvHr9Et5K2L6"
      },
      "source": [
        "# get a synset for the first sense of the word \"car\"\n",
        "motorcar = wn.synset('car.n.01') \n",
        "# get hyponyms (more narrow terms belonging to the more broad concept of a car)\n",
        "types_of_motorcar = motorcar.hyponyms()\n",
        "# what types of cars are there in WordNet?\n",
        "sorted(lemma.name() for synset in types_of_motorcar for lemma in synset.lemmas())"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00052-9e48b9f0-0d4d-4439-a4cb-de9ade6b409e",
        "id": "MYoelys-K2L6"
      },
      "source": [
        "**CODE IT (2)**: edit the code below to find hyponyms of a word sense of your choice. Be creative!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "3c9a664",
        "execution_millis": 1,
        "cell_id": "00053-6369805f-9255-45a1-95a2-d888e113b212",
        "execution_start": 1604410988527,
        "id": "E1NmYrAlK2L6"
      },
      "source": [
        "# your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00054-afb828d8-b008-4330-ab9f-00e81fdf8368",
        "id": "7Vq09YhAK2L6"
      },
      "source": [
        "We can also explore the hierarchy by checking out **hypernyms** (superordinates, or more general \"umbrella terms\").\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "6edb2a23",
        "execution_millis": 1,
        "execution_start": 1603984400394,
        "cell_id": "00055-c32444d9-18a4-4c7a-ada5-9efff6970e42",
        "id": "R0lKSmzdK2L7"
      },
      "source": [
        "motorcar.hypernyms()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00056-97cd94e8-b07e-49b9-98ae-b52ab653bbf9",
        "id": "dszV0sgRK2L7"
      },
      "source": [
        "We can get the **most general hypernyms (or root hypernyms)** of a synset as follows:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "cc1f8f4c",
        "execution_start": 1603984476024,
        "execution_millis": 41,
        "cell_id": "00057-fcac5ba7-0f9c-4c8c-8cc4-0c3b78dc95e9",
        "id": "S0N1t1BIK2L7"
      },
      "source": [
        "motorcar.root_hypernyms()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00058-8b0a3e4c-e031-44c5-b0f9-44c7e60ff53e",
        "id": "mxLlO-9WK2L7"
      },
      "source": [
        "Hypernyms and hyponyms are called **lexical relations**  -  they relate one synset to another. These two relations navigate up and down the \"is-a\" hierarchy. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00059-80cb6508-83d3-4f88-a217-daf357786e61",
        "id": "q9CXMRZwK2L7"
      },
      "source": [
        "Another way to navigate WordNet is from items to their components (**meronyms**) or to the things they are contained in (**holonyms**).\n",
        "\n",
        "A **meronym** represents a component of a larger whole. It is a vast relationship, and NLTK divides the meronym category into part-representing whole (part_meronyms()) and substance-representing whole (substance_meronyms())."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "9454f51",
        "execution_start": 1603984795480,
        "execution_millis": 4,
        "cell_id": "00060-6c4272b0-2a5c-49f1-a933-495562f442a4",
        "id": "DH6ga-EhK2L8"
      },
      "source": [
        "# what parts does a tree consist of?\n",
        "wn.synset('tree.n.01').part_meronyms()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "6332f52e",
        "execution_millis": 0,
        "execution_start": 1603984984542,
        "cell_id": "00061-f99086ec-869d-4eae-a071-b968d8b4c86d",
        "id": "YKaO_i_-K2L8"
      },
      "source": [
        "# what substance does ice consist of?\n",
        "wn.synset('ice.n.01').substance_meronyms()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "b8181db7",
        "execution_start": 1603993777902,
        "execution_millis": 52,
        "cell_id": "00062-4a102eec-60c4-497c-afce-43d412d4f1d7",
        "id": "l5nFnkRQK2L8"
      },
      "source": [
        "wn.synset('kitchen.n.01').part_holonyms() \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "e9edcd81",
        "execution_millis": 0,
        "execution_start": 1603993804624,
        "cell_id": "00063-f62cf24e-c025-4848-b270-442d5d9213a0",
        "id": "hnxh090TK2L8"
      },
      "source": [
        "wn.synset('dog.n.01').member_holonyms() \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "5d302e48",
        "execution_start": 1603993835960,
        "execution_millis": 11,
        "cell_id": "00064-cb222739-77f7-4fdd-917a-5663eabd70b9",
        "id": "icc4QPoqK2L8"
      },
      "source": [
        "wn.synset('tree.n.01').member_holonyms()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "cc9494fb",
        "execution_millis": 4,
        "execution_start": 1603985017523,
        "cell_id": "00065-c1c6a762-88c0-4a97-8467-7bf17b52cd88",
        "id": "wHOcBglvK2L9"
      },
      "source": [
        "wn.synset('ice.n.01').substance_holonyms()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00066-dbb9748d-560d-4259-bdc1-c193b7a2ecce",
        "id": "-jylsMcbK2L9"
      },
      "source": [
        "An **entailment** is an implication. Looking implies seeing. Buying implies choosing and paying."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00067-ef1a9644-bd94-45af-970d-c2e63f10d9bc",
        "id": "hh7T-lbqK2L9"
      },
      "source": [
        "Take the following sentence: \"She walked to her bed and in a few minutes she was snoring loudly”. What exactly is implied by \"snoring\" here? Is it important that she was snoring, or is it just that she was sleeping soundly? Entailment analysis can help investigate it.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "8dd84f37",
        "execution_millis": 0,
        "execution_start": 1604332192421,
        "cell_id": "00068-618f8949-38e0-49ac-bb92-bf4c6deb18c2",
        "id": "Il8RBhbiK2L9"
      },
      "source": [
        "wn.synset('snore.v.01').entailments()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00069-649b513a-d7a8-4a9a-a3ce-0be28c253cea",
        "id": "3eUu78N1K2L-"
      },
      "source": [
        "Some other examples of verb entailment relationship:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "478a67ea",
        "execution_start": 1603985115206,
        "execution_millis": 2,
        "cell_id": "00070-14846c36-17e1-4243-9d88-ae54176f4fe1",
        "id": "C72ksBytK2L-"
      },
      "source": [
        "wn.synset('eat.v.01').entailments()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "e8cca271",
        "execution_millis": 14,
        "execution_start": 1604331722513,
        "cell_id": "00071-9b5acb65-66c4-4bee-8362-2330d3df0a2a",
        "id": "3tupE_ZlK2L-"
      },
      "source": [
        "wn.synset('buy.v.01').entailments()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00072-24bdbd5f-f3a9-455b-85fd-116c23629f18",
        "id": "S3PtUKqaK2L-"
      },
      "source": [
        "Some lexical relationships hold between lemmas, e.g., **antonymy**:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "d0afd08e",
        "execution_millis": 41,
        "execution_start": 1604334477554,
        "cell_id": "00073-6b3c2e07-2f5b-43e6-b0af-c934e059638c",
        "id": "_fgmoM54K2L_"
      },
      "source": [
        "wn.lemma('supply.n.02.supply').antonyms()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "b290a204",
        "execution_millis": 8,
        "execution_start": 1604334481034,
        "cell_id": "00074-38050ad2-82d7-4f4a-970a-896d650cba0e",
        "id": "m3HcSa9CK2L_"
      },
      "source": [
        "wn.lemma('horizontal.a.01.horizontal').antonyms()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00075-0deac3a0-6e48-4159-b871-77705572fed0",
        "id": "stWQZUSTK2MA"
      },
      "source": [
        "### Semantic similarity in WordNet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00076-2a250d74-b087-40fc-b420-0cb615758ec5",
        "id": "S6L1te_CK2MA"
      },
      "source": [
        "Remember we measured semantic similarity with word embeddings? Those measurements were in the context of a specific corpus. WordNet allows us to measure semantic similarity between words based on their definitions and relations as defined in the WordNet itself.\n",
        "It is useful to know which words are semantially related to other words - for example, for indexing a corpus of texts (searching for information about dinosaurs? You may be interested in reptiles or Pterodactylus)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00077-b3700a76-125f-4fb1-a7bd-30a2b93fed61",
        "id": "OK40G8OtK2MA"
      },
      "source": [
        "Let's explore another example: **Right whales and minke whales** are species of large baleen whales. The **killer whale or orca**  is whale belonging to the oceanic dolphin family."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "e6295637",
        "execution_millis": 41,
        "execution_start": 1604334588632,
        "cell_id": "00078-99901a25-c8b2-49eb-8a84-ee4206df8858",
        "id": "XlhN9VDjK2MA"
      },
      "source": [
        "right_whale = wn.synset('right_whale.n.01')\n",
        "orca = wn.synset('orca.n.01')\n",
        "minke_whale = wn.synset('minke_whale.n.01')\n",
        "tortoise = wn.synset('tortoise.n.01')\n",
        "novel = wn.synset('novel.n.01')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00079-fcf1a3e1-1a5c-406e-b124-7d85b23759c5",
        "id": "Fvis7vVgK2MH"
      },
      "source": [
        "**lowest_common_hypernyms()** locates the lowest hypernym (the **broadest sense**) that is shared by two given words.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00080-4baf705d-0e0b-4f6c-8407-bc924c220200",
        "id": "9Mhr3AH7K2MI"
      },
      "source": [
        "Reminder: **hypernym**: a word with a broad meaning constituting a category into which words with more specific meanings fall (superordinate)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "3ae66d89",
        "execution_millis": 16,
        "execution_start": 1604334830220,
        "cell_id": "00081-aa82c47d-66b2-44f0-b41c-4483a3a546b9",
        "id": "MYCn58tMK2MJ"
      },
      "source": [
        "right_whale.lowest_common_hypernyms(minke_whale)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00082-1964f9bf-648d-467e-bcb8-a9775b559f8c",
        "id": "cADmWtZ8K2MJ"
      },
      "source": [
        "Indeed, right whale and minke are both baleen whales!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "c470263b",
        "execution_millis": 8,
        "execution_start": 1604334838281,
        "cell_id": "00083-e162cb0a-0b79-4843-858c-b343aa8907c4",
        "id": "2k8kSWeVK2MK"
      },
      "source": [
        "right_whale.lowest_common_hypernyms(orca)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00084-774021de-91a3-42ad-8fdc-f83ab223b692",
        "id": "vv1nggyTK2MK"
      },
      "source": [
        "Right whale and orca are both whales (but orca is not a baleen whale, so what they have in common is only being a (generic) whale)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "f7249716",
        "execution_millis": 15,
        "execution_start": 1604334847444,
        "cell_id": "00085-82f21894-c5e4-4554-8e23-15ae75d1dca7",
        "id": "xlr7i2UBK2ML"
      },
      "source": [
        "right_whale.lowest_common_hypernyms(tortoise)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00086-5c353171-a677-4c4b-904c-e3b442a99ec2",
        "id": "6DPtZxBTK2ML"
      },
      "source": [
        "Right whale and tortoise are both vertebrate."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "bbc6efee",
        "execution_millis": 38,
        "execution_start": 1604334853165,
        "cell_id": "00087-a95a477d-fc26-441f-abe2-e86d5dfdbe4f",
        "id": "1dc2-qovK2MM"
      },
      "source": [
        "right_whale.lowest_common_hypernyms(novel)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00088-420fac60-59c4-4c4c-ad9f-44017e5e5bef",
        "id": "2vKnqVE_K2MM"
      },
      "source": [
        "...and a right whale and a novel - what do they have in common? Nothing...excep that they are both entities (and not 'feelings' or 'shapes' - return to **unique beginners** if you have forgotten everything about them!)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00089-5ac90fe8-ee2f-489b-a002-a047d01efa44",
        "id": "led3QaRJK2MM"
      },
      "source": [
        "**CODE IT (3):** explore the relationship between three words senses of your choice (choose senses relate to each other in some interesting ways - for example, animal types from the same - and a different -  family etc.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "cell_id": "00090-6e0a556c-f421-40ab-a545-416c20361232",
        "id": "qTL4jdsoK2MN"
      },
      "source": [
        "#insert your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00091-d8a01374-37f7-47ca-8b8a-3185fb2576f2",
        "id": "MHfpMLo_K2MN"
      },
      "source": [
        "\"Whale\" is very specific (and \"baleen whale\" is very very very specific!), and \"vertebrate\" is more general. And \"entity\" is very very very general. \n",
        "We can quantify this **concept of generality** by defining the depth of each synset:\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "5f6889fe",
        "execution_start": 1603986270144,
        "execution_millis": 3,
        "cell_id": "00092-f9265c2b-965b-46a6-ab54-2b7eb0721734",
        "id": "zn9Qb797K2MN"
      },
      "source": [
        "wn.synset('baleen_whale.n.01').min_depth()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "c8815ddb",
        "execution_start": 1603986280265,
        "execution_millis": 3,
        "cell_id": "00093-d27a9537-a9b5-42f5-8e48-e5ffc0460868",
        "id": "1KBYOIoQK2MN"
      },
      "source": [
        "wn.synset('whale.n.02').min_depth()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "ebdf965f",
        "execution_start": 1603986297045,
        "execution_millis": 0,
        "cell_id": "00094-b68975d5-1255-4357-916f-16a03b74a56f",
        "id": "QHNa7BffK2MO"
      },
      "source": [
        "wn.synset('vertebrate.n.01').min_depth()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "d901f076",
        "execution_start": 1603986287142,
        "execution_millis": 4,
        "cell_id": "00095-cc22b176-5eb6-42a8-98c1-f51943683bdb",
        "id": "umYDoyBkK2MO"
      },
      "source": [
        "wn.synset('entity.n.01').min_depth()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00096-b9ebeb27-8a87-4c32-965d-bfac88485111",
        "id": "yr4GCdhVK2MO"
      },
      "source": [
        "**path_similarity** assigns a score (0-1) based on the shortest path that connects the concepts in the hypernym hierarchy.\n",
        "-1 means that a path cannot be found.\n",
        "You will receive 1 when comparing a synset with itself. \n",
        "Although the numbers themselved do not mean anything really, they decrease as we move away from the semantic space of sea creatures to inanimate objects.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "a44e8f71",
        "execution_millis": 217,
        "execution_start": 1604335453669,
        "cell_id": "00097-93d580d4-80b0-4122-adf6-830d9e745d45",
        "id": "q8mO7oGaK2MO"
      },
      "source": [
        "right_whale.path_similarity(minke_whale)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "568eeb8c",
        "execution_millis": 9,
        "execution_start": 1604335459517,
        "cell_id": "00098-e7ae9065-d0df-44cb-95ae-49b68a557826",
        "id": "2Cto_JXtK2MO"
      },
      "source": [
        "right_whale.path_similarity(orca)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "e188f68f",
        "execution_millis": 13,
        "execution_start": 1604335466358,
        "cell_id": "00099-e04eaa96-99d4-4291-8b05-685d54899e2f",
        "id": "fx16wqsPK2MP"
      },
      "source": [
        "right_whale.path_similarity(tortoise)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "eebdca9d",
        "execution_millis": 13,
        "execution_start": 1604335473301,
        "cell_id": "00100-88f8c288-ce10-436b-8880-a51bbc02443e",
        "id": "F3DWF9fCK2MP"
      },
      "source": [
        "right_whale.path_similarity(novel)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00101-d1703872-7b1f-490d-85b3-8b64b2f596cd",
        "id": "3FLxNa-bK2MP"
      },
      "source": [
        "**CODE IT (4) **: Use one of the predefined similarity measures to score the similarity of each of the following pairs of words. Rank the pairs in order of decreasing similarity. How close is your ranking to the order given here, an order that was established experimentally by (Miller & Charles, 1998): car-automobile, gem-jewel, journey-voyage, boy-lad, coast-shore, asylum-madhouse, magician-wizard, midday-noon, furnace-stove, food-fruit, bird-cock, bird-crane, tool-implement, brother-monk, lad-brother, crane-implement, journey-car, monk-oracle, cemetery-woodland, food-rooster, coast-hill, forest-graveyard, shore-woodland, monk-slave, coast-forest, lad-wizard, chord-smile, glass-magician, rooster-voyage, noon-string.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "cell_id": "00101-c2f45a80-3ed4-499c-a000-0c8b11d7a775",
        "id": "wNwB2xlwK2MQ"
      },
      "source": [
        "# your code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00103-540f0a3d-6643-4380-bc98-43d9350badaf",
        "id": "TmDITZN2K2MQ"
      },
      "source": [
        "## Word sense disambiguation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00104-5695c71f-70bc-47bc-868c-4d42dc4cae71",
        "id": "we5uYzM_K2MR"
      },
      "source": [
        "Somtimes categonal ambiguity (\"can\" - is it a verb or a noun?) can be\n",
        "resolved using syntactic aspects of it use (\"I can do it\" - \"can\" here is 100% a verb).\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00105-94f0f9c3-2129-4580-94ae-46c4d28960b2",
        "id": "WYSoNQl4K2MR"
      },
      "source": [
        "** Word sense disambiguation (WSD)** - selecting the correct sense for a word. \n",
        "WSD algorithms take as input a word in context and a fixed inventory of potential word senses and outputs the correct word sense in context."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00106-36d97503-cdcf-4a50-8703-fc481c512389",
        "id": "QffVCK_RK2MR"
      },
      "source": [
        "How does it work in general? \n",
        "When we have a small pre-selected set of target words and an inventory of senses for each word, in such a **lexical sample task** when the set of words and the\n",
        "set of senses are small, we can use simple supervised classification approaches.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00107-aedd0ec2-e19f-4fbe-9506-5e18fb9c217e",
        "id": "rcsq9OioK2MR"
      },
      "source": [
        "However, we often have to disambiguate **all words** in some text. \n",
        "In this **all-words task**, the input is an entire text and a lexicon with an inventory of senses for each entry. \n",
        "We then need to disambiguate every content word in the text. This is somewhat simiar to part-of-speech tagging (though the number of tags is much bigger).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00108-a3d6696a-311e-4454-9ec9-f5f9a7c02eb7",
        "id": "ou6HOVFgK2MS"
      },
      "source": [
        "**Supervised all-word disambiguation tasks** are usually trained from a **semantic\n",
        "concordance** -  a corpus in which each open-class word in each sentence is labeled\n",
        "with its word sense from a specific dictionary or thesaurus, most often WordNet.\n",
        "\n",
        "The **SemCor corpus** is a subset of the Brown Corpus where \n",
        "words were manually tagged with WordNet senses. There are other similar corpora too. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00109-efd32087-65ea-4c8b-8c55-f66062353ccc",
        "id": "zSdn4-IYK2MS"
      },
      "source": [
        "Given each noun, verb, adjective, or adverb word in the hand-labeled test set (say\n",
        "fruit), the SemCor-based WSD task is to choose the correct sense from the possible\n",
        "senses in WordNet. For fruit this would mean choosing between the correct answer\n",
        "fruit1 (the ripened reproductive body of a seed plant), and the other two senses fruit2\n",
        "(yield; an amount of a product) and fruit3\n",
        "(the consequence of some effort or action).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00110-e6295697-015e-4af5-9cea-057efb80c257",
        "id": "rrH9HD3iK2MS"
      },
      "source": [
        "![picture] (https://www.researchgate.net/profile/Devendra_Chaplot/publication/322328811/figure/fig2/AS:631613809496087@1527599875540/An-example-of-the-all-word-WSD-task-Content-words-and-their-possible-senses-are-labeled.png)\n",
        "(Image source: https://www.researchgate.net/figure/An-example-of-the-all-word-WSD-task-Content-words-and-their-possible-senses-are-labeled_fig2_322328811)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00111-41ec1304-97e7-4819-ab3a-17fea40c92ba",
        "id": "8sYkjsozK2MS"
      },
      "source": [
        "A surprisingly strong baseline is simply to choose the most frequent sense for most frequent\n",
        "sense\n",
        "each word from the senses in a labeled corpus (For WordNet, this\n",
        "corresponds to the first sense). A second heuristic, called **one sense per discourse** is based on the work of one sense per\n",
        "discourse who noticed that a word appearing multiple times in a text or\n",
        "discourse often appears with the same sense."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00112-1b7aa1dd-c90e-45d4-b713-18162b1b123e",
        "id": "1RRdiFpuK2MT"
      },
      "source": [
        "## Lesk Algorithm"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00113-b5312c4b-550d-49cf-a393-826cbb62f51e",
        "id": "1-z6Lz3TK2MT"
      },
      "source": [
        "Generating sense labeled corpora like SemCor is difficult and expensive. An\n",
        "alternative class of WSD algorithms - **knowledge-based algorithms**, rely solely on\n",
        "WordNet or other similar resources and do not need labeled data. \n",
        "\n",
        "**Lesk algorithm**  - the oldest and most powerful knowledge-based WSD\n",
        "method, and is a useful baseline. It basically chooses the sense whose dictionary gloss or definition shares the most words with the target\n",
        "word’s neighborhood. It assumes that words in a given neighborhood (section of text) tend to share a common topic. \n",
        "A simplified version of the Lesk algorithm is to compare the dictionary definition of an ambiguous word with the terms contained in its neighbourhood.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "1b3fc799",
        "execution_millis": 1273,
        "cell_id": "00114-5992af19-ca23-4d84-9041-eb3a1f28a865",
        "execution_start": 1604411142470,
        "id": "qYIX1BlmK2MT"
      },
      "source": [
        "from nltk.wsd import lesk"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00115-efc7c96d-3af8-4acc-9f39-080b915519c4",
        "id": "v-28KLBQK2MT"
      },
      "source": [
        "NLTK's Lesk arguments:\n",
        "- context_sentence (a list of words)\n",
        "- ambiguous word (word requiring WSD)\n",
        "- pos - specified part of speech (default = None)\n",
        "\n",
        "The POS tagset here includes 'a' = adjective/adverbs, 's'=satelite adjective, 'n' = nouns and 'v' = verbs.\n",
        "\n",
        "\n",
        "- synsets - possible synsets of the ambiguous word (default=None)\n",
        "\n",
        "It returns a synset for an ambiguous word in a context. Sometimes it will fail...\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "ccd9caa1",
        "execution_start": 1604339315093,
        "execution_millis": 42,
        "cell_id": "00116-78a468e9-2af2-495f-baf9-bb546cc37369",
        "id": "JuPbPHy7K2MT"
      },
      "source": [
        "lesk(['I', 'went', 'to', 'the', 'bank', 'to', 'deposit', 'money', '.'], 'bank', 'n')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "edefa743",
        "execution_millis": 13,
        "execution_start": 1604339332950,
        "cell_id": "00117-0e172997-e3ca-479a-baa4-a73c178241dc",
        "id": "W47XwVgrK2MT"
      },
      "source": [
        "wn.synset('savings_bank.n.02').definition()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "9bf44eec",
        "execution_start": 1603989484129,
        "execution_millis": 5,
        "cell_id": "00118-e511f859-7be7-4657-b467-46f840991e40",
        "id": "Ys_Kgy6hK2MU"
      },
      "source": [
        "lesk(['I','have', 'put','an', 'ice' 'cream','cone', 'in','the', 'fridge'], 'cone')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "6589330d",
        "execution_millis": 6,
        "execution_start": 1604339346216,
        "cell_id": "00119-3bdca3c2-760b-48f8-b8d8-1598c2e0cf1e",
        "id": "irPmCz-KK2MU"
      },
      "source": [
        "wn.synset('cone.n.04').definition()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00120-41694d98-1ce1-4f0c-93ee-8ffe183e19c6",
        "id": "NTzJKongK2MU"
      },
      "source": [
        "**CODE IT (5)** Consider a word which can have different meanings in a language you know well, using the implementation of Lesk algorithm, use example sentences to derive the several definitations of that word.\n",
        "\n",
        "For instance:\n",
        "\n",
        "**terminal**\n",
        "\n",
        "    1.  a point on an electrical device at which electric current enters or leaves.\n",
        "\n",
        "    2.  where transport vehicles load or unload passengers or goods.\n",
        "    \n",
        "    3.  an input-output device providing access to a computer.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "cell_id": "00121-085524e2-8935-48ce-ae04-0c232fe68a52",
        "id": "2DWe8rWUK2MU"
      },
      "source": [
        "#insert your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00122-43047176-628d-4ff1-b240-790fdab031de",
        "id": "_W769W4MK2MV"
      },
      "source": [
        "## WSD with Contextual Embeddings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00123-b2d014b9-6f70-4f96-87ff-9f8f428d714f",
        "id": "3LhIzVCxK2MV"
      },
      "source": [
        "Do you remember what Word Embeddings are?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "61d93d37",
        "execution_start": 1604404599689,
        "execution_millis": 120806,
        "cell_id": "00125-f923e81b-f707-441a-a5cf-2fcaf6ca955e",
        "id": "twmdN5fLK2MV"
      },
      "source": [
        "!pip install gensim\n",
        "import gensim.downloader\n",
        "\n",
        "# Show all available models in gensim-data\n",
        "\n",
        "print(list(gensim.downloader.info()['models'].keys()))\n",
        "['fasttext-wiki-news-subwords-300',\n",
        " 'conceptnet-numberbatch-17-06-300',\n",
        " 'word2vec-ruscorpora-300',\n",
        " 'word2vec-google-news-300',\n",
        " 'glove-wiki-gigaword-50',\n",
        " 'glove-wiki-gigaword-100',\n",
        " 'glove-wiki-gigaword-200',\n",
        " 'glove-wiki-gigaword-300',\n",
        " 'glove-twitter-25',\n",
        " 'glove-twitter-50',\n",
        " 'glove-twitter-100',\n",
        " 'glove-twitter-200',\n",
        " '__testing_word2vec-matrix-synopsis']\n",
        "\n",
        "\n",
        "# Download the \"glove-twitter-25\" embeddings\n",
        "\n",
        "model = gensim.downloader.load('glove-twitter-25')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deepnote_cell_type": "code",
        "tags": [],
        "output_cleared": false,
        "source_hash": "bbfad8ce",
        "execution_millis": 2,
        "execution_start": 1604404921639,
        "cell_id": "00126-0c54ddda-eb69-446b-a713-9912535d4e4d",
        "id": "0c8jqm3ZK2MV"
      },
      "source": [
        "model['mouse'] "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00127-969131da-73a6-4b12-a009-14484a9afb2f",
        "id": "EGIA7QgUK2MV"
      },
      "source": [
        "A word embedding is a numerical representation of a word learnt from huge amounts of data.\n",
        "\n",
        "Each word in a vector space language model is represented by a single vector.\n",
        "\n",
        "**But** what about **homonyms**? \n",
        "\n",
        "Consider the word \"mouse\": \n",
        "\n",
        "What if we had a word embedding for each of the different meanings of **mouse**?\n",
        "![](http://ai.stanford.edu/blog/assets/img/posts/2020-03-24-contextual/contextual_mouse_transparent_1.png)\n",
        "![](http://ai.stanford.edu/blog/assets/img/posts/2020-03-24-contextual/contextual_mouse_transparent_2.png)\n",
        "\n",
        "(Image source: (http://ai.stanford.edu/blog/)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00128-aba2fdd0-af59-4270-b760-faff5f5f7072",
        "id": "sGuh9paCK2MW"
      },
      "source": [
        "Here is where **Contextualized word embedding models** enter in NLP world.\n",
        "![Elmo](https://media.giphy.com/media/yr7n0u3qzO9nG/giphy.gif)![BertUrl](https://media.giphy.com/media/umMYB9u0rpJyE/giphy.gif)\n",
        "\n",
        "[ELMO](https://arxiv.org/pdf/1802.05365.pdf)(Embeddings from Language Models) and [BERT](https://arxiv.org/pdf/1810.04805.pdf)(Bidirectional Transformers for Language Understanding) are two of the most famous and most commonly used contextualized word embeddings.\n",
        "\n",
        "Training these models require huge amounts of data and processing resources(including time and hardware).\n",
        "\n",
        "\n",
        "\n",
        "In layman's terms contextualized word embeddings map each word to a vector considering the context.\n",
        "![Elmo-mouse](https://miro.medium.com/max/1000/1*3mxK9XhkOQstPbrRfRT2vQ.png)\n",
        "(Image source: https://medium.com/@leslie_huang/automatic-extraction-of-word-senses-from-deep-contextualized-word-embeddings-2f09f16e820)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deepnote_cell_type": "markdown",
        "tags": [],
        "cell_id": "00129-f8d4b709-1b91-49cb-bc39-4521eae82e05",
        "id": "LF5xoNTRK2MW"
      },
      "source": [
        "**Homework Exercise** Do a quick research on the task of **textual entailment** in NLP and summarize it using your own words(at least 100 words). \n",
        "\n",
        "What are textual entailment relations(entails, contradicts, non-TE)?\n",
        "\n",
        "Using only wordnet functions such as entails or antonyms, can you suggest a simple algoithm that can detect textual entailment relations? \n",
        "\n",
        "\n",
        "\n"
      ]
    }
  ]
}