notebooks/quick_start.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quick start: Using Jack models\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "**Note:** these commands need to be run in terminal from the root of Jack.\n",
    "\n",
    "Download GloVe [[1]](#ref1) vectors:\n",
    "> `sh data/GloVe/download.sh`\n",
    "\n",
    "Download pretrained FastQA [[2]](#ref2) and DAM [[3]](#ref2) models:\n",
    "> `wget -O fastqa.zip https://www.dropbox.com/s/qb796uljoqj0lvo/fastqa.zip?dl=1`\n",
    "\n",
    "> `wget -O dam.zip http://data.neuralnoise.com/jack/natural_language_inference/dam.zip`\n",
    "\n",
    "Prepare the model for use:\n",
    "> `unzip fastqa.zip`\n",
    "\n",
    "> `unzip dam.zip`\n",
    "\n",
    "First, let's get the imports sorted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "import os\n",
    "os.chdir('..')    # change dir to Jack root"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from jack import readers\n",
    "from jack.core import QASetting\n",
    "from jack.io.load import load_jack\n",
    "from notebooks.prettyprint import QAPrettyPrint, print_nli"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usecase: Question Answering (QA)\n",
    "\n",
    "Load the (previously downloaded) pretrained FastQA [[2]](#ref2) model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./fastqa/model_module\n"
     ]
    }
   ],
   "source": [
    "fastqa_reader = readers.reader_from_file(\"./fastqa\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define a reading comprehension _paragraph_ and a _question_ from the SQuAD [[4]](#ref4) corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "paragraph = \"\"\"It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. \n",
    "At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.\"\"\"\n",
    "\n",
    "question = \"To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We merge them into a single `QASetting` data structure. This structure requires a _question_ and a _list of supporting documents_:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "qa_setting = QASetting(question=question, support=[paragraph])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feed the `qa_setting` (paragraph and the question) structure into the reader to get the _answers_:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "answers = fastqa_reader([qa_setting])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The answer can be found here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Saint Bernadette Soubirous'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "answers[0][0].text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...together with the answer span, which we use to highlight the answer in the text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to <span style=\"background-color: #ff00ff; color: white\">Saint Bernadette Soubirous</span> in 1858. <br>At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary."
      ],
      "text/plain": [
       "<notebooks.prettyprint.QAPrettyPrint at 0x10d03cd30>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "QAPrettyPrint(paragraph, answers[0][0].span)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...and the score of the answer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9918955"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "answers[0][0].score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also predict the top _k_ answers of our model instead of just the best scoring one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer 0:   Saint Bernadette Soubirous \t (score: 0.99190)\n",
      "Answer 1:   Saint Bernadette Soubirous \t (score: 0.99190)\n",
      "Answer 2:   Bernadette Soubirous \t (score: 0.00798)\n",
      "Answer 3:   Saint Bernadette \t (score: 0.00006)\n",
      "Answer 4:   Soubirous \t (score: 0.00003)\n",
      "Answer 5:   Saint Bernadette Soubirous in \t (score: 0.00002)\n",
      "Answer 6:   Saint Bernadette Soubirous in 1858 \t (score: 0.00000)\n",
      "Answer 7:   to Saint Bernadette Soubirous \t (score: 0.00000)\n",
      "Answer 8:   Saint Bernadette Soubirous in 1858. \t (score: 0.00000)\n",
      "Answer 9:   Saint \t (score: 0.00000)\n"
     ]
    }
   ],
   "source": [
    "top_k = 10\n",
    "fastqa_reader.model_module.set_topk(top_k)\n",
    "answers = fastqa_reader([qa_setting])\n",
    "for i, a in enumerate(answers[0]):\n",
    "    print(\"Answer %d:   %s \\t (score: %.5f)\" % (i, a.text, a.score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usecase: Natural Language Inference (NLI)\n",
    "\n",
    "We first load a pretrained DAM [[3]](#ref3) model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./dam/model_module\n"
     ]
    }
   ],
   "source": [
    "dam_reader = readers.reader_from_file(\"./dam\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and next some Natural Language Inference examples from the SNLI corpus [[5]](#ref5):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "premise = \"A wedding party is taking pictures.\"\n",
    "hypothesis1 = \"A group of people is celebrating.\"\n",
    "hypothesis2 = \"A rock band is giving a concert.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the NLI case, the answer is a label among {_\"entailment\"_, _\"neutral\"_, _\"contradiction\"_}.\n",
    "\n",
    "We again use the same `QASetting` data structure as above to feed Jack the input entailment data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "snli_setting1 = QASetting(question=hypothesis1, support=[premise])\n",
    "snli_setting2 = QASetting(question=hypothesis2, support=[premise])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We generate predictions by calling the reader with these inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A wedding party is taking pictures.\t--(entailment)-->\tA group of people is celebrating.\n",
      "A wedding party is taking pictures.\t--(contradiction)-->\tA rock band is giving a concert.\n"
     ]
    }
   ],
   "source": [
    "prediction = dam_reader([snli_setting1])\n",
    "print_nli(premise, hypothesis1, prediction[0][0].text)\n",
    "\n",
    "prediction = dam_reader([snli_setting2])\n",
    "print_nli(premise, hypothesis2, prediction[0][0].text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...and we can again also inspect prediction scores:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.99598\n"
     ]
    }
   ],
   "source": [
    "print(prediction[0][0].score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References:\n",
    "\n",
    "<a id='ref1'>[1]</a> Jeffrey Pennington, Richard Socher, and Christopher Manning. <a href='http://www.aclweb.org/anthology/D14-1162'>\"Glove: Global vectors for word representation.\"</a> Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014.\n",
    "\n",
    "<a id='ref2'>[2]</a> Dirk Weissenborn, Georg Wiese, and Laura Seiffe. <a href='http://www.aclweb.org/anthology/K17-1028'>\"Making neural qa as simple as possible but not simpler.\"</a> Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL). 2017.</a>\n",
    "\n",
    "<a id='ref3'>[3]</a> Ankur Parikh, Oscar Täckström, Dipanjan Das, Jakob Uszkoreit . <a href='http://www.aclweb.org/anthology/D14-1162'>\"A Decomposable Attention Model for Natural Language Inference.\"</a> Proceedings of the 2016 conference on empirical methods in natural language processing (EMNLP). 2016. \n",
    "\n",
    "\n",
    "<a id='ref4'>[4]</a> Pranav Rajpurkar, et al. <a href='http://www.anthology.aclweb.org/D/D16/D16-1264.pdf'>\"SQuAD: 100,000+ Questions for Machine Comprehension of Text.\"</a> Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2016.\n",
    "\n",
    "\n",
    "<a id='ref5'>[5]</a> Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. <a href='http://www.anthology.aclweb.org/D/D16/D16-1264.pdf'>\"A large annotated corpus for learning natural language inference.\"</a> In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2015.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}