USS_QA_IR_Tutorial.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/UCREL/Session_2_Question-Answering-Information-Retrieval/blob/main/USS_QA_IR_Tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ErVy6A2NisJ"
      },
      "source": [
        " # UCREL SUMMER SCHOOL USS 2024\n",
        "\n",
        " ## Session 2: Tutorial on QA and IR\n",
        "\n",
        "- **Goal**: After completing this tutorial, you'll have learned how to build QA-IR pipelines that uses extractive and generative models alongside retrievers to answer given questions.\n",
        "\n",
        "> This tutorial uses Haystack 2.0. To learn more, visit the [Haystack 2.0 Documentation](https://docs.haystack.deepset.ai/docs/intro).\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uClfTB7jN6g-"
      },
      "source": [
        "## Extractive Question Answering\n",
        "\n",
        "What is extractive question answering? The short answer is that extractive models pull verbatim answers out of text. It's good for use cases where accuracy is paramount, and you need to know exactly where in the text that the answer came from.\n",
        "\n",
        "In this tutorial you'll create a pipeline that extracts answers to questions, based on the provided documents.\n",
        "\n",
        "To get data into the extractive pipeline, you'll also build an indexing pipeline to ingest the [Wikipedia pages of Seven Wonders of the Ancient World dataset](https://en.wikipedia.org/wiki/Wonders_of_the_World)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eF_hnatJUEHq"
      },
      "source": [
        "## Preparing the Colab Environment\n",
        "\n",
        "- Login with your Google account\n",
        "- Enable GPU Runtime in Colab (T4 is ok as well)\n",
        "- Read and run the notebook cell by cell"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zQnSZtyyUJVF"
      },
      "source": [
        "#Installation\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rwgpwV4eHVoo"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "pip install -q haystack-ai accelerate deepeval-haystack \"sentence-transformers>=2.2.0\" \"datasets>=2.6.1\" &> /dev/null"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Complete Pipeline"
      ],
      "metadata": {
        "id": "q3ZgdUwEsa7M"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U8I641xobh_w"
      },
      "source": [
        "## Load data into the `DocumentStore`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b2HaHlBrSvLa"
      },
      "source": [
        "Before you can use this data in the extractive pipeline, you'll use an indexing pipeline to fetch it, process it, and load it into the document store.\n",
        "\n",
        "\n",
        "The data has already been cleaned and preprocessed, so turning it into Haystack `Documents` is fairly straightfoward.\n",
        "\n",
        "Using an `InMemoryDocumentStore` here keeps things simple. However, this general approach would work with [any document store that Haystack 2.0 supports](https://docs.haystack.deepset.ai/docs/document-store).\n",
        "\n",
        "The `SentenceTransformersDocumentEmbedder` transforms each `Document` into a vector. Here we've used [`sentence-transformers/multi-qa-mpnet-base-dot-v1`](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1). You can substitute any embedding model you like, as long as you use the same one in your extractive pipeline.\n",
        "\n",
        "Lastly, the `DocumentWriter` writes the vectorized documents to the `DocumentStore`.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ttuq7kLtaV5b"
      },
      "outputs": [],
      "source": [
        "from datasets import load_dataset\n",
        "from haystack import Document\n",
        "from haystack import Pipeline\n",
        "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
        "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n",
        "from haystack.components.readers import ExtractiveReader\n",
        "from haystack.components.embedders import SentenceTransformersDocumentEmbedder\n",
        "from haystack.components.writers import DocumentWriter\n",
        "\n",
        "\n",
        "dataset = load_dataset(\"bilgeyucel/seven-wonders\", split=\"train\")\n",
        "\n",
        "documents = [Document(content=doc[\"content\"], meta=doc[\"meta\"]) for doc in dataset]\n",
        "\n",
        "model = \"sentence-transformers/multi-qa-mpnet-base-dot-v1\"\n",
        "\n",
        "document_store = InMemoryDocumentStore()\n",
        "\n",
        "indexing_pipeline = Pipeline()\n",
        "\n",
        "indexing_pipeline.add_component(instance=SentenceTransformersDocumentEmbedder(model=model), name=\"embedder\")\n",
        "indexing_pipeline.add_component(instance=DocumentWriter(document_store=document_store), name=\"writer\")\n",
        "indexing_pipeline.connect(\"embedder.documents\", \"writer.documents\")\n",
        "\n",
        "indexing_pipeline.run({\"documents\": documents})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r5CL5VXaVQqE"
      },
      "source": [
        "## Build an Extractive QA Pipeline\n",
        "\n",
        "Your extractive QA pipeline will consist of three components: an embedder, retriever, and reader.\n",
        "\n",
        "- The `SentenceTransformersTextEmbedder` turns a query into a vector, usaing the same embedding model defined above.\n",
        "\n",
        "- Vector search allows the retriever to efficiently return relevant documents from the document store. Retrievers are tightly coupled with document stores; thus, you'll use an `InMemoryEmbeddingRetriever`to go with the `InMemoryDocumentStore`.\n",
        "\n",
        "- The `ExtractiveReader` returns answers to that query, as well as their location in the source document, and a confidence score.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xZGGv8yaHZtV"
      },
      "outputs": [],
      "source": [
        "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n",
        "from haystack.components.readers import ExtractiveReader\n",
        "from haystack.components.embedders import SentenceTransformersTextEmbedder\n",
        "\n",
        "\n",
        "retriever = InMemoryEmbeddingRetriever(document_store=document_store)\n",
        "reader = ExtractiveReader()\n",
        "reader.warm_up()\n",
        "\n",
        "extractive_qa_pipeline = Pipeline()\n",
        "\n",
        "extractive_qa_pipeline.add_component(instance=SentenceTransformersTextEmbedder(model=model), name=\"embedder\")\n",
        "extractive_qa_pipeline.add_component(instance=retriever, name=\"retriever\")\n",
        "extractive_qa_pipeline.add_component(instance=reader, name=\"reader\")\n",
        "\n",
        "extractive_qa_pipeline.connect(\"embedder.embedding\", \"retriever.query_embedding\")\n",
        "extractive_qa_pipeline.connect(\"retriever.documents\", \"reader.documents\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yxi6IYXPZMFw"
      },
      "source": [
        "Try extracting some answers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "mvG3XJOtZR79",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "fe78e2fc-ea80-4acb-f8ca-147a31ab7158"
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'extractive_qa_pipeline' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-1-b4ab9983d959>\u001b[0m in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mquery\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Who was Pliny the Elder?\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m output = extractive_qa_pipeline.run(\n\u001b[0m\u001b[1;32m      3\u001b[0m     \u001b[0mdata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m\"embedder\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m\"text\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mquery\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"retriever\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m\"top_k\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;36m3\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"reader\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m\"query\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mquery\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"top_k\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m )\n",
            "\u001b[0;31mNameError\u001b[0m: name 'extractive_qa_pipeline' is not defined"
          ]
        }
      ],
      "source": [
        "query = \"Who was Pliny the Elder?\"\n",
        "output = extractive_qa_pipeline.run(\n",
        "    data={\"embedder\": {\"text\": query}, \"retriever\": {\"top_k\": 3}, \"reader\": {\"query\": query, \"top_k\": 2}}\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "for i,answer in enumerate(output['reader']['answers']):\n",
        "  a = answer.to_dict()['init_parameters']['data']\n",
        "  print(f\"Answer {i}: {a}\")\n",
        "  if a:\n",
        "    c = answer.to_dict()['init_parameters']['document']['content'].replace('\\n',' ')\n",
        "    print(f\"Context {i}: {c}\")\n",
        "    print(f\"Score {i}: {answer.to_dict()['init_parameters']['score']}\\n\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8_IFo0yOmVib",
        "outputId": "18568777-2f35-4315-fa5a-921a8b511686"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Answer 0: Roman writer\n",
            "Context 0: The Roman writer Pliny the Elder, writing in the first century AD, argued that the Great Pyramid had been raised, either \"to prevent the lower classes from remaining unoccupied\", or as a measure to prevent the pharaoh's riches from falling into the hands of his rivals or successors.[60] Pliny does not speculate as to the pharaoh in question, explicitly noting that \"accident [has] consigned to oblivion the names of those who erected such stupendous memorials of their vanity\".[61] In pondering how the stones could be transported to such a vast height he gives two explanations: That either vast mounds of nitre and salt were heaped up against the pyramid which were then melted away with water redirected from the river. Or, that \"bridges\" were constructed, their bricks afterwards distributed for erecting houses of private individuals, arguing that the level of the river is too low for canals to ever bring water up to the pyramid. Pliny also recounts how \"in the interior of the largest Pyramid there is a well, eighty-six cubits [45.1 m; 147.8 ft] deep, which communicates with the river, it is thought\". \n",
            "Score 0: 0.8306006193161011\n",
            "\n",
            "Answer 1: a Roman author\n",
            "Context 1: [21] Pliny the Elder (AD 23/24 – 79) was a Roman author, a naturalist and natural philosopher, a naval and army commander of the early Roman Empire, and a friend of emperor Vespasian. Pliny wrote the encyclopedic Naturalis Historia (Natural History), which became an editorial model for encyclopedias. The Naturalis Historia is one of the largest single works to have survived from the Roman Empire to the modern day and purports to cover the entire field of ancient knowledge. Pliny remarked:  But that which is by far the most worthy of our admiration, is the colossal statue of the Sun, which stood formerly at Rhodes, and was the work of Chares the Lindian, a pupil of the above-named Lysippus; no less than seventy cubits in height. This statue fifty-six years after it was erected, was thrown down by an earthquake; but even as it lies, it excites our wonder and admiration. Few men can clasp the thumb in their arms, and its fingers are larger than most statues. Where the limbs are broken asunder, vast caverns are seen yawning in the interior. \n",
            "Score 1: 0.7280883193016052\n",
            "\n",
            "Answer 2: None\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GOKWgMDCWGRd"
      },
      "source": [
        "## `ExtractiveReader`: a closer look\n",
        "\n",
        "Here's an example answer:\n",
        "```python\n",
        "[ExtractedAnswer(query='Who was Pliny the Elder?', score=0.8306006193161011, data='Roman writer', document=Document(id=bb2c5f3d2e2e2bf28d599c7b686ab47ba10fbc13c07279e612d8632af81e5d71, content: 'The Roman writer Pliny the Elder, writing in the first century AD, argued that the Great Pyramid had...', meta: {'url': 'https://en.wikipedia.org/wiki/Great_Pyramid_of_Giza', '_split_id': 16}\n",
        "```\n",
        "\n",
        "The confidence score ranges from 0 to 1. Higher scores mean the model has more confidence in the answer's relevance.\n",
        "\n",
        "The Reader sorts the answers based on their probability scores, with higher probability listed first. You can limit the number of answers the Reader returns in the optional `top_k` parameter.\n",
        "\n",
        "By default, the Reader sets a `no_answer=True` parameter. This param returns an `ExtractedAnswer` with no text, and the probability that none of the returned answers are correct.\n",
        "\n",
        "```python\n",
        "ExtractedAnswer(query='Who was Pliny the Elder?', score=0.04606167031102615, data=None, document=None, context=None, document_offset=None, context_offset=None, meta={})]}}\n",
        "```\n",
        "\n",
        "`.0.04606167031102615` means the model is fairly confident the provided answers are correct in this case. You can disable this behavior and return only answers by setting the `no_answer` param to `False` when initializing your `ExtractiveReader`.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Sentence Level Retrieval"
      ],
      "metadata": {
        "id": "xHlSWl_Eytul"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from haystack import Document, Pipeline\n",
        "from haystack.components.retrievers.in_memory import InMemoryBM25Retriever\n",
        "from haystack.components.retrievers import SentenceWindowRetrieval\n",
        "from haystack.components.preprocessors import DocumentSplitter\n",
        "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
        "\n",
        "splitter = DocumentSplitter(split_by=\"word\", split_length=2)\n",
        "text = (\n",
        "        \"This is a text with some words. There is a second sentence. And there is also a third sentence. \"\n",
        "        \"It also contains a fourth sentence. And a fifth sentence. And a sixth sentence. And a seventh sentence\"\n",
        ")\n",
        "doc = Document(content=text)\n",
        "docs = splitter.run([doc])\n",
        "doc_store = InMemoryDocumentStore()\n",
        "doc_store.write_documents(docs[\"documents\"])\n",
        "\n",
        "\n",
        "rag = Pipeline()\n",
        "rag.add_component(\"bm25_retriever\", InMemoryBM25Retriever(doc_store, top_k=1))\n",
        "rag.add_component(\"sentence_window_retriever\", SentenceWindowRetrieval(document_store=doc_store, window_size=1))\n",
        "rag.connect(\"bm25_retriever\", \"sentence_window_retriever\")\n",
        "\n",
        "output = rag.run({'bm25_retriever': {\"query\":\"third\"}})\n",
        "print(output['sentence_window_retriever']['context_windows'][0])"
      ],
      "metadata": {
        "id": "FMry_5ZpyzX9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "560ee681-3156-4428-cd0b-82f1aa20c90c"
      },
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "is alsoa thirdsentence. It \n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Reranking Pipeline"
      ],
      "metadata": {
        "id": "iFufeq6rp-P_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from haystack import Pipeline\n",
        "from haystack.components.retrievers import InMemoryBM25Retriever\n",
        "from haystack.components.rankers import TransformersSimilarityRanker\n",
        "\n",
        "# Assuming 'document_store' is already defined and populated with your dataset\n",
        "\n",
        "retriever = InMemoryBM25Retriever(document_store=document_store)\n",
        "reranker = TransformersSimilarityRanker(model=\"cross-encoder/ms-marco-MiniLM-L-6-v2\")  # Replace with your desired cross-encoder model\n",
        "\n",
        "reranking_pipeline = Pipeline()\n",
        "reranking_pipeline.add_component(instance=retriever, name=\"retriever\")\n",
        "reranking_pipeline.add_component(instance=reranker, name=\"ranker\")\n",
        "reranking_pipeline.connect(\"retriever.documents\", \"ranker.documents\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Qz3FH3Sfp9h6",
        "outputId": "b231225b-ca28-4401-ebac-81b89204dee5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<haystack.core.pipeline.pipeline.Pipeline object at 0x7f1e96383c70>\n",
              "🚅 Components\n",
              "  - retriever: InMemoryBM25Retriever\n",
              "  - ranker: TransformersSimilarityRanker\n",
              "🛤️ Connections\n",
              "  - retriever.documents -> ranker.documents (List[Document])"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"When was Rhodes raided?\"\n",
        "output = reranking_pipeline.run(data={\"retriever\": {\"query\": query, \"top_k\": 3},\n",
        "                                   \"ranker\": {\"query\": query, \"top_k\": 2}})"
      ],
      "metadata": {
        "id": "8mY_IPGdvfEE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for i, document in enumerate(output['ranker']['documents']):\n",
        "  d = document.to_dict()\n",
        "  print(f\"Document {i}: {d['content']}\")\n",
        "  print(f\"Link {i}: {d['url']}\")\n",
        "  print(f\"Score {i}: {d['score']}\\n\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CfQlPOHZwOxd",
        "outputId": "71e627df-5e9b-4c90-d2b0-6ca941b9fc48"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Document 0: In 653, an Arab force under Muslim general Muawiyah I raided Rhodes, and according to the Chronicle of Theophanes the Confessor,[7] the remains of the statue constituted part of the booty, being melted down and sold to a Jewish merchant of Edessa who loaded the bronze onto 900 camels.[8] The same story is recorded by Bar Hebraeus, writing in Syriac in the 13th century in Edessa[25] (after the Arab pillage of Rhodes): \"And a great number of men hauled on strong ropes which were tied around the brass Colossus which was in the city and pulled it down. And they weighed from it three thousand loads of Corinthian brass, and they sold it to a certain Jew from Emesa\" (the Syrian city of Homs).[26]\n",
            "Ultimately, Theophanes is the sole source of this account, and all other sources can be traced to him.[27] As Theophanes' source was Syriac, it may have had vague information about a raid and attributed the statue's demise to it, not knowing much more. Or the Arab destruction and the purported sale to a Jew may have originated as a powerful metaphor for Nebuchadnezzar II's dream of the destruction of a great statue.\n",
            "Link 0: https://en.wikipedia.org/wiki/Colossus_of_Rhodes\n",
            "Score 0: 0.9932199716567993\n",
            "\n",
            "Document 1: It was untouched when the city fell to Alexander the Great in 334 BC and still undamaged after attacks by pirates in 62 and 58 BC. It stood above the city's ruins for sixteen centuries. Then a series of earthquakes shattered the columns and sent the bronze chariot crashing to the ground. By AD 1404, only the base of the Mausoleum was still recognizable.\n",
            "The Knights of St John of Rhodes invaded the region and built Bodrum Castle (Castle of Saint Peter). When they decided to fortify it in 1494, they used the stones of the Mausoleum. This is also about when \"imaginative reconstructions\" of the Mausoleum began to appear.[25] In 1522, rumours of a Turkish invasion caused the Crusaders to strengthen the castle at Halicarnassus (which was by then known as Bodrum) and much of the remaining portions of the tomb were broken up and used in the castle walls. Sections of polished marble from the tomb can still be seen there today. Suleiman the Magnificent conquered the base of the knights on the island of Rhodes, who then relocated first briefly to Sicily and later permanently to Malta, leaving the Castle and Bodrum to the Ottoman Empire.\n",
            "Link 1: https://en.wikipedia.org/wiki/Mausoleum_at_Halicarnassus\n",
            "Score 1: 0.23899805545806885\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's add a reader to the pipline"
      ],
      "metadata": {
        "id": "A0lGMzwbwwQ8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "reranking_pipeline = Pipeline()\n",
        "\n",
        "retriever = InMemoryBM25Retriever(document_store=document_store)\n",
        "reranker = TransformersSimilarityRanker(model=\"cross-encoder/ms-marco-MiniLM-L-6-v2\")  # Replace with your desired cross-encoder model\n",
        "reader = ExtractiveReader()\n",
        "reader.warm_up()\n",
        "\n",
        "reranking_pipeline.add_component(instance=retriever, name=\"retriever\")\n",
        "reranking_pipeline.add_component(instance=reranker, name=\"ranker\")\n",
        "reranking_pipeline.add_component(instance=reader, name=\"reader\")\n",
        "reranking_pipeline.connect(\"retriever.documents\", \"ranker.documents\")\n",
        "reranking_pipeline.connect(\"ranker.documents\", \"reader.documents\")\n",
        "\n",
        "query = \"When was Rhodes raided?\"\n",
        "output = reranking_pipeline.run(data={\"retriever\": {\"query\": query, \"top_k\": 3},\n",
        "                                   \"ranker\": {\"query\": query, \"top_k\": 2},\n",
        "                                    \"reader\": {\"query\": query, \"top_k\": 2}})"
      ],
      "metadata": {
        "id": "myJV90xcwzrD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for i,answer in enumerate(output['reader']['answers']):\n",
        "  a = answer.to_dict()['init_parameters']['data']\n",
        "  print(f\"Answer {i}: {a}\")\n",
        "  if a:\n",
        "    c = answer.to_dict()['init_parameters']['document']['content'].replace('\\n',' ')\n",
        "    print(f\"Context {i}: {c}\")\n",
        "    print(f\"Score {i}: {answer.to_dict()['init_parameters']['score']}\\n\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SeaingntxW9M",
        "outputId": "ea72bf1e-55ca-49ae-f17a-287cc44c6302"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Answer 0: 653\n",
            "Context 0: In 653, an Arab force under Muslim general Muawiyah I raided Rhodes, and according to the Chronicle of Theophanes the Confessor,[7] the remains of the statue constituted part of the booty, being melted down and sold to a Jewish merchant of Edessa who loaded the bronze onto 900 camels.[8] The same story is recorded by Bar Hebraeus, writing in Syriac in the 13th century in Edessa[25] (after the Arab pillage of Rhodes): \"And a great number of men hauled on strong ropes which were tied around the brass Colossus which was in the city and pulled it down. And they weighed from it three thousand loads of Corinthian brass, and they sold it to a certain Jew from Emesa\" (the Syrian city of Homs).[26] Ultimately, Theophanes is the sole source of this account, and all other sources can be traced to him.[27] As Theophanes' source was Syriac, it may have had vague information about a raid and attributed the statue's demise to it, not knowing much more. Or the Arab destruction and the purported sale to a Jew may have originated as a powerful metaphor for Nebuchadnezzar II's dream of the destruction of a great statue.\n",
            "Score 0: 0.8582978844642639\n",
            "\n",
            "Answer 1: 62 and 58 BC. It stood above the city's ruins for sixteen centuries. Then a series of earthquakes shattered the columns and sent the bronze chariot crashing to the ground. By AD 1404\n",
            "Context 1: It was untouched when the city fell to Alexander the Great in 334 BC and still undamaged after attacks by pirates in 62 and 58 BC. It stood above the city's ruins for sixteen centuries. Then a series of earthquakes shattered the columns and sent the bronze chariot crashing to the ground. By AD 1404, only the base of the Mausoleum was still recognizable. The Knights of St John of Rhodes invaded the region and built Bodrum Castle (Castle of Saint Peter). When they decided to fortify it in 1494, they used the stones of the Mausoleum. This is also about when \"imaginative reconstructions\" of the Mausoleum began to appear.[25] In 1522, rumours of a Turkish invasion caused the Crusaders to strengthen the castle at Halicarnassus (which was by then known as Bodrum) and much of the remaining portions of the tomb were broken up and used in the castle walls. Sections of polished marble from the tomb can still be seen there today. Suleiman the Magnificent conquered the base of the knights on the island of Rhodes, who then relocated first briefly to Sicily and later permanently to Malta, leaving the Castle and Bodrum to the Ottoman Empire.\n",
            "Score 1: 0.3340127766132355\n",
            "\n",
            "Answer 2: None\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Generative QA"
      ],
      "metadata": {
        "id": "IXqHPreqyQZN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Large Language Models are best suited for Generative QA\n",
        "In this example we will use Flan-T5-Large for computational limits"
      ],
      "metadata": {
        "id": "YdJARYUW2M62"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from haystack import Pipeline\n",
        "from haystack.components.retrievers.in_memory import InMemoryBM25Retriever\n",
        "from haystack.components.builders.prompt_builder import PromptBuilder\n",
        "from haystack.components.generators import HuggingFaceLocalGenerator\n",
        "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
        "from haystack import Document\n",
        "\n",
        "generator = HuggingFaceLocalGenerator(\n",
        "    model=\"google/flan-t5-large\",\n",
        "    task=\"text2text-generation\",\n",
        "    generation_kwargs={\"max_new_tokens\": 100, \"temperature\": 0.9})\n",
        "\n",
        "generator.warm_up()\n",
        "\n",
        "\n",
        "query = \"When was Rhodes raided?\"\n",
        "\n",
        "template = \"\"\"\n",
        "Given the following information, answer the question.\n",
        "\n",
        "Context:\n",
        "{% for document in documents %}\n",
        "    {{ document.content }}\n",
        "{% endfor %}\n",
        "\n",
        "Question: {{ query }}?\n",
        "\"\"\"\n",
        "pipe = Pipeline()\n",
        "\n",
        "pipe.add_component(\"retriever\", InMemoryBM25Retriever(document_store=document_store))\n",
        "pipe.add_component(\"prompt_builder\", PromptBuilder(template=template))\n",
        "pipe.add_component(\"llm\", generator)\n",
        "pipe.connect(\"retriever\", \"prompt_builder.documents\")\n",
        "pipe.connect(\"prompt_builder\", \"llm\")\n",
        "\n",
        "res=pipe.run({\n",
        "    \"prompt_builder\": {\n",
        "        \"query\": query\n",
        "    },\n",
        "    \"retriever\": {\n",
        "        \"query\": query\n",
        "    }\n",
        "})"
      ],
      "metadata": {
        "id": "-T_4v8qHwuF-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a23c5934-17a6-4f82-e25a-c82d4476c665"
      },
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Token indices sequence length is longer than the specified maximum sequence length for this model (2760 > 512). Running this sequence through the model will result in indexing errors\n",
            "/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(res['llm']['replies'][0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lbTQEbQX2K8E",
        "outputId": "224cb4a6-82d8-4f26-d876-862fc3f6a84c"
      },
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "In 653\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### API Call (FREE LLMs)\n",
        "\n",
        "[Login](https://huggingface.co/login) to your HuggingFace account. ([Create](https://huggingface.co/join) one if you dont have it already)\n",
        "\n",
        "[Generate a new API key](https://huggingface.co/settings/tokens/new) of READ type. Copy and Paste it in the code below.\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "e0s001wO3y01"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from haystack.components.generators import HuggingFaceAPIGenerator\n",
        "from haystack.utils import Secret\n",
        "generator = HuggingFaceAPIGenerator(api_type=\"serverless_inference_api\",\n",
        "                                    api_params={\"model\": \"HuggingFaceH4/zephyr-7b-beta\"},\n",
        "                                    token=Secret.from_token(\"your_key_here\"))"
      ],
      "metadata": {
        "id": "v5RWys8X3x9i"
      },
      "execution_count": 96,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "you can use other models e.g., mistralai/Mistral-7B-v0.1\n",
        "\n",
        "for Mistral and other models you need to go to the [model page](https://huggingface.co/mistralai/Mistral-7B-v0.1) and agree with the terms of use before using it.\n"
      ],
      "metadata": {
        "id": "ZEpGrQSG60vf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"When was Rhodes raided?\"\n",
        "\n",
        "template = \"\"\"\n",
        "Given the following information, answer the question.\n",
        "\n",
        "Context:\n",
        "{% for document in documents %}\n",
        "    {{ document.content }}\n",
        "{% endfor %}\n",
        "\n",
        "Question: {{ query }}?\n",
        "\"\"\"\n",
        "pipe = Pipeline()\n",
        "\n",
        "pipe.add_component(\"retriever\", InMemoryBM25Retriever(document_store=document_store))\n",
        "pipe.add_component(\"prompt_builder\", PromptBuilder(template=template))\n",
        "pipe.add_component(\"llm\", generator)\n",
        "pipe.connect(\"retriever\", \"prompt_builder.documents\")\n",
        "pipe.connect(\"prompt_builder\", \"llm\")\n",
        "\n",
        "res=pipe.run({\n",
        "    \"prompt_builder\": {\n",
        "        \"query\": query\n",
        "    },\n",
        "    \"retriever\": {\n",
        "        \"query\": query\n",
        "    }\n",
        "})"
      ],
      "metadata": {
        "id": "V2L-NEIs4NMc"
      },
      "execution_count": 97,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(res['llm']['replies'][0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GtMlet-k4Upm",
        "outputId": "ef58e8af-e67f-4afd-9dbb-7c0d1296b9b9"
      },
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "Answer: In 653, an Arab force under Muslim general Muawiyah I raided Rhodes.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Chatbot"
      ],
      "metadata": {
        "id": "hHfjtvK39VI-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "find a dataset on huggingface that you want to query"
      ],
      "metadata": {
        "id": "1aMwf4rEC15_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Chatbot interface using reranking and generative QA\n",
        "# @\n",
        "def get_answer(dataset,column,split,query):\n",
        "\n",
        "  dataset = load_dataset(dataset, split=split)\n",
        "\n",
        "  documents = [Document(content=doc[column]) for doc in dataset]\n",
        "\n",
        "  document_store = InMemoryDocumentStore()\n",
        "\n",
        "  indexing_pipeline = Pipeline()\n",
        "\n",
        "  indexing_pipeline.add_component(instance=SentenceTransformersDocumentEmbedder(model=\"sentence-transformers/multi-qa-mpnet-base-dot-v1\"), name=\"embedder\")\n",
        "  indexing_pipeline.add_component(instance=DocumentWriter(document_store=document_store), name=\"writer\")\n",
        "  indexing_pipeline.connect(\"embedder.documents\", \"writer.documents\")\n",
        "\n",
        "  indexing_pipeline.run({\"documents\": documents})\n",
        "\n",
        "  if \"generator\" not in globals():\n",
        "      generator = HuggingFaceLocalGenerator(\n",
        "          model=\"google/flan-t5-large\",\n",
        "          task=\"text2text-generation\",\n",
        "          generation_kwargs={\"max_new_tokens\": 100, \"temperature\": 0.9})\n",
        "\n",
        "      generator.warm_up()\n",
        "\n",
        "  template = \"\"\"\n",
        "  Given the following information, answer the question.\n",
        "\n",
        "  Context:\n",
        "  {% for document in documents %}\n",
        "      {{ document.content }}\n",
        "  {% endfor %}\n",
        "\n",
        "  Question: {{ query }}?\n",
        "  \"\"\"\n",
        "  pipe = Pipeline()\n",
        "\n",
        "  pipe.add_component(\"retriever\", InMemoryBM25Retriever(document_store=document_store))\n",
        "  pipe.add_component(\"ranker\", TransformersSimilarityRanker(model=\"cross-encoder/ms-marco-MiniLM-L-6-v2\"))\n",
        "  pipe.add_component(\"prompt_builder\", PromptBuilder(template=template))\n",
        "  pipe.add_component(\"llm\", generator)\n",
        "  pipe.connect(\"retriever.documents\", \"ranker.documents\")\n",
        "  pipe.connect(\"ranker\", \"prompt_builder.documents\")\n",
        "  pipe.connect(\"prompt_builder\", \"llm\")\n",
        "\n",
        "  res=pipe.run({\n",
        "      \"prompt_builder\": {\n",
        "          \"query\": query\n",
        "      },\n",
        "      \"retriever\": {\n",
        "          \"query\": query, \"top_k\": 3\n",
        "      },\n",
        "      \"ranker\": {\"query\": query, \"top_k\": 1}\n",
        "  })\n",
        "\n",
        "  return res['llm']['replies'][0]\n",
        "\n",
        "dataset = \"bilgeyucel/seven-wonders\" # @param {type:\"string\"}\n",
        "\n",
        "column_name = \"content\" # @param {type:\"string\"}\n",
        "split = \"train\" # @param {type:\"string\"}\n",
        "query = \"When was Rhodes raided?\" # @param {type:\"string\"}\n",
        "\n",
        "\n",
        "print(get_answer(dataset,column_name,split,query))\n"
      ],
      "metadata": {
        "cellView": "form",
        "id": "XekGhaR49U7P"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# TASK: Bring Your knowledge base and build your Chatbot\n",
        "Add PDF files to your colab space and query them"
      ],
      "metadata": {
        "id": "E4xY6dhf1dl9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "file_names=['file1.pdf','file2.pdf','file3.pdf']"
      ],
      "metadata": {
        "id": "lrT9lFJP1tdV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from haystack import Pipeline\n",
        "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
        "from haystack.components.converters import PDFMinerToDocument\n",
        "from haystack.components.preprocessors import DocumentCleaner\n",
        "from haystack.components.preprocessors import DocumentSplitter\n",
        "from haystack.components.writers import DocumentWriter\n",
        "\n",
        "document_store = InMemoryDocumentStore()\n",
        "\n",
        "pipeline = Pipeline()\n",
        "pipeline.add_component(\"converter\", PDFMinerToDocument())\n",
        "pipeline.add_component(\"cleaner\", DocumentCleaner())\n",
        "pipeline.add_component(\"splitter\", DocumentSplitter(split_by=\"sentence\", split_length=5))\n",
        "pipeline.add_component(\"writer\", DocumentWriter(document_store=document_store))\n",
        "pipeline.connect(\"converter\", \"cleaner\")\n",
        "pipeline.connect(\"cleaner\", \"splitter\")\n",
        "pipeline.connect(\"splitter\", \"writer\")\n",
        "\n",
        "pipeline.run({\"converter\": {\"sources\": file_names}})"
      ],
      "metadata": {
        "id": "W8YYla9P1dMF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#\n",
        "# COMPLETE THIS\n",
        "# BY ADDING YOUR\n",
        "# QA PIPELINE\n",
        "#"
      ],
      "metadata": {
        "id": "0ZoX9bN2yO9l"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}