graph-builder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "19ae6874-7585-4a0b-848c-965f639def41",
   "metadata": {},
   "source": [
    "# Graph Builder\n",
    "In this notebook, let's explore how to leverage generative AI to build and consume a knowledge graph in Neo4j.\n",
    "\n",
    "## Setup\n",
    "First, let's install the libraries we're going to need for this lab and the following notebook dependent labs.  We'll also want to reboot the kernel once done.  To do that, go to the \"Kernel\" menu and click \"Restart Kernel and Clear All Outputs.\"  That will get rid of everything the install statements printed, leaving us with a cleaner notebook to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4aad4e36-7048-4bbf-a71a-a5d1b24d0ade",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#%pip install --user langchain\n",
    "#%pip install --user neo4j"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dd7c539-68b5-4126-990e-60c85b84fafa",
   "metadata": {},
   "source": [
    "Now restart the kernel. That will allow the Python evironment to import the new packages."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a05b8fb7-fce9-492c-b40b-1b0beff31775",
   "metadata": {},
   "source": [
    "## Start Neo4j\n",
    "We're going to run Neo4j in Docker.  You can do so by running this command in the terminal.  It will start Neo4j with APOC enabled.  We need APOC for Langchain to work.\n",
    "\n",
    "To do -- figure a way to make this run in the notebook.  Background isn't supported.  I'm not sure how screen could do this either."
   ]
  },
  {
   "cell_type": "raw",
   "id": "6360e1d2-64ac-4f46-a6fd-c5c965f55f13",
   "metadata": {
    "tags": []
   },
   "source": [
    "docker run \\\n",
    "    --restart always \\\n",
    "    --publish=7474:7474 --publish=7687:7687 \\\n",
    "    --env NEO4J_dbms_security_procedures_unrestricted=gds.*,apoc.* \\\n",
    "    --env NEO4J_dbms_security_procedures_allowlist=gds.*,apoc.* \\\n",
    "    --env NEO4J_AUTH=neo4j/mypassword \\\n",
    "    -e NEO4J_apoc_export_file_enabled=true \\\n",
    "    -e NEO4J_apoc_import_file_enabled=true \\\n",
    "    -e NEO4J_apoc_import_file_use__neo4j__config=true \\\n",
    "    -e NEO4J_PLUGINS=\\[\\\"apoc\\\"\\] \\\n",
    "    neo4j:5.15.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c40856f2-43bf-4d0c-9530-713015e4d76d",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Parse Data\n",
    "Now we're going to use an LLM to parse a document and create a graph in Neo4j."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f1ebc836-fc60-4c57-981c-75263d9bb892",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "CYPHER_GENERATION_TEMPLATE = \"\"\"You are an expert Neo4j Cypher translator who understands the English and converts it to Cypher strictly based on the Neo4j Schema provided and following the instructions below:\n",
    "1. Only generate Cypher queries that are compatible with Neo4j Version 5.\n",
    "2. Do not use EXISTS, SIZE keywords in the cypher. Use alias when using the WITH keyword.\n",
    "3. Do not use same variable names for different nodes and relationships in the query.\n",
    "4. Always enclose the Cypher output inside 3 backticks\n",
    "5. Always do a case-insensitive and fuzzy search for any properties related search. Eg: to search for a Company name use `toLower(c.name) contains 'neo4j'`\n",
    "6. Candidate node is synonymous to Manager\n",
    "7. Always use aliases to refer the node in the query\n",
    "8. 'Answer' is not a Cypher keyword. Answer should never be used in a query.\n",
    "9. Generate only one Cypher query per question. \n",
    "10. Cypher is not SQL. So, do not mix and match the syntaxes.\n",
    "11. Every Cypher query always starts with a MERGE keyword.\n",
    "\n",
    "Schema:\n",
    "{schema}\n",
    "Samples:\n",
    "Question: Jeff took his dog, Lassie, for a walk.\n",
    "Answer: MERGE (Jeff:Person)\n",
    "\n",
    "Question: {question}\n",
    "Answer: \n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02486049-121b-4fad-b44f-ded37259c932",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.prompts.prompt import PromptTemplate\n",
    "\n",
    "CYPHER_GENERATION_PROMPT = PromptTemplate(\n",
    "    input_variables=['schema','question'], validate_template=True, template=CYPHER_GENERATION_TEMPLATE\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f648c7da-9504-4c10-8358-6095d447e14a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "NEO4J_USERNAME = 'neo4j'\n",
    "NEO4J_PASSWORD = 'mypassword'\n",
    "NEO4J_URI = 'neo4j://localhost'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe4a115c-b70e-4c3b-bf9a-4021bcea82f7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.graphs import Neo4jGraph\n",
    "\n",
    "graph = Neo4jGraph(\n",
    "    url=NEO4J_URI, \n",
    "    username=NEO4J_USERNAME, \n",
    "    password=NEO4J_PASSWORD\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a971ec7-2752-4789-a5e9-4a58cbcd624f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.chains import GraphCypherQAChain\n",
    "from langchain.llms import VertexAI\n",
    "\n",
    "chain = GraphCypherQAChain.from_llm(\n",
    "    graph=graph,\n",
    "    cypher_llm=VertexAI(model_name='code-bison@001', max_output_tokens=2048, temperature=0.0),\n",
    "    qa_llm=VertexAI(model_name='text-bison', max_output_tokens=2048, temperature=0.0),\n",
    "    cypher_prompt=CYPHER_GENERATION_PROMPT,\n",
    "    verbose=True,\n",
    "    return_intermediate_steps=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a239a35f-2794-4ff3-8b6c-f103e6411a7a",
   "metadata": {
    "tags": []
   },
   "source": [
    "## A Simple Example\n",
    "First let's start with a simple example.  We'll take a graph from a single sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3666927c-17a3-40f0-b5b7-ab595bd1c145",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "chain(\"\"\"Jeff took his dog, Lassie, for a walk.\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e151a288-a870-45dc-874b-71d6b1ae6b5a",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Important Note\n",
    "None of the examples below work yet...\n",
    "\n",
    "## Alice in Wonderland\n",
    "Let's see what kind of graph the text of Alice in Wonderland contains!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37f592b3-f73f-4389-aeec-1021ecd9c7ab",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#from urllib.request import urlopen\n",
    "\n",
    "#url = 'https://www.gutenberg.org/cache/epub/11/pg11.txt'\n",
    "#text = urlopen(url).read().decode('utf-8')\n",
    "#chain(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82ff5acd-8bb0-4c96-be55-6c414dd9609b",
   "metadata": {},
   "source": [
    "## Romeo and Juliet\n",
    "This one is tragic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7089891d-34ed-4177-bb7b-8a848015a4d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "#url = 'https://www.gutenberg.org/cache/epub/1513/pg1513.txt'\n",
    "#text = urlopen(url).read().decode('utf-8')\n",
    "#chain(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "428bf256-f369-41e5-8f01-e8c4fb73bd42",
   "metadata": {},
   "source": [
    "## Ford Earnings Transcript\n",
    "This is an earnings transcript from Ford."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "129a2ce0-59e2-40b0-b894-d4210d309f4d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#url = 'https://s201.q4cdn.com/693218008/files/doc_events/2023/Oct/26/ford-q3-earnings-transcript.pdf'\n",
    "#pdf = urlopen(url).read()\n",
    "## to do - convert pdf to text\n",
    "#chain(text)"
   ]
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "conda-root-py",
   "name": "workbench-notebooks.m113",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/workbench-notebooks:m113"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel) (Local)",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}