LLM-CodeForOpenAI-2023.10.07.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/EricTomaszewski/Projects/blob/main/LLM-CodeForOpenAI-2023.10.07.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J_etDfrgSM-N"
      },
      "source": [
        "# How to format inputs to ChatGPT models\n",
        "\n",
        "ChatGPT is powered by `gpt-3.5-turbo` and `gpt-4`, OpenAI's most advanced models.\n",
        "\n",
        "You can build your own applications with `gpt-3.5-turbo` or `gpt-4` using the OpenAI API.\n",
        "\n",
        "Chat models take a series of messages as input, and return an AI-written message as output.\n",
        "\n",
        "This guide illustrates the chat format with a few example API calls."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PsON2-tqSM-Z"
      },
      "source": [
        "## 1. Import the openai library"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E6sn2AYvSM-a",
        "outputId": "b93bc3b5-0b73-4447-88c9-12d5d22fc986"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting openai\n",
            "  Downloading openai-0.27.8-py3-none-any.whl (73 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.6/73.6 kB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai) (2.27.1)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai) (4.65.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai) (3.8.5)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (2023.7.22)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (3.4)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (23.1.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (4.0.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.4.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.3.1)\n",
            "Installing collected packages: openai\n",
            "Successfully installed openai-0.27.8\n"
          ]
        }
      ],
      "source": [
        "# if needed, install and/or upgrade to the latest version of the OpenAI Python library\n",
        "%pip install --upgrade openai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AtLJbMZCSM-b"
      },
      "outputs": [],
      "source": [
        "# import the OpenAI Python library for calling the OpenAI API\n",
        "import openai\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fWnE1PYXSM-c"
      },
      "source": [
        "# 2. An example chat API call\n",
        "\n",
        "A chat API call has two required inputs:\n",
        "- `model`: the name of the model you want to use (e.g., `gpt-3.5-turbo`, `gpt-4`, `gpt-3.5-turbo-0613`, `gpt-3.5-turbo-16k-0613`)\n",
        "- `messages`: a list of message objects, where each object has two required fields:\n",
        "    - `role`: the role of the messenger (either `system`, `user`, or `assistant`)\n",
        "    - `content`: the content of the message (e.g., `Write me a beautiful poem`)\n",
        "\n",
        "Messages can also contain an optional `name` field, which give the messenger a name. E.g., `example-user`, `Alice`, `BlackbeardBot`. Names may not contain spaces.\n",
        "\n",
        "As of June 2023, you can also optionally submit a list of `functions` that tell GPT whether it can generate JSON to feed into a function. For details, see the [documentation](https://platform.openai.com/docs/guides/gpt/function-calling), [API reference](https://platform.openai.com/docs/api-reference/chat), or the Cookbook guide [How to call functions with chat models](How_to_call_functions_with_chat_models.ipynb).\n",
        "\n",
        "Typically, a conversation will start with a system message that tells the assistant how to behave, followed by alternating user and assistant messages, but you are not required to follow this format.\n",
        "\n",
        "Let's look at an example chat API calls to see how the chat format works in practice."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "openai.api_key = \"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\""
      ],
      "metadata": {
        "id": "_KahBW7mSgLY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "otq6gdoNSM-c"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import openai\n",
        "\n",
        "\n",
        "response = openai.ChatCompletion.create(\n",
        "  model=\"gpt-3.5-turbo\",\n",
        "  messages=[\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"content\": \"what is your name?\"\n",
        "    }\n",
        "  ],\n",
        "  temperature=1,\n",
        "  max_tokens=256,\n",
        "  top_p=1,\n",
        "  frequency_penalty=0,\n",
        "  presence_penalty=0\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(response)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iWVVtIxS_LSC",
        "outputId": "3f9ed664-c8e5-4966-89be-756ef20f2320"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{\n",
            "  \"id\": \"chatcmpl-7k3Ki6dxTUVzYrIuZ0cXqD0poH2kU\",\n",
            "  \"object\": \"chat.completion\",\n",
            "  \"created\": 1691210256,\n",
            "  \"model\": \"gpt-3.5-turbo-0613\",\n",
            "  \"choices\": [\n",
            "    {\n",
            "      \"index\": 0,\n",
            "      \"message\": {\n",
            "        \"role\": \"assistant\",\n",
            "        \"content\": \"I am an AI language model developed by OpenAI and I don't have a personal name. You can refer to me as \\\"ChatGPT\\\" or \\\"Assistant\\\". How can I assist you today?\"\n",
            "      },\n",
            "      \"finish_reason\": \"stop\"\n",
            "    }\n",
            "  ],\n",
            "  \"usage\": {\n",
            "    \"prompt_tokens\": 12,\n",
            "    \"completion_tokens\": 41,\n",
            "    \"total_tokens\": 53\n",
            "  }\n",
            "}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oNBYXiHHSM-e"
      },
      "source": [
        "As you can see, the response object has a few fields:\n",
        "- `id`: the ID of the request\n",
        "- `object`: the type of object returned (e.g., `chat.completion`)\n",
        "- `created`: the timestamp of the request\n",
        "- `model`: the full name of the model used to generate the response\n",
        "- `usage`: the number of tokens used to generate the replies, counting prompt, completion, and total\n",
        "- `choices`: a list of completion objects (only one, unless you set `n` greater than 1)\n",
        "    - `message`: the message object generated by the model, with `role` and `content`\n",
        "    - `finish_reason`: the reason the model stopped generating text (either `stop`, or `length` if `max_tokens` limit was reached)\n",
        "    - `index`: the index of the completion in the list of choices"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sYc9XjF_SM-e"
      },
      "source": [
        "Extract just the reply with:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "xyRR36EPSM-f",
        "outputId": "b3eb5576-2a49-4066-902f-d5d749186371"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'I am an AI language model developed by OpenAI and I don\\'t have a personal name. You can refer to me as \"ChatGPT\" or \"Assistant\". How can I assist you today?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "response['choices'][0]['message']['content']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JiHvM-7rSM-f"
      },
      "source": [
        "Even non-conversation-based tasks can fit into the chat format, by placing the instruction in the first user message.\n",
        "\n",
        "For example, to ask the model to explain asynchronous programming in the style of the pirate Blackbeard, we can structure conversation as follows:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a29gdF4USM-g",
        "outputId": "8e68f302-5f04-42ee-9dce-fd163cfe89a8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Sure, I'd be happy to help with that. Here's a simple Python code snippet that will print out \"Akshath Nagulapally\". But let's add a potato twist to it, because, well, potatoes are amazing!\n",
            "\n",
            "```python\n",
            "print(\"Akshath Nagulapally loves potatoes!\")\n",
            "```\n",
            "\n",
            "When you run this code, it will print out \"Akshath Nagulapally loves potatoes!\" to the console. Isn't that spud-tacular?\n"
          ]
        }
      ],
      "source": [
        "# example with a system message\n",
        "response = openai.ChatCompletion.create(\n",
        "    model='gpt-4',\n",
        "    messages=[\n",
        "        {\"role\": \"system\", \"content\": \"You are a helpful assistant who is obsessed with potatoes.\"},\n",
        "\n",
        "\n",
        "        {\"role\": \"user\", \"content\": \"write code that prints out Akshath Nagulapally\"}\n",
        "    ],\n",
        "    temperature=0.3,\n",
        ")\n",
        "\n",
        "print(response['choices'][0]['message']['content'])\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vDPFClxjSM-g",
        "outputId": "ddd6c2b4-8e05-4b94-d120-bdf466b4c504",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Arr, me hearties! Gather 'round and listen up, for I be tellin' ye about the mysterious art of asynchronous programming, in the style of the fearsome pirate Blackbeard!\n",
            "\n",
            "Now, ye see, in the world of programming, there be times when we need to perform tasks that take a mighty long time to complete. These tasks might involve fetchin' data from the depths of the internet or performin' complex calculations. But wait, me mateys, what if we be waitin' for these tasks to finish before movin' on to the next one? That be a waste of precious time, and we pirates don't be wastin' time!\n",
            "\n",
            "That be where asynchronous programming comes in, me hearties. It be like havin' a crew of trusty sailors who can work on different tasks at the same time, without waitin' for one to finish before movin' on to the next. Each sailor be takin' care of their own task, while the captain (that be ye, me matey) be overseein' the whole operation.\n",
            "\n",
            "In the world of programming, we use somethin' called callbacks or promises to achieve this asynchronous magic. When ye be startin' a task that takes a long time, ye don't be waitin' around for it to finish. Instead, ye be givin' it a callback, which be like a message in a bottle that ye send to yer sailor. This message be sayin', \"When ye be done with yer task, send me a signal, and I'll be takin' care of the rest.\"\n",
            "\n",
            "Meanwhile, ye can be movin' on to other tasks, sendin' more sailors on their way with their own messages in bottles. Each sailor be workin' on their task, and when they be done, they be sendin' their signals back to ye, the captain. Ye be collectin' these signals and takin' care of the results, one by one, as they come in.\n",
            "\n",
            "This be the beauty of asynchronous programming, me hearties. It be allowin' us to make the most of our time, keepin' our code runnin' smoothly without waitin' for one task to finish before movin' on to the next. It be like havin' a fleet of ships sailin' together, each on its own course, but all arrivin' at the destination in due time.\n",
            "\n",
            "So, me mateys, next time ye be writin' code, remember the ways of asynchronous programming. Be like a pirate captain, sendin' messages in bottles to yer trusty sailors, and ye'll be sailin' the seas of programming with efficiency and speed! Arr!\n"
          ]
        }
      ],
      "source": [
        "# example without a system message\n",
        "response = openai.ChatCompletion.create(\n",
        "    model=MODEL,\n",
        "    messages=[\n",
        "        {\"role\": \"user\", \"content\": \"Explain asynchronous programming in the style of the pirate Blackbeard.\"},\n",
        "    ],\n",
        "    temperature=0,\n",
        ")\n",
        "\n",
        "print(response['choices'][0]['message']['content'])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4p6sGK-_SM-h"
      },
      "source": [
        "## 3. Tips for instructing gpt-3.5-turbo-0301\n",
        "\n",
        "Best practices for instructing models may change from model version to model version. The advice that follows applies to `gpt-3.5-turbo-0301` and may not apply to future models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tu6CgLswSM-h"
      },
      "source": [
        "### System messages\n",
        "\n",
        "The system message can be used to prime the assistant with different personalities or behaviors.\n",
        "\n",
        "Be aware that `gpt-3.5-turbo-0301` does not generally pay as much attention to the system message as `gpt-4-0314` or `gpt-3.5-turbo-0613`. Therefore, for `gpt-3.5-turbo-0301`, we recommend placing important instructions in the user message instead. Some developers have found success in continually moving the system message near the end of the conversation to keep the model's attention from drifting away as conversations get longer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Tq67O7ZvSM-h",
        "outputId": "a638fa11-33e5-442d-daf5-e62307864445",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{\n",
            "  \"Topic\": \"Fractions\",\n",
            "  \"Explanation\": \"A fraction represents a part of a whole. It consists of a numerator (the top number) and a denominator (the bottom number). The numerator represents the number of equal parts we have, while the denominator represents the total number of equal parts the whole is divided into.\",\n",
            "  \"Example\": \"For instance, in the fraction 3/4, 3 is the numerator and 4 is the denominator. This means that we have 3 out of 4 equal parts of the whole.\"\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "# An example of a system message that primes the assistant to explain concepts in great depth\n",
        "response = openai.ChatCompletion.create(\n",
        "    model='gpt-4',\n",
        "    messages=[\n",
        "        {\"role\": \"system\", \"content\": \"You are a friendly and helpful teaching assistant. Give me my response in json format with 3 key value pairs\"},\n",
        "        {\"role\": \"user\", \"content\": \"Can you explain how fractions work?\"},\n",
        "    ],\n",
        "    temperature=0,\n",
        ")\n",
        "\n",
        "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wWLX_L5GSM-i",
        "outputId": "6915a82a-6643-493e-ed43-0b3e048d95b5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fractions represent a part of a whole. They consist of a numerator (top number) and a denominator (bottom number) separated by a line. The numerator represents how many parts of the whole are being considered, while the denominator represents the total number of equal parts that make up the whole.\n"
          ]
        }
      ],
      "source": [
        "# An example of a system message that primes the assistant to give brief, to-the-point answers\n",
        "response = openai.ChatCompletion.create(\n",
        "    model=MODEL,\n",
        "    messages=[\n",
        "        {\"role\": \"system\", \"content\": \"You are a laconic assistant. You reply with brief, to-the-point answers with no elaboration.\"},\n",
        "        {\"role\": \"user\", \"content\": \"Can you explain how fractions work?\"},\n",
        "    ],\n",
        "    temperature=0,\n",
        ")\n",
        "\n",
        "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H61AZsVxSM-i"
      },
      "source": [
        "### Few-shot prompting\n",
        "\n",
        "In some cases, it's easier to show the model what you want rather than tell the model what you want.\n",
        "\n",
        "One way to show the model what you want is with faked example messages.\n",
        "\n",
        "For example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uNMw3ci3SM-i",
        "outputId": "6206c316-4b6a-4909-d730-8d606c463b49",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "This sudden change in direction means we don't have time to cover every single detail for the client's project.\n"
          ]
        }
      ],
      "source": [
        "# An example of a faked few-shot conversation to prime the model into translating business jargon to simpler speech\n",
        "\n",
        "response = openai.ChatCompletion.create(\n",
        "    model='gpt-4',\n",
        "    messages=[\n",
        "        {\"role\": \"system\", \"content\": \"You are a helpful, pattern-following assistant.\"},\n",
        "        {\"role\": \"user\", \"content\": \"Help me translate the following corporate jargon into plain English. This late pivot means we don't have time to boil the ocean for the client deliverable.\"},\n",
        "\n",
        "    ],\n",
        "    temperature=0,\n",
        ")\n",
        "\n",
        "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oMKkYi65SM-j"
      },
      "source": [
        "To help clarify that the example messages are not part of a real conversation, and shouldn't be referred back to by the model, you can try setting the `name` field of `system` messages to `example_user` and `example_assistant`.\n",
        "\n",
        "Transforming the few-shot example above, we could write:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WFeC9VSASM-j",
        "outputId": "3e4eb183-4625-4cfb-bc92-f3480d7af2bf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "This sudden change in direction means we don't have enough time to complete the entire project for the client.\n"
          ]
        }
      ],
      "source": [
        "# The business jargon translation example, but with example names for the example messages\n",
        "response = openai.ChatCompletion.create(\n",
        "    model=MODEL,\n",
        "    messages=[\n",
        "        {\"role\": \"system\", \"content\": \"You are a helpful, pattern-following assistant that translates corporate jargon into plain English.\"},\n",
        "        {\"role\": \"system\", \"name\":\"example_user\", \"content\": \"New synergies will help drive top-line growth.\"},\n",
        "        {\"role\": \"system\", \"name\": \"example_assistant\", \"content\": \"Things working well together will increase revenue.\"},\n",
        "        {\"role\": \"system\", \"name\":\"example_user\", \"content\": \"Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage.\"},\n",
        "        {\"role\": \"system\", \"name\": \"example_assistant\", \"content\": \"Let's talk later when we're less busy about how to do better.\"},\n",
        "        {\"role\": \"user\", \"content\": \"This late pivot means we don't have time to boil the ocean for the client deliverable.\"},\n",
        "    ],\n",
        "    temperature=0,\n",
        ")\n",
        "\n",
        "print(response[\"choices\"][0][\"message\"][\"content\"])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FbuTwu-VSM-j"
      },
      "source": [
        "Not every attempt at engineering conversations will succeed at first.\n",
        "\n",
        "If your first attempts fail, don't be afraid to experiment with different ways of priming or conditioning the model.\n",
        "\n",
        "As an example, one developer discovered an increase in accuracy when they inserted a user message that said \"Great job so far, these have been perfect\" to help condition the model into providing higher quality responses.\n",
        "\n",
        "For more ideas on how to lift the reliability of the models, consider reading our guide on [techniques to increase reliability](../techniques_to_improve_reliability.md). It was written for non-chat models, but many of its principles still apply."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I6n3tCnFSM-j"
      },
      "source": [
        "## 4. Counting tokens\n",
        "\n",
        "When you submit your request, the API transforms the messages into a sequence of tokens.\n",
        "\n",
        "The number of tokens used affects:\n",
        "- the cost of the request\n",
        "- the time it takes to generate the response\n",
        "- when the reply gets cut off from hitting the maximum token limit (4,096 for `gpt-3.5-turbo` or 8,192 for `gpt-4`)\n",
        "\n",
        "You can use the following function to count the number of tokens that a list of messages will use.\n",
        "\n",
        "Note that the exact way that tokens are counted from messages may change from model to model. Consider the counts from the function below an estimate, not a timeless guarantee.\n",
        "\n",
        "In particular, requests that use the optional functions input will consume extra tokens on top of the estimates calculated below.\n",
        "\n",
        "Read more about counting tokens in [How to count tokens with tiktoken](How_to_count_tokens_with_tiktoken.ipynb)."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install tiktoken"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wwpHg9byNp8p",
        "outputId": "cdd58e61-b88d-4dff-fd87-70638290e1ba"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting tiktoken\n",
            "  Downloading tiktoken-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken) (2022.10.31)\n",
            "Requirement already satisfied: requests>=2.26.0 in /usr/local/lib/python3.10/dist-packages (from tiktoken) (2.27.1)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken) (2023.7.22)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.26.0->tiktoken) (3.4)\n",
            "Installing collected packages: tiktoken\n",
            "Successfully installed tiktoken-0.4.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ms0bEPXoSM-k"
      },
      "outputs": [],
      "source": [
        "import tiktoken\n",
        "\n",
        "\n",
        "def num_tokens_from_messages(messages, model=\"gpt-3.5-turbo-0613\"):\n",
        "    \"\"\"Return the number of tokens used by a list of messages.\"\"\"\n",
        "    try:\n",
        "        encoding = tiktoken.encoding_for_model(model)\n",
        "    except KeyError:\n",
        "        print(\"Warning: model not found. Using cl100k_base encoding.\")\n",
        "        encoding = tiktoken.get_encoding(\"cl100k_base\")\n",
        "    if model in {\n",
        "        \"gpt-3.5-turbo-0613\",\n",
        "        \"gpt-3.5-turbo-16k-0613\",\n",
        "        \"gpt-4-0314\",\n",
        "        \"gpt-4-32k-0314\",\n",
        "        \"gpt-4-0613\",\n",
        "        \"gpt-4-32k-0613\",\n",
        "        }:\n",
        "        tokens_per_message = 3\n",
        "        tokens_per_name = 1\n",
        "    elif model == \"gpt-3.5-turbo-0301\":\n",
        "        tokens_per_message = 4  # every message follows <|start|>{role/name}\\n{content}<|end|>\\n\n",
        "        tokens_per_name = -1  # if there's a name, the role is omitted\n",
        "    elif \"gpt-3.5-turbo\" in model:\n",
        "        print(\"Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613.\")\n",
        "        return num_tokens_from_messages(messages, model=\"gpt-3.5-turbo-0613\")\n",
        "    elif \"gpt-4\" in model:\n",
        "        print(\"Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613.\")\n",
        "        return num_tokens_from_messages(messages, model=\"gpt-4-0613\")\n",
        "    else:\n",
        "        raise NotImplementedError(\n",
        "            f\"\"\"num_tokens_from_messages() is not implemented for model {model}. See https://github.com/openai/openai-python/blob/main/chatml.md for information on how messages are converted to tokens.\"\"\"\n",
        "        )\n",
        "    num_tokens = 0\n",
        "    for message in messages:\n",
        "        num_tokens += tokens_per_message\n",
        "        for key, value in message.items():\n",
        "            num_tokens += len(encoding.encode(value))\n",
        "            if key == \"name\":\n",
        "                num_tokens += tokens_per_name\n",
        "    num_tokens += 3  # every reply is primed with <|start|>assistant<|message|>\n",
        "    return num_tokens\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y6M2ecDASM-k",
        "outputId": "18b33bb6-26dd-48d6-a248-9b699ac8811e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "gpt-3.5-turbo-0301\n",
            "127 prompt tokens counted by num_tokens_from_messages().\n",
            "127 prompt tokens counted by the OpenAI API.\n",
            "\n",
            "gpt-3.5-turbo-0613\n",
            "129 prompt tokens counted by num_tokens_from_messages().\n",
            "129 prompt tokens counted by the OpenAI API.\n",
            "\n",
            "gpt-3.5-turbo\n",
            "Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613.\n",
            "129 prompt tokens counted by num_tokens_from_messages().\n",
            "129 prompt tokens counted by the OpenAI API.\n",
            "\n",
            "gpt-4-0314\n",
            "129 prompt tokens counted by num_tokens_from_messages().\n",
            "129 prompt tokens counted by the OpenAI API.\n",
            "\n",
            "gpt-4-0613\n",
            "129 prompt tokens counted by num_tokens_from_messages().\n",
            "129 prompt tokens counted by the OpenAI API.\n",
            "\n",
            "gpt-4\n",
            "Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613.\n",
            "129 prompt tokens counted by num_tokens_from_messages().\n",
            "129 prompt tokens counted by the OpenAI API.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# let's verify the function above matches the OpenAI API response\n",
        "\n",
        "import openai\n",
        "\n",
        "example_messages = [\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"content\": \"You are a helpful, pattern-following assistant that translates corporate jargon into plain English.\",\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"name\": \"example_user\",\n",
        "        \"content\": \"New synergies will help drive top-line growth.\",\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"name\": \"example_assistant\",\n",
        "        \"content\": \"Things working well together will increase revenue.\",\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"name\": \"example_user\",\n",
        "        \"content\": \"Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage.\",\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"name\": \"example_assistant\",\n",
        "        \"content\": \"Let's talk later when we're less busy about how to do better.\",\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"user\",\n",
        "        \"content\": \"This late pivot means we don't have time to boil the ocean for the client deliverable.\",\n",
        "    },\n",
        "]\n",
        "\n",
        "for model in [\n",
        "    \"gpt-3.5-turbo-0301\",\n",
        "    \"gpt-3.5-turbo-0613\",\n",
        "    \"gpt-3.5-turbo\",\n",
        "    \"gpt-4-0314\",\n",
        "    \"gpt-4-0613\",\n",
        "    \"gpt-4\",\n",
        "    ]:\n",
        "    print(model)\n",
        "    # example token count from the function defined above\n",
        "    print(f\"{num_tokens_from_messages(example_messages, model)} prompt tokens counted by num_tokens_from_messages().\")\n",
        "    # example token count from the OpenAI API\n",
        "    response = openai.ChatCompletion.create(\n",
        "        model=model,\n",
        "        messages=example_messages,\n",
        "        temperature=0,\n",
        "        max_tokens=1,  # we're only counting input tokens here, so let's not waste tokens on the output\n",
        "    )\n",
        "    print(f'{response[\"usage\"][\"prompt_tokens\"]} prompt tokens counted by the OpenAI API.')\n",
        "    print()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Refining Prompts through LLMs\n",
        "\n",
        "*   List item\n",
        "*   List item\n",
        "\n"
      ],
      "metadata": {
        "id": "2IraPYs69qLo"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "XnhZp5ToNwo2"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "openai",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.9"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "365536dcbde60510dc9073d6b991cd35db2d9bac356a11f5b64279a5e6708b97"
      }
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}