Python_Notebook_Sample/BQML+Claude.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "cell_execution_strategy": "setup",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# BQML + Claude Get Started Sample\n",
        "\n",
        "This is a starter notebook for BQML + Claude"
      ],
      "metadata": {
        "id": "gj-XMNm32BHM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Before you begin\n",
        "Follow the required permissions and Enable the neccessary API in this GCP Doc:\n",
        "[link](https://cloud.google.com/bigquery/docs/generate-text#required_permissions)"
      ],
      "metadata": {
        "id": "pRomYsdu4hBM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 1: Enable Claude Model"
      ],
      "metadata": {
        "id": "w-np-2m53WH3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "1.   Visit the Vertex AI Model Garden console and select the model tile for Claude model of your choice. Following this doc [link](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude)\n",
        "2.   Click on the “Enable” button and follow the proceeding instructions."
      ],
      "metadata": {
        "id": "nc50bd8h3qc7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 2: Create a BQ External Connection\n",
        "Follow the same process like this one: [link](https://cloud.google.com/bigquery/docs/generate-text#create_a_connection) However, keep an attension to the** supported region** of Claude models and make your conenction follow the same region for example us-east5 for Claude 3.5.\n"
      ],
      "metadata": {
        "id": "3RxPYQWG5JTH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 3: Give the service account access\n",
        "\n",
        "Follow the same step as in this [link](https://cloud.google.com/bigquery/docs/generate-text#give_the_service_account_access)"
      ],
      "metadata": {
        "id": "npgZhUQv6L4c"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 4 Create a model"
      ],
      "metadata": {
        "id": "aS0XL8ZV6jPR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Same as other LLMs, but the endpoint option will be one of the followings:\n",
        "\n",
        "For Claude 3.5 Sonnet, use claude-3-5-sonnet@20240620.\n",
        "For Claude 3 Opus, use claude-3-opus@20240229.\n",
        "For Claude 3 Haiku, use claude-3-haiku@20240307.\n",
        "For Claude 3 Sonnet, use claude-3-sonnet@20240229.\n",
        "\n",
        "Similar to Vertex AI, we also recommended using a model endpoint that includes a suffix that starts with an @ symbol. (Vertex AI public doc)\n",
        "\n",
        "For supported regions different Claude models, please refer to: https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#regions\n",
        "\n",
        "For quota for different models and regions, please refer to:\n",
        "https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#anthropic_claude_quotas_and_supported_context_length\n",
        "\n",
        "Eample Syntax Below:"
      ],
      "metadata": {
        "id": "xIvW8A2A7YL8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%bigquery\n",
        "CREATE OR REPLACE MODEL\n",
        "`PROJECT_ID.DATASET_ID.MODEL_NAME`\n",
        "REMOTE WITH CONNECTION `PROJECT_ID.REGION.CONNECTION_ID`\n",
        "OPTIONS (ENDPOINT = 'claude-3-5-sonnet@20240620');"
      ],
      "metadata": {
        "id": "dkM11-Q85I1e"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 5: Model Inference\n",
        "Some examples below"
      ],
      "metadata": {
        "id": "kd5L1SBV9MnM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%bigquery\n",
        "-- Example 1: Generate text from text prompt\n",
        "SELECT *\n",
        "FROM ML.GENERATE_TEXT(\n",
        "  MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`,\n",
        "  (\"What's the capital of France.\"),\n",
        "  STRUCT(\n",
        "    TOKENS AS max_output_tokens,\n",
        "    TOP_K AS top_k,\n",
        "    TOP_P AS top_p,\n",
        "    FLATTEN_JSON AS flatten_json_output)\n",
        ");\n"
      ],
      "metadata": {
        "id": "5Irq-AGn2MDM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        " %%bigquery\n",
        "-- Example 2: Generate text from text data from a table\n",
        " SELECT\n",
        "    [Column 1],\n",
        "    [Text Column 2],\n",
        "    ml_generate_text_result AS generated_text\n",
        "  FROM\n",
        "    ML.GENERATE_TEXT(\n",
        "      MODEL `PROJECT_ID.DATASET_ID.MODEL_NAME`,\n",
        "      (\n",
        "        SELECT\n",
        "       [Column 1],\n",
        "       [Text Column 2],\n",
        "       concat(\"summarize this input:\", [Text Column 2])  AS prompt\n",
        "       from `PROJECT_ID.DATASET_ID.TABLE_NAME`\n",
        "      ),\n",
        "        STRUCT(\n",
        "          TOKENS AS max_output_tokens,\n",
        "          TOP_K AS top_k,\n",
        "          TOP_P AS top_p,\n",
        "          FLATTEN_JSON AS flatten_json_output)\n",
        "    )"
      ],
      "metadata": {
        "id": "1RNDJBFi3VXM"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}