(docs) Add docs on using Vertex with Fine Tuning APIs (BerriAI#7491)

* docs add Overview for vertex endpoints * docs add vertex ft api to docs * Advanced use case - Passing `adapter_size` to the Vertex AI API
kp-forks · Jan 1, 2025 · 6705e30 · 6705e30
1 parent 859f6e1
commit 6705e30
Show file tree

Hide file tree

Showing 2 changed files with 107 additions and 52 deletions.
diff --git a/docs/my-website/docs/fine_tuning.md b/docs/my-website/docs/fine_tuning.md
@@ -110,58 +110,6 @@ curl http://localhost:4000/v1/fine_tuning/jobs \
 
 </TabItem>
 
-<TabItem value="Vertex" label="VertexAI">
-
-<Tabs>
-<TabItem value="openai" label="OpenAI Python SDK">
-
-```python
-ft_job = await client.fine_tuning.jobs.create(
-    model="gemini-1.0-pro-002",                  # Vertex model you want to fine-tune
-    training_file="gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl",                 # file_id from create file response
-    extra_body={"custom_llm_provider": "vertex_ai"}, # tell litellm proxy which provider to use
-)
-```
-</TabItem>
-
-<TabItem value="curl" label="curl (Unified API)">
-
-```shell
-curl http://localhost:4000/v1/fine_tuning/jobs \
-    -H "Content-Type: application/json" \
-    -H "Authorization: Bearer sk-1234" \
-    -d '{
-    "custom_llm_provider": "vertex_ai",
-    "model": "gemini-1.0-pro-002",
-    "training_file": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl"
-    }'
-```
-</TabItem>
-
-<TabItem value="curl-vtx" label="curl (VertexAI API)">
-
-:::info
-
-Use this to create Fine tuning Jobs in [the Vertex AI API Format](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning#create-tuning)
-
-:::
-
-```shell
-curl http://localhost:4000/v1/projects/tuningJobs \
-      -H "Content-Type: application/json" \
-      -H "Authorization: Bearer sk-1234" \
-      -d '{
-  "baseModel": "gemini-1.0-pro-002",
-  "supervisedTuningSpec" : {
-      "training_dataset_uri": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl"
-  }
-}'
-```
-
-</TabItem>
-</Tabs>
-
-</TabItem>
 </Tabs>
 
 ### Request Body

diff --git a/docs/my-website/docs/providers/vertex.md b/docs/my-website/docs/providers/vertex.md
@@ -4,13 +4,16 @@ import TabItem from '@theme/TabItem';
 
 # VertexAI [Anthropic, Gemini, Model Garden]
 
+## Overview
 
 | Property | Details |
 |-------|-------|
 | Description | Vertex AI is a fully-managed AI development platform for building and using generative AI. |
 | Provider Route on LiteLLM | `vertex_ai/` |
 | Link to Provider Doc | [Vertex AI ↗](https://cloud.google.com/vertex-ai) |
 | Base URL | [https://{vertex_location}-aiplatform.googleapis.com/](https://{vertex_location}-aiplatform.googleapis.com/) |
+| Supported Operations | [`/chat/completions`](#sample-usage), `/completions`, [`/embeddings`](#embedding-models), [`/audio/speech`](#text-to-speech-apis), [`/fine_tuning`](#fine-tuning-apis), [`/batches`](#batch-apis), [`/files`](#batch-apis), [`/images`](#image-generation-models) |
+
 
 <br />
 <br />
@@ -2500,6 +2503,110 @@ create_batch_response = oai_client.batches.create(
 }
 ```
 
+## **Fine Tuning APIs**
+
+
+| Property | Details |
+|----------|---------|
+| Description | Create Fine Tuning Jobs in Vertex AI (`/tuningJobs`) using OpenAI Python SDK |
+| Vertex Fine Tuning Documentation | [Vertex Fine Tuning](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning#create-tuning) |
+
+### Usage
+
+#### 1. Add `finetune_settings` to your config.yaml
+```yaml
+model_list:
+  - model_name: gpt-4
+    litellm_params:
+      model: openai/fake
+      api_key: fake-key
+      api_base: https://exampleopenaiendpoint-production.up.railway.app/
+
+# 👇 Key change: For /fine_tuning/jobs endpoints
+finetune_settings:
+  - custom_llm_provider: "vertex_ai"
+    vertex_project: "adroit-crow-413218"
+    vertex_location: "us-central1"
+    vertex_credentials: "/Users/ishaanjaffer/Downloads/adroit-crow-413218-a956eef1a2a8.json"
+```
+
+#### 2. Create a Fine Tuning Job
+
+<Tabs>
+<TabItem value="openai" label="OpenAI Python SDK">
+
+```python
+ft_job = await client.fine_tuning.jobs.create(
+    model="gemini-1.0-pro-002",                  # Vertex model you want to fine-tune
+    training_file="gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl",                 # file_id from create file response
+    extra_body={"custom_llm_provider": "vertex_ai"}, # tell litellm proxy which provider to use
+)
+```
+</TabItem>
+
+<TabItem value="curl" label="curl">
+
+```shell
+curl http://localhost:4000/v1/fine_tuning/jobs \
+    -H "Content-Type: application/json" \
+    -H "Authorization: Bearer sk-1234" \
+    -d '{
+    "custom_llm_provider": "vertex_ai",
+    "model": "gemini-1.0-pro-002",
+    "training_file": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl"
+    }'
+```
+</TabItem>
+
+</Tabs>
+
+
+**Advanced use case - Passing `adapter_size` to the Vertex AI API**
+
+Set hyper_parameters, such as `n_epochs`, `learning_rate_multiplier` and `adapter_size`. [See Vertex Advanced Hyperparameters](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning#advanced_use_case)
+
+
+<Tabs>
+<TabItem value="openai" label="OpenAI Python SDK">
+
+```python
+
+ft_job = client.fine_tuning.jobs.create(
+    model="gemini-1.0-pro-002",                  # Vertex model you want to fine-tune
+    training_file="gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl",                 # file_id from create file response
+    hyperparameters={
+        "n_epochs": 3,                      # epoch_count on Vertex
+        "learning_rate_multiplier": 0.1,    # learning_rate_multiplier on Vertex
+        "adapter_size": "ADAPTER_SIZE_ONE"  # type: ignore, vertex specific hyperparameter
+    },
+    extra_body={
+        "custom_llm_provider": "vertex_ai",
+    },
+)
+```
+</TabItem>
+
+<TabItem value="curl" label="curl">
+
+```shell
+curl http://localhost:4000/v1/fine_tuning/jobs \
+    -H "Content-Type: application/json" \
+    -H "Authorization: Bearer sk-1234" \
+    -d '{
+    "custom_llm_provider": "vertex_ai",
+    "model": "gemini-1.0-pro-002",
+    "training_file": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl",
+    "hyperparameters": {
+        "n_epochs": 3,
+        "learning_rate_multiplier": 0.1,
+        "adapter_size": "ADAPTER_SIZE_ONE"
+    }
+    }'
+```
+</TabItem>
+
+</Tabs>
+
 
 ## Extra