From 6705e30d5d7aca00666eac197272c0941e404878 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Tue, 31 Dec 2024 18:50:18 -0800 Subject: [PATCH] (docs) Add docs on using Vertex with Fine Tuning APIs (#7491) * docs add Overview for vertex endpoints * docs add vertex ft api to docs * Advanced use case - Passing `adapter_size` to the Vertex AI API --- docs/my-website/docs/fine_tuning.md | 52 ----------- docs/my-website/docs/providers/vertex.md | 107 +++++++++++++++++++++++ 2 files changed, 107 insertions(+), 52 deletions(-) diff --git a/docs/my-website/docs/fine_tuning.md b/docs/my-website/docs/fine_tuning.md index fd3cbc792dc1..657377aa815b 100644 --- a/docs/my-website/docs/fine_tuning.md +++ b/docs/my-website/docs/fine_tuning.md @@ -110,58 +110,6 @@ curl http://localhost:4000/v1/fine_tuning/jobs \ - - - - - -```python -ft_job = await client.fine_tuning.jobs.create( - model="gemini-1.0-pro-002", # Vertex model you want to fine-tune - training_file="gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl", # file_id from create file response - extra_body={"custom_llm_provider": "vertex_ai"}, # tell litellm proxy which provider to use -) -``` - - - - -```shell -curl http://localhost:4000/v1/fine_tuning/jobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer sk-1234" \ - -d '{ - "custom_llm_provider": "vertex_ai", - "model": "gemini-1.0-pro-002", - "training_file": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl" - }' -``` - - - - -:::info - -Use this to create Fine tuning Jobs in [the Vertex AI API Format](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning#create-tuning) - -::: - -```shell -curl http://localhost:4000/v1/projects/tuningJobs \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer sk-1234" \ - -d '{ - "baseModel": "gemini-1.0-pro-002", - "supervisedTuningSpec" : { - "training_dataset_uri": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl" - } -}' -``` - - - - - ### Request Body diff --git a/docs/my-website/docs/providers/vertex.md b/docs/my-website/docs/providers/vertex.md index 10329b15a4b4..aa551d711c3a 100644 --- a/docs/my-website/docs/providers/vertex.md +++ b/docs/my-website/docs/providers/vertex.md @@ -4,6 +4,7 @@ import TabItem from '@theme/TabItem'; # VertexAI [Anthropic, Gemini, Model Garden] +## Overview | Property | Details | |-------|-------| @@ -11,6 +12,8 @@ import TabItem from '@theme/TabItem'; | Provider Route on LiteLLM | `vertex_ai/` | | Link to Provider Doc | [Vertex AI ↗](https://cloud.google.com/vertex-ai) | | Base URL | [https://{vertex_location}-aiplatform.googleapis.com/](https://{vertex_location}-aiplatform.googleapis.com/) | +| Supported Operations | [`/chat/completions`](#sample-usage), `/completions`, [`/embeddings`](#embedding-models), [`/audio/speech`](#text-to-speech-apis), [`/fine_tuning`](#fine-tuning-apis), [`/batches`](#batch-apis), [`/files`](#batch-apis), [`/images`](#image-generation-models) | +

@@ -2500,6 +2503,110 @@ create_batch_response = oai_client.batches.create( } ``` +## **Fine Tuning APIs** + + +| Property | Details | +|----------|---------| +| Description | Create Fine Tuning Jobs in Vertex AI (`/tuningJobs`) using OpenAI Python SDK | +| Vertex Fine Tuning Documentation | [Vertex Fine Tuning](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning#create-tuning) | + +### Usage + +#### 1. Add `finetune_settings` to your config.yaml +```yaml +model_list: + - model_name: gpt-4 + litellm_params: + model: openai/fake + api_key: fake-key + api_base: https://exampleopenaiendpoint-production.up.railway.app/ + +# 👇 Key change: For /fine_tuning/jobs endpoints +finetune_settings: + - custom_llm_provider: "vertex_ai" + vertex_project: "adroit-crow-413218" + vertex_location: "us-central1" + vertex_credentials: "/Users/ishaanjaffer/Downloads/adroit-crow-413218-a956eef1a2a8.json" +``` + +#### 2. Create a Fine Tuning Job + + + + +```python +ft_job = await client.fine_tuning.jobs.create( + model="gemini-1.0-pro-002", # Vertex model you want to fine-tune + training_file="gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl", # file_id from create file response + extra_body={"custom_llm_provider": "vertex_ai"}, # tell litellm proxy which provider to use +) +``` + + + + +```shell +curl http://localhost:4000/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer sk-1234" \ + -d '{ + "custom_llm_provider": "vertex_ai", + "model": "gemini-1.0-pro-002", + "training_file": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl" + }' +``` + + + + + +**Advanced use case - Passing `adapter_size` to the Vertex AI API** + +Set hyper_parameters, such as `n_epochs`, `learning_rate_multiplier` and `adapter_size`. [See Vertex Advanced Hyperparameters](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning#advanced_use_case) + + + + + +```python + +ft_job = client.fine_tuning.jobs.create( + model="gemini-1.0-pro-002", # Vertex model you want to fine-tune + training_file="gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl", # file_id from create file response + hyperparameters={ + "n_epochs": 3, # epoch_count on Vertex + "learning_rate_multiplier": 0.1, # learning_rate_multiplier on Vertex + "adapter_size": "ADAPTER_SIZE_ONE" # type: ignore, vertex specific hyperparameter + }, + extra_body={ + "custom_llm_provider": "vertex_ai", + }, +) +``` + + + + +```shell +curl http://localhost:4000/v1/fine_tuning/jobs \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer sk-1234" \ + -d '{ + "custom_llm_provider": "vertex_ai", + "model": "gemini-1.0-pro-002", + "training_file": "gs://cloud-samples-data/ai-platform/generative_ai/sft_train_data.jsonl", + "hyperparameters": { + "n_epochs": 3, + "learning_rate_multiplier": 0.1, + "adapter_size": "ADAPTER_SIZE_ONE" + } + }' +``` + + + + ## Extra