Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(go/plugins/vertexai): add context caching to vertexai #1478

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

alonsopec89
Copy link
Contributor

@alonsopec89 alonsopec89 commented Dec 10, 2024

Add context caching to vertexai
#1443

How to test:

func main() {
    fmt.Println("hello world!!!!")
    ctx := context.Background()
    if err := vertexai.Init(ctx, nil); err != nil {
        log.Fatal(err)
    }
    m := vertexai.Model("gemini-1.5-flash-002")
    if m == nil {
        log.Fatal(errors.New("vertexai init failed"))
    }

    strrr, err := ai.GenerateText(ctx, m, ai.WithTextPrompt("Tell me a joke about golang developers."))
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(strrr)

    strrr, err = ai.GenerateText(ctx, m, ai.WithTextPrompt("Tell me a joke about golang developers. "))
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(strrr)
}

Checklist (if applicable):

@alonsopec89 alonsopec89 changed the title feat(go): add context caching to vertexai feat(go/plugins/vertexai): add context caching to vertexai Dec 10, 2024
"gemini-1.5-flash": gemini.Multimodal,
"gemini-1.0-pro": gemini.BasicText,
"gemini-1.5-pro": gemini.Multimodal,
"gemini-1.5-flash-002": gemini.Multimodal,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context caching is working only with stable Gemini model versions using number prefix like https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions#stable-versions-available

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get rid of the plain gemini-1.5-flash. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.

@alonsopec89 alonsopec89 linked an issue Dec 13, 2024 that may be closed by this pull request
@apascal07
Copy link
Collaborator

Please add tests for the caching business logic and a sample where this is used (by setting the cache TTL, which enables context caching). See #1297 for test coverage.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go style guide uses tabs (as 4 spaces) instead of 2 spaces. Please update here and in the other PR.

"gemini-1.5-flash": gemini.Multimodal,
"gemini-1.0-pro": gemini.BasicText,
"gemini-1.5-pro": gemini.Multimodal,
"gemini-1.5-flash-002": gemini.Multimodal,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get rid of the plain gemini-1.5-flash. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.

@alonsopec89 alonsopec89 force-pushed the 1443-add-context-caching-to-vertex-ai branch from 8562c77 to 7ef5090 Compare December 17, 2024 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

[Go] Add context caching to Google AI and Vertex AI plugins
2 participants