-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(go/plugins/vertexai): add context caching to vertexai #1478
base: main
Are you sure you want to change the base?
Conversation
go/plugins/vertexai/vertexai.go
Outdated
"gemini-1.5-flash": gemini.Multimodal, | ||
"gemini-1.0-pro": gemini.BasicText, | ||
"gemini-1.5-pro": gemini.Multimodal, | ||
"gemini-1.5-flash-002": gemini.Multimodal, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
context caching is working only with stable Gemini model versions using number prefix like https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions#stable-versions-available
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want to get rid of the plain gemini-1.5-flash
. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.
Please add tests for the caching business logic and a sample where this is used (by setting the cache TTL, which enables context caching). See #1297 for test coverage. |
go/plugins/vertexai/cache.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go style guide uses tabs (as 4 spaces) instead of 2 spaces. Please update here and in the other PR.
go/plugins/vertexai/vertexai.go
Outdated
"gemini-1.5-flash": gemini.Multimodal, | ||
"gemini-1.0-pro": gemini.BasicText, | ||
"gemini-1.5-pro": gemini.Multimodal, | ||
"gemini-1.5-flash-002": gemini.Multimodal, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want to get rid of the plain gemini-1.5-flash
. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.
8562c77
to
7ef5090
Compare
Add context caching to vertexai
#1443
How to test:
Checklist (if applicable):