Update to the latest google-cloud-aiplatform dependency and properly handle function calls in gemini #16793
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Google updated the cloud-aiplatform dependencies, so in a probably futile attempt to keep them somewhat current I have updated them.
|
(Appreciate this @stfines-clgx -- I don't have access to test this 🙏🏻 ) |
| @@ -495,7 +495,7 @@ def get_tool_calls_from_response( | |||
|
|
|||
| tool_selections = [] | |||
| for tool_call in tool_calls: | |||
| response_dict = MessageToDict(tool_call._pb) | |||
| response_dict = MessageToDict(tool_call._raw_message._pb) | |||
There was a problem hiding this comment.
no way to do this without accessing private attributes hey? 😅 Oh google
There was a problem hiding this comment.
Well, in another branch, I did it by just creating an entirely new object by hand. I kept it this way to minimize the change surface area. I can move over to the other change style if it is an issue, but yeah- their MessageToDict function doesn't work without private access. Kind of silly; but then again, I'm not entirely sure that google makes apis for use outside of google.
The BaseTool object serializes things using pydantic but this a json schema objects which do not currently play well with protobuf. This version is an attempt to handle that issue.
…a_index into dev/1_70_protobuf_only
… Protobuf This changes how plans are serialized so that they are compatible with protobuf. It's a workaround until Google addresses the issue in google-cloud-aiplatform.
…/1_70_protobuf_only
Adds a feature flag for model garden.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This fixes the usage of function and non-function tools with Google's Gemini models. The prior implementation would not address the new
ModelRoleof tool when passing tool requests to the agent and it would therefore cause the Gemini message to fail.Additionally, the non-public imports and beta imports that have been superseded by release versions have been corrected so that the integration now uses publicly supported APIs where they are available.
Works around the issues that google-cloud-aiplatform has with serialization of JSON Schema as well.
Fixes #16678 #16625 #16970
New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.
Suggested Checklist:
make format; make lintto appease the lint gods