Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a description to ReAct module's input fields #2003

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

itay1542
Copy link
Contributor

@itay1542 itay1542 commented Jan 1, 2025

added a short description in the ReAct output fields because some models (including big ones like the latest claude-3-5-sonnet) sometimes output the ReAct keys without the "next" string, for example:

[[ ## next_thought ## ]]

[[ ## tool_name ## ]] <-- missing "next"

[[ ## next_tool_args ## ]]

This should fix a lot of instances of these errors:
ValueError: Expected dict_keys(['next_thought', 'next_tool_name', 'next_tool_args']) but got dict_keys([])

@okhat
Copy link
Collaborator

okhat commented Jan 1, 2025

Hey @itay1542 ! This is SUPER helpful, thank you.

I didn't know that claude 3.5 sonnet struggled with this. That's a very useful report.

I think we must fix that, but I don't think the fundamental fix is by changing the ReAct descriptions. I think that has to happen at the Adapter layer instead. A good adapter should ensure this kind of thing passes for all popular models out of the box, and should fix the issue on retry if it fails the first time.

I will keep this PR open as a "draft" so that we can prioritize adding this unit test (sonnet ReAct) to our adapters in 2.6

@okhat okhat marked this pull request as draft January 1, 2025 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants