Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return finish_reason as an Enum - with wrapped stopping word/sequence/tok #800

Open
ShelbyJenkins opened this issue Sep 27, 2024 · 0 comments
Labels
new feature New feature or request

Comments

@ShelbyJenkins
Copy link

I use stop words/sequences to determine next steps after a response. So if the LLM returns stop word A, we perform action X.

I also do the same with other finish reasons - for example retrying with longer context size if the finish_reason is Length.

Current behavior:

  • finish_reason is a string
  • stop sequences/words are not returned, so knowing which stop sequence was hit is not possible

Proposed behavior

  • finish_reason is an enum. this will allow for easier matching
  • the finish_reason stop sequence/word/tok variant includes the string that caused the end of inference
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant