How to specify/change max_tokens while using the server #242
-
I am currently trying to run the OpenAPI server and I noticed that I keep getting incomplete responses with ("finish_reason": "length"). I couldn't figure out a way to modify the "max_tokens" attribute. |
Beta Was this translation helpful? Give feedback.
Answered by
advaitdeshmukh
May 19, 2023
Replies: 1 comment
-
I managed to figure it out. You can set it in the request body like so: |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
advaitdeshmukh
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I managed to figure it out. You can set it in the request body like so:
{ "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What is black hole?" } ], "max_tokens":2000 }