You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
A lengthy Assistant conversation will exceed token context limit. We need a way to trim the messages and/or summarize the conversation to be able to continue it.
Describe the solution you'd like
Add Langchain::Assistant#summarize_messages! method that summarizes the all of the messages. Need to research how to best summarize a conversation, and whether the summary should go into a prompt in a system message or as a role="user" message.
Add Langchain::Assistant#trim_messages!(token_offset:) to trim messages to free up a token offset. Research best ways to do this as well.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
Zep DB is also summarizing chat histories after a certain time (definable by the user). they used a separate role for this called summary. So introducing a 4th role might help here.
Regarding the best way to summarize: I would set one as default but allow the user to customize the summarization-prompt.
I worked a lot with Sparse Priming Representations and for me it worked fine.
Another great method for summaries is Chain of Density.
Zep is moving from summaries towards "Facts". So they extract specific knowledge from the history and save it in a graph. Maybe that's an approach to consider as well.
Is your feature request related to a problem? Please describe.
A lengthy Assistant conversation will exceed token context limit. We need a way to trim the messages and/or summarize the conversation to be able to continue it.
Describe the solution you'd like
Langchain::Assistant#summarize_messages!
method that summarizes the all of the messages. Need to research how to best summarize a conversation, and whether the summary should go into a prompt in a system message or as arole="user"
message.Langchain::Assistant#trim_messages!(token_offset:)
to trim messages to free up a token offset. Research best ways to do this as well.Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: