Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Assistants methods to summarize and trim messages #786

Open
2 tasks
andreibondarev opened this issue Sep 19, 2024 · 1 comment
Open
2 tasks

Add Assistants methods to summarize and trim messages #786

andreibondarev opened this issue Sep 19, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@andreibondarev
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
A lengthy Assistant conversation will exceed token context limit. We need a way to trim the messages and/or summarize the conversation to be able to continue it.

Describe the solution you'd like

  • Add Langchain::Assistant#summarize_messages! method that summarizes the all of the messages. Need to research how to best summarize a conversation, and whether the summary should go into a prompt in a system message or as a role="user" message.
  • Add Langchain::Assistant#trim_messages!(token_offset:) to trim messages to free up a token offset. Research best ways to do this as well.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@andreibondarev andreibondarev added the enhancement New feature or request label Sep 19, 2024
@cannuri
Copy link

cannuri commented Nov 4, 2024

Zep DB is also summarizing chat histories after a certain time (definable by the user). they used a separate role for this called summary. So introducing a 4th role might help here.
Regarding the best way to summarize: I would set one as default but allow the user to customize the summarization-prompt.
I worked a lot with Sparse Priming Representations and for me it worked fine.
Another great method for summaries is Chain of Density.

Zep is moving from summaries towards "Facts". So they extract specific knowledge from the history and save it in a graph. Maybe that's an approach to consider as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants