Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use metadata filter for chats #423

Open
pmeier opened this issue May 21, 2024 · 0 comments
Open

use metadata filter for chats #423

pmeier opened this issue May 21, 2024 · 0 comments
Labels

Comments

@pmeier
Copy link
Member

pmeier commented May 21, 2024

Feature description

See #422 for work that has to happen before this. See #256 for a discussion.

  • SourceStorage.retrieve needs to take a MetadataFilter object as first input argument rather than a list of documents
  • The builtin source storages need to be refactored in a way to only have a single index. The metadata of each document should be stored alongside the document in the DB. During retrieval, the metadata filter needs to be translated into the dialect of the DB
  • The first input argument of Rag().chat() needs to be renamed to input and should allow a list of Documents as well as a MetadataFilter:
    • Passing a list of documents should retain the current behavior by storing them in the source storage and creating a metadata filter to select only these documents in the retrieval stage.
    • Passing a metadata filter should skip the storing and just set the prepared flag on the chat
      The cases of passing both types or none as explored in "managed" Ragna #256 (comment) will be saved for later. For now only a single input is fine
  • For the REST API, instead of overloading one input parameter as done for the Python API, we should have two mutually exclusive documents and metadata_filter inputs.

Value and/or benefit

No response

Anything else?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Todo
Development

No branches or pull requests

1 participant