Planned support for classification (reranking) models?

[Vinno97]

Hi Guys, nice work on this project!

I was wondering if there's any plan to support more model types than just embedding models. To my knowledge, there isn't that much difference between running an embedding (encoder) model and a classification (encoder) model, except the way the output is processed.

One use-case for this, is that RAG systems benefit from having a classification model (https://huggingface.co/BAAI/bge-reranker-large) re-rank the mode likely candidate documents (https://www.sbert.net/examples/applications/retrieve_rerank/README.html).

[OlivierDehaene]

That's not on our roadmap but it could be something interesting for the future for sure.
For now I guess you can still use TEI and then run the classification head on the client side.

[OlivierDehaene]

See: #76

[Vinno97]

Oh nice, thank you! I didn't actually expect this to be implemented anytime soon, but happy to see it now already! 😃