How to build a service that dynamically loads embedding models?

[kobiche]

Analogous to how Huggingface handles loading models dynamically (just by typing the model name), I wondered how this could be achieved using BentoML.
I see that runners are now considered legacy. So, how could this be achieved by using class and methods decorators?
I am thinking of the following:

1. the service offers an endpoint where the model (and possible serve configurations) is defined, along with the text(s) to be embedded
2. the system recognizes if the model is already loaded on memory (GPU/CPU) or must be initialized first
3. the system outputs the embedding

Optional:

• the system offers an endpoint where the list of loaded models is retrievable
• the system offers a delete endpoint to remove/unload a model from memory