Skip to content

Commit

Permalink
feat(embeddings): do not require to be configured (mudler#1842)
Browse files Browse the repository at this point in the history
Certain engines requires to know during model loading
if the embedding feature has to be enabled, however, it is impractical
to have to set it to ALL the backends that supports embeddings.

There are transformers and sentencentransformers that seamelessly handle
both cases, without having this settings to be explicitly enabled.

The case sussist only for ggml-based models that needs to enable
featuresets during model loading (and thus settings `embedding` is
required), however most of the other engines does not require this.

This change disables the check done at code side, making easier to use
embeddings by not having to specify explicitly `embeddings: true`.

Part of: mudler#1373
  • Loading branch information
mudler authored Mar 15, 2024
1 parent ae2e4fc commit 89351f1
Showing 1 changed file with 0 additions and 4 deletions.
4 changes: 0 additions & 4 deletions core/backend/embeddings.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,6 @@ import (
)

func ModelEmbedding(s string, tokens []int, loader *model.ModelLoader, backendConfig config.BackendConfig, appConfig *config.ApplicationConfig) (func() ([]float32, error), error) {
if !backendConfig.Embeddings {
return nil, fmt.Errorf("endpoint disabled for this model by API configuration")
}

modelFile := backendConfig.Model

grpcOpts := gRPCModelOpts(backendConfig)
Expand Down

0 comments on commit 89351f1

Please sign in to comment.