Rerank API not accessible: {"error":{"code":500,"message":"grpc service not ready","type":""}}

[Sarmingsteiner]

I am trying to connect Dify with LocalAI for reranking. However, the connection to the API (http://localhost:8080/v1/rerank) is not possible and results in a 500 error.

But even the proposed CURL test is not successful.
After some time, the follow error message appears:
{"error":{"code":500,"message":"grpc service not ready","type":""}}

I tried with both latest-aio-cpu as well as latest-cpu packages (version 2.16).

Any ideas?

[mudler]

for using rerankers you would need images shipping python dependencies. the latest-aio-cpu and latest-cpu tags automatically points to core images that doesn't ship python.

See also: https://localai.io/basics/container/#standard-container-images

[Sarmingsteiner]

Thanks for your quick reply!
Unfortunately, upon checking the link, I cannot find the tags for the images including python?

[mudler]

just use latest, or for instance a tagged release e.g. v2.16.0

[walbercardoso]

Im getting this error with tts models (coqui, bark). Only piper is working

failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:43665: connect: connection refused""