2.2.14 Backend: Faster Whisper

faster-whisper-service

Handle: stt URL: http://localhost:34151

faster-whisper-server is an OpenAI API compatible transcription server which uses faster-whisper as it's backend. Features:

• GPU and CPU support.
• Easily deployable using Docker.
• Configurable through environment variables (see config.py).
• OpenAI API compatible.

---

Starting

# [Optional] pre-pull the image
harbor pull stt

# Start the service
harbor up stt

Configuration

Harbor will:

• Automatically run cuda version of the service, if possible.
• Automatically switch Open WebUI to use faster-whisper-server as the default STT backend, when run together.

Tip

When running for the very first time, the service will download the model weights, which may take some time. Track in service logs via:

harbor logs stt

Apart from that, following configuration options are available:

# Get/set STT model to use
harbor stt model
harbor stt model Systran/faster-distil-whisper-large-v3

# Get/set STT version to use
harbor stt version
harbor stt version latest

Run the following command to see all available config options:

harbor config ls | grep STT

Models

You can hit the /models endpoint when the service is running to get a list of supported models.

curl $(harbor url stt)/v1/models

See more examples of the API in the http catalog.