Charmed Ollama

Ollama enables easy deployment of large language models on your own infrastructure.

See the models available in the Ollama library.

Installation

Deploying Ollama on your cloud using Juju

juju deploy ollama --channel=beta

Installing a large language model to use

juju run ollama/0 pull model="llama3.1"

Note: the pull action may take a long time, you can add the --wait parameter (i.e. --wait=5m) to avoid getting a timeout error if your model is too large.

Usage

Generating text using the juju action

juju run ollama/0 generate model="llama3.1" prompt="Why is the sky blue?"

Note: the generate action may take a long time, you can add the --wait parameter (i.e. --wait=5m) to avoid getting a timeout error if your hardware is too slow.

Generating text using the HTTP API

curl http://<unit-ip-address>:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt":"Why is the sky blue?"
}'

Other resources

• See the Juju SDK documentation for more information about developing and improving charms.
• See the Ollama API documentation for all the interactions you can have with Ollama.