Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can this be used with "Ollama" to share models? If so, how can it be modified? #72

Open
Willy-Shenn opened this issue Sep 2, 2024 · 1 comment

Comments

@Willy-Shenn
Copy link

I am currently using two UI systems, but they cannot share models (possibly due to differences in how the models are identified and created). Even after modifying the environment variables, both UIs cannot use models from the same path. Is there anyone who can guide me on how to modify the two UIs so they can use models from the same path? I would be very grateful.

@ink-splatters
Copy link

ink-splatters commented Sep 12, 2024

Option 1

  1. Install gollama. It supports some cli options but defaults should be fine in the most cases
  2. Make sure lms models dir exists, the snippet below assumes its default location:
mkdir -p ~/.cache/lm-studio/models
  1. in gollama TUI use l/ L shortcuts to (re)link particular / all models

Option 2

  1. install jq
  2. Use this script to (re)link all models:
#!/usr/bin/env bash
# NOTE: this script should be properly credited as I only slightly modified it, but unfortunately I don't know the author's name anymore.

OLLAMA_MODELS_DIR="${OLLAMA_MODELS_DIR:-$HOME/.ollama/models}"
LMSTUDIO_MODELS_DIR="${LMSTUDIO_MODELS_DIR:-$HOME/.cache/lm-studio/models}"

OLLAMA_MANIFEST_DIR=$OLLAMA_MODELS_DIR/manifests/registry.ollama.ai
OLLAMA_BLOBS_DIR=$OLLAMA_MODELS_DIR/blobs

mkdir -p "$LMSTUDIO_MODELS_DIR"
find "$LMSTUDIO_MODELS_DIR" -type l -exec rm {} +

find "$OLLAMA_MANIFEST_DIR" -mindepth 3 -maxdepth 3 -type f | while IFS= read -r file; do
    user=$(basename "$(dirname "$(dirname "$file")")" | sed 's/^registry\.ollama\.ai/ollama/')
    model=$(basename "$(dirname "$file")")
    tag=$(basename "$file")

    digest=$(jq -r '.layers[] | select(.mediaType == "application/vnd.ollama.image.model") | .digest' "$file")
    digest="${digest//:/-}"
    mkdir -p "$LMSTUDIO_MODELS_DIR/$user/$model/"
    ln -s "$OLLAMA_BLOBS_DIR/$digest" "$LMSTUDIO_MODELS_DIR/$user/$model/$model-$tag.gguf"

    echo "$user - $model:$tag"
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants