Ollama create Error: unsupported architecture

[simonnxren]

the architecture error when creating models or converting to gguf with llama.cpp

it look like the model has the same architecture as the llama3.2V, could you help me with this?

[XuGW-Kevin]

Hi, This model has exactly the same architecture as the llama3.2V.

[XuGW-Kevin]

In theory, you can run this model on any platform that supports llama3.2V.
I'm not familiar with Ollama, but if you can share the error, I'll try to help with that.

[simonnxren]

In theory, you can run this model on any platform that supports llama3.2V. I'm not familiar with Ollama, but if you can share the error, I'll try to help with that.

cmd:
ollama create llama32V_cot -- this name points to the hf repo i downloaded

output:
transferring model data 100%
converting model
Error: unsupported architecture

ollama run llama3.2-vision, this cmd can run without problem.
I just figured out why it is not working. the official llama3.2vision is converted to gguf already. so it can run directly. But when using ollama create, the server need to convert the safetensor to gguf with llama.cpp in the background, while the mllama is not supported by the llama.cpp yet. therefore, the converting process failed

[simonnxren]

Hi, This model has exactly the same architecture as the llama3.2V.

could you upload a gguf version as well? i am not sure how the official llama3.2v is converted

[XuGW-Kevin]

It seems that one can use:

pip install model2gguf
model2gguf convert --model_id "Xkev/Llama-3.2V-11B-cot" --output_file "output_file.gguf"

I will try to upload one later. (I haven't tried that before, honestly.)

[simonnxren]

It seems that one can use:

pip install model2gguf model2gguf convert --model_id "Xkev/Llama-3.2V-11B-cot" --output_file "output_file.gguf"

I will try to upload one later. (I haven't tried that before, honestly.)

ehh... just tried the package. it is using llama.cpp in the backend as well. It does not work, got the same error:

INFO:hf-to-gguf:Loading model: Llama-3.2V-11B-cot
ERROR:hf-to-gguf:Model MllamaForConditionalGeneration is not supported