Gemma models still not working.

[rombodawg]

Gemma models that have been quantized using Llamacpp are not working. Please look into the issue

error

"llama.cpp error: 'create_tensor: tensor 'output.weight' not found'"

I will open a issue on the llamacpp github aswell addressing this

ggerganov/llama.cpp#5706

System:
Ryzen 5600x
rtx 3080 gpu
b550 motherboard
64gb ddr4 ram
windows 10 OS

[rombodawg]

Im uploading the model files for the merges if anyone wants to do some debugging. Should be in the next 10 hours or so. Sorry slow internet.

Follow the mulit-thread. And check out my model for debugging.

Thread links:
#21
ggerganov/llama.cpp#5706
arcee-ai/mergekit#181
oobabooga/text-generation-webui#5562

https://huggingface.co/rombodawg/Gemme-Merge-Test-7b