Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model download URLs? #59

Open
SoftologyPro opened this issue Jul 17, 2024 · 1 comment
Open

Model download URLs? #59

SoftologyPro opened this issue Jul 17, 2024 · 1 comment

Comments

@SoftologyPro
Copy link

SoftologyPro commented Jul 17, 2024

I see this in the readme
Supports EXL2, GPTQ and FP16 models
but no links to the models themselves?
Can you give me the HF URLs for those recommended models? Or the models you think are "best" for use with ExUI (with a 24 GB VRAM GPU)?
Thanks.

@turboderp
Copy link
Owner

turboderp commented Jul 18, 2024

I have a bunch of EXL2 models on HF here. Other than that you can simply search for "EXL2" or "GPTQ" on HF and you should get lots of results. The FP16 format is just the standard HF format. Supported architectures should load without any quantization if you have enough VRAM.

Which model to choose depends what you're trying to do. Gemma2-27B-it is a very strong model that should work well on 24 GB GPU at about 4-5 bpw.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants