Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support the new format of llama.cpp "gguf"? #993

Closed
0x766c6f6f6b opened this issue Aug 30, 2023 · 1 comment
Closed

Support the new format of llama.cpp "gguf"? #993

0x766c6f6f6b opened this issue Aug 30, 2023 · 1 comment
Labels
enhancement New feature or request primordial Related to the primordial version of PrivateGPT, which is now frozen in favour of the new PrivateGPT

Comments

@0x766c6f6f6b
Copy link

A new file format has been introduced: ggerganov/llama.cpp#2398
I want to use my own model for privateGPT, then i download it from HF, but i realize that the hf format need to convert to ggml.
The previous version of llama.cpp(https://codeload.github.com/ggerganov/llama.cpp/zip/refs/tags/master-9e232f0) have a python script(convert.py) can convert hf format to ggml, but it doesn't work now.(maybe before either)
The latest llama.cpp include a script called convert.py can convert hf format to the new format "gguf",and other format can also convert to "gguf".(i don't know wether it is a good idea to introduce this new format but these guys have done a lot of work for this new format).So the problem is that the privategpt can not gguf format model. Is it a better idea to support the new gguf format?

@0x766c6f6f6b 0x766c6f6f6b added the enhancement New feature or request label Aug 30, 2023
@abhishekbhakat
Copy link

abhishekbhakat commented Aug 30, 2023

It already does #994 (comment)

@imartinez imartinez added the primordial Related to the primordial version of PrivateGPT, which is now frozen in favour of the new PrivateGPT label Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request primordial Related to the primordial version of PrivateGPT, which is now frozen in favour of the new PrivateGPT
Projects
None yet
Development

No branches or pull requests

3 participants