You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'd be very interested in how we could take llm.c models and export them into universal formats, e.g. for very fast inference in llama.cpp, vllm, or etc. Or how they could be made HuggingFace compatible. This would also allow us to run more comprehensive evals on the models that we train in llm.c, because it would (hopefully) slot into other existing infrastructure in those projects.
The text was updated successfully, but these errors were encountered:
@YuchenJin yep exactly what I had in mind! I put up the issue because I am sequencing other things before I get around to it, possibly someone can pick it up in parallel before.
I'd be very interested in how we could take llm.c models and export them into universal formats, e.g. for very fast inference in llama.cpp, vllm, or etc. Or how they could be made HuggingFace compatible. This would also allow us to run more comprehensive evals on the models that we train in llm.c, because it would (hopefully) slot into other existing infrastructure in those projects.
The text was updated successfully, but these errors were encountered: