[Extension Proposal] Huggingface Downloader extension

[lordofthejars]

Description

I am unsure if this should be an extension, but the idea is to create an extension based on jgit that lets Quarkus users download models either programmatically or declaratively.

This is useful for projects like DJL that expect the model to be locally downloaded.

Repository name

quarkus-huggingface-downloader

Short description

Quarkusextension to download huggingface models

Repository Homepage URL

https://docs.quarkiverse.io/quarkus-huggingface-downloader/dev/

Repository Topics

• quarkus-extension
• huggingface
• ai

Team Members

• @lordofthejars

Additional context

No response

[quarkus-bot]

/cc @aloubyansky (extension-proposal), @gastaldi (extension-proposal), @gsmet (extension-proposal), @maxandersen (extension-proposal)

[gastaldi]

Sounds like it could be a CLI extension (or a JBang script)? Not sure how useful would that be for a standalone Quarkiverse extension

[lordofthejars]

Well, the thing is that a user might need to do this programmatically. I know it is not the same but think like Flyway for DBs. You bootup the app and check if you want to apply a DB migration, this would be something similar, you bootup the app, check if the local model is there/updated, and update if necessary.

While I write this, I see that it can be a more complex extension that I though as we could control the versions, tags and so on, and update the model accordantly following some kind of name convention as in Flyway.

Well again, I don't know if this could have the entity of an extension, for this reason I raised the question :)

[geoand]

I am not opposed to this, but I think we need to answer the following questions:

• Does it really need to be an extension? If so, why?
• If it does make sense to have an extension, should it be standalone, or could it be part of Quarkus LangChain4j?

[lordofthejars]

Hi, yes, I don't know where it should be; I have seen that it is not as simple as I thought because of LFS. In theory jgit supports LFS natively in Java or using git lfs. Using the git lfs support works, but when trying to use the native LFS in Java, it is not working, there is no documentation at all, and what I do all the time is find tests to try to make it work, now without much success. So, long story short, it seems that it would be interesting to have this code somewhere.

[maxandersen]

Huggingface has a rest API - can't that be used?

Btw. Just downloading it is just part of it - other is having it put a place where tbe LLM runtime will read it from...ollama, Jlama, podman ai currently read from different locations.

What's your thoughts there ?

[lordofthejars]

I know I can query a repo using a Rest API but what I have not found is downloading the repo using a Rest API from HuggingFace.