Automating Pre-Building of Wheels for All Platforms (v0.3.4 and Beyond)

[FarisZahrani]

Hi everyone,

I’ve been working on an app that uses llama-cpp-python, and I’m currently researching the process of pre-building wheels. Specifically, I’m trying to figure out how the wheels for versions like v0.3.4 and v0.2.90 were built.

The reason I’m looking into automating this is that building wheels from source takes a very long time, and I’d like to explore ways to make this process faster and more efficient. Automating this would not only save time for developers, but also make it much easier for the community to get up and running quickly across different platforms.

I’m interested in the possibility of creating wheels for the available platforms:

• CUDA
• Metal
• hipBLAS (ROCm)
• Vulkan
• SYCL
• RPC

It seems like this could be automated using tools like:

• GitHub Actions for CI/CD workflows.
• Docker for reproducible local builds across platforms.

If anyone, including @abetlen, has any existing workflows, processes, or insights on how this is currently handled, I’d really appreciate learning more. I’m trying to piece this together and would love to contribute where I can to help automate and expand the wheel-building process.

Thanks for any input!

Best,
Faris