Added petals models

[filopedraz]

Description

Added Petals services

• Stable Beluga 2
• Code Llama 34B

Missing

• Falcon 180B

[biswaroop1547]

LGTM, do we plan to add something on the daemon side to choose swarm or run it in petals swarm by default?

[filopedraz]

We use by default always the public swarm (for now).

[casperdcl]

Sorry about the force-pushes - was debugging this questionable GHA CI behaviour 🤯

[casperdcl]

Are premai-io/CodeLlama-34b-Instruct-hf and petals-team/StableBeluga2 only compatible with arm64 and not amd64?

Dockerfile:16
> RUN python3 download.py --model $MODEL_ID
Floating point exception (core dumped)

Or maybe it's a Petals bug.

[filopedraz]

It's a Petals bug. I will investigate it further. Wanna have the arm images to be done so that I can create a PR to the registry and test the entire flow on Mac :)

[casperdcl]

lgtm

• did you wanna add falcon-180b first?
• you can link to Added petals models #104 or specifically 01c62ac if reporting to https://github.com/bigscience-workshop/petals/issues/ :)

[filopedraz]

Falcon requires an HF token in order to fetch the weights due to the terms of services. I am investigating here how to upload the weights to Torrent and then download them with a CLI that we can use inside a dockerfile. :)

[casperdcl]

... or just add the HF token to GH repo secrets :)