New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Milabench Size #254

Open

Delaunay opened this issue Aug 20, 2024 · 0 comments

Collaborator

Delaunay commented Aug 20, 2024

llama3 - 70B is big (132G)
We could generate it on the fly Generate llama instead of downloading it #250
- One possible issue is that it generate 8 checkpoints for 8 GPUs
- Can we load the model on 4 GPUs ?

2.5M    ./cache
325M    ./data/processed
263G    ./data/llama3_70B        # <= Duplicated weights
252M    ./data/raw
15G     ./data/llama3_8B
25G     ./data/FakeImageNet
304G    ./data
2.8M    ./runs
5.7G    ./venv/gnn                    # <= gnn need their own pytorch
6.4G    ./venv/torch           
13G     ./venv
316G    .

The text was updated successfully, but these errors were encountered:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment