Skip to content

Issues: jiaweizzhao/GaLore

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

36 Open 16 Closed
36 Open 16 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

loss figure data
#61 opened Sep 2, 2024 by BaohaoLiao
ValueError: can't optimize a non-leaf Tensor (param.is_leaf=False,param.retains_grad=False)
#60 opened Aug 21, 2024 by liveck
1
Results vs FP32
#59 opened Jul 30, 2024 by tsengalb99
Zero Loss: The algorithm failed to converge because the input matrix is ill-conditioned or has too many repeated singular values
#58 opened Jul 23, 2024 by akjindal53244
1
Figure 1 clarification on batch size and sequence length
#57 opened Jul 17, 2024 by psandovalsegura
1
Questions about glue task report scores
#56 opened Jul 10, 2024 by MYT677
Support for DDP with multi-gpus
#55 opened Jul 8, 2024 by seongjunyun
Why not reproject the internal Adam states during update_proj_gap?
#54 opened Jun 30, 2024 by liuliu
2
Does galore save gradient memory?
#53 opened Jun 17, 2024 by jinqixiao
1
(Question) About glue tasks
#52 opened Jun 13, 2024 by ZhichaoWang091732
3
Galore finetuning #stopped
#51 opened Jun 7, 2024 by j-datta
IndexError: tuple index out of range
#47 opened May 13, 2024 by zyushun
11
When I used galore on orpo, the learning rate was set to 8e-6, but the training rate was 0.01
#46 opened May 10, 2024 by Minami-su
1
torch_run.py lacking autocast and scaling for Automatic Mixed Precision
#45 opened May 9, 2024 by bhavnicksm
1
Questions about reproducing the result of "Benchmark 2: Fine-Tuning RoBERTa on GLUE tasks"
#44 opened May 4, 2024 by JamesSand
2
Galore unstable on Llama 7B beyond 20K steps
#43 opened May 2, 2024 by kyleliang919
1
Questions about Figure 3 in the original paper
#42 opened May 1, 2024 by fy817
ValueError: some parameters appear in more than one parameter group
#41 opened Apr 27, 2024 by jiaohuix
How many GB memory is required to train the 7b model using DDP mode with galore?
#40 opened Apr 23, 2024 by zhangqijun
1
can support llava model ?
#39 opened Apr 14, 2024 by awzhgw
Release of Trained Models
#38 opened Apr 9, 2024 by JLake310
Any plan for the first stable release?
#36 opened Apr 8, 2024 by wsp317
Resume function for optimizer
#35 opened Apr 3, 2024 by bokyeong1015
Support for Jamba (ai21labs/Jamba-v0.1)
#34 opened Apr 2, 2024 by creatorrr
1
Dataset loading issue, integration with Colossal-AI
#33 opened Mar 29, 2024 by Edenzzzz
3
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.