New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimizer offloading 太强了 #311

Open

154912369 opened this issue Aug 2, 2024 · 2 comments

154912369 commented Aug 2, 2024

可以相对低资源的训练较大模型了，感谢大佬们

Collaborator

jerryli1981 commented Aug 6, 2024

谢谢鼓励和支持哈

WuNein commented Nov 4, 2024

H100 开OPTIMIZER_OFFLOAD = auto 性能损失非常小！尤其是batch size比较大的时候！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment