New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FP32/FP16精度训练 #271

Open

KompressorSC opened this issue Nov 8, 2024 · 0 comments

KompressorSC commented Nov 8, 2024

最近在Follow您的工作，有些问题想请教下。
您的训练代码中RWKV可以支持FP16/FP32精度的训练，但将您的模型代码迁移到别的训练框架时不采用BF16精度会报错，看起来似乎是CUDA和CPP文件的原因。请问怎么修改才可以使模型支持FP32/FP32的训练呢？感谢！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment