Replies: 1 comment 1 reply
-
|
没有做改动吧，只有transformers库更新了，用了新的写法，算法是一样的 |
Beta Was this translation helpful? Give feedback.
1 reply
-
这里 _attn_implementation print出来其实是eager
我查了下代码，发现是config.json里的"attn_implementation": "sdpa", 参数实际并没有生效导致的，在运行的时候使用了默认的eager attention实现，才导致长文本OOM的 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
求问是这期间做了什么改动吗
硬件
A100
显存40G
用的是transformers的model.generate接口
Beta Was this translation helpful? Give feedback.
All reactions