Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[fix] Fix the activation checkpointing when using SwiGLUPackedFusedOp
According to the docs (https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function) forward() method should not be called directly, apply() method have to be used instead. After removing forward call, activation checkpointing starts working. (alternative variant 2)
- Loading branch information