Add AdaLomo

KaiLv69 released this 24 Oct 07:23

· 20 commits to main since this release

Add AdaLomo, a more powerful and equally efficient optimizer. https://arxiv.org/abs/2310.10195.
The code is easier to use based on collie (dev-branch currently).

Assets 2