From 38419aefb0252c1e2f10a2db32d31953b4a1e8a5 Mon Sep 17 00:00:00 2001 From: kozistr Date: Fri, 6 Dec 2024 23:39:52 +0900 Subject: [PATCH 1/2] docs: v3.3.0 changelog --- docs/changelogs/v3.3.0.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/changelogs/v3.3.0.md b/docs/changelogs/v3.3.0.md index ddd33246..8c151fdb 100644 --- a/docs/changelogs/v3.3.0.md +++ b/docs/changelogs/v3.3.0.md @@ -14,6 +14,14 @@ * Improve the stability of `ADOPT` optimizer. (#294) * [Note](https://github.com/iShohei220/adopt?tab=readme-ov-file#update-on-nov-22-2024) * Support a new projection type `random` for `GaLoreProjector`. (#294) +* Implement `DeMo` optimizer. (#300, #301) + * [Decoupled Momentum Optimization](https://arxiv.org/abs/2411.19870) +* Implement `Muon` optimizer. (#302) + * [MomentUm Orthogonalized by Newton-schulz](https://github.com/KellerJordan/Muon) +* Implement `ScheduleFreeRAdam` optimizer. (#304) +* Implement `LaProp` optimizer. (#304) + * [Separating Momentum and Adaptivity in Adam](https://arxiv.org/abs/2002.04839) +* Support `Cautious` variant to `LaProp`, `AdamP`, `Adopt` optimizers. (#304). ### Refactor From ccd683d93b23b2cda96db6355ac660913f17eea7 Mon Sep 17 00:00:00 2001 From: kozistr Date: Fri, 6 Dec 2024 23:40:01 +0900 Subject: [PATCH 2/2] docs: v3.3.1 changelog --- docs/changelogs/v3.3.1.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/docs/changelogs/v3.3.1.md b/docs/changelogs/v3.3.1.md index 2a366cdf..9825faee 100644 --- a/docs/changelogs/v3.3.1.md +++ b/docs/changelogs/v3.3.1.md @@ -2,11 +2,3 @@ ### Feature -* Implement `DeMo` optimizer. (#300, #301) - * [Decoupled Momentum Optimization](https://arxiv.org/abs/2411.19870) -* Implement `Muon` optimizer. (#302) - * [MomentUm Orthogonalized by Newton-schulz](https://github.com/KellerJordan/Muon) -* Implement `ScheduleFreeRAdam` optimizer. (#304) -* Implement `LaProp` optimizer. (#304) - * [Separating Momentum and Adaptivity in Adam](https://arxiv.org/abs/2002.04839) -* Support `Cautious` variant to `LaProp`, `AdamP`, `Adopt` optimizers. (#304).