[Feature] Implements D-Adaptation optimizers

[kozistr]

Problem (Why?)

new optimizers!

Solution (What/How?)

• implement DAdaptAdaGrad optimizer
• implement DAdaptAdam optimizer
• implement DAdaptSGD optimizer
• Shampoo Optimizer (related to Updated Shampoo uber slow performance #100)
  • supports no_preconditioning_for_layers_with_dim_gt
  • refactor matrix_power()
  • improve a speed of power_iter()

Other changes (bug fixes, small refactors)

• docs for D-Adaptation optimizers
• docs for Shampoo optimizer utils

Notes

bump version into v2.4.0

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.69%. Comparing base (44c423a) to head (a3a5557).
Report is 1791 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #101      +/-   ##
==========================================
+ Coverage   99.65%   99.69%   +0.03%     
==========================================
  Files          37       38       +1     
  Lines        2618     2940     +322     
==========================================
+ Hits         2609     2931     +322     
  Misses          9        9              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.