[Feature] Implement AdaLOMO optimizer and others

[kozistr]

Problem (Why?)

lots of stuffs

Solution (What/How?)

• implement AdaLOMO optimizer
• support Q-GaLore optimizer
• support more bitbytesands optimizers
• deprecate optional dependency, bitsandbytes

Other changes (bug fixes, small refactors)

• refactor AdaMini optimizer
• move get_rms, approximate_sq_grad functions to BaseOptimizer for reusability.

Notes

N/A

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (acd218b) to head (2acade3).
Report is 152 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##              main      #258    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           90        90            
  Lines         6754      6863   +109     
==========================================
+ Hits          6754      6863   +109     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.