Skip to content

v2.5.0

Compare
Choose a tag to compare
@Goekdeniz-Guelmez Goekdeniz-Guelmez released this 02 Sep 19:55

v2.5.0

  • Adding a MOE model (KANamav5)
  • Adding a better trainer and fixing batching function
  • Adding more examples, with tiny-shakespear.txt and fineweb.jsonl datasets
  • Adding loading bars to the SFTTrainer
  • General fixes