Turbo-Alignment v0.0.2 Release Notes 🚀

🤗 Add SimPO and ORPO Trainers
- Introducing the ORPO and SimPO trainers. These new additions allow you to experiment with cutting-edge preference optimization methods, which are doesn't require reference model.
🔧 Fix SLiC-HF Trainer
- We've resolved issues that prevented the use of the SLiC-HF trainer.

📘 Add Multimodal Tutorial and Docs
- Enhance your skills with our new tutorials and documentation designed for multimodal training pipelines.

🐛 Fix Embeddings Initialization Strategy for GPT-NeoX
- Token embeddings initialization now supports both LLama and GPT-NeoX architectures.
📈 Fix Multiple Logits in Chat Generator
- The chat generator has been updated to handle multiple logits.
🔍 Fix Type of ID for Answer Message in rewards.py
- Answer message IDs are now correctly typed as strings to match pydantic model expectations.
🛑 Add stop_strings into Chat Generator and Fix Multiple EOS Problem
- Now, the chat generator manages multiple End-Of-Sequence tokens and incorporates using strings as EOS tokens.
🔄 .to('cpu') Per Batch into Chat Generator
- We've optimized memory usage for batched chat generation by moving batches individually to the CPU.
✅ Fix model.config.use_cache When Not Using Grad Checkpointing
- The KV-Cache now behaves predictably, ensuring proper functionality even in absence of gradient checkpointing.

You can view the complete list of changes in this release by visiting the changelog on GitHub: Full Changelog.

We hope you enjoy these updates! As always, we welcome your feedback and contributions to make Turbo-Alignment even better.

Don't forget to star ⭐️ the repo if you find it useful, and watch it for future updates.

Thank you for supporting Turbo-Alignment! 🙌

Need help or have questions? Reach out to us on GitHub Issues, and we’ll be there to support you.

Installation

Upgrade to the latest Turbo-Alignment release with:

pip install turbo-alignment==0.0.2

— Turbo-Alignment Team 🤫