update poetry version

.github/workflows/release.yml

@@ -26,5 +26,6 @@ jobs:
         env:
           PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
         run: |
+          poetry version patch
           poetry config pypi-token.pypi $PYPI_TOKEN
           poetry publish --build

README.md

@@ -128,43 +128,48 @@ python -m turbo_alignment train_kto --experiment_settings_path configs/exp/train
 ```
 
 <a name="-multimodal-train"></a>
-⌛️ in progress..
-## Multimodal Tasks 
+## Multimodal train
+⌛️  in progress..
+<!-- ## Multimodal Tasks 
 To start multimodal training, you should:
 - **Prepare the multimodal dataset**. See examples [here](docs/dataset_example.md#-multimodel-dataset).
 - **Preprocess the data (OPTIONAL)**. If you plan to run many experiments on the same dataset, you should preprocess it. The preprocessing stage includes reading `pixel_values` from images, encoding them with the specified encoder, and saving them in safetensors format. Later, during training, the pipeline will skip the stage of reading and encoding images and only extract prepared encodings from the safetensors files.
-- **Suitable config**: [llava.json](tests/fixtures/configs/train/multimodal/llama_llava_base_clip.json),[c_abs.json](tests/fixtures/configs/train/multimodal/llama_c_abs_clip_pickle.json)
+- **Suitable config**: [llava.json](tests/fixtures/configs/train/multimodal/llama_llava_base_clip.json),[c_abs.json](tests/fixtures/configs/train/multimodal/llama_c_abs_clip_pickle.json) -->
 
 <a name="-rag-train"></a>
-⌛️ in progress..
-## RAG
-To launch RAG:
-- **Choose a base encoder**, create a document index.
-- For **end-to-end**:
-  - **Train both** the retriever and the generator.
-  - **Prepare the data** in `"dataset_type": "chat"` **with query -> response.**
-  - **Suitable config**: [end2end_rag](configs/exp/train/rag/end2end_rag.json)
+## RAG (Retrieval-Augmented Generation) 
+
+<a name="-sft-rag-train"></a>
+### SFT-RAG
+- **📚 Dataset type**: prepare your dataset in `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format.
+- **📝 Configs Example**: [sft_with_retrieval_utility](configs/exp/train/sft/llama/sft_with_retrieval_utility.json)
+- **🖥️ CLI launch command**: ```bash
+python -m turbo_alignment train_sft --experiment_settings_path configs/exp/train/sft/llama/sft_with_retrieval_utility.json
+```
 
-- For **sft-rag**:
-  - **Train only** generator
-  - **Prepare the data** in `"dataset_type": "chat"` with **query+retrieved_documents -> response.**
-  - **Suitable config**: [sft_with_retrieval_utility](configs/exp/train/sft/llama/sft_with_retrieval_utility.json)
+<a name="-e2e-rag-train"></a>
+### End2End-RAG
+- **📚 Dataset type**: prepare your dataset in `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format.
+- **📝 Configs Example**: [end2end_rag](configs/exp/train/rag/end2end_rag.json)
+- **🖥️ CLI launch command**: ```bash
+python -m turbo_alignment train_rag --experiment_settings_path configs/exp/train/rag/end2end_rag.json
+```
 
 
 <a name="-inference"></a>
 # Inference
 <a name="-chat-inference"></a>
 ## Chat Inference
 - **📚 Dataset type** prepare your dataset  in the `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format.
-- **📝 Configs Example**: [sft.json](configs/exp/train/sft/sft.json)
+- **📝 Configs Example**: [sft.json](configs/exp/inference/generation/default_llama_adapter.json)
 - **🖥️ CLI launch command**
 ```bash
-python -m turbo_alignment inference_chat --inference_settings_path configs/exp/train/sft/sft.json
+python -m turbo_alignment inference_chat --inference_settings_path configs/exp/inference/generation/default_llama_adapter.json
 ```
 
 <a name="-classification-inference"></a>
 ## Classification Inference
-- **📚 Dataset type** prepare your dataset  in the `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format.
+- **📚 Dataset type** prepare your dataset  in the `ClassificationDataset`, examples available [here](docs/dataset_example.md#-classification-dataset) format.
 - **📝 Configs Example**: [classification_inference.json](configs/exp/inference/classification/classification_inference.json)
 - **🖥️ CLI launch command**
 ```bash
@@ -173,11 +178,11 @@ python -m turbo_alignment inference_classification --inference_settings_path con
 
 <a name="-multimodal-inference"></a>
 ## Multimodal Inference
-- **📚 Dataset type** prepare your dataset  in the `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format.
+- **📚 Dataset type** prepare your dataset  in the `MultimodalDataset`, examples available [here](docs/dataset_example.md#-multimodal-dataset) format.
 - **📝 Configs Example**: [mlp.json](configs/exp/inference/multimodal/mlp.json)
 - **🖥️ CLI launch command**
 ```bash
-python -m turbo_alignment inference_multimodal --inference_settings_path configs/exp/train/sft/sft.json
+python -m turbo_alignment inference_multimodal --inference_settings_path configs/exp/inference/multimodal/mlp.json
 ```
 
 <a name="-rag-inference"></a>
@@ -186,7 +191,7 @@ python -m turbo_alignment inference_multimodal --inference_settings_path configs
 - **📝 Configs Example**: [rag_inference.json](configs/exp/inference/rag/rag_inference.json)
 - **🖥️ CLI launch command**
 ```bash
-python -m turbo_alignment inference_rag --inference_settings_path configs/exp/train/sft/sft.json
+python -m turbo_alignment inference_rag --inference_settings_path configs/exp/inference/rag/rag_inference.json
 ```
 
 <a name="-sampling"></a>
@@ -211,7 +216,7 @@ python -m turbo_alignment rso_sample --experiment_settings_path tests/fixtures/c
 
 <a name="-rm-sampling"></a>
 ## Reward Model Sampling
-- **📚 Dataset type** prepare your dataset  in the `ChatDataset`, examples available [here](docs/dataset_example.md#-sampling-dataset) format.
+- **📚 Dataset type** prepare your dataset  in the `SamplingRMDataset`, examples available [here](docs/dataset_example.md#-sampling-dataset) format.
 - **📝 Configs Example**: [rm.json](tests/fixtures/configs/sampling/rm.json)
 - **🖥️ CLI launch command**
 ```bash
@@ -292,6 +297,13 @@ Tutorials are available [here](tutorials/tutorial.md).
 This project is licensed, see the [LICENSE](https://github.com/turbo-llm/turbo-alignment/-/blob/main/LICENSE) file for details.
 
 
+## References
+
+- DPO Trainer implementation inspired by Leandro von Werra et al. (2020) TRL: Transformer Reinforcement Learning. GitHub repository, GitHub. Available at: [https://github.com/huggingface/trl](https://github.com/huggingface/trl).
+
+- Registry implementation inspired by Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. AllenNLP: A Deep Semantic Natural Language Processing Platform. Available at: [arXiv:1803.07640](https://arxiv.org/abs/1803.07640).
+
+
 ## Citation
 
 ```bibtex

pyproject.toml

@@ -3,7 +3,7 @@ name = "turbo-alignment"
 packages = [
     { include = "turbo_alignment" },
 ]
-version = "0.0.3"
+version = "0.0.2"
 description = "turbo-alignment repository"
 authors = ["T Mega Alignment Team <[email protected]>" ]
 readme = "README.md"