diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 9a520ce..e60ed08 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -26,5 +26,6 @@ jobs: env: PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }} run: | + poetry version patch poetry config pypi-token.pypi $PYPI_TOKEN poetry publish --build diff --git a/README.md b/README.md index 4067f05..110a547 100755 --- a/README.md +++ b/README.md @@ -128,27 +128,32 @@ python -m turbo_alignment train_kto --experiment_settings_path configs/exp/train ``` -⌛️ in progress.. -## Multimodal Tasks +## Multimodal train +⌛️ in progress.. + -⌛️ in progress.. -## RAG -To launch RAG: -- **Choose a base encoder**, create a document index. -- For **end-to-end**: - - **Train both** the retriever and the generator. - - **Prepare the data** in `"dataset_type": "chat"` **with query -> response.** - - **Suitable config**: [end2end_rag](configs/exp/train/rag/end2end_rag.json) +## RAG (Retrieval-Augmented Generation) + + +### SFT-RAG +- **📚 Dataset type**: prepare your dataset in `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format. +- **📝 Configs Example**: [sft_with_retrieval_utility](configs/exp/train/sft/llama/sft_with_retrieval_utility.json) +- **🖥️ CLI launch command**: ```bash +python -m turbo_alignment train_sft --experiment_settings_path configs/exp/train/sft/llama/sft_with_retrieval_utility.json +``` -- For **sft-rag**: - - **Train only** generator - - **Prepare the data** in `"dataset_type": "chat"` with **query+retrieved_documents -> response.** - - **Suitable config**: [sft_with_retrieval_utility](configs/exp/train/sft/llama/sft_with_retrieval_utility.json) + +### End2End-RAG +- **📚 Dataset type**: prepare your dataset in `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format. +- **📝 Configs Example**: [end2end_rag](configs/exp/train/rag/end2end_rag.json) +- **🖥️ CLI launch command**: ```bash +python -m turbo_alignment train_rag --experiment_settings_path configs/exp/train/rag/end2end_rag.json +``` @@ -156,15 +161,15 @@ To launch RAG: ## Chat Inference - **📚 Dataset type** prepare your dataset in the `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format. -- **📝 Configs Example**: [sft.json](configs/exp/train/sft/sft.json) +- **📝 Configs Example**: [sft.json](configs/exp/inference/generation/default_llama_adapter.json) - **🖥️ CLI launch command** ```bash -python -m turbo_alignment inference_chat --inference_settings_path configs/exp/train/sft/sft.json +python -m turbo_alignment inference_chat --inference_settings_path configs/exp/inference/generation/default_llama_adapter.json ``` ## Classification Inference -- **📚 Dataset type** prepare your dataset in the `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format. +- **📚 Dataset type** prepare your dataset in the `ClassificationDataset`, examples available [here](docs/dataset_example.md#-classification-dataset) format. - **📝 Configs Example**: [classification_inference.json](configs/exp/inference/classification/classification_inference.json) - **🖥️ CLI launch command** ```bash @@ -173,11 +178,11 @@ python -m turbo_alignment inference_classification --inference_settings_path con ## Multimodal Inference -- **📚 Dataset type** prepare your dataset in the `ChatDataset`, examples available [here](docs/dataset_example.md#-chat-dataset) format. +- **📚 Dataset type** prepare your dataset in the `MultimodalDataset`, examples available [here](docs/dataset_example.md#-multimodal-dataset) format. - **📝 Configs Example**: [mlp.json](configs/exp/inference/multimodal/mlp.json) - **🖥️ CLI launch command** ```bash -python -m turbo_alignment inference_multimodal --inference_settings_path configs/exp/train/sft/sft.json +python -m turbo_alignment inference_multimodal --inference_settings_path configs/exp/inference/multimodal/mlp.json ``` @@ -186,7 +191,7 @@ python -m turbo_alignment inference_multimodal --inference_settings_path configs - **📝 Configs Example**: [rag_inference.json](configs/exp/inference/rag/rag_inference.json) - **🖥️ CLI launch command** ```bash -python -m turbo_alignment inference_rag --inference_settings_path configs/exp/train/sft/sft.json +python -m turbo_alignment inference_rag --inference_settings_path configs/exp/inference/rag/rag_inference.json ``` @@ -211,7 +216,7 @@ python -m turbo_alignment rso_sample --experiment_settings_path tests/fixtures/c ## Reward Model Sampling -- **📚 Dataset type** prepare your dataset in the `ChatDataset`, examples available [here](docs/dataset_example.md#-sampling-dataset) format. +- **📚 Dataset type** prepare your dataset in the `SamplingRMDataset`, examples available [here](docs/dataset_example.md#-sampling-dataset) format. - **📝 Configs Example**: [rm.json](tests/fixtures/configs/sampling/rm.json) - **🖥️ CLI launch command** ```bash @@ -292,6 +297,13 @@ Tutorials are available [here](tutorials/tutorial.md). This project is licensed, see the [LICENSE](https://github.com/turbo-llm/turbo-alignment/-/blob/main/LICENSE) file for details. +## References + +- DPO Trainer implementation inspired by Leandro von Werra et al. (2020) TRL: Transformer Reinforcement Learning. GitHub repository, GitHub. Available at: [https://github.com/huggingface/trl](https://github.com/huggingface/trl). + +- Registry implementation inspired by Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. AllenNLP: A Deep Semantic Natural Language Processing Platform. Available at: [arXiv:1803.07640](https://arxiv.org/abs/1803.07640). + + ## Citation ```bibtex diff --git a/pyproject.toml b/pyproject.toml index 843713b..543e268 100755 --- a/pyproject.toml +++ b/pyproject.toml @@ -3,7 +3,7 @@ name = "turbo-alignment" packages = [ { include = "turbo_alignment" }, ] -version = "0.0.3" +version = "0.0.2" description = "turbo-alignment repository" authors = ["T Mega Alignment Team " ] readme = "README.md"