Check out our blogpost about how to to fine-tune your RAG pipeline using Fondant!
This repository contains data pipelines and ready-to-use notebooks for tuning RAG systems both manually and automatically using parameter search. To achieve this, it leverages Fondant, a free and open source framework for production-ready, easy and shareable data processing. Check out the Fondant website if you want to learn more and join our Discord if you want to stay up to date.
A notebook with a simple Fondant pipeline to index your data into a RAG system.
A notebook which iteratively runs a Fondant pipeline to evaluate a RAG system using RAGAS.
⚠️ Prerequisites:
- A Python version between 3.8 and 3.10 installed on your system.
- Docker and docker compose installed and configured on your system. More info here.
- A GPU is recommended to run the model-based components of the pipeline.
Clone this repository to your local machine using one of the following commands:
HTTPS
git clone https://github.com/ml6team/fondant-usecase-rag.gitSSH
git clone [email protected]:ml6team/fondant-usecase-rag.gitpip install -r requirements.txtConfirm that Fondant has been installed correctly on your system by executing the following command:
fondant --helpThere are two options to run the pipeline:
- Via python files and the Fondant CLI: how you should run Fondant in production
- Via a Jupyter notebook: ideal to learn about Fondant