This demo demonstrates how to fine tune a LLM and build an ML application: the MLOps master bot! We'll train falcon-7b on Iguazio's MLOps blogs and cover how easy it is to take a model and code from development to production. Even if its a big scary LLM model, MLRun will take care of the dirty work!
We will use:
- HuggingFace - as the main machine learning framework to get the model and tokenizer.
- DeepSpeed - as the distributed training framework.
- and MLRun - as the orchastraitor to operationalize it, moving it from development to production.
The demo contains a single notebook that covers the two main stages in every MLOps project:
- Training Pipeline Automation - Demonstrating how to get an existing model (
falcon-7b) from HuggingFace's Transformers package and operationalize it through all of its life cycle phases: data collection, data ppreparation, training and evaluation, as a fully automated pipeline. - Application Serving Pipeline - Showing how to productize the newly trained LLM as a serverless function.
You can find all the python source code under /src
Be sure to check out Yaron Haviv's video Deploying Hugging Face Models to Production at Scale with GPUs to get a walkthrough of a simillar demo.
This project can run in different development environments:
- Local computer (using PyCharm, VSCode, Jupyter, etc.)
- Inside GitHub Codespaces
- Other managed Jupyter environments
To get started, fork this repo into your GitHub account and clone it into your development environment.
To install the package dependencies (not required in GitHub codespaces) use:
make install-requirements
If you prefer to use Conda use this instead (to create and configure a conda env):
make conda-env
Make sure you open the notebooks and select the
mlrunconda environment
The MLRun service and computation can run locally (minimal setup) or over a remote Kubernetes environment.
If your development environment support docker and have enough CPU resources run:
make mlrun-docker
MLRun UI can be viewed in: http://localhost:8060
If your environment is minimal, run mlrun as a process (no UI):
[conda activate mlrun &&] make mlrun-api
For MLRun to run properly you should set your client environment, this is not required when using codespaces, the mlrun conda environment, or iguazio managed notebooks.
Your environment should include MLRUN_ENV_FILE=<absolute path to the ./mlrun.env file> (point to the mlrun .env file
in this repo), see mlrun client setup instructions for details.
Note: You can also use a remote MLRun service (over Kubernetes), instead of starting a local mlrun, edit the mlrun.env and specify its address and credentials