Skip to content

Latest commit

 

History

History

examples

NVIDIA FLARE Examples

NVIDIA FLARE provides several examples to help you get started using federated learning for your own applications.

The provided examples cover different aspects of NVIDIA FLARE, such as using the provided Controllers for "scatter and gather" or "cyclic weight transfer" workflows and different Executors to implement your own training and validation pipelines. Some examples use the provided "task data" and "task result" Filters for homomorphic encryption and decryption or differential privacy. Furthermore, we show how to use different components for FL algorithms such as FedAvg, FedProx, and FedOpt. We also provide domain-specific examples for deep learning and medical image analysis.

Getting started

To get started with NVIDIA FLARE, please follow the Getting Started Guide in the documentation. This walks you through installation, creating a POC workspace, and deploying your first NVIDIA FLARE Application. The following examples will detail any additional requirements in their requirements.txt.

Set up a virtual environment

We recommend setting up a virtual environment before installing the dependencies of the examples. You need to set up the virtual environment and install nvflare and set additional PYTHONPATH before launch the jupyter lab.

Install dependencies for a virtual environment with:

python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv

(If needed) make all shell scripts executable using:

find . -name ".sh" -exec chmod +x {} \;

Create and activate your virtual environment with the set_env.sh script:

source ./set_env.sh

Install nvflare

(nvflare_example)$ pip install nvflare

In each example folder, install required packages for training:

(nvflare_example)$ pip install --upgrade pip
(nvflare_example)$ pip install -r requirements.txt

(optional) some examples contains script for plotting the TensorBoard event files, if needed, please also install:

(nvflare_example)$ pip install -r plot-requirements.txt

Set up JupyterLab for notebooks

To run examples including notebooks, we recommend using JupyterLab. You need to set up the virtual environment and install nvflare and set additional PYTHONPATH before launch the jupyter lab.

After activating your virtual environment, install JupyterLab:

(nvflare_example)$ pip install jupyterlab

You can register the virtual environment you created, so it is usable in JupyterLab:

(nvflare_example)$ python3 -m ipykernel install --user --name="nvflare_example"

Start a Jupyter Lab:

(nvflare_example)$ jupyter lab .

When you open a notebook, select the kernel nvflare_example using the dropdown menu at the top right. Selecting a JupyterLab kernel

1. Hello World Examples

Example Framework Summary
Notebook for Hello Examples - Notebook for examples below.
Hello FedAvg NumPy Numpy Example using FedAvg controller workflow.
Hello Cross-Site Validation Numpy Example using CrossSiteEval controller workflow, and example using previous results without training workflow.
Hello Cyclic Weight Transfer PyTorch Example using CyclicController controller workflow to implement Cyclic Weight Transfer.
Hello PyTorch PyTorch Example using an image classifier using FedAvg and PyTorch as the deep learning training framework.
Hello TensorFlow TensorFlow Example of using an image classifier using FedAvg and TensorFlow as the deep learning training framework.

2. Step-by-Step Examples

Example Dataset Controller-Type Execution API Type Framework Summary
image_stats CIFAR10 server Executor Pandas Example for federated stats image histogram calculation.
sag CIFAR10 server Client API PyTorch Example for FedAvg with ScatterAndGather controller workflow using the Client API.
sag_deploy_map CIFAR10 server Client API PyTorch Example showcasing site-specific configurations and deploy_map.
sag_model_learner CIFAR10 server ModelLearner PyTorch Example with ScatterAndGather using a ModelLearner.
sag_executor CIFAR10 server Executor PyTorch Example with ScatterAndGather using an Executor.
sag_mlflow CIFAR10 server Client API PyTorch MLflow experiment tracking logs with ScatterAndGather using the Client API.
sag_he CIFAR10 server Client API PyTorch Example with homomorphic encyption using Client API and POC -he mode.
cse CIFAR10 server Client API PyTorch Example using CrossSiteModelEval controller workflow.
cyclic CIFAR10 server Client API PyTorch Example for cyclic weight transfer using CyclicController controller workflow.
cyclic_ccwf CIFAR10 client Client API PyTorch Example for client-controlled cyclic weight transfer using CyclicClientController controller workflow.
swarm CIFAR10 client Client API PyTorch Example for swarm learning and client-controlled cross-site evaluation using SwarmClientController and CrossSiteEvalClientController controller workflows.
tabular_stats HIGGS server Executor Pandas Example for federated stats tabular histogram calculation.
sklearn_linear HIGGS server Client API sklearn Example for federated linear model (logistic regression on binary classification) learning on tabular data.
sklearn_svm HIGGS server Client API sklearn Example for federated SVM model learning on tabular data.
sklearn_kmeans HIGGS server Client API sklearn Example for federated k-Means clustering on tabular data.
xgboost HIGGS server Client API XGBoost Example for federated horizontal xgboost learning on tabular data with bagging collaboration.

3. Tutorial notebooks

Example Summary
Intro to the FL Simulator Shows how to use the FLARE Simulator to run a local simulation.
Hello FLARE API Goes through the different commands of the FLARE API.
NVFLARE in POC Mode Shows how to use POC mode.
Job CLI Walks through the different commands of the Job CLI.

4. FL algorithms

Example Summary
Simulated Federated Learning with CIFAR-10 This example includes instructions on running FedAvg, FedProx, FedOpt, and SCAFFOLD algorithms using NVFlare's FL simulator.
Real-world Federated Learning with CIFAR-10 Includes instructions on running FedAvg with streaming of TensorBoard metrics to the server during training and homomorphic encryption.

5. Traditional ML examples

Example Framework Notebooks Summary
Federated Linear Model with Scikit-learn scikit-learn FL Model with Scikit-learn on HIGGS Dataset Shows how to use the NVIDIA FLARE with scikit-learn, a widely used open-source machine learning library.
Federated K-Means Clustering with Scikit-learn scikit-learn Federated K-Means Clustering with Scikit-learn on Iris Dataset NVIDIA FLARE with scikit-learn and k-Means.
Federated SVM with Scikit-learn scikit-learn Federated SVM with Scikit-learn on Breast Cancer Dataset NVIDIA FLARE with scikit-learn and SVM.
Histogram-based FL for XGBoost XGBoost Histogram-based FL for XGBoost on HIGGS Dataset Histogram-based algorithm for XGBoost
Tree-based Federated Learning for XGBoost XGBoost Tree-based FL for XGBoost on HIGGS Dataset Tree-based algorithms includes bagging and cyclic approaches.
Federated Learning for Random Forest based on XGBoost XGBoost Federated Random Forest on HIGGS Dataset Example of using NVIDIA FLARE with scikit-learn and Random Forest.
Federated Vertical XGBoost XGBoost Federated Vertical XGBoost Example using Private Set Intersection and XGBoost on vertically split HIGGS data.

6. Medical Image Analysis

Example Framework Summary
NVFlare + MONAI integration MONAI For an example of using NVIDIA FLARE to train a 3D medical image analysis model using federated averaging (FedAvg) and MONAI Bundle, see here.
Federated Learning with Differential Privacy for BraTS18 segmentation MONAI Illustrates the use of differential privacy for training brain tumor segmentation models using federated learning.
Federated Learning for Prostate Segmentation from Multi-source Data MONAI Example of training a multi-institutional prostate segmentation model using FedAvg, FedProx, and Ditto.

7. Federated Statistics

Federated Statistics Overview discusses the overall federated statistics features.

Example Notebooks Summary
Federated Statistics for Images Image Histograms Example of gathering local image histogram to compute the global dataset histograms.
Federated Statistics for DataFrame Data Frame Federated Statistics, Visualization Example of gathering local statistics summary from Pandas DataFrame to compute the global dataset statistics.
Federated Hierarchical Statistics for DataFrame Federated Hierarchical Statistics, Visualization Example of generating federated hierarchical statistics for data that can be represented as Pandas DataFrame.

8. Federated Policies

Example Summary
Federated Policies Discuss the federated site policies for authorization, resource and data privacy management.
Custom Authentication Demonstrate the custom authentication policy and secure mode.
Job-Level Authorization Demonstrate the job-level authorization policy and secure mode.
KeyCloak Site Authentication Integration Demonstrate KeyCloak integration for supporting site-specific authentication.

9. Experiment tracking

Example Framework Summary
Hello PyTorch with TensorBoard Streaming PyTorch Example building upon Hello PyTorch showcasing the TensorBoard streaming capability from the clients to the server.
FL Experiment Tracking with MLflow PyTorch Example integrating Hello PyTorch with MLflow streaming capability from clients to the server.
FL Experiment Tracking with Weights and Biases PyTorch Example integrating Hello PyTorch with Weights and Biases streaming capability from clients to the server.
MONAI FLARE Integration Experiment Tracking MONAI Example using FLARE and MONAI integration with experiment tracking streaming from clients to server.

10. NLP

Example Summary
NLP-NER Illustrates both BERT and GPT-2 models from Hugging Face (BERT-base-uncased, GPT-2) on a Named Entity Recognition (NER) task using the NCBI disease dataset.

11. Federated Learning Hub

Example Framework Summary
FL Hub PyTorch/MONAI Allow hierarchical interaction between several levels of nvflare FL systems, e.g. Tier-1 (hub) and Tier-2 (sub-systems).

12. Federated Large Language Model (LLM)

Example Framework Summary
Parameter Efficient Fine Turning NeMo Example utilizing NeMo's PEFT methods to adapt a LLM to a downstream task.
Prompt-Tuning Example NeMo Example for using FLARE with NeMo for prompt learning.
Supervised Fine Tuning (SFT) NeMo Example to fine-tune all parameters of a LLM on supervised data.
LLM Tuning via HuggingFace SFT Trainer NeMo Example for using FLARE with a HuggingFace trainer for LLM tuning tasks.

13. Graph Neural Network (GNN)

Example Framework Summary
Protein Classification PyTorch Geometric Example using GNNs for Protein Classification using PPI dataset using GraphSAGE.
Financial Transaction Classification PyTorch Geometric Example using GNNs for Financial Transaction Classification with Elliptic++ dataset using GraphSAGE.

14. Financial Applications

Example Framework Summary
Financial Application with Federated XGBoost Methods XGBoost Example using XGBoost in various ways to train a federated model to perform fraud detection with a finance dataset.