From fddbab79932eedf1a78041ef38c47df80ab84c90 Mon Sep 17 00:00:00 2001 From: Sayak Paul Date: Sat, 26 Oct 2024 22:13:03 +0900 Subject: [PATCH] [research_projects] Update README.md to include a note about NF5 T5-xxl (#9775) Update README.md --- examples/research_projects/flux_lora_quantization/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/examples/research_projects/flux_lora_quantization/README.md b/examples/research_projects/flux_lora_quantization/README.md index ffec85550e51..51005b640221 100644 --- a/examples/research_projects/flux_lora_quantization/README.md +++ b/examples/research_projects/flux_lora_quantization/README.md @@ -5,7 +5,8 @@ This example shows how to fine-tune [Flux.1 Dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) with LoRA and quantization. We show this by using the [`Norod78/Yarn-art-style`](https://huggingface.co/datasets/Norod78/Yarn-art-style) dataset. Steps below summarize the workflow: -* We precompute the text embeddings in `compute_embeddings.py` and serialize them into a parquet file. +* We precompute the text embeddings in `compute_embeddings.py` and serialize them into a parquet file. + * Even though optional, we load the T5-xxl in NF4 to further reduce the memory foot-print. * `train_dreambooth_lora_flux_miniature.py` takes care of training: * Since we already precomputed the text embeddings, we don't load the text encoders. * We load the VAE and use it to precompute the image latents and we then delete it. @@ -163,4 +164,4 @@ image.save("yarn_merged.png") |-------|-------| | ![Image A](https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/quantized_flux_training/merged.png) | ![Image B](https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/quantized_flux_training/unmerged.png) | -As we can notice the first column result follows the style more closely. \ No newline at end of file +As we can notice the first column result follows the style more closely.