-
Notifications
You must be signed in to change notification settings - Fork 840
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix the reaccording to view suggestion
- Loading branch information
1 parent
e6bf758
commit 9509806
Showing
3 changed files
with
726 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
689 changes: 689 additions & 0 deletions
689
notebooks/277-amused-lightweight-text-to-image/277-amused-lightweight-text-to-image.ipynb
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Lightweight image generation with aMUSEd and OpenVINO™ | ||
|
||
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/277-amused-lightweight-text-to-image/277-amused-lightweight-text-to-image.ipynb) | ||
|
||
|
||
<img src="https://huggingface.co/amused/amused-256/resolve/main/assets/collage_small.png" /> | ||
|
||
[Amused](https://huggingface.co/docs/diffusers/api/pipelines/amused) is a lightweight text to image model based off | ||
of the [muse](https://arxiv.org/pdf/2301.00704.pdf) architecture. Amused is particularly useful in applications that | ||
require a lightweight and fast model such as generating many images quickly at once. | ||
|
||
Amused is a VQVAE token based transformer that can generate an image in fewer forward passes than many diffusion models. | ||
In contrast with muse, it uses the smaller text encoder CLIP-L/14 instead of t5-xxl. Due to its small parameter count | ||
and few forward pass generation process, amused can generate many images quickly. This benefit is seen particularly at | ||
larger batch size | ||
|
||
## Notebook contents | ||
The tutorial consists from following steps: | ||
|
||
- Prerequisites | ||
- Load and run the original pipeline | ||
- Convert the model to OpenVINO IR | ||
- Convert the Text Encoder | ||
- Convert the U-ViT transformer | ||
- Convert VQ-GAN decoder (VQVAE) | ||
- Compiling models and prepare pipeline | ||
- Interactive inference | ||
|
||
## Installation instructions | ||
This is a self-contained example that relies solely on its own code.</br> | ||
We recommend running the notebook in a virtual environment. You only need a Jupyter server to start. | ||
For details, please refer to [Installation Guide](../../README.md). |