Improve torch compile performance (#1082)

Co-authored-by: regisss <[email protected]>
huggingface · Jun 16, 2024 · 595cc3e · 595cc3e
1 parent 8028af7
commit 595cc3e
Show file tree

Hide file tree

Showing 2 changed files with 4 additions and 1 deletion.
diff --git a/examples/stable-diffusion/README.md b/examples/stable-diffusion/README.md
@@ -202,6 +202,9 @@ python text_to_image_generation.py \
 > The first batch of images entails a performance penalty. All subsequent batches will be generated much faster.
 > You can enable this mode with `--use_hpu_graphs`.
 
+> Please note: there is a regression with "--guidance_scale 0.0" for the latest release.
+
+
 ### ControlNet
 
 ControlNet was introduced in [Adding Conditional Control to Text-to-Image Diffusion Models ](https://huggingface.co/papers/2302.05543) by Lvmin Zhang and Maneesh Agrawala.

diff --git a/examples/text-generation/utils.py b/examples/text-generation/utils.py
@@ -177,7 +177,7 @@ def patch_scoped_linear_all_reduce(model):
 
 
 def get_torch_compiled_model(model):
-    model.model = torch.compile(model.model, backend="hpu_backend")
+    model.model = torch.compile(model.model, backend="hpu_backend", options={"keep_input_mutations": True})
     return model