forked from eliahuhorwitz/Academic-project-page-template
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
365 lines (319 loc) · 20.7 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
<!DOCTYPE html>
<html>
<head>
<meta name="google-site-verification" content="UHgzLKwihDXNbKGXneVXpbKAicWpxZvA4islDMsP9HU" />
<meta charset="utf-8">
<!-- Meta tags for social media banners -->
<meta name="description" content="Exploring non-linear correction in diffusion models for enhanced control and precision in machine learning applications.">
<meta property="og:title" content="Characteristic Guidance: improved guidance method for stable diffusion"/>
<meta property="og:description" content="Characteristic Guidance, a new guidance method to improve diffusion model performance at large guidance scales."/>
<meta property="og:url" content="https://scraed.github.io/CharacteristicGuidance/"/>
<meta property="og:image" content="static/image/1girl_handstand_sportswear_gym.jpg" />
<meta property="og:image:width" content="1200"/>
<meta property="og:image:height" content="630"/>
<meta name="twitter:title" content="Characteristic Guidance: improved guidance method for stable diffusion">
<meta name="twitter:description" content="Discover how Characteristic Guidance elevates the performance of diffusion models in generating images at high CFG scale.">
<meta name="twitter:image" content="static/images/1girl_handstand_sportswear_gym.jpg">
<meta name="twitter:card" content="summary_large_image">
<!-- Keywords for indexing -->
<meta name="keywords" content="Guidance, Stable Diffusion, Web UI, Diffusion Models, CFG scale, Machine Learning, Non-Linear Correction, Characteristic Guidance, Research, Artificial Intelligence">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Characteristic Guidance for Diffusion Model: large CFG scale correction</title>
<link rel="icon" type="image/x-icon" href="static/images/witchbearlogo.ico">
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="static/css/bulma.min.css">
<link rel="stylesheet" href="static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="static/css/bulma-slider.min.css">
<link rel="stylesheet" href="static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="static/css/index.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script src="https://documentcloud.adobe.com/view-sdk/main.js"></script>
<script defer src="static/js/fontawesome.all.min.js"></script>
<script src="static/js/bulma-carousel.min.js"></script>
<script src="static/js/bulma-slider.min.js"></script>
<script src="static/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-3 publication-title">Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale</h1>
<div class="is-size-5 publication-authors">
<!-- Paper authors -->
<span class="author-block">
<a href="https://www.math.hkust.edu.hk/people/students/czhengac/" target="_blank">Candi Zheng</a><sup>*^</sup>,</span>
<span class="author-block">
<a href="https://hk.linkedin.com/in/yuan-lan-21095016b" target="_blank">Yuan Lan</a><sup>*</sup>,</span>
</div>
<div class="is-size-6 publication-authors">
<span class="author-block">The Hong Kong University of Science and Technology</span>
<span class="eql-cntrb"><small><br><sup>*</sup>Indicates Equal Contribution <sup>^</sup>Corresponding to [email protected]</small></span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<!-- Arxiv PDF link -->
<span class="link-block">
<a href="https://arxiv.org/pdf/2312.07586.pdf" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fas fa-file-pdf"></i>
</span>
<span>Paper (ICML 2024)</span>
</a>
</span>
<!-- Github link -->
<span class="link-block">
<a href="https://github.com/scraed/CharacteristicGuidanceWebUI" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Stable Diffusion WebUI Extension</span>
</a>
</span>
<!-- ArXiv abstract Link -->
<span class="link-block">
<a href="https://arxiv.org/abs/2312.07586" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="ai ai-arxiv"></i>
</span>
<span>arXiv</span>
</a>
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Paper abstract -->
<section class="section hero">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-left">
<div class="column is-four-fifths">
<h2 class="title is-3">WebUI Extension for Stable Diffusion</h2>
<div class="content is-size-6 has-text-justified">
<p>
We are excited to share our publicly available extension, the <a href="https://github.com/scraed/CharacteristicGuidanceWebUI" target="_blank", class="author-block">Characteristic Guidance Web UI</a>, which provides large CFG (Cassifier-Free Guidance) scale correction for the Stable Diffusion web UI (AUTOMATIC1111). This tool is an application of the methods and theories presented in our paper, offering improved control in sample generation and compatibility with existing sampling methods. For researchers and practitioners interested in exploring our new sampling approach, the extension and its installation instructions can be found on its <a href="https://github.com/scraed/CharacteristicGuidanceWebUI" target="_blank", class="author-block">Github</a> page.
</p>
</div>
</div>
</div>
</div>
</section>
<!-- End paper abstract -->
<!-- Image carousel -->
<section class="hero is-small">
<div class="hero-body">
<div class="container">
<div id="results-carousel" class="carousel results-carousel">
<div class="item">
<!-- Your image here -->
<img src="static/images/1girl_handstand_sportswear_gym.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>1girl, handstand, sportswear, gym</b> <br>
Negative prompt: low quality, worst quality, <br>
Batch size: 1 <br>
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 10, Seed: 0, Size: 1024x1024, Model hash: 1449e5b0b9, Model: animagineXLV3_v30, <br>
<b>Dynamic thresholding</b> <br>
Dynamic thresholding enabled: True, Mimic scale: 7, Separate Feature Channels: True, Scaling Startpoint: MEAN, Variability Measure: AD, Interpolate Phi: 1, Threshold percentile: 100, Mimic mode: Half Cosine Down, Mimic scale minimum: 0, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 50, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.7.0
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/1girl, running, mountain, grass.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>1girl, running, mountain, grass, <br></b>
Negative prompt: low quality, worst quality, <br>
Batch size: 4 <br>
Steps: 30, Sampler: UniPC, CFG scale: 30, Seed: 0, Size: 1024x1024, Model hash: 1449e5b0b9, Model: animagineXLV3_v30, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 5, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 50, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.7.0
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/1girl handstand sports close_up.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>1girl, handstand, sports, close_up</b> <br>
Batch size: 1 <br>
Steps: 40, Sampler: DPM++ 2M Karras, CFG scale: 10, Seed: 2635612998, Size: 512x512, Model hash: 1a189f0be6, Model: v1-5-pruned, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 0.25, Regularization Range Over Time: 0.25, Max Num. Characteristic Iteration: 50, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.6.1
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/newspaper news english.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>newspaper, news, english</b> <br>
Batch size: 1 <br>
Steps: 40, Sampler: DPM++ 2M Karras, CFG scale: 10, Seed: 3470006633, Size: 512x512, Model hash: 1a189f0be6, Model: v1-5-pruned, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 50, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.6.1
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/StrawberryPancake.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>strawberry pancake</b> <br>
Batch size: 4 <br>
Steps: 20, Sampler: UniPC, CFG scale: 30, Seed: 669228313, Size: 512x512, Model hash: c0d1994c73, Model: realisticVisionV20_v20, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1.5, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 30, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.6.1
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/lake night_sky meteor makoto_shinkai.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>lake, night_sky, meteor, makoto_shinkai</b> <br>
Batch size: 1 <br>
Steps: 40, Sampler: DPM++ 2M Karras, CFG scale: 30, Seed: 3255477725, Size: 1024x256, Model hash: 1a189f0be6, Model: v1-5-pruned, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 30, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.6.1
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/A Party with Four Dogs.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>a birthday party with four dogs</b> <br>
Batch size: 4 <br>
Steps: 20, Sampler: UniPC, CFG scale: 10, Seed: 2693088926, Size: 512x512, Model hash: c0d1994c73, Model: realisticVisionV20_v20, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 30, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.6.1
</h2>
</div>
<div class="item">
<!-- Your image here -->
<img src="static/images/1girl earring.jpg" alt="MY ALT TEXT"/>
<h2 class="subtitle is-size-7 has-text-left">
<b>1girl, earring</b> <br>
Batch size: 4 <br>
Steps: 40, Sampler: DPM++ 2M Karras, CFG scale: 30, Seed: 209458356, Size: 512x512, Model hash: 00bafa0e71, Model: anyloraCheckpoint_novaeFp16, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 30, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
Version: v1.6.1
</h2>
</div>
</div>
</div>
</div>
</section>
<!-- End image carousel -->
<!-- Paper abstract -->
<section class="section hero">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-left">
<div class="column is-four-fifths">
<h2 class="title is-4">Preliminary Support for ControlNet</h2>
<div class="content is-size-6 has-text-justified">
<div class="item">
<!-- Your image here -->
<img src="static/images/1girl kimono.jpg" alt="MY ALT TEXT"/>
<p class="content is-size-7 has-text-left">
<b>1girl, kimono</b> <br>
Steps: 40, Sampler: DPM++ 2M Karras, CFG scale: 10, Seed: 0, Size: 512x512, Model hash: 1a189f0be6, Model: v1-5-pruned, VAE hash: 735e4c3a44, VAE: vae-ft-mse-840000-ema-pruned.safetensors, <br>
<b>Characteristic Guidance:</b> <br>
Regularization Strength: 1, Regularization Range Over Time: 1, Max Num. Characteristic Iteration: 30, Num. Basis for Correction: 1, Reuse Correction of Previous Iteration: 0, Log 10 Tolerance for Iteration Convergence: -4, Iteration Step Size: 1, Regularization Annealing Speed: 0.4, Regularization Annealing Strength: 0.5, AA Iteration Memory Size: 2, <br>
<b>ControlNet 0:</b> "Module: openpose, Model: control_v11p_sd15_openpose [cab727d4], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: False, Save Detected Map: True", <br>
Version: v1.6.1
</p>
<p class="content is-size-6 has-text-justified">
We are excited to announce that the characteristic guidance WebUI extension now preliminarily supports ControlNet.
This new integration offering users the ability to leverage ControlNet's robust control capabilities within the familiar webUI environment.
Please note that as this is an initial integration, some features may still be in development, and users might encounter limitations or issues. We are committed to continuously updating and improving this integration and look forward to the community's input and support in this journey.
Stay tuned for more updates, and welcome to contribute to our extension development!
</p>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Paper abstract -->
<section class="section hero">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-left">
<div class="column is-four-fifths">
<h2 class="title is-3">How characteristic guidance works</h2>
<div class="content is-size-6 has-text-justified">
<div class="item">
<!-- Your image here -->
<img src="static/images/CHtheory.PNG" alt="MY ALT TEXT"/>
<p class="content is-size-6 has-text-left">
In comparing classifier-free guidance with characteristic guidance, we examine their behavior during the sampling process at time t. Classifier-free guidance applies a linear combination of positive and negative prompts at a given point along the sampling trajectory. This approach, however, does not adhere to the non-linear dynamics of the Fokker-Planck equation of scores. On the other hand, characteristic guidance leverages the characteristic line of the Fokker-Planck equation of scores, effectively bring the linear combination back to the initial time (t=0). At this point, the Fokker-Planck equation of scores permits linear combinations, thus ensuring that characteristic guidance adheres to the equation’s constraints.
</p>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Paper abstract -->
<section class="section hero is-light">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-left">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content is-size-6 has-text-justified">
<p>
Popular guidance for denoising diffusion probabilistic model (DDPM) linearly combines distinct conditional models together to provide enhanced control over samples. However, this approach overlooks nonlinear effects that become significant when guidance scale is large. To address this issue, we propose characteristic guidance, a novel method that provides non-linear correction for classifier-free guided DDPMs. Such correction forces the guided DDPMs to respect the Fokker-Planck equation of their underlying diffusion process, in a way that is first-principle, training-free, derivative-free, and compatible with existing sampling methods. Experiments show that characteristic guidance is robust to various applications, offers enhanced control over sample generation, suppresses color and exposure issues even for latent space sampling, and can handle physics problems such as the phase transitions of magnets.
</p>
</div>
</div>
</div>
</div>
</section>
<!-- End paper abstract -->
<!--BibTex citation -->
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code>@misc{zheng2023characteristic,
title={Characteristic Guidance: Non-linear Correction for DDPM at Large Guidance Scale},
author={Candi Zheng and Yuan Lan},
year={2023},
eprint={2312.07586},
archivePrefix={arXiv},
primaryClass={cs.CV}
}</code></pre>
</div>
</section>
<!--End BibTex citation -->
<footer class="footer">
<div class="container">
<div class="columns is-centered">
<div class="column is-8">
<div class="content">
<p>
This page was built using the <a href="https://github.com/eliahuhorwitz/Academic-project-page-template" target="_blank">Academic Project Page Template</a> which was adopted from the <a href="https://nerfies.github.io" target="_blank">Nerfies</a> project page.
</p>
</div>
</div>
</div>
</div>
</footer>
<!-- Statcounter tracking code -->
<!-- You can add a tracker to track page visits by creating an account at statcounter.com -->
<!-- End of Statcounter Code -->
</body>
</html>