High-res fix optimization - estimated time reduction up to 40%

[alfonslm]

From my understanding of how the high-res fix works it's basically an option to run an up-scaler and img2img after you have fully generated the first text2img.

I did play around with comfy UI (I never got the weight correct and had some issues with lack of some nodes) you can basically generate the first text2img about 2/3 and then add 1/3 of noise upscale and then start the process as if it already had run about 1/3 of the steps.

With my shit unoptimized doodle estimation you could probably save around 22% time by running the generation in two presteps (50% and about 75% in the doodle) befor the final size

I will assume that you can calculate a second degree equation for the estimated time consumption for the actual cost of running an optimisation step and a third degree equation (the area to border will have a bigger effect on the diffusion model than scaller) for the time consumption for the sampling and either by bruit forcing (I don't expect that many to run more than 100 samples so it wouldn't be that explosive) or some smarter calculation you should be able to get a pretty optimal generation sequence.

From experience you will probably want to manually set the start size since it will determine the smal to large feature ratio and the artistic value is probably more important than the small time save you have in that stage.

With the assumption that I have not done any major logical mistakes I did the lazy thing and asked MathGPT to solve the sum of the time estimation, a, b, c would be the diffuser system constants and d, e the scaller constants:

This is what MathGPT summarised and the optimisation gains do align with the napkin doodle I made so it should be pretty accurate:
Summary of Optimization Problem

1. Original Sum S(n)
   
   where a, b, c, d, e are positive constants
2. Base Case (n = 1)
   
3. Finding Optimal n (n_opt)
   Take derivative of S(n) with respect to n
   Set derivative equal to zero
   Due to complexity, use numerical approach:
   Algorithm:

n = 1
prev_sum = S(1)
while true:
    current_sum = S(n+1)
    if current_sum > prev_sum:
        n_opt = n
        break
    n = n + 1
    prev_sum = current_sum

4. Size Comparison
   Ratio between optimal and base case:
   
   Expected improvement range:
   
5. Final Results
   Typical reduction: 20-40% improvement from n = 1
   n_opt typically falls between 2 and 5
   Actual improvement depends on specific values of constants a, b, c, d, e
6. Why It Works
   Linear terms (na, nd): grow linearly with n
   Mixed terms (b, c terms): grow approximately linearly
   Exponential term:
   
   balances against linear growth

• Optimal n is where marginal benefit of increasing n equals marginal cost

[Juqowel]

What is the conceptual difference from Kohya HRFix Integrated?

[alfonslm]

I have actually not seen that one but I guess that I did the math to enable it to self optimise with just a few more steps so it should be able to be enable by default with no issues