Flux support

[alex-mitov]

Hello,

I've been using the Tiled Diffusion node in ComfyUI with really good results. I was wondering if there are any plans to support the new Flux model by Black Forest Labs.

Thanks in advance!

[Adreitz]

Not the dev, but have you confirmed that it does not work? I haven't tried it yet with Flux dev, but I had tried it with SD3. It seemed to work, but failed when I introduced the SD3 tile controlnet (it applied the whole initial image to each tile, rather than just the corresponding portion of the initial image). Since there is no Flux tile controlnet yet to my knowledge, and Flux seems to hallucinate very little when naively tiling anyway, this might not be a big problem.

[alex-mitov]

I meant to say that the problem happens with the tile controlnet. Sorry for not being clear. Just like you described, it applies the initial image to each tile. :(

There already is a tile control net for Flux as part of the Union controlnet model by Instant X.

[Adreitz]

Thanks for the quick confirmation! While getting the node working correctly with tile is important in the long run, I'd recommend trying it without the controlnet with Flux. Up to now I've just been using Ultimate SD Upscale node with Flux with barely any hallucinations or tile discontinuities (mostly in very smooth backgrounds where one tile might not have any detail to speak of, so Flux might introduce speckles or posterization in the gradations in the second pass), so hopefully it will work well with this node, too.

Example with no controlnets:

[alex-mitov]

Thanks for the quick confirmation! While getting the node working correctly with tile is important in the long run, I'd recommend trying it without the controlnet with Flux. Up to now I've just been using Ultimate SD Upscale node with Flux with barely any hallucinations or tile discontinuities (mostly in very smooth backgrounds where one tile might not have any detail to speak of, so Flux might introduce speckles or posterization in the gradations in the second pass), so hopefully it will work well with this node, too.

Example with no controlnets:

Looks good! Have you been able to creatively resample or/and upscale with Flux? I'm getting much more creative results when resampling/upscaling images with SD 1.5 models and the SD 1.5 tile controlnet.

[Adreitz]

I haven't tried. My personal opinion is that the creative aspect should happen at the initial low-res generation, where the quality will be the best, and that upscaling should simply be a matter of increasing the image detail with as few compositional changes as possible. Flux is working pretty well for me in that regard, but it means I don't really have experience with other ways of working.

[Vigilence]

This node doesn't support the GGUF version but does the fp8 version. I look forward to the gguf version being supported in the future.

[shiimizu]

Flux fp & gguf should be okay now.

[alex-mitov]

Hi, @shiimizu . I was very excited to try the new update to Tiled Diffusion, but it still doesn't work for me. I'm still getting the same image in each tile. Am I doing something wrong? Please see the screenshot below..

I tried both Flux FP16 and Flux FP8, and I also tried Multi Diffusion in both Mixture of Diffusers and MultiDiffusion modes. My ComfyUI is updated to the latest version - 2639 [b643ea] (2024-09-01).

[Vigilence]

Does anyone have a working workflow for flux? I keep getting:

The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 0

[alex-mitov]

Does anyone have a working workflow for flux? I keep getting:

The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 0

Make sure the widths and heights of your latents and images are always divisible by 16. Flux doesn't like dimensions that are not divisible by 16 and will often give errors if you don't follow this rule.

[shiimizu]

@alex-mitov Instantx/union flux controlnets aren't working at the moment. I'll try to fix it. The other depth & canny ones work fine though.

[Vigilence]

Does anyone have a working workflow for flux? I keep getting:
The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 0

Make sure the widths and heights of your latents and images are always divisible by 16. Flux doesn't like dimensions that are not divisible by 16 and will often give errors if you don't follow this rule.

I just checked and they are divisible by 16.

[shiimizu]

FLUX Union ControlNets should work now.

@Vigilence Is that related to TiledDiffusion? If so, open a new issue with a workflow and full error stacktrace.

[alex-mitov]

I can confirm that Tiled Diffusion now works as expected with Flux!

[hakbodi]

I can confirm that Tiled Diffusion now works as expected with Flux!

Can you share a workflow for that please?

[alex-mitov]

I can confirm that Tiled Diffusion now works as expected with Flux!

Can you share a workflow for that please?

Just connect the Tiled Diffusion node to the "model" input of your Ksampler. Nothing complicated. :)

[uzitotte]

I also need a workflow for this..