-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug when using with controlnet #25
Comments
Thanks for reporting this! I ran into the same problem a week ago and hadn't had a chance to file an issue just yet. But I can also confirm that the node breaks when using more than one ControlNet in series. |
I ran into these bugs when working with instant ID on huge images. Even after fixing this using the method I have mentioned, the issues remained on instant ID side. So once we have a proper solution here, I will report the issue in instant ID page! |
Any idea to fix it ? |
I managed to find a quick fix to the multiple controlnet problem in the links I had shared above (https://github.com/aravindhv10/ComfyUI-TiledDiffusion/blob/fix_controlnet/tiled_diffusion.py#L308-L314). Donot know of a solution for instant ID yet. If you just want to use tile diffusion with multiple controlnets for now, you can use the fork and branch from the link. |
Duuuude Youre crazy! Thank you very much ! Can you text me on discord ? Dervlex is my name on Discord |
But i get errors on 3 Control Nets now xD 2 works. 3 not :D |
!!! Exception during processing!!! object of type 'NoneType' has no len() |
Same. Just tested with 4. Doesn't work, but if I disable 2 of them then it will work. Nothing more then 2 will work. |
Hi, sorry for the delayed reply. I made a few more changes in the same fork. It should now atleast work with 3 controlnets for SDXL (I tried with full canny, depth and TTPlanet tile). It was a minor bug in my code. |
I tried your new update with more then one controlnet and it wouldn't work. Disabling everything except one controlnet makes it work. |
I was able to use it with 3 controlnets, can you check you are using the correct branch in my fork? You have can use "git switch fix_controlnet". |
Hi, what was the video memory you have ? Can you try reducing the tile diffusion batch size? |
I managed to run this workflow: https://github.com/aravindhv10/ComfyUI-TiledDiffusion/blob/fix_controlnet/tile_diffusion_sr.json on G5.2x large aws machine (24 gb gpu memory, ubuntu 22.04 native os python with venv) with the packages: The xformers seem to help. |
Hi, I also noticed that this error is very difficult to reproduce and depends on the resolution and other matters of the input image. I found that bypassing instant id resolves the error for me. Probably it comes from combination of insight face (face detection) and other complex interactions from instant id. |
@aravindhv10 I am testing the update now. So far its working very great! I noticed some incompatibility with midas depth node, and zoe depth node, but it works with the regular depth node and leres node. |
@aravindhv10 Hi, you might be doing something similar to what I want, but I keep not getting the desired output. How to use Tiled-diffusion with controlnet depth and lineart and prompt to generate huge images? I want to take in a huge image, and then apply controlnet depth, and use tiled diffusion to generate tile by tile to finally generate a high resolution image, same resolution as the original input image. I want to achieve the results like these two examples below (using multidiffusion): |
Hey, should be related because now I have this error: The type of "control" should a tensor (or np array, I'm not sure), and when I print "control" before this error, it's a: I executed first a part of my workflow with 3 controlnet and tiled diffusion. And this error append after when I execute again my workflow but with only one controlnet. Somewhere a conversion is not done? I can't find. Edit: looks like someone had a similar issue: comfyanonymous/ComfyUI#4809 I can see in log it uses Tiled Diffusion |
Hi, sorry just saw the messages. @edtjulien Again, sorry we are not fully up to date with latest commit of tilediffusion. We use a specific commit which is known to work with atleast 4 controlnets (full controlnets, not control loras) and SDXL (including instant ids). Yet to check this with flux either, we are on it though. |
Can you share the workflow which leads to the error? |
Hello ! Thanks for the fix and the follow up. The fix of @shiimizu solved my issue. So, no more workflow with a problem to share. Thanks again for this!! |
Hi, when using tiled diffusion with more than one controlnet in series, I was running into errors. The workflow is here:
https://github.com/aravindhv10/ComfyUI-TiledDiffusion/blob/fix_controlnet/counter_example.json
The exact error is:
It seems that self.control_params (https://github.com/shiimizu/ComfyUI-TiledDiffusion/blob/main/tiled_diffusion.py#L292) in some circumstances is not initialized properly.
I found a hacky way to fix this:
https://github.com/aravindhv10/ComfyUI-TiledDiffusion/blob/fix_controlnet/tiled_diffusion.py#L308-L314
wanted to know if this could be merged or there was a better way?
Perhaps, these lines:
https://github.com/shiimizu/ComfyUI-TiledDiffusion/blob/main/tiled_diffusion.py#L303-L306
can be rewritten.
Thanks!
The text was updated successfully, but these errors were encountered: