ControlNet issue: TypeError: argument of type 'ControlLora' is not iterable

[Tromazing]

Expected Behavior

Used ControlNet with a PonyXL model to create an image.

Actual Behavior

After one or two successful generations, and no changes to any settings, I get an error message from the KSampler.
File "D:\StableDiffusion\Packages\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 357, in apply_control
if control is not None and name in control and len(control[name]) > 0:
TypeError: argument of type 'ControlLora' is not iterable

I've tried using different KSamplers and different Apply ControlNet nodes, but same result. The error first appeared using the core nodes. I've also tried with different ControlNet models, and still the same.

Restarting ComfyUI solves the issue for one or two generations, then it returns. Bypassing ControlNet removes any issue.

Steps to Reproduce

Json attached.
ControlNetPonyError.json

Debug Logs

got prompt
  0%|          | 0/30 [00:00<?, ?it/s]
!!! Exception during processing !!! argument of type 'ControlLora' is not iterable
Traceback (most recent call last):
  File "D:\StableDiffusion\Packages\ComfyUI\execution.py", line 317, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "D:\StableDiffusion\Packages\ComfyUI\execution.py", line 192, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
  File "D:\StableDiffusion\Packages\ComfyUI\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "D:\StableDiffusion\Packages\ComfyUI\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
  File "D:\StableDiffusion\Packages\ComfyUI\nodes.py", line 1429, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
  File "D:\StableDiffusion\Packages\ComfyUI\nodes.py", line 1396, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample
    return original_sample(*args, **kwargs)  # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 420, in motion_sample
    return orig_comfy_sample(model, noise, *args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\sampling.py", line 116, in acn_sample
    return orig_comfy_sample(model, *args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 117, in uncond_multiplier_check_cn_sample
    return orig_comfy_sample(model, *args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\sample.py", line 43, in sample
    samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 104, in KSampler_sample
    return orig_fn(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-TiledDiffusion\utils.py", line 48, in KSampler_sample
    return orig_fn(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 829, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 122, in sample
    return orig_fn(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 729, in sample
    return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 716, in sample
    output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 695, in inner_sample
    samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 87, in KSAMPLER_sample
    return orig_fn(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-TiledDiffusion\utils.py", line 31, in KSAMPLER_sample
    return orig_fn(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 600, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
  File "D:\StableDiffusion\Packages\ComfyUI\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\k_diffusion\sampling.py", line 612, in sample_dpmpp_sde
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 299, in __call__
    out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 682, in __call__
    return self.predict_noise(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 685, in predict_noise
    return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 162, in sampling_function
    out = orig_fn(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 279, in sampling_function
    out = calc_cond_batch(model, conds, x, timestep, model_options)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\samplers.py", line 228, in calc_cond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
  File "D:\StableDiffusion\Packages\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 69, in apply_model_uncond_cleanup_wrapper
    return orig_apply_model(self, *args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\model_base.py", line 142, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "D:\StableDiffusion\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 858, in forward
    h = apply_control(h, control, 'input')
  File "D:\StableDiffusion\Packages\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 357, in apply_control
    if control is not None and name in control and len(control[name]) > 0:
TypeError: argument of type 'ControlLora' is not iterable

Prompt executed in 0.69 seconds

Other

No response

[edtjulien]

Hey, it's maybe related to the issue in Tiled Diffusion: shiimizu/ComfyUI-TiledDiffusion#25 I got the same error today.

[shiimizu]

Thank you for the workflow. It should be fixed now.