LoRA merge fails for the CPU path with latest A1111

[vanilla-coder]

LoRA merging works fine for the last official A1111 release, but fails on v1.10.0-RC:

Traceback (most recent call last):
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
result = await self.call_function(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 772, in pluslora
theta_0 = newpluslora(theta_0,filenames,lweis,names, isxl,isv2, keychanger)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 868, in newpluslora
theta_0[wkey], theta_0[bkey]= plusweights(theta_0[wkey], module, bias = theta_0[bkey])
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 889, in plusweights
updown = module.calc_updown(weight.to(dtype=torch.float))
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions-builtin/Lora/network_lora.py", line 76, in calc_updown
updown = lyco_helpers.rebuild_conventional(up, down, output_shape, self.network.dyn_dim)
File "/Volumes/External One/A1111/stable-diffusion-webui/extensions-builtin/Lora/lyco_helpers.py", line 15, in rebuild_conventional
return (up @ down).reshape(shape)
RuntimeError: "addmm_impl_cpu" not implemented for 'Half'

[hako-mikan]

It might have been fixed in the current version. Give it a try.

[vanilla-coder]

Unfortunately the current version is even more broken than before, now normal merging doesn't work on the CPU path either. It fails on Macs because it requires having a torch version compiled with CUDA, even when the "use CUDA" box isn't checked (I'm using a Mac M2 so CUDA isn't possible to use).

Traceback (most recent call last):
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
result = await self.call_function(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/mergers.py", line 119, in smergegen
result,currentmodel,modelid,theta_0,metadata = smerge(
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/mergers.py", line 418, in smerge
theta_0[key] = theta_0[key].to("cuda")
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/cuda/init.py", line 289, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

[hako-mikan]

How about current version?

[vanilla-coder]

Now normal merging works again, thanks for that!

But the LoRA merging still fails with the same error as before:

Traceback (most recent call last):
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
result = await self.call_function(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 781, in pluslora
theta_0 = newpluslora(theta_0,filenames,lweis,names, calc_precision, isxl,isv2, keychanger)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 885, in newpluslora
theta_0[wkey], theta_0[bkey]= plusweights(theta_0[wkey], module, bias = theta_0[bkey])
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 906, in plusweights
updown = module.calc_updown(weight.to(dtype=torch.float))
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/A1111/network_lora.py", line 87, in calc_updown
updown = lyco_helpers.rebuild_conventional(up, down, output_shape, self.network.dyn_dim)
File "/Volumes/External One/A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/A1111/lyco_helpers.py", line 15, in rebuild_conventional
return (up @ down).reshape(shape)
RuntimeError: "addmm_impl_cpu" not implemented for 'Half'

[vanilla-coder]

PS.
I'd like to take the opportunity to thank you for this essential tool! While I'm stuck using a very old version that still works, I use it all the time, and this SD model of mine wouldn't exist without it:

https://civitai.com/models/835597?modelVersionId=1264727

[hako-mikan]

May be fixed.

[vanilla-coder]

Unfortunately not. But there's a new error! Progress! ;-)

Traceback (most recent call last):
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1431, in process_api
result = await self.call_function(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Volumes/External One/_A1111/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 794, in pluslora
theta_0 = newpluslora(theta_0,filenames,lweis,names, calc_precision, isxl,isv2, keychanger)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 898, in newpluslora
theta_0[wkey], theta_0[bkey]= plusweights(theta_0[wkey], module, bias = theta_0[bkey])
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/mergers/pluslora.py", line 919, in plusweights
updown = module.calc_updown(weight.to(dtype=torch.float))
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/A1111/network_lora.py", line 87, in calc_updown
updown = lyco_helpers.rebuild_conventional(up, down, output_shape, self.network.dyn_dim)
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/A1111/lyco_helpers.py", line 10, in rebuild_conventional
up = cpufloat(up.reshape(up.size(0), -1))
File "/Volumes/External One/_A1111/stable-diffusion-webui/extensions/sd-webui-supermerger/scripts/A1111/lyco_helpers.py", line 71, in cpufloat
if not module: return module #None対策
RuntimeError: Boolean value of Tensor with more than one value is ambiguous

[hako-mikan]

This time, it should be fixed.

[vanilla-coder]

This time it is indeed fixed!

Thanks a lot!