[1.1.442] New preprocessor normal dsine #2703

sdbds · 2024-03-26T12:23:47Z

sdbds
Mar 26, 2024
Collaborator

Dsine

https://github.com/baegwangbin/DSINE

How to use

Use as normal bae

Model comparison

normal dsine has more details than others

input image

normal bae

normal dsine

normal midas

FurkanGozukara · 2024-03-26T12:44:34Z

FurkanGozukara
Mar 26, 2024

thanks for update

0 replies

DavideAlidosi · 2024-03-27T07:40:34Z

DavideAlidosi
Mar 27, 2024

Some thing goes wrong for me:

*** Error running process: L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "L:\WebUI\webui\modules\scripts.py", line 784, in process
        script.process(p, *script_args)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1279, in process
        self.controlnet_hack(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1264, in controlnet_hack
        self.controlnet_main_entry(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1029, in controlnet_main_entry
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1029, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 986, in preprocess_input_image
        detected_map, is_image = self.preprocessor[unit.module](
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
        return cached_func(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
        return func(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\processor.py", line 720, in normal_dsine
        result = model_normal_dsine(img, new_fov=float(thr_a), iterations=int(thr_b), resulotion=res)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\annotator\normaldsine\__init__.py", line 41, in __call__
        self.load_model()
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\annotator\normaldsine\__init__.py", line 30, in load_model
        model = utils.load_checkpoint(modelpath, model)
      File "L:\WebUI\system\python\lib\site-packages\dsine\utils\utils.py", line 11, in load_checkpoint
        ckpt = torch.load(fpath, map_location='cpu')['model']
      File "L:\WebUI\webui\modules\safe.py", line 108, in load
        return load_with_extra(filename, *args, extra_handler=global_extra_handler, **kwargs)
      File "L:\WebUI\webui\modules\safe.py", line 156, in load_with_extra
        return unsafe_torch_load(filename, *args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\serialization.py", line 1028, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "L:\WebUI\system\python\lib\site-packages\torch\serialization.py", line 1246, in _legacy_load
        magic_number = pickle_module.load(f, **pickle_load_args)
    _pickle.UnpicklingError: invalid load key, '<'.

I'm sorry for that.

2 replies

sdbds Mar 27, 2024
Collaborator Author

Sorry, I wrote the wrong download link.
Please update it and re-download the model.

DavideAlidosi Mar 27, 2024

Update, remove old normaldsine folder from Annotator directory and this is the new error:

*** Error running process: L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "L:\WebUI\webui\modules\scripts.py", line 784, in process
        script.process(p, *script_args)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1279, in process
        self.controlnet_hack(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1264, in controlnet_hack
        self.controlnet_main_entry(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1029, in controlnet_main_entry
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1029, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 986, in preprocess_input_image
        detected_map, is_image = self.preprocessor[unit.module](
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
        return cached_func(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
        return func(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\processor.py", line 718, in normal_dsine
        from annotator.normaldsine import NormalDsineDetector
    ModuleNotFoundError: No module named 'annotator.normaldsine'

Looks like it's unable to download the preprocessor again

sdbds · 2024-03-27T09:14:34Z

sdbds
Mar 27, 2024
Collaborator Author

Update, remove old normaldsine folder from Annotator directory and this is the new error:

*** Error running process: L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "L:\WebUI\webui\modules\scripts.py", line 784, in process
        script.process(p, *script_args)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1279, in process
        self.controlnet_hack(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1264, in controlnet_hack
        self.controlnet_main_entry(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1029, in controlnet_main_entry
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1029, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 986, in preprocess_input_image
        detected_map, is_image = self.preprocessor[unit.module](
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
        return cached_func(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
        return func(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\processor.py", line 718, in normal_dsine
        from annotator.normaldsine import NormalDsineDetector
    ModuleNotFoundError: No module named 'annotator.normaldsine'

Looks like it's unable to download the preprocessor again

you should remove annotator/downloading/normaldsine
not annotator/normaldsine

1 reply

DavideAlidosi Mar 27, 2024

Thanks, now works flowesly! Bests.

IDXoX · 2024-03-28T09:37:49Z

IDXoX
Mar 28, 2024

When running the preprocessor there is such an error（I re-downloaded the dsin.pt file and still had this problem）：

Traceback (most recent call last):
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\scripts\controlnet_ui\controlnet_ui_group.py", line 1015, in run_annotator
    result, is_image = preprocessor(
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
    return cached_func(*args, **kwargs)
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
    return func(*args, **kwargs)
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
    return preprocessor_modules[preprocessor_name](*args, **kwargs)
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\scripts\processor.py", line 720, in normal_dsine
    result = model_normal_dsine(img, new_fov=float(thr_a), iterations=int(thr_b), resulotion=res)
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\annotator\normaldsine\__init__.py", line 41, in __call__
    self.load_model()
  File "F:\sd-webui-aki-v4.6.1\extensions\sd-webui-controlnet\annotator\normaldsine\__init__.py", line 28, in load_model
    model = DSINE()
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\dsine\models\dsine.py", line 56, in __init__
    self.encoder = Encoder(B=5, pretrained=True)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\dsine\models\submodules.py", line 24, in __init__
    basemodel = geffnet.create_model('tf_efficientnet_b%s_ap' % B, pretrained=pretrained)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\geffnet\model_factory.py", line 20, in create_model
    model = create_fn(**model_kwargs)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\geffnet\gen_efficientnet.py", line 1120, in tf_efficientnet_b5_ap
    model = _gen_efficientnet(
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\geffnet\gen_efficientnet.py", line 544, in _gen_efficientnet
    model = _create_model(model_kwargs, variant, pretrained)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\geffnet\gen_efficientnet.py", line 290, in _create_model
    load_pretrained(model, model_urls[variant])
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\geffnet\helpers.py", line 39, in load_pretrained
    state_dict = load_state_dict_from_url(url, progress=False, map_location='cpu')
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\torch\hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
  File "F:\sd-webui-aki-v4.6.1\modules\safe.py", line 108, in load
    return load_with_extra(filename, *args, extra_handler=global_extra_handler, **kwargs)
  File "F:\sd-webui-aki-v4.6.1\modules\safe.py", line 156, in load_with_extra
    return unsafe_torch_load(filename, *args, **kwargs)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\torch\serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "F:\sd-webui-aki-v4.6.1\python\lib\site-packages\torch\serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: unexpected EOF, expected 259011 more bytes. The file might be corrupted.

1 reply

IDXoX Mar 28, 2024

Have already solved

vixenius · 2024-04-27T09:41:43Z

vixenius
Apr 27, 2024

I'm trying to load normal_dsine.pt but it wont show in controlnet. In A1111 it is listed under normal preprocessor but in webforge it just wont load, i have the normal_dsine.pt in these locations: "D:\AINOVO\WebForge\webui\models\ControlNet" and "D:\AINOVO\WebForge\webui\models\ControlNetPreprocessor". What am i doing wrong?

1 reply

sdbds Apr 27, 2024
Collaborator Author

Forge not update for this because we cant merge code in main branch now.

kagevazquez · 2024-04-28T13:32:01Z

kagevazquez
Apr 28, 2024

https://jugghm.github.io/Metric3Dv2/ > :D

0 replies

sdbds · 2024-04-28T15:45:59Z

sdbds
Apr 28, 2024
Collaborator Author

https://jugghm.github.io/Metric3Dv2/ > :D

I've seen it before and it looks good, I'll test it out

0 replies

pflky · 2024-09-04T13:57:21Z

pflky
Sep 4, 2024

I wish someone would cook this up to work with Forge's controlnet, because DSINE is honestly the best controlnet preprocessor, even depth anything pales in comparison.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1.1.442] New preprocessor normal dsine #2703

{{title}}

Replies: 8 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

[1.1.442] New preprocessor normal dsine #2703

sdbds Mar 26, 2024 Collaborator

Dsine

How to use

Model comparison

Replies: 8 comments · 5 replies

sdbds Mar 27, 2024 Collaborator Author

sdbds Mar 27, 2024 Collaborator Author

sdbds Apr 27, 2024 Collaborator Author

sdbds Apr 28, 2024 Collaborator Author

sdbds
Mar 26, 2024
Collaborator

Replies: 8 comments 5 replies

sdbds Mar 27, 2024
Collaborator Author

sdbds
Mar 27, 2024
Collaborator Author

sdbds Apr 27, 2024
Collaborator Author

sdbds
Apr 28, 2024
Collaborator Author