no such file or directory

[23Rj20]

What happened?
I am trying to generate video after loading a finetuned/ trained lora model, although the model gets loaded, it doesnt generate video it says the above error, first the error was the file not found, where the weights of the model are located, if i give a direct path in the code then it says 'NoneType' object has no attribute 'items' Error.

Steps to reproduce the problem
Go to webui and open text2video generation
load the trained lora text to video model in the option given at the bottom
give a prompt and generate a video
after you generate this error should come in the terminal
Screenshot 2024-02-14 144855

file where the webui lora model located along with the weights
111

21

What should have happened?
Normally it should work witout giving any errors

WebUI and Deforum extension Commit IDs
webui commit id -
txt2vid commit id -

Torch version
'2.2.0'

What GPU were you using for launching?
NVIDIA RTX A4000 16GB

On which platform are you launching the webui backend with the extension?
Local PC setup (Windows)

Settings
version: [v1.7.0] •  python: 3.10.6  •  torch: 2.0.1+cu118  •  xformers: N/A  •  gradio: 3.41.2  •  checkpoint: [6ce0161689]

Console logs
C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui>webui-user
venv "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 | packaged by conda-forge | (main, Oct 24 2022, 16:02:16) [MSC v.1916 64 bit (AMD64)]
Version: v1.7.0
Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e
Launching Web UI with arguments:
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
Style database not found: C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\styles.csv
Loading weights [6ce0161689] from C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\extensions\sd-webui-text2video\scripts\text2vid.py:48: GradioDeprecationWarning: The style method is deprecated. Please set these arguments in the constructor instead.
with gr.Row(elem_id='t2v-core').style(equal_height=False, variant='compact'):
Running on local URL: http://127.0.0.1:7860/
Creating model from config: C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\configs\v1-inference.yaml

To create a public link, set share=True in launch().
Startup time: 24.4s (prepare environment: 5.4s, import torch: 6.7s, import gradio: 2.8s, setup paths: 2.6s, initialize shared: 0.6s, other imports: 1.9s, setup codeformer: 0.5s, load scripts: 1.9s, create ui: 0.7s, gradio launch: 1.2s).
Applying attention optimization: Doggettx... done.
Model loaded in 18.4s (load weights from disk: 1.1s, load config: 0.2s, create model: 0.5s, apply weights to model: 10.2s, apply half(): 2.9s, calculate empty prompt: 3.3s).
text2video — The model selected is: (ModelScope-like)
text2video extension for auto1111 webui
Git commit: 989f5cfe
Starting text2video
Pipeline setup
config namespace(framework='pytorch', task='text-to-video-synthesis', model={'type': 'latent-text-to-video-synthesis', 'model_args': {'ckpt_clip': 'open_clip_pytorch_model.bin', 'ckpt_unet': 'text2video_pytorch_model.pth', 'ckpt_autoencoder': 'VQGAN_autoencoder.pth', 'max_frames': 16, 'tiny_gpu': 1}, 'model_cfg': {'unet_in_dim': 4, 'unet_dim': 320, 'unet_y_dim': 768, 'unet_context_dim': 1024, 'unet_out_dim': 4, 'unet_dim_mult': [1, 2, 4, 4], 'unet_num_heads': 8, 'unet_head_dim': 64, 'unet_res_blocks': 2, 'unet_attn_scales': [1, 0.5, 0.25], 'unet_dropout': 0.1, 'temporal_attention': 'True', 'num_timesteps': 1000, 'mean_type': 'eps', 'var_type': 'fixed_small', 'loss_type': 'mse'}}, pipeline={'type': 'latent-text-to-video-synthesis'})
*** Error verifying pickled file from C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\models\Lora\lora\full_weights
*** The file may be malicious, so the program is not going to read it.
*** You can skip this check with --disable-safe-unpickle commandline argument.

Traceback (most recent call last):
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\modules\safe.py", line 137, in load_with_extra
check_pt(filename, extra_handler)
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\modules\safe.py", line 83, in check_pt
with zipfile.ZipFile(filename) as z:
File "C:\Users\INP_Rohit.conda\envs\ttv\lib\zipfile.py", line 1249, in init
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\models\Lora\lora\full_weights\'
Traceback (most recent call last):
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui/extensions/sd-webui-text2video/scripts\t2v_helpers\render.py", line 30, in run
vids_pack = process_modelscope(args_dict, args)
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui/extensions/sd-webui-text2video/scripts\modelscope\process_modelscope.py", line 71, in process_modelscope
stable_lora_processor.process(pipe, *stable_lora_args)
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui/extensions/sd-webui-text2video/scripts\stable_lora\scripts\lora_webui.py", line 203, in process
self.process_lora(*args, undo_merge=False)
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui\venv\lib\site-packages\torch\amp\autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "C:\Users\INP_Rohit\Documents\ImageGeneration\stable-diffusion-webui/extensions/sd-webui-text2video/scripts\stable_lora\stable_utils\lora_processor.py", line 217, in process_lora
for k, v in lora_model.items():
AttributeError: 'NoneType' object has no attribute 'items'
Exception occurred: 'NoneType' object has no attribute 'items'

I have added the link of issue mentioned explicitly with images below:

kabachuha/sd-webui-text2video#241

Can you please help me out with this asap

[codeautopilot]

Potential solution

The plan to solve the bug involves ensuring that the file paths for the LoRA model weights are correct and that the model is loaded successfully before attempting to generate the video. Additionally, we need to add error handling to provide informative messages if the model or weights cannot be found or loaded. This will help the user to quickly identify and fix the issue.

What is causing this bug?

The bug is likely caused by one or more of the following issues:

1. Incorrect file paths for the LoRA model weights, leading to a "No such file or directory" error.
2. Failure to load the LoRA model correctly, resulting in a None object and subsequently causing the "'NoneType' object has no attribute 'items'" error when attempting to access the model's items.
3. Potential issues with the LoRA injection and realization process, which could leave the model in an incomplete or incorrect state.

Code

To address the potential causes, the following code snippets and checks can be implemented:

1. Verify file paths and existence of the LoRA weights:

import os

lora_weights_path = 'path/to/lora/weights'  # Replace with the correct path
if not os.path.exists(lora_weights_path):
    raise FileNotFoundError(f"LoRA weights file not found at {lora_weights_path}")

2. Add error handling after model loading:

model = load_model_function(...)  # Replace with the actual function to load the model
if model is None:
    raise ValueError("Failed to load the LoRA model. Please check the model path and weights.")

3. Ensure LoRA injection and realization are correctly implemented:

# After injecting LoRA layers
if not are_lora_layers_injected(model):  # Replace with the actual check
    raise ValueError("LoRA layers were not injected correctly.")

# After realizing LoRA layers
if not is_lora_realization_successful(model):  # Replace with the actual check
    raise ValueError("LoRA layers were not realized correctly.")

How to replicate the bug

To replicate the bug, follow these steps:

1. Go to the web UI and open the text2video generation feature.
2. Load the trained LoRA text-to-video model using the option provided at the bottom.
3. Enter a prompt and attempt to generate a video.
4. The error should appear in the terminal.

Task

To resolve the issue, the following tasks should be performed:

1. Verify and correct the file paths for the LoRA model weights.
2. Implement error handling to check for successful model loading and provide informative error messages.
3. Review the LoRA injection and realization logic to ensure it is applied correctly.
4. Add unit tests to validate the functionality of LoRA-related operations.

Click here to create a Pull Request with the proposed solution

Files used for this task:

Changes on utils/lora.py

Upon reviewing the utils/lora.py file, it is evident that the code is designed to handle the injection, manipulation, and saving of LoRA (Low-Rank Adaptation) layers into neural network models. The file contains a variety of classes and functions to support these operations.

The user's issue is related to loading a fine-tuned/trained LoRA model and generating a video. The error messages mentioned are "No such file or directory" and "'NoneType' object has no attribute 'items'".

Based on the provided code and the error messages, here are some potential causes and recommendations for the issue:

1. File Not Found Error: The error "No such file or directory" suggests that the code is trying to access a file that does not exist at the specified path. This could be due to an incorrect file path or the file not being present where the code expects it to be.

   • Recommendation: Ensure that the file paths specified in the code for loading the LoRA weights are correct and that the files actually exist at those locations. This includes checking the paths provided to functions like torch.load() which are used to load the LoRA weights.

2. 'NoneType' object Error: The error "'NoneType' object has no attribute 'items'" indicates that a variable expected to be a dictionary or an object with an items() method is actually None. This could happen if the model or weights are not loaded correctly, resulting in None being passed to a function or method that expects a dictionary.

   • Recommendation: Add error handling around the code that loads the LoRA weights to ensure that the loading process is successful. If the weights cannot be loaded, the code should provide a clear error message indicating the problem. This can be done by checking if the result of torch.load() or similar functions is None and handling that case appropriately.

3. Safe Loading of Weights: The code uses a function safe_open from the safetensors library to load weights safely. If the weights file is not in the expected format or is corrupted, it could lead to the "No such file or directory" error.

   • Recommendation: Verify that the weights file is in the correct format expected by the safe_open function. If the file is not a valid safetensors file, it will not be loaded correctly.

4. LoRA Injection and Realization: The code contains methods for injecting LoRA layers (inject_trainable_lora, inject_trainable_lora_extended) and realizing them as part of the model (collapse_lora). If these processes are not executed correctly, it could lead to the model being in an incomplete or incorrect state.

   • Recommendation: Review the logic related to LoRA injection and realization to ensure that it is being applied correctly. Ensure that the LoRA layers are properly initialized and that their weights are being set as expected.

In summary, the user should verify the file paths for the LoRA weights, add error handling to provide informative messages if the weights file cannot be found or loaded, and ensure that the LoRA injection and realization processes are correctly implemented. Additionally, it would be helpful to have unit tests to validate the functionality of the LoRA-related operations to catch any issues early in the development process.

Changes on inference.py

After reviewing the inference.py file, here are some observations and recommendations:

1. Model Loading Path: The initialize_pipeline function loads the model using UNet3DConditionModel.from_pretrained(model, subfolder='unet'). It is important to ensure that the model variable passed to this function contains the correct path to the model directory. If the path is incorrect, the model will not load, which could lead to the 'NoneType' object has no attribute 'items' error.

2. Error Handling: There is no explicit error handling for the case when the model fails to load. It would be beneficial to add checks after the model loading lines to ensure that the model is not None. If the model is None, an informative error message should be displayed to the user.

3. LoRA Weights: The inject_inferable_lora function is called with pipeline, lora_path, and lora_rank as arguments. It is crucial to verify that the lora_path is correct and points to the location where the LoRA weights are stored. If the path is incorrect, the function will not be able to load the LoRA weights, which could be the cause of the error.

4. File Paths in Arguments: The script accepts file paths as arguments (e.g., --model, --init-video). Users should be careful to provide the correct paths when running the script. It might be helpful to add checks to verify that the provided file paths exist before attempting to use them.

5. Model Loading Checks: Before calling functions that use the model (e.g., vid2vid, pipeline), there should be checks to ensure that the model and its components (e.g., scheduler, tokenizer, text_encoder, vae, unet) are loaded correctly and are not None.

6. Output Directory: The script creates an output directory if it does not exist. However, there is no check to see if the directory creation was successful. Adding a check after os.makedirs(output_dir, exist_ok=True) could prevent issues related to directory access or creation.

7. Inference Function: The inference function is the main entry point for generating videos. It is important to ensure that all the necessary components are loaded correctly before proceeding with video generation. If any component fails to load, the function should return an error message.

In summary, the main focus should be on verifying file paths, adding error handling to check for successful model loading, and ensuring that the LoRA weights are correctly loaded. These steps will help prevent the 'No such file or directory' and 'NoneType' object has no attribute 'items' errors.

Disclaimer: This comment was entirely generated using AI. Be aware that the information provided may be incorrect.

Current plan usage: 0.00%

Have feedback or need help?
Discord
Documentation
[email protected]