Temporary directory to Permanent directory on Python Script

[HripsimeS]

Dear MONAI team, @Nic-Ma @yiheng-wang-nv

I am trying to switch from Jupyter Notebook to Python scripts in order to optimize a faster inference mode using your example.
https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/torch/unet_training_dict.py

I don't want to create a temporary directory, just use files from my local permanent directory for the training and testing. Therefore I am receiving an error for the last part of the code I assume. Here is the full code of the training script.
https://github.com/HripsimeS/Computer-Vision/blob/main/train_script_GitHub.ipynb

Can you please help to see how to modify my code to make it work eventually. Look forward to hearing from you!

[yiheng-wang-nv]

Hi @HripsimeS , it looks like there is no data loaded, thus step is 0.
Can you please double check if the data is loaded normally? You can do sth like:

...
for batch_data in train_loader:
    step += 1
    inputs, labels = batch_data["image"].to(device), batch_data["mask"].to(device)
    print(inputs.shape)

In addition, from your notebook, I did not find the log that should be printed (like in this line: print(f"epoch {epoch + 1}/{max_epochs}"), which is at the beginning of each epoch). Therefore, could you keep the full output and save the notebook?

[HripsimeS]

@yiheng-wang-nv Thanks for your reply. The problem is that I am escaping the step when I need to create a temporary directory to generate a synthetic data to {tempdir}. And in the end of notebook I use just main(root_dir) instead of "with tempfile.TemporaryDirectory() as tempdir: main(tempdir)". I tried to add the "check_ds" part to do a data check and received an TypeError: 'NoneType' object is not subscriptable.

https://github.com/HripsimeS/Computer-Vision/blob/main/train_script_GitHub.ipynb

The dataset is loading fine outside of function. But as I don't create a temporarly direction, the beginning and the end of my function is probably wrong. For the log part that should print was in the script just below the the loop "for epoch in range(max_epochs)", unless if you mean something else...
Please can you check the beginning and the end of my function, maybe the error comes from that part. Let me know if you need additional information from my side.

[yiheng-wang-nv]

you have:

root_dir  = '../projects/CTPelvic1K_data/root_dir'

and

    train_images = sorted(glob.glob(os.path.join(root_dir, "*data.nii.gz")))
    train_labels = sorted(glob.glob(os.path.join(root_dir, "*mask_4label.nii.gz")))

Could you ensure that you have images and labels in '../projects/CTPelvic1K_data/root_dir/'? You can use the following to check:

print(os.listdir(root_dir))

If the dir is wrong, there will be no data in data_dicts, and the error will happen.

By the way, maybe you can submit a .py file version, thus it's easier to mention the specific line that may have the issue.

[HripsimeS]

@yiheng-wang-nv thank you very much, you were totally right about the path. I had to put a full name of the path, like this 'C:/Users/Name/OneDrive /projects/CTPelvic1K_data/root_dir' and then it worked out. I did not have an issue before with a short path's name on Jupyter Notebook, but maybe in the script inside of function it is mandatory to put a full name of path. Currently my train.py file is running on command prompt without problem, just calculation of each epoch takes 15 times longer compared if I run the same training on Jupyter Notebook. I will see later how to fix the runtime issue. Thank you again for your help :) 👍 🥇