BUG: Running suite2p interrupted (no file/directory found)

[marytunun]

Describe the issue:

I opened an issue here before about installation: #946
Tried the suggested solution, but the sample data still had a 'strip data' issue, so I tried on tiff images from my actual lab. I have now a folder with those tiff images for analysis, all of them are 256x256.

Running suite2p almost worked, seems that it successfully went through steps of converting to binary, registration, ROI detection, extraction, classification and spike convolution. BUT after that I have errors, and it seems like those errors are not because of something I did..

Is suite2p for some reason not able to create thosee specific files/directories? I require your assistance with this error, thank you.

Reproduce the code example:

{'data_path': ['C:/Users/Maria/Documents/suite2p_docs/lab_data_samples/Ca_imaging_organoids/256tiffs'], 'subfolders': [], 'save_path0': 'C:/Users/Maria/Documents/suite2p_docs/save_path', 'fast_disk': 'C:/Users/Maria/Documents/suite2p_docs/fast_disk', 'input_format': 'tif'}
tif
** Found 6330 tifs - converting to binary **
2000 frames of binary, time 139.29 sec.
4000 frames of binary, time 274.81 sec.
6000 frames of binary, time 406.76 sec.
time 429.00 sec. Wrote 6330 frames per binary for 2 planes
>>>>>>>>>>>>>>>>>>>>> PLANE 0 <<<<<<<<<<<<<<<<<<<<<<
NOTE: not registered / registration forced with ops['do_registration']>1
      (no previous offsets to delete)
NOTE: applying default C:\Users\Maria\.suite2p\classifiers\classifier_user.npy
----------- REGISTRATION
Reference frame, 4.09 sec.
Registered 500/6330 in 11.82s
Registered 1000/6330 in 17.65s
Registered 1500/6330 in 23.84s
Registered 2000/6330 in 29.45s
Registered 2500/6330 in 33.80s
Registered 3000/6330 in 38.73s
Registered 3500/6330 in 43.34s
Registered 4000/6330 in 48.18s
Registered 4500/6330 in 53.37s
Registered 5000/6330 in 57.28s
Registered 5500/6330 in 61.32s
Registered 6000/6330 in 65.27s
Registered 6330/6330 in 67.96s
----------- Total 73.90 sec
Registration metrics, 12.05 sec.
----------- ROI DETECTION
Binning movie in chunks of length 10
Binned movie of size [633,208,208] created in 1.10 sec.
NOTE: estimated spatial scale ~12 pixels, time epochs 1.00, threshold 10.00 
0 ROIs, score=75.32
Detected 13 ROIs, 6.14 sec
After removing overlaps, 11 ROIs remain
----------- Total 7.46 sec.
----------- EXTRACTION
Masks created, 60.64 sec.
Extracted fluorescence from 11 ROIs in 6330 frames, 11.77 sec.
----------- Total 72.49 sec.
----------- CLASSIFICATION
['skew', 'npix_norm', 'compact']
----------- SPIKE DECONVOLUTION
----------- Total 0.01 sec.
Plane 0 processed in 165.99 sec (can open in GUI).
>>>ERROR<<<
Traceback (most recent call last):
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 194, in _run_module_as_main
>>>ERROR<<<
    return _run_code(code, main_globals, None,
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\__main__.py", line 81, in <module>
>>>ERROR<<<
    main()
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\__main__.py", line 74, in main
>>>ERROR<<<
    run_s2p(ops, db)
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 506, in run_s2p
>>>ERROR<<<
    op = np.load(ops_path, allow_pickle=True).item()
  File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\numpy\lib\npyio.py", line 405, in load
>>>ERROR<<<
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/Maria/Documents/suite2p_docs/save_path\\suite2p\\plane1\\ops.npy'
Interrupted by error (not finished)

Error message:

No response

Version information:

suite2p v0.12.2.dev30+g828b5e2

Context for the issue:

Cannot open my tiff images because the running is interrupted

[carsen-stringer]

Perhaps you didn't create the folder you're trying to save in?

[generalciao]

Do you have / are you expecting 2 planes? I.e. do you have a fast focus switching setup (ETL, piezo)? Because the suite2p output implies it thinks your data contains 2 planes, and the error occurs for a 2nd plane folder.

Note at the beginning the following output:
Wrote 6330 frames per binary for 2 planes

And at the end the FileNotFoundError refers to "plane1", which would be the second plane:
'C:/Users/Maria/Documents/suite2p_docs/save_path\\suite2p\\plane1\\ops.npy'

I have no personal experience with multiplane analysis with suite2p, so this might still be an installation bug as you suspect, but it might also be a mismatch between your data and your settings?

[marytunun]

Do you have / are you expecting 2 planes? I.e. do you have a fast focus switching setup (ETL, piezo)? Because the suite2p output implies it thinks your data contains 2 planes, and the error occurs for a 2nd plane folder.

Note at the beginning the following output: Wrote 6330 frames per binary for 2 planes

And at the end the FileNotFoundError refers to "plane1", which would be the second plane: 'C:/Users/Maria/Documents/suite2p_docs/save_path\\suite2p\\plane1\\ops.npy'

I have no personal experience with multiplane analysis with suite2p, so this might still be an installation bug as you suspect, but it might also be a mismatch between your data and your settings?

Actually, my data has 1 plane, and it is also 1 in the 'main settings'. Do you think I should put 0 planes instead? Although that would be really weird...

[marytunun]

Perhaps you didn't create the folder you're trying to save in?

The folder exists for sure. But what I also tried now is to not specify separate folders for save_path and fast_disk, and I got another error that started already during the extraction process:

----------- EXTRACTION
---ERROR---
Traceback (most recent call last):
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 194, in _run_module_as_main
---ERROR---
return run_code(code, main_globals, None,
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\runpy.py", line 87, in run_code
---ERROR---
exec(code, run_globals)
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p_main.py", line 81, in
---ERROR---
main()
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p_main.py", line 74, in main
run_s2p(ops, db)
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 518, in run_s2p
---ERROR---
op = run_plane(op, ops_path=ops_path)
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 333, in run_plane
ops = pipeline(f_reg, f_raw, f_reg_chan2, f_raw_chan2, run_registration, ops,
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\run_s2p.py", line 170, in pipeline
---ERROR---
stat, F, Fneu, F_chan2, Fneu_chan2 = extraction.extraction_wrapper(
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\extraction\extract.py", line 189, in extraction_wrapper
---ERROR---
cell_masks, neuropil_masks0 = create_masks(stat, Ly, Lx, ops)
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\extraction\masks.py", line 13, in create_masks
---ERROR---
cell_pix = create_cell_pix(stats, Ly=Ly, Lx=Lx,
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\suite2p\extraction\masks.py", line 50, in create_cell_pix
filt = percentile_filter(lammap, percentile=lam_percentile,
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\scipy\ndimage_filters.py", line 1467, in percentile_filter
---ERROR---
return _rank_filter(input, percentile, size, footprint, output, mode,
File "C:\Users\Maria\anaconda3\envs\suite2p\lib\site-packages\scipy\ndimage_filters.py", line 1334, in _rank_filter
_nd_image.rank_filter(input, rank, footprint, output, mode, cval,
---ERROR---
MemoryError
Interrupted by error (not finished)

[marytunun]

Didn't really solve the issue, but I just jumped to my lab's data instead of the sample data and got other errors, which I better describe in a separate thread