PYP (tomoswarm) failed due to "FileNotFoundError: slot8_311.st_0000_ctffind4_avrot.txt not found."

[MiaoXiaoPu1121]

Hi dear,

I was using my own data to try nextPYP but it went wrong during pre-processing. Here is part of the log:

234 | 2024-11-02 20:36:46 [INFO] pyp/system/mpi.py:111 | 35 command(s) finished
235 | 2024-11-02 20:39:59 [INFO] /opt/pyp/bin/run/pyp:57 | Apply alignment to tiltseries took: 00h 05m 19s
236 | 2024-11-02 20:39:59 [INFO] pyp/ctf/core.py:722 | Central z-slice for initial CTF estimation is 18,18
237 | 2024-11-02 20:40:14 [INFO] pyp/ctf/core.py:57 | CTF estimation took: 00h 00m 15s
238 | 2024-11-02 20:49:09 [INFO] pyp/merge/core.py:57 | Merging took: 00h 08m 51s
239 | 2024-11-02 20:49:10 [INFO] /opt/pyp/bin/run/pyp:57 | Total time in tomo_swarm 00h 31m 33s seconds
240 | 2024-11-02 20:49:10 [INFO] /opt/pyp/bin/run/pyp:3178 | Filesystem Size Used Avail Use% Mounted on
241 | 2024-11-02 20:49:10 [INFO] /opt/pyp/bin/run/pyp:3178 | /dev/sdb2 1.8T 712G 1.1T 40% /tmp/pyp
242 | joblib.externals.loky.process_executor._RemoteTraceback:
243 | """
244 | Traceback (most recent call last):
245 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 428, in _process_worker
246 | r = call_item()
247 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 275, in call
248 | return self.fn(*self.args, **self.kwargs)
249 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 620, in call
250 | return self.func(*args, **kwargs)
251 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 288, in call
252 | return [func(*args, **kwargs)
253 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 288, in
254 | return [func(*args, **kwargs)
255 | File "/opt/pyp/src/pyp/system/mpi.py", line 174, in wrapper
256 | func(*arg)
257 | File "/opt/pyp/src/pyp/ctf/core.py", line 690, in ctffind_tilt_multiprocessing
258 | [df1, df2, angast, cc, res] = run_ctffind_tilt(
259 | File "/opt/pyp/src/pyp/ctf/core.py", line 1537, in run_ctffind_tilt
260 | ctfprof = np.loadtxt(image_file + "_ctffind4_avrot.txt", comments="#")
261 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/numpy/lib/npyio.py", line 1356, in loadtxt
262 | arr = _read(fname, dtype=dtype, comment=comment, delimiter=delimiter,
263 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/numpy/lib/npyio.py", line 975, in _read
264 | fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
265 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/numpy/lib/_datasource.py", line 193, in open
266 | return ds.open(path, mode, encoding=encoding, newline=newline)
267 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/numpy/lib/_datasource.py", line 533, in open
268 | raise FileNotFoundError(f"{path} not found.")
269 | FileNotFoundError: slot8_311.st_0000_ctffind4_avrot.txt not found.
270 | """
271 |  
272 | The above exception was the direct cause of the following exception:
273 |  
274 | Traceback (most recent call last):
275 | File "/opt/pyp/bin/run/pyp", line 3440, in
276 | tomo_swarm(args.path, args.file, args.debug, args.keep, args.skip)
277 | File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
278 | return func(*args, **kwargs)
279 | File "/opt/pyp/bin/run/pyp", line 1588, in tomo_swarm
280 | mpi.submit_function_to_workers(mpi_funcs, mpi_args, verbose=parameters["slurm_verbose"], silent=True)
281 | File "/opt/pyp/src/pyp/system/mpi.py", line 182, in submit_function_to_workers
282 | parallel(delayed(wrapper)(func, *arg, current_directory=current_directory) for idx, func in enumerate(funcs) for arg in args[idx])
283 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 1098, in call
284 | self.retrieve()
285 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/parallel.py", line 975, in retrieve
286 | self._output.extend(job.get(timeout=self.timeout))
287 | File "/usr/local/envs/pyp/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 567, in wrap_future_result
288 | return future.result(timeout=timeout)
289 | File "/usr/local/envs/pyp/lib/python3.8/concurrent/futures/_base.py", line 437, in result
290 | return self.__get_result()
291 | File "/usr/local/envs/pyp/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
292 | raise self._exception
293 | FileNotFoundError: slot8_311.st_0000_ctffind4_avrot.txt not found.
294 | 2024-11-02 20:49:10 [ERROR] /opt/pyp/bin/run/pyp:3445 | PYP (tomoswarm) failed

I also played with the tutorial data with the same parameter and everything goes right.

Any suggestions about this?

[abartesaghi]

It looks like CTF estimation is failing for some tilts in this tilt-series. This can happen for a variety of reasons, but sometimes adjusting the resolution range used for CTF estimation and/or tightening the upper/lower defocus limits can help solve these problems. Does this happen for all tilt-series in the dataset or just this one?

[MiaoXiaoPu1121]

It looks like CTF estimation is failing for some tilts in this tilt-series. This can happen for a variety of reasons, but sometimes adjusting the resolution range used for CTF estimation and/or tightening the upper/lower defocus limits can help solve these problems. Does this happen for all tilt-series in the dataset or just this one?

Hi Bartesaghi,
Thanks for reply,
I've only used two of my tilt series for testing in nextPYP, so I'm not sure about the status of the others. I will try adjusting the CTF parameters based on your advice.
BTW, does nextPYP integrate only ctfind4 for CTF parameter estimation, or can we use other tools like Gctf? Perhaps I missed that option.

[abartesaghi]

nextPYP only runs ctffind4 for now. However, we may consider adding other programs to estimate the CTF in the future.