Troubleshooting Import Tilt Series

[jakecroft]

Hello, I'm trying to import my data into nextPYP 0.6.3, and it seems to be failing. The job doesn't actually fail or give any error message I can find, but the spinning wheel keeps on going forever with no progress (over a day, I'm only importing one tilt series to try and get it to work). At first I thought it could be an issue with symlinks or permissions, so I hard copied the data into the folder and used chmod +777 to give all users access, but that doesn't seem to fix it. Possibly it could because I don't have mdoc files for this data, I wasn't sure if the import would try to parse the filenames for info like tilt angle or how else to specify that. Do you have guidelines for how the files should be named/organized prior to import? Unsure what else to try since I don't see any kind of output/error for the job.

Thanks,
Jake Croft (University of Washington)

[abartesaghi]

Are you running in standalone mode or with a slurm cluster attached? If you are an admin for the instance (which it looks like you are), go to the Administration section, then the PYP tab, and click on "PYP/WebRPC Ping" button. After a short while, you should get a message similar to this:

Result Type: Success
Exit code: 0

sending ping to "servername" ...
response:
{"id":1,"jsonrpc":"2.0","result":"pong"}

If you don't, you will need to look into the troubleshooting section.

If you can select the files from the file browser, there is probably no issue with file permissions. However, if the symlinks point to a path is not within the list of binds, then that would prevent you from reading the files. To solve that problem, you need to add the actual location of the files to the list of binds.

If your tilt-series is in a single file, the only additional information you need are the tilt angles and the order of acquisition. If you don't have .mdoc files, you need to provide .rawtlt and .order files for each tilt-series containing that information. Examples of these files are included with the classification tutorial data.

You don't need to name the tilt-series files in any particular way, but the files containing the metadata should have the same name as the tilt-series (just different extension), e.g., tilt-series-01.mrc, tilt-series-01.rawtlt and tilt-series-01.order.

Also, when looking for errors in the logs, make sure you look at the Logs tab which is right next to the Stream tab.

[jakecroft]

Thanks for the response. Yep, running in standalone mode as an admin. I ran the ping test and it was successful. And yes, I am able to select the files in the file browser so looks like it should be able to read them.

I don't have the tilt series combined into a single file, since I wasn't sure what happens when you make a stack of multi-frame movies (I usually run motioncor externally, then create a stack before import into other software, but wanted to import the movies into nextPYP since motion correction is integrated). Looking at the data from the normal Tomography tutorial, the only other files I see are a ".spk" file for each tilt series, but I'm unsure what these are and am unable to open these in my text editor. How do you specify the tilt information/order if you're importing the tilt series with each tilt as a separate movies?

Or alternatively, what is the correct way to combine movies into a tilt series stack? (I usually use imod newstack command on pre-aligned mrcs, when I tried this on the movies it seemed to create a stack of the frames hundreds of images tall but the stacks in the Classification Tutorial were only 41 images when i viewed them in imod- do they still have the frames included?)

[abartesaghi]

That should be OK, you don't need to have a single stack with all the frames (in fact, nextPYP does not know how deal with data in that format right now). The .spk files are for importing particle coordinates, so that's not relevant here either.

In order to import tilt-frame data into nextPYP you either need to provide corresponding .mdoc files for each tilt (these are typically generated by the data acquisition software) or you need use the correct value for the Frame pattern parameter (Frame alignment tab in the Pre-processing block). This parameter allows you extract the name of the tilt-series, the order of acquisition and the tilt-angle from the file name of each tilt-file (assuming this information is encoded there). This parameter defaults to TILTSERIES_SCANORD_ANGLE.tif, which corresponds to having frames in the .tif format and files named like this:TS_02_04_6.0.tif, where the tilt-series name is TS_02, the sequence number is 4, and the tilt-angle is 6 degrees.

If you provide an example of how your files are named, we can help you figure out the correct File pattern to use.

[jakecroft]

Ok, thanks for the input. Trying to ask the most basic questions because in cases like this it is the most simple things that cause issues... So it sounds like from what you've said nothing about my data should cause the import job to fail. I wasn't sure when the tilt series order needs to be specified at the beginning, but it seems I can do that in the next step with "Frame Pattern" (I have similar naming with some extra dataset name info at the beginning, an example filename is "Imm-ADA_b4g3-nov122021_100_0001_-3.0.tif". In any case I can probably figure out that when I get to the next step but what is really confusing me is why it won't import.

In case it was something with my installation, I tried with the tomo tutorial (10164) data and this successfully imported almost immediately. Now I can click on the icon next to the job in the jobs panel and see the stream and logs tabs you mentioned. When I try with my own data, the gears next to the job just spin forever the icon that takes me to stream/logs in the tutorial data is a question mark that initiates a popup that says "This job is running under PID xxx". As far as I can tell there is not anything different between my data and the tutorial data. They are both .tif files that open up in imod and look like multi-frame movies, and they seem to have the same file permissions. Only difference is how they are named, but it seems that shouldn't matter until the next step. Mine has a gain reference but I tried importing without it like the tutorial to see if the gain ref file was messing it up and that didn't change anything. I guess the other small difference is that my movies were collected in super-resolution mode and haven't been binned yet. Not sure if that would be relevant... Do you have any other ideas?

[abartesaghi]

If the tilt-series are super resolution, the import job may be running out of memory. Try increasing the memory per task in the import block to something bigger than 4GB. If you are indeed running out of memory, the logs of the failed job should tell you that, but it doesn't look like you can access the logs at all. Actually, if you are running in standalone mode, I don't think we are enforcing the limits on the memory resources, so my suggestion may not work, but probably still worth a try since I'm beginning to run out of ideas here.

[yezhou867]

Sometimes if the files come from the Windows system where you do the data collection. There could be extra quotation mark in the file names, which could also be causing some problems. Not sure if this your case.

[jakecroft]

Just to update here - it seems to be a problem with some specific files. I'm able to import other data collected in the same way on the same microscope without a problem. The files that aren't working aren't super crucial or urgent so I may not put too much time into wrestling with them but I'll report back if I figure out the cause.