Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to register multiple .ome.tif at the same time & issue registering .ome.tif #12

Open
Boehmin opened this issue Sep 4, 2023 · 7 comments

Comments

@Boehmin
Copy link

Boehmin commented Sep 4, 2023

Hi @Yu-AnChen !

I just tried palom on two cycles of FISH and the registration so far seems really really impressive (in particular since one of the cycles is half out of focus). First of all, thank you so much for making this tool available!

I was wondering how it would be possible to register multiple .ome.tif images simultaneously. Additionally, the registration worked fine with one of my datasets (converted from .h5 to .ome.tif which was a big pain already), however I have issues with a smaller .ome.tif dataset. I get the below error message:

(palom) [boehm0002@sgi27 ]$python3 palom_dan_ome_R1toR2.py
2023-09-04 14:12:42.717 | WARNING  | palom.reader:auto_format_pyramid:64 - Unable to detect pyramid levels, it may take a while to compute thumbnails during coarse alignment
2023-09-04 14:12:42.729 | WARNING  | palom.reader:auto_format_pyramid:64 - Unable to detect pyramid levels, it may take a while to compute thumbnails during coarse alignment
Traceback (most recent call last):
  File "/scicore/home/rueegg/boehm0002/spatial_transcriptomics/dans_test1/HiPlex_Example_DJH/palom_dan_ome_R1toR2.py", line 27, in <module>
    ref_thumbnail=c1r.read_level_channels(THUMBNAIL_LEVEL, 2).compute(),
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/dask/threaded.py", line 89, in get
    results = get_async(
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/dask/local.py", line 511, in get_async
    raise_exception(exc, tb)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/dask/local.py", line 319, in reraise
    raise exc
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/dask/local.py", line 224, in execute_task
    result = _execute_task(task, data)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/zarr/core.py", line 821, in __getitem__
    result = self.get_basic_selection(pure_selection, fields=fields)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/zarr/core.py", line 947, in get_basic_selection
    return self._get_basic_selection_nd(selection=selection, out=out,
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/zarr/core.py", line 990, in _get_basic_selection_nd
    return self._get_selection(indexer=indexer, out=out, fields=fields)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/zarr/core.py", line 1285, in _get_selection
    self._chunk_getitem(chunk_coords, chunk_selection, out, out_selection,
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/zarr/core.py", line 1994, in _chunk_getitem
    cdata = self.chunk_store[ckey]
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/zarr/storage.py", line 738, in __getitem__
    return self._mutable_mapping[key]
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 12274, in __getitem__
    return self._getitem(key)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 12951, in _getitem
    keyframe, page, chunkindex, offset, bytecount = self._parse_key(key)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 13050, in _parse_key
    page = series[pageindex]
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 12135, in __getitem__
    return self._getitem(int(key))
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 12115, in _getitem
    return self.parent.pages._getitem(page.index + key)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 7568, in _getitem
    self._seek(key)
  File "/scicore/home/rueegg/boehm0002/miniconda3/envs/palom/lib/python3.10/site-packages/tifffile/tifffile.py", line 7402, in _seek
    raise ValueError('seek of closed file')
ValueError: seek of closed file

Thanks ever so much for your help!

@Yu-AnChen
Copy link
Collaborator

Hi @Boehmin thanks for the kind words :)

Palom might be able to read h5 files depending on the h5 file structure. If you have a small dataset to share I'll find time to take a look.

For the .ome.tif dataset that gave you the error, it looks to me that the file might have some issues. Would you be able to share that?

To register multiple files, our recommended approach is to register everything (cycles 2, 3, 4, and so on) to the first cycle. Please checkout the example script and lmk for any questions.

@Boehmin
Copy link
Author

Boehmin commented Sep 8, 2023

Hi @Yu-AnChen

Thanks for pointing me to the example script. I'll try this!

I have a smaller fused example h5 file (only one timepoint) and the ome tif dataset here. The .h5 consists of z-stacks so I do not know if that would be an issue for palom? If not that would be even better. Channel 4 is DAPI and channel 3 is a fiducial channel so in theory I could register over the fiducial channel as well if that helps?

Again, thanks for looking into this.

@Yu-AnChen
Copy link
Collaborator

Thanks for sharing the images, I was able to reproduce the error when using your ome-tiff files as input. Will look into a solution there.

On the h5 dataset (seems to be generated with BigDataViewer), I was able to convert it into a pyramidal ome-tiff using palom. Note that installing h5py in the environment that you run palom is required.

Since there are Z-stacks, I did a maximum intensity projection on the Z-axis in the following script (the output file size is ~50 MB; it can be opened in QuPath)

import palom
import h5py
import dask.array as da
import pathlib

img_path = pathlib.Path('h5_small/small_fuse_fused.h5')
h5_img = h5py.File(img_path)

# X-Y pixel size at full resolution; unit is µm/pixel
PIXEL_SIZE = 1
CHANNEL_NAMES = ['marker-1', 'marker-2', 'marker-fiducial', 'DNA']

mosaics = [
    # max projection on the Z axis
    da.from_array(h5_img[f"t00000/{channel}/0/cells"]).max(axis=0)
    for channel in h5_img['t00000']
]

palom.pyramid.write_pyramid(
    mosaics=mosaics,
    output_path=img_path.parent / f"{img_path.stem}-pyramid.ome.tif",
    pixel_size=PIXEL_SIZE,
    channel_names=CHANNEL_NAMES,
    downscale_factor=2,
    compression='zlib',
    tile_size=1024,
    save_RAM=True
)

With the output ome-tiffs from the above script, I think you should be able to run palom without running into the ValueError: seek of closed file error in your first comment.

@Yu-AnChen
Copy link
Collaborator

As for the issue of ValueError: seek of closed file, I've found a workaround. Please try it out on your end - I don't have a complete solution to it yet but hopefully that can get you up and running.

@lguerard
Copy link

Since there are Z-stacks, I did a maximum intensity projection on the Z-axis in the following script (the output file size is ~50 MB; it can be opened in QuPath)

Sorry to hijack this, is palom working on 2D files only or did you do a projection to speed it up ?
Thanks !

@Boehmin
Copy link
Author

Boehmin commented Sep 14, 2023

@Yu-AnChen, I tried both solutions and they both worked! Thank you.
Also curious whether this could work in 3D. :)

@Yu-AnChen
Copy link
Collaborator

The current image file reader and writer only works with images that have 2 or 3 dimensions. In our own application, it's (Channel/Band, Y, X). Palom could compute and find out the transformation/mapping between two (C, Z, Y, X) datasets and apply the transformation plane by plane, but the image file reader and writing will need some custom formatting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants