Types, Filesystem Management, and BioformatsReader Fixes

The aicsimageio v4.8.0 release includes:

• aicsimageio now ships with mypy type annotations. If you are using mypy, our types can now be properly imported and type checked against.
• aicsimageio image containers and writer functions now have an fs_kwargs parameter to pass down any arguments for creating the fsspec Filesystem object with. i.e. AICSImage("s3://some-bucket/some-image.ome.tiff", fs_kwargs=dict(anon=True)) for anonymous / no credential S3 file reading.
• BioformatsReader bugfixes for scene management / switching and preparing the reader for the next release of the underlying bioformats_jar version.

We have also dropped support for Python 3.7 and added support for Python 3.10

An additional minor package admin change has been added which runs our aicsimageio test suite multiple times a week but installs with our upstream dependencies main branches instead of released versions. This is so we can help catch bugs earlier in the release process rather than post release.

What's Changed

• admin/ship-mypy-type-annotations-drop-py37-add-py310-manage-test-dependencies-separate-for-each-tox-env by @JacksonMaxfield in #397
• bugfix/pass-series-index-in-biofile-init by @JacksonMaxfield in #401
• feature-and-admin/create-aicsimage-objects-with-fs-kwargs-and-remove-need-for-creds by @JacksonMaxfield in #407
• admin/add-test-upstreams-action by @JacksonMaxfield in #406
• admin/prepare-bioformats_reader-to-work-with-new-bioformats_jar-based-on-scyjava by @tlambert03 in #402

Contributors

@toloudis
@AetherUnbound
@tlambert03
@jrbourbeau
@JacksonMaxfield

Full Changelog: v4.7.0...v4.8.0

BFIO OmeTiledTiffReader

We are pleased to annouce the release of aicsimageio v4.7.0!

This release brings another new reader: the OmeTiffTiledReader.

For certain tiled / mosaic OME-TIFFs, this reader massively speedups read performance and you can see all the benchmarks provided by @Nicholas-Schaub in the PR which added this reader: #387

The new reader can be installed with pip install aicsimageio[bfio]

Changelog

• feature/tiledtiffreader by @Nicholas-Schaub in #387

Reviewers

• @JacksonMaxfield
• @toloudis

Full Changelog: v4.6.4...v4.7.0

Fix Past OMEXML Plane Handling

The previous v3 and below OMEXML.py continues to catch us. This patch should alleviate some issues created by it's functionality in the current reader by fixing out-of-order metadata information prior to reading.

What's Changed

• bugfix/add-logic-to-ensure-OME-XML-plane-elements-occur-last by @Nicholas-Schaub in #385

New Contributors

• @Nicholas-Schaub made their first contribution in #385

Full Changelog: v4.6.2...v4.6.3

Fix Optional Pixel Sizes for CZI to OME Metadata Conversion

A very small patch to fix CZI to OME converted metadata for the physical pixel sizes attribute. As such this change only affects usage of the ome_metadata property for CZIReader.

What's Changed

• admin/update-czi-to-ome-xslt-submodule by @JacksonMaxfield in #382

Full Changelog: v4.6.1...v4.6.2

Fix CZI Physical Pixel Sizes

This is a quick patch put in by @emay2022 to fix CZI physical pixel sizes. Users of AICSImageIO may notice that their CZI physical pixel sizes have changed after upgrading but we believe them to be accurate now as the original implmentation of parsing the CZI metadata was string based and not float calculation / unit conversion based.

What's Changed

• bugfix/czi-physical-size by @emay2022 in #384

Contributors and Reviewers this Release

• @emay2022
• @ianhi
• @toloudis
• @JacksonMaxfield

Full Changelog: v4.6.0...v4.6.1

Managing CZI Licensing and Reader Selection

AICSImageIO 4.6.0

This release wraps up our licensing and dependency management work.

In v4.3.0 we changed the install patterns of BioformatsReader and LifReader due to their GPL licenses. We have followed the same process to change the install pattern for CziReader.

• CziReader -- was installable with pip install aicsimageio[czi] but must now be installed with pip install aicsimageio aicspylibczi>=3.0.5

aicspylibczi is a completely separate libraries that carries with it a GPL license, if you use this reader, be sure to see how your code should now be licensed and add this library to your own dependency list because our install option is no longer available.

Changelog

• Fix many issues with reader selection during AICSImage object init #367
• Upgrade nd2 supporting lib dependency version #379
• Remove CZI install pattern #376
• Add more information and documentation to logged error on corrupt file #380

To review all changes made in this release please see our full CHANGELOG.

Contributors and Reviewers this Release (alphabetical)

Jackson Maxfield Brown (@JacksonMaxfield)
Talley Lambert (@tlambert03)
Madison Swain-Bowden (@AetherUnbound)
Dan Toloudis (@toloudis)

Goodness Gracious, Great Globs of TIFFS

AICSImageIO 4.5.0

We are happy to announce the release of AICSImageIO 4.5.0!

AICSImageIO is a library for image reading, metadata conversion, and image writing for microscopy formats in pure Python. It aims to be able to read microscopy images into a single unified API regardless of size, format, or location, while additionally writing images and converting metadata to a standard common format.

If you are new to the library, please see our full documentation for always up-to-date usage and a quickstart README.

Highlights

TIFF Glob Reading

This release adds a TiffGlobReader! Incredibly useful for all the datasets comprised of image stacks stored as multiple TIFFs. And with it, a specific indexer pattern already stored for MicroManager users.

# Given files with names like "s001_t002_c03_z04.tif"
reader = TiffGlobReader("path/to/data/*.tif")

# We can use this to read single image tiffs generated by MicroManager
# Micromanager creates directories for each position so we need to recursively glob 
# for the images files and pass the list to TiffGlobReader.
# Note that all images are named according to "img_channel000_position001_time000000003_z004.tif"
glob_files = glob.glob("path/to/data/**/*.tif", recursive=True)

# since the numbers in Micromanager files are not in STCZ order we
# need to use a different indexer than default. For convenience
# when working MicroManager generated files you can use the provided indexer: TiffGlobReader.MicroManagerIndexer
mm_reader = TiffGlobReader(glob_files, indexer=TiffGlobReader.MicroManagerIndexer)

# as an example of making a custom indexer
# you can manually create the MicroManagerIndexer like so:
import pandas as pd
from pathlib import Path
import re

def mm_indexer(path_to_img):
    inds = re.findall(r”d+”, Path(path_to_img).name)
    series = pd.Series(inds, index=["C", "S", "T", "Z"]).astype(int)
    return series

mm_reader = TiffGlobReader(glob_files, indexer=mm_indexer)

Thanks to @jrussell25 and @ianhi for these additions!

YX Chunking for Large Files Read by BioformatsReader

If the image you are trying to read using BioformatsReader has YX planes that are incredibly large, you may find the new parameters dask_tiles and tile_size useful to additionally chunk the YX dimensions by the provided tile size.

bf_default = BioformatsReader("my_file.svs")
bf_tiled = BioformatsReader("my_file.svs", dask_tiles=True)
bf_tiled_custom = BioformatsReader("my_file.svs", dask_tiles=True, tile_size=(1024, 1024))

assert bf_default.dask_data.chunksize == (1, 1, 1, 4096, 4096)
assert bf_tiled.dask_data.chunksize == (1, 1, 1, 240, 240)
assert bf_tiled_custom.dask_data.chunksize == (1, 1, 1, 1024, 1024)

Thanks to @NHPatterson for this addition!

Other Changes

The Dimensions object now has a __getitem__.

img = AICSImage("my_file.tiff")
img.dims # <Dimensions T:50, C:1, Z: 1, Y: 480, X: 480>
img.dims["T", "Y"] = (50, 480)

Contributors and Reviewers this Release (alphabetical)

Jackson Maxfield Brown (@JacksonMaxfield)
Ian Hunt-Isaak (@ianhi)
Talley Lambert (@tlambert03)
Heath Patterson (@NHPatterson)
Madison Swain-Bowden (@AetherUnbound)
John Russell (@jrussell25)
Dan Toloudis (@toloudis)

Two Readers

AICSImageIO 4.4.0

We are happy to announce the release of AICSImageIO 4.4.0!

AICSImageIO is a library for image reading, metadata conversion, and image writing for microscopy formats in pure Python. It aims to be able to read microscopy images into a single unified API regardless of size, format, or location, while additionally writing images and converting metadata to a standard common format.

If you are new to the library, please see our full documentation for always up-to-date usage and a quickstart README.

Highlights

Native Python ND2 and DV Support

This release adds two native Python readers specific to the ND2 and DeltaVision file formats. Additionally, both of these file format readers are licensed under BSD and as such we include them in our normal pip extra install options.

pip install aicsimageio[nd2,dv]

nd2_img = AICSImage("my-file.nd2")
dv_img = AICSImage("my-file.dv")

Major thanks to @tlambert03 for both of these additions.

Contributors and Reviewers this Release (alphabetical)

Jackson Maxfield Brown (@JacksonMaxfield)
Talley Lambert (@tlambert03)
Madison Swain-Bowden (@AetherUnbound)
Dan Toloudis (@toloudis)

Managing Dependencies

AICSImageIO 4.3.0

This release has been published to begin the process of managing some of our dependencies and their licenses.

AICSImageIO is built off of many independent file format readers, each with their own licensing. For many of our supported formats, the library that provides us the capability to read the format is licensed under a permissive license such as MIT or BSD. However, some of our formats are released under GPL licenses. Previously, AICSImageIO has included all of these dependencies in our pip extra install options, and released our package under BSD license. However, after our latest release, we received push-back on this practice and as such, we have begun the process of stripping out the specific installation of GPL licensed dependencies.

This does not mean that AICSImageIO no longer supports the various file formats covered by GPL license. We still do, however we will no longer provide a pip extra install option for such formats and we will include specific reference to readers that utilize such GPL licenses.

In this release, the two readers that have had their extra install options removed are:

• LifReader -- was installed with pip install aicsimageio[lif] but must now be installed with pip install aicsimageio readlif>=0.6.4
• BioformatsReader -- was installed with pip install aicsimageio[bioformats] but must now be installed with pip install aicsimageio bioformats_jar

Both readlif and bioformats_jar are completely separate libraries that carry with them their own GPL licenses and as such, if you use either of these readers, be sure to see how your code should now be licensed and add these libraries to your own dependency list because our install option is no longer available.

Moving forward, we plan to use this same installation pattern for any more GPL licensed file format readers. AICSImageIO will make them generally available, but will document that the downstream user must install both aicsimageio and the supporting file format library independent so that AICSImageIO remains BSD licensed.

Changelog

• Support for reading LIF files must now be installed separately due to licensing conflicts. (pip install aicsimagieo readlif>=0.6.4) #332
• Support for reading Bio-Formats supported files must now be installed separately due to licensing conflicts. (pip install aicsimageio bioformats_jar) #329
• More detailed error message and documentation on how to configure BioFormats reading. #324
• Shortened reader import paths for any reader. #326

To review all changes made in this release please see our full CHANGELOG.

Contributors and Reviewers this Release (alphabetical)

Chris Allan (@chris-allan)
Sébastien Besson (@sbesson)
Jackson Maxfield Brown (@JacksonMaxfield)
Talley Lambert (@tlambert03)
Josh Moore (@joshmoore)
Peter Sobolewski (@psobolewskiPhD)
Madison Swain-Bowden (@AetherUnbound)
Jason Swedlow (@jrswedlow)
Dan Toloudis (@toloudis)

Bio-Formats Support

AICSImageIO 4.2.0

We are happy to announce the release of AICSImageIO 4.2.0!

AICSImageIO is a library for image reading, metadata conversion, and image writing for microscopy formats in pure Python. It aims to be able to read microscopy images into a single unified API regardless of size, format, or location, while additionally writing images and converting metadata to a standard common format.

If you are new to the library, please see our full documentation for always up-to-date usage and a quickstart README.

Highlights

Bio-Formats Support

Breaking with our "reading and writing for microscopy formats in pure Python", and with a massive thanks to @tlambert03, we now support reading any file format supported by Bio-Formats.

from aicsimageio import AICSImage

nd2_img = AICSImage("my-file.nd2")
dv_img = AICSImage("my-file.dv")
oir_img = AICSImage("my-file.oir")

Just like our other readers, to add this additional file format support you will need to install with pip install aicsimageio[bioformats].
Note: ensure that you have the Java executable available on your PATH, or exposed with the JAVA_HOME environment variable. Adding Java to a conda virtual environment can be done with conda install -c conda-forge openjdk.)

For formats that can be read by both a native Python reader and the Bio-Formats reader, the native Python reader will always take precidence unless it has not been installed. I.e. CZI support can be added with either pip install aicsimageio[czi] or pip install aicsimageio[bioformats], if both are installed, our native Python reader will take precidence. The different readers may result in different pixel and metadata availablity. For full control over this behavior, specify the reader as a parameter.

from aicsimageio import AICSImage
from aicsimageio.readers.bioformats_reader import BioformatsReader
from aicsimageio.readers.czi_reader import CziReader

img_from_native = AICSImage("my-file.czi", reader=CziReader)
img_from_bf = AICSImage("my-file.czi", reader=BioformatsReader)

Contributing Documentation

Aren't satisfied with the available options for image reading? Want to contribute to the project? Our updated Contributing documentation has an entire section on adding a new file format reader.

We look forward to the continued growth of AICSImageIO in the community!

Experimental CZI-to-OME Metadata

For those following our experiments into "language agnostic methods for metadata translation and access" we have added experimental support for translating CZI metadata into OME metadata.

from aicsimageio import AICSImage

img = AICSImage("my-file.czi")
img.ome_metadata

You can find a comparison between the produced metadata from our XSLT and the produced metadata from Bio-Formats here. This is experimental, it is missing some metadata translations and may raise errors, however if you would like to try out this feature (or the XSLT itself in another language) please let us know how it goes.

Contributors and Reviewers this Release (alphabetical)

Matte Bailey (@MatteBailey)
Sébastien Besson (@sbesson)
Jackson Maxfield Brown (@JacksonMaxfield)
Talley Lambert (@tlambert03)
Josh Moore (@joshmoore)
Madison Swain-Bowden (@AetherUnbound)
Dan Toloudis (@toloudis)