Skip to content

Commit

Permalink
STYLE: Apply pre-commit linters
Browse files Browse the repository at this point in the history
  • Loading branch information
thewtex committed Jul 22, 2024
1 parent 35ee28c commit ef65ae8
Show file tree
Hide file tree
Showing 20 changed files with 514 additions and 296 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/notebook-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ jobs:

- name: Test notebooks
run: |
pixi run test-notebooks
pixi run test-notebooks
6 changes: 3 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Test

on: [push,pull_request]
on: [push, pull_request]

jobs:
test:
Expand All @@ -9,7 +9,7 @@ jobs:
max-parallel: 5
matrix:
os: [ubuntu-22.04, windows-2022, macos-12, macos-14]
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4
Expand All @@ -30,4 +30,4 @@ jobs:
if: ${{ matrix.os != 'mac-14' && matrix.package == '3.8' }}
uses: mikepenz/action-junit-report@v2
with:
report_paths: 'junit/test-results*.xml'
report_paths: "junit/test-results*.xml"
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ repos:
rev: "v2.2.5"
hooks:
- id: codespell
exclude: examples/

- repo: https://github.com/shellcheck-py/shellcheck-py
rev: "v0.9.0.5"
Expand Down
40 changes: 24 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@
[![image](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
[![DOI](https://zenodo.org/badge/379678181.svg)](https://zenodo.org/badge/latestdoi/379678181)

Generate a multiscale, chunked, multi-dimensional spatial image data structure that can serialized to [OME-NGFF].

Each scale is a scientific Python [Xarray] [spatial-image] [Dataset], organized into nodes of an Xarray [Datatree].
Generate a multiscale, chunked, multi-dimensional spatial image data structure
that can serialized to [OME-NGFF].

Each scale is a scientific Python [Xarray] [spatial-image] [Dataset], organized
into nodes of an Xarray [Datatree].

## Installation

Expand All @@ -32,8 +33,8 @@ image = to_spatial_image(array)
print(image)
```

An [Xarray] [spatial-image] [DataArray].
Spatial metadata can also be passed during construction.
An [Xarray] [spatial-image] [DataArray]. Spatial metadata can also be passed
during construction.

```
<xarray.SpatialImage 'image' (y: 128, x: 128)>
Expand Down Expand Up @@ -82,9 +83,11 @@ DataTree('multiscales', parent=None)
image (y, x) uint8 dask.array<chunksize=(16, 16), meta=np.ndarray>
```

Store as an Open Microscopy Environment-Next Generation File Format ([OME-NGFF]) / [netCDF] [Zarr] store.
Store as an Open Microscopy Environment-Next Generation File Format ([OME-NGFF])
/ [netCDF] [Zarr] store.

It is highly recommended to use `dimension_separator='/'` in the construction of the Zarr stores.
It is highly recommended to use `dimension_separator='/'` in the construction of
the Zarr stores.

```python
store = zarr.storage.DirectoryStore('multiscale.zarr', dimension_separator='/')
Expand All @@ -96,10 +99,14 @@ released. We mean it :-).

## Examples

- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FHelloMultiscaleSpatialImageWorld.ipynb) [Hello MultiscaleSpatialImage World!](./examples/HelloMultiscaleSpatialImageWorld.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FConvertITKImage.ipynb) [Convert itk.Image](./examples/ConvertITKImage.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FConvertImageioImageResource.ipynb) [Convert imageio ImageResource](./examples/ConvertImageioImageResource.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FConvertPyImageJDataset.ipynb) [Convert pyimagej Dataset](./examples/ConvertPyImageJDataset.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FHelloMultiscaleSpatialImageWorld.ipynb)
[Hello MultiscaleSpatialImage World!](./examples/HelloMultiscaleSpatialImageWorld.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FConvertITKImage.ipynb)
[Convert itk.Image](./examples/ConvertITKImage.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FConvertImageioImageResource.ipynb)
[Convert imageio ImageResource](./examples/ConvertImageioImageResource.ipynb)
- [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/spatial-image/multiscale-spatial-image/main?urlpath=lab/tree/examples%2FConvertPyImageJDataset.ipynb)
[Convert pyimagej Dataset](./examples/ConvertPyImageJDataset.ipynb)

## Development

Expand Down Expand Up @@ -171,15 +178,16 @@ gzip -9 ../data.tar
python3 -c 'import pooch; print(pooch.file_hash("../data.tar.gz"))'
```

Update the `test_data_sha256` variable in the *test/_data.py* file.
Upload the data to [web3.storage](https://web3.storage).
And update the `test_data_ipfs_cid` [Content Identifier (CID)](https://proto.school/anatomy-of-a-cid/01) variable, which is available in the web3.storage web page interface.
Update the `test_data_sha256` variable in the _test/\_data.py_ file. Upload the
data to [web3.storage](https://web3.storage). And update the
`test_data_ipfs_cid`
[Content Identifier (CID)](https://proto.school/anatomy-of-a-cid/01) variable,
which is available in the web3.storage web page interface.

### Submit the patch

We use the standard [GitHub flow].


[spatial-image]: https://github.com/spatial-image/spatial-image
[Xarray]: https://xarray.pydata.org/en/stable/
[OME-NGFF]: https://ngff.openmicroscopy.org/
Expand All @@ -190,4 +198,4 @@ We use the standard [GitHub flow].
[Dask]: https://docs.dask.org/en/stable/array.html
[netCDF]: https://www.unidata.ucar.edu/software/netcdf/
[pixi]: https://pixi.sh
[GitHub flow]: https://docs.github.com/en/get-started/using-github/github-flow
[GitHub flow]: https://docs.github.com/en/get-started/using-github/github-flow
2 changes: 1 addition & 1 deletion multiscale_spatial_image/__about__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: 2022-present NumFOCUS <[email protected]>
#
# SPDX-License-Identifier: MIT
__version__ = '1.0.0'
__version__ = "1.0.0"
13 changes: 6 additions & 7 deletions multiscale_spatial_image/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@
Generate a multiscale spatial image."""


__all__ = [
"MultiscaleSpatialImage",
"Methods",
"to_multiscale",
"itk_image_to_multiscale",
"__version__",
"MultiscaleSpatialImage",
"Methods",
"to_multiscale",
"itk_image_to_multiscale",
"__version__",
]

from .__about__ import __version__
from .multiscale_spatial_image import MultiscaleSpatialImage
from .to_multiscale import Methods, to_multiscale, itk_image_to_multiscale
from .to_multiscale import Methods, to_multiscale, itk_image_to_multiscale
5 changes: 1 addition & 4 deletions multiscale_spatial_image/multiscale_spatial_image.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
from typing import Union, List
from typing import Union

import xarray as xr
from datatree import DataTree
from datatree.treenode import TreeNode
import numpy as np
from collections.abc import MutableMapping
from pathlib import Path
from zarr.storage import BaseStore
import xarray as xr
from datatree import register_datatree_accessor


Expand Down
4 changes: 3 additions & 1 deletion multiscale_spatial_image/to_multiscale/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
from .to_multiscale import Methods, to_multiscale
from .to_multiscale import Methods, to_multiscale
from .itk_image_to_multiscale import itk_image_to_multiscale

__all__ = ["Methods", "to_multiscale", "itk_image_to_multiscale"]
111 changes: 70 additions & 41 deletions multiscale_spatial_image/to_multiscale/_dask_image.py
Original file line number Diff line number Diff line change
@@ -1,25 +1,28 @@
from spatial_image import to_spatial_image
from dask.array import map_blocks, map_overlap
import numpy as np

from ._support import _align_chunks, _dim_scale_factors, _compute_sigma


def _compute_input_spacing(input_image):
'''Helper method to manually compute image spacing. Assumes even spacing along any axis.
"""Helper method to manually compute image spacing. Assumes even spacing along any axis.
input_image: xarray.core.dataarray.DataArray
The image for which voxel spacings are computed
result: Dict
Spacing along each enumerated image axis
Example {'x': 1.0, 'y': 0.5}
'''
return {dim: float(input_image.coords[dim][1]) - float(input_image.coords[dim][0])
for dim in input_image.dims}
"""
return {
dim: float(input_image.coords[dim][1]) - float(input_image.coords[dim][0])
for dim in input_image.dims
}


def _compute_output_spacing(input_image, dim_factors):
'''Helper method to manually compute output image spacing.
"""Helper method to manually compute output image spacing.
input_image: xarray.core.dataarray.DataArray
The image for which voxel spacings are computed
Expand All @@ -29,14 +32,15 @@ def _compute_output_spacing(input_image, dim_factors):
result: Dict
Spacing along each enumerated image axis
Example {'x': 2.0, 'y': 1.0}
'''
"""
input_spacing = _compute_input_spacing(input_image)
return {dim: input_spacing[dim] * dim_factors[dim] for dim in input_image.dims}

def _compute_output_origin(input_image, dim_factors):
'''Helper method to manually compute output image physical offset.

def _compute_output_origin(input_image, dim_factors):
"""Helper method to manually compute output image physical offset.
Note that this method does not account for an image direction matrix.
input_image: xarray.core.dataarray.DataArray
The image for which voxel spacings are computed
Expand All @@ -46,23 +50,32 @@ def _compute_output_origin(input_image, dim_factors):
result: Dict
Offset in physical space of first voxel in output image
Example {'x': 0.5, 'y': 1.0}
'''
import math

"""

input_spacing = _compute_input_spacing(input_image)
input_origin = {dim: float(input_image.coords[dim][0])
for dim in input_image.dims if dim in dim_factors}
input_origin = {
dim: float(input_image.coords[dim][0])
for dim in input_image.dims
if dim in dim_factors
}

# Index in input image space corresponding to offset after shrink
input_index = {dim: 0.5 * (dim_factors[dim] - 1)
for dim in input_image.dims if dim in dim_factors}
input_index = {
dim: 0.5 * (dim_factors[dim] - 1)
for dim in input_image.dims
if dim in dim_factors
}
# Translate input index coordinate to offset in physical space
# NOTE: This method fails to account for direction matrix
return {dim: input_index[dim] * input_spacing[dim] + input_origin[dim]
for dim in input_image.dims if dim in dim_factors}
return {
dim: input_index[dim] * input_spacing[dim] + input_origin[dim]
for dim in input_image.dims
if dim in dim_factors
}


def _get_truncate(xarray_image, sigma_values, truncate_start=4.0) -> float:
'''Discover truncate parameter yielding a viable kernel width
"""Discover truncate parameter yielding a viable kernel width
for dask_image.ndfilters.gaussian_filter processing. Block overlap
cannot be greater than image size, so kernel radius is more limited
for small images. A lower stddev truncation ceiling for kernel
Expand All @@ -81,23 +94,37 @@ def _get_truncate(xarray_image, sigma_values, truncate_start=4.0) -> float:
result: float
Truncation value found to yield largest possible kernel width without
extending beyond one chunk such that chunked smoothing would fail.
'''
"""

from dask_image.ndfilters._gaussian import _get_border

truncate = truncate_start
stddev_step = 0.5 # search by stepping down by 0.5 stddev in each iteration

border = _get_border(xarray_image.data, sigma_values, truncate)
while any([border_len > image_len for border_len, image_len in zip(border, xarray_image.shape)]):
while any(
[
border_len > image_len
for border_len, image_len in zip(border, xarray_image.shape)
]
):
truncate = truncate - stddev_step
if(truncate <= 0.0):
break
if truncate <= 0.0:
break
border = _get_border(xarray_image.data, sigma_values, truncate)

return truncate

def _downsample_dask_image(current_input, default_chunks, out_chunks, scale_factors, data_objects, image, label=False):

def _downsample_dask_image(
current_input,
default_chunks,
out_chunks,
scale_factors,
data_objects,
image,
label=False,
):
import dask_image.ndfilters
import dask_image.ndinterp

Expand All @@ -116,24 +143,28 @@ def _downsample_dask_image(current_input, default_chunks, out_chunks, scale_fact
input_spacing = _compute_input_spacing(current_input)

# Compute output shape and metadata
output_shape = [int(image_len / shrink_factor)
for image_len, shrink_factor in zip(current_input.shape, shrink_factors)]
output_shape = [
int(image_len / shrink_factor)
for image_len, shrink_factor in zip(current_input.shape, shrink_factors)
]
output_spacing = _compute_output_spacing(current_input, dim_factors)
output_origin = _compute_output_origin(current_input, dim_factors)

if label == 'mode':
if label == "mode":

def largest_mode(arr):
values, counts = np.unique(arr, return_counts=True)
m = counts.argmax()
return values[m]

size = tuple(shrink_factors)
blurred_array = dask_image.ndfilters.generic_filter(
image=current_input.data,
function=largest_mode,
size=size,
mode='nearest',
mode="nearest",
)
elif label == 'nearest':
elif label == "nearest":
blurred_array = current_input.data
else:
input_spacing_list = [input_spacing[dim] for dim in image.dims]
Expand All @@ -142,16 +173,16 @@ def largest_mode(arr):

blurred_array = dask_image.ndfilters.gaussian_filter(
image=current_input.data,
sigma=sigma_values, # tzyx order
mode='nearest',
truncate=truncate
sigma=sigma_values, # tzyx order
mode="nearest",
truncate=truncate,
)

# Construct downsample parameters
image_dimension = len(dim_factors)
transform = np.eye(image_dimension)
for dim, shrink_factor in enumerate(shrink_factors):
transform[dim,dim] = shrink_factor
transform[dim, dim] = shrink_factor
if label:
order = 0
else:
Expand All @@ -161,9 +192,9 @@ def largest_mode(arr):
blurred_array,
matrix=transform,
order=order,
output_shape=output_shape # tzyx order
output_shape=output_shape, # tzyx order
).compute()

downscaled = to_spatial_image(
downscaled_array,
dims=image.dims,
Expand All @@ -173,9 +204,7 @@ def largest_mode(arr):
axis_names={
d: image.coords[d].attrs.get("long_name", d) for d in image.dims
},
axis_units={
d: image.coords[d].attrs.get("units", "") for d in image.dims
},
axis_units={d: image.coords[d].attrs.get("units", "") for d in image.dims},
t_coords=image.coords.get("t", None),
c_coords=image.coords.get("c", None),
)
Expand Down
Loading

0 comments on commit ef65ae8

Please sign in to comment.