Computing statistics of clusters

[mchoblet]

First of all thank you for creating this super useful, straight-forward to use package.
I see loads of applications in the Geosciences where I work in, for instance to cluster extreme events
I have just started using the package, my current use case is the clustering of storms after computing if my data (lat,lon,time) is above a threshold (so basically I just have a huge matrix of ones and zeros).

I have a couple of questions regarding the cc3d.statistics function, which I thought of adapting, but as I have no experience with cython/the algos behind the labelling I prefer to ask before doing anything:

• The statistics function is currently limited in one dimension by uint16 ("Only dimensions shorter than 65536 are supported."). Is there a specific reason for this?. The time dimension I use can be longer than that.
• Is there a straight forward way to add storing the original label to cc3d.statistics? (in my case I'm only interested in the areas with a 1 for instance).
• I realized when computing some own statistics in python with the cc3d.each method, it makes a huge difference in speed if i use the computed bounding boxes to reduce the size of the matrix from the start, anything more like that to take into account? (this is more a numpy related question, I know)

Thanks!

[william-silversmith]

Hi! Thanks for writing in.

The statistics function is currently limited in one dimension by uint16 ("Only dimensions shorter than 65536 are supported."). Is there a specific reason for this?. The time dimension I use can be longer than that.

The reason for this was I was attempting to be conservative with memory. The bounding boxes are 6 * (N + 1) * data_width, which can become pretty large in the worst case (e.g. a volume where all the labels are different). Maybe I can automatically select uint32 when the dimensions are larger than uint16.

Is there a straight forward way to add storing the original label to cc3d.statistics? (in my case I'm only interested in the areas with a 1 for instance).

In your case, 0 is treated as background during CCL computation, so only the 0th index in statistics is background. All other entries are foreground. If for some reason you didn't want this behavior, I'd recommend adding 1 to the volume before putting it into ccl and then subtracting 1 from the results after.

I realized when computing some own statistics in python with the cc3d.each method, it makes a huge difference in speed if i use the computed bounding boxes to reduce the size of the matrix from the start, anything more like that to take into account? (this is more a numpy related question, I know)

That's the main trick. It might be interesting for me to integrate that into cc3d.each for convenience's sake.

Will

[mchoblet]

Hi Will,
Thanks you very much for the quick reply and adjusting the code directly!
I somehow missed out the fact that 0 is treated as background. That is perfect solution for my case, no need for anything more.
Cheers, Mathurin

[william-silversmith]

cc3d 3.12.5 now supports uint32 sized axes in cc3d.statistics