Adding Functionalities for sequential scans

[pkgorde]

Hi Community!

I am working on a research project, where I am developing an entire pipeline for deconstructing DICOM files, which have images sequences of medical scans in them, and delivering a prepared workload at the end, readily usable for ML models which can deal with image sequences or videos.
Since the data I am inherently working on is medical scans, I wanted to know if a functionality like that would be a welcome part of the MONAI codebase. And if yes, are there some guidelines/practices that I should follow while developing this sub-package?
I am primarily using Numpy and OpenCV for the implementation.

[KumoLiu]

Hi @pkgorde, thanks for your interest here.
If you would like do any contribution, you can refer to the CONTRIBUTION guide.
Looking forward to your contribution.

Thanks.

[aylward]

Hi,

I'm interested in the ML approach that you plan to follow/support. Unlike video, clinical data typically isn't acquired at the same interval for every patient or even at the same interval for the same patient, so you will need to encode a representation of time as a network input, in addition to the image data. Or does your problem involve image data acquired at fixed intervals (e.g., perfusion sequences) or perhaps image data acquired when some other clinical measure triggers them (e.g., they are indexed by changes in a lab value instead of being indexed by time or perhaps they are tied to cardiac or respiratory cycle)?

To get started, there are several outstanding toolkits for handling dicom data. Consider looking at pydicom, dcmqi, and ITK (my biased favorite). They will handle most dicom data well.

s

[pkgorde]

Hi, Thanks for the input! I am already using pydicom for handling the dcm files directly, and working with Numpy arrays from then on. The data I have in question is Angiogram scans (Radiology), which are a sequence of scans taken in the entire process. With that being said the number of scans in the data array generated (by pydicom) is the extra time dimension you mentioned

[aylward]

Nice!

It seems like you're perhaps looking at DSA (digital subtraction angiograms, i.e., 2D images rapidly acquired over time) as contrast is injected and/or a catheter is advanced. If so, then for certain analyses that type of data lends itself to being processed as video data. To get started, you might want to look at the video segmentation tutorial:
https://github.com/Project-MONAI/tutorials/tree/main/computer_assisted_intervention

Also, if you're interested in a discussion in making a general framework for sequential data (2D and 3D data over time, non-isotropic voxels in space and time, etc.), I'd be happy to help and others may have suggestions too.