Train a MULTI-CHANNEL model from scratch on dcm-zurich-lesions datasets using nnUNet

[valosekj]

This issue summarizes experiments related to Dataset602_DCMlesionsMultiChannel (multi-channel model trained using T2w_ax and SC seg as input channels to segment lesions).

dataset.json:

    "channel_names": {
        "0": "acq-ax_T2w",
        "1": "SC_seg"
    },
    "labels": {
        "background": 0,
        "lesion": 1
    },

nnUNetPlans.json

{
    "dataset_name": "Dataset602_DCMlesionsMultiChannel",
    "plans_name": "nnUNetPlans",
    "original_median_spacing_after_transp": [
        3.5999999046325684,
        0.5,
        0.5
    ],
    "original_median_shape_after_transp": [
        15,
        320,
        319
    ],
    "image_reader_writer": "SimpleITKIO",
    "transpose_forward": [
        0,
        1,
        2
    ],
    "transpose_backward": [
        0,
        1,
        2
    ],
    "configurations": {
        "2d": {
            "data_identifier": "nnUNetPlans_2d",
            "preprocessor_name": "DefaultPreprocessor",
            "batch_size": 31,
            "patch_size": [
                320,
                320
            ],
            "median_image_size_in_voxels": [
                320.0,
                319.0
            ],
            "spacing": [
                0.5,
                0.5
            ],
            "normalization_schemes": [
                "ZScoreNormalization",
                "ZScoreNormalization"
            ],
            "use_mask_for_norm": [
                false,
                false
            ],
            "UNet_class_name": "PlainConvUNet",
            "UNet_base_num_features": 32,
            "n_conv_per_stage_encoder": [
                2,
                2,
                2,
                2,
                2,
                2,
                2
            ],
            "n_conv_per_stage_decoder": [
                2,
                2,
                2,
                2,
                2,
                2
            ],
            "num_pool_per_axis": [
                6,
                6
            ],
            "pool_op_kernel_sizes": [
                [
                    1,
                    1
                ],
                [
                    2,
                    2
                ],
                [
                    2,
                    2
                ],
                [
                    2,
                    2
                ],
                [
                    2,
                    2
                ],
                [
                    2,
                    2
                ],
                [
                    2,
                    2
                ]
            ],
            "conv_kernel_sizes": [
                [
                    3,
                    3
                ],
                [
                    3,
                    3
                ],
                [
                    3,
                    3
                ],
                [
                    3,
                    3
                ],
                [
                    3,
                    3
                ],
                [
                    3,
                    3
                ],
                [
                    3,
                    3
                ]
            ],
            "unet_max_num_features": 512,
            "resampling_fn_data": "resample_data_or_seg_to_shape",
            "resampling_fn_seg": "resample_data_or_seg_to_shape",
            "resampling_fn_data_kwargs": {
                "is_seg": false,
                "order": 3,
                "order_z": 0,
                "force_separate_z": null
            },
            "resampling_fn_seg_kwargs": {
                "is_seg": true,
                "order": 1,
                "order_z": 0,
                "force_separate_z": null
            },
            "resampling_fn_probabilities": "resample_data_or_seg_to_shape",
            "resampling_fn_probabilities_kwargs": {
                "is_seg": false,
                "order": 1,
                "order_z": 0,
                "force_separate_z": null
            },
            "batch_dice": true
        },
        "3d_fullres": {
            "data_identifier": "nnUNetPlans_3d_fullres",
            "preprocessor_name": "DefaultPreprocessor",
            "batch_size": 2,
            "patch_size": [
                16,
                320,
                320
            ],
            "median_image_size_in_voxels": [
                15.0,
                320.0,
                319.0
            ],
            "spacing": [
                3.5999979972839355,
                0.5,
                0.5
            ],
            "normalization_schemes": [
                "ZScoreNormalization",
                "ZScoreNormalization"
            ],
            "use_mask_for_norm": [
                false,
                false
            ],
            "UNet_class_name": "PlainConvUNet",
            "UNet_base_num_features": 32,
            "n_conv_per_stage_encoder": [
                2,
                2,
                2,
                2,
                2,
                2,
                2
            ],
            "n_conv_per_stage_decoder": [
                2,
                2,
                2,
                2,
                2,
                2
            ],
            "num_pool_per_axis": [
                2,
                6,
                6
            ],
            "pool_op_kernel_sizes": [
                [
                    1,
                    1,
                    1
                ],
                [
                    1,
                    2,
                    2
                ],
                [
                    1,
                    2,
                    2
                ],
                [
                    2,
                    2,
                    2
                ],
                [
                    2,
                    2,
                    2
                ],
                [
                    1,
                    2,
                    2
                ],
                [
                    1,
                    2,
                    2
                ]
            ],
            "conv_kernel_sizes": [
                [
                    1,
                    3,
                    3
                ],
                [
                    1,
                    3,
                    3
                ],
                [
                    3,
                    3,
                    3
                ],
                [
                    3,
                    3,
                    3
                ],
                [
                    3,
                    3,
                    3
                ],
                [
                    3,
                    3,
                    3
                ],
                [
                    3,
                    3,
                    3
                ]
            ],
            "unet_max_num_features": 320,
            "resampling_fn_data": "resample_data_or_seg_to_shape",
            "resampling_fn_seg": "resample_data_or_seg_to_shape",
            "resampling_fn_data_kwargs": {
                "is_seg": false,
                "order": 3,
                "order_z": 0,
                "force_separate_z": null
            },
            "resampling_fn_seg_kwargs": {
                "is_seg": true,
                "order": 1,
                "order_z": 0,
                "force_separate_z": null
            },
            "resampling_fn_probabilities": "resample_data_or_seg_to_shape",
            "resampling_fn_probabilities_kwargs": {
                "is_seg": false,
                "order": 1,
                "order_z": 0,
                "force_separate_z": null
            },
            "batch_dice": false
        }
    },
    "experiment_planner_used": "ExperimentPlanner",
    "label_manager": "LabelManager",
    "foreground_intensity_properties_per_channel": {
        "0": {
            "max": 2102.0,
            "mean": 575.5780639648438,
            "median": 516.0,
            "min": 157.0,
            "percentile_00_5": 241.0,
            "percentile_99_5": 1752.0,
            "std": 235.12808227539062
        },
        "1": {
            "max": 1.0,
            "mean": 1.0,
            "median": 1.0,
            "min": 1.0,
            "percentile_00_5": 1.0,
            "percentile_99_5": 1.0,
            "std": 0.0
        }
    }
}

Dataset602_DCMlesionsMultiChannel will be trained on dcm-zurich-lesions and dcm-zurich-lesions-20231115 datasets using nnUNetv2.

Manual lesion GTs are available for both datasets.

TODO

• convert both datasets from BIDS to nnUNet using dataset_conversion/convert_bids_to_nnUNetv2_multi-channel.py
• run training

[valosekj]

It seems that the model is learning nothing for both fold_0 and fold_1:

fold 0 -- learning nothing -- killed after 500 epochs

fold 1 -- learning nothing -- killed after 500 epochs

[valosekj]

Potential future idea for another MULTI-CHANNEL model:

    "channel_names": {
        "0": "acq-ax_T2w",
        "1": "compression_label"
    },

But we currently do not have the compression labels for dcm-zurich-lesions-20231115.

[valosekj]

Since the multi-channel model with the default nnUNetTrainer trainer was learning nothing (see my comment above), I tried the nnUNetTrainerDiceCELoss_noSmooth trainer (i.e., without the smoothing term of the Dice loss) as done for the region-based model (see here).
The training with the nnUNetTrainerDiceCELoss_noSmooth trainer finished successfully 🎉

fold 0, seed42

fold 1, seed42

[valosekj]

Training the multi-channel nnUNetTrainerDiceCELoss_noSmooth model for seed710 now (model name: Dataset603_DCMlesionsMultiChannelSeed710).

[valosekj]

nnUNetTrainerDiceCELoss_noSmooth__nnUNetPlans__3d_fullres/fold_1

`dcm-zurich-lesions`

Test Phase Metrics [ANIMA], n=3:
	Jaccard --> Mean: 0.383, Std: 0.071
	Dice --> Mean: 0.549, Std: 0.077
	Sensitivity --> Mean: 0.390, Std: 0.079
	Specificity --> Mean: 1.000, Std: 0.000
	PPV --> Mean: 0.970, Std: 0.043
	NPV --> Mean: 1.000, Std: 0.000
	RelativeVolumeError --> Mean: -59.436, Std: 9.721
	HausdorffDistance --> Mean: 4.537, Std: 2.466
	ContourMeanDistance --> Mean: 1.064, Std: 0.241
	SurfaceDistance --> Mean: 0.181, Std: 0.193

`dcm-zurich-lesions-20231115`

Test Phase Metrics [ANIMA], n=8:
	Jaccard --> Mean: 0.437, Std: 0.174
	Dice --> Mean: 0.587, Std: 0.182
	Sensitivity --> Mean: 0.553, Std: 0.293
	Specificity --> Mean: 1.000, Std: 0.000
	PPV --> Mean: 0.806, Std: 0.128
	NPV --> Mean: 1.000, Std: 0.000
	RelativeVolumeError --> Mean: -23.972, Std: 53.596
	HausdorffDistance --> Mean: 5.372, Std: 5.410
	ContourMeanDistance --> Mean: 2.005, Std: 2.353
	SurfaceDistance --> Mean: 0.729, Std: 1.423