这是一篇论文 "MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts" 的相关介绍。 该论文发布于MICCAI 2022。
MUSCLE的主标是通过预训练一个主干网络,来提高深度学习在医学影像分析任务中的性能。
该论文的所有代码均使用PaddlePaddle框架实现。
MUSCLE聚合了从不同人体部位收集的多个Xray图像数据集,并作用于各种Xray影像的分析任务。 我们提出了多数据集动量对比表征学习(MD-MoCo)模块和多任务持续学习模块, 以自我监督的持续学习方式对深度学习框架的主干网络进行预训练。 预训练的模型可以使用特定任务的head对目标任务进行微调,并取得极佳的性能。
Datasets | Body Part | Task | Train | Valid | Test | Total |
---|---|---|---|---|---|---|
Only Used for the first step (MD-MoCo) of MUSCLE | ||||||
NIHCC | Chest | N/A | 112,120 | N/A | N/A | 112,120 |
China-Set-CXR | Chest | N/A | 661 | N/A | N/A | 661 |
Montgomery-Set-CXR | Chest | N/A | 138 | N/A | N/A | 138 |
Indiana-CXR | Chest | N/A | 7,470 | N/A | N/A | 7,470 |
RSNA Bone Age | Hand | N/A | 10,811 | N/A | N/A | 10,811 |
Used for all three steps of MUSCLE | ||||||
Pneumonia | Chest | Classification | 4,686 | 585 | 585 | 5,856 |
MURA | Various Bone | Classification | 32,013 | 3,997 | 3,997 | 40,005 |
Chest Xray Masks and labels | Chest | Segmentation | 718 | 89 | 89 | 896 |
TBX | Chest | Detection | 640 | 80 | 80 | 800 |
Total | N/A | N/A | 169,257 | 4,751 | 4,479 | 178,757 |
- 主干网络
- ResNet-18、 ResNet-50
- 医学影像分析任务
- 肺炎分类任务 (Pneumonia),
- 骨骼异常分类任务 (MURA)
- 肺部分割任务 (Lung)
- 结核病Bounding Box检测 (TBX)
- Head网络
- 分类任务:Fully-Connected (FC) Layer
- 分割任务:DeepLab-V3
- 检测任务:FasterRCNN
- 基线的预训练算法
- Scratch: 模型主干网络使用Kaiming’s initialization进行参数初始化
- ImageNet: 模型主干网络使用官方发布的ImageNet进行参数初始化
- MD-MoCo: 模型主干网络只使用在多数据源的Xray图像进行MoCo学习的参数进行初始化
- MUSCLE−−: 模型的初始化策略和MUSCLE一致,但是不采用我们设计的跨任务记忆与循环和重组学习计划模块
注意:Pneumonia是由胸片图像构成的数据集,而MURA由骨骼图像构成
Datasets | Backbones | Pre-train | Acc. | Sen. | Spe. | AUC(95%CI) |
---|---|---|---|---|---|---|
Pneumonia | ResNet-18 | Scratch | 91.11 | 93.91 | 83.54 | 96.58(95.09-97.81) |
ImageNet | 90.09 | 93.68 | 80.38 | 96.05(94.24-97.33) | ||
MD-MoCo | 96.58 | 97.19 | 94.94 | 98.48(97.14-99.30) | ||
MUSCLE-- | 96.75 | 97.66 | 94.30 | 99.51(99.16-99.77) | ||
MUSCLE | 97.26 | 97.42 | 96.84 | 99.61(99.32-99.83) | ||
ResNet-50 | Scratch | 91.45 | 92.51 | 88.61 | 96.55(95.08-97.82) | |
ImageNet | 95.38 | 95.78 | 94.30 | 98.72(98.03-99.33) | ||
MD-MoCo | 97.09 | 98.83 | 92.41 | 99.53(99.23-99.75) | ||
MUSCLE-- | 96.75 | 98.36 | 92.41 | 99.58(99.30-99.84) | ||
MUSCLE | 98.12 | 98.36 | 97.47 | 99.72(99.46-99.92) | ||
MURA | ResNet-18 | Scratch | 81.00 | 68.17 | 89.91 | 86.62(85.73-87.55) |
ImageNet | 81.88 | 73.49 | 87.70 | 88.11(87.18-89.03) | ||
MD-MoCo | 82.48 | 72.27 | 89,57 | 88.28(87.28-89.26) | ||
MUSCLE-- | 82.45 | 74.16 | 88.21 | 88.41(87.54-89.26) | ||
MUSCLE | 82.62 | 74.28 | 88.42 | 88.5o(87.46-89.57) | ||
RcsNet-50 | Scratch | 80.50 | 65.42 | 90.97 | 86.22(85.22-87.35) | |
ImngeNet | 81.73 | 68.36 | 91.01 | 87.87(86.85-88.85) | ||
MD-MoCo | 82.35 | 73.12 | 88.76 | 87.89(87.06-88.88) | ||
MUSCLE-- | 81.10 | 69.03 | 89.48 | 87.14(86.10-88.22) | ||
MUSCLE | 82.60 | 74.53 | 88.21 | 88.37(87.38-89.32) |
注意:Lung为肺部分割任务,而TBX为检测任务
Backbones | Pre-train | Lung | TBX | |||
---|---|---|---|---|---|---|
Dice | mloU | mAP | AP-Active | AP-Latent | ||
ResNet-18 | Scratch | 95.24 | 94.00 | 30.71 | 56.71 | 4.72 |
ImageNet | 95.26 | 94.10 | 29.46 | 56.27 | 2.66 | |
MD-MoCo | 95.31 | 94.14 | 36.00 | 67.17 | 4.84 | |
MUSCLE-- | 95.14 | 93.90 | 34.70 | 63.43 | 5.97 | |
MUSCLE | 95.37 | 94.22 | 36.71 | 64.84 | 8.59 | |
ResNet-50 |
Scratch | 93.52 | 92.03 | 23.93 | 44.85 | 3.01 |
ImageNet | 93.77 | 92.43 | 35.61 | 58.81 | 12.42 | |
MD-MoCo | 94.33 | 93.04 | 36.78 | 64.37 | 9.18 | |
MUSCLE-- | 95.04 | 93.82 | 35.14 | 57.32 | 12.97 | |
MUSCLE | 95.27 | 94.10 | 37.83 | 63.46 | 12.21 |
如果我们的项目在学术上帮助到你,请考虑以下引用:
@inproceedings{liao2022muscle,
title={MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts},
author={Weibin, Liao and Haoyi, Xiong and Qingzhong, Wang and Yan, Mo and Xuhong, Li and Yi, Liu and Zeyu, Chen and Siyu, Huang and Dejing, Dou},
booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
year={2022},
organization={Springer}
}