语言:
🇨🇳
🇺🇸
«KnowledgeReview»复现了论文Distilling Knowledge via Knowledge Review
| arch_s | top1 | top5 | arch_t | top1 | top5 | dataset | lambda | top1 | top5 |
|---|---|---|---|---|---|---|---|---|---|
| MobileNetv2 | 80.620 | 95.820 | ResNet50 | 83.540 | 96.820 | CIFAR100 | 7.0 | 83.370 | 96.810 |
| MobileNetv2 | 80.620 | 95.820 | ResNet152 | 85.490 | 97.590 | CIFAR100 | 8.0 | 84.530 | 97.470 |
| MobileNetv2 | 80.620 | 95.820 | ResNeXt_32x8d | 85.720 | 97.650 | CIFAR100 | 6.0 | 84.520 | 97.470 |
| ResNet18 | 80.540 | 96.040 | ResNet50 | 83.540 | 96.820 | CIFAR100 | 10.0 | 83.130 | 96.350 |
| ResNet50 | 83.540 | 96.820 | ResNet152 | 85.490 | 97.590 | CIFAR100 | 6.0 | 86.240 | 97.610 |
| ResNet50 | 83.540 | 96.820 | ResNeXt_32x8d | 85.720 | 97.650 | CIFAR100 | 6.0 | 86.220 | 97.490 |
更多内容参见docs
和之前的知识迁移算法不同，RFD使用了跨阶段的教师特征来训练学生特征。同时，它还设计了一个新的残差学习框架用于简化学生特征转换操作，以及设计了ABF（基于融合注意力）模块和HCL（分层内容损失）函数来辅助特征蒸馏训练。
当前实现基于 ZJCV/overhaul 和 dvlab-research/ReviewKD。
$ pip install -r requirements.txt
- 训练
$ CUDA_VISIBLE_DEVICES=0 python tools/train.py -cfg=configs/rfd/resnet/rfd_6_0_r152_pret_r50_c100_224_e100_sgd_mslr.yaml
- 测试
$ CUDA_VISIBLE_DEVICES=0 python tools/test.py -cfg=configs/rfd/resnet/rfd_6_0_r152_pret_r50_c100_224_e100_sgd_mslr.yaml
- zhujian - Initial work - zjykzj
@misc{chen2021distilling,
title={Distilling Knowledge via Knowledge Review},
author={Pengguang Chen and Shu Liu and Hengshuang Zhao and Jiaya Jia},
year={2021},
eprint={2104.09044},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
欢迎任何人的参与！打开issue或提交合并请求。
注意:
GIT提交，请遵守Conventional Commits规范- 语义版本化，请遵守Semantic Versioning 2.0.0规范
README编写，请遵守standard-readme规范
Apache License 2.0 © 2021 zjykzj