SKD

Self-supervised Knowledge Distillation for Few-shot Learning

Introduction

Name:	SKD
Embed.:	Conv64F/ResNet12/ResNet18
Type:	Fine-tuning
Venue:	arXiv'20
Codes:	SKD

• Due to lacking enough GPUs, considering SKD-Gen0 uses episodic tasks for val/test only, we believe that the best model chosen by 5-1 validation is almost the best model chosen by 5-5, so we run SKD-Gen0 once for 2 testing setting.

Cite this work with:

@article{DBLP:journals/corr/abs-2006-09785,
  author    = {Jathushan Rajasegaran and
               Salman Khan and
               Munawar Hayat and
               Fahad Shahbaz Khan and
               Mubarak Shah},
  title     = {Self-supervised Knowledge Distillation for Few-shot Learning},
  journal   = {CoRR},
  volume    = {abs/2006.09785},
  year      = {2020},
  url       = {https://arxiv.org/abs/2006.09785},
  archivePrefix = {arXiv},
  eprint    = {2006.09785}
}

---

Results and Models

Classification

	Embedding	📖 miniImageNet (5,1)	💻 miniImageNet (5,1)	📖miniImageNet (5,5)	💻 miniImageNet (5,5)	📝 Comments
1	Conv64F	-	48.14 ± 0.33 ⬇️ 📋	-	66.36 ± 0.29 ⬇️ 📋	SKD-Gen0-Table2
2	ResNet12	-	66.40 ± 0.36 ⬇️ 📋	-	83.06 ± 0.24⬇️ 📋	SKD-Gen0-Table2
3	ResNet12	-	67.35 ± 0.37 ⬇️ 📋	-	83.31 ± 0.24 ⬇️ 📋	SKD-Gen1-Table2
4	ResNet18	-	66.18 ± 0.37 ⬇️ 📋	-	82.21 ± 0.24⬇️ 📋	SKD-Gen0-Table2
5	ResNet18	-	66.70 ± 0.37 ⬇️ 📋	-	82.60 ± 0.24⬇️ 📋	SKD-Gen1-Table2

	Embedding	📖 tieredImageNet (5,1)	💻 tieredImageNet (5,1)	📖tieredImageNet (5,5)	💻 tieredImageNet (5,5)	📝 Comments
1	Conv64F	-	51.78 ± 0.36 ⬇️ 📋	-	70.65 ± 0.32 ⬇️ 📋	SKD-Gen0-Table2
2	ResNet12	-	71.90 ± 0.39 ⬇️ 📋	-	86.20 ± 0.27 ⬇️ 📋	SKD-Gen0-Table2
3	ResNet18	-	70.00 ± 0.57 ⬇️ 📋	-	84.70 ± 0.41 ⬇️ 📋	SKD-Gen0-Table2