Skip to content

Latest commit

 

History

History
113 lines (89 loc) · 19.4 KB

MODEL_ZOO.md

File metadata and controls

113 lines (89 loc) · 19.4 KB
Raw

Model Zoo

Note

  • All the config.yaml in our exp are NOT the training config actually used, since some hyperparameters are changed in the run.sh or test.sh.
  • #Frame = #input_frame x #crop x #clip
  • #input_frame means how many frames are input for model per inference
  • #crop means spatial crops (e.g., 3 for left/right/center)
  • #clip means temporal clips (e.g., 4 means repeted sampling four clips with different start indices)

K710

Model Frame Model Shell Config
UniFormerV2-B/16 8 ckpt run.sh config.yaml
UniFormerV2-L/14 8 ckpt run.sh config.yaml
UniFormerV2-L/14@336 8 ckpt run.sh config.yaml
Frozen UniFormerV2-L/14@336 8 ckpt run.sh config.yaml

K400

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-B/16 CLIP-400M 8x3x4 84.4 ckpt run.sh config.yaml
UniFormerV2-B/16 CLIP-400M+K710 8x3x4 85.6 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 8x3x4 88.8 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 16x3x4 89.1 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 32x3x2 89.3 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 32x3x2 89.7 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 64x3x2 90.0 ckpt run.sh config.yaml
Frozen Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-L/14@336 CLIP-400M 8x1x3 86.7 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 8x1x3 87.8 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 32x3x4 88.9 ckpt run.sh config.yaml

K600

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-B/16 CLIP-400M 8x3x4 85.0 ckpt run.sh config.yaml
UniFormerV2-B/16 CLIP-400M+K710 8x3x4 86.1 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 8x3x4 89.0 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 16x3x4 89.4 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 32x3x2 89.5 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 32x3x2 89.9 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 64x3x2 90.1 ckpt run.sh config.yaml
Frozen Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-L/14@336 CLIP-400M 8x1x3 87.4 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 8x1x3 88.2 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 32x3x4 89.2 ckpt run.sh config.yaml

K700

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-B/16 CLIP-400M 8x3x4 75.8 ckpt run.sh config.yaml
UniFormerV2-B/16 CLIP-400M+K710 8x3x4 76.3 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 8x3x4 80.8 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 16x3x4 81.2 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710 32x3x2 81.5 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 32x3x2 82.1 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 64x3x2 82.7 ckpt run.sh config.yaml
Frozen Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-L/14@336 CLIP-400M 8x1x3 79.6 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 8x1x3 79.7 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710 32x3x4 80.8 ckpt run.sh config.yaml

Moments in Time V1

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-B/16 CLIP-400M+K710+K400 8x3x4 42.6 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710+K400 8x3x4 47.0 ckpt run.sh config.yaml
UniFormerV2-L/14@336 CLIP-400M+K710+K400 8x3x4 47.8 ckpt run.sh config.yaml

Something-Something V1

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-B/16 CLIP-400M 16x3x1 56.8 ckpt run.sh config.yaml
UniFormerV2-B/16 CLIP-400M 32x3x1 59.4 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M 16x3x1 60.5 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M 32x3x1 62.7 ckpt run.sh config.yaml

Something-Something V2

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-B/16 CLIP-400M 16x3x1 69.5 ckpt run.sh config.yaml
UniFormerV2-B/16 CLIP-400M 32x3x1 70.7 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M 16x3x1 72.1 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M 32x3x1 73.0 ckpt run.sh config.yaml

ActivityNet

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-L/14 CLIP-400M+K710+K400 16x3x10 94.3 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710+K400 32x3x10 94.7 ckpt run.sh config.yaml

HACS

Model Pretraining #Frame Top-1 Model Shell Config
UniFormerV2-L/14 CLIP-400M+K710+K400 16x3x10 95.5 ckpt run.sh config.yaml
UniFormerV2-L/14 CLIP-400M+K710+K400 32x3x10 95.4 ckpt run.sh config.yaml