Skip to content

Latest commit

 

History

History
89 lines (59 loc) · 5.61 KB

README.md

File metadata and controls

89 lines (59 loc) · 5.61 KB
Raw

EDVR (CVPRW'2019)

EDVR: Video Restoration with Enhanced Deformable Convolutional Networks

Task: Video Super-Resolution

Abstract

Video restoration tasks, including super-resolution, deblurring, etc, are drawing increasing attention in the computer vision community. A challenging benchmark named REDS is released in the NTIRE19 Challenge. This new benchmark challenges existing methods from two aspects: (1) how to align multiple frames given large motions, and (2) how to effectively fuse different frames with diverse motion and blur. In this work, we propose a novel Video Restoration framework with Enhanced Deformable networks, termed EDVR, to address these challenges. First, to handle large motions, we devise a Pyramid, Cascading and Deformable (PCD) alignment module, in which frame alignment is done at the feature level using deformable convolutions in a coarse-to-fine manner. Second, we propose a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration. Thanks to these modules, our EDVR wins the champions and outperforms the second place by a large margin in all four tracks in the NTIRE19 video restoration and enhancement challenges. EDVR also demonstrates superior performance to state-of-the-art published methods on video super-resolution and deblurring.

Results and models

Evaluated on RGB channels. The metrics are PSNR and SSIM .

Model Dataset PSNR SSIM Training Resources Download
edvrm_wotsa_x4_8x4_600k_reds REDS 30.3430 0.8664 8 model | log
edvrm_x4_8x4_600k_reds REDS 30.4194 0.8684 8 model | log
edvrl_wotsa_c128b40_8x8_lr2e-4_600k_reds4 REDS 31.0010 0.8784 8 (Tesla V100-PCIE-32GB) model | log
edvrl_c128b40_8x8_lr2e-4_600k_reds4 REDS 31.0467 0.8793 8 (Tesla V100-PCIE-32GB) model | log

Quick Start

Train

Train Instructions

You can use the following commands to train a model with cpu or single/multiple GPUs.

# cpu train
CUDA_VISIBLE_DEVICES=-1 python tools/train.py configs/edvr/edvrm_8xb4-600k_reds.py

# single-gpu train
python tools/train.py configs/edvr/edvrm_8xb4-600k_reds.py

# multi-gpu train
./tools/dist_train.sh configs/edvr/edvrm_8xb4-600k_reds.py 8

For more details, you can refer to Train a model part in train_test.md.

Test

Test Instructions

You can use the following commands to test a model with cpu or single/multiple GPUs.

# cpu test
CUDA_VISIBLE_DEVICES=-1 python tools/test.py python tools/test.py configs/edvr/edvrm_8xb4-600k_reds.py https://download.openmmlab.com/mmediting/restorers/edvr/edvrm_x4_8x4_600k_reds_20210625-e29b71b5.pth

# single-gpu test
python tools/test.py configs/edvr/edvrm_8xb4-600k_reds.py https://download.openmmlab.com/mmediting/restorers/edvr/edvrm_x4_8x4_600k_reds_20210625-e29b71b5.pth

# multi-gpu test
./tools/dist_test.sh configs/edvr/edvrm_8xb4-600k_reds.py https://download.openmmlab.com/mmediting/restorers/edvr/edvrm_x4_8x4_600k_reds_20210625-e29b71b5.pth 8

For more details, you can refer to Test a pre-trained model part in train_test.md.

Citation

@InProceedings{wang2019edvr,
  author    = {Wang, Xintao and Chan, Kelvin C.K. and Yu, Ke and Dong, Chao and Loy, Chen Change},
  title     = {EDVR: Video restoration with enhanced deformable convolutional networks},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
  month     = {June},
  year      = {2019},
}