Myungsub Choi, Heewon Kim, Bohyung Han, Ning Xu, Kyoung Mu Lee

2nd place in [AIM 2019 ICCV Workshop] - Video Temporal Super-Resolution Challenge

Project Paper-AAAI (Download the paper [here] in case the AAAI link is broken) Poster

Paper

Abstract

Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature reshaping operation, referred to as PixelShuffle, with a channel attention, which replaces the optical flow computation module. The main idea behind the design is to distribute the information in a feature map into multiple channels and extract motion information by attending the channels for pixel-level frame synthesis. The model given by this principle turns out to be effective in the presence of challenging motion and occlusion. We construct a comprehensive evaluation benchmark and demonstrate that the proposed approach achieves outstanding performance compared to the existing models with a component for optical flow computation.

Model

Dataset

Download

Results

Video

Video

Citation

If you find this code useful for your research, please consider citing the following paper:

@inproceedings{choi2020cain,
    author = {Choi, Myungsub and Kim, Heewon and Han, Bohyung and Xu, Ning and Lee, Kyoung Mu},
    title = {Channel Attention Is All You Need for Video Frame Interpolation},
    booktitle = {AAAI},
    year = {2020}
}

Acknowledgement

Many parts of this code is adapted from:

We thank the authors for sharing codes for their great works.


For further questions, please contact @myungsub)