Kanchana, V and Somraj, N and Yadwad, S and Soundararajan, R (2022) Revealing Disocclusions in Temporal View Synthesis through Infilling Vector Prediction. In: 22nd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, 4 - 8 January 2022, Washington, pp. 3093-3102.
|
PDF
IEEE-CVF_WACV 2022_3093-3102_2022.pdf - Published Version Download (8MB) | Preview |
Abstract
We consider the problem of temporal view synthesis, where the goal is to predict a future video frame from the past frames using knowledge of the depth and relative camera motion. In contrast to revealing the disoccluded regions through intensity based infilling, we study the idea of an infilling vector to infill by pointing to a non-disoccluded region in the synthesized view. To exploit the structure of disocclusions created by camera motion during their infilling, we rely on two important cues, temporal correlation of infilling directions and depth. We design a learning framework to predict the infilling vector by computing a temporal prior that reflects past infilling directions and a normalized depth map as input to the network. We conduct extensive experiments on a large scale dataset we build for evaluating temporal view synthesis in addition to the SceneNet RGB-D dataset. Our experiments demonstrate that our infilling vector prediction approach achieves superior quantitative and qualitative infilling performance compared to other approaches in literature.
| Item Type: | Conference Paper |
|---|---|
| Publication: | Proceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022 |
| Publisher: | Institute of Electrical and Electronics Engineers Inc. |
| Additional Information: | The copyright for this article belongs to the Authors. |
| Keywords: | Cameras; Color photography; Computer vision; Deep learning; Forecasting; Large dataset; Three dimensional computer graphics, 3D computer vision; Camera motions; Deep learning; Disocclusion; Image and video synthesis 3d computer vision; Images synthesis; Infilling; Video synthesis; View synthesis; Vision for graphics, Vectors |
| Department/Centre: | Autonomous Societies / Centres > Centre for Brain Research Division of Electrical Sciences > Electrical Communication Engineering |
| Date Deposited: | 11 Jul 2022 09:45 |
| Last Modified: | 11 Jul 2022 09:45 |
| URI: | https://eprints.iisc.ac.in/id/eprint/74305 |
Actions (login required)
 |
View Item |
