Kundu, JN and Rakesh, M and Jampani, V and Venkatesh, RM and Venkatesh Babu, R (2020) Appearance Consensus Driven Self-supervised Human Mesh Recovery. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 23 - 28 August 2020, Glasgow, pp. 794-812.
|
PDF
ECCV 2020.pdf - Published Version Download (4MB) | Preview |
Abstract
We present a self-supervised human mesh recovery framework to infer human pose and shape from monocular images in the absence of any paired supervision. Recent advances have shifted the interest towards directly regressing parameters of a parametric human model by supervising them on large-scale datasets with 2D landmark annotations. This limits the generalizability of such approaches to operate on images from unlabeled wild environments. Acknowledging this we propose a novel appearance consensus driven self-supervised objective. To effectively disentangle the foreground (FG) human we rely on image pairs depicting the same person (consistent FG) in varied pose and background (BG) which are obtained from unlabeled wild videos. The proposed FG appearance consistency objective makes use of a novel, differentiable Color-recovery module to obtain vertex colors without the need for any appearance network; via efficient realization of color-picking and reflectional symmetry. We achieve state-of-the-art results on the standard model-based 3D pose estimation benchmarks at comparable supervision levels. Furthermore, the resulting colored mesh prediction opens up the usage of our framework for a variety of appearance-related tasks beyond the pose and shape estimation, thus establishing our superior generalizability.
| Item Type: | Conference Paper |
|---|---|
| Publication: | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
| Publisher: | Springer Science and Business Media Deutschland GmbH |
| Additional Information: | The copyright for this article belongs to the Authors. |
| Keywords: | 3D modeling; Color; Computer vision; Large dataset; Mesh generation; Recovery, 3D pose estimation; Color recovery; Large-scale datasets; Monocular image; Reflectional symmetry; Shape estimation; State of the art; The standard model, Computer graphics |
| Department/Centre: | Division of Interdisciplinary Sciences > Computational and Data Sciences |
| Date Deposited: | 23 Jan 2023 10:20 |
| Last Modified: | 23 Jan 2023 10:20 |
| URI: | https://eprints.iisc.ac.in/id/eprint/79269 |
Actions (login required)
 |
View Item |
