CAPNet: Continuous approximation projection for 3D point cloud reconstruction using 2D supervision

Navaneet, KL and Mandikal, P and Agarwal, M and Venkatesh Babu, R (2019) CAPNet: Continuous approximation projection for 3D point cloud reconstruction using 2D supervision. In: 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Annual Conference on Innovative Applications of Artificial Intelligence, 27 January - 1 February 2019, Hilton Hawaii VillageHonolulu; United States, pp. 8819-8826.

	PDF aaa_con_art_int_8819-8826_2019.pdf - Published Version Restricted to Registered users only Download (7MB) | Request a copy
	PDF CAPNet_suppl_AAAI_2019_arxiv.pdf - Published Supplemental Material Restricted to Registered users only Download (7MB) | Request a copy

Abstract

Knowledge of 3D properties of objects is a necessity in order to build effective computer vision systems. However, lack of large scale 3D datasets can be a major constraint for data-driven approaches in learning such properties. We consider the task of single image 3D point cloud reconstruction, and aim to utilize multiple foreground masks as our supervisory data to alleviate the need for large scale 3D datasets. A novel differentiable projection module, called 'CAPNet', is introduced to obtain such 2D masks from a predicted 3D point cloud. The key idea is to model the projections as a continuous approximation of the points in the point cloud. To overcome the challenges of sparse projection maps, we propose a loss formulation termed 'affinity loss' to generate outlier-free reconstructions. We significantly outperform the existing projection based approaches on a large-scale synthetic dataset. We show the utility and generalizability of such a 2D supervised approach through experiments on a real-world dataset, where lack of 3D data can be a serious concern. To further enhance the reconstructions, we also propose a test stage optimization procedure to obtain reconstructions that display high correspondence with the observed input image. © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Item Type:	Conference Paper
Publication:	33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019
Publisher:	AAAI Press
Additional Information:	The copyright of this article belongs to AAAI Press
Keywords:	Artificial intelligence; Image enhancement; Large dataset; Water conservation, 3D point cloud; Computer vision system; Continuous approximations; Data-driven approach; Input image; Optimization procedures; Projection maps; Single images, Image reconstruction
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	16 Nov 2020 09:23
Last Modified:	30 Aug 2022 06:50
URI:	https://eprints.iisc.ac.in/id/eprint/66713

Actions (login required)

View Item