A Machine Learning-Based Framework for Real-Time 3D Reconstruction and Space Utilization in Built Environments

Mukhopadhyay, A and Patel, S and Sharma, P and Biswas, P (2024) A Machine Learning-Based Framework for Real-Time 3D Reconstruction and Space Utilization in Built Environments. In: UNSPECIFIED.

PDF Cue_wor_pro_3660_2024 - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

The process of 3D reconstruction involves transforming 2D images or data into a three-dimensional representation of an object, model, or environment. While supervised 3D reconstruction has made significant strides using deep neural networks, it is often time-consuming due to extensive image stitching and the requirement for specialized imaging sensors such as 360-degree or depth cameras. This paper introduces a machine learning-based 3D reconstruction framework aimed at making informed decisions regarding space utilization and asset management within any built environment. The proposed system comprises three key components: (I) object detection on 2D frames to identify target objects, (II) calculation of their pose using image processing techniques, and (III) utilization of an artificial neural network to map real and virtual environments. The evaluation using YOLOv7 demonstrated an accuracy of F1 score of 0.70 in detecting objects of interest. Pose estimation analysis indicated that the proposed algorithm could estimate object orientation with an error rate of 8.03�. The mapping algorithm exhibited high-quality performance, achieving a correlation coefficient of R2 = 0.97. Ultimately, all this information is transmitted and visualized in the reconstructed virtual model, enabling remote monitoring and simulation. © 2024 Copyright for this paper by its authors.

Item Type:	Conference Paper
Publication:	CEUR Workshop Proceedings
Publisher:	CEUR-WS
Additional Information:	The copyright for this article belongs to CEUR-WS.
Keywords:	Conformal mapping; Deep neural networks; Gesture recognition; Image reconstruction; Learning systems; Metadata; Object detection; Object recognition; Three dimensional computer graphics; Virtual reality, 3D reconstruction; Built environment; Eye trackers; Hand gesture; Machine-learning; Multimodal Interaction; Snake robots; Soft continuum manipulator; Soft snake robot; Space utilization, Manipulators
Department/Centre:	Division of Mechanical Sciences > Department of Design & Manufacturing (formerly Centre for Product Design & Manufacturing)
Date Deposited:	22 May 2024 04:28
Last Modified:	22 May 2024 04:28
URI:	https://eprints.iisc.ac.in/id/eprint/84846

Actions (login required)

View Item