HaM3D: generalized XR-based multimodal HRI framework with haptic feedback for industry 4.0

Raj, S and Beri, N and Patel, DS and Sinha, Y and Chakrabarti, A and Biswas, P (2024) HaM3D: generalized XR-based multimodal HRI framework with haptic feedback for industry 4.0. In: Journal on Multimodal User Interfaces, 18 . pp. 331-349.

PDF Jou_Mul_Use_Int_2024.pdf - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

The current generation of Industry 4.0 involves the integration of automation and data in manufacturing, while the upcoming generation, Industry 5.0, aims to enable seamless interaction between humans, robots, and machines for sustainable and personalized product and service creation. This necessitates manufacturing components according to customer wishes, leading to frequent changes in the production line. In this context, automation proves unsustainable and could be addressed through Human�Robot Cooperation. In this paper, an Extended Reality (XR)-based Human�Robot Interaction with haptic feedback has been proposed to assist users in remotely performing tasks. The haptic feedback, generated using the Artificial Potential Field method, aids users in controlling robots without collisions. This paper introduces a multimodal XR-based framework for effective communication between humans and robots, utilizing vision models to provide information about the remote side to local users. This seamless communication is facilitated through multiple modalities such as haptic, speech, visual, and text. The system�s effectiveness through pick-and-place activities and adaptation to it for complex industry use cases such as peg in the hole and welding are further evaluated. The results demonstrate the system�s efficacy in two specific human�robot collaborative scenarios in industries: arc welding and assembly tasks. © The Author(s), under exclusive licence to Springer Nature Switzerland AG 2024.

Item Type:	Journal Article
Publication:	Journal on Multimodal User Interfaces
Publisher:	Springer Science and Business Media Deutschland GmbH
Additional Information:	The copyright for this article belongs to Springer Science and Business Media Deutschland GmbH
Keywords:	Carbon dioxide arc welding; Critical path analysis; Haptic interfaces; Industrial robots; Robotic assembly; Submerged arc welding, Current generation; Extended reality; Haptic feedbacks; Human robots; Human-machine; Human-robot-cooperation; Integration of automation; Mixed reality; Multi-modal; Multimodal Interaction, Collaborative robots
Department/Centre:	Division of Mechanical Sciences > Department of Design & Manufacturing (formerly Centre for Product Design & Manufacturing)
Date Deposited:	27 Nov 2024 11:48
Last Modified:	27 Nov 2024 11:48
URI:	http://eprints.iisc.ac.in/id/eprint/86957

Actions (login required)

View Item