Comparing alternative modalities in the context of multimodal human–robot interaction

Saren, S and Mukhopadhyay, A and Ghose, D and Biswas, P (2023) Comparing alternative modalities in the context of multimodal human–robot interaction. In: Journal on Multimodal User Interfaces .

PDF jou_mul_use_int_2023.pdf - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

With the advancement of interactive technology, alternative input modalities are often used, instead of conventional ones, to create intuitive, efficient, and user-friendly avenues of controlling and collaborating with robots. Researchers have examined the efficacy of natural interaction modalities such as gesture or voice in single and dual-task scenarios. These investigations have aimed to address the potential of the modalities on diverse applications encompassing activities like online shopping, precision agriculture, and mechanical component assembly, which involve tasks like object pointing and selection. This article aims to address the impact on user performance in a practical human�robot interaction application where a fixed-base robot is controlled through the utilization of natural alternative modalities. We explored this by investigating the impact of single-task and dual-task conditions on user performance for object picking and dropping. We undertook two user studies�one focusing on single-task scenarios, employing a fixed-base robot for object picking and dropping and the other encompassing dual-task conditions, utilizing a mobile robot for a driving scenario. We measured task completion times and estimated cognitive workload through the NASA Task Load Index (TLX), which offers a subjective, multidimensional scale measuring the perceived cognitive workload of a user. The studies revealed that the ranking of completion times for the alternative modalities remained consistent across both single-task and dual-task scenarios. However, the ranking based on perceived cognitive load was different. In the single-task study, the gesture-based modality resulted the highest TLX score, contrasting with the dual-task study, where the highest TLX score was associated with the eye gaze-based modality. Likewise, the speech-based modality achieved a lower TLX score compared to eye gaze and gesture in the single-task study, but its TLX score in the dual-task study was between gesture and eye gaze. These outcomes suggest that the efficacy of alternative modalities is contingent not only on user preferences but also on the specific situational context. © 2023, The Author(s), under exclusive licence to Springer Nature Switzerland AG.

Item Type:	Journal Article
Publication:	Journal on Multimodal User Interfaces
Publisher:	Springer Science and Business Media Deutschland GmbH
Additional Information:	The copyright for this article belongs to Springer Science and Business Media Deutschland GmbH.
Keywords:	Human–Robot interaction · Multimodal interaction · Alternative input modalities
Department/Centre:	Division of Mechanical Sciences > Aerospace Engineering(Formerly Aeronautical Engineering) Division of Mechanical Sciences > Centre for Product Design & Manufacturing
Date Deposited:	17 Dec 2023 09:59
Last Modified:	17 Dec 2023 09:59
URI:	https://eprints.iisc.ac.in/id/eprint/83460

Actions (login required)

View Item