Robust Hand Gestures Recognition Using a Deep CNN and Thermal Images

Breland, DS and Dayal, A and Jha, A and Yalavarthy, PK and Pandey, OJ and Cenkeramaddi, LR (2021) Robust Hand Gestures Recognition Using a Deep CNN and Thermal Images. In: IEEE Sensors Journal, 21 (23). pp. 26602-26614.

PDF iee_sen_jou_21_26602-26614_2021.pdf - Published Version Restricted to Registered users only Download (3MB) | Request a copy

Abstract

Medical systems and assistive technologies, human-computer interaction, human-robot interaction, industrial automation, virtual environment control, sign language translation, crisis and disaster management, entertainment and computer games, and so on all use RGB cameras for hand gesture recognition. However, their performance is limited especially in low-light conditions. In this paper, we propose a robust hand gesture recognition system based on high-resolution thermal imaging that is light-independent. A dataset of 14,400 thermal hand gestures is constructed, separated into two color tones. We also propose using a deep CNN to classify high-resolution hand gestures accurately. The proposed models were also tested on Raspberry Pi 4 and Nvidia AGX edge computing devices, and the results were compared to the benchmark models. The model also achieves an accuracy of 98.81% and an inference time of 75.138 ms on Nvidia Jetson AGX. In contrast to hand gesture recognition systems based on RGB cameras, which have limited performance in the dark-light conditions, the proposed system based on reliable high resolution thermal images is well-suited to a wide range of applications.

Item Type:	Journal Article
Publication:	IEEE Sensors Journal
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Cameras; Computer control systems; Computer games; Computer vision; Disasters; Face recognition; Gesture recognition; Human robot interaction; Image resolution; Infrared imaging; Learning systems; Medical imaging; Palmprint recognition; Security systems; Tumors; Virtual reality, Deep CNN; Gestures recognition; Hand-gesture recognition; High resolution; High resolution thermal imaging; Humans-robot interactions; Thermal images; Thermal sensors; Thermal-imaging, Human computer interaction
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	18 Apr 2023 09:01
Last Modified:	18 Apr 2023 09:01
URI:	https://eprints.iisc.ac.in/id/eprint/80650

Actions (login required)

View Item