Deep Learning-Based Sign Language Digits Recognition from Thermal Images with Edge Computing System

Breland, DS and Skriubakken, SB and Dayal, A and Jha, A and Yalavarthy, PK and Cenkeramaddi, LR (2021) Deep Learning-Based Sign Language Digits Recognition from Thermal Images with Edge Computing System. In: IEEE Sensors Journal, 21 (9). pp. 10445-10453.

PDF IEEE_sen_jou_21-9_10445-10453_2021.pdf - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

The sign language digits based on hand gestures have been utilized in various applications such as human-computer interaction, robotics, health and medical systems, health assistive technologies, automotive user interfaces, crisis management and disaster relief, entertainment, and contactless communication in smart devices. The color and depth cameras are commonly deployed for hand gesture recognition, but the robust classification of hand gestures under varying illumination is still a challenging task. This work presents the design and deployment of a complete end-to-end edge computing system that can accurately provide the classification of hand gestures captured from thermal images. A thermal dataset of 3200 images was created with each sign language digit having 320 thermal images. The solution presented here utilizes live images taken from a low-resolution thermal camera of 32 × 32 pixels, feeding into a novel light weight deep learning model based on bottleneck motivated from deep residual learning for classification of hand gestures. The edge computing system presented here utilizes Raspberry pi with a thermal camera making it highly portable. The designed system achieves an accuracy of 99.52% on the test data set with an added advantage of accuracy being invariable to background lighting conditions as it is based on thermal imaging. © 2021 IEEE.

Item Type:	Journal Article
Publication:	IEEE Sensors Journal
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Cameras; Disaster prevention; Edge computing; Human computer interaction; Infrared devices; Infrared imaging; Learning systems; Medical robotics; Social robots; Statistical tests; User interfaces, Assistive technology; Computing system; Crisis management; Hand-gesture recognition; Learning models; Lighting conditions; Medical systems; Robust classification, Deep learning
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	22 Feb 2023 03:26
Last Modified:	22 Feb 2023 03:26
URI:	https://eprints.iisc.ac.in/id/eprint/80404

Actions (login required)

View Item