Indian Sign Language Recognition on PYNQ Board

Singh, S and Rathna, GN and Singhal, V (2022) Indian Sign Language Recognition on PYNQ Board. In: Recent Advances in Computer Science and Communications, 15 (1). pp. 98-104.

PDF rec_adv_com_sci_som_15-1_98-104_2022.pdf - Published Version Restricted to Registered users only Download (4MB) | Request a copy

Abstract

Sign language is the only way to communicate for speech-impaired people. But this sign language is not known to normal people so this is a barrier in communication. This is the problem faced by people with speech impairments or disorder. In this paper, we have presented a system which captures hand gestures with a Kinect camera and classifies the hand gesture into its correct symbol. Methods: We used the Kinect camera, not the ordinary web camera, because the ordinary camera does not capture its 3d orientation or depth of an image; however, Kinect camera can capture 3d image and this will make the classification more accurate. Results: Kinect camera produces a different image for hand gestures for ‘2’ and ‘V’ and similarly for ‘1’ and ‘I’; however, a simple web camera cannot distinguish between these two. We used hand gestures for Indian sign language and our dataset contained 46339, RGB images and 46339 depth images. 80% of the total images were used for training and the remaining 20% for testing. In total, 36 hand gestures were considered to capture alphabets and alphabets ranged from A-Z and 10 for nu-merics. Conclusion: Along with real-time implementation, we have also shown the comparison of the performance of various machine learning models in which we found that CNN working on depth-images has more accuracy than other models. All these resulted were obtained on the PYNQ Z2 board. Discussion: We performed labeling of the data set, training, and classification on PYNQ Z2 FPGA board for static images using SVM, logistic regression, KNN, multilayer perceptron, and random forestalgorithms. For this experiment, we used our own 4 different datasets of ISL alphabets pre-pared in our lab. We analyzed both RGB images and depth images.

Item Type:	Journal Article
Publication:	Recent Advances in Computer Science and Communications
Publisher:	Bentham Science Publishers
Additional Information:	The copyright for this article belongs to the Bentham Science Publishers
Keywords:	Cameras; Classification (of information); Decision trees; Real time control; Statistical tests; Support vector machines, Depth image; Hand gesture; Impaired people; Indian sign languages; Kinect cameras; PYNQ-z2; RGB images; Sign language; Sign Language recognition; Web camera, Computer vision
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	04 Jul 2022 06:17
Last Modified:	02 Nov 2022 03:17
URI:	https://eprints.iisc.ac.in/id/eprint/74209

Actions (login required)

View Item