Pronunciation Training on Isolated Kannada Words Using 'Kannada Kali' - A Cloud Based Smart Phone Application

Murthy, S and Anand, A and Kumar, A and Cholin, A and Shetty, A and Bhat, A and Venkatesh, A and Kothiwale, L and Sitaram, D and Kumar, V (2019) Pronunciation Training on Isolated Kannada Words Using 'Kannada Kali' - A Cloud Based Smart Phone Application. In: 7th IEEE International Conference on Cloud Computing in Emerging Markets, CCEM 2018, 23 - 24 November 2018, Bengaluru, pp. 57-64.

PDF CCEM_2018.pdf - Published Version Restricted to Registered users only Download (9MB) | Request a copy

Abstract

Automated feedback on pronunciation system on a smart phone is useful for a student trying to learn a new language at his or her own pace. The objective of our re-search is to implement a pronunciation training system with minimal language specific data. Our proposed system consists of an Android application as a front-end, and a pronunciation evaluation and mispronunciation detection framework as the back-end hosted on a cloud. We conduct our experiments on spoken isolated words in Kannada. Our pronunciation evaluation(for spoken word) implementation on the cloud involves training a classifier with features from Dynamic Time Warping (DTW) with Mel Frequency Cepstral Coefficients (MFCC) and Line Spectral Frequencies (LSF) and, without directly on LSF (without DTW). We study the performance of different machine learning algorithms for pronunciation rating. We propose a novel semi-supervised approach for detecting mispronounced segments of a word using Self Organizing Maps (SOM) that are also deployed on the cloud. Our implementation of SOM learns the features of an automatically segmented reference speech. The trained SOM is then used to determine the deviations in the learner's pronunciation. We evaluate our system on 1169 Kannada audio samples from students around 18 to 25 years of age. The Kannada words considered are taken from textbooks of first and second grade (considering learners as beginners who do not know Kannada) and include 2 to 5 syllable words. We report accuracy on binary classification and multi-class classification for different classifiers. The mispronounced segments detected using SOM correlate with the human ratings. Our approach of pronunciation evaluation and mispronunciation detection is based on minimal data and does not require a speech recognition system. © 2018 IEEE.

Item Type:	Conference Paper
Publication:	Proceedings - 7th IEEE International Conference on Cloud Computing in Emerging Markets, CCEM 2018
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Cloud computing; Commerce; Conformal mapping; Learning algorithms; Machine learning; Search engines; Self organizing maps; Smartphones; Speech recognition, CAPT; Cloud services; Kannada; Mispronunciation detections; Pronunciation evaluations; Pronunciation trainings, Classification (of information)
Department/Centre:	Division of Mechanical Sciences > Divecha Centre for Climate Change Others
Date Deposited:	13 Dec 2022 04:50
Last Modified:	13 Dec 2022 04:50
URI:	https://eprints.iisc.ac.in/id/eprint/78324

Actions (login required)

View Item