Representation Learning Using Convolution Neural Network for Acoustic-to-articulatory Inversion

Illa, A and Ghosh, PK (2019) Representation Learning Using Convolution Neural Network for Acoustic-to-articulatory Inversion. In: 44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019, 12 - 17 May 2019, Brighton, pp. 5931-5935.

PDF ICASSP_2019.pdf - Published Version Restricted to Registered users only Download (14MB) | Request a copy

Abstract

Recent techniques employ end-to-end systems to learn relevant features for several speech related applications, including speech recognition, and speaker verification. In this work, we focus on the task of acoustic-to-articulatory inversion (AAI) for which we propose an end-to-end system that comprises a convolution neural network (CNN) and a bidirectional long short-term memory network (BLSTM). The aim of this work is to understand the nature of the features learnt by the end-to-end model and the importance of pre-emphasis in representation learning for AAI. Further, we propose a subject adaptation scheme to overcome the limitations of the availability of parallel acoustic-articulatory data to train an end-to-end AAI system. The AAI performance is evaluated with ~3.19 hours of acoustic-articulatory data collected from 8 subjects. Experiments reveal that, the frequency response of filters learnt by the CNN in the proposed system resembles those of the mel-scale, and hence, the performance of the proposed system (RMSE=1.47mm) is on par with that using mel-frequency cepstral coefficients (1.42mm) as features. Using pre-emphasis reduces RMSE by 0.13mm, and also the proposed adaptation scheme performs better than a subject-specific AAI model by an RMSE of 0.21mm despite of limited acoustic-articulatory data from a subject.

Item Type:	Conference Paper
Publication:	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Convolution; Frequency response; Speech communication; Speech recognition, Articulatory inversion; BLSTM; Convolution neural network; electromagnetic articulograph; End-to-end systems; Mel frequency cepstral co-efficient; Relevant features; Speaker verification, Audio signal processing
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	15 Dec 2022 08:11
Last Modified:	15 Dec 2022 08:11
URI:	https://eprints.iisc.ac.in/id/eprint/78375

Actions (login required)

View Item