Acoustic-to-articulatory inversion for dysarthric speech by using cross-corpus acoustic-articulatory data

Maharana, SK and Illa, A and Mannem, R and Belur, Y and Shetty, P and Kumar, VP and Vengalil, S and Polavarapu, K and Atchayaram, N and Ghosh, PK (2021) Acoustic-to-articulatory inversion for dysarthric speech by using cross-corpus acoustic-articulatory data. In: 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021, 6-11 Jun 2021, Toronto, pp. 6458-6462.

PDF ICASSP_6458-6462_2021.pdf - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

In this work, we focus on estimating articulatory movements from acoustic features, known as acoustic-to-articulatory inversion (AAI), for dysarthric patients with amyotrophic lateral sclerosis (ALS). Unlike healthy subjects, there are two potential challenges involved in AAI on dysarthric speech. Due to speech impairment, the pronunciation of dysarthric patients is unclear and inaccurate, which could impact the AAI performance. In addition, acoustic-articulatory data from dysarthric patients is limited due to the difficulty in the recording. These challenges motivate us to utilize cross-corpus acoustic-articulatory data. In this study, we propose an AAI model by conditioning speaker information using x-vectors at the input, and multi-target articulatory trajectory outputs for each corpus separately. Results reveal that the proposed AAI model shows relative improvements of the Pearson correlation coefficient (CC) by �13.16 and �16.45 over a randomly initialized baseline AAI model trained with only dysarthric corpus in seen and unseen conditions, respectively. In the seen conditions, the proposed AAI model outperforms the three baseline AAI models, that utilize the cross-corpus, by �3.49, �6.46, and �4.03 in terms of CC. © 2021 IEEE

Item Type:	Conference Paper
Publication:	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Correlation methods, Acoustic features; Amyotrophic lateral sclerosis; Articulatory data; Articulatory inversion; Healthy subjects; Multi-targets; Pearson correlation coefficients, Signal processing
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	03 Dec 2021 08:29
Last Modified:	03 Dec 2021 08:29
URI:	http://eprints.iisc.ac.in/id/eprint/70207

Actions (login required)

View Item