The impact of cross language on acoustic-to-articulatory inversion and its influence on articulatory speech synthesis

Illa, A and Nair, A and Ghosh, PK (2022) The impact of cross language on acoustic-to-articulatory inversion and its influence on articulatory speech synthesis. In: 47th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022, 23 May 2022 through 27 May 2022, Virtual, Online at Singapore, pp. 8267-8271.

PDF IEEE_ICASSP-2022_2022_8267-8271_2022.pdf - Published Version Restricted to Registered users only Download (946kB) | Request a copy

Abstract

Estimating articulatory representations (ARs) from acoustic features is known as acoustic-to-articulatory inversion (AAI). Various factors of input acoustic features impact the performance of AAI. In this work, we investigate the effect of unseen language on the AAI performance in both seen and unseen speaker conditions. We further perform experiments to analyze how these AAI predictions in unseen language and unseen speaker conditions, in turn, impact the articulatory speech synthesis, i.e., articulatory-to-acoustic forward mapping (AAF). We hypothesize that this investigation enables the exploration of alternative approaches to voice conversion across unseen languages using ARs. Experiments are performed on the AAF model trained using English ARs and evaluated on ARs from unseen speakers speaking different native Indian languages, namely, Hindi, Kannada, Telugu, and Tamil. Experiments reveal that, for AAI, there is a drop in performance due to the mismatch in language in both seen and unseen speaker evaluations. For AAF, subjective evaluations reveal that the synthesized speech quality of non-native (mismatched language) speech is comparable with that of English (matched language). © 2022 IEEE

Item Type:	Conference Paper
Publication:	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	The copyright for this article belongs to the Institute of Electrical and Electronics Engineers Inc.
Keywords:	acoustic-to-articulatory inversion; articulatory speech synthesis; articulatory-to-acoustic forward mapping
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	21 Jun 2022 10:27
Last Modified:	21 Jun 2022 10:27
URI:	https://eprints.iisc.ac.in/id/eprint/73936

Actions (login required)

View Item