Reconstruction of articulatory movements during neutral speech from those during whispered speech

Meenakshi, Nisha G and Ghosh, Prasanta Kumar (2018) Reconstruction of articulatory movements during neutral speech from those during whispered speech. In: JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 143 (6). pp. 3352-3364.

PDF Jou_Asc_Soc_Ame_143-6_3352_2018.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

A transformation function (TF) that reconstructs neutral speech articulatory trajectories (NATs) from whispered speech articulatory trajectories (WATs) is investigated, such that the dynamic time warped (DTW) distance between the transformed whispered and the original neutral articulatory movements is minimized. Three candidate TFs are considered: an affine function with a diagonal matrix (A(d)) which reconstructs one NAT from the corresponding WAT, an affine function with a full matrix (A(f)) and a deep neural network (DNN) based nonlinear function which reconstruct each NAT from all WATs. Experiments reveal that the transformation could be approximated well by Af, since it generalizes better across subjects and achieves the least DTW distance of 5.20 (61.27) mm (on average), with an improvement of 7.47%, 4.76%, and 7.64% (relative) compared to that with Ad, DNN, and the best baseline scheme, respectively. Further analysis to understand the differences in neutral and whispered articulation reveals that the whispered articulators exhibit exaggerated movements in order to reconstruct the lip movements during neutral speech. It is also observed that among the articulators considered in the study, the tongue exhibits a higher precision and stability while whispering, implying that subjects control their tongue movements carefully in order to render an intelligible whispered speech. (C) 2018 Acoustical Society of America.

Item Type:	Journal Article
Publication:	JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA
Publisher:	ACOUSTICAL SOC AMER AMER INST PHYSICS, STE 1 NO 1, 2 HUNTINGTON QUADRANGLE, MELVILLE, NY 11747-4502 USA
Additional Information:	Copyright of this article belong to ACOUSTICAL SOC AMER AMER INST PHYSICS, STE 1 NO 1, 2 HUNTINGTON QUADRANGLE, MELVILLE, NY 11747-4502 USA
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	25 Jul 2018 15:28
Last Modified:	25 Jul 2018 15:28
URI:	http://eprints.iisc.ac.in/id/eprint/60301

Actions (login required)

View Item