Fotedar, G and Ghosh, PK (2017) An information theoretic analysis of the temporal synchrony between head gestures and prosodic patterns in spontaneous speech. In: 18th Annual Conference of the International Speech Communication Association, INTERSPEECH 2017, 20 - 24 August 2017, Stockholm, pp. 157-161.
Full text not available from this repository.Abstract
We analyze the temporal co-ordination between head gestures and prosodic patterns in spontaneous speech in a data-driven manner. For this study, we consider head motion and speech data from 24 subjects while they tell a fixed set of five stories. The head motion, captured using a motion capture system, is converted to Euler angles and translations in X, Y and Z-directions to represent head gestures. Pitch and short-time energy in voiced segments are used to represent the prosodic patterns. To capture the statistical relationship between head gestures and prosodic patterns, mutual information (MI) is computed at various delays between the two using data from 24 subjects in six native languages. The estimated MI, averaged across all subjects, is found to be maximum when the head gestures lag the prosodic patterns by 30msec. This is found to be true when subjects tell stories in English as well as in their native language. We observe a similar pattern in the root mean squared error of predicting head gestures from prosodic patterns using Gaussian mixture model. These results indicate that there could be an asynchrony between head gestures and prosody during spontaneous speech where head gestures follow the corresponding prosodic patterns.
| Item Type: | Conference Paper |
|---|---|
| Publication: | Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH |
| Publisher: | International Speech Communication Association |
| Additional Information: | The copyright for this article belongs to International Speech Communication Association. |
| Keywords: | Gaussian distribution; Information theory; Mean square error; Speech, Gaussian Mixture Model; Head gestures; Information-theoretic analysis; Motion capture system; Mutual informations; Prosodic patterns; Root mean squared errors; Statistical relationship, Speech communication |
| Department/Centre: | Division of Electrical Sciences > Electrical Engineering |
| Date Deposited: | 25 Jul 2022 05:08 |
| Last Modified: | 25 Jul 2022 05:08 |
| URI: | https://eprints.iisc.ac.in/id/eprint/74714 |
Actions (login required)
 |
View Item |
