ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Self-Attentive Feature-level Fusion for Multimodal Emotion Detection

Hazarika, Devamanyu and Gorantla, Sruthi and Poria, Soujanya and Zimmermann, Roger (2018) Self-Attentive Feature-level Fusion for Multimodal Emotion Detection. In: IEEE Conference on Multimedia Information Processing and Retrieval 2018, 10-12 April 2018, Miami, FL, USA, pp. 196-201.

[img] PDF
ICMIPR_2018.pdf - Published Version
Restricted to Registered users only

Download (633kB) | Request a copy
Official URL: http://doi.org/10.1109/MIPR.2018.00043


Multimodal emotion recognition is the task of detecting emotions present in user-generated multimedia content. Such resources contain complementary information in multiple modalities. A stiff challenge often faced is the complexity associated with feature-level fusion of these heterogeneous modes. In this paper, we propose a new feature-level fusion method based on self-attention mechanism. We also compare it with traditional fusion methods such as concatenation, outer-product, etc. Analyzed using textual and speech (audio) modalities, our results suggest that the proposed fusion method outperforms others in the context of utterance-level emotion recognition in videos.

Item Type: Conference Proceedings
Additional Information: Copyright for this article belongs to IEEE
Keywords: Multimodal emotion recognition, Feature level Fusion , Self Attention
Department/Centre: Division of Electrical Sciences > Computer Science & Automation
Depositing User: LIS Interns
Date Deposited: 23 May 2019 09:34
Last Modified: 23 May 2019 09:34
URI: http://eprints.iisc.ac.in/id/eprint/62756

Actions (login required)

View Item View Item