ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

A Multi-Head Relevance Weighting Framework for Learning Raw Waveform Audio Representations

Dutta, D and Agrawal, P and Ganapathy, S (2021) A Multi-Head Relevance Weighting Framework for Learning Raw Waveform Audio Representations. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2021-O . pp. 191-195.

[img]
Preview
PDF
IEEE_WASPAA_2021.pdf - Published Version

Download (2MB) | Preview
Official URL: https://doi.org/10.1109/WASPAA52581.2021.9632708

Abstract

In this work, we propose a multi-head relevance weighting framework to learn audio representations from raw waveforms. The audio waveform, split into windows of short-duration, are processed with a 1-D convolutional layer of cosine modulated Gaussian filters acting as a learnable filterbank. The key novelty of the proposed framework is the introduction of multi-head relevance on the learnt filterbank representations. Each head of the relevance network is modelled as a separate sub-network. These heads perform representation enhancement by generating weight masks for different parts of the time-frequency representation learnt by the parametric acoustic filterbank layer. The relevance weighted representations are fed to a neural classifier and the whole system is trained jointly for the audio classification objective. Experiments are performed on the DCASE2020 Task 1A challenge as well as the Urban Sound Classification (USC) tasks. In these experiments, the proposed approach yields relative improvements of 10 and 23 respectively for the DCASE2020 and USC datasets over the mel-spectrogram baseline. Also, the analysis of multi-head relevance weights provides insights on the learned representations. © 2021 IEEE.

Item Type: Journal Article
Publication: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics
Publisher: Institute of Electrical and Electronics Engineers Inc.
Additional Information: The copyright for this article belongs to Authors
Keywords: Filter banks, Audio representation; Audio waveforms; Learn+; Raw waveform modeling; Relevance weighting; Short durations; Sound classification; Sound event classification; Waveform modeling; Waveforms, Audio acoustics
Department/Centre: Division of Electrical Sciences > Electrical Engineering
Date Deposited: 07 Feb 2022 12:16
Last Modified: 07 Feb 2022 12:16
URI: http://eprints.iisc.ac.in/id/eprint/71268

Actions (login required)

View Item View Item