ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

End-to-End lyrics recognition with voice to singing style transfer

Basak, S and Agarwal, S and Ganapathy, S and Takahashi, N (2021) End-to-End lyrics recognition with voice to singing style transfer. In: 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021, 6 June - 11 June 2021, Toronto, pp. 266-270.

ICASSP_2021.pdf - Published Version

Download (1MB) | Preview
Official URL: https://doi.org/10.1109/ICASSP39728.2021.9415096


Automatic transcription of monophonic/polyphonic music is a challenging task due to the lack of availability of large amounts of transcribed data. In this paper, we propose a data augmentation method that converts natural speech to singing voice based on vocoder based speech synthesizer. This approach, called voice to singing (V2S), performs the voice style conversion by modulating the F0 contour of the natural speech with that of a singing voice. The V2S model based style transfer can generate good quality singing voice thereby enabling the conversion of large corpora of natural speech to singing voice that is useful in building an E2E lyrics transcription system. In our experiments on monophonic singing voice data, the V2S style transfer provides a significant gain (relative improvements of 21 %) for the E2E lyrics transcription system. We also discuss additional components like transfer learning and lyrics based language modeling to improve the performance of the lyrics transcription system.

Item Type: Conference Paper
Publication: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Publisher: Institute of Electrical and Electronics Engineers Inc.
Additional Information: The copyright for this article belongs to the Authors.
Keywords: Modeling languages; Signal processing; Speech, Automatic transcription; Data augmentation; Language model; Model-based OPC; Monophonic singing; Natural speech; Singing styles; Speech synthesizer, Transfer learning
Department/Centre: Division of Electrical Sciences > Electrical Engineering
Date Deposited: 06 Jun 2023 10:11
Last Modified: 06 Jun 2023 10:11
URI: https://eprints.iisc.ac.in/id/eprint/81818

Actions (login required)

View Item View Item