Lyrics to audio alignment for karaoke in pop music

Dzhambazov, Georgi Bogomilov; Miron, Marius; Serra, Xavier

Lyrics to audio alignment for karaoke in pop music

License

Citation

Dzhambazov G, Miron M, Serra X. Lyrics to audio alignment for karaoke in pop music. Paper presented at: ISMIR 2016. 17th International Society for Music Information Retrieval Conference; 2016 Aug 7-11; New York City (NY).

Abstract

In this paper we describe an algorithm for automatic lyricsto-audio alignment. It has as a goal the automatic detection of word boundaries in multi-instrumental English pop songs. We rely on a phonetic recognizer based on hidden Markov models: a widely-used method for tracking phonemes in speech processing problems. Tracking lyrics in music audio is harder than tracking text in speech because, unlike speech, the singing voice is mixed with multiple instruments. To address this obstacle we apply a convolution neural networks-based method for singing voice separation. We present a prototype of a practical application based on the alignment method - the highliting of lyrics in a karaoke-like fashion.

Description

Comunicació preseentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA.