Lyrics to audio alignment for karaoke in pop music

Abstract

In this paper we describe an algorithm for automatic lyricsto-audio alignment. It has as a goal the automatic detection of word boundaries in multi-instrumental English pop songs. We rely on a phonetic recognizer based on hidden Markov models: a widely-used method for tracking phonemes in speech processing problems. Tracking lyrics in music audio is harder than tracking text in speech because, unlike speech, the singing voice is mixed with multiple instruments. To address this obstacle we apply a convolution neural networks-based method for singing voice separation. We present a prototype of a practical application based on the alignment method - the highliting of lyrics in a karaoke-like fashion.