Leveraging Carnatic live recordings for singing voice separation using regression-guided latent diffusion

Plaja-Roglans, Genís; Serra, Xavier; Rocamora, Martín

Leveraging Carnatic live recordings for singing voice separation using regression-guided latent diffusion

License

Citation

Plaja-Roglans G, Serra X, Rocamora M. Leveraging Carnatic live recordings for singing voice separation using regression-guided latent diffusion. Paper presented at: 26th International Society for Music Information Retrieval Conference (ISMIR 2025); 2025 Sep 21-25; Daejeon, Korea. 9p.

Abstract

Diffusion models have demonstrated potential to separate individual sources from music mixtures in a generative fashion, enabling a new solution for this challenging problem. However, existing works require clean multi-stem data, which is scarce for several repertoires, consequently compromising generalization. We explore the potential of generative modeling to perform weakly-supervised singing voice separation for Carnatic Music, a music repertoire for which large quantities of multi-stem recordings with bleeding between sources have been collected from live performances. We pre-train a latent diffusion model to perform preliminary vocal separation conditioning on the corresponding mixture. Then, using a regressive model which is separately trained on a clean, smaller, and out-of-domain dataset, we estimate the level of bleeding in the preliminary separations and use that information to guide the diffusion model toward generating cleaner samples. The objective and perceptual evaluations show the potential of the proposed generative system for Carnatic vocal separation.

Description

Comunicació presentada a la 26th International Society for Music Information Retrieval Conference (ISMIR 2025), celebrada a Daejeon (Korea) del 21 a 25 de setembre de 2025.