Visualization of deep audio embeddings for music exploration and rediscovery

Tovstogan, Philip; Serra, Xavier; Bogdanov, Dmitry

Visualization of deep audio embeddings for music exploration and rediscovery

License

Citation

Tovstogan P, Serra X, Bogdanov D. Visualization of deep audio embeddings for music exploration and rediscovery. In: Proceedings of the SMC 2022 Music technology and design; 2022 June 5-12; Saint-Étienne, France. Saint-Étienne: SMC; 2022. p. 493-500.

Abstract

User interfaces for music exploration and discovery have always been an exciting application of music information retrieval (MIR) throughout the years. However, while discovering new music is a common goal of such systems, there has been less attention paid to the exploration and rediscovery within personal music collections, where finding interesting relations between music items already familiar to the user can lead to a different type of highly engaging and rewarding experience. In this paper, we present a novel web interface to visualize music collections using the audio embeddings extracted from music tracks. The system allows exploring the relationship between music tracks from multiple perspectives, displaying embedding and tag spaces extracted by music auto-tagging models, trained using different architectures and datasets, coupled with various 2D projection algorithms. We conduct a user study to analyze the effectiveness of different visualization strategies on the participants’ personal music collections, particularly for playlist creation and music library navigation and rediscovery. Our results show that such an interface provides a good alternative to standard hierarchical library organization by metadata.