Automatic musical instrument recognition in audiovisual recordings by combining image and audio classification strategies

Slizovskaia, Olga; Gómez Gutiérrez, Emilia, 1975-; Haro Ortega, Gloria

Home
→
Recerca: articles, congressos, llibres
→
Departament de Tecnologies de la Informació i les Comunicacions
→
Congressos (Departament de Tecnologies de la Informació i les Comunicacions)
→
View Item

dc.contributor.author	Slizovskaia, Olga
dc.contributor.author	Gómez Gutiérrez, Emilia, 1975-
dc.contributor.author	Haro Ortega, Gloria
dc.date.accessioned	2017-07-10T07:28:15Z
dc.date.available	2017-07-10T07:28:15Z
dc.date.issued	2016
dc.identifier.citation	Slizovskaia O, Gómez E, Haro G. Automatic musical instrument recognition in audiovisual recordings by combining image and audio classification strategies. In: Großmann R, Hajdu G, editors. Proceedings SMC 2016. 13th Sound and Music Computing Conference; 2016 Aug 31; Hamburg, Germany. Hamburg (Germany): ZM4, Hochschule für Musik und Theater Hamburg; 2016. p. 442-7.
dc.identifier.uri	http://hdl.handle.net/10230/32522
dc.description	Comunicació presentada a la 13th Sound and Music Computing Conference, celebrada el 31 d'agost de 2016 a Hamburg, Alemanya.
dc.description.abstract	The goal of this work is to incorporate the visual modality into a musical instrument recognition system. For that, we first evaluate state-of-the-art image recognition techniques in the context of music instrument recognition, using a database of about 20000 images and 12 instrument classes. We then reproduce the results of state-of-the-art methods for audio-based musical instrument recognition, considering standard datasets including more than 9000 sound excerpts and 45 instrument classes. We finally compare the accuracy and confusions in both modalities and we showcase how they can be integrated for audio-visual instrument recognition in music videos. We obtain around 0.75 F1-measure for audio and 0.77 for images and similar confusions between instruments. This study confirms that visual (shape) and acoustic (timbre) properties of music instruments are related to each other and reveals the potential of audiovisual music description systems.
dc.description.sponsorship	This research was partially supported by the Spanish Ministry of Economy and Competitiveness under the María de Maeztu Units of Excellence Programme (MDM-2015-0502).
dc.format.mimetype	application/pdf
dc.language.iso	eng
dc.publisher	Zentrum für Mikrotonale Musik und Multimediale Komposition (ZM4), Hochschule für Musik und Theater Hamburg
dc.relation.ispartof	Großmann R, Hajdu G, editors. Proceedings SMC 2016. 13th Sound and Music Computing Conference; 2016 Aug 31; Hamburg, Germany. Hamburg (Germany): ZM4, Hochschule für Musik und Theater Hamburg; 2016. p. 442-7.
dc.rights	© 2016 Olga Slizovskaia et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
dc.rights.uri	https://creativecommons.org/licenses/by/3.0/
dc.subject.other	Música -- Anàlisi
dc.subject.other	Instruments musicals
dc.subject.other	Reconeixement de formes (Informàtica)
dc.title	Automatic musical instrument recognition in audiovisual recordings by combining image and audio classification strategies
dc.type	info:eu-repo/semantics/conferenceObject
dc.rights.accessRights	info:eu-repo/semantics/openAccess
dc.type.version	info:eu-repo/semantics/publishedVersion