Score-Informed MIDI Velocity Estimation for Piano Performance by FiLM Conditioning

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Kim, Hyon
• dc.contributor.author Miron, Marius
• dc.contributor.author Serra, Xavier
• dc.date.accessioned 2023-05-12T15:58:35Z
• dc.date.available 2023-05-12T15:58:35Z
• dc.date.issued 2023-05-12
• dc.description This work has been accepted at the SMC Sound and Music Computing Conference 2023, at Stockholm, Sweden. June 15-17, 2023.
• dc.description.abstract Piano is one of the most popular instruments among people that learn to play music. When playing the piano, the level of loudness is crucial for expressing emotions as well as manipulating tempo. These elements convey the expressiveness of music performance. Detecting the loudness of each note could provide more valuable feedback for music students, helping to improve their performance dynamics. This can be achieved by visualizing the loudness levels not only for self-learning purposes but also for effective communication between teachers and students. Also, given the polyphonic nature of piano music, which often involves parallel melodic streams, determining the loudness of each note is more informative than analyzing the cumulative loudness of a specific time frame. This research proposes a method using Deep Neural Network (DNN) with score information to estimate note-level MIDI velocity of piano performances from audio input. In addition, when score information is available, we condition the DNN with score information using a Feature-wise Linear Modulation (FiLM) layer. To the best of our knowledge, this is the first attempt to estimate the MIDI velocity using a neural network in an end to end fashion. The model proposed in this study achieved improved accuracy in both MIDI velocity estimation and estimation error deviation, as well as higher recall accuracy for note classification when compared to the DNN model that did not use score informationca
• dc.description.sponsorship This research was carried out under the project Musical AI - PID2019- 111403GB-I00/AEI/10.13039/501100011033, funded by the Spanish Ministerio de Ciencia e Innovacion ́ and the Agencia Estatal de Investigacion.
• dc.format.mimetype application/pdf*
• dc.identifier.uri http://hdl.handle.net/10230/56803
• dc.language.iso engca
• dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00
• dc.rights © 2023 Hyon Kim et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.ca
• dc.rights.accessRights info:eu-repo/semantics/openAccessca
• dc.rights.uri https://creativecommons.org/licenses/by/3.0/ca
• dc.title Score-Informed MIDI Velocity Estimation for Piano Performance by FiLM Conditioningca
• dc.type info:eu-repo/semantics/preprintca
• dc.type.version info:eu-repo/semantics/submittedVersionca