Welcome to the UPF Digital Repository

Score-Informed MIDI Velocity Estimation for Piano Performance by FiLM Conditioning

Show simple item record

dc.contributor.author Kim, Hyon
dc.contributor.author Miron, Marius
dc.contributor.author Serra, Xavier
dc.date.accessioned 2023-05-12T15:58:35Z
dc.date.available 2023-05-12T15:58:35Z
dc.date.issued 2023-05-12
dc.identifier.uri http://hdl.handle.net/10230/56803
dc.description This work has been accepted at the SMC Sound and Music Computing Conference 2023, at Stockholm, Sweden. June 15-17, 2023.
dc.description.abstract Piano is one of the most popular instruments among people that learn to play music. When playing the piano, the level of loudness is crucial for expressing emotions as well as manipulating tempo. These elements convey the expressiveness of music performance. Detecting the loudness of each note could provide more valuable feedback for music students, helping to improve their performance dynamics. This can be achieved by visualizing the loudness levels not only for self-learning purposes but also for effective communication between teachers and students. Also, given the polyphonic nature of piano music, which often involves parallel melodic streams, determining the loudness of each note is more informative than analyzing the cumulative loudness of a specific time frame. This research proposes a method using Deep Neural Network (DNN) with score information to estimate note-level MIDI velocity of piano performances from audio input. In addition, when score information is available, we condition the DNN with score information using a Feature-wise Linear Modulation (FiLM) layer. To the best of our knowledge, this is the first attempt to estimate the MIDI velocity using a neural network in an end to end fashion. The model proposed in this study achieved improved accuracy in both MIDI velocity estimation and estimation error deviation, as well as higher recall accuracy for note classification when compared to the DNN model that did not use score information
dc.description.sponsorship This research was carried out under the project Musical AI - PID2019- 111403GB-I00/AEI/10.13039/501100011033, funded by the Spanish Ministerio de Ciencia e Innovacion ́ and the Agencia Estatal de Investigacion.
dc.format.mimetype application/pdf
dc.language.iso eng
dc.rights © 2023 Hyon Kim et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
dc.rights.uri https://creativecommons.org/licenses/by/3.0/
dc.title Score-Informed MIDI Velocity Estimation for Piano Performance by FiLM Conditioning
dc.type info:eu-repo/semantics/preprint
dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.type.version info:eu-repo/semantics/submittedVersion

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Search DSpace


Advanced Search

Browse

My Account

Statistics

In collaboration with Compliant to Partaking