A method for MIDI velocity estimation for piano performance by a U-Net with attention and FiLM

Kim, Hyon; Serra, Xavier

A method for MIDI velocity estimation for piano performance by a U-Net with attention and FiLM

Citation

Kim H, Serra X. A method for MIDI velocity estimation for piano performance by a U-Net with attention and FiLM. Paper presented at: 25th International Society for Music Information Retrieval Conference (ISMIR2024); 2024 November 10-14; San Francisco, USA.

Abstract

It is a well known fact that the dynamics in piano performance gives significant effect in expressiveness. Taking the polyphonic nature of the instrument into account, analysing information to form dynamics for each performed note has significant meaning to understand piano performance in a quantitative way. It is also a key element in an education context for piano learners. In this study, we developed a model for estimating MIDI velocity for each note, as one of indicators to represent loudness, with a condition of score assuming educational use case, by a Deep Neural Network (DNN) utilizing a U-Net with Scaled Dot-Product Attention (Attention) and Feature-wise Linear Modulation (FiLM) conditioning. As a result, we prove that effectiveness of Attention and FiLM conditioning, improved estimation accuracy and achieved the best result among previous researches using DNNs and showed its robustness across the various domain of test data.

Description

Comunicació presentada a la 25th International Society for Music Information Retrieval Conference (ISMIR2024), celebrada a San Francisco del 10 al 14 de novembre de 2024.