Simulating piano performance mistakes for music learning

Abstract

The development of machine-learning based technologies to support music instrument learning needs large-scale datasets that capture the different stages of learning in a manner that is both realistic and computation-friendly. We are interested in modeling the mistakes of beginnerintermediate piano performances in practice or work-inprogress settings. In the absence of large-scale data representing our target case, our approach is to start by understanding such mistakes from real data and then provide a methodology for their simulation, thus creating synthetic data to support the training of performance assessment models. The main goals of this paper are: a) to propose a taxonomy of performance mistakes, specifically apt for simulating or reproducing/recreating them on mistake-free MIDI performances, and b) to provide a pipeline for creating synthetic datasets based on the former. We incorporate prior research in related contexts to facilitate the understanding of common mistake behaviours. Then, we design a hierarchical mistake taxonomy to categorize two real-world datasets capturing relevant piano performance contexts. Finally, we discuss our approach with 3 music teachers through a listening test and subsequent discussions.