Mapping between dynamic markings and performed loudness: a machine learning approach

Kosta, Katerina; Ramírez, Rafael, 1966-; Bandtlow, Oscar F.; Chew, Elaine

doi:http://dx.doi.org/10.1080/17459737.2016.1193237

Mapping between dynamic markings and performed loudness: a machine learning approach

Citation

Kosta K, Ramírez R, Bandtlow OF, Chew E. Mapping between dynamic markings and performed loudness: a machine learning approach. Journal of Mathematics and Music. 2016 Aug 3;10(2):149-72. DOI: 10.1080/17459737.2016.1193237

Abstract

Loudness variation is one of the foremost tools for expressivity in music performance. Loudness is frequently notated as dynamic markings such as p (piano, meaning soft) or f (forte, meaning loud). While dynamic markings in music scores are important indicators of how music pieces should be interpreted, their meaning is less straightforward than it may seem, and depends highly on the context in which they appear. In this article, we investigate the relationship between dynamic markings in the score and performed loudness by applying machine learning techniques – decision trees, support vector machines, artificial neural networks, and a k-nearest neighbor method – to the prediction of loudness levels corresponding to dynamic markings, and to the classification of dynamic markings given loudness values. The methods are applied to 44 recordings of performances of Chopin's Mazurkas, each by 8 pianists. The results show that loudness values and markings can be predicted relatively well when trained across recordings of the same piece, but fail dismally when trained across the pianist's recordings of other pieces, demonstrating that score features may trump individual style when modeling loudness choices. Evidence suggests that all the features chosen for the task are relevant, and analysis of the results reveals the forms (such as the return of the theme) and structures (such as dynamic-marking repetitions) that influence the predictability of loudness and markings. Modeling of loudness trends in expressive performance appears to be a delicate matter, and sometimes loudness expression can be a matter of the performer's idiosyncracy.