Applying deep learning techniques to estimate patterns of musical gesture

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Dalmazzo, David
• dc.contributor.author Waddell, George
• dc.contributor.author Ramírez, Rafael,1966-
• dc.date.accessioned 2021-02-15T07:51:15Z
• dc.date.available 2021-02-15T07:51:15Z
• dc.date.issued 2021
• dc.description.abstract Repetitive practice is one of the most important factors in improving the performance of motor skills. This paper focuses on the analysis and classification of forearm gestures in the context of violin playing. We recorded five experts and three students performing eight traditional classical violin bow-strokes: martelé, staccato, detaché, ricochet, legato, trémolo, collé, and col legno. To record inertial motion information, we utilized the Myo sensor, which reports a multidimensional time-series signal. We synchronized inertial motion recordings with audio data to extract the spatiotemporal dynamics of each gesture. Applying state-of-the-art deep neural networks, we implemented and compared different architectures where convolutional neural networks (CNN) models demonstrated recognition rates of 97.147%, 3DMultiHeaded_CNN models showed rates of 98.553%, and rates of 99.234% were demonstrated by CNN_LSTM models. The collected data (quaternion of the bowing arm of a violinist) contained sufficient information to distinguish the bowing techniques studied, and deep learning methods were capable of learning the movement patterns that distinguish these techniques. Each of the learning algorithms investigated (CNN, 3DMultiHeaded_CNN, and CNN_LSTM) produced high classification accuracies which supported the feasibility of training classifiers. The resulting classifiers may provide the foundation of a digital assistant to enhance musicians' time spent practicing alone, providing real-time feedback on the accuracy and consistency of their musical gestures in performance.
• dc.description.sponsorship This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
• dc.format.mimetype application/pdf
• dc.identifier.citation Dalmazzo D, Waddell G, Ramírez R. Applying deep learning techniques to estimate patterns of musical gesture. Front Psychol. 2021 Jan 5;11:575971. DOI: 10.3389/fpsyg.2020.575971
• dc.identifier.doi http://dx.doi.org/10.3389/fpsyg.2020.575971
• dc.identifier.issn 1664-1078
• dc.identifier.uri http://hdl.handle.net/10230/46473
• dc.language.iso eng
• dc.publisher Frontiers
• dc.relation.ispartof Frontiers in Psychology. 2021 Jan 5;11:575971.
• dc.relation.projectID info:eu-repo/grantAgreement/ES/1PE/TIN2013-48152-C2-2-R
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/688269
• dc.rights © 2021 Dalmazzo, Waddell and Ramírez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) (https://creativecommons.org/licenses/by/4.0/). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by/4.0/
• dc.subject.keyword Gesture recognition
• dc.subject.keyword Bow-strokes
• dc.subject.keyword Music interaction
• dc.subject.keyword CNN
• dc.subject.keyword LSTM
• dc.subject.keyword Music education
• dc.subject.keyword ConvLSTM
• dc.subject.keyword CNN_LSTM
• dc.title Applying deep learning techniques to estimate patterns of musical gesture
• dc.type info:eu-repo/semantics/article
• dc.type.version info:eu-repo/semantics/publishedVersion