EMG / motion capture-based accessible music interfaces for rehabilitation

Abstract

We extended the functionality of SoundControl — a software that uses machine learning to turn sensor data into sound — to include support for the Myo sensor. This armband includes EMG-sensors, as well as an accelerometer and a gyroscope, which can be used to detect hand gestures and the spacial position of the arm. With SoundControl, the user can train the software to recognize movements and gestures of their choosing, which then triggers a sound they previously linked to it, effectively creating a custom instrument. Intending to test its potential for artistic purposes, we conducted a pilot study where we analyzed the ability of our subjects to keep in rhythm with a metronome beat. We compared their performance to similar studies where the subjects were tapping to a beat. We found that when using our program, the subjects performance only diminished slightly during the simple tasks but that the potential of our tool is limited by the problems arising when more complex play patterns are attempted. We used our results to critically examine its potential as a tool for stroke rehabilitation and concluded that it could be applied successfully to that end.