Acquisition of violin instrumental gestures using an infrared depth camera

Abstract

We present a low-cost six-degrees of freedom violin and bow pose-tracking system. The infrared and depth streams of an infrared-based Microsoft Kinect Mbox One depth camera are utilized to track the 3-dimensional position of 6 infrared markers attached on the violin and the bow. After computing the bow pose on the violin coordinate system, a number of bowing features are extracted. In order to evaluate the system’s performance, we recorded 4 bowing exercises using simultaneously our system along with a commercial two-sensor 3D tracking system based on electromagnetic field (EMF) sensing. The mean pearson coefficient values were 0.996 for the bow position, 0.889 for the bow velocity, 0.966 for the bow tilt, 0.692 for the bowbridge distance and 0.998 for the bow inclination. Compared to existing bowing-tracking solutions, the proposed solution might be appealing because of its low cost, easy setup and high performance.