Remixing music using source separation algorithms to improve the musical experience of cochlear implant users

Pons Puig, Jordi; Janer Mestres, Jordi; Rode, Thilo; Nogueira, Waldo

doi:http://dx.doi.org/10.1121/1.4971424

Remixing music using source separation algorithms to improve the musical experience of cochlear implant users

License

Citation

Pons J, Janer J, Rode T, Nogueira W. Remixing music using source separation algorithms to improve the musical experience of cochlear implant users. J Acoust Soc Am. 2016;140(6):4338-49. DOI: 10.1121/1.4971424

Abstract

Music perception remains rather poor for many Cochlear Implant (CI) users due to the users' deficient pitch perception. However, comprehensible vocals and simple music structures are well perceived by many CI users. In previous studies researchers re-mixed songs to make music more enjoyable for them, favoring the preferred music elements (vocals or beat) attenuating the others. However, mixing music requires the individually recorded tracks (multitracks) which are usually not accessible. To overcome this limitation, Source Separation (SS) techniques are proposed to estimate the multitracks. These estimated multitracks are further re-mixed to create more pleasant music for CI users. However, SS may introduce undesirable audible distortions and artifacts. Experiments conducted with CI users (N = 9) and normal hearing listeners (N = 9) show that CI users can have different mixing preferences than normal hearing listeners. Moreover, it is shown that CI users' mixing preferences are user dependent. It is also shown that SS methods can be successfully used to create preferred re-mixes although distortions and artifacts are present. Finally, CI users' preferences are used to propose a benchmark that defines the maximum acceptable levels of SS distortion and artifacts for two different mixes proposed by CI users.