Expression control in singing voice synthesis: features, approaches, evaluation, and challenges

Umbert Morist, Martí; Bonada, Jordi, 1973-; Goto, Masataka; Nakano, Tomoyasu; Sundberg, Johan

doi:http://dx.doi.org/10.1109/MSP.2015.2424572

Expression control in singing voice synthesis: features, approaches, evaluation, and challenges

Citation

Umbert M, Bonada J, Goto M, Nakano T, Sundberg J. Expression control in singing voice synthesis: features, approaches, evaluation, and challenges. IEEE Signal Process Mag. 2015;32(6):55-73. DOI: 10.1109/MSP.2015.2424572

Abstract

In the context of singing voice synthesis, expression control manipulates a set of voice features related to a particular emotion, style, or singer. Also known as performance modeling, it has been approached from different perspectives and for different purposes, and different projects have shown a wide extent of applicability. The aim of this article is to provide an overview of approaches to expression control in singing voice synthesis. We introduce some musical applications that use singing voice synthesis techniques to justify the need for an accurate control of expression. Then, expression is defined and related to speech and instrument performance modeling. Next, we present the commonly studied set of voice parameters that can change perceptual aspects of synthesized voices. After that, we provide an up-to-date classification, comparison, and description of a selection of approaches to expression control. Then, we describe how these approaches are currently evaluated and discuss the benefits of building a common evaluation framework and adopting perceptually-motivated objective measures. Finally, we discuss the challenges that we currently foresee.