JamSketch: improvisation support system with GA-based melody creation from user’s drawing

Abstract

Improvisation is an enjoyable form of music performance but requires advanced skills and knowledge of music because the player has to create melodies immediately during the performance. To support improvisations by people without skills or knowledge of music, we have to develop (1) a human interface that can be used without skills or knowledge of music and (2) automatic melody generation from the user’s input that may be musically abstract or incomplete. In this paper, we develop an improvisation support system based on melodic outlines, which represent the overall contour of melodies, with a function of melody generation using a genetic algorithm (GA). Once the user draws a melodic outline on the piano-roll display with the mouse or touch screen, the system immediately generates a melody using a GA with a fitness function based on the similarity to the outline, an N-gram probability, and entropy. The generated melody is performed expressively based on expression parameters calculated with an machine learning approach. The results of listening tests for comparing human performances and the system’s performances suggest that generated melodies have quality similar to performances by non-expert human performers.