We propose a novel user interface that enables control of a singing voice synthesizer at a live improvisational performance. The user first registers the lyrics of a song with the system before performance, and the system builds a probabilistic model that models the possible jumps within the lyrics. During performance, the user simultaneously inputs the lyrics of a song with the left hand using a vowel keyboard and the melodies with the right hand using a standard musical keyboard. Our system searches for a portion of the registered lyrics whose vowel sequence matches the current user input using the probabilistic model, and sends the matched lyrics to the singing voice synthesizer. The vowel input keys are mapped onto a standard musical keyboard, enabling experienced keyboard players to learn the system from a standard musical score. We examine the feasibility of the system through a series of evaluations and user studies.
Our system consists of two steps, lyric registration step and actual performance step. At the lyrics registration step, the user registers the lyrics of the songs, and the system analyzes it. At the actual per- formance step, the user simultaneously inputs the vowel sequences and melodies using a musical key- board, and the system estimates the plausible lyrics from them and synthesizes singing voice sounds.
In Japanese, a character consists of a consonant and a vowel (left). For example, ”Ka” is the combination of ”K” and ”a”, ”Su” is the combination of ”S” and ”u”. There are 5 vowels and a special vowel (n) in Japanese. We mapped these vowels on a piano keyboard (center). Because of this, in our system, the lyrics can be represented as a standard musical score as the left hand part (right).
・Kazuhiko Yamamoto, and Takeo Igarashi. 2016. LiVo: Sing a Song with a Vowel Keyboard.
New Interfaces for Musical Expression (NIME), 2015. [PDF]
@inproceedings{10.5555/2993778.2993832,
author = {Yamamoto, Kazuhiko and Igarashi, Takeo},
title = {LiVo: Sing a Song with a Vowel Keyboard},
year = {2015},
publisher = {The School of Music and the Center for Computation and Technology (CCT), Louisiana State University},
booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
pages = {205–208},
series = {NIME 2015}
}
Kazuhiko Yamamoto – www.yamo-n.org
mail: yamo_o(at)acm.org
twitter: @yamo_o