VocaListener is a singing synthesis system that automatically estimates parameters for singing synthesis from a user's singing voice with the help of song lyrics.

The overall workflow for VocaListener appears to be the following. First, an empty song block needs to be selected and the program started inside VOCALOID3 Editor through the JobPlugin menu. VocaListener pops up as its own GUI, complete with waveforms and notes, etc. Once the window is up, the user needs to load the source audio, after which VocaListener automatically analyzes the pitch. Then, the user needs to drag out areas that have singing and type in a string of lyrics that correspond to the lyrics sung. The program will then think and adjust the positioning of the notes, as well as readjusting the pitch and dynamics curves. Often times, the program’s guess for the timing of the lyrics is not correct and need to be adjusted. Other parameters may also be tweaked, including the amount of pitch expression, etc. Everytime there is a change, it’ll take some time for VocaListener to process and redo the pitch and dynamics tweaks. Once the user is satisfied with the tuning, one button closes the window and sends the note, pitch and dynamics data back to VOCALOID3 Editor. There is also the option to load and save VocaListener projects, and launching VocaListener from a track made with VocaListener should bring up the original project associated with it.
VocaListener will require a very clean audio source for it to work well, i.e. a raw audio recording with no reverb, etc. This is because it’ll attempt to match the entire waveform, and effects such as reverb will cause VocaListener to attempt to emulate that with the singing, resulting in very funny results.



How to use :Install in folder with job plugins and add lua-file in V3-editor through Manage Job-Plugins. Vocalistener plugin located in C:\Program Files (x86)\Vocaloid3FE\JobPlugins\Vocalistener\V3VocaListener.lua

run vocaloid-editor through Applocal and type notes only in hiragana