In the last post, I talked about some potential experiments I would have to do on the pitch/dynamics model for my VOCALOID2 recreation project. I had already programmed the base code for Bonada 2005 expression system, actually many months ago now. This was all in anticipation of the day I finally compare my system to samples that were actually generated by VOCALOID2. Well today I decided that day is now... and I was totally wrong about everything.
Well not everything,- I was wrong about the expression system. It was actually the Ortola 2001 expression system that was still used in VOCALOID2 seemingly, except using a different dynamics model from the Predictive Amplitude Shaping algorithm that was used in VOCALOID1.
Anyway, at first this made me even more concerned, because there was much less information about the Ortola 2001 expression system. One of the big mysteries was how the pitch was handled over the note duration. Well, as you can see in the attached image, it is very simple. In fact it is actually the simplest thing possible - it is a straight line.
Another thing that concerned me was the dynamics model, since there was no description of this at all anywhere. But luckly, it proved to be incredibly simple. Simple enough that I was able to recreate it just by fiddling around for a couple hours in Jupyter notebook.
In general, I feel like I have enough from the creators of VOCALOID that I have in a way gain an intuition for their thought process and what mathematical structures they favored. Below, I have presented in a demonstration of a partial recreation of the dynamics model and template-less portamento pitch transition model (my [orange] vs V2 [blue] portamento pictured). For an example of this thought process understanding, the first thing I tried for note dynamics decay was a formula of the form y*(e^-x - 1), which was inspired from the formula used for computing the source curve in the Excitation plus Resonance model, and it worked. Another example is the way the exponent is used in the portamento transition function, which was inspired in a way from the formula used for that same transition mentioned in the Bonada 2005 expression system patent (https://patents.google.com/patent/JP2006330615A).
import numpy as np
def synthesize_note_dynamics(note_on, duration, amplitude, sr=22050):
v = np.zeros(round((duration + note_on + 1.0) * sr))
for itr in range(len(v)):
t = itr / sr
if t >= note_on and t < note_on + 1.25:
amp_attack = np.cos(((t - note_on) / 1.25) * (np.pi / 2.0)) * 0.4 + 0.6
else:
amp_attack = 0.0
if t > note_on + 1.25:
amp_decay = 0.6 + (np.exp(-(t - (note_on + 1.25)) / (duration - 1.25)) - 1.0) * 0.175
else:
amp_decay = 0.0
if t >= note_on - 0.015 and t < note_on + 0.005:
amp_spike1 = (1.0 - abs((t - (note_on - 0.005)) / 0.01)) / 4.0
else:
amp_spike1 = 0.0
# Note: There is also a fade of about 33% that occurs near the end that lasts about 100ms. The rest of the fade is handled by an articulation from phoneme to silence. The fade is not modeled here since its time depends on the length of that articulation.
v[itr] = (amp_spike1 + amp_attack + amp_decay) * amplitude
return v
# In cents
def synthesize_portamento(start_time, duration, start_pitch, end_pitch, sr=22050):
v = np.zeros(round((duration + start_time + 1.0) * sr))
for itr in range(len(v)):
t = itr / sr
if t >= start_time and t < start_time + duration:
if end_pitch >= start_pitch:
v[itr] = start_pitch + (end_pitch - start_pitch) * (((t - start_time) / duration) ** 0.8)
else:
v[itr] = end_pitch + (start_pitch - end_pitch) * ((1.0 - (t - start_time) / duration) ** 1.5)
elif t < start_time:
v[itr] = start_pitch
else:
v[itr] = end_pitch
return v
The captcha doesn't fit either way. The quick reply form should be scrollable!
note that 
thankyou
