Hello,
As you may know, I am working on a project relating to voice signal processing. There several goals to this project. One is creating a general voice signal processing software library; another is creating a realistic singing voice synthesizer based on said library. However, a side project of this project has been recreating the VOCALOID2 synthesis engine.
Previously, I have discussed this, but I had yet to really attempt it specifically. Recently though, I made a breakthrough that has been an impetus towards actually doing it. This breakthrough was not a scientific breakthrough, but simply something I found about. Previously, I did not attempt this because I realized I'd actually need to obtain the VOCALOID2 software to have a good chance at doing this. Previously, I had hoped that I'd be able to find clean raw audio of VOCALOID2 output online and compare it to my results. However, I quickly realized this was not enough. First, I couldn't actually find that much. Although the little I did find was quite helpful (it was the basis of my last post on pitch and dynamics models), I quickly realized I'd need a lot more, and would need agency to test specific things. Furthermore, I'd need the specific database to test.
The VOCALOID2 software has not been available for sale for over a decade. One option would be to order a physical copy, but these are hard to come by. Besides, it requires internet activation to install, which cannot work now since the activation servers have been shut down. Although some voice libraries are still available, these require the software itself as a prerequisite.
However, I found out after some research that apparently demo versions of voice libraries actually come bundled with a limited version of the actual software. It seemed unlikely though that any of these would still be available. However, after a quick search I found one that was surprisingly still available. After signing up, I was able to download it.
Indeed, it did not require online activation. However, the installer would fail to start, without giving any error message. I tried just about every trick in the book, however none worked. I even tried https://github.com/lifenjoiner/ISx as an attempt to directly extract files to install manually, but that also wouldn't work. Finally, I moved the installer out of the directory it came in, which contained invalid characters. I thought I had already tried this, but it seems I hadn't, because it worked perfectly following this.
So recently, I have been working on the VOCALOID2 recreation. The basic idea was to implement it based on papers and patents, implementing several possible variations in cases of uncertainty. The VOCALOID2 software would be used as a reference to compare the outputs to, and also to decide the few things that weren't really mentioned in papers or patents. The latter was basically just the pitch and dynamics model, which is the single most transparent thing since it can be analyzed just by analyzing the resulting synthesis audio.
Already, a database parser has been implemented in C, that parses the VOCALOID2 database format. That format seems to be known as DaisyDB, according to figures in Bonada's PhD thesis, so I have named it accordingly. Although I expected the process to be a huge hassle, it actually turned out to be surprisingly easy. Basically, just by spending a couple days in a hex editor, and cross referencing it with papers and patents, I was able to decipher pretty much the entire format. The parser can already parse whole databases without issue, and there are only a handful of fields whose use is not yet known, many of which I expect will be resolved in due time.
(continued in next post)