Screen Shot 2026-03-02 at 4.18.40 PM.png
- 444.29 KB
(2003x1640)
So I've been working on a project to create a singing voice synthesizer I'm basing it on the description in Jordi Bonada's PhD thesis "Voice Processing and Synthesis by Performance Sampling and Spectral Models". After a lot of trouble with getting TWM f0 estimation to work, I've finally gotten to implementing MFPA (Maximally Flat Phase Alignment". And amazingly, it seems to have worked first try. Compare my results: https://i.ibb.co/dsvgv0fd/Screen-Shot-2026-03-02-at-3-54-48-PM.png To the results in the study: https://i.ibb.co/C3fjdWVd/Screen-Shot-2026-03-02-at-3-55-09-PM.png
>>13207 I base my work on resources I have read, along with my own research. Main works cited are: "Voice Processing And Synthesis By Performance Sampling And Spectral Models" by Jordy Bonada "Understanding Digital Signal Processing", Third Edition, by Richard G. Lyons (RIP) "The Science Of The Singing Voice" by Johan Sundberg In addition to these, I have read many dozens of other papers and theses. Also, I have done my own research. Although I am still working on it, and a lot of it has yet to be tested, I think I may have made some improvements in areas. A big reason for how I have been able to achieve this was actually because I was so quick to cast doubt on my own work. Originally, I thought I had come up with a brilliant innovation. One night, from looking at figures, I actually had a little bit of doubt. Instead of just brushing it aside, I spent hours obsessing over minute details in figures. Eventually, I re-read some things and then I came to the conclusion that my algorithm was based on a fundamental misunderstanding of how the voice system works (although it would actually work some types of sound sources; for example a theramin). I think about a week later, I was back with a new idea. Long story short, this was also wrong. I spent a good portion of a day experimenting and seeing it all break down, which on the other hand strengthened my understanding. As I went to bed, feeling a bit devastated. I began thinking about ideal trains of voice pulses. This led me to come up with a rough idea that the harmonics of a series of overlapping pulses are exactly equal to a sampling of the spectrum of a single pulse. This made me feel a lot better. The next day, I wrote down a proof in LaTeX. A few days later, I came up with a third iteration of the solution to this particular problem. A few days later, I came up with a better idea. A few weeks later, and I came up with an even better idea based on the same principle. Since then, it has stayed mostly the same, although with a few minor tweaks. I think it should work now, although it is still not implemented. This is an example of just one of many things I have worked on within this project. >To find mistakes in your code, for example. I've been hearing a lot from people that LLMs can find the sort of bugs people miss for decades. I very recently caved and decided to run an LLM on most of the codebase. I posed not as myself but as an external reviewer. Needless to say, it found some issues. Many were "pedantic", but this was actually kind of what I wanted to find. There were some false detections. On the other hand, there were some real bugs I thought were false detections at first. Anyway, I should note that every issue that was found were basically minor programming mistakes, but no issues with the program design itself or the algorithms used. Anyway, what I did was I compiled a list of all the issues it found and began looking through them. In the process of fixing the issues, I actually found several more issues, which I also fixed. >Or to explain stuff to you. Like when instead of actually reading the documentation you just ask an LLM.down Not really applicable, since the main library does not have any dependencies. The GUI toolkit uses FLTK, but FLTK has pretty good documentation, which I've read a good amount of.
>>13208 Jordi Bonada* I need to sleep
Screenshot 2026-10-10 at 11.59.19â¯PM.png
- 528.19 KB
(1836x1556)
I have implemented an improvement to the laryngograph onset detection procedure. In typical laryngograph signals, like most signals, there is high-frequency noise. However, in addition to this high-frequency noise, there is also an abundance of low-frequency noise, usually much more than the high-frequency noise. This is due shifting of the neck and laryngograph. The standard DSP solution to this problem would be to use a highpass filter. However, the frequency of this noise, and thus the required transition band, are so low that it would render a standard single-stage FIR highpass filter impractical. The complete procedure I have implemented is follows: 1. The signal is lowpass-filtered (to filter out the high-frequency noise). 2. Peaks (which correspond to glottal closure instants) are detected and interpolated using quadratic interpolation. The topographical prominence is also calculated for each peak, and peaks below a minimum prominence are discarded. 3. In between each adjacent pair of peaks, the minimum valley is found (also using quadratic interpolation). 4. An envelope is created using the natural cubic spline algorithm from this set of minimum valleys. 5. This envelope is subtracted from the original signal. 6. The signal is filtered again, replacing the old filtered signal. 7. Peaks are detected again, replacing the old set of peaks. 8. These peaks are grouped into regions using two heuristics: distance between subsequent peaks and change in subsequent distances. 9. Regions with only one peak are discarded. Another approach would be to use two-stage downsampling. Then, it could be further lowpass-filtered to obtain fine-control (the new sample rate is chosen to be efficient, not necessarily precisely what is desired). These values could then be used in the same interpolation procedure; alternatively, they can be upsampled back to the original sampling rate (also using a two-stage process for efficiency reasons) and this signal subtracted from the laryngograph signal. This approach is more correct in theory. On the other hand, this assumes a perfectly periodic signal. In reality, the laryngograph pulses vary. Furthermore, they are most definitely not period around the start and end of voiced regions. This aperiodicity will distort any filtering operation. On the other hand, the valleys in between pulse generally vary much less than the signal overall. Two stage downsampling is also less efficient than the spline approach.
In previous articles, I have discussed sources of error within the Wide-Band Harmonic Sinusoidal Modeling framework. Although I have not implemented most of the proposed improvements yet, I have derived closed-form solutions for several sources of error and have also provided theoretic justifications for the proposed solutions. Notably though, one source of error - named in the original paper - has still gone unaddressed. This is the intrinsic error that results from the actual vocal tract filter varying continuously, whereas in the model we have assumed that it remains constant over given intervals. In the original WBHSM framework, these intervals were single pitch periods that were presumed to correspond to a sum of an infinite number of identical voice pulses. In my last proposed improvement on WBHSM, which I called Radiated Voice Pulse Modeling, I proposed solutions, in the form of corrective terms, to the errors that resulted from actual voice pulses not all being identical. However, each pulse was stilled modeled as having a constant vocal tract filter. Actually, theoretically it got worse since we were now considering the pulses whole radiated duration (up to a reasonable cut off point) as the pulse instead of a single period, although in practice the added error would not be significant because the proposed implementation only adds error terms that are in proportion to the differences between voice pulses. This problem would seem unresolvable within WBHSM framework, since it is based on frequency-domain modeling, which is based on linear time-invariant filters with defined z-transfer function. Since the actual vocal tract filter varies continuously, it is not time-invariant and thus does not have a z-transfer function nor a frequency response. Thus my proposed solution involves hybridizing the WBHSM technique with another technique that can model time-variant filters, namely Linear Predictive Coding. Linear Predictive Coding has a well-known deficiency, which is that it is an all-pole filter that does not model z-plane zeros. This would seem to negate much of the benefit of using it, but this is actually not true. Disregarding the non-linear aspect for a moment, if we think of the z-transfer function of an infinite impulse response filter, we can appreciate that it can be decomposed into two separate filters: An finite impulse response filter that models the zero, and an all-pole infinite response filter that models the poles. From this angle, inverse filtering just the poles would leave of us with the voice source convolved with an FIR filter that corresponds to the zeros. Since the number of taps in that filter is directly related to the number of zeros, its size can be presumed to be small and thus the convoluted pulses can also be reasonably presumed to not overlap. Furthermore, an all-pole filter has a practical benefit in the deconvolution process, which is that it is not susceptible to numerical stability issues, at least within floating-point arithmetic. One important issue is where the vocal tract filter's estimation is to be sourced from. There are basically two options: Either within LPC or within WBHSM. In the latter case, there is a problem. The current vocal tract model I am using is called Excitation plus Resonance. This model models each resonance as a parallel component of the vocal tract filter. On the other hand, LPC presumes a cascade configuration. One may think to simply modify the EpR model to be cascade instead, but this is not as trivial as it may seem. The procedure I have used for resonance estimation, which I have called Formant Region Iterative Refinement (the first iteration of which was detailed here: https://queuesevenm.wordpress.com/2026/06/02/recreating-the-excitation-plus-resonance-model/), is fundamentally based around adjusting each resonance according to its local surroundings and that inter-resonance effects are minimal. A cascade configuration breaks the second requirement. (continued in next post)
(continued from previous post) One option would be to somehow modify the LPC inverse filtering process to be amenable to a parameter configuration instead. However, I am not sure how to do this, or whether it is even possible (perhaps a good place to start would be to frame the problem as a matrix equation?). On the other hand, another option would be to create a cascade filter from the estimated parallel filter. This could be done by first starting with just the first resonance. This is the same in both cases. Then we go on to add the second one. Before adding it, we calculate the current value in the cascade filter at the next resonance's center frequency, we then scale its amplitude by the reciprocal of this value. This new filter will not be exactly the same, but it should be reasonably close. Actually, it may be a better model of the vocal tract. Perhaps also the FRIR algorithm could be used to refine it, even in the cascade case, since it should have a very low error? The final procedure then, would be to run one iteration of the WBHSM algorithm, to estimate the EpR parameters. Then, for each instant an EpR model has been estimated, only the poles are extracted and are then converted to a cascade filter. Next, the inverse filtering is performed. For each inverse filtering frame (which can be as small as a single frame), the closest vocal tract filter estimates are interpolated using, for example, linear or spline interpolation. Next, the z-transfer function denominator of the resultant filter is expanded using the standard procedure to obtain the LPC coefficient, which are finally used to do the inverse filtering itself. The final step is the second pass of WBHSM, over the inversecfiltered signal. Since it has been inverse filtered, there should be no interference between voice pulses. An additional potential benefit is the harmonic envelope no longer contains resonances, which may improve the estimation of the source curve and zeros, although the same effect could also be achieved in the traditional EpR estimation by using two passes. Finally, a combined EpR model is created from the resonance plus the source and residual model obtained in the last step. This procedure makes one significant presumption, which is that most of the error of WBHSM is concentrated in the non-resonance part of the EpR model. This would seem to be a reasonable assumption to make, given that EpR is fitted using multiple harmonics, and that it could furthermore be filtered across multiple frame using techniques such as lowpass filtering, moving averages, or exponential moving averages. However, I do not yet have empirical evidence for this. One possible case wherein the presumption would not hold would be if the error changes slowly over frequency, for example. Actually, a type of error with precisely this fingerprint does exist; it results from the fundamental frequency changing. However, I have already proposed a solution to it, within the RVPM framework: https://listserv.cuit.columbia.edu/scripts/wa.exe?A2=MUSIC-DSP;31999a11.2608A&S=
only the silliest will survive 
have you returned to continue the Fight? 
good post
i feel useless cuz i'm dum while the bot actually can find the necessary info for me and also help me immensely with troubleshooting
