This project just interpolates?

[Coice]

There is an old trick where you can take 16khz audio and do linear interpolation on it which will fill out the spectrum. This creates very visible "mirroring" at 8khz and 16khz if you interpolate to 48khz audio.

The code in this repo does the same, just a better version of it, but it's not actually creating any real spectrum.

Take a look at this image you'll see what I mean. In the image I provided, the letter "A" is all the same mirrored area.

Can you dig in and take a deep dive into this?

Here's a bonus image, same order, this audio happens to have a bit of noise at the start, you can clearly see the mirror in action:

[PhilipJoey]

Obvious aliasing

[ysharma3501]

@Coice @PhillipJoey This work essentially first linear resamples to 48khz and then uses a modified much smaller version of an amp block to essentially refine and create new frequencies. This is done in other upsampling work like hierspeech’s upsampler as well.

Aliasing and other problems still exist because it is simply it was trained on 100 hours of audio vs sidons 2k hours and there are some improvements the model arch can gain easily. This should come next week as training code came this week.

[DarkAlchy]

I am glad this ended up in ComfyUI has I had a lot of ancient SUNO songs to slap this on. No thanks as every single one lost the wideness of the stereo field. I was almost mono tbh. Luckily this workflow allowed A/B testing.

[ysharma3501]

@DarkAlchy Yes unfortunately right now it’s trained on mono audio and has not been trained on music at all. A checkpoint trained on music, sounds, and multilingual speech is coming this week along with some improvements in architecture.

[DarkAlchy]

@ysharma3501 I figured as much. the node had mono and stereo and there was a difference, but to wide field was pretty zapped. Loved what I saw on the spectrograph, though, as I could see where it was adding.