RefineGAN

[noobmaster777]

Greetings! First thank you for this wonderful program you made. Seems like the original RVC is no longer maintained. So far, I think this is the best alternative. However! I have a problem.... first, my setup.. I have 12 minutes of audio in 3 wav files with my voice. When I use HiFi-Gan, and train it for about 100-300 epochs, it works, but the voice isn't really like my voice (any tips to improve this?) So I decided to use RefineGAN instead (highest audio fidelity) and for pretrained model type: None (cant use default). Using the same wav files, i trained with around 250 epochs. It succeeds but when I test the voice, it is like whispering and robotic? Perhaps im doing something wrong? Or instead of None as pretrained model, perhaps I should use Custom? But how do I set that up? Any help would be greatly appreciated! Thanks in advance. Warm regards!

[BlankFX1]

Using RefineGAN isn't a good idea to use at all if you don't use it with a pretrained model. It would require you to train your model from absolute zero which would require you to have way more training samples and way more computational power (or month of time for the training).

Just stick to HiFi-GAN and check what went wrong. Is your training material good enough? Did you train it properly?
100-300 Epochs alone doesn't give any clue on that.
Your best avg-gen-loss should be quite low, like 2 or 1.5. (unless someone with more experience than I says something different)
In general; the lower the better, though a too low value will degrade the models quality too.

I recently learned that the Batch Size is very important quality-wise, not only in terms of speed.
Even though my system could handle a Batch Size of over 40, this can lead to a low quality model.
It might be better to stick to something between 8 and 16. I recently prefered 14 for a Model I have around 4 hours of training data for.

[noobmaster777]

Using RefineGAN isn't a good idea to use at all if you don't use it with a pretrained model. It would require you to train your model from absolute zero which would require you to have way more training samples and way more computational power (or month of time for the training).

Just stick to HiFi-GAN and check what went wrong. Is your training material good enough? Did you train it properly? 100-300 Epochs alone doesn't give any clue on that. Your best avg-gen-loss should be quite low, like 2 or 1.5. (unless someone with more experience than I says something different) In general; the lower the better, though a too low value will degrade the models quality too.

I recently learned that the Batch Size is very important quality-wise, not only in terms of speed. Even though my system could handle a Batch Size of over 40, this can lead to a low quality model. It might be better to stick to something between 8 and 16. I recently prefered 14 for a Model I have around 4 hours of training data for.

Thanks for your response. Ive been doing Batch Size 32. Maybe that is the problem. You have 4 hours of training data? Thats quite a lot! How's the cloning and accent at that amount? Perhaps I should also increase my training data... Is it generally better to split the wav files (training data) to smaller chunks, and removing silence?

[JackismyShephard]

@noobmaster777 Ultimate RVC now has a pretrained model for RefineGAN so it might work you.

[noobmaster777]

Oh this is great. I will test it out asap then give feedback. Thanks @JackismyShephard