A Cut-off Result or Distortion Sound

[opkaraman]

First of all, thank you for the project.

In Turkish, some sentences aren’t being generated fully. I’m not sure if it’s a coincidence, but it cuts off exactly at the 10-second mark. When I generate the same sentence a few times, it generates it fully, but most of the time it only generates part of it.

It sometimes fixes itself when I change the reference audio. But other errors occur. For example, a sound distortion at the beginning of the audio.

I think these issues stem from the reference audio. How can I fix this?

The reference files are 10 seconds long.

---

Text: "Uygarlık tarihinde insan, toplum ve doğa etkileşiminden doğan geleneksel sosyal sorunların yanı sıra özellikle Batı’da sanayileşmeyle birlikte ortaya çıkan yeni sosyal sorunları çözmek için mesleki ve bilimsel bir gereksinim olarak var olan ve gelişen sosyal çalışma günümüzde kendi içinde uzmanlaşmış bir çok alana ve alt dallara ayrılmıştır."

Reference file 1: Sound distortion at the begining:

0.wav

Reference file 2: Cut off at the 10-second:

1.wav

[a710128]

Hi, thanks for the feedback and the audio files.

For the sound distortion: This might happen if you're using "ultimate cloning" mode with a reference audio that doesn't end cleanly. Also, a bug in older versions (before 2.0.2/VoxCPM) could cause this. Are you on the latest version?

For the 10-second cutoff: That's a very strange issue, I haven't seen it before. I'm not sure what could be causing it. I wonder if anyone else has run into this?

Appreciate the report!

[madey83]

#213

?

[a710128]

I just tested with 0.wav several times and didn't encounter the cutoff problem.

Could you please provide more details about your inputs and the settings you were using when it happened?

[madey83]

I just tested with 0.wav several times and didn't encounter the cutoff problem.

Could you please provide more details about your inputs and the settings you were using when it happened?

Hello,

If you haven't got that issue maybe would be better to provide details about how did you done test and not get issues. This would be more informative to every one..

[a710128]

If you haven't got that issue maybe would be better to provide details about how did you done test and not get issues. This would be more informative to every one..

Open the demo on HuggingFace, upload 0.wav as the reference audio, and then enter the following content in the target text:

Uygarlık tarihinde insan, toplum ve doğa etkileşiminden doğan geleneksel sosyal sorunların yanı sıra özellikle Batı’da sanayileşmeyle birlikte ortaya çıkan yeni sosyal sorunları çözmek için mesleki ve bilimsel bir gereksinim olarak var olan ve gelişen sosyal çalışma günümüzde kendi içinde uzmanlaşmış bir çok alana ve alt dallara ayrılmıştır.

Click the generate button. Several results were generated, and the truncation issue at 10 seconds was not observed.

[madey83]

Ok. Hugging face uses what settings by default?

Cfg 2.0
Steps 10

Or something else?

I will try demo with my an issue in Polish text and reference file.

[a710128]

Ok. Hugging face uses what settings by default?

Cfg 2.0 Steps 10

Or something else?

I will try demo with my an issue in Polish text and reference file.

Yes, that's how I did it. If you want to resolve your issue, it would be better if we discussed it in #213 .

[opkaraman]

I just tested with 0.wav several times and didn't encounter the cutoff problem.

Could you please provide more details about your inputs and the settings you were using when it happened?

Thanks for quick reply.

Here is my inference codes:

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained("openbmb/VoxCPM2", load_denoiser=False)

text = "Uygarlık tarihinde insan, toplum ve doğa etkileşiminden doğan geleneksel sosyal sorunların yanı sıra özellikle Batı’da sanayileşmeyle birlikte ortaya çıkan yeni sosyal sorunları çözmek için mesleki ve bilimsel bir gereksinim olarak var olan ve gelişen sosyal çalışma günümüzde kendi içinde uzmanlaşmış bir çok alana ve alt dallara ayrılmıştır."

prompt_text = "Temel bakım hizmetleri. Ünite 1. Temel bakımın planlanması. Giriş. İnsanlar artık daha uzun yaşamakta, doğum oranları azalmakta ve sonuçta yaşlı nüfus sayı ve oranı gün geçtikçe artmaktadır. Dünya Sağlık Örgütü geriatrik yaş grubunu 65 yaş ve üzeri olarak tanımlanmıştır."

wav = model.generate(
    text=text,
    prompt_wav_path="test_google.wav",
    prompt_text=prompt_text,
    reference_wav_path="test_google.wav",
    cfg_value=2.0,
    inference_timesteps=10,
)

sf.write("generated_audio.wav", wav, model.tts_model.sample_rate)

Here is my reference file:

test_google.wav

I just tried it again. It generated the audio correctly a few times. Then it cut off exactly at the 10-second mark again. Whenever I generate audio with different sentences one after another, one of them always cuts off at the 10-second mark.

I’m using the latest version of VoxCPM2; I installed it yesterday via pip.

I’ll pay attention to the issue you mentioned regarding the sound distortion—specifically, the reference audio cutting off before it fully ends. I was using some reference audio exactly as you described. That must be the main cause of the audio clipping.

#213

?

I read your post as well. My problem is a bit different, but similar. In my case, the audio cuts off not at the end, but exactly at the 10-second mark. I usually expect the audio to last between 15 and 20 seconds, but it cuts off at the 10-second mark (and sometimes as early as the 9-second mark).