How to obtain Zero-shot LM & CFM?

[mukherjeesougata-eros]

As illustrated in Figure 2 of the CosyVoice 3
paper, I would like to understand how to obtain Zero-shot LM and CFM models after large-scale pretraining on my own custom dataset.

Could you please clarify which training script should be used for this purpose? The provided run.sh
script appears to initialize training from an already pretrained model, so I am unsure how to perform training from scratch or conduct large-scale pretraining on custom data.

[aluminumbox]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

[mukherjeesougata-eros]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

In the training script run.sh, may I know why is a pretrained model being used as an argument during the training stage? If the goal is to perform pretraining, my understanding is that the process should start entirely from scratch. Could you please clarify the role of the pretrained model here?

I have run the run.sh script it is giving me the following error:-

Traceback (most recent call last):                                                                                 
  File "/mnt/data0/Sougata/TTS/CosyVoice/examples/libritts/cosyvoice3/tools/extract_embedding.py", line 74, in <mod
ule>                                                                                                               
    ort_session = onnxruntime.InferenceSession(args.onnx_path, sess_options=option, providers=providers)           
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 419, in __init__                                                                   
    self._create_inference_session(providers, provider_options, disabled_optimizers)                               
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 472, in _create_inference_session                                                  
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)               
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from ../.
./../pretrained_models/CosyVoice3-0.5B/campplus.onnx failed:Load model ../../../pretrained_models/CosyVoice3-0.5B/c
ampplus.onnx failed. File doesn't exist

[mukherjeesougata-eros]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

In the training script run.sh, may I know why is a pretrained model being used as an argument during the training stage? If the goal is to perform pretraining, my understanding is that the process should start entirely from scratch. Could you please clarify the role of the pretrained model here?

I have run the run.sh script it is giving me the following error:-

Traceback (most recent call last):                                                                                 
  File "/mnt/data0/Sougata/TTS/CosyVoice/examples/libritts/cosyvoice3/tools/extract_embedding.py", line 74, in <mod
ule>                                                                                                               
    ort_session = onnxruntime.InferenceSession(args.onnx_path, sess_options=option, providers=providers)           
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 419, in __init__                                                                   
    self._create_inference_session(providers, provider_options, disabled_optimizers)                               
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 472, in _create_inference_session                                                  
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)               
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from ../.
./../pretrained_models/CosyVoice3-0.5B/campplus.onnx failed:Load model ../../../pretrained_models/CosyVoice3-0.5B/c
ampplus.onnx failed. File doesn't exist

@aluminumbox Can you kindly please reply to this?

[mukherjeesougata-eros]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

In the training script run.sh, may I know why is a pretrained model being used as an argument during the training stage? If the goal is to perform pretraining, my understanding is that the process should start entirely from scratch. Could you please clarify the role of the pretrained model here?

I have run the run.sh script it is giving me the following error:-

Traceback (most recent call last):                                                                                 
  File "/mnt/data0/Sougata/TTS/CosyVoice/examples/libritts/cosyvoice3/tools/extract_embedding.py", line 74, in <mod
ule>                                                                                                               
    ort_session = onnxruntime.InferenceSession(args.onnx_path, sess_options=option, providers=providers)           
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 419, in __init__                                                                   
    self._create_inference_session(providers, provider_options, disabled_optimizers)                               
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 472, in _create_inference_session                                                  
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)               
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from ../.
./../pretrained_models/CosyVoice3-0.5B/campplus.onnx failed:Load model ../../../pretrained_models/CosyVoice3-0.5B/c
ampplus.onnx failed. File doesn't exist

@ZhihaoDU Can you kindly please reply to this question?

[mukherjeesougata-eros]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

In the training script run.sh, may I know why is a pretrained model being used as an argument during the training stage? If the goal is to perform pretraining, my understanding is that the process should start entirely from scratch. Could you please clarify the role of the pretrained model here?
I have run the run.sh script it is giving me the following error:-

Traceback (most recent call last):                                                                                 
  File "/mnt/data0/Sougata/TTS/CosyVoice/examples/libritts/cosyvoice3/tools/extract_embedding.py", line 74, in <mod
ule>                                                                                                               
    ort_session = onnxruntime.InferenceSession(args.onnx_path, sess_options=option, providers=providers)           
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 419, in __init__                                                                   
    self._create_inference_session(providers, provider_options, disabled_optimizers)                               
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 472, in _create_inference_session                                                  
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)               
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from ../.
./../pretrained_models/CosyVoice3-0.5B/campplus.onnx failed:Load model ../../../pretrained_models/CosyVoice3-0.5B/c
ampplus.onnx failed. File doesn't exist

@aluminumbox Can you kindly please reply to this?

@aluminumbox A gentle reminder for the same.

[mukherjeesougata-eros]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

In the training script run.sh, may I know why is a pretrained model being used as an argument during the training stage? If the goal is to perform pretraining, my understanding is that the process should start entirely from scratch. Could you please clarify the role of the pretrained model here?
I have run the run.sh script it is giving me the following error:-

Traceback (most recent call last):                                                                                 
  File "/mnt/data0/Sougata/TTS/CosyVoice/examples/libritts/cosyvoice3/tools/extract_embedding.py", line 74, in <mod
ule>                                                                                                               
    ort_session = onnxruntime.InferenceSession(args.onnx_path, sess_options=option, providers=providers)           
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 419, in __init__                                                                   
    self._create_inference_session(providers, provider_options, disabled_optimizers)                               
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 472, in _create_inference_session                                                  
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)               
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from ../.
./../pretrained_models/CosyVoice3-0.5B/campplus.onnx failed:Load model ../../../pretrained_models/CosyVoice3-0.5B/c
ampplus.onnx failed. File doesn't exist

@ZhihaoDU Can you kindly please reply to this question?

@ZhihaoDU A gentle reminder for the same.

[mukherjeesougata-eros]

follow run.sh in examples/libritts, change lr from constant_lr to warmup lr

In the training script run.sh, may I know why is a pretrained model being used as an argument during the training stage? If the goal is to perform pretraining, my understanding is that the process should start entirely from scratch. Could you please clarify the role of the pretrained model here?

I have run the run.sh script it is giving me the following error:-

Traceback (most recent call last):                                                                                 
  File "/mnt/data0/Sougata/TTS/CosyVoice/examples/libritts/cosyvoice3/tools/extract_embedding.py", line 74, in <mod
ule>                                                                                                               
    ort_session = onnxruntime.InferenceSession(args.onnx_path, sess_options=option, providers=providers)           
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 419, in __init__                                                                   
    self._create_inference_session(providers, provider_options, disabled_optimizers)                               
  File "/mnt/data0/anaconda_dir/miniconda3/envs/cosyvoice/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime
_inference_collection.py", line 472, in _create_inference_session                                                  
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)               
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from ../.
./../pretrained_models/CosyVoice3-0.5B/campplus.onnx failed:Load model ../../../pretrained_models/CosyVoice3-0.5B/c
ampplus.onnx failed. File doesn't exist

@aluminumbox I would greatly appreciate it if you could kindly help clarify my doubt. If any part of my question is unclear, please let me know which specific sections need more explanation. I will then update the issue with additional details, including precise line references and relevant screenshots.