Add new language tags for whisper

[amilek8]

Can whisper add tags in more than 99 languages?
Please let me know if it doesn't work, and if it does please let me know how to add a new language tag!
I made some attempts to convert new language tags into tokens by modifying the model configuration file downloaded from huggingface and its third-party libraries, but the following problem occurs when calculating the loss during training：

How do I fix it, or is this approach not feasible
RuntimeError: philox_cuda_state for an unexpected CUDA generator used during capture. In regions captured by CUDA graphs, you may only use the default CUDA RNG generator on the device that's current when capture begins. If you need a non-default (user-supplied) generator, or a generator on another device, please file an issue.

[phineas-pta]

modifying model config doesn't change anything

u need to find a way with the tokenizer, if your language is close enough to a language supported u can still use the tokenizer

otherwise try another model like wav2vec2 which support a lot more lang

[amilek8]

Thank you for your answer, another question I would like to ask is I have to fine tune different dialects of the same language so that a model can recognize different dialects of audio, their text is exactly the same are labeled in the tokenizer, do I need to create a separate language tag to represent the different dialects. Or just mix the two dialects for training

[phineas-pta]

if u only need to transcribe without identify dialects, u can just mix them, make sure to gather balanced data

[amilek8]

Thank you for your answer, it helped me a lot!

[jasgithub101]

Can whisper add tags in more than 99 languages? Please let me know if it doesn't work, and if it does please let me know how to add a new language tag! I made some attempts to convert new language tags into tokens by modifying the model configuration file downloaded from huggingface and its third-party libraries, but the following problem occurs when calculating the loss during training：

How do I fix it, or is this approach not feasible RuntimeError: philox_cuda_state for an unexpected CUDA generator used during capture. In regions captured by CUDA graphs, you may only use the default CUDA RNG generator on the device that's current when capture begins. If you need a non-default (user-supplied) generator, or a generator on another device, please file an issue.

Can you please share your code for fine tuning on new language, I'm trying to add new tokens for new language. It's been difficult for me to figure that out!

[amilek8]

If you want to add a new tag, Modify the four places of the model configuration file tokenizer.json/tokenizer_config.json/special_tokens_map.json/add_tokens.json, and then modify the model structure of the model decode for training. It can achieve the effect you want, the specific training details are still being sorted out, and the code will be published after a period of time