Whisper model is unable to segment sentences after LoRA finetune

[bardenthenry]

I have about 90,000 records of data from Common Voice, and approximately 10,000 records from our own recorded meetings. I performed finetuning on Whisper Large-V2 using LoRA, and accelerated it using the ct2-transformers-converter. However, the finetuned Whisper model is unable to segment sentences. How can I adjust to resolve this problem?

[casic]

Hi, what hardware is needed for fine-tune large-v2 ?? I have tried with 3 Tesla T4 16GB , but only small model running. Medium and large-v2 was out of memory :(. Please give minimal configuration to fine tune whisper large-v2. Thanks На пн, 16.10.2023 г. в 5:15 ч. bardenthenry ***@***.***> написа:

…

I have about 90,000 records of data from Common Voice, and approximately 10,000 records from our own recorded meetings. I performed finetuning on Whisper Large-V2 using LoRA, and accelerated it using the ct2-transformers-converter. However, the finetuned Whisper model is unable to segment sentences. How can I adjust to resolve this problem? — Reply to this email directly, view it on GitHub <#1714>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYTSUONDZ2ZRZAZ2AVF2F3X7SKCZAVCNFSM6AAAAAA6BORLFSVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZVG42DANRVHE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[bardenthenry]

Hi @casic.
My hardware is just one RTX 4090, AMD 7950X and 32G DRAM.