Fine-tuning Whisper Large #1247
-
|
Hi, guys, I'm trying fine-tune whisper large's performance on mandarian with huggingface transformer. Each of my sample is about 6~7mb and I'm fine tuning it with 4 v100s, even though I set batch size to 1, pytorch still throw me oom error. It also doesn't work when I trying to run it on a single 3090 24GB GPU with 1 batch size. Any suggestion to get it running:)? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
The audio and the labels should be segmented into 30s or shorter chunks, to match the training distribution. I guess it should make the memory usage low enough. Using mixed precision and gradient checkpointing may further reduce the memory usage during fine-tuning. |
Beta Was this translation helpful? Give feedback.
The audio and the labels should be segmented into 30s or shorter chunks, to match the training distribution. I guess it should make the memory usage low enough. Using mixed precision and gradient checkpointing may further reduce the memory usage during fine-tuning.