Transcribing video file directly instead of its path

[ilyasbendev]

Hello everyone,
I'm working with whisper actually, my goal is to transcribe a video file.
It's ok when i enter the path of file as an input but its not working when i'm dealing with files directly.

Its ok when i'm dealing with audios, this is the code that i wrote (for the audio)

import io
import librosa
import whisper
model = whisper.load_model("base")
buf = io.BytesIO(file.read())
data, sr = librosa.load(buf)
if sr != 16000:
        data = librosa.resample(data, orig_sr=sr, target_sr=16000)
    
model.transcribe(data)

Does anyone have an idea about working with videos also ? (i tried to extract audio from the video and after that execute the code above, but its not working

[threeal]

I've tried using the decode_audio function from faster_whisper to transcribe audio from a WebM file, and it worked. I'm not 100% sure if it's going to work with other video formats, but one thing is for sure: the decode_audio function uses FFmpeg and the av library to load audio files. You can still see the implementation of that function in case it doesn't work with other video formats.

from faster_whisper.audio import decode_audio

data = decode_audio(file)
model.transcribe(data)

[ilyasbendev]

thank you @threeal for your response.
Yes the idea is to manipulate FFmpeg to transform video to audio.