Automatic video editing with in-video spoken commands

[vivekhaldar]

Use case: talking-head videos where you know while recording which parts you want to keep and which parts you flubbed and want to drop.

I embed spoken commands while recording. I say "victor kilo" after a segment I want to keep, "victor delta" after a segment I want to drop. The commands are not important, you can pick your own words.

Then I have a script that uses Whisper modified with stable-ts (https://github.com/jianfch/stable-ts) to get word-level timestamps, look for the commands, and output the final edited video.

Full code: https://gist.github.com/vivekhaldar/63764b085640bd82ebf07482f8a58cb8

Video that explains it, along with a short demo: https://youtu.be/SmWpi-jeAEU

[hlevring]

Pretty cool!

[Besam-juajua]

mark