Automatic Video Transcription / Translation Subtitles

[dsymbol]

I created a script to effortlessly burn whisper AI generated transcription / translation subtitles into provided video using ffmpeg.

Repository: https://github.com/dsymbol/decipher

Thank you openai team, your outstanding work deserves praise.

[shn-dev-1]

This works great. One piece of feedback: subtitles start immediately in the video. Therefore, if the audio starts at 30 seconds, the subtitles will display up til that point, even if there is no dialogue. This is a little bit awkward for the user, who wonders why there are subtitles if there is no dialogue.

[Baenwort]

This also shows up if there are long pauses between talking but some music in the background. The earlier sentence stays up until someone speaks again.

Is there a way to have the subs drop off after a period if there is no dialog?

[dsymbol]

This is how whisper natively generates transcription, I'll look into fixing it however I cannot promise it can be done.

[zeitgeist8080]

Will there be a windows version for newbies, Just can't handle any command line.

[zachweinstein]

Could support for manually choosing the input language be added? Thanks.

[dsymbol]

Added! use --language

[FurkanGozukara]

I also have coded an automatic batch processor for whisper

The source code is available and it is in c#

https://www.youtube.com/watch?v=dP53wzLwqMA

[arekayway]

WARNING: Generating metadata for package whisper produced metadata for project name openai-whisper. Fix your #egg=whisper fragments.
Discarding git+https://github.com/openai/whisper.git: Requested openai-whisper from git+https://github.com/openai/whisper.git (from decipher==0.0.0) has inconsistent name: filename has 'whisper', but metadata has 'openai-whisper'
ERROR: Could not find a version that satisfies the requirement whisper (unavailable) (from decipher) (from versions: 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.9.9, 0.9.10, 0.9.11, 0.9.12, 0.9.13, 0.9.14, 0.9.15, 0.9.16, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.1.9, 1.1.10)
ERROR: No matching distribution found for whisper (unavailable)

[arekayway]

`+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 69C P0 31W / 70W | 0MiB / 15360MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
Hit:1 https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/ InRelease
Hit:2 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu focal InRelease
Ign:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu2004/x86_64 InRelease
Hit:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64 InRelease
Hit:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu2004/x86_64 Release
Hit:6 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:8 http://ppa.launchpad.net/cran/libgit2/ubuntu focal InRelease
Hit:9 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:10 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:11 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal InRelease
Hit:12 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:13 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu focal InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
28 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree
Reading state information... Done
ffmpeg is already the newest version (7:4.2.7-0ubuntu0.1).
The following package was automatically installed and is no longer required:
libnvidia-common-510
Use 'apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting git+https://github.com/dsymbol/decipher
Cloning https://github.com/dsymbol/decipher to /tmp/pip-req-build-z8za9ppj
Running command git clone --filter=blob:none --quiet https://github.com/dsymbol/decipher /tmp/pip-req-build-z8za9ppj
Resolved https://github.com/dsymbol/decipher to commit 6c50511a087d2e86844a4f9f0efd834e8743fb67
Preparing metadata (setup.py) ... done
Collecting whisper@ git+https://github.com/openai/whisper.git
Cloning https://github.com/openai/whisper.git to /tmp/pip-install-agxrjpkd/whisper_b2f177fa788f4eedbb8a6f9d111bd4f7
Running command git clone --filter=blob:none --quiet https://github.com/openai/whisper.git /tmp/pip-install-agxrjpkd/whisper_b2f177fa788f4eedbb8a6f9d111bd4f7
Resolved https://github.com/openai/whisper.git to commit 7858aa9
Preparing metadata (setup.py) ... done
WARNING: Generating metadata for package whisper produced metadata for project name openai-whisper. Fix your #egg=whisper fragments.
Discarding git+https://github.com/openai/whisper.git: Requested openai-whisper from git+https://github.com/openai/whisper.git (from decipher==0.0.0) has inconsistent name: filename has 'whisper', but metadata has 'openai-whisper'
ERROR: Could not find a version that satisfies the requirement whisper (unavailable) (from decipher) (from versions: 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.9.9, 0.9.10, 0.9.11, 0.9.12, 0.9.13, 0.9.14, 0.9.15, 0.9.16, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.1.9, 1.1.10)
ERROR: No matching distribution found for whisper (unavailable)`

[dsymbol]

Should be working now.

[FurkanGozukara]

@arekayway here a tutorial

How to do Free Speech-to-Text Transcription Better Than Google Premium API with OpenAI Whisper Model

[arekayway]

yes, but it is under windows I am interested in under google colab

[Baenwort]

You can use SubtitleEdit (github project) which has a GUI and uses whisperAI as a plugin to generate text files. There are a couple other projects that do the same. Jun 20, 2023 06:07:13 zeitgeist8080 ***@***.***>:

…

Will there be a windows version for newbies, Just can't handle any command line. — Reply to this email directly, view it on GitHub[#298 (reply in thread)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AHUQXAHS5EFHWVNITOVO2VLXMF76DANCNFSM6AAAAAARCPN56M]. You are receiving this because you commented.[Tracking image][https://github.com/notifications/beacon/AHUQXAGU7XVS2AEP7YFUPVLXMF76DA5CNFSM6AAAAAARCPN56OWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAL4GRK.gif]

[jimi202008]

I can't use --model large-v2 I need to be able to customize the model

[elvisisvan]

I created a script to effortlessly burn whisper AI generated transcription / translation subtitles into provided video using ffmpeg.

Repository: https://github.com/dsymbol/decipher

Thank you openai team, your outstanding work deserves praise.

what is the maximum length of a video can this tool translate? i'm intending to translate an 2-hour-plus video, will it do?