Initial download huge

[jmbreuer]

I pulled the updated/latest image from ghcr.io/rishikanthc/scriberr (should be v1.2.0).

Now as it's starting it's downloading huge amounts of data, more than 15 GB so far, and the docker overlay is exceeding 50 GB.

There's a message in the console:

Downloading canary-1b-v2.nemo: 11% (667.1 MB / 5.9 GB)time=08:21:02 level=ERROR msg="Failed to initialize transcription model" model_id=canary error="failed to download Canary model: failed to download Canary model: failed to save file: context deadline exceeded"

Has the fix for #303 perchance changed behavior to download everything on startup, rather than models and such on an as-needed basis?

I'm not sure how to have a usable install of Scriberr at this point, I don't have tens of gigs of space to spare for it.

[jmbreuer]

Also, I see this:

scriberr-1  | time=07:51:02 level="INFO " msg="Initializing registered models in parallel..."

(How) can I "unregister" unnecessary models?

[jmbreuer]

(... and possibly omit all the nVidia stuff - I have an nVidia GPU, but it has too little memory to really be useful for this.)

[tomjridge]

Just adding to what @jmbreuer wrote: I'm on a MacBook Pro M4. Installed using brew. Initial run has taken about 180 minutes and counting (mostly copying onnx files apparently). The README instructions mention "several minutes". If 180+minutes is expected, the README should probably be changed. But I suspect the install is somehow doing lots of things that are unnecessary?

[rishikanthc]

@jmbreuer Unfortunately currently there's no way to unregister or opt out of downloading models. I can add this as a feature though.
Regarding GPU stuff, only the scriberr-cuda image installs GPU specific libraries. The regular image doesn't add any NVIDIA libraries.

Unfortunately transcription models are quite large in size. Here's the breakdown:
Whisper - tiny: ~1GB
Whisper - base: ~1GB
Whisper - small: ~2GB
Whisper - medium: ~5GB
Whisper - large: 10GB

NVIDIA canary - 5.9GB
NVIDIA parakeet - 2.3GB

During initial installation, only the NVIDIA canary and parakeet models are pre-downloaded and cached. The whisper models are downloaded and cached the first time you invoke a transcription using a model. Same with the PyAnnote models. So these aren't downloaded during initial startup.
So far we have only covered the model sizes themselves. The python packages required for running these models like pytorch etc are also large.

The max I can do is provide an opt-out for using NVIDIA models.. That would save you about 8-9GB of space. However, the rest of the memory overhead cannot be addressed. The limitations stem from the ecosystem by itself.. You can mitigate this a little bit by using the quantization options from the UI for whisper models. That should shave off some size.

What I can do is provide an environment variable in the compose to make it optional to download NVIDIA models. Let me know if you think that's a reasonable solution.. Or if you have other ideas am open to those as well and I'm more than happy to implement a solution if it exists.

[rishikanthc]

@tomjridge 180+ minutes is NOT EXPECTED and is NOT NORMAL. When I meant several minutes I meant more like 5-10mins max. I have a M3 Pro and it takes less than 5mins for me. So something is definitely off.. Can you try downloading the pre-compiled binary and running it instead of installing it via homebrew ?
You need to install the following:

• uv: curl -LsSf https://astral.sh/uv/install.sh | sh (https://docs.astral.sh/uv/getting-started/installation/)
• ffmpeg: brew install ffmpeg

If you want youtube video support then you need to install deno and yt-dlp as well..

I am wondering if this is something specific to M4 and homebrew..
So would be of great help to me if you could test out the pre-compiled binaries or docker on your mac and let me know how it goes.. The install process has actually been cleaned up significantly in the last release..
I think this is something to do with pytorch installations for M4.. Could you provide more logs please and describe any other behavior you noticed? Would help me debug..

[ML-CSI]

My experience was similar (downloading a huge amount) but it works and I felt kinda OK with the amount of data.
I'm using parakeet/canary most of the time and im running on CPU (x64) and have the "non-cuda docker image" running.

As my focus is German language mostly I can only use Whisper lage and the two NVIDIA models mentioned above..
My conclusion: It would be nice and convenient to "blacklist" all the others to save disk space (approx. 10GB) but it's not the end of the wold if this isnt possible ;)

[jmbreuer]

During the attempt I mentioned above, I let it run... a bit later my Internet disconnected leading to all sorts of errors in the docker console.

I shut down the container (docker compose down) which immediately freed most of those 50 GB it had been eating up.

After starting it again, it downloaded about 10-15 GB more and used about twice as much disk space, but not nearly the same amount as in the attempt before. It's running now for me with "only" 30 GB used in docker's overlay2 (and a reported size of 2.4 GB image + 12.7 GB container in docker).