Problem loading custom models on Windows

[drHuangMHT]

When supplied with a relative path:

python run_server.py --port 9090 --backend faster_whisper -fw "./models/ggml-large-v3.bin"

reports this when client connects:

// Server
INFO:root:Custom model option was provided. Switching to single model mode.
INFO:websockets.server:connection open
INFO:root:New client connected
INFO:root:Using custom model ./models/ggml-large-v3.bin
INFO:root:Using Device=cpu with precision int8
ERROR:root:Failed to load model: Repo id must be in the form 'repo_name' or 'namespace/repo_name': './models/ggml-large-v3.bin'. Use `repo_type` argument if needed.
INFO:root:Running faster_whisper backend.
INFO:root:Connection closed by client
INFO:root:Cleaning up.

// Client
[INFO]: * recording
[INFO]: Waiting for server ready ...
[INFO]: Opened connection
Message from Server: Failed to load model: ./models/ggml-large-v3.bin
[INFO]: Websocket connection closed: None: None

When supplied with a hg repo:

python run_server.py --port 9090 --backend faster_whisper -fw "Systran/faster-whisper-large-v3"

the server cannot start properly:

Traceback (most recent call last):
  File "WhisperLive\run_server.py", line 45, in <module>
    server.run(
  File "WhisperLive\whisper_live\server.py", line 381, in run
    raise ValueError(f"Custom faster_whisper model '{faster_whisper_custom_model_path}' is not a valid path.")

When supplied with an absolute path:

python run_server.py --port 9090 --backend faster_whisper -fw "D:\WhisperLive\models\ggml-large-v3.bin"

reports:

// Server
INFO:root:Custom model option was provided. Switching to single model mode.
INFO:websockets.server:connection open
INFO:root:New client connected
INFO:root:Using custom model D:\WhisperLive\models\ggml-large-v3.bin
INFO:root:Using Device=cpu with precision int8
ERROR:root:Failed to load model: Invalid model size 'D:\WhisperLive\models\ggml-large-v3.bin', expected one of: tiny.en, tiny, base.en, base, small.en, small, medium.en, medium, large-v1, large-v2, large-v3, large, distil-large-v2, distil-medium.en, distil-small.en, distil-large-v3, large-v3-turbo, turbo
INFO:root:Running faster_whisper backend.
INFO:root:Connection closed by client
INFO:root:Cleaning up.

// Client
[INFO]: * recording
[INFO]: Waiting for server ready ...
[INFO]: Opened connection
Message from Server: Failed to load model: D:\WhisperLive\models\ggml-large-v3.bin
[INFO]: Websocket connection closed: None: None

and same after replacing backslashes with forward slashes.

Client code:

from whisper_live.client import TranscriptionClient

client = TranscriptionClient(
    "localhost",
    9090,
    translate=False,
    use_vad=False,
    save_output_recording=False,
    max_clients=4,
    max_connection_time=600,
    mute_audio_playback=False
)

client()

Maybe I did something wrong, but the log looks rather confusing to me.

[makaveli10]

@drHuangMHT ggml backend is not supported in whisperlive
we support tensorrt, faster-whisper, openvino

[drHuangMHT]

@drHuangMHT ggml backend is not supported in whisperlive we support tensorrt, faster-whisper, openvino

Thank you for your response! I tried https://huggingface.co/Systran/faster-whisper-large-v3 and it still reports

INFO:root:Custom model option was provided. Switching to single model mode.
INFO:websockets.server:connection open
INFO:root:New client connected
INFO:root:Using custom model D:\WhisperLive\models\model.bin
INFO:root:Using Device=cpu with precision int8
ERROR:root:Failed to load model: Invalid model size 'D:\WhisperLive\models\model.bin', expected one of: tiny.en, tiny, base.en, base, small.en, small, medium.en, medium, large-v1, large-v2, large-v3, large, distil-large-v2, distil-medium.en, distil-small.en, distil-large-v3, large-v3-turbo, turbo
INFO:root:Running faster_whisper backend.
INFO:root:Connection closed by client
INFO:root:Cleaning up.

and by replacing backslashes gives me this

ERROR:root:Failed to load model: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'D:/WhisperLive/models/model.bin'. Use `repo_type` argument if needed.

What does "model size" actually mean? Is it the physical size of the model or the parameter size?

[klonikar]

it happens on linux too when i do pip install -e . it happens for any model downloaded from HF. You need to specify path to model at server start time but at model load time it expects model repo. This is after doing git clone. don't know about installing a release package.

[duj12]

You may try:

huggingface-cli download mobiuslabsgmbh/faster-whisper-large-v3-turbo --local-dir models/faster-whisper/large-v3-turbo --local-dir-use-symlinks False
python run_server.py --port 9090 --backend faster_whisper --max_clients 4 --max_connection_time 300 -fw .\models\faster-whisper\large-v3-turbo

use model path, not model.bin as input.