Using the Whisper Python package without internet

[mribeirodantas]

The current whisper Python package always downloads the pre-trained model, which requires an internet connection. It would be cool if we could download the pre-trained model once and use it afterward as many times as we wanted. Does anyone know how to do it?

[jongwook]

Once your model is downloaded under ~/.cache/whisper/, you can pass the downloaded model file's path when calling load_model(), instead of model names like "large", then the code will attempt load the model from the given path:

whisper/whisper/__init__.py

Lines 103 to 104 in 0b5dcfd

	elif os.path.isfile(name):
	checkpoint_file = open(name, "rb").read() if in_memory else name

[bandaider]

I get the distinct impression, however, that Whisper will still try to make a connection to the Internet-based model repo, even if the selected model already exists in the MODEL_ROOT. Can anyone confirm/deny?

[mribeirodantas]

This is great! Thans, @jongwook !

[jwnacnud]

You can use it with no internet connection. Jeffrey Duncan

…

On Tue, Apr 4, 2023 at 9:02 AM bandaider ***@***.***> wrote: I get the distinct impression, however, that Whisper will still try to make a connection to the Internet-based model repo, even if the selected model already exists in the MODEL_ROOT. Can anyone confirm/deny? — Reply to this email directly, view it on GitHub <#718 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGW5A6A7J2BKIGVUZDCPHTW7QZXRANCNFSM6AAAAAATC4DWOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[joezuntz]

I had to set the environment variable TIKTOKEN_CACHE_DIR, run once online to download the files to that dir, and then I could run offline.

[bandaider]

I found that if you specify not the name of the model, but the actual filepath to an already downloaded model, then Whisper will not attempt to make an Internet connection. It will, however, do so, even if the model you have requested is located in the ROOT_DIR. You can look at the code and see this for yourself.

[mstyp]

...Ok, so its possible. But how, exactly? Could somone please provide an example that I can copy paste, because I cant figure out how to do it. Connecting to the internet even once is not an option, the device these are running on are offline only.

I found that if you specify not the name of the model, but the actual filepath to an already downloaded model, then Whisper will not attempt to make an Internet connection

ok @bandaider, but specify how, exactly? I've tried all of the following and it fails every time.

whisper transcribeme.mkv --model tiny.en --model_dir ~/Documents/softwarez/wizpar/ --language en --verbose True --output_format srt
whisper transcribeme.mkv --model ~/Documents/softwarez/wizpar/tiny.en.pt --language en --verbose True --output_format srt
whisper transcribeme.mkv --model_dir ~/Documents/softwarez/wizpar/tiny.en.pt --language en --verbose True --output_format srt
whisper transcribeme.mkv --model_dir ~/Documents/softwarez/wizpar/ --language en --verbose True --output_format srt

I've tried a bunch of other permutations I cant remember, but no matter what I try it either gives me an error like "error: argument --model: invalid choice:" or an http error.

[bandaider]

@mstyp For clarification, I was able to force Whisper to use the model provided without an attempt to download (or verify the file) through the Python API. I can't say that I tried to do so from the command-line, but one would think they would act the same.

It seems that what you are doing is correct so perhaps the behavior is different from the command-line?

[mstyp]

Hrm. Well, I made a file called wiz.py with the following based on the github readme:

import whisper

model = whisper.load_model("~/Documents/softwarez/wizpar/tiny.en.pt")
result = model.transcribe("audio.mp3")

print(result["text"])

Then ran python3 wiz.py but still got an http connection error. specificially:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /gpt-2/encodings/main/vocab.bpe (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fe62a122790>: Failed to establish a new connection: [Errno -2] Name or service not known'))

I've not done as joezuntz suggested and "run once online to download the files to that dir" since I cant run it online once, but i dont know if that matters.

[zakenobi]

Having the exact same issue, did you manage solve this by any chance ?

[jengbou]

If you do not already have /tmp/data-gym-cache stored somewhere that can be accessed by your offline device, then you will need to at least run your script once on a device that has internet connection, which will download two files into /tmp/data-gym-cache folder, and copy over the data-gym-cache folder to your offline device. You might want to save the folder in a more persistent place and create an auto copy mechanism (e.g., using rc-local.service) to make sure the relevant files are in /tmp/data-gym-cache whenever rebooting the offline device.

[nicholasgcotton]

I have a guide for how to setup and use Whisper offline https://github.com/nicholasgcotton/WhisperDO

It's EASIER to do the setup part online and then just USE it offline, but if necessary you can get pip to create offline install packages and move them over without the target machine ever being on the internet.