I can't use MPS in the latest version?

[Yang-9987]

import whisper
import torch

# 加载模型
device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
model = whisper.load_model("small", device)

# load audio and pad/trim it to fit 30 seconds
audio = whisper.load_audio("/Users/yangjianqi/Desktop/口播.mp3")
audio = whisper.pad_or_trim(audio)

mel = whisper.log_mel_spectrogram(audio).to(device)
# detect the spoken language
_, probs = model.detect_language(mel)
print(f"Detected language: {max(probs, key=probs.get)}")

# decode the audio
options = whisper.DecodingOptions(fp16 = False)
result = whisper.decode(model, mel, options)

# print the recognized text
print(result.text)

and there is error

[Yang-9987]

I trird command line to use whisper with ops whisper --device mps --fp16 False --model small 口播.mp3 and then

Traceback (most recent call last):
  File "/Users/yangjianqi/anaconda3/envs/faceScript/bin/whisper", line 8, in <module>
    sys.exit(cli())
  File "/Users/yangjianqi/anaconda3/envs/faceScript/lib/python3.8/site-packages/whisper/transcribe.py", line 458, in cli
    model = load_model(model_name, device=device, download_root=model_dir)
  File "/Users/yangjianqi/anaconda3/envs/faceScript/lib/python3.8/site-packages/whisper/__init__.py", line 156, in load_model
    return model.to(device)
  File "/Users/yangjianqi/anaconda3/envs/faceScript/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1152, in to
    return self._apply(convert)
  File "/Users/yangjianqi/anaconda3/envs/faceScript/lib/python3.8/site-packages/torch/nn/modules/module.py", line 849, in _apply
    self._buffers[key] = fn(buf)
  File "/Users/yangjianqi/anaconda3/envs/faceScript/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1150, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'SparseMPS' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::empty.memory_format' is only available for these backends: [CPU, MPS, Meta, QuantizedCPU, QuantizedMeta, MkldnnCPU, SparseCPU, SparseMeta, SparseCsrCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradMeta, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, BatchedNestedTensor, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PreDispatch, PythonDispatcher].

CPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterCPU.cpp:31357 [kernel]
MPS: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterMPS.cpp:27248 [kernel]
Meta: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterMeta.cpp:26984 [kernel]
QuantizedCPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterQuantizedCPU.cpp:944 [kernel]
QuantizedMeta: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterQuantizedMeta.cpp:105 [kernel]
MkldnnCPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterMkldnnCPU.cpp:515 [kernel]
SparseCPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterSparseCPU.cpp:1387 [kernel]
SparseMeta: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterSparseMeta.cpp:249 [kernel]
SparseCsrCPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterSparseCsrCPU.cpp:1135 [kernel]
BackendSelect: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/build/aten/src/ATen/RegisterBackendSelect.cpp:807 [kernel]
Python: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/core/PythonFallbackKernel.cpp:153 [backend fallback]
FuncTorchDynamicLayerBackMode: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/functorch/DynamicLayer.cpp:498 [backend fallback]
Functionalize: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/FunctionalizeFallbackKernel.cpp:302 [backend fallback]
Named: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
Conjugate: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/ConjugateFallback.cpp:21 [kernel]
Negative: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/native/NegateFallback.cpp:23 [kernel]
ZeroTensor: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/ZeroTensorFallback.cpp:90 [kernel]
ADInplaceOrView: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/core/VariableFallbackKernel.cpp:86 [backend fallback]
AutogradOther: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradCPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradCUDA: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradHIP: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradXLA: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradMPS: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradIPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradXPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradHPU: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradVE: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradLazy: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradMTIA: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradPrivateUse1: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradPrivateUse2: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradPrivateUse3: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradMeta: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
AutogradNestedTensor: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/VariableType_2.cpp:19039 [autograd kernel]
Tracer: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/torch/csrc/autograd/generated/TraceType_2.cpp:17346 [kernel]
AutocastCPU: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/autocast_mode.cpp:378 [backend fallback]
AutocastCUDA: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/autocast_mode.cpp:244 [backend fallback]
FuncTorchBatched: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/functorch/LegacyBatchingRegistrations.cpp:720 [backend fallback]
BatchedNestedTensor: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/functorch/LegacyBatchingRegistrations.cpp:746 [backend fallback]
FuncTorchVmapMode: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/functorch/VmapModeRegistrations.cpp:28 [backend fallback]
Batched: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/LegacyBatchingRegistrations.cpp:1075 [backend fallback]
VmapMode: fallthrough registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]
FuncTorchGradWrapper: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/functorch/TensorWrapper.cpp:203 [backend fallback]
PythonTLSSnapshot: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/core/PythonFallbackKernel.cpp:161 [backend fallback]
FuncTorchDynamicLayerFrontMode: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/functorch/DynamicLayer.cpp:494 [backend fallback]
PreDispatch: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/core/PythonFallbackKernel.cpp:165 [backend fallback]
PythonDispatcher: registered at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1701070740221/work/aten/src/ATen/core/PythonFallbackKernel.cpp:157 [backend fallback]

[glangford]

This problem is not unique to the new version, see

Add aten::empty.memory_format for SparseMPS
pytorch/pytorch#87886

[shuhongwu]

you can use this pip with apple mps: https://pypi.org/project/whisper-mps/

[IsanHu]

Hello, I tried this, it's awesome! But I found it does not support large-v3-turbo. is whisper-mps open sourced?