Speed up using torch.compile()?

[Anjum48]

Has anyone experienced any performance gains using torch.compile() in PyTorch 2.0?

At the moment I'm doing this but not seeing any gains (RTX 3090):

model = whisper.load_model("base")
model_compiled = torch.compile(model, backend="inductor")
result = model_compiled.transcribe("audio.mp3", fp16=True)

Curious to see if anyone has looked into where else Whisper could be accelerated

Edit: I think the correct way of doing this is:

backend = "inductor"
model = whisper.load_model("base")
model.encoder = torch.compile(model.encoder, backend=backend)
model.decoder = torch.compile(model.decoder, backend=backend)

I'm seeing some errors right now but I'll try to see if I can find a backend that works

[Anjum48]

It seems like hooks are not fully functioning at this moment (link)

The issue is that the kv_cache is not being properly set for attn in ResidualAttentionBlock when the forward hooks are used. Weirdly it is being set for cross_attn. The missing bits cause an error as the incorrect offset is returned meaning the positional_embedding has one of its dims = 0

[cyberluke]

@Anjum48 HuggingFace transformer solved this in their Whisper by replacing kv_cache here: https://app.semanticdiff.com/gh/huggingface/transformers/pull/30949/overview

[msaroufim]

Right now torch.compile is not doing anything since it assumes you're compiling the forward() function

You need to do this instead

model.transcribe = torch.compile(model.transcribe, backend="inductor")

Then you should see a whole bunch of logs, there's still tons of small issues (mostly minor source code changes to whisper) preventing this from being performant but we'll take a look cc @malfet

[Artyom17]

Has anyone succeeded in this endeavor? Getting multiple "unsupported" errors while compiling the transcribe:

[2023-07-25 18:23:13,494] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /usr/lib/python3.10/contextlib.py
[2023-07-25 18:23:13,494] torch._dynamo.eval_frame: [DEBUG] skipping __enter__ /usr/lib/python3.10/contextlib.py
[2023-07-25 18:23:13,494] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /usr/lib/python3.10/contextlib.py
[2023-07-25 18:23:13,494] torch._dynamo.eval_frame: [DEBUG] skipping __enter__ /usr/lib/python3.10/contextlib.py
[2023-07-25 18:23:13,494] torch._dynamo.eval_frame: [DEBUG] skipping enable_dynamic /home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py
[2023-07-25 18:23:13,498] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing transcribe
[2023-07-25 18:23:13,498] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/transcribe.py:109
[2023-07-25 18:23:13,498] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_DEREF decode_options []
[2023-07-25 18:23:13,498] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR get [ConstDictVariable()]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST fp16 [GetAttrVariable(ConstDictVariable(), get)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST True [GetAttrVariable(ConstDictVariable(), get), ConstantVariable(str)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [GetAttrVariable(ConstDictVariable(), get), ConstantVariable(str), ConstantVariable(bool)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 18 [ConstantVariable(bool)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR float16 [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 22 [ConstantVariable(dtype)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST dtype [ConstantVariable(dtype)]
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/transcribe.py:110
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_DEREF model []
[2023-07-25 18:23:13,499] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [NNModuleVariable()]
[2023-07-25 18:23:13,500] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object device at 0x7fde447fdf20, file "/home/user/whisper/whisper/model.py", line 266> 
 268           0 LOAD_GLOBAL              0 (next)
              2 LOAD_FAST                0 (self)
              4 LOAD_METHOD              1 (parameters)
              6 CALL_METHOD              0
              8 CALL_FUNCTION            1
             10 LOAD_ATTR                2 (device)
             12 RETURN_VALUE
 

[2023-07-25 18:23:13,500] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/model.py:268
[2023-07-25 18:23:13,500] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL next []
[2023-07-25 18:23:13,500] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(next)]
[2023-07-25 18:23:13,500] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR parameters [BuiltinVariable(next), NNModuleVariable()]
[2023-07-25 18:23:13,500] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [BuiltinVariable(next), UserMethodVariable(<function Module.parameters at 0x7fdea8b8d6c0>, NNModuleVariable())]
[2023-07-25 18:23:14,163] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(next), ListIteratorVariable()]
[2023-07-25 18:23:14,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [TensorVariable()]
[2023-07-25 18:23:14,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TorchVariable(cuda:0)]
[2023-07-25 18:23:14,164] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object device at 0x7fde447fdf20, file "/home/user/whisper/whisper/model.py", line 266>
[2023-07-25 18:23:14,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [TorchVariable(cuda:0)]
[2023-07-25 18:23:14,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [TorchVariable(cuda:0), TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST cpu [TorchVariable(cuda:0), TorchVariable(<class 'torch.device'>)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TorchVariable(cuda:0), TorchVariable(<class 'torch.device'>), ConstantVariable(str)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP == [TorchVariable(cuda:0), ConstantVariable(device)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 86 [ConstantVariable(bool)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/transcribe.py:117
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST dtype []
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [ConstantVariable(dtype)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR float32 [ConstantVariable(dtype), TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP == [ConstantVariable(dtype), ConstantVariable(dtype)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 104 [ConstantVariable(bool)]
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/transcribe.py:121
[2023-07-25 18:23:14,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL log_mel_spectrogram []
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [UserFunctionVariable()]
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL N_SAMPLES [UserFunctionVariable(), TensorVariable()]
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('padding',) [UserFunctionVariable(), TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [UserFunctionVariable(), TensorVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object log_mel_spectrogram at 0x7fdf2dcec920, file "/home/user/whisper/whisper/audio.py", line 110> 
 138           0 LOAD_GLOBAL              0 (torch)
              2 LOAD_METHOD              1 (is_tensor)
              4 LOAD_FAST                0 (audio)
              6 CALL_METHOD              1
              8 POP_JUMP_IF_TRUE        19 (to 38)

139          10 LOAD_GLOBAL              2 (isinstance)
             12 LOAD_FAST                0 (audio)
             14 LOAD_GLOBAL              3 (str)
             16 CALL_FUNCTION            2
             18 POP_JUMP_IF_FALSE       14 (to 28)

140          20 LOAD_GLOBAL              4 (load_audio)
             22 LOAD_FAST                0 (audio)
             24 CALL_FUNCTION            1
             26 STORE_FAST               0 (audio)

141     >>   28 LOAD_GLOBAL              0 (torch)
             30 LOAD_METHOD              5 (from_numpy)
             32 LOAD_FAST                0 (audio)
             34 CALL_METHOD              1
             36 STORE_FAST               0 (audio)

143     >>   38 LOAD_FAST                3 (device)
             40 LOAD_CONST               1 (None)
             42 IS_OP                    1
             44 POP_JUMP_IF_FALSE       28 (to 56)

144          46 LOAD_FAST                0 (audio)
             48 LOAD_METHOD              6 (to)
             50 LOAD_FAST                3 (device)
             52 CALL_METHOD              1
             54 STORE_FAST               0 (audio)

145     >>   56 LOAD_FAST                2 (padding)
             58 LOAD_CONST               2 (0)
             60 COMPARE_OP               4 (>)
             62 POP_JUMP_IF_FALSE       40 (to 80)

146          64 LOAD_GLOBAL              7 (F)
             66 LOAD_METHOD              8 (pad)
             68 LOAD_FAST                0 (audio)
             70 LOAD_CONST               2 (0)
             72 LOAD_FAST                2 (padding)
             74 BUILD_TUPLE              2
             76 CALL_METHOD              2
             78 STORE_FAST               0 (audio)

147     >>   80 LOAD_GLOBAL              0 (torch)
             82 LOAD_METHOD              9 (hann_window)
             84 LOAD_GLOBAL             10 (N_FFT)
             86 CALL_METHOD              1
             88 LOAD_METHOD              6 (to)
             90 LOAD_FAST                0 (audio)
             92 LOAD_ATTR               11 (device)
             94 CALL_METHOD              1
             96 STORE_FAST               4 (window)

148          98 LOAD_GLOBAL              0 (torch)
            100 LOAD_ATTR               12 (stft)
            102 LOAD_FAST                0 (audio)
            104 LOAD_GLOBAL             10 (N_FFT)
            106 LOAD_GLOBAL             13 (HOP_LENGTH)
            108 LOAD_FAST                4 (window)
            110 LOAD_CONST               3 (True)
            112 LOAD_CONST               4 (('window', 'return_complex'))
            114 CALL_FUNCTION_KW         5
            116 STORE_FAST               5 (stft)

149         118 LOAD_FAST                5 (stft)
            120 LOAD_CONST               5 (Ellipsis)
            122 LOAD_CONST               1 (None)
            124 LOAD_CONST               6 (-1)
            126 BUILD_SLICE              2
            128 BUILD_TUPLE              2
            130 BINARY_SUBSCR
            132 LOAD_METHOD             14 (abs)
            134 CALL_METHOD              0
            136 LOAD_CONST               7 (2)
            138 BINARY_POWER
            140 STORE_FAST               6 (magnitudes)

151         142 LOAD_GLOBAL             15 (mel_filters)
            144 LOAD_FAST                0 (audio)
            146 LOAD_ATTR               11 (device)
            148 LOAD_FAST                1 (n_mels)
            150 CALL_FUNCTION            2
            152 STORE_FAST               7 (filters)

152         154 LOAD_FAST                7 (filters)
            156 LOAD_FAST                6 (magnitudes)
            158 BINARY_MATRIX_MULTIPLY
            160 STORE_FAST               8 (mel_spec)

154         162 LOAD_GLOBAL              0 (torch)
            164 LOAD_ATTR               16 (clamp)
            166 LOAD_FAST                8 (mel_spec)
            168 LOAD_CONST               8 (1e-10)
            170 LOAD_CONST               9 (('min',))
            172 CALL_FUNCTION_KW         2
            174 LOAD_METHOD             17 (log10)
            176 CALL_METHOD              0
            178 STORE_FAST               9 (log_spec)

155         180 LOAD_GLOBAL              0 (torch)
            182 LOAD_METHOD             18 (maximum)
            184 LOAD_FAST                9 (log_spec)
            186 LOAD_FAST                9 (log_spec)
            188 LOAD_METHOD             19 (max)
            190 CALL_METHOD              0
            192 LOAD_CONST              10 (8.0)
            194 BINARY_SUBTRACT
            196 CALL_METHOD              2
            198 STORE_FAST               9 (log_spec)

156         200 LOAD_FAST                9 (log_spec)
            202 LOAD_CONST              11 (4.0)
            204 BINARY_ADD
            206 LOAD_CONST              11 (4.0)
            208 BINARY_TRUE_DIVIDE
            210 STORE_FAST               9 (log_spec)

157         212 LOAD_FAST                9 (log_spec)
            214 RETURN_VALUE
 

[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:138
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:14,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR is_tensor [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [TorchVariable(<function is_tensor at 0x7fdef8bc6170>)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TorchVariable(<function is_tensor at 0x7fdef8bc6170>), TensorVariable()]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 38 [ConstantVariable(bool)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:143
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST device []
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE IS_OP 1 [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ConstantVariable(bool)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:145
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST padding []
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [ConstantVariable(int)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP > [ConstantVariable(int), ConstantVariable(int)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 80 [ConstantVariable(bool)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:146
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL F []
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR pad [TorchVariable(<module 'torch.nn.functional' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/nn/functional.py'>)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [TorchVariable(<built-in function pad>)]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in function pad>), TensorVariable()]
[2023-07-25 18:23:14,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST padding [TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int)]
[2023-07-25 18:23:14,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in function pad>), TensorVariable(), TupleVariable()]
[2023-07-25 18:23:14,169] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST audio [TensorVariable()]
[2023-07-25 18:23:14,169] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:147
[2023-07-25 18:23:14,169] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:14,169] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR hann_window [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,169] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL N_FFT [TorchVariable(<built-in method hann_window of type object at 0x7fdef80a6540>)]
[2023-07-25 18:23:14,169] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TorchVariable(<built-in method hann_window of type object at 0x7fdef80a6540>), ConstantVariable(int)]
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR to [TensorVariable()]
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [GetAttrVariable(TensorVariable(), to)]
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [GetAttrVariable(TensorVariable(), to), TensorVariable()]
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), to), TorchVariable(cpu)]
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST window [TensorVariable()]
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:148
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:14,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR stft [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [TorchVariable(<function stft at 0x7fde46f72b90>)]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL N_FFT [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable()]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL HOP_LENGTH [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST window [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int)]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST True [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), TensorVariable()]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('window', 'return_complex') [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), TensorVariable(), ConstantVariable(bool)]
[2023-07-25 18:23:14,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 5 [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), TensorVariable(), ConstantVariable(bool), ConstantVariable(tuple)]
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST stft [TensorVariable()]
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:149
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST stft []
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST Ellipsis [TensorVariable()]
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), ConstantVariable(ellipsis)]
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TensorVariable(), ConstantVariable(ellipsis), ConstantVariable(NoneType)]
[2023-07-25 18:23:14,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TensorVariable(), ConstantVariable(ellipsis), ConstantVariable(NoneType), ConstantVariable(int)]
[2023-07-25 18:23:14,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TensorVariable(), ConstantVariable(ellipsis), SliceVariable()]
[2023-07-25 18:23:14,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TensorVariable(), TupleVariable()]
[2023-07-25 18:23:14,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR abs [TensorVariable()]
[2023-07-25 18:23:14,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), abs)]
[2023-07-25 18:23:14,177] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [TensorVariable()]
[2023-07-25 18:23:14,177] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_POWER None [TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST magnitudes [TensorVariable()]
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:151
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL mel_filters []
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [UserDefinedObjectVariable(_lru_cache_wrapper)]
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [UserDefinedObjectVariable(_lru_cache_wrapper), TensorVariable()]
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST n_mels [UserDefinedObjectVariable(_lru_cache_wrapper), TorchVariable(cpu)]
[2023-07-25 18:23:14,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [UserDefinedObjectVariable(_lru_cache_wrapper), TorchVariable(cpu), ConstantVariable(int)]
[2023-07-25 18:23:14,179] torch._dynamo.symbolic_convert: [DEBUG] FAILED INLINING <code object log_mel_spectrogram at 0x7fdf2dcec920, file "/home/user/whisper/whisper/audio.py", line 110>
[2023-07-25 18:23:14,180] torch._dynamo.output_graph: [DEBUG] restore_graphstate: removed 7 nodes
[2023-07-25 18:23:14,180] torch._dynamo.symbolic_convert: [DEBUG] break_graph_if_unsupported triggered compile
Traceback (most recent call last):
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
    return inner_fn(self, inst)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1014, in CALL_FUNCTION_KW
    self.call_function(fn, args, kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 259, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 92, in call_function
    return tx.inline_user_function_return(
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in inline_user_function_return
    result = InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1806, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1862, in inline_call_
    tracer.run()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
    and self.step()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
    getattr(self, inst.opname)(inst)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
    return inner_fn(self, inst)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 965, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 264, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 230, in call_function
    unimplemented(f"call_function {self} {args} {kwargs}")
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 71, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function UserDefinedObjectVariable(_lru_cache_wrapper) [TorchVariable(cpu), ConstantVariable(int)] {}
[2023-07-25 18:23:14,181] torch._dynamo.output_graph: [DEBUG] restore_graphstate: removed 0 nodes
[2023-07-25 18:23:14,181] torch._dynamo.output_graph: [DEBUG] COMPILING GRAPH due to GraphCompileReason(reason='call_function UserDefinedObjectVariable(_lru_cache_wrapper) [TorchVariable(cpu), ConstantVariable(int)] {}', user_stack=[<FrameSummary file /home/user/whisper/whisper/transcribe.py, line 121 in transcribe>, <FrameSummary file /home/user/whisper/whisper/audio.py, line 151 in log_mel_spectrogram>])
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing log_mel_spectrogram
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:138
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR is_tensor [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [TorchVariable(<function is_tensor at 0x7fdef8bc6170>)]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TorchVariable(<function is_tensor at 0x7fdef8bc6170>), TensorVariable()]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 38 [ConstantVariable(bool)]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:143
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST device []
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE IS_OP 1 [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ConstantVariable(bool)]
[2023-07-25 18:23:14,199] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:145
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST padding []
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [ConstantVariable(int)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP > [ConstantVariable(int), ConstantVariable(int)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 80 [ConstantVariable(bool)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:146
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL F []
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR pad [TorchVariable(<module 'torch.nn.functional' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/nn/functional.py'>)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [TorchVariable(<built-in function pad>)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in function pad>), TensorVariable()]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST padding [TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int)]
[2023-07-25 18:23:14,200] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in function pad>), TensorVariable(), TupleVariable()]
[2023-07-25 18:23:14,201] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST audio [TensorVariable()]
[2023-07-25 18:23:14,201] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:147
[2023-07-25 18:23:14,201] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:14,201] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR hann_window [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,202] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL N_FFT [TorchVariable(<built-in method hann_window of type object at 0x7fdef80a6540>)]
[2023-07-25 18:23:14,202] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TorchVariable(<built-in method hann_window of type object at 0x7fdef80a6540>), ConstantVariable(int)]
[2023-07-25 18:23:14,203] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR to [TensorVariable()]
[2023-07-25 18:23:14,203] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [GetAttrVariable(TensorVariable(), to)]
[2023-07-25 18:23:14,203] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [GetAttrVariable(TensorVariable(), to), TensorVariable()]
[2023-07-25 18:23:14,203] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), to), TorchVariable(cpu)]
[2023-07-25 18:23:14,203] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST window [TensorVariable()]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:148
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR stft [TorchVariable(<module 'torch' from '/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py'>)]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [TorchVariable(<function stft at 0x7fde46f72b90>)]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL N_FFT [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable()]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL HOP_LENGTH [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST window [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int)]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST True [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), TensorVariable()]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('window', 'return_complex') [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), TensorVariable(), ConstantVariable(bool)]
[2023-07-25 18:23:14,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 5 [TorchVariable(<function stft at 0x7fde46f72b90>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), TensorVariable(), ConstantVariable(bool), ConstantVariable(tuple)]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST stft [TensorVariable()]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:149
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST stft []
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST Ellipsis [TensorVariable()]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), ConstantVariable(ellipsis)]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TensorVariable(), ConstantVariable(ellipsis), ConstantVariable(NoneType)]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TensorVariable(), ConstantVariable(ellipsis), ConstantVariable(NoneType), ConstantVariable(int)]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TensorVariable(), ConstantVariable(ellipsis), SliceVariable()]
[2023-07-25 18:23:14,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TensorVariable(), TupleVariable()]
[2023-07-25 18:23:14,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR abs [TensorVariable()]
[2023-07-25 18:23:14,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), abs)]
[2023-07-25 18:23:14,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [TensorVariable()]
[2023-07-25 18:23:14,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_POWER None [TensorVariable(), ConstantVariable(int)]
[2023-07-25 18:23:14,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST magnitudes [TensorVariable()]
[2023-07-25 18:23:14,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /home/user/whisper/whisper/audio.py:151
[2023-07-25 18:23:14,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL mel_filters []
[2023-07-25 18:23:14,209] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST audio [UserDefinedObjectVariable(_lru_cache_wrapper)]
[2023-07-25 18:23:14,209] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [UserDefinedObjectVariable(_lru_cache_wrapper), TensorVariable()]
[2023-07-25 18:23:14,209] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST n_mels [UserDefinedObjectVariable(_lru_cache_wrapper), TorchVariable(cpu)]
[2023-07-25 18:23:14,209] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [UserDefinedObjectVariable(_lru_cache_wrapper), TorchVariable(cpu), ConstantVariable(int)]
[2023-07-25 18:23:14,209] torch._dynamo.symbolic_convert: [DEBUG] break_graph_if_unsupported triggered compile
Traceback (most recent call last):
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 342, in wrapper
    return inner_fn(self, inst)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 965, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 474, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 264, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 230, in call_function
    unimplemented(f"call_function {self} {args} {kwargs}")
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 71, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function UserDefinedObjectVariable(_lru_cache_wrapper) [TorchVariable(cpu), ConstantVariable(int)] {}
[2023-07-25 18:23:14,209] torch._dynamo.output_graph: [DEBUG] restore_graphstate: removed 0 nodes
[2023-07-25 18:23:14,209] torch._dynamo.output_graph: [DEBUG] COMPILING GRAPH due to GraphCompileReason(reason='call_function UserDefinedObjectVariable(_lru_cache_wrapper) [TorchVariable(cpu), ConstantVariable(int)] {}', user_stack=[<FrameSummary file /home/user/whisper/whisper/audio.py, line 151 in log_mel_spectrogram>])
[2023-07-25 18:23:14,210] torch._dynamo.output_graph: [INFO] Step 2: calling compiler function debug_wrapper
First Audio is 5.119916666666667 mins.
<class 'numpy.ndarray'>
<class 'numpy.float64'>
['tiny.en', 'tiny', 'base.en', 'base', 'small.en', 'small', 'medium.en', 'medium', 'large-v1', 'large-v2', 'large']
[2023-07-25 18:23:15,161] torch._inductor.compile_fx: [INFO] Step 3: torchinductor compiling FORWARDS graph 0
[2023-07-25 18:23:15,200] torch._inductor.graph: [INFO] Creating implicit fallback for:
  target: aten.hann_window.default
  args[0]: 400
  kwargs: {'device': device(type='cpu'), 'pin_memory': False}
[2023-07-25 18:23:15,203] torch._inductor.graph: [INFO] Using FallbackKernel: torch.ops.aten.hann_window.default
[2023-07-25 18:23:15,204] torch._inductor.graph: [INFO] Using FallbackKernel: aten.reflection_pad1d
[2023-07-25 18:23:15,223] torch._inductor.graph: [INFO] Using FallbackKernel: aten._fft_r2c
[2023-07-25 18:23:16,776] torch._inductor.debug: [WARNING] model__0_inference_0 debug trace: /home/user/whisper/torch_compile_debug/run_2023_07_25_18_23_13_493950-pid_6424/aot_torchinductor/model__0_inference_0.0
Traceback (most recent call last):
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 670, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.fake_example_inputs())
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py", line 1055, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/__init__.py", line 1390, in __call__
    return compile_fx(model_, inputs_, config_patches=self.config)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 455, in compile_fx
    return aot_autograd(
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/backends/common.py", line 48, in compiler_fn
    cg = aot_module_simplified(gm, example_inputs, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 2822, in aot_module_simplified
    compiled_fn = create_aot_dispatcher_function(
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 2515, in create_aot_dispatcher_function
    compiled_fn = compiler_fn(flat_fn, fake_flat_args, aot_config)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1715, in aot_wrapper_dedupe
    return compiler_fn(flat_fn, leaf_flat_args, aot_config)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_functorch/aot_autograd.py", line 1328, in aot_dispatch_base
    compiled_fw = aot_config.fw_compiler(fw_module, flat_args_with_views_handled)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 430, in fw_compiler
    return inner_compile(
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/debug_utils.py", line 595, in debug_wrapper
    compiled_fn = compiler_fn(gm, example_inputs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/debug.py", line 239, in inner
    return fn(*args, **kwargs)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 177, in compile_fx_inner
    compiled_fn = graph.compile_to_fn()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/graph.py", line 586, in compile_to_fn
    return self.compile_to_module().call
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/graph.py", line 571, in compile_to_module
    code = self.codegen()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/graph.py", line 522, in codegen
    self.scheduler.codegen()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 1177, in codegen
    self.flush()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 1095, in flush
    backend.flush()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/codegen/cpp.py", line 1975, in flush
    self.kernel_group.codegen_define_and_call(V.graph.wrapper_code)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/codegen/cpp.py", line 2004, in codegen_define_and_call
    arg_defs, call_args, arg_types = self.args.cpp_argdefs()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_inductor/codegen/common.py", line 330, in cpp_argdefs
    cpp_dtype = DTYPE_TO_CPP[dtype]
KeyError: torch.complex64

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/whisper/./test.py", line 22, in <module>
    _ = opt_trans(torch.from_numpy(audio_array))
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 209, in _fn
    return fn(*args, **kwargs)
  File "/home/user/whisper/whisper/transcribe.py", line 121, in transcribe
    mel = log_mel_spectrogram(audio, padding=N_SAMPLES)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 337, in catch_errors
    return callback(frame, cache_size, hooks)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 404, in _convert_frame
    result = inner_convert(frame, cache_size, hooks)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 104, in _fn
    return fn(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 262, in _convert_frame_assert
    return _compile(
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 324, in _compile
    out_code = transform_code_object(code, transform)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 445, in transform_code_object
    transformations(instructions, code_options)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 311, in transform
    tracer.run()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1726, in run
    super().run()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 576, in run
    and self.step()
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 540, in step
    getattr(self, inst.opname)(inst)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 372, in wrapper
    self.output.compile_subgraph(self, reason=reason)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 541, in compile_subgraph
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 588, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 163, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/user/whisper/.venv/lib/python3.10/site-packages/torch/_dynamo/output_graph.py", line 675, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: debug_wrapper raised KeyError: torch.complex64

Set torch._dynamo.config.verbose=True for more information


You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True

[glangford]

Curious to see if anyone has looked into where else Whisper could be accelerated

Try the recently announced turbo model -

• `turbo` model release #2363