Qualcomm AI Engine Direct - GA Whisper

[shewu-quic]

Summary:

• Add the unit test for Whisper
• Support multi-method for Whisper
• Add qnn_whisper_runner to run whisper encoder-decoder model

Command:

python examples/qualcomm/oss_scripts/whisper/whisper.py -b build-android  -s <serial> -H <host> -m SM8750 --max_seq_len 1024

Performance:

• SM8750:
  • avg encoding time: 0.037 s
  • avg decoding time: 0.004 s

Accuracy:

• Word Error Rate: 0.1941964328289032

cc: @haowhsu-quic, @cccclai , @winskuo-quic

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12102

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit ff09c9a with merge base bf9cd34 ():

NEW FAILURE - The following job has failed:

• pull / unittest-editable / macos / macos-job (gh)
  backends/xnnpack/test/ops/test_gelu.py::TestGelu::test_fp16_gelu

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[shewu-quic]

@pytorchbot label "release notes: qualcomm"

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

This is great! Curious if whisper needs special handling for better perf?

[shewu-quic]

This is great! Curious if whisper needs special handling for better perf?

Currently, I’m applying these optimizations:

1. Converting linear layers to convolutional
2. Tagging quant IO to avoid Q/DQ for input/output

I believe there’s still room for further performance improvements, such as mha2sha and custom annotation for kv cache.