Arm Zephyr CI Test for Whisper-tiny Encoder and Decoder #13008

BujSet · 2025-07-30T18:15:33Z

Summary

Similar to #12734, this PR includes testing the whisper-tiny encoder and decoder in succession.

Test plan

Adds the whisper_tiny entry to the models matrix for the test-models-arm-zephyr trunk test.

pytorch-bot · 2025-07-30T18:15:36Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13008

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 17a38b9 with merge base bedce91 ():

NEW FAILURES - The following jobs have failed:

trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
RuntimeError: Command docker exec -t 4e5fa80a25cf6da9e51d90babc2a4b5ab29f83e8add98c748c2a63c30a79e2d3 /exec failed with exit code 128
trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
RuntimeError: Command docker exec -t ba090fbff20b636b64aa764f65f4c61014a03a7560c59ed2b58d1147ecd7ba08 /exec failed with exit code 1
trunk / test-models-arm-zephyr (whisper_tiny) / linux-job (gh)
RuntimeError: Command docker exec -t bae4e920695468ff3c09a3f810a070078051ba6ca81aa52a70cbb9449afe0b0a /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2025-07-30T18:16:27Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

BujSet · 2025-07-30T18:17:26Z

@pytorchbot label "release notes: none"

…ting fix

…mmit hash

…he transformers dependency issue

digantdesai · 2025-08-01T10:46:35Z

.github/workflows/trunk.yml

        download_arm_zephyr_sdk
        ./zephyr-sdk-0.16.0/setup.sh -c -t arm-zephyr-eabi

+        SIM_LIMIT=120


Nit

Suggested change

SIM_LIMIT=120

SIM_LIMIT_SECONDS=120

digantdesai · 2025-08-01T10:47:09Z

.github/workflows/trunk.yml

+        if [[ "${MODEL_NAME}" == "whisper_tiny" ]]; then
+          cd $ZEPHYR_PROJ_ROOT
+          # For whisper-tiny, we will use optimum to export the encoder and decoder ptes
+          #OPTIMUM_ET_COMMIT=$(cat .ci/docker/ci_commit_pins/optimum-executorch.txt)


remove or use?

digantdesai · 2025-08-01T10:48:01Z

.github/workflows/trunk.yml

+          pwd
+          ls
+          ls oa_whisper/


digantdesai · 2025-08-01T10:48:58Z

.github/workflows/trunk.yml

+        fi
+        if [[ "${MODEL_NAME}" == "whisper_tiny" ]]; then


digantdesai · 2025-08-01T10:52:00Z

.github/workflows/trunk.yml

+            exit $exit_status
+        fi
+        if [[ "${MODEL_NAME}" == "whisper_tiny" ]]; then
+            python3 build_model.py --pte-file $ZEPHYR_PROJ_ROOT/oa_whisper/decoder.pte


checking error code for encode?

digantdesai

LGTM :)

digantdesai · 2025-08-01T10:52:08Z

.github/workflows/trunk.yml

+            west build -p always -b mps3/corstone300/fvp
+
+            FVP_Corstone_SSE-300_Ethos-U55 -a build/zephyr/zephyr.elf -C mps3_board.visualisation.disable-visualisation=1 -C mps3_board.telnetterminal0.start_telnet=0 -C mps3_board.uart0.out_file='sim.out'  -C cpu0.CFGITCMSZ=15 -C cpu0.CFGDTCMSZ=15 --simlimit $SIM_LIMIT
+        fi


checking error code for decode?

github-actions · 2025-10-01T00:51:27Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 30, 2025

BujSet self-assigned this Jul 30, 2025

BujSet requested review from digantdesai and larryliu0820 July 30, 2025 18:17

pytorch-bot bot added the release notes: none Do not include this in the release notes label Jul 30, 2025

BujSet added module: ci Issues related to continuous integration ciflow/trunk module: microcontrollers For embedded MCUs like Cortex-M, or RTOS like Zephyr, does not track NPU backend like Arm Ethos. and removed release notes: none Do not include this in the release notes labels Jul 30, 2025

BujSet force-pushed the zephyr_ci_whisper_test branch 2 times, most recently from 7904a8b to 62c81b4 Compare July 31, 2025 03:13

BujSet added 10 commits July 31, 2025 04:50

Adding Zephyr CI test for whisper encoder and decoder

a21d0b3

Issues adding dependencies for optimum, trying a fix

d709657

Minor typo fix

ff05f2b

Still issues with setting optimum

a13f167

Optimum change default API, update the CLI call

3b1fb97

Unable to build optimum from scratch, copying setup from other job

8f497ea

Seems like I need some secret token to use hf transformer models, tes…

a568217

…ting fix

Realising I dont need the hugging face login

14e7ff3

Something weird with the optimum commit hash, testing a hard coded co…

e58b80a

…mmit hash

Moving the optimum call after the executorch install, think this is t…

17a38b9

…he transformers dependency issue

BujSet force-pushed the zephyr_ci_whisper_test branch from 7163311 to 17a38b9 Compare July 31, 2025 04:50

digantdesai reviewed Aug 1, 2025

View reviewed changes

.github/workflows/trunk.yml

Comment on lines +118 to +120

pwd

ls

ls oa_whisper/

Copy link

Contributor

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed?

digantdesai reviewed Aug 1, 2025

View reviewed changes

.github/workflows/trunk.yml

Comment on lines +136 to +137

fi

if [[ "${MODEL_NAME}" == "whisper_tiny" ]]; then

Copy link

Contributor

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elif

digantdesai reviewed Aug 1, 2025

View reviewed changes

github-actions bot added the stale PRs inactive for over 60 days label Oct 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Arm Zephyr CI Test for Whisper-tiny Encoder and Decoder #13008

Arm Zephyr CI Test for Whisper-tiny Encoder and Decoder #13008

Uh oh!

BujSet commented Jul 30, 2025

Uh oh!

pytorch-bot bot commented Jul 30, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 30, 2025

Uh oh!

BujSet commented Jul 30, 2025

Uh oh!

digantdesai Aug 1, 2025

Uh oh!

digantdesai Aug 1, 2025

Uh oh!

digantdesai Aug 1, 2025

Uh oh!

digantdesai Aug 1, 2025

Uh oh!

digantdesai Aug 1, 2025

Uh oh!

digantdesai left a comment

Uh oh!

digantdesai Aug 1, 2025

Uh oh!

github-actions bot commented Oct 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Arm Zephyr CI Test for Whisper-tiny Encoder and Decoder #13008

Are you sure you want to change the base?

Arm Zephyr CI Test for Whisper-tiny Encoder and Decoder #13008

Uh oh!

Conversation

BujSet commented Jul 30, 2025

Summary

Test plan

Uh oh!

pytorch-bot bot commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13008

❌ 3 New Failures

Uh oh!

github-actions bot commented Jul 30, 2025

This PR needs a release notes: label

Uh oh!

BujSet commented Jul 30, 2025

Uh oh!

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pytorch-bot bot commented Jul 30, 2025 •

edited

Loading

This PR needs a `release notes:` label