Skip to content

Conversation

@larryliu0820
Copy link
Contributor

This pull request introduces an end-to-end CUDA test for the Voxtral model, adds a new runtime executable for Voxtral, and makes supporting updates to the build system and utility code. The main focus is on enabling automated validation of Voxtral's CUDA export and runtime within CI, including latency measurement and output verification.

End-to-end Voxtral CUDA test integration:

  • Added a new test-voxtral-cuda-e2e job to the .github/workflows/cuda.yml CI workflow, which builds, exports, and runs the Voxtral model using CUDA, and checks for expected output and exit codes.
  • Updated the optimum-executorch commit pin in .ci/docker/ci_commit_pins/optimum-executorch.txt to ensure compatibility with the latest Voxtral export.

Voxtral runtime and build system enhancements:

  • Added a new voxtral_runner executable to backends/cuda/CMakeLists.txt for running exported Voxtral models, linking it with required CUDA and extension libraries.
  • Introduced the implementation of voxtral_runner.cpp, which loads the model, runs the main methods (audio_encoder, token_embedding, text_decoder), prints tensor summaries, and reports method and run latencies.

Utility and compatibility updates:

  • Updated dtype_to_scalar_type in backends/aoti/utils.h to support PyTorch's int64 dtype code, improving tensor type handling for Voxtral inputs.

@pytorch-bot
Copy link

pytorch-bot bot commented Oct 8, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14875

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 8, 2025
@larryliu0820 larryliu0820 marked this pull request as ready for review October 8, 2025 04:02
@larryliu0820 larryliu0820 added the release notes: desktop for desktop/laptop workstream label Oct 8, 2025
@larryliu0820 larryliu0820 merged commit 400b2a5 into main Oct 8, 2025
139 checks passed
@larryliu0820 larryliu0820 deleted the voxtral_runner branch October 8, 2025 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: desktop for desktop/laptop workstream

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants