[ET-VK][Ops] dequantize ops skeleton test framework #11480

ahmtox · 2025-06-09T15:04:55Z

Stack from ghstack (oldest at bottom):

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation

Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

pytorch-bot · 2025-06-09T15:04:58Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11480

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 8005443 with merge base 8cfa858 ():

NEW FAILURE - The following job has failed:

Build documentation / build (buck2) / Build doc (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-06-09T15:06:39Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-09T15:10:32Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-11T17:01:46Z

This pull request was exported from Phabricator. Differential Revision: D76267021

manuelcandales · 2025-06-11T18:36:20Z

backends/vulkan/test/op_tests/dequantize_test.cpp

+//
+
+// Helper function to get the name of a ScalarType for better error messages
+std::string scalar_type_name(c10::ScalarType dtype) {


isn't this the same helper function you used for the quantized tests? if so, please refactor this into some utilities.

manuelcandales · 2025-06-11T18:36:38Z

backends/vulkan/test/op_tests/dequantize_test.cpp

+  }
+}
+
+vkcompute::vkapi::ScalarType from_at_scalartype(c10::ScalarType at_scalartype) {


same as above, refactor into common utilities

manuelcandales · 2025-06-11T18:38:01Z

backends/vulkan/test/op_tests/dequantize_test.cpp

+    at::ScalarType dtype,
+    at::ScalarType out_dtype) {
+  auto out = at::empty_like(input, out_dtype);
+  // Convert at::ScalarType to executorch::ScalarType


you use this conversion here two times, and also in the quantized tests, please refactor into a utility function

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-11T19:01:49Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-11T21:50:45Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-12T14:52:53Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-12T17:01:10Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-12T17:05:42Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-12T19:51:18Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-13T04:15:26Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-13T15:15:27Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-13T18:04:46Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Skeleton framework that is needed to build out the dequantize_per_tensor and dequantize_per_token operators based on cpu implementation Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/) [ghstack-poisoned]

facebook-github-bot · 2025-06-13T22:51:15Z

This pull request was exported from Phabricator. Differential Revision: D76267021

Pull Request resolved: pytorch/executorch#11480 # Context In this diff we plan on creating the skeleton test framework for dequantization. This is necessary as we need a reference to test our vulkan implementation of the dequantization operator against an existing cpu implementation. This test framework is heavily inspired by [sdpa_test.cpp](https://github.com/pytorch/executorch/blob/main/backends/vulkan/test/op_tests/sdpa_test.cpp). We make use of the [op_dequantize.cpp](https://github.com/pytorch/executorch/blob/main/kernels/quantized/cpu/op_dequantize.cpp) cpu implementation of the `dequantize_per_tensor`, and the `dequantize_per_token` operators. An explanation for the operator is included where the actual vulkan implementation is created in a future diff along this stack. # Changes The main thing in this difference is the creation of a new test framework `dequantize_test.cpp`, and also including it in targets.bzl such that we can properly call the test. As this is inspired by sdpa_test.cpp, we also follow a similar format. First we have forward declarations of the functions that we wish to test against (dequantize_per_tensor, and dequantize_per_token). Then we also have wrappers for the functions without context, and finally wrappers for the ATen implementations of the same operators using the `WRAP_TO_ATEN` macro. We don't need context as this is merely for testing. We also have a utility function to test the quantize arguments that will be used when actually using the vulkan implementation. This utility function is just for a sanity check. ghstack-source-id: 290376489 @exported-using-ghexport Differential Revision: [D76267021](https://our.internmc.facebook.com/intern/diff/D76267021/)

ahmtox requested a review from SS-JIA as a code owner June 9, 2025 15:04

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 9, 2025

facebook-github-bot added the fb-exported label Jun 9, 2025

manuelcandales requested changes Jun 11, 2025

View reviewed changes

ahmtox added the release notes: vulkan Changes to the Vulkan backend delegate label Jun 11, 2025

ahmtox mentioned this pull request Jun 11, 2025

[ET-VK][Ops] common test utils for converting aten types to vulkan types #11575

Merged

manuelcandales approved these changes Jun 12, 2025

View reviewed changes

facebook-github-bot merged commit 763269a into gh/ahmtox/13/base Jun 14, 2025
95 of 98 checks passed

facebook-github-bot deleted the gh/ahmtox/13/head branch June 14, 2025 03:46

facebook-github-bot temporarily deployed to cherry-pick-bot June 14, 2025 03:46 — with GitHub Actions Inactive

pytorchbot mentioned this pull request Jun 14, 2025

[ET-VK][Ops] dequantize ops skeleton test framework #11675

Closed

Uh oh!

[ET-VK][Ops] dequantize ops skeleton test framework #11480

[ET-VK][Ops] dequantize ops skeleton test framework #11480

Uh oh!

Conversation

ahmtox commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11480

❌ 1 New Failure, 1 Unrelated Failure

Uh oh!

facebook-github-bot commented Jun 9, 2025

Uh oh!

facebook-github-bot commented Jun 9, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

manuelcandales Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

manuelcandales Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

manuelcandales Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 13, 2025

Uh oh!

facebook-github-bot commented Jun 13, 2025

Uh oh!

facebook-github-bot commented Jun 13, 2025

Uh oh!

facebook-github-bot commented Jun 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ahmtox commented Jun 9, 2025 •

edited

Loading

pytorch-bot bot commented Jun 9, 2025 •

edited

Loading