[TorchFX] `quantize_pt2e` custom quantizers support #3487

daniil-lyakhov · 2025-05-08T15:31:50Z

Changes

TorchAOAdapter is updated with new entity: the ExtendedQuantizerSetup. It contains additional info about dtypes to use in q->dq pairs
TorchFX MinMax algo backend migrated from restrained stip procedure to flexible custom quantization parameters assignment code

Reason for changes

To fully support quantization via quantize_pt2e with custom quantizers (like XNNPACKQuantizer)

Related tickets

#3231

Tests

flexible custom quantization parameters assignment is tested by tests/torch/fx/test_calculation_quantizer_params.py
Conformance test is updated with 2 new configurations:
OV_QUANTIZER_NNCF, OV_QUANTIZER_AO
conformance: post_training_quantization/682/

alexsu52 · 2025-06-07T07:35:36Z

nncf/experimental/quantization/quantizer.py

+    UINT8 = "UINT8"
+
+
+class ExtendedQuantizerSetup(ABC, SingleConfigQuantizerSetup):


This change requires offline discussion.

Discussed offline, ExtendedQuantizerSetup is replaced with ExtendedQuantizerConfig

…#3541) Splitting of huge #3487 PR: ### Changes Default `DuplicateDQPass` which does not work without torch.ao annotations is replaced with working DuplicateDQPassNoAnnotations ### Reason for changes To fix `DuplicateDQPass` ### Example ![image](https://github.com/user-attachments/assets/2f7ec0e5-e1ab-4e44-92ed-9f1fabbf13cb) ### Related tickets #3487 #3231 ### Tests TorchFX Conformance tests references are updated to test fixed DuplicateDQPassNoAnnotations pass

daniil-lyakhov · 2025-07-07T10:09:39Z

@AlexanderDokuchaev, I hope we can merge this PR without the refactoring of TensorDataType and a registry for the new ExtendedQuantizerConfig class. Please let me know ASAP if it is not possible

tests/torch2/conftest.py

src/nncf/tensor/definitions.py

AlexanderDokuchaev · 2025-07-08T07:13:04Z

src/nncf/experimental/quantization/structs.py

+        :param dest_dtype: Target integer data type for quantized values.
+        """
+        super().__init__(num_bits, mode, signedness_to_force, per_channel, narrow_range)
+        if dest_dtype not in [TensorDataType.int8, TensorDataType.uint8]:


General question, why ExtendedQuantizerConfig responsible to check type?
It's responsibilities of algorithms to use only valid type, it's just structure that in general case can contain any type.
Suggest to remove this check, check type in algorithms if need, and remove test_structs.py

Because other types are not supported, no? I don't want to check types in all other places, I want to check them at the beginning. Config with dest_type bfloat16 or int4 does not make any sense to have right now

Validation type is responsibility of algorithms, not a structure that just contains parameters.
Config will be valid with all dest_type, but quantization algorithm supports only uint8 and int8

tests/post_training/pipelines/image_classification_base.py

AlexanderDokuchaev · 2025-07-08T07:46:24Z

tests/post_training/data/ptq_reference_data.yaml

+  metric_value: 0.2429
+torchvision/vit_b_16_backend_X86_QUANTIZER_NNCF:
+  metric_value: 0.80922
+torchvision/vit_b_16_backend_X86_QUANTIZER_AO:


I will not merge tests that are running for several days, even if they are disabled by default.
post_training_quantization/681/ - still in progress after 7 days

What a reason that is takes long time?
Did you check that there is no any bugs?

AlexanderDokuchaev · 2025-07-09T09:21:06Z

src/nncf/experimental/quantization/structs.py

+        :param dest_dtype: Target integer data type for quantized values.
+        """
+        super().__init__(num_bits, mode, signedness_to_force, per_channel, narrow_range)
+        if dest_dtype not in [TensorDataType.int8, TensorDataType.uint8]:


Validation type is responsibility of algorithms, not a structure that just contains parameters.
Config will be valid with all dest_type, but quantization algorithm supports only uint8 and int8

AlexanderDokuchaev · 2025-07-09T17:14:42Z

tests/post_training/data/ptq_reference_data.yaml

+  metric_value: 0.2429
+torchvision/vit_b_16_backend_X86_QUANTIZER_NNCF:
+  metric_value: 0.80922
+torchvision/vit_b_16_backend_X86_QUANTIZER_AO:


I found the issue: validation with using PyTorch runs on all available CPU cores by default, but there's a limit set in the CI environment. As a result, validation runs slower due to CPU throttling.
Tried to fix it by set TORCH_NUM_THREADS env variable in CI.
post_training_quantization/685/

src/nncf/experimental/quantization/structs.py

src/nncf/quantization/algorithms/min_max/torch_fx_backend.py

alexsu52

LGTM

Due to vacation

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 0c628c4 to 8864b02 Compare May 8, 2025 16:03

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 8864b02 to 1fa9940 Compare June 3, 2025 13:20

github-actions bot removed NNCF Common Pull request that updates NNCF Common experimental NNCF ONNX Pull requests that updates NNCF ONNX labels Jun 3, 2025

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 1fa9940 to 8864b02 Compare June 3, 2025 13:30

github-actions bot added NNCF Common Pull request that updates NNCF Common experimental NNCF ONNX Pull requests that updates NNCF ONNX labels Jun 3, 2025

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 8864b02 to 1efa37f Compare June 3, 2025 14:52

github-actions bot removed the NNCF Common Pull request that updates NNCF Common label Jun 3, 2025

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 8814d48 to 9bcaba1 Compare June 6, 2025 18:16

daniil-lyakhov changed the title ~~Dl/fx/dont use nncf q~~ [TorchFX] quantize_pt2e custom quantizers support Jun 6, 2025

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 9bcaba1 to d506459 Compare June 6, 2025 18:28

daniil-lyakhov marked this pull request as ready for review June 6, 2025 18:28

daniil-lyakhov requested a review from a team as a code owner June 6, 2025 18:28

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from d506459 to c85ffe7 Compare June 6, 2025 18:31

alexsu52 self-requested a review June 7, 2025 07:32

alexsu52 suggested changes Jun 7, 2025

View reviewed changes

alexsu52 assigned AlexanderDokuchaev Jun 7, 2025

alexsu52 requested a review from AlexanderDokuchaev June 7, 2025 07:37

daniil-lyakhov mentioned this pull request Jun 10, 2025

[TorchFX] DuplicateDQPass is replaced by DuplicateDQPassNoAnnotations #3541

Merged

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 255f9b2 to 92ef6e2 Compare June 16, 2025 12:15

daniil-lyakhov mentioned this pull request Jul 7, 2025

[NNCF] Use CommonStatefulClassesRegistry to load the SingleConfigQuantizationPoint #3574

Open

1 task

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch 2 times, most recently from 012750c to 69796b5 Compare July 7, 2025 09:44

daniil-lyakhov requested a review from AlexanderDokuchaev July 7, 2025 09:45

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 69796b5 to 7a2c141 Compare July 7, 2025 10:07

github-actions bot removed the NNCF Common Pull request that updates NNCF Common label Jul 7, 2025

Comments

7a2c141

github-actions bot added the NNCF Common Pull request that updates NNCF Common label Jul 7, 2025

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from adaadaa to 01656d1 Compare July 7, 2025 14:29

Comments

01656d1

AlexanderDokuchaev requested changes Jul 8, 2025

View reviewed changes

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from c19a398 to 086758e Compare July 8, 2025 09:32

daniil-lyakhov requested a review from AlexanderDokuchaev July 8, 2025 09:32

daniil-lyakhov added 2 commits July 8, 2025 11:41

Comments

086758e

NNCF_TEST_REGEN_JSON is removed

34d8fa7

AlexanderDokuchaev previously requested changes Jul 9, 2025

View reviewed changes

github-actions bot removed the NNCF Common Pull request that updates NNCF Common label Jul 10, 2025

daniil-lyakhov requested a review from AlexanderDokuchaev July 10, 2025 17:03

Comments

714d77d

alexsu52 suggested changes Jul 15, 2025

View reviewed changes

src/nncf/experimental/quantization/structs.py Outdated Show resolved Hide resolved

src/nncf/experimental/quantization/structs.py Outdated Show resolved Hide resolved

daniil-lyakhov force-pushed the dl/fx/dont_use_nncf_q branch from 6b0fee0 to 45c43a7 Compare July 16, 2025 07:51

daniil-lyakhov requested a review from alexsu52 July 16, 2025 07:52

experimental/structs.py -> quantization/structs.py

45c43a7

alexsu52 reviewed Jul 22, 2025

View reviewed changes

src/nncf/quantization/algorithms/min_max/torch_fx_backend.py Outdated Show resolved Hide resolved

src/nncf/quantization/algorithms/min_max/torch_fx_backend.py Outdated Show resolved Hide resolved

Comments

37f7e05

alexsu52 approved these changes Jul 22, 2025

View reviewed changes

alexsu52 merged commit 73b39a3 into openvinotoolkit:develop Jul 22, 2025
20 checks passed

daniil-lyakhov mentioned this pull request Aug 26, 2025

[release_v2180] Release notes #3629

Merged

		UINT8 = "UINT8"


		class ExtendedQuantizerSetup(ABC, SingleConfigQuantizerSetup):

[TorchFX] quantize_pt2e custom quantizers support #3487

[TorchFX] quantize_pt2e custom quantizers support #3487

Uh oh!

Conversation

daniil-lyakhov commented May 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Reason for changes

Related tickets

Tests

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniil-lyakhov commented Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexsu52 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[TorchFX] `quantize_pt2e` custom quantizers support #3487

[TorchFX] `quantize_pt2e` custom quantizers support #3487

daniil-lyakhov commented May 8, 2025 •

edited

Loading