Skip to content

Conversation

@daniil-lyakhov
Copy link
Collaborator

Changes

  • Weight compression / mixed precision transpose_a support
  • AWQ transpose_a support
  • process_statistics transpose_a param support
  • [ONNX] AWQ gemm support

Reason for changes

  • To apply WC/mixed precision/AWQ to the mamba model family

Related tickets

173277

Tests

  • tests/cross_fw/test_templates/template_test_weights_compression.py::test_mixed_precision expanded with transpose_a param to check base WC/ mixed precision algo
  • tests/cross_fw/test_templates/template_test_weights_compression.py::test_awq_scale_reference expanded with transpose_a and non_mergable_pattern to check non mergeable AWQ branch & activation transpose support
  • tests/cross_fw/test_templates/template_test_weights_compression.py::test_process_stats refactored to test act_ch_axis support in process_statistics fn
  • tests/cross_fw/test_templates/template_test_weights_compression.py::test_compression_skipped_with_transposed_activations moved to common to test ONNX/OV failing with appropriate error when unsupported trahspose_a model supplied to an algorithm

@daniil-lyakhov daniil-lyakhov requested a review from a team as a code owner December 16, 2025 17:28
@github-actions github-actions bot added NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF ONNX Pull requests that updates NNCF ONNX labels Dec 16, 2025
@daniil-lyakhov daniil-lyakhov changed the title Dl/mamba awq [WC] WC/Mixed Precision/AWQ transpose_a support Dec 16, 2025
@github-actions github-actions bot added the NNCF PT Pull requests that updates NNCF PyTorch label Dec 17, 2025
@daniil-lyakhov
Copy link
Collaborator Author

@andrey-churkin, please review updated AWQ ONNX backend

return transformed_model

def _get_activation_node_and_port(self, node: NNCFNode, nncf_graph: NNCFGraph) -> tuple[NNCFNode, int]:
def _get_activation_node_port_and_channel(self, node: NNCFNode, nncf_graph: NNCFGraph) -> tuple[NNCFNode, int]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tuple[NNCFNode, int, int]

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@daniil-lyakhov daniil-lyakhov force-pushed the dl/mamba_awq branch 2 times, most recently from 295211e to 7753663 Compare December 23, 2025 13:04

if any(weight_dim > 3 for weight_dim in all_weight_dims):
max_val = max(weight_dim for weight_dim in all_weight_dims)
msg = f"Compression with {max_val} dimentional weight is not supported"
Copy link
Contributor

@ljaljushkin ljaljushkin Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we always keep convolutions with 4D weights in backup mode
@daniil-lyakhov is it valid condition for error?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

@daniil-lyakhov daniil-lyakhov force-pushed the dl/mamba_awq branch 3 times, most recently from 9017b66 to 1312d77 Compare January 5, 2026 14:55
@daniil-lyakhov
Copy link
Collaborator Author

@andreyanufr @andrey-churkin, please take a look

@daniil-lyakhov daniil-lyakhov force-pushed the dl/mamba_awq branch 2 times, most recently from 7ee7578 to f829a24 Compare January 7, 2026 14:00
@andrey-churkin
Copy link
Contributor

@daniil-lyakhov Could you please start the Weight Compression workflow as well?

Copy link
Contributor

@andrey-churkin andrey-churkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@daniil-lyakhov
Copy link
Collaborator Author

daniil-lyakhov commented Jan 8, 2026

@daniil-lyakhov Could you please start the Weight Compression workflow as well?

Sure
https://github.com/openvinotoolkit/nncf/actions/runs/20814418255
UPD: The run is green

Co-authored-by: andreyanufr <andrey.anufriev@intel.com>
@AlexanderDokuchaev AlexanderDokuchaev merged commit 6dee0b0 into openvinotoolkit:develop Jan 8, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

NNCF ONNX Pull requests that updates NNCF ONNX NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PT Pull requests that updates NNCF PyTorch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants