Skip to content

[Torch] INT2 symmetric decompression support#3971

Draft
ljaljushkin wants to merge 1 commit intoopenvinotoolkit:developfrom
ljaljushkin:nl/2bit_torch_export
Draft

[Torch] INT2 symmetric decompression support#3971
ljaljushkin wants to merge 1 commit intoopenvinotoolkit:developfrom
ljaljushkin:nl/2bit_torch_export

Conversation

@ljaljushkin
Copy link
Contributor

Changes

  • Added INT2SymmetricWeightsDecompressor in nncf/torch/quantization/layers.py:
    stores signed int2 values [-2, 1] as unsigned uint2 [0, 3] using zero point of 2;
    four uint2 values are packed per uint8 byte via pack_uint2/unpack_uint2.
  • Added pack_uint2 and unpack_uint2 functions in quantize_functions.py for
    packing/unpacking 2-bit values in uint8 containers.
  • Updated strip.py to handle INT2 decompressor during model stripping with
    StripFormat.DQ.

Reason for changes

Support INT2 (2-bit) symmetric weight compression of PyTorch models in NNCF,
enabling mixed-precision quantization (e.g., average 3-bit with some layers at 2-bit
and others at 4-bit) for LLM compression with distillation-based QAT+LoRA.

Related tickets

Tests

  • tests/torch/quantization/test_symmetric_2bit.py — 15 tests covering
    QuantizeSymmetric and QuantizeSymmetricTorch at 2-bit: scale conventions,
    level count verification, zero-maps-to-zero, gradient flow, input range formulas
  • tests/torch/function_hook/quantization/test_weights_compression.py::test_pack_uint2
    uint2 pack/unpack round-trip
  • tests/torch/function_hook/quantization/test_weights_compression.py::test_int2_symmetric_weights_decompressor
    INT2 decompressor correctness

@github-actions github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch API Public API-impacting changes labels Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API Public API-impacting changes NNCF PT Pull requests that updates NNCF PyTorch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant