Commit 1233924
authored
Fix compression of first & last layers of bitnet to int8 (#3738)
### Changes
Added u2 type support in NNCF graph
### Reason for changes
int8 compression of first/last layer of bitnet fails with error:
```
raise NotImplementedError(msg)
NotImplementedError: NNCF is not yet supported OpenVINO data type: u2.
```
### Related tickets
CVS-176501
### Tests
manual, since need optimum from main branch and openvino==2025.4
and OV doesn't expose `ov.Type.u2` in Python API (requested for 2026.0
release)
```shell
$ pip install git+https://github.com/huggingface/optimum-intel.git
$ pip install --pre openvino==2025.4.0rc2 openvino-tokenizers==2025.4.0.0rc2 openvino-genai==2025.4.0.0rc2 --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
$ optimum-cli export openvino -m optimum-intel-internal-testing/tiny-random-bitnet --weight-format int8 out_dir --task=text-generation-with-past
$ optimum-cli export openvino -m microsoft/bitnet-b1.58-2B-4T --weight-format int8 out_dir --task=text-generation-with-past
```
<img width="1024" height="143" alt="image"
src="https://github.com/user-attachments/assets/ffd2b816-88d3-4a35-bc61-5048f7cc4f9b"
/>1 parent c234987 commit 1233924
1 file changed
+1
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
| 61 | + | |
61 | 62 | | |
62 | 63 | | |
63 | 64 | | |
| |||
0 commit comments