Feature Request: Add BitNet Notebook

[ekurniaw]

Add bitnet-b1.58-2B-4T notebook based on CVS-169357 using OpenVINO nightly build (openvinotoolkit/openvino#31467) planning for OpenVINO 2025.4.

[sbalandi]

Hi, PR with the addition of the model #3133

[jgespino]

Thanks @sbalandi! @ekurniaw Looks like PR has merged. Could you please verify?

[ekaakurniawan]

@jgespino , I tested successfully using ARL-S CPU. ARL-S NPU throws error. Both ARL-S iGPU and B580 dGPU only outputs '!' characters. Please find details below. Will it be supported on GPU and NPU as well?

Ubuntu: 24.04
OpenVINO: 2025.4 rc2
GPU Driver (Intel Compute Runtime): 25.44.36015.5 and 25.40.35563.4
NPU Driver: 1.24.0

ARL-S CPU

Loading model from bitnet-b1.58-2B-4T/FP16

Input text: What is OpenVINO?
 
OpenVINO is an open-source software development kit (SDK) that provides a set of tools and libraries for developing and deploying machine learning models on various hardware platforms. It is designed to be a cross-platform solution that can run on multiple operating systems, including Windows, Linux, and macOS.

OpenVINO supports a wide range of hardware platforms, including CPUs, GPUs, FPGAs, and specialized hardware accelerators. It also provides support for various machine learning frameworks, such as TensorFlow, PyTorch, and Caffe.

The OpenVINO SDK includes a set of pre-built models and tools for tasks such as image classification,

ARL-S NPU

RuntimeError                              Traceback (most recent call last)
Cell In[17], line 8
      3 import sys
      5 print(f"Loading model from {model_dir}\n")
----> 8 pipe = ov_genai.LLMPipeline(str(model_dir), device.value)
     10 generation_config = ov_genai.GenerationConfig()
     11 generation_config.max_new_tokens = 128

RuntimeError: Exception from src/inference/src/cpp/core.cpp:114:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/intel_npu/src/plugin/npuw/partitioning/partitioning.cpp:1393:
Unable to handle type opset1::Constant __module.model.layers.9.self_attn.q_proj/ov_ext::bit_linear/Constant[0]:u2[]

ARL-S iGPU

Loading model from bitnet-b1.58-2B-4T/FP16

Input text: What is OpenVINO?
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

B580 dGPU

Loading model from bitnet-b1.58-2B-4T/FP16

Input text: What is OpenVINO?
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[ekurniaw]

CPU is supported. Will close for now.