Skip to content

Conversation

@JohnLeFeng
Copy link

During test below sample which listed on README.md, I noticed the process down suddenly and it fix by add execution device when compile model, core.compile_model.

import numpy as np
from transformers import AutoTokenizer
from openvino import compile_model, save_model
from openvino_tokenizers import convert_tokenizer

hf_tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
ov_tokenizer = convert_tokenizer(hf_tokenizer)

compiled_tokenzier = compile_model(ov_tokenizer, "CPU")
text_input = ["Test string"]

hf_output = hf_tokenizer(text_input, return_tensors="np")
ov_output = compiled_tokenzier(text_input)

for output_name in hf_output:
    print(f"OpenVINO {output_name} = {ov_output[output_name]}")
    print(f"HuggingFace {output_name} = {hf_output[output_name]}")
# OpenVINO input_ids = [[ 101 3231 5164  102]]
# HuggingFace input_ids = [[ 101 3231 5164  102]]
# OpenVINO token_type_ids = [[0 0 0 0]]
# HuggingFace token_type_ids = [[0 0 0 0]]
# OpenVINO attention_mask = [[1 1 1 1]]
# HuggingFace attention_mask = [[1 1 1 1]]

# save tokenizer for later use
save_model(ov_tokenizer, "openvino_tokenizer.xml")

loaded_tokenizer = compile_model("openvino_tokenizer.xml", "CPU")
loaded_ov_output = loaded_tokenizer(text_input)
for output_name in hf_output:
    assert np.all(loaded_ov_output[output_name] == ov_output[output_name])

@apaniukov apaniukov enabled auto-merge December 12, 2025 14:24
@Wovchena
Copy link
Contributor

I suggest reporting the bug to AUTO

@JohnLeFeng
Copy link
Author

JohnLeFeng commented Dec 12, 2025

@Wovchena and @apaniukov After tracing source of AUTO, I notice the issue is from my machine as my NPU driver is YB and it cause AUTO plugin and OV cannot work well and then cause the program down suddenly without error trace. Here is a story, CVS-178215, for AUTO to handle YB case.

@apaniukov apaniukov disabled auto-merge December 12, 2025 16:24
@apaniukov apaniukov closed this Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants