Covert error

[Hui-zju]

I tried to do model conversion in windows and encountered different errors in both attempts. The errors are reported as below, how can I fix these problems?

First：
(openvino_env) C:\Users\dell\chatglm3.openvino>python convert.py --model_id F:/LLM/chatglm3-6b-modelscope/chatglm3-6b --precision int4 --output F:/chatglm3-6b-ov
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino
====Exporting IR=====
Framework not specified. Using pt to export the model.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 7/7 [06:13<00:00, 53.39s/it]
Setting eos_token is not supported, use the default one.
Setting pad_token is not supported, use the default one.
Setting unk_token is not supported, use the default one.
Setting eos_token is not supported, use the default one.
Setting pad_token is not supported, use the default one.
Setting unk_token is not supported, use the default one.
Using the export variant default. Available variants are:
- default: The default ONNX variant.
Using framework PyTorch: 2.2.2+cpu
WARNING:root:Cannot apply model.to_bettertransformer because of the exception:
The model type chatglm is not yet supported to be used with BetterTransformer. Feel free to open an issue at https://github.com/huggingface/optimum/issues if you would like this model type to be supported. Currently supported models are: dict_keys(['albert', 'bark', 'bart', 'bert', 'bert-generation', 'blenderbot', 'bloom', 'camembert', 'blip-2', 'clip', 'codegen', 'data2vec-text', 'deit', 'distilbert', 'electra', 'ernie', 'fsmt', 'gpt2', 'gptj', 'gpt_neo', 'gpt_neox', 'hubert', 'layoutlm', 'm2m_100', 'marian', 'markuplm', 'mbart', 'opt', 'pegasus', 'rembert', 'prophetnet', 'roberta', 'roc_bert', 'roformer', 'splinter', 'tapas', 't5', 'vilt', 'vit', 'vit_mae', 'vit_msn', 'wav2vec2', 'xlm-roberta', 'yolos']).. Usage model with stateful=True may be non-effective if model does not contain torch.functional.scaled_dot_product_attention
Overriding 1 configuration item(s)
- use_cache -> True
C:\Users\dell\openvino_env\lib\site-packages\transformers\modeling_utils.py:4225: FutureWarning: _is_quantized_training_enabled is going to be deprecated in transformers 4.39.0. Please use model.hf_quantizer.is_trainable instead
warnings.warn(
C:\Users\dell\openvino_env\lib\site-packages\optimum\exporters\openvino\model_patcher.py:198: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if past_length:
WARNING:nncf:Weight compression expects a single reduction axis, but 2 given. Weight shape: (8192, 32, 2), reduction axes: (1, 2), node name: __module.transformer/aten::index/Gather. The node won't be quantized.
Searching for Mixed-Precision Configuration ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 112/112 • 0:03:55 • 0:00:00INFO:nncf:Statistics of the bitwidth distribution:
+--------------+---------------------------+-----------------------------------+
| Num bits (N) | % all parameters (layers) | % ratio-defining parameters |
| | | (layers) |
+==============+===========================+===================================+
| 8 | 28% (31 / 114) | 21% (29 / 112) |
+--------------+---------------------------+-----------------------------------+
| 4 | 72% (83 / 114) | 79% (83 / 112) |
+--------------+---------------------------+-----------------------------------+
Applying Weight Compression ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 114/114 • 0:04:48 • 0:00:00Exception ignored in: <finalize object at 0x285f445c720; dead>
Traceback (most recent call last):
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\weakref.py", line 591, in call
return info.func(*info.args, **(info.kwargs or {}))
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 859, in _cleanup
cls._rmtree(name, ignore_errors=ignore_errors)
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 750, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 620, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 846, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 750, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 601, in _rmtree_unsafe
onerror(os.scandir, path, sys.exc_info())
File "C:\Users\dell\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 598, in _rmtree_unsafe
with os.scandir(path) as scandir_it:
NotADirectoryError: [WinError 267] 目录名称无效。: 'C:\Users\dell\AppData\Local\Temp\tmpl2bqxzug\openvino_model.bin'
Configuration saved in F:\chatglm3-6b-ov\openvino_config.json
====Exporting tokenizer=====
WARNING:transformers_modules.chatglm3-6b.tokenization_chatglm:Setting eos_token is not supported, use the default one.
WARNING:transformers_modules.chatglm3-6b.tokenization_chatglm:Setting pad_token is not supported, use the default one.
WARNING:transformers_modules.chatglm3-6b.tokenization_chatglm:Setting unk_token is not supported, use the default one.

Second：
(openvino_env) C:\Users\dell\chatglm3.openvino>python convert.py --model_id F:\LLM\chatglm3-6b-modelscope\chatglm3-6b --output F:\chatglm3-6b-OV
INFO:nncf:NNCF initialized successfully. Supported frameworks detected: torch, onnx, openvino
====Exporting IR=====
Framework not specified. Using pt to export the model.
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [06:38<00:00, 56.91s/it]
Setting eos_token is not supported, use the default one.
Setting pad_token is not supported, use the default one.
Setting unk_token is not supported, use the default one.
Setting eos_token is not supported, use the default one.
Setting pad_token is not supported, use the default one.
Setting unk_token is not supported, use the default one.
Using the export variant default. Available variants are:
- default: The default ONNX variant.
Using framework PyTorch: 2.2.2+cpu
WARNING:root:Cannot apply model.to_bettertransformer because of the exception:
The model type chatglm is not yet supported to be used with BetterTransformer. Feel free to open an issue at https://github.com/huggingface/optimum/issues if you would like this model type to be supported. Currently supported models are: dict_keys(['albert', 'bark', 'bart', 'bert', 'bert-generation', 'blenderbot', 'bloom', 'camembert', 'blip-2', 'clip', 'codegen', 'data2vec-text', 'deit', 'distilbert', 'electra', 'ernie', 'fsmt', 'gpt2', 'gptj', 'gpt_neo', 'gpt_neox', 'hubert', 'layoutlm', 'm2m_100', 'marian', 'markuplm', 'mbart', 'opt', 'pegasus', 'rembert', 'prophetnet', 'roberta', 'roc_bert', 'roformer', 'splinter', 'tapas', 't5', 'vilt', 'vit', 'vit_mae', 'vit_msn', 'wav2vec2', 'xlm-roberta', 'yolos']).. Usage model with stateful=True may be non-effective if model does not contain torch.functional.scaled_dot_product_attention
Overriding 1 configuration item(s)
- use_cache -> True
C:\Users\dell\openvino_env\lib\site-packages\transformers\modeling_utils.py:4225: FutureWarning: _is_quantized_training_enabled is going to be deprecated in transformers 4.39.0. Please use model.hf_quantizer.is_trainable instead
warnings.warn(
C:\Users\dell\openvino_env\lib\site-packages\optimum\exporters\openvino\model_patcher.py:198: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if past_length:
Traceback (most recent call last):
File "C:\Users\dell\chatglm3.openvino\convert.py", line 53, in
ov_model = OVModelForCausalLM.from_pretrained(args.model_id, export=True,
File "C:\Users\dell\openvino_env\lib\site-packages\optimum\modeling_base.py", line 401, in from_pretrained
return from_pretrained_method(
File "C:\Users\dell\openvino_env\lib\site-packages\optimum\intel\openvino\modeling_decoder.py", line 268, in _from_transformers
return cls._from_pretrained(
File "C:\Users\dell\openvino_env\lib\site-packages\optimum\intel\openvino\modeling_decoder.py", line 571, in _from_pretrained
model = cls.load_model(model_cache_path, quantization_config=None if load_in_4bit else quantization_config)
File "C:\Users\dell\openvino_env\lib\site-packages\optimum\intel\openvino\modeling_base.py", line 126, in load_model
model = core.read_model(file_name) if not file_name.suffix == ".onnx" else convert_model(file_name)
File "C:\Users\dell\openvino_env\lib\site-packages\openvino\runtime\ie_api.py", line 479, in read_model
return Model(super().read_model(model))
RuntimeError: Exception from src\inference\src\cpp\core.cpp:92:
Check 'false' failed at src\frontends\common\src\frontend.cpp:54:
Converting input model
stoll argument out of range

[2681248863]

I have this problem，too

[openvino-dev-samples]

Hi sorry for late reply, is this issue still there ?
Could you help to update the dependencies with latest requirements.txt and try it again ?

[2681248863]

I have solved this problem, some errors can be ignored

Hi sorry for late reply, is this issue still there ? Could you help to update the dependencies with latest and try it again ?requirements.txt