新代码不能加载本地模型了吗？

[zzutligang]

我把模型下载到本地文件夹，demo的模型路径，它还是去https://huggingface.co加载模型，我是内网，根本访问到不到这个地址，之前版本都已经试验成功了的。使用最新的是最新的代码和最新的模型。

[zRzRzRzRzRzRzR]

可以啊，那个是个警告吧 4.38.1有警告，可以加载本地模型，你降级到4.37.2就没了吧

[zzutligang]

我修改了api_server.py文件，里面指定了我下载到本地的模型路径
MODEL_PATH = os.environ.get('MODEL_PATH', '/home/hfwy/models/chatglm3-6b')
EMBEDDING_PATH = os.environ.get('EMBEDDING_PATH', '/home/hfwy/models/bge-large-zh-v1.5')
然后运行pythone api_server.py
报如下错误：
(base) hfwy@chatglm:~/ChatGLM3/openai_api_demo$ python api_server.py
Could not locate the tokenization_chatglm.py inside THUDM/chatglm3-6b.
Traceback (most recent call last):
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
raise new_e
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
self._validate_conn(conn)
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
conn.connect()
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connection.py", line 616, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connection.py", line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fee28f15750>: Failed to establish a new connection: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/hfwy/miniconda/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /THUDM/chatglm3-6b/resolve/main/tokenization_chatglm.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fee28f15750>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1247, in hf_hub_download
metadata = get_hf_file_metadata(
^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1624, in get_hf_file_metadata
r = _request_wrapper(
^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 402, in _request_wrapper
response = _request_wrapper(
^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 425, in _request_wrapper
response = get_session().request(method=method, url=url, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 63, in send
return super().send(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /THUDM/chatglm3-6b/resolve/main/tokenization_chatglm.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fee28f15750>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 971068ec-c56f-426e-8340-2a7fbcf69ff5)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/hfwy/miniconda/lib/python3.11/site-packages/transformers/utils/hub.py", line 398, in cached_file
resolved_file = hf_hub_download(
^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1377, in hf_hub_download
raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/hfwy/ChatGLM3/openai_api_demo/api_server.py", line 525, in
tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_PATH, trust_remote_code=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 806, in from_pretrained
tokenizer_class = get_class_from_dynamic_module(class_ref, pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 488, in get_class_from_dynamic_module
final_module = get_cached_module_file(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 293, in get_cached_module_file
resolved_module_file = cached_file(
^^^^^^^^^^^^
File "/home/hfwy/miniconda/lib/python3.11/site-packages/transformers/utils/hub.py", line 441, in cached_file
raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like THUDM/chatglm3-6b is not the path to a directory containing a file named tokenization_chatglm.py.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
(base) hfwy@chatglm:~/ChatGLM3/openai_api_demo$
不是警告，就是错误，提示无法连接到https://huggingface.co

[zzutligang]

@zRzRzRzRzRzRzR 麻烦抽空看看我的问题可以吗？模型下载到本地，路径使用绝对路径，但就是一只报无法连接到https://huggingface.co/，几个月前的版本和模型在我这台机器上都是调通的。

[213210828]

我把本地目录下的 config.json,modeling_chatglm.py,tokenizer_config.json 下的路径都成本地路径解决了该问题，可以试试

[zzutligang]

感谢您的热心帮助，我按@jiangqr 的方法，可以加载本地模型了。

[CharonsPluto]

请问方便贴一下你的tokenizer_config.json在哪里吗，我也是内网碰到了一样的问题，但是找不到文件在哪，搜教程还和我的不太一样。我的是在conda虚拟环境运行的

[213210828]

https://huggingface.co/THUDM/chatglm3-6b/tree/main file里面下载好的模型+文件

[zzutligang]

就在你下载模型文件夹下，

打开这个文件，按下图修改：

[jiangqr]

tokenizer_config.json里AutoTokenizer里的THUDM/chatglm3-6b--tokenization_chatglm.ChatGLMTokenizer 改成tokenization_chatglm.ChatGLMTokenizer就行了

[zzutligang]

@jiangqr 十分感谢，修改这个地方后，可以加载本地模型了。
@213210828 也感谢您的热心帮助！

[Crily11]

为什么我修改不了呢

[snahualimi]

你好我的tokenizer_config.json里本来就是tokenization_chatglm.ChatGLMTokenizer但还是有相同的报错

[zRzRzRzRzRzRzR]

检查一下你的transformers版本和huggingface上我们模型的配置

[snahualimi]

transformers 版本用的最新的4.38.1，模型在modelscope下的重新配置了一遍还是会报错