Skip to content

Commit f882fb8

Browse files
authored
[https://nvbugs/5367180][fix] Fix xgrammar import before loading tensorrt_llm binary (#7906)
Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
1 parent 40820e6 commit f882fb8

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

tensorrt_llm/__init__.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,10 @@ def _add_trt_llm_dll_directory():
2929

3030
import sys
3131

32-
# Need to import xgrammar before tensorrt_llm library,
33-
# otherwise `MemoryError: std::bad_alloc` pattern error will be raised.
34-
import xgrammar # noqa
32+
# Need to import torch before tensorrt_llm library, otherwise some shared binary files
33+
# cannot be found for the public PyTorch, raising errors like:
34+
# ImportError: libc10.so: cannot open shared object file: No such file or directory
35+
import torch # noqa
3536

3637
import tensorrt_llm._torch.models as torch_models
3738
import tensorrt_llm.functional as functional

0 commit comments

Comments
 (0)