Bug in importing transformers modules

[ly01325]

Describe the bug

Can not import many transformers module, which are used in several tasks

and other module as well

Steps/Code to reproduce bug

git clone the repo
bash docker/build.sh
the bugs are in the official docker image env

Expected behavior

System information

======================================================================

• Container used (if applicable): ?
• OS (e.g., Ubuntu 22.04, CentOS 7, Windows 10): Ubuntu 24.04.2 LTS
• CPU architecture (x86_64, aarch64): x86_64
• GPU name (e.g. H100, A100, L40S): NVIDIA RTX 6000D
• GPU memory size: 83.6 GB
• Number of GPUs: 2
• Library versions (if applicable):
  • Python: 3.12.3
  • ModelOpt version or commit hash: 0.35.0
  • CUDA: 12.9
  • PyTorch: 2.8.0+cu128
  • Transformers: 4.56.1
    :1297: FutureWarning: The cuda.cuda module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.driver module instead.
    :1297: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
    [2025-09-05 07:35:20] INFO utils.py:148: Note: detected 128 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable.
    [2025-09-05 07:35:20] INFO utils.py:151: Note: NumExpr detected 128 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
    [2025-09-05 07:35:20] INFO utils.py:164: NumExpr defaulting to 16 threads.
    [2025-09-05 07:35:20] INFO config.py:54: PyTorch version 2.8.0 available.
    [2025-09-05 07:35:20] INFO config.py:66: Polars version 1.25.2 available.
    /usr/local/lib/python3.12/dist-packages/modelopt/torch/utils/import_utils.py:32: UserWarning: Failed to import peft plugin due to: ImportError("cannot import name 'PreTrainedModel' from 'transformers' (/usr/local/lib/python3.12/dist-packages/transformers/init.py)"). You may ignore this warning if you do not need this plugin.
    warnings.warn(
    /usr/local/lib/python3.12/dist-packages/modelopt/torch/utils/import_utils.py:32: UserWarning: Failed to import huggingface plugin due to: ImportError("cannot import name 'ModelOptHFTrainer' from 'modelopt.torch.opt.plugins' (/usr/local/lib/python3.12/dist-packages/modelopt/torch/opt/plugins/init.py)"). You may ignore this warning if you do not need this plugin.
    warnings.warn(
    /usr/local/lib/python3.12/dist-packages/modelopt/torch/trace/plugins/init.py:34: UserWarning: Failed to import transformers plugin due to: AttributeError("partially initialized module 'torchvision' has no attribute 'extension' (most likely due to a circular import)")
    _warnings.warn(f"Failed to import transformers plugin due to: {e!r}")
    /usr/local/lib/python3.12/dist-packages/modelopt/torch/utils/import_utils.py:32: UserWarning: Failed to import transformer engine plugin due to: AttributeError("module 'transformer_engine' has no attribute 'pytorch'"). You may ignore this warning if you do not need this plugin.
    warnings.warn(
    /usr/local/lib/python3.12/dist-packages/modelopt/torch/utils/import_utils.py:32: UserWarning: Failed to import transformers plugin due to: AttributeError("partially initialized module 'torchvision' has no attribute 'extension' (most likely due to a circular import)"). You may ignore this warning if you do not need this plugin.
    warnings.warn(
  • TensorRT-LLM: ?
  • ONNXRuntime: 1.22.0
  • TensorRT: 10.11.0.33
• Any other details that may help: ?

[kevalmorabia97]

Can you check if transformers 4.55 works?

[ly01325]

Can you check if transformers 4.55 works?

OK, I tried to upgrade the transformer version. But the problems still remian.4.55.
I also tried 4.53.1, still the same proble

[kevalmorabia97]

Do you see this issue even when nvidia-modelopt is uninstalled? I think this problem is not related to modelopt

[ly01325]

Do you see this issue even when nvidia-modelopt is uninstalled? I think this problem is not related to modelopt

I tested in the base image of modelopt image: nvcr.io/nvidia/tensorrt-llm/release:1.0.0rc6
The transformers version is 4.53.1, which worked successfully and could import transformers modules.
But in the built modelopt image, the transformer version is changed to 0.451 and it failed to import. And I also tried to reinstall the transformers==4.53.1 in the modelopt image, but it doesn't work.
I feel that the dependencies installed in the example have conflicts, which is causing some environment compatibility issues. However, there are too many related libraries in the example, and I am not sure which specific one is causing the problem

[ly01325]

aHA, I think I just comfirmed it.
"I limited the installation of the dependencies in the example to only those I needed. Now it is working properly."

[ly01325]

@kevalmorabia97
I found another issue. The modelopt package installed in the container:/usr/local/lib/python3.12/dist-packages/modelopt/torch/quantization/config.py has a config file that does not include W4A8_NVFP4_FP8_CFG, the names are different, which causes errors during NVFP4 quantization. I need to modify the PYTHONPATH to point to the latest repo to resolve this."

[kevalmorabia97]

What modelopt version is pre-installed in your container? Is it 0.35.0?

[ly01325]

is

What modelopt version is pre-installed in your container? Is it 0.35.0?

yes , 0.35.0

[kevalmorabia97]

Perhaps when the container was built, outdated files were used?

[kevalmorabia97]

Good practice is to just mount your local clone of the repo to the container such that modelopt version is matching the branch from the repo - in your case release/0.35.0 branch. Alternatively you can install modelopt from source as well (pip install -e .[all]) so installation and example scripts are from same branch

[ly01325]

Good practice is to just mount your local clone of the repo to the container such that modelopt version is matching the branch from the repo - in your case release/0.35.0 branch. Alternatively you can install modelopt from source as well (pip install -e .[all]) so installation and example scripts are from same branch

Yes, mount the local repo is the easiest way to solve this.