[Bug]: transformers compatibility issues

[tcherckez-nvidia]

System Info

H100

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

These models fail on issues with transformers lib:
ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/init.py)
python3 /opt/tensorrt-llm/examples/auto_deploy/build_and_run_ad.py --model microsoft/Phi-4-mini-instruct --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/dashboard_default.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/world_size_1.yaml

ImportError: cannot import name 'NEED_SETUP_CACHE_CLASSES_MAPPING' from 'transformers.generation.utils' (/usr/local/lib/python3.12/dist-packages/transformers/generation/utils.py)
python3 /opt/tensorrt-llm/examples/auto_deploy/build_and_run_ad.py --model nvidia/Llama-3_1-Nemotron-Ultra-253B-v1 --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/dashboard_default.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/world_size_8.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/simple_shard_only.yaml

python3 /opt/tensorrt-llm/examples/auto_deploy/build_and_run_ad.py --model nvidia/Llama-3_1-Nemotron-Ultra-253B-v1-FP8 --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/dashboard_default.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/world_size_8.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/simple_shard_only.yaml

python3 /opt/tensorrt-llm/examples/auto_deploy/build_and_run_ad.py --model nvidia/Llama-3_3-Nemotron-Super-49B-v1 --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/dashboard_default.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/world_size_8.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/simple_shard_only.yaml

Expected behavior

models should build

actual behavior

model fail

additional notes

NA

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

[galagam]

This is a known issue due to transformers upgrade:
https://huggingface.co/microsoft/Phi-4-mini-instruct/discussions/39
SafeAILab/EAGLE#306

The bad import is coming from the HF model itself.

[lucaslie]

let's disable them in the dashboard but can we reference the issue in case we want to revisit it