Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Sep 17, 2024.

This update includes:

• Features
  • Supported BFloat16 for fused Gated MLP.
  • Supported calibration with offline dataset for the Python LLM class.
• Bug fixes
  • Fixed a typo in tensorrt_llm/models/model_weights_loader.py, thanks to the contribution from @wangkuiyi in Update model_weights_loader.py #2152.
  • Fixed duplicated import module in tensorrt_llm/runtime/generation.py, thanks to the contribution from @lkm2835 in Fix duplicated import module #2182.

Thanks,
The TensorRT-LLM Engineering Team

[hofong428]

Awesome!