Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Nov 26, 2024.

This update includes:

• Model Support
  • Added support for Stable Diffusion XL. Refer to examples/sdxl/README.md. Thanks for the contribution from @Zars19 in Support SDXL and its distributed inference #1514.
• Features
  • Added support for W4A8 quantization to BF16 models on Ada(SM89).
  • Added PDL support for the FP8 GEMM plugins.
  • Added a runtime max_num_tokens dynamic tuning feature, it can be enabled by setting --enable_max_num_tokens_tuning to gptManagerBenchmark.
• Bug fixes
  • Fixed unnecessary batch logits post processor calls. ([bug] unnecessary batch logits post processor calls #2439)

Thanks,
The TensorRT-LLM Engineering Team