Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (also includes the Triton backend) this December 1st, 2023.

This update includes:

• Model Support
  • Mixture of Experts support(experimental)
• Features
  • fMHA support for chunked attention and paged kv cache
  • Baichuan FP8 quantization support
• Memory optimization
  • Reduced host memory when building engines
• Performance
  • MMHA optimization for MQA and GQA

Thanks,
The TensorRT-LLM Engineering Team