Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this April 30, 2024.

This update includes:

• Model Support
  • [Experimental] Support RecurrentGemma
• Features
  • Support paged KV cache for enc-dec models, note that the support is limited to beam width 1
• Bug fixes
  • Fix a requirement specification on Windows for nvidia-cudnn-cu12 ImportError: DLL load failed while importing tensorrt #1446
  • Fix MMHA relative position calculation error in gpt_attention_plugin for enc-dec models Flan t5 xxl result large difference #1343
• Benchmark
  • [BREAKING CHANGE] Move request rate generation arguments and logic from prepare dataset script to gptManagerBenchmark
• Performance
  • Improve the performance of pipeline parallelism when enabling in-flight batching

Thanks,
The TensorRT-LLM Engineering Team