Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch(also includes the Triton backend) this November 10th, 2023.

This update includes:

• Model Support
  • Mistral 7B
• Features
  • Python binding for C++ runtime.
  • Support for stop and bad words in non-inflight batching.
  • New heuristic for launching Multi-block MMHA.
• Bug fixes
  • Fix missing torch dependencies: [BUG] The batch_manage.a choice error in --cpp-only when torch's cxx_abi version is different with gcc #151, Linking error during compiling google-test & benchmarks #277
  • Fix logits dtype for Baichuan and ChatGLM: segment falut cause bfloat16 decoder not create #335

Thanks,
The TensorRT-LLM Engineering Team