Update on the development branch

[DanBlanaru]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Sep 30, 2024.

This update includes:

• Model Support
  • Supported NemotronNas, see examples/nemotron_nas/README.md.
  • Supported Phi-3.5 models, see examples/phi/README.md.
• API
  • Supported finish_reason and stop_reason for the LLM API.
• Bug fixes
  • Fixed kv_cache_type issue in the Python benchmark, thanks to the contribution from @qingquansong in Fix kv_cache_type issue #2219.
• Infra
  • The dependent TensorRT version is updated to 10.4.0.

Thanks,
The TensorRT-LLM Engineering Team