Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this March 12, 2024.

This update includes:

• Model Support
  • Support VILA (see “LLaVA and VILA” section in examples/multimodal/README.md)
• Features
  • Support loading Gemma from Hugging Face
  • Add support to context chunking to work with KV cache reuse
  • Support auto parallelism planner for high-level API and unified builder workflow
    • See "Auto parallel" section in examples/high-level-api/README.md and examples/high-level-api/run_auto_parallel_examples.sh for an example
  • Enable multi-LoRA for BART LoRA
• API
  • [BREAKING CHANGE] Remove model parameter from gptManagerBenchmark and gptSessionBenchmark
• Bug fixes
  • Fix ChatGLM2-6B building failure on INT8 chatglm2-6b int8+kv8 build failed on 0.8.0 branch #1239
  • Fix wrong relative path in Baichuan documentation Incorrect documentation in examples /baichuan/ #1242
• Performance
  • Remove router tensor parallelism to improve performance for MoE models, thanks to the contribution from @megha95 in moe router tp removed #1091
• Infra
  • TensorRT dependency is updated to 9.3.
  • Base Docker image for TensorRT-LLM is updated to nvcr.io/nvidia/pytorch:24.01-py3
  • Base Docker image for TensorRT-LLM backend is updated to nvcr.io/nvidia/tritonserver:24.01-py3

Thanks,
The TensorRT-LLM Engineering Team