Update of the Development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch this November 7th, 2023.

This update includes:

• Model Support
  • ChatGLM3: Support ChatGLM3 #270
  • InternLM (contributed by @wangruohui): Support InternLM models (deprecated and move to #266) #123
• Features
  • P-Tuning support (enabled in GPT, LLaMA)
  • Baichuan INT8 SQ / INT8 KV Cache support
  • GPT-J
    • AWQ Tensor Parallelism support
    • INT8 KV cache + AWQ/weight-only support
  • Exclude input in output: How to get the newly generated tokens only? #227
• Bug fixes
  • Fix Multi-block MMHA: Difference between max_batch_size in the engine builder and max_num_sequences in TrtGptModelOptionalParams? #65
  • Optimize high CPU memory usage
  • Fix the log message to be more accurate on KV cache: Smaller available space for paged KV cache compared with vLLM #224
  • Fix Windows release wheel installation: Failed to install the release wheel for Windows using pip #261
  • Minor bug fixes
• Perf enhancement
  • Pipeline parallelism improvement: different micro batch size for context and generation phase support

Thanks,
The TensorRT-LLM Engineering Team