txt

Signed-off-by: WoosukKwon <[email protected]>

Author: WoosukKwon

_posts/2025-01-26-v1.md

@@ -1,6 +1,6 @@
 ---
 layout: post
-title: "vLLM V1 Alpha Release"
+title: "vLLM V1: A Major Upgrade to vLLM's Core Architecture"
 author: "vLLM Team"
 ---
 
@@ -116,22 +116,34 @@ Thanks to the extensive architectural enhancements, vLLM V1 achieves state-of-th
 These dramatic performance gains stem from comprehensive CPU overhead reductions across the entire stack.
 The improvements are even more pronounced for vision-language models (VLMs) like Qwen2-VL, thanks to V1's enhanced support for VLMs.
 
-- **Llama 3.1 8B & Llama 3.3 70B**
+- **Text Models: Llama 3.1 8B & Llama 3.3 70B**
 
 <p align="center">
 <picture>
 <img src="/assets/figures/v1/v1_llama.png" width="100%">
 </picture>
 </p>
 
-- **Qwen2-VL (VLM), 1xH100**
+We measured the performance of vLLM V0 and V1 on Llama 3.1 8B and Llama 3.3 70B models using the ShareGPT dataset.
+V1 demonstrated consistently lower latency than V0 especially at high QPS, thanks to the higher throughput it achieves.
+Given that the kernels used for V0 and V1 are almost identical, the performance difference is mainly due to the architectural improvements (reduced CPU overheads) in V1.
+
+- **Vision-language Models: Qwen2-VL, 1xH100**
 
 <p align="center">
 <picture>
 <img src="/assets/figures/v1/v1_qwen2vl.png" width="50%">
 </picture>
 </p>
 
+We evaluated the performance on VLMs by testing Qwen2-VL using the [VisionArena dataset](https://arxiv.org/abs/2412.08687).
+V1 delivered even larger speedups over V0, thanks its improved VLM support, driven by two key improvements: offloading input processing to a separate process and implementing more flexible scheduling for multimodal queries.
+
+- Looking Forward
+
+While these improvements are significant, we view them as just the beginning.
+Our new clean architecture provides a solid foundation for future enhancements, which we plan to roll out in the coming weeks.
+Stay tuned for more updates!
 
 # Limitations & Future Work
 
@@ -141,7 +153,7 @@ While vLLM V1 shows promising results, it is still in its alpha stage and lacks
 V1 supports decoder-only Transformers like Llama, mixture-of-experts (MoE) models like Mixtral, and several VLMs such as Qwen2-VL. All quantization methods are supported. However, V1 currently does not support encoder-decoder architectures like multimodal Llama 3.2, Mamba-based models like Jamba, or embedding models. Please check out [our documentation]() for a more detailed list of the supported models.
 
 **Feature Limitations:**  
-V1 currently lacks support for log probs, prompt log probs sampling parameters, pipeline parallelism, structured decoding, speculative decoding, prometheus metrics, and LoRA. We are actively working to close this feature gap and add new optimizations. Please stay tuned!
+V1 currently lacks support for log probs, prompt log probs sampling parameters, pipeline parallelism, structured decoding, speculative decoding, prometheus metrics, and LoRA. We are actively working to close this feature gap and add brand-new optimizations to the V1 engine.
 
 **Hardware Support:**  
 V1 currently supports only Ampere or later NVIDIA GPUs. We are actively working to extend support to other hardware backends such as TPU.