[Attention] Full CG support for llama4 and remove use of deprecated properties #31851

Closed

LucasWilkinson wants to merge 100 commits intovllm-project:mainfrom

neuralmagic:lwilkinson/full-cg-llama4

Commits on Jan 3, 2026

WIP: Refactor chunked local attention for FULL CG support
LucasWilkinson
committed
Remove CPU<>GPU syncs and debug prints from chunked local attention
LucasWilkinson
committed

Commits on Jan 4, 2026

Add comprehensive tests for chunked local attention Triton kernel
LucasWilkinson
committed
Fix query_start_loc_cpu to use proper CPU tensor
LucasWilkinson
committed

Commits on Jan 5, 2026

make lazy
LucasWilkinson
committed

Commits on Jan 6, 2026

Add Multimodal Processor Benchmark (#29105)

reaganjlee
authored and
LucasWilkinson
committed
[Bugfix] Replace BaseException with specific exceptions in FLA utils (#31590)

c0de128
authored and
LucasWilkinson
committed
[ROCm][CI] Fix failure in Language Models Tests (Extra Standard) by reducing agent pool size (#31553)

AndreasKaratzas
authored and
LucasWilkinson
committed
feat: support LoRA for DeepSeek-OCR(Language Model part) (#31569)

authored and
LucasWilkinson
committed
[Bugfix] Fix block size used in EAGLE slot mapping (#31540)

benchislett
authored and
LucasWilkinson
committed
[Model] Enable LoRA support for tower and connector in LLaVA (#31513)

authored and
LucasWilkinson
committed
[ROCm][CI] Fix ModernBERT token classification test (#31612)

AndreasKaratzas
authored and
LucasWilkinson
committed
[Bugfix] Fix activation quantization for compressed-tensors W4A16 (#31572)

Tmn07
authored and
LucasWilkinson
committed
Remove unused `use_marlin` variable in `Mxfp4MoEMethod` (#31549)

vsourirajan
authored and
LucasWilkinson
committed
[Bugfix][Hardware][AMD] Fix last_page_len calculation in AITER MLA decode (#31282)

c0de128
authored and
LucasWilkinson
committed
[Bugfix] Fix weight_loader v1 block scale (#31103)

kyuyeunk
authored and
LucasWilkinson
committed
Add multimodal input method in the documentation (#31601)

labAxiaoming
authored and
LucasWilkinson
committed
CustomOp: test forward dispatch for grouped_topk (#31530)

xinyu-intel
authored and
LucasWilkinson
committed
[BugFix] Support online dense model DP without overhead (#30739)

njhill
authored and
LucasWilkinson
committed
[MoE] Fix output_shape calculation in Attention layer to handle 3D query inputs (#31596)

AndreasKaratzas
authored and
LucasWilkinson
committed
[MoE Refactor] Split `invoke_fused_moe_kernel` (#31050)

authored and
LucasWilkinson
committed
[MoE Refactor] Explicit construct mk for flashinfer bf16 kernel (#31504)

authored and
LucasWilkinson
committed
[Benchmark] Fix OOM during MoE kernel tuning for large models (#31604)

massif-01
authored and
LucasWilkinson
committed
[Core] Parse vLLM engine required fields from hf_config to model_arch_config (#28454)

charlotte12l
authored and
LucasWilkinson
committed
Improve HF qwen3_omni: preserve audio_sample_rate in kwargs restructuring (#29255)

jeremyteboul
authored and
LucasWilkinson
committed
[CI][Bugfix] Fix token counting in chunked prefill compl test (#31630)

AndreasKaratzas
authored and
LucasWilkinson
committed
[MoE Refactor][13/N] Convert FI to Use PFNoEP (#31533)

authored and
LucasWilkinson
committed
[Docs] Fix argparse include path for mm-processor benchmark (#31654)

reaganjlee
authored and
LucasWilkinson
committed
fix no think of GLM-4.5 / GLM-4.7 (#31449)

zRzRzRzRzRzRzR
authored and
LucasWilkinson
committed
[misc] Sort uvicorn log level description according to verbosity (#31137)

andyxning
authored and
LucasWilkinson
committed
[BugFix] Async scheduling: handle model forward errors more cleanly (#31611)

njhill
authored and
LucasWilkinson
committed
[CI] Skip Phi-MoE test due to old API util (#31632)

AndreasKaratzas
authored and
LucasWilkinson
committed
[ROCm][CI] Fix language generation test accuracy by disabling HF flash_sdp and mem_efficient_sdp (#31597)

AndreasKaratzas
authored and
LucasWilkinson
committed
[Bugfix] Fix AttributeError: 'Stream' object has no attribute 'dp_size' (#31663)

jeejeelee
authored and
LucasWilkinson
committed
[Minor] Small pooler output processing optimization (#31667)

njhill
authored and
LucasWilkinson
committed
[CI/Build] Revive skipped reward models e2e test (#31665)

Isotr0py
authored and
LucasWilkinson
committed
[Platform] Deprecate seed_everything (#31659)

wangxiyuan
authored and
LucasWilkinson
committed
[Misc] Various code simplifications (#31666)

njhill
authored and
LucasWilkinson
committed
[CI Failure] Fix NomicBert max_model_len validation (#31662)

noooop
authored and
LucasWilkinson
committed
Add chat prefix completion feature to DeepSeek v3.2 (#31147)

PHOEBEMOON0802
authored and
LucasWilkinson
committed
[log] enable max_log_len trim only when needed (#31482)

andyxning
authored and
LucasWilkinson
committed
[Bugfix] Fix EPLB state logging error (#31455)

tlrmchlsmth
authored and
LucasWilkinson
committed
[Frontend] [Bugfix] respect server-level default chat template kwargs in reasoning parser (#31581)

authored and
LucasWilkinson
committed
[CI] Bump sentence-transformer from 3.2.1 to 5.2.0 (#31664)

noooop
authored and
LucasWilkinson
committed
[ROCM] Reorder arguments and rename parameters for rope_cached_thd_positions_2c_fwd_inplace (#29993)

tpopp
authored and
LucasWilkinson
committed
[Model] Enable LoRA support for BLIP2 (#31620)

ppppqp
authored and
LucasWilkinson
committed
[LoRA] LoRA PDL improvement (#31660)

jeejeelee
authored and
LucasWilkinson
committed
[KVconnector][LMCache] remove the import of legacy LMCache code (#31704)

ApostaC
authored and
LucasWilkinson
committed
[platform] Support additional forward context for OOT (#31674)

authored and
LucasWilkinson
committed
[Model] Let more models to support the score template. (#31335)

authored and
LucasWilkinson
committed
[v1] Add encoder-only/cross attention support to Triton Attention backend (#31406)

Isotr0py
authored and
LucasWilkinson
committed
[Frontend] [Doc] Exclude log deltas feature (#30322)

authored and
LucasWilkinson
committed
[CI Failure] Disable B200 tests while runner is broken (#31732)

mgoin
authored and
LucasWilkinson
committed
[Bugfix][CPU] Fix RotaryEmbedding fallback causing gibberish with --enforce-eager (#31643)

ricky-chaoju
authored and
LucasWilkinson
committed
[Bugfix] Add missing extra_tensors arg to DeviceCommunicatorBase.disp… (#31644)

kzwrime
authored and
LucasWilkinson
committed
Triton Attention: Support cross-layers blocks (#30687)

orozery
authored and
LucasWilkinson
committed
[Misc] Enable Paligemma's PrefixLM attention mask computation (#31725)

Isotr0py
authored and
LucasWilkinson
committed
Fix GLM-4.6v flash tool calling in transformers 5.x (#31622)

baonudesifeizhai
authored and
LucasWilkinson
committed
[Misc][Model][Refactor] Pass the prefix into Linear layers (#31669)

kunpengW-code
authored and
LucasWilkinson
committed
[BugFix] Fix architecture flags to prevent issues on SM103 (#31150)

LopezCastroRoberto
authored and
LucasWilkinson
committed
pin lora_b moe weights on cpu (#31317)

gnovack
authored and
LucasWilkinson
committed
[docker] install cuda13 version of lmcache and nixl (#30913)

soodoshll
authored and
LucasWilkinson
committed
[Model] Nemotron Parse 1.1 Support (#30864)

authored and
LucasWilkinson
committed
[Cleanup] Remove deprecated fields from CachedRequestData class (#31734)

njhill
authored and
LucasWilkinson
committed
[CI][DeepSeek] Add nightly DeepSeek R1 `lm_eval` tests on H200 (#30356)

authored and
LucasWilkinson
committed
[Bug] Revert torch warning fix (#31585)

yewentao256
authored and
LucasWilkinson
committed
[MoE Refactor] Aiter Experts for BF16 MoE (#31542)

authored and
LucasWilkinson
committed
[Bugfix] Fix Broken ModelOpt NVFP4 MoE (#31742)

authored and
LucasWilkinson
committed
[Bugfix] Properly apply v_scale for mimo_v2_flash (#31175)

mgoin
authored and
LucasWilkinson
committed
[CI/Build] Allow user to configure NVSHMEM version via ENV or command line (#30732)

authored and
LucasWilkinson
committed
[Bugfix] vLLM produces invalid UTF-8 tokens and “�” (#28874)

authored and
LucasWilkinson
committed
Revert "[CI Failure] Disable B200 tests while runner is broken" (#31750)

mgoin
authored and
LucasWilkinson
committed
[Docs] Improve malformed exception caused by backslash line continuations (#31694)

authored and
LucasWilkinson
committed
[Perf] Optimize additional `fill(0)` in cutlass moe, 2.9% E2E throughput improvement, 10.8% TTFT improvement (#31754)

yewentao256
authored and
LucasWilkinson
committed
[Cleanup] Remove redundant `decoder_layer_type` assignment in `Qwen2` (#31760)

maang-h
authored and
LucasWilkinson
committed
[UX] Add `-ep` shorthand for `--enable-expert-parallel` (#30890)

mgoin
authored and
LucasWilkinson
committed
[Bugfix] Add init_workspace_manager to moe kernel benchmarks (#31042)

mgoin
authored and
LucasWilkinson
committed
[CI] Fix CPU MM PRocessor Test (#31764)

authored and
LucasWilkinson
committed
[Bugfix][Hardware][AMD] Fix exception types in AITER MLA FP8 check (#31177)

authored and
LucasWilkinson
committed
[Doc] Show that `use_audio_in_video` is supported in docs (#30837)

DarkLight1337
authored and
LucasWilkinson
committed
[Bugfix][ROCm] Fix Unsupported attention metadata type for speculative decoding in `eagle.py` (#31714)

vllmellm
authored and
LucasWilkinson
committed
[Models]: Use `MMEncoderAttention` for MoonViT (#31738)

authored and
LucasWilkinson
committed
[Bugfix][CI/Build] Fix failing pooling models test due to Triton kernel accuracy diff (#31776)

Isotr0py
authored and
LucasWilkinson
committed
[Chore] Remove more V0 dead code from `sequence.py` (#31783)

DarkLight1337
authored and
LucasWilkinson
committed
[cpu][bench] Add CPU paged attention benchmarks (#31720)

fadara01
authored and
LucasWilkinson
committed
[Misc] Use `deprecated` for `seed_everything` (#31780)

DarkLight1337
authored and
LucasWilkinson
committed
[CI] Increase the MTEB_EMBED_TOL threshold to 5e-4. (#31797)

noooop
authored and
LucasWilkinson
committed
[Doc] Fix format of multimodal_inputs.md (#31800)

BlankRH
authored and
LucasWilkinson
committed
[Chore] Cleanup `mem_utils.py` (#31793)

DarkLight1337
authored and
LucasWilkinson
committed
[Attention][1/n] Remove usage of deprecated `seq_lens_cpu` and `num_computed_tokens_cpu` CommonAttentionMetadata properties (#31773)
LucasWilkinson
committed
[Bugfix] Fix torch.compile error for DP + MoE on CPU Backend (#31650)

kzwrime
authored and
LucasWilkinson
committed
[Misc] Implement `TokenizerLike.convert_tokens_to_ids` (#31796)

DarkLight1337
authored and
LucasWilkinson
committed
[Bugfix]: avoid overriding audio/text kwargs (Qwen3-Omni) (#31790)

Jzz1943
authored and
LucasWilkinson
committed
[Frontend] Support GLM-4.5 / GLM-4.7 with enable_thinking: false (#31788)

chaunceyjiang
authored and
LucasWilkinson
committed
[Model] rename use_pad_token to use_sep_token (#31784)

noooop
authored and
LucasWilkinson
committed
[LoRA]Disable linear LoRA kernel PDL (#31777)

jeejeelee
authored and
LucasWilkinson
committed
[Bugfix]: Fix cross attention backend selection for Turing GPU (#31806)

Isotr0py
authored and
LucasWilkinson
committed
[MoE Refactor] Add Temporary Integration Tests - H100/B200 (#31759)

authored and
LucasWilkinson
committed
[MoE Refactor][14/N] Clean Up FI Quant Config Smuggling (#31593)

authored and
LucasWilkinson
committed
[NemotronH] Use ReplicatedLinear for fc1_latent_proj (#31807)

roikoren755
authored and
LucasWilkinson
committed