Skip to content

Alt Schedules #111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 518 commits into
base: sage/dbo-full-cudagraphs
Choose a base branch
from
Draft

Alt Schedules #111

wants to merge 518 commits into from
+56,896 −65,903

Conversation

LucasWilkinson
Copy link

@LucasWilkinson LucasWilkinson commented Sep 4, 2025 •
edited by github-actions bot
Loading

…bo-full-cudagraphs

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@LucasWilkinson LucasWilkinson force-pushed the lwilkinson/dbo-alt-schedules branch from 5ba544b to 759e737 Compare September 4, 2025 19:48
@LucasWilkinson LucasWilkinson mentioned this pull request Sep 5, 2025
Closed
5 tasks
wuxibin89 and others added 28 commits September 23, 2025 21:49
@wuxibin89 @youkaichao
[BugFix] Register expert_map as named buffer for wake_up and sleep (v…
61d1b35 
…llm-project#25458)

Signed-off-by: wuxibin <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@chaunceyjiang @markmc
[P/D] Support NIXL connector to disconnect during a clean shutdown (v…
f05a4f0 
…llm-project#24423)

Signed-off-by: chaunceyjiang <[email protected]>
Co-authored-by: Mark McLoughlin <[email protected]>
@panpan0000 @NickLucche
[Docs] NixlConnector quickstart guide (vllm-project#24249)
da5e7e4 
Signed-off-by: Peter Pan <[email protected]>
Signed-off-by: Peter Pan <[email protected]>
Signed-off-by: Nicolò Lucchesi<[email protected]>
Co-authored-by: Nicolò Lucchesi <[email protected]>
@faaany
[XPU] Fix MOE DP accuracy issue on XPU (vllm-project#25465)
4c966e4
@mgoin
[UX] Change kv-cache-memory log level to debug (vllm-project#25479)
2c58742 
Signed-off-by: Michael Goin <[email protected]>
@tdoublep
[V1] Remove V0 code paths for Hybrid models (vllm-project#25400)
a903669 
Signed-off-by: Thomas Parnell <[email protected]>
@LucasWilkinson @SageMoore
@yewentao256 @tlrmchlsmth
[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support…
cc1dc7e 
… and Prefill support (vllm-project#24845)

Signed-off-by: Sage Moore <[email protected]>
Signed-off-by: Lucas Wilkinson <[email protected]>
Signed-off-by: yewentao256 <[email protected]>
Signed-off-by: Lucas Wilkinson <[email protected]>
Signed-off-by: Tyler Michael Smith <[email protected]>
Co-authored-by: Sage Moore <[email protected]>
Co-authored-by: yewentao256 <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@hmellor
Add backward compatibility for GuidedDecodingParams (vllm-project#2…
875d6de 
…5422)

Signed-off-by: Harry Mellor <[email protected]>
@bnellnm @mgoin
[Kernels] Support blocked fp8 quantization for compressed tensors MoE (…
f11e3c5 
…vllm-project#25219)

Signed-off-by: Bill Nell <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@rivos-shreeasish
[BugFix] Fix UB in per_token_group_quant.cu (vllm-project#24913)
2357480 
Signed-off-by: Shreeasish Kumar <[email protected]>
@yewentao256
[Log] Optimize kv cache memory log from Bytes to GiB (vllm-project#25204
846197f 
)

Signed-off-by: yewentao256 <[email protected]>
@minosfuture @rahul-tuli
@claude @yewentao256 @houseroad @yeqcharlotte
Use macro guard CUDA functions for back compatibility in grouped_topk…
527821d 
…_kernel.cu (vllm-project#25346)

Signed-off-by: Ming Yang <[email protected]>
Signed-off-by: Rahul Tuli <[email protected]>
Co-authored-by: Rahul Tuli <[email protected]>
Co-authored-by: Claude <[email protected]>
Co-authored-by: Wentao Ye <[email protected]>
Co-authored-by: Lu Fang <[email protected]>
Co-authored-by: Ye (Charlotte) Qi <[email protected]>
@bringlein @cyang49 @yewentao256
[V1][Kernel] Add triton implementation for reshape_and_cache_flash (v…
100b630 
…llm-project#24503)

Signed-off-by: Burkhard Ringlein <[email protected]>
Co-authored-by: Chih-Chieh Yang <[email protected]>
Co-authored-by: Wentao Ye <[email protected]>
@wdhongtw
[Misc] Reduce initialization time of auto_tune (vllm-project#23682)
24e8222 
Signed-off-by: Weida Hong <[email protected]>
@ekagra-ranjan @ywang96
[Spec Decode][CI] Add e2e test for examples/spec_decode.py and prev…
867ecdd 
…ent breaking Acceptance Length (vllm-project#24531)

Signed-off-by: Ekagra Ranjan <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@jeejeelee
[Core] Ensure LoRA linear respect the base_layer's tp_size and tp_rank (
5abb117 
vllm-project#25487)

Signed-off-by: Jee Jee Li <[email protected]>
@amd-hhashemi
[ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (vllm-pr…
a3a7828 
…oject#24988)

Signed-off-by: Hashem Hashemi <[email protected]>
Signed-off-by: Hashem Hashemi <[email protected]>
@Amir-19 @mgoin
[core] add nccl symmetric memory for all reduce (vllm-project#24532)
8c1c81a 
Signed-off-by: Amir Samani <[email protected]>
Signed-off-by: Michael Goin <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@ElizaWszola @ProExpertProg @mgoin
[Performance] Move apply_w8a8_block_fp8_linear to an op class (vllm-p…
6340025 
…roject#24666)

Signed-off-by: ElizaWszola <[email protected]>
Signed-off-by: ElizaWszola <[email protected]>
Signed-off-by: Luka Govedič <[email protected]>
Signed-off-by: Luka Govedič <[email protected]>
Co-authored-by: Luka Govedič <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Luka Govedič <[email protected]>
@mgoin
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEW…
24fab45 
…ISE (vllm-project#25444)

Signed-off-by: mgoin <[email protected]>
@jiahanc
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (…
d5944d5 
…vllm-project#25406)

Signed-off-by: jiahanc <[email protected]>
@mgoin
[Bugfix] Lower gpt-oss max cudagraph size to 992 to be compatible wit…
a8ffc4f 
…h FA3 (vllm-project#25508)

Signed-off-by: mgoin <[email protected]>
@ilmarkov @mgoin
Enable symmetric memory all reduce by default only enabling for TP (v…
8bdd8b5 
…llm-project#25070)

Signed-off-by: ilmarkov <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@yewentao256
[CI] Fix Pre-commit Issue (vllm-project#25497)
8b8a8af 
Signed-off-by: yewentao256 <[email protected]>
@alecsolder
[Bugfix] gpt-oss container tool output bug (vllm-project#25485)
c828d1b 
Signed-off-by: Alec Solder <[email protected]>
Co-authored-by: Alec Solder <[email protected]>
@chaunceyjiang
[Build] Update Xgrammar to 0.1.25 (vllm-project#25467)
08275ec 
Signed-off-by: chaunceyjiang <[email protected]>
@gshtras
[Bugfix] Fix for the import error from vllm-project#24588 (vllm-proje…
690f948 
…ct#25481)

Signed-off-by: Gregory Shtrasberg <[email protected]>
@Isotr0py
[CI/Build] Fix and re-enable v1 PP test on CI (vllm-project#25496)
ae00292 
Signed-off-by: Isotr0py <[email protected]>
aarnphm and others added 26 commits September 29, 2025 23:26
@aarnphm
[V0 Deprecation] Remove vllm.worker and update according imports (v…
6a113d9 
…llm-project#25901)
@qthequartermasterman
Test Prompt Embeds/LoRA compatibility and Enable LoRA Support for OPT…
78a47f8 
… Models (vllm-project#25717)

Signed-off-by: Andrew Sansom <[email protected]>
@yewentao256 @gemini-code-assist
[Bug] Fix Weight Loading for Block FP8 Cutlass SM90 (vllm-project#25909)
89e4050 
Signed-off-by: yewentao256 <[email protected]>
Signed-off-by: Wentao Ye <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@zhuohan123
[Benchmark] Support benchmark throughput for external launcher DP (vl…
d3bd171 
…lm-project#25913)

Signed-off-by: Zhuohan Li <[email protected]>
@hmellor
MoveVllmConfig from config/__init__.py to config/vllm.py (vllm-…
61aedb5 
…project#25271)

Signed-off-by: Harry Mellor <[email protected]>
@LucasWilkinson
[BugFix] Fix DP/EP hang (vllm-project#25906)
23194d8 
Signed-off-by: Lucas Wilkinson <[email protected]>
@acisseJZhong
[BugFix] Pass config_format via try_get_generation_config (vllm-proje…
e47433b 
…ct#25912)
@zhoukezi
[Model][Bugfix] Fix MiDashengLM audio encoder mask by removing incorr…
2e1b8bc 
…ect `logical_not` (vllm-project#25925)

Signed-off-by: zhoukz <[email protected]>
@simondanielsson
[Bugfix]: Clean up chunked prefill logging when using whisper (vllm-p…
e23cacd 
…roject#25075)

Signed-off-by: simondanielsson <[email protected]>
@zyongye @heheda12345
@youkaichao @LucasWilkinson @robertgshaw2-redhat @yewentao256 @mgoin @luccafong @NickLucche @IwakuraRein @MatthewBonanni @mxz297 @Barry-Delaney
[New Model] DeepSeek-V3.2 (Rebased to Main) (vllm-project#25896)
fa7e254 
Signed-off-by: Chen Zhang <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: Lucas Wilkinson <[email protected]>
Signed-off-by: mgoin <[email protected]>
Signed-off-by: NickLucche <[email protected]>
Signed-off-by: Yongye Zhu <[email protected]>
Signed-off-by: Barry Kang <[email protected]>
Signed-off-by: Lucia Fang <[email protected]>
Co-authored-by: Chen Zhang <[email protected]>
Co-authored-by: youkaichao <[email protected]>
Co-authored-by: Lucas Wilkinson <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
Co-authored-by: Lucas Wilkinson <[email protected]>
Co-authored-by: yewentao256 <[email protected]>
Co-authored-by: Wentao Ye <[email protected]>
Co-authored-by: mgoin <[email protected]>
Co-authored-by: Lucia Fang <[email protected]>
Co-authored-by: Lucia Fang <[email protected]>
Co-authored-by: NickLucche <[email protected]>
Co-authored-by: Siyuan Fu <[email protected]>
Co-authored-by: Matthew Bonanni <[email protected]>
Co-authored-by: Xiaozhu Meng <[email protected]>
Co-authored-by: Barry Kang <[email protected]>
@a120092009
[Doc] Add Cambricon MLU support (vllm-project#25942)
8d0afa9 
Signed-off-by: a120092009 <[email protected]>
@sergiopaniego @hmellor
Updated TRL integration docs (vllm-project#25684)
1ad3aca 
Signed-off-by: sergiopaniego <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Sergio Paniego Blanco <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
@CSWYF3634076
[Bugfix][Model]fix ernie45 moe gate&bias dtype to float32 (vllm-proje…
ef6e0e7 
…ct#25936)

Signed-off-by: wangyafeng <[email protected]>
@DarkLight1337
[Model] Move vision_feature_select_strategy into `resolve_visual_en…
d7e34b4 
…coder_outputs` (vllm-project#25938)

Signed-off-by: DarkLight1337 <[email protected]>
@lhtin
[perf] Use CPU tensor to reduce GPU->CPU sync (vllm-project#25884)
e184c9c 
Signed-off-by: Lehua Ding <[email protected]>
@NickLucche @heheda12345
[NIXL] Add support for MLA caches with different latent dim (vllm-pro…
80608ba 
…ject#25902)

Signed-off-by: NickLucche <[email protected]>
Signed-off-by: Chen Zhang <[email protected]>
Co-authored-by: Chen Zhang <[email protected]>
@rzabarazesh @hmellor
[CI] Move applicable tests to CPU (vllm-project#24080)
bc546f7 
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
@ihb2032
[Fix] Improve CPU backend compatibility for RISC-V (vllm-project#25816)
bb6d430 
Signed-off-by: lyd1992 <[email protected]>
Signed-off-by: ihb2032 <[email protected]>
@Josephasafg
[Kernel][Moe Configs] Add more tuned triton configs for ExpertsInt8 a…
35fe398 
…nd FP8 (vllm-project#25858)

Signed-off-by: asafg <[email protected]>
@sergiopaniego @hmellor
Add Hugging Face Inference Endpoints guide to Deployment docs (vllm-p…
099aaee 
…roject#25886)

Signed-off-by: sergiopaniego <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
@Anionex
[Bugfix][Model] Fix inference for Hunyuan dense models (vllm-project#…
f4db5e6 
…25354)

Signed-off-by: anion <[email protected]>
Signed-off-by: Anion <[email protected]>
@pavanimajety
[Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (vl…
ef28354 
…lm-project#25895)

Signed-off-by: Pavani Majety <[email protected]>
@DarkLight1337
[Bugfix] Token type and position embeddings fail to be applied to `in…
9f1c4ec 
…puts_embeds` (vllm-project#25922)

Signed-off-by: DarkLight1337 <[email protected]>
@youkaichao
[bugfix][deepseek] fix flashmla kernel selection (vllm-project#25956)
a2e6fa7 
Signed-off-by: youkaichao <[email protected]>
@LucasWilkinson
Squashed commit of nm/lwilkinson/dbo-alt-schedules changes relative t…
2682bb7 
…o origin/main

fixes

Signed-off-by: Lucas Wilkinson <[email protected]>

fix

Signed-off-by: Lucas Wilkinson <[email protected]>

fix

Signed-off-by: Lucas Wilkinson <[email protected]>

fixes and formatting

Signed-off-by: Lucas Wilkinson <[email protected]>
@LucasWilkinson
get deep ll to run
fcd015c 
Signed-off-by: Lucas Wilkinson <[email protected]>
@LucasWilkinson LucasWilkinson force-pushed the lwilkinson/dbo-alt-schedules branch from 79d0a01 to fcd015c Compare September 30, 2025 18:06
@LucasWilkinson
fix
6dc3492 
Signed-off-by: Lucas Wilkinson <[email protected]>
@LucasWilkinson
cleanup
2d7ca92 
Signed-off-by: Lucas Wilkinson <[email protected]>
@LucasWilkinson
cleanup
4effe25 
Signed-off-by: Lucas Wilkinson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NickLucche NickLucche Awaiting requested review from NickLucche NickLucche will be requested when the pull request is marked ready for review NickLucche is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin will be requested when the pull request is marked ready for review mgoin is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat will be requested when the pull request is marked ready for review robertgshaw2-redhat is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 will be requested when the pull request is marked ready for review yewentao256 is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth will be requested when the pull request is marked ready for review tlrmchlsmth is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao will be requested when the pull request is marked ready for review youkaichao is a code owner

@njhill njhill Awaiting requested review from njhill njhill will be requested when the pull request is marked ready for review njhill is a code owner

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat alexm-redhat will be requested when the pull request is marked ready for review alexm-redhat is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg will be requested when the pull request is marked ready for review ProExpertProg is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.