Skip to content

Conversation

CaveNightingale
Copy link
Contributor

@CaveNightingale CaveNightingale commented Aug 14, 2025

What this PR does / why we need it?

Mainline vLLM fixes its disaggregated prefill in vllm-project/vllm#22598 . But it is still not working in vllm-ascend.
To be concrete, decoder instances crash before vllm's fix and hang after vllm's fix in ascend devices.
This patch allows disaggregated prefill to work.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Qwen3-0.6B 1P1D tp=1 dp=1

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to fix a bug in disaggregated prefill for vllm-ascend, which was causing hangs. The change introduces version-specific logic to handle differences in the ModelRunnerOutput API between vLLM versions. My review found a potential issue with the version checking mechanism, which could lead to crashes on older vLLM versions because it uses an exact version match, which is brittle.

Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Signed-off-by: CaveNightingale <[email protected]>
@CaveNightingale CaveNightingale marked this pull request as ready for review August 15, 2025 03:53
Copy link

codecov bot commented Aug 15, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@1b40665). Learn more about missing BASE report.
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2374   +/-   ##
=======================================
  Coverage        ?   76.18%           
=======================================
  Files           ?      120           
  Lines           ?    13532           
  Branches        ?        0           
=======================================
  Hits            ?    10310           
  Misses          ?     3222           
  Partials        ?        0           
Flag Coverage Δ
unittests 76.18% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wangxiyuan
Copy link
Collaborator

wangxiyuan commented Aug 15, 2025

quick merge to recover PD functin. The CI failure doesn't relate to this PR

@wangxiyuan wangxiyuan merged commit 2bb7e55 into vllm-project:main Aug 15, 2025
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants