Skip to content

[bugfix]fixed block_size incorrect setting issue in dsv3.2#7630

Open
kunpengW-code wants to merge 5 commits intovllm-project:mainfrom
kunpengW-code:bugfix_block_size
Open

[bugfix]fixed block_size incorrect setting issue in dsv3.2#7630
kunpengW-code wants to merge 5 commits intovllm-project:mainfrom
kunpengW-code:bugfix_block_size

Conversation

@kunpengW-code
Copy link
Contributor

@kunpengW-code kunpengW-code commented Mar 25, 2026

What this PR does / why we need it?

vllm-project/vllm#35122 This PR in the vllm community refactors the update mode of block_size. As a result, when the user does not specify --block-size, dsv3.2 obtains an incorrect block_size.

The root cause of the problem is analyzed from the block_size update process as follows:

  1. In NPUPlatform, check_and_update_config calls refresh_block_size to set block_size to 128.
  2. During Modelrunner initialization, the self.block_size parameter is generated. At this time, block_size is still 128. This parameter will be used for operations such as kvcache initialization.
  3. update_block_size_for_backend updates block_size to the size set in attn_backend. The reason why the DSV3.2 is faulty is that it has an additional attn_backend DeepseekV32IndexerBackend, and this backend is not rewritten. The block_size obtained from attn_backend is 64. In this case, only vllm_config.cache_config.block_size is updated, and other parts are not modified. As a result, the block_size on the entire network is inconsistent.

Modification solution:
Skip update_block_size_for_backend and modify block_size only in the check_and_update_config method.

In the future, the block_size update logic can be migrated to the update_block_size_for_backend method. Ensure that all block_size values on the entire network are updated.

Does this PR introduce any user-facing change?

no

How was this patch tested?

Signed-off-by: Wang Kunpeng <1289706727@qq.com>
@github-actions
Copy link
Contributor

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a bug related to incorrect block_size settings, specifically within the dsv3.2 environment. It streamlines the block size determination logic on the Ascend platform by removing redundant checks and centralizing the handling of hybrid models for appropriate block size configuration, ensuring more consistent and correct cache behavior.

Highlights

  • Block Size Configuration Refinement: The update_block_size_for_backend method in vllm_ascend/platform.py has been refactored. Previous logic for handling user-specified block sizes and hybrid models, along with the call to the superclass method, has been removed and replaced with a pass statement and a TODO comment, indicating a future consolidation of block size determination logic.
  • Hybrid Model Block Size Handling: The refresh_block_size utility function in vllm_ascend/utils.py was updated to correctly identify hybrid attention+mamba models using model_config.is_hybrid for block size determination. This change replaces hardcoded checks for specific Qwen model types (qwen3_next, qwen3_5), making the logic more generic and robust.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the block size update mechanism across vllm_ascend/platform.py and vllm_ascend/utils.py. The update_block_size_for_backend method in platform.py has been simplified to a pass statement with a TODO, moving its logic elsewhere. The refresh_block_size function in utils.py now includes specific handling for hybrid models and revised conditions for setting the block size to 128. However, a critical issue has been identified: the refactoring has removed the check for user_specified_block_size from the block size determination process. This omission could lead to user-defined block sizes being unintentionally overridden, causing unexpected behavior, and indicates an incomplete centralization of block size selection logic.

Comment on lines +186 to +188
# TODO: NPU still sets block_size in check_and_update_config.
# Move that logic here so block_size is chosen by the backend.
pass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The update_block_size_for_backend method has been refactored to a pass statement with a TODO. While the is_hybrid model logic has been correctly moved to refresh_block_size (which is called by check_and_update_config), the critical check for cache_config.user_specified_block_size has been removed from this call path. This omission means that user-defined block sizes might be unintentionally overridden, leading to unexpected behavior. The TODO also highlights that the block size selection logic is not yet fully centralized, indicating an incomplete refactoring.

Comment on lines +1100 to +1103
if model_config.is_hybrid:
# Hybrid attention+mamba models rely on the model-specific sizing
# logic rather than the generic platform default.
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The refresh_block_size function now correctly handles hybrid models. However, it currently does not respect cache_config.user_specified_block_size. If a user explicitly sets --block-size, this function might still override it to 128 if enable_prefix_caching or enable_chunked_prefill is enabled. This can lead to unexpected behavior and should be addressed to ensure user configuration is prioritized. Please reintroduce the check for user_specified_block_size before applying default logic.

    if cache_config.user_specified_block_size:
        # User specified --block-size; keep it.
        return

    if model_config.is_hybrid:
        # Hybrid attention+mamba models rely on the model-specific sizing
        # logic rather than the generic platform default.
        return

@chenxi-hh chenxi-hh added ready read for review ready-for-test start test by label for PR labels Mar 25, 2026
@yiz-liu yiz-liu added this to the v0.18.0rc1 milestone Mar 25, 2026
Signed-off-by: Wang Kunpeng <1289706727@qq.com>
Signed-off-by: Wang Kunpeng <1289706727@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:core ready read for review ready-for-test start test by label for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants