Commit 7bbc7a0
committed
[bugfix]fixed block_size incorrect setting issue in dsv3.2 (vllm-project#7630)
### What this PR does / why we need it?
vllm-project/vllm#35122 This PR in the vllm
community refactors the update mode of block_size. As a result, when the
user does not specify `--block-size`, dsv3.2 obtains an incorrect
block_size.
**The root cause of the problem is analyzed from the block_size update
process as follows:**
1. In NPUPlatform, `check_and_update_config` calls `refresh_block_size`
to set block_size to 128.
2. During Modelrunner initialization, the `self.block_size` parameter is
generated. At this time, block_size is still 128. This parameter will be
used for operations such as kvcache initialization.
3. `update_block_size_for_backend` updates block_size to the size set in
attn_backend. The reason why the DSV3.2 is faulty is that it has an
additional attn_backend `DeepseekV32IndexerBackend`, and this backend is
not rewritten. The block_size obtained from attn_backend is 64. In this
case, only `vllm_config.cache_config.block_size` is updated, and other
parts are not modified. As a result, the block_size on the entire
network is inconsistent.
**Modification solution:**
Skip `update_block_size_for_backend` and modify block_size only in the
`check_and_update_config` method.
In the future, the block_size update logic can be migrated to the
`update_block_size_for_backend` method. Ensure that all block_size
values on the entire network are updated.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
- vLLM version: v0.18.0
- vLLM main:
vllm-project/vllm@ed359c4
---------
Signed-off-by: Wang Kunpeng <1289706727@qq.com>1 parent dba34d4 commit 7bbc7a0
2 files changed
+9
-17
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
224 | 224 | | |
225 | 225 | | |
226 | 226 | | |
227 | | - | |
228 | | - | |
229 | | - | |
230 | | - | |
231 | | - | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
237 | | - | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
238 | 230 | | |
239 | 231 | | |
240 | 232 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1101 | 1101 | | |
1102 | 1102 | | |
1103 | 1103 | | |
1104 | | - | |
1105 | | - | |
1106 | | - | |
1107 | | - | |
1108 | | - | |
1109 | | - | |
| 1104 | + | |
| 1105 | + | |
| 1106 | + | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
1110 | 1110 | | |
1111 | 1111 | | |
1112 | 1112 | | |
| |||
0 commit comments