Skip to content

Commit 234a5a4

Browse files
[v0.9.1][Bugfix] Fix guided decoding invalid backend (#2645)
### What this PR does / why we need it? Run: ```bash pytest -sv tests/singlecard/test_guided_decoding.py ``` Come across an error: ```bash FAILED tests/singlecard/test_guided_decoding.py::test_guided_json_completion[guidance:disable-any-whitespace] - NotImplementedError: VLLM_USE_V1=1 is not supported with --guided-decoding-backend=guidance:disable-any-whitespace. FAILED tests/singlecard/test_guided_decoding.py::test_guided_regex[guidance:disable-any-whitespace] - NotImplementedError: VLLM_USE_V1=1 is not supported with --guided-decoding-backend=guidance:disable-any-whitespace. ``` ### How was this patch tested? Run: ```bash pytest -sv tests/singlecard/test_guided_decoding.py ``` Output: ```bash =========================================================================== test session starts =========================================================================== platform linux -- Python 3.10.18, pytest-8.4.1, pluggy-1.6.0 rootdir: /home/sss/github/vllm-v0.9.1/vllm-ascend configfile: pytest.ini plugins: anyio-4.10.0, mock-3.14.1 collected 8 items tests/singlecard/test_guided_decoding.py ss.ss..s [100%] ============================================================================ warnings summary ============================================================================= <frozen importlib._bootstrap>:241 <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute <frozen importlib._bootstrap>:241 <frozen importlib._bootstrap>:241: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute ../../../miniconda3/envs/vllm-v0.9.1/lib/python3.10/site-packages/torch_npu/dynamo/torchair/__init__.py:8 /home/sss/miniconda3/envs/vllm-v0.9.1/lib/python3.10/site-packages/torch_npu/dynamo/torchair/__init__.py:8: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================================================== 3 passed, 5 skipped, 3 warnings in 193.21s (0:03:13) =========================================================== ``` --------- Signed-off-by: shen-shanshan <[email protected]>
1 parent 87a41e2 commit 234a5a4

File tree

2 files changed

+7
-5
lines changed

2 files changed

+7
-5
lines changed

.github/workflows/vllm_ascend_test.yaml

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -179,8 +179,7 @@ jobs:
179179
run: |
180180
if [[ "${{ matrix.os }}" == "linux-arm64-npu-1" ]]; then
181181
VLLM_USE_MODELSCOPE=True pytest -sv tests/singlecard/test_offline_inference.py
182-
# guided decoding doesn't work, fix it later
183-
# pytest -sv tests/singlecard/test_guided_decoding.py.py
182+
pytest -sv tests/singlecard/test_guided_decoding.py
184183
# test_ascend_config.py should be ran separately because it will regenerate the global config many times.
185184
pytest -sv tests/singlecard/test_ascend_config.py
186185
pytest -sv tests/singlecard/test_camem.py
@@ -216,8 +215,7 @@ jobs:
216215
run: |
217216
if [[ "${{ matrix.os }}" == "linux-arm64-npu-1" ]]; then
218217
VLLM_USE_MODELSCOPE=True pytest -sv tests/singlecard/test_offline_inference.py
219-
# guided decoding doesn't work, fix it later
220-
# pytest -sv tests/singlecard/test_guided_decoding.py.py
218+
pytest -sv tests/singlecard/test_guided_decoding.py
221219
pytest -sv tests/singlecard/test_camem.py
222220
# test_ascend_config.py should be ran separately because it will regenerate the global config many times.
223221
pytest -sv tests/singlecard/test_ascend_config.py

tests/singlecard/test_guided_decoding.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
"lm-format-enforcer",
3535
"xgrammar",
3636
]
37-
GuidedDecodingBackendV1 = ["xgrammar", "guidance:disable-any-whitespace"]
37+
GuidedDecodingBackendV1 = ["xgrammar", "guidance"]
3838
GuidedDecodingBackend = list(
3939
set(GuidedDecodingBackendV0 + GuidedDecodingBackendV1))
4040

@@ -94,6 +94,10 @@ def test_guided_json_completion(guided_decoding_backend: str,
9494
# xgrammar does not support json schema, will fall back to outlines, skip it
9595
pytest.skip(
9696
f"{guided_decoding_backend} will fall back to outlines, skip it")
97+
if guided_decoding_backend == "outlines":
98+
pytest.skip(
99+
f"{guided_decoding_backend} will take up too much time for json "
100+
"completion, skip it")
97101
if guided_decoding_backend not in GuidedDecodingBackendV0 and os.getenv(
98102
"VLLM_USE_V1") == "0":
99103
# guidance does not support on v0, skip it

0 commit comments

Comments
 (0)