Skip to content

Conversation

Jack-Khuu
Copy link
Contributor

@Jack-Khuu Jack-Khuu commented Oct 7, 2025

guided_decoding was getting recursively deconstructed. This keeps the object class

HF_HUB_DISABLE_XET=1 python -m tests.sandbox.vllm.main --config tests/sandbox/vllm/llama3_8b.yaml

With guided_decoding on:
image

@Jack-Khuu Jack-Khuu requested a review from pbontrager October 7, 2025 21:44
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 7, 2025
@Jack-Khuu Jack-Khuu changed the title Fix conversion for Spec decoding Fix conversion for Guided decoding Oct 7, 2025
@Jack-Khuu Jack-Khuu merged commit b3870f5 into main Oct 8, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants