generated from runpod-workers/worker-template
-
Notifications
You must be signed in to change notification settings - Fork 24
Open
Description
According to SGLang's Structured Outputs docs, SGLang supports constraining outputs via json_schema, regex, or ebnf, and allows choosing grammar backends (xgrammar, outlines, llguidance)—some of which (like Outlines) enforce JSON property order.
However, the current RunPod worker only accepts json_schema via the OpenAI-compatible response_format, and does not expose:
regexorebnfconstraints,--grammar-backendconfiguration (e.g., to useoutlinesfor deterministic field ordering).
This means ordered structured output is impossible with this worker, even though SGLang supports it upstream.
Example error when trying ebnf:
{
"response_format": { "type": "ebnf", "ebnf": { "schema": "..." } }
}→ ValidationError: Input should be 'text', 'json_object' or 'json_schema'
Request:
- Add support for
regexandebnfinresponse_format.type, or - Allow setting
GRAMMAR_BACKENDvia env var (e.g.,outlinesfor field-order enforcement).
This would enable use cases requiring strict output ordering or non-JSON structured formats.
[this was written by AI]
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels