Faster prompt A/B testing when a variant label is missing #11868
jannikmaierhoefer
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Describe the feature or potential improvement
We recommend using prompt labels for A/B tests (see docs: /docs/prompt-management/features/a-b-testing). In practice, some prompts may not have the “B” label (e.g., prob-b) yet.
When the “B” label is missing, users may implement:
try { getPrompt("prob-b") } catch { getPrompt("prod") }But the failed getPrompt() call can be slow (reported ~2s), which makes prompt resolution expensive in hot paths.
Desired outcome:
Make “label missing” a fast path so A/B prompt resolution is cheap even when variants are not fully configured.
Additional information
No response
Beta Was this translation helpful? Give feedback.
All reactions