feat: add stateless mode configuration for OpenAI Native provider #7791
Conversation
- Add openAiNativeStatelessMode boolean option to provider settings schema - Update OpenAI Native handler to respect stateless mode configuration - When enabled, forces store: false for all Responses API requests - Add comprehensive tests for stateless mode behavior Fixes #7789
dosubot
bot
added
size:L
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
| stream: false, // Non-streaming for completePrompt | ||
| store: false, // Don't store prompt completions | ||
| // Use stateless mode if configured, otherwise don't store prompt completions | ||
| store: this.options.openAiNativeStatelessMode ? false : false, |
There was a problem hiding this comment.
The ternary operator for the 'store' field is redundant here—both branches yield false. If prompt completions should never be stored (even when stateless mode is disabled), consider simply using 'store: false' with an explanatory comment. Otherwise, adjust the logic to allow storing when stateless mode is off.
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| stream: false, // Non-streaming for completePrompt | ||
| store: false, // Don't store prompt completions | ||
| // Use stateless mode if configured, otherwise don't store prompt completions | ||
| store: this.options.openAiNativeStatelessMode ? false : false, |
There was a problem hiding this comment.
This appears to be redundant logic. The expression this.options.openAiNativeStatelessMode ? false : false always evaluates to false regardless of the configuration value.
Since completePrompt is meant for non-conversational completions, it makes sense to always use stateless mode (store: false) here.
| openAiNativeServiceTier: serviceTierSchema.optional(), | ||
| // When true, forces the OpenAI Responses API to run in stateless mode (store: false) | ||
| // This prevents responses from being stored for 30 days in OpenAI's Responses API | ||
| openAiNativeStatelessMode: z.boolean().optional(), |
There was a problem hiding this comment.
Good addition to the schema! However, I notice there's no corresponding UI component to expose this setting to users. Would it be helpful to add a toggle in the provider settings UI so users don't have to manually edit configuration files?
Also, consider adding more detailed documentation about when users should enable this mode (e.g., for privacy-sensitive applications, compliance requirements, etc.).
| }) | ||
| }) | ||
|
|
||
| describe("Stateless mode configuration", () => { |
There was a problem hiding this comment.
Great test coverage! The 6 test cases cover the main scenarios well. Consider adding a few edge case tests:
- What happens if
openAiNativeStatelessModeis set to a non-boolean value? - How does this interact with other providers that might accidentally have this property set?
- Test the behavior when switching between stateless and stateful modes during runtime.
| stream: true, | ||
| store: metadata?.store !== false, // Default to true unless explicitly set to false | ||
| // Use stateless mode if configured, otherwise respect metadata.store (default true) | ||
| store: this.options.openAiNativeStatelessMode ? false : metadata?.store !== false, |
There was a problem hiding this comment.
The implementation correctly overrides the metadata.store value when stateless mode is enabled. However, have you considered making this controllable per-request through metadata? Something like:
This would give users more granular control when needed.
hannesrudolph
added
the
Issue/PR - Triage
Summary
This PR attempts to address Issue #7789 by adding a configuration option to proactively set the OpenAI Responses API provider to run in stateless mode.
Changes
openAiNativeStatelessModeboolean option to the provider settings schemastore: falsefor all Responses API requests, preventing responses from being stored for 30 daysImplementation Details
The new configuration option allows users to:
storesettings when stateless mode is enabledTesting
Privacy Benefits
When
openAiNativeStatelessModeis set totrue:Fixes #7789
Feedback and guidance are welcome!
Important
Adds
openAiNativeStatelessModeto OpenAI Native provider for stateless operation, preventing response storage.openAiNativeStatelessModeboolean option toopenAiNativeSchemainprovider-settings.ts.OpenAiNativeHandlerinopenai-native.tsto respectopenAiNativeStatelessMode, forcingstore: falsewhen enabled.storesettings when stateless mode is enabled.openai-native.spec.tsto cover stateless mode behavior, including default behavior, metadata override, and both streaming and non-streaming scenarios.openAiNativeStatelessModeistrue, responses are not stored, enhancing privacy by preventing 30-day storage in OpenAI's Responses API.This description was created by
for 753a66f. You can customize this summary. It will automatically update as commits are pushed.