feat: add openAiNativeStatelessMode configuration option #7792
Conversation
- Add openAiNativeStatelessMode boolean option to OpenAI Native provider settings - Update provider to check this setting and override metadata.store accordingly - Add comprehensive tests for the new configuration option - Maintain backward compatibility (defaults to current behavior) Fixes #7789
dosubot
bot
added
size:L
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
| stream: false, // Non-streaming for completePrompt | ||
| store: false, // Don't store prompt completions | ||
| // Use stateless mode if configured, otherwise don't store prompt completions | ||
| store: this.options.openAiNativeStatelessMode ? false : false, |
There was a problem hiding this comment.
In completePrompt, the code sets the store field as:
store: this.options.openAiNativeStatelessMode ? false : false
This always yields false. If prompt completions are meant to be non-stored regardless of configuration, consider simplifying to a literal false and adding a comment to explain the rationale.
| store: this.options.openAiNativeStatelessMode ? false : false, | |
| \t\t\t\tstore: false, // prompt completions are never stored (see rationale above) |
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| stream: false, // Non-streaming for completePrompt | ||
| store: false, // Don't store prompt completions | ||
| // Use stateless mode if configured, otherwise don't store prompt completions | ||
| store: this.options.openAiNativeStatelessMode ? false : false, |
There was a problem hiding this comment.
Is this intentional? The expression this.options.openAiNativeStatelessMode ? false : false always evaluates to false. Could we simplify this to just store: false since prompt completions shouldn't be stored regardless of the stateless mode setting?
| stream: true, | ||
| store: metadata?.store !== false, // Default to true unless explicitly set to false | ||
| // Check if stateless mode is enabled in configuration, otherwise use metadata.store | ||
| store: this.options.openAiNativeStatelessMode ? false : metadata?.store !== false, |
There was a problem hiding this comment.
Consider extracting the store property logic into a helper method like getStoreValue(metadata) since this same logic appears in both line 259 and line 1291. This would improve maintainability and ensure consistency.
| expect(secondCallBody.previous_response_id).toBe("resp_789") | ||
| }) | ||
|
|
||
| it("should respect openAiNativeStatelessMode configuration", async () => { |
There was a problem hiding this comment.
Would it be helpful to add a test case for when openAiNativeStatelessMode is true AND metadata.store is explicitly set to true? This would verify that the global setting takes precedence over per-request settings as documented.
hannesrudolph
added
the
Issue/PR - Triage
Closes #7789
Summary
This PR adds a new configuration option
openAiNativeStatelessModethat allows users to globally set the OpenAI provider to stateless mode, preventing responses from being stored on OpenAI servers.Changes
openAiNativeStatelessModeboolean option to the provider settings schemametadata.storeaccordinglyHow to Use
Users can enable stateless mode by adding the following to their settings:
{ "openAiNativeStatelessMode": true }When enabled, all OpenAI API requests will include
store: false, preventing response storage on OpenAI servers. This is useful for users who prefer local state management over server-side state.Testing
Notes
openAiNativeStatelessModeistrue, it overrides any per-requestmetadata.storesettingsprevious_response_idImportant
Adds
openAiNativeStatelessModeoption to control response storage in OpenAI native provider, with tests ensuring correct behavior and backward compatibility.openAiNativeStatelessModeoption toprovider-settings.tsto control response storage in OpenAI native provider.openai-native.ts, modifiesOpenAiNativeHandlerto respectopenAiNativeStatelessMode, settingstore: falsewhen enabled.metadata.storesettings whenopenAiNativeStatelessModeistrue.openai-native.spec.tsto verifyopenAiNativeStatelessModebehavior.storeis set tofalsewhenopenAiNativeStatelessModeis enabled and respectsmetadata.storewhen disabled.This description was created by
for 53df91c. You can customize this summary. It will automatically update as commits are pushed.