Add promptCacheKey for any provider with npm @ai-sdk/openai

[shantur]

v2 for openai-caching when npm uses @ai-sdk/openai
Includes fix for #4386

[rekram1-node]

Yeah this one should be fine, but im curious what is your need for this? @shantur what provider are you using?

[shantur]

@rekram1-node - openai codex via CLIProxyAPI

[shantur]

Having said that this is needed for any provider that supports openai APIs

[rekram1-node]

@shantur hmm but this will still cause errors for certain people, I know people that have special proxies setup internally that use that openai provider but it goes through a proxy, it will error if you set this across the board.

Also a lot of providers handle the caching automatically, like openrouter for example doesn't need that key afaik (could be wrong)

[rekram1-node]

btw you can set this using a plugin if you need to

[shantur]

@rekram1-node - As per the official OpenAI and ai-sdk/openai docs this is the expected behaviour - https://ai-sdk.dev/providers/ai-sdk-providers/openai#responses-models

If people are using for their setups then it's incorrect setup and they should be using - https://ai-sdk.dev/providers/openai-compatible-providers with "@ai-sdk/openai-compatible"

[rekram1-node]

openai compatible doesnt support responses api, so they cant

[shantur]

What would be your suggestion how this could be handled?
Some flag in the provider config? like enablePromptCache which is by default not enabled and when enable allows promptCacheKey

[rekram1-node]

@shantur maybe not the perfect solution but this should work:

import { Plugin } from "./index"

export const SetCacheKey: Plugin = async (ctx) => {
  return {
    "chat.params": async (input, output) => {
      if (input.provider.npm && input.provider.npm === "@ai-sdk/openai") {
        output.options["promptCacheKey"] = input.sessionID
      }
    },
  }
}

[shantur]

@rekram1-node
Thanks for the workaround but I feel this is such a core feature that should be handled in the provider config where we are defining the type of the provider.

[rekram1-node]

thats fair, wanna add a setting then and we can do that?

[duggthangs]

If people are using for their setups then it's incorrect setup and they should be using - https://ai-sdk.dev/providers/openai-compatible-providers with "@ai-sdk/openai-compatible"

So "Fireworks.ai" is a provider that is supported by https://ai-sdk.dev/providers/ai-sdk-providers/fireworks and it complains about this promptCacheKey being there.

I raised the original issue that called for the removal of that code in #4386, which fixed it. This change just adds it back in.

@shantur Does this handle the use case for #4386?

[shantur]

@rekram1-node - Closing this in favor of #4654