Anthropic's endpoints is it possible to set the default in .env or librechat.yml to do not turn on claude3.7 thinking mode?

[harrisonhxy]

I have anthropic endpoints configured now, and the problem I'm having is that when I start the container every time I want to use the non-thinking model of claude 3.7, I need to manually turn off thinking button in the sidebar parameter. it's quite cumbersome.

I was under the impression that using presets, I could save the non-thinking models. but since some of my users are not in the habit of using Preset, I was thinking it would be a lot easier if I could have claude3.7 in a non-thinking mode by default when the container is started. in other hand, I can add the thinking model of claude3.7 via Model Spec, which would greatly simplify the process for the user, and would be much more convenient!

I've tried to use it in librechat.yml so far, but claude 3.7 still has thinking mode turned on by default:

endpoint:
    preset:
        model: "claude-3.7-sonneet"
        thinking: false
        promptCache: true
        thinkingBdget: 

[harrisonhxy]

#5034 (reply in thread)

When browsing issues, I see that prompt caching can be disabled, can it be implemented in the config file?

[Jaifroid]

I would also like a yaml setting to turn thinking off by default, instead of it being on by default (for Claude 3.7 at least).

[danny-avila]

There is no feature to customize parameters or default parameter values, I'm taking this as a feature request

[harrisonhxy]

Excuse me, Will the default shutdown of Claude-3.7-sonnet's thinking be implemented in v0.78?

[nhtruong]

Maybe Overriding Parameter Definitions feature from this PR will solve this for you?

[harrisonhxy]

First of all, thank you for your reply, but unfortunately, I did not realize what I wanted by referring to the contents of the document.

Here is my configuration in librechat for Anthropic:

endpoints:
  custom:
    # Anthropic
    - name: "Claude"
      apiKey: "${CLAUDE_API_KEY}"
      baseURL: "https://anthrpic_api_url"
      iconURL: "anthropic"
      models:
        default: [
          "claude-3.5-haiku",
          "claude-3.5-sonnet",
          "claude-3.7-sonnet"
          ]
        fetch: false
      userIdQuery: false
      titleConvo: true
      titleMethod: "completion"
      titleModel: "claude-3.5-haiku"
      summarize: false
      summaryModel: "claude-3.5-haiku"
      forcePrompt: false
      modelDisplayLabel: "Claude"
      customParams:
        defaultParamsEndpoint: 'anthropic'
        paramDefinitions:
          - key: thinking
            default: false
          - key: thinkingBudget
            default: 4000
          - key: promptCache
            default: true
          - key: maxContextTokens
            default: 20000
          - key: maxOutputTokens
            default: 12800

My question is, after I use defaultParamsEndpoint: 'anthropic', is the format of the API call still in OpenAI format? Or does the API call format point to the native format of anthropic after I use defaultParamsEndpoint: 'anthropic'?

At the moment it seems to only work if I follow the OpenAI API format when I try it.

But if it's still the OpenAI format, how does customParams pass/control anthropic's promptCache,thinking,thinkingBudget parameters? Because I notice that the promptCache and thinking functions are only supported after that using anthropic's native format.

[nhtruong]

Idk what you meant by "is the format of the API call still in OpenAI format?". When you choose 'anthropic' as the defaultParamsEndpoint, you will get anthropic parameters instead of the openAI parameters. I'm not sure what format you are referring to.

[harrisonhxy]

I'm sorry that I couldn't let you understand what I mean quickly. English is not my native language.

I would say that according to the description of custom endpoint in the LC Document, "LibreChat supports OpenAI API compatible services", that is to say, the custom endpoint accepts OpenAI-compatible API format/service.。
Although Anthropic also has some degree of adaptation to the OpenAI API format in terms of compatibility, it is not possible to implement features such as promptCache, thinking content, thinkingBudget, etc.。
Therefore, after configuring defaultParamsEndpoint: 'anthropic', if the request of Anthropic still follows the API format of OpenAI, even if the parameters of promptCache, thinking content and thinkingBudget can be configured, these parameters cannot actually be passed to Anthropic, because native Anthropic API is not used.

[Jaifroid]

@harrisonhxy Have you tried the parameters you list:

      customParams:
        defaultParamsEndpoint: 'anthropic'
        paramDefinitions:
          - key: thinking
            default: false
          - key: thinkingBudget
            default: 4000
          - key: promptCache
            default: true
          - key: maxContextTokens
            default: 20000
          - key: maxOutputTokens
            default: 12800

Do they turn off thinking by default?

[harrisonhxy]

Like my reply above, since my anthropic API is official native format (i.e., it's configured in .env), I can't effectively configure it in the custom endpoint.

I've tried a number of options and none of them have worked so far 😢.