Bedrock backend doesn't update spec field in the conversation

[dekkersp]

What happened?

If changing to a Bedrock based model, the new "spec" is not updated in the conversation. As a result for example the "spec" dropdown revert back to the old value.

In the below example all properties are correct, except the spec property. That one is still pointing to the previous spec and didn't get updated.

{
    "conversations": [
        {
            "_id": "677b96bdfbdbf368544de863",
            "conversationId": "bfe56210-fed1-4a31-b2fc-6beb638c06b9",
            "user": "67488a7994c776fd04351f33",
            "__v": 0,
            "createdAt": "2025-01-06T08:39:25.374Z",
            "endpoint": "bedrock",
            "isArchived": false,
            "messages": [
                "677b96bdfbdbf368544de862",
                "677b96bdfbdbf368544de864",
                "677ba667fbdbf368544de865",
                "677ba667fbdbf368544de866"
            ],
            "model": "anthropic.claude-3-haiku-20240307-v1:0",
            "region": "eu-west-1",
            "tags": [],
            "title": "Wat is de hoofdstad van Frankrijk",
            "updatedAt": "2025-01-06T12:37:33.596Z",
            "greeting": "...",
            "iconURL": "...",
            "promptPrefix": "...",
            "resendFiles": true,
            "spec": "OpenAI | Advanced V1.1"
        }
    ],
    "pages": 1,
    "pageNumber": 1,
    "pageSize": 25
}

This only happens when selecting a Bedrock based model, leading to the assumption that the Bedrock API endpoint doesn't correctly update the spec attribute while other implementations do.

Steps to Reproduce

1. Select a spec that is based on a Bedrock model
2. Submit an any type of request (1 + 1 = ?)
3. See that the dropdown is reversing back to the old spec value.

What browsers are you seeing the problem on?

Chrome

Relevant log output

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi thanks for the report.

The issue cannot be reproduced. It would be helpful if you provide the relevant part of your librechat.yaml file.

You may also have been experiencing the same underlying issue as #5200 since they share functionality, and is now resolved (for the spec property specifically, iconURL still needs to be included for bedrock in upcoming PR).

I tested with the following on the latest build:

modelSpecs:
  enforce: true
  prioritize: true
  list:
    - name: "Max"
      label: "Max"
      description: "Smartest model"
      iconURL: "https://code.librechat.ai/_next/image?url=%2Ficon.png&w=32&q=75"
      preset:
        default: true
        endpoint: "bedrock"
        modelLabel: "Max"
        model: "anthropic.claude-3-haiku-20240307-v1:0"
        greeting: |
          Hello there!
    - name: "Max 2"
      label: "Max 2"
      description: "Smartest model"
      iconURL: "https://code.librechat.ai/_next/image?url=%2Ficon.png&w=32&q=75"
      preset:
        endpoint: "bedrock"
        modelLabel: "Max 2"
        model: "anthropic.claude-3-haiku-20240307-v1:0"
        greeting: |
          Hello there!
    - name: "Max 3"
      label: "Max 3"
      description: "Smartest model"
      iconURL: "https://code.librechat.ai/_next/image?url=%2Ficon.png&w=32&q=75"
      preset:
        endpoint: "openAI"
        model: "chatgpt-4o-latest"
        temperature: 0.90
        modelLabel: "Max 3"
        greeting: |
          Hello! How can I help you today?
        promptPrefix: |
          You are a friendly assistant.

[danny-avila]

Even with default: true on one of the specs, did not experience the issue

[dekkersp]

Thanks, will look further into #5200

These are the models we currently have deployed (part of config as sent to the browser):

 "modelSpecs": {
        "enforce": false,
        "prioritize": false,
        "list": [
            {
                "name": "Everyday V2",
                "label": "Everyday V2",
                "preset": {
                    "endpoint": "bedrock",
                    "modelLabel": "...",
                    "model": "anthropic.claude-3-haiku-20240307-v1:0",
                    "promptPrefix": "...",
                    "greeting": "..."
                },
                "description": "...",
                "iconURL": "https://i.imgur.com/1fWeVGd.png"
            },
            {
                "name": "Advanced V1.1",
                "label": "Advanced V1.1",
                "preset": {
                    "endpoint": "azureOpenAI",
                    "modelLabel": "...",
                    "model": "gpt-4o",
                    "promptPrefix": "...",
                    "greeting": "..."
                },
                "description": "...",
                "iconURL": "https://i.imgur.com/wW0op1F.png"
            },
            {
                "name": "Custom V1",
                "label": "Custom V1",
                "preset": {
                    "endpoint": "azureOpenAI",
                    "modelLabel": "...",
                    "model": "gpt-4o",
                    "promptPrefix": "...",
                    "greeting": "..."
                },
                "description": "...",
                "iconURL": "https://i.imgur.com/95FbaHH.png"
            }
        ]
    },

[dekkersp]

One thing I noticed in the Bedrock endpoint is that the caller of getOptions uses different (1 additional) parameters than the function itself:

const options = await getOptions({
    req,
    res,
    endpointOption,
  });

const getOptions = async ({ req, endpointOption }) => {
  const {
    BEDROCK_AWS_SECRET_ACCESS_KEY,
    BEDROCK_AWS_ACCESS_KEY_ID,
    BEDROCK_AWS_SESSION_TOKEN,
    ...

Didn't do any debugging yet to see if this an issue, but seems wrong.

[danny-avila]

I'm not sure this is relevant, but I'm taking another look, thanks for your patience

[danny-avila]

So I've done a lot of testing now, specs for bedrock are working as expected, besides the icon. The icon not persisting is fixed here:
#5229

Please double check that you've reviewed the model specs documentation as it was recently updated:

https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/model_specs

One thing I noticed is that you have default: true under preset, but it should be beside it as a top-level field:
https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/model_specs#default

[dekkersp]

Thanks for the quick help, much appreciated !!!