Unable to configure conversation parameters, error on token limit

[leoneperdigao]

What happened?

I'm facing an error when I try to interact with a model (via AWS Bedrock). The model is made available via a custom inference profile, just due to region restrictions. However, I have this strange error:

Also, I can't set the parameters on the right side panel:

I did check the documentation and didn't see anything specific about it. I wonder whether it is a misconfiguration on my side.

Steps to Reproduce

Just start a new conversation and enter any prompt.

What browsers are you seeing the problem on?

Chrome

Relevant log output

No response

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[alvieirajr]

Hi @leoneperdigao. I took a quicky look on bedrock.js file from langchain node module and seams that max_tokens wasn't set, maybe because the provider used isn't on list of suported (anthropic, ai21, meta, amazon, cohere, mistral). May be this explain the "undefined".

[leoneperdigao]

Hi @alvieirajr , thanks for the comment. In fact, the provider is AWS. The only thing is that I used a custom inference profile, but the same behaviour happens with llama3 from meta, for example.

[danny-avila]

Thanks @alvieirajr that could really be the issue. There isn't enough information to determine, are you sure the region is configured correctly?

The model name is important, can you share that to test?

[leoneperdigao]

@danny-avila , I believe so:

BEDROCK_AWS_DEFAULT_REGION=eu-west-1 # A default region must be provided
BEDROCK_AWS_MODELS=eu.meta.llama3-2-3b-instruct-v1:0,3k2xuiqap2ha

The custom inference profile I mentioned was created like this:

[cloudshell-user@ip-10-134-60-203 ~]$ aws bedrock create-inference-profile --region 'eu-west-1' \
> --inference-profile-name 'aicoe-claude-3-bedrock-profile' \
> --description 'aicoe-bedrock-profile copy from eu-central-1 into eu-west-1' \
> --model-source '{"copyFrom": "arn:aws:bedrock:eu-central-1::foundation-model/anthropic.claude-3-5-sonnet-20240620-v1:0"}' \
> --tags '[{"key": "CostAllocateTag","value": "aicoe"}]'
{
    "inferenceProfileArn": "arn:aws:bedrock:eu-west-1:<ACCOUNT_NUMBER>:application-inference-profile/3k2xuiqap2ha",
    "status": "ACTIVE"
}
[cloudshell-user@ip-10-134-60-203 ~]$ 

Then the inference profile is 3k2xuiqap2ha. Just for context, I need to use inference profile because of regions restrictions (i.e., the models are accessible via cross-region inference)

[kapustor]

Hey @leoneperdigao did you figure it out? I am also trying to use LibreChat with a custom inference profile, I tried to specify it in the model list in the .env file, but I got an error then when trying to use it:

The provided model identifier is invalid.

[leoneperdigao]

Hey @leoneperdigao did you figure it out? I am also trying to use LibreChat with a custom inference profile, I tried to specify it in the model list in the .env file, but I got an error then when trying to use it:

The provided model identifier is invalid.

@kapustor it doesn't work with custom profiles, so I dropped it and used one of the existing profiles in the end.