Sending Custom Params in the Request Body (LibreChat + Custom FastAPI Server)

[zeeshan-bandar]

I've set up a custom FastAPI server and connected it successfully with LibreChat. Now, I'm trying to send additional custom parameters from the frontend to my FastAPI endpoint.

While passing data through headers works, I'd prefer to send these custom values in the request body instead — especially since I might need to send more complex or nested data.

Has anyone customized LibreChat to allow sending extra fields in the body of the request to a custom endpoint?
If so, could you point me to the right file or code section where the request is being constructed and sent?

Any help or guidance would be appreciated!

[danny-avila]

Look into addParams in the custom endpoint config. as well as customParams:

• https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/custom_endpoint#addparams
• https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/custom_params

[relic664]

I'm not sure either addParams or customParams function as intended.

Here's a snippet of my config:

endpoints:
  custom:
    - name: "LiteLLM"
      apiKey: "notused"
      baseURL: "http://192.168.42.144:4000/v1"
      directEndpoint: true
      addParams:
        safe_prompt: true
      customParams:
        defaultParamsEndpoint: "openAI"
        paramDefinitions:
          - key: temperature
            range: {min: 0, max: 1, step: 0.05}
            default: 0.6
          - key: top_p
            range: {min: 0, max: 1, step: 0.05}
            default: 0.95
          - key: presence_penalty
            range: {min: -2, max: 2, step: 0.1}
            default: 0.3
          - key: frequency_penalty
            range: {min: -2, max: 2, step: 0.1}
            default: 0.2
          - key: web_search
            type: boolean
            label: "Web Search"
            default: false

I can see that the payload from librechat to liteLLM (truncated for clarity)

litellm_pre_call_utils.py:670 - receiving data: ... 'body': {'model': 'glm-45-air', 'user': 'user', 'stream': True, 'messages': [{'role': 'user', 'content': 'Can you tell me a joke?'}]}} ... }}}

So naturally liteLLM reads this as:

LiteLLM: Params passed to completion() {'model': 'glm-45-air', 'functions': None, 'function_call': None, 'temperature': None, 'top_p': None, 'n': None, 'stream': True, 'stream_options': None, 'stop': None, 'max_tokens': None, 'max_completion_tokens': None, 'modalities': None, 'prediction': None, 'audio': None, 'presence_penalty': None, 'frequency_penalty': None, 'logit_bias': None, 'user': '688bdb61e9aa48aa8f50a207', 'custom_llm_provider': 'openai', 'response_format': None, 'seed': None, 'tools': None, 'tool_choice': None, 'max_retries': 0, 'logprobs': None, 'top_logprobs': None, 'extra_headers': None, 'api_version': None, 'parallel_tool_calls': None, 'drop_params': None, 'allowed_openai_params': None, 'reasoning_effort': None, 'additional_drop_params': None, 'messages': [{'role': 'user', 'content': 'Can you tell me a joke?'}], 'thinking': None, 'web_search_options': None}

and passes:

{
    "model": "glm-45-air",
    "user": "user",
    "stream": true,
    "messages": [
        {
            "role": "user",
            "content": "Can you tell me a joke?"
        }
    ]
}

Separately, I tried the example from the docs here to try addparams

By using the example verbatim:

endpoints:
  custom:
    - name: "LiteLLM"
      apiKey: "notused"
      baseURL: "http://192.168.42.144:4000/v1"
      directEndpoint: true
      addParams:
        safe_prompt: true

And I never see it show up in the payload to liteLLM. This is on 0.8.0-rc1 (docker). (Note: strictly speaking LiteLLM would expect an extra_body payload not a raw safe_prompts, but this is still illustrative)

I would expect 1) default values get passed if set and 2) addParams are present in the payload

Related to #7251, that was a few months ago so I'm not sure if it was ever resolved

[danny-avila]

addParams are working as intended on LibreChat side, but it seems the upstream dependency is stripping the params, will look into this very soon.

[danny-avila]

This is resolved now in the latest update, let me know if you still have issues with addParams

[relic664]

addParams is back to working now in Docker tag 7147bce3c3d001a2af4f2e5b53c131e6985b5331