File search tool returning error in Agent mode after updating to v0.7.7 from v0.7.7-rc

[sinansoulier]

What happened?

Hello, I am currently facing an issue with file search, while working on a customized / forked version of LibreChat (the exact same issue is reproductible in the public version of the project, on the main branch). The tool displays an Error label in the frontend, and does not return any response from the rag_api.

One other inconsistency is the type of the tool_call one the file search tool has been called. The name of the tool is 'file_search' but its type is 'tool_call' and not 'file_search'. Should it be considered a normal behavior? Here is a picture below describing the tool call object:

(Sensitive content has been hidden on this screenshot).

Specifications

Model: gpt-4o and text-embedding-3-large through LiteLLM
rag_api version: latest (dev-lite and dev -> issue occurs on both)
runs: Docker exclusively

Thank you in advance for your feedback!

Configuration files

librechat.yaml (Content, without credentials)

version: 1.2.1
 
endpoints:
  agents:
    disableBuilder: false
    capabilities: ["execute_code", "file_search", "actions", "tools"]
  azureAssistants:
    pollIntervalMs: 500
    timeoutMs: 10000
    capabilities: ["code_interpreter", "actions", "tools", "image_vision"]
  custom:
      - name: "<modelDisplayLabel>"
        apiKey: "<api_key>"
        baseURL: "<base_url>"
        models:
          default: ["gpt-4o"]
          fetch: true
        titleConvo: true
        titleModel: "gpt-4o"
        summarize: false
        summaryModel: "gpt-4o"
        forcePrompt: false
        modelDisplayLabel: "<modelDisplayLabel>"

fileConfig:
  endpoints:
    azureOpenAI:
      fileLimit: 5
      fileSizeLimit: 20
      totalSizeLimit: 50
      supportedMimeTypes:
        - "image/.*"
        - "application/pdf"
        - "text/.*"
        - "application/msword"
        - "application/vnd.openxmlformats-officedocument.*"
    default:
      totalSizeLimit: 20
  serverFileSizeLimit: 100

---

docker-compose.override.yml (Content)

services:
  api:
    user: root
    container_name: LibreChat
    ports:
      - "${PORT}:${PORT}"
    depends_on:
      - mongodb
      - rag_api
    image: librechat
    build:
      context: .
      target: node
    environment:
      - HOST=${HOST}
      - MONGO_URI=${MONGO_URI}
      - MEILI_HOST=${MEILI_HOST}
      - RAG_PORT=${RAG_PORT:-8000}
      - RAG_API_URL=${RAG_API_URL}
    volumes:
      - type: bind
        source: ./.env
        target: /app/.env
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml
      - ./images:/app/client/public/images
      - ./uploads:/app/uploads
      - ./logs:/app/api/logs
    extra_hosts:
      - "host.docker.internal:host-gateway"
    env_file:
      - .env

  mongodb:
    container_name: chat-mongodb
    image: mongo
    restart: always
    volumes:
      - ./data-node:/data/db
    command: mongod --noauth
  meilisearch:
    container_name: chat-meilisearch
    image: getmeili/meilisearch:v1.12.3
    restart: always
    environment:
      - MEILI_HOST=${MEILI_HOST}
      - MEILI_NO_ANALYTICS=${MEILI_NO_ANALYTICS}
      - MEILI_MASTER_KEY=${MEILI_MASTER_KEY}
    volumes:
      - ./meili_data_v1.12:/meili_data
  vectordb:
    container_name: vectordb
    image: ankane/pgvector:latest
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    restart: always
    volumes:
      - pgdata2:/var/lib/postgresql/data
  rag_api:
    container_name: rag_api
    image: ghcr.io/danny-avila/librechat-rag-api-dev-lite:latest
    environment:
      - DB_HOST=${DB_HOST}
      - RAG_PORT=${RAG_PORT:-8000}
    restart: always
    depends_on:
      - vectordb
    ports:
      - 8000:8000
    env_file:
      - .env

volumes:
  pgdata2:

---

.env (Content, without credentials)

#=====================================================================#
#                       LibreChat Configuration                       #
#=====================================================================#
# Please refer to the reference documentation for assistance          #
# with configuring your LibreChat environment.                        #
#                                                                     #
# https://www.librechat.ai/docs/configuration/dotenv                  #
#=====================================================================#
 
#==================================================#
#               Server Configuration               #
#==================================================#
 
HOST=0.0.0.0
PORT=3080
 
 
DOMAIN_CLIENT=http://localhost:3080
DOMAIN_SERVER=http://localhost:3080
 
NO_INDEX=true
 
#===============#
# JSON Logging  #
#===============#
 
# Use when process console logs in cloud deployment like GCP/AWS
CONSOLE_JSON=false
 
#===============#
# Debug Logging #
#===============#
 
DEBUG_LOGGING=true
DEBUG_CONSOLE=true
 
#=============#
# Permissions #
#=============#
 
 UID=1000
 GID=1000
 
#===============#
# Configuration #
#===============#
# Use an absolute path, a relative path, or a URL
 
 CONFIG_PATH="./librechat.yaml"
 
#===================================================#
#                     Endpoints                     #
#===================================================#
 
ENDPOINTS=azureOpenAI,agents #,gptPlugins,assistants,azureAssistants
 
PROXY=
 
#===================================#
# Known Endpoints - librechat.yaml  #
#===================================#
# https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints
 
# ANYSCALE_API_KEY=
# APIPIE_API_KEY=
# FIREWORKS_API_KEY=
# GROQ_API_KEY=
# HUGGINGFACE_TOKEN=
# MISTRAL_API_KEY=
# OPENROUTER_KEY=
# PERPLEXITY_API_KEY=
# SHUTTLEAI_API_KEY=
# TOGETHERAI_API_KEY=
 
#============#
# Anthropic  #
#============#
 
ANTHROPIC_API_KEY=user_provided
 
#============#
# Azure      #
#============#

 
#============#
# BingAI     #
#============#
 
BINGAI_TOKEN=user_provided
# BINGAI_HOST=https://cn.bing.com
 
#============#
# Google     #
#============#
 
GOOGLE_KEY=user_provided
 
#============#
# OpenAI     #
#============#
 
OPENAI_API_KEY=user_provided
 
DEBUG_OPENAI=false
 
# TITLE_CONVO=false
# OPENAI_TITLE_MODEL=gpt-3.5-turbo
 
# OPENAI_SUMMARIZE=true
# OPENAI_SUMMARY_MODEL=gpt-3.5-turbo
 
# OPENAI_FORCE_PROMPT=true
 
# OPENAI_REVERSE_PROXY=
 
# OPENAI_ORGANIZATION=
 
#====================#
#   Assistants API   #
#====================#
 
ASSISTANTS_API_KEY=user_provided
#============#
# OpenRouter #
#============#
# !!!Warning: Use the variable above instead of this one. Using this one will override the OpenAI endpoint
# OPENROUTER_API_KEY=
 
#============#
# Plugins    #
#============#
 
DEBUG_PLUGINS=true
 
CREDS_KEY=<creds_key>
CREDS_IV=<creds_IV>
 
# Azure AI Search
#-----------------
AZURE_AI_SEARCH_SERVICE_ENDPOINT=<azure_ai_search_service_endpoint>
AZURE_AI_SEARCH_INDEX_NAME=labai_index_axel
AZURE_AI_SEARCH_API_KEY=<azure_ai_search_key>
 
AZURE_AI_SEARCH_API_VERSION=2023-10-01-preview
AZURE_AI_SEARCH_SEARCH_OPTION_QUERY_TYPE=
AZURE_AI_SEARCH_SEARCH_OPTION_TOP=
AZURE_AI_SEARCH_SEARCH_OPTION_SELECT=
 
# DALL·E
#----------------
# DALLE_API_KEY=
# DALLE3_API_KEY=
# DALLE2_API_KEY=
# DALLE3_SYSTEM_PROMPT=
# DALLE2_SYSTEM_PROMPT=
# DALLE_REVERSE_PROXY=
# DALLE3_BASEURL=
# DALLE2_BASEURL=
 
# DALL·E (via Azure OpenAI)
# Note: requires some of the variables above to be set
#----------------
# DALLE3_AZURE_API_VERSION=
# DALLE2_AZURE_API_VERSION=
 
# Google
#-----------------
GOOGLE_SEARCH_API_KEY=
GOOGLE_CSE_ID=
 
# SerpAPI
#-----------------
SERPAPI_API_KEY=
 
# Stable Diffusion
#-----------------
SD_WEBUI_URL=http://host.docker.internal:7860
 
# Tavily
#-----------------
TAVILY_API_KEY=
 
# Traversaal
#-----------------
TRAVERSAAL_API_KEY=
 
# WolframAlpha
#-----------------
WOLFRAM_APP_ID=
 
# Zapier
#-----------------
ZAPIER_NLA_API_KEY=
 
#==================================================#
#                      Search                      #
#==================================================#
 
SEARCH=true
MEILI_NO_ANALYTICS=true
MEILI_HOST=http://meilisearch:7700
MEILI_MASTER_KEY=<meili_master_key>
 
#===================================================#
#                    User System                    #
#===================================================#
 
#========================#
# Moderation             #
#========================#
 
OPENAI_MODERATION=false
OPENAI_MODERATION_API_KEY=
# OPENAI_MODERATION_REVERSE_PROXY=
 
BAN_VIOLATIONS=true
BAN_DURATION=1000 * 60 * 60 * 2
BAN_INTERVAL=20
 
LOGIN_VIOLATION_SCORE=1
REGISTRATION_VIOLATION_SCORE=1
CONCURRENT_VIOLATION_SCORE=1
MESSAGE_VIOLATION_SCORE=1
NON_BROWSER_VIOLATION_SCORE=20
 
LOGIN_MAX=7
LOGIN_WINDOW=5
REGISTER_MAX=5
REGISTER_WINDOW=60
 
LIMIT_CONCURRENT_MESSAGES=true
CONCURRENT_MESSAGE_MAX=2
 
LIMIT_MESSAGE_IP=true
MESSAGE_IP_MAX=40
MESSAGE_IP_WINDOW=1
 
LIMIT_MESSAGE_USER=false
MESSAGE_USER_MAX=40
MESSAGE_USER_WINDOW=1
 
ILLEGAL_MODEL_REQ_SCORE=5
 
#========================#
# Balance                #
#========================#
 
CHECK_BALANCE=false
 
#========================#
# Registration and Login #
#========================#
 
ALLOW_EMAIL_LOGIN=true
ALLOW_REGISTRATION=true
ALLOW_SOCIAL_LOGIN=false
ALLOW_SOCIAL_REGISTRATION=false
 
SESSION_EXPIRY=1000 * 60 * 15
REFRESH_TOKEN_EXPIRY=(1000 * 60 * 60 * 24) * 7
 
JWT_SECRET=<jwt_token>
JWT_REFRESH_SECRET=<jwt_token>
 
# Discord
DISCORD_CLIENT_ID=
DISCORD_CLIENT_SECRET=
DISCORD_CALLBACK_URL=/oauth/discord/callback
 
# Facebook
FACEBOOK_CLIENT_ID=
FACEBOOK_CLIENT_SECRET=
FACEBOOK_CALLBACK_URL=/oauth/facebook/callback
 
# GitHub
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
GITHUB_CALLBACK_URL=/oauth/github/callback
 
# Google
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
GOOGLE_CALLBACK_URL=/oauth/google/callback
 
# OpenID
OPENID_CLIENT_ID=
OPENID_CLIENT_SECRET=
OPENID_ISSUER=
OPENID_SESSION_SECRET=
OPENID_SCOPE="openid profile email"
OPENID_CALLBACK_URL=/oauth/openid/callback
OPENID_REQUIRED_ROLE=
OPENID_REQUIRED_ROLE_TOKEN_KIND=
OPENID_REQUIRED_ROLE_PARAMETER_PATH=
 
OPENID_BUTTON_LABEL=
OPENID_IMAGE_URL=
 
#========================#
# Email Password Reset   #
#========================#
 
EMAIL_SERVICE=
EMAIL_HOST=
EMAIL_PORT=25
EMAIL_ENCRYPTION=
EMAIL_ENCRYPTION_HOSTNAME=
EMAIL_ALLOW_SELFSIGNED=
EMAIL_USERNAME=
EMAIL_PASSWORD=
EMAIL_FROM_NAME=
[email protected]
 
#========================#
# Firebase CDN           #
#========================#
 
FIREBASE_API_KEY=
FIREBASE_AUTH_DOMAIN=
FIREBASE_PROJECT_ID=
FIREBASE_STORAGE_BUCKET=
FIREBASE_MESSAGING_SENDER_ID=
FIREBASE_APP_ID=
 
#===================================================#
#                        UI                         #
#===================================================#
 
APP_TITLE=EforGroup
# CUSTOM_FOOTER="My custom footer"
HELP_AND_FAQ_URL=https://librechat.ai
 
#==================================================#
#                      Others                      #
#==================================================#
#   You should leave the following commented out
 
# Debug settings
DEBUG_LOGGING=true
DEBUG_CONSOLE=true
 
#==================================================#
#                   RAG API Config                   #
#==================================================#
 
EMBEDDINGS_PROVIDER=openai
RAG_OPENAI_BASEURL=<base_url>
RAG_OPENAI_API_KEY=<api_key>
EMBEDDINGS_MODEL=text-embedding-3-large

RAG_PORT=8000
# Vector Database Configuration
# VECTOR_DB_TYPE=pgvector
POSTGRES_DB=mydatabase
POSTGRES_USER=myuser
POSTGRES_PASSWORD=mypassword
DB_HOST=vectordb
# DB_PORT=5432
# COLLECTION_NAME=testcollection
DEBUG_RAG_API=true
MONGO_URI=mongodb://mongodb:27017/LibreChat
RAG_API_URL=http://rag_api:${RAG_PORT:-8000}
# RAG_OPENAI_API_KEY=sk-your-openai-api-key-example
# VECTOR_DB_TYPE=atlas-mongo
# COLLECTION_NAME=testcollection
# ATLAS_SEARCH_INDEX=eforlabia
 
# Debug Settings
DEBUG_RAG_API=true
USER_AGENT=LibreChat/1.0
#EMBEDDING_SIZE=1536
DEBUG_LOGGING=true
DEBUG_CONSOLE=true
 
# File Upload Settings
FILE_UPLOAD=true

Version Information

v0.7.7 (main branch, commit c4fea9c)

Steps to Reproduce

1. Create an agent named Test without adding a prompt.
2. Upload a PDF file in a new chat, and ask a specific information that can be found in the file (e.g. author list of a research paper)

What browsers are you seeing the problem on?

Chrome

Relevant log output

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Relevant log output from the issue template was not provided.

Can't reproduce:

You'll see an "Error" label when there is truly an error, for which we need to check your logs

[sinansoulier]

Thank you for your feedback!

My apologies, here are the logs from each of the docker compose services, attached to this message.

logs_librechat.txt
logs_rag_api.txt
logs_meilisearch.txt
logs_mongodb.txt
logs_vectordb.txt

UPDATE: For privacy reasons, I reproduced the error on the public version of LibreChat, at the commit aed468c of the main branch

Additionaly, my assumption is that the issue comes from our configuration (either librechat.yaml, docker-compose.override.yml or the environment file .env), as everything seems to be working as intended is the example you attached.

Thank you again for your time and help!

[danny-avila]

Hi @sinansoulier

Try recreating the issue first, then share the relevant part of the logs. I could not find anything related to it in your logs.

Also, error logs were not shared.

[sinansoulier]

Hi, thank you for your feedback. The problem is, as other users have mentioned below, there are no error logs. Everything seems to be either caught or not logged.

[sinansoulier]

The logs I shared above contain logging of my activity while recreating the issue

[doubianimehdi]

Hi @danny-avila thank you again for your AMAZING work ! i'm François' manager, and we have chosen librechat for our internal chatbot and we are facing this issue, it's really preventing us to move forward , and it's baffling us ... any help will be more than welcome ! thank you !

[paulotkh]

+1

[paulotkh]

I am getting the same issue whenever I use any tool with the model gpt-4o.

[sinansoulier]

Are you using LiteLLM? Azure? OpenAI?

[sinansoulier]

Do you have this error as well in your logs? Unhandled error type No stepId found for tool_call_id undefined

[paulotkh]

I'm using LiteLLM and there is nothing in the logs.
When using gpt-4o, it just shows "Error" on the UI. In the logs there is nothing.

[sinansoulier]

same here actually, the error I mentioned was temporary

[rioiart]

Yes, it's not just file search, no tool calling works for gpt-4o for instance. But when the agent is configured with claude everything works. Also connecting through LiteLLM. I actually have a related opened issue, initially I thought it's just about images, but then I realized all tool calls result in this non-informative error when using gpt-4o.

[Haherzke]

Yes, it's not just file search, no tool calling works for gpt-4o for instance. But when the agent is configured with claude everything works. Also connecting through LiteLLM. I actually have a related opened issue, initially I thought it's just about images, but then I realized all tool calls result in this non-informative error when using gpt-4o.

I'm not sure if it's gonna help, because this was already with version 0.7.7 and before, but we had a similar issue with Agents litellm + gpt-4o + Tool Calling. With some debugging we flipped a boolean variable "streamUsage" in the LibreChat file: api\server\controllers\agents\run.js in the "createRun" function to false. Then litellm+gpt-4o+Tools worked.

[doubianimehdi]

@Haherzke thank you for your feedback, my question is , what effect does it have on the answer of the LLM using the agents ? is the behavior different ? or just we have to wait for the complete response to show up ?

[Haherzke]

@doubianimehdi
Not that I saw, just try it out yourself locally. It has something to do with token usage information, as it says in that file.

[danny-avila]

Thanks @Haherzke, that's a good hint. LiteLLM likely does not support the new format for requesting token usage metadata:

  stream_options: {
    include_usage: true
  },

it used to be just usage: true. I'll try to add some special handling here, I believe OpenAI may no longer accept the old way.

If someone wants to report this specific point to LiteLLM, as it should also be handled on their end, let me know. I'm not sure if the "fragmentation" mentioned by @sinansoulier has any bearing on this as that might be an independent issue.

[danny-avila]

I did my part to improve usage field compatibility for custom endpoints here: #6803

LiteLLM may also still need to make the improvement on their end