Losing context

[dtarz]

What happened?

After the first very long message, AI model completely loses the context. It doesn't understand at all what it was talking about. In earlier versions I didn't have this problem, the context was remembered even after a series of very long messages.

Version Information

ghcr.io/danny-avila/librechat-dev-api            latest          39587eae1dea   15 hours ago    1.19GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite   latest          6a1dfa1c7fe0   6 days ago      1.47GB

Steps to Reproduce

1. Install v.0.7.8 via Docker on Ubuntu 22
2. Set API_KEY
3. Send long-context message.
4. Ask something about the first message.

What browsers are you seeing the problem on?

Firefox

Relevant log output

2025-06-26T16:11:55.832Z info: [Optional] Redis not initialized.
2025-06-26T16:11:57.707Z info: [Optional] IoRedis not initialized for rate limiters.
2025-06-26T16:11:57.839Z info: Connected to MongoDB
2025-06-26T16:11:57.840Z info: [indexSync] Starting index synchronization check...
2025-06-26T16:11:58.054Z info: Custom config file loaded:
2025-06-26T16:11:58.057Z debug: Web search serperApiKey: Using environment variable SERPER_API_KEY (not set in environment, user provided value)
2025-06-26T16:11:58.055Z info: {
  "version": "1.2.8",
  "cache": true
}
2025-06-26T16:11:58.055Z debug: Custom config:
{
  version: "1.2.8",
  cache: true,
}
2025-06-26T16:11:58.078Z info: RAG API is running and reachable at http://rag_api:8000.
2025-06-26T16:11:58.097Z debug: [MEILI_SYNC:meili-index-sync] Creating initial flow state

2025-06-26T16:14:50.338Z debug: [BaseClient] Loading history:
{
  conversationId: "b399c657-b445-49bf-a24b-f2892d145c87",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
}
2025-06-26T16:14:50.875Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 31650,
  maxContextTokens: 115200,
}
2025-06-26T16:14:50.875Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 31650,
  maxContextTokens: 115200,
}
2025-06-26T16:14:50.876Z debug: [BaseClient] tokenCountMap:
{
  ba4b4927-25eb-4f32-bd3e-6dd52a9a718d: 83547,
}
2025-06-26T16:14:50.876Z debug: [BaseClient]
{
  promptTokens: 83550,
  remainingContextTokens: 31650,
  payloadSize: 1,
  maxContextTokens: 115200,
}
2025-06-26T16:14:50.877Z debug: [BaseClient] tokenCountMap
{
  ba4b4927-25eb-4f32-bd3e-6dd52a9a718d: 83547,
...

2025-06-26T16:16:15.733Z debug: [spendTokens] conversationId: b399c657-b445-49bf-a24b-f2892d145c87 | Context: message | Token usage:
{
  promptTokens: 224030,
  completionTokens: 5780,
}
2025-06-26T16:16:15.781Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:16:15.784Z debug: [spendTokens] No transactions incurred against balance
2025-06-26T16:16:15.811Z debug: [AgentController] Request closed
2025-06-26T16:16:15.832Z warn: [api/server/controllers/agents/client.js #titleConvo] Error getting endpoint config
2025-06-26T16:16:22.013Z debug: [spendTokens] conversationId: b399c657-b445-49bf-a24b-f2892d145c87 | Context: title | Token usage:
{
  promptTokens: 116724,
  completionTokens: 62,
}
2025-06-26T16:16:22.015Z debug: [saveConvo] api/server/services/Endpoints/agents/title.js
2025-06-26T16:16:22.023Z debug: [spendTokens] No transactions incurred against balance
2025-06-26T16:16:22.031Z debug: [AgentController] Title generation started
2025-06-26T16:16:22.031Z debug: [AgentController] Title generation completed
2025-06-26T16:16:22.032Z debug: [AgentController] Performing cleanup
2025-06-26T16:16:22.032Z debug: [AgentController] Cleaning up abort controller
2025-06-26T16:16:22.032Z debug: [AgentController] Cleanup completed
2025-06-26T16:16:40.310Z debug: [FinalizationRegistry] Cleaning up client for user 685d5a5e17aeea837c97f64


{
  promptTokens: 5802,
  remainingContextTokens: 109398,
  payloadSize: 2,
  maxContextTokens: 115200,
}
2025-06-26T16:21:59.799Z debug: [BaseClient] tokenCountMap
{
  6ffe6fca-20fb-4c06-bb72-961d18b36b3b: 5780,
  2b144a2d-a857-4454-9b0a-52943064ce28: 19,
}
2025-06-26T16:21:59.800Z debug: [BaseClient] userMessage
{
  messageId: "2b144a2d-a857-4454-9b0a-52943064ce28",
  parentMessageId: "6ffe6fca-20fb-4c06-bb72-961d18b36b3b",
  conversationId: "b399c657-b445-49bf-a24b-f2892d145c87",
  sender: "User",
  text: "Tab documents should be was always clean...",
  isCreatedByUser: true,
  tokenCount: 19,
}
2025-06-26T16:21:59.830Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:22:08.878Z debug: [spendTokens] conversationId: b399c657-b445-49bf-a24b-f2892d145c87 | Context: message | Token usage:
{
  promptTokens: 1434,
  completionTokens: 334,
}
2025-06-26T16:22:08.888Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:22:08.890Z debug: [spendTokens] No transactions incurred against balance
2025-06-26T16:22:08.917Z debug: [AgentController] Request closed
2025-06-26T16:22:08.922Z debug: [AgentController] Performing cleanup
2025-06-26T16:22:08.922Z debug: [AgentController] Cleaning up abort controller
2025-06-26T16:22:08.923Z debug: [AgentController] Cleanup completed

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Without an example conversation, there's no way to help you troubleshoot. Try exporting the conversation from the top-right and share if you don't mind. There's also an upper limit for context handling of about 90%, to account for margin of error, so set your max context accordingly.

[dtarz]

Here is the full log... The first message contain the app of 10k lines of code.

2025-06-26T16:49:19.482Z debug: [BaseClient] Loading history:
{
  conversationId: "0690dcbd-cc63-4f75-be37-2c429d9be9f5",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
}
2025-06-26T16:49:20.007Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 24734,
  maxContextTokens: 115200,
}
2025-06-26T16:49:20.008Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 24734,
  maxContextTokens: 115200,
}
2025-06-26T16:49:20.008Z debug: [BaseClient] tokenCountMap:
{
  935b1124-e2cf-4da3-a67b-e6c9fd168eec: 90463,
}
2025-06-26T16:49:20.008Z debug: [BaseClient]
{
  promptTokens: 90466,
  remainingContextTokens: 24734,
  payloadSize: 1,
  maxContextTokens: 115200,
}
2025-06-26T16:49:20.009Z debug: [BaseClient] tokenCountMap
{
  935b1124-e2cf-4da3-a67b-e6c9fd168eec: 90463,
}
2025-06-26T16:49:20.018Z debug: [BaseClient] userMessage
{
  messageId: "935b1124-e2cf-4da3-a67b-e6c9fd168eec",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "0690dcbd-cc63-4f75-be37-2c429d9be9f5",
  sender: "User",
  text: "# Task \n\n\n\n*rnCreate a list of tabs that don't exist in the new system ### old main.html```\n{% ... [truncated]",
  isCreatedByUser: true,
  tokenCount: 90463,
}
2025-06-26T16:49:20.197Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:50:09.156Z debug: [spendTokens] conversationId: 0690dcbd-cc63-4f75-be37-2c429d9be9f5 | Context: message | Token usage:
{
  promptTokens: 241584,
  completionTokens: 2305,
}
2025-06-26T16:50:09.198Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:50:09.201Z debug: [spendTokens] No transactions incurred against balance
2025-06-26T16:50:09.229Z debug: [AgentController] Request closed
2025-06-26T16:50:09.255Z warn: [api/server/controllers/agents/client.js #titleConvo] Error getting endpoint config
2025-06-26T16:50:14.237Z debug: [spendTokens] conversationId: 0690dcbd-cc63-4f75-be37-2c429d9be9f5 | Context: title | Token usage:
{
  promptTokens: 121130,
  completionTokens: 65,
}
2025-06-26T16:50:14.239Z debug: [saveConvo] api/server/services/Endpoints/agents/title.js
2025-06-26T16:50:14.247Z debug: [spendTokens] No transactions incurred against balance
2025-06-26T16:50:14.255Z debug: [AgentController] Title generation started
2025-06-26T16:50:14.255Z debug: [AgentController] Title generation completed
2025-06-26T16:50:14.255Z debug: [AgentController] Performing cleanup
2025-06-26T16:50:14.256Z debug: [AgentController] Cleaning up abort controller
2025-06-26T16:50:14.256Z debug: [AgentController] Cleanup completed
2025-06-26T16:50:28.875Z debug: [BaseClient] Loading history:
{
  conversationId: "0690dcbd-cc63-4f75-be37-2c429d9be9f5",
  parentMessageId: "8a4d751a-b388-4a79-93fa-c49a82a9e971",
}
2025-06-26T16:50:28.885Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 112442,
  maxContextTokens: 115200,
}
2025-06-26T16:50:28.885Z debug: [BaseClient] Difference between original payload (3) and context (2): 1
2025-06-26T16:50:28.885Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 112442,
  maxContextTokens: 115200,
}
2025-06-26T16:50:28.886Z debug: [BaseClient] tokenCountMap:
{
  8a4d751a-b388-4a79-93fa-c49a82a9e971: 2305,
  41a72aa3-52fe-4f27-b5c9-04b69ad01435: 450,
}
2025-06-26T16:50:28.886Z debug: [BaseClient]
{
  promptTokens: 2758,
  remainingContextTokens: 112442,
  payloadSize: 2,
  maxContextTokens: 115200,
}
2025-06-26T16:50:28.886Z debug: [BaseClient] tokenCountMap
{
  8a4d751a-b388-4a79-93fa-c49a82a9e971: 2305,
  41a72aa3-52fe-4f27-b5c9-04b69ad01435: 450,
}
2025-06-26T16:50:28.886Z debug: [BaseClient] userMessage
{
  messageId: "41a72aa3-52fe-4f27-b5c9-04b69ad01435",
  parentMessageId: "8a4d751a-b388-4a79-93fa-c49a82a9e971",
  conversationId: "0690dcbd-cc63-4f75-be37-2c429d9be9f5",
  sender: "User",
  text: "## Answer format\nIf you want to clarify something, ask a question or suggest a solution, t.... [truncated]",
  isCreatedByUser: true,
  tokenCount: 450,
}
2025-06-26T16:50:28.914Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:50:42.087Z debug: [spendStructuredTokens] conversationId: 0690dcbd-cc63-4f75-be37-2c429d9be9f5 | Context: message | Token usage:
{
    promptTokens.input: 20,
    promptTokens.write: 2588,
    promptTokens.read: 0,
  completionTokens: 567,
}
2025-06-26T16:50:42.100Z debug: [spendStructuredTokens] No transactions incurred against balance
2025-06-26T16:50:42.102Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-06-26T16:50:42.123Z debug: [AgentController] Request closed
2025-06-26T16:50:42.127Z debug: [AgentController] Performing cleanup
2025-06-26T16:50:42.127Z debug: [AgentController] Cleaning up abort controller
2025-06-26T16:50:42.128Z debug: [AgentController] Cleanup completed

Anthropic logs:

2025-06-26 19:50:28	req_011CQXBEJp2VM7eBE5NRKEU6	claude-sonnet-4-20250514	
1304
	560	Streaming	Standard	
2025-06-26 19:50:09	req_011CQXBCrwkAfAD5SJczSty6	claude-sonnet-4-20250514	
121130
	65	HTTP	Standard	
2025-06-26 19:49:20	req_011CQXB9FRzKaqBRYADZce22	claude-sonnet-4-20250514	
120792
	2300	Streaming	Standard	
2025-06-26 19:43:23	req_011CQXAgwqbHWuYLEmXFrRBy	claude-sonnet-4-20250514	
731
	441	Streaming	Standard	
2025-06-26 19:43:35	req_011CQXAhp7nHn8tqGQkz4Wdj	claude-sonnet-4-20250514	
765
	58	HTTP	Standard

Also setting the variable MAX_CONTEXT_TOKENS=180000 in .env file won't help at all.