Prompt prefixes are ignored by Ollama agents, but not by Ollama modelspecs

[mediaarchitekt]

What happened?

When defining agents based on Ollama models, no matter if phi4, cogito oder llama3.3, they all ignore the instructions set in the agents builder sidepanel, which I suspect to be the prompt prefixes. I read somewhere, that there are problems with Ollama, but found out that modelspecs based on Ollama use prompt prefixes perfectly. So why does it work in modelspecs but not with agents?

I would like using agents far more, so I can tweak parameters easily and share them with a few clicks.

Version Information

I'm using Docker with a fresh copy of LibreChat from Github, but without using Git (zip download) on my server, so I just put basic version information here:

LibreChat version v0.7.8
Configuration version v1.2.1

Steps to Reproduce

1. Create new agent with name, description and instructions and choose Ollama model
2. Talk to agent and check if he follows the instructions
3. Create modelSpecs with Ollama endpoint and model, set promptPrefix
4. Talk to model defined with modelSpecs and check if he follows the promptPrefix

What browsers are you seeing the problem on?

No response

Relevant log output

2025-05-19T09:41:03.716Z debug: [BaseClient] userMessage
{
  messageId: "777823f4-1709-40ad-a798-da79fac8c34a",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "533c01d3-75b8-437b-86d1-fef8ad4ca091",
  sender: "User",
  text: "Who you are?",
  isCreatedByUser: true,
  tokenCount: 8,
}
2025-05-19T09:41:03.776Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-05-19T09:41:09.212Z debug: [spendTokens] conversationId: 533c01d3-75b8-437b-86d1-fef8ad4ca091 | Context: message | Token usage:
{
  promptTokens: 12,
  completionTokens: 28,
}
2025-05-19T09:41:09.236Z debug: [spendTokens] No transactions incurred against balance
2025-05-19T09:41:09.251Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-05-19T09:41:09.291Z debug: [AgentController] Request closed
2025-05-19T09:41:20.821Z error: [api/server/controllers/agents/client.js #titleConvo] Error No tool calls found in the response.
2025-05-19T09:41:20.823Z debug: [saveConvo] api/server/services/Endpoints/agents/title.js
2025-05-19T09:44:10.521Z debug: [AskController]
{
  text: "Who you are?",
  conversationId: null,
  endpoint: "ollama",
  endpointType: "custom",
  modelLabel: "Albert",
  promptPrefix: "Schlüpfe in die Rolle von Albert Einstein. Sei gern etwas zerstreut und mit ironischem Unterton, abe... [truncated]",
  resendFiles: true,
  iconURL: "https://example.com:3443/images/681e12268186866b7ff6f467/828dfa0d-9a95-4b6e-849d-9a5... [truncated]",
  greeting: "„Wissen ist die Beste Form von Glück.“\n",
  spec: "albert_einstein",
  maxContextTokens: 4096,
    modelOptions.model: "cogito:14b",
    modelOptions.temperature: 0.2,
    modelOptions.max_tokens: 4096,
  modelsConfig: "exists",
}
2025-05-19T09:44:10.526Z debug: [BaseClient] Loading history:
{
  conversationId: "0fba0ff5-62af-4ac4-9101-741724c55263",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
}
2025-05-19T09:44:10.714Z debug: [BaseClient] instructions tokenCount: 127
2025-05-19T09:44:10.715Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 1910,
  maxContextTokens: 2048,
}
2025-05-19T09:44:10.715Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 1910,
  maxContextTokens: 2048,
}
2025-05-19T09:44:10.716Z debug: [BaseClient] tokenCountMap:
{
  1eece54d-c0e8-455f-b7c1-8855a375bc75: 8,
}
2025-05-19T09:44:10.717Z debug: [BaseClient]
{
  promptTokens: 138,
  remainingContextTokens: 1910,
  payloadSize: 2,
  maxContextTokens: 2048,
}
2025-05-19T09:44:10.717Z debug: [BaseClient] tokenCountMap
{
  1eece54d-c0e8-455f-b7c1-8855a375bc75: 8,
  instructions: 127,
}
2025-05-19T09:44:10.718Z debug: [BaseClient] userMessage
{
  messageId: "1eece54d-c0e8-455f-b7c1-8855a375bc75",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "0fba0ff5-62af-4ac4-9101-741724c55263",
  sender: "User",
  text: "Who you are?",
  isCreatedByUser: true,
  tokenCount: 8,
}
2025-05-19T09:44:10.719Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "http://192.168.178.20:11434/v1",
    modelOptions.model: "cogito:14b",
    modelOptions.temperature: 0.2,
    modelOptions.max_tokens: 4096,
    modelOptions.user: "681e12268186866b7ff6f467",
    modelOptions.stream: true,
    // 2 message(s)
    modelOptions.messages: [{"role":"system","content":"Instructions:\nSchlüpfe in die Rolle von Albert Einstein. Sei gern etwas... [truncated],{"role":"user","content":"Who you are?"}],
}
2025-05-19T09:44:10.732Z debug: Making request to http://192.168.178.20:11434/v1/chat/completions
2025-05-19T09:44:10.748Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-05-19T09:44:35.977Z debug: [OpenAIClient] chatCompletion response
{
  object: "chat.completion",
  id: "chatcmpl-981",
  // 1 choice(s)
  choices: [{"message":{"role":"assistant","content":"*streicht sich über den zerzausten Bart*\n\nAh, du fragst ... [truncated]],
  created: 1747647875,
  model: "cogito:14b",
  system_fingerprint: "fp_ollama",
}
2025-05-19T09:44:35.980Z debug: [spendTokens] conversationId: 0fba0ff5-62af-4ac4-9101-741724c55263 | Context: message | Token usage:
{
  promptTokens: 138,
  completionTokens: 185,
}
2025-05-19T09:44:35.991Z debug: [spendTokens] No transactions incurred against balance
2025-05-19T09:44:36.019Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-05-19T09:44:36.061Z debug: [AskController] Request closed
2025-05-19T09:44:36.111Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "http://192.168.178.20:11434/v1",
    modelOptions.model: "cogito:14b",
    modelOptions.temperature: 0.2,
    modelOptions.max_tokens: 16,
    modelOptions.user: "681e12268186866b7ff6f467",
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    // 1 message(s)
    modelOptions.messages: [{"role":"user","content":"Please generate a concise, 5-word-or-less title for the conversation, usin... [truncated]],
}
2025-05-19T09:44:36.113Z debug: Making request to http://192.168.178.20:11434/v1/chat/completions
2025-05-19T09:44:40.866Z debug: [OpenAIClient] chatCompletion response
{
  id: "chatcmpl-177",
  object: "chat.completion",
  created: 1747647880,
  model: "cogito:14b",
  system_fingerprint: "fp_ollama",
  // 1 choice(s)
  choices: [{"index":0,"message":{"role":"assistant","content":"Albert Einstein Introduces Himself"},"finish_rea... [truncated]],
    usage.prompt_tokens: 158,
    usage.completion_tokens: 7,
    usage.total_tokens: 165,
}
2025-05-19T09:44:40.869Z debug: [spendTokens] conversationId: 0fba0ff5-62af-4ac4-9101-741724c55263 | Context: title | Token usage:
{
  promptTokens: 154,
  completionTokens: 6,
}
2025-05-19T09:44:40.872Z debug: [OpenAIClient] Convo Title: Albert Einstein Introduces Himself
2025-05-19T09:44:40.872Z debug: [saveConvo] api/server/services/Endpoints/openAI/addTitle.js
2025-05-19T09:44:40.893Z debug: [spendTokens] No transactions incurred against balance

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Agents do not make use of promptPrefix or other model parameters, besides what is assigned to the agent via the UI. For the equivalent of promptPrefix, you write instructions in the Agent form.

[mediaarchitekt]

Hi Danny,

thanks for your immediate reply. I thought agents do make use of promptPrefix as there is an "instructions" field shown in the agents sidebar at the same place like the "custom instructions" field when i define promptPrefix as a parameter under modelspecs in the librechat.yaml. See the fields in the screenshots marked by the mouse cursor.

Does it mean "custom instructions" are mapped to promptPrefix in the modelspecs, though "instructions" of agents are not mapped to promptPrefix but something else? Why is that and for clarity: to what are "instructions" of agents mapped. They are not recognized by Ollama models, but by ChatGPT and Groq.

[danny-avila]

It's a different framework though they are similar. In the end, the end result should be the same as they are both used to create a System Message to be sent as part of the chat payload.

It might be a specific LLM issue (in your case, cogito) as it worked with me using qwen3:30b-a3b-q8_0:

[mediaarchitekt]

Thanks for investigation! Yes, it works with the first agent, but with the second agent not anymore. That's why I suppose it is more a bug. With modelspecs I can build customized endpoints as much as I want. With agents only one, all the following ignore the instructions. Could you please check this? I know it sounds very unspecific, but I set up more than one version of librechat, searched the web and defined modelspecs, that I at least ask you, to help, because I can't find any solution.

[mediaarchitekt]

Okay, I've got it working. I updated Ollama to the the new v0.7.0 and now it works reliable even with the old installed models. Beside that I'm still not sure, whether the update made the difference or throwing away the modelspecs out of the librechat.yaml or some tweaks in GUI. But I'm fine with that. Thanks again for your time.