[BUG] cache_breakpoint injected into messages for non-Anthropic providers (Groq, OpenAI-compatible)

[Hamza-BinRiaz]

Description

Problem:

mark_cache_breakpoint() is called on all messages in agent_executor.py regardless of provider. Only the Anthropic adapter translates/strips this flag. All other providers (Groq, OpenAI-compatible, etc.) receive cache_breakpoint: true in the raw message dict, which their APIs reject.

Error:

GroqException - 'messages.0': property 'cache_breakpoint' is unsupported

Root cause:

crewai/llms/cache.py defines strip_cache_breakpoint() but it is never called for non-Anthropic providers.

Workaround:

import crewai.llms.cache as _crewai_cache
_crewai_cache.mark_cache_breakpoint = lambda msg: msg

Steps to Reproduce

1. Install crewai==1.14.4 and set a GROQ_API_KEY in your .env
2. Create a basic crew using a Groq model:

from crewai import Agent, Task, Crew, LLM
llm = LLM(model="groq/llama-3.3-70b-versatile")
agent = Agent(role="Researcher", goal="Research things", backstory="You are a researcher", llm=llm)
task = Task(description="Write a summary of AI trends", expected_output="A short summary", agent=agent)
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff()

3. Run the script
4. Crew fails immediately with the error above

Expected behavior

The crew runs successfully. The agent sends clean messages to the Groq API and gets a response back with no errors.

Screenshots/Code snippets

ull traceback:

litellm.BadRequestError: GroqException - {"error": {"message": "'messages.0': for 'role:system'
the following must be satisfied[('messages.0': property 'cache_breakpoint' is unsupported)]",
"type": "invalid_request_error"}}

The injected message looks like this before being sent to Groq:

{
  "role": "system",
  "content": "You are a researcher...",
  "cache_breakpoint": true
}

Operating System

Windows 10

Python Version

3.12

crewAI Version

1.14.4

crewAI Tools Version

1.14.4

Virtual Environment

Venv

Evidence

mark_cache_breakpoint() is imported and called unconditionally in:

• crewai/experimental/agent_executor.py (lines 2589, 2595, 2600, 2603, 2607)
• crewai/agents/crew_agent_executor.py (lines 177, 190, 195, 200)

strip_cache_breakpoint() is defined in crewai/llms/cache.py but never called anywhere outside of that file.
The Anthropic provider (crewai/llms/providers/anthropic/completion.py) handles cache_breakpoint correctly,
but there is no equivalent handling in the OpenAI, OpenAI-compatible, or Groq providers.

Possible Solution

Call strip_cache_breakpoint() in the OpenAI / OpenAI-compatible provider adapters before sending
messages to the API, the same way the Anthropic adapter handles the flag. This would strip the
cache_breakpoint key from messages for any provider that does not natively support it.

Additional context

This affects anyone using CrewAI 1.14.4 with any non-Anthropic provider (Groq, OpenAI-compatible
endpoints, etc.). The bug makes it impossible to run a crew with Groq out of the box without the
monkey-patch workaround listed above.

[leavedrop]

Two open PRs target this: #5887 (Devin bot, narrow strip in LiteLLM path) and #5914 (superShen0916, strip + expanded Anthropic prefix detection). Both look correct but neither covers tests or the streaming path explicitly.

Happy to draft a salvage PR combining them — strip via #5914's expanded prefix detector, then add:

• regression test asserting Groq + OpenAI-compatible providers receive no cache_breakpoint key
• streaming path coverage (current PRs only patch the sync send path; need to verify _streaming_handler.py doesn't re-introduce it)

Worth doing as a salvage or do you want to merge one of the existing PRs first? Don't want to step on @superShen0916 or the Devin bot if a merge is imminent.

[abdullahshahin]

Same shape shows up whenever a provider-agnostic layer leaks a provider-specific annotation downstream — the right invariant here is probably that cache_breakpoint never reaches the LLM adapter at all, with the Anthropic adapter translating it into cache_control at the boundary instead of every other adapter having to strip it. Stripping in each non-Anthropic adapter (per the linked PRs) fixes today's bug but locks in N-1 strip sites for every future provider added. The monkey-patch in the OP works because mark_cache_breakpoint is the single injection point — same place a centralized fix would live. That's the structure we landed on for the adapter layer at https://hivein.ai — one translation site per provider, nothing the upper layers know about.

[leavedrop]

Correction to my earlier comment + current state on main (v1.14.6):

The native provider path is already fixed. BaseLLM._format_messages strips cache_breakpoint, and does it via a per-message copy so the caller's markers survive across tool-loop iterations (the executors reuse their messages buffer). The Anthropic adapter translates the flag into cache_control at its own boundary. So the "translate-at-boundary, strip-everywhere-else" invariant @abdullahshahin describes is already in place for native providers — in one site (base_llm.py), not N.

The remaining gap is the legacy LiteLLM path (llm.py LLM), which doesn't go through BaseLLM._format_messages — grep cache_breakpoint src/crewai/llm.py is empty on main. That's the path Groq / OpenAI-compatible hit.

Of the two open PRs, #5887 already closes exactly that: it strips in _prepare_completion_params and ships regression tests (Groq strip / no-mutation of the caller's list / no-marker passthrough). #5914 strips in _format_messages_for_provider and additionally fixes an anthropic-- prefix-detection case worth keeping.

Suggest merging #5887 as the fix (tests + no-mutation guard) and cherry-picking #5914's anthropic-- prefix bit.

Also correcting myself: my earlier comment said neither PR covered tests — that's wrong, #5887 does. Withdrawing my salvage-PR offer; #5887 covers it, no need for a third PR.