Skip to content

fix: don't inject cache_breakpoint for non-Anthropic LLMs #5954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
felipebridge wants to merge 1 commit into
base: main
Choose a base branch
from
+13 −14
Open

fix: don't inject cache_breakpoint for non-Anthropic LLMs #5954

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 13 additions & 14 deletions lib/crewai/src/crewai/agents/crew_agent_executor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,11 @@ def use_stop_words(self) -> bool:
self.llm.supports_stop_words() if isinstance(self.llm, BaseLLM) else False
)

def _llm_is_anthropic(self) -> bool:
"""Return True if the configured LLM is an Anthropic model."""
model: str = getattr(getattr(self, "llm", None), "model", "") or ""
return model.startswith("claude") or "anthropic" in model
Comment on lines +167 to +170
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Harden Anthropic detection for case and string LLM refs.

_llm_is_anthropic() can false-negative when model casing isn’t lowercase or when self.llm is a string ref (allowed by the field type). Normalizing and handling both shapes keeps behavior consistent for Anthropic users.

Suggested patch 
 def _llm_is_anthropic(self) -> bool:
     """Return True if the configured LLM is an Anthropic model."""
-    model: str = getattr(getattr(self, "llm", None), "model", "") or ""
-    return model.startswith("claude") or "anthropic" in model
+    llm_ref = getattr(self, "llm", None)
+    model: str = (
+        llm_ref if isinstance(llm_ref, str) else getattr(llm_ref, "model", "")
+    ) or ""
+    model = model.lower()
+    return model.startswith("claude") or "anthropic" in model
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/src/crewai/agents/crew_agent_executor.py` around lines 167 - 170,
_llm_is_anthropic can miss Anthropic models when the model string has different
casing or when self.llm is provided as a plain string; update _llm_is_anthropic
to handle both shapes and normalize case: if self.llm is a string use it
directly, otherwise get getattr(self.llm, "model", "") and then lower() the
model value before checking startswith("claude") or "anthropic" in model to
ensure consistent detection; reference the _llm_is_anthropic function and the
self.llm / model variable when making the change.


def _setup_messages(self, inputs: dict[str, Any]) -> None:
"""Set up messages for the agent execution.

Expand All @@ -176,29 +181,23 @@ def _setup_messages(self, inputs: dict[str, Any]) -> None:

from crewai.llms.cache import mark_cache_breakpoint

# cache_breakpoint is an Anthropic-only feature; injecting it for other
# providers (Groq, OpenAI-compatible APIs, etc.) causes a hard
# validation error on the provider side.
_mark = mark_cache_breakpoint if self._llm_is_anthropic() else (lambda m: m)

if self.prompt is not None and "system" in self.prompt:
system_prompt = self._format_prompt(
cast(str, self.prompt.get("system", "")), inputs
)
user_prompt = self._format_prompt(
cast(str, self.prompt.get("user", "")), inputs
)
# Cache breakpoints: end-of-system caches the per-agent stable
# prefix; end-of-user caches the per-task stable prefix across
# ReAct-loop iterations.
self.messages.append(
mark_cache_breakpoint(
format_message_for_llm(system_prompt, role="system")
)
)
self.messages.append(
mark_cache_breakpoint(format_message_for_llm(user_prompt))
)
self.messages.append(_mark(format_message_for_llm(system_prompt, role="system")))
self.messages.append(_mark(format_message_for_llm(user_prompt)))
elif self.prompt is not None:
user_prompt = self._format_prompt(self.prompt.get("prompt", ""), inputs)
self.messages.append(
mark_cache_breakpoint(format_message_for_llm(user_prompt))
)
self.messages.append(_mark(format_message_for_llm(user_prompt)))

provider.post_setup_messages(cast(ExecutorContext, cast(object, self)))

Expand Down