after_llm_call hooks never run on acall(): eight native provider handlers skip _invoke_after_llm_call_hooks

[LHMQ878]

Description

after_llm_call hooks never run on acall(). The native providers invoke
_invoke_after_llm_call_hooks only from their sync handlers, so a hook registered to
redact, rewrite or audit a response is silently skipped for every async direct LLM call — the
unmodified response reaches the caller and nothing is logged.

The gap is per-handler, not per-provider. Coverage on main (3266932):

provider	handler	sync	async
openai	_handle_responses / _ahandle_responses	yes (:1068)	no
openai	_handle_streaming_responses / _ahandle_streaming_responses	yes (:1376)	no
openai	_handle_completion / _ahandle_completion	yes (:2008)	no
openai	_handle_streaming_completion / _ahandle_streaming_completion	yes (:2298)	no
anthropic	_handle_completion / _ahandle_completion	yes (:1104)	no
anthropic	_handle_streaming_completion / _ahandle_streaming_completion	yes (:1307)	no
bedrock	_handle_converse / _ahandle_converse	yes (:823)	no
bedrock	_handle_streaming_converse / _ahandle_streaming_converse	no	yes (:1772)

The bedrock row is the giveaway: within one file, _ahandle_streaming_converse calls the hook
and _ahandle_converse does not, and the sync/async polarity is inverted relative to every
other row. That is not a design decision about async — it is eight handlers each missing the
call their twin makes.

Reproduction

No network. Stub the SDK client and compare call() with acall():

from crewai.hooks.llm_hooks import register_after_llm_call_hook

calls = []
def hook(ctx):
    calls.append(ctx.response)
    return "REDACTED BY HOOK"
register_after_llm_call_hook(hook)

llm = AnthropicCompletion(model="claude-sonnet-4-5", thinking={"type": "enabled", "budget_tokens": 1024},
                          max_tokens=2048, stream=False)
llm._client       = SimpleNamespace(messages=SyncStub())    # returns one thinking + one text block
llm._async_client = SimpleNamespace(messages=AsyncStub())   # same payload

print(llm.call("hi"))
print(asyncio.run(llm.acall("hi")))

On main:

anthropic sync : 'REDACTED BY HOOK'                     hook fired: 1
anthropic async: 'the model said something sensitive'   hook fired: 0
openai    sync : 'REDACTED BY HOOK'                     hook fired: 1
openai    async: 'the model said something sensitive'   hook fired: 0

The hook is never even invoked on the async path, so a hook that only observes (logging,
auditing, metrics) also silently records nothing.

Second defect in the same code: anthropic loses thinking blocks on the async path

While mapping the above I found _ahandle_completion and _ahandle_streaming_completion never
call _extract_thinking_block, so _previous_thinking_blocks stays empty on the async path.
The same probe:

anthropic sync  thinking kept: [{'type': 'thinking', 'thinking': 'internal reasoning', 'signature': 'sig-abc'}]
anthropic async thinking kept: []

_format_messages_for_anthropic:851 replays self._previous_thinking_blocks into the
assistant message on the next turn when thinking is enabled. With the async path leaving that
list empty, the signed thinking block is dropped and the follow-up request loses the reasoning
context the sync path preserves. test_anthropic_thinking_blocks_preserved_across_turns
(tests/llms/anthropic/test_anthropic.py:633) pins this behaviour precisely — for call() only.

Why this matters

after_llm_call is the documented interception point for exactly the things that must not be
skippable: redacting PII before a response is stored or displayed, enforcing an output policy,
rewriting a response, and audit logging. A guardrail that works under call() and silently
stops working under acall() is worse than one that never worked, because the failure is
invisible: no exception, no warning, and the response looks plausible.

Async is not an edge path — acall() is what kickoff_async, async flows and any concurrent
usage reach. Moving a working crew from sync to async silently disables every registered
after_llm_call hook.

Expected behaviour

acall() runs after_llm_call hooks exactly as call() does, and anthropic retains thinking
blocks on both paths.

Fix

Add the missing _invoke_after_llm_call_hooks call to each of the eight handlers, mirroring the
placement its twin already uses (after _emit_call_completed_event, and behind the same
isinstance(result, str) guard the sync streaming handler uses so structured tool-call payloads
are not stringified), plus the two _extract_thinking_block loops in anthropic's async handlers.

_invoke_after_llm_call_hooks already returns the response unchanged when from_agent is not
None, so the agent-driven path (which dispatches POST_MODEL_CALL separately via
agent_utils._setup_after_llm_call_hooks) is unaffected — no double dispatch.

Note on scope: azure and gemini call the hook on neither path. That is a symmetric
gap — the provider does not support the hook at all — rather than sync/async drift, so I left it
out of this fix; it needs a separate decision about whether those providers should support
direct-call hooks. Happy to follow up if wanted.

I have a PR ready with the fix plus 11 tests (sync baselines, async regressions, and negative
controls asserting an unhooked response passes through untouched).

Environment

• crewAI main @ 3266932
• Python 3.12

[impartshadow]

Here's my triage of this one.

This is a symmetry-drift failure — a guard that's wired into one execution path (sync) but never propagated to its structural twin (async) when the twin was added later, so the divergence compiles clean and only shows up as silent behavioral loss. The tell is exactly what you flagged: "within one file, _ahandle_streaming_converse calls the hook and _ahandle_converse does not, and the sync/async polarity is inverted relative to every other row." That inversion is the signature of copy-paste-per-handler implementation with no shared invocation point — eight handlers each independently decide whether to call _invoke_after_llm_call_hooks, so eight independent chances to forget it, and bedrock proves it's not even consistently forgotten in one direction.

The deterministic check: assert call-count symmetry, not behavior. For every provider module, collect all _handle_*/_ahandle_* method pairs via reflection, and for each pair assert _invoke_after_llm_call_hooks (and _extract_thinking_block, since that's the same defect class) appears the same number of times in both bytecode/AST bodies. That's a structural test, not a functional one — it doesn't need network stubs or hook registration, it just diffs the call graph of paired methods. Your repro is the right functional complement, but it only catches providers you remember to stub; the AST-pair check catches the next handler someone adds without needing anyone to think of it.

What would confirm or refute the root cause: check whether _ahandle_* methods were added in separate commits from their sync twins — if git blame shows them landing weeks/months apart rather than in the same diff, that confirms drift-via-separate-authorship rather than a single bad copy-paste.

Advisory SFA-2026-812B11CEA7 · Full redacted write-up: https://impartshadow.github.io/echo-site/failure-cases/after-llm-call-hooks-never-run-on-acall-eight-native-812b11cea7.html — I triage one of these in public every day.
If this diagnosis is useful, react 👍 to this comment; if it is wrong, react 👎 — I use the verdict to choose what I investigate next.