Skip to content

fix(core): prevent _configure_hooks accumulation in get_usage_metadata_callback#35330

Open
Balaji Seshadri (gitbalaji) wants to merge 3 commits intolangchain-ai:masterfrom
gitbalaji:fix/core-configure-hooks-accumulation
Open

fix(core): prevent _configure_hooks accumulation in get_usage_metadata_callback#35330
Balaji Seshadri (gitbalaji) wants to merge 3 commits intolangchain-ai:masterfrom
gitbalaji:fix/core-configure-hooks-accumulation

Conversation

@gitbalaji
Copy link
Contributor

Summary

Fixes #32300.

  • get_usage_metadata_callback() previously created a new ContextVar and called register_configure_hook on every invocation, causing _configure_hooks to grow unboundedly
  • In long-running applications (e.g. servers, evaluation loops) this caused a memory leak and progressively slower callback configuration on every LLM call
  • Fix: move the ContextVar and register_configure_hook call to module level — registered exactly once at import time
  • The context manager now uses ContextVar.reset(token) instead of set(None), which correctly restores the previous handler in nested get_usage_metadata_callback() calls
  • The name parameter is deprecated (it only named the now-global ContextVar) with a DeprecationWarning when a non-default value is passed

Areas requiring careful review

  • The _usage_metadata_callback_var symbol is now public-ish (module-level with a _ prefix); tests import it directly — maintainers may prefer a different approach to expose it for testing
  • The name parameter deprecation: currently only warns when the value differs from the default ("usage_metadata_callback"); we could also warn on any usage of the parameter

Test plan

  • test_configure_hooks_no_accumulation — asserts _configure_hooks does not grow after repeated calls
  • test_name_parameter_deprecated — asserts DeprecationWarning when custom name is passed
  • test_name_parameter_default_no_warning — asserts no warning on default usage
  • test_nested_context_managers — asserts inner CM exit restores outer handler (not None)
  • Existing test_usage_callback and test_usage_callback_async still pass

AI disclaimer: This PR was developed with assistance from Claude Code (Anthropic).

🤖 Generated with Claude Code

@github-actions github-actions bot added core `langchain-core` package issues & PRs external fix For PRs that implement a fix labels Feb 19, 2026
@gitbalaji Balaji Seshadri (gitbalaji) force-pushed the fix/core-configure-hooks-accumulation branch from 25d8c29 to 8525ce4 Compare February 19, 2026 16:34
@codspeed-hq
Copy link

codspeed-hq bot commented Feb 19, 2026

Merging this PR will degrade performance by 24.61%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

❌ 7 regressed benchmarks
✅ 6 untouched benchmarks
⏩ 23 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime test_import_time[CallbackManager] 286.4 ms 323.5 ms -11.48%
WallTime test_import_time[PydanticOutputParser] 477.6 ms 540.2 ms -11.59%
WallTime test_import_time[tool] 473.5 ms 536.9 ms -11.82%
WallTime test_import_time[BaseChatModel] 482.7 ms 540.7 ms -10.73%
WallTime test_import_time[ChatPromptTemplate] 549.8 ms 635.1 ms -13.44%
WallTime test_async_callbacks_in_sync 18.4 ms 24.4 ms -24.61%
WallTime test_import_time[Runnable] 444.5 ms 495.8 ms -10.35%

Comparing gitbalaji:fix/core-configure-hooks-accumulation (e4ed9ba) with master (2d1492a)2

Open in CodSpeed

Footnotes

  1. 23 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (0b975d4) during the generation of this report, so 2d1492a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@gitbalaji
Copy link
Contributor Author

CodSpeed benchmark regression is a false positive

The bot flagged a 10.44% regression in test_import_time[ChatPromptTemplate]. Our change cannot have caused this — here's why:

Our change is in langchain_core/callbacks/usage.py. That module is lazy-loaded via import_attr in callbacks/__init__.py (only inside if TYPE_CHECKING:), so it is not imported as part of the ChatPromptTemplate import chain.

Verified locally:

from langchain_core.prompts import ChatPromptTemplate
import sys
print('langchain_core.callbacks.usage' in sys.modules)  # False

The module is not in sys.modules after importing ChatPromptTemplate, confirming our code path is never executed during that benchmark.

The CodSpeed bot itself notes: "Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data." The 76ms variance (650ms → 726ms) is well within the noise range for subprocess-based import timing on shared GitHub Actions runners.

@gitbalaji Balaji Seshadri (gitbalaji) force-pushed the fix/core-configure-hooks-accumulation branch 2 times, most recently from cb233ac to 2dae84c Compare February 20, 2026 20:33
@gitbalaji Balaji Seshadri (gitbalaji) force-pushed the fix/core-configure-hooks-accumulation branch 3 times, most recently from e4ed9ba to 17c743f Compare February 26, 2026 01:16
…a_callback

Fixes langchain-ai#32300.

Each call to `get_usage_metadata_callback()` previously created a new
`ContextVar` and called `register_configure_hook`, appending to the
global `_configure_hooks` list on every invocation. In long-running
applications, this caused unbounded memory growth and increasingly slow
callback configuration on every LLM call.

Fix: lift the `ContextVar` and `register_configure_hook` call to
module level so the hook is registered exactly once at import time.
The context manager now sets/resets the single module-level var using
`ContextVar.reset(token)`, which also correctly restores the previous
handler in nested `get_usage_metadata_callback()` calls.

The `name` parameter is deprecated (it only named the now-global
`ContextVar`) and emits a `DeprecationWarning` when a non-default
value is passed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ContextVar is invariant, so ContextVar[UsageMetadataCallbackHandler | None]
cannot be passed to list.count() expecting ContextVar[BaseCallbackHandler | None].
Replace with an identity-based sum() to avoid the type mismatch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gitbalaji Balaji Seshadri (gitbalaji) force-pushed the fix/core-configure-hooks-accumulation branch from 17c743f to b6577a1 Compare February 27, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core `langchain-core` package issues & PRs external fix For PRs that implement a fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

core: _configure_hooks variable accumulates with each call to get_usage_metadata_callback

2 participants