Skip to content

Conversation

codyde
Copy link
Contributor

@codyde codyde commented Oct 1, 2025

Summary

Adds Sentry tracing instrumentation for the @anthropic-ai/claude-agent-sdk (Claude Code Agent SDK) following OpenTelemetry Semantic Conventions Sentry's Agent Monitoring.

This integration enables AI monitoring for Claude Code agents with comprehensive telemetry:

  • Agent invocation spans (invoke_agent)
  • LLM chat spans (chat)
  • Tool execution spans (execute_tool)
  • Token usage tracking (including cache metrics)
  • Model info and session tracking
  • Optional input/output recording

Key Implementation Details

Why Not Automatic Like Other AI Integrations?

The Claude Code SDK (@anthropic-ai/claude-agent-sdk) is ESM-only with no CommonJS build, which prevents automatic instrumentation via OpenTelemetry's require() hooks that work for other integrations (Anthropic AI, OpenAI, etc.).

Solution: Helper Function Pattern

Provides createInstrumentedClaudeQuery() - a one-line helper that:

  • Lazy loads the SDK via dynamic import() (avoids bundler issues)
  • Automatically retrieves options from claudeCodeIntegration() config
  • Uses global singleton pattern (patches once, reuses everywhere)
  • Works with any bundler (Next.js/webpack, Vite, etc.)

Usage

// Step 1: Configure in Sentry init
import * as Sentry from '@sentry/node';

Sentry.init({
  dsn: 'your-dsn',
  integrations: [
    Sentry.claudeCodeIntegration({
      recordInputs: true,
      recordOutputs: true
    })
  ],
});

// Step 2: Use in your code (1 line!)
import * as Sentry from '@sentry/node';

const query = Sentry.createInstrumentedClaudeQuery();

// Use query as normal - automatically instrumented
for await (const message of query({
  prompt: 'Hello',
  options: { model: 'claude-sonnet-4-5' }
})) {
  console.log(message);
}

Remaining TODO's

  • Tests
  • Handle PII (disabling prompt transmission etc...)
  • Expanded JavaScript server SDKs

Adds Sentry tracing instrumentation for the @anthropic-ai/claude-agent-sdk
following OpenTelemetry Semantic Conventions for Generative AI.

Key features:
- Captures agent invocation, LLM chat, and tool execution spans
- Records token usage, model info, and session tracking
- Supports input/output recording based on sendDefaultPii setting
- Provides createInstrumentedClaudeQuery() helper for clean DX

Due to ESM-only module constraints, this integration uses a helper function
pattern instead of automatic OpenTelemetry instrumentation hooks.

Usage:
```typescript
import { createInstrumentedClaudeQuery } from '@sentry/node';
const query = createInstrumentedClaudeQuery();
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

…g in Claude Code integration

- Add SEMANTIC_ATTRIBUTE_SENTRY_OP to all span creation calls (invoke_agent, chat, execute_tool)
- Capture exceptions to Sentry in catch block with proper mechanism metadata
- Ensure child spans (currentLLMSpan, previousLLMSpan) are always closed in finally block
- Prevents incomplete traces if generator exits early
@codyde codyde marked this pull request as draft October 1, 2025 23:17
Copy link
Contributor

github-actions bot commented Oct 1, 2025

size-limit report 📦

Path Size % Change Change
@sentry/browser 24.23 kB - -
@sentry/browser - with treeshaking flags 22.75 kB - -
@sentry/browser (incl. Tracing) 40.42 kB - -
@sentry/browser (incl. Tracing, Replay) 78.8 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 68.45 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 83.47 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 95.67 kB - -
@sentry/browser (incl. Feedback) 40.95 kB - -
@sentry/browser (incl. sendFeedback) 28.89 kB - -
@sentry/browser (incl. FeedbackAsync) 33.82 kB - -
@sentry/react 25.96 kB - -
@sentry/react (incl. Tracing) 42.39 kB - -
@sentry/vue 28.75 kB - -
@sentry/vue (incl. Tracing) 42.23 kB - -
@sentry/svelte 24.26 kB - -
CDN Bundle 25.75 kB - -
CDN Bundle (incl. Tracing) 40.31 kB - -
CDN Bundle (incl. Tracing, Replay) 76.55 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 82.06 kB - -
CDN Bundle - uncompressed 75.3 kB - -
CDN Bundle (incl. Tracing) - uncompressed 119.31 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 234.47 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 247.23 kB - -
@sentry/nextjs (client) 44.4 kB - -
@sentry/sveltekit (client) 40.84 kB - -
@sentry/node-core 50.47 kB - -
@sentry/node 154.12 kB +0.02% +29 B 🔺
@sentry/node - without tracing 92.33 kB - -
@sentry/aws-serverless 106.03 kB - -

View base workflow run

Copy link
Contributor

github-actions bot commented Oct 1, 2025

node-overhead report 🧳

Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.

Scenario Requests/s % of Baseline Prev. Requests/s Change %
GET Baseline 8,919 - 9,636 -7%
GET With Sentry 1,406 16% 1,444 -3%
GET With Sentry (error only) 6,110 69% 6,177 -1%
POST Baseline 1,202 - 1,179 +2%
POST With Sentry 525 44% 496 +6%
POST With Sentry (error only) 1,063 88% 1,022 +4%
MYSQL Baseline 3,345 - 3,310 +1%
MYSQL With Sentry 529 16% 375 +41%
MYSQL With Sentry (error only) 2,748 82% 2,658 +3%

View base workflow run

@codyde codyde changed the title feat(node): Add Claude Code Agent SDK instrumentation feat(agent-monitoring): Add Claude Code Agent SDK instrumentation Oct 2, 2025
@codyde codyde changed the title feat(agent-monitoring): Add Claude Code Agent SDK instrumentation feat(javascript): Add Claude Code Agent SDK instrumentation Oct 2, 2025
@RulaKhaled RulaKhaled assigned RulaKhaled and unassigned RulaKhaled Oct 6, 2025
@RulaKhaled RulaKhaled self-requested a review October 6, 2025 08:17

type ClaudeCodeInstrumentationOptions = ClaudeCodeOptions;

const GEN_AI_ATTRIBUTES = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have these attributes in packages/core/src/utils/ai/gen-ai-attributes.ts


const SENTRY_ORIGIN = 'auto.ai.claude-code';

function setTokenUsageAttributes(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you reuse function from packages/core/src/utils/ai/utils.ts?

// Parse query arguments
const [queryParams] = args as [Record<string, unknown>];
const { options: queryOptions, inputMessages } = queryParams || {};
const model = (queryOptions as Record<string, unknown>)?.model ?? 'sonnet';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we revert to unknown if not model found here? i think this might be confusing if it's not accurate

name: CLAUDE_CODE_INTEGRATION_NAME,
options,
setupOnce() {
// Note: Automatic patching via require hooks doesn't work for ESM modules
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i believe InstrumentationModuleDefinition will automatically patch Node.js modules when they're loaded via import, you can find some patterns in other AI integrations e.g anthropic AI

* Patches the Claude Code SDK query function with Sentry instrumentation.
* This function can be called directly to patch an imported query function.
*/
export function patchClaudeCodeQuery(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should hook into the query and use proxy here

);

// Preserve Query interface methods
if (typeof (originalQueryInstance as Record<string, unknown>).interrupt === 'function') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using proxy should clean this up a little

Copy link
Member

@RulaKhaled RulaKhaled left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! For the first pass, the biggest lift here is to try to auto patch the functions we need automatically instead of asking user to import patched method, then we can move to tackling the other TODOs you have

@codyde
Copy link
Contributor Author

codyde commented Oct 6, 2025

Thanks for working on this! For the first pass, the biggest lift here is to try to auto patch the functions we need automatically instead of asking user to import patched method, then we can move to tackling the other TODOs you have

Thanks SO much for all these. I'll get started on them.

I tried REALLY hard to figure out how to hook into the existing query, and I couldn't get it to work no matter what I tried. I'll chat with you in slack on it, but I'd love some advice / guidance. I tried a bunch of different angles - but each time I ran into effectively timing issues where we couldn't hook fast enough. Felt like a limitation on how Claude Code's SDK works - but could be a total skill issue on my side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants