Skip to content

feat: add opt-in AgentPond tracing - #136

Open
marcusschiesser wants to merge 1 commit into
yokingma:mainfrom
marcusschiesser:agent/add-agentpond-tracing
Open

feat: add opt-in AgentPond tracing#136
marcusschiesser wants to merge 1 commit into
yokingma:mainfrom
marcusschiesser:agent/add-agentpond-tracing

Conversation

@marcusschiesser

Copy link
Copy Markdown

Title:

  • Add opt-in, privacy-safe AgentPond tracing for OpenAI-compatible chats

Description:

  • Initializes AgentPond's Files SDK exporter only when AGENTPOND_ENABLED=true.
  • Instruments SearChat's existing OpenAI SDK path with OpenInference spans.
  • Hides prompt and response content while retaining operational metadata such as model, timing, status, and token counts.
  • Documents a local filesystem-backed setup and ignores local trace data.
  • Adds a real OpenAI SDK integration test backed by a local HTTP provider fixture.

AgentPond fits this path because SearChat already centralizes its OpenAI-compatible calls in BaseOpenAIChat; the integration stays server-side, provider-neutral, and entirely opt-in.

Configuration and privacy:

Tracing remains disabled by default. When enabled, spans are written only to the Files SDK store selected by the operator. OpenInference input and output values are redacted, so prompts, responses, API keys, and provider targets are not exported.

Validation:

  • yarn install --immutable
  • yarn workspace deepsearcher build
  • yarn workspace server check-types
  • yarn workspace server test — 11/11 passed
  • yarn workspace server exec eslint src/instrumentation.ts src/core/llm/agentpond.e2e.test.ts src/app.ts
  • yarn build — all three workspaces built

End to end, the new test used SearChat's production BaseOpenAIChat and OpenAI SDK against an in-process HTTP provider, received traced answer, flushed the exporter, and synced the local Files SDK store with npx agentpond sync. npx agentpond traces get b2d052f5425c9e48255435cb18d1286b read back an OpenAI Chat Completions LLM trace with model gpt-test, token counts 2 prompt / 2 completion / 4 total, and both input and output recorded as __REDACTED__.

Limitations:

  • This PR instruments SearChat's OpenAI-compatible SDK path only; other provider adapters are unchanged.

Motivation:

  • Give self-hosters a local, consent-based way to diagnose AI latency, failures, and token usage without exporting conversation content.

Related Issues:

  • None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant