Skip to content

fix(security): remove sensitive data from logs and observability#1729

Open
alex-alecu wants to merge 7 commits intomainfrom
fix/remove-sensitive-data-from-logs
Open

fix(security): remove sensitive data from logs and observability#1729
alex-alecu wants to merge 7 commits intomainfrom
fix/remove-sensitive-data-from-logs

Conversation

@alex-alecu
Copy link
Copy Markdown
Contributor

@alex-alecu alex-alecu commented Mar 30, 2026

Summary

Authentication tokens, credentials, and webhook secrets were being leaked through multiple paths: logged to console (forwarded to Sentry), stored in the database, interpolated into LLM prompts, and sent to Sentry through sendDefaultPii and attachRpcInput.

Why sendDefaultPii and attachRpcInput are dangerous

sendDefaultPii tells Sentry: "attach all request headers, cookies, and IP addresses to every error you capture." The problem is it grabs everything — it can't tell the difference between a harmless Content-Type header and a secret Authorization: Bearer <jwt> header. So when a user's request causes an error, their login token ends up sitting in Sentry where anyone with dashboard access can see it.

attachRpcInput does the same thing but for tRPC calls — it sends the full input of every procedure to Sentry. Some procedures accept personal access tokens as input, so those tokens end up in Sentry too.

Both are now disabled for services that handle user credentials (Next.js app, mobile app). The deploy builder keeps sendDefaultPii enabled because it only handles internal service-to-service requests with a shared backend token — no user secrets flow through it.

What changed

  • Removed full HTTP header dumps and raw JWT logging from token validation
  • Disabled sendDefaultPii in the Next.js client and mobile app Sentry configs
  • Disabled attachRpcInput in the tRPC Sentry middleware
  • Kept sendDefaultPii enabled for the deploy builder (internal service, no user credentials)
  • Added a shared redactSensitiveHeaders utility in @kilocode/worker-utils that strips auth-bearing headers (including custom webhook auth headers) before storage or prompt injection
  • Applied header redaction to GitHub/GitLab webhook handlers, TriggerDO request capture, and prompt template rendering
  • Added a concise logging policy to AGENTS.md to prevent regressions

What we lose for debugging

Most changes have little impact. Here's a honest look at what gets harder:

Change What we lose How bad
Removed header dump in tokens.ts Can't see the full headers on auth failures. We still get a tracking ID to match warnings to requests. Medium — harder to debug weird auth headers
attachRpcInput: false Sentry no longer shows what input a tRPC call had when it failed. For simple calls this is fine, for complex ones it's harder to figure out what went wrong. Most noticeable — this is the change most likely to slow down debugging
sendDefaultPii: false (Next.js, mobile) Sentry stops attaching IP and cookies to errors. User ID is still tagged separately so we still know who hit the error. Small — we rarely need cookies to debug
Webhook header redaction Auth headers like x-hub-signature-256 are replaced with [REDACTED]. All other headers (event type, delivery ID, content-type) are kept. Tiny — we never need auth header values to debug webhook logic
Fake login log Logs just the email instead of the whole object. The object only had { email: "..." } anyway. None

If attachRpcInput: false becomes painful: the targeted fix is to add a beforeSend hook in Sentry that scrubs known sensitive fields from tRPC inputs instead of turning off capture entirely.

Verification

  • pnpm typecheck — passes (all packages)
  • pnpm test in packages/worker-utils — 45/45 tests pass (includes 7 redact-headers tests)
  • pnpm format:check — passes
  • Root pnpm test — all failures are pre-existing DB connection errors (role "postgres" does not exist), unrelated to this change

Visual Changes

N/A

Reviewer Notes

  • The consoleLoggingIntegration in sentry.server.config.ts and sentry.edge.config.ts was intentionally left unchanged — once the sources (tokens.ts) stop emitting sensitive data, the integration no longer amplifies secrets.
  • Header keys from the Web API Headers class are always lowercase, but redactSensitiveHeaders does case-insensitive matching to handle the TriggerDO case where headers come from a custom record.
  • Already-captured secrets in Sentry/Axiom retention are an operational concern outside this PR's scope.

@kilo-code-bot
Copy link
Copy Markdown
Contributor

kilo-code-bot bot commented Mar 30, 2026

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

No new issues found in the incremental diff since 434cb58b375fe143ab1bd2550d0b09be0b26848f.

Other Observations (not in diff)

Issues found in unchanged code that cannot receive inline comments:

File Line Issue
cloudflare-deploy-infra/builder/src/index.ts 302 sendDefaultPii remains enabled even though /deploy-archive accepts secrets via X-Env-Vars, so Sentry can still capture deployment env var values in request headers.

Fix these issues in Kilo Cloud

Files Reviewed (3 files)
  • packages/worker-utils/src/index.ts - 0 issues
  • src/app/api/webhooks/gitlab/route.ts - 0 issues
  • src/lib/integrations/platforms/github/webhook-handler.ts - 0 issues

Reviewed by gpt-5.4-20260305 · 158,159 tokens

The deploy builder only handles internal service-to-service requests
authenticated via BACKEND_AUTH_TOKEN, not user credentials. Keeping
PII capture enabled here preserves debugging context without risk.
Next.js uses moduleResolution: "node" which doesn't resolve
package.json subpath exports. Re-export from the main entrypoint
so the Next.js app can import it.
'{{method}}': request.method,
'{{path}}': request.path,
'{{headers}}': JSON.stringify(request.headers, null, 2),
'{{headers}}': JSON.stringify(redactSensitiveHeaders(request.headers), null, 2),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call doesn't pass extraHeaders, so custom webhook auth headers (e.g. x-webhook-secret) won't be redacted at this layer. Currently mitigated because TriggerDO.captureRequest redacts before DB storage, and the queue consumer reads from DB — so headers arriving here are already scrubbed.

Worth either a code comment explaining why extraHeaders isn't needed here, or threading the webhook auth header through for defense-in-depth. If a future caller passes raw headers directly to renderPromptTemplate, the custom auth header would leak into the LLM prompt.

if (process.env.NODE_ENV !== 'development') {
logExceptInTest(`headers (${traceability_logging_id})`, Object.fromEntries(headers.entries()));
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: removing the header logging block left an extra blank line here.

'set-cookie',
'x-gitlab-token',
'x-hub-signature',
'x-hub-signature-256',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding proxy-authorization to the list — it's a standard HTTP header that carries credentials, same pattern as authorization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants