fix: add text/plain Accept header / Axios detection + Sentry structured logging

[codyde]

Summary

Enhances the existing AI client markdown detection feature with improved Accept header support and structured logging using Sentry's logger format.

Key Improvements

🌐 Enhanced Accept Header Detection

Added text/plain to Accept header detection for better semantic matching:

function wantsMarkdownViaAccept(acceptHeader: string): boolean {
  return (
    acceptHeader.includes('text/markdown') ||
    acceptHeader.includes('text/x-markdown') ||
    acceptHeader.includes('text/plain')        // ← NEW
  );
}

📊 Sentry Structured Logging

Converted console.log statements to Sentry's structured logging format:

Before:

console.log(`[Middleware] ${url.pathname} - 📄 MARKDOWN (Accept header) - User-Agent: ${userAgent}`);

After:

Sentry.logger.info(`Middleware request processed: ${url.pathname}`, {
  urlPath: url.pathname,
  acceptHeader: request.headers.get('accept') || '',
  userAgent: request.headers.get('user-agent') || '',
  contentType: willServeMarkdown ? 'markdown' : 'html',
  detectionMethod: willServeMarkdown ? detectionMethod : null,
});

Real-World Impact

Example: Tools Requesting Plain Text

Common scenario with AI tools using axios:

Accept: application/json, text/plain, */*
User-Agent: axios/1.8.4

Before: Gets HTML (not ideal for programmatic use)
After: Gets markdown via text/plain detection ✅

Sentry Dashboard Analytics

With structured logging, you can now query:

// Content type distribution
contentType:markdown OR contentType:html

// Detection method effectiveness  
detectionMethod:"Accept header" OR detectionMethod:"User-agent"

// Accept header analysis
acceptHeader:"text/plain" urlPath:"/platforms/*"

// User agent patterns
userAgent:axios OR userAgent:claude

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
develop-docs	Ready	Preview	Comment	Sep 29, 2025 4:54pm
sentry-docs	Ready	Preview	Comment	Sep 29, 2025 4:54pm

[codecov]

Bundle Report

Changes will increase total bundle size by 53.01kB (0.23%) ⬆️. This is within the configured threshold ✅

Detailed changes

Bundle name	Size	Change
sentry-docs-edge-server-array-push	406.48kB	53.02kB (15.0%) ⬆️
sentry-docs-client-array-push	10.13MB	-6 bytes (-0.0%) ⬇️
sentry-docs-server-cjs	12.9MB	-9 bytes (-0.0%) ⬇️

Affected Assets, Files, and Routes:

view changes for bundle: sentry-docs-server-cjs

Assets Changed:

Asset Name	Size Change	Total Size	Change (%)
1729.js	-3 bytes	1.78MB	-0.0%
../instrumentation.js	-3 bytes	1.1MB	-0.0%
9523.js	-3 bytes	1.08MB	-0.0%

view changes for bundle: sentry-docs-edge-server-array-push

Assets Changed:

Asset Name	Size Change	Total Size	Change (%)
src/middleware.js	53.02kB	261.71kB	25.41% ⚠️

view changes for bundle: sentry-docs-client-array-push

Assets Changed:

Asset Name	Size Change	Total Size	Change (%)
static/chunks/pages/_app-*.js	-3 bytes	882.71kB	-0.0%
static/chunks/8321-*.js	-3 bytes	425.87kB	-0.0%
server/middleware-*.js	-6.46kB	1.0kB	-86.59%
server/middleware-*.js	6.46kB	7.46kB	645.5% ⚠️
static/Q3GjHDFVH0cJLd2uuFsCQ/_buildManifest.js (New)	684 bytes	684 bytes	100.0% 🚀
static/Q3GjHDFVH0cJLd2uuFsCQ/_ssgManifest.js (New)	77 bytes	77 bytes	100.0% 🚀
static/oniBsuoOqa41jMhL_MA-*.js (Deleted)	-77 bytes	0 bytes	-100.0% 🗑️
static/oniBsuoOqa41jMhL_MA-*.js (Deleted)	-684 bytes	0 bytes	-100.0% 🗑️