Implement Markdown Export Feature: Add API route for .md exports and update middleware for URL handling. Remove llms.txt.

Author: codyde

LLMS_TXT_FEATURE.md

@@ -1,8 +1,26 @@
-# LLMs.txt Feature Documentation
+# Markdown Export Feature Documentation
 
-## Overview
+This feature allows converting any page on the Sentry documentation site to a plain markdown format by simply appending `.md` to the end of any URL (without a trailing slash). The feature extracts the actual page content from the source MDX files and converts it to clean markdown, making the documentation more accessible to Large Language Models and other tools.
 
-This feature allows converting any page on the Sentry documentation site to a plain markdown format by simply appending `llms.txt` to the end of any URL. The feature extracts the actual page content from the source MDX files and converts it to clean markdown, making the documentation more accessible to Large Language Models (LLMs) and other automated tools.
+## Example URLs
+
+- Markdown Export: https://docs.sentry.io/platforms/javascript/guides/react/user-feedback.md
+- Markdown Export: https://docs.sentry.io/platforms/javascript/guides/nextjs/user-feedback.md
+- Markdown Export: https://docs.sentry.io/platforms/javascript/guides/react/tracing.md
+
+## How it works
+
+- URL: /platforms/javascript/guides/react/user-feedback.md
+- Rewrite: /api/md-export/platforms/javascript/guides/react/user-feedback
+
+## Example usage
+
+```
+curl "http://localhost:3000/platforms/javascript/guides/react/user-feedback.md"
+curl "http://localhost:3000/platforms/javascript/guides/nextjs/user-feedback.md"
+curl "http://localhost:3000/platforms/javascript/guides/react/tracing.md"
+curl "http://localhost:3000/platforms/javascript/guides/vue/user-feedback.md"
+```
 
 ## ✅ **Feature Status: FULLY WORKING**
 
@@ -112,128 +130,4 @@ yarn add @sentry/react
 
 ```bash {tabTitle:pnpm}
 pnpm add @sentry/react
-```
-```
-
-## Smart Content Processing
-
-### Enhanced JSX Component Handling
-- **Alert components** → `> **Note:** [content]`
-- **PlatformIdentifier** → `` `traces-sample-rate` ``
-- **PlatformLink** → `[Link Text](/path/to/page)`
-- **PlatformContent includes** → **Actual platform-specific content loaded from files**
-- **Code block preservation** → All existing markdown code blocks preserved
-- **Nested components** → Multi-pass processing ensures complete cleanup
-
-### Content Preservation Technology
-- ✅ **Code Block Protection**: Temporarily replaces code blocks during JSX cleanup
-- ✅ **Include Resolution**: Loads real content from platform-includes directory
-- ✅ **Platform Awareness**: Automatically detects platform/guide from URL path
-- ✅ **Smart Fallbacks**: Graceful degradation when includes aren't found
-- ✅ **Content Reconstruction**: Restores protected content after cleanup
-
-## Response Format with Code Snippets
-
-```markdown
-# Set Up User Feedback
-
-The User Feedback feature allows you to collect user feedback from anywhere inside your application at any time.
-
-> **Note:** 
-If you're using a self-hosted Sentry instance, you'll need to be on version 24.4.2 or higher.
-
-### Installation
-
-The User Feedback integration is **already included** with the React SDK package.
-
-```bash {tabTitle:npm}
-npm install @sentry/react --save
-```
-
-```bash {tabTitle:yarn}
-yarn add @sentry/react
-```
-
-```bash {tabTitle:pnpm}
-pnpm add @sentry/react
-```
-
-### Set Up
-
-```javascript
-Sentry.init({
-  dsn: "___PUBLIC_DSN___",
-
-  integrations: [
-    Sentry.feedbackIntegration({
-      colorScheme: "system",
-    }),
-  ],
-});
-```
-
----
-
-**Original URL**: https://docs.sentry.io/platforms/javascript/guides/react/user-feedback
-**Generated**: 2025-06-10T22:25:15.123Z
-
-*This is the full page content converted to markdown format.*
-```
-
-## Technical Implementation
-
-### Enhanced API Route Features
-```typescript
-// New async function for include resolution
-async function resolvePlatformIncludes(content: string, pathSegments: string[]): Promise<string> {
-  // Platform detection from URL segments
-  // File loading from platform-includes/
-  // Content replacement with error handling
-}
-
-// Code block preservation during cleanup
-cleaned = cleaned.replace(/```[\s\S]*?```/g, (match) => {
-  codeBlocks.push(match);
-  return `${codeBlockPlaceholder}${codeBlocks.length - 1}`;
-});
-```
-
-### File System Integration
-- **Direct file access** to `platform-includes/` directory
-- **Gray-matter parsing** for include files
-- **Multi-path resolution** with intelligent fallbacks
-- **Error handling** with descriptive placeholders
-
-## Benefits
-
-✅ **Complete Content**: Extracts actual page content AND code snippets  
-✅ **Platform-Specific**: Shows correct installation/setup for each framework  
-✅ **LLM-Optimized**: Clean markdown format perfect for AI processing  
-✅ **Smart Conversion**: JSX components converted to appropriate markdown  
-✅ **Code Preservation**: All code blocks and snippets properly maintained  
-✅ **Universal Access**: Works with any documentation page  
-✅ **High Performance**: Cached responses with efficient processing  
-✅ **Error Resilient**: Graceful fallbacks and informative error messages  
-
-## Testing Commands
-
-```bash
-# Test React user feedback with full code snippets
-curl "http://localhost:3000/platforms/javascript/guides/react/user-feedback/llms.txt"
-
-# Test Next.js-specific installation (shows wizard command)
-curl "http://localhost:3000/platforms/javascript/guides/nextjs/user-feedback/llms.txt"
-
-# Test React tracing with platform-specific content
-curl "http://localhost:3000/platforms/javascript/guides/react/tracing/llms.txt"
-
-# Test Vue.js user feedback (different platform)
-curl "http://localhost:3000/platforms/javascript/guides/vue/user-feedback/llms.txt"
-```
-
----
-
-**Status**: ✅ **PRODUCTION READY WITH FULL CODE SNIPPETS**  
-**Last Updated**: December 2024  
-**Content Quality**: Full page content with platform-specific code examples  
-**Code Coverage**: Installation, setup, configuration, and API examples all included
+```

app/api/llms-txt/[...path]/route.ts renamed to app/api/md-export/[...path]/route.ts

@@ -8,6 +8,7 @@ import {getDocsRootNode, nodeForPath} from 'sentry-docs/docTree';
 import {isDeveloperDocs} from 'sentry-docs/isDeveloperDocs';
 import {getFileBySlugWithCache} from 'sentry-docs/mdx';
 
+// This route handles .md export requests for any documentation page
 export async function GET(
   _request: Request,
   {params}: {params: Promise<{path: string[]}>}
@@ -198,7 +199,7 @@ For the complete content with full formatting, code examples, and interactive el
 
     return createResponse(pageTitle, cleanContent, `/${pathSegments.join('/')}`);
   } catch (error) {
-    console.error('Error generating llms.txt:', error);
+    console.error('Error generating .md export:', error);
     return new NextResponse('Internal server error', {status: 500});
   }
 }
@@ -328,7 +329,7 @@ async function resolvePlatformIncludes(
   // Build platform identifier for include files
   let platformId = platform;
   if (guide && guide !== platform) {
-    platformId = `${platform}.${guide}`;
+         platformId = `${platform}.${guide}`;
   }
 
   // Replace PlatformContent includes with actual content
@@ -383,4 +384,4 @@ async function resolvePlatformIncludes(
   }
 
   return result;
-}
+} 

redirects.js

@@ -2,11 +2,6 @@ const isDeveloperDocs = !!process.env.NEXT_PUBLIC_DEVELOPER_DOCS;
 
 /** @type {import('next/dist/lib/load-custom-routes').Redirect[]} */
 const developerDocsRedirects = [
-  // LLMs.txt redirect - converts any page to markdown format
-  {
-    source: '/:path*/llms.txt',
-    destination: '/api/llms-txt/:path*',
-  },
   {
     source: '/sdk/miscellaneous/unified-api/tracing/:path*',
     destination: '/sdk/performance/:path*',
@@ -223,11 +218,6 @@ const developerDocsRedirects = [
 
 /** @type {import('next/dist/lib/load-custom-routes').Redirect[]} */
 const userDocsRedirects = [
-  // LLMs.txt redirect - converts any page to markdown format
-  {
-    source: '/:path*/llms.txt',
-    destination: '/api/llms-txt/:path*',
-  },
   {
     source: '/organization/integrations/telegram-alerts-bot/',
     destination: '/organization/integrations/notification-incidents/telegram-alerts-bot/',

src/middleware.ts

@@ -19,6 +19,13 @@ export const config = {
 
 // This function can be marked `async` if using `await` inside
 export function middleware(request: NextRequest) {
+  // Handle .md export requests by rewriting to API route while preserving URL
+  if (request.nextUrl.pathname.endsWith('.md')) {
+    const pathWithoutMd = request.nextUrl.pathname.slice(0, -3); // Remove .md
+    const rewriteUrl = new URL(`/api/md-export${pathWithoutMd}`, request.url);
+    return NextResponse.rewrite(rewriteUrl);
+  }
+
   // Remove the llms.txt handling - it's now handled by Next.js redirects
   return handleRedirects(request);
 }