NxPage

Serve structured JSON for AI agents/crawlers while keeping normal Next.js behavior for regular users.

What This Solves

Modern AI agents such as ChatGPT/OpenAI crawlers, Claude/Anthropic bots, Perplexity, and similar automation clients usually do not need full browser-ready HTML, JS bundles, and client hydration payloads. nxpage lets you serve a lightweight JSON representation for these agent requests, while normal users still get the standard Next.js HTML app.

This gives practical benefits:

• Faster agent response delivery (smaller payloads)
• Lower network egress and transfer cost
• Reduced CPU usage for bot traffic paths
• Cleaner extraction path for crawlers and AI systems

Depending on page size and app complexity, this can reduce transfer volume by up to ~99% for bot-targeted responses.

---

Getting Started

The fastest way to get up and running is with create-nxpage-app. It scaffolds a new Next.js project with NxPage already wired up — server config, build script, and route filters all included out of the box.

1. Scaffold your app

npx create-nxpage-app myapp
cd myapp

This creates a new project with a ready-to-use server.ts (or server.js) that calls createNxPageServer with a starter config. Open it to customise your route patterns before building:

// server.ts (generated by create-nxpage-app)
import { createNxPageServer } from "nxpage";

createNxPageServer({
  port: Number(process.env.PORT ?? 3000),
  includeRoutePatterns: ["/docs/**", "/blog/**"],
  blockRoutePatterns: ["/docs/internal/**"],
});

Any includeRoutePatterns and blockRoutePatterns you set here are automatically used during the manifest build step as well — you only define your route config once.

2. Build the app

npm run build

The build script runs both steps back to back automatically:

1. next build — produces the standard HTML output in .next/
2. nxpage build — reads the HTML output and generates lightweight JSON pages in .next/nxpage-pages/, using the same route config passed to createNxPageServer

No separate manifest step needed. The config drives everything.

3. Start the server

npm start

That's it. Your app now serves:

• Regular users → standard Next.js HTML (unchanged behaviour)
• AI agents & crawlers → compact JSON from .next/nxpage-pages/ for matched routes

What the scaffolded project looks like

myapp/
├── app/               # your Next.js pages and routes
├── server.ts          # NxPage server entry point (edit route patterns here)
├── package.json       # build script runs next build + nxpage build together
└── ...

---

Project Structure

newupdate/
├── packages/
│   ├── nxpage/
│   └── create-nxpage-app/
├── pnpm-workspace.yaml
├── tsconfig.base.json
└── package.json

Packages

• packages/nxpage — runtime server + manifest builder for bot-oriented JSON responses.
• packages/create-nxpage-app — CLI scaffolder for a Next.js app with NxPage setup.

---

Development

Install Dependencies

cd nxpage
pnpm install

If pnpm is not available on your machine:

npm i -g pnpm

Build All Packages

pnpm -r run build

Build a Single Package

pnpm --filter nxpage run build
pnpm --filter create-nxpage-app run build

Notes: Source is written in TypeScript. Build output is emitted as CommonJS (.cjs) in each package dist folder.

---

Quick Usage Flow

1. Build your Next app (human HTML build).
2. Run NxPage manifest build (bot JSON build).
3. Start NxPage server.
4. AI agents receive JSON for allowed routes; humans continue receiving normal HTML.

Dual-Build Model

Build	Command	Output
Human build	next build	.next/* — normal HTML flow
Bot build	nxpage build	.next/nxpage-pages/* — bot/agent JSON flow

You can control which routes are included/excluded with includeRoutePatterns and blockRoutePatterns.

---

create-nxpage-app

A CLI scaffolder that sets up a new Next.js app with NxPage pre-configured.

Local Development

cd newupdate/packages/create-nxpage-app
npm install
npm run build

---

nxpage

Install

npm i nxpage

Exports

• createNxPageServer(options?)
• generateNxPageManifest(options?)

---

How It Works

NxPage uses the same Next.js build as input, then creates two different response outputs:

1. Human output (HTML build) — Standard Next.js pages generated by next build, served normally for human/browser traffic.
2. Bot output (JSON build) — NxPage reads generated HTML files and creates per-route JSON pages for AI agents.

So in practice you get:

• Normal Next HTML for users
• Lightweight JSON pages for AI agents (ChatGPT/OpenAI, Claude, Perplexity, and others)

Build Flow

# 1) Normal Next.js build (HTML output)
next build

# 2) NxPage bot build (JSON output)
nxpage build

After this, your app has both artifacts:

• Next HTML output in .next/*
• NxPage JSON output in .next/nxpage-pages/*

---

createNxPageServer

Create a custom server that:

• Detects bot/agent traffic
• Serves prebuilt NxPage JSON manifest for matching routes
• Falls back to normal Next request handling for other cases

Options

Option	Type	Default	Description
port	number	3000	Port to listen on
manifestPath	string	.next/nxpage-pages/server/app	Path to JSON manifest
isBot	(req) => boolean	built-in	Custom bot detection function
includeRoutePatterns	(string | RegExp)[]	—	Only these routes are eligible for JSON serving
blockRoutePatterns	(string | RegExp)[]	—	Always excludes matched routes

Route Pattern Behavior

• If includeRoutePatterns is provided, only matching routes are eligible.
• blockRoutePatterns always excludes matched routes.
• Final rule: route must pass include filter (if any) and not match block filter.

Example

import { createNxPageServer } from "nxpage";

createNxPageServer({
  port: 3000,
  includeRoutePatterns: ["/docs/**", "/blog/**"],
  blockRoutePatterns: ["/docs/internal/**"],
});

---

generateNxPageManifest

Scans Next build output HTML and generates JSON manifest files consumed by the server.

Options

Option	Type	Default	Description
distDir	string	.next	Next.js build output directory
manifestPath	string	—	Override output path for the manifest
includeRoutePatterns	(string | RegExp)[]	—	Routes to include
blockRoutePatterns	(string | RegExp)[]	—	Routes to exclude

Example

import { generateNxPageManifest } from "nxpage";

await generateNxPageManifest({
  distDir: ".next",
  includeRoutePatterns: ["/docs/**"],
  blockRoutePatterns: ["/docs/internal/**"],
});

---

CLI

The package includes a CLI command:

nxpage build

This runs manifest generation with default options.

---

End-to-End Usage

1) Build Next.js app

next build

2) Generate NxPage route JSON

import { generateNxPageManifest } from "nxpage";

await generateNxPageManifest({
  distDir: ".next",
  includeRoutePatterns: ["/docs/**", "/blog/**"],
  blockRoutePatterns: ["/docs/internal/**"],
});

3) Start NxPage server

import { createNxPageServer } from "nxpage";

createNxPageServer({
  port: Number(process.env.PORT ?? 3000),
  includeRoutePatterns: ["/docs/**", "/blog/**"],
  blockRoutePatterns: ["/docs/internal/**"],
});

4) Result

• AI agents receive NxPage JSON for allowed routes.
• Blocked or non-included routes fall back to normal Next.js handling.
• Human users continue to receive standard HTML pages.

---

Local Development (Monorepo)

cd newupdate/packages/nxpage
npm install
npm run build