|
| 1 | +--- |
| 2 | +pcx_content_type: concept |
| 3 | +title: NLWeb |
| 4 | +sidebar: |
| 5 | + order: 1 |
| 6 | +--- |
| 7 | + |
| 8 | +Enable conversational search on your website with NLWeb and Cloudflare AutoRAG. This template crawls your site, indexes the content, and deploys NLWeb-standard endpoints to serve both people and AI agents. |
| 9 | + |
| 10 | +:::note |
| 11 | +This is a public preview ideal for experimentation. If you're interested in running this in production workflows, please contact us at [email protected]. |
| 12 | +::: |
| 13 | + |
| 14 | +## What is NLWeb |
| 15 | + |
| 16 | +[NLWeb](https://github.com/nlweb-ai/NLWeb) is an open project developed by Microsoft that defines a standard protocol for natural language queries on websites. Its goal is to make every website as accessible and interactive as a conversational AI app, so both people and AI agents can reliably query site content. It does this by exposing two key endpoints: |
| 17 | + |
| 18 | +- `/ask`: Conversational endpoint for user queries |
| 19 | +- `/mcp`: Structured Model Context Protocol (MCP) endpoint for AI agents |
| 20 | + |
| 21 | +## How to use it |
| 22 | + |
| 23 | +You can deploy NLWeb on your website directly through the AutoRAG dashboard: |
| 24 | + |
| 25 | +1. Log in to your [Cloudflare dashboard](https://dash.cloudflare.com/). |
| 26 | +2. Go to **Compute & AI** > **AutoRAG**. |
| 27 | +3. Select **Create AutoRAG**, then choose the **NLWeb Website** option. |
| 28 | +4. Select your domain from your Cloudflare account. |
| 29 | +5. Click **Start indexing**. |
| 30 | + |
| 31 | +Once complete, AutoRAG will crawl and index your site, then deploy an NLWeb Worker for you. |
| 32 | + |
| 33 | +## What this template includes |
| 34 | + |
| 35 | +Choosing the NLWeb Website option extends a normal AutoRAG by tailoring it for content‑heavy websites and giving you everything that is required to adopt NLWeb as the standard for conversational search on your site. Specifically, the template provides: |
| 36 | + |
| 37 | +- **Website as a data source:** Uses [Website](/autorag/configuration/data-source/website/) as data source option to crawl and ingest pages with the Rendered Sites option. |
| 38 | +- **Defaults for content-heavy websites:** Applies tuned embedding and retrieval configurations ideal for publishing and content‑rich websites. |
| 39 | +- **NLWeb Worker deployment:** Automatically spins up a Cloudflare Worker from the [NLWeb Worker template](https://github.com/cloudflare/templates). |
| 40 | + |
| 41 | +## What the Worker includes |
| 42 | + |
| 43 | +Your deployed Worker provides two endpoints: |
| 44 | + |
| 45 | +- `/ask` — NLWeb’s standard conversational endpoint |
| 46 | + - Powers the conversational UI at the root (`/`) |
| 47 | + - Powers the embeddable preview widget (`/snippet.html`) |
| 48 | +- `/mcp` — NLWeb’s MCP server endpoint for trusted AI agents |
| 49 | + |
| 50 | +These endpoints give both people and agents structured access to your content. |
| 51 | + |
| 52 | +## Using It on Your Website |
| 53 | + |
| 54 | +To integrate NLWeb search directly into your site you can: |
| 55 | + |
| 56 | +1. Find your deployed Worker in the [Cloudflare dashboard](https://dash.cloudflare.com/): |
| 57 | + - Go to **Compute & AI** > **AutoRAG**. |
| 58 | + - Select **Connect**, then go to the **NLWeb** tab. |
| 59 | + - Select **Go to Worker**. |
| 60 | +2. Add a [custom domain](/workers/configuration/routing/custom-domains/) to your Worker (for example, ask.example.com) |
| 61 | +3. Use the `/ask` endpoint on your custom domain to power the search (for example, ask.example.com/ask) |
| 62 | + |
| 63 | +You can also use the embeddable snippet to add a search UI directly into your website. For example: |
| 64 | + |
| 65 | +```html |
| 66 | +<!-- Add css on head --> |
| 67 | + <link rel="stylesheet" href="https://ask.example.com/nlweb-dropdown-chat.css"> |
| 68 | + <link rel="stylesheet" href="https://ask.example.com/common-chat-styles.css"> |
| 69 | + |
| 70 | + <!-- Add container on body --> |
| 71 | + <div id="docs-search-container"></div> |
| 72 | + |
| 73 | + <!-- Include JavaScript --> |
| 74 | + <script type="module"> |
| 75 | + import { NLWebDropdownChat } from 'https://ask.example.com/nlweb-dropdown-chat.js'; |
| 76 | + |
| 77 | + const chat = new NLWebDropdownChat({ |
| 78 | + containerId: 'docs-search-container', |
| 79 | + site: 'https://ask.example.com', |
| 80 | + placeholder: 'Search for docs...', |
| 81 | + endpoint: 'https://ask.example.com' |
| 82 | + }); |
| 83 | + </script> |
| 84 | +``` |
| 85 | + |
| 86 | +This lets you serve conversational AI search directly from your own domain, with control over how people and agents access your content. |
| 87 | + |
| 88 | + |
| 89 | +## Modifying or updating the Worker |
| 90 | + |
| 91 | +You may want to customize your Worker, for example, to adjust the UI for the embeddable snippet. In those cases, we recommend calling the `/ask` endpoint for queries and building your own UI on top of it, however, you may also choose to modify the Worker's code for the embeddable UI. |
| 92 | + |
| 93 | +If the NLWeb standard is updated, you can update your Worker to stay compatible and recieve the latest updates. |
| 94 | + |
| 95 | +The simplest way to apply changes or updates is to redeploy the Worker template: |
| 96 | + |
| 97 | +[](https://deploy.workers.cloudflare.com/?url=https://github.com/cloudflare/templates/tree/main/nlweb-template) |
| 98 | + |
| 99 | +To do so: |
| 100 | + |
| 101 | +1. Select the **Deploy to Cloudflare** button from above to deploy the Worker template to your Cloudflare account. |
| 102 | +2. Enter the name of your AutoRAG in the `RAG_ID` environment variable field. |
| 103 | +3. Click **Deploy**. |
| 104 | +4. Select the **GitHub/GitLab** icon on the Workers Dashboard. |
| 105 | +4. Clone the repository that is created for your Worker. |
| 106 | +5. Make your modifications, then commit and push changes to the repository to update your Worker. |
| 107 | + |
| 108 | +Now you can use this Worker as the new NLWeb endpoint for your website. |
0 commit comments