Skip to content

Commit 00c5934

Browse files
tarastaras
committed
feat: add /llms.txt route for AI agent discovery
Implements the llms.txt standard (https://llmstxt.org/) to make Frontside's open source projects discoverable by AI agents. - Add routes/llms-txt-route.ts with llms.txt content - Wire route in main.tsx before legacy proxy catch-all - Add AGENTS.md documenting implementation plan
1 parent 6a44214 commit 00c5934

File tree

3 files changed

+309
-0
lines changed

3 files changed

+309
-0
lines changed

‎AGENTS.md‎

Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
# AGENTS.md — Frontside.com Agent Contract
2+
3+
This file defines how AI agents should work with the frontside.com codebase and
4+
provides implementation instructions for the llms.txt initiative.
5+
6+
## Repository Overview
7+
8+
frontside.com is the main website for Frontside Software, built with:
9+
10+
- **Runtime**: Deno
11+
- **Framework**: Revolution (Frontside's JSX-based web framework)
12+
- **Styling**: Twind (Tailwind-in-JS)
13+
14+
### Architecture
15+
16+
The site proxies several subsites:
17+
18+
| Path | Target | Source |
19+
|------|--------|--------|
20+
| `/effection/*` | effection.deno.dev | thefrontside/effection (www/) |
21+
| `/interactors/*` | interactors.deno.dev | thefrontside/interactors |
22+
| `/graphgen/*` | graphgen.deno.dev | thefrontside/graphgen |
23+
| `/*` (fallback) | frontside.netlify.app | Legacy Gatsby site |
24+
25+
### Key Files
26+
27+
- `main.tsx` — Application entry point, route definitions
28+
- `routes/` — Route handlers (TSX files)
29+
- `assets/` — Static assets served at `/assets/*`
30+
- `blog/` — Blog posts in markdown with frontmatter
31+
32+
---
33+
34+
## llms.txt Implementation Plan
35+
36+
### Goal
37+
38+
Make Frontside's open source projects discoverable by AI agents through the
39+
[llms.txt standard](https://llmstxt.org/).
40+
41+
### Current State
42+
43+
- `frontside.com/llms.txt` — **404 (does not exist)**
44+
- `frontside.com/effection/assets/llms.txt` — exists (proxied from effection repo)
45+
- No root-level llms.txt routing
46+
47+
### Target State
48+
49+
1. `frontside.com/llms.txt` — Root routing file for all Frontside content
50+
2. `frontside.com/effection/llms.txt` — Detailed Effection docs (served by effection site)
51+
52+
---
53+
54+
## Implementation Tasks
55+
56+
### Task 1: Create `/llms.txt` Route
57+
58+
Add a new route that serves the llms.txt file at the root level.
59+
60+
**File to create**: `routes/llms-txt-route.ts`
61+
62+
```typescript
63+
import type { HTTPMiddleware } from "revolution";
64+
65+
export function llmsTxtRoute(): HTTPMiddleware {
66+
return function* () {
67+
return new Response(LLMS_TXT_CONTENT, {
68+
headers: {
69+
"Content-Type": "text/plain; charset=utf-8",
70+
"Cache-Control": "public, max-age=3600",
71+
},
72+
});
73+
};
74+
}
75+
76+
const LLMS_TXT_CONTENT = `# Frontside
77+
78+
> Frontside is a software consultancy specializing in Developer Experience (DX),
79+
> internal developer platforms, and open source tools for JavaScript.
80+
81+
## Services
82+
83+
- [DX Consulting](https://frontside.com/dx-consulting): Developer experience consulting for Cloud native teams
84+
- [Backstage Implementation](https://frontside.com/backstage): Internal developer portal expertise
85+
86+
## Open Source Projects
87+
88+
- [Effection llms.txt](https://frontside.com/effection/llms.txt): Structured concurrency for JavaScript (detailed docs)
89+
- [Interactors](https://frontside.com/interactors): Page objects for component library testing
90+
- [GraphGen](https://frontside.com/graphgen): Generate realistic graph data for testing
91+
92+
## Blog
93+
94+
Technical articles on DX, testing, Backstage, and JavaScript.
95+
96+
- [All posts](https://frontside.com/blog)
97+
- [Backstage articles](https://frontside.com/blog/tags/backstage)
98+
- [Testing articles](https://frontside.com/blog/tags/testing)
99+
100+
## Optional
101+
102+
- [About Frontside](https://frontside.com/about)
103+
- [Contact](https://frontside.com/contact)
104+
- [Case Studies](https://frontside.com/work/case-studies/resideo)
105+
`;
106+
```
107+
108+
**File to modify**: `main.tsx`
109+
110+
Add the route before the legacy proxy (which catches all unmatched routes):
111+
112+
```typescript
113+
import { llmsTxtRoute } from "./routes/llms-txt-route.ts";
114+
115+
// In the app array, add before proxyRoute(proxies.legacy):
116+
route("/llms.txt", llmsTxtRoute()),
117+
```
118+
119+
### Task 2: Update Effection llms.txt (in effection repo)
120+
121+
The effection repo needs to:
122+
123+
1. Move/copy `www/assets/llms.txt` to be served at `/effection/llms.txt`
124+
2. Add the EffectionX package index to the llms.txt
125+
126+
This is tracked separately in the effection repo.
127+
128+
### Task 3: Register with llms.txt Directories
129+
130+
After deployment, submit to:
131+
132+
- https://llmstxt.site/
133+
- https://directory.llmstxt.cloud/
134+
135+
---
136+
137+
## Testing the Implementation
138+
139+
### Local Testing
140+
141+
```bash
142+
# Start the dev server
143+
deno task dev
144+
145+
# Test the llms.txt endpoint
146+
curl http://localhost:8005/llms.txt
147+
148+
# Verify it returns plain text with correct content
149+
```
150+
151+
### Production Verification
152+
153+
```bash
154+
curl https://frontside.com/llms.txt
155+
# Should return the llms.txt content
156+
157+
curl https://frontside.com/effection/llms.txt
158+
# Should proxy to effection site's llms.txt
159+
```
160+
161+
---
162+
163+
## Effection llms.txt Enhancement (Reference)
164+
165+
The effection repo's llms.txt should be enhanced with the EffectionX package
166+
index. Here's the proposed addition:
167+
168+
```markdown
169+
## EffectionX Package Index
170+
171+
Extension packages for common tasks. When you need to:
172+
173+
| Task | Package | Key Exports |
174+
|------|---------|-------------|
175+
| Run shell commands | [@effectionx/process](https://frontside.com/effection/x/process) | `exec()`, `daemon()` |
176+
| Read/write files | [@effectionx/fs](https://frontside.com/effection/x/fs) | `readFile()`, `writeFile()` |
177+
| WebSocket connections | [@effectionx/websocket](https://frontside.com/effection/x/websocket) | `connect()` |
178+
| Timeout operations | [@effectionx/timebox](https://frontside.com/effection/x/timebox) | `timebox()` |
179+
| Test with Vitest | [@effectionx/vitest](https://frontside.com/effection/x/vitest) | `describe()`, `it()` |
180+
| Bridge to Effect-TS | [@effectionx/effect-ts](https://frontside.com/effection/x/effect-ts) | `fromEffect()`, `toEffect()` |
181+
| Stream transformations | [@effectionx/stream-helpers](https://frontside.com/effection/x/stream-helpers) | `filter()`, `map()`, `take()` |
182+
| Watch files for changes | [@effectionx/watch](https://frontside.com/effection/x/watch) | CLI tool |
183+
| Web Workers | [@effectionx/worker](https://frontside.com/effection/x/worker) | `useWorker()` |
184+
| Request animation frames | [@effectionx/raf](https://frontside.com/effection/x/raf) | `useRAF()` |
185+
| BDD testing harness | [@effectionx/bdd](https://frontside.com/effection/x/bdd) | `describe()`, `it()` |
186+
| Promise chaining patterns | [@effectionx/chain](https://frontside.com/effection/x/chain) | `Chain` |
187+
| Context/algebraic effects | [@effectionx/context-api](https://frontside.com/effection/x/context-api) | context utilities |
188+
| Convergence testing | [@effectionx/converge](https://frontside.com/effection/x/converge) | `converge()` |
189+
| Node.js utilities | [@effectionx/node](https://frontside.com/effection/x/node) | stream adapters |
190+
| Immutable state signals | [@effectionx/signals](https://frontside.com/effection/x/signals) | signal primitives |
191+
| YAML stream parsing | [@effectionx/stream-yaml](https://frontside.com/effection/x/stream-yaml) | YAML parser |
192+
| Task concurrency limits | [@effectionx/task-buffer](https://frontside.com/effection/x/task-buffer) | `taskBuffer()` |
193+
| Test framework adapter | [@effectionx/test-adapter](https://frontside.com/effection/x/test-adapter) | adapter base |
194+
| Tinyexec wrapper | [@effectionx/tinyexec](https://frontside.com/effection/x/tinyexec) | `exec()` |
195+
| JSONL file storage | [@effectionx/jsonl-store](https://frontside.com/effection/x/jsonl-store) | store API |
196+
| Scope evaluation | [@effectionx/scope-eval](https://frontside.com/effection/x/scope-eval) | `scopeEval()` |
197+
| Utilities collection | [@effectionx/fx](https://frontside.com/effection/x/fx) | various helpers |
198+
```
199+
200+
---
201+
202+
## CI/CD Considerations
203+
204+
### Automatic Package Index Generation
205+
206+
The EffectionX package index in llms.txt should be auto-generated. Options:
207+
208+
1. **Build-time generation** (recommended): Generate during effection www build
209+
2. **Webhook trigger**: effectionx repo triggers effection rebuild on release
210+
3. **Periodic refresh**: Cron job to regenerate
211+
212+
### Suggested Implementation
213+
214+
In the effection repo, create a script that:
215+
216+
1. Clones/fetches effectionx repo
217+
2. Reads each package's `package.json` and `README.md`
218+
3. Extracts: name, description, key exports
219+
4. Generates the markdown table
220+
5. Embeds it into llms.txt template
221+
222+
---
223+
224+
## Verification Strategy
225+
226+
### Baseline (Before Implementation)
227+
228+
Test what agents currently know:
229+
230+
```
231+
Prompt: "What packages exist in the @effectionx scope?"
232+
Expected: Low confidence, incomplete answers
233+
```
234+
235+
### After Implementation
236+
237+
1. **Immediate**: Test with context injection
238+
- Provide llms.txt content to an agent
239+
- Ask about effectionx packages
240+
- Verify correct recommendations
241+
242+
2. **Post-deployment**: Test discovery
243+
- Ask agents without context
244+
- Monitor if recommendations improve over time
245+
246+
3. **Directory listing**: Verify presence in llms.txt directories
247+
248+
---
249+
250+
## Related Issues
251+
252+
- effection repo: Enhance llms.txt with EffectionX index
253+
- effectionx repo: Add structured metadata to packages for auto-generation

‎main.tsx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { backstageServicesRoute } from "./routes/backstage.html.tsx";
1010
import { dxConsultingServicesRoute } from "./routes/dx-consulting.html.tsx";
1111
import { pluginWorkshopRoute } from "./routes/advanced-backstage-plugin-development-route.tsx";
1212
import { resideoBackstageCaseStudyRoute } from "./routes/work/case-studies/case-study-resideo.html.tsx";
13+
import { llmsTxtRoute } from "./routes/llms-txt-route.ts";
1314

1415
import { etagPlugin } from "./plugins/etag.ts";
1516
import { currentRequestPlugin } from "./plugins/current-request.ts";
@@ -31,6 +32,7 @@ await main(function* (args) {
3132

3233
let revolution = createRevolution({
3334
app: [
35+
route("/llms.txt", llmsTxtRoute()),
3436
route("/", indexRoute()),
3537
route("/blog", blogIndexRoute()),
3638
route("/blog/:id", blogPostRoute()),

‎routes/llms-txt-route.ts‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
import type { HTTPMiddleware } from "revolution";
2+
3+
/**
4+
* Serves the /llms.txt file for AI agent discovery.
5+
*
6+
* This follows the llms.txt standard (https://llmstxt.org/) to help
7+
* AI agents understand and navigate Frontside's content and projects.
8+
*/
9+
export function llmsTxtRoute(): HTTPMiddleware {
10+
return function* () {
11+
return new Response(LLMS_TXT_CONTENT, {
12+
headers: {
13+
"Content-Type": "text/plain; charset=utf-8",
14+
"Cache-Control": "public, max-age=3600",
15+
},
16+
});
17+
};
18+
}
19+
20+
const LLMS_TXT_CONTENT = `# Frontside
21+
22+
> Frontside is a software consultancy specializing in Developer Experience (DX),
23+
> internal developer platforms, and open source tools for JavaScript.
24+
25+
Frontside helps engineering teams build better developer experiences, from local
26+
development environments to production deployment pipelines. We specialize in
27+
Backstage implementation, testing strategy, and structured concurrency.
28+
29+
## Services
30+
31+
- [DX Consulting](https://frontside.com/dx-consulting): Developer experience consulting for Cloud native teams
32+
- [Backstage Implementation](https://frontside.com/backstage): Internal developer portal expertise and plugin development
33+
34+
## Open Source Projects
35+
36+
- [Effection](https://frontside.com/effection/llms.txt): Structured concurrency for JavaScript - see detailed llms.txt for API docs and extension packages
37+
- [Interactors](https://frontside.com/interactors): Page objects for component library testing
38+
- [GraphGen](https://frontside.com/graphgen): Generate realistic graph data for testing
39+
40+
## Blog
41+
42+
Technical articles on DX, testing, Backstage, and JavaScript. Notable posts:
43+
44+
- [Announcing Effection 4.0](https://frontside.com/blog/2025-12-23-announcing-effection-v4): Major release with improved determinism
45+
- [The Heartbreaking Inadequacy of AbortController](https://frontside.com/blog/2025-08-04-the-heartbreaking-inadequacy-of-abort-controller): Why JavaScript needs structured concurrency
46+
- [Effection: When async/await is not enough](https://frontside.com/blog/2021-10-26-effection-async-await): Introduction to structured concurrency
47+
48+
## Optional
49+
50+
- [All blog posts](https://frontside.com/blog)
51+
- [About Frontside](https://frontside.com/about)
52+
- [Contact](https://frontside.com/contact)
53+
- [Case Study: Resideo](https://frontside.com/work/case-studies/resideo)
54+
`;

0 commit comments

Comments
 (0)