Skip to content

Commit 2c79c49

Browse files
anth-volkclaude
andcommitted
Add parameter path verification and fix scaffold config for dashboard agents
- Add policyengine-parameter-patterns-skill to backend-builder agent - Add Step 3b: Verify All Parameter Paths in backend-builder - Add Check 7: Parameter Path Verification to architecture validator - Add postcss.config.mjs requirement to scaffold agent (required for Tailwind v4) - Replace random port selection with 4000-4100 range macro in Makefile templates - Add copy-pasteable Makefile code for both precomputed and custom-modal patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent ec3bd93 commit 2c79c49

File tree

3 files changed

+190
-39
lines changed

3 files changed

+190
-39
lines changed

agents/dashboard/backend-builder.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ Builds the data layer for a dashboard based on the approved `plan.yaml`.
2222
- **policyengine-interactive-tools-skill** - Data patterns and API integration
2323
- **policyengine-us-skill** or **policyengine-uk-skill** - PolicyEngine variables
2424
- **policyengine-simulation-mechanics-skill** - How simulations work (custom-backend only)
25+
- **policyengine-parameter-patterns-skill** - Parameter YAML structure, bracket path syntax, and Reform.from_dict() paths (custom-backend only)
2526

2627
## Backend Selection Priority
2728

@@ -41,6 +42,7 @@ Pattern C is the most complex and should be the last resort. If the plan specifi
4142
2. `Skill: policyengine-us-skill` (if US dashboard)
4243
3. `Skill: policyengine-uk-skill` (if UK dashboard)
4344
4. `Skill: policyengine-simulation-mechanics-skill` (if custom-backend pattern)
45+
5. `Skill: policyengine-parameter-patterns-skill` (if custom-backend pattern — **required** for correct Reform.from_dict() paths)
4446

4547
## Input
4648

@@ -257,6 +259,39 @@ def run_statewide(params: dict) -> dict:
257259
return {"revenue_change": ..., "winners": ..., "losers": ...}
258260
```
259261

262+
### Step 3b: Verify All Parameter Paths
263+
264+
**CRITICAL — every parameter path used in `Reform.from_dict()` or reform dictionaries MUST be verified against the actual YAML files.** Incorrect paths cause silent failures or runtime errors like "Could not find the parameter". Consult the `policyengine-parameter-patterns-skill` section 6.5 for the bracket path syntax rules.
265+
266+
**Common mistakes:**
267+
- **Off-by-one indexing**: Some parameters use 1-indexed keys (e.g., `gov.irs.income.bracket.rates` has keys `1`-`7`, not `0`-`6`). Always check whether the YAML uses a list (0-indexed) or explicit integer keys (use those exact keys).
268+
- **Missing sub-keys on bracket scales**: Bracket/scale parameters (YAML `brackets:` list) require `.amount` or `.rate` after the index. E.g., `gov.irs.credits.eitc.max[0].amount`, NOT `gov.irs.credits.eitc.max[0]`.
269+
- **Filing-status-indexed parameters**: Some parameters have sub-keys by filing status (e.g., `gov.irs.credits.ctc.phase_out.threshold[SINGLE]`).
270+
271+
**Verification procedure for every parameter path:**
272+
273+
1. Find the YAML file in the `policyengine-us` (or `policyengine-uk`) parameters directory:
274+
```bash
275+
# Convert dotted path to directory path and search
276+
find $(python3 -c "import policyengine_us; import os; print(os.path.dirname(policyengine_us.__file__))") \
277+
-path "*/parameters/gov/irs/income/bracket.yaml" 2>/dev/null
278+
```
279+
280+
2. Read the YAML and check whether the parameter uses:
281+
- **Explicit integer keys** (e.g., `1:`, `2:`, `3:`) → use those exact indices: `path[1]`, `path[2]`
282+
- **A `brackets:` list** → use 0-indexed with sub-key: `path[0].amount`, `path[0].rate`
283+
- **Filing-status sub-keys** → append `[SINGLE]`, `[JOINT]`, etc.
284+
285+
3. Verify programmatically (if policyengine is installed locally):
286+
```python
287+
from policyengine_us import CountryTaxBenefitSystem
288+
p = CountryTaxBenefitSystem().parameters
289+
# Navigate and confirm the path resolves:
290+
print(p.gov.irs.income.bracket.rates[1]("2026-01-01")) # Should return 0.10
291+
```
292+
293+
**Do NOT guess parameter paths from memory.** Always verify against the actual YAML files.
294+
260295
### Step 4: Create Worker App
261296

262297
Generate `backend/app.py`. Only `modal` at module level. Imports business logic **inside each function body**:

agents/dashboard/dashboard-architecture-validator.md

Lines changed: 66 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ Checks that the dashboard uses the correct framework, styling infrastructure, an
1212
## Skills Used
1313

1414
- **policyengine-frontend-builder-spec-skill** — Authoritative spec to validate against
15+
- **policyengine-parameter-patterns-skill** — Bracket path syntax for parameter path verification (custom-modal only)
1516

1617
## First: Load Required Skills
1718

1819
1. `Skill: policyengine-frontend-builder-spec-skill`
20+
2. `Skill: policyengine-parameter-patterns-skill` (if `data_pattern: custom-modal`)
1921

2022
After loading the skill, extract every MUST / MUST NOT statement and validate each one.
2123

@@ -33,15 +35,21 @@ grep -n '@policyengine/ui-kit/theme.css' app/globals.css
3335

3436
# tailwindcss in package.json
3537
grep '"tailwindcss"' package.json
38+
39+
# postcss.config.mjs exists with @tailwindcss/postcss
40+
test -f postcss.config.mjs && grep '@tailwindcss/postcss' postcss.config.mjs
41+
42+
# @tailwindcss/postcss in package.json devDependencies
43+
grep '@tailwindcss/postcss' package.json
3644
```
3745

3846
**Prohibited:**
3947
```bash
4048
# No tailwind.config.ts/js
4149
test ! -f tailwind.config.ts && test ! -f tailwind.config.js
4250

43-
# No postcss.config
44-
test ! -f postcss.config.js && test ! -f postcss.config.mjs
51+
# No old-style postcss.config.js (must be .mjs with @tailwindcss/postcss)
52+
test ! -f postcss.config.js
4553

4654
# No @tailwind directives
4755
grep -rn '@tailwind' app/ --include='*.css'
@@ -157,13 +165,67 @@ grep -n 'policyengine' backend/modal_app.py
157165
# Should find NOTHING — gateway is lightweight
158166
```
159167

168+
### 7. Parameter Path Verification (custom-modal only)
169+
170+
**Only run this check if `plan.yaml` has `data_pattern: custom-modal`.** Skip entirely for other patterns.
171+
172+
This validates that all parameter paths used in `Reform.from_dict()` or reform dictionaries in `backend/simulation.py` resolve to real parameters in the policyengine parameter tree.
173+
174+
**Load required skill first:** `Skill: policyengine-parameter-patterns-skill` — see section 6.5 for bracket path syntax rules.
175+
176+
**Step 1: Extract all parameter paths from simulation.py:**
177+
```bash
178+
# Find all string literals that look like parameter paths (gov.xxx.yyy)
179+
grep -oP '"gov\.[a-z_.]+(\[[A-Z_0-9]+\])*(\.[a-z_]+)*"' backend/simulation.py | sort -u
180+
# Also check for f-string patterns building paths
181+
grep -n 'gov\.' backend/simulation.py | grep -v '#'
182+
```
183+
184+
**Step 2: For each parameter path, find and verify the YAML source:**
185+
```bash
186+
# Convert a dotted path like gov.irs.income.bracket.rates to a file search
187+
# The YAML file is at parameters/gov/irs/income/bracket.yaml with rates as a child node
188+
```
189+
190+
**Step 3: Check indexing correctness:**
191+
- If the YAML has explicit integer keys (`1:`, `2:`, `3:`, ...): verify the code uses those exact indices, NOT 0-based
192+
- If the YAML has a `brackets:` list: verify the code uses 0-based indices WITH `.amount` or `.rate` sub-key
193+
- If the YAML has filing-status sub-keys: verify the code appends `[SINGLE]`, `[JOINT]`, etc.
194+
195+
**Step 4: Verify programmatically (preferred):**
196+
```bash
197+
# If policyengine-us is installed locally (e.g., in backend/):
198+
cd backend && uv run python3 -c "
199+
from policyengine_us import CountryTaxBenefitSystem
200+
p = CountryTaxBenefitSystem().parameters
201+
# Test each path — will raise ParameterNotFoundError if wrong
202+
paths_to_check = [
203+
# Paste extracted paths here
204+
]
205+
for path in paths_to_check:
206+
try:
207+
node = p
208+
# Navigate the path
209+
# ... (manual or eval-based traversal)
210+
print(f'PASS: {path}')
211+
except Exception as e:
212+
print(f'FAIL: {path} — {e}')
213+
"
214+
```
215+
216+
**Common failures to flag:**
217+
- `rates[0]` when the YAML uses 1-indexed keys (`1:` through `7:`) → FAIL
218+
- `eitc.max[0]` without `.amount` suffix on a bracket scale → FAIL
219+
- `rates[0]` without `.rate` suffix on a bracket scale → FAIL
220+
- Filing-status paths missing the `[SINGLE]`/`[JOINT]` index → FAIL
221+
160222
## Report Format
161223

162224
```
163225
## Architecture Compliance Report
164226
165227
### Summary
166-
- PASS: X/6 checks (or X/5 if not custom-modal)
228+
- PASS: X/7 checks (or X/5 if not custom-modal)
167229
- FAIL: Y checks
168230
169231
### Results
@@ -176,6 +238,7 @@ grep -n 'policyengine' backend/modal_app.py
176238
| 4 | Package manager | PASS/FAIL | ... |
177239
| 5 | Tailwind classes used | PASS/FAIL | ... |
178240
| 6 | Modal backend structure | PASS/FAIL/SKIP | ... |
241+
| 7 | Parameter path verification | PASS/FAIL/SKIP | ... |
179242
180243
### Failures (if any)
181244

agents/dashboard/dashboard-scaffold.md

Lines changed: 89 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ Generate a CLAUDE.md following the pattern from existing applets (givecalc, ctc-
151151

152152
```bash
153153
bun install
154-
make dev # Full dev stack (Modal + frontend, random port)
154+
make dev # Full dev stack (Modal + frontend, port 4000-4100)
155155
make dev-frontend # Frontend only
156156
```
157157

@@ -182,6 +182,8 @@ make build
182182
Generate from the fixed tech stack, including:
183183
- `next`, `react`, `react-dom` (^19)
184184
- `tailwindcss` (^4)
185+
- `@tailwindcss/postcss` (dev)
186+
- `postcss` (dev)
185187
- `@policyengine/ui-kit`
186188
- `recharts` (if custom charts beyond ui-kit)
187189
- `react-plotly.js` (if maps in plan)
@@ -201,6 +203,18 @@ const nextConfig: NextConfig = {
201203
export default nextConfig
202204
```
203205

206+
#### postcss.config.mjs
207+
208+
**Required for Tailwind v4.** Without this file, `@import "tailwindcss"` in globals.css is never processed and no utility classes are generated.
209+
210+
```js
211+
export default {
212+
plugins: {
213+
"@tailwindcss/postcss": {},
214+
},
215+
};
216+
```
217+
204218
#### app/globals.css
205219

206220
Generate the Tailwind v4 configuration with the ui-kit theme import:
@@ -393,10 +407,21 @@ Generate a `Makefile` that provides standard development targets. The Makefile c
393407

394408
**IMPORTANT:** Makefile recipes must use literal tab characters for indentation, not spaces.
395409

396-
**For all patterns:** The `dev` and `dev-frontend` targets must use a random available port, never hardcode port 3000. Use this Python one-liner to find a free port:
410+
**For all patterns:** The `dev` and `dev-frontend` targets must try port 4000 first, then increment by 1 up to 4100, erroring out if no port in that range is available. Use this helper script embedded in the Makefile — copy it exactly:
397411

398-
```
399-
python3 -c 'import socket; s=socket.socket(); s.bind(("",0)); print(s.getsockname()[1]); s.close()'
412+
```makefile
413+
# Port selection helper — finds the first available port in 4000-4100
414+
define find_port
415+
$$(python3 -c '\
416+
import socket, sys;\
417+
for p in range(4000, 4101):\
418+
try:\
419+
s = socket.socket(); s.bind(("", p)); s.close(); print(p); sys.exit(0)\
420+
except OSError:\
421+
continue\
422+
print("ERROR: no free port in 4000-4100", file=sys.stderr); sys.exit(1)\
423+
')
424+
endef
400425
```
401426

402427
**For `precomputed`, `policyengine-api`, or `precomputed-csv` patterns:**
@@ -405,26 +430,39 @@ python3 -c 'import socket; s=socket.socket(); s.bind(("",0)); print(s.getsocknam
405430
.PHONY: dev dev-frontend
406431
.PHONY: build test lint clean
407432
408-
# Start development server on a random available port
433+
# Port selection helper — finds the first available port in 4000-4100
434+
define find_port
435+
$$(python3 -c '\
436+
import socket, sys;\
437+
for p in range(4000, 4101):\
438+
try:\
439+
s = socket.socket(); s.bind(("", p)); s.close(); print(p); sys.exit(0)\
440+
except OSError:\
441+
continue\
442+
print("ERROR: no free port in 4000-4100", file=sys.stderr); sys.exit(1)\
443+
')
444+
endef
445+
446+
# Start development server
409447
dev: dev-frontend
410448
411449
# Frontend only (no backend for this data pattern)
412450
dev-frontend:
413-
@PORT=$$(python3 -c 'import socket; s=socket.socket(); s.bind(("",0)); print(s.getsockname()[1]); s.close()'); \
414-
echo "Starting dev server on http://localhost:$$PORT"; \
415-
PORT=$$PORT bun run dev
451+
@PORT=$(find_port); \
452+
echo "Frontend: http://localhost:$$PORT"; \
453+
PORT=$$PORT bun run dev
416454
417455
build:
418-
bun run build
456+
bun run build
419457
420458
test:
421-
bunx vitest run
459+
bunx vitest run
422460
423461
lint:
424-
bun run lint
462+
bun run lint
425463
426464
clean:
427-
rm -rf .next node_modules
465+
rm -rf .next node_modules
428466
```
429467

430468
**For `custom-modal` pattern:**
@@ -437,48 +475,61 @@ The custom-modal pattern uses a **gateway + worker architecture** with frontend
437475
.PHONY: dev dev-frontend dev-backend deploy-worker
438476
.PHONY: build test test-backend lint clean
439477
478+
# Port selection helper — finds the first available port in 4000-4100
479+
define find_port
480+
$$(python3 -c '\
481+
import socket, sys;\
482+
for p in range(4000, 4101):\
483+
try:\
484+
s = socket.socket(); s.bind(("", p)); s.close(); print(p); sys.exit(0)\
485+
except OSError:\
486+
continue\
487+
print("ERROR: no free port in 4000-4100", file=sys.stderr); sys.exit(1)\
488+
')
489+
endef
490+
440491
# Deploy worker functions, then start gateway + frontend
441492
dev:
442-
@echo "Deploying worker functions..."
443-
@unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET && modal deploy backend/app.py
444-
@echo "Starting gateway (ephemeral)..."
445-
@modal serve backend/modal_app.py & MODAL_PID=$$!; \
446-
sleep 5; \
447-
MODAL_URL="https://policyengine--DASHBOARD_NAME-fastapi-app-dev.modal.run"; \
448-
PORT=$$(python3 -c 'import socket; s=socket.socket(); s.bind(("",0)); print(s.getsockname()[1]); s.close()'); \
449-
echo "Gateway: $$MODAL_URL"; \
450-
echo "Frontend: http://localhost:$$PORT"; \
451-
NEXT_PUBLIC_API_URL=$$MODAL_URL PORT=$$PORT bun run dev; \
452-
kill $$MODAL_PID 2>/dev/null
493+
@echo "Deploying worker functions..."
494+
@unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET && modal deploy backend/app.py
495+
@echo "Starting gateway (ephemeral)..."
496+
@modal serve backend/modal_app.py & MODAL_PID=$$!; \
497+
sleep 5; \
498+
MODAL_URL="https://policyengine--DASHBOARD_NAME-fastapi-app-dev.modal.run"; \
499+
PORT=$(find_port); \
500+
echo "Gateway: $$MODAL_URL"; \
501+
echo "Frontend: http://localhost:$$PORT"; \
502+
NEXT_PUBLIC_API_URL=$$MODAL_URL PORT=$$PORT bun run dev; \
503+
kill $$MODAL_PID 2>/dev/null
453504
454505
# Frontend only (uses production API or NEXT_PUBLIC_API_URL if set)
455506
dev-frontend:
456-
@PORT=$$(python3 -c 'import socket; s=socket.socket(); s.bind(("",0)); print(s.getsockname()[1]); s.close()'); \
457-
echo "Starting dev server on http://localhost:$$PORT"; \
458-
PORT=$$PORT bun run dev
507+
@PORT=$(find_port); \
508+
echo "Frontend: http://localhost:$$PORT"; \
509+
PORT=$$PORT bun run dev
459510
460511
# Backend only (gateway in dev mode — worker must already be deployed)
461512
dev-backend:
462-
modal serve backend/modal_app.py
513+
modal serve backend/modal_app.py
463514
464515
# Deploy worker functions to Modal (required before gateway can spawn jobs)
465516
deploy-worker:
466-
unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET && modal deploy backend/app.py
517+
unset MODAL_TOKEN_ID MODAL_TOKEN_SECRET && modal deploy backend/app.py
467518
468519
build:
469-
bun run build
520+
bun run build
470521
471522
test:
472-
bunx vitest run
523+
bunx vitest run
473524
474525
test-backend:
475-
cd backend && uv run pytest
526+
cd backend && uv run pytest
476527
477528
lint:
478-
bun run lint
529+
bun run lint
479530
480531
clean:
481-
rm -rf .next node_modules
532+
rm -rf .next node_modules
482533
```
483534

484535
#### Favicon
@@ -567,7 +618,8 @@ If either fails, fix before proceeding.
567618
- [ ] `CLAUDE.md` follows existing applet patterns
568619
- [ ] `package.json` has all required dependencies (Next.js, Tailwind v4, ui-kit)
569620
- [ ] `globals.css` has `@import "tailwindcss"` + `@import "@policyengine/ui-kit/theme.css"`
570-
- [ ] No `tailwind.config.ts` or `postcss.config.js` (Tailwind v4)
621+
- [ ] `postcss.config.mjs` exists with `@tailwindcss/postcss` plugin
622+
- [ ] No `tailwind.config.ts` (Tailwind v4)
571623
- [ ] No CDN `<link>` for design-system tokens (ui-kit theme provides everything)
572624
- [ ] Inter font is loaded via `next/font/google`
573625
- [ ] Embedding boilerplate is in place
@@ -580,7 +632,7 @@ If either fails, fix before proceeding.
580632
- [ ] `layout.tsx` metadata includes `icons: { icon: '/favicon.svg' }`
581633
- [ ] Header uses `logos.whiteWordmark` or `logos.tealWordmark` (not text-only)
582634
- [ ] `Makefile` has correct targets for the data pattern
583-
- [ ] `make dev` uses a random port (does not hardcode 3000)
635+
- [ ] `make dev` uses port range 4000-4100 (not random, not hardcoded 3000)
584636
- [ ] If custom-modal: `make dev` deploys worker, then starts gateway + frontend
585637
- [ ] If custom-modal: backend has 3-file structure (`_image_setup.py`, `app.py`, `simulation.py`)
586638
- [ ] If custom-modal: `_image_setup.py` has no package imports at module level
@@ -603,7 +655,8 @@ If either fails, fix before proceeding.
603655
- Deploy to Vercel or Modal (that's `/deploy-dashboard`)
604656
- Implement real logic (that's Phase 3 agents)
605657
- Skip the feature branch
606-
- Create `tailwind.config.ts` or `postcss.config.js` (Tailwind v4 uses `@theme` in CSS)
658+
- Create `tailwind.config.ts` (Tailwind v4 uses `@theme` in CSS)
659+
- Omit `postcss.config.mjs` — it IS required for Tailwind v4 (the `@tailwindcss/postcss` plugin processes `@import "tailwindcss"`)
607660
- Rebuild components that exist in `@policyengine/ui-kit`
608661
- Load tokens via CDN `<link>` (use `@import "@policyengine/ui-kit/theme.css"` instead)
609662
- Use `getCssVar()` — it no longer exists. SVG accepts `var()` directly.

0 commit comments

Comments
 (0)