Skip to content

Commit be2c54d

Browse files
authored
Merge pull request #1513 from kodustech/perf/heavy-screens-refactor
refactor(perf): heavy screens + analytics
2 parents a3dc58a + e420c49 commit be2c54d

154 files changed

Lines changed: 21271 additions & 13428 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/skills/perf-debug/SKILL.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
name: perf-debug
3+
description: End-to-end front→back performance debugging for the Kodus web app. Use when a screen is slow, blank, looping, or you need to trace a UI perf problem down to the API query. Drives Playwright/Chrome MCP to open + measure the screen, then reads the API use-case/repository and the DB (Postgres/Mongo indexes, explain) to find and fix the root cause, verifying live.
4+
---
5+
6+
# Perf debug (front → back)
7+
8+
Repeatable loop for resolving Kodus web performance problems, from the rendered
9+
screen down to the DB query. Built from real sessions on the token-usage,
10+
pull-requests, settings and cockpit screens.
11+
12+
## The loop
13+
14+
1. **Open the screen** — Playwright (`mcp__plugin_playwright_playwright__*`) or
15+
Chrome MCP. App is auth-gated; `/` is 404 post-Next16, log in at `/sign-in`
16+
(email → Continue → password). Test user: `Novus@teste.com` (org
17+
`aae1c003-…` has real data). Reset its password via Postgres if needed
18+
(bcryptjs hash; see the memory).
19+
20+
2. **Observe — never trust the first load.** Capture `browser_console_messages`
21+
(errors) + `browser_network_requests` (fan-out). **Reload 2× on a settled
22+
dev server**: HMR produces phantom hydration errors / truncated-parse 500s.
23+
Only what reproduces on a clean load is real.
24+
25+
3. **Measure** via the Performance API (`browser_evaluate`):
26+
`nav.responseStart` (TTFB), FCP, LCP, load. Classify the bottleneck:
27+
- high TTFB → **SSR-data-bound** (server component awaiting slow data)
28+
- big JS / slow FCP after TTFB → **bundle-bound**
29+
- janky interaction → **render-bound** (note: React Compiler is ON, so
30+
manual memo is rarely the fix).
31+
32+
4. **Understand the components** — read the page.tsx / client component. Find the
33+
server/client split and the loading states. Watch for `if (!isMounted) return
34+
null` (blanks the body pre-hydration → show a skeleton instead) and effects
35+
that `router.replace` with `searchParams` in deps (reload loops).
36+
37+
5. **Trace to the API** — from the fan-out, pick the heavy/redundant endpoints
38+
(duplicates, per-page `/api/auth/session`, `/executions`, cockpit fan-out).
39+
40+
6. **Understand the query** — read `apps/api/src/controllers/*`
41+
`libs/**/use-cases` → repository. Then hit the DB directly:
42+
- Postgres: `docker exec kodus_api printenv API_PG_DB_PASSWORD`, then
43+
`docker exec -e PGPASSWORD=… db_postgres psql -U kodusdev -d kodus_db`.
44+
Check `pg_indexes` for the table; look for OFFSET-in-loop, uncached
45+
`COUNT(*)`, N+1 (`relations:[...]` on a to-one is a JOIN, not N+1).
46+
- Mongo: `docker exec mongodb mongosh "mongodb://kodusdev:<pass>@localhost:27017/<db>?authSource=admin"`.
47+
`db.<coll>.getIndexes()` (watch for **partial indexes** matching the
48+
filter), `.explain("executionStats")` — compare `totalKeysExamined` vs
49+
`nReturned`. See the `mongodb-query-optimizer` skill for deeper analysis.
50+
- Kodus DB is generally **well-indexed**; the real backend wins are usually
51+
algorithmic (keyset vs OFFSET) or caching — and only reproduce at prod
52+
scale (dev DB is tiny), so prepare the patch and validate in staging.
53+
54+
7. **Fix surgically + verify live** — one change, then re-measure + re-check
55+
console on a clean reload. Confirm no new hydration errors, no 500.
56+
57+
8. **Isolate when unsure — revert to compare.** If a change might be the cause,
58+
`git checkout -- <file>` (back it up first) and reload: if the symptom
59+
persists with the original, your change is exonerated. (Used to prove the
60+
token-usage blank was a pre-existing reload loop, not the recharts port.)
61+
62+
## Dev-env gotchas (these cost hours if unknown)
63+
64+
- **HMR noise** → hydration/parse errors that vanish on a 2nd clean reload.
65+
- **Turbopack truncated-parse cache** (`Unexpected eof` on a valid file) →
66+
`touch` the file to force a fresh re-read. Root cause: `CHOKIDAR_USEPOLLING=false`
67+
+ `:delegated` bind-mount catches partial writes.
68+
- **OOM** → heavy routes (charts + turbopack) blow the web container's memory
69+
limit. Check `docker stats` / `docker inspect --format '{{.State.OOMKilled}}'`;
70+
bump `deploy.resources.limits.memory` (web needed 4G, 2G OOM'd).
71+
- **Reload loop** → a page hammering its own document (`GET /x` ×1000s, CPU
72+
pegged, blank body). Diagnose via request count in `docker logs`. Cause is
73+
usually an effect that navigates with the value it depends on in its deps.
74+
- **Cache masks slow loads** — the token-usage `$facet` is ~7s cold / ~400ms
75+
cached; bust the cache (uncached filter combo) to observe the real load.
76+
- **Healthcheck false-unhealthy**`kodus_web` pings `/` (404 post-Next16) so
77+
it shows "unhealthy" while working.
78+
79+
## Reference
80+
See memory `reference_perf_debug_flow` for the condensed version and
81+
`project_frontend_perf_next16` for the concrete fixes shipped with this flow.

.dockerignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Dependencies & build output
22
node_modules
33
**/node_modules
4+
.pnpm-store
5+
**/.pnpm-store
46
dist
57
**/dist
68
**/.next

.github/workflows/cli-ci.yml

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,14 +23,15 @@ jobs:
2323
working-directory: apps/cli
2424
steps:
2525
- uses: actions/checkout@v6.0.2
26+
- uses: pnpm/action-setup@v4
2627
- uses: actions/setup-node@v6.4.0
2728
with:
2829
node-version: "22"
29-
cache: yarn
30-
cache-dependency-path: apps/cli/yarn.lock
31-
- run: yarn install --frozen-lockfile
32-
- run: yarn skills:validate
33-
- run: yarn skills:prompt > /dev/null
34-
- run: yarn build
35-
- run: yarn test
36-
- run: yarn vitest run --config vitest.integration.config.ts
30+
cache: pnpm
31+
cache-dependency-path: apps/cli/pnpm-lock.yaml
32+
- run: pnpm install --frozen-lockfile
33+
- run: pnpm skills:validate
34+
- run: pnpm skills:prompt > /dev/null
35+
- run: pnpm build
36+
- run: pnpm test
37+
- run: pnpm vitest run --config vitest.integration.config.ts

.github/workflows/cli-release.yml

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -37,17 +37,18 @@ jobs:
3737
# so the build, test, publish, and release all match
3838
# the artifact we're cutting.
3939
ref: ${{ inputs.tag || github.ref }}
40+
- uses: pnpm/action-setup@v4
4041
- uses: actions/setup-node@v6.4.0
4142
with:
4243
node-version: "24"
43-
cache: yarn
44-
cache-dependency-path: apps/cli/yarn.lock
44+
cache: pnpm
45+
cache-dependency-path: apps/cli/pnpm-lock.yaml
4546
- run: npm --version
46-
- run: yarn install --frozen-lockfile
47-
- run: yarn skills:validate
48-
- run: yarn skills:prompt > /dev/null
49-
- run: yarn build
50-
- run: yarn test
47+
- run: pnpm install --frozen-lockfile
48+
- run: pnpm skills:validate
49+
- run: pnpm skills:prompt > /dev/null
50+
- run: pnpm build
51+
- run: pnpm test
5152
- run: npm publish --provenance --access public
5253

5354
- name: Create GitHub Release

.github/workflows/tests.yml

Lines changed: 10 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -111,25 +111,17 @@ jobs:
111111
- name: Install dependencies
112112
run: pnpm install --frozen-lockfile
113113

114-
- name: Cache yarn store
115-
uses: actions/cache@v5.0.5
116-
with:
117-
path: ~/.cache/yarn
118-
key: ${{ runner.os }}-node22-yarn-${{ hashFiles('apps/web/yarn.lock') }}
119-
restore-keys: |
120-
${{ runner.os }}-node22-yarn-
121-
122114
- name: Install apps/web dependencies
123-
# apps/web is NOT a pnpm workspace member (pnpm-workspace.yaml
124-
# has no `packages:` entry for it) — it's a separate,
125-
# yarn-managed project (see the root `web:install` script).
126-
# The root pnpm install above never touches its
127-
# node_modules, so any spec that imports a real apps/web
128-
# runtime dependency (e.g. @radix-ui/*, class-variance-authority)
129-
# fails with "Cannot find module" unless this also runs.
130-
# This job only started needing it once component specs
131-
# under apps/web/src began rendering real UI components.
132-
run: cd apps/web && yarn install --frozen-lockfile
115+
# apps/web is an ISOLATED pnpm project — its own
116+
# apps/web/pnpm-workspace.yaml makes it a separate root, not a
117+
# member of the backend workspace, so the root `pnpm install`
118+
# above never touches its node_modules. Any spec that imports a
119+
# real apps/web runtime dependency (e.g. @radix-ui/*,
120+
# class-variance-authority) fails with "Cannot find module"
121+
# unless this dedicated install also runs. The pnpm store cache
122+
# above already covers it (hashFiles('**/pnpm-lock.yaml') matches
123+
# apps/web/pnpm-lock.yaml and the store is shared).
124+
run: cd apps/web && pnpm install --frozen-lockfile
133125

134126
- name: Create environment.ts for tests
135127
run: |

.gitignore

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,12 @@ apps/*/dist
55

66
# pnpm's content-addressable store — should live in the global store
77
# (~/Library/pnpm/store). A local copy here is an orphan from a stray
8-
# `pnpm install --store-dir .pnpm-store`; never commit it.
8+
# `pnpm install --store-dir .pnpm-store`; never commit it. The nested pattern
9+
# covers stores leaked into workspace packages (e.g. apps/web/.pnpm-store via
10+
# the dev container bind-mount).
911
/.pnpm-store/
12+
**/.pnpm-store/
13+
apps/web/.pnpm-store/
1014

1115
# selfhosted dev VM state (scripts/selfhosted/*.sh)
1216
.kodus-dev/
@@ -16,6 +20,8 @@ apps/*/dist
1620
# TypeScript compiled files (should NOT be in src/ or packages/)
1721
# These should only exist in dist/ directories
1822
*.d.ts
23+
# ...except hand-written type augmentations, which are source, not build output.
24+
!apps/web/src/types/*.d.ts
1925
*.js.map
2026

2127
# Ignore compiled .js files in src/ and packages/ (they should be in dist/)
@@ -244,3 +250,4 @@ scripts/benchmark/execucao-detalhada.py
244250
scripts/benchmark/golden-context.py
245251
scripts/benchmark/golden-context.py.bak
246252
evals/kody-rules/prod-rules.json
253+
.grok/settings.json

apps/api/src/controllers/__tests__/rbac-matrix.manifest.json

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1319,6 +1319,17 @@
13191319
"contributor": "allow"
13201320
}
13211321
},
1322+
{
1323+
"key": "pullRequest.controller.ts#getAwaitingPullRequests",
1324+
"httpMethod": "GET",
1325+
"urlPath": "/pull-requests/awaiting",
1326+
"expected": {
1327+
"owner": "allow",
1328+
"billing_manager": "deny",
1329+
"repo_admin": "allow",
1330+
"contributor": "allow"
1331+
}
1332+
},
13221333
{
13231334
"key": "pullRequest.controller.ts#getOnboardingSignals",
13241335
"httpMethod": "GET",
@@ -1330,6 +1341,17 @@
13301341
"contributor": "allow"
13311342
}
13321343
},
1344+
{
1345+
"key": "pullRequest.controller.ts#getPullRequestAuthors",
1346+
"httpMethod": "GET",
1347+
"urlPath": "/pull-requests/authors",
1348+
"expected": {
1349+
"owner": "allow",
1350+
"billing_manager": "deny",
1351+
"repo_admin": "allow",
1352+
"contributor": "allow"
1353+
}
1354+
},
13331355
{
13341356
"key": "pullRequest.controller.ts#getPullRequestExecutions",
13351357
"httpMethod": "GET",
@@ -1352,6 +1374,28 @@
13521374
"contributor": "allow"
13531375
}
13541376
},
1377+
{
1378+
"key": "pullRequest.controller.ts#getPullRequestsDailyDigest",
1379+
"httpMethod": "GET",
1380+
"urlPath": "/pull-requests/executions/summary",
1381+
"expected": {
1382+
"owner": "allow",
1383+
"billing_manager": "deny",
1384+
"repo_admin": "allow",
1385+
"contributor": "allow"
1386+
}
1387+
},
1388+
{
1389+
"key": "pullRequest.controller.ts#getPullRequestsFacets",
1390+
"httpMethod": "GET",
1391+
"urlPath": "/pull-requests/executions/facets",
1392+
"expected": {
1393+
"owner": "allow",
1394+
"billing_manager": "deny",
1395+
"repo_admin": "allow",
1396+
"contributor": "allow"
1397+
}
1398+
},
13551399
{
13561400
"key": "pullRequestMessages.controller.ts#createOrUpdatePullRequestMessages",
13571401
"httpMethod": "POST",

0 commit comments

Comments
 (0)