Skip to content

Commit 23f1b32

Browse files
chore: convert Layne to Typescript (#35)
* chore: convert Layne to Typescript * Fix lint and other config
1 parent 8a4126f commit 23f1b32

File tree

302 files changed

+11251
-1435
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

302 files changed

+11251
-1435
lines changed
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"layne": minor
3+
---
4+
5+
Rewrites the codebase from JavaScript to TypeScript for improved type safety and developer experience. No behavioral changes; deployment, configuration schema, and all external interfaces are identical.

CLAUDE.md

Lines changed: 39 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -10,26 +10,30 @@ Layne is a self-hosted GitHub App that centralises security scanning across repo
1010

1111
```bash
1212
# Run the webhook server
13-
npm start # node src/server.js
13+
npm start # node dist/server.js
1414

1515
# Run the job worker
16-
npm run worker # node src/worker.js
16+
npm run worker # node dist/worker.js
17+
18+
# Build TypeScript
19+
npm run build # tsc -p tsconfig.build.json
20+
npm run typecheck # tsc --noEmit
1721

1822
# Tests
1923
npm test # vitest run (single pass)
2024
npm run test:watch # vitest (watch mode)
2125
npm run test:coverage
2226

2327
# Run a single test file
24-
npx vitest run src/__tests__/github.test.js
28+
npx vitest run src/__tests__/github.test.ts
2529

2630
# Lint
2731
npm run lint
2832
```
2933

3034
## Environment variables
3135

32-
Required (checked at startup by `validateEnv()` in `src/env.js`):
36+
Required (checked at startup by `validateEnv()` in `src/env.ts`):
3337
```
3438
GITHUB_APP_ID
3539
GITHUB_APP_PRIVATE_KEY # single-line PEM with literal \n between lines
@@ -51,7 +55,7 @@ DEBUG_MODE # set to "true" for verbose logging
5155

5256
Two separate Node.js processes:
5357

54-
**`src/server.js` - Webhook receiver**
58+
**`src/server.ts` - Webhook receiver**
5559
- Express app with `POST /webhook`, `GET /health`, `GET /metrics` (when enabled), `GET /assets/layne-logo.png`
5660
- Verifies GitHub HMAC signature before processing
5761
- Handles four event types: `pull_request`, `workflow_run`, `workflow_job`, and `issue_comment`
@@ -62,7 +66,7 @@ Two separate Node.js processes:
6266
- Job ID is deduplicated by `{repo}#{pr}@{sha}` - duplicate webhook deliveries are no-ops (Redis lock + queue check)
6367
- Exported `app` and `processWebhookRequest` for use in tests
6468

65-
**`src/worker.js` - Job processor**
69+
**`src/worker.ts` - Job processor**
6670
- BullMQ `Worker` consuming the `scans` queue with concurrency 5
6771
- `processJob()` is exported for direct testing without Redis
6872
- Per-job 10-minute timeout via `Promise.race`
@@ -71,29 +75,29 @@ Two separate Node.js processes:
7175

7276
**Job lifecycle (inside `runScan`):**
7377
1. Mark Check Run `in_progress`
74-
2. Authenticate as installation via `src/auth.js` → short-lived token
75-
3. Resolve merge base SHA via `src/github.js``getMergeBaseSha` (three-dot diff base)
76-
4. Create temp workspace (`src/fetcher.js``createWorkspace`)
77-
5. Partial-clone head and merge-base SHAs with `--filter=blob:none` (`src/fetcher.js``setupRepo`)
78+
2. Authenticate as installation via `src/auth.ts` → short-lived token
79+
3. Resolve merge base SHA via `src/github.ts``getMergeBaseSha` (three-dot diff base)
80+
4. Create temp workspace (`src/fetcher.ts``createWorkspace`)
81+
5. Partial-clone head and merge-base SHAs with `--filter=blob:none` (`src/fetcher.ts``setupRepo`)
7882
6. Diff the two commits to get changed file paths (`getChangedFiles`) and per-file changed line ranges (`getChangedLineRanges`)
7983
7. Sparse-checkout only the changed files - blobs fetched on demand (`checkoutFiles`)
80-
8. Load per-repo config via `src/config.js``loadScanConfig`
81-
9. Run scanners in parallel via `src/dispatcher.js``dispatch()`
82-
10. Validate finding locations against the actual file content (`src/location-validator.js``validateFindingLocations`)
83-
11. Suppress findings that have a `// SECURITY:` comment at the merge base (`src/suppressor.js``suppressFindings`)
84-
12. Filter to actionable findings; stamp each with a deterministic `_findingId` (`LAYNE-xxxxxxxxxxxxxxxx`) via `src/exception-approvals.js``generateFindingId`
85-
13. Convert findings to annotations via `src/reporter.js``buildAnnotations()`
84+
8. Load per-repo config via `src/config.ts``loadScanConfig`
85+
9. Run scanners in parallel via `src/dispatcher.ts``dispatch()`
86+
10. Validate finding locations against the actual file content (`src/location-validator.ts``validateFindingLocations`)
87+
11. Suppress findings that have a `// SECURITY:` comment at the merge base (`src/suppressor.ts``suppressFindings`)
88+
12. Filter to actionable findings; stamp each with a deterministic `_findingId` (`LAYNE-xxxxxxxxxxxxxxxx`) via `src/exception-approvals.ts``generateFindingId`
89+
13. Convert findings to annotations via `src/reporter.ts``buildAnnotations()`
8690
14. If `exceptionApprovers` is configured: load stored exceptions from Redis (`loadExceptions`), remove stale ones whose flagged line changed (`filterStaleExceptions`), resolve approvals that survived a line-number shift via rebase (`resolveDriftedExceptions`), then call `buildExceptionSummary` to potentially override conclusion to `success`
8791
15. Complete Check Run
88-
16. Post PR comment if `comment.enabled` via `src/commenter.js``postComment`
89-
17. Apply/remove PR labels via `src/github.js``ensureLabelsExist` + `setLabels`
90-
18. Notify via `src/notifiers/index.js``notify()` (always fires on exception approval; otherwise only when finding count increases)
92+
16. Post PR comment if `comment.enabled` via `src/commenter.ts``postComment`
93+
17. Apply/remove PR labels via `src/github.ts``ensureLabelsExist` + `setLabels`
94+
18. Notify via `src/notifiers/index.ts``notify()` (always fires on exception approval; otherwise only when finding count increases)
9195
19. Clean up workspace in `finally`
9296

9397
**Scanners (`src/adapters/`):**
94-
- `semgrep.js` - runs `semgrep scan --config auto --json`; exit code 1 = findings found (not an error); maps ERROR→high, WARNING→medium, INFO→low
95-
- `trufflehog.js` - runs `trufflehog filesystem --json --no-update`; exit code 183 = secrets found (not an error); batched at 200 files to stay under ARG_MAX; all findings are severity `high`
96-
- `claude.js` - calls the Anthropic API to detect malicious intent; **disabled by default**, opt in per repo; skips binary files; caps files at 50 KB; batches at 100 KB per API call; errors are caught and logged without failing the scan. Supports two modes (configured per-repo in `config/layne.json`):
98+
- `semgrep.ts` - runs `semgrep scan --config auto --json`; exit code 1 = findings found (not an error); maps ERROR→high, WARNING→medium, INFO→low
99+
- `trufflehog.ts` - runs `trufflehog filesystem --json --no-update`; exit code 183 = secrets found (not an error); batched at 200 files to stay under ARG_MAX; all findings are severity `high`
100+
- `claude.ts` - calls the Anthropic API to detect malicious intent; **disabled by default**, opt in per repo; skips binary files; caps files at 50 KB; batches at 100 KB per API call; errors are caught and logged without failing the scan. Supports two modes (configured per-repo in `config/layne.json`):
97101
- **Prompt mode** (default): single `messages.create` call with a system prompt; use `claude.prompt` to override
98102
- **Skill mode**: uses the Anthropic [API Skills beta](https://platform.claude.com/docs/en/build-with-claude/skills-guide) - adds a `code_execution` tool + an uploaded skill to each batch call, enabling runtime decoding, registry lookups, and richer static analysis; set `claude.skill: { id, version }` to enable; handles `pause_turn` continuations automatically (up to 10 turns per batch)
99103

@@ -113,21 +117,21 @@ Two separate Node.js processes:
113117

114118
| Module | Purpose |
115119
|--------|---------|
116-
| `src/config.js` | Loads and merges `config/layne.json`; cached after first read |
117-
| `src/github.js` | Check Run CRUD + label management (`ensureLabelsExist`, `setLabels`) |
118-
| `src/metrics.js` | Prometheus metric definitions; exports no-op stubs when `METRICS_ENABLED` is not `true` |
119-
| `src/notifiers/index.js` | Notification orchestrator; iterates registered notifiers |
120-
| `src/notifiers/rocketchat.js` | Rocket.Chat incoming webhook notifier |
121-
| `src/queue.js` | Shared Redis + BullMQ queue instance |
122-
| `src/debug.js` | Conditional debug logging via `DEBUG_MODE` |
120+
| `src/config.ts` | Loads and merges `config/layne.json`; cached after first read |
121+
| `src/github.ts` | Check Run CRUD + label management (`ensureLabelsExist`, `setLabels`) |
122+
| `src/metrics.ts` | Prometheus metric definitions; exports no-op stubs when `METRICS_ENABLED` is not `true` |
123+
| `src/notifiers/index.ts` | Notification orchestrator; iterates registered notifiers |
124+
| `src/notifiers/rocketchat.ts` | Rocket.Chat incoming webhook notifier |
125+
| `src/queue.ts` | Shared Redis + BullMQ queue instance |
126+
| `src/debug.ts` | Conditional debug logging via `DEBUG_MODE` |
123127

124128
## Per-repo configuration (`config/layne.json`)
125129

126130
See [docs/2-configuration.md](docs/2-configuration.md) for the full schema and examples.
127131

128132
Key points for code navigation:
129133
- Read once per process startup - **restart both server and worker to pick up changes**
130-
- Loaded and merged by `src/config.js``loadScanConfig`
134+
- Loaded and merged by `src/config.ts``loadScanConfig`
131135
- Supports `$global` key for defaults inherited by all repos
132136
- Scanner blocks: per-repo spread over defaults (`{ ...DEFAULT_CONFIG.semgrep, ...repoOverrides.semgrep }`)
133137
- `trigger`: controls when scanning fires - `pull_request` (default, immediate) or `workflow_run` (deferred until a named CI workflow completes); global default → per-repo override
@@ -141,7 +145,7 @@ Key points for code navigation:
141145

142146
## Metrics
143147

144-
- `src/metrics.js` exports real prom-client objects when `METRICS_ENABLED=true`, silent no-op stubs otherwise
148+
- `src/metrics.ts` exports real prom-client objects when `METRICS_ENABLED=true`, silent no-op stubs otherwise
145149
- No `if (METRICS_ENABLED)` guards needed at call sites - stubs absorb all calls
146150
- Worker: metrics HTTP server + BullMQ queue poller (15 s interval) started only when enabled
147151
- Server: `GET /metrics` route registered only when enabled
@@ -150,9 +154,11 @@ Key points for code navigation:
150154
## Testing conventions
151155

152156
- Tests use Vitest with ESM (`"type": "module"` in package.json)
153-
- `src/__tests__/setup.js` sets all required env vars before each test file; `ANTHROPIC_API_KEY` and `METRICS_ENABLED` are intentionally not set - adapters and metrics are mocked
157+
- All test files are TypeScript (`.ts`); imports use `.js` extensions (NodeNext resolution)
158+
- `src/__tests__/setup.ts` sets all required env vars before each test file; `ANTHROPIC_API_KEY` and `METRICS_ENABLED` are intentionally not set - adapters and metrics are mocked
154159
- External dependencies (`@octokit/auth-app`, `@octokit/rest`, `bullmq`, `ioredis`, `@anthropic-ai/sdk`, `prom-client`) are always mocked - no live connections in tests
155-
- `src/metrics.js` is mocked in worker and server tests with `vi.fn()` stubs; tested in isolation in `src/__tests__/metrics.test.js`
160+
- `src/metrics.ts` is mocked in worker and server tests with `vi.fn()` stubs; tested in isolation in `src/__tests__/metrics.test.ts`
156161
- `processJob` and `dispatch` are exported specifically for unit testing without live infrastructure
157162
- Tests import modules with `await import(...)` after `vi.mock()` calls to handle ESM module caching
158163
- The `@anthropic-ai/sdk` mock uses a regular `function` constructor (not an arrow function) because `new Anthropic()` must be constructable
164+
- Typed mock call access pattern: `(mockFn as ReturnType<typeof vi.fn>).mock.calls[0] as [T1, T2]`

Dockerfile

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,16 @@ WORKDIR /app
55
COPY package*.json ./
66
RUN npm ci --omit=dev
77

8+
FROM node:22-alpine AS build
9+
10+
WORKDIR /app
11+
12+
COPY package*.json ./
13+
RUN npm ci
14+
COPY tsconfig*.json ./
15+
COPY src/ ./src/
16+
RUN npm run build
17+
818
FROM node:22-alpine AS runtime
919

1020
# Pin tool versions for reproducible builds. Update periodically and verify in staging.
@@ -25,8 +35,8 @@ RUN addgroup -S layne && adduser -S layne -G layne
2535
WORKDIR /app
2636

2737
COPY --from=deps /app/node_modules ./node_modules
38+
COPY --from=build /app/dist ./dist
2839
COPY package*.json ./
29-
COPY src/ ./src/
3040
COPY config/ ./config/
3141
COPY assets/ ./assets/
3242

@@ -36,4 +46,4 @@ VOLUME ["/tmp"]
3646

3747
USER layne
3848

39-
CMD ["node", "src/server.js"]
49+
CMD ["node", "dist/server.js"]
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/usr/bin/env node
2+
/**
3+
* Replay a webhook fixture against your local Layne server.
4+
*
5+
* Usage:
6+
* node scripts/replay-webhook.js [fixture-path] [server-url]
7+
*
8+
* Defaults:
9+
* fixture-path fixtures/webhooks/pr_opened.json
10+
* server-url http://localhost:3000
11+
*
12+
* Examples:
13+
* npm run replay
14+
* npm run replay fixtures/webhooks/pr_synchronize.json
15+
* npm run replay fixtures/webhooks/pr_opened.json http://localhost:3001
16+
*/
17+
import 'dotenv/config';
18+
//# sourceMappingURL=replay-webhook.d.ts.map

dist-scripts/scripts/replay-webhook.d.ts.map

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

dist-scripts/scripts/replay-webhook.js

Lines changed: 70 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

dist-scripts/scripts/replay-webhook.js.map

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/usr/bin/env node
2+
/**
3+
* Validates config/layne.json and exits with code 1 if there are errors.
4+
* Run via: npm run validate-config
5+
*/
6+
export {};
7+
//# sourceMappingURL=validate-config.d.ts.map

dist-scripts/scripts/validate-config.d.ts.map

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

dist-scripts/scripts/validate-config.js

Lines changed: 40 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)