Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/rich-socks-say.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"layne": minor
---

Adds a new trigger for Layne: workflow_run
13 changes: 8 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ Two separate Node.js processes:
**`src/server.js` — Webhook receiver**
- Express app with `POST /webhook`, `GET /health`, `GET /metrics` (when enabled), `GET /assets/layne-logo.png`
- Verifies GitHub HMAC signature before processing
- On a qualifying `pull_request` event (opened/synchronize/reopened), creates a GitHub Check Run in `queued` state, then enqueues a BullMQ job and returns 200
- Handles two event types: `pull_request` and `workflow_run`
- **`pull_request` trigger (default):** on opened/synchronize/reopened, creates a Check Run in `queued` state, enqueues a BullMQ job, returns 200
- **`workflow_run` trigger:** on `pull_request` events, caches PR metadata in Redis (TTL 7 days) and creates a `skipped` Check Run; on `workflow_run completed` events matching the configured workflow name and conclusion, looks up cached PR metadata (falls back to GitHub API if cache is cold) then enqueues the scan
- Job ID is deduplicated by `{repo}#{pr}@{sha}` — duplicate webhook deliveries are no-ops (Redis lock + queue check)
- Exported `app` and `processWebhookRequest` for use in tests

Expand Down Expand Up @@ -103,26 +105,27 @@ Two separate Node.js processes:

| Module | Purpose |
|--------|---------|
| `src/config.js` | Loads and merges `config/repos.json`; cached after first read |
| `src/config.js` | Loads and merges `config/layne.json`; cached after first read |
| `src/github.js` | Check Run CRUD + label management (`ensureLabelsExist`, `setLabels`) |
| `src/metrics.js` | Prometheus metric definitions; exports no-op stubs when `METRICS_ENABLED` is not `true` |
| `src/notifiers/index.js` | Notification orchestrator; iterates registered notifiers |
| `src/notifiers/rocketchat.js` | Rocket.Chat incoming webhook notifier |
| `src/queue.js` | Shared Redis + BullMQ queue instance |
| `src/debug.js` | Conditional debug logging via `DEBUG_MODE` |

## Per-repo configuration (`config/repos.json`)
## Per-repo configuration (`config/layne.json`)

See [docs/configuration.md](docs/configuration.md) for the full schema and examples.

Key points for code navigation:
- Read once at worker startup — **restart the worker to pick up changes**
- Read once per process startup — **restart both server and worker to pick up changes**
- Loaded and merged by `src/config.js` → `loadScanConfig`
- Supports `$global` key for defaults inherited by all repos
- Scanner blocks: per-repo spread over defaults (`{ ...DEFAULT_CONFIG.semgrep, ...repoOverrides.semgrep }`)
- `trigger`: controls when scanning fires — `pull_request` (default, immediate) or `workflow_run` (deferred until a named CI workflow completes); global default → per-repo override
- `notifications` and `labels`: per-repo notifier/key wins over global; per-repo absence = inherit global entirely
- `extraArgs` fully replaces the default (not extended)
- `config/repos.json` must be present in the Docker image (`COPY config/ ./config/`)
- `config/layne.json` must be present in the Docker image (`COPY config/ ./config/`)
- Notifier contract: `async function notify({ findings, owner, repo, prNumber, toolConfig })` — must never throw
- Notification dedup key: `layne:scan:count:{owner}/{repo}#{prNumber}` (Redis, 30-day TTL)
- `webhookUrl` values starting with `$` are resolved from `process.env` at runtime
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Closes #42
npm test # run the full test suite
npm run test:watch # watch mode during development
npm run lint # ESLint
npm run validate-config # validate config/repos.json schema
npm run validate-config # validate config/layne.json schema
```

All four must pass before a PR can be merged.
Expand Down
File renamed without changes.
91 changes: 88 additions & 3 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configuration

Scanner behaviour, labels, and notifications are all configured in `config/repos.json`. Layne reads this file once at worker startup — **restart the worker to pick up changes**.
Scanner behaviour, labels, notifications, and trigger conditions are all configured in `config/layne.json`. Layne reads this file once per process startup — **restart both server and worker to pick up changes** (the automated deploy pipeline does this automatically).

---

Expand Down Expand Up @@ -119,7 +119,7 @@ skill = Anthropic().beta.skills.create(
files=files_from_dir("/path/to/skill-folder"), # must contain SKILL.md
betas=["skills-2025-10-02"],
)
print(skill.id) # paste this into repos.json
print(skill.id) # paste this into layne.json
```

### How args are assembled
Expand Down Expand Up @@ -236,7 +236,7 @@ Labels are applied **after** the Check Run is completed. Errors never affect the

### Configuration

Add a `labels` key to `$global` or to any repo entry in `config/repos.json`:
Add a `labels` key to `$global` or to any repo entry in `config/layne.json`:

```json
{
Expand Down Expand Up @@ -289,6 +289,91 @@ If neither `$global` nor the repo defines a `labels` key, the feature is a no-op

---

## Trigger

By default Layne scans every pull request immediately when it is opened, synchronised, or reopened. The `trigger` block lets you defer scanning until a specific CI workflow completes — useful for open-source repositories where workflows from external contributors require maintainer approval before they run.

### Modes

| `on` | Behaviour |
|---|---|
| `pull_request` | *(default)* Scan fires immediately on `opened`, `synchronize`, and `reopened` events |
| `workflow_run` | Scan fires when the named CI workflow completes with a matching conclusion |

### Schema

```json
{
"owner/repo": {
"trigger": {
"on": "workflow_run",
"workflow": "Tests Done",
"conclusions": ["success"]
}
}
}
```

| Key | Type | Default | Description |
|---|---|---|---|
| `on` | `"pull_request"` \| `"workflow_run"` | `"pull_request"` | When to trigger the scan |
| `workflow` | string | — | Name of the GitHub Actions workflow to watch. Required when `on` is `"workflow_run"` |
| `conclusions` | string[] | `["success"]` | Workflow conclusions that trigger the scan. Valid values: `success`, `failure`, `neutral`, `cancelled`, `skipped`, `timed_out`, `action_required` |

> **`trigger` can be set globally.** Set it under `$global` to apply to all repos, then override per-repo as needed.

### How `workflow_run` works

When `on: workflow_run` is configured for a repo:

1. **On `pull_request`** — Layne caches the PR metadata in Redis (7-day TTL) and creates a `skipped` Check Run so the deferral is visible in the PR status UI. No scan is enqueued yet.
2. **On `workflow_run completed`** — When the named workflow finishes with a matching conclusion, Layne looks up the cached PR metadata and enqueues the scan. If the cache is cold (e.g. Layne was offline when the PR was opened), Layne falls back to the GitHub API to find the associated PR.

### Failure mode

If the watched workflow is renamed or removed, Layne never receives the `workflow_run` event and the scan never runs. To fail **closed** (safe) rather than **open** (silent), make Layne's Check Run a **required status check** in branch protection — then a missing check blocks merging and the absence is immediately visible.

### Examples

**Scan only after CI passes (mirrors GitHub's "require approval for workflows" gate):**
```json
{
"owner/repo": {
"trigger": {
"on": "workflow_run",
"workflow": "Tests Done"
}
}
}
```

**Scan regardless of whether CI passes or fails (workflow was approved, code is worth scanning):**
```json
{
"owner/repo": {
"trigger": {
"on": "workflow_run",
"workflow": "Tests Done",
"conclusions": ["success", "failure"]
}
}
}
```

**Apply workflow_run trigger to all repos globally:**
```json
{
"$global": {
"trigger": {
"on": "workflow_run",
"workflow": "CI"
}
}
}
```

---

## Notifications

Layne can send a notification to a chat webhook when a scan finds new issues. Notifications fire after the GitHub Check Run is fully posted — engineers see the check result first, then receive the alert.
Expand Down
2 changes: 1 addition & 1 deletion docs/extending.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ const NOTIFIERS = {
};
```

The notifier key (`pagerduty`) is what operators use in `config/repos.json` under `notifications`.
The notifier key (`pagerduty`) is what operators use in `config/layne.json` under `notifications`.

### 3. Write tests

Expand Down
8 changes: 4 additions & 4 deletions docs/local-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,17 +130,17 @@ Everything else in `.env.example` is commented out with sensible defaults — yo

### 3.3 Add your test repo to the scanner config

Open `config/repos.json`. Add an entry for your test repository:
Open `config/layne.json`. Add an entry for your test repository:

```json
{
"your-org/your-repo": {}
}
```

An empty object uses the global defaults: Semgrep and Trufflehog enabled, Claude disabled, plus any global notifications and labels defined in `config/repos.json`. In the checked-in example config, Rocket.Chat and Slack notifications are enabled globally, but local development still works fine if you leave the webhook env vars unset because the notifiers will log and skip delivery.
An empty object uses the global defaults: Semgrep and Trufflehog enabled, Claude disabled, plus any global notifications and labels defined in `config/layne.json`. In the checked-in example config, Rocket.Chat notifications are enabled globally, but local development still works fine if you leave the webhook env vars unset because the notifier will log and skip delivery.

> **Important:** The worker reads `config/repos.json` once at startup. Restart it after any changes.
> **Important:** Both the server and worker read `config/layne.json` once at startup. Restart both after any changes.

---

Expand Down Expand Up @@ -446,4 +446,4 @@ npm run dev:stop
| `[replay] Error: could not reach the server` | Server is not running | `npm start` in another terminal |
| `[replay] response → 401 Invalid signature` | `GITHUB_WEBHOOK_SECRET` in `.env` does not match what the server expects | They must be identical — both processes load the same `.env` |
| Check Run never appears on GitHub | The worker failed to authenticate | Check worker logs for `getInstallationToken` errors; verify `GITHUB_APP_PRIVATE_KEY` is a valid single-line PEM |
| Config changes not picked up by the worker | Worker caches `config/repos.json` at startup | Restart the worker: `Ctrl-C` then `npm run worker` |
| Config changes not picked up | Server and worker each cache `config/layne.json` at startup | Restart both: `Ctrl-C` on each, then `npm start` and `npm run worker` |
3 changes: 1 addition & 2 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@
| `DEBUG_MODE` | No | off | Set to `true` or `1` to enable verbose debug logging |
| `METRICS_ENABLED` | No | `false` | Set to `true` to enable Prometheus metrics endpoints |
| `METRICS_PORT` | No | `9091` | Port for the worker Prometheus metrics server |
| `ROCKETCHAT_WEBHOOK_URL` | No | — | Global Rocket.Chat webhook URL, referenced as `"$ROCKETCHAT_WEBHOOK_URL"` in `config/repos.json`. Add additional vars (e.g. `PAYMENTS_ROCKETCHAT_WEBHOOK_URL`) for per-repo webhooks. |
| `SLACK_WEBHOOK_URL` | No | — | Global Slack incoming webhook URL, referenced as `"$SLACK_WEBHOOK_URL"` in `config/repos.json`. Add additional vars (e.g. `PAYMENTS_SLACK_WEBHOOK_URL`) for per-repo webhooks. |
| `ROCKETCHAT_WEBHOOK_URL` | No | — | Global Rocket.Chat webhook URL, referenced as `"$ROCKETCHAT_WEBHOOK_URL"` in `config/layne.json`. Add additional vars (e.g. `PAYMENTS_ROCKETCHAT_WEBHOOK_URL`) for per-repo webhooks. |

## Finding Shape

Expand Down
14 changes: 7 additions & 7 deletions scripts/validate-config.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env node
/**
* Validates config/repos.json and exits with code 1 if there are errors.
* Validates config/layne.json and exits with code 1 if there are errors.
* Run via: npm run validate-config
*/

Expand All @@ -10,18 +10,18 @@ import { fileURLToPath } from 'url';
import { validateConfig } from '../src/config-validator.js';

const __dirname = dirname(fileURLToPath(import.meta.url));
const configPath = join(__dirname, '..', 'config', 'repos.json');
const configPath = join(__dirname, '..', 'config', 'layne.json');

let raw;
try {
raw = JSON.parse(await readFile(configPath, 'utf8'));
} catch (err) {
if (err.code === 'ENOENT') {
console.error(`✗ config/repos.json not found at ${configPath}`);
console.error(`✗ config/layne.json not found at ${configPath}`);
} else if (err instanceof SyntaxError) {
console.error(`✗ config/repos.json is not valid JSON: ${err.message}`);
console.error(`✗ config/layne.json is not valid JSON: ${err.message}`);
} else {
console.error(`✗ Failed to read config/repos.json: ${err.message}`);
console.error(`✗ Failed to read config/layne.json: ${err.message}`);
}
process.exit(1);
}
Expand All @@ -30,10 +30,10 @@ const result = validateConfig(raw);

if (result.valid) {
const repoCount = Object.keys(raw).filter(k => k !== '$global').length;
console.log(`✓ config/repos.json is valid (${repoCount} repo(s) configured)`);
console.log(`✓ config/layne.json is valid (${repoCount} repo(s) configured)`);
process.exit(0);
} else {
console.error(`✗ config/repos.json has ${result.errors.length} error(s):\n`);
console.error(`✗ config/layne.json has ${result.errors.length} error(s):\n`);
for (const err of result.errors) {
console.error(` • ${err}`);
}
Expand Down
56 changes: 51 additions & 5 deletions src/__tests__/config.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,28 @@ describe('loadScanConfig()', () => {
DEFAULT_CONFIG = configMod.DEFAULT_CONFIG;
});

it('returns defaults when repos.json is missing (readFile throws)', async () => {
it('returns defaults when layne.json is missing (readFile throws)', async () => {
vi.mocked(readFile).mockRejectedValueOnce(new Error('ENOENT'));
const config = await loadScanConfig({ owner: 'org', repo: 'repo' });
expect(config.semgrep).toEqual(DEFAULT_CONFIG.semgrep);
expect(config.trufflehog).toEqual(DEFAULT_CONFIG.trufflehog);
});

it('returns defaults when repos.json contains malformed JSON', async () => {
it('returns defaults when layne.json contains malformed JSON', async () => {
vi.mocked(readFile).mockResolvedValueOnce('not valid json {{{');
const config = await loadScanConfig({ owner: 'org', repo: 'repo' });
expect(config.semgrep).toEqual(DEFAULT_CONFIG.semgrep);
expect(config.trufflehog).toEqual(DEFAULT_CONFIG.trufflehog);
});

it('returns defaults when repos.json top-level value is an array', async () => {
it('returns defaults when layne.json top-level value is an array', async () => {
vi.mocked(readFile).mockResolvedValueOnce(JSON.stringify([{ foo: 'bar' }]));
const config = await loadScanConfig({ owner: 'org', repo: 'repo' });
expect(config.semgrep).toEqual(DEFAULT_CONFIG.semgrep);
expect(config.trufflehog).toEqual(DEFAULT_CONFIG.trufflehog);
});

it('returns defaults when repos.json top-level value is a number', async () => {
it('returns defaults when layne.json top-level value is a number', async () => {
vi.mocked(readFile).mockResolvedValueOnce('42');
const config = await loadScanConfig({ owner: 'org', repo: 'repo' });
expect(config.semgrep).toEqual(DEFAULT_CONFIG.semgrep);
Expand Down Expand Up @@ -100,7 +100,7 @@ describe('loadScanConfig()', () => {
expect(config.trufflehog.extraArgs).toEqual([]);
});

it('reads repos.json only once across two loadScanConfig calls (cache)', async () => {
it('reads layne.json only once across two loadScanConfig calls (cache)', async () => {
vi.mocked(readFile).mockResolvedValue(JSON.stringify({}));
await loadScanConfig({ owner: 'org', repo: 'a' });
await loadScanConfig({ owner: 'org', repo: 'b' });
Expand Down Expand Up @@ -217,4 +217,50 @@ describe('loadScanConfig()', () => {
const config = await loadScanConfig({ owner: 'org', repo: 'repo' });
expect(config.labels).toEqual({});
});

// --- trigger ---

it('returns the default pull_request trigger when no trigger is configured', async () => {
vi.mocked(readFile).mockResolvedValueOnce(JSON.stringify({}));
const config = await loadScanConfig({ owner: 'org', repo: 'repo' });
expect(config.trigger).toEqual({ on: 'pull_request' });
});

it('returns a workflow_run trigger configured at the repo level', async () => {
vi.mocked(readFile).mockResolvedValueOnce(JSON.stringify({
'acme/frontend': {
trigger: { on: 'workflow_run', workflow: 'Tests Done' },
},
}));
const config = await loadScanConfig({ owner: 'acme', repo: 'frontend' });
expect(config.trigger).toEqual({ on: 'workflow_run', workflow: 'Tests Done' });
});

it('inherits $global trigger when the repo has no trigger block', async () => {
vi.mocked(readFile).mockResolvedValueOnce(JSON.stringify({
'$global': { trigger: { on: 'workflow_run', workflow: 'CI' } },
'acme/frontend': { semgrep: { extraArgs: ['--config', 'auto'] } },
}));
const config = await loadScanConfig({ owner: 'acme', repo: 'frontend' });
expect(config.trigger).toEqual({ on: 'workflow_run', workflow: 'CI' });
});

it('repo-level trigger overrides $global trigger', async () => {
vi.mocked(readFile).mockResolvedValueOnce(JSON.stringify({
'$global': { trigger: { on: 'workflow_run', workflow: 'CI' } },
'acme/frontend': { trigger: { on: 'pull_request' } },
}));
const config = await loadScanConfig({ owner: 'acme', repo: 'frontend' });
expect(config.trigger.on).toBe('pull_request');
});

it('preserves custom conclusions in the trigger', async () => {
vi.mocked(readFile).mockResolvedValueOnce(JSON.stringify({
'acme/frontend': {
trigger: { on: 'workflow_run', workflow: 'CI', conclusions: ['success', 'failure'] },
},
}));
const config = await loadScanConfig({ owner: 'acme', repo: 'frontend' });
expect(config.trigger.conclusions).toEqual(['success', 'failure']);
});
});
Loading