Skip to content

Commit ead7761

Browse files
frostebiteclaude
andcommitted
feat: Discord triage system, open source with MIT license
Add interactive Discord-based triage mode for maintainer-controlled help request workflow. When a help request is detected, the bot posts a color-coded embed with action buttons to a private admin channel. Triage flow: Detect → [Investigate] → Review → [Re-investigate] → [Send] New modules: - src/triage/ — types, notification embeds, button handler, investigation wrapper, response sender - docs/dispatch.md — full dispatch and triage documentation - docs/security.md — security architecture documentation Changes: - Add 'triage' dispatch mode to CLI, config, and live gateway - Add triage channel resolution and InteractionCreate handler in live.ts - Add triage state helpers (getTriageRecord, setTriageRecord) - Support threads and forum channels in triage detection - Add View Investigation button for full response review in threads - Add MIT license and open source README with prerequisites, Discord setup guide, and contributing section - Make repo public Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 6bc4fd1 commit ead7761

File tree

21 files changed

+2069
-70
lines changed

21 files changed

+2069
-70
lines changed

README.md

Lines changed: 309 additions & 43 deletions
Large diffs are not rendered by default.

config.json

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@
9898
{
9999
"name": "game-ci-develop",
100100
"guild_id": "1381241944226664498",
101+
"triage_channel_id": "1481216813437026424",
101102
"system_prompt": "You are helping users in the GameCI Develop Discord server. This is a controlled testing environment. Be concise and helpful. Always offer help proactively.",
102103
"channels": [
103104
{
@@ -116,6 +117,13 @@
116117
"read_threads": true,
117118
"monitor": true
118119
},
120+
{
121+
"name": "help-forum-test",
122+
"system_prompt": "This is a forum channel for testing. Respond to all posts.",
123+
"channel_type": "forum",
124+
"reply_mode": "bot_api",
125+
"monitor": true
126+
},
119127
{
120128
"name": "general",
121129
"channel_type": "text",
@@ -126,6 +134,7 @@
126134
]
127135
}
128136
],
137+
"triage_user_ids": ["274202899347800074"],
129138
"sync_hours": 6,
130139
"ignore_bots": true,
131140
"ignore_prefixes": [

dist/cli.js

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ const report_1 = require("./report");
3232
.option('docs-dir', { type: 'string', description: 'Path to local clone of documentation repo (skips HTTP docs sync)' })
3333
.option('investigation-issues', { type: 'boolean', description: 'Create GitHub issues for each investigation in the target repo' })
3434
.option('investigation-repo', { type: 'string', description: 'Target repo for investigation issues (default: game-ci/help-bot)' })
35-
.option('dispatch-mode', { type: 'string', choices: ['auto', 'approval', 'countdown'], description: 'Dispatch mode: auto (immediate), approval (require reaction), countdown (staged warnings)' })
35+
.option('dispatch-mode', { type: 'string', choices: ['auto', 'approval', 'countdown', 'triage'], description: 'Dispatch mode: auto (immediate), approval (require reaction), countdown (staged warnings)' })
3636
.option('countdown-hours', { type: 'number', description: 'Hours between countdown warning stages (default: 24)' })
3737
.option('model', { type: 'string', description: 'Override LLM model (e.g. claude-opus-4-20250514 for deep investigation)' }), async (args) => {
3838
if (!args['github-only']) {
@@ -63,7 +63,7 @@ const report_1 = require("./report");
6363
.option('provider', { type: 'string', description: 'Override LLM provider' })
6464
.option('github-only', { type: 'boolean', description: 'Skip Discord entirely (no token needed)' })
6565
.option('repos', { type: 'string', array: true, description: 'Override repos to sync' })
66-
.option('dispatch-mode', { type: 'string', choices: ['auto', 'approval', 'countdown'], description: 'Dispatch mode: auto (immediate), approval (require reaction), countdown (staged warnings)' })
66+
.option('dispatch-mode', { type: 'string', choices: ['auto', 'approval', 'countdown', 'triage'], description: 'Dispatch mode: auto (immediate), approval (require reaction), countdown (staged warnings)' })
6767
.option('countdown-hours', { type: 'number', description: 'Hours between countdown warning stages (default: 24)' })
6868
.option('investigation-issues', { type: 'boolean', description: 'Create GitHub issues for each investigation' })
6969
.option('investigation-repo', { type: 'string', description: 'Target repo for investigation issues' })
@@ -84,7 +84,7 @@ const report_1 = require("./report");
8484
});
8585
})
8686
.command('live', 'run as a live Discord bot (persistent Gateway connection)', (y) => y
87-
.option('dispatch-mode', { type: 'string', choices: ['auto', 'approval', 'countdown'], description: 'Dispatch mode for incoming messages (default: discord_mode from config)' })
87+
.option('dispatch-mode', { type: 'string', choices: ['auto', 'approval', 'countdown', 'triage'], description: 'Dispatch mode for incoming messages (default: discord_mode from config)' })
8888
.option('repos', { type: 'string', array: true, description: 'GitHub repos for cross-referencing' })
8989
.option('repo-dir', { type: 'string', description: 'Path to local clone of target repo (Claude reads code directly)' })
9090
.option('docs-dir', { type: 'string', description: 'Path to local docs clone' })

dist/state.js

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ exports.setPostedDiscordResponse = setPostedDiscordResponse;
1313
exports.getLastOnlineAt = getLastOnlineAt;
1414
exports.getFirstOnlineAt = getFirstOnlineAt;
1515
exports.setLastOnlineAt = setLastOnlineAt;
16+
exports.getTriageRecords = getTriageRecords;
17+
exports.setTriageRecords = setTriageRecords;
18+
exports.getTriageRecord = getTriageRecord;
19+
exports.setTriageRecord = setTriageRecord;
1620
exports.getGuildCursor = getGuildCursor;
1721
exports.setGuildCursor = setGuildCursor;
1822
const promises_1 = require("node:fs/promises");
@@ -113,6 +117,21 @@ function setLastOnlineAt(state, iso) {
113117
}
114118
state.meta.lastOnlineAt = now;
115119
}
120+
function getTriageRecords(state) {
121+
return state.meta?.triageRecords ?? {};
122+
}
123+
function setTriageRecords(state, records) {
124+
state.meta ??= {};
125+
state.meta.triageRecords = records;
126+
}
127+
function getTriageRecord(state, key) {
128+
return getTriageRecords(state)[key];
129+
}
130+
function setTriageRecord(state, key, record) {
131+
const records = getTriageRecords(state);
132+
records[key] = record;
133+
setTriageRecords(state, records);
134+
}
116135
// --- Guild-namespaced cursor helpers ---
117136
function getGuildCursor(state, guildName, channelId) {
118137
return state.cursors?.discord?.[guildName]?.[channelId];

docs/dispatch.md

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
# Dispatch system
2+
3+
The dispatch system gates which issues and messages the bot investigates. Instead of processing all eligible items immediately, maintainers can review and approve them first.
4+
5+
## Dispatch modes
6+
7+
| Mode | Behavior |
8+
|------|----------|
9+
| **auto** (default) | All eligible issues are processed immediately. No approval gate. |
10+
| **approval** | Detection issues are created in the target repo. A maintainer must react with an approval emoji before the bot investigates. |
11+
| **countdown** | Like approval, but with staged warnings. Auto-dispatches after all warning stages elapse (default: 3 stages, 24h apart = 72-hour minimum). |
12+
| **triage** | Interactive Discord-based triage. Help requests are posted to a private admin channel with action buttons. Maintainers control the entire lifecycle from Discord. See [Triage mode](#triage-mode) below. |
13+
14+
Set the mode via CLI (`--dispatch-mode <mode>`) or in `config.json`:
15+
16+
```json
17+
{
18+
"dispatch": {
19+
"mode": "auto",
20+
"discord_mode": "triage",
21+
"warnings_required": 3,
22+
"warning_interval_hours": 24,
23+
"approve_reactions": ["+1", "rocket"],
24+
"cancel_reactions": ["-1"],
25+
"max_detections_per_cycle": 10,
26+
"close_on_dispatch": true
27+
}
28+
}
29+
```
30+
31+
## Detection issues (approval / countdown modes)
32+
33+
In `approval` and `countdown` modes, detection issues are created in the target repo (default: `game-ci/help-bot`) with labels `help-bot`, `detection`, and the source repo name.
34+
35+
Maintainer actions on detection issues:
36+
- React with approval emoji (thumbs up or rocket) to approve immediately
37+
- React with cancel emoji (thumbs down) to cancel
38+
- Post any comment to approve (bot-generated warning comments are excluded)
39+
40+
Only reactions/comments from collaborators listed in `config.json` `github.collaborators` are valid.
41+
42+
### Staged countdown safety
43+
44+
The countdown mode prevents bulk auto-dispatch if the bot hasn't run for a while. Only one stage can advance per cycle:
45+
46+
```
47+
Cycle 1: Create detection issue -> stage 0 (pending)
48+
Cycle N: 24h elapsed -> post Warning 1 -> stage 1
49+
Cycle N+: 24h elapsed -> post Warning 2 -> stage 2
50+
Cycle N+: 24h elapsed -> post Warning 3 -> stage 3
51+
Cycle N+: 24h elapsed -> auto-dispatch
52+
```
53+
54+
A maintainer can react to approve or cancel at any point during the countdown.
55+
56+
## Discord dispatch
57+
58+
In `auto`, `approval`, and `countdown` modes, Discord dispatch is never fully automatic -- even with `dispatch.mode: "auto"`, Discord sources are forced to at least `"approval"` mode via `dispatch.discord_mode`.
59+
60+
For full Discord-native control, use `triage` mode instead.
61+
62+
## Triage mode
63+
64+
Triage mode moves the entire approval and review workflow into Discord. When a help request is detected (from any monitored Discord channel, thread, or forum post), the bot posts an interactive triage notification to a private admin channel. Maintainers control every step via buttons.
65+
66+
```bash
67+
gameci-help-bot live --dispatch-mode triage
68+
```
69+
70+
### Triage flow
71+
72+
```
73+
Help request detected (Discord message, thread, forum post, or GitHub issue)
74+
|
75+
v
76+
Bot posts triage notification to #triage channel
77+
(orange embed with source content, author, channel link)
78+
Buttons: [Investigate] [Ignore]
79+
Bot also posts a brief acknowledgment reply to the user
80+
|
81+
+-- Maintainer clicks [Ignore]
82+
| Embed turns gray, marked as ignored
83+
|
84+
+-- Maintainer clicks [Investigate]
85+
|
86+
v
87+
Embed turns blue "INVESTIGATING"
88+
Bot status changes to DND with "Investigating #channel @author"
89+
Claude runs investigation (source code search, docs, cross-references)
90+
|
91+
v
92+
Embed turns green "READY" with response preview
93+
Buttons: [Send Response] [View Investigation] [Re-investigate] [Discard]
94+
Bot status returns to online
95+
|
96+
+-- [View Investigation]
97+
| Creates a thread on the triage message (or uses existing)
98+
| Posts the full response for maintainer review
99+
|
100+
+-- [Re-investigate]
101+
| Reads maintainer messages from the thread as instructions
102+
| Re-runs investigation with previous response marked as rejected
103+
| and maintainer guidance appended to the prompt
104+
|
105+
+-- [Send Response]
106+
| Posts the approved response to the original source
107+
| (Discord reply to the user's message)
108+
| Embed turns gray "SENT"
109+
|
110+
+-- [Discard]
111+
Embed turns gray, response discarded
112+
```
113+
114+
### Providing instructions for re-investigation
115+
116+
After an investigation completes (embed is green "READY"):
117+
118+
1. Click **View Investigation** to see the full response in a thread
119+
2. Reply in that thread with plain English instructions (e.g., "focus on macOS code signing" or "the user is on Unity 2022 LTS, check version-specific issues")
120+
3. Click **Re-investigate** -- the bot reads all non-bot messages from the thread and includes them as `## Maintainer Guidance` in the prompt. The previous response is marked as rejected so Claude generates a fresh answer.
121+
122+
You can post multiple instruction messages and re-investigate as many times as needed. Each re-investigation produces a new response file with a `-reinvN` suffix.
123+
124+
### Triage configuration
125+
126+
Add a `triage_channel_id` to your guild config and list Discord user IDs for triage access:
127+
128+
```json
129+
{
130+
"discord": {
131+
"triage_user_ids": ["YOUR_DISCORD_USER_ID"],
132+
"guilds": [
133+
{
134+
"name": "my-server",
135+
"guild_id": "...",
136+
"triage_channel_id": "...",
137+
"channels": [...]
138+
}
139+
]
140+
}
141+
}
142+
```
143+
144+
| Config key | Description |
145+
|------------|-------------|
146+
| `discord.triage_user_ids` | Array of Discord user IDs (snowflakes) allowed to use triage buttons. Checked alongside `github.collaborators`. |
147+
| `guild.triage_channel_id` | Discord channel ID of the private admin channel where triage notifications are posted. |
148+
| `dispatch.discord_mode` | Set to `"triage"` to enable triage mode, or pass `--dispatch-mode triage` on the CLI. |
149+
150+
### Access control
151+
152+
Button clicks are verified against two lists:
153+
- `github.collaborators` -- GitHub usernames (matched against Discord username)
154+
- `discord.triage_user_ids` -- Discord user IDs (exact snowflake match)
155+
156+
Non-authorized users get an ephemeral "Only maintainers can use triage controls" reply.
157+
158+
### Supported source types
159+
160+
Triage mode handles messages from:
161+
- **Text channels** -- Regular channel messages
162+
- **Threads** -- Messages inside threads (resolved to parent channel config)
163+
- **Forum posts** -- Forum channel posts (resolved to parent forum channel config)
164+
- **GitHub issues/PRs** -- (via cycle-based detection, routed to the same triage UI)
165+
166+
## Dispatch pipeline in cycle mode
167+
168+
The dispatch gate runs after issue filtering and security scanning, before LLM invocation:
169+
170+
1. `createDetections()` -- creates detection issues for new eligible issues
171+
2. `cleanupStaleDetections()` -- cancels detections for issues no longer eligible
172+
3. `checkApprovals()` -- checks reactions/comments, advances countdown stages
173+
4. If no approved issues, skip LLM entirely
174+
5. After investigations complete, `markDispatched()` updates state and `closeDispatchedDetections()` closes detection issues with cross-links
175+
176+
## Source files
177+
178+
| File | Purpose |
179+
|------|---------|
180+
| `src/dispatch/types.ts` | Type definitions: `DispatchMode`, `DetectionRecord`, `DispatchConfig` |
181+
| `src/dispatch/orchestrator.ts` | Main entry point: `runDispatch()`, `runDiscordDispatch()` |
182+
| `src/dispatch/detection.ts` | Creates detection issues (GitHub + Discord) |
183+
| `src/dispatch/approval.ts` | Checks reactions/comments, advances countdown stages |
184+
| `src/dispatch/lifecycle.ts` | Post-dispatch cleanup: mark dispatched, close detections |
185+
| `src/dispatch/sanitize.ts` | Security helpers for untrusted content |
186+
| `src/triage/types.ts` | `TriageRecord`, button ID helpers, guild shortname mapping |
187+
| `src/triage/notification.ts` | Embed builder, button rows, post/update triage messages |
188+
| `src/triage/handler.ts` | InteractionCreate handler, button routing, thread posting |
189+
| `src/triage/investigation.ts` | Claude investigation wrapper, maintainer instruction fetching |
190+
| `src/triage/send.ts` | Post approved responses to original source |

docs/feedback-reporting.md

Lines changed: 29 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,34 @@
1-
# Feedback & reporting
1+
# Feedback and reporting
22

3-
Responses include metadata (reactions, heuristic flags) in `data/responses/feedback.jsonl`. That file stores a JSON line per feedback entry with `responseId`, `verdict` (`good` or `bad`), optional `note`, and timestamp.
3+
The bot tracks response quality through two mechanisms: automatic reaction polling and manual feedback marking.
44

5-
### Commands
5+
## Automatic reaction feedback
66

7-
- `gameci-help-bot feedback mark-good <responseId> [--note "..."]`: Tag the reply as helpful.
8-
- `gameci-help-bot feedback mark-bad <responseId> [--note "..."]`: Mark it for follow-up.
7+
Every posted response includes a footer asking users to react with thumbs up or thumbs down. Each cycle, `syncFeedback()` polls reactions on all tracked bot comments via the GitHub API and records:
98

10-
### Reports
9+
- Thumbs up/down counts and user lists
10+
- Net sentiment classification (positive/negative/neutral/unknown)
1111

12-
- `gameci-help-bot report summary` prints:
13-
- Discord messages synced, responses posted/skipped.
14-
- GitHub issues/releases/tags synced, responses posted/skipped.
15-
- Feedback totals (good vs. bad) plus the timestamp of the last cycle.
16-
- Use the summary to decide whether a cycle needs a rerun, whether certain threads were skipped because an official contributor replied, or whether to tweak `config.json` filters.
12+
Results are written to `data/feedback/feedback-summary.md`, which the LLM reads during investigations to learn from past positive and negative feedback patterns.
13+
14+
## Manual feedback
15+
16+
```bash
17+
gameci-help-bot feedback mark-good <responseId> [--note "..."]
18+
gameci-help-bot feedback mark-bad <responseId> [--note "..."]
19+
```
20+
21+
Manual entries are stored in `data/responses/feedback.jsonl` as one JSON line per entry with `responseId`, `verdict` (`good` or `bad`), optional `note`, and timestamp.
22+
23+
## Reports
24+
25+
```bash
26+
gameci-help-bot report summary
27+
```
28+
29+
Prints:
30+
- Discord messages synced, responses posted/skipped
31+
- GitHub issues/releases/tags synced, responses posted/skipped
32+
- Feedback totals (good vs. bad) plus the timestamp of the last cycle
33+
34+
Use the summary to decide whether a cycle needs a rerun, whether certain threads were skipped because an official contributor replied, or whether to tweak `config.json` filters.

0 commit comments

Comments
 (0)