Skip to content

Commit e74c2b7

Browse files
authored
feat: natural language parsing and multi-instance for /consult (#223)
* feat: add natural language parsing and multi-instance support to /consult The /consult command now accepts free-form natural language input in addition to explicit flags: - "with codex about my auth approach" → tool=codex, question extracted - "ask 3 gemini about this design" → tool=gemini, count=3 - "thoroughly ask claude about errors" → tool=claude, effort=high Multi-instance support (--count=N or "ask N tool") runs N parallel consultations with the same tool, presenting numbered responses with a brief synthesis of agreements and differences. Architecture: command handles NLP parsing + interactive resolution, agent extended with multi-instance parallel execution, skill unchanged. * fix: use placeholder timestamp in multi-session state example Gemini review: hardcoded timestamp in JSON example could mislead the implementing agent. Use dynamic placeholder instead. * fix: address Copilot review feedback - Add count validation (1-5) in Phase 1 and agent Step 2 - Document --continue + --count mutual exclusivity - Add 5th option to Instances picker (was missing 4-5 range)
1 parent a22a469 commit e74c2b7

File tree

6 files changed

+634
-190
lines changed

6 files changed

+634
-190
lines changed

__tests__/consult-command.test.js

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -289,7 +289,7 @@ describe('security constraints', () => {
289289
});
290290

291291
test('command validates --tool against allow-list', () => {
292-
expect(commandContent).toMatch(/validate.*--tool.*allow-list|MUST be one of.*gemini.*codex.*claude/i);
292+
expect(commandContent).toMatch(/validate.*tool.*against.*allow-list|allow-list.*gemini.*codex.*claude|one of.*gemini.*codex.*claude/i);
293293
});
294294

295295
test('skill validates --context=file=PATH is within project', () => {
@@ -358,11 +358,11 @@ describe('interactive selection completeness', () => {
358358
expect(commandContent).toMatch(/AskUserQuestion[\s\S]*?header:\s*"Effort"/);
359359
});
360360

361-
test('Phase 2b has MUST enforcement language for effort', () => {
362-
// Effort picker is in Step 2b (batch selection), model picker in Step 2c
363-
expect(commandContent).toMatch(/Do NOT silently default --effort/);
364-
expect(commandContent).toMatch(/Do NOT skip this step/);
365-
expect(commandContent).toMatch(/Do NOT skip any missing parameter/);
361+
test('Phase 2 has MUST enforcement language for parameters', () => {
362+
// Check that the command enforces interactive resolution for missing params
363+
expect(commandContent).toMatch(/Do NOT silently default any parameter|Do NOT silently use a default model/);
364+
expect(commandContent).toMatch(/Do NOT skip model selection|Do NOT skip this/i);
365+
expect(commandContent).toMatch(/MUST resolve ALL missing parameters/);
366366
});
367367

368368
test('Phase 2 header has MUST language for all missing parameters', () => {
@@ -560,7 +560,7 @@ describe('codex AskUserQuestion transform compatibility', () => {
560560
// The picker should have structured options, not just plain text
561561
// Check that each tool option line has both label and description
562562
const toolPickerSection = commandContent.match(
563-
/Step 2b[\s\S]*?Step 2c/
563+
/Step 2c[\s\S]*?Step 2d/
564564
);
565565
expect(toolPickerSection).not.toBeNull();
566566

@@ -573,9 +573,9 @@ describe('codex AskUserQuestion transform compatibility', () => {
573573
});
574574

575575
test('effort selection options have label+description format', () => {
576-
// Effort picker is in Step 2b (batch selection), not Step 2c (model selection)
576+
// Effort picker is in Step 2c (batch selection), model picker in Step 2d
577577
const effortPickerSection = commandContent.match(
578-
/Step 2b[\s\S]*?Step 2c/
578+
/Step 2c[\s\S]*?Step 2d/
579579
);
580580
expect(effortPickerSection).not.toBeNull();
581581

adapters/codex/skills/consult/SKILL.md

Lines changed: 130 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,59 +1,86 @@
11
<!-- AUTO-GENERATED by scripts/gen-adapters.js - DO NOT EDIT -->
22
---
33
name: consult
4-
description: "Use when user asks to \"consult gemini\", \"ask codex\", \"get second opinion\", \"cross-check with claude\", \"consult another AI\", \"ask opencode\", \"copilot opinion\". Queries another AI CLI tool and returns the response."
4+
description: "Use when user asks to \"consult gemini\", \"ask codex\", \"get second opinion\", \"cross-check with claude\", \"consult another AI\", \"ask opencode\", \"copilot opinion\", \"ask 3 codex\", \"multi-consult\". Queries another AI CLI tool and returns the response."
55
---
66

77
# /consult - Cross-Tool AI Consultation
88

9-
You are executing the /consult command. Your job is to consult another AI CLI tool, get its response, and present the results to the user.
9+
You are executing the /consult command. Your job is to parse the user's request (natural language or flags), resolve missing parameters interactively, and execute the consultation.
1010

1111
## Constraints
1212

1313
- NEVER expose API keys in commands or output
1414
- NEVER run with permission-bypassing flags (`--dangerously-skip-permissions`, `bypassPermissions`)
1515
- MUST use safe-mode defaults (`-a suggest` for Codex, `--allowedTools "Read,Glob,Grep"` for Claude)
1616
- MUST enforce 120s timeout on all tool executions
17-
- MUST validate `--tool` against allow-list: gemini, codex, claude, opencode, copilot (reject all others)
17+
- MUST validate tool names against allow-list: gemini, codex, claude, opencode, copilot (reject all others)
1818
- MUST validate `--context=file=PATH` is within the project directory (reject absolute paths outside cwd)
1919
- MUST quote all user-provided values in shell commands to prevent injection
2020
- NEVER execute tools the user has not explicitly requested
2121

22-
## Arguments
22+
## Execution
2323

24-
Parse from $ARGUMENTS:
24+
### Phase 1: Parse Input (Flags + Natural Language)
2525

26-
- **question**: What to ask the consulted tool (required unless --continue)
27-
- **--tool**: Target tool: `gemini`, `codex`, `claude`, `opencode`, `copilot` (interactive picker if omitted)
28-
- **--effort**: Thinking effort: `low`, `medium`, `high`, `max` (interactive picker if omitted)
29-
- **--model**: Specific model name (interactive picker if omitted). Free text.
30-
- **--context**: Auto-include context: `diff` (git diff), `file=PATH` (attach specific file), `none` (default)
31-
- **--continue**: Continue last consultation session, or `--continue=SESSION_ID` for specific session
26+
Parse `$ARGUMENTS` using both explicit flags and natural language extraction. Flags always take priority over NLP when both provide the same parameter.
3227

33-
## Execution
28+
#### Step 1a: Extract explicit flags
29+
30+
Look for and remove these flags from `$ARGUMENTS`:
31+
32+
1. `--tool=VALUE` or `--tool VALUE` where VALUE is one of: gemini, codex, claude, opencode, copilot
33+
2. `--effort=VALUE` or `--effort VALUE` where VALUE is one of: low, medium, high, max
34+
3. `--model=VALUE` or `--model VALUE` (any string, including quoted)
35+
4. `--context=VALUE` where VALUE is: diff, file=PATH, or none
36+
5. `--continue` (optionally `--continue=SESSION_ID`)
37+
6. `--count=N` where N is 1-5
38+
39+
Remove all matched flags and their values from `$ARGUMENTS`.
40+
41+
#### Step 1b: Natural language extraction (on remaining text)
42+
43+
After removing flags, parse the remaining text for these patterns:
3444

35-
### Phase 1: Parse Arguments
45+
**Tool extraction** (case-insensitive):
46+
- "with {tool}" (e.g., "with codex") -> tool
47+
- "ask {tool}" (e.g., "ask gemini") -> tool
48+
- "consult {tool}" -> tool
49+
- "{tool} about" (e.g., "codex about") -> tool
50+
- Tool names: claude, gemini, codex, opencode, copilot
3651

37-
Extract these values from `$ARGUMENTS`:
52+
**Count extraction**:
53+
- "ask {N} {tool}" (e.g., "ask 3 codex") -> count=N, tool
54+
- "{N} {tool}" (e.g., "3 codex") -> count=N, tool
55+
- "{N} instances" -> count=N
56+
- "few instances" / "multiple" / "several" -> count=ambiguous (ask user in Phase 2)
3857

39-
1. Look for `--tool=VALUE` or `--tool VALUE` where VALUE MUST be one of: gemini, codex, claude, opencode, copilot (reject others)
40-
2. Look for `--effort=VALUE` or `--effort VALUE` where VALUE MUST be one of: low, medium, high, max
41-
3. Look for `--model=VALUE` or `--model VALUE` (any string, including quoted strings like `"my model"`)
42-
4. Look for `--context=VALUE` where VALUE is: diff, file=PATH, or none
43-
5. Look for `--continue` (optionally `--continue=SESSION_ID`)
44-
6. Remove all matched flags (including their values) from `$ARGUMENTS`. Handle quoted flag values (e.g., `--model "gpt 4"`) by removing the entire quoted string. Everything remaining is the **question**.
58+
**Count validation**: After extracting count (from flags or NLP), validate: 1 <= count <= 5. If count < 1 or count > 5, show `[ERROR] Instance count must be 1-5. Got: {count}` and stop.
4559

46-
If no question text and no `--continue` flag found, show:
60+
**Effort extraction**:
61+
- "quick" / "fast" / "brief" -> effort=low
62+
- "thorough" / "deep" / "carefully" / "detailed" -> effort=high
63+
- "maximum" / "max effort" / "exhaustive" -> effort=max
64+
65+
**Question extraction**:
66+
- Text after "about" is the question (e.g., "with codex about my auth approach" -> question="my auth approach")
67+
- If no "about" pattern, everything remaining after removing tool/count/effort markers is the question
68+
69+
**Precedence rule**: Flags from Step 1a always override NLP from Step 1b.
70+
71+
If no question text and no `--continue` flag found after both steps:
4772
```
48-
[ERROR] Usage: /consult "your question" [--tool=gemini|codex|claude|opencode|copilot] [--effort=low|medium|high|max]
73+
[ERROR] Usage: /consult "your question" or /consult with gemini about your question
4974
```
5075

51-
### Phase 2: Interactive Parameter Selection
76+
### Phase 2: Interactive Parameter Resolution
5277

53-
MUST resolve ALL missing parameters interactively before Phase 3. ONLY skip this phase if ALL of --tool, --effort, AND --model are explicitly provided by the user in $ARGUMENTS. Do NOT silently default any parameter.
78+
MUST resolve ALL missing parameters interactively. ONLY skip this phase if ALL required params (tool, effort, model) are resolved AND either a question exists or --continue is present. Do NOT silently default any parameter.
5479

5580
#### Step 2a: Handle --continue
5681

82+
**Note:** `--continue` and `--count > 1` are mutually exclusive. Session resume applies to a single tool session. If both are present, show `[ERROR] Cannot use --continue with --count > 1. Use --continue for single session resume.` and stop.
83+
5784
If `--continue` is present:
5885
1. Read the session file at `{AI_STATE_DIR}/consult/last-session.json`
5986

@@ -64,9 +91,9 @@ If `--continue` is present:
6491
2. If the file exists, restore the saved tool, session_id, and model from it
6592
3. If the file does not exist, show `[WARN] No previous session found` and proceed as a fresh consultation
6693

67-
#### Step 2b: Batch Selection (tool + effort)
94+
#### Step 2b: Detect installed tools
6895

69-
First, detect which tools are installed by running all 5 checks **in parallel** via Bash:
96+
Run all 5 checks **in parallel** via Bash:
7097

7198
- `where.exe <tool> 2>nul && echo FOUND || echo NOTFOUND` (Windows)
7299
- `which <tool> 2>/dev/null && echo FOUND || echo NOTFOUND` (Unix)
@@ -75,13 +102,15 @@ Check for: claude, gemini, codex, opencode, copilot.
75102

76103
If zero tools are installed: `[ERROR] No AI CLI tools found. Install at least one: npm i -g @anthropic-ai/claude-code, npm i -g @openai/codex, npm i -g opencode-ai`
77104

78-
Then use a SINGLE request_user_input call to ask all missing parameters at once. Include only the questions for parameters NOT already provided in $ARGUMENTS:
105+
#### Step 2c: Batch selection for missing params
106+
107+
Use a SINGLE request_user_input call to ask all missing parameters at once. Include ONLY questions for parameters NOT already resolved from Phase 1:
79108

80109
```
81110
request_user_input:
82111
> **Codex**: Each question MUST include a unique `id` field (e.g., `id: "q1"`).
83112
questions:
84-
- header: "AI Tool" # SKIP if --tool provided
113+
- header: "AI Tool" # SKIP if tool resolved
85114
question: "Which AI tool should I consult?"
86115
options (only if installed):
87116
- label: "Claude" description: "Deep code reasoning"
@@ -90,23 +119,35 @@ request_user_input:
90119
- label: "OpenCode" description: "Flexible model choice"
91120
- label: "Copilot" description: "GitHub-integrated AI"
92121
93-
- header: "Effort" # SKIP if --effort provided
122+
- header: "Effort" # SKIP if effort resolved
94123
question: "What thinking effort level?"
95124
options:
96125
- label: "Medium (Recommended)" description: "Balanced speed and quality"
97126
- label: "Low" description: "Fast, minimal reasoning"
98127
- label: "High" description: "Thorough analysis"
99128
- label: "Max" description: "Maximum reasoning depth"
129+
130+
- header: "Instances" # SKIP if count resolved or not hinted
131+
question: "How many parallel consultations?"
132+
options:
133+
- label: "1 (Single)" description: "Standard single consultation"
134+
- label: "2 (Compare)" description: "Two responses to compare"
135+
- label: "3 (Panel)" description: "Three perspectives"
136+
- label: "5 (Full spread)" description: "Five diverse perspectives"
100137
```
101138

102-
Map tool choice to lowercase: "Claude" -> "claude", "Codex" -> "codex", etc.
103-
Map effort choice: "Medium (Recommended)" -> "medium", "Low" -> "low", "High" -> "high", "Max" -> "max".
139+
ONLY show the Instances question if:
140+
- The user explicitly mentioned multiple instances (e.g., "few", "multiple", "several")
141+
- The count was set but ambiguous
142+
Do NOT show Instances question for simple single-tool requests. Default count=1 silently when no multi-instance intent detected.
104143

105-
IMPORTANT: Do NOT skip any missing parameter. Do NOT silently default --effort to "medium" or --tool to any value. Present pickers for ALL unresolved parameters.
144+
Map tool choice to lowercase: "Claude" -> "claude", "Codex" -> "codex", etc.
145+
Map effort choice: "Medium (Recommended)" -> "medium", "Low" -> "low", etc.
146+
Map count choice: "1 (Single)" -> 1, "2 (Compare)" -> 2, "3 (Panel)" -> 3.
106147

107-
#### Step 2c: Model Selection (MUST ask if no --model)
148+
#### Step 2d: Model selection (MUST ask if no --model)
108149

109-
After tool is resolved (from Step 2b or $ARGUMENTS), present a model picker with options specific to the selected tool. The user can always type a custom model name via the "Other" option.
150+
After tool is resolved, present a model picker with options specific to the selected tool. The user can always type a custom model name via the "Other" option.
110151

111152
**For Claude:**
112153
```
@@ -173,17 +214,21 @@ request_user_input:
173214
options:
174215
- label: "claude-sonnet-4-5" description: "Default Copilot model"
175216
- label: "claude-opus-4-6" description: "Most capable Claude model"
176-
- label: "gpt-5.3-codex" description: "OpenAI GPT-5.3 Codex"
217+
- label: "gpt-5.3-codex" description: "OpenAI GPT-5.3 Codex"
177218
- label: "gemini-3-pro" description: "Google Gemini 3 Pro"
178219
```
179220

180-
Map the user's choice to the model string (strip " (Recommended)" suffix if present). Pass the selected model to the skill via `--model`.
221+
Map the user's choice to the model string (strip " (Recommended)" suffix if present).
181222

182-
IMPORTANT: Do NOT skip this step. Do NOT silently use a default model. If --model was not explicitly provided in $ARGUMENTS, you MUST present this picker. The model lists above are current as of Feb 2026 - the user may type any model name supported by their tool via the "Other" option.
223+
IMPORTANT: Do NOT skip model selection. Do NOT silently use a default model. If --model was not explicitly provided, you MUST present this picker.
183224

184-
### Phase 3: Invoke Consult Skill
225+
### Phase 3: Execute Consultation
185226

186-
With all parameters resolved (tool, effort, model, question, and optionally context, continue), invoke the `consult` skill using the Skill tool:
227+
With all parameters resolved (tool, effort, model, question, count, and optionally context/continue):
228+
229+
#### Single instance (count=1, the default)
230+
231+
Invoke the `consult` skill directly using the Skill tool:
187232

188233
```
189234
Skill: consult
@@ -192,15 +237,35 @@ Args: "[question]" --tool=[tool] --effort=[effort] --model=[model] [--context=[c
192237
Example: "Is this the right approach?" --tool=gemini --effort=high --model=gemini-3-pro
193238
```
194239

195-
The skill handles the full consultation lifecycle: it resolves the model from the effort level, builds the CLI command, packages any context, executes the command via Bash with a 120-second timeout, and returns a plain JSON result.
240+
The skill handles the full consultation lifecycle: model resolution, command building, context packaging, execution with 120s timeout, and returns a plain JSON result.
241+
242+
#### Multi-instance (count > 1)
196243

197-
### Phase 4: Parse Skill Output
244+
Spawn the `consult:consult-agent` via the Task tool with all resolved parameters:
198245

199-
The skill returns a plain JSON object containing: `tool`, `model`, `effort`, `duration_ms`, `response`, `session_id`, and `continuable`. Parse the JSON directly from the skill output.
246+
```
247+
Task:
248+
subagent_type: "consult:consult-agent"
249+
prompt: |
250+
Execute a multi-instance consultation with these pre-resolved parameters:
251+
- tool: [tool]
252+
- model: [model]
253+
- effort: [effort]
254+
- question: [question]
255+
- count: [count]
256+
- context: [context or none]
257+
258+
Run [count] parallel consultations with the same tool and parameters.
259+
Return all responses formatted with numbered headers and a brief synthesis.
260+
```
261+
262+
The agent handles parallel execution, temp file management, result parsing, and synthesis.
200263

201-
### Phase 5: Present Results
264+
### Phase 4: Present Results
202265

203-
After parsing the JSON, format and display the result as human-friendly text:
266+
#### Single instance
267+
268+
Parse the skill's plain JSON output and display:
204269

205270
```
206271
Tool: {tool}, Model: {model}, Effort: {effort}, Duration: {duration_ms}ms.
@@ -211,35 +276,46 @@ The results of the consultation are:
211276

212277
For continuable tools (Claude, Gemini, Codex, OpenCode), display: `Session: {session_id} - use /consult --continue to resume`
213278

214-
Save session state for continuable tools (Claude, Gemini, Codex, OpenCode) to `{AI_STATE_DIR}/consult/last-session.json`.
279+
Save session state to `{AI_STATE_DIR}/consult/last-session.json`.
280+
281+
#### Multi-instance
215282

216-
Platform state directory:
217-
- Claude Code: `.claude/`
218-
- OpenCode: `.opencode/`
219-
- Codex CLI: `.codex/`
283+
Display the agent's formatted output directly. The agent returns numbered responses with a synthesis section.
220284

221285
On failure: `[ERROR] Consultation Failed: {specific error message}`
222286

223287
## Error Handling
224288

225289
| Error | Output |
226290
|-------|--------|
227-
| No question provided | `[ERROR] Usage: /consult "your question" [--tool=gemini\|codex\|claude\|opencode\|copilot] [--effort=low\|medium\|high\|max]` |
291+
| No question provided | `[ERROR] Usage: /consult "your question" or /consult with gemini about your question` |
228292
| Tool not installed | `[ERROR] {tool} is not installed. Install with: {install command from skill}` |
229293
| Tool execution fails | `[ERROR] {tool} failed: {error}. Try a different tool with --tool=[other]` |
230294
| Timeout (>120s) | `[ERROR] {tool} timed out after 120s. Try --effort=low for faster response` |
231295
| No tools available | `[ERROR] No AI CLI tools found. Install: npm i -g @anthropic-ai/claude-code` |
232296
| Session not found | `[WARN] No previous session found. Starting fresh consultation.` |
233297
| API key missing | `[ERROR] {tool} requires API key. Set {env var} (see skill for details)` |
298+
| Count out of range | `[ERROR] Instance count must be 1-5. Got: {count}` |
299+
| Multi-instance partial failure | Show successful responses, note failures |
234300

235301
## Example Usage
236302

237303
```bash
238-
/consult "Is this the right approach for error handling?" --tool=gemini --effort=high
239-
/consult "Review this function for performance issues" --tool=codex
240-
/consult "What alternative patterns would you suggest?" --tool=claude --effort=max
304+
# Natural language (NLP parsing)
305+
/consult with codex about my auth approach
306+
/consult ask 3 codex about this design
307+
/consult gemini should I use redis or postgres
308+
/consult thoroughly ask claude about error handling
309+
/consult codex few instances about performance
310+
311+
# Explicit flags (backward compatible)
312+
/consult "Is this the right approach?" --tool=gemini --effort=high
313+
/consult "Review this function" --tool=codex --count=3
241314
/consult "Suggest improvements" --tool=opencode --model=github-copilot/claude-opus-4-6
242-
/consult "Continue from where we left off" --continue
315+
/consult --continue
243316
/consult "Explain this error" --context=diff --tool=gemini
244317
/consult "Review this file" --context=file=src/index.js --tool=claude
318+
319+
# Mixed (flags + natural language)
320+
/consult with gemini --effort=max about database schema design
245321
```

0 commit comments

Comments
 (0)