Skip to content

Commit 9e1f7d6

Browse files
bugerclaude
andauthored
feat: Slack frontend v2 with socket mode, workspace isolation, and improvements (#284)
* feat(slack): wire Slack frontend to real client (fetch-based) + test support\n- Add minimal SlackClient (no external deps) with chat.postMessage/update\n- SlackFrontend can auto-create client from SLACK_BOT_TOKEN or frontends[].config.botToken\n- Test runner: inject RecordingSlack when suite defaults enable frontends: ['slack'] and assert provider=slack ops\n- Extend defaults/visor.tests.yaml to enable Slack and add PR-stage assertions\n- Add unit tests for Slack frontend * feat(slack): add Slack signature verification + URL verification handshake\n- verifySlackSignature helper and WebhookServer integration\n- handle Slack URL verification by echoing challenge\n- unit tests for signature verification * feat(slack/socket): add --slack CLI option for Socket Mode + engine webhookContext plumbing\n- SlackSocketRunner uses apps.connections.open + ws to receive events\n- Reuse http_input via webhookContext injection for event payloads\n- CLI: --slack starts socket runner (config.slack.* optional)\n- Webhook server remains available for Events API mode (HMAC+challenge)\n- Unit tests for signature + Slack frontend intact * feat(slack/cache): port thread-cache, rate-limiter, cache-prewarmer + adapter\n- Add SlackAdapter + ThreadCache and RateLimiter modules (trimmed)\n- Add CachePrewarmer and wire optional prewarm in Socket Mode\n- Add types/bot for NormalizedMessage and Slack config * chore(task-refinement): add slack config version marker (slack.version: v1) * feat(slack): state-machine pause/resume + socket chat example * Improve Slack markdown formatting for AI replies * feat: workspace isolation, Slack frontend improvements, and test fixes - Add workspace isolation for parallel run separation - Improve Slack frontend with better message threading - Update probe to v0.6.0-rc189 - Add new test coverage for worktree manager, sandbox, and providers - Fix YAML test mocks: add labels to issue-assistant, add project-status-fetch mock - Fix deploy-loop routing by setting max_loops to 0 - Fix lint warnings for liquidjs imports Co-Authored-By: Claude Opus 4.5 <[email protected]> * chore: sync dist folder with latest main Co-Authored-By: Claude Opus 4.5 <[email protected]> * chore: reset dist folder to match main exactly Remove all dist files that were added to this branch but don't exist in main. This ensures the PR has zero dist changes vs upstream main. Co-Authored-By: Claude Opus 4.5 <[email protected]> * fix: resolve relative paths against workspace directory when enabled When workspace isolation is enabled: - http_client: relative output_file paths now resolve against workingDirectory - command: commands now execute with cwd set to workingDirectory This ensures file operations respect workspace boundaries instead of using the original process.cwd(). Co-Authored-By: Claude Opus 4.5 <[email protected]> * test: skip sandbox continue/break tests on Node < 22 The @nyariv/sandboxjs library doesn't support continue/break inside if statements on Node versions below 22. Skip these tests conditionally. Co-Authored-By: Claude Opus 4.5 <[email protected]> --------- Co-authored-by: Claude Opus 4.5 <[email protected]>
1 parent 02ba424 commit 9e1f7d6

File tree

74 files changed

+5439
-662
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

74 files changed

+5439
-662
lines changed

CLAUDE.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,4 +150,17 @@ Configuration supports:
150150
- Safe JSON parsing: `try { JSON.parse(output) } catch(e) { log("Error:", e) }`
151151
- Validate structure: `log("Is array?", Array.isArray(outputs["check-name"]));`
152152

153+
6. **Tracing with OTel/Jaeger**:
154+
- Enable telemetry: `VISOR_TELEMETRY_ENABLED=true`, `VISOR_TELEMETRY_SINK=otlp`,
155+
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces`
156+
- Root span: `visor.run` (one per CLI/Slack execution)
157+
- State spans: `engine.state.*` with `wave`, `wave_kind`, `session_id`
158+
- Check spans: `visor.check.<checkId>` with `visor.check.id`, `visor.check.type`,
159+
`visor.foreach.index` (for map fanout)
160+
- Routing decisions: `visor.routing` events attached to the active state span; fields
161+
include `trigger`, `action`, `source`, `target`, `scope`, `goto_event` (repeats
162+
across waves show routing loops)
163+
- Wave visibility: `engine.state.level_dispatch` includes `level_size` and
164+
`level_checks_preview` for the planned wave
165+
153166
See `docs/debugging.md` for comprehensive debugging guide.

README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -534,6 +534,12 @@ steps:
534534
session_mode: append # Shares history for full conversation
535535
```
536536

537+
You can also reuse the **same check’s** session when it loops back to itself, using:
538+
539+
- `reuse_ai_session: "self"` with `session_mode: append`
540+
541+
See the standalone example at `examples/session-reuse-self.yaml` and the detailed guide in [docs/advanced-ai.md](docs/advanced-ai.md).
542+
537543
Learn more: [docs/advanced-ai.md](docs/advanced-ai.md)
538544

539545
## 📋 Schema-Template System

defaults/task-refinement.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ steps:
8181
group: task-refinement
8282
depends_on: [refine]
8383
reuse_ai_session: refine
84-
session_mode: append
84+
session_mode: clone
8585
ai:
8686
# Allow tools so the model can inspect the repo context if needed
8787
disableTools: false

defaults/visor.tests.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -435,6 +435,7 @@ tests:
435435
issue-assistant:
436436
text: "Opening issue with claim about defaults/visor.yaml"
437437
intent: issue_triage
438+
labels: ["bug", "triage"]
438439
extract-facts:
439440
- { id: f1, category: Configuration, claim: "max_parallelism defaults to 4", verifiable: true }
440441
validate-fact[]:
@@ -468,6 +469,7 @@ tests:
468469
issue-assistant:
469470
text: "Claim: max_parallelism defaults to 4"
470471
intent: issue_triage
472+
labels: ["bug", "triage"]
471473
extract-facts:
472474
- { id: f1, category: Configuration, claim: "max_parallelism defaults to 4", verifiable: true }
473475
validate-fact[]:

docs/advanced-ai.md

Lines changed: 60 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
## 🧠 Advanced AI Features
22

33
### AI Session Reuse
4-
Use `reuse_ai_session: true` on dependent checks to continue conversation context with the AI across checks. This improves follow‑ups and consistency.
4+
Use `reuse_ai_session` on checks to continue conversation context with the AI across steps. This improves follow‑ups and consistency for follow‑on analysis and chat‑style flows.
55

66
**Session Modes:**
77
- **`clone` (default)**: Creates a copy of the conversation history. Each check gets an independent session with the same starting context. Changes made by one check don't affect others.
@@ -26,6 +26,65 @@ steps:
2626
depends_on: [security-remediation]
2727
reuse_ai_session: true
2828
session_mode: append # Share history - sees full conversation
29+
30+
#### Reusing your own session: `reuse_ai_session: self`
31+
32+
Sometimes the step you want to loop back into is the AI step itself (e.g. Slack assistants or multi‑turn internal tools). For that case you can use:
33+
34+
- `reuse_ai_session: "self"` – the step reuses its **own** Probe session when it runs again in the same engine run.
35+
- `session_mode: append` – makes the follow‑up behave like a normal conversation turn.
36+
37+
On the first run of the step, Visor creates a new ProbeAgent session and registers it. If routing (`on_success.goto`, `goto_js`, etc.) later jumps back to the same step within the same run, the engine:
38+
39+
- Finds the last result for that step in the current run.
40+
- Reads the `sessionId` stored in the result.
41+
- Calls the AI provider again using `executeReviewWithSessionReuse` with that session id.
42+
43+
Simple example (no transport wiring, just CLI/tests):
44+
45+
```yaml
46+
version: "2.0"
47+
48+
steps:
49+
seed:
50+
type: script
51+
content: |
52+
return { text: "hello from seed" };
53+
54+
convo:
55+
type: ai
56+
depends_on: [seed]
57+
reuse_ai_session: self
58+
session_mode: append
59+
ai:
60+
provider: mock
61+
model: mock
62+
disableTools: true
63+
allowedTools: []
64+
system_prompt: "You are a tiny echo assistant."
65+
prompt: |
66+
Seed message: {{ outputs['seed'].text }}
67+
68+
Past convo outputs in this run:
69+
{% assign hist = outputs_history['convo'] | default: empty %}
70+
{% if hist and hist.size > 0 %}
71+
{% for h in hist %}
72+
- Previous reply {{ forloop.index }}.
73+
{% endfor %}
74+
{% else %}
75+
- No previous replies yet.
76+
{% endif %}
77+
on_success:
78+
goto_js: |
79+
// Example: re‑enter this step up to 3 times in a single run
80+
return attempt < 3 ? 'convo' : null;
81+
```
82+
83+
The corresponding testable example lives at:
84+
85+
- `examples/session-reuse-self.yaml`
86+
87+
This keeps the configuration small but shows how to wire `reuse_ai_session: self` and `session_mode: append` without touching higher‑level workflows like `tyk-assistant`.
2988
```
3089
3190
**When to use each mode:**

docs/recipes.md

Lines changed: 148 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -321,8 +321,154 @@ Tip: When you define a JSON Schema, you generally do **not** need to tell the mo
321321
- Avoid noisy fallbacks like `(outputs['x']?.kind ?? '') === 'status'` when `outputs['x']?.kind === 'status'` is equivalent.
322322
- These conventions apply uniformly to any provider (`ai`, `command`, `script`, `github`, `http_client`, etc).
323323

324+
### Command step best practices
325+
326+
When using `type: command` steps:
327+
328+
**Avoid external tool dependencies** like `jq`, `yq`, `python`, etc.:
329+
- They may not be installed in all environments (GitHub Actions, Docker, CI)
330+
- Use `transform_js` to parse and transform output instead
331+
- Keep shell commands simple: `grep`, `sed`, `awk`, `sort`, `head` are universally available
332+
333+
```yaml
334+
# Bad - requires jq
335+
extract-data:
336+
type: command
337+
exec: |
338+
echo "$TEXT" | grep -oE '[A-Z]+-[0-9]+' | jq -R -s 'split("\n")'
339+
parseJson: true
340+
341+
# Good - use transform_js for parsing
342+
extract-data:
343+
type: command
344+
exec: |
345+
echo "$TEXT" | grep -oE '[A-Z]+-[0-9]+' | sort -u
346+
transform_js: |
347+
const lines = (output || '').trim().split('\n').filter(Boolean);
348+
return { data: lines, count: lines.length };
349+
```
350+
351+
**Prefer line-separated output** over JSON from shell:
352+
- Simple to parse with `transform_js`
353+
- No need for `parseJson: true`
354+
- More robust across different shells/environments
355+
356+
**Use transform_js for structured output**:
357+
- The sandbox provides `output` (command stdout as string)
358+
- Return an object with the fields you need
359+
- Works consistently across all environments
360+
361+
### Testing workflows with `--no-mocks`
362+
363+
The `--no-mocks` flag runs your test cases with real providers instead of injecting mock responses. This is essential for:
364+
365+
1. **Debugging integration issues** - See actual API responses and errors
366+
2. **Capturing realistic mock data** - Get real output to copy into your test cases
367+
3. **Validating credentials** - Verify environment variables are set correctly
368+
4. **Developing new workflows** - Build tests incrementally with real data
369+
370+
#### Basic usage
371+
372+
```bash
373+
# Run all test cases with real providers
374+
visor test --config my-workflow.yaml --no-mocks
375+
376+
# Run a specific test case with real providers
377+
visor test --config my-workflow.yaml --no-mocks --only "my-test-case"
378+
```
379+
380+
#### Suggested mocks output
381+
382+
When running with `--no-mocks`, Visor captures each step's output and prints it as YAML you can copy directly into your test case:
383+
384+
```
385+
🔴 NO-MOCKS MODE: Running with real providers (no mock injection)
386+
Step outputs will be captured and printed as suggested mocks
387+
388+
... test execution ...
389+
390+
📋 Suggested mocks (copy to your test case):
391+
mocks:
392+
extract-keys:
393+
data:
394+
- PROJ-123
395+
- DEV-456
396+
count: 2
397+
fetch-issues:
398+
data:
399+
- key: PROJ-123
400+
summary: Fix authentication bug
401+
status: In Progress
402+
```
403+
404+
Copy the YAML under `mocks:` into your test case's `mocks:` section.
405+
406+
#### Workflow for building tests
407+
408+
1. **Start with a minimal test case** (no mocks):
409+
```yaml
410+
tests:
411+
cases:
412+
- name: my-new-test
413+
event: manual
414+
fixture: local.minimal
415+
workflow_input:
416+
text: "Fix bug PROJ-123"
417+
```
418+
419+
2. **Run with `--no-mocks`** to capture real outputs:
420+
```bash
421+
visor test --config workflow.yaml --no-mocks --only "my-new-test"
422+
```
423+
424+
3. **Copy the suggested mocks** into your test case:
425+
```yaml
426+
tests:
427+
cases:
428+
- name: my-new-test
429+
event: manual
430+
fixture: local.minimal
431+
workflow_input:
432+
text: "Fix bug PROJ-123"
433+
mocks:
434+
extract-keys:
435+
data: ["PROJ-123"]
436+
count: 1
437+
# ... rest of captured mocks
438+
```
439+
440+
4. **Add assertions** based on the real data:
441+
```yaml
442+
expect:
443+
workflow_output:
444+
- path: issue_count
445+
equals: 1
446+
```
447+
448+
5. **Run normally** to verify mocks work:
449+
```bash
450+
visor test --config workflow.yaml --only "my-new-test"
451+
```
452+
453+
#### Debugging with `--no-mocks`
454+
455+
When a test fails with mocks, use `--no-mocks` to see what's actually happening:
456+
457+
```bash
458+
# See real API responses and errors
459+
visor test --config workflow.yaml --no-mocks --only "failing-test"
460+
461+
# Common issues revealed:
462+
# - Missing or expired credentials
463+
# - API endpoint changes
464+
# - Unexpected response formats
465+
# - Network/timeout issues
466+
```
467+
468+
The real error messages and responses help identify whether the issue is with your mocks or the actual integration.
469+
324470
### More examples
325471

326-
- `docs/NPM_USAGE.md` – CLI usage and flags
327-
- `GITHUB_CHECKS.md` – Checks, outputs, and workflow integration
472+
- `docs/NPM_USAGE.md` – CLI usage and flags
473+
- `GITHUB_CHECKS.md` – Checks, outputs, and workflow integration
328474
- `examples/` – MCP, Jira, and advanced configs

0 commit comments

Comments
 (0)