Skip to content

Commit f8fc0d2

Browse files
ANcpLuaclaude
andauthored
feat(qyl-instrumentation): Teams API orchestration + competition polish (#149)
* docs: add spec-0002 qyl Claude Code observability Comprehensive spec for building Claude Code session observability into qyl's AI telemetry dashboard. Zero-instrumentation approach using Claude Code's native OTLP telemetry export (4 env vars). Covers: OTLP data flow, DuckDB schema, 5 API endpoints, React hooks, 4 dashboard components, SSE live streaming, and 4-phase implementation plan. Correlation via prompt.id across all events. Also fixes CLAUDE.md skill count (6 → 4 after council/invoke and feature-dev/code-review skill deletions). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(qyl-instrumentation): rebuild as Teams API orchestration (v2.0.0) Restructure from 3 standalone agents to 1 Opus captain + 4 Sonnet specialists. Captain pre-reads otelwiki bundled semconv docs before spawning specialists — eliminates runtime web search/fetch. New /observe command implements full TeamCreate → spawn → cross-pollinate → synthesize → TeamDelete lifecycle. New: opus-captain agent, qyl-platform-specialist agent, /observe command. Changed: removed WebSearch/WebFetch from all specialist tools, added Team Protocol sections with SendMessage coordination patterns. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(qyl-instrumentation): competition-ready polish (v2.1.0) Add hero scenario (proactive secretary), 8-layer trace example, attribute decision tree, multi-turn agent traces, GenAI failure modes, TypeSpec-to-dashboard flow, MCP/SSE patterns, SEMCONV_CONTEXT shape, verification checklists, and example run walkthrough across all 6 agent/command files. All stay under 250 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent ae3a154 commit f8fc0d2

File tree

10 files changed

+1362
-1
lines changed

10 files changed

+1362
-1
lines changed

.claude-plugin/marketplace.json

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"url": "https://github.com/ANcpLua"
66
},
77
"metadata": {
8-
"description": "Claude Code plugin marketplace: 8 plugins, 23 commands, 4 skills, 14 agents. Multi-agent orchestration, cognitive amplification, OpenTelemetry docs, and .NET build enforcement."
8+
"description": "Claude Code plugin marketplace: 9 plugins, 24 commands, 4 skills, 19 agents. Multi-agent orchestration, cognitive amplification, OpenTelemetry docs, .NET build enforcement, and compile-time instrumentation."
99
},
1010
"plugins": [
1111
{
@@ -55,6 +55,12 @@
5555
"description": "Multi-agent workflow orchestration (9 commands + 2 skills: eight-gates, hades): fix, turbo-fix, fix-pipeline, tournament, mega-swarm, deep-think, batch-implement, red-blue-review, baryon-mode. Hades, eight-gates, and red-blue-review use Teams API for reactive collaboration.",
5656
"version": "2.1.0",
5757
"source": "./plugins/exodia"
58+
},
59+
{
60+
"name": "qyl-instrumentation",
61+
"description": "Teams API observability orchestration for qyl: 1 Opus captain + 4 Sonnet specialists. Captain pre-reads otelwiki semconv docs, dispatches servicedefaults, collector, genai, and platform specialists. Zero runtime web search.",
62+
"version": "2.1.0",
63+
"source": "./plugins/qyl-instrumentation"
5864
}
5965
]
6066
}

CHANGELOG.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,24 @@ and the project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
1212

1313
### Changed
1414

15+
- **`qyl-instrumentation` (2.0.0 → 2.1.0)**: Competition-ready polish for all 6 agent/command files. Added hero scenario (proactive secretary notification handler), 8-layer trace example, attribute decision tree, performance profile, multi-turn agent trace, GenAI failure modes with 15-second async window, TypeSpec-to-dashboard end-to-end flow, MCP tool pattern, SSE consumption pattern, SEMCONV_CONTEXT shape, spawn/synthesis verification checklists, and example run walkthrough. All files stay under 250 lines
16+
17+
### Added
18+
19+
- **`docs/specs/spec-0002-qyl-claude-code-observability.md`**: Comprehensive spec for building Claude Code session observability into qyl's AI telemetry dashboard. Covers OTLP data flow (native `claude_code.*` metrics + events), DuckDB schema, 5 API endpoints, React hooks, 4 dashboard components, SSE live streaming, and 4-phase implementation plan. Zero-instrumentation approach — uses Claude Code's built-in OTLP telemetry export via 4 env vars
20+
- **`qyl-instrumentation/commands/observe.md`**: Teams API orchestration command — Opus captain pre-reads otelwiki bundled semconv docs, assembles SEMCONV_CONTEXT + SHARED_AWARENESS, spawns 4 Sonnet specialists in parallel, coordinates cross-pollination via SendMessage, synthesizes. Zero runtime web search
21+
- **`qyl-instrumentation/agents/opus-captain.md`**: Opus captain agent — orchestrates context assembly and team coordination, reads otelwiki docs before any specialist spawns
22+
- **`qyl-instrumentation/agents/qyl-platform-specialist.md`**: 4th Sonnet specialist covering MCP server, React dashboard, browser OTLP SDK, SSE streaming, and Copilot extensibility
23+
24+
### Changed
25+
26+
- **`qyl-instrumentation`**: Rebuilt from 3 standalone agents (v1.0.0) to Teams API orchestration (v2.0.0). 1 Opus captain + 4 Sonnet specialists. Captain pre-reads otelwiki bundled docs — specialists receive pre-assembled semconv context in spawn prompts instead of web searching at runtime
27+
- **`qyl-instrumentation` agents**: Removed `WebSearch` and `WebFetch` from all 3 existing specialist tool lists. Added Team Protocol sections documenting SendMessage coordination patterns and SEMCONV_CONTEXT injection
28+
- **`qyl-instrumentation/agents/otel-genai-architect.md`**: Convention verification now references captain's SEMCONV_CONTEXT instead of WebSearch
29+
- **`marketplace.json`**: Updated qyl-instrumentation description and version (1.0.0 → 2.0.0), agent count 17 → 19, command count 23 → 24
30+
31+
### Changed
32+
1533
- **`exodia/skills/hades`**: Migrated from vague Teams references to explicit Teams API. SKILL.md now uses `TeamCreate`, `TeamDelete`, `SendMessage` (shutdown_request/shutdown_response), `TaskCreate`/`TaskList`/`TaskUpdate` with explicit parameters. Removed fallback subagent path and duplicate STEP -1 block. All 4 teammate templates (auditors, eliminators, verifiers, goggles) updated: vague `MESSAGE``SendMessage (recipient: "...")`, vague task list → `TaskCreate`/`TaskUpdate`, team context preamble and shutdown protocol added
1634
- **`exodia/eight-gates` Gate 7 EXECUTE**: Removed dual Mode A (Task subagents) / Mode B (Agent Teams) pattern. Teams API is now the single execution mode. Lane workers coordinate via `SendMessage` and claim work via `TaskCreate`/`TaskUpdate`. Collision avoidance uses teammate messaging
1735
- **`exodia/skills/hades` allowed-tools**: Added `TeamCreate`, `TeamDelete`, `TaskCreate`, `TaskList`, `TaskUpdate`, `SendMessage` to frontmatter
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"name": "qyl-instrumentation",
3+
"version": "2.1.0",
4+
"description": "Teams API observability orchestration for qyl: 1 Opus captain pre-reads otelwiki semconv docs, dispatches 4 Sonnet specialists (servicedefaults, collector, genai, platform) via /observe command. Zero runtime web search.",
5+
"author": {
6+
"name": "ANcpLua",
7+
"email": ""
8+
},
9+
"repository": "https://github.com/ANcpLua/ancplua-claude-plugins",
10+
"license": "MIT",
11+
"keywords": [
12+
"qyl",
13+
"observability",
14+
"opentelemetry",
15+
"teams-api",
16+
"instrumentation",
17+
"genai"
18+
],
19+
"commands": "./commands"
20+
}
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# qyl-instrumentation
2+
3+
Teams API observability orchestration for qyl's compile-time OTel instrumentation system.
4+
5+
## Architecture
6+
7+
```text
8+
/qyl-instrumentation:observe [task]
9+
10+
v
11+
captain (Opus) ── pre-reads otelwiki bundled semconv docs
12+
│ assembles SEMCONV_CONTEXT + SHARED_AWARENESS
13+
14+
├── servicedefaults (Sonnet) ── compile-time attributes, generators
15+
├── collector (Sonnet) ──────── OTLP ingestion, DuckDB, API, TypeSpec
16+
├── genai (Sonnet) ──────────── GenAI semconv, agent/LLM tracing
17+
└── platform (Sonnet) ───────── MCP server, dashboard, browser SDK
18+
│ │ │ │
19+
└────────────────┴───────────────┴────────────────┘
20+
cross-pollinate via SendMessage
21+
```
22+
23+
## Key Innovation
24+
25+
**Zero runtime web search.** The Opus captain pre-reads otelwiki's bundled semantic
26+
convention docs BEFORE spawning specialists. Each Sonnet specialist receives
27+
pre-assembled semconv context in their spawn prompt — they never need to web search
28+
or web fetch.
29+
30+
This means:
31+
32+
- Specialists work with VERIFIED, CURRENT attribute data (from otelwiki's synced docs)
33+
- No runtime latency from web searches
34+
- No hallucinated attribute names from stale training data
35+
- Captain verifies doc freshness — warns if otelwiki needs sync
36+
37+
## What This Plugin Provides
38+
39+
### Command
40+
41+
| Command | Purpose |
42+
|---------|---------|
43+
| `/qyl-instrumentation:observe` | Full team orchestration — context assembly, spawn, coordinate, synthesize |
44+
45+
### Agents
46+
47+
| Agent | subagent_type | Model | Domain |
48+
|-------|---------------|-------|--------|
49+
| Opus Captain | `opus-captain` | Opus 4.6 | Orchestration, context assembly |
50+
| ServiceDefaults Specialist | `servicedefaults-specialist` | Sonnet 4.6 | Source generator, attribute vocabulary, interceptor pipelines |
51+
| qyl Observability Specialist | `qyl-observability-specialist` | Sonnet 4.6 | OTLP ingestion, DuckDB, collector architecture, TypeSpec schemas |
52+
| OTel GenAI Architect | `otel-genai-architect` | Sonnet 4.6 | GenAI semantic conventions, agent tracing, LLM telemetry |
53+
| qyl Platform Specialist | `qyl-platform-specialist` | Sonnet 4.6 | MCP server, dashboard, browser SDK, SSE streaming |
54+
55+
## Usage
56+
57+
Full team orchestration:
58+
59+
```text
60+
/qyl-instrumentation:observe Add tracing to the new OrderService with GenAI telemetry
61+
/qyl-instrumentation:observe Design DuckDB schema for agent invocation metrics
62+
/qyl-instrumentation:observe Instrument the MCP server with request/response tracing
63+
```
64+
65+
Individual specialist (bypasses team, no pre-fetched context):
66+
67+
```text
68+
Task(subagent_type="qyl-instrumentation:servicedefaults-specialist", prompt="Add [Traced] to PaymentService")
69+
Task(subagent_type="qyl-instrumentation:otel-genai-architect", prompt="Validate GenAI attributes against semconv")
70+
```
71+
72+
## Shared Awareness
73+
74+
All agents are aware of:
75+
76+
- **AI monitoring use cases:** silent failures, malformed responses, performance
77+
bottlenecks, agent loops, cost attribution, context overflow, tool failures,
78+
safety violations
79+
- **Quality criteria:** accuracy (20-25%), reasoning (20-25%), creativity (15%),
80+
UX (15%), reliability/safety (20%)
81+
- **Enterprise integration:** External MCP (8pts), Connected Agents Architecture
82+
(15pts), OAuth (5pts), Adaptive Cards (5pts)
83+
84+
## Dependencies
85+
86+
- **otelwiki plugin** — captain reads bundled docs for semconv context (graceful degradation if missing)
87+
- **ancplua-project-routing** — detects qyl workspace and injects agent types into session
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
name: opus-captain
3+
description: >-
4+
qyl observability captain. Pre-reads otelwiki bundled semconv docs, creates team,
5+
dispatches 4 Sonnet specialists with pre-assembled context, coordinates cross-pollination,
6+
synthesizes final output. Uses claude-opus-4-6.
7+
model: claude-opus-4-6
8+
tools:
9+
- Task
10+
- Read
11+
- Grep
12+
- Glob
13+
- Bash
14+
---
15+
16+
# qyl Observability Captain
17+
18+
You orchestrate the qyl observability team. Your job is context assembly and coordination — you never implement directly.
19+
20+
## Protocol
21+
22+
1. **Pre-read** otelwiki bundled docs
23+
(Glob for `**/otelwiki/docs/semantic-conventions/gen-ai/*.md` and key instrumentation files)
24+
2. **Assemble** compact SEMCONV_CONTEXT from the attribute tables and conventions you read
25+
3. **Create team** via TeamCreate
26+
4. **Spawn** 4 Sonnet specialists in ONE message, injecting SEMCONV_CONTEXT + SHARED_AWARENESS into each prompt
27+
5. **Coordinate** — read messages, create follow-up tasks, resolve cross-cutting conflicts
28+
6. **Synthesize** — verify consistency across all specialists' work
29+
7. **Shutdown** — send shutdown_request to all, then TeamDelete
30+
31+
## Specialists
32+
33+
| Name | subagent_type | Domain |
34+
|------|---------------|--------|
35+
| servicedefaults | `qyl-instrumentation:servicedefaults-specialist` | Compile-time instrumentation, attributes, generators |
36+
| collector | `qyl-instrumentation:qyl-observability-specialist` | OTLP ingestion, DuckDB, API, TypeSpec |
37+
| genai | `qyl-instrumentation:otel-genai-architect` | GenAI semconv, agent/LLM tracing |
38+
| platform | `qyl-instrumentation:qyl-platform-specialist` | MCP server, dashboard, browser SDK |
39+
40+
## SEMCONV_CONTEXT Shape
41+
42+
Compact attribute listing assembled from otelwiki bundled docs:
43+
44+
```text
45+
GENAI TRACE: gen_ai.system, gen_ai.request.model,
46+
gen_ai.response.model, gen_ai.request.max_tokens,
47+
gen_ai.request.temperature, gen_ai.request.top_p,
48+
gen_ai.usage.input_tokens, gen_ai.usage.output_tokens,
49+
gen_ai.response.finish_reasons, gen_ai.operation.name
50+
51+
AGENT: gen_ai.agent.name, gen_ai.agent.description,
52+
gen_ai.tool.name, gen_ai.tool.description,
53+
gen_ai.tool.call.id
54+
55+
METRICS: gen_ai.client.token.usage (Histogram, {token}),
56+
gen_ai.client.operation.duration (Histogram, s)
57+
58+
DEPRECATED (do NOT use):
59+
ai.model.id → gen_ai.request.model,
60+
ai.usage.tokens → gen_ai.usage.input_tokens + output_tokens
61+
```
62+
63+
Rule: only include attributes that qyl's generator actually emits. If a
64+
specialist references an attribute not in SEMCONV_CONTEXT, captain must
65+
verify it exists in otelwiki docs before approving.
66+
67+
## Spawn Verification Checklist
68+
69+
**Before spawn:**
70+
71+
- [ ] SEMCONV_CONTEXT assembled from otelwiki docs (not from memory)
72+
- [ ] Freshness verified (VERSION.md < 30 days)
73+
- [ ] Task decomposed into specialist-appropriate work items
74+
- [ ] SHARED_AWARENESS block included in each spawn prompt
75+
76+
**After convergence:**
77+
78+
- [ ] Attribute names consistent across all specialists
79+
- [ ] No deprecated attribute names used
80+
- [ ] Build verified (`nuke` compiles without errors)
81+
- [ ] Cross-specialist conflicts resolved
82+
83+
## Key Rule
84+
85+
You are the ONLY agent that reads otelwiki docs. Specialists receive semconv knowledge
86+
FROM YOU in their spawn prompts. They do not web search or web fetch — you eliminate that need.

0 commit comments

Comments
 (0)