Skip to content

Commit 7f4acca

Browse files
Copilottikazyq
andcommitted
Address log format differences with Agent Adapter Pattern
Add comprehensive solution for handling different AI agent log formats: - Implement pluggable Agent Adapter Pattern with base interface - Add detailed adapter examples (Copilot, Claude) - Update implementation checklist with adapter development tasks - Expand architecture diagrams to show adapter layer - Document adapter development guide and strategy Co-authored-by: tikazyq <[email protected]>
1 parent bfc3765 commit 7f4acca

File tree

4 files changed

+261
-13
lines changed

4 files changed

+261
-13
lines changed

docs/design/ai-agent-observability-design.md

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,202 @@ class AgentEventCollectionService {
213213
}
214214
```
215215

216+
#### Handling Different Agent Log Formats
217+
218+
**Challenge**: Each AI coding tool (GitHub Copilot, Cursor, Claude Code, etc.) produces logs in different formats with varying structures, fields, and semantics.
219+
220+
**Solution: Agent Adapter Pattern**
221+
222+
We implement a pluggable adapter pattern where each AI agent has a dedicated adapter that translates its native log format into our standardized `AgentEvent` schema:
223+
224+
```typescript
225+
// Base adapter interface
226+
interface AgentAdapter {
227+
agentId: string;
228+
agentVersion: string;
229+
230+
// Parse raw log entry to standard event
231+
parseEvent(rawLog: any): AgentEvent | null;
232+
233+
// Validate if this adapter can handle the log
234+
canHandle(rawLog: any): boolean;
235+
236+
// Extract session information
237+
extractSessionInfo(rawLogs: any[]): SessionInfo;
238+
}
239+
240+
// Example: GitHub Copilot Adapter
241+
class CopilotAdapter implements AgentAdapter {
242+
agentId = 'github-copilot';
243+
agentVersion = '1.x';
244+
245+
parseEvent(rawLog: CopilotLogEntry): AgentEvent | null {
246+
// Copilot-specific log format:
247+
// { timestamp, action, file, completion, metadata }
248+
249+
return {
250+
id: generateEventId(rawLog),
251+
timestamp: rawLog.timestamp,
252+
type: this.mapActionToEventType(rawLog.action),
253+
agentId: this.agentId,
254+
agentVersion: this.agentVersion,
255+
sessionId: this.extractSessionId(rawLog),
256+
projectId: this.extractProjectId(rawLog),
257+
context: {
258+
filePath: rawLog.file,
259+
workingDirectory: rawLog.metadata?.cwd,
260+
},
261+
data: {
262+
completion: rawLog.completion,
263+
accepted: rawLog.metadata?.accepted,
264+
},
265+
metrics: {
266+
tokenCount: rawLog.metadata?.tokens,
267+
},
268+
};
269+
}
270+
271+
canHandle(rawLog: any): boolean {
272+
return rawLog.source === 'copilot' ||
273+
rawLog.agent === 'github-copilot';
274+
}
275+
276+
private mapActionToEventType(action: string): AgentEventType {
277+
const mapping = {
278+
'completion': 'llm_response',
279+
'file_edit': 'file_write',
280+
'command': 'command_execute',
281+
// ... more mappings
282+
};
283+
return mapping[action] || 'user_interaction';
284+
}
285+
}
286+
287+
// Example: Claude Code Adapter
288+
class ClaudeAdapter implements AgentAdapter {
289+
agentId = 'claude-code';
290+
agentVersion = '1.x';
291+
292+
parseEvent(rawLog: ClaudeLogEntry): AgentEvent | null {
293+
// Claude-specific log format:
294+
// { time, event_type, tool_use, content, metadata }
295+
296+
return {
297+
id: generateEventId(rawLog),
298+
timestamp: rawLog.time,
299+
type: this.mapEventType(rawLog.event_type),
300+
agentId: this.agentId,
301+
agentVersion: this.agentVersion,
302+
sessionId: this.extractSessionId(rawLog),
303+
projectId: this.extractProjectId(rawLog),
304+
context: {
305+
filePath: rawLog.tool_use?.path,
306+
workingDirectory: rawLog.metadata?.working_dir,
307+
},
308+
data: {
309+
toolName: rawLog.tool_use?.tool_name,
310+
content: rawLog.content,
311+
},
312+
metrics: {
313+
tokenCount: rawLog.metadata?.input_tokens + rawLog.metadata?.output_tokens,
314+
},
315+
};
316+
}
317+
318+
canHandle(rawLog: any): boolean {
319+
return rawLog.provider === 'anthropic' ||
320+
rawLog.model?.includes('claude');
321+
}
322+
323+
private mapEventType(eventType: string): AgentEventType {
324+
const mapping = {
325+
'tool_use': 'tool_invocation',
326+
'text_generation': 'llm_response',
327+
'file_operation': 'file_write',
328+
// ... more mappings
329+
};
330+
return mapping[eventType] || 'user_interaction';
331+
}
332+
}
333+
334+
// Adapter Registry
335+
class AgentAdapterRegistry {
336+
private adapters: Map<string, AgentAdapter> = new Map();
337+
338+
register(adapter: AgentAdapter): void {
339+
this.adapters.set(adapter.agentId, adapter);
340+
}
341+
342+
getAdapter(agentId: string): AgentAdapter | null {
343+
return this.adapters.get(agentId) || null;
344+
}
345+
346+
detectAdapter(rawLog: any): AgentAdapter | null {
347+
for (const adapter of this.adapters.values()) {
348+
if (adapter.canHandle(rawLog)) {
349+
return adapter;
350+
}
351+
}
352+
return null;
353+
}
354+
}
355+
356+
// Usage in collection service
357+
class AgentEventCollectionService {
358+
private adapterRegistry: AgentAdapterRegistry;
359+
360+
async collectRawLog(rawLog: any): Promise<void> {
361+
// Auto-detect which adapter to use
362+
const adapter = this.adapterRegistry.detectAdapter(rawLog);
363+
364+
if (!adapter) {
365+
console.warn('No adapter found for log:', rawLog);
366+
return;
367+
}
368+
369+
// Parse to standard format
370+
const event = adapter.parseEvent(rawLog);
371+
372+
if (event) {
373+
await this.collectEvent(event);
374+
}
375+
}
376+
}
377+
```
378+
379+
**Adapter Implementation Strategy**:
380+
381+
1. **Phase 1 Adapters** (Weeks 1-4):
382+
- GitHub Copilot adapter
383+
- Claude Code adapter
384+
- Generic MCP adapter (fallback)
385+
386+
2. **Phase 2 Adapters** (Weeks 5-8):
387+
- Cursor adapter
388+
- Gemini CLI adapter
389+
- Cline adapter
390+
391+
3. **Phase 3+ Adapters**:
392+
- Aider adapter
393+
- Community-contributed adapters
394+
- Custom enterprise adapters
395+
396+
**Benefits of Adapter Pattern**:
397+
- **Extensibility**: Easy to add new agents without changing core code
398+
- **Maintainability**: Each adapter is isolated and can evolve independently
399+
- **Testability**: Adapters can be unit tested with sample logs
400+
- **Flexibility**: Adapters can handle version differences and format variations
401+
- **Community**: Open for community contributions of new adapters
402+
403+
**Adapter Development Guide**:
404+
Each adapter implementation should:
405+
1. Study the agent's log format (JSON, plain text, structured logs)
406+
2. Identify key fields and their semantics
407+
3. Map agent-specific event types to standard `AgentEventType`
408+
4. Handle missing or optional fields gracefully
409+
5. Preserve agent-specific metadata in the `data` field
410+
6. Include comprehensive unit tests with real log samples
411+
216412
#### 1.2 Agent Session Management
217413
**Objective**: Track complete agent working sessions with full context
218414

docs/design/ai-agent-observability-executive-summary.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@ Value: Meet regulatory requirements, reduce risk
9898

9999
### Collection Layer
100100
- Universal event schema for all AI agents
101+
- **Agent Adapter Pattern**: Pluggable adapters normalize different log formats
101102
- Real-time event capture (>10k events/sec)
102103
- Multiple collection methods (MCP, logs, APIs)
103104
- Automatic context enrichment

docs/design/ai-agent-observability-implementation-checklist.md

Lines changed: 41 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -95,12 +95,29 @@ This document provides a detailed, actionable checklist for implementing the AI
9595
- [ ] Write tool documentation
9696
- [ ] Add integration tests
9797

98-
- [ ] **Event Collector for GitHub Copilot**
99-
- [ ] Create `packages/mcp/src/collectors/copilot-collector.ts`
100-
- [ ] Implement log file monitoring
101-
- [ ] Implement event parsing
102-
- [ ] Map Copilot events to standard schema
103-
- [ ] Test with real Copilot sessions
98+
- [ ] **Agent Adapter Pattern Infrastructure**
99+
- [ ] Create `packages/core/src/adapters/agent-adapter.ts` (base interface)
100+
- [ ] Create `packages/core/src/adapters/adapter-registry.ts`
101+
- [ ] Implement adapter detection logic
102+
- [ ] Add adapter testing utilities
103+
- [ ] Write adapter development guide
104+
105+
- [ ] **GitHub Copilot Adapter**
106+
- [ ] Create `packages/core/src/adapters/copilot-adapter.ts`
107+
- [ ] Study Copilot log format (JSON structure, fields)
108+
- [ ] Implement `parseEvent()` with Copilot-specific parsing
109+
- [ ] Implement `canHandle()` for Copilot log detection
110+
- [ ] Map Copilot actions to standard event types
111+
- [ ] Write unit tests with real Copilot log samples
112+
- [ ] Test with live Copilot sessions
113+
114+
- [ ] **Claude Code Adapter**
115+
- [ ] Create `packages/core/src/adapters/claude-adapter.ts`
116+
- [ ] Study Claude Code log format
117+
- [ ] Implement parsing for Claude-specific fields
118+
- [ ] Map Claude event types to standard schema
119+
- [ ] Write unit tests with Claude log samples
120+
- [ ] Test with live Claude Code sessions
104121

105122
- [ ] **Basic Event Viewer UI**
106123
- [ ] Create `apps/web/src/app/projects/[name]/agent-events/page.tsx`
@@ -316,6 +333,13 @@ This document provides a detailed, actionable checklist for implementing the AI
316333

317334
### Week 12: Comparative Analysis & Reporting
318335

336+
- [ ] **Additional Agent Adapters**
337+
- [ ] Create Cursor adapter (`packages/core/src/adapters/cursor-adapter.ts`)
338+
- [ ] Create Gemini CLI adapter (`packages/core/src/adapters/gemini-adapter.ts`)
339+
- [ ] Create Cline adapter (`packages/core/src/adapters/cline-adapter.ts`)
340+
- [ ] Add adapter version detection
341+
- [ ] Update adapter registry with new adapters
342+
319343
- [ ] **Agent Comparison**
320344
- [ ] Create `AgentComparison` component
321345
- [ ] Compare performance metrics
@@ -522,12 +546,17 @@ This document provides a detailed, actionable checklist for implementing the AI
522546
- [ ] Elastic auto-scaling
523547
- [ ] Cold storage archival
524548

525-
### Additional Agents
526-
- [ ] Windsurf integration
527-
- [ ] Continue.dev integration
528-
- [ ] Tabnine integration
529-
- [ ] Cody integration
530-
- [ ] Amazon Q integration
549+
### Additional Agents & Adapters
550+
- [ ] Aider adapter
551+
- [ ] Windsurf adapter
552+
- [ ] Continue.dev adapter
553+
- [ ] Tabnine adapter
554+
- [ ] Cody adapter
555+
- [ ] Amazon Q adapter
556+
- [ ] Generic log format adapter (for unknown agents)
557+
- [ ] Adapter versioning system (handle format changes)
558+
- [ ] Community adapter contribution guidelines
559+
- [ ] Adapter marketplace/registry
531560

532561
## Testing Checklist
533562

docs/design/ai-agent-observability-quick-reference.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,11 @@ AI Agent Observability provides complete visibility into AI coding agent activit
7373
┌─────────────────────┐
7474
│ AI Agents │ (Copilot, Claude, Cursor, etc.)
7575
└──────────┬──────────┘
76-
│ MCP / SDKs
76+
│ Different log formats
77+
┌──────────▼──────────┐
78+
│ Agent Adapters │ (Normalize to standard schema)
79+
└──────────┬──────────┘
80+
│ Standard events
7781
┌──────────▼──────────┐
7882
│ Event Collection │ (Real-time capture)
7983
└──────────┬──────────┘
@@ -91,6 +95,24 @@ AI Agent Observability provides complete visibility into AI coding agent activit
9195
└─────────────────────┘
9296
```
9397

98+
## Handling Different Agent Log Formats
99+
100+
**Challenge**: Each AI tool has its own log format
101+
**Solution**: Agent Adapter Pattern
102+
103+
Each agent gets a dedicated adapter that translates its native log format into our standardized event schema:
104+
105+
- **Copilot Adapter**: Handles GitHub Copilot's JSON logs
106+
- **Claude Adapter**: Handles Claude Code's event format
107+
- **Cursor Adapter**: Handles Cursor's log structure
108+
- **Generic Adapter**: Fallback for unknown formats
109+
110+
Benefits:
111+
- Easy to add new agents
112+
- Isolated, maintainable code
113+
- Version handling per agent
114+
- Community contributions welcome
115+
94116
## Event Types
95117

96118
Core events captured from AI agents:

0 commit comments

Comments
 (0)