Skip to content

Commit 348f8b4

Browse files
Dwij1704bboynton97
andauthored
Smolagents instrumentation (#1070)
* Add SmoLAgents instrumentation * Remove deprecated instrumentor definitions from `__init__.py` to streamline the instrumentation module. This change enhances code clarity and prepares for future updates. * Add SmoLAgents instrumentation * Refactor multi_smolagents_system notebook to streamline agent management. Removed ManagedAgent and adjusted CodeAgent to directly manage web_agent, enhancing clarity and simplifying the agent structure. * Add a blank line for improved readability in `__init__.py` * Refactor SmoLAgents instrumentation for improved clarity and consistency. Updated class names and removed unused dynamic span naming functions. Simplified attribute extraction for agents and models, enhancing observability in telemetry data. * feat: Add multi-agent system notebook example * fix: Update execution counts and correct session ending method in multi-agent notebook * fix: Correct markdown formatting and remove deprecated multi-agent example notebook * feat: Add Smolagents integration documentation and examples * fix: Add missing newline before InstrumentorLoader class definition * feat: Add smolagents integration card to examples documentation * feat: Update examples and introduction documentation to include Smolagents integration --------- Co-authored-by: Braelyn Boynton <[email protected]>
1 parent d81108d commit 348f8b4

File tree

15 files changed

+1691
-47
lines changed

15 files changed

+1691
-47
lines changed

agentops/instrumentation/common/attributes.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,3 +252,27 @@ def get_base_span_attributes(span: Any) -> AttributeMap:
252252
attributes[CoreAttributes.PARENT_ID] = parent_id
253253

254254
return attributes
255+
256+
257+
def extract_token_usage(response: Any) -> Dict[str, int]:
258+
"""Extract token usage information from a response.
259+
260+
Args:
261+
response: The response object to extract token usage from
262+
263+
Returns:
264+
Dictionary containing token usage information
265+
"""
266+
usage = {}
267+
268+
# Try to extract token counts from response
269+
if hasattr(response, "usage"):
270+
usage_data = response.usage
271+
if hasattr(usage_data, "prompt_tokens"):
272+
usage["prompt_tokens"] = usage_data.prompt_tokens
273+
if hasattr(usage_data, "completion_tokens"):
274+
usage["completion_tokens"] = usage_data.completion_tokens
275+
if hasattr(usage_data, "total_tokens"):
276+
usage["total_tokens"] = usage_data.total_tokens
277+
278+
return usage
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# SmoLAgents Instrumentation
2+
3+
This module provides OpenTelemetry instrumentation for the SmoLAgents framework. It captures telemetry data from model operations, agent executions, and tool usage.
4+
5+
## Features
6+
7+
- Model operation tracking
8+
- Text generation
9+
- Token usage
10+
- Streaming responses
11+
- Latency metrics
12+
13+
- Agent execution monitoring
14+
- Step-by-step execution
15+
- Planning phases
16+
- Tool usage
17+
- Execution time
18+
19+
- Tool usage analytics
20+
- Tool call patterns
21+
- Success/failure rates
22+
- Execution time
23+
- Error tracking
24+
25+
## Usage
26+
27+
```python
28+
from agentops import init
29+
from agentops.instrumentation.smolagents import SmoLAgentsInstrumentor
30+
31+
# Initialize AgentOps with your API key
32+
init(api_key="your-api-key")
33+
34+
# The instrumentation will be automatically activated
35+
# All SmoLAgents operations will now be tracked
36+
```
37+
38+
## Metrics Collected
39+
40+
1. Token Usage
41+
- Input tokens
42+
- Output tokens
43+
- Total tokens per operation
44+
45+
2. Timing Metrics
46+
- Operation duration
47+
- Time to first token (streaming)
48+
- Tool execution time
49+
- Planning phase duration
50+
51+
3. Agent Metrics
52+
- Step counts
53+
- Planning steps
54+
- Tools used
55+
- Success/failure rates
56+
57+
4. Error Tracking
58+
- Generation errors
59+
- Tool execution errors
60+
- Parsing errors
61+
62+
## Architecture
63+
64+
The instrumentation is built on OpenTelemetry and follows the same pattern as other AgentOps instrumentors:
65+
66+
1. Attribute Extractors
67+
- Model attributes
68+
- Agent attributes
69+
- Tool call attributes
70+
71+
2. Wrappers
72+
- Method wrappers for sync operations
73+
- Stream wrappers for async operations
74+
- Context propagation handling
75+
76+
3. Metrics
77+
- Histograms for distributions
78+
- Counters for events
79+
- Custom attributes for filtering
80+
81+
## Contributing
82+
83+
When adding new features or modifying existing ones:
84+
85+
1. Follow the established pattern for attribute extraction
86+
2. Maintain context propagation
87+
3. Add appropriate error handling
88+
4. Update tests and documentation
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""SmoLAgents instrumentation for AgentOps."""
2+
3+
LIBRARY_NAME = "smolagents"
4+
LIBRARY_VERSION = "1.16.0"
5+
6+
from agentops.instrumentation.smolagents.instrumentor import SmolAgentsInstrumentor # noqa: E402
7+
8+
__all__ = ["SmolAgentsInstrumentor"]

0 commit comments

Comments
 (0)