OpenAI Agents Framework instrumentation (#917)

Co-authored-by: Alex Hall <[email protected]>

Author: samuelcolvin

docs/images/logfire-screenshot-openai-agents-tools.png

@@ -4,7 +4,12 @@ integration: logfire
 
 ## Introduction
 
-Logfire supports instrumenting calls to OpenAI with one extra line of code.
+We support instrumenting both the [standard OpenAI SDK](https://github.com/openai/openai-python) package and [OpenAI "agents"](https://github.com/openai/openai-agents-python) framework.
+
+### OpenAI SDK
+
+Logfire supports instrumenting calls to OpenAI with one extra line of code, here's an example of instrumenting
+the OpenAI SDK:
 
 ```python hl_lines="7"
 import openai
@@ -45,14 +50,15 @@ With that you get:
   <figcaption>Span arguments including response details</figcaption>
 </figure>
 
-## Methods covered
+### Methods covered
 
 The following OpenAI methods are covered:
 
 - [`client.chat.completions.create`](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) — with and without `stream=True`
 - [`client.completions.create`](https://platform.openai.com/docs/guides/text-generation/completions-api) — with and without `stream=True`
 - [`client.embeddings.create`](https://platform.openai.com/docs/guides/embeddings/how-to-get-embeddings)
 - [`client.images.generate`](https://platform.openai.com/docs/guides/images/generations)
+- [`client.responses.create`](https://platform.openai.com/docs/api-reference/responses)
 
 All methods are covered with both `openai.Client` and `openai.AsyncClient`.
 
@@ -87,7 +93,7 @@ Gives:
   <figcaption>OpenAI image generation span</figcaption>
 </figure>
 
-## Streaming Responses
+### Streaming Responses
 
 When instrumenting streaming responses, Logfire creates two spans — one around the initial request and one
 around the streamed response.
@@ -134,3 +140,81 @@ Shows up like this in Logfire:
   ![Logfire OpenAI Streaming](../../images/logfire-screenshot-openai-stream.png){ width="500" }
   <figcaption>OpenAI streaming response</figcaption>
 </figure>
+
+## OpenAI Agents
+
+We also support instrumenting the [OpenAI "agents"](https://github.com/openai/openai-agents-python) framework.
+
+```python hl_lines="5"
+import logfire
+from agents import Agent, Runner
+
+logfire.configure()
+logfire.instrument_openai_agents()
+
+agent = Agent(name="Assistant", instructions="You are a helpful assistant")
+
+result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
+print(result.final_output)
+```
+
+_For more information, see the [`instrument_openai_agents()` API reference][logfire.Logfire.instrument_openai_agents]._
+
+Which shows up like this in Logfire:
+
+<figure markdown="span">
+  ![Logfire OpenAI Agents](../../images/logfire-screenshot-openai-agents.png){ width="500" }
+  <figcaption>OpenAI Agents</figcaption>
+</figure>
+
+In this example we add a function tool to the agents:
+
+```python
+from typing_extensions import TypedDict
+
+import logfire
+from httpx import AsyncClient
+from agents import RunContextWrapper, Agent, function_tool, Runner
+
+logfire.configure()
+logfire.instrument_openai_agents()
+
+
+class Location(TypedDict):
+    lat: float
+    long: float
+
+
+@function_tool
+async def fetch_weather(ctx: RunContextWrapper[AsyncClient], location: Location) -> str:
+    """Fetch the weather for a given location.
+
+    Args:
+        ctx: Run context object.
+        location: The location to fetch the weather for.
+    """
+    r = await ctx.context.get('https://httpbin.org/get', params=location)
+    return 'sunny' if r.status_code == 200 else 'rainy'
+
+
+agent = Agent(name='weather agent', tools=[fetch_weather])
+
+
+async def main():
+    async with AsyncClient() as client:
+        logfire.instrument_httpx(client)
+        result = await Runner.run(agent, 'Get the weather at lat=51 lng=0.2', context=client)
+    print(result.final_output)
+
+
+if __name__ == '__main__':
+    import asyncio
+    asyncio.run(main())
+```
+
+We see spans from within the function call nested within the agent spans:
+
+<figure markdown="span">
+  ![Logfire OpenAI Agents](../../images/logfire-screenshot-openai-agents-tools.png){ width="500" }
+  <figcaption>OpenAI Agents</figcaption>
+</figure>

docs/images/logfire-screenshot-openai-agents.png

@@ -139,6 +139,8 @@ def instrument_anthropic(self, *args, **kwargs) -> ContextManager[None]:
             def instrument_openai(self, *args, **kwargs) -> ContextManager[None]:
                 return nullcontext()
 
+            def instrument_openai_agents(self, *args, **kwargs) -> None: ...
+
             def instrument_aiohttp_client(self, *args, **kwargs) -> None: ...
 
             def instrument_system_metrics(self, *args, **kwargs) -> None: ...
@@ -168,6 +170,7 @@ def shutdown(self, *args, **kwargs) -> None: ...
         instrument_pydantic = DEFAULT_LOGFIRE_INSTANCE.instrument_pydantic
         instrument_fastapi = DEFAULT_LOGFIRE_INSTANCE.instrument_fastapi
         instrument_openai = DEFAULT_LOGFIRE_INSTANCE.instrument_openai
+        instrument_openai_agents = DEFAULT_LOGFIRE_INSTANCE.instrument_openai_agents
         instrument_anthropic = DEFAULT_LOGFIRE_INSTANCE.instrument_anthropic
         instrument_asyncpg = DEFAULT_LOGFIRE_INSTANCE.instrument_asyncpg
         instrument_celery = DEFAULT_LOGFIRE_INSTANCE.instrument_celery

docs/integrations/llms/openai.md

@@ -30,6 +30,7 @@
 instrument_wsgi = DEFAULT_LOGFIRE_INSTANCE.instrument_wsgi
 instrument_fastapi = DEFAULT_LOGFIRE_INSTANCE.instrument_fastapi
 instrument_openai = DEFAULT_LOGFIRE_INSTANCE.instrument_openai
+instrument_openai_agents = DEFAULT_LOGFIRE_INSTANCE.instrument_openai_agents
 instrument_anthropic = DEFAULT_LOGFIRE_INSTANCE.instrument_anthropic
 instrument_asyncpg = DEFAULT_LOGFIRE_INSTANCE.instrument_asyncpg
 instrument_httpx = DEFAULT_LOGFIRE_INSTANCE.instrument_httpx
@@ -117,6 +118,7 @@ def loguru_handler() -> Any:
     'instrument_pydantic',
     'instrument_fastapi',
     'instrument_openai',
+    'instrument_openai_agents',
     'instrument_anthropic',
     'instrument_asyncpg',
     'instrument_httpx',

logfire-api/logfire_api/__init__.py

@@ -1,11 +1,13 @@
 from __future__ import annotations
 
+import json
 import os
 import re
 import sys
 import typing
 from collections.abc import Sequence
 from functools import partial
+from json import JSONDecodeError
 from pathlib import Path
 from typing import Any, Mapping, cast
 
@@ -47,6 +49,7 @@ def exported_spans_as_dict(
         include_instrumentation_scope: bool = False,
         _include_pending_spans: bool = False,
         _strip_function_qualname: bool = True,
+        parse_json_attributes: bool = False,
     ) -> list[dict[str, Any]]:
         """The exported spans as a list of dicts.
 
@@ -55,6 +58,7 @@ def exported_spans_as_dict(
             strip_filepaths: Whether to strip the filepaths from the exported spans.
             include_resources: Whether to include the resource attributes in the exported spans.
             include_instrumentation_scope: Whether to include the instrumentation scope in the exported spans.
+            parse_json_attributes: Whether to parse strings containing JSON arrays/objects.
 
         Returns:
             A list of dicts representing the exported spans.
@@ -64,6 +68,7 @@ def exported_spans_as_dict(
             fixed_line_number=fixed_line_number,
             strip_filepaths=strip_filepaths,
             strip_function_qualname=_strip_function_qualname,
+            parse_json_attributes=parse_json_attributes,
         )
 
         def build_context(context: trace.SpanContext) -> dict[str, Any]:
@@ -131,6 +136,7 @@ def process_attribute(
     strip_filepaths: bool,
     fixed_line_number: int | None,
     strip_function_qualname: bool,
+    parse_json_attributes: bool = False,
 ) -> Any:
     if name == 'code.filepath' and strip_filepaths:
         try:
@@ -148,6 +154,11 @@ def process_attribute(
     if name == ResourceAttributes.SERVICE_INSTANCE_ID:
         if re.match(r'^[0-9a-f]{32}$', value):
             return '0' * 32
+    if parse_json_attributes and isinstance(value, str) and (value.startswith('{') or value.startswith('[')):
+        try:
+            return json.loads(value)
+        except JSONDecodeError:  # pragma: no cover
+            pass
     return value
 
 
@@ -156,11 +167,12 @@ def build_attributes(
     strip_filepaths: bool,
     fixed_line_number: int | None,
     strip_function_qualname: bool,
+    parse_json_attributes: bool,
 ) -> dict[str, Any] | None:
     if attributes is None:  # pragma: no cover
         return None
     attributes = {
-        k: process_attribute(k, v, strip_filepaths, fixed_line_number, strip_function_qualname)
+        k: process_attribute(k, v, strip_filepaths, fixed_line_number, strip_function_qualname, parse_json_attributes)
         for k, v in attributes.items()
     }
     if 'telemetry.sdk.version' in attributes:
@@ -191,12 +203,14 @@ def exported_logs_as_dicts(
         include_resources: bool = False,
         include_instrumentation_scope: bool = False,
         _strip_function_qualname: bool = True,
+        parse_json_attributes: bool = False,
     ) -> list[dict[str, Any]]:
         _build_attributes = partial(
             build_attributes,
             fixed_line_number=fixed_line_number,
             strip_filepaths=strip_filepaths,
             strip_function_qualname=_strip_function_qualname,
+            parse_json_attributes=parse_json_attributes,
         )
 
         def build_log(log_data: LogData) -> dict[str, Any]:

logfire/__init__.py

@@ -77,6 +77,8 @@ def _instrumentation_setup(**kwargs: Any) -> Any:
             return None, None, kwargs
 
         message_template, span_data, stream_state_cls = get_endpoint_config_fn(kwargs['options'])
+        if not message_template:
+            return None, None, kwargs
 
         span_data['async'] = is_async
 

logfire/_internal/exporters/test.py

@@ -9,6 +9,8 @@
 from openai.types.completion import Completion
 from openai.types.create_embedding_response import CreateEmbeddingResponse
 from openai.types.images_response import ImagesResponse
+from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.trace import get_current_span
 
 from ...utils import handle_internal_errors
 from .types import EndpointConfig, StreamState
@@ -36,11 +38,22 @@ def get_endpoint_config(options: FinalRequestOptions) -> EndpointConfig:
         json_data = {}
 
     if url == '/chat/completions':
+        if is_current_agent_span('Chat completion with {gen_ai.request.model!r}'):
+            return EndpointConfig(message_template='', span_data={})
+
         return EndpointConfig(
             message_template='Chat Completion with {request_data[model]!r}',
             span_data={'request_data': json_data},
             stream_state_cls=OpenaiChatCompletionStreamState,
         )
+    elif url == '/responses':
+        if is_current_agent_span('Responses API'):
+            return EndpointConfig(message_template='', span_data={})
+
+        return EndpointConfig(  # pragma: no cover
+            message_template='Responses API with {request_data[model]!r}',
+            span_data={'request_data': json_data},
+        )
     elif url == '/completions':
         return EndpointConfig(
             message_template='Completion with {request_data[model]!r}',
@@ -64,6 +77,16 @@ def get_endpoint_config(options: FinalRequestOptions) -> EndpointConfig:
         )
 
 
+def is_current_agent_span(span_name: str):
+    current_span = get_current_span()
+    return (
+        isinstance(current_span, ReadableSpan)
+        and current_span.instrumentation_scope
+        and current_span.instrumentation_scope.name == 'logfire.openai_agents'
+        and current_span.name == span_name
+    )
+
+
 def content_from_completions(chunk: Completion | None) -> str | None:
     if chunk and chunk.choices:
         return chunk.choices[0].text