Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assets/langwatch-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
181 changes: 181 additions & 0 deletions docs/observability/langwatch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Agent Observability with LangWatch

[LangWatch](https://langwatch.ai) is a comprehensive observability platform for monitoring, debugging, and improving LLM applications and AI Agents. It provides automatic tracing, evaluation, and monitoring capabilities for your Google ADK applications through seamless OpenTelemetry integration. To get started, sign up for a [free account](https://app.langwatch.ai).

## Overview

LangWatch can automatically collect traces from Google ADK using [OpenInference instrumentation](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-google-adk), allowing you to:

- **Trace agent interactions** - Automatically capture every agent run, tool call, model request, and response with context and metadata
- **Evaluate performance** - Assess agent behavior using custom or pre-built evaluators and run experiments to test agent configurations
- **Monitor in production** - Set up real-time dashboards and alerts to track performance
- **Debug issues** - Analyze detailed traces to quickly identify bottlenecks, failed tool calls, and any unexpected agent behavior

## Installation

Install the required packages:

```bash
pip install langwatch google-adk openinference-instrumentation-google-adk
```

## Setup

### 1. Configure Environment Variables { #configure-environment-variables }

Set your LangWatch API key and Google API key:

```bash
export LANGWATCH_API_KEY=[your_langwatch_api_key_here]
export GOOGLE_API_KEY=[your_google_api_key_here]
```

### 2. Connect your application to LangWatch { #connect-your-application-to-langwatch }

```python
import langwatch
from openinference.instrumentation.google_adk import GoogleADKInstrumentor

# Initialize LangWatch with the Google ADK instrumentor
langwatch.setup(
instrumentors=[GoogleADKInstrumentor()]
)
```

That's it! All Google ADK agent activity will now be traced and sent to your LangWatch dashboard automatically.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, I was able to send a test trace to LangWatch, but I kept seeing this message at the top of the Analytics page:

Setup pending
Your project is not set up yet so you won't be able to see any data on the dashboard, please go to the setup page to get started.

But I couldn't figure out what other actions I needed to do at this point to go deeper into the platform or to view details of the trace that was sent.


## Observe

Now that you have tracing setup, all Google ADK SDK requests will be streamed to LangWatch for observability and evaluation.

```python
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Python code does not work for me, I get the following error (I'm running with ADK 1.15.1):

2025-10-02 15:04:44,311 - ERROR - adk_web_server.py:1284 - Error in event_generator: Fail to load 'langwatch-test' module. 'await' outside function (agent.py, line 55)
Traceback (most recent call last):
  File "/Users/koverholt/miniforge3/lib/python3.11/site-packages/google/adk/cli/adk_web_server.py", line 1264, in event_generator
    runner = await self.get_runner_async(req.app_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/koverholt/miniforge3/lib/python3.11/site-packages/google/adk/cli/adk_web_server.py", line 444, in get_runner_async
    agent_or_app = self.agent_loader.load_agent(app_name)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/koverholt/miniforge3/lib/python3.11/site-packages/google/adk/cli/utils/agent_loader.py", line 234, in load_agent
    agent_or_app = self._perform_load(agent_name)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/koverholt/miniforge3/lib/python3.11/site-packages/google/adk/cli/utils/agent_loader.py", line 207, in _perform_load
    if root_agent := self._load_from_module_or_package(actual_agent_name):
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/koverholt/miniforge3/lib/python3.11/site-packages/google/adk/cli/utils/agent_loader.py", line 104, in _load_from_module_or_package
    raise e
  File "/Users/koverholt/miniforge3/lib/python3.11/site-packages/google/adk/cli/utils/agent_loader.py", line 70, in _load_from_module_or_package
    module_candidate = importlib.import_module(agent_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/koverholt/miniforge3/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/koverholt/Desktop/langwatch-test/__init__.py", line 1, in <module>
    from . import agent
  File "/Users/koverholt/Desktop/langwatch-test/agent.py", line 55
    await session_service.create_session(
    ^
SyntaxError: Fail to load 'langwatch-test' module. 'await' outside function

But I switched to using the sample agent in the ADK quickstart, and I did get traces sent to LangWatch.

Edit: I did try running the (different) sample code in the Automatic Tracing section of the LangWatch docs, which works great. So consider using that code here (or linking to it) instead.

import nest_asyncio
nest_asyncio.apply()

from google.adk.agents import Agent
from google.adk.runners import InMemoryRunner
from google.genai import types

# Define a tool function
def get_weather(city: str) -> dict:
"""Retrieves the current weather report for a specified city.

Args:
city (str): The name of the city for which to retrieve the weather report.

Returns:
dict: status and result or error msg.
"""
if city.lower() == "new york":
return {
"status": "success",
"report": (
"The weather in New York is sunny with a temperature of 25 degrees"
" Celsius (77 degrees Fahrenheit)."
),
}
else:
return {
"status": "error",
"error_message": f"Weather information for '{city}' is not available.",
}

# Create an agent with tools
agent = Agent(
name="weather_agent",
model="gemini-2.0-flash-exp",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest changing to gemini-2.0-flash here and throughout, as the -exp variant is no longer available. Or even better, referencing the latest gemini-2.5-flash model throughout the code examples.

description="Agent to answer questions using weather tools.",
instruction="You must use the available tools to find an answer.",
tools=[get_weather]
)

app_name = "weather_app"
user_id = "test_user"
session_id = "test_session"
runner = InMemoryRunner(agent=agent, app_name=app_name)
session_service = runner.session_service

await session_service.create_session(
app_name=app_name,
user_id=user_id,
session_id=session_id
)

# Run the agent (all interactions will be traced)
async for event in runner.run_async(
user_id=user_id,
session_id=session_id,
new_message=types.Content(role="user", parts=[
types.Part(text="What is the weather in New York?")]
)
):
if event.is_final_response():
print(event.content.parts[0].text.strip())
```

## Advanced Configuration

### Using Decorators for Additional Context

If you want to add additional context or metadata to your traces, you can optionally use the `@langwatch.trace()` decorator:

```python
import langwatch
from google.adk import Agent, Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from openinference.instrumentation.google_adk import GoogleADKInstrumentor

langwatch.setup(
instrumentors=[GoogleADKInstrumentor()]
)

@langwatch.trace(name="Google ADK Agent Run")
def run_agent_interaction(user_message: str):
# Update the current trace with additional metadata
current_trace = langwatch.get_current_trace()
if current_trace:
current_trace.update(
metadata={
"user_id": "user_123",
"session_id": "session_abc",
"agent_name": "weather_agent",
"model": "gemini-2.0-flash-exp"
}
)

# Your agent execution code here
# ... agent execution code ...

return "Agent response"
```

## View Results in LangWatch

Once your application is running with the instrumentation, you can view all traces, metrics, and evaluations in your LangWatch dashboard:

- **Traces**: See detailed execution flows of your agents
- **Metrics**: Monitor performance and usage patterns
- **Evaluations**: Assess agent behavior and run experiments
- **Alerts**: Set up monitoring and get notified of issues

![LangWatch Dashboard](../assets/langwatch-dashboard.png)

## Notes

- You do **not** need to set any OpenTelemetry environment variables or configure exporters manually—`langwatch.setup()` handles everything.
- The `@langwatch.trace()` decorator is **optional** - the OpenInference instrumentor will capture all ADK activity automatically.
- For advanced configuration (custom attributes, endpoint, etc.), see the [LangWatch Python integration guide](https://docs.langwatch.ai/integration/python).

## Troubleshooting

- Make sure your `LANGWATCH_API_KEY` is set in the environment.
- If you see no traces in LangWatch, check that the instrumentor is included in `langwatch.setup()` and that your agent code is being executed.
- Ensure you have the correct Google API key set for Gemini access.
- Check that the `openinference-instrumentation-google-adk` package is properly installed.

## Support and Resources

- [LangWatch Documentation](https://docs.langwatch.ai)
- [LangWatch Google ADK Integration Guide](https://docs.langwatch.ai/integration/python/integrations/google-ai)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note (not related to this PR): Would it make more sense to move the ADK Guide under Frameworks in the LangWatch docs rather than the Model Providers section?

- [OpenInference Package](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-google-adk)
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ nav:
- Cloud Trace: observability/cloud-trace.md
- AgentOps: observability/agentops.md
- Arize AX: observability/arize-ax.md
- LangWatch: observability/langwatch.md
- Phoenix: observability/phoenix.md
- W&B Weave: observability/weave.md
- Evaluate:
Expand Down