Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Get started with Camunda agentic orchestration by building and running your firs

Understand the fundamental concepts of Camunda agentic orchestration.

<AoGrid ao={fundamentalCards} columns={2}/>
<AoGrid ao={fundamentalCards} columns={3}/>

## Explore further resources

Expand Down
208 changes: 208 additions & 0 deletions docs/components/agentic-orchestration/monitor-agents-operate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
---
id: monitor-agents-operate
title: Monitor your AI agents with Operate
sidebar_label: Monitor your AI agents
description: "Monitor and troubleshoot your AI agent process instances in Camunda 8 using Operate."
keywords: ["agentic ai", "AI agents"]
---

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

Monitor and troubleshoot your AI agent process instances in Camunda 8 using Operate.

## About

In this guide, you will:

- Inspect an AI agent process instance in Operate.
- Understand agent’s tool usage and metadata such as tool call inputs and results.
- Analyze the agent context and how it is stored.

:::note
Operate enables inspection of execution paths, tool usage, and agent metadata. However, certain runtime artifacts, such as document storage contents, may require additional configuration.
:::

After completing this guide, you will be able to inspect, debug, and monitor AI agent executions in Camunda 8.

## Prerequisites

- You have the [AI Agent Chat Quick Start](https://marketplace.camunda.com/en-US/apps/587865) model blueprint deployed in [Modeler](/components/modeler/about-modeler.md).
- You are familiar with [Operate](/components/operate/operate-introduction.md).

:::important
This guide is a follow-up to [Build your first AI agent](../../guides/getting-started-agentic-orchestration.md), where you will use the same example AI agent process. Therefore, the same [prerequisites apply](../..//guides/getting-started-agentic-orchestration.md#prerequisites). However, it can be applied to other AI agent process implementations.
:::

## Step 1: Run your AI agent process

Run your process instance using a prompt to trigger the AI Agent connector.

For example, enter "Visit _docs.camunda.io_ and tell me about it" in the **How can I help you today?** field, and click **Start instance**.

## Step 2: Open the process instance in Operate

1. Open [Operate](/components/operate/operate-introduction.md). If working locally with Self-Managed, open it in your browser at http://localhost:8080/operate.
2. Locate the process instance created by your prompt. See [View a deployed process](/components/operate/userguide/basic-operate-navigation.md#view-a-deployed-process) for more details.
3. Open your process instance view by clicking on its process instance key.

At this point, you should see the process progressing through your model.

## Step 3: Understand what Operate shows

With Operate, you can track the agent activity and see which tool tasks are called.

1. To show how many times each BPMN element is triggered, select **Execution count**.

For this particular prompt example, you can see:

- The AI Agent connector was triggered once.
- Within it, the agent executed the **Fetch URL** tool. This aligns with your prompt example: Visit a website URL and extract information from it.

2. Select the **Fetch URL** tool element.

- In the bottom-left pane, you can see where the element belongs in the execution tree.
- In the bottom-right pane, the element details are displayed, including the [**Variables**](components/concepts/variables.md) and [**Input/Output Mappings**](/components/concepts/variables.md#inputoutput-variable-mappings) columns, among others.
However, the actual tool inputs and results are stored in a **parent scope** and are accessible via the element's inner instance in the execution tree.

## Step 4: Inspect tool calls

Each tool execution produces an inner instance where you can find:

- The inputs passed into the tool.
- The results.

To see the **Fetch URL** tool input and results:

1. In the execution tree, select the **AI_Agent#innerInstance** parent element of the **Fetch URL** tool. You will see:

- The **toolCall** variable (the _input_). In its value field, you can see the URL you asked the agent to fetch information from.
- The **toolCallResult** variable (the _results_). See [Tool call responses](/components/connectors/out-of-the-box-connectors/agentic-ai-aiagent-tool-definitions.md#tool-call-responses) for more details.

2. To better inspect the results, click the pencil icon to enter edit mode for **toolCallResult**.
3. Click the two-squares icon to open the JSON editor modal. With this, you can inspect the full payload of the variable value. The result is a reference to a document where the actual output is stored. You will learn more about this in [Step 6](#step-6-understand-how-agent-memory-is-stored).

Here's the **toolCallResult** value:

```json
{
"storeId": "gcp",
"documentId": "1aa0e4f6-7cee-4c19-ac2e-cebee3c7b151",
"contentHash": "649a8575bf8ced157fbbba7acc02ba33058bb2e3f40fb37b0f303730fb64fba0",
"metadata": {
"contentType": "text/html; charset=utf-8",
"size": 33722,
"fileName": "1aa0e4f6-7cee-4c19-ac2e-cebee3c7b151"
},
"camunda.document.type": "camunda"
}
```

:::note
If a tool is executed more than once, select the desired tool invocation in **Instance History**, then open the corresponding inner instance to view the actual inputs and results.
:::

## Step 5: Analyze the agent context

Within the AI Agent connector, you can examine the agent context.
To view it:

1. Select the **AI Agent** element in the execution tree.
2. To better inspect the value, click the pencil icon to enter edit mode for the **agentContext** variable.
3. Click the two-squares icon to open the JSON editor modal. With this, you can inspect the full payload of the variable value.

In the JSON payload, you can find information about:

- Defined tools.
- The conversation, including your prompts and agent's replies.
- Tool calls invoked by the agent.
- Tool call inputs and results.
- Additional metadata, such as reasoning traces and token usage.

Here's the example conversation stored in the agent's context:

```json
"type": "in-process",
"conversationId": "130bbad7-b316-4cac-9653-a77e37e11f8b",
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are a helpful, generic chat agent which can answer a wide amount of questions based on your knowledge and an optional set of available tools.\n\nIf tools are provided, you should prefer them instead of guessing an answer. You can call the same tool multiple times by providing different input values. Don't guess any tools which were not explicitely configured. If no tool matches the request, try to generate an answer. If you're not able to find a good answer, return with a message stating why you're not able to.\n\nIf you are prompted to interact with a person, never guess contact details, but use available user/person lookup tools instead and return with an error if you're not able to look up appropriate data.\n\nThinking, step by step, before you execute your tools, you think using the template `<thinking><context></context><reflection></reflection></thinking>`"
}
]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "Visit https://docs.camunda.io/ and tell me about it"
}
],
"metadata": {
"timestamp": "2026-02-23T11:48:03.90923378Z"
}
},
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "<thinking>\n<context>\nThe user wants me to visit a specific URL (https://docs.camunda.io/) and tell them about it. I have a tool called \"Fetch_URL\" that can fetch the contents of a given URL. This seems like a perfect match for the request.\n</context>\n<reflection>\nI should use the Fetch_URL tool to get the contents of https://docs.camunda.io/ and then analyze what I find to provide information about it to the user.\n</reflection>\n</thinking>"
}
],
"toolCalls": [
{
"id": "tooluse_kBcf61pS50OS31KYQnRFzy",
"name": "Fetch_URL",
"arguments": {
"url": "https://docs.camunda.io/"
}
}
],
```

## Step 6: Understand how agent memory is stored

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Can we explicitly clarify the trade-off between visibility and scalability?

“In Process” → better Operate visibility, limited scalability for long conversations.
“Document Storage” → scalable for production but content is not rendered inline in Operate.

Copy link
Contributor Author

@afgambin afgambin Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added as a note box at the end of the Step 6


In Modeler, within the AI Agent sub-process, you can define how the conversation memory is stored using the **Memory storage type** field.

By default, agent memory uses the **In Process** type, which stores it as part of the agent context.
With this option, you can view it in Operate within the agent context, as you did in the previous step, [Analyze the agent context](#step-5-analyze-the-agent-context).

With the **Camunda Document Storage** option instead:

- You can't view the full conversation and chain-of-thought traces in Operate. Operate only show a **document reference** and metadata.
- Use this option for long conversations, where Operate variable limits might be exceeded.

See [Memory](/components/connectors/out-of-the-box-connectors/agentic-ai-aiagent-subprocess.md#memory) for more details.

:::note Agent memory storage

- Use **In Process** for testing and debugging scenarios: Better visibility in Operate.
- Use **Camunda Document Storage** for production scenarios: Better scalability and runtime behavior for long contexts.

:::

## Step 7: Review the results

In the **User Feedback** element, you will see the execution count in green. This means the process instance execution is stopped there and waiting for action.

In this case, the required action is to provide feedback on the agent results. To do so:

1. Select the **User Feedback** element.
2. Open [Tasklist](/components/tasklist/introduction-to-tasklist.md).
3. Select the User Feedback task and assign to yourself by clicking **Assign to me**.
4. Analyze the result. You will see an overview of the website URL requested in the prompt.
5. You can follow up with more prompts to continue testing your AI agent.
6. Select the **Are you satisfied with the result?** checkbox when you want to finish the process, then click **Complete task**.
7. Go back to Operate. You will see the process instance is now completed, and the end event has been triggered.

## Next steps

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Right before Next Steps, based on enterprise production use cases, can we add a short section titled:

Production Monitoring Recommendations

For example:
Monitor escalation paths and repeated retries.
nspect evaluation outcomes (if using judge agents).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, but I’d need more input on this. Also, regarding judge agents: they aren’t included in the example presented here, so I would keep this for another guide, or cover it when we provide it as part of a model blueprint that we can use as an example.


Now that you know how to monitor your AI agents with Operate, you can:

- Learn more about [Camunda agentic orchestration](/components/agentic-orchestration/agentic-orchestration-overview.md) and the [AI Agent connector](/components/connectors/out-of-the-box-connectors/agentic-ai-aiagent.md).
- Explore other [AI agent model blueprints](https://marketplace.camunda.com/en-US/listing?q=ai&cat=107793&locale=en-US) from Camunda marketplace.
8 changes: 7 additions & 1 deletion docs/components/react-components/_ao-card-data.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,20 @@ export const fundamentalCards = [
title: "AI agents",
image: IconAoAgentImg,
description:
"Learn how to build and integrate AI agents into your end-to-end processes.",
"Build and integrate AI agents into your end-to-end processes.",
},
{
link: "../ao-design/",
title: "Design and architecture",
image: IconAoDesignImg,
description: "Plan and design your agentic orchestration solutions.",
},
{
link: "../monitor-agents-operate/",
title: "Monitor your AI agents",
image: IconAoAgentImg,
description: "Monitor your AI agents with Operate.",
},
];

// Recommendation cards on introduction page
Expand Down
1 change: 1 addition & 0 deletions docs/guides/getting-started-agentic-orchestration.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,7 @@ For example:

You can also:

- [Monitor your AI agent with Operate](/components/agentic-orchestration/monitor-agents-operate.md).
- Learn more about [Camunda agentic orchestration](/components/agentic-orchestration/agentic-orchestration-overview.md) and the [AI Agent connector](/components/connectors/out-of-the-box-connectors/agentic-ai-aiagent.md).
- Read the [Building Your First AI Agent in Camunda](https://camunda.com/blog/2025/02/building-ai-agent-camunda/) blog.
- Explore other [AI blueprints](https://marketplace.camunda.com/en-US/listing?q=ai&cat=107793&locale=en-US) from Camunda marketplace.
Expand Down
1 change: 1 addition & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ module.exports = {
items: [
"components/agentic-orchestration/ai-agents",
"components/agentic-orchestration/ao-design",
"components/agentic-orchestration/monitor-agents-operate",
"components/agentic-orchestration/choose-right-model-agentic",
"components/agentic-orchestration/model-recommendations-agentic",
],
Expand Down
Loading