wip

Author: mme

docs/docs.json

@@ -45,6 +45,19 @@
               "concepts/tools"
             ]
           },
+          {
+            "group": "Draft Changes",
+            "pages": [
+              "drafts/overview",
+              "drafts/activity-events",
+              "drafts/reasoning",
+              "drafts/serialization",
+              "drafts/multimodal-messages",
+              "drafts/interrupts",
+              "drafts/generative-ui",
+              "drafts/meta-events"
+            ]
+          },
           {
             "group": "Tutorials",
             "pages": ["tutorials/cursor", "tutorials/debugging"]

docs/drafts/activity-events.mdx

@@ -0,0 +1,219 @@
+---
+title: Activity Events
+description: Proposal for representing ongoing agent progress between chat messages
+---
+
+# Activity Events Proposal
+
+## Summary
+
+### Problem Statement
+Users want to render "activity" updates inline with chat, not just at run start or end. Currently, there's no standardized way to represent ongoing agent progress between chat messages.
+
+### Motivation
+AG-UI is extended with **ActivityEvents** and **ActivityMessages** to represent ongoing agent progress in between chat messages. This allows frameworks to surface fine-grained activity updates chronologically, giving users immediate visibility into what an agent is doing without waiting for the next message or run boundary.
+
+## Status
+
+- **Status**: Draft
+- **Author(s)**: AG-UI Team
+
+## Background
+
+Users want real-time visibility into agent activities as they happen. Consider this example UI:
+
+```
++------------------------------------------------------------+
+| I will search the internet for relevant information        | <- TextMessage
++------------------------------------------------------------+
++------------------------------------------------------------+
+| ✓ checking reddit                                          | <- ActivityMessage
+| searching X.com...                                         |
++------------------------------------------------------------+
+```
+
+### Use Cases
+
+- **Workflows**: Step-by-step progress through workflow execution
+- **Planning**: Intermediate planning or tool use visibility
+- **Custom frameworks**: Signals representing ongoing work in any agent system
+
+## Challenges
+
+- **Flexibility**: Must handle arbitrary activity data from different frameworks
+- **Serializability**: Events must be replayable and rehydrated for session recovery
+- **Extensibility**: Developers should define custom renderers per activity type, with a generic fallback
+- **Chronology**: Activities must interleave naturally with chat and run events
+
+## Detailed Specification
+
+### Overview
+
+This proposal introduces two new concepts to the AG-UI protocol:
+1. **ActivityEvent**: A new event type in the event stream
+2. **ActivityMessage**: A new message type alongside TextMessage, ToolMessage, etc.
+
+Frameworks may emit ActivityEvents, and frontends can render them inline with chat.
+
+### New Event: ActivityEvent
+
+```typescript
+type ActivityEvent = BaseEvent & {
+  type: EventType.ACTIVITY
+  /**
+   * Unique identifier for the ActivityMessage this event belongs to.
+   */
+  messageId: string
+  /**
+   * Activity type, e.g. "PLAN", "SEARCH", "SCRAPE"
+   */
+  activityType: string
+  /**
+   * Snapshot of the full activity state (optional).
+   */
+  snapshot?: Record<string, any>
+  /**
+   * Patch to apply to the prior snapshot (optional).
+   * Follows JSON Patch semantics.
+   */
+  patch?: Record<string, any>
+}
+```
+
+#### Example Events
+
+Initial activity snapshot:
+```json
+{
+  "id": "evt_001",
+  "ts": 1714064100000,
+  "type": "ACTIVITY",
+  "messageId": "msg_789",
+  "activityType": "PLAN",
+  "snapshot": {
+    "tasks": ["check reddit", "search X.com"]
+  }
+}
+```
+
+Incremental update via patch:
+```json
+{
+  "id": "evt_002",
+  "ts": 1714064120000,
+  "type": "ACTIVITY",
+  "messageId": "msg_789",
+  "activityType": "PLAN",
+  "patch": {
+    "op": "replace",
+    "path": "/tasks/0",
+    "value": "✓ check reddit"
+  }
+}
+```
+
+### New Message: ActivityMessage
+
+```typescript
+type ActivityMessage = {
+  id: string
+  role: "activity"
+  activityType: string
+  /**
+   * Finalized activity content as of compaction.
+   */
+  content: Record<string, any>
+}
+```
+
+### Rendering Strategy
+
+- **Generic renderer**: Displays raw snapshot/patch as JSON or formatted text
+- **Custom renderer**: Developers can register a renderer per `activityType`:
+  - `"PLAN"` → Interactive checklist component
+  - `"SEARCH"` → Live status with progress indicators
+  - `"WORKFLOW"` → Step-by-step workflow visualization
+
+## Implementation Considerations
+
+### Client SDK Changes
+
+TypeScript SDK additions:
+- New `ActivityEvent` type in `@ag-ui/core`
+- New `ActivityMessage` type in message unions
+- Activity renderer registry in `@ag-ui/client`
+
+Python SDK additions:
+- New `ActivityEvent` class in `ag_ui.core.events`
+- New `ActivityMessage` class in message types
+- Activity serialization/deserialization support
+
+### Integration Impact
+
+- **Planning Frameworks**: Can emit ActivityEvents during planning or tool execution phases
+- **Workflow Systems**: Can surface step-by-step workflow progress as ActivityEvents
+- **Other frameworks**: May emit ActivityEvents freely; AG-UI will serialize them like other events
+
+## Breaking Changes
+
+None. This is an additive change that maintains backward compatibility.
+
+## Examples and Use Cases
+
+### Example 1: Web Search Activity
+
+```typescript
+// Agent emits search activity
+agent.emitActivity({
+  messageId: "msg_123",
+  activityType: "SEARCH",
+  snapshot: {
+    sources: [
+      { name: "Reddit", status: "pending" },
+      { name: "X.com", status: "pending" },
+      { name: "Google", status: "pending" }
+    ]
+  }
+})
+
+// Update as search progresses
+agent.emitActivity({
+  messageId: "msg_123",
+  activityType: "SEARCH",
+  patch: {
+    op: "replace",
+    path: "/sources/0/status",
+    value: "complete"
+  }
+})
+```
+
+### Use Case: Multi-Step Workflow Visibility
+
+A data analysis agent performing multiple steps:
+1. Loading dataset → ActivityEvent shows progress bar
+2. Cleaning data → ActivityEvent shows rows processed
+3. Running analysis → ActivityEvent shows current computation
+4. Generating report → ActivityEvent shows sections completed
+
+Each step appears inline with chat, giving users real-time feedback.
+
+## Testing Strategy
+
+- Unit tests for ActivityEvent serialization/deserialization
+- Integration tests with mock frameworks emitting ActivityEvents
+- E2E tests in AG-UI Dojo demonstrating activity rendering
+- Performance benchmarks for high-frequency activity updates
+
+## Open Questions
+
+1. Should we enforce a schema for common activity types (e.g., standardized "PLAN" format)?
+2. How should activities behave across run boundaries?
+3. Should there be a maximum number of patches before requiring a new snapshot?
+4. How do activities interact with message editing/regeneration?
+
+## References
+
+- [JSON Patch RFC 6902](https://tools.ietf.org/html/rfc6902)
+- [AG-UI Events Documentation](/concepts/events)
+- [AG-UI Messages Documentation](/concepts/messages)