Skip to content

Commit 5c8f9ac

Browse files
afavi3talmatteo-grella
authored andcommitted
Add workflowrunner: experimental orchestration layer for multi-agent workflows
This introduces the workflowrunner package, a proof-of-concept orchestration layer that enables declarative multi-agent workflows on top of the OpenAI Agents Go SDK. Key features: - Validates and materializes WorkflowRequest payloads into configured agents - Runs workflows asynchronously with event streaming capabilities - Integrates with OpenAI tracing and supports MCP tools - Provides callback modes for HTTP and stdout output - Includes execution state tracking and approval workflows The package includes comprehensive documentation, examples, and architecture diagrams to help users get started with declarative agent flows.
1 parent 7f25b99 commit 5c8f9ac

File tree

15 files changed

+2315
-0
lines changed

15 files changed

+2315
-0
lines changed

workflowrunner/README.md

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Workflow Runner (POC)
2+
3+
The workflow runner is an experimental orchestration layer that lets you execute
4+
declarative multi-agent workflows on top of the OpenAI Agents Go SDK. It takes a
5+
JSON payload that describes agents, tools, guardrails, and session settings,
6+
turns it into live SDK objects, and runs the conversation while streaming back
7+
events, state updates, and final results.
8+
9+
> **Status:** Proof of concept / work in progress. The API, wiring, and
10+
> filesystem layout may change without notice as we refine the feature.
11+
12+
## Why it exists
13+
- Provide a hosted-workflow-style interface for Go services that want to run
14+
declarative agent flows without hand-coding orchestration logic.
15+
- Make multi-agent runs observable: every lifecycle event can be streamed to
16+
callbacks, persisted for dashboards, or printed to stdout for local debugging.
17+
- Enable production patterns such as resumable sessions, tracing, tool approval
18+
queues, and safety guardrails using the existing Agents SDK features.
19+
20+
## Core capabilities
21+
- Validates and materializes `WorkflowRequest` payloads into configured agents,
22+
guardrails, tools, and output types (`Builder`).
23+
- Runs the workflow asynchronously through `RunnerService.Execute`, returning an
24+
`asynctask` handle for polling or awaiting.
25+
- Streams events to HTTP endpoints or stdout printers while keeping an
26+
`ExecutionStateStore` in sync (in-memory by default, pluggable for shared
27+
storage).
28+
- Integrates with OpenAI tracing so each run shows up in traces with workflow
29+
metadata.
30+
- Supports hosted MCP tools and guardrail registries out of the box.
31+
32+
## Architecture overview
33+
34+
```mermaid
35+
flowchart TD
36+
Caller["Client\n(service or CLI)"]
37+
Request["WorkflowRequest JSON"]
38+
RunnerService["RunnerService"]
39+
Builder["Builder\n(agent graph & run config)"]
40+
AgentsSDK["Agents SDK\n(agents.Runner)"]
41+
StateStore["ExecutionStateStore"]
42+
Callback["CallbackPublisher\n(HTTP | Stdout)"]
43+
44+
Caller -->|constructs| Request
45+
Request --> RunnerService
46+
RunnerService --> Builder
47+
Builder --> AgentsSDK
48+
RunnerService -->|RunStreamed| AgentsSDK
49+
RunnerService --> StateStore
50+
RunnerService --> Callback
51+
AgentsSDK -->|stream events| RunnerService
52+
RunnerService -->|final summary| Caller
53+
```
54+
55+
## Execution timeline
56+
57+
```mermaid
58+
sequenceDiagram
59+
participant Client
60+
participant RunnerService
61+
participant Builder
62+
participant AgentsRunner as agents.Runner
63+
participant Callback as CallbackPublisher
64+
participant Store as ExecutionStateStore
65+
66+
Client->>RunnerService: Execute(ctx, WorkflowRequest)
67+
RunnerService->>Builder: Build(ctx, request)
68+
Builder-->>RunnerService: BuildResult (agents, session, runner)
69+
RunnerService->>AgentsRunner: RunStreamed(startingAgent, query)
70+
RunnerService->>Store: OnRunStarted / Save
71+
loop streamed events
72+
AgentsRunner-->>RunnerService: StreamEvent
73+
RunnerService->>Store: Update state
74+
RunnerService->>Callback: Publish run.event (async best-effort)
75+
end
76+
AgentsRunner-->>RunnerService: FinalOutput + run summary
77+
RunnerService->>Store: OnRunCompleted / Save
78+
RunnerService->>Callback: Publish run.completed
79+
RunnerService-->>Client: Task result (RunSummary)
80+
```
81+
82+
## Getting started
83+
84+
```go
85+
builder := workflowrunner.NewDefaultBuilder()
86+
service := workflowrunner.NewRunnerService(builder)
87+
88+
req := workflowrunner.WorkflowRequest{
89+
Query: "List three fun facts about Mars.",
90+
Session: workflowrunner.SessionDeclaration{
91+
SessionID: "demo-simple",
92+
Credentials: workflowrunner.CredentialDeclaration{
93+
UserID: "user-123",
94+
AccountID: "acct-456",
95+
},
96+
HistorySize: 10,
97+
MaxTurns: 8,
98+
},
99+
Callback: workflowrunner.CallbackDeclaration{
100+
Mode: "stdout", // "http" is also supported
101+
},
102+
Workflow: workflowrunner.WorkflowDeclaration{
103+
Name: "simple_assistant",
104+
StartingAgent: "assistant",
105+
Agents: []workflowrunner.AgentDeclaration{
106+
{
107+
Name: "assistant",
108+
Instructions: "You are an enthusiastic planetary science assistant.",
109+
Model: &workflowrunner.ModelDeclaration{
110+
Model: "gpt-4o-mini",
111+
Temperature: floatPtr(0.3),
112+
},
113+
},
114+
},
115+
},
116+
}
117+
118+
task, err := service.Execute(ctx, req)
119+
if err != nil {
120+
log.Fatalf("build or run failed: %v", err)
121+
}
122+
123+
summary := task.Await()
124+
if summary.Error != nil {
125+
log.Printf("run failed: %v", summary.Error)
126+
} else {
127+
log.Printf("final output: %v", summary.Value.FinalOutput)
128+
}
129+
```
130+
131+
- See `workflowrunner/examples/simple` and `workflowrunner/examples/complex` for
132+
runnable end-to-end demos.
133+
- The runner requires an OpenAI API key (`OPENAI_API_KEY`) to be present in the
134+
environment because agents ultimately call OpenAI models.
135+
136+
## Callback modes
137+
- `mode: "http"` (default): events are POSTed to the provided `target` URL as
138+
JSON payloads (`run.started`, `run.event`, `run.completed`, `run.failed`).
139+
- `mode: "stdout"` / `"stdout_verbose"`: events are printed to stdout in a human
140+
friendly format for local testing; verbose mode also dumps final output.
141+
142+
## State tracking & approvals
143+
- Every run persists a `WorkflowExecutionState` entry containing status,
144+
last-agent information, last response ID, and optional final output.
145+
- MCP approval requests automatically push the execution into the
146+
`waiting_approval` status so a UI can pause the run until a response arrives.
147+
- The default store is in-memory; to make runs resumable across processes,
148+
implement `ExecutionStateStore` against your data layer (e.g., Postgres,
149+
Redis, Firestore).
150+
151+
## Limitations & roadmap
152+
- SQLite-backed session factory targets local experimentation; production builds
153+
may need pluggable stores and rotation policies.
154+
- Only a subset of tool types and guardrails are registered; expand by adding
155+
new factories to `Builder`.
156+
- Error handling is best-effort: callback publishing failures are logged but
157+
currently do not short-circuit runs.
158+
- Tracing and state APIs may evolve—expect breaking changes until this moves out
159+
of POC status.
160+
161+
## What’s next
162+
1. Support multiple model providers directly inside declarative workflow
163+
definitions so agents can route across backends.
164+
2. Integrate LLM observability (e.g., LangSmith, custom tracing exporters) into
165+
the runner lifecycle.
166+
3. Verify human-in-the-loop flows where executions halt on approval callbacks
167+
and resume from the exact same point once cleared.
168+
4. Expand workflow examples and automated tests to cover richer scenarios and
169+
guard future changes.
170+
171+
## Resources
172+
- [`workflowrunner/runner.go`](runner.go) – async orchestration entry point.
173+
- [`workflowrunner/builder.go`](builder.go) – declarative-to-agent translation.
174+
- [`workflowrunner/state.go`](state.go) – execution state tracker and store.
175+
- [`workflowrunner/examples`](examples) – runnable samples to experiment with.
176+
177+
We welcome feedback and issues while this matures; please treat the interface as
178+
unstable and pin revisions carefully if you integrate it early.

0 commit comments

Comments
 (0)