Skip to content

Commit 88edfe2

Browse files
authored
Add Performance section and fix architecture diagram alignment (#250)
1 parent 0c1f8c0 commit 88edfe2

File tree

1 file changed

+51
-47
lines changed

1 file changed

+51
-47
lines changed

README.md

Lines changed: 51 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
[![OpenSSF Best Practices](https://img.shields.io/cii/percentage/12085?label=OpenSSF%20Best%20Practices&logo=opensourcesecurity)](https://www.bestpractices.dev/projects/12085)
1414
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/microsoft/agent-governance-toolkit/badge)](https://scorecard.dev/viewer/?uri=github.com/microsoft/agent-governance-toolkit)
1515

16-
[Quick Start](#quick-start) · [Packages](#packages) · [Integrations](#framework-integrations) · [OWASP Coverage](#owasp-agentic-top-10-coverage) · [Deploy on Azure](docs/deployment/README.md) · [Architecture Notes](#architecture-notes) · [Contributing](CONTRIBUTING.md)
16+
[Quick Start](#quick-start) · [Packages](#packages) · [Integrations](#framework-integrations) · [OWASP Coverage](#owasp-agentic-top-10-coverage) · [Performance](#performance) · [Deploy on Azure](docs/deployment/README.md) · [Architecture Notes](#architecture-notes) · [Contributing](CONTRIBUTING.md)
1717

1818
</div>
1919

@@ -38,38 +38,40 @@ Addresses **10 of 10 [OWASP Agentic Top 10](https://genai.owasp.org/resource/owa
3838
## Architecture
3939

4040
```
41-
┌─────────────────────────────────────────────────────────────────┐
42-
│ Agent Governance Toolkit │
43-
│ pip install ai-agent-compliance[full] │
44-
├─────────────────────────────────────────────────────────────────┤
45-
│ (Python middleware layer) │
46-
│ ┌───────────────────┐ ┌───────────────────────────┐ │
47-
│ │ Agent OS Engine │◄────►│ AgentMesh │ │
48-
│ │ │ │ │ │
49-
│ │ Policy Engine │ │ Zero-Trust Identity │ │
50-
│ │ Capability Model │ │ Ed25519 / SPIFFE Certs │ │
51-
│ │ Audit Logging │ │ Trust Scoring (0-1000) │ │
52-
│ │ Action Interception│ │ A2A + MCP Protocol Bridge│ │
53-
│ └────────┬──────────┘ └─────────────┬─────────────┘ │
54-
│ │ │ │
55-
│ ▼ ▼ │
56-
│ ┌───────────────────┐ ┌───────────────────────────┐ │
57-
│ │ Agent Runtime │ │ Agent SRE │ │
58-
│ │ │ │ │ │
59-
│ │ Execution Rings │ │ SLO Engine + Error Budget│ │
60-
│ │ Resource Limits │ │ Replay & Chaos Testing │ │
61-
│ │ Runtime Sandboxing│ │ Progressive Delivery │ │
62-
│ │ Termination Ctrl │ │ Circuit Breakers │ │
63-
│ └───────────────────┘ └───────────────────────────┘ │
64-
│ │
65-
│ ┌───────────────────┐ ┌───────────────────────────┐ │
66-
│ │ Agent Marketplace │ │ Agent Lightning │ │
67-
│ │ │ │ │ │
68-
│ │ Plugin Discovery │ │ RL Training Governance │ │
69-
│ │ Signing & Verify │ │ Policy Rewards │ │
70-
│ └───────────────────┘ └───────────────────────────┘ │
71-
│ │
72-
└─────────────────────────────────────────────────────────────────┘
41+
╔═════════════════════════════════════════════════════════════════════════════╗
42+
║ ║
43+
║ ════════ AGENT GOVERNANCE TOOLKIT ═══════════ ║
44+
║ pip install ai-agent-compliance[full] ║
45+
║ ║
46+
║ Agent Action ───► POLICY CHECK ───► Allow / Deny (< 0.1 ms) ║
47+
║ ║
48+
║ ┌─────────────────────────────┐ ┌─────────────────────────────────┐ ║
49+
║ │ AGENT OS ENGINE │◄───►│ AGENTMESH │ ║
50+
║ │ │ │ │ ║
51+
║ │ ● Policy Engine │ │ ● Zero-Trust Identity │ ║
52+
║ │ ● Capability Model │ │ ● Ed25519 / SPIFFE Certs │ ║
53+
║ │ ● Audit Logging │ │ ● Trust Scoring (0-1000) │ ║
54+
║ │ ● Action Interception │ │ ● A2A + MCP Protocol Bridge │ ║
55+
║ └──────────────┬──────────────┘ └────────────────┬────────────────┘ ║
56+
║ │ │ ║
57+
║ ▼ ▼ ║
58+
║ ┌─────────────────────────────┐ ┌─────────────────────────────────┐ ║
59+
║ │ AGENT RUNTIME │ │ AGENT SRE │ ║
60+
║ │ │ │ │ ║
61+
║ │ ● Execution Rings │ │ ● SLO Engine + Error Budgets │ ║
62+
║ │ ● Resource Limits │ │ ● Replay & Chaos Testing │ ║
63+
║ │ ● Runtime Sandboxing │ │ ● Progressive Delivery │ ║
64+
║ │ ● Termination Control │ │ ● Circuit Breakers │ ║
65+
║ └─────────────────────────────┘ └─────────────────────────────────┘ ║
66+
║ ║
67+
║ ┌─────────────────────────────┐ ┌─────────────────────────────────┐ ║
68+
║ │ AGENT MARKETPLACE │ │ AGENT LIGHTNING │ ║
69+
║ │ │ │ │ ║
70+
║ │ ● Plugin Discovery │ │ ● RL Training Governance │ ║
71+
║ │ ● Signing & Verification │ │ ● Policy Rewards │ ║
72+
║ └─────────────────────────────┘ └─────────────────────────────────┘ ║
73+
║ ║
74+
╚═════════════════════════════════════════════════════════════════════════════╝
7375
```
7476

7577
## Packages
@@ -152,6 +154,22 @@ Works with **12+ agent frameworks** including:
152154
| Human-Agent Trust Deficit | ASI-09 | ✅ Full audit trails + flight recorder |
153155
| Rogue Agents | ASI-10 | ✅ Kill switch + ring isolation + behavioral anomaly detection ([Agent SRE](packages/agent-sre/src/agent_sre/anomaly/)) |
154156

157+
## Performance
158+
159+
Governance overhead is **sub-millisecond** — negligible compared to any LLM API call (typically 200–2,000 ms).
160+
161+
| Metric | Latency (p50) | Throughput |
162+
|---|---|---|
163+
| Policy evaluation (1 rule) | 0.012 ms | 72K ops/sec |
164+
| Policy evaluation (100 rules) | 0.029 ms | 31K ops/sec |
165+
| Kernel enforcement | 0.091 ms | 9.3K ops/sec |
166+
| Adapter overhead | 0.004–0.006 ms | 130K–230K ops/sec |
167+
| Concurrent throughput (50 agents) || 35,481 ops/sec |
168+
169+
**Bottom line:** Governance adds **< 0.1 ms per action** — roughly 10,000× faster than an LLM API call.
170+
171+
Full methodology, per-adapter breakdowns, and memory profiling: **[BENCHMARKS.md](BENCHMARKS.md)**. Benchmarks are reproducible via the scripts in each package's `benchmarks/` directory and run on every release via CI ([`.github/workflows/benchmarks.yml`](.github/workflows/benchmarks.yml)).
172+
155173
## Documentation
156174

157175
- **[Azure Deployment Guides](docs/deployment/README.md)** — AKS, Azure AI Foundry, Container Apps, OpenClaw sidecar
@@ -198,20 +216,6 @@ Default score for new agents: **500** (Standard tier). Score changes are driven
198216

199217
Policy enforcement benchmarks are measured on a **30-scenario test suite** covering the OWASP Agentic Top 10 risk categories. Results (e.g., policy violation rates, latency) are specific to this test suite and should not be interpreted as universal guarantees. See [`packages/agent-os/modules/control-plane/benchmark/`](packages/agent-os/modules/control-plane/benchmark/) for methodology, datasets, and reproduction instructions.
200218

201-
### Performance
202-
203-
Full benchmark results with p50/p95/p99 latencies, throughput numbers, and memory profiling are published in **[BENCHMARKS.md](BENCHMARKS.md)**. Headlines:
204-
205-
| Metric | Value |
206-
|---|---|
207-
| Policy evaluation (single rule) | 0.012 ms p50 — 72K ops/sec |
208-
| Policy evaluation (100 rules) | 0.029 ms p50 — 31K ops/sec |
209-
| Kernel enforcement overhead | 0.091 ms p50 — 9.3K ops/sec |
210-
| Adapter governance overhead | 0.004–0.006 ms p50 — 130K–230K ops/sec |
211-
| Concurrent throughput (50 agents) | 35,481 ops/sec |
212-
213-
Benchmarks run on every release via CI ([`.github/workflows/benchmarks.yml`](.github/workflows/benchmarks.yml)).
214-
215219
### Known Limitations & Roadmap
216220

217221
- **ASI-10 Behavioral Detection**: Fully implemented in Agent SRE — tool-call frequency analysis (z-score spike detection), action entropy scoring, and capability profile violation detection. See [`packages/agent-sre/src/agent_sre/anomaly/`](packages/agent-sre/src/agent_sre/anomaly/) (72 tests passing)

0 commit comments

Comments
 (0)