Skip to content

Commit 3d43f8d

Browse files
terceltercel
committed
feat: Introduce retry middleware, conformance tests, and utilities for normalization, call chains, and error propagation.
1 parent 20a24e5 commit 3d43f8d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+5327
-98
lines changed

‎.forge/feature-manifest.md‎

Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
# Feature Manifest: apcore-python Protocol Compliance (v0.7.1 → v0.8.0)
2+
3+
**Generated:** 2026-03-04
4+
**Project:** apcore-python SDK
5+
**Goal:** Achieve Level 1 full conformance (100%) and Level 2 substantial conformance (≥90%)
6+
**Current State:** Level 1 ~85%, Level 2 ~70%
7+
8+
---
9+
10+
## Dependency Graph
11+
12+
```
13+
F01 Config (A12) ─────────┬──→ F02 Timeout (A22)
14+
├──→ F07 Guard Call Chain (A20)
15+
├──→ F08 Version Negotiation (A14)
16+
└──→ F09 Safe Hot-Reload (A21)
17+
18+
F04 ID Normalization (A02) ──→ (standalone)
19+
20+
F05 Error Code Collision (A17) ──→ (standalone, enhances errors.py)
21+
22+
F06 ACL Specificity (A10) ──→ F10 ACL Audit Logging
23+
24+
F11 Retry Middleware ──→ (standalone, uses error.retryable)
25+
26+
F12 Streaming Deep Merge ──→ (standalone bug fix)
27+
28+
F03 Conformance Tests ──→ depends on F01, F02, F04, F05, F06, F07 (runs last)
29+
```
30+
31+
---
32+
33+
## Phase 1: Foundation (No Dependencies)
34+
35+
### F01: Config System (A12) — MUST, P0
36+
- **Scope:** Replace `config.py` stub with full YAML loading, env var overrides, validation
37+
- **Files:** `src/apcore/config.py` (rewrite), `tests/test_config.py` (new)
38+
- **Spec:** Algorithm A12 `validate_config()`
39+
- **Estimated Size:** ~250 LOC source + ~400 LOC tests
40+
- **Dependencies:** None (foundation for F02, F07, F08, F09)
41+
- **Blocked by:** Nothing
42+
- **Blocks:** F02, F07, F08, F09
43+
- **Key Requirements:**
44+
- YAML file loading with `Config.load(path)`
45+
- Environment variable overrides: `APCORE_{SECTION}_{KEY}` prefix
46+
- Merge priority: env vars > config file > defaults
47+
- Schema validation: required fields, type checking, constraint checking
48+
- Dot-path access preserved (backward compatible)
49+
- Hot-reload via `Config.reload()`
50+
- Required fields: `version`, `extensions.root`, `schema.root`, `acl.root`, `acl.default_effect`, `project.name`
51+
- Constraint validation: sampling_rate ∈ [0.0, 1.0], max_depth ∈ [1, 16], etc.
52+
53+
### F04: Cross-Language ID Normalization (A02) — MUST, P1
54+
- **Scope:** New utility function for cross-language module ID conversion
55+
- **Files:** `src/apcore/utils/normalize.py` (new), `tests/test_normalize.py` (new)
56+
- **Spec:** Algorithm A02 `normalize_to_canonical_id()`
57+
- **Estimated Size:** ~100 LOC source + ~200 LOC tests
58+
- **Dependencies:** None
59+
- **Blocked by:** Nothing
60+
- **Blocks:** F03
61+
- **Key Requirements:**
62+
- Input: `(local_id: str, language: str)` where language ∈ {python, rust, go, java, typescript}
63+
- Language-specific separators: Python ".", Rust "::", Go ".", Java ".", TypeScript "."
64+
- Case normalization: PascalCase/camelCase → snake_case
65+
- Acronym handling: `HttpJsonParser` → `http_json_parser` (not `h_t_t_p_...`)
66+
- Output validated against Canonical ID EBNF grammar
67+
- Export from `apcore.__init__`
68+
69+
### F05: Error Code Collision Detection (A17) — MUST, P1
70+
- **Scope:** Error code registry with collision detection
71+
- **Files:** `src/apcore/errors.py` (extend), `tests/test_error_codes.py` (new)
72+
- **Spec:** Algorithm A17 `detect_error_code_collisions()`
73+
- **Estimated Size:** ~80 LOC source + ~150 LOC tests
74+
- **Dependencies:** None
75+
- **Blocked by:** Nothing
76+
- **Blocks:** F03
77+
- **Key Requirements:**
78+
- `ErrorCodeRegistry` class with `register(module_id, codes)` method
79+
- Framework reserved codes: prefixes `MODULE_`, `SCHEMA_`, `ACL_`, `GENERAL_`, `CONFIG_`, `CIRCULAR_`, `DEPENDENCY_`
80+
- Detect: module code collides with framework code → error
81+
- Detect: module code collides with another module's code → error
82+
- Return complete code registry set
83+
- Run at framework startup (integrate with Registry.discover())
84+
- Thread-safe
85+
86+
### F06: ACL Specificity Scoring (A10) — SHOULD, P1
87+
- **Scope:** Pattern specificity calculation for ACL debugging
88+
- **Files:** `src/apcore/utils/pattern.py` (extend), `tests/test_specificity.py` (new)
89+
- **Spec:** Algorithm A10 `calculate_specificity()`
90+
- **Estimated Size:** ~40 LOC source + ~100 LOC tests
91+
- **Dependencies:** None
92+
- **Blocked by:** Nothing
93+
- **Blocks:** F10
94+
- **Key Requirements:**
95+
- `calculate_specificity(pattern: str) -> int`
96+
- Scoring: `"*"` → 0, exact segment → +2, partial wildcard segment → +1
97+
- Examples: `"*"` → 0, `"api.*"` → 2, `"api.handler.*"` → 4, `"api.handler.task_submit"` → 6
98+
- Export from `apcore.__init__`
99+
100+
### F12: Streaming Deep Merge (Bug Fix) — P0
101+
- **Scope:** Fix shallow merge in executor streaming accumulation
102+
- **Files:** `src/apcore/executor.py` (fix ~5 lines), `tests/test_executor_stream.py` (extend)
103+
- **Estimated Size:** ~30 LOC source + ~50 LOC tests
104+
- **Dependencies:** None
105+
- **Blocked by:** Nothing
106+
- **Blocks:** Nothing
107+
- **Key Requirements:**
108+
- Replace `{**accumulated, **chunk}` with recursive deep merge
109+
- Nested dicts merged recursively, lists replaced (not concatenated)
110+
- Existing flat streaming behavior unchanged
111+
- New test: overlapping nested keys across chunks
112+
113+
---
114+
115+
## Phase 2: Executor Enhancements (Depend on F01)
116+
117+
### F02: Timeout Enforcement (A22) — MUST, P0
118+
- **Scope:** Cooperative cancellation with grace period for sync modules
119+
- **Files:** `src/apcore/executor.py` (refactor timeout section), `tests/test_executor.py` (extend)
120+
- **Spec:** Algorithm A22 `enforce_timeout()`
121+
- **Estimated Size:** ~100 LOC source + ~150 LOC tests
122+
- **Dependencies:** F01 (reads timeout config)
123+
- **Blocked by:** F01
124+
- **Blocks:** F03
125+
- **Key Requirements:**
126+
- If timeout_ms == 0: skip timeout enforcement
127+
- Cooperative cancellation: set CancelToken before force-killing
128+
- Grace period: 5 seconds after cancel signal before giving up
129+
- Async path: `asyncio.wait_for()` + cancel token (already partially works)
130+
- Sync path: thread + cancel token signal + join(grace_period)
131+
- Log warning when thread cannot be killed (Python limitation)
132+
- Timing starts from first middleware before()
133+
134+
### F07: Guard Call Chain (A20) — MUST, P1
135+
- **Scope:** Extract call chain safety into standalone algorithm
136+
- **Files:** `src/apcore/utils/call_chain.py` (new), `src/apcore/executor.py` (refactor), `tests/test_call_chain.py` (new)
137+
- **Spec:** Algorithm A20 `guard_call_chain()`
138+
- **Estimated Size:** ~80 LOC source + ~120 LOC tests
139+
- **Dependencies:** F01 (reads max_depth, max_repeat config)
140+
- **Blocked by:** F01
141+
- **Blocks:** F03
142+
- **Key Requirements:**
143+
- `guard_call_chain(module_id, call_chain, config) -> None` (raises on violation)
144+
- Three checks: depth limit, circular detection, frequency throttling
145+
- Extract from `executor._check_safety()`, keep executor calling the utility
146+
- Configurable via Config: `executor.max_call_depth`, `executor.max_module_repeat`
147+
- Same error types: `CallDepthExceededError`, `CircularCallError`, `CallFrequencyExceededError`
148+
149+
### F08: Version Negotiation (A14) — MUST, P2
150+
- **Scope:** Semver compatibility checking between declared and SDK versions
151+
- **Files:** `src/apcore/version.py` (new), `tests/test_version.py` (new)
152+
- **Spec:** Algorithm A14 `negotiate_version()`
153+
- **Estimated Size:** ~80 LOC source + ~150 LOC tests
154+
- **Dependencies:** F01 (reads declared version from config)
155+
- **Blocked by:** F01
156+
- **Blocks:** F03
157+
- **Key Requirements:**
158+
- `negotiate_version(declared_version: str, sdk_version: str) -> str`
159+
- Parse semver (major.minor.patch, optional pre-release)
160+
- Major mismatch → `VERSION_INCOMPATIBLE` error
161+
- Declared minor > SDK minor → error (SDK too old)
162+
- Declared minor < SDK minor by >2 → deprecation warning
163+
- Same minor → effective = max(declared, sdk)
164+
- New error class: `VersionIncompatibleError`
165+
- Integrate into Config.load() or Executor initialization
166+
167+
---
168+
169+
## Phase 3: Advanced Features
170+
171+
### F09: Safe Hot-Reload (A21) — SHOULD, P2
172+
- **Scope:** Reference-counted safe module unregistration
173+
- **Files:** `src/apcore/registry/registry.py` (extend), `src/apcore/executor.py` (add ref counting), `tests/registry/test_hot_reload.py` (new)
174+
- **Spec:** Algorithm A21 `safe_unregister()`
175+
- **Estimated Size:** ~120 LOC source + ~200 LOC tests
176+
- **Dependencies:** F01 (config for timeout), F07 (executor awareness)
177+
- **Blocked by:** F01
178+
- **Blocks:** F03
179+
- **Key Requirements:**
180+
- `Registry.safe_unregister(module_id) -> bool`
181+
- Reference counting: executor increments on call start, decrements on call end
182+
- Mark module state as "UNLOADING" (new calls get MODULE_NOT_FOUND)
183+
- Wait for ref_count == 0 with configurable timeout (default 30s)
184+
- Call `on_unload()` hook after all executions finish
185+
- Idempotent: unregistering non-existent module returns True
186+
- Force-unload on timeout with logging
187+
- Thread-safe with atomic state transitions
188+
189+
### F10: ACL Audit Logging — P2
190+
- **Scope:** Structured audit trail for ACL decisions
191+
- **Files:** `src/apcore/acl.py` (extend), `tests/test_acl_audit.py` (new)
192+
- **Estimated Size:** ~80 LOC source + ~100 LOC tests
193+
- **Dependencies:** F06 (includes specificity in audit entries)
194+
- **Blocked by:** F06
195+
- **Blocks:** Nothing
196+
- **Key Requirements:**
197+
- `AuditEntry` dataclass: timestamp, caller_id, target_id, decision, matched_rule, specificity, context_summary
198+
- `ACL.set_audit_handler(handler)` — pluggable handler protocol
199+
- Default: no-op (zero overhead when not configured)
200+
- Built-in: `InMemoryAuditHandler` for testing, `LoggingAuditHandler` for production
201+
- Every `check()` call produces an audit entry
202+
- Thread-safe
203+
204+
### F11: Retry Middleware — P2
205+
- **Scope:** Configurable retry strategy middleware
206+
- **Files:** `src/apcore/middleware/retry.py` (new), `tests/test_retry_middleware.py` (new)
207+
- **Estimated Size:** ~120 LOC source + ~200 LOC tests
208+
- **Dependencies:** None (uses existing middleware + error.retryable)
209+
- **Blocked by:** Nothing
210+
- **Blocks:** Nothing
211+
- **Key Requirements:**
212+
- `RetryMiddleware(max_retries=3, strategy="exponential", base_delay_ms=100, max_delay_ms=10000)`
213+
- Strategies: "exponential" (base * 2^attempt), "fixed" (constant delay), "linear" (base * attempt)
214+
- Only retries if `error.retryable is True`
215+
- Jitter: optional random jitter to prevent thundering herd
216+
- Respects module timeout (total retry time < module timeout)
217+
- Implemented via `on_error()` hook — returns recovery by re-executing
218+
- Configurable per-module overrides via annotations
219+
220+
---
221+
222+
## Phase 4: Validation
223+
224+
### F03: Cross-Language Conformance Test Suite — P0
225+
- **Scope:** JSON fixture-based tests validating protocol compliance
226+
- **Files:** `tests/conformance/` (new directory), `tests/conformance/fixtures/` (JSON fixtures)
227+
- **Estimated Size:** ~500 LOC tests + ~300 LOC fixtures
228+
- **Dependencies:** F01, F02, F04, F05, F06, F07
229+
- **Blocked by:** All Phase 1-2 features
230+
- **Blocks:** Nothing
231+
- **Key Requirements:**
232+
- JSON fixtures define input → expected output for each algorithm
233+
- Fixtures shareable with TypeScript SDK (same JSON, different test runner)
234+
- Coverage sections:
235+
- §2: ID normalization (A01, A02)
236+
- §6: ACL pattern matching (A08), evaluation (A09), specificity (A10)
237+
- §8: Error codes and propagation (A11, A17)
238+
- §9: Config validation (A12)
239+
- §10: Redaction (A13)
240+
- §12: Executor pipeline (10 steps), timeout (A22), call chain (A20)
241+
- §13: Version negotiation (A14)
242+
- Each fixture: `{"algorithm": "A02", "input": {...}, "expected": {...}, "description": "..."}`
243+
- pytest parametrize over fixture files
244+
245+
---
246+
247+
## Implementation Order (Recommended)
248+
249+
```
250+
Week 1: Foundation
251+
F12 Streaming Deep Merge (bug fix, 1h)
252+
F06 ACL Specificity (small, 2h)
253+
F05 Error Code Collision (small, 3h)
254+
F04 ID Normalization (medium, 4h)
255+
F01 Config System (large, 8h)
256+
257+
Week 2: Executor & Protocol
258+
F02 Timeout Enforcement (medium, 6h)
259+
F07 Guard Call Chain (medium, 4h)
260+
F08 Version Negotiation (medium, 4h)
261+
F11 Retry Middleware (medium, 6h)
262+
263+
Week 3: Advanced + Validation
264+
F10 ACL Audit Logging (medium, 4h)
265+
F09 Safe Hot-Reload (large, 8h)
266+
F03 Conformance Tests (large, 8h)
267+
```
268+
269+
---
270+
271+
## Version Bump Plan
272+
273+
After all features complete:
274+
- Version: 0.7.1 → **0.8.0** (minor bump — new features, no breaking changes)
275+
- `Config` API is additive (new class methods, existing `get()` preserved)
276+
- `ACL` API is additive (new `set_audit_handler()`, specificity export)
277+
- `Executor` behavior change: timeout now cooperative (may affect edge cases)
278+
- `errors.py` additive (new `ErrorCodeRegistry`, `VersionIncompatibleError`)
279+
- New public exports: `normalize_to_canonical_id`, `calculate_specificity`, `guard_call_chain`, `negotiate_version`, `ErrorCodeRegistry`, `RetryMiddleware`, `AuditEntry`
280+
281+
---
282+
283+
## Risk Assessment
284+
285+
| Risk | Impact | Mitigation |
286+
|------|--------|------------|
287+
| Config rewrite breaks existing users | High | Preserve `Config(data=dict)` constructor + `get()` method |
288+
| Timeout change affects sync modules | Medium | Grace period + cooperative cancel before force |
289+
| Ref counting in executor adds overhead | Low | Atomic counter, negligible cost per call |
290+
| New dependencies needed | Low | All features use existing deps (pydantic, pyyaml) |
291+
| Conformance fixtures diverge from TS | Medium | Generate fixtures from spec, not from implementation |

‎CHANGELOG.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,62 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [0.8.0] - 2026-03-05
9+
10+
### Added
11+
12+
#### Executor Enhancements
13+
- **Dual-timeout model** — Global deadline enforcement (`executor.global_timeout`) alongside per-module timeout. The shorter of the two is applied, preventing nested call chains from exceeding the global budget.
14+
- **Cooperative cancellation** — On module timeout, the executor sends `CancelToken.cancel()` and waits a 5-second grace period before raising `ModuleTimeoutError`. Modules that check `cancel_token` can clean up gracefully.
15+
- **Error propagation (Algorithm A11)** — All execution paths (sync, async, stream) now wrap exceptions via `propagate_error()`, ensuring middleware always receives `ModuleError` instances with trace context.
16+
- **Deep merge for streaming** — Streaming chunk accumulation uses recursive deep merge (depth-capped at 32) instead of shallow merge, correctly handling nested response structures.
17+
18+
#### Error System
19+
- **ErrorCodeRegistry** — Custom module error codes are validated against framework prefixes and other modules to prevent collisions. Raises `ErrorCodeCollisionError` on conflict.
20+
- **VersionIncompatibleError** — New error class for SDK/config version mismatches with `negotiate_version()` utility.
21+
- **MiddlewareChainError** — Now explicitly `_default_retryable = False` per PROTOCOL_SPEC §8.6.
22+
23+
#### Utilities
24+
- **`guard_call_chain()`** — Standalone Algorithm A20 implementation for call chain safety checks (depth, circular, frequency). Executor delegates to this utility.
25+
- **`propagate_error()`** — Standalone Algorithm A11 implementation for error wrapping and trace context attachment.
26+
- **`normalize_to_canonical_id()`** — Cross-language module ID normalization (Python snake_case, Go PascalCase, etc.).
27+
- **`calculate_specificity()`** — ACL pattern specificity scoring for deterministic rule ordering.
28+
- **`parse_docstring()`** — Docstring parser for extracting parameter descriptions from function docstrings.
29+
30+
#### ACL Enhancements
31+
- **Audit logging** — `ACL` constructor accepts optional `audit_logger` callback. All access decisions emit `AuditEntry` with timestamp, caller/target IDs, matched rule, identity, and trace context.
32+
- **Condition-based rules** — ACL rules support `conditions` for identity type, role, and call depth filtering.
33+
34+
#### Config System
35+
- **Full validation** — `Config.validate()` checks schema structure, value types, and range constraints.
36+
- **Hot reload** — `Config.reload()` re-reads the YAML source and re-validates.
37+
- **Environment overrides** — `APCORE_*` environment variables override config values (e.g., `APCORE_EXECUTOR_DEFAULT_TIMEOUT=5000`).
38+
- **`Config.from_defaults()`** — Factory method for default configuration.
39+
40+
#### Middleware
41+
- **RetryMiddleware** — Configurable retry with exponential/fixed backoff, jitter, and max delay. Only retries errors marked `retryable=True`.
42+
43+
#### Registry Enhancements
44+
- **ID conflict detection** — Registry detects and prevents registration of conflicting module IDs.
45+
- **Safe unregister** — `safe_unregister()` with drain timeout for graceful module removal.
46+
47+
#### Context
48+
- **Generic `services` typing** — `Context[T]` supports typed dependency injection via the `services` field.
49+
50+
#### Testing
51+
- **Conformance test suite** — JSON fixture-driven tests for error codes, call chain safety, ACL evaluation, pattern matching, specificity, ID normalization, and version negotiation.
52+
- **New unit tests** — 17 new test files covering all added features.
53+
54+
### Changed
55+
56+
#### Executor Internals
57+
- `_check_safety()` now delegates to standalone `guard_call_chain()` instead of inline logic.
58+
- Error handling wraps exceptions with `propagate_error()` and re-raises with `raise wrapped from exc`.
59+
- Global deadline set on root call only, propagated to child contexts via `Context._global_deadline`.
60+
61+
#### Public API
62+
- Expanded `__all__` in `apcore.__init__` with new exports: `RetryMiddleware`, `RetryConfig`, `ErrorCodeRegistry`, `ErrorCodeCollisionError`, `VersionIncompatibleError`, `negotiate_version`, `guard_call_chain`, `propagate_error`, `normalize_to_canonical_id`, `calculate_specificity`, `AuditEntry`, `parse_docstring`.
63+
864
## [0.7.0] - 2026-03-01
965

1066
### Added
@@ -300,6 +356,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
300356

301357
---
302358

359+
[0.8.0]: https://github.com/aipartnerup/apcore-python/compare/v0.7.0...v0.8.0
303360
[0.7.0]: https://github.com/aipartnerup/apcore-python/compare/v0.6.0...v0.7.0
304361
[0.6.0]: https://github.com/aipartnerup/apcore-python/compare/v0.5.0...v0.6.0
305362
[0.5.0]: https://github.com/aipartnerup/apcore-python/compare/v0.4.0...v0.5.0

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "apcore"
7-
version = "0.7.1"
7+
version = "0.8.0"
88
description = "Schema-driven module development framework for AI-perceivable interfaces"
99
readme = "README.md"
1010
requires-python = ">=3.11"

0 commit comments

Comments
 (0)