feat: Add comprehensive documentation for new features

- Introduced Decorator and YAML Bindings documentation outlining the `@module` decorator, automatic Pydantic model generation, and YAML binding file loading.
- Added Middleware System documentation detailing the composable middleware pipeline, lifecycle phases, and built-in middleware like `LoggingMiddleware`.
- Created Observability System documentation covering distributed tracing, metrics collection, and structured logging with OpenTelemetry integration.
- Developed Module Registry and Discovery System documentation explaining the 8-step discovery pipeline, module registration, and event system.
- Added Schema System documentation detailing schema loading, validation, `$ref` resolution, and export capabilities for structured module interfaces.

Author: tercel

docs/features/acl-system.md

@@ -0,0 +1,124 @@
+# Access Control System
+
+## Overview
+
+Pattern-based Access Control List (ACL) with first-match-wins evaluation for module access control. The system enforces which callers may invoke which target modules, using wildcard patterns, special identity patterns (`@external`, `@system`), and optional conditions based on identity type, roles, and call depth. Configuration can be loaded from YAML files and hot-reloaded at runtime.
+
+## Requirements
+
+- Implement first-match-wins rule evaluation: rules are evaluated in order, and the first rule whose patterns match the caller and target determines the access decision (allow or deny).
+- Support wildcard patterns for caller and target matching (e.g., `admin.*`, `*`), delegating to a shared pattern-matching utility.
+- Handle special patterns: `@external` matches calls with no caller (external entry points), and `@system` matches calls where the execution context has a system-type identity.
+- Support conditional rules with `identity_types` (identity type must be in list), `roles` (at least one role must overlap), and `max_call_depth` (call chain length must not exceed threshold).
+- Provide `default_effect` fallback (allow or deny) when no rule matches.
+- Load ACL configuration from YAML files via `ACL.load()`, with strict validation of structure and rule fields.
+- Support runtime rule management: `add_rule()` inserts at highest priority (position 0), `remove_rule()` removes by caller/target pattern match.
+- Support hot reload from the original YAML file via `reload()`.
+- All public methods must be thread-safe.
+
+## Technical Design
+
+### Architecture
+
+The ACL system consists of two primary components: the `ACLRule` dataclass representing individual rules, and the `ACL` class that manages a rule list and evaluates access decisions.
+
+#### Rule Evaluation
+
+```
+check(caller_id, target_id, context)
+  |
+  +--> effective_caller = "@external" if caller_id is None else caller_id
+  |
+  +--> for each rule in rules (first-match-wins):
+  |      1. Test caller patterns (OR logic: any pattern matching is sufficient)
+  |      2. Test target patterns (OR logic)
+  |      3. Test conditions (AND logic: all conditions must pass)
+  |      4. If all pass -> return rule.effect == "allow"
+  |
+  +--> No rule matched -> return default_effect == "allow"
+```
+
+#### Pattern Matching
+
+Pattern matching is handled at two levels:
+- **Special patterns** (`@external`, `@system`) are resolved directly in `ACL._match_pattern()` using caller identity and context.
+- **All other patterns** (exact strings, wildcard `*`, prefix wildcards like `executor.*`) are delegated to the foundation `match_pattern()` utility in `utils/pattern.py`, which implements Algorithm A08 with support for `*` wildcards matching any character sequence including dots.
+
+#### Conditional Rules
+
+When a rule has a `conditions` dict, all specified conditions must be satisfied (AND logic):
+- `identity_types`: Context identity's type must be in the provided list.
+- `roles`: At least one of the context identity's roles must overlap with the condition's role list (set intersection).
+- `max_call_depth`: The length of `context.call_chain` must not exceed the threshold.
+
+If no context is provided but conditions are present, the rule does not match.
+
+### Components
+
+- **`ACLRule`** -- Dataclass with fields: `callers` (list of patterns), `targets` (list of patterns), `effect` ("allow" or "deny"), optional `description`, and optional `conditions` dict.
+- **`ACL`** -- Main class managing an ordered rule list. Provides `check()`, `add_rule()`, `remove_rule()`, `reload()`, and the `ACL.load()` classmethod for YAML loading. All public methods are protected by `threading.Lock`.
+- **`match_pattern()`** -- Wildcard pattern matcher in `utils/pattern.py`. Supports `*` as a wildcard matching any character sequence. Handles prefix, suffix, and infix wildcards via segment splitting.
+
+### Thread Safety
+
+The `ACL` class uses an internal `threading.Lock` on all public methods. The `check()` method copies the rule list and default effect under the lock, then performs evaluation outside the lock. `add_rule()`, `remove_rule()`, and `reload()` all hold the lock for the duration of their mutations.
+
+### YAML Configuration Format
+
+```yaml
+version: "1.0"
+default_effect: deny
+rules:
+  - callers: ["api.*"]
+    targets: ["db.*"]
+    effect: allow
+    description: "API modules can access database modules"
+  - callers: ["@external"]
+    targets: ["public.*"]
+    effect: allow
+  - callers: ["*"]
+    targets: ["admin.*"]
+    effect: deny
+    conditions:
+      identity_types: ["service"]
+      roles: ["admin"]
+      max_call_depth: 5
+```
+
+## Key Files
+
+| File | Lines | Purpose |
+|------|-------|---------|
+| `src/apcore/acl.py` | 279 | `ACLRule` dataclass and `ACL` class with pattern matching, YAML loading, and runtime management |
+| `src/apcore/utils/pattern.py` | 46 | `match_pattern()` wildcard utility (Algorithm A08) |
+
+## Dependencies
+
+### Internal
+- `apcore.context.Context` -- Provides `identity`, `call_chain`, and other context fields for conditional rule evaluation.
+- `apcore.context.Identity` -- Dataclass with `id`, `type`, and `roles` fields used by `@system` pattern and condition checks.
+- `apcore.errors.ACLRuleError` -- Raised for invalid ACL configuration (bad YAML structure, missing keys, invalid effect values).
+- `apcore.errors.ConfigNotFoundError` -- Raised when the YAML file path does not exist.
+- `apcore.utils.pattern.match_pattern` -- Foundation wildcard matching for non-special patterns.
+
+### External
+- `yaml` (PyYAML) -- YAML parsing for configuration loading.
+- `threading` (stdlib) -- Lock for thread-safe access to the rule list.
+- `os` (stdlib) -- File existence checks in `ACL.load()`.
+- `logging` (stdlib) -- Debug-level logging of access decisions.
+
+## Testing Strategy
+
+### Unit Tests (`tests/test_acl.py`)
+
+- **Pattern matching**: Tests for `@external` matching None callers (and not matching string callers), `@system` matching system-type identities (and failing for None or non-system identities), exact patterns, wildcard `*`, and prefix wildcards like `executor.*`.
+- **First-match-wins evaluation**: Verifies that the first matching allow returns True, first matching deny returns False, and that rule order takes precedence over specificity.
+- **Default effect**: Tests both `default_effect="deny"` and `default_effect="allow"` when no rule matches.
+- **YAML loading**: Validates correct loading of rules with descriptions and conditions, and error handling for missing files (`ConfigNotFoundError`), invalid YAML, missing `rules` key, non-list `rules`, missing required keys (`callers`, `targets`, `effect`), invalid effect values, and non-list `callers`.
+- **Conditional rules**: Tests `identity_types` matching and failing, `roles` intersection matching and failing, `max_call_depth` within and exceeding limits, and conditions failing when context or identity is None.
+- **Runtime modification**: `add_rule()` inserts at position 0, `remove_rule()` returns True/False, `reload()` re-reads the YAML file and updates rules.
+- **Context interaction**: Verifies `caller_id=None` maps to `@external`, and context is forwarded to conditional evaluation.
+- **Thread safety**: Concurrent `check()` calls (10 threads x 200 iterations) with no errors, and concurrent `add_rule()` + `check()` with no corruption.
+
+### Integration Tests (`tests/integration/test_acl_enforcement.py`)
+- End-to-end tests exercising ACL enforcement through the `Executor` pipeline.

docs/features/core-executor.md

@@ -0,0 +1,93 @@
+# Core Execution Engine
+
+## Overview
+
+The Core Execution Engine is the central orchestration component of apcore. It processes module calls through a structured 10-step pipeline, handling everything from context creation and safety checks to module execution with timeout enforcement and result validation. The engine supports both synchronous and asynchronous execution paths, bridging between the two via threading and an async event loop bridge.
+
+## Requirements
+
+- Orchestrate module calls through a well-defined, sequential pipeline with clear separation of concerns at each step.
+- Enforce safety constraints including maximum call depth limits, circular call detection, and frequency throttling to prevent runaway or abusive execution.
+- Look up modules from the Registry and enforce access control lists (ACL) before execution.
+- Validate inputs and outputs using Pydantic models, with automatic redaction of fields marked as `x-sensitive`.
+- Support middleware chains that execute before and after the core module invocation, enabling cross-cutting concerns such as logging, metrics, and transformation.
+- Execute modules with configurable timeout enforcement, using daemon threads for synchronous modules and an async bridge for asynchronous modules.
+- Return structured results that include execution metadata and any errors encountered during the pipeline.
+
+## Technical Design
+
+### 10-Step Execution Pipeline
+
+The executor processes every module call through the following pipeline:
+
+1. **Context Creation** -- A `Context` object is constructed carrying the caller identity, call metadata, and any propagated state from parent calls. This context flows through every subsequent step.
+
+2. **Safety Checks** -- Three safety mechanisms are evaluated before proceeding:
+   - *Call depth check*: Rejects calls that exceed the configured maximum nesting depth, preventing unbounded recursion.
+   - *Circular call detection*: Inspects the call chain recorded in the context to detect and reject circular module invocations.
+   - *Frequency throttling*: Tracks call frequency per module and rejects calls that exceed the configured rate, protecting against tight-loop abuse.
+
+3. **Module Lookup from Registry** -- The target module is resolved by name from the Registry. If the module is not found or not loaded, the pipeline terminates with a descriptive error.
+
+4. **ACL Enforcement** -- The caller's `Identity` (extracted from the context) is checked against the module's access control list. Unauthorized calls are rejected before any execution occurs.
+
+5. **Input Validation with Pydantic + Sensitive Field Redaction** -- The call's input payload is validated against the module's input schema (a dynamically generated Pydantic model). Fields annotated with `x-sensitive` are redacted from logs and error messages using the `redact_sensitive` utility.
+
+6. **Middleware Before Chain** -- All registered "before" middleware functions are executed in order. Each middleware receives the context and validated input, and may modify or enrich them before the module runs.
+
+7. **Module Execution with Timeout** -- The module's handler is invoked. Timeout enforcement is implemented via daemon threads for synchronous handlers and an async bridge for asynchronous handlers. If the handler exceeds the configured timeout, the call is cancelled and a timeout error is returned.
+
+8. **Output Validation** -- The module's return value is validated against its output schema. Invalid output triggers an error rather than allowing malformed data to propagate.
+
+9. **Middleware After Chain** -- All registered "after" middleware functions are executed in order with access to the context, input, and output. These may perform logging, transformation, or cleanup.
+
+10. **Result Return** -- The final validated output (or error) is packaged into a structured result and returned to the caller.
+
+### Key Classes
+
+- **Executor** -- The main engine class that implements the 10-step pipeline. Manages middleware registration, timeout configuration, and the execution loop.
+- **Context** -- Immutable data class carrying call metadata: caller identity, call chain history, depth counter, and propagated key-value state.
+- **Identity** -- Represents the caller's identity for ACL enforcement. Carries roles, permissions, and an identifier.
+- **Config** -- Configuration data class holding executor-level settings such as max call depth, timeout defaults, and throttle limits.
+
+### Sync/Async Bridge
+
+The executor exposes both `execute()` (sync) and `execute_async()` (async) entry points. Internally:
+- Synchronous modules called from an async context are dispatched to a daemon thread via `asyncio.to_thread`.
+- Asynchronous modules called from a synchronous context are executed through a temporary event loop on a daemon thread.
+- An async module cache lock protects concurrent access to shared module state.
+
+### Sensitive Field Redaction
+
+The `redact_sensitive` utility walks the input/output dictionaries and replaces values of fields marked `x-sensitive: true` in the schema with a placeholder string. This ensures sensitive data never appears in logs or error reports.
+
+### Validation
+
+The `validate()` method on the executor provides a standalone validation path that runs input through the Pydantic model without executing the module, useful for pre-flight checks.
+
+## Key Files
+
+| File | Lines | Purpose |
+|------|-------|---------|
+| `executor.py` | 634 | Core execution engine implementing the 10-step pipeline |
+| `context.py` | 66 | Context and Identity data classes |
+| `config.py` | 29 | Executor configuration data class |
+| `errors.py` | 395 | Structured error types for every failure mode in the pipeline |
+
+## Dependencies
+
+### External
+- `pydantic>=2.0` -- Used for input/output schema validation, dynamic model generation, and field metadata.
+
+### Internal
+- **Registry** -- Module lookup (step 3) depends on the Registry system to resolve module names to loaded module instances.
+- **Schema System** -- Input and output validation (steps 5 and 8) depend on the Schema System for Pydantic model generation from YAML schemas.
+
+## Testing Strategy
+
+- **Unit tests** cover each pipeline step in isolation, verifying that context creation, safety checks, ACL enforcement, validation, middleware chains, and result packaging all behave correctly for both success and failure cases.
+- **Timeout tests** verify that both synchronous and asynchronous modules are correctly cancelled when exceeding configured timeouts, and that daemon threads do not leak.
+- **Safety check tests** exercise call depth limits, circular detection with various call chain topologies, and frequency throttle edge cases.
+- **Redaction tests** confirm that `x-sensitive` fields are properly masked in logs and error messages while remaining intact in the actual data passed to the module.
+- **Integration tests** run full pipeline executions through the executor with real Registry and Schema instances to verify end-to-end behavior.
+- Test naming follows the `test_<unit>_<behavior>` convention.