|
| 1 | +# Provider/Model Fallback Explanation Model |
| 2 | + |
| 3 | +Epic 12 Task 12.1 defines the trace contract for explaining model and provider fallback decisions. |
| 4 | + |
| 5 | +## Goals |
| 6 | + |
| 7 | +- make every routing decision explainable in deterministic order |
| 8 | +- keep default output readable while preserving deep diagnostics for debugging |
| 9 | +- avoid exposing secrets or sensitive provider identifiers in normal traces |
| 10 | + |
| 11 | +## Trace structure |
| 12 | + |
| 13 | +A trace represents one routing decision in three stages: |
| 14 | + |
| 15 | +1. `requested`: what the caller asked for |
| 16 | +2. `attempted`: ordered candidates that were evaluated |
| 17 | +3. `selected`: final model/provider outcome |
| 18 | + |
| 19 | +Reference shape: |
| 20 | + |
| 21 | +```json |
| 22 | +{ |
| 23 | + "requested": { |
| 24 | + "category": "deep", |
| 25 | + "model": "openai/gpt-5.3-codex", |
| 26 | + "source": "user_override" |
| 27 | + }, |
| 28 | + "attempted": [ |
| 29 | + { |
| 30 | + "rank": 1, |
| 31 | + "model": "openai/gpt-5.3-codex", |
| 32 | + "provider": "openai", |
| 33 | + "result": "unavailable", |
| 34 | + "reason": "model_not_in_available_set" |
| 35 | + }, |
| 36 | + { |
| 37 | + "rank": 2, |
| 38 | + "model": "openai/gpt-5-mini", |
| 39 | + "provider": "openai", |
| 40 | + "result": "accepted", |
| 41 | + "reason": "category_default_fallback" |
| 42 | + } |
| 43 | + ], |
| 44 | + "selected": { |
| 45 | + "model": "openai/gpt-5-mini", |
| 46 | + "provider": "openai", |
| 47 | + "reason": "fallback_unavailable_model_to_category" |
| 48 | + } |
| 49 | +} |
| 50 | +``` |
| 51 | + |
| 52 | +## Output levels |
| 53 | + |
| 54 | +### Compact (default) |
| 55 | + |
| 56 | +- include only `requested.model/category`, final `selected`, and 1-line fallback reason |
| 57 | +- include attempted count but not full per-attempt details |
| 58 | +- optimized for `/routing status` and routine debugging |
| 59 | + |
| 60 | +### Verbose |
| 61 | + |
| 62 | +- include full `attempted` chain with rank and rejection reason per candidate |
| 63 | +- include resolution source metadata (`system_default`, `category_default`, `user_override`) |
| 64 | +- include deterministic timestamps or sequence ids when available |
| 65 | + |
| 66 | +## Redaction rules |
| 67 | + |
| 68 | +Always redact in both compact and verbose modes: |
| 69 | + |
| 70 | +- API keys, tokens, bearer strings, authorization headers |
| 71 | +- full endpoint query strings containing credentials |
| 72 | +- account-scoped identifiers that can reveal tenant internals |
| 73 | + |
| 74 | +Redaction behavior: |
| 75 | + |
| 76 | +- preserve structural placeholders (`***redacted***`) so traces remain parseable |
| 77 | +- keep provider class labels (`openai`, `anthropic`, etc.) unless explicitly marked sensitive |
| 78 | +- if a field is fully sensitive, replace value and attach a redaction reason code |
| 79 | + |
| 80 | +## Determinism requirements |
| 81 | + |
| 82 | +- attempted candidates must be listed in exact evaluation order |
| 83 | +- fallback reason codes must use stable identifiers (no free-form prose) |
| 84 | +- identical inputs and availability state must yield identical trace output |
| 85 | + |
| 86 | +## Integration targets |
| 87 | + |
| 88 | +Task 12.2 should emit this trace model from runtime routing. |
| 89 | +Task 12.3 should expose compact and verbose views via user-facing commands. |
| 90 | +Task 12.4 should verify deterministic traces and redaction safety. |
0 commit comments