[Draft] Bootstrap Python quality gates and remove the OBO import cycle

[Paulo Lacerda (placerda)]

Status

Incomplete adoption / draft. Do not merge or activate required checks.

Implements the orchestrator scope of Azure/GPT-RAG#681. Coordination:
Azure/GPT-RAG#689. Published documentation is coordinated through the draft
docs-branch PR Azure/GPT-RAG#688, not published by this feature PR.

Current head: 2be1223.
Branch: feature/python-module-boundaries; target: develop.
The earlier freeze was lifted. This is an active implementation checkpoint,
not a claim that the remaining broad-handler adoption is complete.

Delivered changes

• Development-only exact pins: Ruff 0.16.5, mypy 2.3.1, Import Linter
  2.14, Grimp 3.16; tested with Python 3.12.9. Existing Hatchling,
  src layout, pytest, runtime dependency source and frontend jobs remain.
• Full static AST graph including flat modules, namespace packages, deferred
  and type-only imports, facades, cross-package private members and exact
  dynamic-import sites. No runtime source import during analysis or cycle
  baseline. Import Linter/Grimp independently cross-check package coverage.
• Five blocking typing seeds, permanent automatic coverage for newly adopted
  modules across successive PRs and moves, individual diagnostic/source debt
  identities, annotation/suppression ratchet, empty baseline. Imported
  diagnostics outside declared blocking scope remain visible/nonblocking.
• Closed protected policy and report schemas, exact broad-handler inventory
  including Ruff-exempt logging/rethrow, exact exception fingerprints and
  executed named JUnit evidence. Candidate policies cannot approve themselves.
• Same-workflow quality-gate depends on actual Python/frontend and all five
  quality jobs. Protected evaluator, immutable Actions, read-only permissions,
  verified CODEOWNERS, independent policy/repository/base/head/toolchain/run
  identity validation and stale/skipped/malformed artifact rejection.
• Source-only static execution isolation at 703e67f:
  python -I -S, trusted venv wheel bootstrap without site initialization,
  sanitized environment/private snapshot cwd, no candidate plugin/custom
  interpreter/config extension, fresh caches and an execution audit hook.
  Real shadow-module, startup-hook, PYTHONPATH and plugin/cache fixtures.
  This is not an OS sandbox or protection against compromised trusted wheels.
• Complete 78 ModuleSurface records, all 11 fields, at
  61b26e4. Source-backed exports, ownership,
  importers, private relationships, aliases, immutable provenance and repeated
  move checks. Storage refinement reported to parent: current records live in
  protected .quality/module-surfaces.json; policy.modules remains the
  immutable adoption-name inventory so typing coverage cannot disappear.
• Removed Search/Foundry IQ OBO cycle through connectors.obo, preserving
  Search compatibility callables, request scope, identity and token cache
  behavior. Legacy retrieval plugin now uses existing get_genai_client and
  awaits embeddings, retaining typed error-result translation.

Runtime failure milestones

Parent-delivered and separately committed MAF correction:
f06d0cd.
MafLite and MAF Agent Service no longer convert outer failures into raw
ordinary answer text. Failures propagate through the existing typed turn/SSE
channel, with failed audit outcome and a constant-safe enclosing span/log.
The real chain covers early/partial output, initialization failure, success,
cancellation, history/finally persistence and an in-memory SDK exporter.
Exactly one generic SSE error is emitted; cancellation remains distinct.
This bounded correction does not certify every third-party span or other
strategy/legacy diagnostic as safe.

Earlier 5883d0a parser/retry milestone:
JWT base64/UTF8/JSON, blob URL, OBO JSON/expiry/fingerprint catches now use
actual parsing exceptions instead of arbitrary Exception. Existing malformed
input/error envelopes and zero-TTL behavior remain. Unexpected parser failures
propagate. Removed unnecessary environment-host and saved logger-level catches.
Real provider evidence exposed Tenacity's incorrectly bound retry callback;
the static one-argument callback now executes the configured retry budget,
and lookup recovery catches exhausted RetryError. Optional defaults/allow-none
and required-value errors remain. Five attempts/backoff configuration is unchanged.

Real AppConfig provider-to-startup/required-strategy evidence distinguishes
missing endpoint, authentication failure, unavailable provider and loaded
configuration with environment overrides on/off. Retrieval evidence separately
covers keyword fallback, empty provider context, tolerant composite siblings
and strict versus anonymous explicit search_failed connector outcomes.
These outcomes are not homogenized into fatal errors or mislabelled as audit.

The f3cf4d6 failure-boundary milestone:
optional SDK converter absence catches only ImportError, while unexpected
initialization failures propagate. Nested provider RetryError now remains
inside the actual retry budget instead of becoming a premature missing value.
Four exact inactive proposals cover the demonstrated AppConfig availability,
Search keyword/empty-context and strict/anonymous connector contracts. Those
logs retain classification/severity but no raw exception text/traceback.
Two redundant inner audit sanitizer catches and their obsolete proposals are
removed: the enclosing boundary still omits unreadable/partial containers and
releases cycle-detection state. Four SQL/Fabric catches now use actual
ODBC/Azure exception families; identical caller failures, token scopes and
credential cleanup remain, with bounded connector diagnostics.
No active approval or lint-avoidance logging exemption was added.

The 7565839 milestone closes the already-known hosted transport and MCP cleanup
boundaries: generic hosted errors remain deduplicated, partial output is
preserved without success closing frames, initialization failure is translated,
and cancellation propagates distinctly. MCP cleanup preserves the primary
failure/cancellation, including cleanup cancellation; without a primary failure
the identical cleanup error propagates. Two raw exception logs are now bounded
constant diagnostics. Three exact inactive proposals bind these contracts.
This deliberately adds two Ruff findings formerly exempted by traceback logging;
it does not hide either handler or claim broader telemetry confidentiality.
Parent owns the disjoint src/util/blob_sas.py and new
tests/test_blob_sas_failure_boundaries.py slice until handback; it is not
included in this head.

The acaefa4 milestone removes 11 broad handlers: six Cosmos SDK boundaries,
two Search token boundaries and three optional telemetry configuration catches.
Cosmos preserves Azure SDK not-found/forbidden/transport unavailable results,
principal partitions, document mutations and soft deletes. Unexpected programming
failures now propagate instead of being called missing documents. Actual HTTP
evidence proves SDK read failures retain legacy404, unexpected failures use500,
foreign-principal documents remain403, and cancellation propagates. Rename/delete
success bodies and SDK/unexpected mutation500 remain. The existing SDK
unavailable/404 mapping is preserved, not certified as a new availability policy.
Four exact inactive HTTP proposals and bounded route/connector logs accompany
the narrowing. HTTP500 cannot prove a remote mutation was rolled back.

Telemetry now uses get_value(..., allow_none=True) rather than masking failures:
missing/unavailable/default/environment precedence remains; callback defects
surface. The HTTP log-level setting is read once, not four times inside a logger
loop. Search token failures propagate unchanged without issuing an HTTP request.
Two legacy not-found fixtures now raise actual CosmosResourceNotFoundError
instead of generic Exception. No blanket exception tuple or active approval.

Current 2be1223 narrows JWT diagnostic parsing to ValueError and issuer
verification to PyJWTError; unexpected verification/provider failures retain
the existing fail-closed generic401, now with a bounded exception-class log.
One exact inactive proposal binds that HTTP boundary. Real, in-memory generated
RSA keys and signed JWTs prove v1/v2 issuer handling, expiry/audience/tenant/
signature/algorithm rejection, trusted role shape, rotation/alternate JWKS,
required-setting500, cancellation and tenant-scoped cache clearing. No signing
keys are committed; no identity/audience/cache policy changes are introduced.
Existing debug claim diagnostics are not globally certified by this change.

Exact current evidence

.\.venv\Scripts\python -m pip install -r requirements.txt
.\.venv\Scripts\python -m pip install pytest pytest-asyncio pytest-mock jsonschema
.\.venv\Scripts\python -m pip install -r requirements-quality.txt
.\.venv\Scripts\python -m pytest -q tests\test_quality_policy.py --junitxml=.artifacts\quality-policy.xml --tb=short
.\.venv\Scripts\python -m pytest -q --junitxml=.artifacts\pytest.xml --tb=short
.\.venv\Scripts\python -I -S .github\scripts\check-quality.py --check all --base-ref origin/develop --report .artifacts\quality.json --test-results .artifacts\pytest.xml
.\.venv\Scripts\python -m ruff check .github\scripts --isolated --select E9,F63,F7,F82,BLE001
.\.venv\Scripts\python .github\scripts\validate-agentic-assets.py
Push-Location frontend
npm run build
Pop-Location
git diff --check

Install commands are contributor setup, not claims of new installs in the
current batch. Runtime/tool pins were not changed in these follow-ups.
Individual checker selections: lint, typing, architecture, exceptions,
policy. Exit1 is violations; exit2 is incomplete/invalid analysis.
-I -S is required for static checker/aggregate startup; ordinary -I still
executes installed .pth/sitecustomize hooks.

Evidence	Result
ModuleSurface quality suite at61b26e4	228 passed, 57.66s
Parent MAF milestone integration selection	220 passed, 6 existing warnings, 10.03s
Parsing/config/retrieval/telemetry selection	153 passed, 6 existing warnings, 7.10s; three additional regression cases included in full suite
Provider/retrieval/MAF selection for current batch	222 passed, 6 existing warnings, 8.14s
Audit omission/export/lifecycle selection	91 passed, 7 existing warnings, 5.27s
SQL/Fabric and existing consumer selection	54 passed, 7.52s
Full suite on source committed asf3cf4d6	1102 passed, including 232 quality cases, 7 existing warnings, 95.20s
Frontend npm run build	Passed; existing large-chunk warning
Hosted/MCP regression before bounded logging	8 failed /6 passed, reproducing raw exception diagnostics
Current quality/hosted/MCP combined selection	383 passed, including 235 quality cases, 61.61s
Final hosted/MCP selection after stronger identity/status assertions	148 passed, 5.14s
Exact7565839 isolated full quality	Exit1: lint91, typing0, architecture0, exceptions153, policy1, using .artifacts\boundary-pytest.xml
Full local suite on source committed asacaefa4	1217 passed, including 239 quality cases, 7 warnings, 75.72s
Exactacaefa4 isolated full quality	Exit1: lint83, typing0, architecture0, exceptions146, policy1, using full .artifacts\pytest.xml
JWT regression before bounded diagnostics	2 failed /15 passed
Expanded real JWT/dashboard/config/parser selection	106 passed, 6 warnings, 6.13s
Current exact proposal/JWT selection	33 passed, 228 deselected, 12.96s; quality suite now 240 cases
Exact2be1223 isolated full quality	Exit1: lint81, typing0, architecture0, exceptions145, policy1
Current broad inventory / ledger	122 sites; 23 proposed / 0 active; 99 sites without proposals
Coverage / baseline	5/78 blocking, 73 explicitly uncovered; 0 debt entries
Linux CI at61b26e4 andf06d0cd	Python/frontend/typing/architecture/assets passed; lint/exceptions/policy/aggregate failed
Linux CI atf3cf4d6, run34060247333	Tests/frontend/typing/architecture passed; lint/exceptions/policy/aggregate failed; assets34060247323 passed
Linux CI at7565839, run34060631349	1116 passed, 8 warnings, 36.26s, Python3.12.14; frontend/typing/architecture/assets passed; lint/exceptions/policy/aggregate failed
Linux CI atacaefa4, run34061635704	1217 passed, 8 warnings, 40.44s; frontend/typing/architecture/assets passed; lint/exceptions/policy/aggregate failed
Current Linux CI at2be1223, run34062040196	1239 passed, 8 warnings, 36.22s; frontend/typing/architecture passed; lint/exceptions/policy/aggregate failed; assets34062040212 passed

The initial parser reproduction had three genuine unexpected-error swallow
failures and one incorrect nesting-depth fixture assumption. Correcting that
fixture and asserting actual provider call counts exposed two real callback
failures: 5 failed /44 passed before remediation. A first expanded selector
attempt used nonexistent auth test filenames and exited4 without running tests;
the corrected existing-file selection above passed. No failed command is
represented as a passing result.

Current batch regression receipts: optional-import/nested-retry selection
initially 3 failed /20 passed; bounded provider diagnostics 6 failed /23
passed; SQL/Fabric raw diagnostics 10 failed /5 passed. Each was repaired.
The new audit characterization initially assumed an object instead of the
existing JSON-string attribute; correcting the assertion gave 46 passed
before and after catch consolidation. The first consolidated suite exposed
the old hard-coded ten-site audit inventory assertion (1 failed /1082
passed); it now matches the eight remaining exact sites and still forbids
self-approval. The final consolidated receipt is the1102-pass result above.

Immutable MAF CI:
https://github.com/Azure/gpt-rag-orchestrator/actions/runs/34050328693
Current head CI is separate; prior-head results are not its execution receipt.
Exact current-head Linux receipt:
https://github.com/Azure/gpt-rag-orchestrator/actions/runs/34062040196
The parent-owned blob-signing slice remains outstanding and is not included.
Current batch commands include the full contributor suite/checker above and:

.\.venv\Scripts\python -m pytest -q tests\test_quality_policy.py tests\test_hosted_responses.py tests\test_mcp_strategy.py --tb=short --junitxml=.artifacts\boundary-pytest.xml
.\.venv\Scripts\python -m pytest -q tests\test_hosted_responses.py tests\test_mcp_strategy.py --tb=short --junitxml=.artifacts\boundary-pytest.xml
.\.venv\Scripts\python -I -S .github\scripts\check-quality.py --check all --base-ref origin/develop --report .artifacts\boundary-quality.json --test-results .artifacts\boundary-pytest.xml
.\.venv\Scripts\python -m pytest -q tests\test_cosmos_failure_boundaries.py tests\test_retrieval_logging.py tests\test_audit_telemetry.py tests\test_appconfig_failure_boundaries.py tests\test_audit_lifecycle.py --tb=short --junitxml=.artifacts\sdk-boundary-pytest.xml
.\.venv\Scripts\python -m pytest -q tests\test_auth_failure_boundaries.py tests\test_dashboard.py tests\test_appconfig_failure_boundaries.py tests\test_parsing_failure_boundaries.py --tb=short --junitxml=.artifacts\auth-boundary-pytest.xml
.\.venv\Scripts\python -m pytest -q tests\test_quality_policy.py tests\test_auth_failure_boundaries.py -k 'compatibility_proposal or auth_failure_boundaries' --tb=short --junitxml=.artifacts\auth-proposal-pytest.xml
.\.venv\Scripts\python -I -S .github\scripts\check-quality.py --check all --base-ref origin/develop --report .artifacts\auth-boundary-quality.json --test-results .artifacts\auth-proposal-pytest.xml

The SDK/configuration selection passed147 cases before adding eight real HTTP
mutation cases. Regression receipts were Cosmos32failed/12passed, Search4/2,
telemetry6/6 and API7/16 after correcting a new fixture's keyword signature.
The mutation success assertion was corrected for Pydantic's existing UTC Z
serialization, not a runtime change. Initial full suite1215passed/2failed caught
the legacy generic not-found fixtures; both corrected selectors passed and the
final integrated result is1217passed above.

The independent703 isolation review reported no significant new issues; no
repeat isolation implementation or pin changes are included in this batch.

The JUnit selector correction at93e4d5c adds no CLI flag: use --junitxml
and --test-results. Class-based selectors resolve against actual test modules;
duplicates/unknown modules fail, skipped/failed cases earn no credit. JUnit is
evidence, not approval or a cryptographic execution receipt.

Task accounting

Task	Code/evidence status
T002	Delivered source/workflow inventory and reconciled develop;78 records now complete
T006	Delivered exact pins, strict policy parsing and complete ModuleSurface records; storage refinement explicitly coordinated
T009	Delivered ratchet/tool-error/aggregate/stale/self-approval fixtures; seven independent original reviewer defects closed at550a6ee
T012	Delivered protected incremental Ruff/mypy enforcement; legacy lint adoption remains red rather than baselined away
T015	Delivered same-workflow protected execution, aggregate, immutable Actions and verified ownership; admin activation separate
T018	Delivered full graph/private/dynamic/broad-handler mutations, now included in240 quality cases
T021	Delivered OBO cycle removal and compatibility wrappers
T024	Enforcement delivered; handler disposition incomplete:99 sites still lack exact proposals, all23 proposals unapproved
T027	Bounded OBO/MCP/retrieval/actual MAF/audit/Cosmos-to-HTTP/real-JWT/cancellation evidence delivered; not a global legacy failure-contract certification
T039	AGENTS, PR commands, compatibility/recovery and coordinated documentation handoff delivered
T045 portion	Local1217 plus exact current Linux1239/frontend and current static evidence recorded; final adoption/live/admin acceptance incomplete

Remaining acceptance and recovery

Substantive code/review work remains, not just a GitHub settings switch:
81 BLE001 findings, per-site disposition/evidence for99 unproposed handlers,
independent review of23 exact proposals, and unresolved legacy primary
fallback/raw diagnostic behavior outside the corrected MAF boundary.
There are no active exceptions or cycle/debt baselines. Do not invent approvals,
use broad exception tuples to green lint, or infer approval from characterization.
The seven original tooling findings are independently closed; the subsequent
703 isolation review found no significant new issues. ModuleSurface and
bootstrap/policy adoption approval remain separate.

The23 proposals comprise eight audit, three classic stream, three hosted/MCP,
four conversation HTTP, four configuration/retrieval and one JWT boundary.
They are neither active exceptions nor permission for other sites.
Cosmos's narrower SDK unavailable results are preserved, not an approval of
legacy persistence availability semantics. Auth configuration fallback,
OBO acquisition recovery, legacy strategy terminal
handling, optional startup/provider cleanup and typed plugin translations
still require their own source/consumer disposition. This is remaining
implementation/review work, not merely human administration.

Concrete unresolved runtime decision sent to parent:
NL2SQLStrategy.initiate_agent_flow and
MultimodalStrategy.initiate_agent_flow still turn thrown failures into raw
ordinary answer/history and successful completion. Their source is untouched.
Multimodal buffers before yielding; NL2SQL explicit typed validation/execution
error answers are distinct from thrown failures. The parent MAF correction did
not itself authorize changing these two public outcomes. The two-path
blob-signing delegation also remains outstanding. These are not approvals or
claims that every other unproposed site is human-only work.

Bootstrap deliberately reports its missing trusted base policy. An independent
administrator must review bootstrap/policy, require actual quality-gate and
tests on develop/main, require latest-head code-owner review, dismiss stale
approvals and restrict bypasses. No settings were changed. A separately reviewed
policy repair, not candidate self-approval, is required for policy changes.

Compatibility remains with shipped UI v2.6.2 and ingestion v2.7.3;
no dependency on companion unmerged runtime PRs, schema migration, configuration
rename, identity/cache redesign or orchestrator reorganization is introduced.
The MAF failure outcome intentionally corrects the legacy raw-answer path using
the already-existing generic SSE error contract.

Recovery is the preceding orchestrator artifact v4.1.1, without peer/data
migration. Artifact rollback does not undo later data, configuration or settings
changes. Live Azure compatibility/recovery and administrative clean/failing-PR
merge-eligibility acceptance remain unperformed. No merge, settings change,
release/tag/image publication or deployment occurred.