Skip to content

docs: document Python quality checkpoints and contributor setup - #688

Draft
Paulo Lacerda (placerda) wants to merge 6 commits into
docsfrom
feature/python-quality-documentation
Draft

docs: document Python quality checkpoints and contributor setup#688
Paulo Lacerda (placerda) wants to merge 6 commits into
docsfrom
feature/python-quality-documentation

Conversation

@placerda

@placerda Paulo Lacerda (placerda) commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Status and scope

Documentation companion for #681, coordinated in #689. DRAFT: do not merge or publish. T005 and checkpoint T042 are complete for the coordinator-confirmed inputs below. This does not complete runtime acceptance, approve handlers, make quality gates green, or activate required checks.

Base: docs, from dfa4481 (documented exception to runtime develop). Branch: feature/python-quality-documentation. Consolidated documentation head: b81befa396f45657990fe2bad14a6e9911100fb7. The authorized final reconciliation is one commit after fa65c33 (+90/-32), with the required Copilot co-author trailer.

Changed path Full PR diff Scope
docs/contributing.md +281/-1 Branch/local-preview instructions; actual component setup and evidence interfaces; typing and inactive proposals; UI package/ownership/assets and bounded installed/container results
docs/howto_authentication.md +14/-2 Separate authorization from success; qualify successful soft refresh; explicitly unmerged ingestion Apply/PUT outcomes
docs/ingestion_nl2sql_data_source.md +9/-0 Explicitly unmerged confirmed-deletion and failed scan/delete/count propagation note

Total: 3 files, +304/-3. No new pages or navigation changes. Changed candidate behavior is labeled preview/unshipped, not current-release guidance.

Frozen component dependencies and actual CI evidence

All three companion PRs remain drafts targeting develop. Source guides, policy records and runtime links use immutable SHAs.

Component Frozen candidate Exact CI evidence Quality remains red
Azure/gpt-rag-orchestrator#346 8d0ac0532b06b2fe83a084d5748646c3009c7283 34043989448, tests job 101515653945: 978 passed, 8 warnings, 21.40s; frontend, typing and architecture pass 99 lint findings; 153 broad sites plus 11 proposed records; bootstrap 1; aggregate failed; 0 active exceptions
Azure/gpt-rag-ingestion#296 3a46472b19049631fa4427699a79968134a46769 34046006654, unit job 101521050678: 422 passed, 96 warnings, 15.24s; typing and architecture pass 131 lint; 210 exception findings = 202 sites + 8 proposals; policy 9; aggregate failed; 0 active exceptions
Azure/gpt-rag-ui#110 871106dbe891a1ccde373b4964c5e56a71c4f4cc 34045788683: unit job 101520472716, 471 cases, 74.981s; container job 101520472598, 410 behavioral cases, 4.397s, no skips; typing and architecture pass 20 lint; 63 exception findings; bootstrap 1; aggregate failed; 0 active exceptions

The exact job conclusions and behavioral totals above were read from GitHub API/logs. Quality counts and the bounded closure of the original checker findings are also supplied by the coordinator/owners; passing those replays is not blanket acceptance of every policy or runtime boundary.

Ingestion CI installed the declared azure-appconfiguration-provider==2.5.0 and azure-search-documents==11.5.2. The final 3a46472 fixture uses the real provider's paged loading/deduplication interface, asserts that the provider (not an environment fallback) was created, and retains the existing unlabelled-last result. The earlier 0fdfc46 run with 421 passed/1 failed is superseded; local provider 2.1.0 is not the final CI evidence. 3a46472 changes only that test fixture after the runtime restoration in f8c3f84 and checker work in 0fdfc46.

Actual contributor interfaces

  • Python 3.12 (owner local validation used 3.12.9); development pins Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14 and Grimp 3.16. Runtime requirements stay authoritative. Unavailable research versions are not presented as exercised tools.
  • Orchestrator: existing runtime/test dependencies plus requirements-quality.txt; pytest JUnit then check-quality.py --check all --base-ref <fetched-protected-target-sha> --report ... --test-results ....
  • Ingestion: pytest with --junitxml and -o junit_family=legacy; quality-evidence.py --junit ... --base-ref ... --report ...; quality checker consumes that JSON through --test-evidence.
  • UI: requirements, quality requirements, python -m pip install --no-deps -e .; existing unittest discovery remains supported. For a receipt, set QUALITY_RUN_ID once, run run-unittest.py --base-ref $Base --report .artifacts\unittest.json, then check-quality.py ... --test-evidence .artifacts\unittest.json. --test-evidence belongs to the checker, not the unittest runner. UI and ingestion JSON formats are not interchangeable, nor is either orchestrator JUnit.
  • Preserve base/head/source/run binding and actual job outcomes. A checksum, test name, skipped/expected-failure case or candidate approval string is not executed passing evidence or administrative approval.
  • Incremental typing seeds, stable identities and individual diagnostic debt remain explicit. Backend baselines remain empty; the exception ledgers now contain 11 orchestrator and 8 ingestion proposals, not empty ledgers or active approvals. UI's ledger is empty. Namespace discovery/explicit mypy package bases do not change runtime import paths.

UI package evidence and remaining boundary

The package installs src/gpt_rag_ui and explicit legacy adapters. Docker retains non-editable pip install --no-deps ., /app, and uvicorn main:app --host 0.0.0.0 --port 8080. requirements.txt remains the runtime authority. Assets stay external: public/, .chainlit/, chainlit.config.yaml, chainlit.md, VERSION; the existing asset-root resolution and defaults are unchanged.

Ten installed methods cover a clean venv with runtime requirements and pip check, non-editable wheel origins, the existing 410 behavioral cases copied as tests only, synthetic Entra/Copilot startup, missing/invalid active configuration, and actual Chainlit HTTPSession.persist_file/cleanup beneath the staged root. These are not claims of live identity sign-in or an external HTTP upload endpoint test.

Linux CI builds the existing Dockerfile into an ephemeral image and runs tests/container_smoke.py with --network none: 410 behavioral cases/no skips, real Uvicorn main:app loopback ready/not-ready responses, staged CSS/VERSION and installed origins. Earlier system-site-packages and globally missing Linux-evidence limitations are superseded. A missing local Docker pipe is only a local limitation. No image was published or deployed by this docs work.

Citation rendering receives explicit conversation_id, principal_id, copilot_session_id, without reading Chainlit context. The history API/service seam remains pending: api.history registers the layer, while its factory, orchestrator-backed BaseDataLayer and Chainlit session adapter remain in services.history. Preserve datalayer.OrchestratorDataLayer and datalayer.get_data_layer in a later split. This is not a universally framework-free DTO rewrite, a Cosmos-history migration, or closure of all U1/U4.

The orchestrator's real MAF raw-error/ordinary-response/audit-completion path remains characterization and an unaccepted legacy risk, not approved error handling. More generally, passing characterization tests does not approve inherited fallbacks. Live peer/identity/Azure integration, production recovery, remaining failure-boundary dispositions, independent approvals and administrative acceptance are not completed here.

Bounded operator impact and no-change assessment

Read the parent spec/plan/contracts and documentation-consistency skill; searched operator source for config endpoints, status wording, purge, Search deletion and late-scan outcomes. The earlier blanket operator no-change assessment is superseded by these two narrow changes:

  • Authentication diagram now says authorization permits attempting the operation, not guaranteed 200/202. The Apply paragraph describes the successful soft-refresh path. A pinned unmerged note records genuine ingestion schedule-application failure as HTTP 500, while durable PUT /api/config writes retain HTTP 200/applied if only their best-effort local refresh fails and there are no other write/scheduling failures. No new 207 contract is documented for that case. Explicit Reload/Apply remain separate operations.
  • NL2SQL's pinned unmerged note counts only confirmed Search deletions; failed/partial deletion, scan (including late pages) or counts propagate instead of producing a finished summary. No public status/event/summary field is renamed. Nominal purge/log examples do not promise success for failed operations.

Selected-provider read failures in the ingestion candidate now propagate with bounded Azure retries; real missing keys/defaults, environment opt-in, selector order and constructor fallbacks remain unchanged. These and Apply/delete corrections are observable failure-path fixes, not merely checker changes. Code rollback does not undo completed deletions or durable configuration writes.

docs/deploy.md, docs/hosted_continuity_platform_contract.md, docs/services_ingestion.md, Blob/SharePoint operator pages and mkdocs.yml are unchanged. Keep azd commands, isolation/sticky topology, CHAT_BACKEND, delegated BFF versus downstream OBO, HOSTED_CONTINUITY_ENABLED=false, owner validation and off/HTTP 503 behavior. No runtime, manifest/release pin, asset/config key, deployment setting or parent tasks file was edited.

Merge/publication gate

Keep this PR draft. Do not merge to docs while its dependencies are unresolved bootstrap checkpoints. Resolve and independently review component work, merge it through the owning PRs to the intended contributor targets, then reconcile any further changes and obtain the parent's separate docs-publication approval. Preview wording must reflect verified adoption, not be removed merely because tests pass.

Workflow/CODEOWNERS files do not activate required enforcement. Separately authorized administrators must configure required checks/latest-head code-owner review, dismiss stale approvals, restrict bypass and prove clean/negative PR merge eligibility. No handler approval, full SC-005, rule activation, live integration, merge, release, tag/image/package publication, deployment or settings change is claimed or performed by this documentation PR.

Documentation validation and handoff

Used only existing tooling. Local MkDocs 1.6.1/Python 3.12.9; unchanged docs workflow uses Python 3.11.

  • python -m mkdocs build --site-dir <session-artifacts>\docs-candidate-site passed after the final three-page edit, in 5.23s. Rendered immutable references, receipt commands and both unmerged operator notes were inspected.
  • git diff --check passed; untouched deployment/continuity/navigation diff passed. Same pre-existing INFO for orchestrator_visual_guide.md outside navigation; no new build issue.
  • Component evidence came from exact source, coordinator handoffs and actual CI API/logs; component suites were not run from this docs checkout. No new tools/dependencies installed, publishing workflow invoked or mkdocs gh-deploy used.
  • T005 impact assessment and T042 consolidated checkpoint guidance are complete for the frozen inputs. Remaining runtime/approval/adoption work remains with its owners; parent alone updates its tasks.

Related to #681; does not close it.

Prepare the docs-branch contributor handoff for #681 without publishing proposed component behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@placerda

Copy link
Copy Markdown
Contributor Author

Ingestion contributor interface is now stable in draft Azure/gpt-rag-ingestion#296, head bbe52923dbaf2b8ce4f6f371e492ad32ae7ffe45, target develop. Please keep publication/activation gated. See its docs/python-quality.md and PR body for the exact commands and ownership map.

Python 3.12.9; Ruff 0.16.5 / mypy 2.3.1 / Import Linter 2.14 / Grimp 3.16. Install requirements-quality.txt; run python .github/scripts/check-quality.py --check all --base-ref <protected-sha> --report .artifacts/quality.json. Optional --test-evidence takes same-candidate pytest node evidence; --repository is for base-side evaluation. Exit 0 pass / 1 violations / 2 error. Typing blocks audit_contract, audit_sanitizer and new jobs.runtime; independently discovered new modules become blocking. Individual multiset diagnostic identity, no count baseline/no global missing-import ignore. Zero baseline and exception records. 295 maintained tests passed at committed SHA, architecture passed, but 218 unapproved handlers and 131 Ruff findings keep this a draft. Required-check activation and live/recovery acceptance are not claimed. The plan's unavailable candidate pins are documented as a deviation, not represented as tested.

@placerda

Copy link
Copy Markdown
Contributor Author

Orchestrator contributor facts are stable in Azure/gpt-rag-orchestrator#346 (draft to develop), ef649eeab6144156b4c90c4422d62f229454dedc. Python 3.12.9; requirements-quality.txt pins Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16. CLI: python .github/scripts/check-quality.py --check all --base-ref --report .artifacts/quality.json --test-results .artifacts/pytest.xml. Produce evidence via python -m pytest -q --junitxml=.artifacts/pytest.xml (840 passed). Type scope: four planned seed modules plus connectors.obo, 5/78 modules, zero baseline entries; new modules join scope, imported uncovered errors remain visible. Full graph green; no exception exemptions approved: 154 broad handlers / 100 BLE001 findings are blocking, as is missing protected bootstrap policy. AGENTS.md supplies exact setup/ownership/activation guidance. Gate documentation must remain proposed, not active. Maintainer review, strict-policy follow-up and live/admin acceptance remain outstanding; see draft PR for all evidence and compatibility refs.

Ground contributor setup, typing scope and exception review in the actual orchestrator and ingestion drafts for #681. Keep enforcement and publication explicitly pending.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@placerda Paulo Lacerda (placerda) changed the title docs: prepare Python quality contributor guidance docs: document Python quality checkpoints and contributor setup Sep 6, 2026
@placerda

Copy link
Copy Markdown
Contributor Author

UI checkpoint is now Azure/gpt-rag-ui#110, an open draft targeting develop, at 4959ecaf04ea94ce0d35837c73b43ea3884c5be8; base c635bc6696714b543feec24b4a062a8a8f3ff6d0. This supersedes the earlier implementation-only ce76b00 checkpoint. The PR body carries the complete command/result/task/remaining-work handoff.

All 32 researched inventory entries have canonical package owners; this is not merely a skeleton. config/util, auth/clients, services, api/telemetry and bootstrap contain the implementations. Legacy root/connector modules are thin adapters; citations are service-owned and the app/history cycle is removed. Shared state and callback registration retain one canonical owner. Existing history still inherits Chainlit BaseDataLayer, and citation configuration is service-owned rather than fully context-injected; final contract review remains explicit.

Contributor commands:

python -m pip install -r requirements.txt
python -m pip install -r requirements-quality.txt
python -m pip install --no-deps -e .
python -m unittest discover -s tests -v
uvicorn main:app --host 0.0.0.0 --port 8080

Non-editable install: python -m pip install --no-deps ., or python -m pip wheel --no-deps --wheel-dir .artifacts/wheel . followed by installation of that wheel. Runtime dependencies remain exclusively in requirements.txt. Docker retains /app and uvicorn main:app; it adds non-editable package installation.

The artifact is code plus externally staged .chainlit, public, chainlit.config.yaml, chainlit.md, VERSION, not writable assets in site-packages. Existing CHAINLIT_APP_ROOT, an applicable source-adjacent root, or cwd resolves the resource root before Chainlit imports.

Evidence: 410 original/migrated tests passed; final complete suite passed 436 tests, including 17 quality, 3 legacy compatibility and 6 installed-package tests. Installed tests build a real wheel and run isolated subprocesses outside the checkout, assert non-editable UI module origins, exercise both import orders/once-only registration and ASGI startup/resource cases. They reuse third-party dependencies via system-site-packages: independent clean dependency resolution and the full installed security matrix remain pending. Actual Uvicorn import/ASGI TestClient evidence is not a deployed-listener test. Agent asset validation passed. Full-suite checkpoint was 34273ff; final 4959eca only changes documentation line endings and two EOF blank lines.

Quality CLI:

python .github/scripts/check-quality.py --check all --base-ref c635bc6696714b543feec24b4a062a8a8f3ff6d0 --report .artifacts/quality.json

Actual pins: Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16, setuptools 80.9.0. Research's higher Ruff/Import Linter/Grimp pins were unavailable. Blocking typing covers canonical config.chat_backend, config.panel_config, config.hosted_continuity_config, new modules and legacy adapters; moved legacy owners keep stable IDs/prior uncovered status. Typing baseline and approved exceptions are both empty. 119 imported legacy type diagnostics remain visible, not full-repository typing success.

At final head: typing and architecture pass; lint has 20 BLE001 findings, exceptions have 62 unapproved inherited handlers, policy has 1 bootstrap-review finding; combined exit 1. Do not document completed/activated quality adoption. Remaining checker work includes exhaustive schema/suppression/policy mutations, move/split debt identity, exact adapters, dynamic-import allowances and executed exception-test evidence. Real required-check/latest-head enforcement is not established.

T023 and T031-T036 implementation is present; T004/T030/T038 compatibility closure, T008/T011/T014/T017/T020/T026/T029 gate/error-policy closure, T037 container evidence and UI T045 final acceptance remain incomplete. T041 ownership/install/rollback guidance is in the PR and docs/python-development.md; parent/docs reconciliation should retain these limitations.

Linux container execution is blocked by the unavailable Docker Desktop Linux daemon. Live exact-peer integration (orchestrator v4.1.1, ingestion v2.7.3), recovery rehearsal and administrator/negative-PR enforcement evidence remain unperformed. No merges, releases/tags/images, deployment, settings changes or parent file edits occurred.

Dedicated cross-session/PR-creation tools were not exposed; this PR comment supplies the durable coordination handoff via GitHub CLI.

Record actual editable installation, canonical ownership and staged asset resolution at UI ce76b00 without claiming installed-wheel acceptance or active quality enforcement.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@placerda

Copy link
Copy Markdown
Contributor Author

T041 correction: ce76b00 was the implementation-only commit. The contributor/AGENTS update landed in 34273fffb1da549252e82ed0337e3cdd22636042 and is included in the current Azure/gpt-rag-ui#110 draft head 4959ecaf04ea94ce0d35837c73b43ea3884c5be8, targeting develop.

Use these immutable final-checkpoint references:

Confirmed UI CLI has only --check, --base-ref, --report (plus help); do not copy backend evidence flags.

Please distinguish completed local installed-wheel evidence from pending acceptance: six non-editable wheel/import/resource tests passed inside the 436-test local suite. Independent clean dependency resolution, the full installed security/configuration matrix, Linux container parity and CI/administrative acceptance remain unproven; the quality proposal remains red/incomplete. The PR body and previous coordination comment provide exact commands/results and scope. No further source changes or checkpoint expansion are needed for this documentation correction.

Link UI PR #110 and exact contributor revision; distinguish non-editable package evidence from pending clean dependency, Linux and expanded compatibility acceptance. Complete checkpoint documentation for #681 without publishing or claiming enforcement.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@placerda

Copy link
Copy Markdown
Contributor Author

Orchestrator follow-up facts for Azure/gpt-rag-orchestrator#346: head 4e7fdcad98af6e217d37d4d7fa6f3a314a3d91ca, 874 tests passed, pins/CLI unchanged. Five exact audit exception records are proposed and source/test-bound; zero are active. 154 broad sites / 100 BLE001 findings and bootstrap review remain blocking. Strict parsing and dynamic/member resolution now have expanded negative cases. The stronger resolver uncovered and repaired a stale AzureOpenAIClient import in the legacy retrieval plugin; it reuses get_genai_client and awaits embeddings, preserving the security filter/error envelope. I searched published docs/contributing.md, services_orchestrator.md, howto_retrieval_optimization.md and orchestrator_visual_guide.md for old/new connector/plugin names: no affected operator references. AGENTS and Unreleased changelog updated. Please keep gate claims proposed; current CI failures are not an active or completed enforcement rollout.

@placerda

Copy link
Copy Markdown
Contributor Author

Latest orchestrator evidence update: Azure/gpt-rag-orchestrator#346 at 29df99d9b1e1393d6b775d9d5362c445a7fdd021. 884 tests pass, including 117 policy fixtures; 10 exact audit recovery/propagation records are proposed, 0 active. Pins/CLI/type scope/zero baseline unchanged. All 154 broad sites remain unapproved (144 non-audit sites still need individual disposition); 100 BLE001 and bootstrap policy remain blocking. Proposed CODEOWNERS now uses two verified existing admins, placerda and gxjorge; settings are still untouched. Keep publication gated and do not infer completed enforcement from the passing behavioral suite.

@placerda

Copy link
Copy Markdown
Contributor Author

Ingestion documentation follow-up for Azure/gpt-rag-ingestion#296 is committed at 46d08d31e5090045e481f22da699e09f444159dc, still draft/develop. Please reconcile published ingestion/operator/audit guidance with these explicit bug-fix semantics: Search deletion uses SDK delete and confirmed matching counts; NL2SQL failed scans/partial deletion propagate; missing/malformed/duplicate Search results cannot emit positive audit events; governance provider failure no longer silently disables governance; config apply returns 500 on failed scheduling; remote config writes followed by failed local refresh use existing 207 partial-failure responses. Selected write/refresh/reschedule/manual scheduling diagnostics omit exception payloads. Successful API shapes, auth, labels, schema bytes and shipped peer versions are unchanged. Cron/manual failures propagate; ordered independent startup jobs still continue after one fails; audit side effects remain best-effort.

AGENTS/README/Unreleased/docs/python-quality.md and the component PR describe this. Gate remains incomplete: 348 tests pass, typing 3 modules/0 baseline entries, full graph no findings, but 202 unapproved handlers + 131 BLE001 + 4 proposed-record findings + 9 policy findings block acceptance. Four exact audit proposals, zero active approvals; proposed records cannot waive lint/exception checks. An exact active protected record may waive only its BLE001 header, with separate required same-run failure evidence. No activated enforcement/live validation claim. Parent #689 has the full follow-up handoff and owns published coordination.

Pin UI guidance to 043d89b and record clean runtime dependency installation, pip check and installed behavioral coverage. Preserve backend refs, contributor commands and pending runtime/container/quality/administrator acceptance.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@placerda

Copy link
Copy Markdown
Contributor Author

Stable orchestrator tooling repair: Azure/gpt-rag-orchestrator#346, exact SHA 550a6eedc022c879658964cbac919835c6d3b96d. Contributor dependency/test/check-quality commands and pins unchanged: Python 3.12.9; Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16 (development only). 185 quality fixtures / 952 suite tests pass. Important clarification for contributor page: new modules stay blocking relative to the protected adoption inventory, not only their first PR; imported uncovered diagnostics remain visible/nonblocking under accepted incremental scope. Source policy catches typing.no_type_check and aliases; malformed mypy JSON is an analysis error. Report schema v2 binds protected policy/toolchain, repo/base/head and CI run/attempt; aggregate independently validates from exact protected base, and earlier-attempt reports fail (rerun all quality jobs together). Local check-quality CLI unchanged, internal aggregate additionally takes --repository --run-id --run-attempt. Counts unchanged: 5/78 blocking, 73 uncovered, zero debt, 154 broad sites, 10 proposed/0 active; lint/exceptions/bootstrap and overall gate remain red. PR body has exact commands/results. No settings or publication authority implied.

@placerda

Copy link
Copy Markdown
Contributor Author

Scoped quality follow-up delivered

Azure/gpt-rag-ui#110 remains a draft targeting develop, at ae9d0d7d41556e0d8d4c4116fbb17765fbc1210f. This closes the requested quality-tooling follow-up, not overall quality adoption or administrator/live-environment acceptance.

Child commits:

  • ab523d84dde80e4c62bdf2c6cd7cdf52f5a50ad0: schema/source/evidence/aggregation hardening and the explicitly reassigned Linux container helper/job.
  • ae9d0d7d41556e0d8d4c4116fbb17765fbc1210f: additional parent-reviewed lexical alias, package initializer, next-PR typing coverage and malformed-diagnostic fixes.

Parent commits 043d89b87bd2504a9df1429df35c4ed05b4c2d60 and e9620fce81daa879c0d945113911e58ae4b574e3 were preserved. I did not edit or stage runtime modules, packaging metadata, test_installed_package.py, test_module_compatibility.py or .dockerignore. The working tree is clean. Both child commits include the required co-author trailer.

Delivered enforcement

  • Closed four-record schemas reject missing/unknown fields, duplicate JSON keys/IDs/sites, invalid versions/dates/types/paths, unknown scope references and duplicate move destinations.
  • Protected coverage retains stable identities, automatically types new modules and requires those additions to be persisted before merge, preventing coverage loss in the next PR. Unchanged one-to-one moves retain identity; ambiguous/changed moves and splits need explicit protected allocation. Debt retirement works without permitting growth, duplication or revival.
  • Qualified signatures cover class/nested functions, overload occurrences, variadic arguments, variables, decorators and module-level type aliases. Suppressions are syntax-site-bound rather than count-only. Lexical import aliases cannot be overwritten by unrelated functions; package-relative resolution honors inventoried __init__.py ownership.
  • Exact adapter imports, export targets, call/order fingerprints and declared facade chains must terminate at real canonical exports. Current runtime adapters pass; no runtime repair was made.
  • Dynamic loaders include literal keyword/alias forms. Variable sites need exact function/context fingerprints, nonempty permitted targets and executed behavior tests; a record cannot cover a second function with an identical call.
  • Handler fingerprints bind the protected operation, catch/outcome and distinct occurrence. Active protected records require exact source/types, date/stage validity and same-run, non-skipped passing tests. No approvals were fabricated; contractual best-effort outcomes remain representable.
  • Standard unittest execution receipts bind test method/source/outcome, candidate/base/source digest and run ID/attempt. Unknown mypy severities/malformed diagnostics fail execution. Protected aggregation checks actual job results and independently expected Git/source/run/tool values, rejecting even coordinated, resealed artifact forgeries.

The supplementary inventory has 63 sites. The previous checker reported 62 because two separate handlers in AppConfigClient.__init__ had identical handler-body fingerprints and collapsed into one dictionary entry. The new count is not a new runtime handler.

Commands and evidence

At final head, local focused execution:

$env:QUALITY_RUN_ID = [guid]::NewGuid().ToString()
.\.venv\Scripts\python.exe .github\scripts\run-unittest.py --base-ref c635bc6696714b543feec24b4a062a8a8f3ff6d0 --pattern test_quality_policy.py --report .artifacts\quality-receipt-ae9d0d7.json
.\.venv\Scripts\python.exe .github\scripts\check-quality.py --check all --base-ref c635bc6696714b543feec24b4a062a8a8f3ff6d0 --test-evidence .artifacts\quality-receipt-ae9d0d7.json --report .artifacts\quality-ae9d0d7.json

37 focused unittest tests passed in 37.037 seconds. These include real disposable Git repositories and unittest/aggregate subprocesses, not solely mocked success reports. New negative fixtures were run before fixes: the initial batch reproduced 13 failures/eight errors; the additional cross-cutting batch reproduced three failures/one missing-parser error. Dynamic alias/keyword cases were separately demonstrated failing before repair.

The final actual quality CLI exits 1:

Check Result
lint 20 inherited BLE001 violations
typing Passed, zero blocking findings
architecture Passed, zero graph/private/adapter findings
exceptions 63 unapproved inherited sites
policy One missing-protected-base bootstrap-review violation

Typing baseline and approved exception entries remain empty. Tool pins remain Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16; setuptools remains 80.9.0. Scoped Ruff checks and the existing agent-asset validator passed.

Actual Linux CI, not a missing-daemon assumption

Final-head run: https://github.com/Azure/gpt-rag-ui/actions/runs/34043024295

unit-tests, container-tests, typing and architecture succeeded. lint, exceptions, policy and quality-gate failed as expected from the outstanding violations above. This is real final-head CI evidence, not a pending or simulated Linux result.

The child-owned tests/container_smoke.py is outside normal test discovery. CI builds the existing Dockerfile into an ephemeral image, mounts the tests read-only and runs the helper with --network none. It verifies installed canonical origins, staged assets, original main:app Uvicorn listeners (ready and disconnected-not-ready) and all 410 existing behavioral cases without skips. Dependencies/images download only during the build; no image is pushed or deployed. Local helper validation also passed both listeners and 410 cases against a non-editable wheel; that local harness reused third-party dependencies and is separate from the parent's clean-environment proof.

The parent's installed tests now use clean dependencies and pip check, not system-site-packages; its ten-method coverage includes copied behavioral tests, real Entra/Copilot startup/auth order and upload write/cleanup cases. The previous installed-environment limitation is retired, not hidden.

Handoff and remaining acceptance

UI CLI now adds --test-evidence to --check, --base-ref and --report. The separate unittest runner supports --pattern for focused work; CI aggregation requires complete test_*.py discovery. CI supplies run ID/attempt automatically; local receipt commands set QUALITY_RUN_ID once. The required real jobs now include container-tests. Contributor/AGENTS guidance documents these controls and their limits; hashes are integrity checks, not authentication or substitutes for protected execution and actual job outcomes.

This advances T008/T011/T014/T017/T020/T026 and quality T041/T045; T037 now has actual Linux image/runtime evidence. Parent retains runtime/package/U4 closure and FR-008 handler decisions. Remaining adoption work is classification/narrowing or genuine exact approval of the inherited handlers, the bootstrap/protected-policy review route, administrator/latest-head/negative-PR enforcement evidence and unavailable live exact-peer integration/recovery. No all-fallbacks-forbidden rule, blanket ledger or self-approval was introduced.

No merge, tag/release/image publication, deployment, settings change or parent repository file edit occurred. Pausing this scoped turn at the committed, verified handoff.

@placerda

Copy link
Copy Markdown
Contributor Author

Orchestrator draft Azure/gpt-rag-orchestrator#346 now at 8d0ac0532b06b2fe83a084d5748646c3009c7283. Contributor CLI and Python/tool pins unchanged. Current counts: 978 tests, 190 quality fixtures; 5/78 blocking, zero baseline; 153 broad sites, 11 proposed (10 audit+1 turn), zero active. JUnit class selectors resolve to real source modules and duplicate/unknown evidence is rejected. Quality remains red: lint99, exceptions164, bootstrap1. Source-backed provider→startup/strategy and real MafLite→turn→HTTP SSE characterization now documents the legacy raw-error text/logging and success-shaped audit outcome risk; do NOT describe all strategies as safely propagating errors or these tests as confidentiality acceptance. Only runtime change in this latest batch removes the unnecessary host-formatting broad catch after getenv string/missing checks. All other fallback/error contracts remain unchanged. PR body has exact commands and residual acceptance; no settings activation or live validation performed.

@placerda

Copy link
Copy Markdown
Contributor Author

Final orchestrator scoped freeze confirmed at Azure/gpt-rag-orchestrator#346 head 8d0ac0532b06b2fe83a084d5748646c3009c7283, clean tree, draft/develop. Immutable-head rerun: 978 pass, 7 existing warnings; 190 quality cases; exact-head CI Python/frontend/typing/architecture pass. Current 153 broad/11 proposed/0 active; 142 unproposed. Contributor CLI/pins unchanged. Clarification for 93e4d5c: --test-results still consumes pytest --junitxml output; resolver now maps classname to an existing test module followed by ::Class::test, rejects unknown modules/duplicate selectors, excludes failed/error/skipped evidence. --root is respected internally. JUnit itself is not signed or an approval; provenance comes from protected execution/actual needs/same-workflow artifact download and report-v2 run/attempt checks. Final PR body now explicitly says recovery to v4.1.1 is unexercised and does not restore subsequent conversation data, persisted config or administrative settings. Seven original reviewer findings closed independently at 550a6ee; overall adoption, raw-error/MAF semantics, genuine approvals, activation and live compatibility/recovery remain open. No publication/settings change.

@placerda

Copy link
Copy Markdown
Contributor Author

Compatibility correction from the immutable ingestion assessment: the pending follow-up restores PUT /api/config durable-write success (200/applied) even when its PRE-EXISTING best-effort local refresh fails; do not publish the earlier proposed 207 behavior for that case. /config/apply 200-to-500 for genuine application failure and worker/provider exception propagation are observable contract-restoring changes, not identical failure semantics. Selected-provider read failures propagate while real missing-key defaults, env opt-in, bootstrap fallbacks and selector order remain. Eight proposed records, zero active. Purge cleanup and pinned SDK/audit independence evidence are being finalized. Recovery must explicitly state that code rollback does not recreate deleted Search documents or undo persisted config; /ingest-documents direct upload is outside wrapper proof. Component README/contributor/changelog guidance is updated locally; exact runtime follow-up SHA will follow.

@placerda

Copy link
Copy Markdown
Contributor Author

Frozen final UI checkpoint: Azure/gpt-rag-ui#110, draft to develop, head 871106dbe891a1ccde373b4964c5e56a71c4f4cc. Working tree is clean. The updated PR body contains the three-repair fixture mapping, exact commands, changed surfaces and remaining acceptance. No further implementation is underway.

Counts: 48 focused quality tests pass. Final CI https://github.com/Azure/gpt-rag-ui/actions/runs/34045788683 passed 471 unit tests, typing and architecture; offline Linux acceptance passed both real main:app listener states and 410 behavioral cases. Adoption remains red: 20 lint /63 unapproved handlers /1 bootstrap-policy finding, with quality-gate correctly failing. Baseline and approved exception entries remain empty.

CLI unchanged: check-quality.py --check all --base-ref <protected-base-sha> --test-evidence <unittest-receipt> --report <quality-report>. The separate standard-unittest runner retains --pattern; local evidence sets QUALITY_RUN_ID once, CI uses run ID/attempt. No new tools or pins; mypy-only explicit bases are src and ..

All three parent repros are now independently confirmed fixed at the immutable head. Runtime and parent-owned files were preserved. U1's bounded Chainlit/history seam remains pending under parent ownership; no DTO/storage redesign, approval fabrication, merge, publication, deployment or settings change occurred. Paused for the single docs/umbrella reconciliation.

Pin orchestrator 8d0ac05, ingestion 3a46472 and UI 871106d. Document actual UI receipts and installed/container evidence, inactive exception proposals and pending history separation. Clarify authorization versus success and preview confirmed ingestion Apply/purge failures without changing defaults or claiming adoption. Companion to #681 and #689; remains draft on docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@placerda

Copy link
Copy Markdown
Contributor Author

Final ingestion docs pin: Azure/gpt-rag-ingestion#296 at 3a46472b19049631fa4427699a79968134a46769, frozen clean draft/develop. Parent #689 and the rewritten component body contain the consolidated exact commands/task/recovery evidence.

Use current counts: local/CI 422 passed, checker subset116, bounded SDK/runtime subset60; graph53/145 (Grimp41), typing3 blocking/0 baseline/0 in-scope and344 imported local diagnostics. Eight proposed, zero active exceptions. Full gate exits1 with350 findings (202 handlers +131 BLE001 +8 pending +9 policy); CI unit/typing/architecture/assets pass, aggregate fails for those violations. Current CI34046006654 has no remaining unit fixture failure. All five original tooling findings independently closed.

Publish PUT200/applied retained after successful durable write + failed best-effort local refresh, not the superseded207 claim. Genuine /config/apply200-to-500 failures and worker/provider propagation are observable contract-restoring changes, not unchanged error semantics. Real provider2.5.0 preserves selector order and last-selected duplicate precedence; Search11.5.2 models are deserialized; prior ambient2.1/11.7 evidence is superseded, with no runtime pin change. Audit remains best-effort; cleanup warnings cannot replace a primary failure or invalidate confirmed work.

Direct /ingest-documents remains outside wrapper failure proofs. Rollback cannot restore deleted Search documents or undo persisted configuration; those need separately authorized data/config recovery. Shipped peers remain orchestrator4.1.1 andUI2.6.2; no schema/audit bytes, credentials, labels, VERSION or frontend workflow changes. No merge/publish/deploy/settings changes. This is the final handoff for your consolidated docs/umbrella commit, not green quality adoption.

@placerda

Copy link
Copy Markdown
Contributor Author

Atualizacao CLI estabilizada no milestone orch 703e67f94f4f36971bdb388e0eacb920560f9b6e (draft Azure/gpt-rag-orchestrator#346/develop): checker agora exige python -I -S .github\scripts\check-quality.py --check all --base-ref $Base --report .artifacts\quality.json --test-results .artifacts\pytest.xml. Mesmo -I -S no aggregate. Demais comandos de dependencias/pytest inalterados; Python3.12.9, Ruff0.16.5, mypy2.3.1, ImportLinter2.14, Grimp3.16. Ferramentas analisam snapshot fonte sem import de runtime, startup hooks desativados, caches novos e configs executaveis rejeitadas. CI quality usa deps runtime da base protegida; testes seguem deps candidatas em job separado. 201 casos tooling passaram; lint/exceptions/adocao ainda nao verdes. Zero aprovacoes ativas/zero divida typing; imported-outscope continua visivel nao bloqueante. Nao publicar como gate ativado. Revisao T006 ModuleSurface e correcoes runtime continuam; parent assumiu MAF.

@placerda

Copy link
Copy Markdown
Contributor Author

Contributor metadata refinement stable at orchestrator61b26e4c829c4c2ca81b46926a248cd317a87961 (draft Azure/gpt-rag-orchestrator#346): .quality/module-surfaces.json now holds complete78 ModuleSurface records/all11 planned fields. policy.modules remains immutable adoption names so new modules cannot later become uncovered. Full surface file joins protected digest/review; consumers/exports/provenance and stable move identity are enforced. 5 explicit blocking seeds, empty debt, 0active approvals unchanged. CLI/pins unchanged from703e67: Python3.12.9; Ruff0.16.5/mypy2.3.1/ImportLinter2.14/Grimp3.16; python -I -S .github/scripts/check-quality.py --check all --base-ref --report .artifacts/quality.json --test-results .artifacts/pytest.xml. Fresh JUnit is required for exceptions. Quality fixture suite228passed. Parent currently owns an uncommitted MAF correction, so final runtime behavior/docs SHA still pending; do not publish characterization as approved behavior.

@placerda

Copy link
Copy Markdown
Contributor Author

Runtime documentation update now stable at orchestrator f06d0cd7204c63e61ce1b6c80ae768c5f7f0597c (draft Azure/gpt-rag-orchestrator#346): MafLite and MAF Agent Service failures now use existing generic typed error/SSE channel, never raw exception text as ordinary answer. Audit rejects/fails the request instead of successful completion; cancellation remains distinct; existing frame/schema/headers and finally persistence preserved. Enclosing stream_response SDK span disables automatic exception capture and has constant internal_error status; SSE log is constant. This does NOT promise all provider telemetry sanitized. Parent authored bounded runtime correction; own rerun220passing confirms relevant strategies/hosted/history/audit/schema.13 exact exception proposals/0active now; adoption gate still red. Dependencies/tool pins/isolated CLI unchanged. Final full-suite SHA remains pending remaining parser work.

@placerda

Copy link
Copy Markdown
Contributor Author

Stable parser/runtime milestone:5883d0a4abdd98146b53d2c4858ccf971d707655 on draft Azure/gpt-rag-orchestrator#346 (develop). Full1071passed/7warnings; frontend passed. Tool CLI unchanged from isolated703e67f: python -I -S .github\scripts\check-quality.py --check all --base-ref origin/develop --report .artifacts\quality.json --test-results .artifacts\pytest.xml. Use pytest -q --junitxml=.artifacts\pytest.xml first. Python3.12.9; Ruff0.16.5/mypy2.3.1/ImportLinter2.14/Grimp3.16.78surface records,5blocking/73uncovered; baseline0;142broad sites/13proposed/0active/129unproposed. Lint91,typing0,architecture0,exceptions155,policy bootstrap1. Beyond prior safe MAF outcome: malformed parsing retains recovery, unexpected parser failures propagate; actual Tenacity callback repaired so existing5attempt retry budget works before optional default/required-setting failure. No config key/default change. Remaining legacy diagnostics/fallbacks not certified. PR body now replaces obsolete frozen/rawMAF/isolation-unverified claims. Local passing source evidence is distinct from pending exact-head CI; no admin/live acceptance or publication.

@placerda

Copy link
Copy Markdown
Contributor Author

Stable pushed orchestrator receipt: f3cf4d60470867647bedf650b4a716fcf0d7f7ef, draft Azure/gpt-rag-orchestrator#346/develop. Local full1102 pass; isolated all lint89/typing0/architecture0/exceptions150/policy1,135 broad/15 proposed/0 active/120 unproposed,5/78 blocking/0 debt. Exact Linux CI34060247333 completed: tests/frontend/typing/architecture passed; lint/exceptions/policy/aggregate failed; assets34060247323 passed. SQL/Fabric driver/Azure exceptions now narrow with unchanged propagation/credential cleanup; optional converter ImportError-only; nested provider RetryError retried; bounded AppConfig/Search logs; redundant inner audit catches removed. Contributor CLI/pins unchanged: Python3.12.9, Ruff0.16.5/mypy2.3.1/Import Linter2.14/Grimp3.16, checker must use -I -S and --test-results JUnit. Documentation must retain draft/adoption/approval/admin/live limitations. Parent now owns a disjoint blob_sas batch; I am closing exact hosted transport/MCP cleanup evidence, not altering wire contracts or reauditing isolation. Final head will follow.

@placerda

Copy link
Copy Markdown
Contributor Author

Docs stable orchestrator head now acaefa49e46735dd506750fd442b87077d29f200 (draft Azure/gpt-rag-orchestrator#346). Full1217pass/7warnings;239qualitycases; lint83/typing0/architecture0/exceptions146/policy1;124broad/22proposed/0active/102unproposed;5/78blocking/0debt. CLI/pins unchanged. Latest behavior: typed Cosmos SDK failures preserve existing unavailable results and SDK-read404/API-mutation500; unexpected programming errors now propagate to generic500 instead of being mislabeled missing, cancellation/principal guards unchanged. Four exact HTTP proposals and bounded diagnostic logs. Telemetry optional settings use allow_none; defaults/env precedence/unavailability retained, callback bugs surfaced, HTTP setting read once. Search token failures never issue a request and propagate unchanged. No persistence migration or claim of rollback after failed writes. Current PR body has exact commands/results/recovery/remaining gaps; parent blob-signing work not yet included. Linux756 receipt1116pass/8warnings/Python3.12.14; currentaca CI separate.

@placerda

Copy link
Copy Markdown
Contributor Author

Stable head2be12230003c140ed3fcd3204ea73e6afbb25661 now has exact Linux receipt34062040196:1239passed/8warnings/36.22s; frontend/typing/architecture/assets passed; lint/exceptions/policy/aggregate red as documented. Local prior full1217, current auth/consumer106 and exact-record/auth33 pass. Current inventory122broad/23proposed/0active/99unproposed;240qualitycases; lint81/typing0/architecture0/exceptions145/policy1;5/78blocking/0debt. Pins and -I -S/JUnit CLI unchanged. JWT parser/verifier narrowing preserves actual signed v1/v2, rotation/alternate JWKS, role/issuer/audience/tenant checks, required-setting500 and cancellation; unexpected provider/verifier failures still deny401 with bounded class-only diagnostics. No auth/cache policy change. Draft Azure/gpt-rag-orchestrator#346 body now includes exact source/task/command/CI/recovery accounting, pending legacy NL2SQL+multimodal thrown-failure disposition and outstanding parent blob-signing delegation. No global confidentiality/adoption/required-settings/live-recovery claim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant