docs: coordinate Python quality gates and UI package migration - #689
docs: coordinate Python quality gates and UI package migration#689Paulo Lacerda (placerda) wants to merge 15 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record the maintainer-approved design, compatibility contracts and 46 executable tasks for #681. Keep runtime and administrative acceptance separate from planning. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
⛔ Pull requests to
|
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve all 46 task IDs and recorded progress. Make contract coverage, policy ownership, acceptance evidence and execution dependencies explicit for the requested speckit-tasks refresh. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Orchestrator source reconciliation is unchanged at c6d0ccb01a40071f82f30bd17c9fe566b3d0ad18; isolated branch feature/python-module-boundaries. Python 3.12.9 environment restored from requirements.txt. The researched Ruff 0.16.6 / Import Linter 2.15 / Grimp 3.17 pins are not published on the available package index; installed alternatives are Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16. Full AST plus Grimp/import-linter architecture analysis passes after the local OBO extraction (including flat, deferred and type-only edges); 209 focused tests pass. Important scope conflict: source contains 155 pre-existing broad handlers (154 after narrowing the JSON parser), including configuration/retrieval best-effort fallbacks beyond audit. I will not bulk-approve or silently change these runtime contracts. Building a fail-closed draft PR with exact inventory and review blockers unless these can be individually resolved with existing failure evidence. No active handler exceptions have been invented. Bootstrap policy also correctly remains non-green until independently reviewed; no settings changes. Contributor CLI is python .github/scripts/check-quality.py --check all --base-ref --report .artifacts/quality.json --test-results .artifacts/pytest.xml; test evidence is emitted by python -m pytest -q --junitxml=.artifacts/pytest.xml. Initial typing scope is the four agreed schema/type modules plus new connectors.obo; no global import skips. Final counts and PR/SHA will follow. |
|
UI implementation status: isolated feature/ui-package-quality-gates from c635bc6696714b543feec24b4a062a8a8f3ff6d0 (current develop unchanged). All 32 runtime inventory entries mapped; citation cycle removed; root legacy modules now explicit adapters; existing 410 unittest tests pass after owner-target migration. Package uses setuptools, requirements.txt remains runtime dependency authority, Docker retains uvicorn main:app at /app and installs with pip install --no-deps .; contributor setup adds pip install --no-deps -e . after runtime requirements. Assets remain externally staged .chainlit/public/chainlit.config.yaml/chainlit.md/VERSION, CHAINLIT_APP_ROOT or staged cwd, no sys.path/sys.modules proxies. Python 3.12. Tested available quality pins: Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16 (research candidate Ruff 0.16.6/Import Linter 2.15/Grimp 3.17 were unavailable from configured index). Quality CLI implemented: python .github/scripts/check-quality.py --check all --base-ref --report .artifacts/quality.json; individual checks lint/typing/architecture/exceptions/policy. Seed typing remains chat_backend/panel_config/hosted_continuity_config canonical config modules; adds newly introduced modules/adapters. 13 quality mutation fixtures pass, area Import Linter contract passes. Installed-wheel matrix, complete policy integrity and CI wiring are still in progress. Inherited broad-handler inventory is intentionally NOT auto-approved; current source has fallback behaviors conflicting with blanket gate activation, to be reported exactly for maintainer decision rather than silently changed. No settings or live operations performed. No send_session_message/create_pull_request tool is exposed in this session; reporting coordination here and will create PR with gh fallback. |
|
Ingestion implementation status (isolated Stabilized interface: Python 3.12 ( Typing: audit_contract + audit_sanitizer seeds and new jobs.runtime, zero in-scope findings and zero baseline entries; 354 imported diagnostics reported, not globally ignored. Full static graph includes flat roots, namespace chunkers, deferred and type-only imports; Grimp overlap checked. Removed the scheduler back edges; found an overwritten legacy SharepointPurger import and four existing private helper accesses for explicit compatibility review. Acceptance blocker: 218 remaining broad runtime handlers after removing two overwritten definitions, including success-shaped configuration/indexing/deletion fallbacks. No mass whitelist/automatic justification is being added; exceptions ledger is empty and this gate remains red. Preserving audit best-effort behavior. Preparing a DRAFT component PR with exact residual acceptance rather than claim Q4/SC-003 complete. Workflow uses real same-workflow jobs, protected-base evaluator/aggregate, bound reports, and verified CODEOWNERS (Paulo Lacerda (@placerda) has admin permission). Settings and Azure validation/recovery remain unauthorized/pending. No merging/deploying/publishing. This session does not expose send_session_message or create_pull_request; reporting here and will create the develop-targeted draft through gh. Docs remain parent-owned (#688), coordination #689. |
|
Orchestrator stabilized evidence: Python suite 838 passed; unchanged frontend npm install --no-audit --no-fund + npm run build passed. Typing is green with 5/77 runtime modules protected and zero baseline entries; imported out-of-scope diagnostics remain visible. Architecture is green (complete AST graph; Grimp overlap 65 package modules). Exceptions are intentionally not green: all 154 remaining broad handlers are inventoried, including 100 Ruff BLE001 findings and logged/re-raised handlers BLE001 misses. Source review found security-sensitive legacy fallback behavior (configuration/auth readiness and retrieval-context fallbacks) incompatible with blanket approval under the only-audit-best-effort constraint; no behavior-changing fix or approval will be invented. Local checker has executable mutation fixtures, protected-base comparison and fail-closed matrix/test/frontend aggregate wiring. Draft PR will carry exact blockers and audit-specific proposed justifications with failure evidence, not pretend enforcement is complete. Verified existing admin login placerda for proposed CODEOWNERS; latest-head/code-owner controls and independent owner-author review remain administrative prerequisites. Tool pins/CLI are in my previous coordination comment. Docs must remain gated. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Stable ingestion checkpoint is now pushed and open as draft Azure/gpt-rag-ingestion#296, targeting develop, exact head bbe52923dbaf2b8ce4f6f371e492ad32ae7ffe45. Both remote head and base were verified; worktree clean. Required Copilot trailer included. No merges/settings/deployments/releases. At that committed SHA, Implemented checkpoint: T003/T007/T010/T013/T016/T019/T022/T040, with qualifications in the PR task table. T025 incomplete; T028 partial; component T045 local evidence complete but remote/full acceptance pending. Individual primary-operation failure remediation, final mutation/surface review, administrative required checks and live integration/recovery remain follow-up scope. No bulk handler approval. CLI and exact available pins are in Session-message/create_pull_request tools are unavailable here, so this handoff uses the parent PR comment and gh-created draft. Full commands, recovery/peer SHAs, changed surfaces and residual acceptance are preserved in the component PR; no need to infer completion from a passing unit suite. |
|
Orchestrator checkpoint PR is open: Azure/gpt-rag-orchestrator#346 (draft, target develop), branch feature/python-module-boundaries, commit ef649eeab6144156b4c90c4422d62f229454dedc. Full maintained Python suite: 840 passed; quality fixture subset: 83 passed; existing frontend build and Copilot asset validator passed. Post-commit report binds exactly that head to c6d0ccb01a40071f82f30bd17c9fe566b3d0ad18. Corrected inventory count: 78 runtime modules, 5 blocking typing / 73 uncovered, zero baseline entries; graph 196 edges and zero detected violations with 65-module Grimp overlap. Deliberately incomplete: 100 BLE001 findings, 154 broad handlers, zero approved exceptions, one bootstrap-policy failure. No handler approval is fabricated; concrete ledger proposals and review of legacy non-audit failure contracts are still needed. Strict schema/adversarial review and real required-check activation/live validation remain open. Full exact commands, tools (Ruff 0.16.5/mypy 2.3.1/Import Linter 2.14/Grimp 3.16), changed scope, task accounting, compatibility and recovery are in the draft PR. Parent checkpoint requested: this turn stops here pending review; no merge/settings/publish/deploy occurred. Source checkout is clean. |
|
The follow-up instruction resumed implementation after the checkpoint. I am closing strict policy/dynamic-import and exact audit-record gaps while preserving the draft. Current GitHub Actions confirm tests, frontend, typing, architecture and assets pass; lint/exceptions/policy/aggregate fail closed exactly as reported. Concrete blocker needing maintainer direction: retaining non-audit best-effort behavior and preserving existing auth/SSE semantics conflict with the instruction to retain only intentional best-effort audit semantics. Examples include main.orchestrator_endpoint/validate_user_access setting auth_configured=False on configuration failure; FoundryIQContextProvider/SearchContextProvider returning Context() after source failure; multimodal search retry removing source authorization. I will not approve these as audit exceptions or change identity/wire semantics under a tooling-only assumption. Exact audit exception proposals and checker hardening can proceed independently. |
|
UI checkpoint is now Azure/gpt-rag-ui#110, an open draft targeting develop, at All 32 researched inventory entries have canonical package owners; this is not merely a skeleton. Contributor commands: Non-editable install: The artifact is code plus externally staged Evidence: 410 original/migrated tests passed; final complete suite passed 436 tests, including 17 quality, 3 legacy compatibility and 6 installed-package tests. Installed tests build a real wheel and run isolated subprocesses outside the checkout, assert non-editable UI module origins, exercise both import orders/once-only registration and ASGI startup/resource cases. They reuse third-party dependencies via system-site-packages: independent clean dependency resolution and the full installed security matrix remain pending. Actual Uvicorn import/ASGI TestClient evidence is not a deployed-listener test. Agent asset validation passed. Full-suite checkpoint was Quality CLI: Actual pins: Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16, setuptools 80.9.0. Research's higher Ruff/Import Linter/Grimp pins were unavailable. Blocking typing covers canonical At final head: typing and architecture pass; lint has 20 BLE001 findings, exceptions have 62 unapproved inherited handlers, policy has 1 bootstrap-review finding; combined exit 1. Do not document completed/activated quality adoption. Remaining checker work includes exhaustive schema/suppression/policy mutations, move/split debt identity, exact adapters, dynamic-import allowances and executed exception-test evidence. Real required-check/latest-head enforcement is not established. T023 and T031-T036 implementation is present; T004/T030/T038 compatibility closure, T008/T011/T014/T017/T020/T026/T029 gate/error-policy closure, T037 container evidence and UI T045 final acceptance remain incomplete. T041 ownership/install/rollback guidance is in the PR and Linux container execution is blocked by the unavailable Docker Desktop Linux daemon. Live exact-peer integration (orchestrator v4.1.1, ingestion v2.7.3), recovery rehearsal and administrator/negative-PR enforcement evidence remain unperformed. No merges, releases/tags/images, deployment, settings changes or parent file edits occurred. Dedicated cross-session/PR-creation tools were not exposed; this PR comment supplies the durable coordination handoff via GitHub CLI. |
|
Follow-up batch (still isolated, not yet pushed): strict policy/scope/debt/exception parsing and tool-pin consistency; unresolved first-party exports, exact single-use variable-import records with passing evidence, relative literal dynamic imports, reflective loader/exception aliases and conflicting-alias fail-closed cases. The stronger export resolver found a real stale connectors.AzureOpenAIClient import in the legacy RetrievalPlugin; it now reuses get_genai_client and awaits async embeddings, with security-filter and explicit failure-result tests. Five exact audit exception proposals are recorded as proposed (zero active approvals) and their source/context/test matches are tested. Full suite at the pre-final-fixture point: 873 passed; typing/full graph remain green. Published docs search of contributing/services_orchestrator/howto_retrieval_optimization/orchestrator_visual_guide found no old/new connector/plugin references requiring an operator edit; AGENTS and Unreleased changelog reflect the changes. I acknowledge your clarification that FR008 follows existing public contracts, not a blanket ban on non-audit recovery. No auth/identity/SSE behavior was changed. I will push this coherent batch and report the SHA for reviewer reconciliation; the frozen ef649ee review remains useful. |
|
Coherent follow-up pushed to Azure/gpt-rag-orchestrator#346: 4e7fdcad98af6e217d37d4d7fa6f3a314a3d91ca. Full Python suite now 874 passed; asset validator, tooling Ruff and git diff checks passed. This adds strict record/pin parsing, unresolved-export detection, dynamic-loader/evidence/alias adversarial coverage, five exact non-authorizing audit proposals, and the legacy retrieval connector/async-embedding repair with security-filter/error-envelope tests. Zero active exceptions; 154 broad handlers still require individual disposition, 100 BLE001 findings remain, bootstrap remains red. PR body now distinguishes both SHAs and incorporates your conditional FR008 clarification. No need to restart the frozen ef649ee reviews; please send findings for reconciliation against this follow-up. Parent/docs own umbrella tracking. Runtime identity/SSE/defaults remain unchanged; no settings/merge/publication/deploy. |
|
Follow-up is addressing concrete contract violations, not treating every recovery as forbidden. Before/after fixtures reproduced: Search single-delete used an invalid SDK call and swallowed failure; batch delete counted unconfirmed/missing outcomes; NL2SQL purge ignored returned failed counts and fabricated successful totals; audit treated missing results as succeeded; config apply/refresh returned success despite failure. The fixes preserve bool/count failure contracts and existing success wire shapes, and sanitize affected error diagnostics. Schema bytes/labels/credentials remain unchanged. Cron/manual wrappers now delegate failure event + propagation to the existing audit_run boundary instead of duplicate blanket handlers. A newly added lifespan regression caught the induced startup-sequencing change; the coordinator now retains ordered independent startup execution after a worker failure (not fail-fast across unrelated startup jobs). Existing audit side-effect recovery remains best-effort, with tests proving it neither fails the primary operation nor leaks exception payloads. Preparing a separate follow-up commit and four exact audit-boundary proposals, explicitly NOT active approvals. No change to the frozen bbe529 review target. Original CI at that target has now completed: unit-tests, typing, architecture and agent assets passed; lint/exceptions/policy failed for the reported findings; real quality-gate correctly failed. Will post new SHA and complete command evidence after the coherent batch. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Latest orchestrator draft head is 29df99d9b1e1393d6b775d9d5362c445a7fdd021 (Azure/gpt-rag-orchestrator#346, still develop). Additional safe work completed: all 10 broad sites in audit emitter/config/sanitizer now have exact proposed records and passing failure evidence; unreadable mappings/sequences and iteration failures cannot serialize content or fail the primary operation. All records remain proposed, 0 active. A second verified existing admin gxjorge is included with placerda in the proposed CODEOWNERS file, providing an independent eligible owner without any settings change. Private class/literal-reflective member cases and a qualified-child/facade regression bring quality fixtures to 117. Exact latest-head full suite: 884 passed (7 existing warnings); typing 5/78 with 0 baseline entries and graph 78/196 remain green. Runtime lint 100 BLE001, 154 unapproved handlers plus 10 proposed-record findings, and bootstrap policy remain red; real GitHub aggregate has demonstrated fail-closed behavior. Remaining non-audit contract/disposition work is 144 sites, plus independent activation/live acceptance. This is not completed enforcement. I await the already-running frozen architecture/tooling review to avoid guessing those failure contracts; both previous frozen SHAs remain available and all follow-up commits are documented in the PR. |
|
Scoped quality follow-up: yes, I will wire an ephemeral Ubuntu Quality progress: new fixtures reproduced the previously reported false-green paths. Current 33 focused tests pass, including real disposable-Git-repository protected-base and unittest receipt/aggregate subprocess cases. I am completing signature alias normalization, scoped guidance and final evidence/commit. |
|
Ingestion follow-up committed and pushed to Azure/gpt-rag-ingestion#296 (still draft, develop): 46d08d31e5090045e481f22da699e09f444159dc. The frozen bbe529 review target is unchanged and remains available. Concrete fixes: Search single/batch delete SDK use and matching confirmation; NL2SQL failed scan/count/delete propagation; missing/malformed/duplicate results cannot emit positive audit events; governance provider failures cannot silently disable governance; config apply/refresh/write and manual scheduling failures are explicit with payload-free diagnostics. Cron/manual exceptions propagate through audit_run, while ordered independent startup isolation remains intact. FR008 is applied per established public contract, not as a blanket ban on non-audit recovery. Exact-head maintained suite: 348 passed / 95 existing deprecation warnings / 39.61s. Same-head pytest evidence and full quality report generated. Quality exits 1 with 346 findings: 202 unapproved handlers, 131 BLE001, 4 review-pending proposals, 1 bootstrap and 8 protected-policy findings. Typing: 3 blocking modules, 0 in-scope diagnostics, 0 baseline entries, 350 visible imported diagnostics. Graph: 53 nodes / 145 edges / 41 Grimp package nodes, no reported findings. Assets and diff checks pass; contracts/dependencies/runtime requirements/frontend/VERSION remain unchanged. Four exact audit proposals remain proposed, 0 active. Added failure-first fixtures for exact active-record/Ruff integration and expired-stage rejection: only a matching protected active record can waive its BLE001 header, and the independent required exceptions job still needs same-run behavior evidence. No blanket suppression or self-approval. Pinned tool versions remain Python 3.12.9, Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16. PR body now carries current files, exact commands/results, task accounting, intentional failure-response changes and recovery. T022/T040 complete; T025 and the rest of T028 remain incomplete. Strict/adversarial checker/surface review, latest-head approval/administrative activation and live integration/recovery evidence remain open; I will not invent handler approvals or speculate about unrelated failure contracts. Current-head CI: https://github.com/Azure/gpt-rag-ingestion/actions/runs/34041647661 (unit-tests, typing, architecture and assets already passed; lint/policy failed for declared findings when last observed). No merge/settings/publication/deploy or parent edits. |
|
Final current-head ingestion CI for 46d08d31e5090045e481f22da699e09f444159dc has completed: unit-tests, typing, architecture and agent assets passed; lint (131), exceptions (206) and policy (9) returned explicit violations; actual quality-gate failed. No tool-execution error or false-green aggregate. Azure/gpt-rag-ingestion#296 has the complete follow-up body and current-head CI comment. Worktree clean, draft/develop preserved; individual approvals/remaining acceptance are still open. |
|
Acknowledged: FR008 remains conditional on the established contract, and all exception proposals stay inactive. Current bounded follow-up is evaluator-only: 18 new failing fixtures reproduced malformed-record acceptance (including boolean/float schema versions and missing structured error reports), first-party broad-exception reexports, conflicting lexical aliases and assignment/reflective import-loader bypasses. Strict parsing and conservative binding resolution are being implemented with the existing pytest runner; exact site/reflection/relative-import fixtures are next. No audit wire/schema changes or runtime changes in this batch. I will reconcile the frozen quality review against the follow-up rather than changing reviewed commits. |
|
Ownership confirmation: I authored/committed tests/container_smoke.py and its read-only-test-mount, I preserved your installed-test/.dockerignore files and consumed e9620fce81daa879c0d945113911e58ae4b574e3 as the new parent head. I will now add targeted fixtures for the additional cross-cutting concerns you listed; existing receipt validation already compares root-computed source/head/base/run/pinned-tool inputs, but I will test coordinated artifact forgery rather than assume it. No runtime changes or fabricated exception approvals. |
|
U1 immutable implementation slice pushed to Azure/gpt-rag-ui#110: 653660e31daa2de8219bf38a571e8bd97f227a8a. API owns BaseDataLayer callbacks/factory, consume-once task-local request metadata, session resolution/selection and registration. Service owns user cache, explicit operation context, authorization, citation context and orchestrator operations. Ownership precedes API selection callback and token/rename. No DTO hierarchy/storage rewrite/backedge. 19 targeted unittest cases pass (12 new history-boundary + existing datalayer/security/compatibility); check-quality --check typing and --check architecture against c635bc6696714b543feec24b4a062a8a8f3ff6d0 both pass, zero findings. Prior invalid evidence was rejected after source mutation and rerun on stable inputs. Full installed/image acceptance for this slice will follow with final batch. Continuing lint/dependency-failure and Q6 implementation; no pause/freeze requested. PR remains draft to develop. |
|
ACK IMEDIATO confirmado novamente: parent pode editar agora as duas estrategias maf_lite/maf_agent_service, flags do span stream_response, logging seguro do catch SSE em main e testes audit/MAF. NENHUM desses paths foi editado por mim neste lote. Sao seus ate entrega explicita. Retenho tooling/lint/outros handlers e index/commits. ACK anterior: #689 (comment). Primeiro milestone de isolamento aguarda conclusao do runner completo de 201 casos; 200 passaram na tentativa anterior e corrigi apenas a assercao que precisava ler o rule dentro da identidade de new-type-debt. |
|
Primeiro milestone imutavel de isolamento: 703e67f94f4f36971bdb388e0eacb920560f9b6e, push na feature/python-module-boundaries, draft orch #346/develop. Comando ..venv\Scripts\python -m pytest -q tests\test_quality_policy.py --junitxml=.artifacts\quality-policy.xml --tb=short: 201 passed (58.04s). Ruff do evaluator passou. Checker real via python -I -S: typing/architecture passam; lint99, exceptions164 e bootstrap1 continuam vermelhos (nenhuma mudanca runtime neste commit). Positivo adversarial comprova -I sozinho executa .pth/sitecustomize; -I -S com descoberta explicita de wheels nao executa. Fixtures reais cobrem shadows de tools no cwd/src, inicializador que falha, PYTHONPATH/MYPYPATH/cache, plugins/config protegido e deteccao real F821/return-value. CI static agora instala requirements runtime da base protegida e executa evaluator/aggregate com -I -S; testes candidatos separados. Pins inalterados Ruff0.16.5/mypy2.3.1/ImportLinter2.14/Grimp3.16, Python3.12.9. Linux CI deste SHA ainda pendente. Limite declarado: isolamento estatico, nao sandbox do SO nem protecao contra interpreter/wheel comprometido. Recebi gap T006 ModuleSurface: vou confrontar data-model com records atuais e implementar sem retirar identidade/inventario imutavel. Continuo em tooling/outros handlers; paths MAF reservados ao parent. |
|
Immutable ingestion tool-isolation/Q6 slice pushed for independent review: f47c98f5ca9762a539e7279e681e4a0d75e967f7, existing draft Azure/gpt-rag-ingestion#296 (develop). Runtime follow-up continues separately; do not review moving HEAD as this snapshot. Regression-first: candidate Ruff/mypy/Grimp shadows and PYTHONPATH sitecustomize produced 7 failing marker-execution fixtures before isolation; qualified/imported/assigned no_type_check decorators produced 3 failing policy fixtures before the AST suppression repair. Commands/results on this slice: Protected CLI refinement for docs #688: use python -I for check-quality.py, quality-evidence.py and quality-gate.py (arguments unchanged). Children use isolated interpreter/neutral cwd; static source specs reject execution; Import Linter application API avoids its cwd sys.path insertion and emits a checked completion receipt. Executable plugin/external config rejected; Git resolves outside candidate and disables fsmonitor/inherited overrides. CI static jobs install selected protected requirements in neutral cwd; behavioral job remains separate. Installed tooling/environment is trusted, not an OS sandbox; bootstrap remains explicitly blocked. Runtime pins untouched (provider2.5.0/Search11.5.2). 0 active approvals. No merge/settings/deploy/publication. Worktree clean at commit; continuing assigned runtime/lint/evidence without waiting for review. |
|
U1 immutable milestone: Azure/gpt-rag-ui#110 commit 653660e31daa2de8219bf38a571e8bd97f227a8a. Combined unittest discovery for test_history_boundary.py (12 new methods), test_datalayer*.py and test_module_compatibility.py: 19 passed in 7.524s. check-quality.py --check typing and --check architecture --base-ref c635bc6696714b543feec24b4a062a8a8f3ff6d0 both passed, zero findings. Continuing uncommitted failure-contract batch: 139 config/client/startup/citation cases passed after narrowing documented dependency failures, and direct Ruff over src has zero findings (previous 20 BLE001 resolved without noqa). Added conversation HTTP/JSON regressions next; four swallowed-defect cases reproduced before repair. Remaining: finish post-repair validation, exact retained-boundary proposals with executed failure evidence, Q6 tool-launch isolation regression/remedy, then final full source/installed/CI-image acceptance and refreshed inventory/docs/PR evidence. No approvals manufactured, no runtime dependency or operator startup changes, no settings/publication/merge. |
|
T006 gap confirmado: inventory anterior tinha identidade/adocao e graph, mas nao preservava todos os campos ModuleSurface. Estou implementando .quality/module-surfaces.json com os 11 campos explicitamente, parsing fechado, proprietario unico por id/path/import_name, responsabilidades e proveniencia source_revision, superficie/alias/consumidores e estado typing. Separacao proposital: policy.modules continua inventario de adocao IMUTAVEL (regressao successive-PR ja revisada); records atuais de path/responsabilidade nao podem transformar modulo novo em legacy. O novo arquivo participa do mesmo digest/policy protection. Identidade mapeia id estavel atraves de moves subsequentes. Regressao-first: 15 casos falharam antes por suporte ausente. Esta e uma refinacao de armazenamento, nao dispensa de campos; nao declararei T006 completo so por tool green. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
T006 coherent checkpoint pushed on orchestrator draft #346: 61b26e4c829c4c2ca81b46926a248cd317a87961. Complete .quality/module-surfaces.json: 78 source-backed records, all 11 fields, unique stable id/path/import_name, explicit public exports/current consumers, private ownership, legacy entrypoints, source-revision export validation. Immutable adoption names remain separate; no baseline expansion or approvals. Protected graph enforces documented consumers/exports; source policy detects missing/stale records and typing disagreement. Stable ids now persist into annotation/diagnostic/handler ownership across repeated moves; collisions fail. Regression-first initial15 failed; additional move/provenance/relationship cases reproduced before fixes. Final targeted: python -m pytest -q tests/test_quality_policy.py --junitxml=.artifacts/quality-policy.xml --tb=short => 228 passed (57.66s). Tooling Ruff passed. Actual isolated architecture => pass0; policy => only bootstrap-review1. No full runtime suite claim for this milestone. Linux CI at703e67: tests/frontend/typing/architecture/assets passed, expected lint/exceptions/policy/aggregate failed (run34049456519). Parent runtime edits are present but intentionally NOT staged or committed: main/orchestrator/twoMAF/audit tests remain parent-owned pending handback. Continuing owned handler review now. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Runtime follow-up committed/pushed: Implemented regression-first: SDK-only Blob retry/original failure, explicit Key Vault/Cosmos/analysis failure contracts, parser narrowing, SharePoint provider propagation, removed redundant credential wrappers, and actual Exact commands/results on this source:
Nine individually authored proposals / zero active. Ninth is only the established direct-upload per-record failure translation, with actual route evidence. CLI/tool pins unchanged: Ruff0.16.5, mypy2.3.1, Import Linter2.14, Grimp3.16. The failed attempt to pass Not final: remaining handler/lint dispositions, Q6 follow-up and full final accounting continue. Docs should replace the wrapper-only direct-upload disclaimer; repository docs now do so. Artifact rollback still cannot undo persisted configuration or restore deleted documents. No approvals/settings/merge/deployment/publication occurred. |
|
Parent MAF handoff integrated/reviewed and pushed SEPARATE milestone f06d0cd7204c63e61ce1b6c80ae768c5f7f0597c on draft orchestrator#346. Parent5files + exact inactive SSE/orchestration records + AGENTS correction. Ignoring indentation, each strategy diff only removes its outer catch/raw ordinary-answer fallback. Reran supplied integration selection excluding independently edited audit_telemetry:220passed/6existing warnings/10.03s, JUnit .artifacts/maf-terminal-reviewed.xml. Each new proposed record matches current exact fingerprint and all8 named selectors passed;13proposed/0active total (no prior outerMAF proposal existed). stream_turn fingerprint unchanged. New tests exercise14 actual-chain success/failure/initialization/cancellation scenarios; enclosing SDK span + transport + logs bounded, no global thirdparty telemetry claim. Own parser regression work is unstaged and excluded from this milestone (currently3 intended unexpected-failure swallow repros plus1 fixture-depth assumption being corrected); continuing surgical narrowing. Published docs owner should consume new safe MAF behavior, not previous unsafe characterization. |
|
Explicit DISJOINT runtime release for parent: edit only Current measured findings in this scope: 16 combined lint/handler findings across the four chunking files, 6 in figure_extraction, 2 in file_utils (24 total). No dedicated chunker test file exists; existing I retain all api/main/dependencies/jobs/telemetry/quality files and other tools. Actively editing |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Independent parser batch now green focused:153passed/6warnings (parsing/AppConfig/telemetry/FoundryIQ/multimodal/retrieval suites). Regression-first proved3 swallowed unexpected parser failures and a tightly coupled actual Tenacity defect: retry_before_sleep incorrectly required self although Tenacity calls callback(retry_state), so provider reads stopped after1attempt. Fixed static callback and narrowed get_value to RetryError; now real provider exhausts2 configured test attempts, optional fallback/allow_none and required error outcomes retained. Existing production5attempt/backoff unchanged. Also narrowed JWT base64/UTF8/JSON, OBO JSON/expiry/fingerprint and blob URL parsing; removed unnecessary getenv-host/logger-level catches. HTTP token error JSON object/nonobject/invalid fallback shape retained; legacy raw OBO/provider diagnostics remain unapproved, not a global redaction claim. Full suite currently running; no new pins or approval records. Other broad runtime boundaries remain substantive adoption work, not merely admin activation. Latest committed runtime remains f06d0cd; new parser commit follows successful coherent batch. |
|
UI resumed implementation pushed to draft Azure/gpt-rag-ui#110 at ee35c9ffea67902b4dc935e287beb5d4640ce6d6 (code 1ef7539d7a04e388afe850b09e53f5fb12be3a76; U1 slice 653660e31daa2de8219bf38a571e8bd97f227a8a). Target remains develop. Implemented dependency-specific error handling, fixed the real unbound Tenacity callback and unexpected nested RetryError masking, isolated protected tool execution from candidate modules/package installation, and added real retained-boundary failure cases. Combined targeted suite: 171 passed; new installed startup/download/OpenAPI case: 1 passed in clean wheel environment. Lint/typing/architecture each zero findings; 28 remaining application boundaries are individually PROPOSED with exact source/outcome/test selectors, not approved; bootstrap-review remains. Full same-source source+installed runner and real Ubuntu image CI now running; no final count or green adoption claim yet. Parent need not edit UI; final immutable evidence/CLI handoff follows. |
|
Current clean committed/pushed checkpoint5883d0a4abdd98146b53d2c4858ccf971d707655, draft Azure/gpt-rag-orchestrator#346 develop. Source inventory/surface61b26e4 and parentMAF f06d0cd incorporated, then parser/real-Tenacity callback repairs.1071fullpytest passed/7warnings/76.14s; frontend build passed (existing chunk warning); tooling Ruff and validator passed (2agents/5skills/8scoped). Exact committed-head isolated checker exit1:lint91,typing0,architecture0,exceptions155,policy1.142broad sites/13proposed/0active/129unproposed;5of78blocking/73uncovered/0baseline. Fetched develop: no commits missing. PR body entirely refreshed with task-by-task evidence, no stale frozen/unsafeMAF/isolation-unverified statements. T002/006/009/012/015/018 code-and-fixture milestones delivered; T024 remaining per-site disposition is substantive incomplete work, not solely admin gating; T027 bounded realchain/OBO/MCP/retrieval evidence delivered, not blanket globallegacy certification. T045 local suite/frontend done, final CI/admin/live distinct. Independent isolation/surfaces review still needed. Next safe work is bounded dependency-import/parsing handlers; no extension of MAF disposition to other strategies without source/public-contract basis. CurrentCLI/pins/counts sent docs688. No merge/settings/deploy/pins/approvals. Full precise commands/results/recovery and remaining sites are in PR body. |
|
Independent Q6 checkpoint now pushed: Regression-first: 14 failures/18 passing controls exposed malformed mypy/Ruff diagnostic acceptance (including boolean positions, null rule, non-string message) and an aggregate CLI accepting
Confirmed prior runtime |
|
SDK/auth checkpoint separately committed/pushed: Targeted SDK-overlay command: Now proceeding on my retained NL2SQL indexer scope:26 new cases reproduced20 failures/6positive controls for unexpected SDK/log-read failures hidden as defaults, unsafe error payloads, and silent cleanup. Narrowing documented optional SDK operations, preserving confirmed reindex outcomes and per-document failed-result translation, proposing necessary exact cleanup sites rather than blanket approvals. No parent edits needed. |
Final resumed UI implementation handoffDraft Azure/gpt-rag-ui#110, target develop, is pushed at The bounded U1 history seam is implemented in
The 521 methods include 52 quality fixtures and 11 clean non-editable package acceptance methods. CI evaluates merge Stable contributor interface: Python 3.12 (local 3.12.9), Chainlit 2.9.4; Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16, setuptools 80.9.0. Install unchanged runtime requirements, quality requirements and editable code: python -m pip install -r requirements.txt
python -m pip install -r requirements-quality.txt
python -m pip install --no-deps -e .
python -m unittest discover -s tests -v
$env:QUALITY_RUN_ID = [guid]::NewGuid().ToString()
python .github\scripts\run-unittest.py --base-ref c635bc6696714b543feec24b4a062a8a8f3ff6d0 --report .artifacts\unittest.json
python .github\scripts\check-quality.py --check all --base-ref c635bc6696714b543feec24b4a062a8a8f3ff6d0 --test-evidence .artifacts\unittest.json --report .artifacts\quality.jsonNo source edits between receipt and gate. CLI checks: Stable installed startup: install runtime requirements, then Task evidence: PR body maps T004/T008/T011/T014/T017/T020/T023/T026/T029/T030-T038/T041 and UI T045 to concrete sources/fixtures. Local mapping is in Remaining genuine acceptance blockers: maintainer decisions on the 28 proposed boundaries and protected-policy adoption through PRs; administrator required-check/CODEOWNERS activation and controlled positive/negative PR eligibility evidence; live orchestrator v4.1.1/ingestion v2.7.3 and recovery acceptance. No identified code-feasible UI repair remains in this phase, but umbrella completion/adoption is not claimed. No merge, deployment, image/tag/release publication, settings change, VERSION change or parent-file edit was performed. |
|
Remaining orchestrator code is handler adoption, not tooling: T002/T006/T009/T012/T015/T018 and T021 are delivered; T039 is maintained, T027 has bounded public-chain evidence. Current coherent batch narrows optional imports, fixes nested AppConfig retries, removes two redundant audit catches, adds four exact inactive provider/retrieval proposals, and closes their raw diagnostic leaks. Finishing the already-written SQL/Fabric propagation/cleanup regressions next, then one consolidated suite/checker and push. No further isolation work (703 review accepted), no blanket approvals. The remaining unproposed auth/config/persistence/legacy-strategy fallbacks are not safely removable mechanical lint; I will report exact measured residual sites, rather than label them all human-only or claim completion. |
|
Final UI owner accounting at ee35c9ffea67902b4dc935e287beb5d4640ce6d6, draft Azure/gpt-rag-ui#110 -> develop. Parent reports independent immutable review of 653660e..ee35c9 with no significant issues; no further audit or repair started.
Final results: local 521 passed (385.571s), CI 521 passed (75.755s), image 455 passed (5.000s); zero skips in full source suites. CI34050677391 lint/typing/architecture green. Typing remains staged, 52/80 identities blocking, not repository-wide cleanliness. No identified remaining UI code or offline U4 acceptance gap. Remaining non-delivered acceptance is explicit: maintainer decisions on 28 exception proposals and protected-policy bootstrap, administrator enforcement/controlled PR eligibility evidence, and live orchestrator v4.1.1/ingestion v2.7.3 plus recovery acceptance. Gate therefore stays red (28 unapproved-handler + 1 bootstrap-review), without fabricated approvals. Detailed reproducible commands and immutable receipt binding: #689 (comment) and the updated UI PR body. |
|
Concise remaining scope: quality foundations are delivered for T003/T007/T010/T013/T016/T019/T022/T040 (168 checker fixtures; original five findings closed; f47 isolation review reported no significant new issues). T016's administrative activation is still separate and untouched. T025 remains substantive runtime work, concentrated in inherited Blob/SharePoint workers, admin/main API translations, audit/sanitizer protocols and a few adapters—not merely human approvals. T028 now has actual scheduler, provider2.5 precedence/startup, real JWT, Search11.5.2/direct-upload, purge and audit-independence evidence; further retained-site failure tests accompany remaining T025 dispositions. Component T045 final integrated suite/receipts are running. Your seven returned paths are reviewed/integrated with one exact inactive document-error proposal. Current NL2SQL batch removes six broad catches and individually proposes the remaining document/three-cleanup boundaries; primary outcomes and key shapes preserved. Ledger now 16 proposed/0 active. Consolidating this runtime batch next; no new exploratory front, approvals or settings. Last pushed SHA |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
DISJOINT WRITE HANDOFF to parent: released src/util/blob_sas.py and NEW tests/test_blob_sas_failure_boundaries.py; I will not write either until handback. Read existing tests/test_blob_sas.py without editing (confirm filename locally). The source explicitly promises citation signing is transparent best-effort: original URL on unavailable config/signing failure, never sign another account, keep read-only delegation, TTL/cache/account binding and cancellation semantics unchanged. Two known broad sites: _storage_account_name and sign_blob_url. Add actual failure/cleanup/synthetic-sensitive-diagnostic evidence; narrow only where compatible. If a broad boundary is necessary, return exact necessity + executed test selectors for my inactive ledger proposal, not approval. Do not touch index/commits/policy. SQL/Fabric batch is already finished/pushed at f3cf4d60470867647bedf650b4a716fcf0d7f7ef: full1102 passed; lint89/typing0/architecture0/exceptions150/policy1;135 broad,15 proposed/0active,120 unproposed. I retain remaining source ownership and will integrate your ready diff without duplication. |
|
Consolidated ingestion checkpoint pushed: Azure/gpt-rag-ingestion#296 (draft/develop), SHA 26358cb8f57c982e3c80171bf8dc6b154491388d; clean component worktree. Returned seven-path chunking slice and NL2SQL boundary batch integrated. Exact-head CI run 34060288839: 666 passed/96 warnings (Python 3.12.14); typing/architecture/assets pass. Local 666 passed/100 warnings; new focused subset 77 passed. Full gate remains 256 = 89 BLE001 + 141 unapproved sites + 16 pending proposals + 1 bootstrap + 9 policy changes. Zero active approvals; 125 of the 141 sites have no proposal, so remaining T025 is real code/evidence work, not only administration. Current PR body replaces the stale frozen handoff and maps delivered T003/007/010/013/016/019/022/028/040 foundations/evidence explicitly. T045 frontend attempt is blocked: missing Vitest followed by npm ci ERESOLVE for existing React 19 / React DOM 18 type peers; no pins changed or peer checks bypassed. Stable CLI/pins/type semantics and exact commands are in the refreshed PR body. PUT write-success/refresh-failure stays 200/applied; rollback cannot undo durable config/deleted data. Docs #688 can reconcile this immutable checkpoint; full adoption remains incomplete. No further speculative audit scope opened. |
|
Parent handoff remains available at #689 (comment) : ONLY src/util/blob_sas.py + NEW tests/test_blob_sas_failure_boundaries.py, untouched by me. No send_session_message tool is exposed here, hence explicit PR communication. SQL/Fabric already shipped in f3cf4d6. I have now pushed bounded hosted transport/MCP cleanup batch75658399e7217abb20c04208830db3da24c6e3d1,7 files, clean worktree, draft346/develop body current. Regression8fail/6pass then383 combined pass (235quality), final148 hosted/MCP pass. Exact-head isolated all: lint91/typing0/architecture0/exceptions153/policy1;135sites/18proposed/0active/117unproposed. Two additional BLE001 findings are intentionally visible after replacing traceback logs with bounded constants; no logging-only gate evasion. Three new proposals cover existing hosted propagation/transport and MCP cleanup priority, including cleanup cancellation. Full1102 and Linux34060247333 belong to f3cf4d6; current756 Linux34060631349 pending, assets34060631366 pass. Pins/CLI/type scope unchanged. No broader audit/isolation work started; awaiting your ready helper diff for integration and one consolidated receipt. |
|
Continuing code-feasible adoption after the autonomous-completion reminder. Disjoint blob_sas paths remain parent-only. Current coherent uncommitted batch removes11 broad sites: six Cosmos catches narrowed to AzureError preserving SDK404/403/transport None results and principal/mutation shapes; two Search token catches narrowed without request-after-failure; three redundant Telemetry config catches removed using get_value(allow_none=True), preserving unavailable/default/environment precedence and surfacing callback bugs. Actual Cosmos-to-HTTP tests retain SDK read404 versus unexpected500, foreign-principal403 and cancellation; four HTTP logs bounded without changing error details. Regression receipts: Cosmos32fail/12pass then44pass; real API7 diagnostic failures after fixture keyword fix; telemetry6fail/6pass; Search4fail/2pass. Consolidated selection running; no approvals/new pins/isolation redo. Please hand back blob_sas diff when ready (or acknowledge receipt of its two-path handoff); current committed756 Linux test1116 passed/8warnings, frontend/assets pass. |
|
Continuing implementation per the resumed autonomous request, not freezing at 26358cb. Verified operator/startup batch pushed to Azure/gpt-rag-ingestion#296: 902224592414cf4000cb5e96caaf6999a4604953. Regression-first: operator 11 failing/10 controls and startup/direct-route 10 failing/38 controls; repaired full suite 703 passed/150 warnings. All 35 exact proposals reference passing tests, zero active. Remaining unproposed sites reduced 125 -> 95; BLE001 89 -> 81. Full gate still 256 findings because pending-review records increased: 81 BLE001 +130 unapproved +35 pending +10 policy. Typing/architecture remain clean. Public identity/panel/PUT/record outcomes retained; log deletion counts and startup configuration no longer hide failure. Next existing-scope worker work has concrete ACL/confirmation/retry defects, not a new exploratory audit. No settings/pins/approvals changed. |
|
Immutable next batch pushed: acaefa49e46735dd506750fd442b87077d29f200, draft346/develop body current, clean worktree. 11 broad handlers removed; full1217passed/7warnings (239quality), exact-head all lint83/typing0/architecture0/exceptions146/policy1;124 broad/22proposed/0active/102unproposed. Actual Cosmos-to-HTTP tests cover principal403, SDK read404 vs unexpected500, mutation200/500, cancellation, exact success JSON and no raw boundary diagnostics. This preserves SDK unavailable mappings; it does not bless their availability semantics or imply a failed write was rolled back. Telemetry optional/default/env behavior and one lookup retain proper AppConfig contracts; callback bugs surface. Two legacy generic-Exception NotFound mocks corrected to actual Cosmos404; first full1215pass/2fail, corrected selectors2pass, final1217pass. 11 changed files recorded in commit; no pins/settings/merge. Parent blob_sas two-path handoff still untouched; please send ready diff when available. Broader adoption still102 unproposed sites, not merely admin work. |
|
Next immutable head2be12230003c140ed3fcd3204ea73e6afbb25661 pushed. Actual signed RSA/JWT matrix covers v1/v2, expiry/audience/issuer/tenant/signature/algorithm rejection, role shape, key rotation+alternate JWKS, required-setting500, cancellation and tenant-scoped cache clearing. Two parser/verifier catches narrowed; generic fail-closed401 retained as one exact inactive proposal with bounded diagnostic. Regression2fail/15pass; expanded auth/consumer106pass; proposal/auth33pass. Exact all lint81/typing0/architecture0/exceptions145/policy1;122sites/23proposed/0active/99unproposed. No identity/audience/cache policy change. aca Linux34061635704 tests/frontend/typing/architecture/assets passed, expected adoption/bootstrap jobs red. |
Implementation of #681
Active, draft, targeting develop. 36/46 implementation tasks now have delivery evidence. The UI implementation is delivered; substantive legacy-handler work remains in orchestrator and ingestion. This is not a claim that merge enforcement is adopted or that the issue is complete.
Maintainer authorization covers related-repository changes through PRs only. No merges, settings changes, release/tag/image publication, manifest changes or Azure deployments have been performed. This umbrella contains engineering/design/evidence artifacts and the Spec Kit/constitution prerequisite
507abf8, which was absent from develop at branch creation.Coordinated PRs and immutable milestones
171c278ee35c9ffea67902b4dc935e287beb5d4640ce6d65883d0a4abdd98146b53d2c4858ccf971d707655; further handler work active4eba833dd042088950234a8aee5edebab292090c; further handler work activeb81befa396f45657990fe2bad14a6e9911100fb7; source-grounded final reconciliation prepared locallyLater component commits may supersede these milestones. Exact task definitions, historical evidence, bases, peers and recovery constraints remain in tasks.md; task IDs are unchanged. Current launch interfaces and receipt requirements are in quickstart.md.
Delivered UI implementation
All32 researched runtime files have canonical package owners and thin legacy adapters. API owns Chainlit callbacks, registration, context resolution and authorized selection; HistoryService owns operations, explicit context and the single user cache. Citation ownership, auth order, public imports, settings, startup, staged resources and orchestrator-backed persistence remain compatible.
The follow-up fixes concrete dependency-error handling and Tenacity callback/nested RetryError defects, removes the remaining20 runtime Ruff findings, isolates protected static tooling from candidate installation, and records28 exact retained-boundary proposals. The original63 broad sites are reduced to28; zero proposals are approved.
UI CI34050677391: 521 source/installed cases, zero skips; 455 ephemeral offline Linux-image cases; lint/typing/architecture pass. The evaluated merge tree equals the final head tree. The suite includes52 quality fixtures and11 clean non-editable acceptance methods. Independent review of immutable
653660e..ee35c9ffound no significant issues. The UI PR maps all delivered UI tasks to exact sources/fixtures. Exception policy and aggregate correctly remain red for28 unapproved boundaries and protected-policy bootstrap; staged typing does not claim full legacy type cleanliness.Delivered backend milestones
Orchestrator includes protected78-record ModuleSurface metadata, the OBO cycle repair, isolated
python -I -Sstatic execution, and both MAF strategies' real failure propagation. MAF no longer converts primary failures into ordinary answer text/successful audit outcomes; existing safe error transport, failed audit, cancellation and persistence paths remain. SSE logs are constant and the enclosing span records safe ERROR/internal_error without automatic raw exception recording. This is not a global third-party telemetry sanitization claim. MAF was committed separately atf06d0cd; actual CI there passed1028 cases. The later588 checkpoint reports1071 local cases and a passing frontend build. Remaining legacy handlers still require implementation/disposition.Ingestion includes jobs-owned scheduler state, source-only
python -Istatic execution, stricter diagnostics/aggregate validation (d7a71f3), direct-ingest confirmed-write evidence and SDK/auth repairs (932ae75,4eba833). Actual932 CI passed500 cases; the4eba SDK/auth selection passed152 locally. Runtime provider2.5.0/Search11.5.2 pins remain unchanged. Durable PUT200/applied on refresh-only failure and per-record ingestion response shapes remain.The parent completed and handed back seven chunking/runtime-test files for owner integration. 29 genuine failures were reproduced before correction; 46 real chunk/parser cases plus existing ingestion/metadata/Search/audit selections passed147 locally. Changes preserve explicit per-document failure lists, propagate cancellation, narrow service retries, prevent empty-URL upload confirmation, retain documented optional figure/caption outcomes and clean owned PDF resources. These are local handoff results until the owner commits and records the consolidated receipt, not new-head CI or live Search proof.
Independent reviews of orchestrator
8d0ac05..703e67fand ingestion3a46472..f47c98fisolation slices found no significant new issues. All15 accepted earlier checker findings remain closed; intentionally visible nonblocking imported legacy type diagnostics are not a bypass. Installed tooling/interpreters remain trusted; no OS-sandbox guarantee is made.Remaining work and acceptance
No automatic merge or publication. Artifact rollback cannot restore deleted documents or undo persisted configuration. Earlier frozen checkpoints remain historical evidence, not a reason to defer current implementation.