RFC: E2E and QA Lab scorecard consolidation

[RomneyDa]

Summary

• add a draft RFC for scorecard-backed e2e and QA Lab consolidation
• define normal CI core e2e, release full e2e, pluggable mock/live backends, Multipass channel coverage, script disposition, and sibling repo boundaries
• lay out a PR-by-PR implementation plan for OpenClaw test evidence

Verification

• pnpm docs:list
• git -C ../rfcs diff --check
• checked RFC script inventory covers every test/scripts/*.ts file in openclaw/openclaw

[clawsweeper]

Codex review: found issues before merge. Reviewed June 7, 2026, 7:43 PM ET / 23:43 UTC.

Summary
Adds a draft RFC and three Markdown sidecars proposing scorecard-backed E2E/QA Lab consolidation, release evidence gates, mock/live backend lanes, and an implementation plan.

Reproducibility: not applicable. this is an RFC/documentation PR, not a runtime bug. The review path is source inspection against README conventions and the PR diff.

Review metrics: 2 noteworthy metrics.

• RFC surface added: 4 files added, 725 insertions. The PR records a substantial cross-repo process proposal rather than a small documentation correction.
• Markdown sidecars: 3 Markdown files under rfcs/0007/. The added sidecars are the main layout concern because current repo guidance describes that directory as asset-only.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• [P2] Get maintainer acceptance and update the RFC status and implementation issue before merge.
• Inline the Markdown sidecars into the RFC, move them to an accepted content location, or explicitly update the repository layout convention.

Risk before merge

• [P1] Merging now would record a broad QA, CI, release-evidence, scorecard, and sibling-repo policy before the repository's documented acceptance step is complete.
• [P1] The three Markdown sidecars create a second proposal-content location under a directory that current README guidance reserves for assets unless maintainers explicitly change that convention.

Maintainer options:

1. Accept Then Normalize (recommended)
   After maintainer discussion accepts the direction, update the frontmatter to accepted with an implementation issue and reconcile the Markdown sidecars with the repository layout policy before merge.
2. Keep As Discussion Draft
   If maintainers are not ready to own the scorecard and QA Lab consolidation policy, leave the PR open as a discussion artifact or close it in favor of narrower RFCs.
3. Approve A Layout Exception
   Maintainers can intentionally allow Markdown content in numbered sidecar directories, but that should be an explicit repository convention change rather than an accidental precedent.

Next step before merge

• [P2] Manual review is required because the remaining blockers are maintainer acceptance and repository convention choices, not a narrow automated repair.

Security
Cleared: The diff is Markdown-only RFC content and does not add code execution, dependency, workflow, credential, or supply-chain changes.

Review findings

• [P2] Do not merge while the RFC is still draft — rfcs/0007-e2e-qa-lab-scorecard-consolidation.md:7-8
• [P2] Keep proposal content out of the asset sidecar folder — rfcs/0007/example-scorecard-checklist.md:1

Review details

Best possible solution:

Keep the RFC open through maintainer discussion, then accept it with an implementation issue and repository-compliant layout or split/close it if maintainers do not want this broad policy direction.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is an RFC/documentation PR, not a runtime bug. The review path is source inspection against README conventions and the PR diff.

Is this the best way to solve the issue?

No, not as a merge-ready branch. The RFC direction may be useful, but the current branch still needs maintainer acceptance and layout reconciliation before it becomes the best repository record.

Full review comments:

• [P2] Do not merge while the RFC is still draft — rfcs/0007-e2e-qa-lab-scorecard-consolidation.md:7-8
  The repo lifecycle says an accepted RFC should update to status: accepted, set the implementation issue, and then merge. This RFC is still status: draft with issue blank, so landing it now would skip the acceptance record.
  Confidence: 0.93
• [P2] Keep proposal content out of the asset sidecar folder — rfcs/0007/example-scorecard-checklist.md:1
  The README says the sibling numbered directory is only for RFC assets, but this PR adds Markdown checklist, plan, and inventory documents there. Inline the material into the RFC, update the repo convention first, or otherwise avoid creating a second proposal-content location.
  Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: not found in the target repository.

Codex review notes: model gpt-5.5, reasoning high; reviewed against e938e93198f4.

Label changes

Label justifications:

• P3: This is a low-urgency RFC/process proposal that needs maintainer acceptance rather than urgent runtime repair.
• merge-risk: 🚨 other: Merging would record broad QA/release policy and a layout exception that normal CI cannot validate.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: This is a member-authored RFC/documentation PR, so after-fix real behavior proof is not a merge gate.

Evidence reviewed

What I checked:

• Protected author signal: GitHub reports the PR author association as MEMBER, so conservative cleanup should not auto-close this item even if it has merge blockers. (8a85bd790330)
• RFC lifecycle policy: Current main says new RFCs should not merge while still draft, and accepted RFCs should set an implementation issue before merge. (README.md:75, e938e93198f4)
• Asset-only sidecar policy: Current main describes the sibling numbered directory as optional and used only for assets referenced by the RFC, with authoring notes limiting it to diagrams, screenshots, or other supporting files. (README.md:37, e938e93198f4)
• PR frontmatter remains pre-acceptance: The latest PR head still adds the RFC with status: draft and a blank issue field. (rfcs/0007-e2e-qa-lab-scorecard-consolidation.md:7, 8a85bd790330)
• Sidecar content remains Markdown proposal material: The latest PR adds three Markdown files under rfcs/0007/, so the sidecar directory would carry checklist, implementation plan, and script inventory content rather than only assets. (rfcs/0007/example-scorecard-checklist.md:1, 8a85bd790330)
• Not implemented on current main: Current main has no existing RFC content for QA Lab scorecard consolidation, so the PR is not obsolete or already implemented. (e938e93198f4)

Likely related people:

• kevinlin-openai: Blame and history show this person added the current RFC lifecycle and sidecar-layout guidance that this PR needs to satisfy before merge. (role: RFC process and layout owner; confidence: high; commits: e366ea9825a4, f4fdf38f4717, bbb4058da234; files: README.md, rfcs/0000-template.md)
• Gio Della-Libera: Recent substantial RFC content and refactoring history make this person a useful reviewer for proposal-shape consistency, though not the direct lifecycle-policy author. (role: adjacent RFC contributor; confidence: medium; commits: e46c2a113cc9, 1cd9e911af24, 2d0e47f03c1d; files: rfcs/0006-feeds.md, rfcs/needs_refactoring/0002-doctor-health-upgrades.md, rfcs/needs_refactoring/0003-policy-conformance.md)
• Omar Shahine: Authored and aligned recent RFC documents in the current repository, so this person is relevant for consistency with accepted RFC shape. (role: adjacent RFC contributor; confidence: medium; commits: f346050b2878, 4b98ff01752e, 1a545b1de86d; files: rfcs/0005-approval-prompt-markdown.md, rfcs/needs_refactoring/0004-imessage-channel-configuration-cleanup.md)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.