fix(e2e): handle top-level payloads in openclaw agent JSON output#4030
Draft
hunglp6d wants to merge 1 commit into
Draft
fix(e2e): handle top-level payloads in openclaw agent JSON output#4030hunglp6d wants to merge 1 commit into
hunglp6d wants to merge 1 commit into
Conversation
OpenClaw 2026.5.18 moved the `payloads` array from `result.payloads`
to the top level of the `openclaw agent --json` output object. Five
E2E tests extracted the reply via `doc.get("result") or {}` which now
returns an empty dict, causing `result.get("payloads")` to yield no
payloads and every agent-reply assertion to see an empty string.
Change the fallback to `doc.get("result") or doc` so the extraction
works with both the old nested schema and the new flat schema.
Affected tests:
- test-bedrock-runtime-compatible-anthropic.sh
- test-messaging-compatible-endpoint.sh
- test-openclaw-inference-switch.sh
- test-launchable-smoke.sh
- test-sandbox-operations.sh
Signed-off-by: Hung Le <hple@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
E2E Advisor RecommendationRequired E2E: None Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
PR Review AdvisorRecommendation: blocked This is an automated advisory review. A human maintainer must make the final merge decision. Limitations: This advisory review is based on trusted metadata and the provided diff; no scripts, tests, package-manager commands, or workflows were executed.; Issue #4031 has no comments in the trusted context, so acceptance extraction is limited to the issue body.; The raw nightly failure logs referenced by issue #4031 were not independently inspected here; clauses about raw payload output are therefore marked unknown where appropriate.; The PR is draft and CodeRabbit skipped review, so there may be future review feedback not present in the current context.; Open PR #3925 overlaps all changed files and may alter the final merge conflict/drift assessment. Full advisor summaryPR Review AdvisorBase: The five-line E2E parser compatibility change matches the linked issue, but the PR is still draft/mergeState BLOCKED and overlaps active work in PR #3925. Gate status
🔴 Blockers
🟡 Warnings
🔵 Suggestions
Acceptance coverage
Security review
Test / E2E status
✅ What looks good
Review completeness
|
wscurran
added
E2E
Contributor
|
✨ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4031
Summary
Five nightly E2E tests fail because their inline Python extractors assume
openclaw agent --jsonoutput nests payloads underresult.payloads, but OpenClaw ≥ 2026.5.18 now returnspayloadsat the top level of the JSON envelope. The fix makes the extraction resilient to both schemas:doc.get("result") or docfalls back to the document root when there is noresultwrapper.Root cause
OpenClaw's agent JSON output changed from:
{"result": {"payloads": [{"text": "..."}]}}to:
{"payloads": [{"text": "..."}]}The E2E test scripts hardcoded
doc.get("result") or {}which yields{}under the new schema, so.get("payloads")always returns[]→ empty reply → assertion failure.Changes
test/e2e/test-bedrock-runtime-compatible-anthropic.sh(doc.get("result") or {}).get("payloads")→(doc.get("result") or doc).get("payloads")test/e2e/test-launchable-smoke.shdoc.get('result') or {}→doc.get('result') or doctest/e2e/test-messaging-compatible-endpoint.shdoc.get('result') or {}→doc.get('result') or doctest/e2e/test-openclaw-inference-switch.shdoc.get("result") or {}→doc.get("result") or doctest/e2e/test-sandbox-operations.shdoc.get('result') or {}→doc.get('result') or docNightly run
sandbox-operations-e2e,bedrock-runtime-compatible-anthropic-e2e,messaging-compatible-endpoint-e2e,openclaw-inference-switch-e2e,launchable-smoke-e2eValidation
The extraction pattern
doc.get("result") or docis backwards-compatible:{"result": {"payloads": [...]}}):doc.get("result")returns truthy dict → uses it →.get("payloads")works{"payloads": [...]}):doc.get("result")returnsNone→ falls back todoc→.get("payloads")worksCustom E2E workflow validation was not run (PAT lacks
workflowscope to push.github/workflows/changes). The fix is a minimal, mechanically-verified single-expression change across 5 files.Test plan
Signed-off-by: Hung Le hple@nvidia.com