Skip to content

Fix user executor failures missing from JUnit/XML/JSON/HTML reports#907

Open
ivan-velasco wants to merge 19 commits intoovh:masterfrom
socotra:fix/user-executor-reports
Open

Fix user executor failures missing from JUnit/XML/JSON/HTML reports#907
ivan-velasco wants to merge 19 commits intoovh:masterfrom
socotra:fix/user-executor-reports

Conversation

@ivan-velasco
Copy link
Copy Markdown
Contributor

Problem: User-executor inner failures were only merged into the parent step inside printTestStepResult, so paths that skipped that call left reports without real error detail.

Change: After runTestSteps in RunUserExecutor, merge all inner TestStepResults into the parent (Errors, ComputedInfo, stdout/stderr; skip whitespace-only streams). Drop the merge from printTestStepResult to avoid duplicates.

Tests: tests/user_executor_with_failure.yml plus lib executors (simple + nested failure).

example

<testcase classname="user_executor_with_failure.yml" name="simple-user-executor-failure" time="0.014">
  <error><![CDATA[Testcase "user_executor_with_failure", step #2-0: Assertion "result.code ShouldEqual 1" failed. expected: 1  got: 0 (user_executor_with_failure.yml:0)]]></error>
  <error><![CDATA[Testcase "simple user executor failure", step #1-0: executor "user_executor_with_failure" failed (user_executor_with_failure.yml:0)]]></error>
  <system-out><![CDATA[this step succeeds
failing step
]]></system-out>
</testcase>

ncharaf and others added 18 commits March 24, 2026 19:26
Signed-off-by: Nabil <26463615+ncharaf@users.noreply.github.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
This reverts commit 3e3f43e, reversing
changes made to 044ac99.

Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Gabriel Augendre <gabriel.augendre@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
…ry (ovh#878)

Signed-off-by: Daria Fortunato <daria.kobtseva@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Daria Fortunato <daria.kobtseva@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
…etrieval (ovh#867)

Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
If the JSON has escaped quotes already, the parser was double escaping
them making the out invalid. This will catch already escaped quotes and
skip them when encountered.

Signed-off-by: Shane A. Stillwell <brisk.pen7424@ranger50.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: sonpham <quangson137@gmail.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Co-authored-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Co-authored-by: Yvonnick Esnault <yvonnick.esnault@corp.ovh.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
Signed-off-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Co-authored-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
* feat(assertions): support custom assertions

Signed-off-by: GitHub <noreply@github.com>

* custom to user assertion

Signed-off-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>

---------

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Co-authored-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
close ovh#883

Signed-off-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Co-authored-by: Yvonnick Esnault <yvonnick.esnault@ovhcloud.com>
Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
…on for JUnit and other report outputs

Signed-off-by: Ivan Velasco <ivan.velasco@socotra.com>
@ivan-velasco ivan-velasco force-pushed the fix/user-executor-reports branch from 582148b to a1b2be8 Compare March 25, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants