feat: add test_step_outcome input to detect silent test failures#175
feat: add test_step_outcome input to detect silent test failures#175
Conversation
Pass the test runner step outcome to set MERGIFY_TEST_EXIT_CODE, allowing the CLI to detect cases where the runner crashed but the JUnit report appears clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Ie46d801917402e93726e1d182c8df3ba2c819b01
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🔴 🤖 Continuous IntegrationThis rule is failing.
🔴 👀 Review RequirementsThis rule is failing.
🔴 🔎 ReviewsThis rule is failing.
🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 📕 PR descriptionWonderful, this rule succeeded.
|
There was a problem hiding this comment.
Pull request overview
Adds a new optional test_step_outcome input to the composite GitHub Action so it can set MERGIFY_TEST_EXIT_CODE for mergify ci junit-process, enabling detection of “silent” test-runner crashes even when JUnit output looks clean.
Changes:
- Introduce
test_step_outcomeinput inaction.ymlfor passingsteps.<id>.outcome. - Derive and export
MERGIFY_TEST_EXIT_CODEfromtest_step_outcomewhen running thejunit-processaction.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| test_step_outcome: | ||
| description: | | ||
| Outcome of the test runner step (e.g. steps.<id>.outcome). | ||
| Pass this to detect silent failures where the test runner crashed | ||
| but the JUnit report appears clean. | ||
| Values: 'success', 'failure', 'cancelled', or omit entirely. | ||
| required: false |
There was a problem hiding this comment.
README.md action docs are auto-generated from action.yml and CI’s autodoc job fails if README.md isn’t regenerated. Since this PR adds a new input, README.md needs to be updated by running ./generate-doc.sh (or tj-actions/auto-doc) and committing the result.
| Outcome of the test runner step (e.g. steps.<id>.outcome). | ||
| Pass this to detect silent failures where the test runner crashed | ||
| but the JUnit report appears clean. | ||
| Values: 'success', 'failure', 'cancelled', or omit entirely. |
There was a problem hiding this comment.
steps.<id>.outcome can also be skipped in GitHub Actions. This input’s description currently lists only success, failure, and cancelled, but the implementation will treat any other non-empty value (including skipped) as a failing exit code. Please either document skipped explicitly (and how it should map), or add an input validation check to reject unexpected values to avoid confusing/incorrect behavior when users pass steps.<id>.outcome.
| Values: 'success', 'failure', 'cancelled', or omit entirely. | |
| Values: 'success', 'failure', 'cancelled', 'skipped', or omit entirely. | |
| Note: any non-empty value other than 'success' is treated as a failing outcome. |
|
Duplicate of #174 |
Pass the test runner step outcome to set MERGIFY_TEST_EXIT_CODE,
allowing the CLI to detect cases where the runner crashed but the
JUnit report appears clean.
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com