Skip to content

Conversation

@jviotti
Copy link
Member

@jviotti jviotti commented Dec 26, 2025

See: #589
Signed-off-by: Juan Cruz Viotti [email protected]

@jviotti jviotti force-pushed the ctrf-json branch 2 times, most recently from 8c4493a to 64d8e2a Compare December 26, 2025 18:25
@jviotti jviotti changed the title [WIP] Emit CTRF output on test when --json is passed Emit CTRF output on test when --json is passed Dec 26, 2025
@jviotti jviotti marked this pull request as ready for review December 26, 2025 18:25
@augmentcode
Copy link

augmentcode bot commented Dec 26, 2025

🤖 Augment PR Summary

Summary: This PR extends the test command to emit results in CTRF (Common Test Report Format) when --json/-j is provided, enabling easier integration with CI dashboards and standardized reporting.

Changes:

  • Refactors src/command_test.cc into reusable helpers (suite parsing, text reporting, CTRF reporting)
  • Implements CTRF JSON report generation (tool metadata, summary counts, per-test entries with duration/file/line)
  • Updates CLI help text and testing documentation to mention CTRF output via --json
  • Adds new UNIX shell tests that validate CTRF output against a vendored CTRF JSON Schema and compare normalized output
  • Adjusts an existing test to expect clean JSON error output when --json is used
  • Vendors CTRF schema artifacts under vendor/ctrf/ and registers the CTRF dependency

Technical Notes: The CTRF output is printed via sourcemeta::core::prettify and failures still use exit code 2 to distinguish test failures from execution errors.

🤖 Was this summary useful? React with 👍 or 👎

Copy link

@augmentcode augmentcode bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

@jviotti jviotti force-pushed the ctrf-json branch 2 times, most recently from 30efcc0 to 7a8526c Compare December 26, 2025 18:46
@jviotti jviotti merged commit 6f84cda into main Dec 26, 2025
12 checks passed
@jviotti jviotti deleted the ctrf-json branch December 26, 2025 19:03
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Jan 5, 2026
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [sourcemeta/jsonschema](https://github.com/sourcemeta/jsonschema) | minor | `v13.2.1` → `v13.6.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>sourcemeta/jsonschema (sourcemeta/jsonschema)</summary>

### [`v13.6.0`](https://github.com/sourcemeta/jsonschema/releases/tag/v13.6.0)

[Compare Source](sourcemeta/jsonschema@v13.5.0...v13.6.0)

#### What's Changed

- Emit CTRF output on `test` when `--json` is passed by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;593](sourcemeta/jsonschema#593)
- Disable `orphan_definitions` for now by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;594](sourcemeta/jsonschema#594)

**Full Changelog**: <sourcemeta/jsonschema@v13.5.0...v13.6.0>

### [`v13.5.0`](https://github.com/sourcemeta/jsonschema/releases/tag/v13.5.0)

[Compare Source](sourcemeta/jsonschema@v13.4.0...v13.5.0)

#### What's Changed

- Fix minor shell auto-completion mismatches by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;587](sourcemeta/jsonschema#587)
- Lint property names that are not code-generation friendly by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;588](sourcemeta/jsonschema#588)

**Full Changelog**: <sourcemeta/jsonschema@v13.4.0...v13.5.0>

### [`v13.4.0`](https://github.com/sourcemeta/jsonschema/releases/tag/v13.4.0)

[Compare Source](sourcemeta/jsonschema@v13.3.0...v13.4.0)

#### What's Changed

- Add various linter and canonicalizer new rules by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;586](sourcemeta/jsonschema#586)

**Full Changelog**: <sourcemeta/jsonschema@v13.3.0...v13.4.0>

### [`v13.3.0`](https://github.com/sourcemeta/jsonschema/releases/tag/v13.3.0)

[Compare Source](sourcemeta/jsonschema@v13.2.1...v13.3.0)

#### What's Changed

- Implement a `canonicalize` command by [@&#8203;jviotti](https://github.com/jviotti) in [#&#8203;585](sourcemeta/jsonschema#585)

**Full Changelog**: <sourcemeta/jsonschema@v13.2.1...v13.3.0>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi42NC4xIiwidXBkYXRlZEluVmVyIjoiNDIuNjYuOCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90IiwiYXV0b21hdGlvbjpib3QtYXV0aG9yZWQiLCJkZXBlbmRlbmN5LXR5cGU6Om1pbm9yIl19-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants