From 62bd12f3873d5971d1a2b7515bdaee03cd538c7c Mon Sep 17 00:00:00 2001 From: Sofus Albertsen <4633181+sofusalbertsen@users.noreply.github.com> Date: Fri, 12 Dec 2025 10:12:58 +0100 Subject: [PATCH] Docs: adding new tutorial about CTRF custom attestations --- .../tutorials/custom-attestation-ctrf.md | 70 +++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 docs.kosli.com/content/tutorials/custom-attestation-ctrf.md diff --git a/docs.kosli.com/content/tutorials/custom-attestation-ctrf.md b/docs.kosli.com/content/tutorials/custom-attestation-ctrf.md new file mode 100644 index 000000000..d7800d084 --- /dev/null +++ b/docs.kosli.com/content/tutorials/custom-attestation-ctrf.md @@ -0,0 +1,70 @@ +--- +title: "Creating custom CTRF attestation type" +bookCollapseSection: false +weight: 508 +summary: "In this tutorial, we will create a custom attestation type with schema and evaluation for Common Test Report Format" +--- + +## Custom Attestations + +Custom attestations in Kosli allow you to report and verify any type of data as part of your software delivery process. When defining a custom attestation type, you provide two powerful mechanisms to ensure data integrity and compliance: + +1. **Schema**: A [JSON schema](https://json-schema.org/learn) that validates the format of the data being reported. This ensures that the input strictly adheres to the expected structure, preventing malformed data from entering your system. +2. **JQ Rule**: A [jq](https://jqlang.github.io/jq/) expression used to evaluate the data against your compliance policies. This rule determines whether the reported data passes or fails your requirements (e.g., "zero failed tests"). + +## Common Test Report Format (CTRF) + +The [Common Test Report Format (CTRF)](https://ctrf.io/) is a standardized JSON schema for test execution reports. It solves the problem of fragmented test output formats by providing a universal structure, regardless of the testing framework (Jest, Pytest, Mocha, etc.) or language used. + +Adopting a common format like CTRF significantly simplifies compliance evaluation. Instead of writing unique parsing logic and compliance rules for every testing tool in your stack, you can define a single Kosli attestation type for CTRF. This allows you to uniformly enforce quality gates—such as "no failed tests"—across all your projects. + +## Creating the Custom Type + +To start reporting CTRF results to Kosli, you first need to create a custom attestation type. We will name it `ctrf`. + +### 1. Define the Schema + +First, ensure you have a JSON schema that matches the CTRF specification. This schema will be used to validate that the reports sent to Kosli are valid CTRF reports. + +Download the [official schema](https://ctrf.io/docs/full-schema) to a file named `ctrf-schema.json` with the necessary validation structure. + +### 2. Create the Type via CLI + +Use the `kosli create attestation-type` command to define the new type. We will add a jq rule to ensure that the number of failed tests in the summary is zero. + +```shell +kosli create attestation-type ctrf \ + --description "Attestation for Common Test Report Format (CTRF)" \ + --schema ctrf-schema.json \ + --jq '.results.summary.failed == 0' +``` + +In this command: + +* `--schema ctrf-schema.json`: Points to the JSON schema file for validation. +* `--jq '.results.summary.failed == 0'`: Sets the compliance rule. The attestation will only pass if the `failed` count in the report summary is exactly 0. + +### 3. Report the Attestation + +Once your tests have run and generated a CTRF report (e.g., `ctrf-report.json`), you can report it to Kosli using the `kosli attest custom` command. + +```shell +kosli attest custom \ + --type ctrf \ + --name playwright-tests \ + --attestation-data ctrf-report.json +``` + +In this command: + +* `--name playwright-tests`: A name for this specific attestation instance (e.g., identifying the test suite). +* `--attestation-data ctrf-report.json`: The path to the actual CTRF report file generated by your test runner. +* `--type ctrf`: Specifies the custom attestation type we created earlier. + +Kosli will validate `ctrf-report.json` against the schema and evaluate the jq rule. If the report is valid and `.results.summary.failed` is 0, the attestation will be marked as compliant. + +Resources: + +* [create custom attestation type](/client_reference/kosli_create_attestation-type) +* [report custom attestation to an artifact or a trail](/client_reference/kosli_attest_custom/) for usage details and examples. +* [Naming convention for attestation types in Kosli](/implementation_guide/phase_2/plan_organizational_structure/naming_conventions/attestation_types/)