|
| 1 | +--- |
| 2 | +title: "Creating custom CTRF attestation type" |
| 3 | +bookCollapseSection: false |
| 4 | +weight: 508 |
| 5 | +summary: "In this tutorial, we will create a custom attestation type with schema and evaluation for Common Test Report Format" |
| 6 | +--- |
| 7 | + |
| 8 | +## Custom Attestations |
| 9 | + |
| 10 | +Custom attestations in Kosli allow you to report and verify any type of data as part of your software delivery process. When defining a custom attestation type, you provide two powerful mechanisms to ensure data integrity and compliance: |
| 11 | + |
| 12 | +1. **Schema**: A [JSON schema](https://json-schema.org/learn) that validates the format of the data being reported. This ensures that the input strictly adheres to the expected structure, preventing malformed data from entering your system. |
| 13 | +2. **JQ Rule**: A [jq](https://jqlang.github.io/jq/) expression used to evaluate the data against your compliance policies. This rule determines whether the reported data passes or fails your requirements (e.g., "zero failed tests"). |
| 14 | + |
| 15 | +## Common Test Report Format (CTRF) |
| 16 | + |
| 17 | +The [Common Test Report Format (CTRF)](https://ctrf.io/) is a standardized JSON schema for test execution reports. It solves the problem of fragmented test output formats by providing a universal structure, regardless of the testing framework (Jest, Pytest, Mocha, etc.) or language used. |
| 18 | + |
| 19 | +Adopting a common format like CTRF significantly simplifies compliance evaluation. Instead of writing unique parsing logic and compliance rules for every testing tool in your stack, you can define a single Kosli attestation type for CTRF. This allows you to uniformly enforce quality gates—such as "no failed tests"—across all your projects. |
| 20 | + |
| 21 | +## Creating the Custom Type |
| 22 | + |
| 23 | +To start reporting CTRF results to Kosli, you first need to create a custom attestation type. We will name it `ctrf`. |
| 24 | + |
| 25 | +### 1. Define the Schema |
| 26 | + |
| 27 | +First, ensure you have a JSON schema that matches the CTRF specification. This schema will be used to validate that the reports sent to Kosli are valid CTRF reports. |
| 28 | + |
| 29 | +Download the [official schema](https://ctrf.io/docs/full-schema) to a file named `ctrf-schema.json` with the necessary validation structure. |
| 30 | + |
| 31 | +### 2. Create the Type via CLI |
| 32 | + |
| 33 | +Use the `kosli create attestation-type` command to define the new type. We will add a jq rule to ensure that the number of failed tests in the summary is zero. |
| 34 | + |
| 35 | +```shell |
| 36 | +kosli create attestation-type ctrf \ |
| 37 | + --description "Attestation for Common Test Report Format (CTRF)" \ |
| 38 | + --schema ctrf-schema.json \ |
| 39 | + --jq '.results.summary.failed == 0' |
| 40 | +``` |
| 41 | + |
| 42 | +In this command: |
| 43 | + |
| 44 | +* `--schema ctrf-schema.json`: Points to the JSON schema file for validation. |
| 45 | +* `--jq '.results.summary.failed == 0'`: Sets the compliance rule. The attestation will only pass if the `failed` count in the report summary is exactly 0. |
| 46 | + |
| 47 | +### 3. Report the Attestation |
| 48 | + |
| 49 | +Once your tests have run and generated a CTRF report (e.g., `ctrf-report.json`), you can report it to Kosli using the `kosli attest custom` command. |
| 50 | + |
| 51 | +```shell |
| 52 | +kosli attest custom \ |
| 53 | + --type ctrf \ |
| 54 | + --name playwright-tests \ |
| 55 | + --attestation-data ctrf-report.json |
| 56 | +``` |
| 57 | + |
| 58 | +In this command: |
| 59 | + |
| 60 | +* `--name playwright-tests`: A name for this specific attestation instance (e.g., identifying the test suite). |
| 61 | +* `--attestation-data ctrf-report.json`: The path to the actual CTRF report file generated by your test runner. |
| 62 | +* `--type ctrf`: Specifies the custom attestation type we created earlier. |
| 63 | + |
| 64 | +Kosli will validate `ctrf-report.json` against the schema and evaluate the jq rule. If the report is valid and `.results.summary.failed` is 0, the attestation will be marked as compliant. |
| 65 | + |
| 66 | +Resources: |
| 67 | + |
| 68 | +* [create custom attestation type](/client_reference/kosli_create_attestation-type) |
| 69 | +* [report custom attestation to an artifact or a trail](/client_reference/kosli_attest_custom/) for usage details and examples. |
| 70 | +* [Naming convention for attestation types in Kosli](/implementation_guide/phase_2/plan_organizational_structure/naming_conventions/attestation_types/) |
0 commit comments