Skip to content

Commit fe95083

Browse files
docs: add tutorial for creating custom CTRF attestation type
1 parent 0eb8fc8 commit fe95083

File tree

1 file changed

+70
-0
lines changed

1 file changed

+70
-0
lines changed
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
title: "Creating custom CTRF attestation type"
3+
bookCollapseSection: false
4+
weight: 508
5+
summary: "In this tutorial, we will create a custom attestation type with schema and evaluation for Common Test Report Format"
6+
---
7+
8+
## Custom Attestations
9+
10+
Custom attestations in Kosli allow you to report and verify any type of data as part of your software delivery process. When defining a custom attestation type, you provide two powerful mechanisms to ensure data integrity and compliance:
11+
12+
1. **Schema**: A [JSON schema](https://json-schema.org/learn) that validates the format of the data being reported. This ensures that the input strictly adheres to the expected structure, preventing malformed data from entering your system.
13+
2. **JQ Rule**: A [jq](https://jqlang.github.io/jq/) expression used to evaluate the data against your compliance policies. This rule determines whether the reported data passes or fails your requirements (e.g., "zero failed tests").
14+
15+
## Common Test Report Format (CTRF)
16+
17+
The [Common Test Report Format (CTRF)](https://ctrf.io/) is a standardized JSON schema for test execution reports. It solves the problem of fragmented test output formats by providing a universal structure, regardless of the testing framework (Jest, Pytest, Mocha, etc.) or language used.
18+
19+
Adopting a common format like CTRF significantly simplifies compliance evaluation. Instead of writing unique parsing logic and compliance rules for every testing tool in your stack, you can define a single Kosli attestation type for CTRF. This allows you to uniformly enforce quality gates—such as "no failed tests"—across all your projects.
20+
21+
## Creating the Custom Type
22+
23+
To start reporting CTRF results to Kosli, you first need to create a custom attestation type. We will name it `ctrf`.
24+
25+
### 1. Define the Schema
26+
27+
First, ensure you have a JSON schema that matches the CTRF specification. This schema will be used to validate that the reports sent to Kosli are valid CTRF reports.
28+
29+
Download the [official schema](https://ctrf.io/docs/full-schema) to a file named `ctrf-schema.json` with the necessary validation structure.
30+
31+
### 2. Create the Type via CLI
32+
33+
Use the `kosli create attestation-type` command to define the new type. We will add a jq rule to ensure that the number of failed tests in the summary is zero.
34+
35+
```shell
36+
kosli create attestation-type ctrf \
37+
--description "Attestation for Common Test Report Format (CTRF)" \
38+
--schema ctrf-schema.json \
39+
--jq '.results.summary.failed == 0'
40+
```
41+
42+
In this command:
43+
44+
* `--schema ctrf-schema.json`: Points to the JSON schema file for validation.
45+
* `--jq '.results.summary.failed == 0'`: Sets the compliance rule. The attestation will only pass if the `failed` count in the report summary is exactly 0.
46+
47+
### 3. Report the Attestation
48+
49+
Once your tests have run and generated a CTRF report (e.g., `ctrf-report.json`), you can report it to Kosli using the `kosli attest custom` command.
50+
51+
```shell
52+
kosli attest custom \
53+
--type ctrf \
54+
--name playwright-tests \
55+
--attestation-data ctrf-report.json
56+
```
57+
58+
In this command:
59+
60+
* `--name playwright-tests`: A name for this specific attestation instance (e.g., identifying the test suite).
61+
* `--attestation-data ctrf-report.json`: The path to the actual CTRF report file generated by your test runner.
62+
* `--type ctrf`: Specifies the custom attestation type we created earlier.
63+
64+
Kosli will validate `ctrf-report.json` against the schema and evaluate the jq rule. If the report is valid and `.results.summary.failed` is 0, the attestation will be marked as compliant.
65+
66+
Resources:
67+
68+
* [create custom attestation type](/client_reference/kosli_create_attestation-type)
69+
* [report custom attestation to an artifact or a trail](/client_reference/kosli_attest_custom/) for usage details and examples.
70+
* [Naming convention for attestation types in Kosli](/implementation_guide/phase_2/plan_organizational_structure/naming_conventions/attestation_types/)

0 commit comments

Comments
 (0)