Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions docs.kosli.com/content/tutorials/custom-attestation-ctrf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: "Creating custom CTRF attestation type"
bookCollapseSection: false
weight: 508
summary: "In this tutorial, we will create a custom attestation type with schema and evaluation for Common Test Report Format"
---

## Custom Attestations

Custom attestations in Kosli allow you to report and verify any type of data as part of your software delivery process. When defining a custom attestation type, you provide two powerful mechanisms to ensure data integrity and compliance:

1. **Schema**: A [JSON schema](https://json-schema.org/learn) that validates the format of the data being reported. This ensures that the input strictly adheres to the expected structure, preventing malformed data from entering your system.
2. **JQ Rule**: A [jq](https://jqlang.github.io/jq/) expression used to evaluate the data against your compliance policies. This rule determines whether the reported data passes or fails your requirements (e.g., "zero failed tests").

## Common Test Report Format (CTRF)

The [Common Test Report Format (CTRF)](https://ctrf.io/) is a standardized JSON schema for test execution reports. It solves the problem of fragmented test output formats by providing a universal structure, regardless of the testing framework (Jest, Pytest, Mocha, etc.) or language used.

Adopting a common format like CTRF significantly simplifies compliance evaluation. Instead of writing unique parsing logic and compliance rules for every testing tool in your stack, you can define a single Kosli attestation type for CTRF. This allows you to uniformly enforce quality gates—such as "no failed tests"—across all your projects.

## Creating the Custom Type

To start reporting CTRF results to Kosli, you first need to create a custom attestation type. We will name it `ctrf`.

### 1. Define the Schema

First, ensure you have a JSON schema that matches the CTRF specification. This schema will be used to validate that the reports sent to Kosli are valid CTRF reports.

Download the [official schema](https://ctrf.io/docs/full-schema) to a file named `ctrf-schema.json` with the necessary validation structure.

### 2. Create the Type via CLI

Use the `kosli create attestation-type` command to define the new type. We will add a jq rule to ensure that the number of failed tests in the summary is zero.

```shell
kosli create attestation-type ctrf \
--description "Attestation for Common Test Report Format (CTRF)" \
--schema ctrf-schema.json \
--jq '.results.summary.failed == 0'
```

In this command:

* `--schema ctrf-schema.json`: Points to the JSON schema file for validation.
* `--jq '.results.summary.failed == 0'`: Sets the compliance rule. The attestation will only pass if the `failed` count in the report summary is exactly 0.

### 3. Report the Attestation

Once your tests have run and generated a CTRF report (e.g., `ctrf-report.json`), you can report it to Kosli using the `kosli attest custom` command.

```shell
kosli attest custom \
--type ctrf \
--name playwright-tests \
--attestation-data ctrf-report.json
```

In this command:

* `--name playwright-tests`: A name for this specific attestation instance (e.g., identifying the test suite).
* `--attestation-data ctrf-report.json`: The path to the actual CTRF report file generated by your test runner.
* `--type ctrf`: Specifies the custom attestation type we created earlier.

Kosli will validate `ctrf-report.json` against the schema and evaluate the jq rule. If the report is valid and `.results.summary.failed` is 0, the attestation will be marked as compliant.

Resources:

* [create custom attestation type](/client_reference/kosli_create_attestation-type)
* [report custom attestation to an artifact or a trail](/client_reference/kosli_attest_custom/) for usage details and examples.
* [Naming convention for attestation types in Kosli](/implementation_guide/phase_2/plan_organizational_structure/naming_conventions/attestation_types/)