feat(policies): document policies (#1106)

jiparis · web-flow · commit 3ca0d61844ed · 2024-07-18T17:05:38.000+02:00
Signed-off-by: Jose I. Paris &lt;jiparis@chainloop.dev&gt;
diff --git a/docs/docs/reference/policies.md b/docs/docs/reference/policies.md
@@ -0,0 +1,157 @@
+---
+title: Policies
+---
+
+Starting with Chainloop [0.93.8](https://github.com/chainloop-dev/chainloop/releases/tag/v0.93.8), operators can attach policies to contracts. 
+These policies will be evaluated against the different materials and the statement metadata, if required. The result of the evaluation is informed as a list of possible violations and added to the attestation statement
+before signing and sending it to Chainloop. 
+
+Currently, policy violations won't block `attestation push` commands, but instead, we chose to include them in the attestation so that they can 
+be used for building server side control gates.
+
+### Policy specification
+A policy can be defined in a YAML document, like this:
+```yaml
+# cyclonedx-licenses.yaml
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: cyclonedx-licenses # (1)
+spec:
+  type: SBOM_CYCLONEDX_JSON # (2)
+  embedded: | # (3)
+    package main
+
+    deny[msg] {
+      count(without_license) > 0
+      msg := "SBOM has components without licenses"
+    }
+
+    without_license = {comp.purl |
+      some i
+      comp := input.components[i]
+      not comp.licenses
+    }
+```
+In this particular example, we see:
+* (1) policies have a name
+* (2) they can be optionally applied to a specific type of material (check [the documentation](./operator/contract#material-schema) for the supported types). If no type is specified, a material name will need to be provided explicitly in the contract.
+* (3) they have a policy script that it's evaluated against the material (in this case a CycloneDX SBOM report). Currently, only [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/#learning-rego) policies are supported.
+
+Policy scripts could also be specified in a detached form:
+```yaml
+...
+spec:
+  type: SBOM_CYCLONEDX_JSON
+  path: my-script.rego
+```
+
+### Applying policies to contracts
+When defining a contract, a new `policies` section can be specified. Policies can be applied to any material, but also to the attestation statement as a whole.
+```yaml
+schemaVersion: v1
+materials:
+  - name: sbom
+    type: SBOM_CYCLONEDX_JSON
+  - name: another-sbom
+    type: SBOM_CYCLONEDX_JSON
+  - name: my-image
+    type: CONTAINER_IMAGE
+policies:
+  materials: # policies applied to materials
+    - ref: cyclonedx-licenses.yaml # (1)
+  attestation: # policies applied to the whole attestation
+    - ref: https://github.com/chainloop/chainloop-dev/blob/main/docs/examples/policies/chainloop-commit.yaml # (2)
+```
+Here we can see that:
+- (1) materials will be validated against `cyclonedx-licenses.yaml` policy. But, since that policy has a `type` property set to `SBOM_CYCLONEDX_JSON`, only SBOM materials (`sbom` and `another-sbom` in this case) will be evaluated. 
+  
+  If we wanted to only evaluate the policy against the `sbom` material, and skip the other, we should filter them by name:
+  ```yaml
+  policies:
+    materials:
+      - ref: cyclonedx-licenses.yaml
+        selector: # (3)
+          name: sbom
+  ```
+  Here, in (3), we are making explicit that only `sbom` material must be evaluated by the `cyclonedx-licenses.yaml` policy.
+- (2) the attestation in-toto statement as a whole will be evaluated against the remote policy `chainloop-commit.yaml`, which has a `type` property set to `ATTESTATION`. 
+  This brings the opportunity to validate global attestation properties, like annotations, the presence of a material, etc. You can see this policy and other examples in the [examples folder](https://github.com/chainloop-dev/chainloop/tree/main/docs/examples/policies).
+
+Finally, note that material policies are evaluated during `chainloop attestation add` commands, while attestation policies are evaluated in `chainloop attestation push` command.
+
+### Embedding or referencing policies
+There are two ways to attach a policy to a contract:
+* **By referencing it**, as it can be seen in the examples above. `ref` property admits a local (filesystem) or remote reference (HTTPS). For example:
+  ```yaml
+  policies:
+    materials: 
+      - ref: cyclonedx-licenses.yaml # local reference
+  ```
+  and
+  ```yaml
+  policies:
+    materials:
+      - ref: https://github.com/chainloop/chainloop-dev/blob/main/docs/examples/policies/cyclonedx-licenses.yaml
+  ```
+  are both equivalent. The advantage of having remote policies is that they can be easily reused, allowing organizations to create policy catalogs.
+
+* If preferred, authors could create self-contained contracts **embedding policy specifications**. The main advantage of this method is that it ensures that the policy source cannot be changed, as it's stored and versioned within the contract:
+  ```yaml
+  schemaVersion: v1
+  materials:
+    - name: sbom
+      type: SBOM_CYCLONEDX_JSON
+  policies:
+    materials:
+      - policy: # (1)
+          apiVersion: workflowcontract.chainloop.dev/v1
+          kind: Policy
+          metadata:
+            name: sbom-licenses 
+          spec:
+            type: SBOM_CYCLONEDX_JSON 
+            embedded: | 
+              package main
+
+              deny[msg] {
+                count(without_license) > 0
+                msg := "SBOM has components without licenses"
+              }
+
+              without_license = {comp.purl |
+                some i
+                comp := input.components[i]
+                not comp.licenses
+              }
+    ```
+  In the example above, we can see that, when referenced by the `policy` attribute (1), a full policy can be embedded in the contract.
+  
+### Rego scripts
+Currently, policy scripts are assumed to be written in [Rego language](https://www.openpolicyagent.org/docs/latest/policy-language/#learning-rego). Other policy engines might be implemented in the future.
+The only requirement of the policy is the existence of one or multiple `deny` rules, which evaluate to a **list of violation strings**.
+For example, this policy script:
+```yaml
+package main
+
+deny[msg] {
+  not is_approved
+  
+  msg:= "Container image is not approved"
+}
+
+is_approved {
+  input.predicate.materials[_].annotations["chainloop.material.type"] == "CONTAINER_IMAGE"
+  
+  input.predicate.annotations.approval == "true"
+}
+```
+when evaluated against an attestation, will generate the following output if the expected annotation is not present:
+```json
+{
+    "deny": [
+        "Container image is not approved"
+    ]
+}
+```
+Make sure you test your policies in https://play.openpolicyagent.org/, since you might get different results when using Rego V1 syntax, as there are [some breaking changes](https://www.openpolicyagent.org/docs/latest/opa-1/).
diff --git a/docs/examples/policies/chainloop-commit.yaml b/docs/examples/policies/chainloop-commit.yaml
@@ -0,0 +1,34 @@
+# Copyright 2024 The Chainloop Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: chainloop-commit
+spec:
+  type: ATTESTATION
+  embedded: |
+    package main
+    
+    deny[msg] {
+      not has_commit
+      msg := "missing commit in attestation material"
+    }
+    
+    has_commit {
+      some i
+      input.subject[i].name == "git.head"
+      input.subject[i].digest.sha1
+    }
+
diff --git a/docs/examples/policies/chainloop-qa.yaml b/docs/examples/policies/chainloop-qa.yaml
@@ -0,0 +1,37 @@
+# Copyright 2024 The Chainloop Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Checks that there are is a container material with a custom annotation "chainloop-qa-approval=true"
+# chainloop att push
+# This can be used as a control gate to allow e.g. a production deployment
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: sarif-errors
+spec:
+  type: ATTESTATION
+  embedded: |
+    package main
+    
+    deny[msg] {
+      not is_approved
+      
+      msg:= "Container image is not approved"
+    }
+    
+    is_approved {
+      input.predicate.materials[_].annotations["chainloop.material.type"] == "CONTAINER_IMAGE"
+      
+      input.predicate.annotations.approval == "true"
+    }
diff --git a/docs/examples/policies/cyclonedx-licenses.yaml b/docs/examples/policies/cyclonedx-licenses.yaml
@@ -0,0 +1,34 @@
+# Copyright 2024 The Chainloop Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Checks that all components have a license
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: cyclonedx-licenses
+spec:
+  type: SBOM_CYCLONEDX_JSON
+  embedded: |
+    package main
+    
+    deny[msg] {
+      count(without_license) > 0
+      msg := "SBOM has components without licenses"
+    }
+      
+    without_license = {comp.purl |
+      some i
+      comp := input.components[i]
+      not comp.licenses
+    }
diff --git a/docs/examples/policies/sarif-errors.yaml b/docs/examples/policies/sarif-errors.yaml
@@ -0,0 +1,31 @@
+# Copyright 2024 The Chainloop Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Checks that there are no errors in the SARIF report
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: sarif-errors
+spec:
+  embedded: |
+    package main
+    
+    deny[msg] {
+      has_errors
+      msg := "There are errors in the SARIF report"
+    }
+    
+    has_errors {
+      input.runs[_].results[_].level == "error"
+    }
diff --git a/docs/examples/policies/sbom-present.yaml b/docs/examples/policies/sbom-present.yaml
@@ -0,0 +1,43 @@
+# Copyright 2024 The Chainloop Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: sbom-present
+spec:
+  type: ATTESTATION
+  embedded: |
+    package main
+      
+    # Verifies there is a SBOM material, even if not enforced by contract
+    
+    import future.keywords.contains
+    import future.keywords.in
+    
+    deny[msg] {
+      not has_sbom
+      msg := "missing SBOM material"
+    }
+    
+    # Collect all material types
+    kinds contains kind {
+      some material in input.predicate.materials
+      kind := material.annotations["chainloop.material.type"]
+    }
+      
+    has_sbom {
+      values := ["SBOM_SPDX_JSON","SBOM_CYCLONEDX_JSON"]      
+      kinds[_] == values[_]
+    }
diff --git a/docs/examples/policies/sbom-syft.yaml b/docs/examples/policies/sbom-syft.yaml
@@ -0,0 +1,37 @@
+# Copyright 2024 The Chainloop Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: workflowcontract.chainloop.dev/v1
+kind: Policy
+metadata:
+  name: made-with-syft
+spec:
+  type: SBOM_SPDX_JSON
+  embedded: |
+    package main
+
+    import future.keywords.in
+
+    # Verifies that the SPDX was created with Syft
+    
+    deny[msg] {
+      not made_with_syft
+
+      msg := "Not made with syft"
+    }
+
+    made_with_syft {
+      some creator in input.creationInfo.creators
+      contains(creator, "syft")
+    }
diff --git a/docs/examples/policies/trivy-vulns.yaml b/docs/examples/policies/trivy-vulns.yaml
diff --git a/docs/sidebars.js b/docs/sidebars.js
