Skip to content

Commit 5449030

Browse files
rhmdndrhmdnd
authored
CMP-3765: Bootstrap samples for OpenShift Virt profile (#813)
Let's use the config samples to show how users can write CustomRule resources to harden an OpenShift virtualization cluster. These updates are isolated to an openshift-virtualization directory with a kustomization file so that they're easy to reuse and apply in existing clusters (e.g., oc apply -k config/samples/custom-rules/openshift-virtualization/). This idea can be expanded on to show how other contributors and teams can ship their own content for the Compliance Operator, and bundle it such that it can be reused by others. One potential future workflow would be reuse this idea by putting necessary rules in a directory, build the rules and rbac changes into a scratch image, upload the image to registry, and reuse the profile bundle feature to parse the rules and load them into the compliance operator.
1 parent bf01f7e commit 5449030

9 files changed

+226
-0
lines changed

‎config/samples/custom-rules/openshift-virtualization/README.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# OpenShift Virtualization Hardening with `CustomRule`
2+
3+
The Compliance Operator (version 1.8.0 and newer) includes a `CustomRule`
4+
Custom Resource Definition (CRD). This feature allows you to write your own
5+
compliance checks.
6+
7+
This collection of samples demonstrates how to use `CustomRule` to harden an
8+
OpenShift Virtualization cluster.
9+
10+
## Running a Scan
11+
12+
The following command applies all the `CustomRule` resources in this directory,
13+
bundles them into a `TailoredProfile`, and immediately starts a compliance
14+
scan:
15+
16+
```bash
17+
$ oc apply -k config/samples/custom-rules/openshift-virtualization
18+
```
19+
20+
This scan will produce `ComplianceCheckResult` resources, one for each rule.
21+
You can monitor the progress of the scan and view the results using this
22+
command:
23+
24+
```bash
25+
$ oc get suites,scans,compliancecheckresults
26+
```
27+
28+
The `kustomization.yaml` file in this directory contains the full workflow
29+
using the rules, profiles, bindings, and permissions for the Compliance
30+
Operator to execute these checks.

‎config/samples/custom-rules/openshift-virtualization/kubevirt-no-permitted-host-devices.yaml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
apiVersion: compliance.openshift.io/v1alpha1
2+
kind: CustomRule
3+
metadata:
4+
name: kubevirt-no-permitted-host-devices
5+
namespace: openshift-compliance
6+
spec:
7+
title: "KubeVirt Must Not Permit Host Devices"
8+
id: kubevirt_no_permitted_host_devices
9+
description: |-
10+
Host devices should not be permitted to virtualization workloads unless
11+
absolutely necessary for workload execution. Allowing host devices provides
12+
direct access to host hardware, which can introduce security risks including
13+
unauthorized access to sensitive hardware resources, potential for privilege
14+
escalation, and bypass of virtualization security boundaries.
15+
16+
By default, no host devices should be trusted or permitted for use by
17+
virtualization workloads.
18+
failureReason: |-
19+
The '.spec.permittedHostDevices' field is set in the 'kubevirt-hyperconverged'
20+
resource, allowing host devices to be used by virtualization workloads.
21+
severity: Medium
22+
checkType: Platform
23+
scannerType: CEL
24+
inputs:
25+
- name: hcoList
26+
kubernetesInputSpec:
27+
apiVersion: hco.kubevirt.io/v1beta1
28+
resource: hyperconvergeds
29+
expression: |
30+
hcoList.items.filter(h,
31+
h.metadata.name == 'kubevirt-hyperconverged' &&
32+
h.metadata.namespace == 'openshift-cnv'
33+
).size() == 1 &&
34+
hcoList.items.filter(h,
35+
h.metadata.name == 'kubevirt-hyperconverged' &&
36+
h.metadata.namespace == 'openshift-cnv'
37+
).all(h,
38+
!has(h.spec.permittedHostDevices) ||
39+
h.spec.permittedHostDevices == null ||
40+
(has(h.spec.permittedHostDevices.pciHostDevices) && size(h.spec.permittedHostDevices.pciHostDevices) == 0) &&
41+
(has(h.spec.permittedHostDevices.mediatedDevices) && size(h.spec.permittedHostDevices.mediatedDevices) == 0)
42+
)

‎config/samples/custom-rules/openshift-virtualization/kubevirt-nonroot-feature-gate-is-enabled.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
apiVersion: compliance.openshift.io/v1alpha1
2+
kind: CustomRule
3+
metadata:
4+
name: kubevirt-nonroot-feature-gate-is-enabled
5+
namespace: openshift-compliance
6+
spec:
7+
title: "KubeVirt nonRoot Feature Gate Must Be Enabled"
8+
id: kubevirt_nonroot_feature_gate_is_enabled
9+
description: |-
10+
Unauthorized access to a root account without restrictions implemented by
11+
the nonRoot feature introduces the risk of unintended or unauthorized
12+
access to privilege elevation and the ability to perform administrative
13+
tasks.
14+
failureReason: |-
15+
The '.spec.featureGates.nonRoot' field is missing or not set to 'true' in
16+
the 'kubevirt-hyperconverged' resource.
17+
severity: Medium
18+
checkType: Platform
19+
scannerType: CEL
20+
inputs:
21+
- name: hcoList
22+
kubernetesInputSpec:
23+
apiVersion: hco.kubevirt.io/v1beta1
24+
resource: hyperconvergeds
25+
expression: |
26+
hcoList.items.filter(h,
27+
h.metadata.name == 'kubevirt-hyperconverged' &&
28+
h.metadata.namespace == 'openshift-cnv'
29+
).size() == 1 &&
30+
hcoList.items.filter(h,
31+
h.metadata.name == 'kubevirt-hyperconverged' &&
32+
h.metadata.namespace == 'openshift-cnv'
33+
).all(h,
34+
has(h.spec.featureGates) &&
35+
has(h.spec.featureGates.nonRoot) &&
36+
h.spec.featureGates.nonRoot == true
37+
)

‎config/samples/custom-rules/openshift-virtualization/kubevirt-persistent-reservation-disabled.yaml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
apiVersion: compliance.openshift.io/v1alpha1
2+
kind: CustomRule
3+
metadata:
4+
name: kubevirt-persistent-reservation-disabled
5+
namespace: openshift-compliance
6+
spec:
7+
title: "KubeVirt Persistent Reservation Feature Gate Must Be Disabled"
8+
id: kubevirt_persistent_reservation_disabled
9+
description: |-
10+
The persistent reservation feature gate in KubeVirt allows virtual machines
11+
to use SCSI persistent reservations, which provide exclusive access to shared
12+
storage. This feature should be disabled unless explicitly required for
13+
workload operation, as it can introduce security risks by allowing VMs to
14+
claim exclusive access to storage resources, potentially impacting availability
15+
and enabling resource manipulation outside normal access controls.
16+
failureReason: |-
17+
The '.spec.featureGates.persistentReservation' field is missing, not set,
18+
or not set to 'false' in the 'kubevirt-hyperconverged' resource.
19+
severity: Medium
20+
checkType: Platform
21+
scannerType: CEL
22+
inputs:
23+
- name: hcoList
24+
kubernetesInputSpec:
25+
apiVersion: hco.kubevirt.io/v1beta1
26+
resource: hyperconvergeds
27+
expression: |
28+
hcoList.items.filter(h,
29+
h.metadata.name == 'kubevirt-hyperconverged' &&
30+
h.metadata.namespace == 'openshift-cnv'
31+
).size() == 1 &&
32+
hcoList.items.filter(h,
33+
h.metadata.name == 'kubevirt-hyperconverged' &&
34+
h.metadata.namespace == 'openshift-cnv'
35+
).all(h,
36+
has(h.spec.featureGates) &&
37+
has(h.spec.featureGates.persistentReservation) &&
38+
h.spec.featureGates.persistentReservation == false
39+
)

‎config/samples/custom-rules/openshift-virtualization/kustomization.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
resources:
2+
- ocp-virt-permissions.yaml
3+
- ocp-virt-rbac-binding.yaml
4+
- kubevirt-nonroot-feature-gate-is-enabled.yaml
5+
- kubevirt-no-permitted-host-devices.yaml
6+
- kubevirt-persistent-reservation-disabled.yaml
7+
- tailored-profile.yaml
8+
- scan-setting-binding.yaml

‎config/samples/custom-rules/openshift-virtualization/ocp-virt-permissions.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: rbac.authorization.k8s.io/v1
2+
kind: ClusterRole
3+
metadata:
4+
name: ocp-virt-api-resource-collector
5+
rules:
6+
- apiGroups:
7+
- hco.kubevirt.io
8+
resources:
9+
- "*"
10+
verbs:
11+
- get
12+
- list
13+
- watch

‎config/samples/custom-rules/openshift-virtualization/ocp-virt-rbac-binding.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
kind: ClusterRoleBinding
3+
apiVersion: rbac.authorization.k8s.io/v1
4+
metadata:
5+
name: ocp-virt-api-resource-collector
6+
subjects:
7+
- kind: ServiceAccount
8+
name: api-resource-collector
9+
namespace: openshift-compliance
10+
roleRef:
11+
kind: ClusterRole
12+
name: ocp-virt-api-resource-collector
13+
apiGroup: rbac.authorization.k8s.io

‎config/samples/custom-rules/openshift-virtualization/scan-setting-binding.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: compliance.openshift.io/v1alpha1
2+
kind: ScanSettingBinding
3+
metadata:
4+
name: openshift-virt-binding
5+
namespace: openshift-compliance
6+
profiles:
7+
- apiGroup: compliance.openshift.io/v1alpha1
8+
kind: TailoredProfile
9+
name: openshift-virt-platform-checks
10+
settingsRef:
11+
apiGroup: compliance.openshift.io/v1alpha1
12+
kind: ScanSetting
13+
name: default

‎config/samples/custom-rules/openshift-virtualization/tailored-profile.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
apiVersion: compliance.openshift.io/v1alpha1
2+
kind: TailoredProfile
3+
metadata:
4+
name: openshift-virt-platform-checks
5+
namespace: openshift-compliance
6+
spec:
7+
description: Custom security checks for OpenShift Virtualization
8+
enableRules:
9+
- kind: CustomRule
10+
name: kubevirt-nonroot-feature-gate-is-enabled
11+
rationale: |-
12+
Unauthorized access to a root account without restrictions implemented
13+
by the nonRoot feature introduces the risk of unintended or
14+
unauthorized access to privilege elevation and the ability to perform
15+
administrative tasks.
16+
- kind: CustomRule
17+
name: kubevirt-no-permitted-host-devices
18+
rationale: |-
19+
Host devices should not be permitted to virtualization workloads unless
20+
absolutely necessary. Allowing host devices provides direct access to
21+
host hardware, which can introduce security risks including
22+
unauthorized access to sensitive hardware resources and potential
23+
privilege escalation.
24+
- kind: CustomRule
25+
name: kubevirt-persistent-reservation-disabled
26+
rationale: |-
27+
The persistent reservation feature gate should be disabled unless
28+
explicitly required. This feature allows VMs to claim exclusive access
29+
to storage resources, potentially impacting availability and enabling
30+
resource manipulation outside normal access controls.
31+
title: Platform checks for OpenShift Virtualization

0 commit comments

Comments
 (0)