CMP-2870, CMP-2869, CMP-2870, CMP-2872, CMP-2868: Compliance SDK Compliance Operator Implementation (#811)

* feat: Add CustomRule CRD for custom compliance rules

    Adds a new CustomRule CRD to support user-defined compliance rules with CEL expressions

* Enhance TailoredProfile to support CustomRules

Add CustomRule handling logic
Validate rule types consistency
Update ownership management logic
Ensure correct product type annotation for CustomRUle
Ensure correct scannerType annotation is set

* feat: Add initial CEL expression scanner pod implmentation

* Fix CustomRule api verify-bundle error

Fix an issues that fails verify-bundle make target, we need to move rulepayload under Spec

* Skip not used component for CEL scan

We are going to disable the raw stroage related resources as well as runtime kubeletconfig for CEL as those are not going to be used

* Skip aggregator for cel scanner based Scan

* Handle CustomRule in ScanSettingBinding Controller

We are modifying the ScanSettingBinding controller so that if tailoredProfile is referencing CustomRule will be handled accordingly, we are also removing the not used scannerImage field in ComplianceScan CRD

* Update ScanHandler to support CEL

We are updating the Platform ScanHandler to be able to process CEL based CustomRule.

* Add CCR generation in CEL-Scanner

We are adding the ability to generate ComplianceCheckResult directly in CEL scanner

* Update TailoredProfile status for CustomRule

* Fix tailoredProfile controller annotation generation for Custom CEL Rule

* Add custom rule annotation for ComplianceSuite Controller

* Add CEL-scanner cmd

* Fix UUID generation for CustomRule

* Add CEL demo files

* Fix issues in cel scanner cmd

* Fill correct tp information for CustomRule in the Scan

* Add rbac for CEL Scanner

* Fix cmd flag and client creation for CEL scanner

* Adjust demo, add rbac for kubevirt rule

* Fix issue with CEl scanner

Use correct resouce as input for cel scanner

* Add necessary permission so scanner can set ownership to CCR

* Make KubeResource Input as inline config

This will align with enhancement, make input KubeResource as inline config under input instead
of having it under input.KubeResource

* Add more demo files for CEL

* Update tailoredProfile controller to watch for CustomRule changes

* Fix the CEL aggregation phase

Updating platformScanTypeHandler to handle compliance result
for CEL correctly during aggregator phase so that we can move to
Done state.

* Change check status to Fail for not fetched resource

We are switching the ComplianceCheckStatus to Fail
Instead of error for resource that we are not able to fetch. We will only set it to error when we were having other issues evaluating the rule.
This matches the openscap default behavior.

* Fix duplicated scanner pod for OpenScap scan

This is to revert changes made to scanner pod config, so that OpenScap can run without issues.

* Add support for custom variables in CEL scanner and tailored profile

* Add README for writing and running CEL rules in Compliance Operator

* remove unwanted changes

* Make OpenSCAP as default value for scannerType

This makes changes backward compatible and will also address some e2e failures with rerunner

* Updating scanner field description

* Add CEL scanner module with comprehensive architecture and input handling

This commit introduces a new CEL scanner module designed for compliance checking using Google's Common Expression Language (CEL). Key features include:

- A flexible architecture supporting multiple input types (Kubernetes, filesystem, system commands).
- Comprehensive test coverage with unit, integration, and benchmark tests.
- Detailed documentation including usage guides and architecture principles.
- New input fetchers for Kubernetes and filesystem resources, along with system command execution capabilities.

This implementation enhances the compliance operator's ability to evaluate rules against various data sources, ensuring a robust compliance scanning solution.

* Update go.sum to remove unused dependencies and update existing ones from mergy conflict with master

* Install compliance-sdk go1.23 dependency

* Remove deprecated cel-scanner-refactored.go and eliminate unused components.

* Refactor KubeResource structure in customrule_types.go to integrate KubernetesInputSpec from compliance-sdk

This update modifies the KubeResource type to include a KubernetesInputSpec field, enhancing the input handling for compliance rules. The InputPayload structure has also been adjusted to reflect this change, improving the overall architecture for compliance scanning.

* Add custom rule controller and unit tests for validation

This commit introduces a new controller for managing custom rules within the compliance operator. It includes the implementation of the `add_customrule.go` file, which registers the custom rule controller with the manager. Additionally, comprehensive unit tests have been added in `customrule_controller_test.go` to validate the reconciliation logic and structure of custom rules, ensuring proper handling of various scenarios including valid and invalid rules.

* Update compliance-sdk dependency and enhance CustomRule CRD with validation features

This commit updates the compliance-sdk dependency to a newer version, improving compatibility and functionality. Additionally, it enhances the CustomRule Custom Resource Definition (CRD) by adding additional printer columns for better visibility of rule status and severity. The validation structure for CustomRule inputs has been refined, ensuring more robust error handling and validation during rule processing. Unit tests have been updated to reflect these changes, ensuring comprehensive coverage of the new features.

* Add support for deprecated tailored profile notifications in ComplianceScan for CustomRule

This commit enhances the ComplianceScan controller to notify users when a deprecated tailored profile is in use for CEL scanner types.

* Refactor CEL scanner to integrate compliance-sdk and enhance logging

This commit refactors the CEL scanner implementation to utilize the compliance-sdk for improved resource fetching and scanning capabilities.

* Remove deprecated CEL demo rules and associated resources; add new custom rule for pod security context as for the demo.

* Add e2e tests for customRule

This commit adds tests for CustomRule functionality, including tests for tailored profiles, validation of CEL expressions, and handling of multiple inputs.

* Revert image registry cleanup in Makefile

* Remove unused DeriveResourcePath function and clean up imports in common.go

* Move end-to-end tests for CustomRule functionality to Parallel

* Refactor CustomRule validation tests to improve error handling

This commit updates the error handling in the CustomRule validation tests, it was asserting the wrong result

* Update CustomRule and TailoredProfile descriptions for clarity

This commit refines the descriptions in the CustomRule and TailoredProfile CRDs to enhance clarity, specifically changing references from "XCCDF ID" to "ID of the Rule" and improving the wording of scanner type descriptions. Additionally, it updates the CustomRule validation logic to use the new CustomRulePayload structure, ensuring consistency across the codebase. Unit tests have been adjusted accordingly to reflect these changes.

* Clean Up not used CRD and RBAC config

* Add checkType to CustomRule examples.

* Refactor CustomRule to replace ErrorMessage with FailureReason

* Enhance ScanSettingBinding status handling in controller

This commit introduces logic to transition the ScanSettingBinding status from Invalid to Ready when the associated TailoredProfile is fixed. It updates the status check function to account for this transition and adds corresponding unit tests to verify the behavior.

* Remove unused label from custom security checks profile in pod security configuration

* Update CustomRule descriptions for improved clarity

* Revert the id field of Rule and TailoredProfile to be XCCDF ID

* Enhance descriptions for rule reference in TailoredProfiles and CRDs to improve clarity. Specify the type of rule reference as "Rule" or "CustomRule" and clarify default behavior.

* Update ComplianceScanSpec and CustomRule definitions for improved clarity and validation.

Enhance ScannerType description and add validation enum for ScannerType.
Remove unused Severity column from CustomRule resource definition.

* Rename getCelScannerClient to getRuntimeClient for improved clarity in client creation function.

* Refactor CelScanner configuration to use ApiResourceCacheDir instead of ApiResourcePath for API resources are cached

ApiResourceCacheDir unambiguously refers to a local filesystem directory where API resources are cached

* Update failure reason in CustomRule for pod security context validation

* Enhance error logging and refactor variable handling in CelScanner.

Improved log messages to include additional context for profile and tailored profile errors.

* Change scan result output format from XML to JSON in CelScanner and update serialization error messages for clarity.

* Add error handling for ComplianceCheckResult operations in CelScanner

Implement os.Exit with CelExitCodeError for error scenarios when listing, creating/updating, and deleting ComplianceCheckResults to ensure proper termination on failure.

* Refactor object retrieval logic in CelScanner and introduce utility function for improved error handling

Replace the existing getObjectIfFoundCEL function with a new utility function GetObjectIfFound in the utils package,
which implements retry logic with exponential backoff for retrieving Kubernetes objects. Additionally, correct
spelling errors in method names related to scanner type validation in ComplianceScan.

* Update CustomRuleProfileAnnotation to clarify its purpose as tailored profile containing custom rules

* Refactor CustomRule validation logic to remove redudent checks performed by kubebuilder

* Update CustomRule ID in TestCustomRuleTailoredProfile to just use customRuleName

* Fix TestCustomRuleTailoredProfile e2e test

* Add readiness check for ScanSettingBinding in TestCustomRuleWithMultipleInputs

* Fix spelling error in scanner type validation method in ComplianceScan

* Add TestCustomRuleCascadingStatusUpdate to validate cascading status changes

This test checks the behavior of CustomRule, TailoredProfile, and ScanSettingBinding when a CustomRule is updated with an invalid expression. It verifies that the resources transition to error states and subsequently recover when the expression is fixed. Additionally, it updates the ID fields in existing tests to use consistent naming conventions.

* Refactor TestCustomRuleTailoredProfile to improve pod creation logic and update compliance expectations.

The test now verifies that a pod without the test label is ignored, while a pod with the label is expected to be non-compliant due to the absence of a security context.

* Add validation to prevent mixing CustomRules and regular Rules in TailoredProfile

* Add FailureReason to warnings in ComplianceCheckResult for failed scans

* Adjusted the rule in TestTailoredProfileRejectsMixedRuleTypes to fix error in tailoredProfile states caused by node rule platform mixing.

* Refactor TailoredProfile controller logic and enhance test validations. Updated condition for custom rules check and improved error messaging for unsupported rule types. Enhanced tests to validate CustomRule results, ensuring accurate compliance status reporting.

* Update scan name assignment in CustomRule tests to use TailoredProfile name.

This change clarifies the expected behavior for TailoredProfiles in the tests, ensuring accurate compliance check results.

* Added validation to ensure CustomRules only use 'Platform' as the check type and 'CEL' as the scanner type. Updated descriptions in CRDs for clarity on rule ID and check type restrictions.

* Update error logging in CelScanner to use TailoredProfile name for improved clarity in scan failure messages.

* Refactor error handling in TestCustomRuleCheckTypeAndScannerTypeValidation to improve clarity on invalid scannerType creation. Removed unnecessary status checks and streamlined test logic for better readability.

Author: Vincent056

PROJECT

@@ -1,3 +1,7 @@
+# Code generated by tool. DO NOT EDIT.
+# This file is used to track the info used to scaffold your project
+# and allow the plugins properly work.
+# More info: https://book.kubebuilder.io/reference/project-config.html
 domain: openshift.io
 layout:
 - go.kubebuilder.io/v3
@@ -61,4 +65,12 @@ resources:
   kind: TailoredProfile
   path: github.com/ComplianceAsCode/compliance-operator/api/v1alpha1
   version: v1alpha1
+- api:
+    crdVersion: v1
+    namespaced: true
+  domain: openshift.io
+  group: compliance
+  kind: CustomRule
+  path: github.com/ComplianceAsCode/compliance-operator/api/v1alpha1
+  version: v1alpha1
 version: "3"

bundle/manifests/compliance-operator.clusterserviceversion.yaml

@@ -206,6 +206,12 @@ spec:
       kind: ComplianceSuite
       name: compliancesuites.compliance.openshift.io
       version: v1alpha1
+    - description: CustomRule represents a rule that can be used with TailoredProfiles
+        to execute arbitrary checks against the cluster.
+      displayName: Custom Rule
+      kind: CustomRule
+      name: customrules.compliance.openshift.io
+      version: v1alpha1
     - description: ProfileBundle is the Schema for the profilebundles API
       displayName: Profile Bundle
       kind: ProfileBundle
@@ -1034,16 +1040,47 @@ spec:
           - get
           - list
           - watch
+        - apiGroups:
+          - compliance.openshift.io
+          resources:
+          - compliancescans/finalizers
+          - compliancecheckresults/finalizers
+          verbs:
+          - update
         - apiGroups:
           - compliance.openshift.io
           resources:
           - compliancesuites
+          - compliancescans
           - scansettings
           - scansettingbindings
+          - tailoredprofiles
+          - customrules
+          - variables
+          verbs:
+          - get
+          - list
+          - watch
+        - apiGroups:
+          - hco.kubevirt.io
+          resources:
+          - '*'
           verbs:
           - get
           - list
           - watch
+        - apiGroups:
+          - compliance.openshift.io
+          resources:
+          - compliancecheckresults
+          verbs:
+          - get
+          - list
+          - watch
+          - update
+          - patch
+          - create
+          - delete
         - apiGroups:
           - machineconfiguration.openshift.io
           resources:

bundle/manifests/compliance.openshift.io_compliancescans.yaml

@@ -273,6 +273,14 @@ spec:
                 default: Node
                 description: The type of Compliance scan.
                 type: string
+              scannerType:
+                default: OpenSCAP
+                description: The scanner used to evaluate the rules in a Profile or
+                  TailoredProfile.
+                enum:
+                - OpenSCAP
+                - CEL
+                type: string
               showNotApplicable:
                 default: false
                 description: Determines whether to hide or show results that are not

bundle/manifests/compliance.openshift.io_compliancesuites.yaml

@@ -292,6 +292,14 @@ spec:
                       default: Node
                       description: The type of Compliance scan.
                       type: string
+                    scannerType:
+                      default: OpenSCAP
+                      description: The scanner used to evaluate the rules in a Profile
+                        or TailoredProfile.
+                      enum:
+                      - OpenSCAP
+                      - CEL
+                      type: string
                     showNotApplicable:
                       default: false
                       description: Determines whether to hide or show results that

bundle/manifests/compliance.openshift.io_customrules.yaml

@@ -0,0 +1,204 @@
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: v0.17.1
+  creationTimestamp: null
+  name: customrules.compliance.openshift.io
+spec:
+  group: compliance.openshift.io
+  names:
+    kind: CustomRule
+    listKind: CustomRuleList
+    plural: customrules
+    singular: customrule
+  scope: Namespaced
+  versions:
+  - additionalPrinterColumns:
+    - jsonPath: .status.phase
+      name: Status
+      type: string
+    - jsonPath: .metadata.creationTimestamp
+      name: Age
+      type: date
+    name: v1alpha1
+    schema:
+      openAPIV3Schema:
+        description: CustomRule represents a rule that can be used with TailoredProfiles
+          to execute arbitrary checks against the cluster.
+        properties:
+          apiVersion:
+            description: |-
+              APIVersion defines the versioned schema of this representation of an object.
+              Servers should convert recognized schemas to the latest internal value, and
+              may reject unrecognized values.
+              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
+            type: string
+          kind:
+            description: |-
+              Kind is a string value representing the REST resource this object represents.
+              Servers may infer this from the endpoint the client submits requests to.
+              Cannot be updated.
+              In CamelCase.
+              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
+            type: string
+          metadata:
+            type: object
+          spec:
+            properties:
+              availableFixes:
+                description: The Available fixes
+                items:
+                  description: |-
+                    FixDefinition Specifies a fix or remediation
+                    that applies to a rule
+                  properties:
+                    disruption:
+                      description: |-
+                        An estimate of the potential disruption or operational
+                        degradation that this fix will impose in the target system
+                      type: string
+                    fixObject:
+                      description: an object that should bring the rule into compliance
+                      type: object
+                      x-kubernetes-embedded-resource: true
+                      x-kubernetes-preserve-unknown-fields: true
+                    platform:
+                      description: The platform that the fix applies to
+                      type: string
+                  type: object
+                nullable: true
+                type: array
+                x-kubernetes-list-type: atomic
+              checkType:
+                description: |-
+                  What type of check will this rule execute:
+                  Platform, Node or none (represented by an empty string)
+                  For CustomRules, only Platform is supported.
+                type: string
+              description:
+                description: The description of the Rule
+                type: string
+              expression:
+                description: Expression is the CEL expression to evaluate
+                minLength: 1
+                type: string
+              failureReason:
+                description: FailureReason is displayed when the rule evaluation fails
+                minLength: 1
+                type: string
+              id:
+                description: The ID of the Rule
+                type: string
+              inputs:
+                description: Inputs defines the Kubernetes resources that need to
+                  be fetched before evaluating the expression
+                items:
+                  properties:
+                    kubernetesInputSpec:
+                      description: KubernetesInputSpec is the specification of the
+                        Kubernetes resource to fetch
+                      properties:
+                        apiVersion:
+                          description: APIVersion is the API version (e.g., "v1",
+                            "v1beta1")
+                          minLength: 1
+                          type: string
+                        group:
+                          description: Group is the API group (e.g., "apps", "" for
+                            core resources)
+                          type: string
+                        resource:
+                          description: |-
+                            Resource is the resource type (e.g., "pods", "configmaps")
+                            Use the plural form of the resource
+                          minLength: 1
+                          type: string
+                        resourceName:
+                          description: |-
+                            ResourceName is the specific resource name
+                            Leave empty to fetch all resources of this type
+                          type: string
+                        resourceNamespace:
+                          description: |-
+                            ResourceNamespace is the namespace to search in
+                            Leave empty for cluster-scoped resources or to search all namespaces
+                          type: string
+                      required:
+                      - apiVersion
+                      - resource
+                      type: object
+                    name:
+                      description: Name is the variable name used to reference this
+                        resource in the CEL expression
+                      minLength: 1
+                      type: string
+                  required:
+                  - kubernetesInputSpec
+                  - name
+                  type: object
+                minItems: 1
+                type: array
+              instructions:
+                description: Instructions for auditing this specific rule
+                type: string
+              rationale:
+                description: The rationale of the Rule
+                type: string
+              scannerType:
+                description: ScannerType denotes the scanning implementation to use
+                  when evaluating rules
+                enum:
+                - CEL
+                type: string
+              severity:
+                description: The severity level
+                type: string
+              title:
+                description: The title of the Rule
+                type: string
+              warning:
+                description: A discretionary warning about the of the Rule
+                type: string
+            required:
+            - expression
+            - failureReason
+            - id
+            - inputs
+            - scannerType
+            - title
+            type: object
+          status:
+            description: Status contains the validation status and other runtime information
+            properties:
+              errorMessage:
+                description: ErrorMessage contains any validation error message
+                type: string
+              lastValidationTime:
+                description: LastValidationTime is the timestamp of the last validation
+                format: date-time
+                type: string
+              observedGeneration:
+                description: ObservedGeneration represents the .metadata.generation
+                  that the status was set based upon
+                format: int64
+                type: integer
+              phase:
+                description: Phase describes the current phase of the CustomRule (Ready
+                  or Error)
+                enum:
+                - Ready
+                - Error
+                type: string
+            type: object
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}
+status:
+  acceptedNames:
+    kind: ""
+    plural: ""
+  conditions: null
+  storedVersions: null

bundle/manifests/compliance.openshift.io_rules.yaml

@@ -54,12 +54,13 @@ spec:
             description: |-
               What type of check will this rule execute:
               Platform, Node or none (represented by an empty string)
+              For CustomRules, only Platform is supported.
             type: string
           description:
             description: The description of the Rule
             type: string
           id:
-            description: The XCCDF ID
+            description: The ID of the Rule
             type: string
           instructions:
             description: Instructions for auditing this specific rule

bundle/manifests/compliance.openshift.io_tailoredprofiles.yaml

@@ -57,6 +57,12 @@ spec:
                   description: RuleReferenceSpec specifies a rule to be selected/deselected,
                     as well as the reason why
                   properties:
+                    kind:
+                      description: |-
+                        Specifies the type of rule reference: either "Rule" or "CustomRule".
+                        If not set, "Rule" is used by default. The "CustomRule" type is only applicable
+                        when referencing CustomRule resources in TailoredProfiles via enableRules.
+                      type: string
                     name:
                       description: Name of the rule that's being referenced
                       type: string
@@ -75,6 +81,12 @@ spec:
                   description: RuleReferenceSpec specifies a rule to be selected/deselected,
                     as well as the reason why
                   properties:
+                    kind:
+                      description: |-
+                        Specifies the type of rule reference: either "Rule" or "CustomRule".
+                        If not set, "Rule" is used by default. The "CustomRule" type is only applicable
+                        when referencing CustomRule resources in TailoredProfiles via enableRules.
+                      type: string
                     name:
                       description: Name of the rule that's being referenced
                       type: string
@@ -97,6 +109,12 @@ spec:
                   description: RuleReferenceSpec specifies a rule to be selected/deselected,
                     as well as the reason why
                   properties:
+                    kind:
+                      description: |-
+                        Specifies the type of rule reference: either "Rule" or "CustomRule".
+                        If not set, "Rule" is used by default. The "CustomRule" type is only applicable
+                        when referencing CustomRule resources in TailoredProfiles via enableRules.
+                      type: string
                     name:
                       description: Name of the rule that's being referenced
                       type: string