Config Structural Validation Schema Part 4: Monitoring

[Zash]

Warning

This is a public repository, ensure not to disclose:

• personal data beyond what is necessary for interacting with this pull request, nor
• business confidential information, such as customer names.

What kind of PR is this?

Required: Mark one of the following that is applicable:

• kind/feature
• kind/improvement
• kind/deprecation
• kind/documentation
• kind/clean-up
• kind/bug
• kind/other

Optional: Mark one or more of the following that are applicable:

Important

Breaking changes should be marked kind/admin-change or kind/dev-change depending on type
Critical security fixes should be marked with kind/security

• kind/admin-change
• kind/dev-change
• kind/security
• kind/adr

What does this PR do / why do we need this PR?

This adds structural validation for the following top-level config sections:

• alerts
• grafana
• grafanaLabelEnforcer
• kubeStateMetrics
• metricsServer
• openstackMonitoring
• prometheus
• prometheusBlackboxExporter
• prometheusNodeExporter
• prometheusOperator
• s3Exporter
• thanos
• wcProbeIngress
• welcomingDashboard
• fluentd
• opensearch
• elasticsearch

For more context including the required scripts and see #1862
Part of issue #1427

Information to reviewers

ck8s validate should now perform validation of the above sections.

While the CI is failing:
Please help determine whether it is the schema that is lacking or whether it detected a config mistake.

Suggestions and improvements very welcome!

Also see the README for brief information about the schema format

Obs: The $defs are really added in #2063 but the commit is included here too for CI.

Checklist

• Proper commit message prefix on all commits
• Change checks:
  • The change is transparent
  • The change is disruptive
  • The change requires no migration steps
  • The change requires migration steps
  • The change upgrades CRDs
  • The change updates the config and the schema
• Metrics checks:
  • The metrics are still exposed and present in Grafana after the change
  • The metrics names didn't change (Grafana dashboards and Prometheus alerts are not affected)
  • The metrics names did change (Grafana dashboards and Prometheus alerts were fixed)
• Logs checks:
  • The logs do not show any errors after the change
• Pod Security Policy checks:
  • Any changed pod is covered by Pod Security Admission
  • Any changed pod is covered by Gatekeeper Pod Security Policies
  • The change does not cause any pods to be blocked by Pod Security Admission or Policies
• Network Policy checks:
  • Any changed pod is covered by Network Policies
  • The change does not cause any dropped packets in the NetworkPolicy Dashboard
• Audit checks:
  • The change does not cause any unnecessary Kubernetes audit events
  • The change requires changes to Kubernetes audit policy
• Falco checks:
  • The change does not cause any alerts to be generated by Falco
• Bug checks:
  • The bug fix is covered by regression tests

[lunkan93]

This for example seems like a config mistake and should be {} in the defaults
https://github.com/elastisys/compliantkubernetes-apps/blob/main/config/config/common-config.yaml#L265
https://github.com/elastisys/compliantkubernetes-apps/blob/main/config/config/common-config.yaml#L290

[anders-elastisys]

Seems like there is a missing schema for extraConfigMaps for wc:

✗ ./bin/ck8s validate wc
[ck8s] Validating wc config
...
[ck8s] Failed schema validation:
wc-config.yaml: fluentd: Additional property extraConfigMaps is not allowed

[Zash]

Seems like there is a missing schema for extraConfigMaps

What is the format of extraConfigMaps? Map of string filenames to string content?

[lunkan93]

Seems like there is a missing schema for extraConfigMaps

What is the format of extraConfigMaps? Map of string filenames to string content?

Yeah should be the same as this for SC https://github.com/elastisys/compliantkubernetes-apps/blob/main/helmfile.d/values/fluentd/forwarder-service-cluster.yaml.gotmpl#L6

[Zash]

I will rebase this once #2063 has been merged.