Incorrect rules are skipped without mention, giving the impression all is well

[fliphess]

Hey :) Thanks for writing this utility.

I have some questions regarding missing keys or checks that are suppose to fail but dont.

I have the following yaml:

apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: some-chart
  namespace: default
  annotations:
    fluxcd.io/automated: false

When I on purpose remove a few chars so that the assertion is supposed to fail:

- id: CHECK_HELMRELEASE
  severity: FAILURE
  message: Check validity for helmrelease
  resource: "HelmRelease"
  assertions:
  - key: metadata
    op: has-properties
    value: name,namespace,annotat

.... The check still succeeds.

Is this intended behaviour? It looks like a regex is used or a string in string comparison where the full key should be checked instead.

[fliphess]

Apparently I have the same behaviour with several other operators (not-empty, present, in) that return OKAY when there are purposefully created bugs that should fail the check.

Using -verbose shows me the check on itself is faulty and as a result completely ignored, which is not shown when using -validate.

Would it be possible to change this behaviour into a hard fail?

I think it's much safer to bail out when a rule is incorrect rather than skipping it without mention which implies all is well, like when the check succeeds.