RFC: Alert CR support for wildcard namespace

[dm3ch]

/relates fluxcd/notification-controller#504
/relates fluxcd/notification-controller#71
/relates #4095

RFC covering support of monitoring all namespaces by Flux Alert.

[dm3ch]

@matheuscscp You proposed to bring this feature discussions as a RFC.
Since I'm new to this RFC process and didn't understand what the sponsor means in the readme, I'm kindly asking you to assist or at least consult me how to start the discussion process properly.

[matheuscscp]

This RFC will hardly get through. I don't see anybody in the maintainers team interested in evaluating or considering the possibility of introducing a mechanism that breaks Kubernetes namespace isolation in such a... simplistic, or niche way. The Flux project has been historically trying to move away from this (a.k.a. --no-cross-namespace-refs), and I will not agree with introducing such a security boundary breakage the way it's proposed here. The --no-cross-namespace-refs flag came later to fix the unfortunate introduction of cross-namespace references in Flux. In my opinion, it should not be used as a standard for introducing more cross-namespace reference features. In hindsight, cross-namespace reference features would never exist in Flux.

If there's a big desire from the Flux community to have proper support for cross-namespace references, I'd lean more towards the ReferenceGrant API. Looking at the KEPs it didn't seem like they were utterly rejected, but rather simply deprioritized. I wouldn't be surprised if they eventually converged to something better, given that ReferenceGrant is looking really bad under the API group gateway.networking.k8s.io, and other SIGs have already started using it. I'm pretty sure that if somebody showed up willing to push and contribute their time in this standardization they would probably appreciate the help. In an ideal world, Flux should integrate with a consolidated ReferenceGrant API (and not build one under something.toolkit.fluxcd.io).

ReferenceGrant will, however, take a long time.

A specific point brought up in this similar RFC that I would be willing to consider is:

https://github.com/rsafonseca/flux2/blob/090a3e6c616eac5986af878d365d6699419f89a7/rfcs/0005-global-objects/README.md?plain=1#L27-L29

It seems reasonable to me that cluster administrators may not want to share notification provider secrets with tenants. For this I would be willing to consider a controller-level flag that allows specifying a fallback secret in the same namespace of the controller pod (like we're gonna do here). Would this be useful?

[zparnold]

FWIW, I don't represent the whole flux community but as someone who administers several clusters of geo-distributed teams (that are not multi-tenant, but multi-ns), I'm not usually aware of what new Kustomizations/HelmReleases are going in while I'm asleep but I am on the hook if something goes wrong. Also, HelmReleases I thought could create new NS'es which means that it isn't always immediately obvious that one would need another new Alert object for that NS if we wanted to track HelmRelease changes. The --no-cross-namespace-refs flag I think is sufficient to also cover this use-case, don't you think? Meaning, from your argument, this flag was regrettably introduced to ensure that multi-tenant users couldn't accidentally violate NS boundaries in flux. That still holds here with a wildcard, right? Even if cross-ns access wasn't a design goal (or shouldn't have been as you mentioned) of flux, the reality of a cluster operator (me) is that we are often keeping track of many different namespaces and not having to follow behind teams to setup new alert objects would be nice.

Could I write a webhook that deploys a new Alert object on each namespace creation? Sure. Should I have to? I would argue, no. It was a little surprising as a flux user to find this still open after so long. But again. I'm one person not a representative sample of the community.