Why having gotk-sync.yaml changed is a bad idea?

[nikp123]

Ok, this may sound dumb at first, but I do have a reason for this.

Following the documentation for the sops-based age secrets management, you're supposed to have your secrets in a separate directory where a custom defined flux Kustomization picks it up and those get decrypted on the fly and applied to the cluster.

The setup I want is to have a single sops key for my entire cluster rather than setting up per-app sops keys.

Considering that, I find this setup quite limiting, because I'd like to have my secrets be a part of my apps definition.
The idea is to have a directory structure akin to this:

.
├── clusters
│   └── home
│       ├── apps
│       │   ├── actualfinance
│       │   │   ├── helmrelease.yaml
│       │   │   ├── ingressroute.yaml
│       │   │   ├── kustomization.yaml
│       │   │   ├── pvc.yaml
│       │   │   └── secrets.yaml
│       │   └── ....
│       ├── kustomization.yaml
│       └── ...
├── flux-system
│   ├── gotk-components.yaml
│   ├── gotk-sync.yaml
│   ├── kustomization.yaml
│   └── repos
│       └── ....

This way whenever I need to disable an app I just comment it out of the parent Kustomization managing my apps. This also removes the secret so that I don't have to manually manage them separately.

The documented way doesn't let me have this because both the flux-system kustomization provided in gotk-sync.yaml and whatever I make will start fighting over secrets and values because they both point to the same directory, thus overriding eachothers' values.

The gotk-sync.yaml file explicitly says "DO NOT MODIFY" in the header, however I don't find any convenient way of just including my sops key into it without modification.

Is it fine to edit gotk-sync here if all I need to add is this:

spec:
  decryption:
    provider: sops
    secretRef:
      name: sops-age

[matheuscscp]

It's a bad idea because it's an auto-generated file. If you use flux bootstrap to bootstrap and upgrade clusters, next time it will undo all your changes. You should add patches to kustomization.yaml instead.

Even better, you should stop using gotk-sync.yaml and gotk-components.yaml altogether and start using Flux Operator's FluxInstance:

https://fluxoperator.dev/docs/guides/migration/

This way you don't have a tool committing files to your repo at all.

[nikp123]

So migrating to that will just have GitHub's CI push OCI images and that gets picked up by my cluster instead?

How does it handle secrets though, that seems a bit unclear? Sorry, I am very new to this.

[matheuscscp]

https://fluxoperator.dev/docs/instance/customization/#cluster-sync-sops-decryption

[nikp123]

Thank you, that's just what I needed.