feat(llmisvc): add manager options hook for distribution-specific cache configuration#1192
Conversation
…he configuration Extracts ctrl.Options into a variable and introduces a customizeManagerOptions hook using the build-tag companion file pattern. The distro implementation replaces the simple label-based Secret cache with a namespace-aware one that also watches the platform CA signing secret needed for workload TLS certificates. Signed-off-by: Bartosz Majsak <bartosz.majsak@gmail.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bartoszmajsak The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
📝 WalkthroughWalkthroughThe changes introduce a customization hook pattern for controller-runtime manager initialization. The main.go file is refactored to create manager options in a dedicated configuration object, then calls a customizeManagerOptions hook before instantiation. Two implementations are provided via build tags: a no-op default (!distro) and a distro-specific variant (distro) that configures namespace-aware caching for Secret objects to watch a platform CA signing secret alongside standard label-filtered Secrets. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Key areas requiring attention:
🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@cmd/llmisvc/main.go`:
- Around line 176-177: Change customizeManagerOptions to return an error and
propagate failures at the callsite: update the signatures in
manager_options_ocp.go and manager_options_default.go to func
customizeManagerOptions(*ctrl.Options) error, have those implementations return
any bootstrap/cache-install errors (e.g., "Secret entry not found"), and in
cmd/llmisvc/main.go call err := customizeManagerOptions(&mgrOpts) and if err !=
nil log the error and exit before calling ctrl.NewManager so distro-specific
cache override failures stop startup instead of causing latent runtime faults.
In `@cmd/llmisvc/manager_options_ocp.go`:
- Around line 35-46: The current code replaces opts.Cache.ByObject[obj]
wholesale and wipes any upstream cache knobs; instead, read the existing
cache.ByObject (bo := opts.Cache.ByObject[obj]), copy or initialize
bo.Namespaces (preserve other bo fields like Label, Field, Transform,
UnsafeDisableDeepCopy, EnableWatchBookmarks), shallow-copy the Namespaces map if
non-nil or create a new map, then set/override the two keys
llmisvc.ServiceCASigningSecretNamespace (with FieldSelector for metadata.name ==
llmisvc.ServiceCASigningSecretName) and cache.AllNamespaces (with LabelSelector
cfg.Label), assign the new Namespaces back to bo and finally store bo into
opts.Cache.ByObject[obj] so only Namespaces is modified and all other settings
remain intact.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Central YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: 942a7f41-b039-446b-a8af-03d59d75eb0d
📒 Files selected for processing (3)
cmd/llmisvc/main.gocmd/llmisvc/manager_options_default.gocmd/llmisvc/manager_options_ocp.go
| customizeManagerOptions(&mgrOpts) | ||
| mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), mgrOpts) |
There was a problem hiding this comment.
Propagate hook failures before starting the manager.
cmd/llmisvc/manager_options_ocp.go already has a known failure path (Secret entry not found), but this call site gives it no way to stop startup. In distro builds that turns a cache-setup error into a latent runtime failure when pkg/controller/v1alpha2/llmisvc/workload_tls_cert_ocp.go:108-114 needs the CA signing secret. Make customizeManagerOptions return error and exit here if the distro-specific cache override cannot be installed. As per coding guidelines, review priorities include bug-prone patterns and error handling gaps.
Suggested change
- customizeManagerOptions(&mgrOpts)
+ if err := customizeManagerOptions(&mgrOpts); err != nil {
+ setupLog.Error(err, "unable to customize manager options")
+ os.Exit(1)
+ }
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), mgrOpts)Also mirror the signature change in cmd/llmisvc/manager_options_default.go and cmd/llmisvc/manager_options_ocp.go:
func customizeManagerOptions(*ctrl.Options) error🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@cmd/llmisvc/main.go` around lines 176 - 177, Change customizeManagerOptions
to return an error and propagate failures at the callsite: update the signatures
in manager_options_ocp.go and manager_options_default.go to func
customizeManagerOptions(*ctrl.Options) error, have those implementations return
any bootstrap/cache-install errors (e.g., "Secret entry not found"), and in
cmd/llmisvc/main.go call err := customizeManagerOptions(&mgrOpts) and if err !=
nil log the error and exit before calling ctrl.NewManager so distro-specific
cache override failures stop startup instead of causing latent runtime faults.
| opts.Cache.ByObject[obj] = cache.ByObject{ | ||
| Namespaces: map[string]cache.Config{ | ||
| llmisvc.ServiceCASigningSecretNamespace: { | ||
| FieldSelector: fields.SelectorFromSet(map[string]string{ | ||
| "metadata.name": llmisvc.ServiceCASigningSecretName, | ||
| }), | ||
| }, | ||
| cache.AllNamespaces: { | ||
| LabelSelector: cfg.Label, | ||
| }, | ||
| }, | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
In sigs.k8s.io/controller-runtime v0.21.0, what fields exist on cache.ByObject and cache.Config, and what is the correct way to preserve an existing ByObject cache configuration when adding per-namespace settings?
💡 Result:
Fields in sigs.k8s.io/controller-runtime/pkg/cache v0.21.0
cache.ByObject fields: Namespaces map[string]cache.Config, Label labels.Selector, Field fields.Selector, Transform toolscache.TransformFunc, UnsafeDisableDeepCopy *bool, EnableWatchBookmarks *bool. [1]
cache.Config fields: LabelSelector labels.Selector, FieldSelector fields.Selector, Transform toolscache.TransformFunc, UnsafeDisableDeepCopy *bool, EnableWatchBookmarks *bool. [1]
Preserving existing ByObject config when adding per-namespace settings
Don’t overwrite the whole ByObject entry (or its Namespaces map). Instead, read the existing ByObject, copy/initialize its Namespaces map, modify it, then write it back—so you keep the existing selectors/transform/deep-copy/bookmark settings. Also note the semantics: ByObject.Namespaces == nil means “default to cache.Options.DefaultNamespaces”; an empty map means “cache all namespaces”. [1]
obj := &corev1.Secret{} // example key; must be the same type you configured earlier
bo := opts.ByObject[obj] // existing ByObject (zero value if absent)
// IMPORTANT: setting Namespaces from nil -> non-nil changes semantics,
// but that's required if you want explicit per-namespace settings.
ns := bo.Namespaces
if ns == nil {
ns = map[string]cache.Config{}
} else {
// shallow copy so we don't mutate a shared map
cp := make(map[string]cache.Config, len(ns)+1)
for k, v := range ns { cp[k] = v }
ns = cp
}
ns["my-namespace"] = cache.Config{
// set only what you want to override for that namespace
// (unset fields will be defaulted per the precedence described in ByObject.Namespaces docs)
}
bo.Namespaces = ns
opts.ByObject[obj] = boIf you instead do opts.ByObject[obj] = cache.ByObject{Namespaces: ...}, you’ll drop any previously-set Label/Field/Transform/... on that object’s cache configuration. [1]
Sources: [1] (pkg.go.dev)
Citations:
Preserve the existing Secret cache configuration when adding per-namespace settings.
This code rebuilds the entire cache.ByObject entry from scratch, discarding any upstream-configured Label, Field, Transform, UnsafeDisableDeepCopy, or EnableWatchBookmarks settings. If cmd/llmisvc/main.go later adds any of these cache knobs, distro builds will silently drop them—reintroducing the upstream-drift problem this hook prevents.
Instead, read the existing cache.ByObject for the Secret, preserve all its fields, and only modify or extend the Namespaces map:
Correct pattern
bo := opts.Cache.ByObject[obj] // existing ByObject (zero value if absent)
ns := bo.Namespaces
if ns == nil {
ns = map[string]cache.Config{}
} else {
cp := make(map[string]cache.Config, len(ns)+1)
for k, v := range ns { cp[k] = v }
ns = cp
}
ns["llmisvc.ServiceCASigningSecretNamespace"] = cache.Config{
FieldSelector: fields.SelectorFromSet(map[string]string{
"metadata.name": llmisvc.ServiceCASigningSecretName,
}),
}
ns[cache.AllNamespaces] = cache.Config{
LabelSelector: cfg.Label,
}
bo.Namespaces = ns
opts.Cache.ByObject[obj] = bo🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@cmd/llmisvc/manager_options_ocp.go` around lines 35 - 46, The current code
replaces opts.Cache.ByObject[obj] wholesale and wipes any upstream cache knobs;
instead, read the existing cache.ByObject (bo := opts.Cache.ByObject[obj]), copy
or initialize bo.Namespaces (preserve other bo fields like Label, Field,
Transform, UnsafeDisableDeepCopy, EnableWatchBookmarks), shallow-copy the
Namespaces map if non-nil or create a new map, then set/override the two keys
llmisvc.ServiceCASigningSecretNamespace (with FieldSelector for metadata.name ==
llmisvc.ServiceCASigningSecretName) and cache.AllNamespaces (with LabelSelector
cfg.Label), assign the new Namespaces back to bo and finally store bo into
opts.Cache.ByObject[obj] so only Namespaces is modified and all other settings
remain intact.
|
/retest-required |
1 similar comment
|
/retest-required |
|
/retest |
|
@bartoszmajsak: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
2 participants
What this PR does / why we need it:
Adds a
customizeManagerOptionsbuild-tag hook so the distro build can extend the controller manager's cache configuration without modifyingmain.godirectly.The distro implementation replaces the simple label-based Secret cache with a namespace-aware one that also watches the platform CA signing secret (
openshift-service-ca/signing-key) needed for workload TLS certificates. Without this, cert rotation requires inline changes tomain.gothat conflict on every upstream sync.Follows the established
_default.go/_ocp.gocompanion file pattern.Feature/Issue validation/testing:
go vet ./cmd/llmisvc/...passes (both default anddistrotags)ServiceCASigningSecretName/ServiceCASigningSecretNamespaceconstants resolve fromworkload_tls_cert_ocp.goSpecial notes for your reviewer:
Upstream hook landed in kserve/kserve (pending merge). This PR adds both the upstream infrastructure (
main.gorefactor +_default.go) and the distro implementation (_ocp.go).Checklist:
Summary by CodeRabbit