Skip to content

Commit 4cdaa89

Browse files
committed
Rabbitmq vhost and user support
Add new messagingBus and notificationsBus interfaces to hold cluster, user and vhost names for optional usage. The controller adds these values to the TransportURL create request when present. Additionally, we migrate RabbitMQ cluster name to RabbitMq config struct using DefaultRabbitMqConfig from infra-operator to automatically populate the new Cluster field from legacy RabbitMqClusterName. Example usage: spec: messagingBus: cluster: rpc-rabbitmq user: rpc-user vhost: rpc-vhost notificationsBus: cluster: notifications-rabbitmq user: notifications-user vhost: notifications-vhost Finally, we add the rabbitmquser crs to the secret so they can be stored for dataplane finalizers management and do auto cleanup of orphaned users after credential rotations. Jira: https://issues.redhat.com/browse/OSPRH-22697
1 parent 6dba625 commit 4cdaa89

File tree

18 files changed

+496
-56
lines changed

18 files changed

+496
-56
lines changed

api/bases/nova.openstack.org_nova.yaml

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -540,6 +540,23 @@ spec:
540540
MemcachedInstance is the name of the Memcached CR that the services in the cell will use.
541541
If defined then this takes precedence over Nova.Spec.MemcachedInstance for this cel
542542
type: string
543+
messagingBus:
544+
description: MessagingBus configuration (username, vhost, and
545+
cluster)
546+
properties:
547+
cluster:
548+
description: Name of the cluster
549+
minLength: 1
550+
type: string
551+
user:
552+
description: User - RabbitMQ username
553+
type: string
554+
vhost:
555+
description: Vhost - RabbitMQ vhost name
556+
type: string
557+
required:
558+
- cluster
559+
type: object
543560
metadataServiceTemplate:
544561
description: |-
545562
MetadataServiceTemplate - defines the metadata service dedicated for the
@@ -1340,6 +1357,22 @@ spec:
13401357
description: MemcachedInstance is the name of the Memcached CR that
13411358
all nova service will use.
13421359
type: string
1360+
messagingBus:
1361+
description: MessagingBus configuration (username, vhost, and cluster)
1362+
properties:
1363+
cluster:
1364+
description: Name of the cluster
1365+
minLength: 1
1366+
type: string
1367+
user:
1368+
description: User - RabbitMQ username
1369+
type: string
1370+
vhost:
1371+
description: Vhost - RabbitMQ vhost name
1372+
type: string
1373+
required:
1374+
- cluster
1375+
type: object
13431376
metadataContainerImageURL:
13441377
description: MetadataContainerImageURL
13451378
type: string
@@ -1648,6 +1681,23 @@ spec:
16481681
NodeSelector here acts as a default value and can be overridden by service
16491682
specific NodeSelector Settings.
16501683
type: object
1684+
notificationsBus:
1685+
description: NotificationsBus configuration (username, vhost, and
1686+
cluster) for notifications
1687+
properties:
1688+
cluster:
1689+
description: Name of the cluster
1690+
minLength: 1
1691+
type: string
1692+
user:
1693+
description: User - RabbitMQ username
1694+
type: string
1695+
vhost:
1696+
description: Vhost - RabbitMQ vhost name
1697+
type: string
1698+
required:
1699+
- cluster
1700+
type: object
16511701
notificationsBusInstance:
16521702
description: |-
16531703
NotificationsBusInstance is the name of the RabbitMqCluster CR to select

api/go.mod

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ require (
1818
github.com/cespare/xxhash/v2 v2.3.0 // indirect
1919
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
2020
github.com/emicklei/go-restful/v3 v3.12.2 // indirect
21-
github.com/evanphx/json-patch v5.9.11+incompatible // indirect
2221
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
2322
github.com/fsnotify/fsnotify v1.9.0 // indirect
2423
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
@@ -45,6 +44,7 @@ require (
4544
github.com/prometheus/client_model v0.6.2 // indirect
4645
github.com/prometheus/common v0.65.0 // indirect
4746
github.com/prometheus/procfs v0.16.1 // indirect
47+
github.com/rabbitmq/cluster-operator/v2 v2.16.0 // indirect
4848
github.com/spf13/pflag v1.0.7 // indirect
4949
github.com/x448/float16 v0.8.4 // indirect
5050
go.yaml.in/yaml/v2 v2.4.2 // indirect
@@ -70,6 +70,9 @@ require (
7070
sigs.k8s.io/yaml v1.6.0 // indirect
7171
)
7272

73+
// Use lmiccini's infra-operator branch with status.RabbitmqUserRef support
74+
replace github.com/openstack-k8s-operators/infra-operator/apis => github.com/lmiccini/infra-operator/apis v0.0.0-20260116071214-b5e11cdcbab3
75+
7376
// mschuppert: map to latest commit from release-4.16 tag
7477
// must consistent within modules and service operators
7578
replace github.com/openshift/api => github.com/openshift/api v0.0.0-20250711200046-c86d80652a9e //allow-merging

api/go.sum

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
12
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
23
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
34
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
@@ -64,6 +65,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
6465
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
6566
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
6667
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
68+
github.com/lmiccini/infra-operator/apis v0.0.0-20260116071214-b5e11cdcbab3 h1:S1WlXTdXw0PA0yZT1acQ1Jwxf/zLGdF2R9DAxbtpVRw=
69+
github.com/lmiccini/infra-operator/apis v0.0.0-20260116071214-b5e11cdcbab3/go.mod h1:ZXwFlspJCdZEUjMbmaf61t5AMB4u2vMyAMMoe/vJroE=
6770
github.com/mailru/easyjson v0.9.0 h1:PrnmzHw7262yW8sTBwxi1PdJA3Iw/EKBa8psRf7d9a4=
6871
github.com/mailru/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
6972
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -78,10 +81,10 @@ github.com/onsi/ginkgo/v2 v2.27.4 h1:fcEcQW/A++6aZAZQNUmNjvA9PSOzefMJBerHJ4t8v8Y
7881
github.com/onsi/ginkgo/v2 v2.27.4/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
7982
github.com/onsi/gomega v1.38.3 h1:eTX+W6dobAYfFeGC2PV6RwXRu/MyT+cQguijutvkpSM=
8083
github.com/onsi/gomega v1.38.3/go.mod h1:ZCU1pkQcXDO5Sl9/VVEGlDyp+zm0m1cmeG5TOzLgdh4=
81-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260108154501-11e5091cddf1 h1:zAbZVtpldi1TU/CO9aU2ZByzcsi+N3aIv6snpSjBVLY=
82-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260108154501-11e5091cddf1/go.mod h1:ZXwFlspJCdZEUjMbmaf61t5AMB4u2vMyAMMoe/vJroE=
8384
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20251230215914-6ba873b49a35 h1:pF3mJ3nwq6r4qwom+rEWZNquZpcQW/iftHlJ1KPIDsk=
8485
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20251230215914-6ba873b49a35/go.mod h1:kycZyoe7OZdW1HUghr2nI3N7wSJtNahXf6b/ypD14f4=
86+
github.com/openstack-k8s-operators/rabbitmq-cluster-operator/v2 v2.6.1-0.20250929174222-a0d328fa4dec h1:saovr368HPAKHN0aRPh8h8n9s9dn3d8Frmfua0UYRlc=
87+
github.com/openstack-k8s-operators/rabbitmq-cluster-operator/v2 v2.6.1-0.20250929174222-a0d328fa4dec/go.mod h1:Nh2NEePLjovUQof2krTAg4JaAoLacqtPTZQXK6izNfg=
8588
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
8689
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
8790
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=

api/v1beta1/nova_types.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ limitations under the License.
1717
package v1beta1
1818

1919
import (
20+
rabbitmqv1 "github.com/openstack-k8s-operators/infra-operator/apis/rabbitmq/v1beta1"
2021
topologyv1 "github.com/openstack-k8s-operators/infra-operator/apis/topology/v1beta1"
2122
condition "github.com/openstack-k8s-operators/lib-common/modules/common/condition"
2223
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -49,6 +50,10 @@ type NovaSpecCore struct {
4950
// communicate.
5051
APIMessageBusInstance string `json:"apiMessageBusInstance"`
5152

53+
// +kubebuilder:validation:Optional
54+
// MessagingBus configuration (username, vhost, and cluster)
55+
MessagingBus rabbitmqv1.RabbitMqConfig `json:"messagingBus,omitempty"`
56+
5257
// +kubebuilder:validation:Optional
5358
// +kubebuilder:default={cell0: {cellDatabaseAccount: nova-cell0, hasAPIAccess: true}, cell1: {cellDatabaseAccount: nova-cell1, cellDatabaseInstance: openstack-cell1, cellMessageBusInstance: rabbitmq-cell1, hasAPIAccess: true}}
5459
// Cells is a mapping of cell names to NovaCellTemplate objects defining
@@ -131,6 +136,10 @@ type NovaSpecCore struct {
131136
// Avoid colocating it with RabbitMqClusterName, APIMessageBusInstance or CellMessageBusInstance used for RPC.
132137
// For particular Nova cells, notifications cannot be disabled, nor configured differently.
133138
NotificationsBusInstance *string `json:"notificationsBusInstance,omitempty"`
139+
140+
// +kubebuilder:validation:Optional
141+
// NotificationsBus configuration (username, vhost, and cluster) for notifications
142+
NotificationsBus *rabbitmqv1.RabbitMqConfig `json:"notificationsBus,omitempty"`
134143
}
135144

136145
// NovaSpec defines the desired state of Nova

api/v1beta1/nova_webhook.go

Lines changed: 116 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ import (
2626
"fmt"
2727

2828
"github.com/google/go-cmp/cmp"
29+
rabbitmqv1 "github.com/openstack-k8s-operators/infra-operator/apis/rabbitmq/v1beta1"
2930
service "github.com/openstack-k8s-operators/lib-common/modules/common/service"
3031
"github.com/robfig/cron/v3"
3132

@@ -88,6 +89,24 @@ func (spec *NovaSpecCore) Default() {
8889
spec.APITimeout = novaDefaults.APITimeout
8990
}
9091

92+
// Default MessagingBus.Cluster from APIMessageBusInstance if not already set
93+
if spec.MessagingBus.Cluster == "" {
94+
spec.MessagingBus.Cluster = spec.APIMessageBusInstance
95+
}
96+
97+
// Default NotificationsBus if NotificationsBusInstance is specified
98+
if spec.NotificationsBusInstance != nil && *spec.NotificationsBusInstance != "" {
99+
if spec.NotificationsBus == nil {
100+
// Initialize empty NotificationsBus - credentials will be created dynamically
101+
// to ensure separation from MessagingBus (RPC and notifications should never share credentials)
102+
spec.NotificationsBus = &rabbitmqv1.RabbitMqConfig{}
103+
}
104+
// Default cluster name if not already set
105+
if spec.NotificationsBus.Cluster == "" {
106+
spec.NotificationsBus.Cluster = *spec.NotificationsBusInstance
107+
}
108+
}
109+
91110
for cellName, cellTemplate := range spec.CellTemplates {
92111

93112
if cellTemplate.MetadataServiceTemplate.Enabled == nil {
@@ -106,6 +125,11 @@ func (spec *NovaSpecCore) Default() {
106125
}
107126
}
108127

128+
// Default MessagingBus.Cluster from CellMessageBusInstance if not already set
129+
if cellTemplate.MessagingBus.Cluster == "" {
130+
cellTemplate.MessagingBus.Cluster = cellTemplate.CellMessageBusInstance
131+
}
132+
109133
// "cellTemplate" is a by-value copy, so we need to re-inject the updated version of it into the map
110134
spec.CellTemplates[cellName] = cellTemplate
111135
}
@@ -138,18 +162,34 @@ func (spec *NovaSpecCore) ValidateCellTemplates(basePath *field.Path, namespace
138162
cell.TopologyRef, *basePath.Child("topologyRef"), namespace)...)
139163

140164
if name != Cell0Name {
141-
if dupName, ok := cellMessageBusNames[cell.CellMessageBusInstance]; ok {
165+
// Determine which rabbit cluster this cell is using
166+
// Prefer the new MessagingBus.Cluster field, fall back to deprecated CellMessageBusInstance
167+
var cellCluster string
168+
if cell.MessagingBus.Cluster != "" {
169+
cellCluster = cell.MessagingBus.Cluster
170+
} else {
171+
cellCluster = cell.CellMessageBusInstance
172+
}
173+
174+
// Check if this rabbit cluster is already used by another cell
175+
if dupName, ok := cellMessageBusNames[cellCluster]; ok {
176+
// Determine which field to report the error on
177+
fieldPath := cellPath.Child("messagingBus").Child("cluster")
178+
if cell.MessagingBus.Cluster == "" {
179+
fieldPath = cellPath.Child("cellMessageBusInstance")
180+
}
181+
142182
errors = append(errors, field.Invalid(
143-
cellPath.Child("cellMessageBusInstance"),
144-
cell.CellMessageBusInstance,
183+
fieldPath,
184+
cellCluster,
145185
fmt.Sprintf(
146186
"RabbitMqCluster CR need to be uniq per cell. It's duplicated with cell: %s",
147187
dupName),
148188
),
149189
)
150190
}
151191

152-
cellMessageBusNames[cell.CellMessageBusInstance] = name
192+
cellMessageBusNames[cellCluster] = name
153193
}
154194
if *cell.MetadataServiceTemplate.Enabled && *spec.MetadataServiceTemplate.Enabled {
155195
errors = append(
@@ -315,7 +355,78 @@ func (spec *NovaSpec) ValidateUpdate(old NovaSpec, basePath *field.Path, namespa
315355
// expected to be called by the validation webhook in the higher level meta
316356
// operator
317357
func (spec *NovaSpecCore) ValidateUpdate(old NovaSpecCore, basePath *field.Path, namespace string) field.ErrorList {
318-
errors := spec.ValidateCellTemplates(basePath, namespace)
358+
var errors field.ErrorList
359+
360+
// Validate deprecated fields and their new equivalents
361+
// Don't allow setting both old and new fields with different values
362+
// Users can either set both to the same value, or null out the old field and set the new one
363+
if spec.APIMessageBusInstance != "" && spec.MessagingBus.Cluster != "" &&
364+
spec.APIMessageBusInstance != spec.MessagingBus.Cluster {
365+
errors = append(errors, field.Invalid(
366+
basePath.Child("messagingBus").Child("cluster"),
367+
spec.MessagingBus.Cluster,
368+
fmt.Sprintf("messagingBus.cluster cannot differ from deprecated apiMessageBusInstance (%s). "+
369+
"Either use the new messagingBus.cluster field or the deprecated apiMessageBusInstance, but not both with different values",
370+
spec.APIMessageBusInstance)))
371+
}
372+
373+
// Reject changes to deprecated field unless nulling it out
374+
if spec.APIMessageBusInstance != old.APIMessageBusInstance && spec.APIMessageBusInstance != "" {
375+
errors = append(errors, field.Forbidden(
376+
basePath.Child("apiMessageBusInstance"),
377+
"apiMessageBusInstance is deprecated and cannot be changed. Please use messagingBus.cluster instead"))
378+
}
379+
380+
// Similar validation for NotificationsBusInstance
381+
// Don't allow setting both old and new fields with different values
382+
if spec.NotificationsBusInstance != nil && *spec.NotificationsBusInstance != "" &&
383+
spec.NotificationsBus != nil && spec.NotificationsBus.Cluster != "" &&
384+
*spec.NotificationsBusInstance != spec.NotificationsBus.Cluster {
385+
errors = append(errors, field.Invalid(
386+
basePath.Child("notificationsBus").Child("cluster"),
387+
spec.NotificationsBus.Cluster,
388+
fmt.Sprintf("notificationsBus.cluster cannot differ from deprecated notificationsBusInstance (%s). "+
389+
"Either use the new notificationsBus.cluster field or the deprecated notificationsBusInstance, but not both with different values",
390+
*spec.NotificationsBusInstance)))
391+
}
392+
393+
// Reject changes to deprecated field unless nulling it out
394+
if spec.NotificationsBusInstance != nil && old.NotificationsBusInstance != nil &&
395+
*spec.NotificationsBusInstance != *old.NotificationsBusInstance &&
396+
*spec.NotificationsBusInstance != "" {
397+
errors = append(errors, field.Forbidden(
398+
basePath.Child("notificationsBusInstance"),
399+
"notificationsBusInstance is deprecated and cannot be changed. Please use notificationsBus.cluster instead"))
400+
}
401+
402+
// Check cell template changes
403+
for cellName, cellTemplate := range spec.CellTemplates {
404+
if oldCell, exists := old.CellTemplates[cellName]; exists {
405+
cellPath := basePath.Child("cellTemplates").Key(cellName)
406+
407+
// Don't allow setting both old and new fields with different values
408+
// Users can either set both to the same value, or null out the old field and set the new one
409+
if cellTemplate.CellMessageBusInstance != "" && cellTemplate.MessagingBus.Cluster != "" &&
410+
cellTemplate.CellMessageBusInstance != cellTemplate.MessagingBus.Cluster {
411+
errors = append(errors, field.Invalid(
412+
cellPath.Child("messagingBus").Child("cluster"),
413+
cellTemplate.MessagingBus.Cluster,
414+
fmt.Sprintf("messagingBus.cluster cannot differ from deprecated cellMessageBusInstance (%s). "+
415+
"Either use the new messagingBus.cluster field or the deprecated cellMessageBusInstance, but not both with different values",
416+
cellTemplate.CellMessageBusInstance)))
417+
}
418+
419+
// Reject changes to deprecated field unless nulling it out
420+
if cellTemplate.CellMessageBusInstance != oldCell.CellMessageBusInstance &&
421+
cellTemplate.CellMessageBusInstance != "" {
422+
errors = append(errors, field.Forbidden(
423+
cellPath.Child("cellMessageBusInstance"),
424+
"cellMessageBusInstance is deprecated and cannot be changed. Please use messagingBus.cluster instead"))
425+
}
426+
}
427+
}
428+
429+
errors = append(errors, spec.ValidateCellTemplates(basePath, namespace)...)
319430
// Validate top-level TopologyRef
320431
errors = append(errors, topologyv1.ValidateTopologyRef(
321432
spec.TopologyRef, *basePath.Child("topologyRef"), namespace)...)

api/v1beta1/novacell_types.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ limitations under the License.
1717
package v1beta1
1818

1919
import (
20+
rabbitmqv1 "github.com/openstack-k8s-operators/infra-operator/apis/rabbitmq/v1beta1"
2021
topologyv1 "github.com/openstack-k8s-operators/infra-operator/apis/topology/v1beta1"
2122
condition "github.com/openstack-k8s-operators/lib-common/modules/common/condition"
2223
"github.com/openstack-k8s-operators/lib-common/modules/common/tls"
@@ -51,6 +52,10 @@ type NovaCellTemplate struct {
5152
// communicate in this cell. For cell0 it is unused.
5253
CellMessageBusInstance string `json:"cellMessageBusInstance"`
5354

55+
// +kubebuilder:validation:Optional
56+
// MessagingBus configuration (username, vhost, and cluster)
57+
MessagingBus rabbitmqv1.RabbitMqConfig `json:"messagingBus,omitempty"`
58+
5459
// +kubebuilder:validation:Required
5560
// HasAPIAccess defines if this Cell is configured to have access to the
5661
// API DB and message bus.

api/v1beta1/zz_generated.deepcopy.go

Lines changed: 8 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)