Skip to content

Expose pod property for resource QoS #339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 26, 2025
Merged

Expose pod property for resource QoS #339

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions api/bases/mariadb.openstack.org_galeras.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,61 @@ spec:
maximum: 3
minimum: 0
type: integer
resources:
description: Resources QoS configuration for pods
properties:
claims:
description: |-
Claims lists the names of resources, defined in spec.resourceClaims,
that are used by this container.


This is an alpha field and requires enabling the
DynamicResourceAllocation feature gate.


This field is immutable. It can only be set for containers.
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: |-
Name must match the name of one entry in pod.spec.resourceClaims of
the Pod where this field is used. It makes that resource available
inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: |-
Limits describes the maximum amount of compute resources allowed.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: |-
Requests describes the minimum amount of compute resources required.
If Requests is omitted for a container, it defaults to Limits if that is explicitly specified,
otherwise to an implementation-defined value. Requests cannot exceed Limits.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
type: object
type: object
secret:
description: Name of the secret to look for password keys
type: string
Expand Down
8 changes: 6 additions & 2 deletions api/v1beta1/galera_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,13 @@ limitations under the License.
package v1beta1

import (
topologyv1 "github.com/openstack-k8s-operators/infra-operator/apis/topology/v1beta1"
condition "github.com/openstack-k8s-operators/lib-common/modules/common/condition"
"github.com/openstack-k8s-operators/lib-common/modules/common/tls"
"github.com/openstack-k8s-operators/lib-common/modules/common/util"
topologyv1 "github.com/openstack-k8s-operators/infra-operator/apis/topology/v1beta1"
"k8s.io/apimachinery/pkg/util/validation/field"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/validation/field"
)

const (
Expand Down Expand Up @@ -87,6 +88,9 @@ type GaleraSpecCore struct {
// TopologyRef to apply the Topology defined by the associated CR referenced
// by name
TopologyRef *topologyv1.TopoRef `json:"topologyRef,omitempty"`
// +kubebuilder:validation:Optional
// Resources QoS configuration for pods
Resources corev1.ResourceRequirements `json:"resources,omitempty"`
}

// GaleraAttributes holds startup information for a Galera host
Expand Down
1 change: 1 addition & 0 deletions api/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 55 additions & 0 deletions config/crd/bases/mariadb.openstack.org_galeras.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,61 @@ spec:
maximum: 3
minimum: 0
type: integer
resources:
description: Resources QoS configuration for pods
properties:
claims:
description: |-
Claims lists the names of resources, defined in spec.resourceClaims,
that are used by this container.


This is an alpha field and requires enabling the
DynamicResourceAllocation feature gate.


This field is immutable. It can only be set for containers.
items:
description: ResourceClaim references one entry in PodSpec.ResourceClaims.
properties:
name:
description: |-
Name must match the name of one entry in pod.spec.resourceClaims of
the Pod where this field is used. It makes that resource available
inside a container.
type: string
required:
- name
type: object
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
limits:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: |-
Limits describes the maximum amount of compute resources allowed.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
type: object
requests:
additionalProperties:
anyOf:
- type: integer
- type: string
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
description: |-
Requests describes the minimum amount of compute resources required.
If Requests is omitted for a container, it defaults to Limits if that is explicitly specified,
otherwise to an implementation-defined value. Requests cannot exceed Limits.
More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
type: object
type: object
secret:
description: Name of the secret to look for password keys
type: string
Expand Down
3 changes: 2 additions & 1 deletion hack/crd-schema-checker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ for crd in config/crd/bases/*.yaml; do
if git show "$BASE_REF:$crd" > "$TMP_DIR/$crd"; then
$CHECKER check-manifests \
--existing-crd-filename="$TMP_DIR/$crd" \
--new-crd-filename="$crd"
--new-crd-filename="$crd" \
--disabled-validators NoMaps
fi
done
2 changes: 2 additions & 0 deletions pkg/mariadb/statefulset.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ func getGaleraInitContainers(g *mariadbv1.Galera) []corev1.Container {
},
},
}},
Resources: g.Spec.Resources,
VolumeMounts: getGaleraInitVolumeMounts(g),
}}
}
Expand Down Expand Up @@ -149,6 +150,7 @@ func getGaleraContainers(g *mariadbv1.Galera, configHash string) []corev1.Contai
ContainerPort: 4567,
Name: "galera",
}},
Resources: g.Spec.Resources,
VolumeMounts: getGaleraVolumeMounts(g),
StartupProbe: &corev1.Probe{
ProbeHandler: corev1.ProbeHandler{
Expand Down
113 changes: 113 additions & 0 deletions tests/chainsaw/tests/qos/chainsaw-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: qos
spec:
steps:
- name: Deploy 1-node cluster without pod QoS
description: Deploy a 1-node cluster and wait for readiness
bindings:
- name: replicas
value: 1
try:
- apply:
file: ../../common/galera.yaml
- assert:
# check if all nodes are started and one pod is reacheable via the service endpoint
file: ../../common/galera-assert.yaml
- script: &check
# check if galera can be accessed and cluster is correct
content: |
../../scripts/mysql-cli.sh -sNEe "show status like 'wsrep_cluster_size';" | tail -1 | tr -d '\n'
check:
(to_number($stdout)): ($replicas)

- name: Update QoS to Burstable
description: Ensure that requested resource is configured in the pod
try:
- patch:
resource:
apiVersion: mariadb.openstack.org/v1beta1
kind: Galera
metadata:
name: openstack
spec:
resources: &res_burstable
requests:
memory: "128M"
cpu: "700m"
- assert:
resource:
apiVersion: v1
kind: Pod
metadata:
name: openstack-galera-0
spec:
initContainers:
- resources: *res_burstable
containers:
- name: galera
resources: *res_burstable
status:
qosClass: Burstable
containerStatuses:
- name: galera
ready: true

- name: Update QoS to Guaranteed
description: Ensure that requested and limit resources are configured in the pod
try:
- patch:
resource:
apiVersion: mariadb.openstack.org/v1beta1
kind: Galera
metadata:
name: openstack
spec:
resources: &res_guaranteed
limits:
memory: "256M"
cpu: "700m"
requests:
memory: "256M"
cpu: "700m"
- assert:
resource:
apiVersion: v1
kind: Pod
metadata:
name: openstack-galera-0
spec:
initContainers:
- resources: *res_guaranteed
containers:
- name: galera
resources: *res_guaranteed
status:
qosClass: Guaranteed
containerStatuses:
- name: galera
ready: true

- name: Remove pod QoS
description: Ensure that a pod QoS can be removed
try:
- patch:
resource:
apiVersion: mariadb.openstack.org/v1beta1
kind: Galera
metadata:
name: openstack
spec:
resources:
- assert:
resource:
apiVersion: v1
kind: Pod
metadata:
name: openstack-galera-0
status:
qosClass: BestEffort
containerStatuses:
- name: galera
ready: true