Skip to content

Commit 9581ba9

Browse files
silenium-devFxKu
silenium-dev
and
FxKu
authored
Add hugepages 2Mi and 1Gi fields to ResourceDescription and pass them to the statefulset (#2311)
* Add hugepages-2Mi and 1Gi to ResourceDescription type and crd (#1549, #1788) * Add tests for hugepages resource requests/limits * Add tests for hugepages resource requests/limits on sidecars, too * Add docs for hugepages support * Add link to kubernetes docs on hugepages * Add tests for hugepages not being set on container if not requested in custom resource * Add hugepages resources fields to manifest docs * Add hugepages resources fields to complete manifest example * Add hugepages resources fields to chart crd --------- Co-authored-by: Felix Kunde <[email protected]>
1 parent 3ca26d0 commit 9581ba9

File tree

9 files changed

+247
-9
lines changed

9 files changed

+247
-9
lines changed

‎charts/postgres-operator/crds/postgresqls.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -441,6 +441,12 @@ spec:
441441
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
442442
# Note: the value specified here must not be zero or be higher
443443
# than the corresponding limit.
444+
hugepages-2Mi:
445+
type: string
446+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
447+
hugepages-1Gi:
448+
type: string
449+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
444450
requests:
445451
type: object
446452
properties:
@@ -450,6 +456,12 @@ spec:
450456
memory:
451457
type: string
452458
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
459+
hugepages-2Mi:
460+
type: string
461+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
462+
hugepages-1Gi:
463+
type: string
464+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
453465
schedulerName:
454466
type: string
455467
serviceAnnotations:

‎docs/reference/cluster_manifest.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,14 @@ CPU and memory requests for the Postgres container.
359359
memory requests for the Postgres container. Optional, overrides the
360360
`default_memory_request` operator configuration parameter.
361361

362+
* **hugepages-2Mi**
363+
hugepages-2Mi requests for the sidecar container.
364+
Optional, defaults to not set.
365+
366+
* **hugepages-1Gi**
367+
1Gi hugepages requests for the sidecar container.
368+
Optional, defaults to not set.
369+
362370
### Limits
363371

364372
CPU and memory limits for the Postgres container.
@@ -371,6 +379,14 @@ CPU and memory limits for the Postgres container.
371379
memory limits for the Postgres container. Optional, overrides the
372380
`default_memory_limits` operator configuration parameter.
373381

382+
* **hugepages-2Mi**
383+
hugepages-2Mi requests for the sidecar container.
384+
Optional, defaults to not set.
385+
386+
* **hugepages-1Gi**
387+
1Gi hugepages requests for the sidecar container.
388+
Optional, defaults to not set.
389+
374390
## Parameters defining how to clone the cluster from another one
375391

376392
Those parameters are applied when the cluster should be a clone of another one
@@ -500,6 +516,14 @@ CPU and memory requests for the sidecar container.
500516
memory requests for the sidecar container. Optional, overrides the
501517
`default_memory_request` operator configuration parameter. Optional.
502518

519+
* **hugepages-2Mi**
520+
hugepages-2Mi requests for the sidecar container.
521+
Optional, defaults to not set.
522+
523+
* **hugepages-1Gi**
524+
1Gi hugepages requests for the sidecar container.
525+
Optional, defaults to not set.
526+
503527
### Limits
504528

505529
CPU and memory limits for the sidecar container.
@@ -512,6 +536,14 @@ CPU and memory limits for the sidecar container.
512536
memory limits for the sidecar container. Optional, overrides the
513537
`default_memory_limits` operator configuration parameter. Optional.
514538

539+
* **hugepages-2Mi**
540+
hugepages-2Mi requests for the sidecar container.
541+
Optional, defaults to not set.
542+
543+
* **hugepages-1Gi**
544+
1Gi hugepages requests for the sidecar container.
545+
Optional, defaults to not set.
546+
515547
## Connection pooler
516548

517549
Parameters are grouped under the `connectionPooler` top-level key and specify

‎docs/user.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -690,6 +690,27 @@ manifest the operator will raise the limits to the configured minimum values.
690690
If no resources are defined in the manifest they will be obtained from the
691691
configured [default requests](reference/operator_parameters.md#kubernetes-resource-requests).
692692

693+
### HugePages support
694+
695+
The operator supports [HugePages](https://www.postgresql.org/docs/15/kernel-resources.html#LINUX-HUGEPAGES).
696+
To enable HugePages, set the matching resource requests and/or limits in the manifest:
697+
698+
```yaml
699+
spec:
700+
resources:
701+
requests:
702+
hugepages-2Mi: 250Mi
703+
hugepages-1Gi: 1Gi
704+
limits:
705+
hugepages-2Mi: 500Mi
706+
hugepages-1Gi: 2Gi
707+
```
708+
709+
There are no minimums or maximums and the default is 0 for both HugePage sizes,
710+
but Kubernetes will not spin up the pod if the requested HugePages cannot be allocated.
711+
For more information on HugePages in Kubernetes, see also
712+
[https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/](https://kubernetes.io/docs/tasks/manage-hugepages/scheduling-hugepages/)
713+
693714
## Use taints, tolerations and node affinity for dedicated PostgreSQL nodes
694715

695716
To ensure Postgres pods are running on nodes without any other application pods,

‎manifests/complete-postgres-manifest.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,9 +107,13 @@ spec:
107107
requests:
108108
cpu: 10m
109109
memory: 100Mi
110+
# hugepages-2Mi: 128Mi
111+
# hugepages-1Gi: 1Gi
110112
limits:
111113
cpu: 500m
112114
memory: 500Mi
115+
# hugepages-2Mi: 128Mi
116+
# hugepages-1Gi: 1Gi
113117
patroni:
114118
failsafe_mode: false
115119
initdb:

‎manifests/postgresql.crd.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -439,6 +439,12 @@ spec:
439439
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
440440
# Note: the value specified here must not be zero or be higher
441441
# than the corresponding limit.
442+
hugepages-2Mi:
443+
type: string
444+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
445+
hugepages-1Gi:
446+
type: string
447+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
442448
requests:
443449
type: object
444450
properties:
@@ -448,6 +454,12 @@ spec:
448454
memory:
449455
type: string
450456
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
457+
hugepages-2Mi:
458+
type: string
459+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
460+
hugepages-1Gi:
461+
type: string
462+
pattern: '^(\d+(e\d+)?|\d+(\.\d+)?(e\d+)?[EPTGMK]i?)$'
451463
schedulerName:
452464
type: string
453465
serviceAnnotations:

‎pkg/apis/acid.zalan.do/v1/crds.go

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,11 @@ package v1
33
import (
44
"fmt"
55

6-
acidzalando "github.com/zalando/postgres-operator/pkg/apis/acid.zalan.do"
7-
"github.com/zalando/postgres-operator/pkg/util"
86
apiextv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
97
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
8+
9+
acidzalando "github.com/zalando/postgres-operator/pkg/apis/acid.zalan.do"
10+
"github.com/zalando/postgres-operator/pkg/util"
1011
)
1112

1213
// CRDResource* define names necesssary for the k8s CRD API
@@ -684,6 +685,14 @@ var PostgresCRDResourceValidation = apiextv1.CustomResourceValidation{
684685
Type: "string",
685686
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
686687
},
688+
"hugepages-2Mi": {
689+
Type: "string",
690+
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
691+
},
692+
"hugepages-1Gi": {
693+
Type: "string",
694+
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
695+
},
687696
},
688697
},
689698
"requests": {
@@ -697,6 +706,14 @@ var PostgresCRDResourceValidation = apiextv1.CustomResourceValidation{
697706
Type: "string",
698707
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
699708
},
709+
"hugepages-2Mi": {
710+
Type: "string",
711+
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
712+
},
713+
"hugepages-1Gi": {
714+
Type: "string",
715+
Pattern: "^(\\d+(e\\d+)?|\\d+(\\.\\d+)?(e\\d+)?[EPTGMK]i?)$",
716+
},
700717
},
701718
},
702719
},

‎pkg/apis/acid.zalan.do/v1/postgresql_type.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -153,8 +153,10 @@ type PostgresqlParam struct {
153153

154154
// ResourceDescription describes CPU and memory resources defined for a cluster.
155155
type ResourceDescription struct {
156-
CPU string `json:"cpu"`
157-
Memory string `json:"memory"`
156+
CPU string `json:"cpu"`
157+
Memory string `json:"memory"`
158+
HugePages2Mi string `json:"hugepages-2Mi"`
159+
HugePages1Gi string `json:"hugepages-1Gi"`
158160
}
159161

160162
// Resources describes requests and limits for the cluster resouces.

‎pkg/cluster/k8sres.go

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,10 @@ import (
2020
"k8s.io/apimachinery/pkg/types"
2121
"k8s.io/apimachinery/pkg/util/intstr"
2222

23+
"golang.org/x/exp/maps"
24+
batchv1 "k8s.io/api/batch/v1"
25+
"k8s.io/apimachinery/pkg/labels"
26+
2327
acidv1 "github.com/zalando/postgres-operator/pkg/apis/acid.zalan.do/v1"
2428
"github.com/zalando/postgres-operator/pkg/spec"
2529
"github.com/zalando/postgres-operator/pkg/util"
@@ -28,9 +32,6 @@ import (
2832
"github.com/zalando/postgres-operator/pkg/util/k8sutil"
2933
"github.com/zalando/postgres-operator/pkg/util/patroni"
3034
"github.com/zalando/postgres-operator/pkg/util/retryutil"
31-
"golang.org/x/exp/maps"
32-
batchv1 "k8s.io/api/batch/v1"
33-
"k8s.io/apimachinery/pkg/labels"
3435
)
3536

3637
const (
@@ -266,6 +267,19 @@ func fillResourceList(spec acidv1.ResourceDescription, defaults acidv1.ResourceD
266267
}
267268
}
268269

270+
if spec.HugePages2Mi != "" {
271+
requests[v1.ResourceHugePagesPrefix+"2Mi"], err = resource.ParseQuantity(spec.HugePages2Mi)
272+
if err != nil {
273+
return nil, fmt.Errorf("could not parse hugepages-2Mi quantity: %v", err)
274+
}
275+
}
276+
if spec.HugePages1Gi != "" {
277+
requests[v1.ResourceHugePagesPrefix+"1Gi"], err = resource.ParseQuantity(spec.HugePages1Gi)
278+
if err != nil {
279+
return nil, fmt.Errorf("could not parse hugepages-1Gi quantity: %v", err)
280+
}
281+
}
282+
269283
return requests, nil
270284
}
271285

‎pkg/cluster/k8sres_test.go

Lines changed: 126 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,8 @@ import (
55
"fmt"
66
"reflect"
77
"sort"
8-
"time"
9-
108
"testing"
9+
"time"
1110

1211
"github.com/stretchr/testify/assert"
1312

@@ -3081,6 +3080,131 @@ func TestGenerateResourceRequirements(t *testing.T) {
30813080
ResourceLimits: acidv1.ResourceDescription{CPU: "1", Memory: "2Gi"},
30823081
},
30833082
},
3083+
{
3084+
subTest: "test HugePages are not set on container when not requested in manifest",
3085+
config: config.Config{
3086+
Resources: configResources,
3087+
PodManagementPolicy: "ordered_ready",
3088+
},
3089+
pgSpec: acidv1.Postgresql{
3090+
ObjectMeta: metav1.ObjectMeta{
3091+
Name: clusterName,
3092+
Namespace: namespace,
3093+
},
3094+
Spec: acidv1.PostgresSpec{
3095+
Resources: &acidv1.Resources{
3096+
ResourceRequests: acidv1.ResourceDescription{},
3097+
ResourceLimits: acidv1.ResourceDescription{},
3098+
},
3099+
TeamID: "acid",
3100+
Volume: acidv1.Volume{
3101+
Size: "1G",
3102+
},
3103+
},
3104+
},
3105+
expectedResources: acidv1.Resources{
3106+
ResourceRequests: acidv1.ResourceDescription{
3107+
CPU: "100m",
3108+
Memory: "100Mi",
3109+
},
3110+
ResourceLimits: acidv1.ResourceDescription{
3111+
CPU: "1",
3112+
Memory: "500Mi",
3113+
},
3114+
},
3115+
},
3116+
{
3117+
subTest: "test HugePages are passed through to the postgres container",
3118+
config: config.Config{
3119+
Resources: configResources,
3120+
PodManagementPolicy: "ordered_ready",
3121+
},
3122+
pgSpec: acidv1.Postgresql{
3123+
ObjectMeta: metav1.ObjectMeta{
3124+
Name: clusterName,
3125+
Namespace: namespace,
3126+
},
3127+
Spec: acidv1.PostgresSpec{
3128+
Resources: &acidv1.Resources{
3129+
ResourceRequests: acidv1.ResourceDescription{
3130+
HugePages2Mi: "128Mi",
3131+
HugePages1Gi: "1Gi",
3132+
},
3133+
ResourceLimits: acidv1.ResourceDescription{
3134+
HugePages2Mi: "256Mi",
3135+
HugePages1Gi: "2Gi",
3136+
},
3137+
},
3138+
TeamID: "acid",
3139+
Volume: acidv1.Volume{
3140+
Size: "1G",
3141+
},
3142+
},
3143+
},
3144+
expectedResources: acidv1.Resources{
3145+
ResourceRequests: acidv1.ResourceDescription{
3146+
CPU: "100m",
3147+
Memory: "100Mi",
3148+
HugePages2Mi: "128Mi",
3149+
HugePages1Gi: "1Gi",
3150+
},
3151+
ResourceLimits: acidv1.ResourceDescription{
3152+
CPU: "1",
3153+
Memory: "500Mi",
3154+
HugePages2Mi: "256Mi",
3155+
HugePages1Gi: "2Gi",
3156+
},
3157+
},
3158+
},
3159+
{
3160+
subTest: "test HugePages are passed through on sidecars",
3161+
config: config.Config{
3162+
Resources: configResources,
3163+
PodManagementPolicy: "ordered_ready",
3164+
},
3165+
pgSpec: acidv1.Postgresql{
3166+
ObjectMeta: metav1.ObjectMeta{
3167+
Name: clusterName,
3168+
Namespace: namespace,
3169+
},
3170+
Spec: acidv1.PostgresSpec{
3171+
Sidecars: []acidv1.Sidecar{
3172+
{
3173+
Name: "test-sidecar",
3174+
DockerImage: "test-image",
3175+
Resources: &acidv1.Resources{
3176+
ResourceRequests: acidv1.ResourceDescription{
3177+
HugePages2Mi: "128Mi",
3178+
HugePages1Gi: "1Gi",
3179+
},
3180+
ResourceLimits: acidv1.ResourceDescription{
3181+
HugePages2Mi: "256Mi",
3182+
HugePages1Gi: "2Gi",
3183+
},
3184+
},
3185+
},
3186+
},
3187+
TeamID: "acid",
3188+
Volume: acidv1.Volume{
3189+
Size: "1G",
3190+
},
3191+
},
3192+
},
3193+
expectedResources: acidv1.Resources{
3194+
ResourceRequests: acidv1.ResourceDescription{
3195+
CPU: "100m",
3196+
Memory: "100Mi",
3197+
HugePages2Mi: "128Mi",
3198+
HugePages1Gi: "1Gi",
3199+
},
3200+
ResourceLimits: acidv1.ResourceDescription{
3201+
CPU: "1",
3202+
Memory: "500Mi",
3203+
HugePages2Mi: "256Mi",
3204+
HugePages1Gi: "2Gi",
3205+
},
3206+
},
3207+
},
30843208
}
30853209

30863210
for _, tt := range tests {

0 commit comments

Comments
 (0)