OSPC-1927: memcached epoxy changes

[manojacloud]

Summary:

• Genestack will use openstack helm chart (https://github.com/openstack/openstack-helm/tree/master/memcached) instead of bitnami helm chart (https://github.com/bitnami/charts/tree/main/bitnami/memcached) for memcached.
• Currently we have memcached version 1.6.39 in bitnami helm chart. We will use 1.6.32 version provided by chart version 2025.2.2+c31cd6c96 of openstack helm memcached. So it will be a minor version downgrade from 1.6.39 to 1.6.32.
• However it will not impact anything on openstack as openstack doesn't define any specific version of memcached to use in it documentation.
• Verified release notes in 1.6.32 (https://github.com/memcached/memcached/wiki/ReleaseNotes1632) and 1.6.39 (https://github.com/memcached/memcached/wiki/ReleaseNotes1639) version. There are changes related to proxy and tls which is not used by the memcached used openstack.

Key changes:

• The env vars MEMCACHED_THREADS and MEMCACHED_MAX_ITEM_SIZE (

  genestack/base-helm-configs/memcached/memcached-helm-overrides.yaml

  Lines 135 to 138 in 3499bd5

  	- name: MEMCACHED_THREADS
  	value: "8"
  	- name: MEMCACHED_MAX_ITEM_SIZE
  	value: "4194304"

  ) were used in bitnami chart and openstack helm chart don't provide that. So, kustomize post renderer will be used to add those extra variables and include them during installation.
• max_connections for memcached is set to 8192 as default in openstack-helm memcached chart. However genestack was using 4096 (

  genestack/base-helm-configs/memcached/memcached-helm-overrides.yaml

  Lines 133 to 134 in 3499bd5

  	- name: MEMCACHED_MAX_CONNECTIONS
  	value: "4096"

  ) in its bitnami memcached chart. So it will be set to 4096 to match old settings.
• Memory was set to 9216 Mi in previously used bitnami memcached chart (

  genestack/base-helm-configs/memcached/memcached-helm-overrides.yaml

  Lines 131 to 132 in 065150e

  	- name: MEMCACHED_CACHE_SIZE
  	value: "9216"

  ). It will be set to 3072 Mi as the limit in resources is set to 3072 Mi.
• If we set stats_cachedump to false in memcached-helm-overrides.yaml then we will have to add one additional argument "-X" in https://github.com/manojacloud/genestack/blob/285f0500318d4f97dd093dab551e838c17389bed/base-kustomize/memcached/base/memcached-bin-configmap-patch.yaml. This is because openstack-helm memcached chart doesn't have some variables as compared to previously used bitnami memcached chart so we are using kustomize post renderer to add additional variable and this dynamic value of stats_cachedump will not set automatically there. Currently we are not passing "-X" as stats_cachedump is enabled, if we disable then need to pass "-X".
• Bitnami chart had persistence enabled for memcached. However openstack-helm chart for memcached don't have persistence enabled.
• Kept the service name of memcached same "memcached.openstack.svc.cluster.local" as previuos bitnami chart in openstack-helm memcached chart as it is used by all openstack services. Hence, need to remove existing memcached installed by bitnami in genestack using command "helm uninstall memcached -n openstack" before installing new memcached. Then install memcached of openstack-helm using the usual installation script.

Validation:

• Tested the changes in Hyperconverged lab.
• Removed existing memcached chart of bitnami and installed openstack-helm memcached using install-memcached.sh script.

root@hyperconverged-103-0:~# helm get metadata memcached -n openstack
NAME: memcached
CHART: memcached
VERSION: 2025.2.2+c31cd6c96
APP_VERSION: v1.5.5
ANNOTATIONS:
DEPENDENCIES: helm-toolkit
NAMESPACE: openstack
REVISION: 1
STATUS: deployed
DEPLOYED_AT: 2026-03-19T13:45:54Z

root@hyperconverged-103-0:~# kubectl exec memcached-memcached-0 -n openstack -ti -- bash
Defaulted container "memcached" out of: memcached, init (init)
nobody@memcached-memcached-0:/$ memcached --version
memcached 1.6.32

• Created a temporary test pod and verified that all required variables are present in memcached pod.

root@hyperconverged-103-0:~# kubectl run tmp --rm -it --image=busybox -n openstack -- sh
/ # echo stats | nc memcached.openstack.svc.cluster.local 11211 | egrep 'max_connections|threads|limit_maxbytes'
STAT max_connections 4096
STAT limit_maxbytes 3221225472
STAT threads 8
echo stats settings | nc memcached.openstack.svc.cluster.local 11211 | grep item_size_max
STAT item_size_max 4194304

[awfabian-rs]

looks like base-kustomize/memcached/base/hpa.yaml HPA will immediately have to kick this to 2 to hit minReplicas

[awfabian-rs]

commented that replicas should probably be '2' so HPA doesn't immediately have to kick in and fix it

I tried a helm dry run and modified the replica in the overrides.

the rendered HPA object says:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: memcached
  namespace: openstack
spec:
  maxReplicas: 9
  metrics:
  - resource:
      name: cpu
      target:
        averageUtilization: 80
        type: Utilization
    type: Resource
  - resource:
      name: memory
      target:
        averageUtilization: 80
        type: Utilization
    type: Resource
  minReplicas: 2
  scaleTargetRef:
    apiVersion: apps/v1
    kind: StatefulSet
    name: memcached-memcached

Then the original statefulset with the replicas: 1, then after I changed it to replicas: 2:

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  annotations:
    openstackhelm.openstack.org/release_uuid: ""
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/instance: memcached
    app.kubernetes.io/name: memcached
    application: memcached
    component: server
    release_group: memcached
  name: memcached-memcached
spec:
  podManagementPolicy: Parallel
  replicas: 1

apiVersion: apps/v1
kind: StatefulSet
metadata:
  annotations:
    openstackhelm.openstack.org/release_uuid: ""
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/instance: memcached
    app.kubernetes.io/name: memcached
    application: memcached
    component: server
    release_group: memcached
  name: memcached-memcached
spec:
  podManagementPolicy: Parallel
  replicas: 2
  selector: