Skip to content

ACK RDS controller false diff when creating DBParameterGroup with log_checkpoints: '1' #2616

@eTimS

Description

@eTimS

Hello,

Describe the bug
Creating a DBParameterGroup with log_checkpoints: '1' in the parameterOverrides leads to permanent diff.

Steps to reproduce
Use rds-chart:1.6.0 (bug also observed with rds-chart:1.4.1)
with the following values (all settings have been copied in case they are relevant):

aws:
  region: eu-west-3
deployment:
  replicas: 3
leaderElection:
  enabled: true
metrics:
  service:
    create: true
reconcile:
  defaultMaxConcurrentSyncs: 10
  defaultResyncPeriod: 60 # Debug value to show the diff message in the controller logs every minute instead of 10 hours
serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: <insert your IRSA ARN here>

Apply the two following manifests:

  • dpg.yaml :
    apiVersion: rds.services.k8s.aws/v1alpha1
    kind: DBParameterGroup
    metadata:
      name: dpg-postgres
      namespace: default
    spec:
      description: Parameter group with no diff issue
      family: postgres17
      name: dpg-postgres
      parameterOverrides:
        hot_standby_feedback: '1'
        # log_checkpoints: '1'
        log_connections: '1'
        log_lock_waits: '1'
        log_min_duration_statement: '10000'
        log_rotation_age: '1440'
        shared_preload_libraries: pg_stat_statements
  • dpg-permanent-diff.yaml :
    apiVersion: rds.services.k8s.aws/v1alpha1
    kind: DBParameterGroup
    metadata:
      name: dpg-permanent-diff-postgres
      namespace: default
    spec:
      description: Parameter group with permanent diff issue
      family: postgres17
      name: dpg-permanent-diff-postgres
      parameterOverrides:
        hot_standby_feedback: '1'
        log_checkpoints: '1'
        log_connections: '1'
        log_lock_waits: '1'
        log_min_duration_statement: '10000'
        log_rotation_age: '1440'
        shared_preload_libraries: pg_stat_statements

The only difference is adding the log_checkpoints: '1' which causes permanent drift detection.
After one minute, notice in the controller logs that a diff is noticed for the DBParameterGroup named "dpg-permanent-diff-postgres" but not for "dpg-postgres" :

{"level":"info","ts":"2025-09-01T14:50:05.906Z","logger":"ackrt","msg":"desired resource state has changed","kind":"DBParameterGroup","namespace":"default","name":"dpg-permanent-diff-postgres","account":"[redacted]","role":"","region":"eu-west-3","is_adopted":false,"generation":1,"diff":[{"Path":{"Parts":["Spec","ParameterOverrides"]},"A":{"hot_standby_feedback":"1","log_checkpoints":"1","log_connections":"1","log_lock_waits":"1","log_min_duration_statement":"10000","log_rotation_age":"1440","shared_preload_libraries":"pg_stat_statements"},"B":{"hot_standby_feedback":"1","log_connections":"1","log_lock_waits":"1","log_min_duration_statement":"10000","log_rotation_age":"1440","shared_preload_libraries":"pg_stat_statements"}}]}

The logs seems to indicate that the controller "doesn't see" "log_checkpoints":"1" in the parameter group and wants to modify the parameter group to reconcile. Double checking with the AWS RDS Console proved the "log_checkpoints":"1" is actually correctly set on the parameter group.

Expected outcome
We expect no diff to be detected since the "dpg-permanent-diff-postgres" is appropriately configured and synced.

Environment

  • Kubernetes version
    1.31
  • Using EKS (yes/no), if so version?
    eks.34
  • AWS service targeted (S3, RDS, etc.)
    RDS

Metadata

Metadata

Assignees

Labels

service/rdsIndicates issues or PRs that are related to rds-controller.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions