Skip to content

Persistent reconciliation diff when adopting DBCluster and DBInstance with RDS Controller 1.6.2 #2653

@asaldalop

Description

@asaldalop

Describe the bug
When adopting an RDS Aurora cluster with instances, the reconciliation loop consistently detects differences in the desired state, causing the instances to enter the "Modifying" state every 2–3 minutes.

Steps to reproduce
Adopt an RDS Aurora cluster and instance by using the following manifest:

apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBCluster
metadata:
  name: test-rds-cluster
  namespace: default
  annotations:
    services.k8s.aws/deletion-policy: retain
    services.k8s.aws/adoption-policy: "adopt-or-create"
spec:
  engine: aurora-mysql
  engineVersion: 8.0.mysql_aurora.3.08.0
  ...
---
apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBInstance
metadata:
  name: test-rds-instance-0
  namespace: default
  annotations:
    services.k8s.aws/deletion-policy: retain
    services.k8s.aws/adoption-policy: "adopt-or-create"
spec:
  dbInstanceClass: db.t4g.medium
  dbInstanceIdentifier: test-rds-instance-0
  ...

The controller logs show attempts to reconcile due to detected differences, even though no actual changes have been made:

dbcluster.log
{
    "level": "info",
    "ts": "2025-10-10T16:47:17.676Z",
    "logger": "ackrt",
    "msg": "desired resource state has changed",
    "kind": "DBCluster",
    "namespace": "default",
    "name": "test-rds-cluster",
    "account": "XXXXXXXXXXXX",
    "role": "",
    "region": "us-east-1",
    "is_adopted": true,
    "generation": 9,
    "diff":
    [
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "NetworkType"
                ]
            },
            "A": null,
            "B": "IPV4"
        }
    ]
}
dbinstance.log
{
    "level": "info",
    "ts": "2025-10-10T16:45:10.820Z",
    "logger": "ackrt",
    "msg": "desired resource state has changed",
    "kind": "DBInstance",
    "namespace": "default",
    "name": "test-rds-instance-0",
    "account": "XXXXXXXXXXXX",
    "role": "",
    "region": "us-east-1",
    "is_adopted": true,
    "generation": 21,
    "diff":
    [
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "AllocatedStorage"
                ]
            },
            "A": null,
            "B": 1
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "AutoMinorVersionUpgrade"
                ]
            },
            "A": null,
            "B": false
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "BackupRetentionPeriod"
                ]
            },
            "A": null,
            "B": 7
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "CopyTagsToSnapshot"
                ]
            },
            "A": null,
            "B": false
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "DBName"
                ]
            },
            "A": null,
            "B": "testdb"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "DatabaseInsightsMode"
                ]
            },
            "A": null,
            "B": "standard"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "DeletionProtection"
                ]
            },
            "A": null,
            "B": false
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "KMSKeyID"
                ]
            },
            "A": null,
            "B": "arn:aws:kms:us-east-1:XXXXXXXXXXXX:key/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "LicenseModel"
                ]
            },
            "A": null,
            "B": "general-public-license"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "MasterUsername"
                ]
            },
            "A": null,
            "B": "admin"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "MonitoringInterval"
                ]
            },
            "A": null,
            "B": 0
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "MultiAZ"
                ]
            },
            "A": null,
            "B": false
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "PreferredBackupWindow"
                ]
            },
            "A": null,
            "B": "08:00-09:00"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "PreferredMaintenanceWindow"
                ]
            },
            "A": null,
            "B": "mon:02:00-mon:03:00"
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "PromotionTier"
                ]
            },
            "A": null,
            "B": 0
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "PubliclyAccessible"
                ]
            },
            "A": null,
            "B": false
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "StorageEncrypted"
                ]
            },
            "A": null,
            "B": true
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "StorageThroughput"
                ]
            },
            "A": null,
            "B": 0
        },
        {
            "Path":
            {
                "Parts":
                [
                    "Spec",
                    "StorageType"
                ]
            },
            "A": null,
            "B": "aurora"
        }
    ]
}

Expected outcome
Having the resources adopted without any differences detected, and the instances remained stable without constant modifications during each reconciliation loop.

Environment

  • Kubernetes version: 1.32.6
  • Using EKS (yes/no), if so version? yes
  • AWS service targeted (S3, RDS, etc.): RDS and ACK RDS Controller v.1.6.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.service/rdsIndicates issues or PRs that are related to rds-controller.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions