[DynamoDB] Unable to create tables replicas in PROVISIONED billingMode

[vishalkatikineni]

Describe the bug
Unable to create tables replicas in PROVISIONED mode.

Steps to reproduce

apiVersion: dynamodb.services.k8s.aws/v1alpha1
kind: Table
metadata:
  annotations:
    services.k8s.aws/region: us-east-1
  name: example-global-table
  namespace: test-resources
spec:
  attributeDefinitions:
  - attributeName: PK
    attributeType: S
  - attributeName: SK
    attributeType: S
  billingMode: PROVISIONED
  continuousBackups:
    pointInTimeRecoveryEnabled: true
  deletionProtectionEnabled: false
  keySchema:
  - attributeName: PK
    keyType: HASH
  - attributeName: SK
    keyType: RANGE
  provisionedThroughput:
    readCapacityUnits: 1
    writeCapacityUnits: 1
  streamSpecification:
    streamEnabled: true
    streamViewType: NEW_AND_OLD_IMAGES
  tableClass: STANDARD
  tableName: example-global-table
  tableReplicas:
  - provisionedThroughputOverride:
      readCapacityUnits: 2
    regionName: us-west-2
  tags:
  - key: Name
    value: example-global-table
status:
  ackResourceMetadata:
    arn: arn:aws:dynamodb:us-east-1:1234567890:table/example-global-table
    ownerAccountID: "1234567890"
    region: us-east-1
  conditions:
  - message: 'api error ValidationException: Table write capacity should either be
      Pay-Per-Request or AutoScaled.'
    status: "True"
    type: ACK.Terminal
  - lastTransitionTime: "2025-08-20T20:54:02Z"
    message: Resource not synced
    reason: resource is in terminal condition
    status: "False"
    type: ACK.ResourceSynced
  creationDateTime: "2025-08-20T20:53:40Z"
  itemCount: 0
  latestStreamARN: arn:aws:dynamodb:us-east-1:1234567890:table/example-global-table/stream/2025-08-20T20:53:40.423
  latestStreamLabel: 2025-08-20T20:53:40.423
  tableID: 9b7a588d-9e94-4efd-b390-cf95b21e8250
  tableSizeBytes: 0
  tableStatus: ACTIVE

Expected outcome
Expect the replicas to be created with the same provisioned throughput. As a workaround we are enabling auto-scaling manually through console, and this enables the DynamoDB controller to reconcile and create the replicas.

Environment
Dev

• Kubernetes version
• Using EKS (yes/no), if so version?
• AWS service targeted (S3, RDS, etc.) DynamoDB

[github-actions]

Hello @vishalkatikineni 👋 Thank you for opening an issue in ACK! A maintainer will triage this issue soon.

We encourage community contributions, so if you're interested in tackling this yourself or suggesting a solution, please check out our Contribution and Code of Conduct guidelines.

You can find more information about ACK on our website.

[vishalkatikineni]

• creating the tableReplicas with billingMode as PAY_PER_REQUEST to start with and later switch to PROVISIONED mode works fine. But if we start creating the table with PROVISIONED mode and tableReplicas that's when you get the error api error ValidationException: Table write capacity should either be Pay-Per-Request or AutoScaled.
• Converting from PAY_PER_REQUEST to PROVISIONED mode with GSIs and tableReplicas error out with message cannot update table operation error DynamoDB: UpdateTable, https response error StatusCode: 400, RequestID: GBC1LCPD82UPBUD6P9BTKHVSH3VV4KQNSO5AEMVJF66Q9ASUAAJG, api error ValidationException: One or more parameter values were invalid: ProvisionedThroughput must be specified for index: MyGSI

spec:
  attributeDefinitions:
  - attributeName: PK
    attributeType: S
  - attributeName: SK
    attributeType: S
  - attributeName: GSI_PK
    attributeType: S
  - attributeName: GSI_SK
    attributeType: S
  billingMode: PROVISIONED
  continuousBackups:
    pointInTimeRecoveryEnabled: true
  deletionProtectionEnabled: false
  globalSecondaryIndexes:
  - indexName: MyGSI
    keySchema:
    - attributeName: GSI_PK
      keyType: HASH
    - attributeName: GSI_SK
      keyType: RANGE
    projection:
      projectionType: KEYS_ONLY
    provisionedThroughput:
      readCapacityUnits: 1
      writeCapacityUnits: 1
  keySchema:
  - attributeName: PK
    keyType: HASH
  - attributeName: SK
    keyType: RANGE
  provisionedThroughput:
    readCapacityUnits: 1
    writeCapacityUnits: 1
  streamSpecification:
    streamEnabled: true
    streamViewType: NEW_AND_OLD_IMAGES
  tableClass: STANDARD
  tableName: example-keyvaluedb-global-2
  tableReplicas:
  - globalSecondaryIndexes:
    - indexName: MyGSI
    regionName: us-west-2

[knottnt]

When it comes to creating table replicas with the PROVISIONED billing mode I'm not sure we'll be able to solve this within the dynamodb-controller due to how the DynamoDB API is implemented. To create a table replica for a DynamoDB table with billing mode set to PROVISIONED it's required that application auto-scaling is enabled for the table. However, the DynamoDB API doesn't provide any way to enable this. Instead one needs to directly interact with the Application Autoscaling API to setup the scaling policy.

Regarding the issue with GSIs when switching from PAY_PER_REQUEST to PROVISIONED I think we may be able to fix that since it seems to be entirely contained in the DynamoDB API.

Relevant AWS documentation
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/AutoScaling.html
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/AutoScaling.HowTo.SDK.html

[vishalkatikineni]

@knottnt for table replicas with the PROVISIONED billing mode if we can mimic similar to what terraform does https://registry.terraform.io/modules/terraform-aws-modules/dynamodb-table/aws/5.1.0?tab=inputs that will be great.

[knottnt]

@vishalkatikineni any Terraform behaviors you're looking for in particular?

[vishalkatikineni]

@knottnt Instead of relying on the applicationautoscaling-controller We are looking for a solution with in dynamodb-controller similar to how terraform enables autoscaling_defaults, autoscaling_enabled, autoscaling_read, autoscaling_write and autoscaling_indexes resources.

There is also an open issue related to autoscaling #2033

[knottnt]

@vishalkatikineni we should be able to support allowing the provisioned throughput values to be set by Application Autoscaling as described in #2033 using an implementation similar to the eks.services.k8s.aws/desired-size-managed-by annotation.

However, we won't be able to setup the Application Autoscaling policy from the dynamodb-controller as the DynamoDB API doesn't expose this capability. The linked Terraform module appears to be using providers for both Application Autoscaling and DynamoDB to interact with both AWS services to setup the individual resources. In the case of ACK a higher level construct like a KRO RGD could be used to provision both the DynamoDB table and Application Autoscaling policy.