Skip to content

feat(datastore): add per-model syncPageSize and maxRecordsToSync configuration#14732

Open
danrivett wants to merge 2 commits intoaws-amplify:mainfrom
danrivett:datastore/feat/per-model-sync-page-size
Open

feat(datastore): add per-model syncPageSize and maxRecordsToSync configuration#14732
danrivett wants to merge 2 commits intoaws-amplify:mainfrom
danrivett:datastore/feat/per-model-sync-page-size

Conversation

@danrivett
Copy link

Description of changes

Adds an optional third parameter to syncExpression() that allows configuring syncPageSize and maxRecordsToSync on a per-model basis. Models without per-model configuration continue to use the global defaults.

Problem: DataStore has a single global syncPageSize (default: 1000) that applies to all models during sync. When one model with large/denormalized records exceeds the AppSync 1MB response limit, users must lower syncPageSize globally, causing a significant performance penalty for all other (smaller) models.

Solution: Extend syncExpression() with an optional PerModelSyncConfig parameter:

DataStore.configure({
  syncPageSize: 1000,              // global default
  maxRecordsToSync: 10000,         // global default
  syncExpressions: [
    // Per-model config: use smaller page size for this model only
    syncExpression(LargeModel, () => Predicates.ALL, {
      syncPageSize: 100,
    }),
    // Existing usage unchanged (backward compatible)
    syncExpression(SmallModel, () => m => m.active.eq(true)),
  ]
});

Changes:

  • packages/datastore/src/types.ts — Add PerModelSyncConfig interface, extend SyncExpression type and syncExpression() function with optional third parameter
  • packages/datastore/src/datastore/datastore.ts — Extract per-model config in processSyncExpressions(), pass through amplifyConfig to SyncProcessor. Also replace non-null assertion on getModelDefinition() with an explicit null check and descriptive error
  • packages/datastore/src/sync/processors/sync.ts — Look up per-model overrides when computing limit and done condition in the sync pagination loop, falling back to global defaults via nullish coalescing

This is fully backward compatible — the third parameter is optional and the existing two-parameter syncExpression() signature works unchanged.

Issue #, if available

Closes #7310

Description of how you validated changes

  • Full DataStore test suite passes (yarn test in packages/datastore)
  • ESLint passes with 0 errors
  • TypeScript compilation passes
  • Added unit tests for syncExpression() with per-model config (4 tests in syncExpression.test.ts)
  • Added integration tests in connectivityHandling.test.ts verifying:
    • Per-model syncPageSize overrides the global default
    • Models without per-model config fall back to the global syncPageSize
    • Multiple models with different per-model syncPageSize values
    • Per-model maxRecordsToSync correctly caps the limit below syncPageSize
  • Validated in a consumer application by configuring per-model syncPageSize: 50 for a model with large records while other models sync at the default of 1000

Checklist

  • PR description included
  • yarn test passes
  • Unit Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)

Checklist for repo maintainers

  • Verify E2E tests for existing workflows are working as expected or add E2E tests for newly added workflows
  • New source file paths included in this PR have been added to CODEOWNERS, if appropriate

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@danrivett danrivett requested a review from a team as a code owner February 26, 2026 02:20
@changeset-bot
Copy link

changeset-bot bot commented Feb 26, 2026

🦋 Changeset detected

Latest commit: a192946

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@aws-amplify/datastore Minor
aws-amplify Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@danrivett danrivett force-pushed the datastore/feat/per-model-sync-page-size branch from 89f410d to d0892e8 Compare March 6, 2026 16:20
…iguration (aws-amplify#7310)

Allow syncExpression() to accept an optional third parameter with
per-model sync settings (syncPageSize, maxRecordsToSync) that override
global defaults. Models without per-model config continue to use the
global values. This lets users lower the page size for models with
large records (e.g., exceeding AppSync's 1MB response limit) without
penalizing sync performance for all other models.
@danrivett danrivett force-pushed the datastore/feat/per-model-sync-page-size branch from d0892e8 to a192946 Compare March 23, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

More flexibility with syncPageSize on DataStore

1 participant