feat(datastore): add per-model syncPageSize and maxRecordsToSync configuration#14732
Open
danrivett wants to merge 2 commits intoaws-amplify:mainfrom
Open
feat(datastore): add per-model syncPageSize and maxRecordsToSync configuration#14732danrivett wants to merge 2 commits intoaws-amplify:mainfrom
danrivett wants to merge 2 commits intoaws-amplify:mainfrom
Conversation
🦋 Changeset detectedLatest commit: a192946 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
89f410d to
d0892e8
Compare
…iguration (aws-amplify#7310) Allow syncExpression() to accept an optional third parameter with per-model sync settings (syncPageSize, maxRecordsToSync) that override global defaults. Models without per-model config continue to use the global values. This lets users lower the page size for models with large records (e.g., exceeding AppSync's 1MB response limit) without penalizing sync performance for all other models.
d0892e8 to
a192946
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
Adds an optional third parameter to
syncExpression()that allows configuringsyncPageSizeandmaxRecordsToSyncon a per-model basis. Models without per-model configuration continue to use the global defaults.Problem: DataStore has a single global
syncPageSize(default: 1000) that applies to all models during sync. When one model with large/denormalized records exceeds the AppSync 1MB response limit, users must lowersyncPageSizeglobally, causing a significant performance penalty for all other (smaller) models.Solution: Extend
syncExpression()with an optionalPerModelSyncConfigparameter:Changes:
packages/datastore/src/types.ts— AddPerModelSyncConfiginterface, extendSyncExpressiontype andsyncExpression()function with optional third parameterpackages/datastore/src/datastore/datastore.ts— Extract per-model config inprocessSyncExpressions(), pass throughamplifyConfigtoSyncProcessor. Also replace non-null assertion ongetModelDefinition()with an explicit null check and descriptive errorpackages/datastore/src/sync/processors/sync.ts— Look up per-model overrides when computinglimitand done condition in the sync pagination loop, falling back to global defaults via nullish coalescingThis is fully backward compatible — the third parameter is optional and the existing two-parameter
syncExpression()signature works unchanged.Issue #, if available
Closes #7310
Description of how you validated changes
yarn testinpackages/datastore)syncExpression()with per-model config (4 tests insyncExpression.test.ts)connectivityHandling.test.tsverifying:syncPageSizeoverrides the global defaultsyncPageSizesyncPageSizevaluesmaxRecordsToSynccorrectly caps the limit belowsyncPageSizesyncPageSize: 50for a model with large records while other models sync at the default of 1000Checklist
yarn testpassesChecklist for repo maintainers
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.