fix(FileProvider): enable pagination for large folder enumeration #9286
+480
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes an issue where folders with 1,500+ files would stop updating in Finder when using macOS FileProvider/VFS. The bug was caused by pagination being disabled in the enumerateItems method in Enumerator.swift: - pageSettings was set to nil, preventing pagination requests to server - nextPage was unconditionally set to nil, discarding pagination tokens This caused only the first page of items (~50-100 files) to be detected during enumeration, while remaining files were never processed. Changes: - Enable pagination by passing proper pageSettings to readServerUrl() - Remove unconditional nextPage = nil that was discarding pagination - Add unit tests for large folder enumeration with pagination - Add integration test for RemoteChangeObserver with large folders Closes nextcloud#9283 Signed-off-by: Dev Admin <[email protected]>
ihsanmokhlisse
requested review from
Aiiaiiio,
camilasan,
i2h3,
mgallien and
nilsding
as code owners
Collaborator
|
Hello, please do not proceed for the moment. I will search for the reference if it was fixed already... |
Collaborator
Collaborator
|
the backend fix was merged. we will look into it |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #9283 - macOS FileProvider/VFS folders with 1,500+ files not updating
Problem
Folders with many files (1,500+) would stop updating in Finder when using macOS FileProvider/VFS. The folder would sync correctly initially but eventually stop showing new files, even after forcing a VFS sync.
Root Cause
The bug was in
Enumerator.swiftwhere pagination was being disabled:pageSettings: nil- prevented pagination requests to the servernextPage = nil- unconditionally discarded pagination tokens from server responsesThis caused only the first page of items (~50-100 files) to be enumerated, while all remaining files were never processed.
Solution
pageSettingstoreadServerUrl()nextPage = nilthat was discarding pagination tokensTesting
Added comprehensive unit tests following the existing testing strategy:
testLargeFolderChangeEnumerationWithPagination- Core bug fix verificationtestLargeFolderItemEnumerationWithPagination- Item enumeration with paginationtestIncrementalLargeFolderUpdate- Simulates the exact bug scenario (phone scan uploads)testVeryLargeFolderEnumerationWith150Items- Tests pagination across 3 pagestestVeryLargeFolderChangeEnumerationWith150Items- Change detection with many itemstestLargeFolderChangeRecognised- Integration test with RemoteChangeObserverChecklist