Revalidating infinite query pages one at a time instead of all at once

[christianjuth]

I'm using Tanstack Query persisted. When a user opens my app, I may have an infinite query with 10 pages. Instead of waiting for all 10 persisted pages to refresh, could you add a setting to update the cache as each individual page finishes fetching?

My reasoning:

Say you have a social media app. The feed consists of 10 persisted pages of 50 posts each. You launch the app and you start at the top of the feed. You don't really need to wait for all 10 pages to refresh before updating the first page. You can update the first page immediately, and the user gets to see those new posts quickly. And by the time they make it through the first 50 posts in page 1, the rest of the pages have been updated.

Currently, I'm forced to pick one of these non ideal solutions:

1. Wait a long time on app launch for all 10 pages to refresh
2. Truncate the pages in the cache to prevent > 2 (or some small number) pages from being persisted

Note: I chose 10 arbitrarily to demonstrate my point. Could be more than 10 pages if the user scrolls more than 500 posts in the feed.

Side effects

I understand that this ask could potentially lead to duplicate items in the feed. That's why I think this should be opt in behavior. I dedupe the items across all pages anyway, so duplicate items isn't that hard to solve.

Some extra context

This is one of a few issues I've run into with persisting Tanstack query. I've also run into API rate limiting issues since it revalidates all pages of all mounted infinitive queries app launch. This was an easy fix using Tanstack pacer to artificially throttle refetching the 10 pages. Unfortunately that makes waiting for all 10 pages to refresh an ever bigger problem. But updating the cache page by page would allow me to continue throttling the infinite queries.

Note: This throttled infinite query wrapper I wrote is open source, though a bit messy. I'm happy to explain more, if that context is helpful.