Infinite scroll with mutations

[t-blackwell]

Hi, I'm in the process of rebuilding my site mycrossword.co.uk and I'm having trouble combining infinite scrolling with mutations. The home page of the site has a list of cards and they're loaded 12 cards per page when scrolling - see the existing home page.

I managed to get the infinite scroll to work after following this post but I found when I added in a mutation, like pressing a card's upvote button, it started to complicate things.

I'm currently holding page and crosswords array in state, every time you scroll to the bottom 12 new crossword records are added to the array and the page is incremented. When I press the upvote button it calls fetcher.submit, which hits the action and upvotes the crossword and then hits the loader and gets a page worth of crosswords. The only problem is that the loader has no knowledge of the page state therefore will always default to grabbing the first 12 crosswords. It may be the case that I've scrolled down to page 5 and I upvote crossword number 60 at the bottom. That means I can't overwrite all the existing data but only the first 12 of 60 records.

I can't help but feeling I'm missing something.

I tried to change tack by amending the way the API worked to always return all records i.e. page 1 returns 12 records, page 2 returns 24, page 3 return 36 and so on. That way I don't need to hold the crosswords in state and I can just use the loader data, and instead of calling fetcher.load('?page=2') I can use setSearchParams('?page=2') to refetch all the records (1-24) rather than just that page (12-24). The downside of this is that not only do you get an ugly URL but the scroll restoration jumps back to the top every time you load the next page of records.

Are either of these two approaches correct? Can anyone point me in the right direction?

Thanks in advance.

Tom

[MichaelDeBoey]

@t-blackwell We now have an infinite scroll example. Maybe that one can help you out?

[t-blackwell]

Thanks @MichaelDeBoey that's really useful. I'll take a look over the weekend and see if I can apply any of that code to my use case.

[taylorcantwell]

@t-blackwell Hi Tom, did you find a solution for this? I've run into the same issue.

[t-blackwell]

Unfortunately not - I did take a look at the examples but couldn't find anything of use.

[brophdawg11]

@t-blackwell I think this exact same use case just came up earlier this week and is captured in #2775. Can you confirm if that sounds like what you'll need?

If so we can probably close this and track through there. That fetcher revalidate flag is being worked on with the new react-router work, so would become available in remix once that's done and remix is updated to run on the new react router

[t-blackwell]

Hi @brophdawg11, sorry about the slow reply - I was away for the Easter weekend. Thanks for linking to the use case above, it does look very similar but with one difference - I don't have a child route. Looking at Ryan's example code, I really feel like I'm doing something wrong but I can't see what exactly. Here's a gist of the route in case anyone wanted to take look at what I'm playing around with.

If anyone has an example of infinite scrolling with a mutation, even if it's just using json-server, that would be super useful.

[gielcobben]

Did anyone find a solution for this?

[andreslemusm]

I think there's no good solution with the Remix APIs to build this kind of UX in a simple way. I built something similar a while ago and the best way I found was using React Query + fetching the loader directly (without fetcher.load or fetcher.submit)

[rajeshdavidbabu]

Makes sense. I am thinking of the same infact !

[meoyawn]

yeah I feel like when moving to remix we have to redesign the pagination to be

First Prev 1 2 3 4 5 Next Last

[rickbatka]

@t-blackwell I have the same problem. You're not doing anything wrong. The key to understanding what's going on here is "shouldRevalidate". When you trigger a POST or PUT, either through Form or Fetcher, remix reloads all the loaders on the page and rerenders any components whose new data has changed. Obviously, it thinks your scroller has changed, because it is only getting one page of results.

I hacked something together on my page where I override "ShouldRevalidate" to return false, and now the revalidation no longer happens and the list doesn't get reset when I trigger a POST...

BUT my page doesn't update if I change something.

I wonder if there is a solution here that does things the more "Remix"-y way, even if more wasteful:

Add a "?page=" parameter to the mix, but when loading pages, always load EVERYTHING up to and including the current end point.

So if there are 5 items per "page", when loading page 1, you get the first 5. When loading page 2, you get the first 10. And so on.

It would be important that the page number parameter be updated in the URL, not just in the fetcher.

When your revalidation happens, and you're on page 2, the loader gets all items 1-10 and determines the loaderdata matches what was last fetched, and doesn't destroy your list.

Of course it will be more wasteful to send all that data every time, but I can't think of another way to imlpement it that plays nicer with Remix, without moving to manual pagination like @meoyawn stated.

I wonder if anyone else thinks this would work?

[rajeshdavidbabu]

I have been dabbling with something similar, from what I can think off is that infinite scroll requires you to maintain your data in a client-state.

If you allow revalidation even fetchers that were used to load your page data will take part in revalidation and will cause weird issues.

There is a tradeoff here, if you block revalidation then your infinite list might be out of sync with server. If you allow ravalidation then it will revalidate more than what you want.

I would rather block remix revalidation and do my own revalidation using lets say react-query.

[akomm]

Fundamentally its a general infinite scroll & mutations problem.

Why do you implement it? So that you don't have to load all data at once. That means you don't reflect the state in the URL when you reload page fully, you start over. You could theoretically implement a query param to specify how many items should be loaded at once, but then someone could just blow it up by passing any arbitrary number.

So you can only do it with CLIENT State, where you fetch more and more. If you try to mix it with the URL state of remix, it does not make any sense. When you use fetcher / submit in remix, in principle it is close to the regular browser behavior of reloading the entire page on submission. Though its "smart" in augmenting it by just calling all the active loaders and update the prop / hook values.

You could "hack" around try to solve it with useFetcher / submit, but it wouldn't be sound entirely. But in remix you can always do the same thing, you would do without remix: Solve it entirely on the client side. Use react-query, swr or handle web fetch yourself, any choice that fits you best.

[rajeshdavidbabu]

Alright. I solved it a differently. And lets see if this helps people.

Infinite scroll most often times uses client-state and if you are using client-state then we need to ensure that this view is not revalidated at all.

In my app https://www.gitposter.dev I have implemented an infinite scroll, but on actions fired I am using "shouldRevalidate" to block my infinite-list from revalidation. Instead I only update the list-item using useRouteLoaderData hook.

Example:

/gitposts -> infinite list of posts
/gitposts/:postId -> Post view that fires actions like/comment.

Once the action is fired and the /gitposts/:postId gets revalidated, I ensure that /gitposts is not revalidated using "shouldRevalidate" and instead only update the list-item of the /gitposts view using useRouteLoaderData("/gitposts/:postId").

The full code is here https://github.com/rajeshdavidbabu/remix-supabase-social/blob/master/app/routes/resources.like.tsx#L25

[akomm]

Yes. As explained here: #2501 (comment) its best to keep it client state fully. As long as the infinite-scroll data is not derived from url / server state, you should be fine for any reload. For example if you have an input state and some reload happens, but the input is still rendered, the state is not lost. You only lose it if it was somehow derived from the previously mentioned things. So normally you should not even need "shouldRevalidate". But obviously edge-cases might exist.

[rajeshdavidbabu]

In my case, I use fetcher.load to populate my local-state. During revalidation the last loaded fetcher.load url would fire-again. So "shouldRevalidate" is required if we ever user fetcher.load.

[akomm]

Not saying you don't need it, but maybe your case requires something like a use ref, memo or state with a lazy initialize function?

Example with useState:

const loaderData = useLoaderData()
const [localState, setLocalState] = useState(() => /* initialize using loaderData */)

[akomm]

More sophisticated outlining of the idea:

const {initialItems} = useLoaderData()
const [items, setItems] = useState(initialItems) // in this case no lazy init needed
const fetchMore = useCallback(() => { // evtl. you can also pass offset / after as parameter
  /* error handling omitted */
  fetch("...", {/*...*/}).then(res => res.json()).then(({items}) => {
    setItems(prev => [...prev, ...items])
  })
}, []) // or dependency, like after parameter or however you handle load more offset

[rajeshdavidbabu]

I see your point, from what I have checked, in anycase /gitposts would revalidate if there is an action fired from /gitposts/:postId page. Its not just the fetcher.load ! Sorry for the miscommunication.

Skipping revalidation for that page on that specific action has been the only thing that worked for me.

[gustavopch]

I think there's a much easier solution that I learned from @sergiodxa.

• Make a resource route that returns a page of crosswords.
• Delegate loading and rendering the crosswords of each page to a separate <Page> component.
• Make your route component render as many <Page>s as needed.

Pseudocode:

// routes/resources/crosswords.ts
export loader = async ({ request }) => {
  const searchParams = new URL(request.url).searchParams
  const page = searchParams.get('page') ?? 1
  const crosswords = await loadCrosswords({ page })
  return json(crosswords)
}

// routes/crosswords.ts
export default function Component() {
  const [visiblePages, setVisiblePages] = useState([1])
  // TODO: Write the logic to append/prepend the next/previous number
  // to `visiblePages` as you reach the bottom/top of the document.
  return (
    <>
      <h1>Crosswords</h1>
      {visiblePages.map(page => (
        <Page page={page} />
      ))}
    </>
  )
}

export const Page = ({ page }) => {
  const fetcher = useFetcher()
  useEffect(() => {
    fetcher.load(`/resources/crosswords?page=${page}`)
  }, [])
  return (
    <ul>
      {(fetcher.data ?? []).map(crossword => (
        <li>{crossword.text}</li>
      )}
    </ul>
  )
}

In summary, when visiblePages goes from [1] to [1, 2], it will render a new <Page> which will fetch the second page and display. The first page will be intact, won't reload, because its useEffect() won't run again.

So no need to deal with storing the data from previous pages manually.

[akomm]

Then if you mutate data you need etag or dete based caching if you want to update stale / drop optimistic in case of failure.

[gustavopch]

@rickbatka #7634 and #8092.

[rajeshdavidbabu]

Is it a remixy way of implementing a bi-directional virtual list ?

In anycase, bi-directional virtual lists that load one page at a time would solve this problem.

[gustavopch]

@rajeshdavidbabu Kind of, I'd say it's a way to let Remix manage most of the state, but it's agnostic to the way you decide to implement the virtual list.

[gnapse]

This looks interesting, but I tried it, and it has a problem: once you have a few of these pages mounted, and a revalidation is needed, yes, they'll all be updated by Remix. But you do not have control over in what order all those revalidation responses come in. So you end up with a UI that sometimes flickers or jumps a bit when revalidated.