Throttling for parallel queries

[williamjulianvicary]

I have a dashboard that loads up-to many hundreds of rows of data, once loaded these rows then need to be hydrated via an API request.

I'd like to throttle the requests to the endpoint to N concurrent requests, i.e, say, 3 at a time as the endpoint this hits actually makes a backend request to other endpoints which impose rate limits.

I don't believe this is currently possible, right?

[TkDodo]

no, I don't think it's possible. You would need to do this on the actual api layer that makes the requests.

[williamjulianvicary]

It would be possible without React Query via a throttle of some description. I appreciate I could likely do this within the HTTP callback instead of using React Query to handle the state of each request but I was hoping for something a little more built-in so each row can render once it's ready but I appreciate that's unlikely to be possible.

However, thinking this through a little, would a centralised throttle work with child components, so (pseudo). I'm not totally sure how you'd go about not breaking reactivity within the usePost() but conceptually dependent queries may be able to be used to achieve a throttle?

function usePosts() {
  return useQuery(["posts"], async () => {
    const { data } = await axios.get(
      "https://jsonplaceholder.typicode.com/posts"
    );
    return data;
  });
}

const getPostById = async (id) => {
  const { data } = await axios.get(
    `https://jsonplaceholder.typicode.com/posts/${id}`
  );
  return data;
};

function usePost(postId, isEnabled) {
  return useQuery(["post", postId], () => getPostById(postId), {
    enabled: !!postId && isEnabled
  });
}

function Posts(){
  const { status, data, error, isFetching } = usePosts();
  const throttle = useThrottle('throttle-id', maxConcurrency: 5); // concept, would need building
  return <>{map.data => (post) => (<Post postId={post.id} throttle={throttle}/>)}</>
}

function Post({postId, throttle}) {
  const { status, data, error, isFetching } = usePost(postId, throttle.acquireLock(postId));
  return [...output post...]
}

[TkDodo]

Could a library like this help?

• https://github.com/tannerlinsley/swimmer

I'm guessing you could:

• create a custom pool
• have each query function add a task to the pool and await it with await pool.add(...)

would be cool to see if that works :)

[pavitra-infocusp]

Tanstack now has a new Pacer library which provides Async Queues. Here's an example of using Queue Pacer with Query - https://tanstack.com/pacer/latest/docs/framework/react/examples/react-query-queued-prefetch. However, I couldn't get how this really works with Query, so I gave some thought and I really happy with what I came up with!

function useTokenDetails(addresses: string[]) {
  return useQueries({
    queries: addresses.map((address, i) => ({
      queryKey: ['token-details', address],
      queryFn: async () => {
        // simulate batched requests
        const concurrency = 2;
        const delay = Math.floor(i / concurrency) * 1000;
        console.debug('Queued', { address, delay });
        await new Promise(resolve => setTimeout(resolve, delay));
        console.debug('Fetched', { address, delay });

        // Perform the query
        console.debug('Fetching token details for:', address);
        return await getTokenDetails(address);
      },
    })),
  });
}