how to cancel useInfiniteQuery's process of refetchOnWindowFocus

[rickvian]

I am using useInfiniteQuery, and has refetchOnWindowFocus: true,
when i focused to the window, it executes codes inside getNextPageParam,
during the sequence of fetching, user can choose to back to previous page, (we are not unmounting the component for some reason)

i tried to run queryClient.cancelQueries(['searchShipmentQuery']) when user clicked back to previous page it doesn't cancel the background fetching. It still running and console logging. (See console.log('getNextParam Triggered'))

when clicked back to previous page, i do set the enabled: false (by setting the searchPayload to empty, resolved to false)
yet the background fetching still running until it is out of pages,
how to cancel it?

const {
    data: searchShipmentQueryData,
    fetchNextPage,
    hasNextPage,
    isFetching,
    isFetchingNextPage,
    isLoading,
  } = useInfiniteQuery(
    ['searchShipmentQuery', searchPayload],
    ({ pageParam = 0 }) =>
      getShipmentsWithDimension(searchPayload, limit, pageParam),
    {
      keepPreviousData: false,
      refetchOnWindowFocus: !_.isEmpty(searchPayload),
      enabled: !_.isEmpty(searchPayload),
      retry: 0,
      staleTime: 0,
      select: ({ pageParams, pages }) => {
        const respondShipments = _.flatMap(pages, 'data.shipments')

        return {
          pageParams,
          pages: respondShipments.map(extendShipmentWithDimension),
          count: _.get(pages[0], 'data.count', 0),
        }
      },
      getNextPageParam: (lastPage, pages) => { //i want to stop sequence of this block executing when user pressing back.
        console.log('getNextParam Triggered')

        const currentTotalData = _.sum(
          _.map(pages, (page) => _.get(page, 'data.shipments', []).length)
        )
        const count = _.get(_.first(pages), 'data.count', 0)

        if (count === 0) {
          showSearchEmptyResultModal()
          setSearchPayload({}) 
        } else {

          if (!_.isEmpty(searchPayload)) {
            setPageType(PageType.Results) 
            if (currentTotalData < count) {
              return _.get(_.last(lastPage.data.shipments), 'shipment_id')
            }
          }
        }
      },
      onError: () => {
        setSearchPayload({}) //! this will disable the useInfiniteQuery
      },
    }
  )

here is code block executed if user clicked back button

 const handleBackToMain = () => {
    queryClient.cancelQueries(['searchShipmentQuery'])
    setSearchPayload({}) //!disable infiniteQuery
    setExtendedData({ shipments: [], count: 0 }) //reset table state
    setPageType(PageType.Filter) //set the page type back to filter page, instead of result page
  }

[TkDodo]

Judging from the code, cancelling a query should abort the page refetching. Can you create a minimal codesandbox reproduction that shows your issue please?