Can't get global filtering to work with manual pagination

[nhowlandatl]

I'm trying to get global filtering working with manual pagination in react-table. I'm in the unfortunate position of inheriting an incomplete web app with lots of broken features that I'm tasked to fix, but I've hit a block on this issue. We have manual pagination enabled, which seems to work just fine switching between pages and viewing the table data. The problem is that global filtering (such as text search) is not working across the table data. It is only searching the current page of data visible in the table.

Global filtering kind of works when manual pagination is disabled, but this creates odd behavior where I'm not seeing all of the data in the table when toggling between pages or searching.

Here is the react-table component:

export interface TableProps {
  columns?: Column<any>[];
  data?: object[];
  rowClick?: any;
  isRowClick?: boolean;
  paginationColor?: string;
  paginationHoverColor?: string;
  buttonColor?: string;
  hasSearch?: boolean;
  hasExport?: boolean;
  handleCSVExport?: () => void;
  hasFilters?: boolean;
  filters?: TableFiltersProps;
  updateFilters?: (filters: TableFiltersProps) => void;
  /**
   * for handling server-side pagination
   */
  setCurrentPage?: (page: number) => void;
  initialPageIndex?: number;
  pageCount?: number;
  // =====
  customButton?: any;
  title?: string;
  pageSize?: number;
  hasBadgeFilters?: boolean;
  filterBadges?: any;
  searchOptions?: any;
  selectOptions?: any;
  tableOptions?: Omit<TableOptions<object>, 'columns' | 'data' | 'initialState'>;
}

export const Table = ({
  columns = [],
  data = [],
  rowClick,
  paginationColor = Theme.colors.primaries_000,
  paginationHoverColor = Theme.colors.primaries_neg100,
  buttonColor = Theme.colors.primaries_000,
  hasSearch,
  hasExport,
  handleCSVExport = () => {},
  filters = {},
  updateFilters,
  setCurrentPage,
  initialPageIndex,
  hasBadgeFilters,
  customButton,
  title,
  pageSize,
  pageCount,
  isRowClick = false,
  tableOptions
}: TableProps) => {
  const [showFilters, setShowFilters] = React.useState(false);
  const [openFilters, setOpenFilters] = React.useState(false);
  const [filterInputValues, setFilterInputValues] = React.useState({});

  const hasFilters = Object.keys(filters).length > 0;

  const tableInstance = useTable(
    {
      columns,
      data,
      pageCount,
      initialState: {
        pageIndex: initialPageIndex,
        pageSize: pageSize || 10
      },
      ...tableOptions,
      manualPagination: true
    },
    useGlobalFilter,
    useFilters,
    useSortBy,
    usePagination
  );

  const {
    getTableProps,
    getTableBodyProps,
    headerGroups,
    page,
    prepareRow,
    state,
    setGlobalFilter,
    gotoPage,
    nextPage,
    previousPage,
    allColumns,
    setFilter
  } = tableInstance;

  const { globalFilter, pageIndex } = state;

  useEffect(() => {
    setCurrentPage?.(pageIndex);
  }, [pageIndex, setCurrentPage]);

  const pages = Array.from(Array(pageCount || 1).keys());

  const cancelFilters = () => {
    setFilterInputValues({});
    allColumns.map((column) => {
      setFilter(column.id, undefined);
      return column;
    });
    setOpenFilters(false);
  };

  return (
    <>
      <TableWrapper>
        <Filter
          width={'296px'}
          height={'924px'}
          title={'Filters'}
          isShown={openFilters}
          cancelFilters={() => cancelFilters()}
          setOpenFilters={() => setOpenFilters(false)}>
          {allColumns.map((column) => {
            if (column.canFilter) {
              return (
                <FilterContentWrapper key={`column-filter-${column.id}`}>
                  <SubtitleM color={Theme.colors.neutrals_100}>{`${column.Header}`}</SubtitleM>
                  <div style={{ height: '8px', width: '100%' }} />
                  <TextInput
                    //@ts-ignore
                    value={filterInputValues[column.id] || ''}
                    onChange={(value: any) => {
                      const newFilterInputValues: any = {
                        ...filterInputValues
                      };
                      newFilterInputValues[column.id] = value;
                      setFilterInputValues(newFilterInputValues);
                      setFilter(column.id, value);
                    }}
                    isError={false}
                    placeholder={`${column?.Header}`}
                  />
                </FilterContentWrapper>
              );
            }
            return null;
          })}
        </Filter>
        <SearchWrapper>
          {title && (
            <TitleContainer>
              <TitleXS>{title}</TitleXS>
            </TitleContainer>
          )}
          {hasSearch && (
            <SearchContainer>
              <TextInput
                icon
                value={globalFilter}
                onChange={(value: any) => {
                  setGlobalFilter(value);
                }}
                img={<SearchIcon />}
                isError={false}
                placeholder={'Search'}
              />
            </SearchContainer>
          )}
          {(hasExport || hasFilters) && (
            <OptionalButtonContainer>
              {hasExport && (
                <ButtonContainer>
                  <Button
                    hoverColor={Theme.colors.primaries_neg100}
                    icon
                    outlineButton
                    color={buttonColor}
                    width={'116px'}
                    buttonText={'Export'}
                    img={<FolderDownloadIcon width={'15px'} />}
                    onClick={() => handleCSVExport()}
                  />
                </ButtonContainer>
              )}
              {hasFilters && (
                <ButtonContainer>
                  <Button
                    hoverColor={Theme.colors.primaries_neg100}
                    icon
                    outlineButton
                    color={buttonColor}
                    width={'116px'}
                    buttonText={'Filters'}
                    img={<AdjustmentsIcon width={'13.5px'} />}
                    onClick={() => {
                      if (hasBadgeFilters) {
                        setShowFilters(!showFilters);
                        return;
                      }
                      setOpenFilters(true);
                    }}
                  />
                </ButtonContainer>
              )}
            </OptionalButtonContainer>
          )}
          {<CustomButtonContainer>{customButton}</CustomButtonContainer>}
        </SearchWrapper>
        {showFilters && hasFilters && (
          <TableFilters filters={filters} setFilters={(fitler) => updateFilters?.(fitler)} />
        )}
        <TableStyle {...getTableProps()}>
          <thead>
            {headerGroups.map((headerGroup) => (
              <HeaderRow {...headerGroup.getHeaderGroupProps()}>
                {headerGroup.headers.map((column) => (
                  <TableH {...column.getHeaderProps(column.getSortByToggleProps())}>
                    <HeaderWrapper>
                      <HeaderText sorted={column.isSorted}>{column.render('Header')}</HeaderText>
                      {column.isSorted ? column.isSortedDesc ? <DownArrow /> : <UpArrow /> : ''}
                    </HeaderWrapper>
                  </TableH>
                ))}
              </HeaderRow>
            ))}
          </thead>
          <tbody {...getTableBodyProps()}>
            {page.map((row, rowIndex) => {
              prepareRow(row);
              return (
                <CellRow
                  {...row.getRowProps()}
                  isRowClick={isRowClick}
                  isEnd={rowIndex + 2 > page.length}
                  onClick={() => (isRowClick ? rowClick(row) : null)}>
                  {row.cells.map((cell) => {
                    return (
                      <TableData {...cell.getCellProps()}>
                        <ParagraphS
                          style={{ margin: 0 }}
                          color={
                            //@ts-ignore
                            cell?.column?.textColor
                              ? //@ts-ignore
                                cell?.column?.textColor
                              : Theme.colors.neutrals_100
                          }>
                          {cell.render('Cell')}
                        </ParagraphS>
                      </TableData>
                    );
                  })}
                </CellRow>
              );
            })}
          </tbody>
        </TableStyle>
      </TableWrapper>
      {pages.length > 1 ? (
        <PaginationWrapper>
          <Pagination
            pages={pages}
            pageCount={pageCount || 1}
            nextPage={nextPage}
            prevPage={previousPage}
            index={pageIndex}
            gotoPage={gotoPage}
            selectedColor={paginationColor}
            hoverColor={paginationHoverColor}
          />
        </PaginationWrapper>
      ) : null}
    </>
  );
};

And then in the component that generates the react-table:

interface ScreenWrapperProps {
  isLoading?: boolean;
}

const AssetAndUtilityColumnComponent = ({ total, cls, opn }: any) => {
  const kilowatt = total ? `${total} kw` : '0.0 kw';
  return (
    <ColumnContainer>
      <RowContainer style={{ marginBottom: '0.5rem', marginLeft: '0.25rem' }}>
        <ParagraphS style={{ marginRight: '0.5rem' }} color={Theme.colors.neutrals_000}>
          {cls || '00'}
        </ParagraphS>
        <Breakers />
      </RowContainer>
      <RowContainer style={{ marginBottom: '0.5rem', marginLeft: '0.25rem' }}>
        <ParagraphS style={{ marginRight: '0.5rem' }} color={Theme.colors.neutrals_000}>
          {opn || '00'}
        </ParagraphS>
        <Breakers isActive />
      </RowContainer>
      <RowContainer>
        <ParagraphS color={Theme.colors.neutrals_200}>{kilowatt}</ParagraphS>
      </RowContainer>
    </ColumnContainer>
  );
};

const SitesScreen = () => {
  const { filters, updateFilters, values } = useTableFilters({
    initialFilters: defaultSitesFilter
  });
  let filterValues: any = values;

  const [searchParams] = useSearchParams();
  const runGroupId = searchParams.get('runGroupId');
  if (runGroupId) {
    filterValues = { ...filterValues, run_group_ids: [parseInt(runGroupId, 10)] };
  }
  const [currentPage, setCurrentPage] = useState(0);
  const { data, isLoading, isSuccess } = useSites({
    page: currentPage,
    limit: 20,
    includeStatus: true,
    filter_by: filterValues
  });

  useEffect(() => {
    setCurrentPage(0);
  }, [filters]);

  const entries = data?.entries || [];
  const pageCount = data?.pagination?.last || 0;
  console.log(data?.pagination);

  const sitesDetailsColumns: any[] = useMemo(
    () => [
      {
        Header: 'Site',
        accessor: 'name',
        Cell: (cell: any) => {
          const { name, city, state, id } = cell.row.original;
          return (
            <div style={{ display: 'flex', flexDirection: 'column' }}>
              <ParagraphS>
                <NameLink to={ROUTES.SITES.DETAILS.NAV.OVERVIEW.path(id)}>{name}</NameLink>
              </ParagraphS>
              <ParagraphS color={Theme.colors.neutrals_200}>
                {city}
                {', '}
                {state}
              </ParagraphS>
            </div>
          );
        }
      },
      {
        Header: 'Utility',
        accessor: 'assets',
        Cell: (cell: any) => {
          const { site_status } = cell.row.original;
          const values = {
            cls: site_status.utilityBreakersClosed,
            opn: site_status.utilityBreakersOpen,
            total: site_status.utilityPower
          };
          return AssetAndUtilityColumnComponent(values);
        }
      },
      {
        Header: 'Asset',
        accessor: 'asset',
        Cell: (cell: any) => {
          const { site_status } = cell.row.original;
          const values = {
            cls: site_status.assetBreakersClosed,
            opn: site_status.assetBreakersOpen,
            total: site_status.assetPower
          };
          return AssetAndUtilityColumnComponent(values);
        }
      },
      {
        Header: 'Fuel',
        accessor: 'generators.fuel',
        Cell: (cell: any) => {
          const { site_status } = cell.row.original;
          return FuelAndDEFRolloverCell(site_status.fuelLevels);
        }
      },
      {
        Header: 'DEF',
        accessor: 'generators.DEF',
        Cell: (cell: any) => {
          const { site_status } = cell.row.original;
          return FuelAndDEFRolloverCell(site_status.defLevels);
        }
      },
      {
        Header: 'Alarms',
        accessor: 'alarms',
        Cell: (cell: any) => {
          const { site_status } = cell.row.original;
          const alarms = [
            {
              code: `C ${site_status.criticalAlarms}`,
              text: `Critical ${site_status.criticalAlarms}`
            },
            {
              code: `NC ${site_status.nonCriticalAlarms}`,
              text: `Non Critical ${site_status.nonCriticalAlarms}`
            }
          ];
          const colors = [Theme.colors.danger_000, Theme.colors.neutrals_200];
          return TwoColumnStackedCard(
            alarms,
            <Tooltip
              textColor={Theme.colors.danger_000}
              position={'bottom'}
              graph={false}
              graphLineColor={'none'}
              text={
                <div style={{ display: 'flex', flexDirection: 'column' }}>
                  <ParagraphS>{alarms[0].text}</ParagraphS>
                  <ParagraphS>{alarms[1].text}</ParagraphS>
                </div>
              }>
              <InformationCircleIcon
                width={'18px'}
                height={'18px'}
                color={Theme.colors.danger_000}
              />
            </Tooltip>,
            colors
          );
        }
      },
      {
        Header: 'Activity',
        accessor: 'activity.state',
        Cell: (cell: any) => {
          const { site_status } = cell.row.original;
          const categoryName = site_status.activeEvent?.category_name;
          const subCategoryName =
            site_status.activeEvent?.sub_category_name ||
            site_status.activeEvent?.solo_category_name;
          const colorsForStackedCardTexts = [Theme.colors.danger_000, Theme.colors.danger_000];
          return StackedInfoCard([categoryName, subCategoryName], colorsForStackedCardTexts);
        }
      }
    ],
    []
  );

  console.log(isSuccess);

  if (isLoading && !isSuccess) {
    return (
      <ScreenWrapper isLoading={isLoading}>
        <TailSpin
          height={'150'}
          width={'150'}
          color={Theme.colors.primaries_000}
          ariaLabel={'tail-spin-loading'}
          radius={'1'}
          wrapperStyle={{}}
          wrapperClass={''}
          visible
        />
      </ScreenWrapper>
    );
  }

  return (
    <ScreenWrapper>
      <TitleS style={{ marginBottom: '1.5rem' }}>{'Sites'}</TitleS>
      <Table
        hasSearch
        hasFilters
        hasExport
        pageSize={10}
        pageCount={pageCount}
        initialPageIndex={currentPage}
        setCurrentPage={setCurrentPage}
        handleCSVExport={() => {
          /**
           * Once filters are in state we can
           * pass those along here
           */
          localDownload('sites.csv', `/site?includeStatus=true&format=csv`);
        }}
        filters={filters}
        updateFilters={updateFilters}
        hasBadgeFilters
        data={entries || []}
        columns={sitesDetailsColumns}
      />
    </ScreenWrapper>
  );
};

export default SitesScreen;

[ChristianAgren]

Hello!

Global filtering kind of works when manual pagination is disabled, but this creates odd behavior where I'm not seeing all of the data in the table when toggling between pages or searching.

I'm not sure why you're not seeing some of the data with manual pagination being disabled. Are the columns being filtered (https://tanstack.com/table/v8/docs/api/features/filters?from=reactTableV7)? Or are you missing entire rows?

As I have understood the API, manual pagination should only be used when you're filtering/paginating in your backend (when fetching the data from your endpoints). If you're filtering on all of the data in the client, you should let the table paginate for you (manual pagination should be off).

[KevinVandy]

If you're using manualPagination, you're obviously also going to have to use manualFiltering. The table instance cannot filter based on data that is not passed to it. You'll have to do all of the filtering in the back-end too. And then apply pagination in the back-end logic, after filtering, sorting, etc.