chore: add memory bank for pagination

Author: astandrik

memory-bank/activeContext.md

@@ -0,0 +1,27 @@
+# Active Context: Pagination Implementation
+
+## Current Focus
+
+The current focus is on understanding and documenting the pagination mechanism implemented in the YDB Embedded UI project, specifically the `PaginatedTable` component and its related files.
+
+### Key Components
+
+1. **PaginatedTable.tsx** - Main component that orchestrates the pagination
+2. **useScrollBasedChunks.ts** - Custom hook that determines which chunks are active based on scroll position
+3. **TableChunk.tsx** - Component that renders a chunk of data and manages its loading state
+4. **TableRow.tsx** - Component that renders individual table rows
+5. **types.ts** - Type definitions for the pagination components
+6. **constants.ts** - Constants used throughout the pagination implementation
+
+## Recent Changes
+
+N/A - This is the initial documentation of the pagination implementation.
+
+## Open Questions/Issues
+
+1. How does the pagination implementation handle very large datasets (10,000+ rows)?
+2. Are there any performance optimizations that could be applied to the current implementation?
+3. How does the pagination interact with filtering and sorting?
+4. How could this implementation be improved for accessibility?
+
+[2025-04-27 13:07:52] - Initial documentation of pagination active context

memory-bank/decisionLog.md

@@ -0,0 +1,109 @@
+# Decision Log: Pagination Implementation
+
+## [2025-04-27 13:47:10] - Pure Chunk-Based Virtualization Approach
+
+**Decision**: The pagination system implements purely chunk-based virtualization without any row-based virtualization.
+
+**Context**: In web UI development, there are common approaches to efficiently rendering large datasets:
+
+- Chunk-based virtualization (implemented in this project): Only rendering chunks of rows that are visible
+- Row-based virtualization (alternative approach, not implemented): Selectively rendering individual rows based on visibility
+
+**Analysis of Current Implementation**:
+The implementation uses exclusively chunk-based virtualization:
+
+- In `useScrollBasedChunks.ts`, the calculations only determine which chunks are visible:
+
+  ```typescript
+  // Lines 49-53
+  const start = Math.max(Math.floor(visibleStart / rowHeight / chunkSize) - overscanCount, 0);
+  const end = Math.min(
+    Math.floor(visibleEnd / rowHeight / chunkSize) + overscanCount,
+    Math.max(chunksCount - 1, 0),
+  );
+  ```
+
+- In `TableChunk.tsx`, when a chunk is active, it renders ALL rows in that chunk:
+
+  ```typescript
+  // Lines 136-144
+  return currentData.data.map((rowData, index) => (
+      <TableRow
+          key={index}
+          row={rowData as T}
+          columns={columns}
+          height={rowHeight}
+          getRowClassName={getRowClassName}
+      />
+  ));
+  ```
+
+- There is no code that selectively renders individual rows based on their visibility
+
+**Rationale**:
+
+- Simpler implementation and maintenance
+- More efficient data fetching by requesting groups of rows at once
+- Better compatibility with standard HTML table structure
+- Reduced complexity in handling scroll events
+
+**Implications**:
+
+- All rows in an active chunk are rendered, even if only some are visible
+- Works well for moderate-sized chunks but may be less optimal for very large chunks
+
+## [2025-04-27 13:47:10] - Custom Hook for Scroll Tracking
+
+**Decision**: Implemented scroll tracking logic in a separate custom hook (`useScrollBasedChunks`).
+
+**Context**: The pagination system needs to track scroll position to determine which chunks should be rendered.
+
+**Rationale**:
+
+- Separation of concerns: Isolates scroll tracking logic from rendering logic
+- Reusability: The hook can be used in different table implementations
+- Testability: Makes it easier to test the scroll tracking logic independently
+- Maintainability: Simplifies the main component by extracting complex logic
+
+**Implications**:
+
+- Requires careful management of dependencies and re-renders
+- Introduces a level of indirection that might make the code flow harder to follow
+
+## [2025-04-27 13:47:10] - Independent Chunk Loading
+
+**Decision**: Each chunk independently manages its own loading state and data fetching.
+
+**Context**: When scrolling through a large dataset, multiple chunks may need to be loaded at different times.
+
+**Rationale**:
+
+- Parallel loading: Multiple chunks can load simultaneously
+- Fault isolation: Errors in one chunk don't affect others
+- Progressive rendering: Chunks can render as soon as their data is available
+- Better user experience: Visible parts of the table appear faster
+
+**Implications**:
+
+- Requires careful management of API requests to avoid overwhelming the server
+- May result in duplicated requests if not properly integrated with a caching mechanism
+- Needs proper error handling at the chunk level
+
+## [2025-04-27 13:47:10] - RTK Query Integration
+
+**Decision**: Use Redux Toolkit Query for data fetching in chunks.
+
+**Context**: The pagination system needs to fetch data for each chunk efficiently.
+
+**Rationale**:
+
+- Built-in caching: Avoids duplicate requests for the same data
+- Automatic refetching: Supports automatic refresh intervals
+- Loading and error states: Provides standardized ways to track loading and error states
+- Cancellation: Supports cancelling requests when components unmount
+
+**Implications**:
+
+- Creates a dependency on Redux and RTK Query
+- Requires proper configuration of query parameters for efficient caching
+- Adds complexity to the application's state management

memory-bank/productContext.md

@@ -0,0 +1,43 @@
+# Product Context: YDB Embedded UI Pagination
+
+## Overview
+
+The YDB Embedded UI is a web interface for managing and interacting with YDB (Yandex Database) instances. As a database management tool, it frequently needs to display large datasets such as tables with thousands of rows, query results, logs, and other paginated information.
+
+## Pagination Requirements
+
+### Core Requirements
+
+1. **Performance**: Must efficiently handle large datasets (potentially thousands of rows) without degrading UI responsiveness
+2. **Progressive Loading**: Should load data incrementally as needed, rather than all at once
+3. **Smooth Scrolling**: Should provide a smooth scrolling experience without visible jumps or layout shifts
+4. **Error Resilience**: Should handle fetch errors gracefully and allow for retries
+5. **Visual Feedback**: Should provide clear loading states and empty states
+
+### User Experience Considerations
+
+1. **Familiar Interface**: Should feel like a standard table with traditional scrolling behavior
+2. **Responsive**: Should adapt to different screen sizes and container dimensions
+3. **Sortable Columns**: Should support sorting data by different columns
+4. **Filterable Data**: Should support filtering data based on various criteria
+5. **Accessibility**: Should be usable with keyboard navigation and screen readers
+
+## Technical Context
+
+The pagination system is used across multiple areas of the application where large datasets need to be displayed:
+
+1. Database tables and query results
+2. Logs and diagnostic information
+3. Monitoring data and metrics
+4. Configuration and settings listings
+
+The implementation needs to work well with the React-based frontend architecture and integrate with the application's data fetching mechanisms, state management (Redux), and UI component library.
+
+## Related Systems
+
+- **Data Fetching**: Integrates with RTK Query for API requests
+- **State Management**: Uses Redux for state management
+- **UI Components**: Built on the project's component library
+- **Filtering and Sorting**: Works with the application's filtering and sorting mechanisms
+
+[2025-04-27 13:38:20] - Initial documentation of pagination product context

memory-bank/progress.md

@@ -0,0 +1,46 @@
+# Progress Log: Pagination Analysis
+
+## Analysis and Documentation Progress
+
+### [2025-04-27 13:51:05] - Initial Analysis of PaginatedTable Components
+
+**Status**: Completed
+
+**Tasks Completed**:
+
+- Analyzed core pagination components:
+  - PaginatedTable.tsx
+  - TableChunk.tsx
+  - TableRow.tsx
+  - useScrollBasedChunks.ts
+  - constants.ts
+  - types.ts
+- Documented the main pagination patterns in systemPatterns.md
+- Created product context documentation in productContext.md
+- Documented key architectural decisions in decisionLog.md
+
+**Key Findings**:
+
+- The pagination implementation uses a chunk-based virtualization approach
+- Each chunk manages its own loading state and data fetching
+- The system efficiently handles large datasets by only rendering visible chunks
+- Integration with RTK Query provides caching and automatic refetching
+- The implementation includes proper error handling at the chunk level
+
+**Future Work**:
+
+- Consider performance testing with very large datasets (10,000+ rows)
+- Explore accessibility improvements for the pagination implementation
+- Investigate how the pagination implementation interacts with filtering and sorting
+- Document best practices for component customization and extension
+
+## Current Status
+
+The memory bank for the pagination implementation is now established with comprehensive documentation of:
+
+- The product context and requirements (productContext.md)
+- The system patterns and architecture (systemPatterns.md)
+- Key architectural decisions and their rationale (decisionLog.md)
+- Progress tracking and future work (progress.md)
+
+This documentation provides a solid foundation for understanding, maintaining, and potentially enhancing the pagination implementation in the YDB Embedded UI.

memory-bank/systemPatterns.md

@@ -0,0 +1,82 @@
+# System Patterns: Pagination Implementation
+
+## Scroll-Based Chunk Virtualization
+
+The YDB Embedded UI implements a scroll-based chunk virtualization pattern for table pagination, which is a common approach for efficiently rendering large datasets in web applications. The implementation is based on the following key concepts:
+
+### 1. Chunking
+
+The data is divided into "chunks" of rows (e.g., 20 rows per chunk by default), and only the chunks that are currently visible or near the viewport are rendered. This approach significantly reduces the DOM size and improves performance for large datasets.
+
+### 2. Scroll Position Tracking
+
+The system tracks the user's scroll position to determine which chunks should be rendered. As the user scrolls, new chunks come into view and are rendered, while chunks that move far from the viewport are unmounted to conserve memory and improve performance.
+
+### 3. Virtual Height Calculation
+
+To maintain proper scrollbar behavior, the system calculates the total virtual height of all rows (including those not currently rendered) and applies it to container elements. This provides a consistent scrolling experience without actually rendering all rows.
+
+### 4. Chunk Loading Pattern
+
+The chunk loading mechanism is sophisticated and includes several key features:
+
+- **On-demand loading**: Each chunk loads its data only when it becomes "active" (visible in the viewport or within the overscan area)
+- **Debounced fetching**: Data fetching has a default timeout of 200ms to avoid unnecessary requests during rapid scrolling
+- **Loading placeholders**: While a chunk is loading, it displays skeleton loaders through the `LoadingTableRow` component to maintain correct layout and provide visual feedback
+- **Error handling**: Each chunk independently handles and displays errors using the `EmptyTableRow` with either a custom error renderer or a default `ResponseError` component, ensuring errors in one chunk don't affect others
+- **RTK Query integration**: Uses Redux Toolkit Query for data fetching, caching, and auto-refresh capabilities
+
+### 5. Chunk-Level Rendering
+
+Unlike some virtualization implementations that virtualize at the row level, this system virtualizes at the chunk level. When a chunk is active, all rows within that chunk are rendered. This approach simplifies the implementation while still providing good performance for reasonably sized chunks.
+
+## Component Communication Pattern
+
+The pagination implementation follows a hierarchical component communication pattern:
+
+1. **PaginatedTable** (Parent Component)
+
+   - Manages the overall state and configuration
+   - Provides context and props to child components
+   - Uses useScrollBasedChunks to determine which chunks are active
+
+2. **TableChunk** (Container Component)
+
+   - Receives a subset of data to render
+   - Manages loading states and data fetching
+   - Only renders content when isActive is true
+   - Maintains proper height to ensure correct scrollbar behavior
+
+3. **TableRow Components** (Presentational Components)
+   - **TableRow**: Renders a single row of data with actual content
+   - **LoadingTableRow**: Renders skeleton placeholders using the Skeleton UI component with consistent dimensions
+   - **EmptyTableRow**: Renders a message or error state with full table width (colSpan)
+   - All variants maintain consistent layout and height for smooth transitions
+
+## Performance Optimization Patterns
+
+1. **Throttled Scroll Handling**
+
+   - Scroll events are throttled to avoid excessive calculations
+   - Default throttle delay of 100ms balances responsiveness and performance
+
+2. **Debounced Data Fetching**
+
+   - Data fetching is debounced to prevent unnecessary API calls during rapid scrolling
+   - Default debounce timeout of 200ms
+
+3. **Memoization**
+
+   - Components use React.useMemo and typedMemo to prevent unnecessary re-renders
+   - Particularly important for TableChunk to avoid performance issues with large tables
+
+4. **Overscan**
+
+   - The system renders additional chunks beyond the visible area (overscan)
+   - This provides smoother scrolling by having content ready before it comes into view
+
+5. **Display Strategy**
+   - Non-active chunks use display: block to maintain proper height without rendering content
+   - Active chunks use display: table-row-group for proper table layout
+
+[2025-04-27 13:35:00] - Initial documentation of pagination system patterns