Add cursor-based pagination

[seb-jean]

This PR introduces cursor-based (keyset) pagination as an alternative to the existing offset-based Paginator.

Why cursor pagination?

Offset-based pagination has well-known performance issues with large datasets:

• OFFSET requires the database to scan and discard rows, making deep pages increasingly slow
• Results can be inconsistent when data is inserted/deleted between page requests

Cursor pagination solves these problems by:

• Using indexed columns (typically the ORDER BY columns) to seek directly to the next page
• Providing stable pagination even when underlying data changes

Implementation

The implementation consists of three main classes:

• Cursor: An immutable value object that encodes/decodes pagination state as a URL-safe base64 string
• CursorPaginator: The main paginator class implementing IteratorAggregate and Countable
• CursorWalker: A TreeWalkerAdapter that modifies the AST to inject cursor conditions and handle ORDER BY reversal for previous page navigation

Usage

  $query = $em->createQuery('SELECT p FROM Post p ORDER BY p.createdAt DESC, p.id DESC');

  $paginator = new CursorPaginator($query);
  $paginator->paginate(limit: 20, cursor: $cursor); // $cursor is a string that corresponds to $_GET['cursor']

  foreach ($paginator as $post) {
      // ...
  }

  $hasNextPage = $paginator->hasNextPage();
  $hasPreviousPage = $paginator->hasPreviousPage();
  $hasToPaginate = $paginator->hasToPaginate();
  $nextCursor = $paginator->getNextCursor();
  $previousCursor = $paginator->getPreviousCursor();

Navigation API

The paginator provides a complete API for building pagination UI:

Method	Header
hasNextPage(): bool	Whether more results exist after the current page
hasPreviousPage(): bool	Whether results exist before the current page
hasToPaginate(): bool	Whether pagination controls should be displayed (has next or previous)
getNextCursor(): ?Cursor	Returns the cursor for the next page, or null if none
getPreviousCursor(): ?Cursor	Returns the cursor for the previous page, or null if none

Requirements

• The query must have an ORDER BY clause (cursor pagination requires deterministic ordering)
• For best performance, ORDER BY columns should be indexed

Notes

• Supports multiple ORDER BY columns with proper tie-breaking logic
• Handles both forward (next) and backward (previous) navigation
• Works with both Query and QueryBuilder

I hope you like this PR :)

[theofidry]

Thank you a lot for the proposal!

I've only had a cursory look, I couldn't really get to try it out. Besides the questions in the review, I wonder if it wouldn't make more sense to return an Item<T> = { value: T, cursor: Cursor } rather than directly the value T. If you expose those results to the API, you would otherwise need to do something like:

$paginator = new CursorPaginator($query);
$paginator->paginate(limit: 20, cursor: $cursor); // $cursor is a string that corresponds to $_GET['cursor']

$items = array_map(
  static fn (Post $post) => [
    'cursor' => $paginator->getCursorForItem($post),
    'value' => $post,
  ],
  iterator_to_array($paginator),
);

I notice in our paginator we have:

::getItems()
::getValues()
::getOffsets() (we named your "cursor" as `"offset")

so I guess there is multiple cases

[theofidry]

How do you handle the direction with this API? I.e. "I have this cursor, I want the page of X items before/after this cursor"

[seb-jean]

The direction is encoded within the cursor itself. The cursor is a Base64-encoded JSON containing the pivot values and a _isNext boolean flag. When decoded, this flag tells the paginator whether to fetch the next or previous page. The paginator exposes getNextCursor() (produces a cursor with _isNext: true) and getPreviousCursor() (produces a cursor with _isNext: false), so the direction is baked into the opaque cursor string — no additional parameter needed.

For example I have the following cursor eyJhLmNyZWF0ZWRBdCI6IjIwMTktMTEtMDEgMTk6MTU6MzciLCJhLmVtYWlsIjoiaGVucmkyNEBtdW5vei5vcmciLCJhLmlkIjo5NDIsIl9pc05leHQiOmZhbHNlfQ.

The associated JSON is:

{
   "a.createdAt":"2019-11-01 19:15:37",
   "a.email":"henri24@munoz.org",
   "a.id":942,
   "_isNext":false
}

[theofidry]

Is that correct?

I may not have correctly understood neither am I deeply familiar with cursor based paginations. I've merely employed it in one of my projects, I admit without worrying too much what was the standard.

One usage we had was for a given cursor, we had to collect items both before and after. It seems a simple use case but I struggle to see how you would achieve this if you make the direction implicit in the API and enforced into the cursor value.

[seb-jean]

Yes, that's correct. _isNext allows you to specify the pagination direction.

So if in your JSON the pagination direction is:

• true, you will be able to see the next items
• false, you will be able to see the previous items

[theofidry]

but if that's part of the generated cursor you are hardcoding that result with that direction, which seems wrong.

Let's say you have your cursor for an arbitrary blog post:

query {
  posts(query: $query, pageSize: $pageSize, after: $after) { ... } // will only work if the direction for which the cursor ways used is the same
  posts(query: $query, pageSize: $pageSize, before: $before) { ... } // if the previous works, the latter will not
}

Or am I missing something here?

[seb-jean]

In your example, what exactly do $after and $before represent, please?

[seb-jean]

Thank you a lot for the proposal!

I've only had a cursory look, I couldn't really get to try it out. Besides the questions in the review, I wonder if it wouldn't make more sense to return an Item<T> = { value: T, cursor: Cursor } rather than directly the value T. If you expose those results to the API, you would otherwise need to do something like:

$paginator = new CursorPaginator($query);
$paginator->paginate(limit: 20, cursor: $cursor); // $cursor is a string that corresponds to $_GET['cursor']

$items = array_map(
  static fn (Post $post) => [
    'cursor' => $paginator->getCursorForItem($post),
    'value' => $post,
  ],
  iterator_to_array($paginator),
);

I notice in our paginator we have:

::getItems()
::getValues()
::getOffsets() (we named your "cursor" as `"offset")

so I guess there is multiple cases

I added this improvement.