Skip to content

Conversation

@wschurman
Copy link
Member

@wschurman wschurman commented Feb 5, 2026

Why

This is the main goal of the stack that started in #407. It adds a paginated loader for entities for applications that use the knex postgres database adapter.

How

Many many iterations of claude.

The general principle is hoisted from Expo server application code: fetch limit + 1 entities to know if there are more pages.

This makes use of the sql loader added in #414 to do the actual page load. Exact implementation described in docblocks.

Next PRs:

  • Add includeTotal option and use postgres windows if possible SELECT *, COUNT(*) OVER() as total_count
  • Add search (ilike and trigram)

Test Plan

Full test coverage and example usage in tests.

@codecov
Copy link

codecov bot commented Feb 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (64712a7) to head (1218be7).

Additional details and impacted files
@@                                          Coverage Diff                                          @@
##           wschurman/02-09-fix_move_entityprivacyutils_back_into_core_package      #422    +/-   ##
=====================================================================================================
  Coverage                                                              100.00%   100.00%            
=====================================================================================================
  Files                                                                     108       108            
  Lines                                                                   15073     15616   +543     
  Branches                                                                 1320      1388    +68     
=====================================================================================================
+ Hits                                                                    15073     15616   +543     
Flag Coverage Δ
integration 24.94% <96.88%> (+2.64%) ⬆️
unittest 95.92% <65.75%> (-0.43%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@wschurman wschurman changed the base branch from wschurman/02-01-feat_add_sql_template_and_loader_method to graphite-base/422 February 6, 2026 19:37
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch from 5d121ae to aabff5d Compare February 6, 2026 19:38
@wschurman wschurman changed the base branch from graphite-base/422 to wschurman/02-06-fix_use_tselectedfields_for_knex_loader_order_by_method February 6, 2026 19:38
@wschurman wschurman changed the base branch from wschurman/02-06-fix_use_tselectedfields_for_knex_loader_order_by_method to graphite-base/422 February 6, 2026 20:26
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch from aabff5d to da0673d Compare February 6, 2026 20:26
@wschurman wschurman changed the base branch from graphite-base/422 to wschurman/02-06-fix_type_idfield_in_entityconfiguration_as_tidfield February 6, 2026 20:26
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch 4 times, most recently from 75f7914 to 4be6e84 Compare February 9, 2026 23:16
@wschurman wschurman force-pushed the wschurman/02-06-fix_type_idfield_in_entityconfiguration_as_tidfield branch from 6164b63 to fbce886 Compare February 9, 2026 23:28
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch 2 times, most recently from 586186d to 806058c Compare February 9, 2026 23:31
@wschurman wschurman force-pushed the wschurman/02-06-fix_type_idfield_in_entityconfiguration_as_tidfield branch 2 times, most recently from 29a3d8c to 0c6daeb Compare February 10, 2026 00:02
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch from 806058c to 11222bc Compare February 10, 2026 00:03
@wschurman wschurman changed the base branch from wschurman/02-06-fix_type_idfield_in_entityconfiguration_as_tidfield to graphite-base/422 February 10, 2026 03:01
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch from 11222bc to 33e8e6f Compare February 10, 2026 03:33
@wschurman wschurman changed the base branch from graphite-base/422 to wschurman/02-09-fix_move_entityprivacyutils_back_into_core_package February 10, 2026 03:33
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch 2 times, most recently from b347527 to 0b4f87d Compare February 10, 2026 04:02
@wschurman wschurman marked this pull request as ready for review February 10, 2026 04:07
@wschurman wschurman requested review from ide and quinlanj February 10, 2026 04:07
@wschurman wschurman changed the title feat!: Add paginated loader to entity-database-adapter-knex feat: Add paginated loader to entity-database-adapter-knex Feb 10, 2026
@wschurman wschurman force-pushed the wschurman/02-05-feat_add_paginated_loader_to_entity-database-adapter-knex branch from 0b4f87d to 1218be7 Compare February 11, 2026 03:30
Copy link
Member

@quinlanj quinlanj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but will need to account for the tricky Date case in the future, if we want to support that

private encodeCursor(fieldObject: Readonly<TFields>, fields: readonly (keyof TFields)[]): string {
const cursorData: Partial<TFields> = {};
for (const field of fields) {
cursorData[field] = fieldObject[field];
Copy link
Member

@quinlanj quinlanj Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for most fields but if we want to support the Date field it will be trickier because Postgres stores dates with a finer granularity, and it gets converted to the JS date when it comes out of the DB at a rougher granularity. This is a big problem when you rely on the Date field to order the objects.

We'll either have to encode the date as a string to preserve granularity or have an option to encode the cursor with just the entity ID and lookup the cursor in a WITH .... statement appended to the main SQL so we never expose the Date in JS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants