Skip to content

Conversation

@dolfinus
Copy link
Member

@dolfinus dolfinus commented Sep 22, 2025

Change Summary

Run and Operation models have primary key (id, created_at). When we have only filters like created_at >= :since, this leads to scanning all primary keys, because B-tree indexes are prefix-based.

As ids are UUIDv7, and created_at is always UUID internal timestamp, we can generate artificial filters like created_at >= since AND id >= :min_id, to limit range of ids to be scanned, improving query plan. Same for created_at <= :until and id >= :max_id.

This affects only some REST API endpoints, as consumer already access entries by specific id, not the date range.

Also, tables like Input, Output, ColumnLineage has primary key (created_at, id), and the only filter used here is created_at >= :since, so these remain unchanged.

Related issue number

Checklist

  • Commit message and PR title is comprehensive
  • Keep the change as small as possible
  • Unit and integration tests for the changes exist
  • Tests pass on CI and coverage does not decrease
  • Documentation reflects the changes where applicable
  • docs/changelog/next_release/<pull request or issue id>.<change type>.rst file added describing change
    (see CONTRIBUTING.rst for details.)
  • My PR is ready to review.

@dolfinus dolfinus self-assigned this Sep 22, 2025
@dolfinus dolfinus changed the base branch from develop to feature/DOP-28871 September 22, 2025 12:47
@dolfinus dolfinus added the ci:skip-changelog Add this label to skip changelog file check label Sep 22, 2025
@github-actions
Copy link

github-actions bot commented Sep 22, 2025

Coverage

Coverage Report •
FileStmtsMissCoverMissing
data_rentgen/db/repositories
   operation.py611181%132–133, 159–160, 163–164, 166–167, 169, 171, 179
   run.py902967%187–188, 190–191, 193, 195, 203, 206–207, 209–210, 217–218, 221–224, 227–228, 231, 246–248, 256, 273–275, 277–278
data_rentgen/utils
   uuid.py54983%111–112, 117–123
TOTAL696239794% 

@dolfinus dolfinus marked this pull request as ready for review September 22, 2025 12:56
Base automatically changed from feature/DOP-28871 to develop September 23, 2025 08:36
@dolfinus dolfinus merged commit 9ba07f0 into develop Sep 24, 2025
10 of 11 checks passed
@dolfinus dolfinus deleted the feature/DOP-28871-1 branch September 24, 2025 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:skip-changelog Add this label to skip changelog file check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants