Skip to content

Skip predicate evaluation while seeking parquet iterators to a target row#7635

Open
mapno wants to merge 5 commits into
grafana:mainfrom
mapno:synciterator-seek-skip-filter-smaller
Open

Skip predicate evaluation while seeking parquet iterators to a target row#7635
mapno wants to merge 5 commits into
grafana:mainfrom
mapno:synciterator-seek-skip-filter-smaller

Conversation

@mapno

@mapno mapno commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What this PR does:

Skips predicate evaluation for values scanned past during SyncIterator.SeekTo.

SeekTo scans toward the seek target by calling next(), which evaluates the column predicate on every value — only for SeekTo to discard everything before the target.

When seeking, next skips values before the target without evaluating the filter, then filters normally from the target onward — one loop, no separate seek path. The target is kept as iterator scratch state and compared in place by an inlineable helper, keeping 32-byte RowNumber copies out of the hot loop. Results are identical.

  • Filtered metrics query ({span.http.host != `` && span.http.flavor=2} | rate() by (...)): −18.3%
  • Plain Next() iteration and unfiltered scans: unchanged
benchstat: vParquet5 block benchmarks (main vs PR, n=10)
                                                                                                        │ /tmp/final_macro_main.txt │       /tmp/final_macro_pr.txt       │
                                                                                                        │          sec/op           │   sec/op     vs base                │
BackendBlockTraceQL/spanAttValMatch-12                                                                                  1.969m ± 8%   1.959m ± 2%        ~ (p=0.684 n=10)
BackendBlockTraceQL/mixedValMixedMatchAnd-12                                                                            431.5µ ± 3%   433.8µ ± 2%        ~ (p=0.436 n=10)
BackendBlockTraceQL/struct-12                                                                                           59.77m ± 1%   59.71m ± 0%        ~ (p=1.000 n=10)
BackendBlockQueryRange/{span.http.host_!=_``_&&_span.http.flavor=`2`}_|_rate()_by_(span.http.flavor)-12                 62.50m ± 2%   51.10m ± 1%  -18.25% (p=0.000 n=10)
geomean                                                                                                                 7.505m        7.136m        -4.93%
benchstat: BenchmarkSyncIteratorSeekTo (main vs PR, n=8)
                                              │ /tmp/final_micro_main.txt │      /tmp/final_micro_pr.txt       │
                                              │          sec/op           │   sec/op     vs base               │
SyncIteratorSeekTo/int/nopred/stride=10-12                  1568.6µ ± 12%   908.4µ ± 2%  -42.09% (p=0.000 n=8)
SyncIteratorSeekTo/int/nopred/stride=500-12                 1438.4µ ± 11%   707.4µ ± 2%  -50.82% (p=0.000 n=8)
SyncIteratorSeekTo/int/between/stride=10-12                  1.479m ±  1%   1.009m ± 1%  -31.80% (p=0.000 n=8)
SyncIteratorSeekTo/int/between/stride=500-12                1273.9µ ±  2%   765.9µ ± 2%  -39.88% (p=0.000 n=8)
SyncIteratorSeekTo/string/regex/stride=10-12                 3.102m ±  1%   2.092m ± 3%  -32.57% (p=0.000 n=8)
SyncIteratorSeekTo/string/regex/stride=500-12               2873.3µ ±  2%   716.5µ ± 2%  -75.06% (p=0.000 n=8)
geomean                                                      1.833m         951.9µ       -48.06%

Which issue(s) this PR fixes:
N/A

Checklist

  • Tests updated
  • Documentation added
  • Changelog entry added under .chloggen/ (run make chlog-new, or make chlog-new FILENAME=<name> to override the default branch-name file; see .chloggen/README.md)

mapno added 3 commits July 16, 2026 18:10
… row

   SyncIterator.SeekTo scanned toward the seek target by calling next(),
   which evaluated the column predicate on every value and returned them
   one call at a time, only for SeekTo to discard everything before the
   target. The predicate's verdict on those values is irrelevant.

   The seek scan now compares row numbers first and only evaluates the
   predicate for values at/after the target, consuming the read buffer
   inline instead of one next() call per value.

   TraceQL benchmarks on a real vparquet5 block: geomean -5.5% (up to
   -9.2%) on search, -18.2% on a filtered metrics query. Unfiltered scans
   and plain Next() iteration unchanged.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes SyncIterator.SeekTo in pkg/parquetquery by avoiding per-value predicate evaluation for values that are scanned and discarded while seeking to a target row, improving performance for join-style TraceQL queries while keeping result semantics unchanged.

Changes:

  • Updates SyncIterator.SeekTo/next to skip predicate evaluation for values strictly before the seek target, while still applying the predicate at/after the target.
  • Refactors iterator buffer refill/page advancement logic into a dedicated fill() helper.
  • Adds targeted tests for SeekTo+predicate interaction and a benchmark modeling repeated forward seeks.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
pkg/parquetquery/iters.go Implements seek-aware scanning that bypasses predicate checks for pre-target values and factors buffer refill logic into fill().
pkg/parquetquery/iters_test.go Adds tests asserting predicate behavior around SeekTo and introduces a benchmark for forward-seek access patterns.
.chloggen/synciterator-seek-skip-filter.yaml Adds a changelog entry documenting the iterator seek performance enhancement.

@mapno
mapno marked this pull request as ready for review July 21, 2026 10:08
Copilot AI review requested due to automatic review settings July 21, 2026 10:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread pkg/parquetquery/iters.go
Comment on lines +882 to 888
if seeking && c.beforeSeekTarget() {
continue
}

if c.filter != nil && !c.filter.KeepValue(*v) {
continue
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've already tried this variant while developing the PR: flipping seeking mid-loop converts it from a loop-invariant (free, perfectly-predicted branch) into loop-carried state, which grows the frame and regressed seek microbenchmarks by ~14%

Comment thread pkg/parquetquery/iters.go

// Seek scratch state owned by next(). Copied from its arguments so the
// hot loop can compare in place without growing next's stack frame.
seekTo RowNumber

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be simplified by passing them into the BeforeSeekTo function? Instead of member vars?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants