Add SegmentOrder utility class for per-query segment re-ordering #15591
Conversation
This allows client code that is querying sorted indexes to ensure that segments most likely to produce entries in a top-k search are queried first, allowing efficient skipping of other segments.
github-actions
bot
added
module:core/index
module:core/search
module:test-framework
labels
|
|
||
| @Override | ||
| public LeafCollector getLeafCollector(LeafReaderContext context) throws IOException { | ||
| String segmentId = context.toString().substring(19, 20); |
There was a problem hiding this comment.
This is something of a hack, but I couldn't think of an easier way to work out what the 'original' ordinal for a segment is. Less hacky solution ideas are welcome!
There was a problem hiding this comment.
Isn't ord a public field in LeafReaderContext?
There was a problem hiding this comment.
It is, but it is generated by BaseCompositeReader after the segment sorter has been applied. So the ord of the first segment in the leaves array is always 0, and we can't use it to see if the segments have been re-ordered.
There was a problem hiding this comment.
Ok, segment id - nevermind. Don't know. The substring hack seems... very fragile.
There was a problem hiding this comment.
I updated the test to just compare the order of leaves before and after the re-ordering. The collector checks were necessary when I was wiring everything through IndexSearcher but aren't needed now.
…che#15591) This allows client code that is querying sorted indexes to ensure that segments most likely to produce entries in a top-k search are queried first, allowing efficient skipping of other segments.
2 participants
This allows client code that is querying sorted indexes to ensure that segments most likely to produce entries in a top-k search are queried first, allowing efficient skipping of other segments.