Skip to content

Commit 4a2c3fd

Browse files
churromoralesshubhamsrkdevmsfroh
committed
Add Temporal Merge Policy for time-series data (#15620)
Introduces TemporalMergePolicy, a new merge policy designed for time-series workloads where documents contain a timestamp field. The policy groups segments into time windows and merges segments within the same window, but never merges segments across different time windows. This preserves temporal locality and improves query performance for time-range queries. --------- Co-authored-by: Shubham Sharma <shubham234727@gmail.com> Co-authored-by: Michael Froh <msfroh@apache.org>
1 parent 876e77a commit 4a2c3fd

File tree

4 files changed

+1665
-17
lines changed

4 files changed

+1665
-17
lines changed

lucene/CHANGES.txt

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -155,28 +155,30 @@ New Features
155155
`Lucene104HnswScalarQuantizedVectorsFormat` replaces the now legacy `Lucene99HnswScalarQuantizedVectorsFormat`
156156
(Trevor McCulloch)
157157

158-
* GITHUB#15271, GITHUB#15353: Extend `Lucene104ScalarQuantizedVectorsFormat` and `Lucene104HnswScalarQuantizedVectorsFormat` to
159-
allow asymmetric quantization. The initially supported bits are single bit with 4 bit queries. This is a replacement
160-
for the now legacy `Lucene102HnswBinaryQuantizedVectorsFormat` and `Lucene102BinaryQuantizedVectorsFormat`. To use
161-
the new format for asymmetric binary quantization,
162-
`Lucene104HnswScalarQuantizedVectorsFormat(ScalarEncoding.SINGLE_BIT_QUERY_NIBBLE, int, int)`
163-
(Ben Trent)
164-
165-
* GITHUB#15564: Further extend `Lucene104ScalarQuantizedVectorsFormat` and `Lucene104HnswScalarQuantizedVectorsFormat` to
166-
allow asymmetric quantization with 2 bits and 4 bit queries. This can provide increased recall over single bit
167-
with little additional storage cost and latency.
158+
* GITHUB#15271, GITHUB#15353: Extend `Lucene104ScalarQuantizedVectorsFormat` and `Lucene104HnswScalarQuantizedVectorsFormat` to
159+
allow asymmetric quantization. The initially supported bits are single bit with 4 bit queries. This is a replacement
160+
for the now legacy `Lucene102HnswBinaryQuantizedVectorsFormat` and `Lucene102BinaryQuantizedVectorsFormat`. To use
161+
the new format for asymmetric binary quantization,
162+
`Lucene104HnswScalarQuantizedVectorsFormat(ScalarEncoding.SINGLE_BIT_QUERY_NIBBLE, int, int)`
168163
(Ben Trent)
169164

170-
* GITHUB#15415: Add fallback support to Lucene104ScalarQuantizedVectorsFormat getFloatVectorValues when there are
171-
no full-precision vectors present (Pulkit Gupta)
165+
* GITHUB#15564: Further extend `Lucene104ScalarQuantizedVectorsFormat` and `Lucene104HnswScalarQuantizedVectorsFormat` to
166+
allow asymmetric quantization with 2 bits and 4 bit queries. This can provide increased recall over single bit
167+
with little additional storage cost and latency.
168+
(Ben Trent)
169+
170+
* GITHUB#15415: Add fallback support to Lucene104ScalarQuantizedVectorsFormat getFloatVectorValues when there are
171+
no full-precision vectors present (Pulkit Gupta)
172+
173+
* GITHUB#15468: Add support for `@SuppressAssertingFormats` annotation for fine-grained control over `AssertingCodec` formats (Prudhvi Godithi)
174+
175+
* GITHUB#15518: Add support for post-collection faceting to the new faceting API in the sandbox module. (Egor Potemkin)
172176

173-
* GITHUB#15468: Add support for `@SuppressAssertingFormats` annotation for fine-grained control over `AssertingCodec` formats (Prudhvi Godithi)
177+
* GITHUB#15620: Add Temporal Merge Policy for time-series data
174178

175-
* GITHUB#15518: Add support for post-collection faceting to the new faceting API in the sandbox module. (Egor Potemkin)
179+
* GITHUB#14792: Introduced OffHeapQuantizedFloatVectorValues class to access float vectors when only quantized byte vectors are available in the index. (Pulkit Gupta)
176180

177-
* GITHUB#14792: Introduced OffHeapQuantizedFloatVectorValues class to access float vectors when only quantized byte vectors are available in the index. (Pulkit Gupta)
178-
179-
* GITHUB#15591: Add SegmentOrder helper class to re-order segments at query time to align with sorts. (Alan Woodward)
181+
* GITHUB#15591: Add SegmentOrder helper class to re-order segments at query time to align with sorts. (Alan Woodward)
180182

181183
Improvements
182184
---------------------

0 commit comments

Comments
 (0)