Skip to content

Commit 2550eb2

Browse files
authored
Speed up flushing of live docs. (#14998)
This uses `Bits#applyMask` (introduced in #14134) to speed up flushing live docs, instead of checking bits one by one.
1 parent 6b12055 commit 2550eb2

File tree

2 files changed

+18
-11
lines changed

2 files changed

+18
-11
lines changed

lucene/CHANGES.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,8 @@ Optimizations
129129
---------------------
130130
* GITHUB#15140: Optimize TopScoreDocCollector with TernaryLongHeap for improved performance over Binary-LongHeap. (Ramakrishna Chilaka)
131131

132+
* GITHUB#14998: Speed up flushing of live docs. (Adrien Grand)
133+
132134
Bug Fixes
133135
---------------------
134136
* GITHUB#14161: PointInSetQuery's constructor now throws IllegalArgumentException

lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90LiveDocsFormat.java

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -138,18 +138,23 @@ public void writeLiveDocs(
138138
}
139139

140140
private int writeBits(IndexOutput output, Bits bits) throws IOException {
141-
int delCount = 0;
142-
final int longCount = FixedBitSet.bits2words(bits.length());
143-
for (int i = 0; i < longCount; ++i) {
144-
long currentBits = 0;
145-
for (int j = i << 6, end = Math.min(j + 63, bits.length() - 1); j <= end; ++j) {
146-
if (bits.get(j)) {
147-
currentBits |= 1L << j; // mod 64
148-
} else {
149-
delCount += 1;
150-
}
141+
int delCount = bits.length();
142+
// Copy bits in batches of 1024 bits at once using Bits#applyMask, which is faster than checking
143+
// bits one by one.
144+
FixedBitSet copy = new FixedBitSet(1024);
145+
for (int offset = 0; offset < bits.length(); offset += copy.length()) {
146+
int numBitsToCopy = Math.min(bits.length() - offset, copy.length());
147+
copy.set(0, copy.length());
148+
if (numBitsToCopy < copy.length()) {
149+
// Clear ghost bits
150+
copy.clear(numBitsToCopy, copy.length());
151+
}
152+
bits.applyMask(copy, offset);
153+
delCount -= copy.cardinality();
154+
int longCount = FixedBitSet.bits2words(numBitsToCopy);
155+
for (int i = 0; i < longCount; ++i) {
156+
output.writeLong(copy.getBits()[i]);
151157
}
152-
output.writeLong(currentBits);
153158
}
154159
return delCount;
155160
}

0 commit comments

Comments
 (0)