Skip to content

Commit b699981

Browse files
authored
Allow FST builder to use different writer (#12543) (#12624)
* Move size() to FSTStore * Remove size() completely * Allow FST builder to use different DataOutput * access BytesStore byte[] directly for copying * Rename BytesStore * Change class to final * Reorder methods * Remove unused methods * Rename truncate to setPosition() and remove skipBytes() * Simplify the writing operations * Update comment * remove unused parameter * Simplify BytesStore operation * tidy code * Rename copyBytes to writeTo * Simplify BytesStore operations * Embed writeBytes() to FSTCompiler * Fix the write bytes method * Remove the default block bits constant * add assertion * Rename method parameter names * Move reverse to FSTCompiler * Revert setPosition call * Address comments * Return immediately when writing 0 bytes * Add comment & * Rename variables * Fix the compile error * Remove isReadable() * Remove isReadable() * Optimize ReadWriteDataOutput * tidy code * Freeze the DataOutput once finished() * Refactor * freeze the DataOutput before use * Address comments and add off-heap FST tests * Remove the hardcoded random * Ignore the Test2BFSTOffHeap test * Simplify ReadWriteDataOutput * Update Javadoc * Address comments
1 parent f55f2af commit b699981

File tree

15 files changed

+923
-945
lines changed

15 files changed

+923
-945
lines changed

lucene/core/src/java/org/apache/lucene/codecs/lucene90/blocktree/Lucene90BlockTreeTermsWriter.java

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616
*/
1717
package org.apache.lucene.codecs.lucene90.blocktree;
1818

19+
import static org.apache.lucene.util.fst.FSTCompiler.getOnHeapReaderWriter;
20+
1921
import java.io.IOException;
2022
import java.util.ArrayList;
2123
import java.util.Arrays;
@@ -525,7 +527,7 @@ public void compileIndex(
525527
// Disable suffixes sharing for block tree index because suffixes are mostly dropped
526528
// from the FST index and left in the term blocks.
527529
.suffixRAMLimitMB(0d)
528-
.bytesPageBits(pageBits)
530+
.dataOutput(getOnHeapReaderWriter(pageBits))
529531
.build();
530532
// if (DEBUG) {
531533
// System.out.println(" compile index for prefix=" + prefix);

0 commit comments

Comments
 (0)