Skip to content

Add missing Lucene99 codec and relax term vectors constraint whem merging #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
david-sitsky wants to merge 2 commits into
base: main
Choose a base branch
from
+8 −1
Open

Add missing Lucene99 codec and relax term vectors constraint whem merging #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9dae173
Add Lucene99 codec to manifests.
Feb 10, 2026
ec2cf69
Disable "strictly consistent" checking as Lucene 8/9 semantics differ.
Feb 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion src/main/java/org/trypticon/luceneupgrader/lucene9/internal/lucene/index/FieldInfos.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,12 @@ static final class FieldNumbers {
this.omitNorms = new HashMap<>();
this.storeTermVectors = new HashMap<>();
this.softDeletesFieldName = softDeletesFieldName;
this.strictlyConsistent = indexCreatedVersionMajor >= 9;

// Don't perform "strictly consistent" checking as Lucene 8/9 semantics differ. Lucene 8 sets "term vectors" to false
// even if a field was declared to use "term vectors", but it emitted no terms in the entire segment.
// Lucene 9 seems to rely on the declared field. Disabling this check to avoid these headaches.
this.strictlyConsistent = false;
// this.strictlyConsistent = indexCreatedVersionMajor >= 9;
Copy link
Author

@david-sitsky david-sitsky Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change may not be safe after all.. but the other changes are worth putting in.

Copy link
Owner

@hakanai hakanai Mar 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like Lucene themselves may have done something similar?

https://github.com/apache/lucene/pull/842/changes

Maybe the thing to do is update the baked Lucene to 9.2+... but yeah, having more codecs in the list makes sense independently, maybe split the PR?

this.parentFieldName = parentFieldName;
if (softDeletesFieldName != null
&& parentFieldName != null
Expand Down
1 change: 1 addition & 0 deletions ...urces/META-INF/services/org.trypticon.luceneupgrader.lucene9.internal.lucene.codecs.Codec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

org.trypticon.luceneupgrader.lucene9.internal.lucene.codecs.lucene912.Lucene912Codec

org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene99.Lucene99Codec
org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene95.Lucene95Codec
org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene80.Lucene80Codec
org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene84.Lucene84Codec
Expand Down
1 change: 1 addition & 0 deletions ...A-INF/services/org.trypticon.luceneupgrader.lucene9.internal.lucene.codecs.PostingsFormat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

org.trypticon.luceneupgrader.lucene9.internal.lucene.codecs.lucene912.Lucene912PostingsFormat

org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene99.Lucene99PostingsFormat
org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene90.Lucene90PostingsFormat
org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene84.Lucene84PostingsFormat
org.trypticon.luceneupgrader.lucene9.internal.lucene.backward_codecs.lucene50.Lucene50PostingsFormat
Loading