Docs and simplifications to support for Lucene ancient versions #124053
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
javanna
added
>refactoring
:Search Foundations/Search
Collaborator
|
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
elasticsearchmachine
added
the
Team:Search Foundations
javanna
commented
Mar 4, 2025
| throw new IllegalArgumentException(formatName); | ||
| } | ||
|
|
||
| private class FieldsWriter extends FieldsConsumer { |
Member
Author
There was a problem hiding this comment.
there is no need to have a FieldsWriter that throws exceptions, we can simply throw exception directly when a fields writer is currently requested, as that's only used for writing.
javanna
commented
Mar 4, 2025
| * Holds all state required for {@link Lucene50PostingsReader} to produce a {@link | ||
| * org.apache.lucene.index.PostingsEnum} without re-seeking the terms dict. | ||
| */ | ||
| public static final class IntBlockTermState extends BlockTermState { |
Member
Author
There was a problem hiding this comment.
this is public, we can rely on the Lucene original variant where needed (only in tests with the removal of Lucene50PostingsReader)
The old lucene versions plugin allows users to read indices created by ancient Elasticsearch version, starting from 5.0. Especially for 5.x which relied on Lucene 6.x, some special logic is required around postings format support. That revolves around reading of FSTs, but has a consequence of requiring quite a few fork of other Lucene classes due to their visibility. This commit attempts to add javadocs to clarify the intent of some of these classes. It also includes some simplifications, in that Lucene50PostingsReader can be reused as-is and some other classes are only needed in tests hence are moved to the test folder.
javanna
force-pushed
the
refactoring/old_lucene_versions_legacy_adapting_docs
branch
from
f08c513 to
d56f308
Compare
Member
Author
|
@drempapis if it makes reviewing easier I can split this into a couple smaller PRs, let me know what you think. |
javanna
commented
Mar 5, 2025
| */ | ||
| public abstract class LegacyAdaptingPerFieldPostingsFormat extends PostingsFormat { | ||
| @UpdateForV10(owner = UpdateForV10.Owner.SEARCH_FOUNDATIONS) | ||
| public final class LegacyAdaptingPerFieldPostingsFormat extends PostingsFormat { |
Member
Author
There was a problem hiding this comment.
this no longer needs to be abstract and subclassed, because the two only subclasses we had did exactly the same thing in their getPostingsFormat method:
if (formatName.equals("Lucene50")) {
return new BWCLucene50PostingsFormat();
} else {
return new BWCCodec.EmptyPostingsFormat();
}
This logic is now incorporated in this class.
drempapis
reviewed
Mar 10, 2025
| * </ul> | ||
| * </dl> | ||
| * This is a fork of {@link org.apache.lucene.backward_codecs.lucene50.Lucene50PostingsFormat} that allows to read from indices | ||
| * written by versions older than Lucene 7.0. |
drempapis
reviewed
Mar 10, 2025
| public FieldsProducer fieldsProducer(SegmentReadState state) throws IOException { | ||
| PostingsReaderBase postingsReader = new Lucene50PostingsReader(state); | ||
| // The postings reader is the original one from Lucene 5.0 | ||
| PostingsReaderBase postingsReader = new org.apache.lucene.backward_codecs.lucene50.Lucene50PostingsReader(state); |
Contributor
There was a problem hiding this comment.
Can't an import be used here? The xpack equivalent class has been deleted.
Member
Author
There was a problem hiding this comment.
yes I am not sure why I did not do so, fixing
javanna
added
the
auto-backport
javanna
deleted the
refactoring/old_lucene_versions_legacy_adapting_docs
branch
Collaborator
💚 Backport successful
|
georgewallace
pushed a commit
to georgewallace/elasticsearch
that referenced
this pull request
Mar 11, 2025
…tic#124053) The old lucene versions plugin allows users to read indices created by ancient Elasticsearch version, starting from 5.0. Especially for 5.x which relied on Lucene 6.x, some special logic is required around postings format support. That revolves around reading of FSTs, but has a consequence of requiring quite a few fork of other Lucene classes due to their visibility. This commit attempts to add javadocs to clarify the intent of some of these classes. It also includes some simplifications, in that Lucene50PostingsReader can be reused as-is and some other classes are only needed in tests hence are moved to the test folder.
javanna
added a commit
that referenced
this pull request
Mar 11, 2025
…#124053) (#124509) * Docs and simplifications to support for Lucene ancient versions (#124053) The old lucene versions plugin allows users to read indices created by ancient Elasticsearch version, starting from 5.0. Especially for 5.x which relied on Lucene 6.x, some special logic is required around postings format support. That revolves around reading of FSTs, but has a consequence of requiring quite a few fork of other Lucene classes due to their visibility. This commit attempts to add javadocs to clarify the intent of some of these classes. It also includes some simplifications, in that Lucene50PostingsReader can be reused as-is and some other classes are only needed in tests hence are moved to the test folder. * iter * iter
albertzaharovits
pushed a commit
to albertzaharovits/elasticsearch
that referenced
this pull request
Mar 13, 2025
…tic#124053) The old lucene versions plugin allows users to read indices created by ancient Elasticsearch version, starting from 5.0. Especially for 5.x which relied on Lucene 6.x, some special logic is required around postings format support. That revolves around reading of FSTs, but has a consequence of requiring quite a few fork of other Lucene classes due to their visibility. This commit attempts to add javadocs to clarify the intent of some of these classes. It also includes some simplifications, in that Lucene50PostingsReader can be reused as-is and some other classes are only needed in tests hence are moved to the test folder.
jfreden
pushed a commit
to jfreden/elasticsearch
that referenced
this pull request
Mar 13, 2025
…tic#124053) The old lucene versions plugin allows users to read indices created by ancient Elasticsearch version, starting from 5.0. Especially for 5.x which relied on Lucene 6.x, some special logic is required around postings format support. That revolves around reading of FSTs, but has a consequence of requiring quite a few fork of other Lucene classes due to their visibility. This commit attempts to add javadocs to clarify the intent of some of these classes. It also includes some simplifications, in that Lucene50PostingsReader can be reused as-is and some other classes are only needed in tests hence are moved to the test folder.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The old lucene versions plugin allows users to read indices created by ancient Elasticsearch version, starting from 5.0. Especially for 5.x which relied on Lucene 6.x, some special logic is required around postings format support. That revolves around reading of FSTs, but has a consequence of requiring quite a few fork of other Lucene classes due to their visibility.
This commit attempts to add javadocs to clarify the intent of some of these classes. It also includes some simplifications, in that Lucene50PostingsReader can be reused as-is and some other classes are only needed in tests hence are moved to the test folder.