feat: Extend the evaluators to support external predictions stored in files#185
Merged
nikos-livathinos merged 21 commits intomainfrom Dec 8, 2025
Merged
feat: Extend the evaluators to support external predictions stored in files#185nikos-livathinos merged 21 commits intomainfrom
nikos-livathinos merged 21 commits intomainfrom
Conversation
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…mmy entries in all evaluators. Extend the CLI to support the --external-predictions-path Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…various formats Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…th. Add unit test Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…d unit test. Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
Contributor
|
✅ DCO Check Passed Thanks @nikos-livathinos, all your commits are properly signed off. 🎉 |
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 Require two reviewer for test updatesWonderful, this rule succeeded.When test data is updated, we require two reviewers
|
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…it test. Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…unit test Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…dd unit test. Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…. Add unit test Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…dd unit test Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…ngOrderEvaluator. Fix main Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…oclingDocument from doctags and the GT image. - Introduce the staticmethod load_doctags() which covers all cases on page image loading. - Refactor the FilePredictionProvider to use the load_doctags() from ExternalDoclingDocumentLoader. - Refactor all evaluators to use the new ExternalDoclingDocumentLoader. Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
…sing the API and the CLI. Signed-off-by: Nikos Livathinos <nli@zurich.ibm.com>
cau-git
approved these changes
Dec 8, 2025
vagenas
approved these changes
Dec 8, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support external predictions according to the description given in #112
external_predictions_path.DoclingDocumentobjects from files instead of the parquet dataset.docling-eval evaluate: