add classmethod IngestionCache.from_persist_dir and add/fix some test… #20222
+69
−7
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I added the
from_persist_dirinterface to theIngestionCacheclass. This is intended for use when utilizingIngestionPipelinewithout usingload, instead directly passing pre-generated instances ofdocstoreandcache. It was implemented to resolve the imbalance wherefrom_persist_direxists on thedocstoreside but not on thecacheside.Additionally, while creating the test case for the newly added from_persist_dir (test_pipeline_with_preload_from_persist_dir @test_pipeline.py), I discovered that The existing test cases
test_save_load_pipelineandtest_save_load_pipeline_without_docstorewere re-running the first pipeline after the second pipeline generation to verify dedup. This seemed inconsistent with the intended behavior, so I corrected it as well.After adding
from_persist_dirand fixing the test cases, I confirmed all tests pass.New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.
Suggested Checklist:
uv run make format; uv run make lintto appease the lint gods