Merged
Conversation
dakinggg
approved these changes
Sep 11, 2025
Collaborator
dakinggg
left a comment
dakinggg
left a comment
There was a problem hiding this comment.
Thanks, looks good to me!
Collaborator
|
Looks like a couple lint errors to resolve |
cthoyt
force-pushed
the
tfidf-ann-caching
branch
from
e882d4a to
5011a8e
Compare
Contributor
Author
|
@dakinggg thanks for the review (and sorry I missed the styling). I have updated it and squashed my commits together. This should be ready for merge now. |
Contributor
Author
|
@dakinggg do you think you could make a new release following this PR? thanks! |
Collaborator
|
Sure, I can probably do one next week. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before,
create_tfidf_ann_indexwould only write cache files to a given directory.This PR implements functionality for loading cache files, if they're already there. It also does a minor refactor to reduce code duplication for the loading of such cache objects, which was also implemented in
CandidateGenerator.__init__()I've found this to be very useful for #542, where I don't want to have to spend a time- and compute-intensive process to rebuild the index each time, but I also want to have my code be fully reproducible / not require manually running scripts ahead of time to create cache files