-
Notifications
You must be signed in to change notification settings - Fork 419
feat(mito2): expose puffin index metadata #7042
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
8039fe1
to
0187c15
Compare
ef13217
to
2c57625
Compare
Signed-off-by: Zhenchi <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds the ability to expose puffin index metadata from the MitoEngine, enabling inspection of various index types including bloom filters, fulltext indexes, and inverted indexes. The implementation provides structured metadata extraction from puffin files and converts it to a queryable format.
Key changes:
- Added
PuffinIndexMetaEntry
schema and record batch conversion utilities - Implemented puffin file traversal to collect metadata from different index types (bloom, fulltext, inverted)
- Added
MitoEngine::all_index_metas()
method with comprehensive integration tests
Reviewed Changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
File | Description |
---|---|
src/store-api/src/sst_entry.rs | Added PuffinIndexMetaEntry struct with schema and record batch conversion |
src/mito2/src/engine/puffin_index.rs | New module for collecting index metadata from puffin files |
src/mito2/src/engine.rs | Added all_index_metas() method to MitoEngine |
src/index/src/target.rs | New IndexTarget enum for encoding/decoding index target keys |
src/mito2/src/sst/index/*/creator.rs | Updated to use IndexTarget for consistent target key encoding |
src/mito2/src/sst/index/*/applier.rs | Updated to use IndexTarget for consistent target key decoding |
src/mito2/src/sst/index/*.rs | Made blob type constants public for metadata collection |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost LGTM
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
PR Checklist
Please convert it to a draft if some of the following conditions are not met.