Skip to content

Conversation

zhongzc
Copy link
Contributor

@zhongzc zhongzc commented Sep 29, 2025

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

  • add PuffinIndexMetaEntry schema plus record-batch helpers
  • traverse puffin blobs to collect bloom/fulltext/inverted metadata
  • wire MitoEngine::all_index_metas and cover it with integration tests

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

@zhongzc zhongzc requested review from a team, evenyag, v0y4g3r and waynexia as code owners September 29, 2025 04:35
@zhongzc zhongzc changed the title Zhongzc/index meta api feat(mito2): expose puffin index metadata Sep 29, 2025
@github-actions github-actions bot added the docs-not-required This change does not impact docs. label Sep 29, 2025
Signed-off-by: Zhenchi <[email protected]>
Signed-off-by: Zhenchi <[email protected]>
@zhongzc zhongzc force-pushed the zhongzc/index-meta-api branch from ef13217 to 2c57625 Compare September 29, 2025 15:04
Signed-off-by: Zhenchi <[email protected]>
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds the ability to expose puffin index metadata from the MitoEngine, enabling inspection of various index types including bloom filters, fulltext indexes, and inverted indexes. The implementation provides structured metadata extraction from puffin files and converts it to a queryable format.

Key changes:

  • Added PuffinIndexMetaEntry schema and record batch conversion utilities
  • Implemented puffin file traversal to collect metadata from different index types (bloom, fulltext, inverted)
  • Added MitoEngine::all_index_metas() method with comprehensive integration tests

Reviewed Changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/store-api/src/sst_entry.rs Added PuffinIndexMetaEntry struct with schema and record batch conversion
src/mito2/src/engine/puffin_index.rs New module for collecting index metadata from puffin files
src/mito2/src/engine.rs Added all_index_metas() method to MitoEngine
src/index/src/target.rs New IndexTarget enum for encoding/decoding index target keys
src/mito2/src/sst/index/*/creator.rs Updated to use IndexTarget for consistent target key encoding
src/mito2/src/sst/index/*/applier.rs Updated to use IndexTarget for consistent target key decoding
src/mito2/src/sst/index/*.rs Made blob type constants public for metadata collection

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost LGTM

@zhongzc zhongzc enabled auto-merge October 15, 2025 10:10
@zhongzc zhongzc added this pull request to the merge queue Oct 17, 2025
Merged via the queue into main with commit 7b396bb Oct 17, 2025
42 checks passed
@zhongzc zhongzc deleted the zhongzc/index-meta-api branch October 17, 2025 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs. documentation size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants