Skip to content

Feature: support custom key prefix for issue 957 #958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Feature: support custom key prefix for issue 957 #958

wants to merge 5 commits into from
+78 −25

Conversation

@uniqueni
Copy link
Contributor

@uniqueni uniqueni commented Oct 23, 2025 •
edited
Loading

Proposed Solution

Add MC_STORE_ETCD_KEY_PREFIX and MC_METADATA_ETCD_KEY_PREFIX environment variable to customize the key prefix

Use Cases

  • Multi-environment: Share etcd between dev/test/prod
  • Multi-cluster: Run multiple clusters on same infrastructure
  • Multi-tenancy: Isolate metadata per tenant/application

@uniqueni uniqueni mentioned this pull request Oct 23, 2025
Open
1 task
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @uniqueni, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new feature that allows users to customize the metadata key prefix for etcd via the MC_METADATA_KEY_PREFIX environment variable. This change enables better isolation and organization of metadata in multi-environment, multi-cluster, and multi-tenancy deployments without requiring any code changes or affecting other storage backends.

Highlights

  • New Feature: Introduces the MC_METADATA_KEY_PREFIX environment variable to allow customization of the metadata key prefix specifically for etcd.
  • Use Cases: Supports multi-environment, multi-cluster, and multi-tenancy scenarios by isolating metadata.
  • Backward Compatibility: Maintains backward compatibility by defaulting to the current behavior if the environment variable is not set.
  • Implementation Details: Modifies the TransferMetadata class to read the MC_METADATA_KEY_PREFIX environment variable and construct dynamic prefixes for metadata keys.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist[bot]
gemini-code-assist bot reviewed Oct 23, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a useful feature to support custom metadata key prefixes for etcd, which will help in multi-tenant or multi-environment setups. The implementation is straightforward, adding the prefix logic within the TransferMetadata class and isolating it to the etcd backend.

The code changes are mostly correct, but I've found one issue regarding protocol detection that could cause the feature to not work with certain valid connection strings. I've left a specific comment with a suggestion to fix it.

Additionally, this new functionality is not covered by any tests. It would be beneficial to add unit tests to transfer_metadata_test.cpp to verify:

  • The custom prefix is correctly applied when MC_METADATA_KEY_PREFIX is set for an etcd connection.
  • The default prefix is used when the environment variable is not set.
  • The environment variable is ignored for non-etcd backends (like redis or http).

Adding these tests will ensure the feature works as expected and prevent future regressions.

@uniqueni uniqueni changed the title Feature: support custom metadata key prefix for issue 957 Feature: support custom key prefix for issue 957 Oct 24, 2025
@ykwd
Copy link
Collaborator

ykwd commented Oct 27, 2025

Thank you for this work. Being able to isolate keys in etcd under a multi-tenant environment is meaningful. I’ve left some comments for the mooncake-store part, and the transfer engine part will need @alogfans to take a look.

  1. After changing MASTER_VIEW_KEY from a fixed string to a member variable of MasterViewHelper, it would be better to compute it once during class initialization and avoid recalculating it later.

  2. The environment variable name MC_STORE_ETCD_KEY_PREFIX is suggested to be renamed to MC_STORE_CLUSTER_ID. Currently, the path isolation format is:

    mooncake-store (to distinguish from other apps) / cluster_id (to distinguish between clusters) / key_name

  3. If the environment variable is not found, we can provide a default cluster ID, such as mooncake.

alogfans
alogfans reviewed Oct 27, 2025
View reviewed changes
@uniqueni
Copy link
Contributor Author

Thank you for this work. Being able to isolate keys in etcd under a multi-tenant environment is meaningful. I’ve left some comments for the mooncake-store part, and the transfer engine part will need @alogfans to take a look.

  1. After changing MASTER_VIEW_KEY from a fixed string to a member variable of MasterViewHelper, it would be better to compute it once during class initialization and avoid recalculating it later.
  2. The environment variable name MC_STORE_ETCD_KEY_PREFIX is suggested to be renamed to MC_STORE_CLUSTER_ID. Currently, the path isolation format is: 
    mooncake-store (to distinguish from other apps) / cluster_id (to distinguish between clusters) / key_name
  3. If the environment variable is not found, we can provide a default cluster ID, such as mooncake.

Okay, I'll optimize it.

@uniqueni uniqueni force-pushed the feature-custom-prefix branch 2 times, most recently from 888f553 to 96e3b35 Compare October 28, 2025 09:15
@uniqueni
Copy link
Contributor Author

@ykwd @alogfans
I have changed env name to MC_STORE_CLUSTER_ID and MC_METADA_CLUSTER_ID
please review it again thx

@ykwd
Copy link
Collaborator

ykwd commented Oct 28, 2025

@ykwd @alogfans I have changed env name to MC_STORE_CLUSTER_ID and MC_METADA_CLUSTER_ID please review it again thx

Thanks. The moocake store part looks good to me. I will merge this PR once @alogfans finishes reviewing the transfer engine part.

@uniqueni uniqueni force-pushed the feature-custom-prefix branch from 96e3b35 to ea60d01 Compare October 28, 2025 10:20
@stmatengss stmatengss requested a review from alogfans October 29, 2025 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alogfans alogfans Awaiting requested review from alogfans

1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@uniqueni @ykwd @alogfans