Skip to content

doc: add MEP for Import Two-Phase Commit (primary/secondary replication)#29

Open
bigsheeper wants to merge 1 commit intomilvus-io:mainfrom
bigsheeper:feat/import-replication-2pc
Open

doc: add MEP for Import Two-Phase Commit (primary/secondary replication)#29
bigsheeper wants to merge 1 commit intomilvus-io:mainfrom
bigsheeper:feat/import-replication-2pc

Conversation

@bigsheeper
Copy link
Contributor

Summary

Adds a Milvus Enhancement Proposal (MEP) for Import Two-Phase Commit support in primary/secondary replication clusters.

The design enables bulk-loaded data to become visible atomically and consistently across all clusters by:

  1. Holding import data invisible (is_importing=true) until an explicit CommitImportMessage is delivered via WAL
  2. CDC replicating the WAL message to all secondary clusters, ensuring every cluster commits at the same logical position
  3. New auto_commit option (default true) maintains full backward compatibility for non-replication clusters

New public APIs:

  • POST /v2/vectordb/jobs/import/commit
  • POST /v2/vectordb/jobs/import/abort
  • New Uncommitted and Committing states in GetImportProgress

Related:

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bigsheeper
To complete the pull request process, please assign jiaoew1991 after the PR has been reviewed.
You can assign the PR to them by writing /assign @jiaoew1991 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants