Skip to content

doc: add design proposal for topology-aware cluster selection#5986

Open
WMP wants to merge 1 commit intoceph:develfrom
WMP:proposal/topology-aware-cluster-selection
Open

doc: add design proposal for topology-aware cluster selection#5986
WMP wants to merge 1 commit intoceph:develfrom
WMP:proposal/topology-aware-cluster-selection

Conversation

@WMP
Copy link

@WMP WMP commented Jan 28, 2026

Describe what this PR does

Design proposal for topology-aware multi-cluster volume provisioning.
This enables the CSI driver to dynamically select the appropriate Ceph
cluster at CreateVolume time based on the node's topology zone.

The proposal introduces two new configuration mechanisms:

  • topologyDomainLabels field in config.json cluster entries — associates
    each cluster with Kubernetes topology labels
  • clusterIDs StorageClass parameter — a comma-separated list of candidate
    cluster IDs for topology-based selection

This is a design-only PR. Implementation will follow in a separate PR
once the design is approved.

Is there anything that requires special attention

  • The design is fully backward compatible. Existing configs and
    StorageClasses with a single clusterID work unchanged.
  • volumeBindingMode: WaitForFirstConsumer is required for topology-based
    selection (Kubernetes must provide AccessibilityRequirements).
  • The existing clusterID parameter takes priority when present —
    topology selection is only used as a fallback via the new clusterIDs
    parameter.

Related issues

Ref: #5177

Future concerns

  • Making clusterID fully optional when clusterIDs is provided
  • Combining topology-based cluster selection with topologyConstrainedPools
    for selecting both cluster and pool
  • E2E tests with multi-cluster topology setup

Checklist:

  • Commit Message Formatting: follows developer guide
  • Reviewed the developer guide on Submitting a Pull Request
  • Pending release notes updated
  • Documentation has been updated, if necessary.
  • Unit tests have been added, if necessary.
  • Integration tests have been added, if necessary.

@mergify mergify bot added ci/skip/e2e skip running e2e CI jobs ci/skip/multi-arch-build skip building on multiple architectures component/docs Issues and PRs related to documentation labels Jan 28, 2026
Add a design document describing topology-aware multi-cluster volume
provisioning. This enables the CSI driver to dynamically select the
appropriate Ceph cluster at CreateVolume time based on the node's
topology zone.

The proposal introduces two configuration mechanisms:
- topologyDomainLabels field in config.json cluster entries
- clusterIDs StorageClass parameter (comma-separated list)

Ref: ceph#5177

Signed-off-by: Marcin Janowski <marcin.janowski@assecobs.pl>
@WMP WMP force-pushed the proposal/topology-aware-cluster-selection branch from 76965d0 to 6c1c9d7 Compare January 28, 2026 09:02
@Rakshith-R
Copy link
Contributor

Hey @WMP ,

Thanks for the contributions !

Please note that all of this code is written by Claude, and at this point, it has not been built or tested. This is only a proposal to add this functionality. At the moment, it is really just vibe coding. However, if you agree that the idea of implementing this functionality is appropriate, I will continue my work and perform tests on live ceph and k8s clusters.

We would love to review and accept contributions.
Reviewing designs and code consumes a lot of effort.

Before you provide design on a new feature, have you tested and understood the current working of topology based provisioning in k8s, csi and cephcsi as of today ?

This would help you understand the feature and review design for proposed improvements yourself in depth.

It would certainly boost our confidence for reviewing this design document.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/skip/e2e skip running e2e CI jobs ci/skip/multi-arch-build skip building on multiple architectures component/docs Issues and PRs related to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants