Skip to content

Commit 22372a7

Browse files
committed
update the KEP with cluster access mechanism
1 parent 382fe4b commit 22372a7

File tree

1 file changed

+58
-18
lines changed
  • keps/sig-multicluster/4322-cluster-inventory

1 file changed

+58
-18
lines changed

keps/sig-multicluster/4322-cluster-inventory/README.md

Lines changed: 58 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,11 @@ tags, and then generate with `hack/update-toc.sh`.
106106
- [Version](#version)
107107
- [Properties](#properties)
108108
- [Conditions](#conditions)
109+
- [Cluster Access](#cluster-access)
110+
- [Pull Model with Work API](#pull-model-with-work-api)
111+
- [Push Model with Identity Federation (Recommended)](#push-model-with-identity-federation-recommended)
112+
- [Push Model via Credentials in Secret (Not Recommended)](#push-model-via-credentials-in-secret-not-recommended)
113+
- [Secret format](#secret-format)
109114
- [API Example](#api-example)
110115
- [Scalability implication](#scalability-implication)
111116
- [Test Plan](#test-plan)
@@ -155,7 +160,7 @@ Items marked with (R) are required *prior to targeting to a milestone / release*
155160
- [ ] (R) Design details are appropriately documented
156161
- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
157162
- [ ] e2e Tests for all Beta API Operations (endpoints)
158-
- [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
163+
- [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
159164
- [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
160165
- [ ] (R) Graduation criteria is in place
161166
- [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
@@ -240,7 +245,7 @@ management. Examples of consumers includes:
240245
this area.
241246
* GitOps tools (ArgoCD, flux etc) are having the requirement to deploy
242247
workload to multiple clusters. They either need to build the cluster
243-
concept by themselves or understand cluster API from each different
248+
concept by themselves or understand APIs representing a cluster from each
244249
cluster management project. A common ClusterProfile API can provide
245250
a thin compatible layer for different projects.
246251
* Operation tools or customized external consumers: this API gives a
@@ -279,6 +284,7 @@ and make progress.
279284
* Provide a standard reference implementation.
280285
* Define specific implementation details beyond general API behavior.
281286
* Offering functionalities related to multi-cluster orchestration.
287+
* Define the Consumer registration API
282288

283289

284290
## Proposal
@@ -310,14 +316,14 @@ the API proposed by this KEP aims to
310316
- **Member Cluster**: A kubernetes cluster that is part of a cluster inventory.
311317

312318
- **Cluster Manager**: An entity that creates the ClusterProfile API object per member cluster,
313-
and keeps their status up-to-date. Each cluster manager MUST be identified with a unique name.
314-
Each ClusterProfile resource SHOULD be managed by only one cluster manager. A cluster manager SHOULD
319+
and keeps their status up-to-date. Each cluster manager MUST be identified with a unique name.
320+
Each ClusterProfile resource SHOULD be managed by only one cluster manager. A cluster manager SHOULD
315321
have sufficient permission to access the member cluster to fetch the information so it can update the status
316322
of the ClusterProfile API resource.
317323

318-
- **ClusterProfile API Consumer**: the person running the cluster managers
319-
or the person developing extensions for cluster managers for the purpose of
320-
workload distribution, operation management etc.
324+
- **ClusterInventory Consumer**: Controllers or tools that leverage the ClusterProfile for the purpose of
325+
workload distribution, operation management etc. Their name must be unique for a single inventory. They
326+
might need to register themselves with the Cluster Manager which is not defined in this KEP.
321327

322328
### User Stories (Optional)
323329

@@ -392,19 +398,19 @@ if all its member clusters adhere to the [namespace sameness](https://github.com
392398
Note that a cluster can only be in one ClusterSet while there is not such restriction for a cluster inventory.
393399

394400
#### How should the API be consumed?
395-
We recommend that all ClusterProfile objects within the same cluster inventory reside on
401+
We recommend that all ClusterProfile objects within the same cluster inventory reside on
396402
a dedicated Kubernetes cluster (aka. the hub cluster). This approach allows consumers to have a single integration
397403
point to access all the information within a cluster inventory. Additionally, a multi-cluster aware
398404
controller can be run on the dedicated cluster to offer high-level functionalities over this inventory of clusters.
399405

400406
#### How should we organize ClusterProfile objects on a hub cluster?
401-
While there are no strict requirements, we recommend making the ClusterProfile API a namespace-scoped object.
407+
While there are no strict requirements, we recommend making the ClusterProfile API a namespace-scoped object.
402408
This approach allows users to leverage Kubernetes' native namespace-based RBAC if they wish to restrict access to
403-
certain clusters within the inventory.
409+
certain clusters within the inventory.
404410

405411
However, if a cluster inventory represents a ClusterSet, all its ClusterProfile objects MUST be part of the same clusterSet
406412
and namespace must be used as the grouping mechanism. In addition, the namespace must have a label with the key "clusterset.multicluster.x-k8s.io"
407-
and the value as the name of the clusterSet.
413+
and the value as the name of the clusterSet.
408414

409415
#### Uniqueness of the ClusterProfile object
410416
While there are no strict requirements, we recommend that there is only one ClusterProfile object representing any member cluster
@@ -442,21 +448,22 @@ represent a cluster and support the above use case at the minimum scope.
442448

443449
The target consumers of the API are users who manage the clusters and
444450
other tools to understand the clusters concept from the API for
445-
multicluster scheduling, workload distribution and cluster management.
451+
multi-cluster scheduling, workload distribution and cluster management.
446452
The API aims to provide the information for the consumers to answer the
447453
below questions:
448454

449455
* Is the cluster under management?
450456
* How can I select a cluster?
451457
* Is the cluster healthy?
458+
* How to access the cluster?
452459
* Does the cluster have certain capabilities or properties?
453460
* Does the cluster have sufficient resources?
454461

455462
### Cluster Name
456463

457464
It is required that cluster name is unique for each cluster, and it
458465
should also be unique among different providers (cluster manager). It
459-
is cluster managers's responsibility to ensure the name uniqueness.
466+
is cluster manager's responsibility to ensure the name uniqueness.
460467

461468
It's the responsibility of the cluster manager platform administrator
462469
to ensure cluster name uniqueness.
@@ -543,6 +550,39 @@ Predefined condition types:
543550
The status of the cluster SHOULD be updated by the cluster manager under
544551
this condition.
545552

553+
### Cluster Access
554+
There are multiple methods for a ClusterInventory Consumer to gain access to the cluster represented by a ClusterProfile API.
555+
This KEP does not define the exact mechanism for each approach, but it is recommended that ClusterInventory Consumers avoid
556+
using a secret if possible. Here are the three approaches:
557+
558+
#### Pull Model with Work API
559+
The ClusterInventory Consumer can leverage the [Work API](https://multicluster.sigs.k8s.io/concepts/work-api/) so that
560+
the work API agent on the Member Cluster can *pull* the necessary objects from the management cluster where the ClusterInventory
561+
Consumer operates to the Member Cluster. This approach allows the Cluster Manager to manage the RBAC for each Consumer on the
562+
management cluster, ensuring that their access is restricted to the corresponding namespaces of the ClusterProfile objects.
563+
564+
#### Push Model with Identity Federation (Recommended)
565+
The ClusterInventory consumer can utilize identity federation mechanisms, such as [Azure Workload Identity Federation](https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation)
566+
or [GCP Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation). This allows the
567+
ClusterInventory Consumer to use an identity that already has access to the clusters in the Cluster Inventory.
568+
While the Cluster Manager can assist in setting up the federation, it is not a mandatory requirement.
569+
570+
#### Push Model via Credentials in Secret (Not Recommended)
571+
The ClusterInventory Consumer can obtain credentials to access the cluster represented by a ClusterProfile object by reading
572+
from a secret. In this approach, the Cluster Manager generates secrets containing the necessary credentials within the namespace
573+
accessible to the ClusterInventory Consumer. For this to function correctly, Cluster Managers must be aware of the following details
574+
about the consumer: their name, whether credentials are required, and the preferred unique namespace for reading credentials as secrets.
575+
Those information can be obtained during the "registering" process but this is out of the scope of this KEP.
576+
577+
##### Secret format
578+
- The secret *MUST* reside in the namespace with the label `x-k8s.io/cluster-inventory-consumer` with the value being the name of the ClusterInventory Consumer.
579+
- The secret *MUST* contain the label `x-k8s.io/cluster-profile` with the value being the name of the ClusterProfile object that the secret is associated with.
580+
- The access information in the secret must contain the following fields
581+
- **Config**: This field contains cluster access information compatible with the
582+
[kubeconfig format](https://github.com/kubernetes/kubernetes/blob/v1.31.2/staging/src/k8s.io/client-go/tools/clientcmd/api/types.go#L31).
583+
- Since a single [Kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) supports access to multiple clusters, the Cluster manager *MUST* ensure that each secret contains access information for only a single consumer.
584+
585+
546586
## API Example
547587

548588
```yaml
@@ -553,18 +593,18 @@ metadata:
553593
labels:
554594
x-k8s.io/cluster-manager: some-cluster-manager
555595
spec:
556-
displayName: some-cluster
596+
displayName: cluster-us-east
557597
clusterManager:
558598
name: some-cluster-manager
559599
status:
560-
version:
561-
kubernetes: 1.28.0
562-
properties:
600+
version:
601+
kubernetes: 1.28.0
602+
properties:
563603
- name: clusterset.k8s.io
564604
value: some-clusterset
565605
- name: location
566606
value: apac
567-
conditions:
607+
conditions:
568608
- type: ControlPlaneHealthy
569609
status: True
570610
lastTransitionTime: "2023-05-08T07:56:55Z"

0 commit comments

Comments
 (0)