You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -155,7 +160,7 @@ Items marked with (R) are required *prior to targeting to a milestone / release*
155
160
-[ ] (R) Design details are appropriately documented
156
161
-[ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
157
162
-[ ] e2e Tests for all Beta API Operations (endpoints)
158
-
-[ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
163
+
-[ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
159
164
-[ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
160
165
-[ ] (R) Graduation criteria is in place
161
166
-[ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
@@ -240,7 +245,7 @@ management. Examples of consumers includes:
240
245
this area.
241
246
* GitOps tools (ArgoCD, flux etc) are having the requirement to deploy
242
247
workload to multiple clusters. They either need to build the cluster
243
-
concept by themselves or understand cluster API from each different
248
+
concept by themselves or understand APIs representing a cluster from each
244
249
cluster management project. A common ClusterProfile API can provide
245
250
a thin compatible layer for different projects.
246
251
* Operation tools or customized external consumers: this API gives a
@@ -279,6 +284,7 @@ and make progress.
279
284
* Provide a standard reference implementation.
280
285
* Define specific implementation details beyond general API behavior.
281
286
* Offering functionalities related to multi-cluster orchestration.
287
+
* Define the Consumer registration API
282
288
283
289
284
290
## Proposal
@@ -310,14 +316,14 @@ the API proposed by this KEP aims to
310
316
-**Member Cluster**: A kubernetes cluster that is part of a cluster inventory.
311
317
312
318
-**Cluster Manager**: An entity that creates the ClusterProfile API object per member cluster,
313
-
and keeps their status up-to-date. Each cluster manager MUST be identified with a unique name.
314
-
Each ClusterProfile resource SHOULD be managed by only one cluster manager. A cluster manager SHOULD
319
+
and keeps their status up-to-date. Each cluster manager MUST be identified with a unique name.
320
+
Each ClusterProfile resource SHOULD be managed by only one cluster manager. A cluster manager SHOULD
315
321
have sufficient permission to access the member cluster to fetch the information so it can update the status
316
322
of the ClusterProfile API resource.
317
323
318
-
-**ClusterProfile API Consumer**: the person running the cluster managers
319
-
or the person developing extensions for cluster managers for the purpose of
320
-
workload distribution, operation management etc.
324
+
-**ClusterInventory Consumer**: Controllers or tools that leverage the ClusterProfile for the purpose of
325
+
workload distribution, operation management etc. Their name must be unique for a single inventory. They
326
+
might need to register themselves with the Cluster Manager which is not defined in this KEP.
321
327
322
328
### User Stories (Optional)
323
329
@@ -392,19 +398,19 @@ if all its member clusters adhere to the [namespace sameness](https://github.com
392
398
Note that a cluster can only be in one ClusterSet while there is not such restriction for a cluster inventory.
393
399
394
400
#### How should the API be consumed?
395
-
We recommend that all ClusterProfile objects within the same cluster inventory reside on
401
+
We recommend that all ClusterProfile objects within the same cluster inventory reside on
396
402
a dedicated Kubernetes cluster (aka. the hub cluster). This approach allows consumers to have a single integration
397
403
point to access all the information within a cluster inventory. Additionally, a multi-cluster aware
398
404
controller can be run on the dedicated cluster to offer high-level functionalities over this inventory of clusters.
399
405
400
406
#### How should we organize ClusterProfile objects on a hub cluster?
401
-
While there are no strict requirements, we recommend making the ClusterProfile API a namespace-scoped object.
407
+
While there are no strict requirements, we recommend making the ClusterProfile API a namespace-scoped object.
402
408
This approach allows users to leverage Kubernetes' native namespace-based RBAC if they wish to restrict access to
403
-
certain clusters within the inventory.
409
+
certain clusters within the inventory.
404
410
405
411
However, if a cluster inventory represents a ClusterSet, all its ClusterProfile objects MUST be part of the same clusterSet
406
412
and namespace must be used as the grouping mechanism. In addition, the namespace must have a label with the key "clusterset.multicluster.x-k8s.io"
407
-
and the value as the name of the clusterSet.
413
+
and the value as the name of the clusterSet.
408
414
409
415
#### Uniqueness of the ClusterProfile object
410
416
While there are no strict requirements, we recommend that there is only one ClusterProfile object representing any member cluster
@@ -442,21 +448,22 @@ represent a cluster and support the above use case at the minimum scope.
442
448
443
449
The target consumers of the API are users who manage the clusters and
444
450
other tools to understand the clusters concept from the API for
445
-
multicluster scheduling, workload distribution and cluster management.
451
+
multi-cluster scheduling, workload distribution and cluster management.
446
452
The API aims to provide the information for the consumers to answer the
447
453
below questions:
448
454
449
455
* Is the cluster under management?
450
456
* How can I select a cluster?
451
457
* Is the cluster healthy?
458
+
* How to access the cluster?
452
459
* Does the cluster have certain capabilities or properties?
453
460
* Does the cluster have sufficient resources?
454
461
455
462
### Cluster Name
456
463
457
464
It is required that cluster name is unique for each cluster, and it
458
465
should also be unique among different providers (cluster manager). It
459
-
is cluster managers's responsibility to ensure the name uniqueness.
466
+
is cluster manager's responsibility to ensure the name uniqueness.
460
467
461
468
It's the responsibility of the cluster manager platform administrator
462
469
to ensure cluster name uniqueness.
@@ -543,6 +550,39 @@ Predefined condition types:
543
550
The status of the cluster SHOULD be updated by the cluster manager under
544
551
this condition.
545
552
553
+
### Cluster Access
554
+
There are multiple methods for a ClusterInventory Consumer to gain access to the cluster represented by a ClusterProfile API.
555
+
This KEP does not define the exact mechanism for each approach, but it is recommended that ClusterInventory Consumers avoid
556
+
using a secret if possible. Here are the three approaches:
557
+
558
+
#### Pull Model with Work API
559
+
The ClusterInventory Consumer can leverage the [Work API](https://multicluster.sigs.k8s.io/concepts/work-api/) so that
560
+
the work API agent on the Member Cluster can *pull* the necessary objects from the management cluster where the ClusterInventory
561
+
Consumer operates to the Member Cluster. This approach allows the Cluster Manager to manage the RBAC for each Consumer on the
562
+
management cluster, ensuring that their access is restricted to the corresponding namespaces of the ClusterProfile objects.
563
+
564
+
#### Push Model with Identity Federation (Recommended)
565
+
The ClusterInventory consumer can utilize identity federation mechanisms, such as [Azure Workload Identity Federation](https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation)
566
+
or [GCP Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation). This allows the
567
+
ClusterInventory Consumer to use an identity that already has access to the clusters in the Cluster Inventory.
568
+
While the Cluster Manager can assist in setting up the federation, it is not a mandatory requirement.
569
+
570
+
#### Push Model via Credentials in Secret (Not Recommended)
571
+
The ClusterInventory Consumer can obtain credentials to access the cluster represented by a ClusterProfile object by reading
572
+
from a secret. In this approach, the Cluster Manager generates secrets containing the necessary credentials within the namespace
573
+
accessible to the ClusterInventory Consumer. For this to function correctly, Cluster Managers must be aware of the following details
574
+
about the consumer: their name, whether credentials are required, and the preferred unique namespace for reading credentials as secrets.
575
+
Those information can be obtained during the "registering" process but this is out of the scope of this KEP.
576
+
577
+
##### Secret format
578
+
- The secret *MUST* reside in the namespace with the label `x-k8s.io/cluster-inventory-consumer` with the value being the name of the ClusterInventory Consumer.
579
+
- The secret *MUST* contain the label `x-k8s.io/cluster-profile` with the value being the name of the ClusterProfile object that the secret is associated with.
580
+
- The access information in the secret must contain the following fields
581
+
-**Config**: This field contains cluster access information compatible with the
- Since a single [Kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) supports access to multiple clusters, the Cluster manager *MUST* ensure that each secret contains access information for only a single consumer.
0 commit comments