Skip to content

Commit e751fbc

Browse files
Merge pull request #275252 from shashankbarsin/build-2024-fleet-workload
Build 2024 fleet workload
2 parents 2e55d5c + dac8957 commit e751fbc

9 files changed

+315
-65
lines changed

articles/kubernetes-fleet/concepts-choosing-fleet.md

Lines changed: 23 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
title: "Choose an Azure Kubernetes Fleet Manager option"
33
description: This article provides a conceptual overview of the various Azure Kubernetes Fleet Manager options and why you may choose a specific configuration.
44
ms.date: 05/01/2024
5-
author: nickomang
6-
ms.author: nickoman
5+
author: shashankbarsin
6+
ms.author: shasb
77
ms.service: kubernetes-fleet
88
ms.topic: conceptual
99
---
@@ -14,52 +14,51 @@ This article provides an overview of the various Azure Kubernetes Fleet Manager
1414

1515
## Fleet types
1616

17-
There are two main types of Fleet resources—hubless fleets and hubful fleets. As the names suggest, a hubful fleet has an associate Azure Kubernetes Service (AKS) cluster that acts as a hub to store and propagate configuration, while a hubless fleet doesn't. Both options are valid and in active development. There's no expectation that you need to migrate to hubful fleets unless you want to take advantage of the full set of features.
17+
A Kubernetes Fleet resource can be created with or without a hub cluster. A hub cluster is a managed Azure Kubernetes Service (AKS) cluster that acts as a hub to store and propagate Kubernetes resources.
1818

19-
The following table compares the two options.
19+
The following table compares the scenarios enabled by the hub cluster:
2020

21-
||Hubless fleet|Hubful fleet|
21+
| Capability | Kubernetes Fleet resource without hub cluster | Kubernetes Fleet resource with hub cluster |
2222
|----|----|----|
23-
|**Hub cluster hosting**|<span class='red-x'>&#10060;</span>|<span class='green-check'>&#9989;</span>||
23+
|**Hub cluster hosting**|<span class='red-x'>&#10060;</span>|<span class='green-check'>&#9989;</span>|
2424
|**Member cluster limit**|Up to 100 clusters|Up to 20 clusters|
2525
|**Update orchestration**|<span class='green-check'>&#9989;</span>|<span class='green-check'>&#9989;</span>|
2626
|**Workload orchestration**|<span class='red-x'>&#10060;</span>|<span class='green-check'>&#9989;</span>|
2727
|**Layer 4 load balancing**|<span class='red-x'>&#10060;</span>|<span class='green-check'>&#9989;</span>|
2828
|**Billing considerations**|No cost|You pay cost associated with the hub, which is a standard-tier AKS-cluster.|
29-
|**Converting fleet types**|Can convert a hubless fleet to a hubful fleet.|Can't convert a hubful fleet to a hubless fleet.|
29+
|**Converting fleet types**|Can be upgraded to a Kubernetes Fleet resource with a hub cluster.|Can't be downgraded to a Kubernetes Fleet resource without a hub cluster.|
3030

31-
## Hubless fleets
31+
## Kubernetes Fleet resource without hub clusters
3232

33-
Without a hub cluster, Fleet acts solely as a grouping entity in Azure Resource Manager. Certain scenarios, such as update runs, don't require a Kubernetes API and thus don't require a hub cluster. To take full advantage of all the features available on Fleet, you need a hubful fleet.
33+
Without a hub cluster, Kubernetes Fleet acts solely as a grouping entity in Azure Resource Manager (ARM). Certain scenarios, such as update runs, don't require a Kubernetes API and thus don't require a hub cluster. To take full advantage of all the features available, you need a Kubernetes Fleet resource with a hub cluster.
3434

35-
For more information, see [Create a hubless fleet][create-hubless-fleet].
35+
For more information, see [Create a Kubernetes Fleet resource without a hub cluster][create-fleet-without-hub].
3636

37-
## Hubful fleets
37+
## Kubernetes Fleet resource with hub clusters
3838

39-
A hubful fleet has an AKS-managed hub cluster, which is used to store configuration for workload orchestration and layer-4 load balancing.
39+
A Kubernetes Fleet resource with a hub cluster has an associated AKS-managed cluster, which is used to store the configuration for workload orchestration and layer-4 load balancing.
4040

41-
Upon the creation of a hubful fleet, a hub cluster is automatically created in the same subscription under a managed resource group named `FL_*`. To improve reliability, hub clusters are locked down by denying any user initiated mutations to the corresponding AKS clusters (under the Fleet-managed resource group `FL_*`) and their underlying Azure resources (under the AKS-managed resource group `MC_FL_*`), such as VMs, via Azure deny assignments. Control plane operations, such as changing the hub cluster's configuration through Azure Resource Manager (ARM) or deleting the cluster entirely, are denied. Data plane operations, such as connecting to the hub cluster's Kubernetes API server in order to configure workload orchestration, are not denied.
41+
Upon the creation of a Kubernetes Fleet resource with a hub cluster, a hub AKS cluster is automatically created in the same subscription under a managed resource group that begins with `FL_`. To improve reliability, hub clusters are locked down by denying any user-initiated mutations to the corresponding AKS clusters (under the Fleet-managed resource group `FL_`) and their underlying Azure resources (under the AKS-managed resource group `MC_FL_*`), such as virtual machines (VMs), via Azure deny assignments. Control plane operations, such as changing the hub cluster's configuration through Azure Resource Manager (ARM) or deleting the cluster entirely, are denied. Data plane operations, such as connecting to the hub cluster's Kubernetes API server in order to configure workload orchestration, are not denied.
4242

4343
Hub clusters are exempted from [Azure policies][azure-policy-overview] to avoid undesirable policy effects upon hub clusters.
4444

45-
### Public and private hubful fleets
45+
### Network access modes for hub cluster
4646

47-
For hubful fleets, there are two subtypes:
47+
For a Kubernetes Fleet resource with a hub cluster, there are two network access modes:
4848

49-
- **Public hubful fleets** expose the hub cluster to the internet. This means that with the right credentials, anyone on the internet can connect to the hub server. This configuration can be useful during the development and testing phase, but represents a security concern, which is largely undesirable in production.
49+
- **Public hub clusters** expose the hub cluster to the internet. This means that with the right credentials, anyone on the internet can connect to the hub cluster. This configuration can be useful during the development and testing phase, but represents a security concern, which is largely undesirable in production.
5050

51-
For more information, see [Create a public hubful fleet][create-public-hubful-fleet].
51+
For more information, see [Create a Kubernetes Fleet resource with a public hub cluster][create-public-hub-cluster].
5252

53-
- **Private hubful fleets** use a [private AKS cluster][aks-private-cluster] as the hub, which prevents open access over the internet. All considerations for a private AKS cluster apply, so review the prerequisites and limitations to determine whether a private hubful fleet meets your needs.
53+
- **Private hub clusters** use a [private AKS cluster][aks-private-cluster] as the hub, which prevents open access over the internet. All considerations for a private AKS cluster apply, so review the prerequisites and limitations to determine whether a Kubernetes Fleet resource with a private hub cluster meets your needs.
5454

5555
Some other details to consider:
5656

5757
- Whether you choose a public or private hub, the type can't be changed after creation.
58-
- When using an AKS private cluster, you have the ability to configure fully qualified domain names (FQDNs) and FQDN subdomains. This functionality doesn't apply to the private cluster used in a private hubful fleet.
58+
- When using an AKS private cluster, you have the ability to configure fully qualified domain names (FQDNs) and FQDN subdomains. This functionality doesn't apply to the private hub cluster of the Kubernetes Fleet resource.
5959
- When you connect to a private hub cluster, you can use the same methods that you would use to [connect to any private AKS cluster][aks-private-cluster-connect]. However, connecting using AKS command invoke and private endpoints aren't currently supported.
60-
- When you use private hubful fleets, you're required to specify the subnet in which the Fleet hub cluster's node VMs reside. This process differs slightly from the AKS private cluster equivalent. For more information, see [Create a private hubful fleet][create-private-hubful-fleet].
60+
- When you use private hub clusters, you're required to specify the subnet in which the Kubernetes Fleet hub cluster's node VMs reside. This process differs slightly from the AKS private cluster equivalent. For more information, see [create a Kubernetes Fleet resource with a private hub cluster][create-private-hub-cluster].
6161

62-
<!-- TODO: NEED REVIEW ON THE WORDING OF ABOVE BULLETS -->
6362

6463
## Next steps
6564

@@ -70,6 +69,6 @@ Now that you understand the different types of Kubernetes fleet resources, see [
7069
[aks-private-cluster-connect]: /azure/aks/private-clusters?tabs=azure-portal#options-for-connecting-to-the-private-cluster
7170
[azure-policy-overview]: /azure/governance/policy/overview
7271
[quickstart-create-fleet]: quickstart-create-fleet-and-members.md
73-
[create-hubless-fleet]: quickstart-create-fleet-and-members.md?tabs=hubless#create-a-fleet-resource
74-
[create-public-hubful-fleet]: quickstart-create-fleet-and-members.md?tabs=hubful#public-hub
75-
[create-private-hubful-fleet]: quickstart-create-fleet-and-members.md?tabs=hubful#private-hub
72+
[create-fleet-without-hub]: quickstart-create-fleet-and-members.md?tabs=without-hub-cluster#create-a-fleet-resource
73+
[create-public-hub-cluster]: quickstart-create-fleet-and-members.md?tabs=with-hub-cluster#public-hub-cluster
74+
[create-private-hub-cluster]: quickstart-create-fleet-and-members.md?tabs=with-hub-cluster#private-hub-cluster

articles/kubernetes-fleet/concepts-rbac.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,19 +12,19 @@ ms.topic: conceptual
1212

1313
[Azure role-based access control (Azure RBAC)][azure-rbac-overview] is an authorization system built on Azure Resource Manager that provides fine-grained access management to Azure resources.
1414

15-
This article provides an overview of the various built-in Azure RBAC roles that you can use to access Azure Kubernetes Fleet Manager (Fleet) resources.
15+
This article provides an overview of the various built-in Azure RBAC roles that you can use to access Azure Kubernetes Fleet Manager (Kubernetes Fleet) resources.
1616

1717
## Control plane
1818

19-
This role grants access to Azure Resource Manager (ARM) Fleet resources and subresources, and is applicable both to hubless and hubful Fleets.
19+
This role grants access to Azure Resource Manager (ARM) Fleet resources and subresources, and is applicable both Kubernetes Fleet resource with and without a hub cluster.
2020

2121
|Role name|Description|Usage|
2222
|---------|-----------|-----|
23-
|[Azure Kubernetes Fleet Manager Contributor][azure-rbac-fleet-manager-contributor-role]|This role grants read and write access to Azure resources provided by Azure Kubernetes Fleet Manager, including fleets, fleet members, fleet update strategies, fleet update runs, and more.|You can use this role to grant Contributor permissions that apply solely to Fleet resources and subresources. For example, this role can be given to an Azure administrator tasked with defining and maintaining Fleet resources.|
23+
|[Azure Kubernetes Fleet Manager Contributor][azure-rbac-fleet-manager-contributor-role]|This role grants read and write access to Azure resources provided by Azure Kubernetes Fleet Manager, including fleets, fleet members, fleet update strategies, fleet update runs, and more.|You can use this role to grant Contributor permissions that apply solely to Kubernetes Fleet resources and subresources. For example, this role can be given to an Azure administrator tasked with defining and maintaining Fleet resources.|
2424

2525
## Data plane
2626

27-
These roles grant access to Fleet hub Kubernetes objects, and are therefore only applicable to hubful Fleets.
27+
These roles grant access to Fleet hub Kubernetes objects, and are therefore only applicable to Kubernetes Fleet resources with a hub cluster.
2828

2929
You can assign data plane roles at the Fleet hub cluster scope, or at an individual Kubernetes namespace scope by appending `/namespace/<namespace>` to the role assignment scope.
3030

@@ -37,7 +37,7 @@ You can assign data plane roles at the Fleet hub cluster scope, or at an individ
3737

3838
## Example role assignments
3939

40-
You can grant Azure RBAC roles using the [Azure CLI][azure-cli-overview]. For example, to create a role assignment at the Fleet hub cluster scope:
40+
You can grant Azure RBAC roles using the [Azure CLI][azure-cli-overview]. For example, to create a role assignment at the Kubernetes Fleet hub cluster scope:
4141

4242
```azurecli-interactive
4343
IDENTITY=$(az ad signed-in-user show --output tsv --query id)
@@ -46,7 +46,7 @@ FLEET_ID=$(az fleet show --resource-group $GROUP --name $FLEET --output tsv --qu
4646
az role assignment create --role 'Azure Kubernetes Fleet Manager RBAC Reader' --assignee "$IDENTITY" --scope "$FLEET_ID"
4747
```
4848

49-
You can also scope role assignments to an individual Kubernetes namespace. For example, to create a role assignment for a Fleet hub's default Kubernetes namespace:
49+
You can also scope role assignments to an individual Kubernetes namespace. For example, to create a role assignment for a Kubernetes Fleet hub's default Kubernetes namespace:
5050

5151
```azurecli-interactive
5252
IDENTITY=$(az ad signed-in-user show --output tsv --query id)

articles/kubernetes-fleet/concepts-resource-propagation.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,12 @@ The `ClusterResourcePlacement` object supports [using ConfigMap to envelope the
6262

6363
For more information, see the [`ClusterResourcePlacement` API reference][clusterresourceplacement-api].
6464

65-
Once you select the resources, multiple placement policies are available:
65+
When creating the `ClusterResourcePlacement`, the following affinity types can be specified:
66+
67+
- **requiredDuringSchedulingIgnoredDuringExecution**: As this affinity is of the required type during scheduling, it **filters** the clusters based on their properties.
68+
- **preferredDuringSchedulingIgnoredDuringExecution**: As this affinity is only of the preferred type, but is not required during scheduling, it provides preferential ranking to clusters based on properties specified by you such as cost or resource availability.
69+
70+
Multiple placement types are available for controlling the number of clusters to which the Kubernetes resource needs to be propagated:
6671

6772
* `PickAll` places the resources into all available member clusters. This policy is useful for placing infrastructure workloads, like cluster monitoring or reporting applications.
6873
* `PickFixed` places the resources into a specific list of member clusters by name.

articles/kubernetes-fleet/concepts-update-orchestration.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ An update run can be in one of the following states:
4949
- **NotStarted**: State of the update run before it is started.
5050
- **Running**: Upgrade is in progress for at least one of the clusters in the update run.
5151
- **Pending**:
52-
- **Member cluster**: A member cluster can be in the pending state for any of the following reasons and are surfaced under the message field -
52+
- **Member cluster**: A member cluster can be in the pending state for any of the following reasons and are surfaced under the message field.
5353
- Maintenance window is not open. Message indicates next opening time.
5454
- Target Kubernetes version is not yet available in the region of the member. Message links to the release tracker so that you can check status of the release across regions.
5555
- Target node image version is not yet available in the region of the member. Message links to the release tracker so that you can check status of the release across regions.
@@ -64,7 +64,7 @@ An update run can be in one of the following states:
6464
- When consistent node image is chosen for an upgrade run, if it's not possible to find the target image version for one of the node pools, then upgrade is skipped for that cluster. An example situation for this is when a new node pool with a new VM SKU is added after an update run has started.
6565
- **Group**:
6666
- All member clusters were detected as `Skipped` by the system.
67-
- You initiated a skip at the group level
67+
- You initiated a skip at the group level.
6868
- **Stage**:
6969
- All groups in the stage were detected as `Skipped` by the system.
7070
- You initiated a skip at the stage level.

0 commit comments

Comments
 (0)