Skip to content

Commit 224c1a0

Browse files
tfosmarktfosmark
authored
Merge pull request #115491 from mlearned/mdl-haiku-public-preview-arc-k8s
Mdl haiku public preview arc k8s
2 parents 3cb0d8f + 381c4fb commit 224c1a0

12 files changed

+1347
-0
lines changed

‎articles/azure-arc/kubernetes/connect-cluster.md

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
---
2+
title: "Connect an Azure Arc-enabled Kubernetes cluster (Preview)"
3+
services: azure-arc
4+
ms.service: azure-arc
5+
#ms.subservice: azure-arc-kubernetes coming soon
6+
ms.date: 05/19/2020
7+
ms.topic: article
8+
author: mlearned
9+
ms.author: mlearned
10+
description: "Connect an Azure Arc-enabled Kubernetes cluster with Azure Arc"
11+
keywords: "Kubernetes, Arc, Azure, K8s, containers"
12+
---
13+
14+
# Connect an Azure Arc-enabled Kubernetes cluster (Preview)
15+
16+
Connect a Kubernetes cluster to Azure Arc.
17+
18+
## Before you begin
19+
20+
Verify you have the following requirements ready:
21+
22+
* A Kubernetes cluster that is up and running
23+
* You'll need access with kubeconfig, and cluster-admin access.
24+
* The user or service principal used with `az login` and `az connectedk8s connect` commands must have the 'Read' and 'Write' permissions on the 'Microsoft.Kubernetes/connectedclusters' resource type.
25+
* Latest version of the *connectedk8s* and *k8sconfiguration* extensions
26+
27+
## Supported regions
28+
29+
* East US
30+
* West Europe
31+
32+
## Network requirements
33+
34+
Azure Arc agents require the following protocols/ports/outbound URLs to function.
35+
36+
* TCP on port 443 --> `https://:443`
37+
* TCP on port 9418 --> `git://:9418`
38+
39+
| Endpoint (DNS) | Description |
40+
| ------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- |
41+
| `https://management.azure.com` | Required for the agent to connect to Azure and register the cluster |
42+
| `https://eastus.dp.kubernetesconfiguration.azure.com`, `https://westeurope.dp.kubernetesconfiguration.azure.com` | Data plane endpoint for the agent to push status and fetch configuration information |
43+
| `https://docker.io` | Required to pull container images |
44+
| `https://github.com`, git://github.com | Example GitOps repos are hosted on GitHub. Configuration agent requires connectivity to whichever git endpoint you specify. |
45+
| `https://login.microsoftonline.com` | Required to fetch and update Azure Resource Manager tokens |
46+
| `https://azurearcfork8s.azurecr.io` | Required to pull container images for Azure Arc agents |
47+
48+
## Register the two providers for Azure Arc enabled Kubernetes:
49+
50+
```console
51+
az provider register --namespace Microsoft.Kubernetes
52+
Registering is still on-going. You can monitor using 'az provider show -n Microsoft.Kubernetes'
53+
54+
az provider register --namespace Microsoft.KubernetesConfiguration
55+
Registering is still on-going. You can monitor using 'az provider show -n Microsoft.KubernetesConfiguration'
56+
```
57+
58+
Registration is an asynchronous process. Registration may take approximately 10 minutes. You can monitor the registration process with the following commands:
59+
60+
```console
61+
az provider show -n Microsoft.Kubernetes -o table
62+
```
63+
64+
```console
65+
az provider show -n Microsoft.KubernetesConfiguration -o table
66+
```
67+
68+
## Install CLI extensions
69+
70+
Install the `connectedk8s` extension, which helps you connect Kubernetes clusters to Azure:
71+
72+
```console
73+
az extension add --name connectedk8s
74+
```
75+
76+
Install the `k8sconfiguration` extension:
77+
78+
```console
79+
az extension add --name k8sconfiguration
80+
```
81+
82+
Run the following commands to update the extensions to the latest versions.
83+
84+
```console
85+
az extension update --name connectedk8s
86+
az extension update --name k8sconfiguration
87+
```
88+
89+
## Create a Resource Group
90+
91+
Use a resource group to store metadata for your cluster.
92+
93+
First, create a resource group to hold the connected cluster resource.
94+
95+
```console
96+
az group create --name AzureArcTest -l EastUS -o table
97+
```
98+
99+
**Output:**
100+
101+
```console
102+
Location Name
103+
---------- ------------
104+
eastus AzureArcTest
105+
```
106+
107+
## Connect a cluster
108+
109+
Next, we will connect our Kubernetes cluster to Azure. The workflow for `az connectedk8s connect` is as follows:
110+
111+
1. Verify connectivity to your Kubernetes cluster: via `KUBECONFIG`, `~/.kube/config`, or `--kube-config`
112+
1. Deploy Azure Arc Agents for Kubernetes using Helm 3, into the `azure-arc` namespace
113+
114+
```console
115+
az connectedk8s connect --name AzureArcTest1 --resource-group AzureArcTest
116+
```
117+
118+
**Output:**
119+
120+
```console
121+
Command group 'connectedk8s' is in preview. It may be changed/removed in a future release.
122+
Helm release deployment succeeded
123+
124+
{
125+
"aadProfile": {
126+
"clientAppId": "",
127+
"serverAppId": "",
128+
"tenantId": ""
129+
},
130+
"agentPublicKeyCertificate": "...",
131+
"agentVersion": "0.1.0",
132+
"id": "/subscriptions/57ac26cf-a9f0-4908-b300-9a4e9a0fb205/resourceGroups/AzureArcTest/providers/Microsoft.Kubernetes/connectedClusters/AzureArcTest1",
133+
"identity": {
134+
"principalId": null,
135+
"tenantId": null,
136+
"type": "None"
137+
},
138+
"kubernetesVersion": "v1.15.0",
139+
"location": "eastus",
140+
"name": "AzureArcTest1",
141+
"resourceGroup": "AzureArcTest",
142+
"tags": {},
143+
"totalNodeCount": 1,
144+
"type": "Microsoft.Kubernetes/connectedClusters"
145+
}
146+
```
147+
148+
## Verify connected cluster
149+
150+
List your connected clusters:
151+
152+
```console
153+
az connectedk8s list -g AzureArcTest
154+
```
155+
156+
**Output:**
157+
158+
```console
159+
Command group 'connectedk8s' is in preview. It may be changed/removed in a future release.
160+
Name Location ResourceGroup
161+
------------- ---------- ---------------
162+
AzureArcTest1 eastus AzureArcTest
163+
```
164+
165+
Azure Arc enabled Kubernetes deploys a few operators into the `azure-arc` namespace. You can view these deployments and pods here:
166+
167+
```console
168+
kubectl -n azure-arc get deploy,po
169+
```
170+
171+
**Output:**
172+
173+
```console
174+
NAME READY UP-TO-DATE AVAILABLE AGE
175+
deployment.apps/cluster-metadata-operator 1/1 1 1 16h
176+
deployment.apps/clusteridentityoperator 1/1 1 1 16h
177+
deployment.apps/config-agent 1/1 1 1 16h
178+
deployment.apps/controller-manager 1/1 1 1 16h
179+
deployment.apps/flux-logs-agent 1/1 1 1 16h
180+
deployment.apps/metrics-agent 1/1 1 1 16h
181+
deployment.apps/resource-sync-agent 1/1 1 1 16h
182+
183+
NAME READY STATUS RESTART AGE
184+
pod/cluster-metadata-operator-7fb54d9986-g785b 2/2 Running 0 16h
185+
pod/clusteridentityoperator-6d6678ffd4-tx8hr 3/3 Running 0 16h
186+
pod/config-agent-544c4669f9-4th92 3/3 Running 0 16h
187+
pod/controller-manager-fddf5c766-ftd96 3/3 Running 0 16h
188+
pod/flux-logs-agent-7c489f57f4-mwqqv 2/2 Running 0 16h
189+
pod/metrics-agent-58b765c8db-n5l7k 2/2 Running 0 16h
190+
pod/resource-sync-agent-5cf85976c7-522p5 3/3 Running 0 16h
191+
```
192+
193+
## Azure Arc agents for Kubernetes
194+
195+
Azure Arc enabled Kubernetes consists of a few agents (operators) that run in your cluster deployed to the `azure-arc` namespace.
196+
197+
* `deploy/config-agent`: watches the connected cluster for source control configuration resources applied on the cluster and updates compliance state
198+
* `deploy/controller-manager`: is an operator of operators and orchestrates interactions between Azure Arc components
199+
200+
## Delete a connected cluster
201+
202+
You can delete a `Microsoft.Kubernetes/connectedcluster` resource using the CLI or Azure portal.
203+
204+
The Azure CLI command `az connectedk8s delete` removes the `Microsoft.Kubernetes/connectedCluster` resource in Azure. The Azure CLI deletes any associated `sourcecontrolconfiguration` resources in Azure. The Azure CLI uses helm uninstall to remove the agents in the cluster.
205+
206+
The Azure portal deletes the `Microsoft.Kubernetes/connectedcluster` resource in Azure, and deletes any associated `sourcecontrolconfiguration` resources in Azure.
207+
208+
To remove the agents in the cluster you need to run `az connectedk8s delete` or `helm uninstall azurearcfork8s`.
209+
210+
## Next steps
211+
212+
* [Use GitOps in a connected cluster](./use-gitops-connected-cluster.md)
213+
* [Use Azure Policy to govern cluster configuration](./use-azure-policy.md)

‎articles/azure-arc/kubernetes/create-onboarding-service-principal.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
title: "Create an Azure Arc-enabled onboarding Service Principal (Preview)"
3+
services: azure-arc
4+
ms.service: azure-arc
5+
#ms.subservice: azure-arc-kubernetes coming soon
6+
ms.date: 05/19/2020
7+
ms.topic: article
8+
author: mlearned
9+
ms.author: mlearned
10+
description: "Create an Azure Arc-enabled onboarding Service Principal "
11+
keywords: "Kubernetes, Arc, Azure, containers"
12+
---
13+
14+
# Create an Azure Arc-enabled onboarding Service Principal (Preview)
15+
16+
## Overview
17+
18+
When a cluster is onboarded to Azure, the agents running in your cluster must authenticate to Azure Resource Manager as part of registration. The `connectedk8s` CLI extension has automated Service Principal creation. However, there may be a few scenarios where the CLI automation does not work:
19+
20+
* Your organization generally restricts the creation of Service Principals
21+
* The user onboarding the cluster does not have sufficient permissions to create Service Principals
22+
23+
Instead, let's create the Service Principal out of band, and then pass the principal to the CLI extension.
24+
25+
## Create a new Service Principal
26+
27+
Create a new Service Principal with an informative name. Note that this name must be unique for your Azure Active Directory tenant:
28+
29+
```console
30+
az ad sp create-for-RBAC --skip-assignment --name "https://azure-arc-for-k8s-onboarding"
31+
```
32+
33+
**Output:**
34+
35+
```console
36+
{
37+
"appId": "22cc2695-54b9-49c1-9a73-2269592103d8",
38+
"displayName": "azure-arc-for-k8s-onboarding",
39+
"name": "https://azure-arc-for-k8s-onboarding",
40+
"password": "09d3a928-b223-4dfe-80e8-fed13baa3b3d",
41+
"tenant": "72f988bf-86f1-41af-91ab-2d7cd011db47"
42+
}
43+
```
44+
45+
## Assign permissions
46+
47+
After creating the new Service Principal, assign the "Azure Arc for Kubernetes Onboarding" role to the newly created principal. This is a built-in Azure role with limited permissions, which only allows the principal to register clusters to Azure. The principal cannot update, delete, or modify any other clusters or resources within the subscription.
48+
49+
Given the limited abilities, customers can easily re-use this principal to onboard multiple clusters.
50+
51+
Permissions may be further limited by passing in the appropriate `--scope` argument when assigning the role. This allows customers to restrict cluster registration. The following scenarios are supported by various `--scope` parameters:
52+
53+
| Resource | `scope` argument| Effect |
54+
| ------------- | ------------- | ------------- |
55+
| Subscription | `--scope /subscriptions/0b1f6471-1bf0-4dda-aec3-111122223333` | Service principal can register any cluster in an existing Resource Group in the given subscription |
56+
| Resource Group | `--scope /subscriptions/0b1f6471-1bf0-4dda-aec3-111122223333/resourceGroups/myGroup` | Service principal can __only__ register clusters in the Resource Group `myGroup` |
57+
58+
```console
59+
az role assignment create \
60+
--role 34e09817-6cbe-4d01-b1a2-e0eac5743d41 \ # this is the id for the built-in role
61+
--assignee 22cc2695-54b9-49c1-9a73-2269592103d8 \ # use the appId from the new SP
62+
--scope /subscriptions/<<SUBSCRIPTION_ID>> # apply the apropriate scope
63+
```
64+
65+
**Output:**
66+
67+
```console
68+
{
69+
"canDelegate": null,
70+
"id": "/subscriptions/<<SUBSCRIPTION_ID>>/providers/Microsoft.Authorization/roleAssignments/fbd819a9-01e8-486b-9eb9-f177ba400ba6",
71+
"name": "fbd819a9-01e8-486b-9eb9-f177ba400ba6",
72+
"principalId": "ddb0ddb4-ba84-4cde-b936-affc272a4b90",
73+
"principalType": "ServicePrincipal",
74+
"roleDefinitionId": "/subscriptions/<<SUBSCRIPTION_ID>>/providers/Microsoft.Authorization/roleDefinitions/34e09817-6cbe-4d01-b1a2-e0eac5743d41",
75+
"scope": "/subscriptions/<<SUBSCRIPTION_ID>>",
76+
"type": "Microsoft.Authorization/roleAssignments"
77+
}
78+
```
79+
80+
## Use Service Principal with CLI
81+
82+
Reference the newly created Service Principal:
83+
84+
```console
85+
az login --service-principal -u mySpnClientId -p mySpnClientSecret --tenant myTenantID
86+
az connectedk8s connect -n myConnectedClusterName -g myResoureGroupName
87+
```
88+
89+
## Next steps
90+
91+
* [Use Azure Policy to govern cluster configuration](./use-azure-policy.md)

‎articles/azure-arc/kubernetes/deploy-azure-iot-edge-workloads.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
title: "Deploy Azure IoT Edge workloads (Preview)"
3+
services: azure-arc
4+
ms.service: azure-arc
5+
#ms.subservice: azure-arc-kubernetes coming soon
6+
ms.date: 05/19/2020
7+
ms.topic: article
8+
author: mlearned
9+
ms.author: mlearned
10+
description: "Deploy Azure IoT Edge workloads"
11+
keywords: "Kubernetes, Arc, Azure, K8s, containers"
12+
---
13+
14+
15+
# Deploy Azure IoT Edge workloads (Preview)
16+
17+
## Overview
18+
19+
Azure Arc and Azure IoT Edge complement each other's capabilities quite well. Azure Arc provides mechanisms for cluster operators to the configure the foundational components of a cluster as well as apply and enforce cluster policies. And IoT Edge allows application operators to remotely deploy and manage the workloads at scale with convenient cloud ingestion and bi-directional communication primitives. The diagram below illustrates this:
20+
21+
![](./media/edge-arc.png)
22+
23+
## Pre-requisites
24+
25+
* [Register an IoT Edge device](https://docs.microsoft.com/azure/iot-edge/quickstart-linux#register-an-iot-edge-device) and [deploy the simulated temperature sensor module](https://docs.microsoft.com/azure/iot-edge/quickstart-linux#deploy-a-module). Be sure to note the device's connection string.
26+
27+
* Use [IoT Edge's support for Kubernetes](https://aka.ms/edgek8sdoc) to deploy it via Azure Arc's Flux operator.
28+
29+
* Download the [**values.yaml**](https://github.com/Azure/iotedge/blob/master/kubernetes/charts/edge-kubernetes/values.yaml) file for IoT Edge Helm chart and replace the **deviceConnectionString** placeholder at the end of the file with the one noted in Step 1. You can set any other supported chart installation options as required. Create a namespace for the IoT Edge workload and create add a secret in it:
30+
31+
```
32+
$ kubectl create ns iotedge
33+
34+
$ kubectl create secret generic dcs --from-file=fully-qualified-path-to-values.yaml --namespace iotedge
35+
```
36+
37+
>You can also set this up remotely using the [cluster config example](./use-gitops-connected-cluster.md).
38+
39+
## Connect a cluster
40+
41+
Use the `az` CLI `connectedk8s` extension to connect a Kubernetes cluster to Azure Arc:
42+
43+
```
44+
az connectedk8s connect --name AzureArcIotEdge --resource-group AzureArcTest
45+
```
46+
47+
## Create a configuration for IoT Edge
48+
49+
Example repo: https://github.com/veyalla/edgearc
50+
51+
This repo points to the IoT Edge Helm chart and references the secret created in the pre-requisites section.
52+
53+
1. Use the `az` CLI `k8sconfiguration` extension to create a configuration to link the connected cluster to the git repo:
54+
55+
```
56+
az k8sconfiguration create --name iotedge --cluster-name AzureArcIotEdge --resource-group AzureArcTest --operator-instance-name iotedge --operator-namespace azure-arc-iot-edge --enable-helm-operator --helm-operator-chart-version 0.6.0 --helm-operator-chart-values "--set helm.versions=v3" --repository-url "git://github.com/veyalla/edgearc.git" --cluster-scoped
57+
```
58+
59+
In a minute or two, you should see the IoT Edge workload modules deployed into the `iotedge` namespace in your cluster. You can view the logs of the `SimulatedTemperatureSensor` pod in that namespace to see the sample values being generated. You can also watch the messages arrive at your IoT hub by using the [Azure IoT Hub Toolkit extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=vsciot-vscode.azure-iot-toolkit).
60+
61+
## Cleanup
62+
63+
You can remove the configuration using:
64+
65+
```
66+
az k8sconfiguration delete -g AzureArcTest --cluster-name AzureArcIotEdge --name iotedge
67+
```
68+
69+
## Next steps
70+
71+
[Use Azure Policy to govern cluster configuration](./use-azure-policy.md)

0 commit comments

Comments
 (0)