Skip to content

Commit b666246

Browse files
bwrenbwren
committed
cli
1 parent c5a4378 commit b666246

File tree

2 files changed

+120
-78
lines changed

2 files changed

+120
-78
lines changed

‎articles/azure-monitor/essentials/edge-pipeline-configure.md

Lines changed: 120 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,11 @@ The following components are required to enable and configure the Azure Monitor
2828
| Data flow | Combination of receivers and exporters that run on the pipeline controller instance. Receivers accept data from clients, and exporters to deliver that data to Azure Monitor. |
2929
| Pipeline configuration | Configuration file that defines the data flows for the pipeline instance. Each data flow includes a receiver and an exporter. The receiver listens for incoming data, and the exporter sends the data to the destination. |
3030
| Data collection endpoint (DCE) | Endpoint where the data is sent to the Azure Monitor pipeline. The pipeline configuration includes a property for the URL of the DCE so the pipeline instance knows where to send the data. |
31+
32+
| Configuration | Description |
33+
|:---|:---|
3134
| Data collection rule (DCR) | Configuration file that defines how the data is received in the cloud pipeline and where it's sent. The DCR can also include a transformation to filter or modify the data before it's sent to the destination. |
35+
| Pipeline configuration | Configuration that defines the data flows for the pipeline instance, including the data flows and cache. |
3236

3337
## Supported configurations
3438

@@ -79,6 +83,14 @@ The following table and diagram describe the detailed steps and components in th
7983
:::image type="content" source="media/edge-pipeline/cloud-pipeline-data-flow.png" lightbox="media/edge-pipeline/cloud-pipeline-data-flow.png" alt-text="Detailed diagram of the steps and components for data collection using Azure Monitor cloud pipeline." border="false":::
8084

8185
## Segmented network
86+
[Network segmentation](/azure/architecture/networking/guide/network-level-segmentation) is a model where you use software defined perimeters to create a different security posture for different parts of your network. In this model, you may have a network segment that can't connect to the internet or to other network segments. The edge pipeline can be used to collect data from these network segments and send it to the cloud pipeline.
87+
88+
:::image type="content" source="media/edge-pipeline/layered-network.png" lightbox="media/edge-pipeline/layered-network.png" alt-text="Diagram of a layered network for Azure Monitor edge pipeline." border="false":::
89+
90+
To use Azure Monitor pipeline in a layered network configuration, you must add the following URLs to the allowlist for the Arc-enabled Kubernetes cluster. See [Configure Azure IoT Layered Network Management Preview on level 4 cluster](/azure/iot-operations/manage-layered-network/howto-configure-l4-cluster-layered-network?tabs=k3s#configure-layered-network-management-preview-service).
91+
92+
- `*.ingest.monitor.azure.com`
93+
- Url of DCE.
8294

8395

8496
## Create table in Log Analytics workspace
@@ -92,6 +104,7 @@ az monitor log-analytics workspace table create --workspace-name my-workspace --
92104
```
93105

94106

107+
95108
## Enable cache
96109
Edge devices in some environments may experience intermittent connectivity due to various factors such as network congestion, signal interference, power outage, or mobility. In these environments, you can configure the edge pipeline to cache data by creating a [persistent volume](https://kubernetes.io) in your cluster. The process for this will vary based on your particular environment, but the configuration must meet the following requirements:
97110

@@ -147,100 +160,42 @@ Following are the steps required to create and configure the components required
147160

148161

149162
### Edge pipeline extension
150-
The following ARM template adds the edge pipeline extension to your Arc-enabled Kubernetes cluster. Replace the properties in the following table before deploying the template.
163+
The following command adds the edge pipeline extension to your Arc-enabled Kubernetes cluster. Replace the properties in the following table before deploying the template.
151164

152-
| Parameter | Description |
153-
|:---|:--|
154-
| `name` | Name of the pipeline extension. Must be unique for the subscription. |
155-
| `scope` | Resource ID of your Arc-enabled Kubernetes cluster. |
156-
| `releaseNamespace` | Namespace in the cluster where the extension will be deployed. |
165+
```azurecli
166+
az k8s-extension create --name <pipeline-extension-name> --extension-type microsoft.monitor.pipelinecontroller --scope cluster --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type connectedClusters --release-train Preview
157167
158-
```json
159-
{
160-
"type": "Microsoft.KubernetesConfiguration/extensions",
161-
"apiVersion": "2022-11-01",
162-
"name": "my-pipeline-extension",
163-
"scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Kubernetes/connectedClusters/my-arc-cluster",
164-
"identity": {
165-
"type": "SystemAssigned"
166-
},
167-
"properties": {
168-
"aksAssignedIdentity": {
169-
"type": "SystemAssigned"
170-
},
171-
"autoUpgradeMinorVersion": false,
172-
"extensionType": "microsoft.monitor.pipelinecontroller",
173-
"releaseTrain": "preview",
174-
"scope": {
175-
"cluster": {
176-
"releaseNamespace": "my-strato-ns"
177-
}
178-
},
179-
"version": "0.37.3-privatepreview"
180-
}
181-
}
168+
## Example
169+
az k8s-extension create --name my-pipe --extension-type microsoft.monitor.pipelinecontroller --scope cluster --cluster-name my-cluster --resource-group my-resource-group --cluster-type connectedClusters --release-train Preview
182170
```
183171

184172
### Custom location
185173
The following ARM template creates the custom location for to your Arc-enabled Kubernetes cluster. Replace the properties in the following table before deploying the template.
186174

187-
| Parameter | Description |
188-
|:---|:--|
189-
| `name` | Name of the custom location. Must be unique for the cluster. |
190-
| `location` | Location of the custom location. |
191-
| `hostResourceId` | Resource ID of the Arc-enabled Kubernetes cluster. |
192-
| `namespace` | Namespace for the custom location. Can use the custom location name. |
193-
| `clusterExtensionIds` | Resource ID of the edge pipeline extension created in the previous step. |
194-
195-
196-
```json
197-
{
198-
"type": "Microsoft.ExtendedLocation/customLocations",
199-
"name": "custom-location-name",
200-
"location": "eastus",
201-
"apiVersion": "2021-08-15",
202-
"properties": {
203-
"hostResourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Kubernetes/connectedClusters/my-arc-cluster",
204-
"namespace": "custom-location-name",
205-
"clusterExtensionIds": [
206-
"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Kubernetes/connectedClusters/strato-01/providers/Microsoft.KubernetesConfiguration/extensions/my-pipeline-extension"
207-
],
208-
"hostType": "Kubernetes"
209-
}
210-
}
211-
```
212175

213176
```azurecli
177+
az connectedk8s enable-features -n <clusterName> -g <resourceGroupName> --features cluster-connect custom-locations
178+
az customlocation create -n <customLocationName> -g <resourceGroupName> --namespace <name of namespace> --host-resource-id <connectedClusterId> --cluster-extension-ids <extensionId>
214179
180+
## Example
181+
az connectedk8s enable-features -n <clusterName> -g <resourceGroupName> --features cluster-connect custom-locations
182+
az customlocation create -n <customLocationName> -g <resourceGroupName> --namespace <name of namespace> --host-resource-id <connectedClusterId> --cluster-extension-ids <extensionId>
215183
```
216184

217185

218186

219187
### DCE
220188
The following ARM template creates the [data collection endpoint (DCE)](./data-collection-endpoint-overview.md) required for the edge pipeline to connect to the cloud pipeline. You can use an existing DCE if you already have one in the same region. Replace the properties in the following table before deploying the template.
221189

222-
| Parameter | Description |
223-
|:---|:--|
224-
| `name` | Name of the DCE. Must be unique for the subscription. |
225-
| `location` | Location of the DCE. Must match the location of the DCR. |
190+
```azurecli
191+
az monitor data-collection endpoint create -g "myResourceGroup" -l "eastus2euap" --name "myCollectionEndpoint" --public-network-access "Enabled"
226192
227-
```json
228-
{
229-
"type": "Microsoft.Insights/dataCollectionEndpoints",
230-
"name": "my-dce",
231-
"location": "eastus",
232-
"apiVersion": "2021-04-01",
233-
"properties": {
234-
"configurationAccess": {},
235-
"logsIngestion": {},
236-
"networkAcls": {
237-
"publicNetworkAccess": "Enabled"
238-
}
239-
}
240-
}
193+
## Example
194+
az monitor data-collection endpoint create --name strato-06-dce --resource-group strato --public-network-access "Enabled"
241195
```
242196

243197

198+
244199
### DCR
245200
The DCR is stored in Azure Monitor and defines how the data will be processed when its received from the edge pipeline. The edge pipeline configuration specifies the `immutable ID` of the DCR and the `stream` in the DCR that will process the data.
246201

@@ -265,10 +220,6 @@ Replace the properties in the following table before deploying the template. See
265220

266221
```json
267222
{
268-
"type": "Microsoft.Insights/dataCollectionRules",
269-
"name": "my-dcr",
270-
"location": "eastus",
271-
"apiVersion": "2021-09-01-preview",
272223
"properties": {
273224
"dataCollectionEndpointId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Insights/dataCollectionEndpoints/my-dce",
274225
"streamDeclarations": {
@@ -341,6 +292,17 @@ Replace the properties in the following table before deploying the template. See
341292
}
342293
```
343294

295+
Install the DCR using the following command:
296+
297+
```azurecli
298+
az monitor data-collection rule create --name 'myDCRName' --location <location> --resource-group <resource-group> --rule-file '<dcr-file-path.json>'
299+
300+
## Example
301+
az monitor data-collection rule create --name my-pipeline-dcr --location westus2 --resource-group 'my-resource-group' --rule-file 'C:\MyDCR.json'
302+
303+
```
304+
305+
344306
### DCR access
345307
The Arc-enabled Kubernetes cluster must have access to the DCR to send data to the cloud pipeline. You can use commands in the Azure CLI to grant the necessary permissions.
346308

@@ -350,7 +312,7 @@ Use the following command to retrieve the object id of the System Assigned Ident
350312
az k8s-extension show --name <extension-name> --cluster-name <cluster-name> --resource-group <resource-group> --cluster-type connectedClusters --query "identity.principalId" -o tsv
351313
352314
## Example:
353-
az k8s-extension show --name my-pipeline-extension--cluster-name my-cluster --resource-group my-resource-group --cluster-type connectedClusters --query "identity.principalId" -o tsv
315+
az k8s-extension show --name my-pipeline-extension --cluster-name my-cluster --resource-group my-resource-group --cluster-type connectedClusters --query "identity.principalId" -o tsv
354316
```
355317

356318
Use the output from this command as input to the following command to give Azure Monitor pipeline the authority to send its telemetry to the DCR.
@@ -848,8 +810,88 @@ You can deploy all of the required components for the Azure Monitor edge pipelin
848810
}
849811
```
850812

813+
### [ARM](#tab/arm)
814+
815+
| Parameter | Description |
816+
|:---|:--|
817+
| `name` | Name of the pipeline extension. Must be unique for the subscription. |
818+
| `scope` | Resource ID of your Arc-enabled Kubernetes cluster. |
819+
| `releaseNamespace` | Namespace in the cluster where the extension will be deployed. |
820+
821+
```json
822+
{
823+
"type": "Microsoft.KubernetesConfiguration/extensions",
824+
"apiVersion": "2022-11-01",
825+
"name": "my-pipeline-extension",
826+
"scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Kubernetes/connectedClusters/my-arc-cluster",
827+
"identity": {
828+
"type": "SystemAssigned"
829+
},
830+
"properties": {
831+
"aksAssignedIdentity": {
832+
"type": "SystemAssigned"
833+
},
834+
"autoUpgradeMinorVersion": false,
835+
"extensionType": "microsoft.monitor.pipelinecontroller",
836+
"releaseTrain": "preview",
837+
"scope": {
838+
"cluster": {
839+
"releaseNamespace": "my-strato-ns"
840+
}
841+
},
842+
"version": "0.37.3-privatepreview"
843+
}
844+
}
845+
846+
847+
848+
| Parameter | Description |
849+
|:---|:--|
850+
| `name` | Name of the custom location. Must be unique for the cluster. |
851+
| `location` | Location of the custom location. |
852+
| `hostResourceId` | Resource ID of the Arc-enabled Kubernetes cluster. |
853+
| `namespace` | Namespace for the custom location. Can use the custom location name. |
854+
| `clusterExtensionIds` | Resource ID of the edge pipeline extension created in the previous step. |
855+
856+
857+
```json
858+
{
859+
"type": "Microsoft.ExtendedLocation/customLocations",
860+
"name": "custom-location-name",
861+
"location": "eastus",
862+
"apiVersion": "2021-08-15",
863+
"properties": {
864+
"hostResourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Kubernetes/connectedClusters/my-arc-cluster",
865+
"namespace": "custom-location-name",
866+
"clusterExtensionIds": [
867+
"/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-resource-group/providers/Microsoft.Kubernetes/connectedClusters/strato-01/providers/Microsoft.KubernetesConfiguration/extensions/my-pipeline-extension"
868+
],
869+
"hostType": "Kubernetes"
870+
}
871+
}
872+
```
873+
851874

875+
| Parameter | Description |
876+
|:---|:--|
877+
| `name` | Name of the DCE. Must be unique for the subscription. |
878+
| `location` | Location of the DCE. Must match the location of the DCR. |
852879

880+
```json
881+
{
882+
"type": "Microsoft.Insights/dataCollectionEndpoints",
883+
"name": "my-dce",
884+
"location": "eastus",
885+
"apiVersion": "2021-04-01",
886+
"properties": {
887+
"configurationAccess": {},
888+
"logsIngestion": {},
889+
"networkAcls": {
890+
"publicNetworkAccess": "Enabled"
891+
}
892+
}
893+
}
894+
```
853895
---
854896

855897
## Verify configuration

‎articles/azure-monitor/essentials/media/edge-pipeline/layered-network.png

55.7 KB
Loading

0 commit comments

Comments
 (0)