Skip to content

Commit 421b154

Browse files
PatAltimorePatAltimore
committed
Merge branch 'release-aio-m2' of https://github.com/MicrosoftDocs/azure-docs-pr into release-aio-m2
2 parents 7a0fd59 + 574edd3 commit 421b154

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+882
-517
lines changed

‎.openpublishing.publish.config.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,12 @@
236236
"branch": "main",
237237
"branch_mapping": {}
238238
},
239+
{
240+
"path_to_root": "azure-iot-operations-samples-m2",
241+
"url": "https://github.com/Azure-Samples/explore-iot-operations",
242+
"branch": "release-m2",
243+
"branch_mapping": {}
244+
},
239245
{
240246
"path_to_root": "azure-iot-sdk-node",
241247
"url": "https://github.com/Azure/azure-iot-sdk-node",

‎articles/iot-operations/configure-observability-monitoring/howto-configure-observability.md

Lines changed: 183 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ ms.date: 02/27/2024
1515

1616
[!INCLUDE [public-preview-note](../includes/public-preview-note.md)]
1717

18-
Observability provides visibility into every layer of your Azure IoT Operations configuration. It gives you insight into the actual behavior of issues, which increases the effectiveness of site reliability engineering. Azure IoT Operations offers observability through custom curated Grafana dashboards that are hosted in Azure. These dashboards are powered by Azure Monitor managed service for Prometheus and by Container Insights. This article shows you how to configure the services you need for observability.
18+
Observability provides visibility into every layer of your Azure IoT Operations configuration. It gives you insight into the actual behavior of issues, which increases the effectiveness of site reliability engineering. Azure IoT Operations offers observability through custom curated Grafana dashboards that are hosted in Azure. These dashboards are powered by Azure Monitor managed service for Prometheus and by Container Insights. This article shows you how to configure the services you need for observability.
1919

2020
## Prerequisites
2121

@@ -36,121 +36,202 @@ az provider register -n "Microsoft.AlertsManagement"
3636
```
3737

3838
## Install observability components
39-
The steps in this section install shared monitoring resources and configure your Arc enabled cluster to emit observability signals to these resources. The shared monitoring resources include Azure Managed Grafana, Azure Monitor Workspace, Azure Managed Prometheus, Azure Log Analytics, and Container Insights.
4039

41-
1. In your console, go to the local folder where you want to clone the Azure IoT Operations repo:
42-
> [!NOTE]
43-
> The repo contains the deployment definition of Azure IoT Operations, and samples that include the sample dashboards used in this article.
40+
The steps in this section install shared monitoring resources and configure your Arc enabled cluster to emit observability signals to these resources. The shared monitoring resources include Azure Managed Grafana, Azure Monitor Workspace, Azure Managed Prometheus, Azure Log Analytics, and Container Insights. In this section, you also deploy an [OpenTelemetry (Otel) Collector](https://opentelemetry.io/docs/collector/)
41+
42+
1. In your console, go to the local folder where you want to clone the Azure IoT Operations repo:
43+
44+
> [!NOTE]
45+
> The repo contains the deployment definition of Azure IoT Operations, and samples that include the sample dashboards used in this article.
4446
4547
1. Clone the repo to your local machine, using the following command:
4648

47-
```shell
48-
git clone https://github.com/Azure/azure-iot-operations.git
49-
```
49+
```shell
50+
git clone https://github.com/Azure/azure-iot-operations.git
51+
```
5052

5153
1. Browse to the following path in your local copy of the repo:
5254

53-
*azure-iot-operations\tools\setup-3p-obs-infra*
54-
55-
1. To deploy the observability components, run the following command. Use the subscription ID and resource group of your Arc-enabled cluster that you want to monitor.
56-
57-
> [!NOTE]
58-
> To discover other optional parameters you can set, see the [bicep file](https://github.com/Azure/azure-iot-operations/blob/main/tools/setup-3p-obs-infra/observability-full.bicep). The optional parameters can specify things like alternative locations for cluster resources.
59-
60-
```azurecli
61-
az deployment group create \
62-
--subscription <subscription-id> \
63-
--resource-group <cluster-resource-group> \
64-
--template-file observability-full.bicep \
65-
--parameters grafanaAdminId=$(az ad user show --id $(az account show --query user.name --output tsv) --query=id --output tsv) \
66-
clusterName=<cluster-name> \
67-
sharedResourceGroup=<shared-resource-group> \
68-
sharedResourceLocation=<shared-resource-location> \
69-
--query=properties.outputs
70-
```
71-
72-
The previous command grants admin access for the newly created Grafana instance to the user who runs it. If that access isn't what you want, run the following command instead. You need to set up permissions manually before anyone can access the Grafana instance.
73-
74-
```azurecli
75-
az deployment group create \
76-
--subscription <subscription-id> \
77-
--resource-group <cluster-resource-group> \
78-
--template-file observability-full.bicep \
79-
--parameters clusterName=<cluster-name> \
80-
sharedResourceGroup=<shared-resource-group> \
81-
sharedResourceLocation=<shared-resource-location> \
55+
*azure-iot-operations\tools\setup-3p-obs-infra*
56+
57+
1. Create a file called `otel-collector-values.yaml` and past the following code into it to define an OpenTelemetry Collector:
58+
59+
```yml
60+
File: otel-collector-values.yaml
61+
mode: deployment
62+
fullnameOverride: aio-otel-collector
63+
image:
64+
repository: otel/opentelemetry-collector
65+
tag: 0.107.0
66+
config:
67+
processors:
68+
memory_limiter:
69+
limit_percentage: 80
70+
spike_limit_percentage: 10
71+
check_interval: '60s'
72+
receivers:
73+
jaeger: null
74+
prometheus: null
75+
zipkin: null
76+
otlp:
77+
protocols:
78+
grpc:
79+
endpoint: ':4317'
80+
http:
81+
endpoint: ':4318'
82+
exporters:
83+
prometheus:
84+
endpoint: ':8889'
85+
resource_to_telemetry_conversion:
86+
enabled: true
87+
service:
88+
extensions:
89+
- health_check
90+
pipelines:
91+
metrics:
92+
receivers:
93+
- otlp
94+
exporters:
95+
- prometheus
96+
logs: null
97+
traces: null
98+
telemetry: null
99+
extensions:
100+
memory_ballast:
101+
size_mib: 0
102+
resources:
103+
limits:
104+
cpu: '100m'
105+
memory: '512Mi'
106+
ports:
107+
metrics:
108+
enabled: true
109+
containerPort: 8889
110+
servicePort: 8889
111+
protocol: 'TCP'
112+
jaeger-compact:
113+
enabled: false
114+
jaeger-grpc:
115+
enabled: false
116+
jaeger-thrift:
117+
enabled: false
118+
zipkin:
119+
enabled: false
120+
```
121+
122+
1. Deploy the collector by running the following commands:
123+
124+
```shell
125+
kubectl get namespace azure-iot-operations || kubectl create namespace azure-iot-operations
126+
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
127+
128+
helm repo update
129+
helm upgrade --install aio-observability open-telemetry/opentelemetry-collector -f otel-collector-values.yaml --namespace azure-iot-operations
130+
```
131+
132+
1. Deploy the observability components by running one of the following commands. Use the subscription ID and resource group of the Arc-enabled cluster that you want to monitor.
133+
134+
> [!NOTE]
135+
> To discover other optional parameters you can set, see the [bicep file](https://github.com/Azure/azure-iot-operations/blob/main/tools/setup-3p-obs-infra/observability-full.bicep). The optional parameters can specify things like alternative locations for cluster resources.
136+
137+
The following command grants admin access for the newly created Grafana instance to the user:
138+
139+
```azurecli
140+
az deployment group create \
141+
--subscription <subscription-id> \
142+
--resource-group <cluster-resource-group> \
143+
--template-file observability-full.bicep \
144+
--parameters grafanaAdminId=$(az ad user show --id $(az account show --query user.name --output tsv) --query=id --output tsv) \
145+
clusterName=<cluster-name> \
146+
sharedResourceGroup=<shared-resource-group> \
147+
sharedResourceLocation=<shared-resource-location> \
148+
--query=properties.outputs
149+
```
150+
151+
If that access isn't what you want, the following command that doesn't configure permissions. Then, set up permissions manually using [role assignments](../../managed-grafana/how-to-share-grafana-workspace.md#add-a-grafana-role-assignment) before anyone can access the Grafana instance. Assign one of the Grafana roles (Grafana Admin, Grafana Editor, Grafana Viewer) depending on the level of access desired.
152+
153+
```azurecli
154+
az deployment group create \
155+
--subscription <subscription-id> \
156+
--resource-group <cluster-resource-group> \
157+
--template-file observability-full.bicep \
158+
--parameters clusterName=<cluster-name> \
159+
sharedResourceGroup=<shared-resource-group> \
160+
sharedResourceLocation=<shared-resource-location> \
82161
--query=properties.outputs
83-
```
84-
85-
To set up permissions manually, [add a role assignment](../../managed-grafana/how-to-share-grafana-workspace.md#add-a-grafana-role-assignment) to the Grafana instance for any users who should have access. Assign one of the Grafana roles (Grafana Admin, Grafana Editor, Grafana Viewer) depending on the level of access desired.
162+
```
86163

87-
If the deployment succeeds, a few pieces of information are printed at the end of the command output. The information includes the Grafana URL and the resource IDs for both the Log Analytics and Azure Monitor resources that were created. The Grafana URL allows you to go to the Grafana instance that you configure in [Deploy dashboards to Grafana](#deploy-dashboards-to-grafana). The two resource IDs enable you to configure other Arc enabled clusters by following the steps in [Add an Arc-enabled cluster to existing observability infrastructure](howto-add-cluster.md).
164+
If the deployment succeeds, a few pieces of information are printed at the end of the command output. The information includes the Grafana URL and the resource IDs for both the Log Analytics and Azure Monitor resources that were created. The Grafana URL allows you to go to the Grafana instance that you configure in [Deploy dashboards to Grafana](#deploy-dashboards-to-grafana). The two resource IDs enable you to configure other Arc enabled clusters by following the steps in [Add an Arc-enabled cluster to existing observability infrastructure](howto-add-cluster.md).
88165

89166
## Configure Prometheus metrics collection
90-
1. Copy and paste the following configuration to a new file named *ama-metrics-prometheus-config.yaml*, and save the file:
91-
92-
```yml
93-
apiVersion: v1
94-
data:
95-
prometheus-config: |2-
96-
scrape_configs:
97-
- job_name: e4k
98-
scrape_interval: 1m
99-
static_configs:
100-
- targets:
101-
- aio-internal-diagnostics-service.azure-iot-operations.svc.cluster.local:9600
102-
- job_name: nats
103-
scrape_interval: 1m
104-
static_configs:
105-
- targets:
106-
- aio-dp-msg-store-0.aio-dp-msg-store-headless.azure-iot-operations.svc.cluster.local:7777
107-
- job_name: otel
108-
scrape_interval: 1m
109-
static_configs:
110-
- targets:
111-
- aio-otel-collector.azure-iot-operations.svc.cluster.local:8889
112-
- job_name: aio-annotated-pod-metrics
113-
kubernetes_sd_configs:
114-
- role: pod
115-
relabel_configs:
116-
- action: drop
117-
regex: true
118-
source_labels:
119-
- __meta_kubernetes_pod_container_init
120-
- action: keep
121-
regex: true
122-
source_labels:
123-
- __meta_kubernetes_pod_annotation_prometheus_io_scrape
124-
- action: replace
125-
regex: ([^:]+)(?::\\d+)?;(\\d+)
126-
replacement: $1:$2
127-
source_labels:
128-
- __address__
129-
- __meta_kubernetes_pod_annotation_prometheus_io_port
130-
target_label: __address__
131-
- action: replace
132-
source_labels:
133-
- __meta_kubernetes_namespace
134-
target_label: kubernetes_namespace
135-
- action: keep
136-
regex: 'azure-iot-operations'
137-
source_labels:
138-
- kubernetes_namespace
139-
scrape_interval: 1m
140-
kind: ConfigMap
141-
metadata:
142-
name: ama-metrics-prometheus-config
143-
namespace: kube-system
144-
```
145-
146-
1. To apply the configuration file you created, run the following command:
147-
148-
`kubectl apply -f ama-metrics-prometheus-config.yaml`
167+
168+
1. Copy and paste the following configuration to a new file named `ama-metrics-prometheus-config.yaml`, and save the file:
169+
170+
```yml
171+
apiVersion: v1
172+
data:
173+
prometheus-config: |2-
174+
scrape_configs:
175+
- job_name: e4k
176+
scrape_interval: 1m
177+
static_configs:
178+
- targets:
179+
- aio-internal-diagnostics-service.azure-iot-operations.svc.cluster.local:9600
180+
- job_name: nats
181+
scrape_interval: 1m
182+
static_configs:
183+
- targets:
184+
- aio-dp-msg-store-0.aio-dp-msg-store-headless.azure-iot-operations.svc.cluster.local:7777
185+
- job_name: otel
186+
scrape_interval: 1m
187+
static_configs:
188+
- targets:
189+
- aio-otel-collector.azure-iot-operations.svc.cluster.local:8889
190+
- job_name: aio-annotated-pod-metrics
191+
kubernetes_sd_configs:
192+
- role: pod
193+
relabel_configs:
194+
- action: drop
195+
regex: true
196+
source_labels:
197+
- __meta_kubernetes_pod_container_init
198+
- action: keep
199+
regex: true
200+
source_labels:
201+
- __meta_kubernetes_pod_annotation_prometheus_io_scrape
202+
- action: replace
203+
regex: ([^:]+)(?::\\d+)?;(\\d+)
204+
replacement: $1:$2
205+
source_labels:
206+
- __address__
207+
- __meta_kubernetes_pod_annotation_prometheus_io_port
208+
target_label: __address__
209+
- action: replace
210+
source_labels:
211+
- __meta_kubernetes_namespace
212+
target_label: kubernetes_namespace
213+
- action: keep
214+
regex: 'azure-iot-operations'
215+
source_labels:
216+
- kubernetes_namespace
217+
scrape_interval: 1m
218+
kind: ConfigMap
219+
metadata:
220+
name: ama-metrics-prometheus-config
221+
namespace: kube-system
222+
```
223+
224+
1. Apply the configuration file by running the following command:
225+
226+
```shell
227+
kubectl apply -f ama-metrics-prometheus-config.yaml
228+
```
149229

150230
## Deploy dashboards to Grafana
231+
151232
Azure IoT Operations provides a collection of dashboards designed to give you many of the visualizations you need to understand the health and performance of your Azure IoT Operations deployment.
152233

153-
Complete the following steps to install the Azure IoT Operations curated Grafana dashboards.
234+
Complete the following steps to install the Azure IoT Operations curated Grafana dashboards.
154235

155236
1. Sign in to the Grafana console, then in the upper right area of the Grafana application, select the **+** icon
156237

@@ -163,4 +244,4 @@ Complete the following steps to install the Azure IoT Operations curated Grafana
163244
## Related content
164245

165246
- [Azure Monitor overview](/azure/azure-monitor/overview)
166-
- [How to Deploy observability resources manually](howto-configure-observability-manual.md)
247+
- [How to deploy observability resources manually](howto-configure-observability-manual.md)

‎articles/iot-operations/deploy-iot-ops/howto-deploy-iot-operations.md

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -27,24 +27,6 @@ Learn how to deploy Azure IoT Operations Preview to a Kubernetes cluster and the
2727

2828
In this article, when we talk about deploying Azure IoT Operations we mean the full set of components that make up a *deployment*. Once the deployment exists, you can view, manage, and update the *instance*.
2929

30-
## Choose your features
31-
32-
Azure IoT Operations offers two deployment modes. You can choose to deploy a basic subset of features that are simpler to get started with for evaluation scenarios, or you can choose to deploy the full feature set.
33-
34-
* Basic feature deployment:
35-
36-
* Does not configure secrets or user-assigned managed identity capabilities.
37-
* Is meant to enable the end-to-end quickstart sample for evaluation purposes, so does support the OPC PLC simulator and connect to cloud resources using system-assigned managed identity.
38-
* Can be upgraded to include the full set of features. For the steps to enable secrets and user-assigned managed identity, see [Configure secrets on your cluster](./howto-manage-secrets.md)
39-
40-
* Full feature deployment:
41-
42-
* Includes the steps to enable secrets and user-assignment managed identity, which are important capabilities for developing a production-ready scenario. Secrets are used whenever Azure IoT Operations components connect to a resource outside of the cluster; for example, an OPC UA server or a dataflow source or destination endpoint.
43-
44-
If you want to deploy the basic subset, see [Quickstart: Run Azure IoT Operations Preview in GitHub Codespaces with K3s](../get-started-end-to-end-sample/quickstart-deploy.md).
45-
46-
This article provides steps to deploy the full feature set for Azure IoT Operations.
47-
4830
## Prerequisites
4931

5032
Cloud resources:

0 commit comments

Comments
 (0)