Skip to content

Commit 6092ed2

Browse files
huypubhuypub
authored
Merge pull request #228493 from MicrosoftDocs/release-preview-azure-operator-distributed-services
Release preview azure operator distributed services--scheduled release at 2PM of 3/26
2 parents 4c52d81 + 34ea69f commit 6092ed2

File tree

54 files changed

+6304
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+6304
-0
lines changed

‎articles/operator-nexus/TOC.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
- name: Azure Operator Nexus documentation
2+
href: index.yml
3+
- name: Overview
4+
items:
5+
- name: What is Azure Operator Nexus?
6+
href: overview.md
7+
- name: Concepts
8+
items:
9+
- name: Resource Types
10+
href: concepts-resource-types.md
11+
- name: Observability
12+
href: concepts-observability.md
13+
- name: Quickstarts
14+
items:
15+
- name: Create Network fabric Controller and Cluster Manager
16+
href: quickstart-network-fabric-controller-cluster-manager-create.md
17+
- name: Before you start platform deployment
18+
href: quickstarts-platform-prerequisites.md
19+
- name: Platform deployment
20+
href: quickstarts-platform-deployment.md
21+
- name: Before you start workload deployment
22+
href: quickstarts-tenant-workload-prerequisites.md
23+
- name: Tenant workload deployment
24+
href: quickstarts-tenant-workload-deployment.md
25+
- name: How-To Guides
26+
items:
27+
- name: BareMetal Functions
28+
href: howto-baremetal-functions.md
29+
- name: Cluster Manager
30+
href: howto-cluster-manager.md
31+
- name: isolation-domain
32+
href: howto-configure-isolation-domain.md
33+
- name: Network fabric
34+
href: howto-configure-network-fabric.md
35+
- name: Network fabric Controller
36+
href: howto-configure-network-fabric-controller.md
37+
- name: AKS-Hybrid
38+
href: howto-hybrid-aks.md
39+
- name: Monitor AKS-Hybrid Cluster
40+
href: howto-monitoring-aks-h-cluster.md
41+
- name: Monitor VMs for VNF
42+
href: howto-monitoring-virtualized-network-functions-virtual-machines.md
43+
- name: Pre-certification
44+
href: howto-precertification.md
45+
- name: Sample Deployment
46+
items:
47+
- name: VNF Deployment
48+
href: template-virtualized-network-function-deployment.md
49+
- name: CNF Deployment
50+
href: template-cloud-native-network-function-deployment.md
51+
- name: Reference
52+
items:
53+
- name: Instance to On-Premises WAN Connectivity
54+
href: reference-customer-edge-provider-edge-connectivity.md
55+
- name: List of Metrics Collected
56+
href: List-of-metrics-collected.md
57+

‎articles/operator-nexus/concepts-observability.md

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
---
2+
title: "Azure Operator Nexus: observability using Azure Monitor"
3+
description: Operator Nexus uses Azure Monitor and collects and aggregates data in Azure Log Analytics workspace. The analysis, visualization, and alerting is performed on this collected data.
4+
author: mukesh-dua #Required; your GitHub user alias, with correct capitalization.
5+
ms.author: mukeshdua #Required; microsoft alias of author; optional team alias.
6+
ms.service: azure #Required
7+
ms.topic: conceptual #Required; leave this attribute/value as-is.
8+
ms.date: 01/31/2023 #Required; mm/dd/yyyy format.
9+
ms.custom: template-concept #Required; leave this attribute/value as-is.
10+
---
11+
12+
# Azure Operator Nexus observability
13+
14+
The Operator Nexus observability framework provides operational insights into your on-premises instances.
15+
The framework supports logging, monitoring, and alerting (LMA), analytics, and visualization of operational (platform and workloads) data and metrics.
16+
17+
<!--- IMG ![ Operator Nexus Logging, Monitoring and Alerting (LMA) Framework](Docs/media/log-monitoring-analytics-framework.png) IMG --->
18+
:::image type="content" source="media/log-monitoring-analytics-framework.png" alt-text="Screenshot of Operator Nexus Logging, Monitoring and Alerting (LMA) Framework.":::
19+
20+
21+
Figure: Operator Nexus Logging, Monitoring and Alerting (LMA) Framework
22+
23+
The key highlights of Operator Nexus observability framework are:
24+
25+
* **Centralized data collection**: Operator Nexus observability solution is based on a collection of all the data in a central place. In this place, you can observe the monitoring data from all of your on-premises instances.
26+
* **Well-defined and tested tooling**: The solution relies on Azure Monitor that collects, analyzes, and acts on telemetry data from your cloud and on-premises instances.
27+
* **Easy to learn and use**: The solution makes it easy for you to analyze and debug problems with the ability to search the data from within or across all of your cloud and on-premises instances.
28+
* **Visualization tools**: You create customized dashboards and workbooks per your needs.
29+
* **Integrated Alert tooling**: You create alerts based on custom thresholds. You can create and reuse alert templates across all of your instances.
30+
31+
This article helps you understand Operator Nexus observability framework that consists of a stack of components:
32+
33+
- Azure Monitor collects and aggregates logging data from the Operator Nexus components
34+
- Azure Log Analytics workspace collects and aggregates logging data from multiple Azure subscriptions and tenants
35+
- Analysis, visualization, and alerting are performed on the aggregated log data.
36+
37+
## Platform Monitoring
38+
39+
Operator Nexus gives you visibility into the performance of your deployments
40+
that consist of [infrastructure resources](./concepts-resource-types.md#platform-components).
41+
You need the logs and metrics to be collected and analyzed from these platform resources.
42+
You gain valuable insights from the centralized collection and aggregation of data from all sources, compared with from dis-aggregated data.
43+
44+
These logs and metrics are used to observe the state of the platform. You can see the performance and analyze what's wrong. You can analyze what caused the situation. Visualization helps you configure the required alerts and under what conditions. For example, you can configure the alerts to be generated when resources are behaving abnormally, or when thresholds have been reached. You can use the collected logs and analytics to debug any problems in the environment.
45+
46+
### Monitoring Data
47+
48+
Operator Nexus observability allows you to collect the same kind of data as other Azure
49+
resources. The data collected from each of your instances can be viewed in your LAW (Log Analytics workspace).
50+
51+
You can learn about monitoring Azure resources [here](/azure/azure-monitor/essentials/monitor-azure-resource#monitoring-data).
52+
53+
### Collection and Routing
54+
55+
Operator Nexus observability allows you to collect data for each infrastructure resource.
56+
The set of infrastructure components includes:
57+
58+
* Network fabric that includes CEs, TORs, NPBs, management switches, and the terminal server.
59+
* Compute that includes Bare Metal Servers.
60+
* Undercloud Control Plane (Kubernetes cluster responsible for deployment and managing lifecycle of overall Platform).
61+
62+
Collection of log data from these layers is enabled by default during the creation of your Operator Nexus
63+
instance. These collected logs are routed to your Azure Monitor Log
64+
Analytics Workspace.
65+
66+
You can also collect data from the tenant layers
67+
created for running Containerized and Virtualized Network Functions. The log data that can be collected includes:
68+
69+
* Collection of syslog from Virtual Machines (used for either VNFs or CNF workloads).
70+
* Collection of logs from AKS-Hybrid clusters and the applications deployed on top.
71+
72+
You'll need to enable the collection of the logs from the tenant AKS-Hybrid clusters and Virtual Machines.
73+
You should follow the steps to deploy the [Azure monitoring agents](/azure/azure-monitor/agents/agents-overview#install-the-agent-and-configure-data-collection). The data would be collected in your Azure Log
74+
Analytics Workspace.
75+
76+
### Operator Nexus Logs storage
77+
78+
Data in Azure Monitor Logs is stored in tables where each table has its own set
79+
of unique properties.
80+
81+
All resource logs in Azure Monitor have the same fields followed by service-specific fields; see the [common schema](/azure/azure-monitor/essentials/resource-logs-schema#top-level-common-schema).
82+
83+
The logs from Operator Nexus platform are stored in the following tables:
84+
85+
| Table | Description |
86+
| ---------------------- | -------------------------------------------------------------------------------- |
87+
| Syslog | Syslog events on Linux computers using the Log Analytics agent |
88+
| ContainerInventory | Details and current state of each container. |
89+
| ContainerLog | Log lines collected from stdout and stderr streams for containers |
90+
| ContainerNodeInventory | Details of nodes that serve as container hosts. |
91+
| InsightMetrics | Metrics collected from Server, K8s, Containers. |
92+
| KubeEvents | Kubernetes events and their properties. |
93+
| KubeMonAgentEvents | Events logged by Azure Monitor Kubernetes agent for errors and warnings. |
94+
| KubeNodeInventory | Details for nodes that are part of Kubernetes cluster |
95+
| KubePodInventory | Kubernetes pods and their properties |
96+
| KubePVInventory | Kubernetes persistent volumes and their properties. |
97+
| KubeServices | Kubernetes services and their properties |
98+
| Heartbeat | Records logged by Log Analytics agents once per minute to report on agent health |
99+
100+
#### Operator nexus metrics
101+
102+
The 'InsightMetrics' table in the Logs section contains the metrics collected from Bare Metal Machines and the undercloud Kubernetes cluster. In addition, a few selected metrics collected from the undercloud can be observed by opening the Metrics tab from the Azure Monitor menu.
103+
104+
<!--- IMG ![Azure Monitor Metrics Selection](Docs/media/azure-monitor-metrics-selection.png) IMG --->
105+
:::image type="content" source="media/azure-monitor-metrics-selection.png" alt-text="Screenshot of Azure Monitor Metrics Selection.":::
106+
107+
Figure: Azure Monitor Metrics Selection
108+
109+
See **[Getting Started with Azure Metrics Explorer](/azure/azure-monitor/essentials/metrics-getting-started)** for details on using this tool.
110+
111+
#### Workbooks
112+
113+
Workbooks combine text, log queries, metrics, and parameters for data analysis and the creation of multiple kinds of rich visualizations.
114+
You can use the sample Azure Resource Manager workbook templates for [Operator Nexus Logging and Monitoring](https://github.com/microsoft/AzureMonitorCommunity/tree/master/Azure%20Services/Azure%20Operator%20Distributed%20Services) to deploy Azure Workbooks within your Azure Log Analytics Workspace.
115+
116+
#### Alerts
117+
118+
You can use the sample Azure Resource Manager alarm templates for [Operator Nexus alerting rules](https://github.com/microsoft/AzureMonitorCommunity/tree/master/Azure%20Services/Azure%20Operator%20Distributed%20Services#alert-rules). You should specify thresholds and conditions for the alerts. You can then deploy these alert templates on your on-premises environment.
119+
120+
## Log analytic workspace
121+
122+
A [Log Analytics workspace (LAW)](/azure/azure-monitor/logs/log-analytics-workspace-overview)
123+
is a unique environment to log data from Azure Monitor and
124+
other Azure services. Each workspace has its own data repository and configuration but may
125+
combine data from multiple services. Each workspace consists of multiple data tables.
126+
127+
A single Log Analytics workspace can be created to collect all relevant data or multiple workspaces based on operator requirements.

‎articles/operator-nexus/concepts-resource-types.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
title: Azure Operator Nexus resource types
3+
description: Operator Nexus platform and tenant resource types
4+
author: jashobhit #Required; your GitHub user alias, with correct capitalization.
5+
ms.author: shobhitjain #Required; microsoft alias of author; optional team alias.
6+
ms.service: azure
7+
ms.topic: conceptual #Required; leave this attribute/value as-is.
8+
ms.date: 01/25/2023 #Required; mm/dd/yyyy format.
9+
ms.custom: template-concept #Required; leave this attribute/value as-is.
10+
---
11+
12+
# Azure Operator Nexus resource types
13+
14+
This article introduces you to the Operator Nexus components represented as Azure resources in Azure Resource Manager.
15+
16+
<!--- IMG ![Resource Types](Docs/media/resource-types.png) IMG --->
17+
:::image type="content" source="media/resource-types.png" alt-text="Screenshot of Resource Types.":::
18+
19+
Figure: Resource model
20+
21+
## Platform components
22+
23+
Your Operator Nexus Cluster (or simply instance) platform components include the infrastructure resources and the platform software resources used to manage these infrastructure resources.
24+
25+
### Network fabric controller
26+
27+
The Network fabric Controller (NFC) is a resource that automates the life cycle management of all network devices (including storage appliance) deployed in an Operator Nexus instance.
28+
The NFC resource is created in the Resource group specified by you in your Azure subscription.
29+
NFC is hosted in a [Microsoft Azure Virtual Network](/azure/virtual-network/virtual-networks-overview) in an Azure region.
30+
The region should be connected to your on-premises network via [Microsoft Azure ExpressRoute](/azure/expressroute/expressroute-introduction).
31+
An NFC can manage the network fabric of many (subject to limits) Operator Nexus on-premises instances.
32+
33+
### Network fabric
34+
35+
The Network fabric resource models a collection of network devices, compute servers, and storage appliances, and their interconnections. The network fabric resource also includes the networking required for your Network Functions and workloads. Each Operator Nexus instance has one Network fabric.
36+
37+
The Network fabric Controller (NFC) performs the lifecycle management of the network fabric.
38+
It configures and bootstraps the network fabric resources.
39+
40+
### Cluster manager
41+
42+
A Cluster Manager (CM) is hosted on Azure and manages the lifecycle of all on-premises clusters.
43+
Like NFC, a CM can manage multiple Operator Nexus instances.
44+
The CM and the NFC are hosted in the same Azure subscription.
45+
46+
### Operator nexus cluster
47+
48+
An Operator Nexus cluster models a collection of racks, bare metal machines, storage, and networking.
49+
Each cluster (sometimes also referred as Operator Nexus instance) is mapped to the on-premises Network fabric. An Operator Nexus cluster provides a holistic view of the deployed capacity.
50+
Cluster capacity examples include the number of vCPUs, the amount of memory, and the amount of storage space. An Operator Nexus cluster is also the basic unit for compute and storage upgrades.
51+
52+
### Network rack
53+
54+
The Network rack consists of Consumer Edge (CE) routers, Top of Rack switches (ToRs), storage appliance, Network Packet Broker (NPB), and the Terminal Server.
55+
The rack also models the connectivity to the operator's Physical Edge switches (PEs) and the ToRs on the other racks.
56+
57+
### Rack
58+
59+
The Rack (or a compute rack) resource represents the compute servers (Bare Metal Machines), management servers, management switch and ToRs. The Rack is created, updated or deleted as part of the Cluster lifecycle management.
60+
61+
### Storage appliance
62+
63+
Storage Appliances represent storage arrays used for persistent data storage in the Operator Nexus instance. All user and consumer data is stored in these appliances local to your premises. This local storage complies with some of the most stringent local data storage requirements.
64+
65+
### Bare Metal Machine
66+
67+
Bare Metal Machines represent the physical servers in a rack. They're lifecycle managed by the Cluster Manager.
68+
Bare Metal Machines are used by workloads to host Virtual Machines and AKS-Hybrid clusters.
69+
70+
## Workload components
71+
72+
Workload components are resources that you use in hosting your workloads.
73+
74+
### Network resources
75+
76+
The Network resources represent the virtual networking in support of your workloads hosted on VMs or AKS-Hybrid clusters.
77+
There are five Network resource types that represent a network attachment to an underlying isolation-domain.
78+
79+
- **Cloud Services Network Resource**: provides VMs/AKS-Hybrid clusters access to cloud services such as DNS, NTP, and user-specified Azure PaaS services. You must create at least one Cloud Services Network in each of your Operator Nexus instances. Each Cloud Service Network can be reused by many VMs and/or AKS-Hybrid clusters.
80+
81+
- **Default CNI Network Resource**: supports configuring of the AKS-Hybrid cluster network resources.
82+
83+
- **Layer 2 Network Resource**: enables "East-West" communication between VMs or AKS-Hybrid clusters.
84+
85+
- **Layer 3 Network Resource**: facilitate "North-South" communication between your VMs/AKS-Hybrid clusters and the external network.
86+
87+
- **Trunked Network Resource**: provides a VM or an AKS-Hybrid cluster access to multiple layer 3 networks and/or multiple layer 2 networks.
88+
89+
### Virtual machine
90+
91+
You can use VMs to host your Virtualized Network Function (VNF) workloads.
92+
93+
### AKS-hybrid cluster
94+
95+
An AKS-Hybrid cluster is Azure Kubernetes Service cluster modified to run on your on-premises Operator Nexus instance. The AKS-Hybrid cluster is designed to host your Containerized Network Function (CNF) workloads.

0 commit comments

Comments
 (0)