|
| 1 | +--- |
| 2 | +title: Data collection endpoints in Azure Monitor (preview) |
| 3 | +description: Overview of data collection endpoints (DCEs) in Azure Monitor including their contents and structure and how you can create and work with them. |
| 4 | +ms.topic: conceptual |
| 5 | +author: shseth |
| 6 | +ms.author: shseth |
| 7 | +ms.date: 1/5/2022 |
| 8 | +ms.custom: references_region |
| 9 | + |
| 10 | +--- |
| 11 | + |
| 12 | +# Data collection endpoints in Azure Monitor (preview) |
| 13 | +Data Collection Endpoints (DCEs) allow you to uniquely configure ingestion settings for your machines, giving you greater control over your networking requirements. This article provides an overview of data collection endpoints including their contents and structure and how you can create and work with them. |
| 14 | + |
| 15 | +## Components of a data collection endpoint |
| 16 | +A data collection endpoint includes the following components. |
| 17 | + |
| 18 | +| Component | Description | |
| 19 | +|:---|:---| |
| 20 | +| Configuration access endpoint | The endpoint used to access the configuration service to fetch associated data collection rules (DCR). Example: `<unique-dce-identifier>.<regionname>.handler.control` | |
| 21 | +| Logs ingestion endpoint | The endpoint used to ingest logs to Log Analytics workspace(s). Example: `<unique-dce-identifier>.<regionname>.ingest` | |
| 22 | +| Network Access Control Lists (ACLs) | Network access control rules for the endpoints |
| 23 | + |
| 24 | + |
| 25 | +## Regionality |
| 26 | +Data collection endpoints are ARM resources created within specific regions. An endpoint in a given region can only be **associated with machines in the same region**, although you can have more than one endpoint within the same region as per your needs. |
| 27 | + |
| 28 | +## Limitations |
| 29 | +Data collection endpoints only support Log Analytics as a destination for collected data. [Custom Metrics (preview)](../essentials/metrics-custom-overview.md) collected and uploaded via the Azure Monitor Agent are not controlled by Data Collection endpoints nor can they be configured over private links. |
| 30 | + |
| 31 | +## Create endpoint and association in Azure portal |
| 32 | +You can use the Azure portal to create a data collection endpoint and associate virtual machines in your subscription to that rule. |
| 33 | + |
| 34 | +> [!NOTE] |
| 35 | +> The data collection endpoint should be created in the **same region** where your virtual machines exist. |
| 36 | +
|
| 37 | +In the **Azure Monitor** menu in the Azure portal, select **Data Collection Endpoint** from the **Settings** section. Click **Create** to create a new Data Collection Rule and assignment. |
| 38 | + |
| 39 | +[](media/data-collection-endpoint-overview/data-collection-endpoint-overview.png#lightbox) |
| 40 | + |
| 41 | +Click **Create** to create a new endpoint. Provide a **Rule name** and specify a **Subscription**, **Resource Group** and **Region**. This specifies where the DCE will be created. |
| 42 | + |
| 43 | +[](media/data-collection-endpoint-overview/data-collection-endpoint-basics.png#lightbox) |
| 44 | + |
| 45 | +Click **Review + create** to review the details of the data collection endpoint. Click **Create** to create it. |
| 46 | + |
| 47 | +Next, you can use 'Data collection rules' in the portal to associate endpoints with a resource (e.g. a virtual machine) or a set of resources. |
| 48 | +Create a new rule or open an existing rule. In the **Resources** tab, click on the **Data collection endpoint** drop-down to associate an existing endpoint for your resource in the same region (or select multiple resources in the same region to bulk-assign an endpoint for them). Doing this creates an association per resource which links the endpoint to the resource. The Azure Monitor agent running on these resources will now start using the endpoint instead for uploading data to Azure Monitor. |
| 49 | + |
| 50 | +[](media/data-collection-rule-azure-monitor-agent/data-collection-rule-virtual-machines-with-endpoint.png#lightbox) |
| 51 | + |
| 52 | +## Create endpoint and association using REST API |
| 53 | + |
| 54 | +> [!NOTE] |
| 55 | +> The data collection endpoint should be created in the **same region** where your virtual machines exist. |
| 56 | +
|
| 57 | +1. Create data collection endpoint(s) using these [DCE REST APIs](/rest/api/monitor/datacollectionendpoints). |
| 58 | +2. Create association(s) to link the endpoint(s) to your target machines or resources, using these [DCRA REST APIs](/rest/api/monitor/datacollectionruleassociations/create#examples). |
| 59 | + |
| 60 | + |
| 61 | +## Sample data collection endpoint |
| 62 | +The sample data collection endpoint below is for virtual machines with Azure Monitor agent, with public network access disabled so that agent only uses private links to communicate and send data to Azure Monitor/Log Analytics. |
| 63 | + |
| 64 | +```json |
| 65 | +{ |
| 66 | + "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myCollectionEndpoint", |
| 67 | + "name": "myCollectionEndpoint", |
| 68 | + "type": "Microsoft.Insights/dataCollectionEndpoints", |
| 69 | + "location": "eastus", |
| 70 | + "tags": { |
| 71 | + "tag1": "A", |
| 72 | + "tag2": "B" |
| 73 | + }, |
| 74 | + "properties": { |
| 75 | + "configurationAccess": { |
| 76 | + "endpoint": "https://mycollectionendpoint-abcd.eastus-1.control.monitor.azure.com" |
| 77 | + }, |
| 78 | + "logsIngestion": { |
| 79 | + "endpoint": "https://mycollectionendpoint-abcd.eastus-1.ingest.monitor.azure.com" |
| 80 | + }, |
| 81 | + "networkAcls": { |
| 82 | + "publicNetworkAccess": "Disabled" |
| 83 | + } |
| 84 | + }, |
| 85 | + "systemData": { |
| 86 | + "createdBy": "user1", |
| 87 | + "createdByType": "User", |
| 88 | + "createdAt": "yyyy-mm-ddThh:mm:ss.sssssssZ", |
| 89 | + "lastModifiedBy": "user2", |
| 90 | + "lastModifiedByType": "User", |
| 91 | + "lastModifiedAt": "yyyy-mm-ddThh:mm:ss.sssssssZ" |
| 92 | + }, |
| 93 | + "etag": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" |
| 94 | +} |
| 95 | +``` |
| 96 | + |
| 97 | +## Enable network isolation for the Azure Monitor Agent |
| 98 | +You can use data collection endpoints to enable the Azure Monitor agent to communicate to the internet via private links. To do so, you must: |
| 99 | +1. Create data collection endpoint(s), at least one per region, as shown above |
| 100 | +2. Add the data collection endpoints to a new or existing [Azure Monitor Private Link Scopes (AMPLS)](../logs/private-link-configure.md#connect-azure-monitor-resources) resource. This adds the DCE endpoints to your private DNS zone (see [how to validate](../logs/private-link-configure.md#review-and-validate-your-private-link-setup)) and allows communication via private links. You can do this from either the AMPLS resource or from within an existing DCE resource's 'Network Isolation' tab. |
| 101 | + > [!NOTE] |
| 102 | + > Other Azure Monitor resources like the Log Analytics workspace(s) configured in your data collection rules that you wish to send data to, must be part of this same AMPLS resource. |
| 103 | +3. For your data collection endpoint(s), ensure **Accept access from public networks not connected through a Private Link Scope** option is set to **No** under the 'Network Isolation' tab of your endpoint resource in Azure portal, as shown below. This ensures that public internet access is disabled, and network communication only happen via private links. |
| 104 | +4. Associate the data collection endpoints to the target resources, using the data collection rules experience in Azure portal. This results in the agent using the configured the data collection endpoint(s) for network communications. See [Configure data collection for the Azure Monitor agent](../agents/data-collection-rule-azure-monitor-agent.md). |
| 105 | + |
| 106 | +  |
| 107 | + |
| 108 | +## Next steps |
| 109 | +- [Associate endpoint to machines](data-collection-rule-azure-monitor-agent.md#create-rule-and-association-in-azure-portal) |
| 110 | +- [Add endpoint to AMPLS resource](../logs/private-link-configure.md#connect-azure-monitor-resources) |
0 commit comments