Skip to content

Commit 4295cd4

Browse files
bwrenbwren
committed
Add DCE back to logs ingestion
1 parent 30b2e7e commit 4295cd4

File tree

2 files changed

+168
-1
lines changed

2 files changed

+168
-1
lines changed

‎articles/azure-monitor/logs/tutorial-logs-ingestion-api.md

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,80 @@ Start by registering a Microsoft Entra application to authenticate against the A
6464

6565
:::image type="content" source="media/tutorial-logs-ingestion-portal/new-app-secret-value.png" lightbox="media/tutorial-logs-ingestion-portal/new-app-secret-value.png" alt-text="Screenshot that shows the secret value for the new app.":::
6666

67+
## Create data collection endpoint
68+
69+
## [DCR endpoint](#tab/dcr)
70+
A DCE isn't required if you use the DCR ingestion endpoint.
71+
72+
## [DCE](#tab/dce)
73+
74+
A [DCE](../essentials/data-collection-endpoint-overview.md) is required to accept the data being sent to Azure Monitor. After you configure the DCE and link it to a DCR, you can send data over HTTP from your application. The DCE must be located in the same region as the DCR and the Log Analytics workspace where the data will be sent.
75+
76+
1. In the Azure portal's search box, enter **template** and then select **Deploy a custom template**.
77+
78+
:::image type="content" source="media/tutorial-workspace-transformations-api/deploy-custom-template.png" lightbox="media/tutorial-workspace-transformations-api/deploy-custom-template.png" alt-text="Screenshot that shows how to deploy a custom template.":::
79+
80+
1. Select **Build your own template in the editor**.
81+
82+
:::image type="content" source="media/tutorial-workspace-transformations-api/build-custom-template.png" lightbox="media/tutorial-workspace-transformations-api/build-custom-template.png" alt-text="Screenshot that shows how to build a template in the editor.":::
83+
84+
1. Paste the following ARM template into the editor and then select **Save**. You don't need to modify this template because you'll provide values for its parameters.
85+
86+
:::image type="content" source="media/tutorial-workspace-transformations-api/edit-template.png" lightbox="media/tutorial-workspace-transformations-api/edit-template.png" alt-text="Screenshot that shows how to edit an ARM template.":::
87+
88+
89+
```json
90+
{
91+
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
92+
"contentVersion": "1.0.0.0",
93+
"parameters": {
94+
"dataCollectionEndpointName": {
95+
"type": "string",
96+
"metadata": {
97+
"description": "Specifies the name of the Data Collection Endpoint to create."
98+
}
99+
},
100+
"location": {
101+
"type": "string",
102+
"defaultValue": "westus2",
103+
"metadata": {
104+
"description": "Specifies the location for the Data Collection Endpoint."
105+
}
106+
}
107+
},
108+
"resources": [
109+
{
110+
"type": "Microsoft.Insights/dataCollectionEndpoints",
111+
"name": "[parameters('dataCollectionEndpointName')]",
112+
"location": "[parameters('location')]",
113+
"apiVersion": "2021-04-01",
114+
"properties": {
115+
"networkAcls": {
116+
"publicNetworkAccess": "Enabled"
117+
}
118+
}
119+
}
120+
],
121+
"outputs": {
122+
"dataCollectionEndpointId": {
123+
"type": "string",
124+
"value": "[resourceId('Microsoft.Insights/dataCollectionEndpoints', parameters('dataCollectionEndpointName'))]"
125+
}
126+
}
127+
}
128+
```
129+
130+
1. On the **Custom deployment** screen, specify a **Subscription** and **Resource group** to store the DCR and then provide values like a **Name** for the DCE. The **Location** should be the same location as the workspace. The **Region** will already be populated and will be used for the location of the DCE.
131+
132+
:::image type="content" source="media/tutorial-logs-ingestion-api/data-collection-endpoint-custom-deploy.png" lightbox="media/tutorial-logs-ingestion-api/data-collection-endpoint-custom-deploy.png" alt-text="Screenshot to edit custom deployment values.":::
133+
134+
1. Select **Review + create** and then select **Create** after you review the details.
135+
136+
1. Select **JSON View** to view other details for the DCE. Copy the **Resource ID** and the **logsIngestion endpoint** which you'll need in a later step.
137+
138+
:::image type="content" source="media/tutorial-logs-ingestion-api/data-collection-endpoint-json.png" lightbox="media/tutorial-logs-ingestion-api/data-collection-endpoint-json.png" alt-text="Screenshot that shows the DCE resource ID.":::
139+
140+
---
67141

68142
## Create new table in Log Analytics workspace
69143
The custom table must be created before you can send data to it. The table for this tutorial will include five columns shown in the schema below. The `name`, `type`, and `description` properties are mandatory for each column. The properties `isHidden` and `isDefaultDisplay` both default to `false` if not explicitly specified. Possible data types are `string`, `int`, `long`, `real`, `boolean`, `dateTime`, `guid`, and `dynamic`.
@@ -146,6 +220,8 @@ The [DCR](../essentials/data-collection-rule-overview.md) defines how the data w
146220
- `destinations`: Destination workspace.
147221
- `dataFlows`: Matches the stream with the destination workspace and specifies the transformation query and the destination table. The output of the destination query is what will be sent to the destination table.
148222

223+
## [DCR endpoint](#tab/dcr)
224+
149225
```json
150226
{
151227
"$schema": "https://schema.management.azure.com/schemas/2023-03-11/deploymentTemplate.json#",
@@ -183,6 +259,7 @@ The [DCR](../essentials/data-collection-rule-overview.md) defines how the data w
183259
"location": "[parameters('location')]",
184260
"apiVersion": "2021-09-01-preview",
185261
"properties": {
262+
"dataCollectionEndpointId": "[parameters('endpointResourceId')]",
186263
"streamDeclarations": {
187264
"Custom-MyTableRawData": {
188265
"columns": [
@@ -241,6 +318,96 @@ The [DCR](../essentials/data-collection-rule-overview.md) defines how the data w
241318
}
242319
```
243320

321+
```json
322+
{
323+
"$schema": "https://schema.management.azure.com/schemas/2023-03-11/deploymentTemplate.json#",
324+
"contentVersion": "1.0.0.0",
325+
"parameters": {
326+
"dataCollectionRuleName": {
327+
"type": "string",
328+
"metadata": {
329+
"description": "Specifies the name of the Data Collection Rule to create."
330+
}
331+
},
332+
"location": {
333+
"type": "string",
334+
"metadata": {
335+
"description": "Specifies the location in which to create the Data Collection Rule."
336+
}
337+
},
338+
"workspaceResourceId": {
339+
"type": "string",
340+
"metadata": {
341+
"description": "Specifies the Azure resource ID of the Log Analytics workspace to use."
342+
}
343+
}
344+
},
345+
"resources": [
346+
{
347+
"type": "Microsoft.Insights/dataCollectionRules",
348+
"name": "[parameters('dataCollectionRuleName')]",
349+
"location": "[parameters('location')]",
350+
"apiVersion": "2021-09-01-preview",
351+
"properties": {
352+
"streamDeclarations": {
353+
"Custom-MyTableRawData": {
354+
"columns": [
355+
{
356+
"name": "Time",
357+
"type": "datetime"
358+
},
359+
{
360+
"name": "Computer",
361+
"type": "string"
362+
},
363+
{
364+
"name": "AdditionalContext",
365+
"type": "string"
366+
},
367+
{
368+
"name": "CounterName",
369+
"type": "string"
370+
},
371+
{
372+
"name": "CounterValue",
373+
"type": "real"
374+
}
375+
]
376+
}
377+
},
378+
"destinations": {
379+
"logAnalytics": [
380+
{
381+
"workspaceResourceId": "[parameters('workspaceResourceId')]",
382+
"name": "myworkspace"
383+
}
384+
]
385+
},
386+
"dataFlows": [
387+
{
388+
"streams": [
389+
"Custom-MyTableRawData"
390+
],
391+
"destinations": [
392+
"myworkspace"
393+
],
394+
"transformKql": "source | extend jsonContext = parse_json(AdditionalContext) | project TimeGenerated = Time, Computer, AdditionalContext = jsonContext, CounterName=tostring(jsonContext.CounterName), CounterValue=toreal(jsonContext.CounterValue)",
395+
"outputStream": "Custom-MyTable_CL"
396+
}
397+
]
398+
}
399+
}
400+
],
401+
"outputs": {
402+
"dataCollectionRuleId": {
403+
"type": "string",
404+
"value": "[resourceId('Microsoft.Insights/dataCollectionRules', parameters('dataCollectionRuleName'))]"
405+
}
406+
}
407+
}
408+
```
409+
---
410+
244411
4. On the **Custom deployment** screen, specify a **Subscription** and **Resource group** to store the DCR. Then provide values defined in the template. The values include a **Name** for the DCR and the **Workspace Resource ID** that you collected in a previous step. The **Location** should be the same location as the workspace. The **Region** will already be populated and will be used for the location of the DCR.
245412

246413
:::image type="content" source="media/tutorial-workspace-transformations-api/custom-deployment-values.png" lightbox="media/tutorial-workspace-transformations-api/custom-deployment-values.png" alt-text="Screenshot that shows how to edit custom deployment values.":::

‎articles/azure-monitor/logs/tutorial-logs-ingestion-code.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -403,7 +403,7 @@ The following PowerShell code sends data to the endpoint by using HTTP REST fund
403403
$appSecret = "0000000000000000000000000000000000000000" #Secret created for the application
404404
405405
# information needed to send data to the DCR endpoint
406-
$endpoint_uri = "https://logs-ingestion-rzmk.eastus2-1.ingest.monitor.azure.com" #Logs ingestion URI for the DCR
406+
$endpoint_uri = "https://my-url.monitor.azure.com" #Logs ingestion URI for the DCR
407407
$dcrImmutableId = "dcr-00000000000000000000000000000000" #the immutableId property of the DCR object
408408
$streamName = "Custom-MyTableRawData" #name of the stream in the DCR that represents the destination table
409409

0 commit comments

Comments
 (0)