Skip to content

Commit 54b395b

Browse files
Jak-MSJak-MS
authored
Merge pull request #89598 from cijothomas/cithomas/fix1
Corrections to Telemetry Initializer/Processor docs
2 parents f6189ff + 144a529 commit 54b395b

File tree

1 file changed

+108
-79
lines changed

1 file changed

+108
-79
lines changed

‎articles/azure-monitor/app/api-filtering-sampling.md

Lines changed: 108 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -17,24 +17,24 @@ ms.author: mbullwin
1717
---
1818
# Filtering and preprocessing telemetry in the Application Insights SDK
1919

20+
You can write and configure plug-ins for the Application Insights SDK to customize how telemetry can be enriched and processed before it's sent to the Application Insights service.
2021

21-
You can write and configure plug-ins for the Application Insights SDK to customize how telemetry is captured and processed before it is sent to the Application Insights service.
22-
23-
* [Sampling](../../azure-monitor/app/sampling.md) reduces the volume of telemetry without affecting your statistics. It keeps together related data points so that you can navigate between them when diagnosing a problem. In the portal, the total counts are multiplied to compensate for the sampling.
24-
* Filtering with Telemetry Processors [for ASP.NET](#filtering) or [Java](../../azure-monitor/app/java-filter-telemetry.md) lets you select or modify telemetry in the SDK before it is sent to the server. For example, you could reduce the volume of telemetry by excluding requests from robots. But filtering is a more basic approach to reducing traffic than sampling. It allows you more control over what is transmitted, but you have to be aware that it affects your statistics - for example, if you filter out all successful requests.
25-
* [Telemetry Initializers add properties](#add-properties) to any telemetry sent from your app, including telemetry from the standard modules. For example, you could add calculated values; or version numbers by which to filter the data in the portal.
22+
* [Sampling](sampling.md) reduces the volume of telemetry without affecting your statistics. It keeps together related data points so that you can navigate between them when diagnosing a problem. In the portal, the total counts are multiplied to compensate for the sampling.
23+
* Filtering with Telemetry Processors lets you filter out telemetry in the SDK before it is sent to the server. For example, you could reduce the volume of telemetry by excluding requests from robots. Filtering is a more basic approach to reducing traffic than sampling. It allows you more control over what is transmitted, but you have to be aware that it affects your statistics - for example, if you filter out all successful requests.
24+
* [Telemetry Initializers add or modify properties](#add-properties) to any telemetry sent from your app, including telemetry from the standard modules. For example, you could add calculated values; or version numbers by which to filter the data in the portal.
2625
* [The SDK API](../../azure-monitor/app/api-custom-events-metrics.md) is used to send custom events and metrics.
2726

2827
Before you start:
2928

30-
* Install the Application Insights [SDK for ASP.NET](../../azure-monitor/app/asp-net.md) or [SDK for Java](../../azure-monitor/app/java-get-started.md) in your app.
29+
* Install the appropriate SDK for you application. [ASP.NET](asp-net.md) or [ASP.NET Core](asp-net-core.md) or [Non HTTP/Worker for .NET/.NET Core](worker-service.md) or [Java](../../azure-monitor/app/java-get-started.md) in your app.
3130

3231
<a name="filtering"></a>
3332

3433
## Filtering: ITelemetryProcessor
35-
This technique gives you more direct control over what is included or excluded from the telemetry stream. You can use it in conjunction with Sampling, or separately.
3634

37-
To filter telemetry, you write a telemetry processor and register it with the SDK. All telemetry goes through your processor, and you can choose to drop it from the stream, or add properties. This includes telemetry from the standard modules such as the HTTP request collector and the dependency collector, as well as telemetry you have written yourself. You can, for example, filter out telemetry about requests from robots, or successful dependency calls.
35+
This technique gives you direct control over what is included or excluded from the telemetry stream. Filtering can be used to drop telemetry items from being sent to Application Insights. You can use it in conjunction with Sampling, or separately.
36+
37+
To filter telemetry, you write a telemetry processor and register it with the `TelemetryConfiguration`. All telemetry goes through your processor, and you can choose to drop it from the stream or give it to the next processor in the chain. This includes telemetry from the standard modules such as the HTTP request collector and the dependency collector, and telemetry you have tracked yourself. You can, for example, filter out telemetry about requests from robots, or successful dependency calls.
3838

3939
> [!WARNING]
4040
> Filtering the telemetry sent from the SDK using processors can skew the statistics that you see in the portal, and make it difficult to follow related items.
@@ -44,59 +44,48 @@ To filter telemetry, you write a telemetry processor and register it with the SD
4444
>
4545
4646
### Create a telemetry processor (C#)
47-
1. Verify that the Application Insights SDK in your project is version 2.0.0 or later. Right-click your project in Visual Studio Solution Explorer and choose Manage NuGet Packages. In NuGet package manager, check Microsoft.ApplicationInsights.Web.
48-
2. To create a filter, implement ITelemetryProcessor. This is another extensibility point like telemetry module, telemetry initializer, and telemetry channel.
49-
50-
Notice that Telemetry Processors construct a chain of processing. When you instantiate a telemetry processor, you pass a link to the next processor in the chain. When a telemetry data point is passed to the Process method, it does its work and then calls the next Telemetry Processor in the chain.
51-
52-
```csharp
53-
using Microsoft.ApplicationInsights.Channel;
54-
using Microsoft.ApplicationInsights.Extensibility;
55-
56-
public class SuccessfulDependencyFilter : ITelemetryProcessor
57-
{
58-
59-
private ITelemetryProcessor Next { get; set; }
60-
61-
// You can pass values from .config
62-
public string MyParamFromConfigFile { get; set; }
63-
64-
// Link processors to each other in a chain.
65-
public SuccessfulDependencyFilter(ITelemetryProcessor next)
66-
{
67-
this.Next = next;
68-
}
69-
public void Process(ITelemetry item)
70-
{
71-
// To filter out an item, just return
72-
if (!OKtoSend(item)) { return; }
73-
// Modify the item if required
74-
ModifyItem(item);
7547

76-
this.Next.Process(item);
77-
}
48+
1. To create a filter, implement `ITelemetryProcessor`.
7849

79-
// Example: replace with your own criteria.
80-
private bool OKtoSend (ITelemetry item)
81-
{
82-
var dependency = item as DependencyTelemetry;
83-
if (dependency == null) return true;
50+
Notice that Telemetry Processors construct a chain of processing. When you instantiate a telemetry processor, you are given a reference to the next processor in the chain. When a telemetry data point is passed to the Process method, it does its work and then calls (or not calls) the next Telemetry Processor in the chain.
8451

85-
return dependency.Success != true;
86-
}
52+
```csharp
53+
using Microsoft.ApplicationInsights.Channel;
54+
using Microsoft.ApplicationInsights.Extensibility;
8755

88-
// Example: replace with your own modifiers.
89-
private void ModifyItem (ITelemetry item)
90-
{
91-
item.Context.Properties.Add("app-version", "1." + MyParamFromConfigFile);
92-
}
93-
}
94-
```
56+
public class SuccessfulDependencyFilter : ITelemetryProcessor
57+
{
58+
private ITelemetryProcessor Next { get; set; }
59+
60+
// next will point to the next TelemetryProcessor in the chain.
61+
public SuccessfulDependencyFilter(ITelemetryProcessor next)
62+
{
63+
this.Next = next;
64+
}
65+
66+
public void Process(ITelemetry item)
67+
{
68+
// To filter out an item, return without calling the next processor.
69+
if (!OKtoSend(item)) { return; }
70+
71+
this.Next.Process(item);
72+
}
73+
74+
// Example: replace with your own criteria.
75+
private bool OKtoSend (ITelemetry item)
76+
{
77+
var dependency = item as DependencyTelemetry;
78+
if (dependency == null) return true;
79+
80+
return dependency.Success != true;
81+
}
82+
}
83+
```
9584

96-
3. Add your processor
85+
2. Add your processor.
9786

9887
**ASP.NET apps**
99-
Insert this in ApplicationInsights.config:
88+
Insert this snippet in ApplicationInsights.config:
10089

10190
```xml
10291
<TelemetryProcessors>
@@ -107,19 +96,16 @@ Insert this in ApplicationInsights.config:
10796
</TelemetryProcessors>
10897
```
10998

110-
(This is the same section where you initialize a sampling filter.)
111-
11299
You can pass string values from the .config file by providing public named properties in your class.
113100

114101
> [!WARNING]
115102
> Take care to match the type name and any property names in the .config file to the class and property names in the code. If the .config file references a non-existent type or property, the SDK may silently fail to send any telemetry.
116103
>
117-
>
118104

119-
**Alternatively,** you can initialize the filter in code. In a suitable initialization class - for example AppStart in Global.asax.cs - insert your processor into the chain:
105+
**Alternatively,** you can initialize the filter in code. In a suitable initialization class - for example AppStart in `Global.asax.cs` - insert your processor into the chain:
120106

121107
```csharp
122-
var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder;
108+
var builder = TelemetryConfiguration.Active.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
123109
builder.Use((next) => new SuccessfulDependencyFilter(next));
124110

125111
// If you have more processors:
@@ -130,13 +116,12 @@ builder.Build();
130116

131117
TelemetryClients created after this point will use your processors.
132118

133-
**ASP.NET Core apps**
119+
**ASP.NET Core/ Worker Service apps**
134120

135121
> [!NOTE]
136-
> Adding initializer using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications.
137-
122+
> Adding processor using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications or if you are using Microsoft.ApplicationInsights.WorkerService SDK.
138123
139-
For [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) applications, adding a new `TelemetryInitializer` is done by adding it to the Dependency Injection container, as shown below. This is done in `ConfigureServices` method of your `Startup.cs` class.
124+
For apps written using [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) or [WorkerService](worker-service.md#adding-telemetry-processors), adding a new `TelemetryProcessor` is done by using `AddApplicationInsightsTelemetryProcessor` extension method on `IServiceCollection`, as shown below. This method is called in `ConfigureServices` method of your `Startup.cs` class.
140125

141126
```csharp
142127
public void ConfigureServices(IServiceCollection services)
@@ -151,8 +136,10 @@ For [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) applications, ad
151136
```
152137

153138
### Example filters
139+
154140
#### Synthetic requests
155-
Filter out bots and web tests. Although Metrics Explorer gives you the option to filter out synthetic sources, this option reduces traffic by filtering them at the SDK.
141+
142+
Filter out bots and web tests. Although Metrics Explorer gives you the option to filter out synthetic sources, this option reduces traffic and ingestion size by filtering them at the SDK itself.
156143

157144
```csharp
158145
public void Process(ITelemetry item)
@@ -165,6 +152,7 @@ public void Process(ITelemetry item)
165152
```
166153

167154
#### Failed authentication
155+
168156
Filter out requests with a "401" response.
169157

170158
```csharp
@@ -175,19 +163,21 @@ public void Process(ITelemetry item)
175163
if (request != null &&
176164
request.ResponseCode.Equals("401", StringComparison.OrdinalIgnoreCase))
177165
{
178-
// To filter out an item, just terminate the chain:
166+
// To filter out an item, return without calling the next processor.
179167
return;
180168
}
181-
// Send everything else:
169+
170+
// Send everything else
182171
this.Next.Process(item);
183172
}
184173
```
185174

186175
#### Filter out fast remote dependency calls
176+
187177
If you only want to diagnose calls that are slow, filter out the fast ones.
188178

189179
> [!NOTE]
190-
> This will skew the statistics you see on the portal. The dependency chart will look as if the dependency calls are all failures.
180+
> This will skew the statistics you see on the portal.
191181
>
192182
>
193183
@@ -205,17 +195,18 @@ public void Process(ITelemetry item)
205195
```
206196

207197
#### Diagnose dependency issues
208-
[This blog](https://azure.microsoft.com/blog/implement-an-application-insights-telemetry-processor/) describes a project to diagnose dependency issues by automatically sending regular pings to dependencies.
209198

199+
[This blog](https://azure.microsoft.com/blog/implement-an-application-insights-telemetry-processor/) describes a project to diagnose dependency issues by automatically sending regular pings to dependencies.
210200

211201
<a name="add-properties"></a>
212202

213203
## Add properties: ITelemetryInitializer
214-
Use telemetry initializers to define global properties that are sent with all telemetry; and to override selected behavior of the standard telemetry modules.
215204

216-
For example, the Application Insights for Web package collects telemetry about HTTP requests. By default, it flags as failed any request with a response code >= 400. But if you want to treat 400 as a success, you can provide a telemetry initializer that sets the Success property.
205+
Use telemetry initializers to enrich telemetry with additional information and/or to override telemetry properties set by the standard telemetry modules.
217206

218-
If you provide a telemetry initializer, it is called whenever any of the Track*() methods are called. This includes methods called by the standard telemetry modules. By convention, these modules do not set any property that has already been set by an initializer.
207+
For example, the Application Insights for Web package collect telemetry about HTTP requests. By default, it flags as failed any request with a response code >= 400. But if you want to treat 400 as a success, you can provide a telemetry initializer that sets the Success property.
208+
209+
If you provide a telemetry initializer, it is called whenever any of the Track*() methods are called. This includes `Track()` methods called by the standard telemetry modules. By convention, these modules do not set any property that has already been set by an initializer. Telemetry initializers are called before calling telemetry processors. So any enrichments done by initializers are visible to processors.
219210

220211
**Define your initializer**
221212

@@ -248,10 +239,11 @@ namespace MvcWebRole.Telemetry
248239
{
249240
// If we set the Success property, the SDK won't change it:
250241
requestTelemetry.Success = true;
242+
251243
// Allow us to filter these requests in the portal:
252-
requestTelemetry.Context.Properties["Overridden400s"] = "true";
244+
requestTelemetry.Properties["Overridden400s"] = "true";
253245
}
254-
// else leave the SDK to set the Success property
246+
// else leave the SDK to set the Success property
255247
}
256248
}
257249
}
@@ -283,12 +275,12 @@ protected void Application_Start()
283275

284276
[See more of this sample.](https://github.com/Microsoft/ApplicationInsights-Home/tree/master/Samples/AzureEmailService/MvcWebRole)
285277

286-
**ASP.NET Core apps: Load your initializer**
278+
**ASP.NET Core/ Worker Service apps: Load your initializer**
287279

288280
> [!NOTE]
289-
> Adding initializer using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications.
281+
> Adding initializer using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications or if you are using Microsoft.ApplicationInsights.WorkerService SDK.
290282
291-
For [ASP.NET Core](asp-net-core.md#adding-telemetryinitializers) applications, adding a new `TelemetryInitializer` is done by adding it to the Dependency Injection container, as shown below. This is done in `ConfigureServices` method of your `Startup.cs` class.
283+
For apps written using [ASP.NET Core](asp-net-core.md#adding-telemetryinitializers) or [WorkerService](worker-service.md#adding-telemetryinitializers), adding a new `TelemetryInitializer` is done by adding it to the Dependency Injection container, as shown below. This is done in `Startup.ConfigureServices` method.
292284

293285
```csharp
294286
using Microsoft.ApplicationInsights.Extensibility;
@@ -364,25 +356,62 @@ Insert a telemetry initializer immediately after the initialization code that yo
364356

365357
For a summary of the non-custom properties available on the telemetryItem, see [Application Insights Export Data Model](../../azure-monitor/app/export-data-model.md).
366358

367-
You can add as many initializers as you like.
359+
You can add as many initializers as you like, and they are called in the order they are added.
360+
361+
### Example TelemetryInitializers
362+
363+
#### Add custom property
364+
365+
The following sample initializer adds a custom property to every tracked telemetry.
366+
367+
```csharp
368+
public void Initialize(ITelemetry item)
369+
{
370+
var itemProperties = item as ISupportProperties;
371+
if(itemProperties != null && !itemProperties.ContainsKey("customProp"))
372+
{
373+
itemProperties.Properties["customProp"] = "customValue";
374+
}
375+
}
376+
```
377+
378+
#### Add cloud role name
379+
380+
The following sample initializer sets cloud role name to every tracked telemetry.
381+
382+
```csharp
383+
public void Initialize(ITelemetry telemetry)
384+
{
385+
if(string.IsNullOrEmpty(telemetry.Context.Cloud.RoleName))
386+
{
387+
telemetry.Context.Cloud.RoleName = "MyCloudRoleName";
388+
}
389+
}
390+
```
368391

369392
## ITelemetryProcessor and ITelemetryInitializer
393+
370394
What's the difference between telemetry processors and telemetry initializers?
371395

372-
* There are some overlaps in what you can do with them: both can be used to add properties to telemetry.
396+
* There are some overlaps in what you can do with them: both can be used to add or modify properties of telemetry, though it is recommended to use initializers for that purpose.
373397
* TelemetryInitializers always run before TelemetryProcessors.
398+
* TelemetryInitializers may be called more than once. By convention, they do not set any property that has already been set.
374399
* TelemetryProcessors allow you to completely replace or discard a telemetry item.
375-
* TelemetryProcessors don't process performance counter telemetry.
400+
* All registered TelemetryInitializers are guaranteed to be called for every telemetry item. For Telemetry processors, SDK guarantees calling the very first telemetry processor. Whether the rest of the processors are called or not, is decided by the preceding telemetry processors.
401+
* Use TelemetryInitializers to enrich telemetry with additional properties, or override existing one. Use TelemetryProcessor to filter out telemetry.
376402

377403
## Troubleshooting ApplicationInsights.config
404+
378405
* Confirm that the fully qualified type name and assembly name are correct.
379406
* Confirm that the applicationinsights.config file is in your output directory and contains any recent changes.
380407

381408
## Reference docs
409+
382410
* [API Overview](../../azure-monitor/app/api-custom-events-metrics.md)
383411
* [ASP.NET reference](https://msdn.microsoft.com/library/dn817570.aspx)
384412

385413
## SDK Code
414+
386415
* [ASP.NET Core SDK](https://github.com/Microsoft/ApplicationInsights-aspnetcore)
387416
* [ASP.NET SDK](https://github.com/Microsoft/ApplicationInsights-dotnet)
388417
* [JavaScript SDK](https://github.com/Microsoft/ApplicationInsights-JS)

0 commit comments

Comments
 (0)