Corrections to Filtering and Enrichment doc

cijothomas · cijothomas · commit c665b4c972bc · 2019-09-24T12:55:41.000-07:00
diff --git a/articles/azure-monitor/app/api-filtering-sampling.md b/articles/azure-monitor/app/api-filtering-sampling.md
@@ -21,7 +21,7 @@ ms.author: mbullwin
 You can write and configure plug-ins for the Application Insights SDK to customize how telemetry is captured and processed before it is sent to the Application Insights service.
 
 * [Sampling](../../azure-monitor/app/sampling.md) reduces the volume of telemetry without affecting your statistics. It keeps together related data points so that you can navigate between them when diagnosing a problem. In the portal, the total counts are multiplied to compensate for the sampling.
-* Filtering with Telemetry Processors [for ASP.NET](#filtering) or [Java](../../azure-monitor/app/java-filter-telemetry.md) lets you select or modify telemetry in the SDK before it is sent to the server. For example, you could reduce the volume of telemetry by excluding requests from robots. But filtering is a more basic approach to reducing traffic than sampling. It allows you more control over what is transmitted, but you have to be aware that it affects your statistics - for example, if you filter out all successful requests.
+* Filtering with Telemetry Processors [for ASP.NET or ASP.NET Core](#filtering) or [Java](../../azure-monitor/app/java-filter-telemetry.md) lets you select or modify telemetry in the SDK before it is sent to the server. For example, you could reduce the volume of telemetry by excluding requests from robots. But filtering is a more basic approach to reducing traffic than sampling. It allows you more control over what is transmitted, but you have to be aware that it affects your statistics - for example, if you filter out all successful requests.
 * [Telemetry Initializers add properties](#add-properties) to any telemetry sent from your app, including telemetry from the standard modules. For example, you could add calculated values; or version numbers by which to filter the data in the portal.
 * [The SDK API](../../azure-monitor/app/api-custom-events-metrics.md) is used to send custom events and metrics.
 
@@ -32,9 +32,10 @@ Before you start:
 <a name="filtering"></a>
 
 ## Filtering: ITelemetryProcessor
-This technique gives you more direct control over what is included or excluded from the telemetry stream. You can use it in conjunction with Sampling, or separately.
 
-To filter telemetry, you write a telemetry processor and register it with the SDK. All telemetry goes through your processor, and you can choose to drop it from the stream, or add properties. This includes telemetry from the standard modules such as the HTTP request collector and the dependency collector, as well as telemetry you have written yourself. You can, for example, filter out telemetry about requests from robots, or successful dependency calls.
+This technique gives you more direct control over what is included or excluded from the telemetry stream. Filtering can be used to drop telemetry items from being sent to Application Insights. You can use it in conjunction with Sampling, or separately.
+
+To filter telemetry, you write a telemetry processor and register it with the `TelemetryConfiguration`. All telemetry goes through your processor, and you can choose to drop it from the stream, or add properties. This includes telemetry from the standard modules such as the HTTP request collector and the dependency collector, as well as telemetry you have written yourself. You can, for example, filter out telemetry about requests from robots, or successful dependency calls.
 
 > [!WARNING]
 > Filtering the telemetry sent from the SDK using processors can skew the statistics that you see in the portal, and make it difficult to follow related items.
@@ -44,56 +45,45 @@ To filter telemetry, you write a telemetry processor and register it with the SD
 >
 
 ### Create a telemetry processor (C#)
-1. Verify that the Application Insights SDK in your project is  version 2.0.0 or later. Right-click your project in Visual Studio Solution Explorer and choose Manage NuGet Packages. In NuGet package manager, check Microsoft.ApplicationInsights.Web.
-2. To create a filter, implement ITelemetryProcessor. This is another extensibility point like telemetry module, telemetry initializer, and telemetry channel.
 
-    Notice that Telemetry Processors construct a chain of processing. When you instantiate a telemetry processor, you pass a link to the next processor in the chain. When a telemetry data point is passed to the Process method, it does its work and then calls the next Telemetry Processor in the chain.
+1. To create a filter, implement ITelemetryProcessor. This is another extensibility point like telemetry module, telemetry initializer, and telemetry channel.
+
+    Notice that Telemetry Processors construct a chain of processing. When you instantiate a telemetry processor, you are given a reference to the next processor in the chain. When a telemetry data point is passed to the Process method, it does its work and then calls (or not calls) the next Telemetry Processor in the chain.
 
 ```csharp
 using Microsoft.ApplicationInsights.Channel;
 using Microsoft.ApplicationInsights.Extensibility;
 
 public class SuccessfulDependencyFilter : ITelemetryProcessor
 {
+    private ITelemetryProcessor Next { get; set; }
 
-	private ITelemetryProcessor Next { get; set; }
-
-	// You can pass values from .config
-	public string MyParamFromConfigFile { get; set; }
-
-	// Link processors to each other in a chain.
-	public SuccessfulDependencyFilter(ITelemetryProcessor next)
-	{
-		this.Next = next;
-	}
-	public void Process(ITelemetry item)
-	{
-		// To filter out an item, just return
-		if (!OKtoSend(item)) { return; }
-		// Modify the item if required
-		ModifyItem(item);
+    // next will point to the next TelemetryProcessor in the chain.
+    public SuccessfulDependencyFilter(ITelemetryProcessor next)
+    {
+        this.Next = next;
+    }
 
-		this.Next.Process(item);
-	}
+    public void Process(ITelemetry item)
+    {
+        // To filter out an item, return without calling the next processor.
+        if (!OKtoSend(item)) { return; }
 
-	// Example: replace with your own criteria.
-	private bool OKtoSend (ITelemetry item)
-	{
-		var dependency = item as DependencyTelemetry;
-		if (dependency == null) return true;
+        this.Next.Process(item);
+    }
 
-		return dependency.Success != true;
-	}
+    // Example: replace with your own criteria.
+    private bool OKtoSend (ITelemetry item)
+    {
+        var dependency = item as DependencyTelemetry;
+        if (dependency == null) return true;
 
-	// Example: replace with your own modifiers.
-	private void ModifyItem (ITelemetry item)
-	{
-		item.Context.Properties.Add("app-version", "1." + MyParamFromConfigFile);
-	}
+        return dependency.Success != true;
+    }
 }
 ```
 
-3. Add your processor
+2. Add your processor
 
 **ASP.NET apps**
 Insert this in ApplicationInsights.config:
@@ -119,7 +109,7 @@ You can pass string values from the .config file by providing public named prope
 **Alternatively,** you can initialize the filter in code. In a suitable initialization class - for example AppStart in Global.asax.cs - insert your processor into the chain:
 
 ```csharp
-var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder;
+var builder = TelemetryConfiguration.Active.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
 builder.Use((next) => new SuccessfulDependencyFilter(next));
 
 // If you have more processors:
@@ -130,13 +120,12 @@ builder.Build();
 
 TelemetryClients created after this point will use your processors.
 
-**ASP.NET Core apps**
+**ASP.NET Core/ Worker Service apps**
 
 > [!NOTE]
-> Adding initializer using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications. 
-
+> Adding processor using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications or if you are using Microsoft.ApplicationInsights.WorkerService SDK.
 
-For [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) applications, adding a new `TelemetryInitializer` is done by adding it to the Dependency Injection container, as shown below. This is done in `ConfigureServices` method of your `Startup.cs` class.
+For apps written using [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) pr [WorkerService](worker-service.md#adding-telemetry-processors), adding a new `TelemetryProcessor` is done by using `AddApplicationInsightsTelemetryProcessor` extension method on `IServiceCollection`, as shown below. This is done in `ConfigureServices` method of your `Startup.cs` class.
 
 ```csharp
     public void ConfigureServices(IServiceCollection services)
@@ -151,8 +140,10 @@ For [ASP.NET Core](asp-net-core.md#adding-telemetry-processors) applications, ad
 ```
 
 ### Example filters
+
 #### Synthetic requests
-Filter out bots and web tests. Although Metrics Explorer gives you the option to filter out synthetic sources, this option reduces traffic by filtering them at the SDK.
+
+Filter out bots and web tests. Although Metrics Explorer gives you the option to filter out synthetic sources, this option reduces traffic and ingestion size by filtering them at the SDK itself.
 
 ```csharp
 public void Process(ITelemetry item)
@@ -165,6 +156,7 @@ public void Process(ITelemetry item)
 ```
 
 #### Failed authentication
+
 Filter out requests with a "401" response.
 
 ```csharp
@@ -175,19 +167,21 @@ public void Process(ITelemetry item)
     if (request != null &&
     request.ResponseCode.Equals("401", StringComparison.OrdinalIgnoreCase))
     {
-        // To filter out an item, just terminate the chain:
+        // To filter out an item, return without calling the next processor.
         return;
     }
-    // Send everything else:
+
+    // Send everything else
     this.Next.Process(item);
 }
 ```
 
 #### Filter out fast remote dependency calls
+
 If you only want to diagnose calls that are slow, filter out the fast ones.
 
 > [!NOTE]
-> This will skew the statistics you see on the portal. The dependency chart will look as if the dependency calls are all failures.
+> This will skew the statistics you see on the portal.
 >
 >
 
@@ -205,17 +199,18 @@ public void Process(ITelemetry item)
 ```
 
 #### Diagnose dependency issues
-[This blog](https://azure.microsoft.com/blog/implement-an-application-insights-telemetry-processor/) describes a project to diagnose dependency issues by automatically sending regular pings to dependencies.
 
+[This blog](https://azure.microsoft.com/blog/implement-an-application-insights-telemetry-processor/) describes a project to diagnose dependency issues by automatically sending regular pings to dependencies.
 
 <a name="add-properties"></a>
 
 ## Add properties: ITelemetryInitializer
-Use telemetry initializers to define global properties that are sent with all telemetry; and to override selected behavior of the standard telemetry modules.
+
+Use telemetry initializers to enrich telemetry with additional information and/or to override telemetry properties set by the standard telemetry modules.
 
 For example, the Application Insights for Web package collects telemetry about HTTP requests. By default, it flags as failed any request with a response code >= 400. But if you want to treat 400 as a success, you can provide a telemetry initializer that sets the Success property.
 
-If you provide a telemetry initializer, it is called whenever any of the Track*() methods are called. This includes methods called by the standard telemetry modules. By convention, these modules do not set any property that has already been set by an initializer.
+If you provide a telemetry initializer, it is called whenever any of the Track*() methods are called. This includes methods called by the standard telemetry modules. By convention, these modules do not set any property that has already been set by an initializer. Telemetry initializers are called before calling telemetry processors. So any enrichment done by initializers are visible to processors.
 
 **Define your initializer**
 
@@ -249,9 +244,9 @@ namespace MvcWebRole.Telemetry
 			// If we set the Success property, the SDK won't change it:
 			requestTelemetry.Success = true;
 			// Allow us to filter these requests in the portal:
-			requestTelemetry.Context.Properties["Overridden400s"] = "true";
+			requestTelemetry.Properties["Overridden400s"] = "true";
 		}
-		// else leave the SDK to set the Success property      
+		// else leave the SDK to set the Success property
 	}
   }
 }
@@ -288,7 +283,7 @@ protected void Application_Start()
 > [!NOTE]
 > Adding initializer using `ApplicationInsights.config` or using `TelemetryConfiguration.Active` is not valid for ASP.NET Core applications. 
 
-For [ASP.NET Core](asp-net-core.md#adding-telemetryinitializers) applications, adding a new `TelemetryInitializer` is done by adding it to the Dependency Injection container, as shown below. This is done in `ConfigureServices` method of your `Startup.cs` class.
+For apps written using [ASP.NET Core](asp-net-core.md#adding-telemetryinitializers) or [WorkerService](worker-service.md#adding-telemetryinitializers), adding a new `TelemetryInitializer` is done by adding it to the Dependency Injection container, as shown below. This is done in `Startup.ConfigureServices` method.
 
 ```csharp
  using Microsoft.ApplicationInsights.Extensibility;
@@ -364,25 +359,29 @@ Insert a telemetry initializer immediately after the initialization code that yo
 
 For a summary of the non-custom properties available on the telemetryItem, see [Application Insights Export Data Model](../../azure-monitor/app/export-data-model.md).
 
-You can add as many initializers as you like.
+You can add as many initializers as you like, and they are called in the order they are added.
 
 ## ITelemetryProcessor and ITelemetryInitializer
+
 What's the difference between telemetry processors and telemetry initializers?
 
-* There are some overlaps in what you can do with them: both can be used to add properties to telemetry.
+* There are some overlaps in what you can do with them: both can be used to add properties to telemetry, though it is recommended to use initializers for that purpose.
 * TelemetryInitializers always run before TelemetryProcessors.
+* TelemetryInitializers may be called more than once. By convention, they do not set any property that has already been set.
 * TelemetryProcessors allow you to completely replace or discard a telemetry item.
-* TelemetryProcessors don't process performance counter telemetry.
 
 ## Troubleshooting ApplicationInsights.config
+
 * Confirm that the fully qualified type name and assembly name are correct.
 * Confirm that the applicationinsights.config file is in your output directory and contains any recent changes.
 
 ## Reference docs
+
 * [API Overview](../../azure-monitor/app/api-custom-events-metrics.md)
 * [ASP.NET reference](https://msdn.microsoft.com/library/dn817570.aspx)
 
 ## SDK Code
+
 * [ASP.NET Core SDK](https://github.com/Microsoft/ApplicationInsights-aspnetcore)
 * [ASP.NET SDK](https://github.com/Microsoft/ApplicationInsights-dotnet)
 * [JavaScript SDK](https://github.com/Microsoft/ApplicationInsights-JS)
