Merge pull request #220544 from Saglodha/patch-4

Updating entity level app group creation

Author: v-shils

articles/event-hubs/media/resource-governance-with-app-groups/create-application-groups-with-event-hub-shared-access-key.png

@@ -14,6 +14,7 @@ This article shows you how to perform the following tasks:
 - Create an application group.
 - Enable or disable an application group
 - Define threshold limits and apply throttling policies to an application group
+- Validate throttling with Diagnostic Logs
 
 > [!NOTE] 
 > Application groups are available only in **premium** and **dedicated** tiers. 
@@ -32,27 +33,35 @@ You can create an application group using the Azure portal by following these st
 1. On the **Add application group** page, follow these steps:
     1. Specify a **name** for the application group.
     1. Confirm that **Enabled** is selected. To have the application group in the disabled state first, clear the **Enabled** option. This flag determines whether the clients of an application group can access Event Hubs or not.
-    1. For **Security context type**, select **Shared access policy** or **AAD application**. When you create the application group, you should associate with either a shared access signatures (SAS) or Azure Active Directory(Azure AD) application ID, which is used by client applications. 
-    1. If you selected **Shared access policy**:
-        1. For **SAS key name**, select the SAS policy that can be used as a security context for this application group. Application group supports the selection of SAS key at either namespace or at entity (event hub) level. You can select **Add SAS Policy** to add a new policy and then associate with the application group. 
-        1. Review the auto-generated **Client group ID**, which is the unique ID associated with the application group. You can update it if you like. The following table  shows  auto generated Client Group ID for different level keys: 
+    1. For **Security context type**, select **Namespace Shared access policy**, **event hub Shared Access Policy** or **AAD application**.Application group supports the selection of SAS key at either namespace or at entity (event hub) level. When you create the application group, you should associate with either a shared access signatures (SAS) or Azure Active Directory(Azure AD) application ID, which is used by client applications. 
+    1. If you selected **Namespace Shared access policy**:
+        1. For **SAS key name**, select the SAS policy that can be used as a security context for this application group.You can select **Add SAS Policy** to add a new policy and then associate with the application group. 
+	
+              :::image type="content" source="./media/resource-governance-with-app-groups/create-application-groups-with-namespace-shared-access-key.png" alt-text="Screenshot of the Add application group page with Namespace Shared access policy option selected.":::
+     1. If you selected **Event Hubs Shared access policy**:
+        1. For **SAS key name**, copy the SAS policy name from Event Hubs "Shared Access Policies" Page and paste into textbox
 
-              | Key type | Auto-generated client group ID |
-              | -------- | ------------------------------ |
-              | Namespace-level key | `NamespaceSASKeyName=RootManageSharedAccessKey` |
-              | Entity-level Key | `EntitySASKeyName=RootManageSharedAccessKey` | 
-    
-              > [!NOTE]
-              > All existing application groups created with namespace level key would continue to work with client group ID starting with `SASKeyName`. However all new application groups would have updated client group ID as shown above. 
-
-    
-              :::image type="content" source="./media/resource-governance-with-app-groups/add-app-group.png" alt-text="Screenshot of the Add application group page with Shared access policy option selected.":::
+              :::image type="content" source="./media/resource-governance-with-app-groups/create-application-groups-with-event-hub-shared-access-key.png" alt-text="Screenshot of the Add application group page with event hub Shared access policy option selected.":::
+     
       1. If you selected **AAD application**:
           1. For **AAD Application (client) ID**, specify the Azure Active Directory (Azure AD) application or client ID. 
-          1. Review the auto-generated **Client group ID**, which is the unique ID associated with the application group. You can update it if you like. The scope of application governance (namespace or entity level) would depend on the access level for the used Azure AD application ID.  
+          
+          	:::image type="content" source="./media/resource-governance-with-app-groups/add-app-group-active-directory.png" alt-text="Screenshot of the Add application group page with Azure AD option.":::
+
+### [Supported Security Context type](#supported-security-context-type)
+Review the auto-generated **Client group ID**, which is the unique ID associated with the application group. The scope of application governance (namespace or entity level) would depend on the access level for the used Azure AD application ID. The following table  shows auto generated Client Group ID for different security Context type: 
+    
+| Security Context type | Auto-generated client group ID|
+| ---| ---		| 
+| Namespace shared access key	| `NamespaceSASKeyName=<NamespaceLevelKeyName>` |
+| Azure AD Application		| `AADAppID=<AppID>`				|
+| Event Hubs shared access key 	| `EntitySASKeyName=<EntityLevelKeyName>` 	| 
+		
+> [!NOTE]
+ > All existing application groups created with namespace shared access key would continue to work with client group ID starting with `SASKeyName`. However all new application groups would have updated client group ID as shown above.
 
-            :::image type="content" source="./media/resource-governance-with-app-groups/add-app-group-active-directory.png" alt-text="Screenshot of the Add application group page with Azure AD option.":::
-    1. To add a policy, follow these steps:
+            
+   1. To add a policy, follow these steps:
         1. Enter a **name** for the policy.
         1. For **Type**, select **Throttling policy**. 
         1. For **Metric ID**, select one of the following options: **Incoming messages**, **Outgoing messages**, **Incoming bytes**, **Outgoing bytes**. In the following example, **Incoming messages** is selected. 
@@ -72,28 +81,30 @@ You can create an application group using the Azure portal by following these st
 
 
 ### [Azure CLI](#tab/cli)
-Use the CLI command: [`az eventhubs namespace application-group create`](/cli/azure/eventhubs/namespace/application-group#az-eventhubs-namespace-application-group-create) to create an application group in an Event Hubs namespace. 
+Use the CLI command: [`az eventhubs namespace application-group create`](/cli/azure/eventhubs/namespace/application-group#az-eventhubs-namespace-application-group-create) to create an application group at Event Hubs namespace or event hub level. You must set --client-app-group-identifier based on the security
+context type you are choosing. Please review the [table](#supported-security-context-type) above to know supported Security context type
 
 The following example creates an application group named `myAppGroup` in the namespace `mynamespace` in the Azure resource group `MyResourceGroup`. It uses the following configurations.
 
 - Shared access policy is used as the security context
-- Client app group ID is set to `SASKeyName=<NameOfTheSASkey>`.
+- Client app group ID is set to `NamespaceSASKeyName=<NameOfTheSASkey>`.
 - First throttling policy for the `Incoming messages` metric with `10000` as the threshold.
 - Second throttling policy for the `Incoming bytes` metric with `20000` as the threshold. 
 
 ```azurecli-interactive
 az eventhubs namespace application-group create --namespace-name mynamespace \
                                                 -g MyResourceGroup \
                                                 --name myAppGroup \
-                                                --client-app-group-identifier SASKeyName=keyname \
+                                                --client-app-group-identifier NamespaceSASKeyName=keyname \
                                                 --throttling-policy-config name=policy1 metric-id=IncomingMessages rate-limit-threshold=10000 \
                                                 --throttling-policy-config name=policy2 metric-id=IncomingBytes rate-limit-threshold=20000
 ```
 
 To learn more about the CLI command, see [`az eventhubs namespace application-group create`](/cli/azure/eventhubs/namespace/application-group#az-eventhubs-namespace-application-group-create). 
 
 ### [Azure PowerShell](#tab/powershell)
-Use the PowerShell command: [`New-AzEventHubApplicationGroup`](/powershell/module/az.eventhub/new-azeventhubapplicationgroup) to create an application group in an Event Hubs namespace. 
+Use the PowerShell command: [`New-AzEventHubApplicationGroup`](/powershell/module/az.eventhub/new-azeventhubapplicationgroup) to create an application group  at Event Hubs namespace or event hub level. You must set -ClientAppGroupIdentifier based on the security
+context type you are choosing. Please review the [table](#supported-security-context-type) above to know supported Security context type
 
 The following example uses the [`New-AzEventHubThrottlingPolicyConfig`](/powershell/module/az.eventhub/new-azeventhubthrottlingpolicyconfig) to create two policies that will be associated with the application.
 
@@ -108,13 +119,13 @@ $policy1 = New-AzEventHubThrottlingPolicyConfig -Name policy1 -MetricId Incoming
 $policy2 = New-AzEventHubThrottlingPolicyConfig -Name policy2 -MetricId IncomingMessages -RateLimitThreshold 23416
 
 New-AzEventHubApplicationGroup -ResourceGroupName myresourcegroup -NamespaceName mynamespace -Name myappgroup 
-		-ClientAppGroupIdentifier SASKeyName=myauthkey -ThrottlingPolicyConfig $policy1, $policy2
+		-ClientAppGroupIdentifier NamespaceSASKeyName=myauthkey -ThrottlingPolicyConfig $policy1, $policy2
 ```
 
 To learn more about the PowerShell command, see [`New-AzEventHubApplicationGroup`](/powershell/module/az.eventhub/new-azeventhubapplicationgroup).
 
 ### [ARM template](#tab/arm)
-The following example shows how to create an application group using an ARM template. In this example, the application group is associated with an existing SAS policy name `contososaspolicy` by setting the client `AppGroupIdentifier` as `SASKeyName=contososaspolicy`. The application group policies are also defined in the ARM template. 
+The following example shows how to create an application group using an ARM template. In this example, the application group is associated with an existing SAS policy name `contososaspolicy` by setting the client `AppGroupIdentifier` as `NamespaceSASKeyName=contososaspolicy`. The application group policies are also defined in the ARM template. You must set ClientAppGroupIdentifier based on the security context type you are choosing. Please review the [table](#supported-security-context-type) above to know supported Security context type
 
 
 ```json
@@ -127,7 +138,7 @@ The following example shows how to create an application group using an ARM temp
 		"[resourceId('Microsoft.EventHub/namespaces/authorizationRules', parameters('eventHubNamespaceName'),parameters('namespaceAuthorizationRuleName'))]"
 	],
 	"properties": {
-		"ClientAppGroupIdentifier": "SASKeyName=contososaspolicy",
+		"ClientAppGroupIdentifier": "NamespaceSASKeyName=contososaspolicy",
 		"policies": [{
 				"Type": "ThrottlingPolicy",
 				"Name": "ThrottlingPolicy1",
@@ -269,7 +280,7 @@ Set-AzEventHubApplicationGroup -ResourceGroupName myresourcegroup -NamespaceName
 ```
 
 ### [ARM template](#tab/arm)
-The following ARM template shows how to update an existing namespace (`contosonamespace`) to add throttling policies. The identifier for the app group is `SASKeyName=RootManageSharedAccessKey`. 
+The following ARM template shows how to update an existing namespace (`contosonamespace`) to add throttling policies. The identifier for the app group is `NamespaceSASKeyName=RootManageSharedAccessKey`. 
 
 ```json
 {
@@ -281,7 +292,7 @@ The following ARM template shows how to update an existing namespace (`contosona
       "type": "String"
     },
     "client-app-group-identifier": {
-      "defaultValue": "SASKeyName=RootManageSharedAccessKey",
+      "defaultValue": "NamespaceSASKeyName=RootManageSharedAccessKey",
       "type": "String"
     }
   },
@@ -317,12 +328,12 @@ The following ARM template shows how to update an existing namespace (`contosona
 
 ### Decide threshold value for throttling policies 
 
-Azure Event Hubs supports [runtime audit logs](monitor-event-hubs-reference.md#runtime-audit-logs) functionality to help you decide on a threshold value for your usual throughput to throttle the application group. You can follow these steps to find out threshold value to explore a good threshold value: 
+Azure Event Hubs supports [Application Metric Logs ](monitor-event-hubs-reference.md#application-metrics-logs) functionality to observe usual throughput within your system and accordingly decide on the threshold value for application group. You can follow these steps to decide on a threshold value:
 
-1. Turn on [diagnostic settings](monitor-event-hubs.md#collection-and-routing) in Event Hubs with **runtime audit logs** as selected category and choose **Log Analytics** as destination.  
+1. Turn on [diagnostic settings](monitor-event-hubs.md#collection-and-routing) in Event Hubs with **Application Metric logs** as selected category and choose **Log Analytics** as destination.  
 2. Create an empty application group without any throttling policy.  
 3. Continue sending messages/events to event hub at usual throughput. 
-4. Go to **Log Analytics workspace** and query for the right activity name (based on the metric ID) in **AzureDiagnostics** table. The following sample query is set to track threshold value for incoming messages:  
+4. Go to **Log Analytics workspace** and query for the right activity name (based on the (resource-governance-overview.md#throttling-policy---threshold-limits)) in **AzureDiagnostics** table. The following sample query is set to track threshold value for incoming messages:  
 
     ```kusto
     AzureDiagnostics 
@@ -334,15 +345,30 @@ Azure Event Hubs supports [runtime audit logs](monitor-event-hubs-reference.md#r
     :::image type="content" source="./media/resource-governance-with-app-groups/azure-monitor-logs.png" lightbox="./media/resource-governance-with-app-groups/azure-monitor-logs.png" alt-text="Screenshot of the Azure Monitor logs page in the Azure portal.":::
     
     In this example, you can see that the usual throughput never crossed more than 550 messages (expected current throughput). This observation helps you define the actual threshold value.   
-6. Once you decide the best threshold value, add a new throttling policy inside the application group. 
+6. Once you decide the threshold value, add a new throttling policy inside the application group. 
 
 ## Publish or consume events 
 Once you successfully add throttling policies to the application group, you can test the throttling behavior by either publishing or consuming events using client applications that are part of the `contosoAppGroup` application group. To test, you can use either an [AMQP client](event-hubs-dotnet-standard-getstarted-send.md) or a [Kafka client](event-hubs-quickstart-kafka-enabled-event-hubs.md) application and same SAS policy name or Azure AD application ID that's used to create the application group. 
 
 > [!NOTE]
 > When your client applications are throttled, you should experience a slowness in publishing or consuming data. 
 
+### Validate Throttling with Application Groups
+
+Similar to [Deciding Threshold limits for Throttling Policies](resource-governance-with-app-groups.md#decide-threshold-value-for-throttling-policies), you can use Application Metric logs to validate throttling and find more details. 
 
+You can use the below example query to find out all the throttled requests in certain timeframe. You must update the ActivityName to match the operation that you expect to be throttled. 
+
+
+  ```kusto
+  
+    AzureDiagnostics 
+    |  where Category =="ApplicationMetricsLogs"
+    | where ActivityName_s =="IncomingMessages" 
+    | where Outcome_s =="Throttled"  
+	
+  ``` 
+Due to restrictions at protocol level, throttled request logs are not generated for consumer operations within event hub ( `OutgoingMessages` or `OutgoingBytes`). when requests are throttled at consumer side, you would observe sluggish egress throughput. 
 
 ## Next steps