Merge pull request #202402 from ninallam/ninallam-criteria

Additional metrics and statistics

Author: PMEds28

articles/load-testing/how-to-define-test-criteria.md

@@ -26,23 +26,29 @@ By defining test criteria, you can specify the performance expectations of your
 
 ## Load test pass/fail criteria
 
-This section discusses the syntax you use to define Azure Load Testing pass/fail criteria.
+This section discusses the syntax of Azure Load Testing pass/fail criteria. When a criterion evaluates to `true`, the load test gets the *failed* status.
 
-You use `Aggregate_function (client_metric) condition value` syntax. When a criterion evaluates to `true`, the load test gets the *failed* status.
+The structure of a pass/fail criterion is: `Request: Aggregate_function (client_metric) condition threshold`.
+
+The following table describes the different components:
 
 |Parameter  |Description  |
 |---------|---------|
+|`Request`     | *Optional.* Name of the sampler in the JMeter script to which the criterion applies. If you don't specify a request name, the criterion applies to the aggregate of all the requests in the script.  |
 |`Client metric`     | *Required.* The client metric on which the criteria should be applied.  |
 |`Aggregate function`     |  *Required.* The aggregate function to be applied on the client metric.  |
 |`Condition`     | *Required.* The comparison operator.        |
-|`Threshold`     |  *Required.* The numeric value to compare with the client metric.<BR>The threshold evaluates against the aggregated value. |
+|`Threshold`     |  *Required.* The numeric value to compare with the client metric. |
 
-Load Testing supports the following combination of parameters:
+Azure Load Testing supports the following metrics:
 
 |Metric  |Aggregate function  |Threshold  |Condition  |
 |---------|---------|---------|---------|
-|`response_time_ms`     |  `avg` (average)       | Integer value, representing number of milliseconds (ms)     |   `>` (greater than)      |
-|`error`     |  `percentage`       | Numerical values in the range 0-100, representing a percentage      |   `>` (greater than)      |
+|`response_time_ms`     |  `avg` (average)<BR> `min` (minimum)<BR> `max` (maximum)<BR> `pxx` (percentile), xx can be 50, 90, 95, 99     | Integer value, representing number of milliseconds (ms).     |   `>` (greater than)<BR> `<` (less than)      |
+|`latency_ms`     |  `avg` (average)<BR> `min` (minimum)<BR> `max` (maximum)<BR> `pxx` (percentile), xx can be 50, 90, 95, 99     | Integer value, representing number of milliseconds (ms).     |   `>` (greater than)<BR> `<` (less than)      |
+|`error`     |  `percentage`       | Numerical value in the range 0-100, representing a percentage.      |   `>` (greater than) <BR> `<` (less than)      |
+|`requests_per_sec`     |  `avg` (average)       | Numerical value with up to two decimal places.      |   `>` (greater than) <BR> `<` (less than)     |
+|`requests`     |  `count`       | Integer value.      |   `>` (greater than) <BR> `<` (less than)     |
 
 ## Define test pass/fail criteria in the Azure portal
 
@@ -82,6 +88,7 @@ In this section, you learn how to define load test pass/fail criteria for contin
     failureCriteria: 
         - avg(response_time_ms) > 300
         - percentage(error) > 20
+        - GetCustomerDetails: avg(latency_ms) >200
     ```
 
 1. Save the YAML configuration file.

articles/load-testing/media/how-to-define-test-criteria/configure-test-test-criteria.png

@@ -30,7 +30,7 @@ A test configuration uses the following keys:
 | `engineInstances` | integer |  | *Required*. Number of parallel instances of the test engine to execute the provided test plan. You can update this property to increase the amount of load that the service can generate. |
 | `configurationFiles` | array |  | List of relevant configuration files or other files that you reference in the Apache JMeter script. For example, a CSV data set file, images, or any other data file. These files will be uploaded to the Azure Load Testing resource alongside the test script. If the files are in a subfolder on your local machine, use file paths that are relative to the location of the test script. <BR><BR>Azure Load Testing currently doesn't support the use of file paths in the JMX file. When you reference an external file in the test script, make sure to only specify the file name. |
 | `description` | string |  | Short description of the test run. |
-| `failureCriteria` | object |  | Criteria that indicate failure of the test. Each criterion is in the form of:<BR>`[Aggregate_function] ([client_metric]) > [value]`<BR><BR>- `[Aggregate function] ([client_metric])` is either `avg(response_time_ms)` or `percentage(error).`<BR>- `value` is an integer number. |
+| `failureCriteria` | object |  | Criteria that indicate when a test should fail. The structure of a pass/fail criterion is: `Request: Aggregate_function (client_metric) condition threshold`. For more information on the supported values, see [Load test pass/fail criteria](./how-to-define-test-criteria.md#load-test-passfail-criteria). |
 | `properties` | object |  | List of properties to configure the load test. |
 | `properties.userPropertyFile` | string |  | File to use as an Apache JMeter [user properties file](https://jmeter.apache.org/usermanual/test_plan.html#properties). The file will be uploaded to the Azure Load Testing resource alongside the JMeter test script and other configuration files. If the file is in a subfolder on your local machine, use a path relative to the location of the test script. |
 | `splitAllCSVs` | boolean | False | Split the input CSV files evenly across all test engine instances. For more information, see [Read a CSV file in load tests](./how-to-read-csv-data.md#split-csv-input-data-across-test-engines). |
@@ -57,6 +57,7 @@ configurationFiles:
 failureCriteria:
   - avg(response_time_ms) > 300
   - percentage(error) > 50
+  - GetCustomerDetails: avg(latency_ms) >200
 splitAllCSVs: True
 env:
   - name: my-variable