Skip to content

Commit 20572b4

Browse files
authored
Merge pull request #263011 from ntrogh/alt-freshness-0124
[Azure Load Testing] Content freshness - January 2024
2 parents 415d58b + 5787263 commit 20572b4

13 files changed

+70
-91
lines changed

articles/load-testing/how-to-compare-multiple-test-runs.md

Lines changed: 34 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,92 +1,78 @@
11
---
2-
title: Compare load test runs to find regressions
2+
title: Compare load test runs
33
titleSuffix: Azure Load Testing
44
description: 'Learn how you can visually compare multiple test runs with Azure Load Testing to identify and analyze performance regressions.'
55
services: load-testing
66
ms.service: load-testing
77
ms.author: nicktrog
88
author: ntrogh
9-
ms.date: 01/18/2023
9+
ms.date: 01/11/2024
1010
ms.topic: how-to
11-
ms.custom: contperf-fy22q3
1211
---
1312

14-
# Identify performance regressions by comparing test runs in Azure Load Testing
13+
# Compare load test runs in Azure Load Testing
1514

16-
In this article, you'll learn how you can identify performance regressions by comparing test runs in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each run, which allows you to quickly analyze performance issues. You will also learn how to view and analyze the trends in client-side performance metrics.
15+
In this article, you learn how you can compare test runs in Azure Load Testing. You can view trends across the last 10 test runs, or you can select and compare up to five individual test runs. Optionally, you can mark a test run as a baseline to compare against.
1716

18-
To identify performance regressions, you can quickly glance over the client-side metrics from your recent test runs to understand if your performance is trending favorably or unfavorably. Optionally, you can compare the recent metrics with a baseline to understand if the performance is meeting your expectations. To dive deeper into a performance regression, you can compare upto five test runs.
17+
To identify regressions over time, you can use the client-side metrics trends of the last 10 test runs, such as the response time, error rate, and more. In combination with [CI/CD integration](./quickstart-add-load-test-cicd.md), the trends data might help you identify which application build introduced a performance issue.
1918

20-
You can compare load test runs for the following scenarios:
19+
When you want to compare the client-side metrics trends against a specific reference test run, you can mark that test run as your baseline. For example, before you implement performance optimizations in your application, you might first create a baseline load test run, and then validate the effects of your optimizations against your baseline.
2120

22-
- Identify performance regressions between application builds or configurations. You could run a load test at each development sprint to ensure that the previous sprint didn't introduce performance issues.
23-
- Identify which application component is responsible for a performance problem (root cause analysis). For example, an application redesign might result in slower application response times. Comparing load test runs might reveal that the root cause was a lack of database resources.
21+
To compare both client-side and server-side metrics, you can select up to five test runs, and compare them in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each test run. By also comparing server-side application metrics in the dashboard, you can identify which application component was the root cause for a sudden performance degradation.
2422

2523
## Prerequisites
2624

2725
- An Azure account with an active subscription. If you don't have an Azure subscription, [create a free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
2826

29-
- An Azure Load Testing resource with a test plan that has multiple test runs. To create a Load Testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).
27+
- An Azure load testing resource, which has a test with multiple test runs. To create a load testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).
3028

31-
## Select test runs
29+
## Compare multiple load test runs
3230

33-
To compare test runs in Azure Load Testing, you'll first have to select up to five runs within a load test. You can only compare runs that belong to the same load test.
31+
To compare test runs in Azure Load Testing, you first have to select up to five runs within a load test. You can only compare runs that belong to the same load test. After you select the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
3432

3533
A test run needs to be in the *Done*, *Stopped*, or *Failed* state to compare it.
3634

3735
Use the following steps to select the test runs:
3836

3937
1. Sign in to the [Azure portal](https://portal.azure.com) by using the credentials for your Azure subscription.
4038

41-
1. Go to your Azure Load Testing resource and then, on the left pane, select **Tests**.
42-
43-
:::image type="content" source="media/how-to-compare-multiple-test-runs/choose-test-from-list.png" alt-text="Screenshot that shows the list of tests for a Load Testing resource.":::
39+
1. Go to your load testing resource, and then select **Tests** in the left pane.
4440

45-
You can also use the filters to find your load test.
41+
> [!TIP]
42+
> You can also use the filters to find your load test.
4643
4744
1. Select the test whose runs you want to compare by selecting its name.
4845

49-
1. Select two or more test runs by selecting the corresponding checkboxes in the list.
50-
51-
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png" alt-text="Screenshot that shows a list of test runs and the 'Compare' button.":::
46+
1. Select two or more test runs, and then select **Compare** to compare test runs.
5247

5348
You can choose a maximum of five test runs to compare.
5449

55-
## Compare multiple test runs
56-
57-
After you've selected the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
58-
59-
1. Select the **Compare** button to open the load test dashboard.
60-
61-
Each test run is shown as an overlay in the different graphs.
62-
63-
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-screen.png" alt-text="Screenshot of the 'Compare' page, displaying a comparison of two test runs.":::
50+
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png" alt-text="Screenshot that shows a list of test runs and the 'Compare' button in the Azure portal." lightbox="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png":::
6451

65-
1. Optionally, use the filters to customize the graphs.
52+
1. On the dashboard, each test run is shown as an overlay in the different graphs.
6653

67-
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-client-side-filters.png" alt-text="Screenshot of the client-side filter controls on the load test dashboard.":::
54+
The dashboard enables you to compare both client-side metrics and server-side metrics. You can view the color-coding for each test run in the **Test run details** section.
6855

69-
> [!TIP]
56+
> [!NOTE]
7057
> The time filter is based on the duration of the tests. A value of zero indicates the start of the test, and the maximum value marks the duration of the longest test run.
7158
72-
## View metrics trends across test runs
59+
:::image type="content" source="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png" alt-text="Screenshot of the load testing dashboard in the Azure portal, comparing two test runs." lightbox="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png":::
60+
61+
## View metrics trends across load test runs
7362

74-
To view metrics trends across test runs in Azure Load Testing, you'll need to have at least two test runs in the *Done*, or *Stopped* state. You can only view trends from runs that belong to the same load test.
63+
To view metrics trends across test runs in Azure Load Testing, you need to have at least two test runs in the *Done*, or *Stopped* state. You can only view trends from runs that belong to the same load test.
7564

7665
Use the following steps to view metrics trends across test runs:
7766

7867
1. Sign in to the [Azure portal](https://portal.azure.com) by using the credentials for your Azure subscription.
7968

8069
1. Go to your Azure Load Testing resource and then, on the left pane, select **Tests**.
8170

82-
:::image type="content" source="media/how-to-compare-multiple-test-runs/choose-test-from-list.png" alt-text="Screenshot that shows the list of tests for a Load Testing resource." lightbox="media/how-to-compare-multiple-test-runs/choose-test-from-list.png":::
83-
84-
You can also use the filters to find your load test.
8571
1. Select the test for which you want to view metrics trends by selecting its name.
8672

87-
1. On the **Test details** pane, select **Trends**
73+
1. Select the **Trends** tab to view the metrics trends for the load test.
8874

89-
The graphs show the trends for total requests, response time, error percentage, and throughput for the ten most recent test runs.
75+
The graphs show the trends for total requests, response time, error percentage, and throughput for the 10 most recent test runs.
9076

9177
:::image type="content" source="media/how-to-compare-multiple-test-runs/choose-trends-from-test-details.png" alt-text="Screenshot that shows the details of a Test in a Load Testing resource." lightbox="media/how-to-compare-multiple-test-runs/choose-trends-from-test-details.png":::
9278

@@ -96,28 +82,29 @@ Use the following steps to view metrics trends across test runs:
9682

9783
You can select a test run that you want to analyze and open the results dashboard for that test run.
9884

99-
## Use a baseline test run
85+
## Compare load test runs against a baseline
10086

101-
You can mark a test run as baseline to compare the client-side metrics of the recent test runs with those of the baseline.
87+
You can mark a test run as a baseline to compare the client-side metrics of the recent test runs with the metrics of the baseline.
10288

10389
Use the following steps to mark a test run as baseline:
10490

105-
1. On the **Trends** pane, select **Mark baseline**
91+
1. On the **Trends** tab, select **Mark baseline**.
10692

10793
:::image type="content" source="media/how-to-compare-multiple-test-runs/select-mark-baseline.png" alt-text="Screenshot that shows Mark baseline button in the Trends pane." lightbox="media/how-to-compare-multiple-test-runs/select-mark-baseline.png":::
10894

109-
1. In the right context pane, select the checkbox for the test run that you want to mark as baseline, and then select **Mark baseline**
95+
1. From the list of test runs, select the checkbox for the test run that you want to mark as baseline, and then select **Mark baseline**.
11096

11197
:::image type="content" source="media/how-to-compare-multiple-test-runs/mark-test-run-as-baseline.png" alt-text="Screenshot that shows the context pane to mark a test run as baseline." lightbox="media/how-to-compare-multiple-test-runs/mark-test-run-as-baseline.png":::
112-
113-
You can also use the filters to find your load test run.
11498

115-
The baseline value is shown as a horizontal line in the charts. In the table view, an additional row with the baseline test run details is shown. For the recent test runs, an arrow mark next to the metrics
116-
value indicates whether the metric is trending favorably or unfavorably as compared to the baseline metric value.
99+
1. On the **Trends** tab, you can now view the baseline test run in the table and charts.
100+
101+
The baseline value is shown as a horizontal line in the charts. In the table view, an extra row with the baseline test run details is shown.
102+
103+
In the table, an arrow icon indicates whether the metric is trending favorably or unfavorably as compared to the baseline metric value.
117104

118105
:::image type="content" source="media/how-to-compare-multiple-test-runs/trends-view-with-baseline.png" alt-text="Screenshot that shows trends in metrics when a baseline is selected." lightbox="media/how-to-compare-multiple-test-runs/trends-view-with-baseline.png":::
119106

120-
## Next steps
107+
## Related content
121108

122109
- Learn more about [exporting the load test results for reporting](./how-to-export-test-results.md).
123110
- Learn more about [diagnosing failing load tests](./how-to-diagnose-failing-load-test.md).

0 commit comments

Comments
 (0)