You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/load-testing/how-to-compare-multiple-test-runs.md
+34-47Lines changed: 34 additions & 47 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,92 +1,78 @@
1
1
---
2
-
title: Compare load test runs to find regressions
2
+
title: Compare load test runs
3
3
titleSuffix: Azure Load Testing
4
4
description: 'Learn how you can visually compare multiple test runs with Azure Load Testing to identify and analyze performance regressions.'
5
5
services: load-testing
6
6
ms.service: load-testing
7
7
ms.author: nicktrog
8
8
author: ntrogh
9
-
ms.date: 01/18/2023
9
+
ms.date: 01/11/2024
10
10
ms.topic: how-to
11
-
ms.custom: contperf-fy22q3
12
11
---
13
12
14
-
# Identify performance regressions by comparing test runs in Azure Load Testing
13
+
# Compare load test runs in Azure Load Testing
15
14
16
-
In this article, you'll learn how you can identify performance regressions by comparing test runs in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each run, which allows you to quickly analyze performance issues. You will also learn how to view and analyze the trends in client-side performance metrics.
15
+
In this article, you learn how you can compare test runs in Azure Load Testing. You can view trends across the last 10 test runs, or you can select and compare up to five individual test runs. Optionally, you can mark a test run as a baseline to compare against.
17
16
18
-
To identify performance regressions, you can quickly glance over the client-side metrics from your recent test runs to understand if your performance is trending favorably or unfavorably. Optionally, you can compare the recent metrics with a baseline to understand if the performance is meeting your expectations. To dive deeper into a performance regression, you can compare upto five test runs.
17
+
To identify regressions over time, you can use the client-side metrics trends of the last 10 test runs, such as the response time, error rate, and more. In combination with [CI/CD integration](./quickstart-add-load-test-cicd.md), the trends data might help you identify which application build introduced a performance issue.
19
18
20
-
You can compare load test runs for the following scenarios:
19
+
When you want to compare the client-side metrics trends against a specific reference test run, you can mark that test run as your baseline. For example, before you implement performance optimizations in your application, you might first create a baseline load test run, and then validate the effects of your optimizations against your baseline.
21
20
22
-
- Identify performance regressions between application builds or configurations. You could run a load test at each development sprint to ensure that the previous sprint didn't introduce performance issues.
23
-
- Identify which application component is responsible for a performance problem (root cause analysis). For example, an application redesign might result in slower application response times. Comparing load test runs might reveal that the root cause was a lack of database resources.
21
+
To compare both client-side and server-side metrics, you can select up to five test runs, and compare them in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each test run. By also comparing server-side application metrics in the dashboard, you can identify which application component was the root cause for a sudden performance degradation.
24
22
25
23
## Prerequisites
26
24
27
25
- An Azure account with an active subscription. If you don't have an Azure subscription, [create a free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
28
26
29
-
- An Azure Load Testing resource with a test plan that has multiple test runs. To create a Load Testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).
27
+
- An Azure load testing resource, which has a test with multiple test runs. To create a load testing resource, see [Create and run a load test](./quickstart-create-and-run-load-test.md).
30
28
31
-
## Select test runs
29
+
## Compare multiple load test runs
32
30
33
-
To compare test runs in Azure Load Testing, you'll first have to select up to five runs within a load test. You can only compare runs that belong to the same load test.
31
+
To compare test runs in Azure Load Testing, you first have to select up to five runs within a load test. You can only compare runs that belong to the same load test. After you select the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
34
32
35
33
A test run needs to be in the *Done*, *Stopped*, or *Failed* state to compare it.
36
34
37
35
Use the following steps to select the test runs:
38
36
39
37
1. Sign in to the [Azure portal](https://portal.azure.com) by using the credentials for your Azure subscription.
40
38
41
-
1. Go to your Azure Load Testing resource and then, on the left pane, select **Tests**.
42
-
43
-
:::image type="content" source="media/how-to-compare-multiple-test-runs/choose-test-from-list.png" alt-text="Screenshot that shows the list of tests for a Load Testing resource.":::
39
+
1. Go to your load testing resource, and then select **Tests** in the left pane.
44
40
45
-
You can also use the filters to find your load test.
41
+
> [!TIP]
42
+
> You can also use the filters to find your load test.
46
43
47
44
1. Select the test whose runs you want to compare by selecting its name.
48
45
49
-
1. Select two or more test runs by selecting the corresponding checkboxes in the list.
50
-
51
-
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png" alt-text="Screenshot that shows a list of test runs and the 'Compare' button.":::
46
+
1. Select two or more test runs, and then select **Compare** to compare test runs.
52
47
53
48
You can choose a maximum of five test runs to compare.
54
49
55
-
## Compare multiple test runs
56
-
57
-
After you've selected the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
58
-
59
-
1. Select the **Compare** button to open the load test dashboard.
60
-
61
-
Each test run is shown as an overlay in the different graphs.
62
-
63
-
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-screen.png" alt-text="Screenshot of the 'Compare' page, displaying a comparison of two test runs.":::
50
+
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png" alt-text="Screenshot that shows a list of test runs and the 'Compare' button in the Azure portal." lightbox="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png":::
64
51
65
-
1.Optionally, use the filters to customize the graphs.
52
+
1.On the dashboard, each test run is shown as an overlay in the different graphs.
66
53
67
-
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-client-side-filters.png" alt-text="Screenshot of the client-side filter controls on the load test dashboard.":::
54
+
The dashboard enables you to compare both client-side metrics and server-side metrics. You can view the color-coding for each test run in the **Test run details** section.
68
55
69
-
> [!TIP]
56
+
> [!NOTE]
70
57
> The time filter is based on the duration of the tests. A value of zero indicates the start of the test, and the maximum value marks the duration of the longest test run.
71
58
72
-
## View metrics trends across test runs
59
+
:::image type="content" source="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png" alt-text="Screenshot of the load testing dashboard in the Azure portal, comparing two test runs." lightbox="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png":::
60
+
61
+
## View metrics trends across load test runs
73
62
74
-
To view metrics trends across test runs in Azure Load Testing, you'll need to have at least two test runs in the *Done*, or *Stopped* state. You can only view trends from runs that belong to the same load test.
63
+
To view metrics trends across test runs in Azure Load Testing, you need to have at least two test runs in the *Done*, or *Stopped* state. You can only view trends from runs that belong to the same load test.
75
64
76
65
Use the following steps to view metrics trends across test runs:
77
66
78
67
1. Sign in to the [Azure portal](https://portal.azure.com) by using the credentials for your Azure subscription.
79
68
80
69
1. Go to your Azure Load Testing resource and then, on the left pane, select **Tests**.
81
70
82
-
:::image type="content" source="media/how-to-compare-multiple-test-runs/choose-test-from-list.png" alt-text="Screenshot that shows the list of tests for a Load Testing resource." lightbox="media/how-to-compare-multiple-test-runs/choose-test-from-list.png":::
83
-
84
-
You can also use the filters to find your load test.
85
71
1. Select the test for which you want to view metrics trends by selecting its name.
86
72
87
-
1.On the **Test details**pane, select **Trends**
73
+
1.Select the **Trends**tab to view the metrics trends for the load test.
88
74
89
-
The graphs show the trends for total requests, response time, error percentage, and throughput for the ten most recent test runs.
75
+
The graphs show the trends for total requests, response time, error percentage, and throughput for the 10 most recent test runs.
90
76
91
77
:::image type="content" source="media/how-to-compare-multiple-test-runs/choose-trends-from-test-details.png" alt-text="Screenshot that shows the details of a Test in a Load Testing resource." lightbox="media/how-to-compare-multiple-test-runs/choose-trends-from-test-details.png":::
92
78
@@ -96,28 +82,29 @@ Use the following steps to view metrics trends across test runs:
96
82
97
83
You can select a test run that you want to analyze and open the results dashboard for that test run.
98
84
99
-
## Use a baseline test run
85
+
## Compare load test runs against a baseline
100
86
101
-
You can mark a test run as baseline to compare the client-side metrics of the recent test runs with those of the baseline.
87
+
You can mark a test run as a baseline to compare the client-side metrics of the recent test runs with the metrics of the baseline.
102
88
103
89
Use the following steps to mark a test run as baseline:
104
90
105
-
1. On the **Trends**pane, select **Mark baseline**
91
+
1. On the **Trends**tab, select **Mark baseline**.
106
92
107
93
:::image type="content" source="media/how-to-compare-multiple-test-runs/select-mark-baseline.png" alt-text="Screenshot that shows Mark baseline button in the Trends pane." lightbox="media/how-to-compare-multiple-test-runs/select-mark-baseline.png":::
108
94
109
-
1.In the right context pane, select the checkbox for the test run that you want to mark as baseline, and then select **Mark baseline**
95
+
1.From the list of test runs, select the checkbox for the test run that you want to mark as baseline, and then select **Mark baseline**.
110
96
111
97
:::image type="content" source="media/how-to-compare-multiple-test-runs/mark-test-run-as-baseline.png" alt-text="Screenshot that shows the context pane to mark a test run as baseline." lightbox="media/how-to-compare-multiple-test-runs/mark-test-run-as-baseline.png":::
112
-
113
-
You can also use the filters to find your load test run.
114
98
115
-
The baseline value is shown as a horizontal line in the charts. In the table view, an additional row with the baseline test run details is shown. For the recent test runs, an arrow mark next to the metrics
116
-
value indicates whether the metric is trending favorably or unfavorably as compared to the baseline metric value.
99
+
1. On the **Trends** tab, you can now view the baseline test run in the table and charts.
100
+
101
+
The baseline value is shown as a horizontal line in the charts. In the table view, an extra row with the baseline test run details is shown.
102
+
103
+
In the table, an arrow icon indicates whether the metric is trending favorably or unfavorably as compared to the baseline metric value.
117
104
118
105
:::image type="content" source="media/how-to-compare-multiple-test-runs/trends-view-with-baseline.png" alt-text="Screenshot that shows trends in metrics when a baseline is selected." lightbox="media/how-to-compare-multiple-test-runs/trends-view-with-baseline.png":::
119
106
120
-
## Next steps
107
+
## Related content
121
108
122
109
- Learn more about [exporting the load test results for reporting](./how-to-export-test-results.md).
123
110
- Learn more about [diagnosing failing load tests](./how-to-diagnose-failing-load-test.md).
0 commit comments