You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/load-testing/tutorial-cicd-azure-pipelines.md
+18-17Lines changed: 18 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -407,13 +407,17 @@ In addition, the [load test results file](./how-to-export-test-results.md) is av
407
407
408
408
---
409
409
410
-
## Define test pass/fail criteria
410
+
## Define test fail criteria
411
411
412
-
In this section, you'll add criteria to determine whether your load test passes or fails. If at least one of the pass/fail criteria evaluates to `true`, the load test is unsuccessful.
412
+
Azure Load Testing enables you to define load test fail criteria. These are criteria that determine when a load test should pass or fail. For example, your load test should fail when the average response time is greater than a specific value, or when too many errors occur.
413
413
414
-
You can specify these criteria in the test configuration YAML file:
414
+
When you run a load test as part of a CI/CD pipeline, the status of the pipeline run will reflect the status of the load test. This allows you to quickly identify performance regressions, or degraded application behavior when the application is experiencing high load.
415
415
416
-
1. Edit the *SampleApp.yml* file in your GitHub repository.
416
+
In this section, you'll configure test fail criteria based on the average response time and the error rate.
417
+
418
+
You can specify load test fail criteria for Azure Load Testing in the test configuration YAML file. Learn more about [configuring load test fail criteria](./how-to-define-test-criteria.md).
419
+
420
+
1. Edit the *SampleApp.yml* file in your fork of the sample application GitHub repository.
417
421
418
422
1. Add the following snippet at the end of the file:
419
423
@@ -430,40 +434,37 @@ You can specify these criteria in the test configuration YAML file:
430
434
431
435
1. Commit and push the changes to the main branch of the repository.
432
436
433
-
The changes will trigger the Azure Pipelines CI/CD workflow.
437
+
The changes will trigger the CI/CD workflow.
434
438
435
-
1. On the page for pipeline runs, select the most recent entry from the list.
439
+
1. After the test finishes, notice that the CI/CD pipeline run has failed.
436
440
437
-
After the load test finishes, you'll notice that the pipeline failed because the average response time was higher than the number that you specified in the pass/fail criteria.
441
+
In the CI/CD output log, you find that the test failed because one of the fail criteria was met. The load test average response time was higher than the value that you specified in the pass/fail criteria.
438
442
439
443
:::image type="content" source="./media/tutorial-cicd-azure-pipelines/test-criteria-failed.png" alt-text="Screenshot that shows pipeline logs after failed test criteria.":::
440
444
441
445
The Azure Load Testing service evaluates the criteria during the test run. If any of these conditions fails, Azure Load Testing service returns a nonzero exit code. This code informs the CI/CD workflow that the test has failed.
442
446
443
-
1. Edit the *SampleApp.yml* file and change the test's pass/fail criteria:
447
+
1. Edit the *SampleApp.yml* file and change the test's pass/fail criteria to increase the criterion for average response time:
444
448
445
449
```yaml
446
450
failureCriteria:
447
451
- avg(response_time_ms) > 5000
448
452
- percentage(error) > 20
449
453
```
450
454
451
-
1. Commit the changes to trigger the Azure Pipelines CI/CD workflow.
452
-
453
-
:::image type="content" source="./media/tutorial-cicd-azure-pipelines/test-criteria-passed.png" alt-text="Screenshot that shows pipeline logs after all test criteria pass.":::
455
+
1. Commit the changes to trigger the CI/CD workflow again.
454
456
455
-
The load test now succeeds and the pipeline finishes successfully.
457
+
After the test finishes, you notice that the load test and the CI/CD workflow run complete successfully.
You've now created an Azure Pipelines CI/CD workflow that uses Azure Load Testing for automatically running load tests. By using pass/fail criteria, you can set the status of the CI/CD workflow. With parameters, you can make the running of load tests configurable.
465
+
You've now created a CI/CD workflow that uses Azure Load Testing to automate running load tests. By using load test fail criteria, you can set the status of the CI/CD workflow and quickly identify performance and application behavior degradations.
464
466
465
-
* Learn more about [comparing results from multiple test runs](./how-to-compare-multiple-test-runs.md).
466
-
* Learn more about the [Azure Load Testing task](/azure/devops/pipelines/tasks/test/azure-load-testing).
467
-
* Learn more about [Parameterizing a load test](./how-to-parameterize-load-tests.md).
468
-
* Learn more [Define test pass/fail criteria](./how-to-define-test-criteria.md).
469
467
* Learn more about [Configuring server-side monitoring](./how-to-monitor-server-side-metrics.md).
468
+
* Learn more about [Comparing results across multiple test runs](./how-to-compare-multiple-test-runs.md).
469
+
* Learn more about [Parameterizing a load test](./how-to-parameterize-load-tests.md).
470
+
* Learn more about [Defining test pass/fail criteria](./how-to-define-test-criteria.md).
0 commit comments