Skip to content

Commit 925d081

Browse files
JoeAldingerJoeAldinger
authored
Merge pull request #69462 from slovern/TELCODOCS-1513
/lgtm TELCODOCS-1513 - CNF-7583 - CNF tests - split dependencies
2 parents caf7b02 + 2007d47 commit 925d081

11 files changed

+46
-116
lines changed

‎modules/cnf-latency-tests-discovery-mode.adoc

Lines changed: 0 additions & 24 deletions
This file was deleted.

‎modules/cnf-measuring-latency.adoc

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,12 +48,9 @@ The tests introduce the following environment variables:
4848

4949
|`MAXIMUM_LATENCY`
5050
|Unified variable that specifies the maximum acceptable latency in microseconds. Applicable for all available latency tools.
51-
52-
|`LATENCY_TEST_RUN`
53-
|Boolean parameter that indicates whether the tests should run. `LATENCY_TEST_RUN` is set to `false` by default. To run the latency tests, set this value to `true`.
5451
|====
5552

5653
[NOTE]
5754
====
5855
Variables that are specific to a latency tool take precedence over unified variables. For example, if `OSLAT_MAXIMUM_LATENCY` is set to 30 microseconds and `MAXIMUM_LATENCY` is set to 10 microseconds, the `oslat` test will run with maximum acceptable latency of 30 microseconds.
59-
====
56+
====

‎modules/cnf-performing-end-to-end-tests-disconnected-mode.adoc

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -37,10 +37,11 @@ where:
3737
+
3838
[source,terminal,subs="attributes+"]
3939
----
40-
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
41-
-e DISCOVERY_MODE=true -e FEATURES=performance -e IMAGE_REGISTRY="<disconnected_registry>" \
40+
podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
41+
-e IMAGE_REGISTRY="<disconnected_registry>" \
4242
-e CNF_TESTS_IMAGE="cnf-tests-rhel8:v{product-version}" \
43-
/usr/bin/test-run.sh -ginkgo.focus="\[performance\]\ Latency\ Test"
43+
-e LATENCY_TEST_RUNTIME=<time_in_seconds> \
44+
<disconnected_registry>/cnf-tests-rhel8:v{product-version} /usr/bin/test-run.sh --ginkgo.v
4445
----
4546

4647
[discrete]
@@ -56,8 +57,8 @@ You can run the latency tests using a custom test image and image registry using
5657
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
5758
-e IMAGE_REGISTRY="<custom_image_registry>" \
5859
-e CNF_TESTS_IMAGE="<custom_cnf-tests_image>" \
59-
-e FEATURES=performance \
60-
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} /usr/bin/test-run.sh
60+
-e LATENCY_TEST_RUNTIME=<time_in_seconds> \
61+
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} /usr/bin/test-run.sh --ginkgo.v
6162
----
6263
+
6364
where:
@@ -142,8 +143,8 @@ registry.redhat.io/openshift4/cnf-tests-rhel8:{product-version} \
142143
[source,terminal,subs="attributes+"]
143144
----
144145
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
145-
-e DISCOVERY_MODE=true -e FEATURES=performance -e IMAGE_REGISTRY=image-registry.openshift-image-registry.svc:5000/cnftests \
146-
cnf-tests-local:latest /usr/bin/test-run.sh -ginkgo.focus="\[performance\]\ Latency\ Test"
146+
-e LATENCY_TEST_RUNTIME=<time_in_seconds> \
147+
-e IMAGE_REGISTRY=image-registry.openshift-image-registry.svc:5000/cnftests cnf-tests-local:latest /usr/bin/test-run.sh --ginkgo.v
147148
----
148149

149150
[discrete]

‎modules/cnf-performing-end-to-end-tests-junit-test-output.adoc

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -18,17 +18,20 @@ Use the following procedures to generate a JUnit latency test output and test fa
1818

1919
* Create a JUnit-compliant XML report by passing the `--junit` parameter together with the path to where the report is dumped:
2020
+
21+
[NOTE]
22+
====
23+
You must create the `junit` folder before running this command.
24+
====
25+
+
2126
[source,terminal,subs="attributes+"]
2227
----
23-
$ podman run -v $(pwd)/:/kubeconfig:Z -v $(pwd)/junitdest:<junit_folder_path> \
24-
-e KUBECONFIG=/kubeconfig/kubeconfig -e DISCOVERY_MODE=true -e FEATURES=performance \
25-
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
26-
/usr/bin/test-run.sh --junit <junit_folder_path> \
27-
-ginkgo.focus="\[performance\]\ Latency\ Test"
28+
$ podman run -v $(pwd)/:/kubeconfig:Z -v $(pwd)/junit:/junit \
29+
-e KUBECONFIG=/kubeconfig/kubeconfig registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
30+
/usr/bin/test-run.sh --ginkgo.junit-report junit/<file-name>.xml --ginkgo.v
2831
----
2932
+
3033
where:
3134
+
3235
--
33-
<junit_folder_path> :: Is the path to the folder where the junit report is generated
34-
--
36+
`junit` :: Is the folder where the junit report is stored.
37+
--

‎modules/cnf-performing-end-to-end-tests-running-cyclictest.adoc

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,6 @@
88

99
The `cyclictest` tool measures the real-time kernel scheduler latency on the specified CPUs.
1010

11-
[IMPORTANT]
12-
====
13-
**Always** run the latency tests with `DISCOVERY_MODE=true` set. If you don't, the test suite will make changes to the running cluster configuration.
14-
====
15-
1611
[NOTE]
1712
====
1813
When executing `podman` commands as a non-root or non-privileged user, mounting paths can fail with `permission denied` errors. To make the `podman` command work, append `:Z` to the volumes creation; for example, `-v $(pwd)/:/kubeconfig:Z`. This allows `podman` to do the proper SELinux relabeling.
@@ -33,10 +28,9 @@ When executing `podman` commands as a non-root or non-privileged user, mounting
3328
[source,terminal,subs="attributes+"]
3429
----
3530
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
36-
-e LATENCY_TEST_RUN=true -e DISCOVERY_MODE=true -e FEATURES=performance -e ROLE_WORKER_CNF=worker-cnf \
3731
-e LATENCY_TEST_CPUS=10 -e LATENCY_TEST_RUNTIME=600 -e MAXIMUM_LATENCY=20 \
3832
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
39-
/usr/bin/test-run.sh -ginkgo.v -ginkgo.focus="cyclictest"
33+
/usr/bin/test-run.sh --ginkgo.focus="cyclictest" --ginkgo.v
4034
----
4135
+
4236
The command runs the `cyclictest` tool for 10 minutes (600 seconds). The test runs successfully when the maximum observed latency is lower than `MAXIMUM_LATENCY` (in this example, 20 μs). Latency spikes of 20 μs and above are generally not acceptable for {rds} workloads.
@@ -56,7 +50,7 @@ I0908 13:01:59.193776 27 request.go:601] Waited for 1.046228824s due to cli
5650
Running Suite: CNF Features e2e integration tests
5751
=================================================
5852
Random Seed: 1662642118
59-
Will run 1 of 194 specs
53+
Will run 1 of 3 specs
6054
6155
[...]
6256

@@ -66,7 +60,7 @@ Summarizing 1 Failure:
6660
/remote-source/app/vendor/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/4_latency/latency.go:220
6761

6862
Ran 1 of 194 Specs in 161.151 seconds
69-
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 193 Skipped
63+
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 2 Skipped
7064
--- FAIL: TestTest (161.48s)
7165
FAIL
7266
----

‎modules/cnf-performing-end-to-end-tests-running-hwlatdetect.adoc

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,6 @@
88

99
The `hwlatdetect` tool is available in the `rt-kernel` package with a regular subscription of {op-system-base-full} {op-system-version}.
1010

11-
[IMPORTANT]
12-
====
13-
**Always** run the latency tests with `DISCOVERY_MODE=true` set. If you don't, the test suite will make changes to the running cluster configuration.
14-
====
15-
1611
[NOTE]
1712
====
1813
When executing `podman` commands as a non-root or non-privileged user, mounting paths can fail with `permission denied` errors. To make the `podman` command work, append `:Z` to the volumes creation; for example, `-v $(pwd)/:/kubeconfig:Z`. This allows `podman` to do the proper SELinux relabeling.
@@ -31,10 +26,9 @@ When executing `podman` commands as a non-root or non-privileged user, mounting
3126
[source,terminal,subs="attributes+"]
3227
----
3328
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
34-
-e LATENCY_TEST_RUN=true -e DISCOVERY_MODE=true -e FEATURES=performance -e ROLE_WORKER_CNF=worker-cnf \
3529
-e LATENCY_TEST_RUNTIME=600 -e MAXIMUM_LATENCY=20 \
3630
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
37-
/usr/bin/test-run.sh -ginkgo.v -ginkgo.focus="hwlatdetect"
31+
/usr/bin/test-run.sh --ginkgo.focus="hwlatdetect" --ginkgo.v
3832
----
3933
+
4034
The `hwlatdetect` test runs for 10 minutes (600 seconds). The test runs successfully when the maximum observed latency is lower than `MAXIMUM_LATENCY` (20 μs).
@@ -54,7 +48,7 @@ I0908 15:25:20.023712 27 request.go:601] Waited for 1.046586367s due to cli
5448
Running Suite: CNF Features e2e integration tests
5549
=================================================
5650
Random Seed: 1662650718
57-
Will run 1 of 194 specs
51+
Will run 1 of 3 specs
5852
5953
[...]
6054

@@ -104,7 +98,7 @@ Summarizing 1 Failure:
10498
/remote-source/app/vendor/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/4_latency/latency.go:476
10599

106100
Ran 1 of 194 Specs in 365.797 seconds
107-
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 193 Skipped
101+
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 2 Skipped
108102
--- FAIL: TestTest (366.08s)
109103
FAIL
110104
----

‎modules/cnf-performing-end-to-end-tests-running-in-single-node-cluster.adoc

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,6 @@
88

99
You can run latency tests on {sno} clusters.
1010

11-
[IMPORTANT]
12-
====
13-
**Always** run the latency tests with `DISCOVERY_MODE=true` set. If you don't, the test suite will make changes to the running cluster configuration.
14-
====
15-
1611
[NOTE]
1712
====
1813
When executing `podman` commands as a non-root or non-privileged user, mounting paths can fail with `permission denied` errors. To make the `podman` command work, append `:Z` to the volumes creation; for example, `-v $(pwd)/:/kubeconfig:Z`. This allows `podman` to do the proper SELinux relabeling.
@@ -24,21 +19,24 @@ When executing `podman` commands as a non-root or non-privileged user, mounting
2419
2520
* You have logged in as a user with `cluster-admin` privileges.
2621
22+
* You have applied a cluster performance profile by using the Node Tuning Operator.
23+
2724
.Procedure
2825

2926
* To run the latency tests on a {sno} cluster, run the following command:
3027
+
3128
[source,terminal,subs="attributes+"]
3229
----
3330
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
34-
-e DISCOVERY_MODE=true -e FEATURES=performance -e ROLE_WORKER_CNF=master \
35-
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
36-
/usr/bin/test-run.sh -ginkgo.focus="\[performance\]\ Latency\ Test"
31+
-e LATENCY_TEST_RUNTIME=<time_in_seconds> registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
32+
/usr/bin/test-run.sh --ginkgo.v
3733
----
3834
+
3935
[NOTE]
4036
====
41-
`ROLE_WORKER_CNF=master` is required because master is the only machine pool to which the node belongs. For more information about setting the required `MachineConfigPool` for the latency tests, see "Prerequisites for running latency tests".
37+
The default runtime for each test is 300 seconds.
38+
For valid latency test results, run the tests for at least 12 hours by updating the `LATENCY_TEST_RUNTIME` variable.
39+
To run the buckets latency validation step, you must specify a maximum latency. For details on maximum latency variables, see the table in the "Measuring latency" section.
4240
====
4341
+
4442
After running the test suite, all the dangling resources are cleaned up.

‎modules/cnf-performing-end-to-end-tests-running-oslat.adoc

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,6 @@
88

99
The `oslat` test simulates a CPU-intensive DPDK application and measures all the interruptions and disruptions to test how the cluster handles CPU heavy data processing.
1010

11-
[IMPORTANT]
12-
====
13-
**Always** run the latency tests with `DISCOVERY_MODE=true` set. If you don't, the test suite will make changes to the running cluster configuration.
14-
====
15-
1611
[NOTE]
1712
====
1813
When executing `podman` commands as a non-root or non-privileged user, mounting paths can fail with `permission denied` errors. To make the `podman` command work, append `:Z` to the volumes creation; for example, `-v $(pwd)/:/kubeconfig:Z`. This allows `podman` to do the proper SELinux relabeling.
@@ -30,13 +25,12 @@ When executing `podman` commands as a non-root or non-privileged user, mounting
3025
[source,terminal,subs="attributes+"]
3126
----
3227
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
33-
-e LATENCY_TEST_RUN=true -e DISCOVERY_MODE=true -e FEATURES=performance -e ROLE_WORKER_CNF=worker-cnf \
3428
-e LATENCY_TEST_CPUS=10 -e LATENCY_TEST_RUNTIME=600 -e MAXIMUM_LATENCY=20 \
3529
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
36-
/usr/bin/test-run.sh -ginkgo.v -ginkgo.focus="oslat"
30+
/usr/bin/test-run.sh --ginkgo.focus="oslat" --ginkgo.v
3731
----
3832
+
39-
`LATENCY_TEST_CPUS` specifies the list of CPUs to test with the `oslat` command.
33+
`LATENCY_TEST_CPUS` specifies the number of CPUs to test with the `oslat` command.
4034
+
4135
The command runs the `oslat` tool for 10 minutes (600 seconds). The test runs successfully when the maximum observed latency is lower than `MAXIMUM_LATENCY` (20 μs).
4236
+
@@ -55,7 +49,7 @@ I0908 12:51:55.999393 27 request.go:601] Waited for 1.044848101s due to cli
5549
Running Suite: CNF Features e2e integration tests
5650
=================================================
5751
Random Seed: 1662641514
58-
Will run 1 of 194 specs
52+
Will run 1 of 3 specs
5953
6054
[...]
6155

@@ -77,7 +71,7 @@ Summarizing 1 Failure:
7771
/remote-source/app/vendor/github.com/openshift/cluster-node-tuning-operator/test/e2e/performanceprofile/functests/4_latency/latency.go:177
7872

7973
Ran 1 of 194 Specs in 161.091 seconds
80-
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 193 Skipped
74+
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 2 Skipped
8175
--- FAIL: TestTest (161.42s)
8276
FAIL
8377
----

‎modules/cnf-performing-end-to-end-tests-running-the-tests.adoc

Lines changed: 8 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,6 @@
88

99
Run the cluster latency tests to validate node tuning for your Cloud-native Network Functions (CNF) workload.
1010

11-
[IMPORTANT]
12-
====
13-
**Always** run the latency tests with `DISCOVERY_MODE=true` set. If you don't, the test suite will make changes to the running cluster configuration.
14-
====
15-
1611
[NOTE]
1712
====
1813
When executing `podman` commands as a non-root or non-privileged user, mounting paths can fail with `permission denied` errors. To make the `podman` command work, append `:Z` to the volumes creation; for example, `-v $(pwd)/:/kubeconfig:Z`. This allows `podman` to do the proper SELinux relabeling.
@@ -29,31 +24,18 @@ You provide the test image with a `kubeconfig` file in current directory and its
2924
[source,terminal,subs="attributes+"]
3025
----
3126
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
32-
-e LATENCY_TEST_RUN=true -e DISCOVERY_MODE=true -e FEATURES=performance registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
33-
/usr/bin/test-run.sh -ginkgo.focus="\[performance\]\ Latency\ Test"
27+
-e LATENCY_TEST_RUNTIME=<time_in_seconds>\
28+
-e MAXIMUM_LATENCY=<time_in_microseconds> \
29+
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} /usr/bin/test-run.sh \
30+
--ginkgo.v
3431
----
3532

36-
. Optional: Append `-ginkgo.dryRun` to run the latency tests in dry-run mode. This is useful for checking what the tests run.
37-
38-
. Optional: Append `-ginkgo.v` to run the tests with increased verbosity.
33+
. Optional: Append `--ginkgo.dryRun` flag to run the latency tests in dry-run mode. This is useful for checking what commands the tests run.
3934

40-
. Optional: To run the latency tests against a specific performance profile, run the following command, substituting appropriate values:
41-
+
42-
[source,terminal,subs="attributes+"]
43-
----
44-
$ podman run -v $(pwd)/:/kubeconfig:Z -e KUBECONFIG=/kubeconfig/kubeconfig \
45-
-e LATENCY_TEST_RUN=true -e FEATURES=performance -e LATENCY_TEST_RUNTIME=600 -e MAXIMUM_LATENCY=20 \
46-
-e PERF_TEST_PROFILE=<performance_profile> registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
47-
/usr/bin/test-run.sh -ginkgo.focus="[performance]\ Latency\ Test"
48-
----
49-
+
50-
where:
51-
+
52-
--
53-
<performance_profile> :: Is the name of the performance profile you want to run the latency tests against.
54-
--
35+
. Optional: Append `--ginkgo.v` flag to run the tests with increased verbosity.
5536
+
5637
[IMPORTANT]
5738
====
58-
For valid latency test results, run the tests for at least 12 hours.
39+
The default runtime for each test is 300 seconds.
40+
For valid latency test results, run the tests for at least 12 hours by updating the `LATENCY_TEST_RUNTIME` variable.
5941
====

‎modules/cnf-performing-end-to-end-tests-test-failure-report.adoc

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,8 @@ Use the following procedures to generate a JUnit latency test output and test fa
2121
[source,terminal,subs="attributes+"]
2222
----
2323
$ podman run -v $(pwd)/:/kubeconfig:Z -v $(pwd)/reportdest:<report_folder_path> \
24-
-e KUBECONFIG=/kubeconfig/kubeconfig -e DISCOVERY_MODE=true -e FEATURES=performance \
25-
registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
26-
/usr/bin/test-run.sh --report <report_folder_path> \
27-
-ginkgo.focus="\[performance\]\ Latency\ Test"
24+
-e KUBECONFIG=/kubeconfig/kubeconfig registry.redhat.io/openshift4/cnf-tests-rhel8:v{product-version} \
25+
/usr/bin/test-run.sh --report <report_folder_path> --ginkgo.v
2826
----
2927
+
3028
where:

0 commit comments

Comments
 (0)