Skip to content

Commit 174f9e1

Browse files
author
Kenny Scharm
authored
Updated e2e node test docs (#5148)
* Updated e2e node test docs to include more information about focus, skip, and testgrid * Fix spelling error * Address requested changes
1 parent aa70504 commit 174f9e1

File tree

1 file changed

+63
-25
lines changed

1 file changed

+63
-25
lines changed

contributors/devel/sig-node/e2e-node-tests.md

Lines changed: 63 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,6 @@
1-
# Node End-To-End tests
1+
# Node End-To-End (e2e) tests
22

3-
Node e2e tests are component tests meant for testing the Kubelet code on a custom host environment.
4-
5-
Tests can be run either locally or against a host running on GCE.
6-
7-
Node e2e tests are run as both pre- and post- submit tests by the Kubernetes project.
3+
Node e2e tests are component tests meant for testing the Kubelet code on a custom host environment. Tests can be run either locally or against a host running on Google Compute Engine (GCE). Node e2e tests are run as both pre- and post- submit tests by the Kubernetes project.
84

95
*Note: Linux only. Mac and Windows unsupported.*
106

@@ -14,13 +10,13 @@ Node e2e tests are run as both pre- and post- submit tests by the Kubernetes pro
1410

1511
## Locally
1612

17-
Why run tests *Locally*? Much faster than running tests Remotely.
13+
Why run tests *locally*? It is much faster than running tests remotely.
1814

1915
Prerequisites:
20-
- [Install etcd](https://github.com/coreos/etcd/releases) on your PATH
16+
- [Install etcd](https://github.com/coreos/etcd/releases) and include the path to the installation in your PATH
2117
- Verify etcd is installed correctly by running `which etcd`
2218
- Or make etcd binary available and executable at `/tmp/etcd`
23-
- [Install ginkgo](https://github.com/onsi/ginkgo) on your PATH
19+
- [Install ginkgo](https://github.com/onsi/ginkgo) and include the path to the installation in your PATH
2420
- Verify ginkgo is installed correctly by running `which ginkgo`
2521

2622
From the Kubernetes base directory, run:
@@ -29,7 +25,7 @@ From the Kubernetes base directory, run:
2925
make test-e2e-node
3026
```
3127

32-
This will: run the *ginkgo* binary against the subdirectory *test/e2e_node*, which will in turn:
28+
This will run the *ginkgo* binary against the subdirectory *test/e2e_node*, which will in turn:
3329
- Ask for sudo access (needed for running some of the processes)
3430
- Build the Kubernetes source code
3531
- Pre-pull docker images used by the tests
@@ -48,12 +44,10 @@ make test-e2e-node PRINT_HELP=y
4844

4945
## Remotely
5046

51-
Why Run tests *Remotely*? Tests will be run in a customized pristine environment. Closely mimics what will be done
52-
as pre- and post- submit testing performed by the project.
47+
Why run tests *remotely*? Tests will be run in a customized testing environment. This environment closely mimics the pre- and post- submit testing performed by the project.
5348

5449
Prerequisites:
55-
- [join the googlegroup](https://groups.google.com/forum/#!forum/kubernetes-dev)
56-
50+
- [Join the googlegroup](https://groups.google.com/forum/#!forum/kubernetes-dev) `[email protected]`
5751
- *This provides read access to the node test images.*
5852
- Setup a [Google Cloud Platform](https://cloud.google.com/) account and project with Google Compute Engine enabled
5953
- Install and setup the [gcloud sdk](https://cloud.google.com/sdk/downloads)
@@ -69,7 +63,7 @@ make test-e2e-node REMOTE=true
6963
This will:
7064
- Build the Kubernetes source code
7165
- Create a new GCE instance using the default test image
72-
- Instance will be called something like **test-cos-beta-81-12871-44-0**
66+
- Instance will be named something like **test-cos-beta-81-12871-44-0**
7367
- Lookup the instance public ip address
7468
- Copy a compressed archive file to the host containing the following binaries:
7569
- ginkgo
@@ -86,7 +80,7 @@ This will:
8680
- **Leave the GCE instance running**
8781

8882
**Note: Subsequent tests run using the same image will *reuse the existing host* instead of deleting it and
89-
provisioning a new one. To delete the GCE instance after each test see
83+
provisioning a new one. To delete the GCE instance after each test see
9084
*[DELETE_INSTANCE](#delete-instance-after-tests-run)*.**
9185

9286

@@ -127,7 +121,7 @@ This is useful if you want recreate the instance for each test run to trigger fl
127121
make test-e2e-node REMOTE=true DELETE_INSTANCES=true
128122
```
129123

130-
## Keep instance, test binaries, and *processes* around after tests run
124+
## Keep instance, test binaries, and processes around after tests run
131125

132126
This is useful if you want to manually inspect or debug the kubelet process run as part of the tests.
133127

@@ -155,7 +149,7 @@ test in parallel against different instances of the same image.
155149
make test-e2e-node REMOTE=true INSTANCE_PREFIX="my-prefix"
156150
```
157151

158-
## Run tests using a custom image config
152+
## Run tests using a custom image configuration
159153

160154
This is useful if you want to test out different runtime configurations. First, make a local
161155
(temporary) copy of the base image config from the test-infra repo:
@@ -179,7 +173,27 @@ Finally, run the tests with your custom config:
179173
make test-e2e-node REMOTE=true IMAGE_CONFIG_FILE="<local file>" [...]
180174
```
181175

182-
# Additional Test Options for both Remote and Local execution
176+
Image configuration files can further influence how FOCUS and SKIP match test cases.
177+
178+
For example:
179+
180+
```sh
181+
--focus="\[NodeFeature:.+\]" --skip="\[Flaky\]|\[Serial\]"
182+
```
183+
184+
runs all e2e tests labeled `"[NodeFeature:*]"` and will skip any tests labeled `"[Flaky]"` or `"[Serial]"`.
185+
186+
Two example e2e tests that match this expression are:
187+
* https://github.com/kubernetes/kubernetes/blob/0e2220b4462130ae8a22ed657e8979f7844e22c1/test/e2e_node/security_context_test.go#L155
188+
* https://github.com/kubernetes/kubernetes/blob/0e2220b4462130ae8a22ed657e8979f7844e22c1/test/e2e_node/security_context_test.go#L175
189+
190+
However, image configuration files select test cases based on the `tests` field.
191+
192+
See https://github.com/kubernetes/test-infra/blob/4572dc3bf92e70f572e55e7ac1be643bdf6b2566/jobs/e2e_node/benchmark-config.yaml#L22-23 for an example configuration.
193+
194+
If the [Prow e2e job configuration](https://github.com/kubernetes/test-infra/blob/master/jobs/e2e_node/image-config.yaml) does **not** specify the `tests` field, FOCUS and SKIP will run as expected.
195+
196+
# Additional test options for both remote and local execution
183197

184198
## Only run a subset of the tests
185199

@@ -202,6 +216,26 @@ For example, the [`ci-kubernetes-node-kubelet`](https://github.com/kubernetes/te
202216
make test-e2e-node REMOTE=true FOCUS="\[NodeConformance\]" SKIP="\[Flaky\]|\[Serial\]"
203217
```
204218

219+
See http://onsi.github.io/ginkgo/#focused-specs in the Grinkgo documentation to learn more about how FOCUS and SKIP work.
220+
221+
## Run a single test
222+
223+
To run a particular e2e test, simply pass the Grinkgo `It` string to the `--focus` argument.
224+
225+
For example, suppose we have the following test case: https://github.com/kubernetes/kubernetes/blob/0e2220b4462130ae8a22ed657e8979f7844e22c1/test/e2e_node/security_context_test.go#L175. We could select this test case by adding the argument:
226+
227+
```sh
228+
--focus="should not show its pid in the non-hostpid containers \[NodeFeature:HostAccess\]"
229+
```
230+
231+
## Run all tests related to a feature
232+
233+
In contrast, to run all node e2e tests related to the "HostAccess" feature one could run:
234+
235+
```sh
236+
--focus="\[NodeFeature:HostAccess\]"
237+
```
238+
205239
## Run tests continually until they fail
206240

207241
This is useful if you are trying to debug a flaky test failure. This will cause ginkgo to continually
@@ -253,15 +287,19 @@ For testing with the QoS Cgroup Hierarchy enabled, you can pass --cgroups-per-qo
253287
```sh
254288
make test_e2e_node TEST_ARGS="--cgroups-per-qos=true"
255289
```
290+
## Testgrid
291+
292+
TestGrid (https://testgrid.k8s.io) is a publicly hosted and configured automated testing framework developed by Google.
293+
294+
Here (https://testgrid.k8s.io/sig-node-containerd#containerd-node-features) we see an example of an e2e Prow job running e2e tests. Each gray row in the grid corresponds
295+
to an e2e test or a stage of the job (i.e. created a VM, downloaded some files). Passed tests are colored green and failed tests are colored red.
256296

257297
# Notes on tests run by the Kubernetes project during pre-, post- submit.
258298

259299
The node e2e tests are run by the [Prow](https://prow.k8s.io/) for each Pull Request and the results published
260-
in the status checks box at
261-
the bottom of the Pull Request below all comments. To have prow re-run the node e2e tests against a PR add the comment
262-
`/test pull-kubernetes-node-e2e` and **include a link to the test
263-
failure logs if caused by a flake.**
264-
Note that [commands to prow](https://prow.k8s.io/command-help#test) must be on separate lines from any commentary.
300+
in the status checks box at the bottom of the Pull Request below all comments. To have Prow re-run the node e2e tests against a PR add the comment
301+
`/test pull-kubernetes-node-e2e` and **include a link to the test failure logs if caused by a flake.**
302+
Note that [commands to Prow](https://prow.k8s.io/command-help#test) must be on separate lines from any commentary.
265303

266304
For example,
267305

@@ -270,7 +308,7 @@ For example,
270308

271309
The PR builder runs tests against the images listed in [image-config.yaml](https://github.com/kubernetes/test-infra/blob/master/jobs/e2e_node/image-config.yaml).
272310

273-
Other [node e2e prow jobs](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-node)
311+
Other [node e2e Prow jobs](https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes/sig-node)
274312
run against different images depending on the configuration chosen in the
275313
[test-infra repo](https://github.com/kubernetes/test-infra/tree/master/jobs/e2e_node).
276314
The source code for these tests comes from the [kubernetes/kubernetes repo](https://github.com/kubernetes/kubernetes/tree/master/test/e2e_node).

0 commit comments

Comments
 (0)