|
1 | 1 | # wpt.fyi Webdriver tests |
2 | 2 |
|
3 | | -This directory covers Webdriver tests for the `webapp`, written in a 3rd-party |
4 | | -Golang Webdriver client, [tebeka/selenium](https://github.com/tebeka/selenium). |
| 3 | +This directory containers integration tests for webapp/. These tests bring up |
| 4 | +the full webserver then use a Golang Webdriver client, |
| 5 | +[tebeka/selenium](https://github.com/tebeka/selenium), to load pages in a |
| 6 | +browser (Chrome or Firefox) and assert that the webapp behaves as expected. |
5 | 7 |
|
6 | 8 | To run the tests, from the root `wpt.fyi` directory, run: |
7 | 9 |
|
8 | | - make go_webdriver_test |
| 10 | + make go_large_test |
9 | 11 |
|
10 | | -If you want to actually see the tests in action, disable the frame buffer. |
| 12 | +You can run just one of Chrome or Firefox via: |
11 | 13 |
|
12 | | - make webdriver_deps |
13 | | - go test --frame_buffer=false -tags=large ./webdriver |
| 14 | + make go_chrome_test |
| 15 | + make go_firefox_test |
14 | 16 |
|
15 | | -If you want to use a custom installed location of selenium / browser / driver |
16 | | -binaries, the required flags are shown in [the Makefile](../Makefile) |
17 | | -`go_webdriver_test' rule. |
| 17 | +Note that when running these tests outside of docker (see [Running in |
| 18 | +docker](#running-in-docker)), they will use your locally installed browser and |
| 19 | +webdriver clients, so it is worth making sure they are the versions you expect. |
| 20 | + |
| 21 | +## Debugging |
| 22 | + |
| 23 | +Debugging the webdriver/ tests can be difficult as they are integration tests |
| 24 | +and the problem can occur anywhere from controlling the browser, to the webapp |
| 25 | +frontend, to the backend - and other weird bugs inbetween! This section |
| 26 | +contains some tips on how to effectively debug them. |
| 27 | + |
| 28 | +After running one of the above `make` commands at least once, one can directly |
| 29 | +run the golang tests via: |
| 30 | + |
| 31 | +``` |
| 32 | +# You can copy GECKODRIVER_PATH out of the make output; it should be installed |
| 33 | +# locally under webapp/node_modules/selenium-standalone/... |
| 34 | +go test -v -timeout=15m -tags=large ./webdriver -args \ |
| 35 | + -firefox_path=/usr/bin/firefox \ |
| 36 | + -geckodriver_path=$GECKODRIVER_PATH \ |
| 37 | + -chrome_path=/usr/bin/google-chrome \ |
| 38 | + -chromedriver_path=/usr/bin/chromedriver \ |
| 39 | + -frame_buffer=true \ |
| 40 | + -staging=false \ |
| 41 | + -browser=chrome # Or firefox |
| 42 | +``` |
| 43 | + |
| 44 | +It is worth comparing this command-line against the Makefile, in case this |
| 45 | +documentation becomes out of date. |
| 46 | + |
| 47 | +### Running only one test |
| 48 | + |
| 49 | +If you only need to run one test, you can use the golang test [`-run` |
| 50 | +parameter](https://golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks). |
| 51 | +For example: |
| 52 | + |
| 53 | +``` |
| 54 | +go test -v -timeout=15m -tags=large ./webdriver \ |
| 55 | + -run TestProductParam_Order/Order \ |
| 56 | + -args \ |
| 57 | + -firefox_path=/usr/bin/firefox \ |
| 58 | + -geckodriver_path=$GECKODRIVER_PATH \ |
| 59 | + -chrome_path=/usr/bin/google-chrome \ |
| 60 | + -chromedriver_path=/usr/bin/chromedriver \ |
| 61 | + -frame_buffer=true \ |
| 62 | + -staging=false \ |
| 63 | + -browser=chrome # Or firefox |
| 64 | +``` |
| 65 | + |
| 66 | +### Visual Output |
| 67 | + |
| 68 | +Many of the tests run some javascript (or click on an element, etc) and expect |
| 69 | +to find some resulting change on the page. When that doesn't occur, they |
| 70 | +usually just timeout and it can be difficult to know why. One very useful trick |
| 71 | +is to enable visual output, so that you can actually see the webpage as the |
| 72 | +test runs. |
| 73 | + |
| 74 | +To do this, set the `frame_buffer` argument to `false`, e.g.: |
| 75 | + |
| 76 | +``` |
| 77 | +go test -v -timeout=15m -tags=large ./webdriver -args \ |
| 78 | + -firefox_path=/usr/bin/firefox \ |
| 79 | + -geckodriver_path=$GECKODRIVER_PATH \ |
| 80 | + -chrome_path=/usr/bin/google-chrome \ |
| 81 | + -chromedriver_path=/usr/bin/chromedriver \ |
| 82 | + -frame_buffer=false \ |
| 83 | + -staging=false \ |
| 84 | + -browser=chrome # Or firefox |
| 85 | +``` |
| 86 | + |
| 87 | +### Verbose webdriver output |
| 88 | + |
| 89 | +By default, webdriver output is hidden as it is very noisy. You can re-enable |
| 90 | +it by passing `-debug=true` to the tests, e.g.: |
| 91 | + |
| 92 | +``` |
| 93 | +go test -v -timeout=15m -tags=large ./webdriver -args \ |
| 94 | + -firefox_path=/usr/bin/firefox \ |
| 95 | + -geckodriver_path=$GECKODRIVER_PATH \ |
| 96 | + -chrome_path=/usr/bin/google-chrome \ |
| 97 | + -chromedriver_path=/usr/bin/chromedriver \ |
| 98 | + -frame_buffer=true \ |
| 99 | + -staging=false \ |
| 100 | + -browser=chrome \ |
| 101 | + -debug=true |
| 102 | +``` |
| 103 | + |
| 104 | +Redirecting stderr to stdout and saving it to a log-file is recommended due to |
| 105 | +the verbosity of webdriver logs (append `2>&1 | tee my-log.txt` to the above |
| 106 | +command). |
| 107 | + |
| 108 | +### Running in docker |
| 109 | + |
| 110 | +Sometimes bugs only occur in a docker-like environment. This can be difficult |
| 111 | +to reproduce, but a first step is to run the tests inside of docker. To do |
| 112 | +this, first start the docker container in one terminal tab: |
| 113 | + |
| 114 | +``` |
| 115 | +./util/docker-dev/run.sh |
| 116 | +``` |
| 117 | + |
| 118 | +Then, in another tab, we need to get the instance id of the container, exec |
| 119 | +'bash' inside of it, and run our test: |
| 120 | + |
| 121 | +``` |
| 122 | +docker container ls |
| 123 | +[ note the CONTAINER ID ] |
| 124 | +docker exec -it $CONTAINER_ID bash |
| 125 | +user@abce84dd426d:~/wpt.fyi$ |
| 126 | +[now you can run 'make go_chrome_test', or 'go test ...' directly, etc] |
| 127 | +``` |
| 128 | + |
| 129 | +Note that this maps the host machine's wpt.fyi checkout into docker, so any |
| 130 | +code edits you make on the host are reflected in the container and vice-versa. |
0 commit comments