Testing your changes

There are several groups of tests you can run on the Redash code. Any code that is submitted via a PR will need to pass all of them before it is merged into our master branch.

• Lint tests - very quick testing of the layout and structure of your code. Helps to ensure consistency through the code base
• Unit tests - Fairly quick testing of functions in the code. Helps catch problems without needing to run the time consuming full End-to-End (E2E) tests
• E2E tests - This runs the entire application locally in docker containers, doing all the needed (test) setup, then runs a fairly comprehensive test suite on the whole thing. Can take about 1/2 an hour to run, depending on the hardware its running on

The Lint and Unit tests both have backend (Python) and front end (javascript / typescript) tests available. The E2E tests only has one option, as it tests both the front and back end at the same time.

Quick testing

It's fairly quick and easy to run both the lint and unit tests together.

First, make sure you've set up a virtual environment (as per this page), then run make test:

$ make test
./bin/flake8_tests.sh
6.0.0 (mccabe: 0.7.0, pycodestyle: 2.10.0, pyflakes: 3.0.1) CPython 3.8.10 on Linux
pip 20.0.2 from /home/jc/redashvenv2/lib/python3.8/site-packages/pip (python 3.8)
0
[lots of extra stuff here]

The Python lint tests (flake) generally show a lot of very verbose output. We should probably fix most of them some day... 😁

In that same make test run, after the flake8 Python linting it will run the Python backend unit tests:

========================= test session starts =========================
platform linux -- Python 3.8.17, pytest-5.2.1, py-1.11.0, pluggy-0.13.1
rootdir: /app, inifile: pytest.ini
plugins: cov-2.8.1, anyio-3.7.0
collected 736 items

tests/test_authentication.py ...................................
[etc]
============ 736 passed, 307 warnings in 111.48s (0:01:51) ============

At the end of those backend unit tests, it should show many tests passed and probably a lot of warnings. There should be no errors.

Next, it will then automatically run the frontend (javascript / typescript) unit tests:

yarn test
yarn run v1.22.19
$ run-s type-check jest
$ tsc --noEmit --project client/tsconfig.json
$ TZ=Africa/Khartoum jest
 PASS  client/app/components/ApplicationArea/ErrorMessage.test.js
 PASS  client/app/services/auth.test.js
[etc]
 PASS  client/app/pages/users/components/ReadOnlyUserProfile.test.js
 PASS  client/app/components/queries/ScheduleDialog.test.js

Test Suites: 13 passed, 13 total
Tests:       1 skipped, 77 passed, 78 total
Snapshots:   12 passed, 12 total
Time:        3.028s
Ran all test suites.
Done in 6.23s.

Again, this will likely show several things passing, and possible some warnings. There should be no errors though.

Comprehensive E2E testing (Cypress)

We use Cypress version 5.3.0 (very old) for End to End (E2E) testing. It mostly works ok, though can take a bit of time to get used to.

When we previously ran the yarn command during the earlier setup process, it would have installed the cli based Cypress pieces but not the Cypress GUI. The Cypress GUI can be useful in development, so lets install that now:

$ ./node_modules/cypress/bin/cypress install

Before we start the Cypress GUI we need to start Redash in the background with the test database:

$ make up test_db

Note that occasionally the test_db step there will fail with a PostgreSQL error of some sort (varies which exactly). eg:

docker-compose exec postgres sh -c 'psql -U postgres -c "drop database if exists tests;" && psql -U postgres -c "create database tests;"'
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
make: *** [Makefile:11: test_db] Error 2

When that happens, just run make test_db again, and this attempt should work fine.

With that done, we can launch the Cypress GUI:

$ ./node_modules/cypress/bin/cypress open

It should look like this: