Testing

Testing

webKnossos support both unit tests and integration/end-to-end (E2E) tests.

Requirements

• Google Chrome with webdriver support (v45+)
• node v0.12+
• java for Selenium

Running Tests

npm test  # For frontend unit tests
sbt test  # For E2E tests only (using a dedicated test DB)

E2E can also be run on directly, as long as a wK instance is running in background

npm run test-e2e  # For E2E tests

Jasmine

Both unit/E2E tests are run using the Jasmine test runner and are written in ES6.

describe("A suite", function() {
  it("contains spec with an expectation", function() {
    expect(true).toBe(true);
  });
});

Unit Tests

All unit tests live in the app/assets/javascripts/test directory and end with .spec.js. They are run by jasmine-node inside the node environment.

Here's is a commented unit test skeleton:

// mock-require is used to mock require modules
import mockRequire from "mock-require";
// sinon is used for function mocks
import sinon from "sinon";
// run-async is a helper to run code (e.g. assertions) asynchronously
import runAsync from "../../helpers/run-async";

mockRequire("../../../some/unit/dependency", {
  // Construct mock to be passed to the unit under test...
});

import Unit from "../../../unit/under/test";

describe("Unit", function() {

  it("should call x.foo() asynchronously", function(done) {

    const x = {foo : sinon.stub()};
    Unit.bar(x);

    runAsync([
      => {
        expect(x.foo.called).toBe(true);
        done();
      }
    ]);
  })
})

End-to-End (E2E) tests

All E2E tests live in the app/assets/javascripts/test directory and end with .e2e.js. These tests are executed against a running server with a testing database. E2E tests may use ava's snapshot testing functionality, which has an interface similar to snapshot(key, value). If no value was saved for a key, the given value is saved. Otherwise, the given value is compared to the saved value and the test passes if the values are equal.

For example, the test/enzyme folder contains tests which render react components including data from the server, while using snapshots.

Run E2E Tests

docker-compose up e2e-tests

Handling failed snapshot tests

Check the diff which ava shows. The snapshot change is likely due to code change which may be either right or wrong. If it is wrong, fix the corresponding code. Otherwise, update the snapshots by removing them (yarn remove-snapshots) and rerunning the e2e tests.

Troubleshooting

The tests fail.

Ensure that ...

• ... you have not another server and/or mongo instance running.
• ... dependencies are installed (yarn install) and the JS files are transpiled (yarn test-prepare-watch).
• ... that docker is executed with the necessary rights (sudo might be necessary)

Docker messed up the file permissions

sudo chown -R $(whoami) webknossosRepository

Test DB

For the E2E tests, the DB dump stored in testdb/ is imported into the oxalis-testing DB. This happens automatically.

In order to change the test db, perform the following steps:

# Drop the play-oxalis db
tools/dropDB.sh

# Import the testdb into play-oxalis
tools/import_export/import.sh play-oxalis testdb

# Do appropriate changes to the db...

# Export play-oxalis, overwriting the existing testdb dump
tools/import_export/export.sh play-oxalis testdb

WebdriverIO

Link to API.

WebdriverIO is a wedriver/selenium abstraction that allows for easy access to DOM elements. See Helpers section below for advice. Running webdriverio test for the first time should download the selenium_standalone_server.jarthat is need to actually control any browser.

Webdriver is configured with the wdio.conf.js file. Almost all WebdriverIO's command are async and can be chained together nicely

browser
  .click("#mybutton")
  .getText("#mybutton")
  .then(function(text) {
    console.log(text);
  });

Helpers

PageObject Pattern

All major wK pages (e.g. Dashboard, Users List, etc) should be wrapped in Page Class that has convenience methods for navigating to the page and access certain buttons / DOM elements.

class Dashboard {


  get() {
    browser.url("/dashboard");
  }

  clickButton() {
     return browser
       .waitForExists("#mybutton")
       .click("#mybutton");
  }
}

All webdriverio calls to elements should make sure that the element is already rendered / available in the DOM by waiting for it.

Debugging Tests

There is several things you can do:

• Edit wdio.conf.js to enable debugging and verbose output:

  • Set debug: true to enable Node.js debugging for debugging the actual test logic with the node-debugger.
  • Set logLevel: 'verbose' to get detailed information about the Selenium selectors and results. Which buttons were pressed, what values were retrieved and which selector is invalid...
  • Set { jasmineNodeOpts: defaultTimeoutInterval: 1000000 } to increase the Jasmine timeouts. (Useful for inspecting a side with the Chrome DevTools.
  • Set waitforTimeout: 2500000 to increase the timeout for wdio's waitFor* commands.

• You can use console.log() in all your tests. The output is visible in the CLI.

• You can interrupt the test and go into the browser to use the Chrome DevTools by calling browser.debug(); in your tests.

• To visually confirm problems try browser.pause(intervalInMS); or browser.saveScreenshot(foo.png);.

• Read the WebriverIO Debug Page

• Ask on the WebdriverIO gitter.

CI

All tests are run by our CI on each commit. More details are on the respective wiki page.