UnitTests

Unit Tests

(XXX macro: "PageOutline")

---

The following is currently under discussion. It's not finalized.

Please add comments for improvements and more elegant solutions on anything and try to run the configurations. Lots of tests cannot be properly loaded, or be initialized, they might run forever, or freeze the test process completely. Certainly many tests fail due to site specific settings. And of course many tests just fail.

Our first task is to get the tests running for all sites, or, as second and worse choice, identifying which of them shall be disabled for a specific site and how to do so automatically.

General

Ideally, CSS code is developed in a test-driven manner. Writing the test first has many advantages:

• Helps to brainstorm what the code should do
• Almost automatically breaks the code into testable and thus modular sections
• Test code serves as a how-to-use documentation for other developers
• Test code serves as a specification, in principle allowing anybody to perform the implementation
• When the test passes, you know at least something "works" as intended

Last not least, you can use the test later to assert that existing functionality is still available. Tests can even be executed as part of an integration build.

Types of Tests

Technically, there are different types of tests which we identify by their file names:

• Plain JUnit tests: *[[UnitTest]].java
• JUnit Plug-in tests that need the Eclipse runtime but no GUI: *[[HeadlessTest]].java
• JUnit Plug-in tests that need the runtime and the GUI: *[[UiPluginTest]].java
• A demo that requires user input or other interaction. Not a real test, but happens to be implemented with JUnit: *Demo.java

Test Guidelines

All developers should import all available plugins into their development environment. The reason is that missing a bundle means missing the tests for this bundle and potentially modifying another bundle that is a prerequisite for the missing one such that this bundle is corrupted (tests would fail but they are not executed).

Tests should have the following features:

• Automatically executable AND terminating without any user interaction or user supervision.
• Not taking an exceptional long time, if it is not necessary for the test itself.
  • E.g. check for database connections in the test setup once. Do not wait for connection timeout for any test to figure out that the test eventually fails.
  • Do not use excessive logging (there's no need if no one's watching), concentrate on important test debug info.
• Use a Logger, not the standard console, if possible.

Once a properly functioning test environment has been established, the hg push procedure should look like follows:

• pulling the latest snapshot from the sourceforge repo
• locking the repo that nobody else may push in between corrupting your current snapshot
• merging your changesets with the latest snapshot
• run all tests of all bundles
  • green - pushing your changesets and unlocking the repo
  • red - unlocking the repo and start debugging the failing tests

Whether such an ideal pushing routine is possible with our number of developers has yet to be found out - push wars might probably occur as the tests run for quite a while. (XXX macro: "BR") Possible ways to easen the trouble of push/commit wars are 'push queues' in which you reserve your place to commit and wait for your turn (or negotiate your place in the queue with somebody else). Or having a DVS after all, any site or group collects their changesets locally before the push, reducing the number of pushes to the sourceforge repo.

A first setup in org.csstudio.testsuite

There is a plugin org.csstudio.testsuite that implements a testsuite that is configured by different launch configurations.

• AllTests.launch (any *Test.class) started vs the common css product with a lot of perm gen space
• HeadlessTests.launch (any *HeadlessTest.class) started vs the headless application
• UnitTests.launch (any *UnitTest.class) started vs the headless application
• !UiPluginTests.launch (any *!UiPluginTest.class) started vs the common css product

Any of these configurations starts a JUnit Plugin test that is supposed to load all classes from all plugins present and analyse them for being a test.

Valid bundle roots are currently:

• org.csstudio
• org.remotercp
• de.desy

Blacklisted bundles (not considered):

• org.csstudio.platform.libs (is supposed to contain loads of classes from 3rd parties)

Blacklisted packages (as they are part of valid bundles):

• !org.apache

Any loaded class is analysed to be either an extension of junit.framework.TestCase or containing public methods annotated with @org.junit.Test, if so and it is a member of a valid bundle, the class is added to the test suite. If the class is a test, but does not end on *Test.class, a warning is logged, as this class is not added to the testsuite. If the class is not a test, but ends on *Test.class, another warning is logged (often observed for demo applications).

Test configuration and test data for different sites

Many tests require different configurations and expected data to function properly on different sites. For example, test code that accesses the control system or a database will need PV names, URLs, passwords which are specific to each site.

A first solution is provided by the org.csstudio.platform.test.!TestDataProvider.

A !TestDataProvider example can be found in test classes (very easy to understand in org.csstudio.email.EMailSenderHeadlessTest, a bit more complicated in org.csstudio.utility.ldap.service.impl.LdapServiceImplHeadlessTest). Here, the site specific properties and parameters have been extracted into desyTestConfiguration.ini and snsTestConfiguration.ini (empty) files. Any test class requiring these site specific properties can include the following code:

    private static [[TestDataProvider]] PROV;
    static {
        try { ...
            PROV = [[TestDataProvider]].getInstance(Activator.ID);
        } catch ...
    }

Once started the test runs with these properties, making database connections, configuring specific search queries, and comparing specific return values. And it does not need the bundle framework, a simple 'Run as JUnit' will work as well (without bundle framework, the current working dir is parsed for the test configuration files). The properties are just loaded once per bundle, even when this code snippet is put in many test classes of a bundle.

It allows modular test configurations by automatically selecting the site-specific configuration file within the plugin. If you have sensitive data that you wouldn't like to put into your xxxTestConfiguration.ini, then you can simply do that:

Include this line in xxxTestConfiguration.ini

sensitive[[ConfigFilePath]]=relative/path/to/your/secret/file/xxx.ini
...

and put passwords and secret connections in this xxx.ini. Finally, take care that you don't publish your xxx.ini by mistake. That wouldn't make much sense, would it?

A simpler suggestion is in org.csstudio.apputil.test (SNS).

If you'd like to improve/change the code, please do so and then inform all potential users! This idea is called collective code ownership! It's a little bit crazy.

How does the plugin including the tests know which of the test configuration files shall be loaded?

Go to the Eclipse menu, Windows->Preferences->Java->Installed JREs. Choose any installed JRE and edit the jvm args to contain:(XXX macro: "BR") -[[DsiteId]]=DESY or -[[DsiteId]]=SNS or -[[DsiteId]]=ITER or -[[DsiteId]]=[[WhatEver]]

The valid values for this siteId parameter can be found in org.csstudio.platform.test.SiteKey. There a configuration file prefix is defined for any site:

• DESY : desy => desyTestConfiguration.ini
• SNS : sns => snsTestConfiguration.ini
• WHATEVER : blubb => blubbTestConfiguration.ini

Until now, only three keys (DESY, SNS, ITER) have been added. Please add or modify the list for your needs. A bit tedious with the jvm arg, I just did not find a better and quick way to implement the site switch outside the test code.