UnitTests

= Unit Tests =

PageOutline

---

'''The following is currently under discussion. It's not finalized.'''

Please add comments for improvements and more elegant solutions on anything and try to run the configurations. Lots of tests cannot be properly loaded, or be initialized, they might run forever, or freeze the test process completely. Certainly many tests fail due to site specific settings. And of course many tests just fail.

Our first task is to get the tests running for all sites, or, as second and worse choice, identifying which of them shall be disabled for a specific site and how to do so automatically.

== General == Ideally, CSS code is developed in a test-driven manner. Writing the test first has many advantages:

• Helps to brainstorm what the code should do
• Almost automatically breaks the code into testable and thus modular sections
• Test code serves as a how-to-use documentation for other developers
• Test code serves as a specification, in principle allowing anybody to perform the implementation
• When the test passes, you know at least something "works" as intended

Last not least, you can use the test later to assert that existing functionality is still available. Tests can even be executed as part of an integration build.

== Types of Tests == Technically, there are different types of tests which we identify by their file names:

• Plain JUnit tests: {{{*UnitTest.java}}}
• JUnit Plug-in tests that need the Eclipse runtime but no GUI: {{{*HeadlessTest.java}}}
• JUnit Plug-in tests that need the runtime and the GUI: {{{*UiPluginTest.java}}}
• A demo that requires user input or other interaction. Not a real test, but happens to be implemented with JUnit: {{{*Demo.java}}}

== Test Guidelines == All developers should import all available plugins into their development environment. The reason is that missing a bundle means missing the tests for this bundle and potentially modifying another bundle that is a prerequisite for the missing one such that this bundle is corrupted (tests would fail but they are not executed).

Tests should have the following features:

• Automatically executable AND terminating without any user interaction or user supervision.
• Not taking an exceptional long time, if it is not necessary for the test itself.
  • E.g. check for database connections in the test setup once. Do not wait for connection timeout for any test to figure out that the test eventually fails.
  • Do not use excessive logging (there's no need if no one's watching), concentrate on important test debug info.
• Use a Logger, not the standard console, if possible.

Once a properly functioning test environment has been established, the hg push procedure should look like follows:

• pulling the latest snapshot from the sourceforge repo
• locking the repo that nobody else may push in between corrupting your current snapshot
• merging your changesets with the latest snapshot
• '''run all tests of all bundles'''
  • green - pushing your changesets and unlocking the repo
  • red - unlocking the repo and start debugging the failing tests

Whether such an ideal pushing routine is possible with our number of developers has yet to be found out - push wars might probably occur as the tests run for quite a while. BR Possible ways to easen the trouble of push/commit wars are 'push queues' in which you reserve your place to commit and wait for your turn (or negotiate your place in the queue with somebody else). Or having a DVS after all, any site or group collects their changesets locally before the push, reducing the number of pushes to the sourceforge repo.

== A first setup in {{{org.csstudio.testsuite}}} == There is a plugin org.csstudio.testsuite that implements a testsuite that is configured by different launch configurations.

• !AllTests.launch (any *Test.class) started vs the common css product with a lot of perm gen space
• !HeadlessTests.launch (any *!HeadlessTest.class) started vs the headless application
• !UnitTests.launch (any *!UnitTest.class) started vs the headless application
• !UiPluginTests.launch (any *!UiPluginTest.class) started vs the common css product

Any of these configurations starts a JUnit Plugin test that is supposed to load all classes from all plugins present and analyse them for being a test.

Valid bundle roots are currently:

• org.csstudio
• org.remotercp
• de.desy

Blacklisted bundles (not considered):

• org.csstudio.platform.libs (is supposed to contain loads of classes from 3rd parties)

Blacklisted packages (as they are part of valid bundles):

• !org.apache

Any loaded class is analysed to be either an extension of junit.framework.!TestCase or containing public methods annotated with @org.junit.Test, if so and it is a member of a valid bundle, the class is added to the test suite. If the class is a test, but does not end on {{{*Test.class}}}, a warning is logged, as this class is not added to the testsuite. If the class is not a test, but ends on *Test.class, another warning is logged (often observed for demo applications).

== Test configuration and test data for different sites == Many tests require different configurations and expected data to function properly on different sites. For example, test code that accesses the control system or a database will need PV names, URLs, passwords which are specific to each site.

A first solution is provided by the org.csstudio.platform.test.!TestDataProvider.

A !TestDataProvider example can be found in test classes (very easy to understand in org.csstudio.email.EMailSenderHeadlessTest, a bit more complicated in org.csstudio.utility.ldap.service.impl.LdapServiceImplHeadlessTest). Here, the site specific properties and parameters have been extracted into desyTestConfiguration.ini and snsTestConfiguration.ini (empty) files. Any test class requiring these site specific properties can include the following code: {{{ private static TestDataProvider PROV; static { try { ... PROV = TestDataProvider.getInstance(Activator.ID); } catch ... } }}} Once started the test runs with these properties, making database connections, configuring specific search queries, and comparing specific return values. The properties are just loaded once per bundle, even when this code snippet is put in many test classes of a bundle.

It allows modular test configurations by automatically selecting the site-specific configuration file within the plugin. If you'd like to run the tests without the framework do so, you won't notice, the file is found via the current working dir from where you start your test and if you'd like to extract sensitive data, just reference your sensitive data file from your configuration file with e.g. include this line in xxxTestConfiguration.ini {{{ sensitiveConfigFilePath=relative/path/to/desys/secret/file/xxx.ini ... }}}

A simpler suggestion is in org.csstudio.apputil.test (SNS).

If you'd like to improve the code, please do so and inform all others! This idea is called collective code ownership!

=== How does the plugin including the tests know which of the test configuration files shall be loaded? === Go to the Eclipse menu, Windows->Preferences->Java->Installed JREs. Choose any installed JRE and edit the jvm args to contain:BR {{{-DsiteId=DESY}}} or {{{-DsiteId=SNS}}} or {{{-DsiteId=ITER}}} or {{{-DsiteId=WhatEver}}}

The valid values for this siteId parameter can be found in org.csstudio.platform.test.!SiteKey. There a configuration file prefix is defined for any site:

• DESY : desy => desyTestConfiguration.ini
• SNS : sns => snsTestConfiguration.ini
• WHATEVER : blubb => blubbTestConfiguration.ini

Until now, only three keys (DESY, SNS, ITER) have been added. Please add or modify the list for your needs. A bit tedious with the jvm arg, I just did not find a better and quick way to implement the site switch outside the test code.