Graduated Testing & Test Numbers

Our goal is to employ graduated testing to optimize our use of the machine resources at our disposal. This is accomplished by tagging the tests by level, and choosing to run the 'sanity' level nightly, and the 'sanity' + 'extended' level against the release builds. Each test target may contain many (1000's of) tests. Some targets are rerun with variations of command-line options, in those cases, the target is not counted multiple times, but the number of tests within a target are counted again, as the new input create a unique test case (a different code path is tested).

sanity = nightly
sanity + extended = per release

Test group	# of sanity targets	# of sanity tests	# of extended targets	# of extended tests	Notes
openjdk
system
external
perf
functional					not yet enabled (running at openj9)