Various "JUnit Plugin Test" launch configurations bugs

[iloveeclipse]

First issue I've found while looking at eclipse-platform/eclipse.platform.ui#3380 and trying to run org.eclipse.ui.editors.tests.SegmentedModeTest test from IDE.

Precondition: have both Java 17 and 21 installed in IDE (I have much more, but I assume it is enough to have these two).
Import org.eclipse.ui.editors.tests project and all dependent test projects in the workspace, select SegmentedModeTest in the Packages Explorer and say "Debug as -> JUnit Plug-in Test".

The console view shows following printout:

WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.vector
WARNING: JNI local refs: 698, exceeds capacity: 32
WARNING: JNI local refs: 696, exceeds capacity: 33
WARNING: JNI local refs: 696, exceeds capacity: 33
<title>Incompatible JVM</title>Version 17.0.16 of the JVM is not suitable for this product. Version: 21 or greater is required.

and error log shows

eclipse.buildId=4.38.0.I20251007-1800
java.version=25
java.vendor=Eclipse Adoptium
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Command-line arguments:  -os linux -ws gtk -arch x86_64 -data /data/4x_platform_workspace

org.eclipse.jdt.junit
Error
Wed Oct 08 13:44:22 CEST 2025
No tests found with test runner 'JUnit 5'.

The test bundle requires Java 17 but obviously it depends on the launch argument (added from "Running" target platform eclipse.ini) which says -Dosgi.requiredJavaVersion=21.

The original (badly broken) launch config generated looks like:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.pde.ui.JunitLaunchConfig">
    <booleanAttribute key="append.args" value="true"/>
    <booleanAttribute key="askclear" value="false"/>
    <booleanAttribute key="automaticAdd" value="false"/>
    <booleanAttribute key="automaticIncludeRequirements" value="true"/>
    <booleanAttribute key="automaticValidate" value="true"/>
    <stringAttribute key="bootstrap" value=""/>
    <stringAttribute key="checked" value="[NONE]"/>
    <booleanAttribute key="clearConfig" value="true"/>
    <booleanAttribute key="clearws" value="true"/>
    <booleanAttribute key="clearwslog" value="false"/>
    <stringAttribute key="configLocation" value="${workspace_loc}/.metadata/.plugins/org.eclipse.pde.core/pde-junit"/>
    <booleanAttribute key="default" value="false"/>
    <booleanAttribute key="includeOptional" value="true"/>
    <stringAttribute key="location" value="${workspace_loc}/../junit-workspace"/>
    <listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_PATHS">
        <listEntry value="/org.eclipse.ui.editors.tests/src/org/eclipse/ui/editors/tests/SegmentedModeTest.java"/>
    </listAttribute>
    <listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_TYPES">
        <listEntry value="1"/>
    </listAttribute>
    <stringAttribute key="org.eclipse.jdt.junit.CONTAINER" value=""/>
    <booleanAttribute key="org.eclipse.jdt.junit.KEEPRUNNING_ATTR" value="false"/>
    <stringAttribute key="org.eclipse.jdt.junit.TEST_KIND" value="org.eclipse.jdt.junit.loader.junit5"/>
    <stringAttribute key="org.eclipse.jdt.launching.MAIN_TYPE" value="org.eclipse.ui.editors.tests.SegmentedModeTest"/>
    <stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-os ${target.os} -ws ${target.ws} -arch ${target.arch} -nl ${target.nl} -consoleLog"/>
    <stringAttribute key="org.eclipse.jdt.launching.PROJECT_ATTR" value="org.eclipse.ui.editors.tests"/>
    <stringAttribute key="org.eclipse.jdt.launching.SOURCE_PATH_PROVIDER" value="org.eclipse.pde.ui.workbenchClasspathProvider"/>
    <stringAttribute key="org.eclipse.jdt.launching.VM_ARGUMENTS" value="-XX:CompileCommand=quiet -XX:CompileCommand=exclude,org.eclipse.jdt.internal.core.dom.rewrite.ASTRewriteAnalyzer::getExtendedRange -Xms256m -Xcheck:jni -Xmx16G -XX:G1PeriodicGCInterval=10000 -XX:G1PeriodicGCSystemLoadThreshold=3.0 -XX:NativeMemoryTracking=summary -XX:TrimNativeHeapInterval=5000 -Dosgi.requiredJavaVersion=21 -Dosgi.dataAreaRequiresExplicitInit=true -Dorg.eclipse.swt.graphics.Resource.reportNonDisposed=true -Declipse.log.size.max=20000 -Dorg.eclipse.swt.internal.gtk.noThemingFixes -Dorg.eclipse.swt.internal.enableStrictChecks -Declipse.p2.unsignedPolicy=allow -Dorg.eclipse.equinox.p2.director.checkAuthorities=false -DSWT_GTK_INPUT_HINT_NO_EMOJI=true -Dp2.skipRepositoryProtocolCheck=true -Djava.security.manager=allow --add-modules=ALL-SYSTEM -Dorg.slf4j.simpleLogger.defaultLogLevel=off -Dorg.eclipse.swt.internal.gtk.cssFile=/opt/ate/devtools/bin/theme_patches.css"/>
    <stringAttribute key="pde.version" value="3.3"/>
    <stringAttribute key="product" value="org.eclipse.sdk.ide"/>
    <booleanAttribute key="run_in_ui_thread" value="true"/>
    <setAttribute key="selected_workspace_bundles">
        <setEntry value="org.eclipse.ui.editors.tests*3.13.1000.qualifier"/>
    </setAttribute>
    <booleanAttribute key="tracing" value="false"/>
    <booleanAttribute key="useCustomFeatures" value="false"/>
    <booleanAttribute key="useDefaultConfig" value="true"/>
    <booleanAttribute key="useDefaultConfigArea" value="false"/>
    <booleanAttribute key="useProduct" value="true"/>
</launchConfiguration>

Now only open "Debug Configurations" dialog (it will by default open with the selected test launch config), and without changing anything, click on "Close" button.

The launch config changes to

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.pde.ui.JunitLaunchConfig">
    <booleanAttribute key="append.args" value="true"/>
    <booleanAttribute key="askclear" value="false"/>
    <booleanAttribute key="automaticAdd" value="false"/>
    <booleanAttribute key="automaticIncludeRequirements" value="true"/>
    <booleanAttribute key="automaticValidate" value="true"/>
    <stringAttribute key="bootstrap" value=""/>
    <stringAttribute key="checked" value="[NONE]"/>
    <booleanAttribute key="clearConfig" value="true"/>
    <booleanAttribute key="clearws" value="true"/>
    <booleanAttribute key="clearwslog" value="false"/>
    <stringAttribute key="configLocation" value="${workspace_loc}/.metadata/.plugins/org.eclipse.pde.core/pde-junit"/>
    <booleanAttribute key="default" value="false"/>
    <setAttribute key="deselected_workspace_bundles"/>
    <booleanAttribute key="includeOptional" value="true"/>
    <stringAttribute key="location" value="${workspace_loc}/../junit-workspace"/>
    <booleanAttribute key="org.eclipse.debug.core.ATTR_FORCE_SYSTEM_CONSOLE_ENCODING" value="false"/>
    <listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_PATHS">
        <listEntry value="/org.eclipse.ui.editors.tests/src/org/eclipse/ui/editors/tests/SegmentedModeTest.java"/>
    </listAttribute>
    <listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_TYPES">
        <listEntry value="1"/>
    </listAttribute>
    <stringAttribute key="org.eclipse.jdt.junit.CONTAINER" value=""/>
    <booleanAttribute key="org.eclipse.jdt.junit.KEEPRUNNING_ATTR" value="false"/>
    <stringAttribute key="org.eclipse.jdt.junit.TESTNAME" value=""/>
    <stringAttribute key="org.eclipse.jdt.junit.TEST_KIND" value="org.eclipse.jdt.junit.loader.junit5"/>
    <booleanAttribute key="org.eclipse.jdt.launching.ATTR_ATTR_USE_ARGFILE" value="false"/>
    <booleanAttribute key="org.eclipse.jdt.launching.ATTR_SHOW_CODEDETAILS_IN_EXCEPTION_MESSAGES" value="true"/>
    <stringAttribute key="org.eclipse.jdt.launching.JRE_CONTAINER" value="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-21"/>
    <stringAttribute key="org.eclipse.jdt.launching.MAIN_TYPE" value="org.eclipse.ui.editors.tests.SegmentedModeTest"/>
    <stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-os ${target.os} -ws ${target.ws} -arch ${target.arch} -nl ${target.nl} -consoleLog"/>
    <stringAttribute key="org.eclipse.jdt.launching.PROJECT_ATTR" value="org.eclipse.ui.editors.tests"/>
    <stringAttribute key="org.eclipse.jdt.launching.SOURCE_PATH_PROVIDER" value="org.eclipse.pde.ui.workbenchClasspathProvider"/>
    <stringAttribute key="org.eclipse.jdt.launching.VM_ARGUMENTS" value="-XX:CompileCommand=quiet -XX:CompileCommand=exclude,org.eclipse.jdt.internal.core.dom.rewrite.ASTRewriteAnalyzer::getExtendedRange -Xms256m -Xcheck:jni -Xmx16G -XX:G1PeriodicGCInterval=10000 -XX:G1PeriodicGCSystemLoadThreshold=3.0 -XX:NativeMemoryTracking=summary -XX:TrimNativeHeapInterval=5000 -Dosgi.requiredJavaVersion=21 -Dosgi.dataAreaRequiresExplicitInit=true -Dorg.eclipse.swt.graphics.Resource.reportNonDisposed=true -Declipse.log.size.max=20000 -Dorg.eclipse.swt.internal.gtk.noThemingFixes -Dorg.eclipse.swt.internal.enableStrictChecks -Declipse.p2.unsignedPolicy=allow -Dorg.eclipse.equinox.p2.director.checkAuthorities=false -DSWT_GTK_INPUT_HINT_NO_EMOJI=true -Dp2.skipRepositoryProtocolCheck=true -Djava.security.manager=allow --add-modules=ALL-SYSTEM -Dorg.slf4j.simpleLogger.defaultLogLevel=off -Dorg.eclipse.swt.internal.gtk.cssFile=/opt/ate/devtools/bin/theme_patches.css"/>
    <stringAttribute key="pde.version" value="3.3"/>
    <stringAttribute key="product" value="org.eclipse.sdk.ide"/>
    <booleanAttribute key="run_in_ui_thread" value="true"/>
    <setAttribute key="selected_target_bundles"/>
    <setAttribute key="selected_workspace_bundles">
        <setEntry value="org.eclipse.ui.editors.tests@default:default"/>
    </setAttribute>
    <booleanAttribute key="show_selected_only" value="false"/>
    <booleanAttribute key="tracing" value="false"/>
    <booleanAttribute key="useCustomFeatures" value="false"/>
    <booleanAttribute key="useDefaultConfig" value="true"/>
    <booleanAttribute key="useDefaultConfigArea" value="false"/>
    <booleanAttribute key="useProduct" value="true"/>
</launchConfiguration>

Interestingly, this part

    <setAttribute key="selected_workspace_bundles">
        <setEntry value="org.eclipse.ui.editors.tests*3.13.1000.qualifier"/>
    </setAttribute>

changes to this:

    <setAttribute key="selected_target_bundles"/>
    <setAttribute key="selected_workspace_bundles">
        <setEntry value="org.eclipse.ui.editors.tests@default:default"/>
    </setAttribute>

The another important change now is that JRE_CONTAINER is added and set to Java 21.

The JRE change allows the launch config go one step further.
Now (after closing the dialog and starting again from the menu) we see the SegmentedModeTest properly starting but it doesn't fail as it does on SDK build.

This brings us to the second problem.

The set of plugins selected by the PDE includes everything in my workspace. After playing with the launch config, I know there is some bundle (no idea yet which) that "fixes" the test, so I can't reproduce what SDK tests are doing.

So let try to fix that.

This is how the launch config looks like:

Let "fix" that config by using "Restore defaults" button.

This selects 496 bundles.

Test fails now (OK), but this is obviously way too much (I also see bundles included in the launch config now that have no relationship at all to SDK or UI tests, like Eclemma, Copilot etc).

So let try to fix that.

As a naive user, let click on "Deselect All" to have a "clean" state and after that click on "Select Required".
This selects 151 bundles, which sounds reasonable, but...

The test wil not start anymore, because of

eclipse.buildId=4.38.0.I20251007-1800
java.version=25
java.vendor=Eclipse Adoptium
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Command-line arguments:  -os linux -ws gtk -arch x86_64 -data /data/4x_platform_workspace

org.eclipse.jdt.junit
Error
Wed Oct 08 15:37:46 CEST 2025
No tests found with test runner 'JUnit 5'.

For whatever reason, the actual test bundle is not selected anymore after clicking on "Select Required", so no tests could be found?!?

Finally, after selecting org.eclipse.ui.editors.tests bundle manually in the launch config, clicking once again on "Select Required", PDE selects 159 bundles, the test starts but does not fail...

So I stuck between minimal state of 159 bundles with green test and "full blown" state of 496 bundles with broken test.

This last problem is surely not a PDE problem, as neither "default" nor "minimal required" set of bundles must match what SDK tests are containing.

But here are the PDE (or platform) bugs that must be fixed:

1. Starting tests from scratch should produce same launch config as if started from the launch configurations dialog for the first time. So dialog pages do more with launch config as action triggered from the menu.
2. If launch requires minimal JRE as 21 via VM attribute -Dosgi.requiredJavaVersion=21 (added by PDE by default), PDE shouldn't select Java 17 just because it is JRE what bundle requires.
3. "Select Required" button should not deselect actual test bundle that contains the test.
4. If nothing is selected, "Select Required" should actually not select anything or select all required bundles including test bundle for the test), but it selects some but not all bundles required by the test bundle with the test, excluding test bundle. I don't see any logic behind.
5. "Select Required" naming is also wrong. It doesn't select required bundles for the test bundle, it selects bundles required by already selected bundles, but if nothing is selected, it still does something. Weird. Calling it multiple times after adding something to selection produces different results, so it seem to be some kind of "Add all Required to Selection".
6. "Restore Defaults" obviously doesn't restore defaults (as the launch config never goes back to the original state after that), so the button name is also wrong. It restores "default" plugins set (whatever default is?), and this state doesn't match the selection state of the original launch config with a single test bundle selected.

[laeubi]

I can't reproduce all this if I run the test with latest Eclipse-SDK IDE and make sure there is no old launch config I get:

and it selects exactly one bundle for the run (I think we can agree that it can't be more minimal):

and the launch is successful

and test passes

So everything works as expected for the use case of running a single tests as a JUnit Plugin Test ... if you want "like SDK tests" you need to look here https://wiki.eclipse.org/Platform-releng/Eclipse_Test_Framework

[iloveeclipse]

So everything works as expected

... for your configuration / target platform. The bugs are real and there.

[merks]

May I continue?

[laeubi]

So everything works as expected

... for your configuration / target platform. The bugs are real and there.

This does not make a difference, I showed you how the feature can be used, if you insist not to try it out with a fresh run config / recommended configuration I fear there is no way to make progress on these bugs (where some are just not bugs but working as documented in the eclipse help).

[iloveeclipse]

This does not make a difference, I showed you how the feature can be used, if you insist not to try it out with a fresh run config / recommended configuration

Please read bug once again. If it is not clear, I never heard about SegmentedModeTest before and so I started with NO EXISTING LAUNCH CONFIG, so whatever was created by PDE was done fresh by the IDE and if IDE doesn't ask me anything, I assume it supposed to do the "recommended" thing by default.

[iloveeclipse]

May I continue?

Sure. Anyone can contribute to fix the problems, so why not?

[merks]

Sorry, my tiny iphone easily mixes me up on issues. I'm trying to deal with the junit 6 problems, not this one.

[laeubi]

Please read bug once again. If it is not clear, I never heard about SegmentedModeTest before and so I started with NO EXISTING LAUNCH CONFIG, so whatever was created by PDE was done fresh by the IDE and if IDE doesn't ask me anything, I assume it supposed to do the "recommended" thing by default.

Then please use latest SDK/Eclipse for test and make sure you selected "The Plugin of the test"

https://eclipse.dev/eclipse/markdown/?f=news/4.37/pde.md#junit-plugin-test-launches

please make sure to delete any old launch config as otherwise PDE will not create one from scratch but reuse what you have configured.

[iloveeclipse]

Then please use latest SDK/Eclipse for test

I always use latest nightly SDK build.

and make sure you selected "The Plugin of the test"

Once again: I start PDE JUnit tests by right clicking on the Java class in the Package Explorer.
Quote:

select SegmentedModeTest in the Packages Explorer and say "Debug as -> JUnit Plug-in Test".

This obviously uses a different path in PDE code to create launch config as the code that creates it in the launch config dialog, and that is the root cause of the bug 1 in my list.

[laeubi]

Once again: I start PDE JUnit tests by right clicking on the Java class in the Package Explorer.

So do I ... no one has ever claimed something different, I showed you the preferences that control how PDE creates new launch configs if you are using the shortcuts from the menu. No one ever talked about the launch config dialog and I suppose 99% of the developers will never create a JUnit launch that way...

So the real difference seems that you choose "Debug As" while I used "Run As" ... what is quite unexpected but maybe intentional. It would really help to focus on one problem at a time its completely confusing here and the important details are just getting lost between all kind of unrelated complains.

[trancexpress]

Another bug to add here, if I try to run org.eclipse.pde.junit.runtime.tests.JUnitExecutionTest (from the context menu of the Java Editor, as a JUnit plug-in test), I get:

org.eclipse.core.runtime.CoreException: Launching failed. Bootstrap code cannot be found.
	at org.eclipse.pde.launching.JUnitLaunchConfigurationDelegate.abort(JUnitLaunchConfigurationDelegate.java:178)
	at org.eclipse.pde.launching.JUnitLaunchConfigurationDelegate.getClasspathAndModulepath(JUnitLaunchConfigurationDelegate.java:444)
	at org.eclipse.jdt.junit.launcher.JUnitLaunchConfigurationDelegate.getVMRunnerConfiguration(JUnitLaunchConfigurationDelegate.java:192)
	at org.eclipse.jdt.junit.launcher.JUnitLaunchConfigurationDelegate.launch(JUnitLaunchConfigurationDelegate.java:275)
	at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:778)
	at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:689)
	at org.eclipse.debug.internal.ui.DebugUIPlugin.buildAndLaunch(DebugUIPlugin.java:1052)
	at org.eclipse.debug.ui.DebugUITools.buildAndLaunch(DebugUITools.java:1044)
	at org.eclipse.pde.junit.runtime.tests.TestExecutionUtil.launchAndWaitForTermination(TestExecutionUtil.java:126)
	at org.eclipse.pde.junit.runtime.tests.TestExecutionUtil.runTest(TestExecutionUtil.java:81)
	at org.eclipse.pde.junit.runtime.tests.JUnitExecutionTest.executePackage(JUnitExecutionTest.java:111)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.eclipse.pde.ui.tests.runtime.TestUtils$1.evaluate(TestUtils.java:267)

If I use Restore Defaults in the Plugins tab of the launch configuration, the error is no longer reported.

[laeubi]

It really does not make sense to dump anything that might be inconvenient here in one big trash-bin.

If there is a specific problem, please create an own issue with a description on how to reproduce the problem at best with a small example project that can be executed with the Eclipse SDK.

If one likes it can then be made an "sub-issue" of this one... otherwise likely no one will look at this and its just bit-rot and even if there is no way to see what is already solved and what not.

[trancexpress]

Sure, I created: #2024