103 fails in core.tests.resources, 6 in core.tests.runtime and 3 in core.tests.resources.saveparticipant

[iloveeclipse]

Fiollow up on eclipse-jdt/eclipse.jdt#146 / eclipse-platform/eclipse.platform.releng.aggregator#3430

See https://download.eclipse.org/eclipse/downloads/drops4/I20251026-0750/testResults.php

We have platform tests that do not timeout anymore, but are failing now with similar error Process returned non-zero code: 13 :

• 103 fails in core.tests.resources,
• 6 in core.tests.runtime
• 3 in core.tests.resources.saveparticipant

[iloveeclipse]

The problem is the multiple junit 5/6 library versions at runtime.

In IDE I can force the test to fail if I select "Restore defaults" in the launch config. Default launch config that only uses required dependencies doesn't fail.

The fail looks like

!SESSION 2025-10-27 17:04:06.802 -----------------------------------------------
eclipse.buildId=unknown
java.version=21.0.8
java.vendor=Eclipse Adoptium
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments:  -version 3 -port 36307 -testLoaderClass org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader -loaderpluginname org.eclipse.jdt.junit5.runtime -classNames org.eclipse.core.tests.internal.preferences.TestBug388004 -application org.eclipse.pde.junit.runtime.coretestapplication -testpluginname org.eclipse.core.tests.runtime
Command-line arguments:  -os linux -ws gtk -arch x86_64 -consoleLog -version 3 -port 36307 -testLoaderClass org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader -loaderpluginname org.eclipse.jdt.junit5.runtime -classNames org.eclipse.core.tests.internal.preferences.TestBug388004 -application org.eclipse.pde.junit.runtime.coretestapplication -data /data/4x_platform_workspace/../junit-workspace -dev file:///data/4x_platform_workspace/.metadata/.plugins/org.eclipse.pde.core/pde-junit/dev.properties -os linux -ws gtk -arch x86_64 -consoleLog -testpluginname org.eclipse.core.tests.runtime

!ENTRY org.eclipse.osgi 4 0 2025-10-27 17:04:09.237
!MESSAGE Application error
!STACK 1
java.util.ServiceConfigurationError: org.junit.platform.engine.TestEngine: org.junit.jupiter.engine.JupiterTestEngine not a subtype
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1244)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
	at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
	at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
	at java.base/java.lang.Iterable.forEach(Iterable.java:74)
	at org.junit.platform.launcher.core.LauncherFactory.collectTestEngines(LauncherFactory.java:161)
	at org.junit.platform.launcher.core.LauncherFactory.createDefaultLauncher(LauncherFactory.java:139)
	at org.junit.platform.launcher.core.LauncherFactory.lambda$create$2(LauncherFactory.java:132)
	at org.junit.platform.launcher.core.DefaultLauncherSession.lambda$new$0(DefaultLauncherSession.java:66)
	at org.junit.platform.launcher.core.ClasspathAlignmentCheckingLauncherInterceptor.intercept(ClasspathAlignmentCheckingLauncherInterceptor.java:25)
	at org.junit.platform.launcher.core.DefaultLauncherSession.<init>(DefaultLauncherSession.java:66)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.createSession(SessionPerRequestLauncher.java:78)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.discover(SessionPerRequestLauncher.java:58)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.<init>(JUnit5TestReference.java:47)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.createUnfilteredTest(JUnit5TestLoader.java:88)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.createTest(JUnit5TestLoader.java:69)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.loadTests(JUnit5TestLoader.java:56)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:504)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:748)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:443)
	at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:111)
	at org.eclipse.pde.internal.junit.runtime.CoreTestApplication.start(CoreTestApplication.java:28)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:219)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:149)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:115)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:467)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:298)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:615)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:563)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1415)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1387)

I believe we need this: eclipse-jdt/eclipse.jdt.ui#2574 (have to check).

[iloveeclipse]

I believe we need this: eclipse-jdt/eclipse.jdt.ui#2574 (have to check).

Nope. The fix eclipse-jdt/eclipse.jdt.ui#2574 is incomplete, the TestKind from jUnit 5 doesn't supply JUnit version as expected and calls new constructor with null as mentioned here.

@trancexpress : FYI.

[iloveeclipse]

Nope. The fix eclipse-jdt/eclipse.jdt.ui#2574 is incomplete, the TestKind from jUnit 5 doesn't supply JUnit version as expected and calls new constructor with null as mentioned here.

Not quite correct. There are still calls to org.eclipse.jdt.internal.junit.launcher.JUnitRuntimeClasspathEntry.JUnitRuntimeClasspathEntry(String, String, String) with null version, but for org.eclipse.jdt.junit5.runtime bundle only, which seem to be not important because later correct bundles are selected in org.eclipse.jdt.junit.launcher.JUnitLaunchConfigurationDelegate.ClasspathLocalizer.localURL(JUnitRuntimeClasspathEntry).

The place where something goes wring seem to be different to code changed in eclipse-jdt/eclipse.jdt.ui#2574.

With the latest version from eclipse-jdt/eclipse.jdt.ui#2574 I see classpatch is constructed via

	at org.eclipse.jdt.junit.launcher.JUnitLaunchConfigurationDelegate.getVMRunnerConfiguration(JUnitLaunchConfigurationDelegate.java:236)
	at org.eclipse.jdt.junit.launcher.JUnitLaunchConfigurationDelegate.launch(JUnitLaunchConfigurationDelegate.java:276)
	at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:778)
	at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:689)
	at org.eclipse.debug.internal.ui.DebugUIPlugin.buildAndLaunch(DebugUIPlugin.java:1052)
	at org.eclipse.debug.internal.ui.DebugUIPlugin$1.run(DebugUIPlugin.java:1254)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

and contains

• junit-jupiter-api 5.14.0
• junit-jupiter-engine 5.14.0
• junit-platform-launcher 1.14.0
  which seem to be expected, but launching still fails with same java.util.ServiceConfigurationError: org.junit.platform.engine.TestEngine: org.junit.jupiter.engine.JupiterTestEngine not a subtype.

[trancexpress]

The problem is the multiple junit 5/6 library versions at runtime.

In IDE I can force the test to fail if I select "Restore defaults" in the launch config. Default launch config that only uses required dependencies doesn't fail.

Are you sure this is not yet another problem?

The fix eclipse-jdt/eclipse.jdt.ui#2574 is incomplete

Note that I'm only guessing for eclipse-jdt/eclipse.jdt.ui#2572. I've not reproduced the fail so I can't be sure this is the actual problem.

[iloveeclipse]

Are you sure this is not yet another problem?

I guess it is same for following reasons:

• In IDE test works fine if I provide minimal required runtime config (about 100 bundles)
• In IDE test crashes if config is "bloated" by "restore defaults" (about 600 bundles)
• In SDK runtime likely contain all bundles from SDK, and tests crash too Process returned non-zero code: 13.

I'm using org.eclipse.core.tests.internal.preferences.TestBug388004 test because it can be even run headless.

[iloveeclipse]

If I manually deselect JUnit 6 bundles in the "bloated" launch config, test works fine:

[laeubi]

I only wanted to note that mostly a change in jdt.ui will not change what happens in PDE and I'm not 100% sure how the testrun are constructed exactly e.g. they use the tests.xml ant scripts that migth work differently than the IDE itself.

Apart from that I think pressing "restore defaults" is not useful at all, whatever these defaults are meant to be it likely gives wrong results. I wanted to look into this but first wait for the junit part to settle down to not create more disruption here.

[trancexpress]

If I manually deselect JUnit 6 bundles in the "bloated" launch config, test works fine:

I've not seen any code that tries to fix a bad configuration.

The only code I've seen is for adding bundles that are missing. The problems I've seen so far come from adding the wrong version for such missing bundles. If the user adds JUnit 5 and JUnit 6 on purpose (or via whatever this restore is doing), the error that you see is expected: eclipse-pde/eclipse.pde#2055

We should probably look into the restore once all the JUnit 6 work is done, it seems to be doing odd things?

[iloveeclipse]

If the user adds JUnit 5 and JUnit 6 on purpose

I didn't added anything on purpose, I disabled what PDE added.

I think pressing "restore defaults" is not useful at all, whatever these defaults are meant to be it likely gives wrong results

The point I'm trying to explain is the similarity of what I observe with the SDK failures, I do not want to discuss the "restore defaults" behavior here, I just want to understand and fix the test failures we observe in SDK tests.

And as mentioned above I believe the problem is the coexistence of both junit 5 and 6 versions in same installation. How this happens (by user or PDE) is a different question.

I've not seen any code that tries to fix a bad configuration

This is probably the root cause.

Regarding "bad" configuration: default SDK contains now both junit versions, so why it is considered to be bad?

[trancexpress]

I didn't added anything on purpose, I disabled what PDE added.

I dont understand this. PDE should be adding a few extra JUnit bundles, potentially with the wrong major version. If you would disable that behavior, then there should be no conflicting bundles. The restore operation must be adding bundles itself.

Regarding "bad" configuration: default SDK contains now both junit versions, so why it is considered to be bad?

The 5 and 6 JUnit bundles dont seem to be able to coexist in the same JVM, when running tests.

I didnt examine why exactly, since I doubt anyone expects this to work. Probably its a plain classloading issue, due to having JUnit classes sharing FQN but residing in different bundles.

[iloveeclipse]

I dont understand this. PDE should be adding a few extra JUnit bundles, potentially with the wrong major version. If you would disable that behavior, then there should be no conflicting bundles. The restore operation must be adding bundles itself.

Let us don't discuss "restore" behavior here, please. I've explained many times, it was just a demonstration of a behavior that was very similar to what we see in SDK tests.

The 5 and 6 JUnit bundles dont seem to be able to coexist in the same JVM, when running tests. I doubt anyone expects this to work. Probably its a plain classloading issue, due to having JUnit classes sharing FQN but residing in different bundles.

I was using latest SDK build to run the test, and it worked (at least for JUnit 4 & 5). Latest SDK contains both junit 5 and 6, so I don't understand why we don't expect it to work if we have already both junit versions in same SDK.

[laeubi]

Regarding "bad" configuration: default SDK contains now both junit versions, so why it is considered to be bad?

The 5 and 6 JUnit bundles dont seem to be able to coexist in the same JVM, when running tests.

To avoid some confusion, having JUnit 5+6 in the same JVM (like an Eclipse SDK) is not a problem at all in general.

It only become a bit more difficult if one actually executes Junit runner because that uses the java SPI to load implementations and then two kind of issues can arise:

• Flat Classpath (e.g. jdt run) order is important now because otherwise there might be Method Not Found and maybe things are discovered twice (because SPI sees two services folder)
• Separate Classloaders (e.g. pde run) it can happen that classloaders are incompatible and SPI complains (X is not a subtype)

For the second it is important to carefully review the stacktrace, because usually the problem is the following:

1. As we have separate classloaders and JUnit / Java SPI was designed with a flat classpath in mind it can not discover other implementations unless we "help" a little bit usually setting a ThreadContextClassLoader
2. If that is to broad and not carefully checks compatibilty, the we will discover incompatible things and run into the problem

One example where I just hacked it a little bit is this https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/pull/3431/files#diff-ed0a19c972177a1842c2081c1a0e650e572a0b6a32a0554a32fa9f8c1c6ff833
A more smart approach would be like I did here https://github.com/eclipse-pde/eclipse.pde/pull/2037/files#diff-13d91ef6c54164891d89bca8b1c02500932a253eecd8f0ff7e7c98670d5946caR907-R933 where one looks at the requirements and what is compatible, but this requires to first determine who "leads" whats needed

In Tycho I implemented it in a way that I look who is the caller and if the bundles of caller + provider match:
https://github.com/eclipse-tycho/tycho/blob/main/tycho-test-plugin/src/main/java/org/eclipse/tycho/testing/plugin/SPIBundleClassLoader.java

this is actually a "forward" implementation of the general feature we plan in Equionox/OSGi that is automatically detects SPI interfaces without special classloaderhacks.

So in this particular-case by @iloveeclipse one needs to find out why RemotePluginTestRunner thinks the providers are compatible while they are not ... I'll try to take a short look here.