Improve substitution package handling and filter uses locally #1124

laeubi · 2025-08-10T15:35:22Z

Currently substitution packages handling can lead to a situation where
the number of performed permutation increase considerably and the
finding of a solution takes very long. In some cases it even times out
and results in a failed resolve state.

This now is improved by the following changes:

When a permutation is removed from the stack, every requirement is
checked if it is a substitution package and then resolve it to either
external or internal case first:

if more candidates exits for this requirement a permutation is added
to account for this alternative solution
now all other providers except the current one are dropped to make
this a single provider as the choice can not be reverted and therefore
other permutations in this path will be invalid
then it must be checked if the package resolves to an external
provider in which case we need to drop the exported package capabilities
of this bundle for the given package
after that all requirements are checked if they have only a single
provider (what can happen independently or as part of the resolving to
either external or internal) the use constraints are checked if they are
in conflict with a currently selected provider and dropping such invalid
choices
finally after that we check the package space consistency (that is a
far more expensive check) what probably will result in more permutations
created and checked until we found the best result to use

As this is a rather fundamental change there is a kill-switch

When setting -Dfelix.resolver.substitution.legacy=true on the JVM it switches back to the old behavior.

github-actions · 2025-08-10T16:00:46Z

Test Results

540 files - 270 540 suites - 270 1h 49m 33s ⏱️ + 24m 43s
2 222 tests + 1 2 170 ✅ - 2 50 💤 + 1 2 ❌ +2
4 454 runs - 2 223 4 350 ✅ - 2 178 100 💤 - 49 4 ❌ +4

For more details on these failures, see this check.

Results for commit 8c4e76c. ± Comparison against base commit 25234a4.

This pull request skips 1 test.

AutomatedTests org.eclipse.osgi.tests.services.datalocation.AllTests org.eclipse.osgi.tests.services.datalocation.BasicLocationTests ‑ testUNC

♻️ This comment has been updated with latest results.

laeubi · 2025-08-10T17:14:23Z

I now finally sorted out the problems with the test.

For my testcase it is

Before: Resolve takes 66 sec and needed to process 2663 permutations
After: Resolve takes 1 sec and needed to process 15 permutations

laeubi · 2025-08-10T18:21:44Z

I was now able to capture this into a database dump and use it as a testcase here:

Add a new testcase to demonstrate a long running resolve operation #1125

The testcase takes 71,5 seconds on my system and requires to process 2661 permutations without my change.
With this change this goes down to 7,9 seconds and only requires 117 permutations to be processed.

Most notable, it reduces the uses permutations created from 31623 to 1(!) and the package permutations from 9272 to 172 meaning it can capture most of uses-constraint violation before they are actually surfacing!

laeubi · 2025-08-11T05:18:59Z

testLargeSet goes down from 2m 11sec > 13 sec now!

vogella · 2025-08-11T07:40:49Z

testLargeSet goes down from 2m 11sec > 13 sec now!

Hubba bubba 😀

eclipse-equinox-bot · 2025-09-03T04:42:20Z

This pull request changes some projects for the first time in this development cycle.
Therefore the following files need a version increment:

bundles/org.eclipse.osgi/META-INF/MANIFEST.MF
bundles/org.eclipse.osgi/pom.xml
features/org.eclipse.equinox.core.feature/feature.xml
features/org.eclipse.equinox.core.sdk/feature.xml
features/org.eclipse.equinox.server.core/feature.xml

An additional commit containing all the necessary changes was pushed to the top of this PR's branch. To obtain these changes (for example if you want to push more changes) either fetch from your fork or apply the git patch.

Git patch

From bff5c2061b400a468e713ddb3351b24273577d3b Mon Sep 17 00:00:00 2001
From: Eclipse Equinox Bot <[email protected]>
Date: Wed, 3 Sep 2025 16:22:45 +0000
Subject: [PATCH] Version bump(s) for 4.38 stream


diff --git a/bundles/org.eclipse.osgi/META-INF/MANIFEST.MF b/bundles/org.eclipse.osgi/META-INF/MANIFEST.MF
index f5b9fc2b8..fa7846660 100644
--- a/bundles/org.eclipse.osgi/META-INF/MANIFEST.MF
+++ b/bundles/org.eclipse.osgi/META-INF/MANIFEST.MF
@@ -107,7 +107,7 @@ Bundle-Activator: org.eclipse.osgi.internal.framework.SystemBundleActivator
 Bundle-Description: %systemBundle
 Bundle-Copyright: %copyright
 Bundle-Vendor: %eclipse.org
-Bundle-Version: 3.23.200.qualifier
+Bundle-Version: 3.23.300.qualifier
 Bundle-Localization: systembundle
 Bundle-DocUrl: http://www.eclipse.org
 Eclipse-ExtensibleAPI: true
diff --git a/bundles/org.eclipse.osgi/pom.xml b/bundles/org.eclipse.osgi/pom.xml
index 88c714ef8..ab530c414 100644
--- a/bundles/org.eclipse.osgi/pom.xml
+++ b/bundles/org.eclipse.osgi/pom.xml
@@ -19,7 +19,7 @@
 </parent>
   <groupId>org.eclipse.platform</groupId>
   <artifactId>org.eclipse.osgi</artifactId>
-  <version>3.23.200-SNAPSHOT</version>
+  <version>3.23.300-SNAPSHOT</version>
   <packaging>eclipse-plugin</packaging>
   <properties>
 	  <!-- The actual TCKs are executed in the org.eclipse.osgi.tck module because of reference to other service implementations -->
diff --git a/features/org.eclipse.equinox.core.feature/feature.xml b/features/org.eclipse.equinox.core.feature/feature.xml
index 97598b06b..c9d008726 100644
--- a/features/org.eclipse.equinox.core.feature/feature.xml
+++ b/features/org.eclipse.equinox.core.feature/feature.xml
@@ -2,7 +2,7 @@
 <feature
       id="org.eclipse.equinox.core.feature"
       label="%featureName"
-      version="1.15.600.qualifier"
+      version="1.15.700.qualifier"
       provider-name="%providerName"
       license-feature="org.eclipse.license"
       license-feature-version="0.0.0">
diff --git a/features/org.eclipse.equinox.core.sdk/feature.xml b/features/org.eclipse.equinox.core.sdk/feature.xml
index 794863737..e719b10b1 100644
--- a/features/org.eclipse.equinox.core.sdk/feature.xml
+++ b/features/org.eclipse.equinox.core.sdk/feature.xml
@@ -2,7 +2,7 @@
 <feature
       id="org.eclipse.equinox.core.sdk"
       label="%featureName"
-      version="3.25.600.qualifier"
+      version="3.25.700.qualifier"
       provider-name="%providerName"
       license-feature="org.eclipse.license"
       license-feature-version="0.0.0">
diff --git a/features/org.eclipse.equinox.server.core/feature.xml b/features/org.eclipse.equinox.server.core/feature.xml
index 91734c800..7b0fb3a4d 100644
--- a/features/org.eclipse.equinox.server.core/feature.xml
+++ b/features/org.eclipse.equinox.server.core/feature.xml
@@ -2,7 +2,7 @@
 <feature
       id="org.eclipse.equinox.server.core"
       label="%featureName"
-      version="1.16.600.qualifier"
+      version="1.16.700.qualifier"
       provider-name="%providerName"
       license-feature="org.eclipse.license"
       license-feature-version="0.0.0">
-- 
2.51.0

Further information are available in Common Build Issues - Missing version increments.

laeubi · 2025-09-03T16:32:12Z

I now fixed some issues and now the testcase even goes down to 0,4 sec on my system and is able to complete with only one performed permutation. That mean that through local optimization we can compute a solution immediately.

I'll try to add one more testcase and have to analyze once case where we produce a permutation that was previously not required (but that might be due to the changed algorithm) and then think it would be good to merge this to discover possible problems but from the usual test-cases I can't find any issues with this anymore and it incredibly reduces the computation times.

Currently substitution packages handling can lead to a situation where the number of performed permutation increase considerably and the finding of a solution takes very long. In some cases it even times out and results in a failed resolve state. This now is improved by the following changes: When a permutation is removed from the stack, every requirement is checked if it is a substitution package and then resolve it to either external or internal case first: - if more candidates exits for this requirement a permutation is added to account for this alternative solution - now all other providers except the current one are dropped to make this a single provider as the choice can not be reverted and therefore other permutations in this path will be invalid - then it must be checked if the package resolves to an external provider in which case we need to drop the exported package capabilities of this bundle for the given package - after that all requirements are checked if they have only a single provider (what can happen independently or as part of the resolving to either external or internal) the use constraints are checked if they are in conflict with a currently selected provider and dropping such invalid choices - finally after that we check the package space consistency (that is a far more expensive check) what probably will result in more permutations created and checked until we found the best result to use

laeubi force-pushed the substitution_packages2 branch 3 times, most recently from 9c46d7b to 5971533 Compare August 10, 2025 17:05

laeubi mentioned this pull request Aug 11, 2025

Improve substitution package handling #1110

Closed

laeubi force-pushed the substitution_packages2 branch from ac6eee2 to 78b2377 Compare August 11, 2025 04:33

laeubi changed the title ~~Resolve Substitution packages~~ Improve substitution package handling and filter uses locally Aug 11, 2025

laeubi marked this pull request as ready for review August 11, 2025 04:34

laeubi marked this pull request as draft August 11, 2025 07:48

laeubi force-pushed the substitution_packages2 branch 2 times, most recently from ec6b508 to 850d63c Compare August 13, 2025 13:16

laeubi force-pushed the substitution_packages2 branch 5 times, most recently from 31cc157 to 3d6f728 Compare August 24, 2025 16:12

laeubi force-pushed the substitution_packages2 branch from 3d6f728 to 26d8e8f Compare September 3, 2025 04:34

laeubi force-pushed the substitution_packages2 branch 2 times, most recently from 87182b6 to c0dd976 Compare September 3, 2025 16:17

laeubi force-pushed the substitution_packages2 branch from a15d037 to 5fbdbc2 Compare September 4, 2025 06:49

laeubi mentioned this pull request Sep 4, 2025

Drop invalid package providers when adding a permutation #1086

Closed

laeubi force-pushed the substitution_packages2 branch from 5fbdbc2 to 680aece Compare September 4, 2025 14:20

laeubi marked this pull request as ready for review September 4, 2025 14:21

laeubi force-pushed the substitution_packages2 branch from 680aece to e9eee54 Compare September 6, 2025 05:57

laeubi mentioned this pull request Sep 6, 2025

Add a Candidates#toString method to make debugging easier #1142

Merged

laeubi force-pushed the substitution_packages2 branch from e9eee54 to f2341a8 Compare September 6, 2025 09:10

laeubi mentioned this pull request Sep 22, 2025

Broken Git operations via SSH with ed25519 keys eclipse-platform/eclipse.platform.ui#3308

Closed

2 tasks

laeubi force-pushed the substitution_packages2 branch from f2341a8 to 8c4e76c Compare September 22, 2025 13:28

chrisrueger mentioned this pull request Oct 19, 2025

Resolver runs in circles till it eats all the memory bndtools/bnd#4172

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve substitution package handling and filter uses locally #1124

Improve substitution package handling and filter uses locally #1124

Uh oh!

laeubi commented Aug 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 10, 2025 •

edited

Loading

Uh oh!

laeubi commented Aug 10, 2025

Uh oh!

laeubi commented Aug 10, 2025 •

edited

Loading

Uh oh!

laeubi commented Aug 11, 2025

Uh oh!

vogella commented Aug 11, 2025 •

edited

Loading

Uh oh!

eclipse-equinox-bot commented Sep 3, 2025 •

edited

Loading

Uh oh!

laeubi commented Sep 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Improve substitution package handling and filter uses locally #1124

Are you sure you want to change the base?

Improve substitution package handling and filter uses locally #1124

Uh oh!

Conversation

laeubi commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

As this is a rather fundamental change there is a kill-switch

Uh oh!

github-actions bot commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

laeubi commented Aug 10, 2025

Uh oh!

laeubi commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laeubi commented Aug 11, 2025

Uh oh!

vogella commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eclipse-equinox-bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laeubi commented Sep 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

laeubi commented Aug 10, 2025 •

edited

Loading

github-actions bot commented Aug 10, 2025 •

edited

Loading

laeubi commented Aug 10, 2025 •

edited

Loading

vogella commented Aug 11, 2025 •

edited

Loading

eclipse-equinox-bot commented Sep 3, 2025 •

edited

Loading