Are touchpoints still relevant or useful with more modern Eclipse?

[andrew-tram]

Admittedly, a bit of a silly question, but are touchpoints still relevant or useful with more modern Eclipse?

Ive recently taken ownership of a project and I'm encountering some issues due to touchpoint, timing, and possibly hot swapping bugs when using "Install New Software..." on plugins that use touchpoint. Specifically, things like this....

## Ensuring some p2 touchpoint plugin installed before this plugin
metaRequirements.0.namespace=org.eclipse.equinox.p2.iu
metaRequirements.0.name=com.my.company.some.plugin
metaRequirements.0.range=[1.0.0,2.0.0)

The thing is that the plugin that's required and the plugin that has the p2.inf that's calling it out will never be installed separately. Theyre always going to be tied together. Though, it's not listed as a required-bundle in the manifest.mf...

I honestly want to avoid using touchpoint as it seems messy and I'm not terribly familiar with it,

I also came across this, which I thought was semi-relevant - https://eclipse.dev/eclipse/news/4.31/platform.html#eclipse_launcher_reloads_eclipse_ini_on_restart

[laeubi]

In general touch-points are relevant and actually used in every installation (but you probably don't notice them) if they are still required to be maintained manually depends on your use-case:

1. P2 will install everything that is required automatically, e.g. if Bundle A requires Bundle B in any way (require-bundle, import package, ...) an install of A will always install B. Thats also the reason why we do not recommend to add third party libraries to a feature anymore as it will happen anyways
2. If you want to group things together, (e.g. A+B have no dependencies to each others but belong to each other and should be installed together) then use a feature and install this
3. There are very very rare cases where one would need additional p2 instructions (e.g. SWT) but then it should be used with care and you should be aware of the consequences.

[merks]

Unfortunately I think no matter where you move it no one is going to feel obligated to fix it. 😢

So it's either self-service or find someone with the skills and available resource to fix it. Sorry...

[andrew-tram]

I understand - my thought was that if I moved it to the right place, that community would have some watchers who have some thoughts

[merks]

No, I think the folks who work on the platform work in an environment where the registry does not change dynamically because a bundle is updated without a restart...

[mickaelistria]

FWIW, the discussion has been quite long and addressed different topics. It was easy to lost track of the conversation. If the conclusion is that there is a bug somewhere in latest release or latest I-Build, then report open a bug with the simplest possible reproducers and comparison of expected vs actual behavior.
Reporting it i a more concise and "formatted" way may attract new interest, especially if the report gives a feeling of being already actionable.

[gireeshpunathil]

@mickaelistria - no, we haven't concluded that there is an issue in the latest release.

[gireeshpunathil]

while the Registry event dispatcher thread is engaged in accessing the registry:

org.eclipse.core.runtime.InvalidRegistryObjectException: Invalid registry object
	at org.eclipse.core.internal.registry.RegistryObjectManager.basicGetObject(RegistryObjectManager.java:293)
	at org.eclipse.core.internal.registry.RegistryObjectManager.getObject(RegistryObjectManager.java:283)
	at org.eclipse.core.internal.registry.ConfigurationElementHandle.getConfigurationElement(ConfigurationElementHandle.java:29)
	at org.eclipse.core.internal.registry.ConfigurationElementHandle.getAttribute(ConfigurationElementHandle.java:38)
	at org.eclipse.ui.internal.services.RegistryPersistence.readRequired(RegistryPersistence.java:428)
	at org.eclipse.ui.internal.services.RegistryPersistence.readRequired(RegistryPersistence.java:407)
	at org.eclipse.ui.internal.commands.CommandImagePersistence.readImagesFromRegistry(CommandImagePersistence.java:91)
	at org.eclipse.ui.internal.commands.CommandImagePersistence.read(CommandImagePersistence.java:225)
        ...

, another thread (Refresh Thread: Equinox Container) is seen as actively removing entries from it. while methods in the class are synchronised to prevent cluttered access of the cache, apparently there is no check to see if the same entry as deleted is being requested for fetching, nor a provision to handle if the cache doesn't contain what is requested; instead it throws Exception in the premise that the cache is expected to contain the entry.

at org/eclipse/core/internal/registry/RegistryObjectManager.removeContribution(RegistryObjectManager.java:486)
at org/eclipse/core/internal/registry/ExtensionRegistry.basicRemove(ExtensionRegistry.java:264)
at org/eclipse/core/internal/registry/ExtensionRegistry.remove(ExtensionRegistry.java:635)
at org/eclipse/core/internal/registry/ExtensionRegistry.remove(ExtensionRegistry.java:607)
at org/eclipse/core/internal/registry/osgi/EclipseBundleListener.removeBundle(EclipseBundleListener.java:112)
at org/eclipse/core/internal/registry/osgi/EclipseBundleListener.bundleChanged(EclipseBundleListener.java:86)
at org/eclipse/osgi/internal/framework/BundleContextImpl.dispatchEvent(BundleContextImpl.java:972(Compiled Code))
at org/eclipse/osgi/framework/eventmgr/EventManager.dispatchEvent(EventManager.java:230(Compiled Code))
at org/eclipse/osgi/framework/eventmgr/ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151(Compiled Code))
at org/eclipse/osgi/internal/framework/EquinoxEventPublisher.publishBundleEventPrivileged(EquinoxEventPublisher.java:235(Compiled Code))
at org/eclipse/osgi/internal/framework/EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:134(Compiled Code))
at org/eclipse/osgi/internal/framework/EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:126(Compiled Code))
at org/eclipse/osgi/internal/framework/EquinoxContainerAdaptor.publishModuleEvent(EquinoxContainerAdaptor.java:229(Compiled Code))
at org/eclipse/osgi/container/ModuleContainer.unresolve0(ModuleContainer.java:1289)
at org/eclipse/osgi/container/ModuleContainer.unresolve(ModuleContainer.java:1132)
at org/eclipse/osgi/container/ModuleContainer.refresh(ModuleContainer.java:1340)
at org/eclipse/osgi/container/ModuleContainer$ContainerWiring.dispatchEvent(ModuleContainer.java:1735)
at org/eclipse/osgi/container/ModuleContainer$ContainerWiring.dispatchEvent(ModuleContainer.java:1)
at org/eclipse/osgi/framework/eventmgr/EventManager.dispatchEvent(EventManager.java:230(Compiled Code))
at org/eclipse/osgi/framework/eventmgr/EventManager$EventThread.run(EventManager.java:341)

my next step is to see if the requested object id was removed recently.

[tjwatson]

I thought I stated this before, but don't see my comment. I believe the issue is likely to be a long standing problem and I have my doubts any recent changes either broke it or fixed the issue.

Any scenario where p2 is calling refreshPackages or refreshBundles while the platform is running by default is a bug in p2 in my opinion. Moving this conversation over to another location, like platform, isn't going to help. The rest of the platform is not prepared to handle dynamic changes to the bundles (particularly any core bundles that may get swept into the refresh operation). A very long time ago p2 had the notion of allowing the user to either select "restart now" or "apply now" after a p2 install process. Where the "apply now" would attempt to provision the bundles into the runtime OSGi Framework while the platform was running.

This caused a huge number of problems that the community simply did not have the time to fix nor the motivation since doing the operation on a restart was always the safe way to guarantee things worked properly after an update/install p2 operation. So p2 did away with giving the user a choice and always offered the "restart now" option after an install to apply the changes.

Any scenario that has p2 trying to apply some newly installed content into the running Eclipse instance is very likely a hole in the overall design of p2. Maybe an old artifact of the days where the p2 developers strove to allow the dynamic updates while the platform is running?

In my opinion we need to understand what scenario needs to install the newly provisioned content into the running Eclipse instance and where in the p2 code that is being driven. If that has any hopes of working I believe it must NOT use a refresh operation on the framework to avoid any risk of refreshing any running bundle in Eclipse. If there is a valid reason p2 must install some of the newly provisioned content, then it must only use a resolve operation against the bundles it newly installs. It must never uninstall any existing bundles nor do a refresh operation while the platform is running. This will only cause some scenario to fail in the future. Even if you somehow get past your immediate issue with some work around, something else will break again someday where some core bundle gets refreshed again while the platform is running.

[merks]

One thing I should remind as well is that this specific scenario involves meta-requirements and somehow the x that requires the touchpoint y to be installed in order to install x is tied in some other way to y, e.g., both are included by some features or even in the same feature. One must keep y as independent of all other things as possible including and especially of x. One should generally ensure that y can update (pre-updated) all by itself. I think it this specific case that such due caution has not been taken and I that should be revisited first and foremost because as @tjwatson suggests, the platform in general is not designed to deal properly with bundles updated dynamically in place, though for touchpoints that is supposed to work. Also, why does y need to be updated at all? Has some touchpoint behavior changed in some way that the newer x cannot be touched properly by the older y touchpoint? I'm doubtful of that as well. The exact details of all the IUs involved and the version ranges of the requirements and meta-requirements have not been fully spelled out but the example looked very bad in this regard...

[andrew-tram]

Something I observed - when replacing these two specific profiles in 4.31 (p2\org.eclipse.equinox.p2.engine\profileRegistry\epp.package.rcp.profile) with those profiles in 4.32. Then the install works. No idea why though.

What I know for 4.31 is:

• installing something random like spotbugs causes the touchpoint update-site to install
• installing a "empty" or feature/plugin that has no source/ content gives the same registry install when trying the toupoint update-site

Does the first point cause the profile to "revalid" or "self correct" itself and thus why installing the touchpoint update-site works? Launching with -clean from a clean unzip of of 4.31 didn't help when trying to install the touchpoint update site. Nor did simply delete these profiles.

[andrew-tram]

Is there something "bad" in the profiles that was in 4.31 (well between 4.29 - 4.31) profile?

[gireeshpunathil]

here is the sequence of events led to the issue as per my understanding:

• the touchpoint installation triggers upgrades to the IU org.eclipse.osgi.services.
• this is because: i) this bundle is marked as deprecated in 4.31 ii) touchpoint dependency resolution is greedy by default.
  (i) explains why it doesn't fail in any other versions - this bundle is removed in 4.32
  (ii) explains why it doesn't fail with other installations.
• the said IU get upgraded by pulling compatible latest from external repo
• as a natural consequence, 285 IUs (in my case, it can be more for IDEs with more features installed) get refreshed.
• this triggers the busy activities around the registry cache.
• while one thread is actively removing items from the cache, the other thread attempting to resolve / refresh some other items may fail when attempting to first check if the item is already present in the cache. the cache is protected from cluttered access at JVM level using sync, but it doesn’t help on the top level logic flaw.
• the bundle-to-bundle dependency within the IU is not an issue, p2 is able to handle it.
• we came up with a working solution to mark touchpoint-to-o.e.o.s resolution non-greedy.

bottom line: the p2 registry cache corruption issue still persists in master, it is masked off due to absence of right recipe. I will come up with a clean repro and raise an issue.