API tools considers nested, noreference classes as public API in baseline

[HeikoKlare]

In Eclipse SWT, we faced the issue that when switching to the 4.38 development stream, some classes were erroneously reported as removed from API even though they were marked as @noreference before. The according comments can be found starting here: eclipse-platform/eclipse.platform.swt#2467

Errors reported looked like this:

Eclipse SWT/common/org/eclipse/swt/graphics/Point.java:133 The constructor org.eclipse.swt.graphics.Point.OfFloat.OfFloat(int, int) is no longer an API

Some specialities of this situation are:

• We have a host/fragment setup with the fragments using Platform filters
• The package of the classes is composed of multiple source folder
• The classes are nested, extend their outer classes and those outer classes are part of the public API

The exepected behavior is that classes (no matter whether nested, in packages composed from multiple source folder or the like) that are marked as @noreference should not be taken into account for baseline comparison.

I have stripped down the baseline (SWT host + Win32 fragment) and the current bundles to the few necessary classes to reproduce the problem (and even made the Win32 fragment independent from Windows, so it can be used on any platform for reproduction): ws-apitooling.zip
You need to import the two projects into a clean workspace and set the baseline to the "p2" folder. After rebuilding, API errors for the classes and their methods Point$OfFloat, Point$WithMonitor, Rectangle$OfFloat and Rectangle$WithMonitor should be reported.

The issue seems to be caused by the .api_description file being incomplete. The restrictions of the nested classes are not written into that file. That conforms to what I found when debugging (see eclipse-platform/eclipse.platform.swt#2467 (comment)), where those classes were not part of the baseline. When removing that file from the artifacts in the p2 repo, the errors do not occur, but only incompatible changes to the classes will be properly reported as API errors.
Is this .api_description file even needed, given that the on-the-fly calculation seems to be more accurate?

[laeubi]

Is this .api_description file even needed, given that the on-the-fly calculation seems to be more accurate?

You are using javadoc comment style annotations that are only present in the source code, so how should API tools ever know from the classfile of an artifact what the baseline was defined of?

e.g. assume we have public class A and I later add @noreference then API tools need to complain that you are degrading A from API to a non-api type, but how to know that without API description? Or your case when you want to remove a type, API tools need know that it was marked as @norefernece before...

[HeikoKlare]

You are using javadoc comment style annotations that are only present in the source code, so how should API tools ever know from the classfile of an artifact what the baseline was defined of?

That indeed makes sense - didn't think about the class files not containing all necessary information (e.g., annotations with non-class rentention). Then in the described case the expectation should be that the information is persisted in the .api_description file.

[laeubi]

Then in the described case the expectation should be that the information is persisted in the .api_description file.

The description file is generated with this mojo here:

https://github.com/eclipse-tycho/tycho/blob/main/tycho-apitools-plugin/src/main/java/org/eclipse/tycho/apitools/ApiFileGenerationMojo.java

so I think the best way forward would be to check there why it is not included an either fix it in PDE or enhance Tycho.