Support notarization of MAC dmg files in eclipse-dmg-packager plugin

[HannesWell]

Currently the eclipse-dmg-packager CBI plugin can only sign but not notarize the built package, which requires an additional subsequent step, for example in the eclipse-platform:

• https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/f3e4494394dcd3e8ee83cd57899549d1058e2b48/cje-production/mbscripts/mb300_gatherEclipseParts.sh#L79-L85
• https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/f3e4494394dcd3e8ee83cd57899549d1058e2b48/cje-production/scripts/notarizeMacApp.sh

It would be great if the eclipse-dmg-packager CBI plugin could already do it when packing a dmg file out of the box.
It would simplify the build configuration and could increase parallelism.

This could be enabled via a notarize property, similar to the already existing sign property:

org.eclipse.cbi/maven-plugins/eclipse-dmg-packager/src/main/java/org/eclipse/cbi/maven/plugins/dmgpackager/CreateDMGMojo.java

Lines 169 to 175 in b332cc9

	/**
	* Controls signing of the dmg file
	*
	* @since 1.1.4
	*/
	@Parameter(property = "cbi.dmgpackager.sign", defaultValue = "false")
	private boolean sign;

If you are fine with the proposed solution I can offer to work on implementing it.

But maybe it would be even better to make it possible to handle it directly in the dmg-packer back-end so that we don't have to download the dmg package and upload it again just for notarization.

[mbarbero]

Thank you, @HannesWell, for raising this.

I understand that it would be more convenient if signing and notarization could be handled together. If you are willing to propose a solution, I’m ready to review it.

The notarization step is quite complex and can also be quite long. As you've seen from https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/f3e4494394dcd3e8ee83cd57899549d1058e2b48/cje-production/scripts/notarizeMacApp.sh, it requires polling because the Apple tooling that the backend calls is designed this way. Also, Apple has never provided a high SLA for notarization. IIRC, at least at the beginning of mandatory notarization, they were saying that once an application was submitted for notarization, it could take up to 24 hours for the process to complete. That never really seems to have been the case, but it’s important to keep in mind that notarization is not a fixed-time process.

Note that when the CBI notarization service was created, Apple had no public API available to submit an app for notarization (we had to use the macOS-only CLI xcrun notarytool). Since then, they have provided publicly supported REST endpoints: https://developer.apple.com/documentation/notaryapi/submitting-software-for-notarization-over-the-web.

I think that if we add support for notarization directly in the DMG packager maven plugin, it could be beneficial to call the Apple API directly rather than the Eclipse notarization “proxy”. We can issue API keys for the App Store Connect API to projects that request them. If the "Java wrapper" supporting the Apple notarization API could be made reusable, in its own JAR, that would be even better.

WDYT?

[HannesWell]

That all sounds good and I'll try to implement this. But it might take a bit until I have sufficient time for this particular task (as it's now larger than initially expected).

it requires polling because the Apple tooling that the backend calls is designed this way.

OK. Do I understand it correctly, that the notarization state is stored at Apple and we don't get anything back (like a signature/certificate) except for the state of the process? If that's correct, I suggest to have two modes:

1. Block/wait within the mojo until the notarization process has completed.
2. provide a call-back (e.g. in form of a script) that can be called later and returns if the notarization has succeeded, is still in progress or has failed.

With the later, e.g. the Eclipse platform I-build could perform other tasks in the meantime instead of just waiting for that process to complete. I.e. it would just have to check the notarization state at the very end.

[mbarbero]

That all sounds good and I'll try to implement this. But it might take a bit until I have sufficient time for this particular task (as it's now larger than initially expected).

👍 Feel free to share things early.

it requires polling because the Apple tooling that the backend calls is designed this way.

OK. Do I understand it correctly, that the notarization state is stored at Apple and we don't get anything back (like a signature/certificate) except for the state of the process?

When notarizing, you also have the option to “staple” the notarization ticket to the app (this prevents the client where the app is installed from having to make a remote call to Apple’s servers to verify the notarization state). We do this by default because it’s simply good practice. Of course, to be able to staple a ticket, the notarization step must be completed first. So one needs to wait before stapling can happen.

If that's correct, I suggest to have two modes:

1. Block/wait within the mojo until the notarization process has completed.
2. provide a call-back (e.g. in form of a script) that can be called later and returns if the notarization has succeeded, is still in progress or has failed.

With the later, e.g. the Eclipse platform I-build could perform other tasks in the meantime instead of just waiting for that process to complete. I.e. it would just have to check the notarization state at the very end.

I understand the idea and the benefit, but in my opinion introducing asynchronous steps into the build adds considerable complexity and risks future debugging headaches.

Finally, I wanted to mention that there is a macOS independent notary tool implemented in Rust (https://gregoryszorc.com/docs/apple-codesign/main/index.html) that does all of this. Wrapping this tool in the dmg-packager-plugin is also an option rather than implementing everything in pure Java. @netomi started investigating using it for the signing backend in order to be able to deploy the service on Linux machines #642