Generate Java classes from OTLP proto files

[felixbarny]

Downloads the latest OTLP protocol buffer release and generates Java classes from them.

Some noteworthy aspects of this:

• Manually downloads the proto release from GitHub and verifies the SHA-256
• The protobuf plugin needs a protoc version that's specific to the current platform. That makes it hard to generate all the verification metadata that's used on other platforms, like on CI or for other contributors. I've added a script to download and create the SHA-256 hash for all protoc platform variants.
• The generated file are not checked in to git but are generated on demand via Gradle. The integration into IntelliJ seems to work nicely after syncing the Gradle build.

Part of #133057

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[elasticsearchmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[felixbarny]

Hm, so one of the issues with this is that the :x-pack:plugin:otel-data:thirdPartyAudit task complains that protobuf uses sun.misc.Unsafe in the following classes:

Classes with violations:
  * com.google.protobuf.MessageSchema
  * com.google.protobuf.UnsafeUtil
  * com.google.protobuf.UnsafeUtil$1
  * com.google.protobuf.UnsafeUtil$Android32MemoryAccessor
  * com.google.protobuf.UnsafeUtil$Android64MemoryAccessor
  * com.google.protobuf.UnsafeUtil$JvmMemoryAccessor
  * com.google.protobuf.UnsafeUtil$MemoryAccessor

Is that something we'd be willing to make an exception for protobuf or should we look at alternatives?

[felixbarny]

Looks like there's precedent for ignoring these classes, so I did the same.

[breskeby]

We want avoid afterEvaluate as its kind of deprecated gradle api. I understand the external plugin relies on it.

[breskeby]

We are in the process of removing all usages of antBuilder in our build scripts as it has a couple of flaws and breaks new features in gradle.

[breskeby]

I will have a go on looking into simplifing the gradle logic here to match our generall gradle api usages and best practices.

[breskeby]

@felixbarny I approved the PR. We can address the rework of the build later on as this should not block you from merging imo

[felixbarny]

@breskeby there's one more challenge related to the :x-pack:plugin:otel-data:splitPackagesAudit task:

Caused by: org.gradle.api.GradleException: Unsupported classpath element: /opt/buildkite-agent/.gradle/caches/modules-2/files-2.1/com.google.protobuf/protoc/4.32.0/15132b0f5c33b2e3212543b264c8d1e73df636fb/protoc-4.32.0-linux-aarch_64.exe	
	at org.elasticsearch.gradle.internal.precommit.SplitPackagesAuditTask$SplitPackagesAuditAction.readPackages(SplitPackagesAuditTask.java:290)	
	at org.elasticsearch.gradle.internal.precommit.SplitPackagesAuditTask$SplitPackagesAuditAction.getDependencyPackages(SplitPackagesAuditTask.java:181)	
	at org.elasticsearch.gradle.internal.precommit.SplitPackagesAuditTask$SplitPackagesAuditAction.execute(SplitPackagesAuditTask.java:143)	
	at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:68)
	...

Do you have a suggestion on how to fix this? Some of the options I'm seeing are to disable the task or to change the implementation to ignore .exe file extensions instead of failing.

[felixbarny]

Ah, nvm, I think I fixed it by using the runtimeOnly scope instead of compileOnly.