Metrics Exporter: Distribution |explicit_buckets.bounds| does not have at least one entry

[cfranzen-dbg]

Metric Exporter is unable to handle Distribution metrices correctly that are collected via Micrometer instrumentation. For metrics like "jvm.gc.pause" typically the Distribtion does not have explicit bucket bounds, but the Stackdriver API is expecting at least one bound. You get a stacktrace like this:

com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: One or more TimeSeries could not be written: timeSeries[2] (metric.type="workload.googleapis.com/jvm.gc.pause", metric.labels={"service_name": "my-dummy-service", "instrumentation_version": "", "cause": "G1 Evacuation Pause", "instrumentation_source": "io.opentelemetry.micrometer-1.5", "gc": "G1 Young Generation", "action": "end of minor GC", "service_instance_id": "061d7963-a69f-4391-adc6-3cf5f139b13b"}): Field points[0].distributionValue had an invalid value: Distribution |explicit_buckets.bounds| does not have at least one entry.
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:92)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:98)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:84)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
	at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1307)
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1070)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:819)
	at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:568)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:569)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564)
	at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
	Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
		at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
		at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
		at com.google.cloud.monitoring.v3.MetricServiceClient.createTimeSeries(MetricServiceClient.java:1908)
		at com.google.cloud.monitoring.v3.MetricServiceClient.createTimeSeries(MetricServiceClient.java:1836)
		at com.google.cloud.opentelemetry.metric.CloudMetricClientImpl.createTimeSeries(CloudMetricClientImpl.java:40)
		at com.google.cloud.opentelemetry.metric.InternalMetricExporter.lambda$export$0(InternalMetricExporter.java:245)
		at com.google.cloud.opentelemetry.metric.InternalMetricExporter.createTimeSeriesBatch(InternalMetricExporter.java:276)
		at com.google.cloud.opentelemetry.metric.InternalMetricExporter.export(InternalMetricExporter.java:248)
		at com.google.cloud.opentelemetry.metric.GoogleCloudMetricExporter.export(GoogleCloudMetricExporter.java:90)
		at io.opentelemetry.sdk.metrics.export.PeriodicMetricReader$Scheduled.doRun(PeriodicMetricReader.java:167)
		at io.opentelemetry.sdk.metrics.export.PeriodicMetricReader$Scheduled.run(PeriodicMetricReader.java:153)
		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
		at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:305)
		at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java)
		at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
		... 3 common frames omitted

In think in the implementation of the Metrics Export there is some maturity missing, especially here: https://github.com/GoogleCloudPlatform/opentelemetry-operations-java/blob/main/exporters/metrics/src/main/java/com/google/cloud/opentelemetry/metric/MetricTranslator.java#L145

Compare it to the Stackdriver Exporter from the Micrometer project, there is lots of special handling for bucket bounds. Maybe you can learn from their implementation: https://github.com/micrometer-metrics/micrometer/blob/main/implementations/micrometer-registry-stackdriver/src/main/java/io/micrometer/stackdriver/StackdriverMeterRegistry.java#L505