How to configure OpenTelemetry? #44956

dhoffer · 2024-12-05T20:49:08Z

dhoffer
Dec 5, 2024

I have configured OpenTelemetry as explained here: https://quarkus.io/guides/observability-devservices-lgtm#access-grafana

But at runtime I get a lot of errors, see below. I'm using Quarkus 3.17.3 on Windows using Quarkus Dev mode. I do not find grafana in the dev mode UI.

2024-12-05 13:46:11,243 WARNING [io.qua.ope.run.exp.otl.sen.VertxGrpcSender] (vert.x-eventloop-thread-7) Failed to export LogsRequestMarshalers. The request could not be executed. Full error message: Connection refused: getsockopt: localhost/127.0.0.1:4317
2024-12-05 13:46:11,243 SEVERE [io.ope.exp.int.grp.GrpcExporter] (vert.x-eventloop-thread-7) Failed to export logs. The request could not be executed. Error message: Connection refused: getsockopt: localhost/127.0.0.1:4317: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: getsockopt: localhost/127.0.0.1:4317
Suppressed: java.lang.IllegalStateException: Retries exhausted: 3/3
at io.smallrye.mutiny.helpers.ExponentialBackoff$1.lambda$apply$0(ExponentialBackoff.java:46)
at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
at io.smallrye.mutiny.groups.MultiOnItem.lambda$transformToUni$6(MultiOnItem.java:268)
at io.smallrye.mutiny.operators.multi.MultiConcatMapOp$MainSubscriber.onItem(MultiConcatMapOp.java:117)
at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drainWithDownstream(UnicastProcessor.java:108)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drain(UnicastProcessor.java:139)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.onNext(UnicastProcessor.java:205)
at io.smallrye.mutiny.operators.multi.processors.SerializedProcessor.onNext(SerializedProcessor.java:104)
at io.smallrye.mutiny.subscription.SerializedSubscriber.onItem(SerializedSubscriber.java:74)
at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
at io.smallrye.mutiny.operators.multi.MultiRetryWhenOp$RetryWhenOperator.onFailure(MultiRetryWhenOp.java:127)
at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
at io.smallrye.mutiny.converters.uni.UniToMultiPublisher$UniToMultiSubscription.onFailure(UniToMultiPublisher.java:104)
at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.dispatch(UniOnFailureFlatMap.java:85)
at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.onFailure(UniOnFailureFlatMap.java:60)
at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:60)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
at io.vertx.core.Future.lambda$toCompletionStage$3(Future.java:604)
at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.future.Mapping.onFailure(Mapping.java:45)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.http.impl.HttpClientImpl.lambda$doRequest$4(HttpClientImpl.java:398)
at io.vertx.core.net.impl.pool.Endpoint.lambda$getConnection$0(Endpoint.java:52)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:162)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:123)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:342)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335)
at io.vertx.core.net.impl.pool.SimpleConnectionPool$ConnectFailed$1.run(SimpleConnectionPool.java:380)
at io.vertx.core.net.impl.pool.Task.runNextTasks(Task.java:43)
at io.vertx.core.net.impl.pool.CombinerExecutor.submit(CombinerExecutor.java:91)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.execute(SimpleConnectionPool.java:244)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.lambda$connect$2(SimpleConnectionPool.java:258)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint.lambda$connect$2(SharedClientHttpStreamEndpoint.java:104)
at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.future.Composition$1.onFailure(Composition.java:66)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FailedFuture.addListener(FailedFuture.java:98)
at io.vertx.core.impl.future.Composition.onFailure(Composition.java:55)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:342)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335)
at io.vertx.core.net.impl.NetClientImpl.failed(NetClientImpl.java:351)
at io.vertx.core.net.impl.NetClientImpl.lambda$connectInternal2$6(NetClientImpl.java:323)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:110)
at io.vertx.core.net.impl.ChannelProvider.lambda$handleConnect$0(ChannelProvider.java:157)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:326)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:342)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: getsockopt: localhost/127.0.0.1:4317
Suppressed: [CIRCULAR REFERENCE: java.lang.IllegalStateException: Retries exhausted: 3/3]
Caused by: java.net.ConnectException: Connection refused: getsockopt
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:973)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: [CIRCULAR REFERENCE: java.net.ConnectException: Connection refused: getsockopt]

2024-12-05 13:46:26,619 WARN [io.mic.reg.otl.OtlpMeterRegistry] (otlp-metrics-publisher) Failed to publish metrics to OTLP receiver (context: url=http://localhost:4318/v1/metrics, resource-attributes={}): java.net.ConnectException: Connection refused: getsockopt
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
at java.base/java.net.Socket.connect(Socket.java:751)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:178)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:531)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:636)
at java.base/sun.net.www.http.HttpClient.(HttpClient.java:280)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1304)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1237)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1123)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1052)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1446)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1417)
at io.micrometer.core.ipc.http.HttpUrlConnectionSender.send(HttpUrlConnectionSender.java:98)
at io.micrometer.core.ipc.http.HttpSender$Request$Builder.send(HttpSender.java:306)
at io.micrometer.registry.otlp.OtlpMeterRegistry.publish(OtlpMeterRegistry.java:172)
at io.micrometer.core.instrument.push.PushMeterRegistry.publishSafelyOrSkipIfInProgress(PushMeterRegistry.java:64)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)

2024-12-05 13:46:53,368 WARNING [io.qua.ope.run.exp.otl.sen.VertxGrpcSender] (vert.x-eventloop-thread-8) Failed to export MetricsRequestMarshalers. The request could not be executed. Full error message: Connection refused: getsockopt: localhost/127.0.0.1:4317
2024-12-05 13:46:53,368 SEVERE [io.ope.exp.int.grp.GrpcExporter] (vert.x-eventloop-thread-8) Failed to export metrics. The request could not be executed. Error message: Connection refused: getsockopt: localhost/127.0.0.1:4317: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: getsockopt: localhost/127.0.0.1:4317
Suppressed: java.lang.IllegalStateException: Retries exhausted: 3/3
at io.smallrye.mutiny.helpers.ExponentialBackoff$1.lambda$apply$0(ExponentialBackoff.java:46)
at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
at io.smallrye.mutiny.groups.MultiOnItem.lambda$transformToUni$6(MultiOnItem.java:268)
at io.smallrye.mutiny.operators.multi.MultiConcatMapOp$MainSubscriber.onItem(MultiConcatMapOp.java:117)
at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drainWithDownstream(UnicastProcessor.java:108)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drain(UnicastProcessor.java:139)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.onNext(UnicastProcessor.java:205)
at io.smallrye.mutiny.operators.multi.processors.SerializedProcessor.onNext(SerializedProcessor.java:104)
at io.smallrye.mutiny.subscription.SerializedSubscriber.onItem(SerializedSubscriber.java:74)
at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
at io.smallrye.mutiny.operators.multi.MultiRetryWhenOp$RetryWhenOperator.onFailure(MultiRetryWhenOp.java:127)
at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
at io.smallrye.mutiny.converters.uni.UniToMultiPublisher$UniToMultiSubscription.onFailure(UniToMultiPublisher.java:104)
at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.dispatch(UniOnFailureFlatMap.java:85)
at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.onFailure(UniOnFailureFlatMap.java:60)
at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:60)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
at io.vertx.core.Future.lambda$toCompletionStage$3(Future.java:604)
at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.future.Mapping.onFailure(Mapping.java:45)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.http.impl.HttpClientImpl.lambda$doRequest$4(HttpClientImpl.java:398)
at io.vertx.core.net.impl.pool.Endpoint.lambda$getConnection$0(Endpoint.java:52)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:162)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:123)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:342)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335)
at io.vertx.core.net.impl.pool.SimpleConnectionPool$ConnectFailed$1.run(SimpleConnectionPool.java:380)
at io.vertx.core.net.impl.pool.Task.runNextTasks(Task.java:43)
at io.vertx.core.net.impl.pool.CombinerExecutor.submit(CombinerExecutor.java:91)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.execute(SimpleConnectionPool.java:244)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.lambda$connect$2(SimpleConnectionPool.java:258)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint.lambda$connect$2(SharedClientHttpStreamEndpoint.java:104)
at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.future.Composition$1.onFailure(Composition.java:66)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FailedFuture.addListener(FailedFuture.java:98)
at io.vertx.core.impl.future.Composition.onFailure(Composition.java:55)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:342)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335)
at io.vertx.core.net.impl.NetClientImpl.failed(NetClientImpl.java:351)
at io.vertx.core.net.impl.NetClientImpl.lambda$connectInternal2$6(NetClientImpl.java:323)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:110)
at io.vertx.core.net.impl.ChannelProvider.lambda$handleConnect$0(ChannelProvider.java:157)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:326)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:342)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: getsockopt: localhost/127.0.0.1:4317
Suppressed: [CIRCULAR REFERENCE: java.lang.IllegalStateException: Retries exhausted: 3/3]
Caused by: java.net.ConnectException: Connection refused: getsockopt
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:973)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: [CIRCULAR REFERENCE: java.net.ConnectException: Connection refused: getsockopt]

2024-12-05 13:48:27,615 WARNING [io.qua.ope.run.exp.otl.sen.VertxGrpcSender] (vert.x-eventloop-thread-7) Failed to export LogsRequestMarshalers. The request could not be executed. Full error message: Connection refused: getsockopt: localhost/127.0.0.1:4317
2024-12-05 13:48:27,616 SEVERE [io.ope.exp.int.grp.GrpcExporter] (vert.x-eventloop-thread-7) Failed to export logs. The request could not be executed. Error message: Connection refused: getsockopt: localhost/127.0.0.1:4317: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: getsockopt: localhost/127.0.0.1:4317
Suppressed: java.lang.IllegalStateException: Retries exhausted: 3/3
at io.smallrye.mutiny.helpers.ExponentialBackoff$1.lambda$apply$0(ExponentialBackoff.java:46)
at io.smallrye.context.impl.wrappers.SlowContextualFunction.apply(SlowContextualFunction.java:21)
at io.smallrye.mutiny.groups.MultiOnItem.lambda$transformToUni$6(MultiOnItem.java:268)
at io.smallrye.mutiny.operators.multi.MultiConcatMapOp$MainSubscriber.onItem(MultiConcatMapOp.java:117)
at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drainWithDownstream(UnicastProcessor.java:108)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.drain(UnicastProcessor.java:139)
at io.smallrye.mutiny.operators.multi.processors.UnicastProcessor.onNext(UnicastProcessor.java:205)
at io.smallrye.mutiny.operators.multi.processors.SerializedProcessor.onNext(SerializedProcessor.java:104)
at io.smallrye.mutiny.subscription.SerializedSubscriber.onItem(SerializedSubscriber.java:74)
at io.smallrye.mutiny.subscription.MultiSubscriber.onNext(MultiSubscriber.java:61)
at io.smallrye.mutiny.operators.multi.MultiRetryWhenOp$RetryWhenOperator.onFailure(MultiRetryWhenOp.java:127)
at io.smallrye.mutiny.subscription.MultiSubscriber.onError(MultiSubscriber.java:73)
at io.smallrye.mutiny.converters.uni.UniToMultiPublisher$UniToMultiSubscription.onFailure(UniToMultiPublisher.java:104)
at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.dispatch(UniOnFailureFlatMap.java:85)
at io.smallrye.mutiny.operators.uni.UniOnFailureFlatMap$UniOnFailureFlatMapProcessor.onFailure(UniOnFailureFlatMap.java:60)
at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:60)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
at io.vertx.core.Future.lambda$toCompletionStage$3(Future.java:604)
at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.future.Mapping.onFailure(Mapping.java:45)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.http.impl.HttpClientImpl.lambda$doRequest$4(HttpClientImpl.java:398)
at io.vertx.core.net.impl.pool.Endpoint.lambda$getConnection$0(Endpoint.java:52)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:162)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint$Request.handle(SharedClientHttpStreamEndpoint.java:123)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:342)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335)
at io.vertx.core.net.impl.pool.SimpleConnectionPool$ConnectFailed$1.run(SimpleConnectionPool.java:380)
at io.vertx.core.net.impl.pool.Task.runNextTasks(Task.java:43)
at io.vertx.core.net.impl.pool.CombinerExecutor.submit(CombinerExecutor.java:91)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.execute(SimpleConnectionPool.java:244)
at io.vertx.core.net.impl.pool.SimpleConnectionPool.lambda$connect$2(SimpleConnectionPool.java:258)
at io.vertx.core.http.impl.SharedClientHttpStreamEndpoint.lambda$connect$2(SharedClientHttpStreamEndpoint.java:104)
at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.future.Composition$1.onFailure(Composition.java:66)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FailedFuture.addListener(FailedFuture.java:98)
at io.vertx.core.impl.future.Composition.onFailure(Composition.java:55)
at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:342)
at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:335)
at io.vertx.core.net.impl.NetClientImpl.failed(NetClientImpl.java:351)
at io.vertx.core.net.impl.NetClientImpl.lambda$connectInternal2$6(NetClientImpl.java:323)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
at io.netty.util.concurrent.DefaultPromise.setFailure(DefaultPromise.java:110)
at io.vertx.core.net.impl.ChannelProvider.lambda$handleConnect$0(ChannelProvider.java:157)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:326)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:342)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: getsockopt: localhost/127.0.0.1:4317
Suppressed: [CIRCULAR REFERENCE: java.lang.IllegalStateException: Retries exhausted: 3/3]
Caused by: java.net.ConnectException: Connection refused: getsockopt
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:973)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: [CIRCULAR REFERENCE: java.net.ConnectException: Connection refused: getsockopt]

Lot more errors...

@brunobat · 2024-12-05T20:49:10Z

quarkus-bot[bot]
bot Dec 5, 2024

/cc @brunobat (opentelemetry), @radcortez (config,opentelemetry)

0 replies

radcortez · 2024-12-05T22:11:09Z

radcortez
Dec 5, 2024
Collaborator

From the log, it does seem that the exporter cannot connect to the service, but we cannot assess much more.

Can you provide a small reproducer and/or full logs of your application? Do you see the DevService starting?

1 reply

brunobat Dec 7, 2024
Collaborator

The stack trace is due to the lack of a collector. It is "harmless" and was fixed, but not released yet here: #44931

This message is weird:
Failed to export LogsRequestMarshalers. The request could not be executed. Full error message: Connection refused: getsockopt: localhost/127.0.0.1:4317
because the endpoint getsockopt: localhost/127.0.0.1:4317 doesn´t make sense.
I wonder if this is windows related... Will try later on a windows machine.

dhoffer · 2024-12-06T00:09:16Z

dhoffer
Dec 6, 2024
Author

@radcortez Sure I have attached the log after running for a while. Yes, I can get to Dev UI http://localhost:8080/q/dev-ui/extensions

quarkus-console.log

2 replies

radcortez Dec 6, 2024
Collaborator

There should be something like [io.qua.obs.tes.LgtmContainer]] (build-133) Starting LGTM ... in the logs, but I can't find it. Either the logs are not complete, or the dev services didn't start.

I've also noticed that you have on the logs:
Unrecognized configuration key "quarkus.otel.tracing.enabled" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo

But that is not the right property. Please check: https://quarkus.io/guides/opentelemetry#configuration-reference

brunobat Dec 7, 2024
Collaborator

There is no need to activate tracing. It's on by default.

dhoffer · 2024-12-06T18:26:47Z

dhoffer
Dec 6, 2024
Author

@radcortez You were right we had DevService disabled. With that enabled the grafana report does load/display but its failing to gather data. Here is the current log file.

The UI says:
Failed to fetch

&

Templating
Template variable service failed Failed to fetch

quarkus-console.log

Can you tell what is wrong?

Here are my config options.

quarkus.otel.attribute.count.limit=128
quarkus.otel.blrp.export.timeout=10S
quarkus.otel.blrp.max.export.batch.size=512
quarkus.otel.blrp.max.queue.size=2048
quarkus.otel.blrp.schedule.delay=1S
quarkus.otel.bsp.export.timeout=10S
quarkus.otel.bsp.max.export.batch.size=512
quarkus.otel.bsp.schedule.delay=5S
quarkus.otel.enabled=true
quarkus.otel.experimental.shutdown-wait-time=2S
quarkus.otel.exporter.otlp.compression=none
quarkus.otel.exporter.otlp.endpoint=http://localhost:4317
quarkus.otel.exporter.otlp.logs.compression=none
quarkus.otel.exporter.otlp.logs.endpoint=http://localhost:4317/
quarkus.otel.exporter.otlp.logs.protocol=grpc
quarkus.otel.exporter.otlp.logs.timeout=10S
quarkus.otel.exporter.otlp.metrics.compression=none
quarkus.otel.exporter.otlp.metrics.endpoint=http://localhost:4317/
quarkus.otel.exporter.otlp.metrics.protocol=grpc
quarkus.otel.exporter.otlp.metrics.temporality-preference=cumulative
quarkus.otel.exporter.otlp.metrics.timeout=10S
quarkus.otel.exporter.otlp.protocol=grpc
quarkus.otel.exporter.otlp.timeout=10S
quarkus.otel.exporter.otlp.traces.compression=none
quarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317/
quarkus.otel.exporter.otlp.traces.protocol=grpc
quarkus.otel.exporter.otlp.traces.timeout=10S
quarkus.otel.instrument.grpc=true
quarkus.otel.instrument.http-server-metrics=true
quarkus.otel.instrument.jvm-metrics=true
quarkus.otel.instrument.messaging=true
quarkus.otel.instrument.rest=true
quarkus.otel.instrument.vertx-event-bus=true
quarkus.otel.instrument.vertx-http=true
quarkus.otel.instrument.vertx-redis-client=true
quarkus.otel.instrument.vertx-sql-client=true
quarkus.otel.logs.enabled=true
quarkus.otel.logs.exporter=cdi
quarkus.otel.logs.handler.enabled=true
quarkus.otel.max.queue.size=2048
quarkus.otel.metrics.enabled=true
quarkus.otel.metrics.export.interval=10S
quarkus.otel.metrics.exporter=cdi
quarkus.otel.mp.compatibility=false
quarkus.otel.propagators=tracecontext,baggage
quarkus.otel.sdk.disabled=false
quarkus.otel.security-events.enabled=true
quarkus.otel.security-events.event-types=all
quarkus.otel.simple=false
quarkus.otel.span.attribute.count.limit=128
quarkus.otel.span.event.count.limit=128
quarkus.otel.span.link.count.limit=128
quarkus.otel.traces.arg=1.0d
quarkus.otel.traces.eusp.enabled=true
quarkus.otel.traces.exporter=cdi
quarkus.otel.traces.include-static-resources=true
quarkus.otel.traces.sampler=parentbased_always_on
quarkus.otel.traces.suppress-non-application-uris=false

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to configure OpenTelemetry? #44956

Uh oh!

{{title}}

Uh oh!

Replies: 4 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

How to configure OpenTelemetry? #44956

Uh oh!

dhoffer Dec 5, 2024

Replies: 4 comments · 3 replies

Uh oh!

quarkus-bot[bot] bot Dec 5, 2024

Uh oh!

radcortez Dec 5, 2024 Collaborator

Uh oh!

brunobat Dec 7, 2024 Collaborator

Uh oh!

Uh oh!

dhoffer Dec 6, 2024 Author

Uh oh!

radcortez Dec 6, 2024 Collaborator

Uh oh!

brunobat Dec 7, 2024 Collaborator

Uh oh!

Uh oh!

dhoffer Dec 6, 2024 Author

dhoffer
Dec 5, 2024

Replies: 4 comments 3 replies

quarkus-bot[bot]
bot Dec 5, 2024

radcortez
Dec 5, 2024
Collaborator

brunobat Dec 7, 2024
Collaborator

dhoffer
Dec 6, 2024
Author

radcortez Dec 6, 2024
Collaborator

brunobat Dec 7, 2024
Collaborator

dhoffer
Dec 6, 2024
Author