-
Notifications
You must be signed in to change notification settings - Fork 32
Description
Version
4.5.15
Context
I am testing vert.x event bus otel tracing support and encountered following issue.
I have an application that uses vert.x event bus "publish" and "send" feature to transmit message, however the tracing broke when consumers handling messages from "publish".
Here is an example:
The publisher trace with trace id 04a3c068b66b60475dcb9ac82fab95f8, the publisher tracing is working fine.
17, 2025 8:36:41 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'publish' : 04a3c068b66b60475dcb9ac82fab95f8 2abf4cf3467787c8 CLIENT [tracer: io.vertx:] AttributesMap{data={messaging.destination.name=example.address.receiver.publish, messaging.operation=publish, message_bus.destination=example.address.receiver.publish, messaging.system=vertx-eventbus}, capacity=128, totalAddedValues=4}
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'send' : 04a3c068b66b60475dcb9ac82fab95f8 15e4df70618cd5a1 CLIENT [tracer: io.vertx:] AttributesMap{data={messaging.destination.name=example.address.receiver.reply, messaging.operation=publish, message_bus.destination=example.address.receiver.reply, messaging.system=vertx-eventbus}, capacity=128, totalAddedValues=4}
succeed io.vertx.core.eventbus.impl.clustered.ClusteredMessage@67538cc1
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : 04a3c068b66b60475dcb9ac82fab95f8 876aa62b83a32abf SERVER [tracer: io.vertx:] AttributesMap{data={http.scheme=http, http.method=GET, http.response.status_code=200, http.url=http://localhost:8079/async2, http.request.method=GET, url.path=/async2, http.status_code=200}, capacity=128, totalAddedValues=7}
The consumer trace, I turned on TracingPolicy.ALWAYS to illustrate the problem, some spans have the correct trace id 04a3c068b66b60475dcb9ac82fab95f8, some just created its own trace. Noticed that the server spans are all have the correct trace id, meaning context is propagated through event bus. The requests send to www.splunk.com/1, www.splunk.com/2, www.splunk.com/3 have the correct trace id, they are part of the 'send' event bus message consumer. However requests send to www.splunk.com/4, www.splunk.com/5, www.splunk.com/6 which are part of the 'publish' consumer, created their own trace.
INFO: 'publish' : 04a3c068b66b60475dcb9ac82fab95f8 cc82c35963507588 SERVER [tracer: io.vertx:] AttributesMap{data={messaging.system=vertx-eventbus, message_bus.destination=example.address.receiver.publish, messaging.destination.name=example.address.receiver.publish, messaging.operation=publish}, capacity=128, totalAddedValues=4}
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : 04a3c068b66b60475dcb9ac82fab95f8 0071d2cddd64af8b CLIENT [tracer: io.vertx:] AttributesMap{data={http.status_code=404, http.request.method=GET, http.url=https://www.splunk.com/1, http.method=GET, url.full=https://www.splunk.com/1, http.response.status_code=404}, capacity=128, totalAddedValues=6}
1
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : 1fa01345a2bd87b4e3f07a74c31125bc 147581edefa6218c CLIENT [tracer: io.vertx:] AttributesMap{data={http.status_code=404, http.request.method=GET, http.url=https://www.splunk.com/4, http.method=GET, url.full=https://www.splunk.com/4, http.response.status_code=404}, capacity=128, totalAddedValues=6}
4
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : 04a3c068b66b60475dcb9ac82fab95f8 081bb382bdb83096 CLIENT [tracer: io.vertx:] AttributesMap{data={http.status_code=404, http.request.method=GET, http.url=https://www.splunk.com/2, http.method=GET, url.full=https://www.splunk.com/2, http.response.status_code=404}, capacity=128, totalAddedValues=6}
2
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : 04a3c068b66b60475dcb9ac82fab95f8 c2ba0737acdb5595 CLIENT [tracer: io.vertx:] AttributesMap{data={http.status_code=404, http.request.method=GET, http.url=https://www.splunk.com/3, http.method=GET, url.full=https://www.splunk.com/3, http.response.status_code=404}, capacity=128, totalAddedValues=6}
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'send' : 04a3c068b66b60475dcb9ac82fab95f8 19b240a603a9df02 SERVER [tracer: io.vertx:] AttributesMap{data={messaging.system=vertx-eventbus, message_bus.destination=example.address.receiver.reply, messaging.destination.name=example.address.receiver.reply, messaging.operation=publish}, capacity=128, totalAddedValues=4}
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : 104107f16d56de77000b9ab83eb8ebc0 dc60db41e9c5cc73 CLIENT [tracer: io.vertx:] AttributesMap{data={http.status_code=404, http.request.method=GET, http.url=https://www.splunk.com/5, http.method=GET, url.full=https://www.splunk.com/5, http.response.status_code=404}, capacity=128, totalAddedValues=6}
5
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
INFO: 'GET' : a829126e271278364e8050973149e106 e41b6a2cd049d6e6 CLIENT [tracer: io.vertx:] AttributesMap{data={http.status_code=301, http.request.method=GET, http.url=https://www.splunk.com/6, http.method=GET, url.full=https://www.splunk.com/6, http.response.status_code=301}, capacity=128, totalAddedValues=6}
Jun 17, 2025 8:36:42 AM io.opentelemetry.exporter.logging.LoggingSpanExporter export
I have put a mini example to reproduce the issue.
Steps to reproduce
Instruction on the repo readme
https://github.com/pureklkl/min-vertx-event-bus-otel-example
Do you have a reproducer?
https://github.com/pureklkl/min-vertx-event-bus-otel-example