-
Notifications
You must be signed in to change notification settings - Fork 10
Description
We need to implement integration for universal profiling in order to enable trace correlation. Differently than the java agent I'd like to implement most of the code in Python and only pass the data to the C extension to store it in the well known variables.
After playing a bit with implementing a python extension I think the issue is more on sorting out the life cycle of things more than in the coding itself. For the process wide data this is mostly static so just have to do it at startup, we need to add a thread (ThreadManager?) that handles the communication with the profiling agent via the shared socket. For the threading specific one need to sort out where to update the data, maybe there's some hook in the context handling?
Another requirements is to buffer the spans for samples-delay-ms as sent in the profiler registration message, in order to let the profiler send the corresponding traces ids via the messaging socket. That would require a way for the thread reading the socket to send data to its parent process (use a Queue to send data and then implement of a cache that the exporter can use?).
Open issues from the profiler side:
- profiler registration message still not implemented
- renaming after OTel acceptance if any
Open issues with upstream OTel
- Context change hook Callback on context change open-telemetry/opentelemetry-python#3915
Open issues from our side
- handle renaming if any
- sort out build and distributions with the robots team
- agree on the shared object built from the C extension file name
- handling of socket from thread
- when to update thread specific data
- how to pass data from thread reading socket
- hooks into batch span exporter to buffer spans for
samples-delay-ms