Avoid holding frames captured by another thread

When tracing starts in one thread, we use `PyThreadState_GetFrame` to
capture the current Python call stack in every other thread, so that we
can associate allocations made later in pre-existing threads with the
correct call stack.

Unfortunately, our approach doesn't play nicely with trace functions,
like those installed by Coverage. The stack we get by walking backwards
from `PyThreadState_GetFrame` sees call frames that a profile function
never sees (because `PyThreadState_GetFrame` gets frames that correspond
to calls to trace functions, but profile functions don't get called for
calls into trace functions). Because of this, we wind up referring to
frames even after they've been popped off the call stack. Because we're
holding borrowed references, this results in a use-after-free, rather
than just incorrect stacks.

We could hold owned references instead of borrowed references. That
fixes the problem of use-after-free, though we'd still report incorrect
stacks. Unfortunately, it's not easy to do, as there are times when we
need to drop those references while the GIL isn't held and we're not
able to decrement the reference count.

We can't easily trim frames off the `PyThreadState_GetFrame` to make it
match the stack that profile functions see, either. That would require
us to be able to recognize frames that correspond to calls to trace
functions. That's easy if the thread never changes its trace function,
but it's impossible in the general case where the trace function
installs a different trace function.

The approach taken by this commit is that, instead of referencing the
frame objects captured when tracking started, we instead copy
information out of them and reference that information when allocations
occur. Later, when our profile function runs in that thread, we switch
to our normal shadow stack approach, using `PyEval_GetFrame` to populate
it from a place where we know there can be no trace function calls on
the stack. This sidesteps the use-after-free problem, but it still means
that we'll misreport stacks until the first time our profile function
gets called in any thread that already existed when the tracking session
started. Notably, because the profile function doesn't get called when
changing from one line to another within a Python function, we'll report
incorrect line numbers for allocations made in background threads until
the first Python function call or return on that thread.

Signed-off-by: Matt Wozniski <[email protected]>

Author: godlygeek

news/823.bugfix.rst

@@ -0,0 +1 @@
+Fix a crash that could occur if tracking was started in one thread while another thread was inside of a trace function installed with ``sys.settrace``. This crash wasn't possible to hit with ``memray run``, but could happen when using ``pytest-memray`` and ``pytest-cov`` together.