fix(di): dynamic function discovery fallback [backport 3.10] (#13999)

Backport b55f08f from #13947 to 3.10.

We implement a dynamic function discovery fallback when the
function-from-code resolution via the GC fails. This can happen if the
target application has interacted with the GC, e.g. by freezing it at a
time that will prevent the current discovery from being able to resolve
the function from the referenced code object.

## Testing Strategy

The original issue was reproducible with a local deployment of
[synapse](https://github.com/element-hq/synapse). The investigation led
to the conclusion that the issue was caused by the way the application
interacts with the GC
https://github.com/element-hq/synapse/blob/1dc29563c1504a2523e467aa7bef6a7ac05cc60c/synapse/app/_base.py#L623C1-L629C35.
Commenting out these lines makes the issue disappear. We have tested the
fix against the unmodified application to verify that the proposed fix
works.

Refs: [DYNIS-28](https://datadoghq.atlassian.net/browse/DYNIS-28)

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)


[DYNIS-28]:
https://datadoghq.atlassian.net/browse/DYNIS-28?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

Co-authored-by: Gabriele N. Tornetta <[email protected]>

Author: github-actions[bot]

ddtrace/debugging/_function/discovery.py

@@ -326,11 +326,43 @@ def at_line(self, line: int) -> List[FullyNamedFunction]:
 
         return []
 
+    def _resolve_pair(self, pair: _FunctionCodePair, fullname: str) -> FullyNamedFunction:
+        try:
+            return pair.resolve()
+        except ValueError as e:
+            # We failed to lookup the function by its code object via the GC.
+            # This should not happen, unless the target application is doing
+            # something unusual with the GC. In this case we try our best to
+            # look up the function dynamically from the module. If the function
+            # is decorated, there are chances this might fail.
+            target = (module := self._module)
+            part = None
+            for part in fullname.replace(f"{module.__name__}.", "", 1).split("."):
+                try:
+                    target = getattr(target, part)
+                except AttributeError:
+                    raise e
+
+            if target is module:
+                # We didn't move from the module so we don't have a function.
+                raise e
+
+            code = pair.code
+            assert code is not None  # nosec
+            f = undecorated(cast(FunctionType, target), cast(str, part), Path(code.co_filename).resolve())
+            if not (isinstance(f, FunctionType) and f.__code__ is code):
+                raise e
+
+            # Cache the lookup result
+            pair.function = f
+
+            return cast(FullyNamedFunction, f)
+
     def by_name(self, qualname: str) -> FullyNamedFunction:
         """Get the function by its qualified name."""
         fullname = f"{self._module.__name__}.{qualname}"
         try:
-            return self._fullname_index[fullname].resolve()
+            return self._resolve_pair(self._fullname_index[fullname], fullname)
         except ValueError:
             pass
         except KeyError:
@@ -342,11 +374,11 @@ def by_name(self, qualname: str) -> FullyNamedFunction:
                     if qualname == name or qualname.endswith(f".{name}"):
                         for fcp in list(fcps):
                             try:
-                                f = fcp.resolve()
+                                f = self._resolve_pair(fcp, fullname)
 
                                 # We have resolved the function so we can now
                                 # get its full name
-                                self._fullname_index[f"{self._module.__name__}.{f.__qualname__}"] = fcp
+                                self._fullname_index[fullname] = fcp
 
                                 # We can remove the entry from the name index
                                 fcps.pop(0)

releasenotes/notes/fix-di-dynamic-discovery-fallback-3a5623e18584cd79.yaml

@@ -0,0 +1,6 @@
+---
+fixes:
+  - |
+    dynamic instrumentation: improve support for function probes with frameworks
+    and applications that interact with the Python garbage collector (e.g.
+    synapse).