fix(ci_visibility): unblock ddtrace import on Python 3.15 for dd_coverage#18321
fix(ci_visibility): unblock ddtrace import on Python 3.15 for dd_coverage#18321gnufede wants to merge 10 commits into
Conversation
…rage Replace the module-level NotImplementedError guards in the bytecode-wrapping subsystem with no-op stubs so that ddtrace can be imported on Python 3.15 without crashing. Wrapping functionality remains intentionally unimplemented until #17849 (wrapping context for 3.15) lands. Changes: - ddtrace/internal/wrapping/asyncs.py: pass + wrap_async no-op on >=3.15 - ddtrace/internal/wrapping/generators.py: pass + wrap_generator no-op on >=3.15 - ddtrace/internal/wrapping/context.py: pass + CONTEXT_RETURN_CONST stub + _UniversalWrappingContext.wrap/unwrap no-ops on >=3.15 - ddtrace/internal/bytecode_injection/__init__.py: pass on >=3.15 (HookType type alias still works; inject/eject not used by the sys.monitoring path) - tests/conftest.py: gate auto_enable_crashtracking on crashtracking.is_available (crashtracker feature is intentionally excluded from the 3.15 build in setup.py) Verified: all 85 dd_coverage tests pass on Python 3.15-dev. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
gnufede
added
the
changelog/no-changelog
|
Codeowners resolved as |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1238661cdb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if PY >= (3, 15): | ||
| raise NotImplementedError("Python >= 3.15 is not supported yet") | ||
| pass # AIDEV-TODO: Populate assembly for 3.15 bytecode (tracked in #17849). |
There was a problem hiding this comment.
When PY >= (3, 15), this leaves INJECTION_ASSEMBLY and _INJECT_HOOK_OPCODES empty, but inject_hook()/inject_hooks() still splice that empty assembly and report success. Callers such as SCA's Instrumenter.instrument() mark a target instrumented immediately after inject_hook(...), so on Python 3.15 SCA/debugging probes can be reported as installed while no hook will ever fire. Please gate or fail the injection APIs on unsupported versions instead of silently succeeding.
Useful? React with 👍 / 👎.
| def wrap_async(instrs: list[bc.Instr], code: CodeType, lineno: int) -> None: | ||
| if PY >= (3, 15): | ||
| # AIDEV-TODO: Async wrapping not yet implemented for 3.15; no-op until | ||
| # #17849 lands. Remove this guard once the assemblies above are populated. | ||
| return |
There was a problem hiding this comment.
On Python 3.15 this returns to wrap_bytecode() after it has already emitted the generic CALL_RETURN trampoline, and wrap() still preserves the original coroutine/async-generator flags. For async targets wrapped by integrations such as pymongo async methods, the resulting coroutine returns the wrapper coroutine object instead of awaiting it (and async generators similarly do not yield through the wrapper), so enabled instrumentation breaks rather than no-ops. Please make wrap() reject/skip unsupported async targets or keep the await/yield adapter.
Useful? React with 👍 / 👎.
The profiling Cython extensions (_lock, _task, etc.) are not built for Python 3.15 (setup.py gates them behind sys.version_info < (3, 15), tracked in #17817). Mirror that guard in smoke_test.py so the create-venv job doesn't fail before dd_coverage tests get a chance to run. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BenchmarksBenchmark execution time: 2026-06-01 15:31:53 Comparing candidate commit 5803f8d in PR branch Found 0 performance improvements and 5 performance regressions! Performance is the same for 615 metrics, 10 unstable metrics. scenario:iastaspects-lstrip_aspect
scenario:iastaspects-translate_aspect
scenario:iastaspectsospath-ospathbasename_aspect
scenario:span-start
scenario:tracer-small
|
pydantic==2.12.0a1 requires pydantic-core~=2.37.x, which bundles PyO3 0.25.1. PyO3 0.25.1 rejects Python 3.15 at build time with "maximum supported version (3.14)". The pydantic==2.13+ releases use pydantic-core 2.46.x (PyO3 0.28) which builds fine on 3.15. Testing an old pydantic alpha against Python 3.15 is not meaningful, so cap the affected sub-venvs at max_version="3.14". The unpinned pydantic-ai-slim==1.63.0 venv continues to cover 3.15 using the latest pydantic/pydantic-core. Pruned stale 3.15 lockfiles: 14a7a5f.txt, 1f96562.txt. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The IAST taint-tracking native extension (_taint_tracking._native, built with pybind11) is not compiled for Python 3.15+ (gated in setup.py behind sys.version_info < (3, 15)). Without it asm_config._iast_enabled stays False, so print_iast_report() returns early and the "Datadog Code Security Report" header never appears — causing both tests in this class to fail. Skip the class on 3.15+ with a clear reason pointing to the tracking issue. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
These were added before #17672 fixed dd_coverage for Python 3.14+. Since the sys.monitoring coverage instrumentation now works correctly on 3.14 and 3.15 (verified: 85/85 dd_coverage tests pass on 3.15), the skip condition is no longer valid. Slipcover incompatibility guard is kept as it remains relevant. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ng clash) On Python 3.14+, coverage.py switched from CTracer to SysMonitor which also uses sys.monitoring.COVERAGE_ID (tool 1). When the outer test session already holds that ID as "datadog", pytester.inline_run() cannot hand it to pytest-cov and raises ValueError: tool 1 is already in use. The tests cannot simply switch to runpytest_subprocess because their patch() mocks would not cross the process boundary. The proper fix is in ddtrace's coverage instrumentation (releasing COVERAGE_ID before spawning inline sub-sessions or detecting the clash gracefully); this skip is a temporary guard until that lands. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On Python 3.14+, coverage.py switched to SysMonitor which uses sys.monitoring.COVERAGE_ID (tool 1) — the same slot ddtrace claims. When an outer test session already holds the slot as "datadog", pytester.inline_run(--cov) inside that session would crash with "ValueError: tool 1 is already in use" in coverage.py's SysMonitor.start(). The existing reset_coverage_state() call in pytest_sessionstart already handles Python-level state resets for the inline_run case. Add the symmetric sys.monitoring release: - instrumentation_py3_12.py: deregister_monitoring() frees COVERAGE_ID (clears callback, calls free_tool_id, empties _CODE_HOOKS) - instrumentation_py3_16.py: no-op stub (not yet implemented for 3.16) - instrumentation.py: dispatch deregister_monitoring alongside instrument_all_lines; no-op stubs for Python < 3.12 - _plugin_v2.py: call deregister_monitoring() in pytest_sessionstart so that each new inline sub-session releases the slot before pytest-cov can claim it; instrument_all_lines() re-registers lazily on the next module import Reverts the temporary skip on test_pytest_coverage_report_upload.py that was added while this root-cause fix was pending. Verified: 85/85 dd_coverage tests still pass on Python 3.15. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…abled
Tighten the sys.monitoring deregistration to only fire when the new
session is actually using pytest-cov (--cov / --cov=<src>).
Registration order analysis:
- pytest-cov: eagerly registers COVERAGE_ID in pytest_load_initial_conftests
- ddtrace: lazily registers COVERAGE_ID in instrument_all_lines (during tests)
In a fresh process pytest-cov always wins the race: ddtrace's existing
"if get_tool() != 'datadog': return" guard in instrument_all_lines
then correctly yields and no conflict occurs.
The clash only arises when a previous inline sub-session left us holding
COVERAGE_ID ('datadog') and the next session's pytest-cov tries to claim
it in pytest_load_initial_conftests.
By gating on _is_pytest_cov_enabled() we avoid touching COVERAGE_ID in
pure --ddtrace sessions, which preserves our coverage tracking when
pytest-cov is absent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The same sys.monitoring.COVERAGE_ID clash that can occur in the CI Visibility v2 plugin also affects the ddtrace.testing v3 plugin: if an outer v3-plugin session claims COVERAGE_ID as "datadog" and then spawns a pytest sub-session via pytester.inline_run(--cov), the inner session's pytest-cov cannot claim the slot. Apply the same targeted fix: deregister_monitoring() only when _is_pytest_cov_enabled() is True, so the slot is released before pytest-cov's pytest_load_initial_conftests runs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
This pull request has been automatically closed after a period of inactivity. |
1 participant
Description
Part of the Python 3.15 integration parity effort (parent tracker: #17809).
Closes #17842
Branches off
vlad/enable-py-315-repo-wide(#18146) which provides therepo-wide 3.15 infrastructure (SUPPORTED_PYTHON_VERSIONS, lockfiles, CI
matrices). This PR adds the dd_coverage-specific fixes on top.
Investigation result: the dd_coverage coverage instrumentation code
already supports Python 3.15 (added in #17672: CACHE opcode handling,
PEP 810 IMPORT_NAME arg shift, threading.local ContextVar fallback). The
only blockers were import-time
NotImplementedErrorguards in thebytecode-wrapping subsystem — unrelated to coverage — that prevented
ddtrace from being imported at all on Python 3.15.
Changes
Wrapping subsystem no-ops (temporary, until #17849 lands)
ddtrace/internal/wrapping/asyncs.pyraise→pass;wrap_asyncreturns early on ≥3.15ddtrace/internal/wrapping/generators.pywrap_generatorddtrace/internal/wrapping/context.pypass+CONTEXT_RETURN_CONSTstub +_UniversalWrappingContext.wrap/unwrapno-ops on ≥3.15ddtrace/internal/bytecode_injection/__init__.pypasson ≥3.15 (HookTypestill works;inject/ejectnot used by thesys.monitoringpath)All four changes are marked with
# AIDEV-TODO: ... (tracked in #17849)so they're easy to remove once #17849 provides full bytecode support.Crashtracking fixture
tests/conftest.py:auto_enable_crashtrackingnow gates oncrashtracking.is_availableas well as Linux.setup.pyintentionally excludes the crashtracker Rust feature on Python ≥3.15, sois_available=False→ the autouse fixture skipped the assert that was blocking every test.Checklist
max_versioncap already lifted in fix(ci_visibility): fix coverage instrumentation for Python 3.13+ #17672changelog/no-changeloglabel (test/CI-only change)Testing
Run locally with
python3.15 -m pytest --no-cov tests/coverage -s -qon Python 3.15.0b1+ (pyenv3.15-dev).Risks
passstubs make async/generator wrapping and bytecode injectionsilent no-ops on Python 3.15. This was already the effective behaviour
(the
raiseprevented any use), so no regression. The# AIDEV-TODOmarkers ensure chore: wrapping context support for Python 3.15 #17849 will clean these up.
the native crashtracker wasn't compiled in, instead of asserting False on
every test.
Additional Notes
Once #18146 (base branch) merges to main this PR can be rebased onto main
with no conflicts — the diff is entirely additive on top of vlad's changes.