From 9ea93219432cfddfc9fc00af75a1c7e6d73cdd04 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Wed, 8 Jul 2026 14:49:43 +0200 Subject: [PATCH 1/9] Consolidate Claude config: shared vs personal split + project.md Add a portable, repo-side convention so each developer can keep personal Claude config alongside the shared team config without committing it, and fold project-level knowledge into the committed common config. - .claude/.gitignore: ignore per-developer paths (personal/, commands/, scripts/, skills/omc-reference/, settings.local.json, worktrees/, .omc/) in place; keep personal/README.md committed. Previously this relied on each developer's global gitignore, which isn't portable and silently hid new shared files. - .claude/personal/: git-ignored drop zone for per-developer config; CLAUDE.md optionally imports personal/CLAUDE.md if present. - .claude/project.md: project knowledge (overview, layout, architecture, INI, conventions) in the terse house style, cross-referencing the existing build/CI/debug docs instead of duplicating them. Reflects the ext/ -> tracer/ split (#3912); versions reference VERSION / Cargo.toml / rust-toolchain.toml rather than being pinned inline. - CLAUDE.md: link general.md, project.md, ci/index.md and import personal. --- .claude/.gitignore | 23 +++++++++ .claude/personal/README.md | 24 ++++++++++ .claude/project.md | 98 ++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 5 +- 4 files changed, 149 insertions(+), 1 deletion(-) create mode 100644 .claude/.gitignore create mode 100644 .claude/personal/README.md create mode 100644 .claude/project.md diff --git a/.claude/.gitignore b/.claude/.gitignore new file mode 100644 index 00000000000..701c78d3837 --- /dev/null +++ b/.claude/.gitignore @@ -0,0 +1,23 @@ +# Per-developer personal Claude config — never committed. +# +# Everything committed under .claude/ is the *shared* team config. Keep your +# own memory, notes, commands, scripts, and skills out of git using the +# locations below. See personal/README.md. + +# Free-form personal config, notes, and memory. CLAUDE.md optionally imports +# personal/CLAUDE.md if you create one. +/personal/* +!/personal/README.md + +# Personal slash-commands and helper scripts (these must live in their +# canonical locations to be discovered, so they are ignored in place). +/commands/ +/scripts/ + +# Personal skills that are not part of the shared set. +/skills/omc-reference/ + +# Local settings and runtime artifacts. +/settings.local.json +/worktrees/ +/.omc/ diff --git a/.claude/personal/README.md b/.claude/personal/README.md new file mode 100644 index 00000000000..47c5d1c0752 --- /dev/null +++ b/.claude/personal/README.md @@ -0,0 +1,24 @@ +# Personal Claude config + +This folder is **git-ignored** (except this README). Put anything personal +here — it never gets committed and won't collide with the shared team config. + +## What goes here + +- `personal/CLAUDE.md` — your personal memory/instructions. The committed + root `CLAUDE.md` imports it automatically (`@.claude/personal/CLAUDE.md`) + if it exists, so it's loaded on top of the shared config. +- Anything else personal: notes, plans, analyses, scratch docs. + +## What lives elsewhere (also git-ignored, see `.claude/.gitignore`) + +- Personal slash-commands must stay in `.claude/commands/` to be discovered. +- Personal helper scripts stay in `.claude/scripts/`. +- Personal skills (e.g. `.claude/skills/omc-reference/`) stay under + `.claude/skills/`. + +## Shared config (committed) + +`CLAUDE.md`, `.claude/general.md`, `.claude/project.md`, `.claude/ci/**`, +`.claude/skills/{check-ci,crash-analysis,release-notes}`, and the debugging +docs are the team's shared config — edit those in a PR, not here. diff --git a/.claude/project.md b/.claude/project.md new file mode 100644 index 00000000000..b2c9874c53c --- /dev/null +++ b/.claude/project.md @@ -0,0 +1,98 @@ +# Project knowledge + +The Datadog PHP tracer (`ddtrace`): a PHP extension bringing APM, distributed +tracing, profiling, and application security to PHP. Multi-language: + +- **C** extension (`ext/`, `tracer/`) — low-level hooks and runtime glue. +- **Rust** (`components-rs/`, `libdatadog/` submodule, `profiling/`, appsec + helper) — shared cross-tracer functionality, sidecar, crashtracker. +- **PHP** userland (`src/`) — high-level instrumentation. + +User docs: . +For build/test/CI, see the pointers at the bottom — do not duplicate them here. + +## Layout + +``` +ext/ Extension infra shared across products: sidecar, + telemetry, remote_config, otel_config, process_tags, + agent_info, crashtracking, configuration, logging, + signal/pcntl handlers, startup. +tracer/ Tracer-specific C: ddtrace.c, hook/ (userland hooks), + integrations/, limiter/, priority_sampling/, + tracer_tag_propagation/, distributed tracing, + code_origins, asm_event, collect_backtrace, + live_debugger, inferred_proxy_headers, + endpoint_guessing, coms/auto_flush (trace sender). +src/ PHP userland: DDTrace/, api/, bridge/, dogstatsd/. +components/ PHP-agnostic C, one .h/.c + tests/ each (CMake). +components-rs/ PHP-specific Rust wrapping libdatadog. +libdatadog/ Datadog shared Rust library (git submodule). +zend_abstract_interface/ Zend engine abstraction across PHP versions (ZAI). +appsec/ Application security (extension + C++/Rust helpers). +profiling/ Rust profiler extension. +loader/ SSI loader. +tea/ Test harness for ZAI/components. +tests/ .phpt tests + PHPUnit (tests/Integration/). +tooling/ Packaging, installers, artifact helpers. +dockerfiles/ Dev/CI images and package verification. +``` + +> The tracer C was split out of `ext/` into `tracer/` (#3912). Stray +> `ext/hook/`, `ext/integrations/` etc. are build artifacts, not sources — +> the tracked sources live under `tracer/`. + +## Architecture + +**PHP version support.** PHP 7.0 → 8.5+. Version-specific behavior gates on +`PHP_VERSION_ID` macros (e.g. `PHP_VERSION_ID >= 80500`); the +`zend_abstract_interface/` layer (ZAI) absorbs Zend API differences. Common C +is gradually extracted into PHP-agnostic `components/`. + +**Trace sender.** Traces are encoded with msgpack and uploaded asynchronously +so PHP request threads never block. On Linux this is routed through the sidecar +(see below); the legacy in-process background sender remains as fallback. See +`architecture.md` for the full design. + +**Sidecar.** A Rust background service (in `libdatadog/datadog-sidecar*`, +driven from `ext/sidecar.{c,h}`) that offloads I/O off request threads: +telemetry, trace upload, crashtracking, DogStatsD, remote config, live +debugger, and appsec data. PHP ↔ sidecar over IPC (`ddog_SidecarTransport`). +It survives request crashes and is shared across language tracers. + +- `DD_TRACE_SIDECAR_CONNECTION_MODE` = `auto` (default) | `subprocess` | + `thread`. `auto` tries subprocess, falls back to thread. +- **Thread mode is not `pcntl_fork()`-safe** — apps that fork must use + subprocess mode. There is no `docs/SIDECAR_CONNECTION_MODES.md`; the modes + are documented in comments in `ext/sidecar.c`. + +**Rust integration.** `libdatadog/` compiles into the extension; `compile_rust.sh` +(invoked from the Makefile) drives it. Minimize FFI surface; headers are +generated with cbindgen. Toolchain is pinned — see `Cargo.toml` +(`rust-version`) and `profiling/rust-toolchain.toml`, not a hardcoded version. + +## Configuration & INI + +- Runtime config: `DD_*` env vars and `datadog.*` INI keys, defined in + `ext/configuration.h` (x-macro table). +- System INI is installed as `.../conf.d/98-ddtrace.ini`. +- Inspect: `php --ri ddtrace`, or `php -i | grep -E 'datadog\.|ddtrace\.'`. + +## Coding conventions + +- **C/C++:** follow existing patterns; gate version differences on + `PHP_VERSION_ID`; keep `components/` free of the Zend API; treat the trace + sender / sidecar paths as thread-safe. +- **PHP:** PSR-2. `composer lint`, `composer fix-lint`. Userland in `src/`. +- **Rust:** standard conventions; keep the FFI boundary small. + +## Pointers (don't duplicate these here) + +- Operating rules: [general.md](general.md) +- Building any artifact locally: [ci/building-locally.md](ci/building-locally.md) +- Reproducing CI jobs locally: [ci/index.md](ci/index.md) +- Debugging (gdb / appsec integration / system tests): + [debugging.md](debugging.md), [gdb.md](gdb.md), + [debugging-system-tests.md](debugging-system-tests.md) +- Version is in `VERSION`; supported framework versions in + `integration_versions.md`; libdatadog updates in `LIBDATADOG.md`. diff --git a/CLAUDE.md b/CLAUDE.md index 4d3d3bf46ba..1751eb30de7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1 +1,4 @@ -See [.claude/general.md](.claude/general.md) and [.claude/ci/index.md](.claude/ci/index.md). +See [.claude/general.md](.claude/general.md), [.claude/project.md](.claude/project.md) and [.claude/ci/index.md](.claude/ci/index.md). + + +@.claude/personal/CLAUDE.md From 71979cc43874220e9d20d63c685652b28ddd575a Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Wed, 8 Jul 2026 14:54:31 +0200 Subject: [PATCH 2/9] Auto-load common config via @import (general.md, project.md) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plain markdown links in CLAUDE.md are not injected into context by Claude Code — only @-imports are. Import general.md and project.md so the shared operating rules and project knowledge are always loaded alongside each developer's personal config. ci/index.md stays a link (large on-demand CI reference). --- CLAUDE.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1751eb30de7..caccab94ced 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,4 +1,13 @@ -See [.claude/general.md](.claude/general.md), [.claude/project.md](.claude/project.md) and [.claude/ci/index.md](.claude/ci/index.md). +# Datadog PHP tracer — Claude config - +Always-loaded shared config (imported into context): + +@.claude/general.md +@.claude/project.md + +On-demand reference (read when reproducing CI jobs), not auto-loaded: +[.claude/ci/index.md](.claude/ci/index.md). + + @.claude/personal/CLAUDE.md From 549eae446a23792a78af48f6943c7e46d6d55c39 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Wed, 8 Jul 2026 17:24:43 +0200 Subject: [PATCH 3/9] Restructure shared config: scripts/ and debugging/ folders; shareable commands Addresses PR review feedback: - .claude/.gitignore: commands/ is now personal-by-default via `/commands/*` (instead of ignoring the whole dir), so the team CAN commit shared commands by un-ignoring them while personal ones stay ignored. Claude Code discovers commands by filesystem regardless of git status, so both coexist. - Scripts: shared helper scripts moved into .claude/scripts/ (dd_php_release_url, find_map_region.py, parse_ucontext.py); personal scripts go in .claude/scripts/local/ (git-ignored). crash-analysis skill refs updated. - Debugging docs grouped under .claude/debugging/ with an index.md (like ci/): gdb.md, appsec-integration.md (was debugging.md), system-tests.md (was debugging-system-tests.md). All inbound/outbound links updated. Note: ci/ keeps its own co-located tooling (dockerh, check-ci, ...) since those are referenced throughout the ci/ docs as a self-contained module. --- .claude/.gitignore | 19 +++++++++++++------ .claude/ci/appsec-gradle-integration.md | 2 +- .claude/ci/system-tests.md | 2 +- .../appsec-integration.md} | 2 +- .claude/{ => debugging}/gdb.md | 0 .claude/debugging/index.md | 13 +++++++++++++ .../system-tests.md} | 4 ++-- .claude/project.md | 5 ++--- .claude/{ => scripts}/dd_php_release_url | 0 .claude/{ => scripts}/find_map_region.py | 0 .claude/{ => scripts}/parse_ucontext.py | 0 .claude/skills/crash-analysis/SKILL.md | 8 ++++---- 12 files changed, 37 insertions(+), 18 deletions(-) rename .claude/{debugging.md => debugging/appsec-integration.md} (98%) rename .claude/{ => debugging}/gdb.md (100%) create mode 100644 .claude/debugging/index.md rename .claude/{debugging-system-tests.md => debugging/system-tests.md} (97%) rename .claude/{ => scripts}/dd_php_release_url (100%) rename .claude/{ => scripts}/find_map_region.py (100%) rename .claude/{ => scripts}/parse_ucontext.py (100%) diff --git a/.claude/.gitignore b/.claude/.gitignore index 701c78d3837..b45c6665eee 100644 --- a/.claude/.gitignore +++ b/.claude/.gitignore @@ -1,7 +1,7 @@ # Per-developer personal Claude config — never committed. # -# Everything committed under .claude/ is the *shared* team config. Keep your -# own memory, notes, commands, scripts, and skills out of git using the +# Everything else committed under .claude/ is the *shared* team config. Keep +# your own memory, notes, commands, scripts, and skills out of git using the # locations below. See personal/README.md. # Free-form personal config, notes, and memory. CLAUDE.md optionally imports @@ -9,10 +9,17 @@ /personal/* !/personal/README.md -# Personal slash-commands and helper scripts (these must live in their -# canonical locations to be discovered, so they are ignored in place). -/commands/ -/scripts/ +# Slash-commands are PERSONAL by default so each dev keeps their own. Claude +# Code's commands/ dir is flat and discovered regardless of git status, so we +# ignore it by default. To SHARE a command with the team, un-ignore it here and +# commit it, e.g.: !/commands/some-shared-command.md +/commands/* + +# Helper scripts: shared ones live in scripts/ (committed); personal ones go in +# scripts/local/ (git-ignored). Scripts aren't auto-discovered, so subfolders +# are fine. +/scripts/local/ +__pycache__/ # Personal skills that are not part of the shared set. /skills/omc-reference/ diff --git a/.claude/ci/appsec-gradle-integration.md b/.claude/ci/appsec-gradle-integration.md index 34edea060dc..78b3cc764d1 100644 --- a/.claude/ci/appsec-gradle-integration.md +++ b/.claude/ci/appsec-gradle-integration.md @@ -278,7 +278,7 @@ If you need to inspect sidecar/helper or PHP issues: (`pref -f dd-ipc-helper`) or to PHP (usually an apache or an FPM worker -- if you're investigating code run during processes it will not be the master process). sidecar requires as a first command `file /proc//exe`). -* See [gdb.md](../gdb.md) for more information on how to run gdb. Always read +* See [gdb.md](../debugging/gdb.md) for more information on how to run gdb. Always read this file before attempting to use gdb. ## Gotchas diff --git a/.claude/ci/system-tests.md b/.claude/ci/system-tests.md index 5383b989f32..0ee328919c8 100644 --- a/.claude/ci/system-tests.md +++ b/.claude/ci/system-tests.md @@ -300,7 +300,7 @@ cd /path/to/system-tests architecture in the filename. CI always sets this variable. Alternatively, build an arm64 `.so` with `make` (not `make static`) and use the `.so` override path — see - [../debugging-system-tests.md](../debugging-system-tests.md). + [../debugging/system-tests.md](../debugging/system-tests.md). - Artifacts are collected from `system-tests/logs_parametric/` and `system-tests/logs/` -- these directories are always uploaded diff --git a/.claude/debugging.md b/.claude/debugging/appsec-integration.md similarity index 98% rename from .claude/debugging.md rename to .claude/debugging/appsec-integration.md index b61b55b8116..bed2f14e591 100644 --- a/.claude/debugging.md +++ b/.claude/debugging/appsec-integration.md @@ -8,7 +8,7 @@ The appsec Gradle integration tests run inside Docker containers. Use attach gdb to processes inside the container (PHP-FPM workers, sidecar). See [gdb.md](gdb.md) for gdb-specific instructions. -See [ci/appsec-gradle-integration.md](ci/appsec-gradle-integration.md) for +See [ci/appsec-gradle-integration.md](../ci/appsec-gradle-integration.md) for Gradle test details. ## Workflow diff --git a/.claude/gdb.md b/.claude/debugging/gdb.md similarity index 100% rename from .claude/gdb.md rename to .claude/debugging/gdb.md diff --git a/.claude/debugging/index.md b/.claude/debugging/index.md new file mode 100644 index 00000000000..81f9e0c3316 --- /dev/null +++ b/.claude/debugging/index.md @@ -0,0 +1,13 @@ +# Debugging guides + +How to debug this project's components locally. Each file covers one scenario. + +| Guide | Covers | +|---|---| +| [gdb.md](gdb.md) | gdb-via-tmux fundamentals: attaching, scripting, watchpoints, reading optimized-out vars, C-vs-Rust language mode, attaching to the sidecar. Start here for any native (C/Rust) debugging. | +| [appsec-integration.md](appsec-integration.md) | Debugging the appsec Gradle integration tests: driving containers with `jdb` (`--debug-jvm`) + gdb, breakpoint strategy, keeping the container alive, the sidecar watchdog. | +| [system-tests.md](system-tests.md) | Debugging system tests locally (arm64): pytest `--pdb` + gdb inside the weblog container. | + +Related: for *building* the binaries you debug, see +[../ci/building-locally.md](../ci/building-locally.md); for reproducing a +specific CI job, see [../ci/index.md](../ci/index.md). diff --git a/.claude/debugging-system-tests.md b/.claude/debugging/system-tests.md similarity index 97% rename from .claude/debugging-system-tests.md rename to .claude/debugging/system-tests.md index 21595d61524..5aa5b72194d 100644 --- a/.claude/debugging-system-tests.md +++ b/.claude/debugging/system-tests.md @@ -2,7 +2,7 @@ Combines Python debugging (pytest `--pdb`) with gdb inside the weblog container. For build/run instructions see -[ci/system-tests.md](ci/system-tests.md). For gdb fundamentals see +[ci/system-tests.md](../ci/system-tests.md). For gdb fundamentals see [gdb.md](gdb.md). ## arm64-specific build notes @@ -20,7 +20,7 @@ emulation is too slow. ## Building ddtrace.so with Rust linked -See [ci/building-locally.md](ci/building-locally.md#for-system-tests-centos-7-release-like-build) +See [ci/building-locally.md](../ci/building-locally.md#for-system-tests-centos-7-release-like-build) for the build command, CARGO_HOME workaround, and `make` vs `make static` explanation. diff --git a/.claude/project.md b/.claude/project.md index b2c9874c53c..31a69ecb0c6 100644 --- a/.claude/project.md +++ b/.claude/project.md @@ -91,8 +91,7 @@ generated with cbindgen. Toolchain is pinned — see `Cargo.toml` - Operating rules: [general.md](general.md) - Building any artifact locally: [ci/building-locally.md](ci/building-locally.md) - Reproducing CI jobs locally: [ci/index.md](ci/index.md) -- Debugging (gdb / appsec integration / system tests): - [debugging.md](debugging.md), [gdb.md](gdb.md), - [debugging-system-tests.md](debugging-system-tests.md) +- Debugging (gdb, appsec integration, system tests): + [debugging/index.md](debugging/index.md) - Version is in `VERSION`; supported framework versions in `integration_versions.md`; libdatadog updates in `LIBDATADOG.md`. diff --git a/.claude/dd_php_release_url b/.claude/scripts/dd_php_release_url similarity index 100% rename from .claude/dd_php_release_url rename to .claude/scripts/dd_php_release_url diff --git a/.claude/find_map_region.py b/.claude/scripts/find_map_region.py similarity index 100% rename from .claude/find_map_region.py rename to .claude/scripts/find_map_region.py diff --git a/.claude/parse_ucontext.py b/.claude/scripts/parse_ucontext.py similarity index 100% rename from .claude/parse_ucontext.py rename to .claude/scripts/parse_ucontext.py diff --git a/.claude/skills/crash-analysis/SKILL.md b/.claude/skills/crash-analysis/SKILL.md index cdb6703c78b..ec674a1c83f 100644 --- a/.claude/skills/crash-analysis/SKILL.md +++ b/.claude/skills/crash-analysis/SKILL.md @@ -35,7 +35,7 @@ file. Do all extractions in parallel where possible. | Native stacktrace | `jq -c '.error.stack.frames' $EVENT` | | PHP stacktrace | `jq -c .experimental.runtime_stack.frames $EVENT` | | Mapped files | `jq -c '.files["/proc/self/maps"]' $EVENT` | -| Registers | `jq -c '.ucontext // .experimental.ucontext' $EVENT \| .claude/parse_ucontext.py` | +| Registers | `jq -c '.ucontext // .experimental.ucontext' $EVENT \| .claude/scripts/parse_ucontext.py` | | PHP version | `jq -r '.language_version // .runtime_version // (.metadata.tags[] \| select(startswith("runtime_version:")) \| split(":")[1])' $EVENT` | | OS / arch | `jq -r '(.os_info.os_type // .host.os) + " " + (.os_info.architecture // .host.arch // "unknown") + " (kernel " + (.host.version // "?") + ")"' $EVENT` | @@ -164,7 +164,7 @@ Runtime Callback 1.0"`) contain: > appropriate Docker container to obtain the exact source. Map each native stacktrace frame to source code: -1. Use `.claude/find_map_region.py
` to identify which +1. Use `.claude/scripts/find_map_region.py
` to identify which binary each instruction pointer belongs to. In schema 1.6+ events, frames include a `module_base_address` field that already identifies the binary — cross-check against the mapped files to confirm the binary name. @@ -208,11 +208,11 @@ If the stacktrace correlation is ambiguous or the crash is in Datadog code: 1. Download the release binaries: - **SSI** (`libdatadog_php.so` in maps): fetches from ECR public, no credentials needed: ``` - .claude/dd_php_release_url --ssi '' '' + .claude/scripts/dd_php_release_url --ssi '' '' ``` - **Monolithic**: fetches from GitHub releases: ``` - .claude/dd_php_release_url '' '' '' '' + .claude/scripts/dd_php_release_url '' '' '' '' ``` Both print a temp directory with the extracted package. Use the version exactly as it appears in `tracer_version` (or reconstructed from From 35d2e268ca2de2df7ed15bff07daa00b1c93103e Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Thu, 9 Jul 2026 14:22:28 +0200 Subject: [PATCH 4/9] Keep personal skill ignores out of shared gitignore Personal skills live in .claude/skills/ alongside shared ones (Claude Code discovers by filesystem, ignoring git status). Their dir name is the skill name, so there's no generic pattern; document that each dev ignores their own personal skill dirs in .git/info/exclude (per-clone) instead of publishing personal skill names into the committed shared config. --- .claude/.gitignore | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/.claude/.gitignore b/.claude/.gitignore index b45c6665eee..afff1043106 100644 --- a/.claude/.gitignore +++ b/.claude/.gitignore @@ -4,25 +4,10 @@ # your own memory, notes, commands, scripts, and skills out of git using the # locations below. See personal/README.md. -# Free-form personal config, notes, and memory. CLAUDE.md optionally imports -# personal/CLAUDE.md if you create one. /personal/* !/personal/README.md -# Slash-commands are PERSONAL by default so each dev keeps their own. Claude -# Code's commands/ dir is flat and discovered regardless of git status, so we -# ignore it by default. To SHARE a command with the team, un-ignore it here and -# commit it, e.g.: !/commands/some-shared-command.md -/commands/* - -# Helper scripts: shared ones live in scripts/ (committed); personal ones go in -# scripts/local/ (git-ignored). Scripts aren't auto-discovered, so subfolders -# are fine. /scripts/local/ -__pycache__/ - -# Personal skills that are not part of the shared set. -/skills/omc-reference/ # Local settings and runtime artifacts. /settings.local.json From ebed68e0e9247b46d6b8d8eab3f5a781106a96a9 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Thu, 9 Jul 2026 15:17:27 +0200 Subject: [PATCH 5/9] Drop in-repo personal mechanism; personal config lives in ~/.claude Personal Claude config now lives entirely in ~/.claude (global), so the repo's .claude/ is purely shared team config: - Remove .claude/personal/ (README) and the @.claude/personal/CLAUDE.md import from CLAUDE.md. - Slim .claude/.gitignore to just local/runtime artifacts (settings.local.json, worktrees/, .omc/); document that personal config goes in ~/.claude. --- .claude/.gitignore | 15 ++++----------- .claude/personal/README.md | 24 ------------------------ CLAUDE.md | 5 ++--- 3 files changed, 6 insertions(+), 38 deletions(-) delete mode 100644 .claude/personal/README.md diff --git a/.claude/.gitignore b/.claude/.gitignore index afff1043106..23f74040a1a 100644 --- a/.claude/.gitignore +++ b/.claude/.gitignore @@ -1,15 +1,8 @@ -# Per-developer personal Claude config — never committed. +# Everything committed under .claude/ is shared team config. Personal Claude +# config is NOT kept in this repo — it lives in ~/.claude (global): your +# ~/.claude/CLAUDE.md, ~/.claude/skills/, ~/.claude/commands/, etc. # -# Everything else committed under .claude/ is the *shared* team config. Keep -# your own memory, notes, commands, scripts, and skills out of git using the -# locations below. See personal/README.md. - -/personal/* -!/personal/README.md - -/scripts/local/ - -# Local settings and runtime artifacts. +# Only local/runtime artifacts are ignored here: /settings.local.json /worktrees/ /.omc/ diff --git a/.claude/personal/README.md b/.claude/personal/README.md deleted file mode 100644 index 47c5d1c0752..00000000000 --- a/.claude/personal/README.md +++ /dev/null @@ -1,24 +0,0 @@ -# Personal Claude config - -This folder is **git-ignored** (except this README). Put anything personal -here — it never gets committed and won't collide with the shared team config. - -## What goes here - -- `personal/CLAUDE.md` — your personal memory/instructions. The committed - root `CLAUDE.md` imports it automatically (`@.claude/personal/CLAUDE.md`) - if it exists, so it's loaded on top of the shared config. -- Anything else personal: notes, plans, analyses, scratch docs. - -## What lives elsewhere (also git-ignored, see `.claude/.gitignore`) - -- Personal slash-commands must stay in `.claude/commands/` to be discovered. -- Personal helper scripts stay in `.claude/scripts/`. -- Personal skills (e.g. `.claude/skills/omc-reference/`) stay under - `.claude/skills/`. - -## Shared config (committed) - -`CLAUDE.md`, `.claude/general.md`, `.claude/project.md`, `.claude/ci/**`, -`.claude/skills/{check-ci,crash-analysis,release-notes}`, and the debugging -docs are the team's shared config — edit those in a PR, not here. diff --git a/CLAUDE.md b/CLAUDE.md index caccab94ced..7d9beb47e22 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,6 +8,5 @@ Always-loaded shared config (imported into context): On-demand reference (read when reproducing CI jobs), not auto-loaded: [.claude/ci/index.md](.claude/ci/index.md). - -@.claude/personal/CLAUDE.md +Personal config is not kept in this repo — put it in `~/.claude` (global): +`~/.claude/CLAUDE.md`, `~/.claude/skills/`, etc. From 66e237c8a18ed193e97b758d5cd8758de4d5e078 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Thu, 9 Jul 2026 15:39:42 +0200 Subject: [PATCH 6/9] Remove .claude/.gitignore; personal tool artifacts ignored globally MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The repo .claude/ carries no ignore file now — it's purely shared config. Local/runtime artifacts are handled outside the shared config: settings.local.json via the repo root .gitignore, and per-developer tooling output (OMC .omc/, Claude Code agent worktrees/checkpoints/etc.) via each dev's own global ~/.gitignore. --- .claude/.gitignore | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 .claude/.gitignore diff --git a/.claude/.gitignore b/.claude/.gitignore deleted file mode 100644 index 23f74040a1a..00000000000 --- a/.claude/.gitignore +++ /dev/null @@ -1,8 +0,0 @@ -# Everything committed under .claude/ is shared team config. Personal Claude -# config is NOT kept in this repo — it lives in ~/.claude (global): your -# ~/.claude/CLAUDE.md, ~/.claude/skills/, ~/.claude/commands/, etc. -# -# Only local/runtime artifacts are ignored here: -/settings.local.json -/worktrees/ -/.omc/ From 7592d54fd53f5aebd5ba4a0a196f2b42e8142830 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Thu, 9 Jul 2026 16:09:36 +0200 Subject: [PATCH 7/9] Expand project.md into a project/ code-guide folder MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Turn the single project.md into a ci/-style folder: a concise always-imported index.md (overview, layout, subsystem map, architecture summary, config, conventions, pointers) plus one terse orientation file per subsystem — ext, tracer, userland (src/), components (+components-rs/ZAI), sidecar, appsec, profiling. Each covers what it is / key files & dirs / how it fits / gotchas, cross-linked, with build/CI/debug content linked out (not duplicated). Produced via an analyze→write→review pass over the actual source: per-subsystem code analysis, drafting, then a source-accuracy + house-style review. Versions are referenced by source-of-truth (VERSION, Cargo.toml, rust-toolchain.toml); no hardcoded versions or source line numbers. CLAUDE.md now imports project/index.md. --- .claude/project/appsec.md | 50 ++++++++++++++++++++ .claude/project/components.md | 44 +++++++++++++++++ .claude/project/ext.md | 46 ++++++++++++++++++ .claude/{project.md => project/index.md} | 60 +++++++++++++++--------- .claude/project/profiling.md | 48 +++++++++++++++++++ .claude/project/sidecar.md | 45 ++++++++++++++++++ .claude/project/tracer.md | 49 +++++++++++++++++++ .claude/project/userland.md | 42 +++++++++++++++++ CLAUDE.md | 2 +- 9 files changed, 363 insertions(+), 23 deletions(-) create mode 100644 .claude/project/appsec.md create mode 100644 .claude/project/components.md create mode 100644 .claude/project/ext.md rename .claude/{project.md => project/index.md} (60%) create mode 100644 .claude/project/profiling.md create mode 100644 .claude/project/sidecar.md create mode 100644 .claude/project/tracer.md create mode 100644 .claude/project/userland.md diff --git a/.claude/project/appsec.md b/.claude/project/appsec.md new file mode 100644 index 00000000000..786e5030230 --- /dev/null +++ b/.claude/project/appsec.md @@ -0,0 +1,50 @@ +# appsec/ — application security + +## What it is + +A separate PHP extension (`ddappsec.so`) plus a standalone WAF helper +process (`libddappsec-helper.so`, C++ original or Rust rewrite). The helper +runs libddwaf on request data, handles remote config and security actions; +matches tag spans via `asm_event`. + +## Key files & dirs + +- `appsec/src/extension/` — `ddappsec.c`, + `commands/{client_init,request_init,request_exec,request_shutdown}.c`, + `helper_process.c`, `configuration.c`, `msgpack_helpers.c`, + `ip_extraction.c`. +- `appsec/src/helper/` — C++ helper: `main.cpp`, + `client/`, `runner/`, `engine/`, `service/`, `remote_config/`, + `subscriber/`. +- `appsec/helper-rust/` — Rust helper: + `src/{lib,server,client,service,config,rc,telemetry}.rs`, `build.rs`, + `CLAUDE.md`. +- `appsec/third_party/{libddwaf,libddwaf-rust,msgpack-c,cpp-base64}/`. + +## How it fits + +Extension MINIT/RINIT spawns the helper (subprocess) and connects over a +Unix socket. Per request, the extension sends headers/body/query via +msgpack; the helper runs the WAF and returns block/redirect decisions plus +matched rules. Matches emit `asm_event`, which [tracer/](tracer.md) attaches +to the root span on serialize. + +Two decoupled `.so` files (can be disabled independently); talks to the +helper over msgpack; integrates with the tracer via `asm_event` and reads +remote config (ASM rules) via the [sidecar](sidecar.md). + +## Gotchas + +- Two helper implementations: C++ (reference) vs Rust (modern, the CMake + default). `-PuseHelperCpp` is an appsec integration-test Gradle property that + runs the tests against the C++ helper — not a build-time selector. +- Submodules required: `libddwaf`, `libddwaf-rust`, `msgpack-c`, + `cpp-base64`, `libdatadog` — see + [building-locally.md](../ci/building-locally.md#submodule-initialisation). +- The C++ helper needs C++17 / devtoolset on CentOS. +- `appsec/helper-rust/CLAUDE.md` has the deep guide for the Rust helper. +- Helper build differs by implementation: Gradle/Docker for Rust, CMake for + C++. +- For build/test detail see + [../ci/building-locally.md](../ci/building-locally.md) and + [../debugging/index.md](../debugging/index.md). diff --git a/.claude/project/components.md b/.claude/project/components.md new file mode 100644 index 00000000000..f11e8da0f1d --- /dev/null +++ b/.claude/project/components.md @@ -0,0 +1,44 @@ +# components/ + components-rs/ + zend_abstract_interface/ (ZAI) + +## What it is + +- `components/` — PHP-agnostic C utilities (no Zend API), each with its own + `.h`/`.c` + CMake tests. +- `components-rs/` — Rust FFI bridge to libdatadog (cbindgen-generated + headers). +- `zend_abstract_interface/` (ZAI) — Zend engine abstraction gating + PHP-version differences. + +## Key files & dirs + +- `components/{string_view,log,sapi,stack-sample,container_id}/` (+ + polyfill headers). +- `components-rs/lib.rs` — modules: `agent_info`, `log`, `remote_config`, + `sidecar`, `stats`, `telemetry`, `trace_filter`, `bytes`. +- `components-rs/{common,datadog,sidecar,telemetry,crashtracker, + library-config,live-debugger}.h` — cbindgen-generated. +- `components-rs/Cargo.toml`. +- `zend_abstract_interface/{hook,config,env,zai_string, + sandbox/{php7,php8},interceptor/{php7,php8},jit_utils,exceptions,headers, + json,uri_normalization}/`. + +## How it fits + +[ext/](ext.md) and [tracer/](tracer.md) `#include` component headers +(symbols prefixed `datadog_php_`); components compile via `config.m4`. +`components-rs` compiles into `libdatadog_php.a` and links in; C calls +`#[no_mangle] extern "C"` functions. `make generate_cbindgen` regenerates +headers (CI checks they're up to date). + +ZAI: `tracer/` calls `zai_hook_install()`; ZAI absorbs PHP7 vs PHP8 +(`zend_observer`) differences so callers avoid scattered `PHP_VERSION_ID` +checks. + +## Gotchas + +- No Zend API allowed in `components/`. +- Version gating is concentrated in ZAI (`hook.c`, `sandbox/php{7,8}`, + `interceptor/php{7,8}`) — put new version-dependent logic there, not in + scattered `PHP_VERSION_ID` checks elsewhere. +- Keep the FFI surface small: don't leak libdatadog structs into C. +- Regenerate cbindgen headers whenever `lib.rs` changes. diff --git a/.claude/project/ext.md b/.claude/project/ext.md new file mode 100644 index 00000000000..4c8ed7379bb --- /dev/null +++ b/.claude/project/ext.md @@ -0,0 +1,46 @@ +# ext/ — extension infra shared across products + +## What it is + +Runtime bedrock shared by tracer, profiler, and appsec: config, sidecar IPC, +telemetry, remote config, signal handlers, logging, agent connectivity. +Tracer-specific C was split out into [tracer/](tracer.md) (#3912); stray +`ext/hook/`, `ext/integrations/` etc. are build artifacts, not sources. + +## Key files & dirs + +- `ext/datadog.c` — module entry (MINIT/RINIT/RSHUTDOWN/MSHUTDOWN). +- `ext/configuration.{c,h}` — shared/infra x-macro INI/env table (tracer- + specific config is in `tracer/configuration.h`). +- `ext/sidecar.{c,h}` — IPC to the sidecar (see [sidecar.md](sidecar.md)). +- `ext/telemetry.{c,h}`, `ext/remote_config.{c,h}` — sidecar-backed features. +- `ext/logging.{c,h}` — signal-safe logging for the background sender. +- `ext/signals.{c,h}` — SIGTERM/INT/CHLD, `pcntl_fork` interception. +- `ext/endpoints.c` — agent/dogstatsd URL resolution. +- `ext/startup_logging.{c,h}` — first-RINIT diagnostics. +- `ext/process_tags.{c,h}`, `ext/agent_info.c`, `ext/git.c`. +- `ext/otel_config.{c,h}` — `OTEL_*` → `DD_*` bridging. +- `ext/crashtracking_windows.c` — Windows only; Unix crashtracking lives in + libdatadog Rust. + +## How it fits + +MINIT registers INI, config, sidecar, remote_config, signals. RINIT runs +remote_config + a one-time `dd_rinit_once` (process tags, startup diagnostics, +signals) + agent_info + sidecar + tracer init. MSHUTDOWN tears down in +reverse. + +`ext/configuration.h` holds the shared/infra x-macro config table (tracer- +specific config lives in `tracer/configuration.h`). ext/ exposes the +sidecar/telemetry/remote_config public API consumed by [tracer/](tracer.md), +[appsec/](appsec.md), and [profiling/](profiling.md). + +## Gotchas + +- Config is x-macro-driven: add a macro + parser, not ad hoc code. +- `DD_TRACE_SIDECAR_CONNECTION_MODE`: `subprocess` is fork-safe, `thread` is + not — see [sidecar.md](sidecar.md). +- `PHP_VERSION_ID` gating throughout; prefer ZAI (see + [components.md](components.md)) for anything non-trivial. +- Signal handlers use a best-effort sidecar pointer (may be null/stale). +- Telemetry redacts repo paths before upload. diff --git a/.claude/project.md b/.claude/project/index.md similarity index 60% rename from .claude/project.md rename to .claude/project/index.md index 31a69ecb0c6..5c5b521858b 100644 --- a/.claude/project.md +++ b/.claude/project/index.md @@ -9,7 +9,7 @@ tracing, profiling, and application security to PHP. Multi-language: - **PHP** userland (`src/`) — high-level instrumentation. User docs: . -For build/test/CI, see the pointers at the bottom — do not duplicate them here. +For build/test/CI, see the pointers below — do not duplicate them here. ## Layout @@ -42,23 +42,36 @@ dockerfiles/ Dev/CI images and package verification. > `ext/hook/`, `ext/integrations/` etc. are build artifacts, not sources — > the tracked sources live under `tracer/`. +## Subsystem map + +| Area | What | Guide | +|---|---|---| +| `ext/` | Shared runtime infra: config, sidecar IPC, telemetry, remote config, signals | [ext.md](ext.md) | +| `tracer/` | Instrumentation engine: spans, sampling, serialization, hooks, trace sender | [tracer.md](tracer.md) | +| `src/` | PHP userland: Tracer/Span API, propagators, ~40 integrations | [userland.md](userland.md) | +| `components/`, `components-rs/`, ZAI | PHP-agnostic C, Rust FFI bridge, Zend version abstraction | [components.md](components.md) | +| sidecar | Rust background service for async I/O (trace/telemetry/RC upload) | [sidecar.md](sidecar.md) | +| `appsec/` | Application security extension + WAF helper process | [appsec.md](appsec.md) | +| `profiling/` | Rust profiler extension (CPU/wall/alloc/exception) | [profiling.md](profiling.md) | + ## Architecture **PHP version support.** PHP 7.0 → 8.5+. Version-specific behavior gates on -`PHP_VERSION_ID` macros (e.g. `PHP_VERSION_ID >= 80500`); the -`zend_abstract_interface/` layer (ZAI) absorbs Zend API differences. Common C -is gradually extracted into PHP-agnostic `components/`. +`PHP_VERSION_ID` macros; `zend_abstract_interface/` (ZAI) absorbs Zend API +differences (see [components.md](components.md)). Common C is gradually +extracted into PHP-agnostic `components/`. **Trace sender.** Traces are encoded with msgpack and uploaded asynchronously -so PHP request threads never block. On Linux this is routed through the sidecar -(see below); the legacy in-process background sender remains as fallback. See -`architecture.md` for the full design. +so PHP request threads never block. On Linux this is routed through the +sidecar (see below); the legacy in-process background sender (`tracer/coms.c`) +remains as fallback. See [tracer.md](tracer.md). -**Sidecar.** A Rust background service (in `libdatadog/datadog-sidecar*`, -driven from `ext/sidecar.{c,h}`) that offloads I/O off request threads: -telemetry, trace upload, crashtracking, DogStatsD, remote config, live -debugger, and appsec data. PHP ↔ sidecar over IPC (`ddog_SidecarTransport`). -It survives request crashes and is shared across language tracers. +**Sidecar.** A Rust background service (`libdatadog/datadog-sidecar*`, driven +from `ext/sidecar.{c,h}`) that offloads I/O off request threads: telemetry, +trace upload, crashtracking, DogStatsD, remote config, live debugger, and +appsec data. PHP ↔ sidecar over IPC (`ddog_SidecarTransport`). It survives +request crashes and is shared across language tracers. See +[sidecar.md](sidecar.md). - `DD_TRACE_SIDECAR_CONNECTION_MODE` = `auto` (default) | `subprocess` | `thread`. `auto` tries subprocess, falls back to thread. @@ -66,15 +79,17 @@ It survives request crashes and is shared across language tracers. subprocess mode. There is no `docs/SIDECAR_CONNECTION_MODES.md`; the modes are documented in comments in `ext/sidecar.c`. -**Rust integration.** `libdatadog/` compiles into the extension; `compile_rust.sh` -(invoked from the Makefile) drives it. Minimize FFI surface; headers are -generated with cbindgen. Toolchain is pinned — see `Cargo.toml` -(`rust-version`) and `profiling/rust-toolchain.toml`, not a hardcoded version. +**Rust integration.** `libdatadog/` compiles into the extension; +`compile_rust.sh` (invoked from the Makefile) drives it. Minimize FFI surface; +headers are generated with cbindgen (see [components.md](components.md)). +Toolchain is pinned — see `Cargo.toml` (`rust-version`) and +`profiling/rust-toolchain.toml`, not a hardcoded version. ## Configuration & INI -- Runtime config: `DD_*` env vars and `datadog.*` INI keys, defined in - `ext/configuration.h` (x-macro table). +- Runtime config: `DD_*` env vars and `datadog.*` INI keys, in two x-macro + tables — `ext/configuration.h` (shared/infra) and `tracer/configuration.h` + (tracer-specific, the majority of keys). - System INI is installed as `.../conf.d/98-ddtrace.ini`. - Inspect: `php --ri ddtrace`, or `php -i | grep -E 'datadog\.|ddtrace\.'`. @@ -88,10 +103,11 @@ generated with cbindgen. Toolchain is pinned — see `Cargo.toml` ## Pointers (don't duplicate these here) -- Operating rules: [general.md](general.md) -- Building any artifact locally: [ci/building-locally.md](ci/building-locally.md) -- Reproducing CI jobs locally: [ci/index.md](ci/index.md) +- Operating rules: [../general.md](../general.md) +- Building any artifact locally: + [../ci/building-locally.md](../ci/building-locally.md) +- Reproducing CI jobs locally: [../ci/index.md](../ci/index.md) - Debugging (gdb, appsec integration, system tests): - [debugging/index.md](debugging/index.md) + [../debugging/index.md](../debugging/index.md) - Version is in `VERSION`; supported framework versions in `integration_versions.md`; libdatadog updates in `LIBDATADOG.md`. diff --git a/.claude/project/profiling.md b/.claude/project/profiling.md new file mode 100644 index 00000000000..dea4aa23613 --- /dev/null +++ b/.claude/project/profiling.md @@ -0,0 +1,48 @@ +# profiling/ — Rust profiler extension + +## What it is + +A separate Rust `cdylib` extension for continuous CPU/wall-time profiling, +heap allocation, exception, and I/O profiling. Uploads pprof directly to the +agent. + +## Key files & dirs + +- `profiling/Cargo.toml` — `cdylib`; depends on `libdd-profiling`/`alloc`/ + `common` from libdatadog. +- `profiling/rust-toolchain.toml` — pins Rust (distinct from the workspace + toolchain). +- `profiling/src/lib.rs` — module entry: `minit`/`rinit`/`prshutdown`, Zend + interrupt registration. +- `profiling/src/profiling/` — `mod.rs` (`Profiler` + `SampleValues`), + `interrupts.rs`, `backtrace.rs`, `stack_walking.rs`, `uploader.rs`, + `thread_utils.rs`. +- `profiling/src/allocation/` — `allocation_ge84.rs` (PHP 8.4+), + `allocation_le83.rs` (≤8.3). +- `profiling/src/config.rs`, `profiling/src/capi.rs`. +- `profiling/build.rs` — bindgen + `php-config` feature detection. +- `profiling/src/php_ffi.{c,h}`. + +## How it fits + +`minit` registers a hybrid module + zend_extension; `rinit` starts a +per-request `Profiler` sampling on VM interrupt (~10ms) and hooks +`zend_execute_internal`. Samples wall/CPU/alloc/heap/exception; uploads +pprof via a background thread; `prshutdown` flushes. + +Builds independently from the tracer; depends on libdatadog for pprof. The +main tracer looks up `ddog_php_prof_interrupt_function` by symbol to call on +interrupt (see [tracer.md](tracer.md)). Not sidecar-dependent — it uploads +directly (contrast with [sidecar.md](sidecar.md)). `build.rs` reads +`../VERSION`. + +## Gotchas + +- Pinned `rust-toolchain.toml`, not the workspace default. +- `CARGO_TARGET_DIR` must be set (the Makefile uses `tmp/build_profiler`). +- `cdylib` is release-only — phpt tests fail on debug builds. +- Allocation hook differs: PHP 8.4+ vs ≤8.3 use different modules. +- `io_profiling` is Linux/macOS only. +- `trigger_time_sample` is a debug-only build feature. +- For build/test detail see + [../ci/building-locally.md](../ci/building-locally.md). diff --git a/.claude/project/sidecar.md b/.claude/project/sidecar.md new file mode 100644 index 00000000000..fc68a3b43e6 --- /dev/null +++ b/.claude/project/sidecar.md @@ -0,0 +1,45 @@ +# sidecar — Rust background service + +## What it is + +A Rust service (`libdatadog/datadog-sidecar`) run as a subprocess or thread, +offloading I/O off request threads: batches/compresses/uploads to the agent +and to Datadog. Survives request crashes and is shared across language +tracers. + +## Key files & dirs + +- PHP/C: `ext/sidecar.{c,h}` (see [ext.md](ext.md)). +- Rust FFI bridge: `components-rs/sidecar.{rs,h}` + (`ddog_sidecar_connect_php` bridges C config → Rust; see + [components.md](components.md)). +- Rust core: `libdatadog/datadog-sidecar/`, + `libdatadog/datadog-sidecar-ffi/src/lib.rs`, + `libdatadog/datadog-sidecar-macros/`. +- Submission points: `ext/telemetry.c`, `ext/remote_config.c`, + `tracer/coms.c` / `tracer/span.c` (traces; see [tracer.md](tracer.md)), + `tracer/live_debugger.c`, dogstatsd, crashtracking, [appsec](appsec.md). + +## How it fits + +MINIT records the master PID (thread mode starts the master listener). +Setup builds instance/runtime IDs and connects (`auto` mode tries subprocess, +falls back to thread). RINIT does a per-thread connection plus +service/env tags. Global shutdown flushes and drops the transport. +`handle_fork` drops the inherited transport and reconnects after +`pcntl_fork()`. + +Traces, telemetry, remote config (via shared memory), DogStatsD, +crashtracking, live debugger, and appsec data all flow through the sidecar — +request threads never block on network I/O. + +## Gotchas + +- `DD_TRACE_SIDECAR_CONNECTION_MODE` = `auto` | `subprocess` | `thread`. +- Thread mode is **not** `pcntl_fork()`-safe — apps that fork must use + subprocess mode. +- The transport type `ddog_SidecarTransport` is opaque, created in Rust. +- Reconnect happens automatically on sidecar crash. +- Connection modes are documented only in code comments in `ext/sidecar.c` + — there is no `docs/SIDECAR_CONNECTION_MODES.md`. +- Config defaults live in `ext/configuration.h`. diff --git a/.claude/project/tracer.md b/.claude/project/tracer.md new file mode 100644 index 00000000000..baee5fcce7d --- /dev/null +++ b/.claude/project/tracer.md @@ -0,0 +1,49 @@ +# tracer/ — tracer-specific C core + +## What it is + +The instrumentation engine (split from [ext/](ext.md) in #3912). Creates and +manages span lifecycles, sampling, metadata, msgpack serialization, and sends +traces (via the sidecar or the legacy in-process background sender). + +## Key files & dirs + +- `tracer/ddtrace.c` — tracer lifecycle hooks (`ddtrace_startup`/rinit/…) + invoked from `ext/datadog.c`; routes sidecar vs in-process sender. +- `tracer/span.c` — span alloc/close stacks, ring buffer. +- `tracer/serializer.c` — msgpack encode → libdatadog. +- `tracer/auto_flush.c` — flush orchestration. +- `tracer/coms.c` + `tracer/comms_php.c` — legacy in-process background + sender (Linux-only). +- `tracer/hook/` — `uhook*.c` userland hooks (`trace_function`/ + `trace_method`, sandboxed). +- `tracer/integrations/` — 40+ framework hooks. +- `tracer/priority_sampling/`, `tracer/limiter/` (token-bucket on closed + spans), `tracer/tracer_tag_propagation/` (W3C + Datadog headers). +- `tracer/distributed_tracing_headers.c`, `tracer/code_origins.c`, + `tracer/asm_event.c` (appsec events on spans), `tracer/collect_backtrace.c`, + `tracer/endpoint_guessing.c`, `tracer/inferred_proxy_headers.c`, + `tracer/live_debugger.c`, `tracer/dogstatsd_client.c`. + +## How it fits + +Startup registers the ZAI interceptor and the profiling symbol. RINIT reads +distributed headers and optionally creates a root span. A hook on function +entry allocates a span; on return it closes the span and applies sampling. +Auto-flush triggers when the open-span threshold is hit or on RSHUTDOWN: +serialize → sidecar or background sender. + +Sits above [ext/](ext.md) infra and ZAI hooks (see +[components.md](components.md)); consumes libdatadog Rust. The sidecar (see +[sidecar.md](sidecar.md)) is the default sender; `coms.c` is the legacy +fallback (not on Windows). [PHP userland](userland.md) (`src/DDTrace`) wraps +these hooks as objects. + +## Gotchas + +- Per-request span stacks, no locks; PHP 8.1+ fibers get separate span + stacks. +- Sampling is decided at span close time, not open time. +- Userland hooks are sandboxed (exceptions in hook code don't crash the + request). +- The root span closes last. diff --git a/.claude/project/userland.md b/.claude/project/userland.md new file mode 100644 index 00000000000..0c1ef3077d1 --- /dev/null +++ b/.claude/project/userland.md @@ -0,0 +1,42 @@ +# src/ — PHP userland + +## What it is + +High-level instrumentation in PHP: span/scope management, config, +propagation, and ~40 library/framework integrations. C hooks fire; userland +builds and tags spans. + +## Key files & dirs + +- `src/api/` — Contracts: `Tracer`/`Span`/`SpanContext`/`ScopeManager`; + logging, sampling, tags. +- `src/DDTrace/Tracer.php` — main implementation. +- `src/DDTrace/{Span,SpanContext,Scope,ScopeManager}.php`. +- `src/DDTrace/Transport/Internal.php` — calls `\DDTrace\flush()` in C. +- `src/DDTrace/Propagators/` — W3C/B3. +- `src/DDTrace/Integrations/` — base + per-library integrations, some with + `V2`/`V3` subdirs for library versions. +- `src/bridge/` — `_files_*` / `_generated_*` file-list pairs (api, tracer, + opentelemetry, openfeature) loaded by `tracer/autoload_php_files.c`, which + registers the autoloader. +- `src/dogstatsd/`. +- OpenTelemetry / OpenTracer / OpenFeature compat layers under `src/DDTrace/`. + +## How it fits + +`tracer/autoload_php_files.c` registers the autoloader and loads the bridge +file lists at startup (path from `DD_TRACE_SOURCES_PATH`, +`datadog.trace.sources_path`). Userland calls `\DDTrace\GlobalTracer::get()` +then `startSpan` / `startActiveSpan`. Integrations load only if +`ddtrace_config_integration_enabled($name)` (a C config check). + +C hooks in [tracer/hook, tracer/integrations](tracer.md) fire; PHP +integrations respond by creating/tagging spans; `Tracer` collects and flushes +via C. + +## Gotchas + +- `DD_TRACE_SOURCES_PATH` (`datadog.trace.sources_path`) is set by the loader; + if missing, autoload is silently skipped. +- `DD_AUTOLOAD_NO_COMPILE` toggles `_files_` vs `_generated_` file lists. +- Before the extension loads, spans are no-ops. diff --git a/CLAUDE.md b/CLAUDE.md index 7d9beb47e22..dbae8f8c595 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -3,7 +3,7 @@ Always-loaded shared config (imported into context): @.claude/general.md -@.claude/project.md +@.claude/project/index.md On-demand reference (read when reproducing CI jobs), not auto-loaded: [.claude/ci/index.md](.claude/ci/index.md). From 1f6fedcb12f65e66220d29374feb268924fe0ee5 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Thu, 9 Jul 2026 16:54:45 +0200 Subject: [PATCH 8/9] project/: add data-flow to tracer/ext/sidecar; link architecture & CONTRIBUTING MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Targeted depth-increase (on-demand area files only; index stays lean): - tracer.md: ordered span→upload data-flow (RINIT → headers → span create → uhook → close → sampling → serializer → auto_flush → sidecar/coms → agent). - ext.md: fix the incorrect "MSHUTDOWN tears down in reverse" claim — it isn't (sidecar shuts down late, after config is freed); tighten MINIT/RINIT/RSHUTDOWN ordering (logging-first, zend_extension registration, dd_rinit_once). - sidecar.md: submit→upload flow (serialize → send_traces_to_sidecar → IPC → TraceFlusher batches ~5s/~1MB → agent upload; fork preserves session ID). - index.md: link architecture.md (background sender / components / version code) and CONTRIBUTING.md (setup/testing) from Pointers. Produced via the OMC analyze→write→review pipeline over live source; verified accurate (flush thresholds, ordering, function anchors). No line numbers. --- .claude/project/ext.md | 16 ++++++++++++---- .claude/project/index.md | 8 +++++++- .claude/project/sidecar.md | 17 +++++++++++++++++ .claude/project/tracer.md | 19 ++++++++++++++++++- 4 files changed, 54 insertions(+), 6 deletions(-) diff --git a/.claude/project/ext.md b/.claude/project/ext.md index 4c8ed7379bb..461ecb1ea2a 100644 --- a/.claude/project/ext.md +++ b/.claude/project/ext.md @@ -25,10 +25,18 @@ Tracer-specific C was split out into [tracer/](tracer.md) (#3912); stray ## How it fits -MINIT registers INI, config, sidecar, remote_config, signals. RINIT runs -remote_config + a one-time `dd_rinit_once` (process tags, startup diagnostics, -signals) + agent_info + sidecar + tracer init. MSHUTDOWN tears down in -reverse. +MINIT: logging init first, then config, `zend_extension` registration, +sidecar, remote_config, signals (tracer pre/early/late phases interleave). + +RINIT: remote_config → one-time `dd_rinit_once` (process tags, signals, +startup diagnostics, tracer first-rinit) → agent_info → sidecar → tracer. + +RSHUTDOWN: remote_config → tracer → sidecar (finalize) → telemetry → +sidecar (rshutdown) → git. + +MSHUTDOWN is **not** a reverse of MINIT — notably the sidecar shuts down +late (after config is freed). Order: tracer → remote_config → signals → log +→ config → sidecar → process_tags. `ext/configuration.h` holds the shared/infra x-macro config table (tracer- specific config lives in `tracer/configuration.h`). ext/ exposes the diff --git a/.claude/project/index.md b/.claude/project/index.md index 5c5b521858b..eccc886ea7e 100644 --- a/.claude/project/index.md +++ b/.claude/project/index.md @@ -64,7 +64,9 @@ extracted into PHP-agnostic `components/`. **Trace sender.** Traces are encoded with msgpack and uploaded asynchronously so PHP request threads never block. On Linux this is routed through the sidecar (see below); the legacy in-process background sender (`tracer/coms.c`) -remains as fallback. See [tracer.md](tracer.md). +remains as fallback. See [tracer.md](tracer.md) and +[../../architecture.md](../../architecture.md) for the background-sender +design. **Sidecar.** A Rust background service (`libdatadog/datadog-sidecar*`, driven from `ext/sidecar.{c,h}`) that offloads I/O off request threads: telemetry, @@ -109,5 +111,9 @@ Toolchain is pinned — see `Cargo.toml` (`rust-version`) and - Reproducing CI jobs locally: [../ci/index.md](../ci/index.md) - Debugging (gdb, appsec integration, system tests): [../debugging/index.md](../debugging/index.md) +- Component design + background sender + PHP-version code: + [../../architecture.md](../../architecture.md) +- Contributor setup / linting / local testing: + [../../CONTRIBUTING.md](../../CONTRIBUTING.md) - Version is in `VERSION`; supported framework versions in `integration_versions.md`; libdatadog updates in `LIBDATADOG.md`. diff --git a/.claude/project/sidecar.md b/.claude/project/sidecar.md index fc68a3b43e6..5a66d5d35c8 100644 --- a/.claude/project/sidecar.md +++ b/.claude/project/sidecar.md @@ -33,6 +33,23 @@ Traces, telemetry, remote config (via shared memory), DogStatsD, crashtracking, live debugger, and appsec data all flow through the sidecar — request threads never block on network I/O. +## Submit → upload + +- Closed spans serialize in `tracer/span.c` → `ddog_TracesBytes`. +- At end-of-request `ddtrace_flush_tracer` (`tracer/auto_flush.c`) calls + `ddog_send_traces_to_sidecar`, passing the batch over the + `ddog_SidecarTransport` IPC (Unix socket / shared memory). +- Sidecar side: `TraceFlusher` + (`libdatadog/datadog-sidecar/src/service/tracing/trace_flusher.rs`) + enqueues incoming send-data, batches (by endpoint), and flushes on + interval (~5s) or size threshold (~1MB), then uploads to the agent (or + Datadog directly if agentless). +- Telemetry, remote config, DogStatsD, live debugger, and appsec enqueue + separately and are multiplexed to the shared endpoint. +- Fork: the child drops the inherited transport, regenerates instance/runtime + IDs (the session ID is preserved across fork), and reconnects + (`datadog_sidecar_handle_fork` in `ext/sidecar.c`). + ## Gotchas - `DD_TRACE_SIDECAR_CONNECTION_MODE` = `auto` | `subprocess` | `thread`. diff --git a/.claude/project/tracer.md b/.claude/project/tracer.md index baee5fcce7d..904d8b58435 100644 --- a/.claude/project/tracer.md +++ b/.claude/project/tracer.md @@ -11,7 +11,7 @@ traces (via the sidecar or the legacy in-process background sender). - `tracer/ddtrace.c` — tracer lifecycle hooks (`ddtrace_startup`/rinit/…) invoked from `ext/datadog.c`; routes sidecar vs in-process sender. - `tracer/span.c` — span alloc/close stacks, ring buffer. -- `tracer/serializer.c` — msgpack encode → libdatadog. +- `tracer/serializer.c` — PHP spans → libdatadog spans (msgpack in libdatadog). - `tracer/auto_flush.c` — flush orchestration. - `tracer/coms.c` + `tracer/comms_php.c` — legacy in-process background sender (Linux-only). @@ -39,6 +39,23 @@ Sits above [ext/](ext.md) infra and ZAI hooks (see fallback (not on Windows). [PHP userland](userland.md) (`src/DDTrace`) wraps these hooks as objects. +## Data flow + +Span→upload path, in order: + +- RINIT (`tracer/ddtrace.c`): init per-request span stacks. +- `tracer/distributed_tracing_headers.c`: extract trace context from headers. +- `tracer/span.c`: create root span (inherit trace/parent IDs if enabled). +- `tracer/hook/uhook.c`: on function entry a ZAI hook fires → allocate span. +- `tracer/span.c`: on return → close span. +- `tracer/priority_sampling/`: sampling decided at close. +- `tracer/serializer.c`: PHP spans → libdatadog spans (msgpack in libdatadog). +- `tracer/auto_flush.c`: flush on threshold or RSHUTDOWN, then handoff → + `ext/sidecar.c` (default) or `tracer/coms.c` (legacy) → agent. + +See [../../architecture.md](../../architecture.md) for the background sender +design. + ## Gotchas - Per-request span stacks, no locks; PHP 8.1+ fibers get separate span From 348677d6e585d8124f2a9393a92bf530dffe3f84 Mon Sep 17 00:00:00 2001 From: Alexandre Rulleau Date: Thu, 9 Jul 2026 17:18:00 +0200 Subject: [PATCH 9/9] =?UTF-8?q?project/:=20address=20Codex=20review=20?= =?UTF-8?q?=E2=80=94=20fix=204=20doc=20inaccuracies?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four verified against live source: - appsec.md: C++ helper client/runner/engine/service are .cpp/.hpp files, not dirs (only network/, remote_config/, subscriber/ are dirs). - appsec.md: the helper is enabled via the sidecar — ext/sidecar.c calls the appsec module's dd_appsec_maybe_enable_helper -> ddog_sidecar_enable_appsec (resolved Rust/C++ path) before ddappsec's first RINIT — not spawned by ddappsec MINIT/RINIT. - tracer.md + index.md: the sidecar sender is only the default on PHP 8.3+ / Windows (DD_SIDECAR_TRACE_SENDER_DEFAULT); on PHP 7.0-8.2 the in-process coms.c sender is the default. - components.md: drop container_id from the component list — it's a stale build-artifact dir with no source; container ID is in the Rust bridge (ddtrace_get_container_id). --- .claude/project/appsec.md | 16 +++++++++------- .claude/project/components.md | 5 +++-- .claude/project/index.md | 10 +++++----- .claude/project/tracer.md | 12 +++++++----- 4 files changed, 24 insertions(+), 19 deletions(-) diff --git a/.claude/project/appsec.md b/.claude/project/appsec.md index 786e5030230..43028c12ef3 100644 --- a/.claude/project/appsec.md +++ b/.claude/project/appsec.md @@ -13,8 +13,8 @@ matches tag spans via `asm_event`. `commands/{client_init,request_init,request_exec,request_shutdown}.c`, `helper_process.c`, `configuration.c`, `msgpack_helpers.c`, `ip_extraction.c`. -- `appsec/src/helper/` — C++ helper: `main.cpp`, - `client/`, `runner/`, `engine/`, `service/`, `remote_config/`, +- `appsec/src/helper/` — C++ helper: `main.cpp`, `client`/`runner`/`engine`/ + `service` (`.cpp`/`.hpp`), and dirs `network/`, `remote_config/`, `subscriber/`. - `appsec/helper-rust/` — Rust helper: `src/{lib,server,client,service,config,rc,telemetry}.rs`, `build.rs`, @@ -23,11 +23,13 @@ matches tag spans via `asm_event`. ## How it fits -Extension MINIT/RINIT spawns the helper (subprocess) and connects over a -Unix socket. Per request, the extension sends headers/body/query via -msgpack; the helper runs the WAF and returns block/redirect decisions plus -matched rules. Matches emit `asm_event`, which [tracer/](tracer.md) attaches -to the root span on serialize. +AppSec is enabled through the sidecar: during sidecar setup `ext/sidecar.c` +calls the appsec module's `dd_appsec_maybe_enable_helper` → +`ddog_sidecar_enable_appsec` with the resolved helper path (Rust helper if +present, else C++), before ddappsec's first RINIT. Per request the extension +sends headers/body/query to the helper via msgpack; the helper runs the WAF +and returns block/redirect decisions + matched rules. Matches emit +`asm_event`, which [tracer/](tracer.md) attaches to the root span on serialize. Two decoupled `.so` files (can be disabled independently); talks to the helper over msgpack; integrates with the tracer via `asm_event` and reads diff --git a/.claude/project/components.md b/.claude/project/components.md index f11e8da0f1d..5332ec15261 100644 --- a/.claude/project/components.md +++ b/.claude/project/components.md @@ -11,8 +11,7 @@ ## Key files & dirs -- `components/{string_view,log,sapi,stack-sample,container_id}/` (+ - polyfill headers). +- `components/{string_view,log,sapi,stack-sample}/` (+ polyfill headers). - `components-rs/lib.rs` — modules: `agent_info`, `log`, `remote_config`, `sidecar`, `stats`, `telemetry`, `trace_filter`, `bytes`. - `components-rs/{common,datadog,sidecar,telemetry,crashtracker, @@ -37,6 +36,8 @@ checks. ## Gotchas - No Zend API allowed in `components/`. +- `components/container_id/` is a stale build-artifact dir (no source); + container ID access is in the Rust bridge (`ddtrace_get_container_id`). - Version gating is concentrated in ZAI (`hook.c`, `sandbox/php{7,8}`, `interceptor/php{7,8}`) — put new version-dependent logic there, not in scattered `PHP_VERSION_ID` checks elsewhere. diff --git a/.claude/project/index.md b/.claude/project/index.md index eccc886ea7e..383dbdaca68 100644 --- a/.claude/project/index.md +++ b/.claude/project/index.md @@ -62,11 +62,11 @@ differences (see [components.md](components.md)). Common C is gradually extracted into PHP-agnostic `components/`. **Trace sender.** Traces are encoded with msgpack and uploaded asynchronously -so PHP request threads never block. On Linux this is routed through the -sidecar (see below); the legacy in-process background sender (`tracer/coms.c`) -remains as fallback. See [tracer.md](tracer.md) and -[../../architecture.md](../../architecture.md) for the background-sender -design. +so PHP request threads never block. The default sender is version-gated: the +sidecar (below) on PHP 8.3+/Windows, the in-process `tracer/coms.c` sender on +PHP 7.0–8.2 (`DD_SIDECAR_TRACE_SENDER_DEFAULT`); either is overridable. See +[tracer.md](tracer.md) and [../../architecture.md](../../architecture.md) for +the background-sender design. **Sidecar.** A Rust background service (`libdatadog/datadog-sidecar*`, driven from `ext/sidecar.{c,h}`) that offloads I/O off request threads: telemetry, diff --git a/.claude/project/tracer.md b/.claude/project/tracer.md index 904d8b58435..5d35f06fca7 100644 --- a/.claude/project/tracer.md +++ b/.claude/project/tracer.md @@ -34,10 +34,12 @@ Auto-flush triggers when the open-span threshold is hit or on RSHUTDOWN: serialize → sidecar or background sender. Sits above [ext/](ext.md) infra and ZAI hooks (see -[components.md](components.md)); consumes libdatadog Rust. The sidecar (see -[sidecar.md](sidecar.md)) is the default sender; `coms.c` is the legacy -fallback (not on Windows). [PHP userland](userland.md) (`src/DDTrace`) wraps -these hooks as objects. +[components.md](components.md)); consumes libdatadog Rust. The sender default +is version-gated (`DD_SIDECAR_TRACE_SENDER_DEFAULT`, `tracer/configuration.h`): +the sidecar (see [sidecar.md](sidecar.md)) on PHP 8.3+/Windows, the in-process +`coms.c` sender on PHP 7.0–8.2 (coms.c isn't built on Windows); either is +overridable. [PHP userland](userland.md) (`src/DDTrace`) wraps these hooks as +objects. ## Data flow @@ -51,7 +53,7 @@ Span→upload path, in order: - `tracer/priority_sampling/`: sampling decided at close. - `tracer/serializer.c`: PHP spans → libdatadog spans (msgpack in libdatadog). - `tracer/auto_flush.c`: flush on threshold or RSHUTDOWN, then handoff → - `ext/sidecar.c` (default) or `tracer/coms.c` (legacy) → agent. + `ext/sidecar.c` (PHP 8.3+/Windows) or `tracer/coms.c` (PHP ≤8.2) → agent. See [../../architecture.md](../../architecture.md) for the background sender design.