Skip to content

Commit 3c53c18

Browse files
committed
fix(pack): the build machine does not travel, and packages ship stripped (#460)
A `kind = "shared"` package kept the DT_RUNPATH the link gave it — a list of absolute paths into the BUILD MACHINE's store — so on any other machine the consumer died with `libstdc++.so.6: cannot open shared object file`. The issue's suggested fix does not work, and that is the whole design. Measured on a real package with the build machine's store made unreachable: stale absolute DT_RUNPATH consumer's DT_RPATH inherited? no rc=127 no tag at all YES ok DT_RUNPATH = $ORIGIN no rc=127 DT_RUNPATH = "" (what --set-rpath '' writes) no rc=127 An object carrying ANY DT_RUNPATH makes the loader skip the whole inherited DT_RPATH chain for that object's dependencies. So the criterion is "there is no tag", and removing it is the right answer rather than a compromise: the consumer's own DT_RPATH is the same closure — payload, package dir, SubOS farm — resolved on the machine that will actually run it. New `mcpp.pack.relocate` edits PT_DYNAMIC in process (delete the slot, shift the tail, pad with DT_NULL; same file length) instead of shelling out to patchelf. Library packs are cross-target by construction and have no host gate, so `sandbox_patchelf` resolves to nothing on a macOS or Windows host — and the application packer's shape for that is `if (!patchelf.empty())`, i.e. silently do nothing. ELF32 and big-endian are covered by unit tests; no CI job produces one and `--target` can. Mach-O LC_RPATH is read and reported, not yet rewritten. Also refuses `mcpp pack <program>` for a Mach-O artifact. That path resolves the dependency closure with `LD_TRACE_LOADED_OBJECTS=1 '<binary>'`, which is glibc's variable — dyld ignores it and RUNS THE PROGRAM, then parses its stdout as a dependency table and reports `Packed`. Keyed on the format, not the host, like the `_WIN32` refusal beside it. Never noticed because the e2e harness grants the `pack` capability only where elf+patchelf exist, i.e. Linux. And the third silent one: the SONAME alias' copy fallback read `leg.artifact` rather than the staged file. Byte-identical while nothing modified the staging copy; with relocate and strip in place it would ship an unprocessed library under the exact name the loader asks for, on the machines where create_symlink fails. Packaging now builds release and strips what it ships. Only the profile FALLBACK changes (dev -> release); `--profile` and `[build] default-profile` still win, so pack never produces flags `mcpp build` would not. Stripping follows dh_strip's division, and the archive row is measured: `--strip-all` on a `.a` removes the archive symbol index and the consumer's link fails with `archive has no index; run ranlib to add one`, while `--strip-debug` links and runs. Shared libraries get `--strip-unneeded` (keeps .dynsym), executables `--strip-all`, and bundled third-party .so files nothing at all. New `--profile` / `--no-strip` / `--debug-symbols DIR` and `[pack] strip` / `[pack] debug_symbols`; `--debug-symbols` separates rather than discards and adds a .gnu_debuglink. e2e 264 puts the defect BACK with patchelf and requires the consumer to FAIL before restoring it: 251 consumed the package on the machine that built it, so it was green throughout this bug's life. The guard reads the DYNAMIC ENTRIES, never the file's bytes — the dead string stays in .dynstr (patchelf leaves the identical residue; .dynstr is tail-merged and deleting it cannot be shown safe), and a byte-pattern check would also have flipped to green for an unrelated reason the day stripping landed.
1 parent 282bc41 commit 3c53c18

28 files changed

Lines changed: 3209 additions & 132 deletions

.agents/docs/2026-08-20-issue460-shared-library-runpath.md

Lines changed: 605 additions & 0 deletions
Large diffs are not rendered by default.

.agents/docs/2026-08-20-pack-and-consumer-model-review.md

Lines changed: 416 additions & 0 deletions
Large diffs are not rendered by default.

CHANGELOG.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,94 @@
33
> 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。
44
> 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)
55
6+
## [2026.8.20.1] — 2026-08-20
7+
8+
### 修复
9+
10+
- **`mcpp pack` 打出的 `kind = "shared"` 库带走了构建机,产物在别人机器上起不来
11+
(#460)。**
12+
13+
打包只是把链接产物 `copy_file` 进包里,于是 `.so` 保留了链接期的
14+
`DT_RUNPATH`——一串指向**构建机** `~/.mcpp/` 的绝对路径。消费者在另一台机器上
15+
拿到的是 `libstdc++.so.6: cannot open shared object file`
16+
17+
⚠️ **issue 里建议的 `$ORIGIN` 修不好它,空串也修不好。** 在真实包 + 真实消费者上
18+
实测(把构建机 store 变成不可达):
19+
20+
| 发货 `.so` 上的状态 | 消费方 `DT_RPATH` 被继承 | 结果 |
21+
|---|---|---|
22+
| 失效绝对路径的 `DT_RUNPATH`(此前的行为) || rc=127 |
23+
| **没有这条 tag** | **** | ok |
24+
| `DT_RUNPATH = $ORIGIN` || rc=127 |
25+
| `DT_RUNPATH = ""` || rc=127 |
26+
27+
关掉继承的是这条 tag 的**存在**而不是内容,所以判据是「tag 不存在」。删掉它也
28+
不是妥协:消费方自己的 `DT_RPATH` 是同一个闭包,只不过是在真正要运行它的机器上
29+
解析的。
30+
31+
实现是**进程内改写 `PT_DYNAMIC`**(删槽、后移、补 `DT_NULL`,文件尺寸不变),
32+
不是调 patchelf——库打包支持 `--target` 且没有宿主门,非 Linux 宿主上
33+
`sandbox_patchelf` 会解析为空,照抄应用侧的 `if (!patchelf.empty())` 等于把这个
34+
缺陷留给一半的宿主。新增 `mcpp.pack.relocate`,覆盖 ELF32/64 × 大小端(单测),
35+
Mach-O 只读报告 `LC_RPATH`
36+
37+
- **`mcpp pack` 一个 Mach-O 程序会**执行用户的程序**,然后报告 `Packed`**
38+
39+
非 PE 路径靠 `LD_TRACE_LOADED_OBJECTS=1 '<binary>'` 向动态链接器要依赖表,而这个
40+
变量是 glibc 的;dyld 不认它,于是那条命令在 macOS 上就是把程序跑起来,程序的
41+
输出被当成依赖表解析(解析出零条),然后写出一个只含二进制的包。有副作用的程序
42+
会把副作用做一遍,交互式的会把打包器挂住。现在按**产物格式**(而非宿主)拒绝,
43+
理由与旁边那条 `_WIN32` 拒绝完全同源。macOS 上 `kind = "lib"` / `"shared"` 照常
44+
打包——库打包从不运行产物。
45+
46+
为什么此前没人发现:e2e`pack` 能力 = `elf` + `patchelf`,只有 `Linux)` 分支
47+
给,所以应用打包在 macOS 上**一条 e2e 都没跑过**
48+
49+
- **共享库包的 SONAME 别名在符号链接失败时会拷到未处理的原始产物。**
50+
51+
别名的 copy 回退读的是 `leg.artifact`(构建树里的文件)而不是暂存后的 `dst`
52+
在没有重定位/strip 之前两者逐字节相同,所以看不出来;之后它会在
53+
`create_symlink` 失败的机器上,用装载器真正要打开的那个名字,发出一份未重定位、
54+
未 strip 的库。
55+
56+
### 变更
57+
58+
- **`mcpp pack` 默认走 release 并 strip 发货产物。**
59+
60+
此前两个打包器发的都是 dev 构建:未 strip、带着发布者的绝对源码路径。现在
61+
profile 的**兜底**`dev` 改为 `release`——其余优先级不变(`--profile` >
62+
`[build] default-profile` > 兜底),所以声明过 profile 的工程仍然拿到它声明的
63+
那个。
64+
65+
剥什么取决于产物**是什么**,用的是 dh_strip 的分档:
66+
67+
| 产物 | 参数 | 为什么不能更狠 |
68+
|---|---|---|
69+
| 可执行文件 | `--strip-all` | 没有人链接它 |
70+
| 共享库 | `--strip-unneeded` | 保留 `.dynsym`——那**就是**导出表 |
71+
| 静态归档 | `--strip-debug --enable-deterministic-archives` | ⚠️ `--strip-all` 会删掉归档的**符号索引**,消费方链接时报 `archive has no index; run ranlib to add one`(实测) |
72+
73+
被捆绑进 bundle 的第三方 `.so` ****剥——它们不是 mcpp 构建的。
74+
75+
新增 `--profile` / `--no-strip` / `--debug-symbols <DIR>``[pack] strip`
76+
`[pack] debug_symbols``--debug-symbols` 是分离而不是丢弃:写出
77+
`<dir>/<产物>.debug` 并给发货产物加 `.gnu_debuglink`
78+
79+
> `[pack] strip``[profile.<name>].strip` 是两个决定:后者给**链接**`-s`
80+
> (碰不到静态归档,也分离不出任何东西),前者管**包里带什么**
81+
82+
### 内部
83+
84+
- `mcpp::toolchain::binutils_tool(tc, name)`:四个工具链家族对同一个 binutils 工具
85+
的四种拼法,此前只有 `ar` 知道。`archive_tool` 现在由它表达(MSVC 的 `lib.exe`
86+
仍是特例,因为它不是 binutils 的名字)。
87+
- `tests/e2e/_elf_tag.sh`:215 与新增的 264 共用同一个 ELF 读取器,两份拷贝会变成
88+
「构建机路径」的两个定义。
89+
- **判据只查动态段,不查文件字节**:重定位删的是条目,字符串留在 `.dynstr`
90+
(`patchelf --remove-rpath` 实测残留完全相同,`.dynstr` 有尾部合并,删不安全)。
91+
一个 `grep` 式的判据会把正确重定位的产物报成脏的,而且会在 strip 落地那天因为
92+
另一个原因变绿——两次都不是因为重定位。
93+
694
## [2026.8.18.3] — 2026-08-18
795

896
### 新增

docs/02-pack-and-release.md

Lines changed: 70 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -133,13 +133,57 @@ mcpp pack --target aarch64-linux-musl # ARM64 equivalent
133133
mcpp pack --format dir # output as a directory, no tarball
134134
mcpp pack -o myapp.tar.gz # filename only: lands at target/dist/myapp.tar.gz
135135
mcpp pack -o /abs/path/myapp.tar.gz # includes a directory: output to the literal path
136+
mcpp pack --profile dev # build with a different profile (default: release)
137+
mcpp pack --no-strip # ship the artifacts as built
138+
mcpp pack --debug-symbols dbg/ # write the separated *.debug files under dbg/
136139
```
137140

138141
When `-o` is given a bare filename, the output is placed under `target/dist/`;
139142
when it includes a directory (relative or absolute), the literal path is used.
140143

141144
For the full set of options, see `mcpp pack --help`.
142145

146+
### What a packed artifact is built with, and what travels inside it
147+
148+
Two things differ from `mcpp build`, and both exist because a package leaves
149+
this machine:
150+
151+
**The profile falls back to `release`, not `dev`.** Precedence is unchanged
152+
otherwise — `--profile` beats `[build] default-profile`, which beats the
153+
fallback. Only the last step differs, so a project that states a profile still
154+
gets the one it stated, and `mcpp pack` never produces an artifact built with
155+
flags `mcpp build` would not.
156+
157+
**Debug information is stripped, and the publisher's paths go with it.** An
158+
unstripped artifact carries DWARF, and DWARF carries the absolute paths of the
159+
producer's source tree and build directory. What is removed depends on what the
160+
artifact *is* — this is dh_strip's division, and the archive row is the one that
161+
matters:
162+
163+
| artifact | strip flags | why not more |
164+
|---|---|---|
165+
| executable | `--strip-all` | nothing links against it |
166+
| shared library | `--strip-unneeded` | keeps `.dynsym` — that IS the export list |
167+
| static archive | `--strip-debug --enable-deterministic-archives` | `--strip-all` removes the archive **symbol index**, and the consumer's link then fails with `archive has no index; run ranlib to add one` |
168+
169+
All three also drop `.comment` and `.note`. Section removal is by exact name, so
170+
`.note.gnu.build-id` survives and still pairs with `--add-gnu-debuglink`.
171+
172+
`--no-strip` (or `[pack] strip = false`) ships the artifacts exactly as built.
173+
`--debug-symbols <dir>` separates the information instead of discarding it:
174+
`<dir>/<artifact>.debug` is written and the shipped artifact gets a
175+
`.gnu_debuglink` pointing at it, which is what a debugger and `debuginfod`
176+
follow.
177+
178+
> `[pack] strip` is not `[profile.<name>].strip`. The profile key appends `-s`
179+
> to the **link**, which never touches a static archive and cannot separate
180+
> anything; this one governs what the **package** carries. Two different
181+
> decisions, two different names.
182+
183+
**Bundled libraries are never stripped.** They came out of the store or off the
184+
host, mcpp did not build them, and rewriting somebody else's shared payload for
185+
this bundle's benefit is not the packer's business.
186+
143187
## Output Layout
144188

145189
The tarball contents are wrapped in a single top-level directory whose name
@@ -310,16 +354,33 @@ The reverse direction — packing a Linux or macOS artifact *from* Windows —
310354
still does not work, and for the original reason: that closure is resolved by
311355
the target's own dynamic linker, which a Windows host has no way to run.
312356

357+
#### Packing a Mach-O program is refused — on every host, including macOS
358+
359+
The same closure step asks the dynamic linker for the dependency list by running
360+
the artifact with `LD_TRACE_LOADED_OBJECTS=1`. That variable is glibc's; dyld
361+
has never heard of it. So on a Mac the command does not trace anything — **it
362+
runs the program**, and whatever the program prints is then parsed as a
363+
dependency table. mcpp refuses instead, and says which mechanism is missing.
364+
365+
The refusal is keyed on the artifact's **format**, not on the host, for the same
366+
reason the Windows one is: `LD_TRACE_LOADED_OBJECTS` cannot trace a Mach-O from
367+
Linux either.
368+
369+
A `kind = "lib"` / `"shared"` target packs normally on macOS — a library package
370+
never runs the artifact. This restriction is only for programs.
371+
313372
## Configuration
314373

315374
Packaging behavior is configured via the `[pack]` section in `mcpp.toml`. The
316375
common fields are:
317376

318377
```toml
319378
[pack]
320-
default_mode = "static" # override the normal vendored default for bare `mcpp pack`
321-
include = ["share/**", "config/*.toml"] # extra files to bundle
322-
exclude = ["debug/**"]
379+
default_mode = "static" # override the normal vendored default for bare `mcpp pack`
380+
strip = true # default. false ships the artifacts as built
381+
debug_symbols = "dist/debug" # separate the debug info here instead of discarding it
382+
include = ["share/**", "config/*.toml"] # extra files to bundle
383+
exclude = ["debug/**"]
323384

324385
# Fine-tune the vendored filtering policy. The configuration key keeps its
325386
# established `bundle-project` spelling.
@@ -339,7 +400,11 @@ The `static` mode additionally requires a musl toolchain configured under
339400

340401
## Planned Support
341402

342-
macOS dylib, Windows DLL, and distribution formats such as `.deb` / `.rpm` /
343-
AppImage are still on the roadmap. This document evolves alongside the
403+
macOS **program** bundling (the Mach-O dependency closure, via `otool -L` /
404+
`LC_LOAD_DYLIB`, and `install_name_tool` for relocation) is still on the
405+
roadmap; until it lands `mcpp pack <program>` refuses on that format rather than
406+
producing something that only looks like a bundle. Windows DLL bundling beyond
407+
the current `.zip`, and distribution formats such as `.deb` / `.rpm` / AppImage,
408+
are also on the roadmap. This document evolves alongside the
344409
`mcpp pack` implementation; for the latest options, refer to
345410
`mcpp pack --help`.

docs/12-binary-distribution.md

Lines changed: 66 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -256,16 +256,77 @@ That is a degradation, not a break, and it is the right direction. But it means
256256
**the gate protects new clients only**, which belongs in the release notes of any
257257
package published to a mixed-version audience.
258258

259+
## What travels inside a package, and what deliberately does not
260+
261+
A published package must work on a machine that is not the publisher's. Two
262+
steps enforce that, and both run on every artifact the packer stages.
263+
264+
### The build machine's loader paths are removed
265+
266+
A dev build bakes the toolchain's own directories into every shared object:
267+
268+
```text
269+
DT_RUNPATH = <home>/registry/data/xpkgs/xim-x-glibc/2.44/lib64
270+
: <home>/registry/data/xpkgs/xim-x-gcc/16.1.0/lib64
271+
: <home>/registry/subos/default/lib
272+
```
273+
274+
That is correct for a dev build and fatal for a package (issue #460), because
275+
of one rule in the ELF loader: **an object that carries any `DT_RUNPATH` makes
276+
the loader skip the entire inherited `DT_RPATH` chain when resolving that
277+
object's own dependencies.** The consumer's `DT_RPATH` — payload, package
278+
directory, SubOS farm, all computed on the machine that will actually run it —
279+
is therefore not consulted, and the program dies with
280+
281+
```text
282+
error while loading shared libraries: libstdc++.so.6: cannot open shared object file
283+
```
284+
285+
⚠️ **`$ORIGIN` is not the fix.** Measured on a real package with the build
286+
machine's store made unreachable:
287+
288+
| state on the shipped `.so` | consumer's `DT_RPATH` inherited? | result |
289+
|---|---|---|
290+
| stale absolute `DT_RUNPATH` | no | fails |
291+
| **no tag at all** | **yes** | **runs** |
292+
| `DT_RUNPATH = $ORIGIN` | no | fails |
293+
| `DT_RUNPATH = ""` | no | fails |
294+
295+
It is the tag's *presence* that disables inheritance, not its contents. So
296+
`mcpp pack` removes the entry rather than rewriting it — and removing it is not
297+
a compromise, it is the right answer: the consumer's own `DT_RPATH` is the same
298+
closure, resolved where it means something.
299+
300+
The path *string* stays in `.dynstr`, unreferenced. `.dynstr` is tail-merged by
301+
the linker, so a shorter live string can begin inside the dead one and deleting
302+
those bytes cannot be shown safe; `patchelf --remove-rpath` leaves the identical
303+
residue at the identical file size. **A guard for this must therefore read the
304+
dynamic entries, never `grep` the file's bytes** — see
305+
`tests/e2e/_elf_tag.sh`.
306+
307+
On Mach-O the packer reads `LC_RPATH` and warns when a package would carry one;
308+
rewriting it (`install_name_tool -delete_rpath`) is not automated yet, because
309+
no test in this suite produces a `.dylib` to measure the edit on.
310+
311+
### Debug information is removed
312+
313+
See [docs/02](02-pack-and-release.md) for the flags, the per-shape table, and
314+
`--debug-symbols`. The rule that matters for a *library* package: a static
315+
archive is only ever `--strip-debug`ed, because `--strip-all` removes the
316+
archive symbol index and the consumer's link then fails with `archive has no
317+
index; run ranlib to add one`.
318+
259319
## Current limitations
260320

261321
| | status |
262322
|---|---|
263323
| `kind = "lib"` (static) | ✅ every target, tested on all three |
264-
| `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME |
324+
| `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME, and no build-machine loader path |
265325
| `kind = "shared"` on PE / MinGW (`*-windows-gnu`) | ✅ — the package carries the `.dll` **and** its import library |
266-
| `kind = "shared"` on Mach-O (`*-macos`) | ✅ — install name is `@rpath/<file>`, so the `.dylib` relocates |
326+
| `kind = "shared"` on Mach-O (`*-macos`) | ✅ — install name is `@rpath/<file>`, so the `.dylib` relocates. `LC_RPATH` is reported, not yet rewritten |
267327
| `kind = "shared"` on PE / MSVC (`*-windows-msvc`) | ✅ — mcpp generates the `.def`; see below |
268328
| `kind = "shared"` on `*-musl` | ❌ a musl target links statically |
329+
| one package carrying two ABIs for the same triple (gcc **and** clang) | ❌ leg selection is `cfg(arch/os/env)`; publish one package per ABI |
269330
| shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact |
270331
| bundling dependencies into the package | ❌ declare them instead (above) |
271332
| consuming a package with **native `cl.exe`** | ✅ — via the neutral link intent; see below |
@@ -415,6 +476,9 @@ The e2e suite gates each test on host capabilities, so "the suite is green" and
415476
| Mach-O shared library relocating out of its build tree ||||
416477
| MSVC refusing `kind = "shared"` for the export reason ||||
417478
| a released mcpp consuming a package this one produced | local only | local only | local only |
479+
| a packed `.so` carries no build-machine loader path, **and the guard can see the defect when it is put back** ||||
480+
| a stripped static archive still links; a stripped shared library still loads; `--no-strip` / `[pack] strip` / `--debug-symbols` from both sides ||||
481+
| the ELF editor on ELF32 and big-endian | unit test | unit test | unit test |
418482

419483
*impossible* is not a gap: a macOS host can serve exactly one target
420484
(`host_can_serve`, `registry.cppm`), so a package with two legs cannot be produced

0 commit comments

Comments
 (0)