Skip to content

Commit 84e6520

Browse files
committed
feat(pack): ship a library as interface + prebuilt binaries (#433)
A closed-source library, an offline site, or a build farm that already compiled this once had no route through mcpp: `mcpp publish` sends source, `mcpp pack` bundles a program's run-time closure, and neither is "a library someone else links". Collecting the artifacts by hand was the answer. mcpp pack mathkit # a static library package mcpp pack mathkit --target x86_64-linux-gnu \ --target aarch64-linux-gnu # one package, two legs WHAT IT PRODUCES IS AN ORDINARY PACKAGE. A normal `mcpp.toml`, read through the route mcpp already had for payloads that carry their own manifest. Zero new manifest sections and zero new keys: what to pack is `[targets.<n>].kind` (so there is no --lib and no --artifact), which interface to publish is `[lib]` plus the module graph, which headers are public is `[build].include_dirs`, and each leg's ABI tag and digest ride on `[[runtime.artifacts]]`, whose fields were already documented as optional evidence. An older mcpp still BUILDS against these packages; it just does not run the two gates below. Two interface modes coexist — `include/` (text, `#include`, never compiled) and `interface/` (module units the consumer compiles). Measured: one package consumed three ways (header only, module only, both) x static and shared = six combinations, all green. WHICH .cppm TRAVEL IS COMPUTED, not declared. It is the module closure of the lib root, and the two ways of getting it wrong are asymmetric: too few fails loudly in the consumer's compile, too many silently publishes a closed-source implementation partition's SOURCE. `.m.o` is not the rule — an implementation partition produces one too. The same closure decides which archive members to drop, and getting THAT wrong was measured as well: dropping every `.m.o` also drops the partition's real code and every target fails to link. Both lists are printed, because "what is not travelling" is half of what a publisher needs. TWO GATES ON THE CONSUMER SIDE, both for failures that are otherwise silent. The interface still matches its binaries: swap two `int` members of a shipped struct — the Itanium ABI does not mangle field order — and before this the consumer compiled, linked, ran, and printed transposed data with no diagnostic from any tool. And the binaries were built for this toolchain, with the refusal listing the tags the package does have, because "not found" sends people looking for a package already on their disk. Also fixes two defects this work found, each with its own regression test: * `[target.'<triple>'.build]` never matched a native build. `matches()` short-circuited on the raw --target string while `context_for()` fell back to the host for `cfg(...)`, so two spellings of one statement disagreed — green in CI, silently inert on a developer's machine. The resolved triple now lives in `cfgpred::Ctx`; there is no second answerer. * `sources = []` was byte-identical to omitting the key, so no author could say "compile nothing". A header-only package needs that, and without it any leftover file under `src/` is compiled into the consumer's build. Docs: docs/12-binary-distribution.md (+ zh), examples 05-lib-dist and 06-lib-consume. Design: .agents/docs/2026-08-17-library-distribution-design.md. Tests: 23 unit assertions across two new suites; e2e 242-248.
1 parent 9c5cc8d commit 84e6520

49 files changed

Lines changed: 5830 additions & 40 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md

Lines changed: 1615 additions & 0 deletions
Large diffs are not rendered by default.

.agents/docs/2026-08-17-library-distribution-design.md

Lines changed: 585 additions & 0 deletions
Large diffs are not rendered by default.

CHANGELOG.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,74 @@
55
66
## [Unreleased]
77

8+
### 新增
9+
10+
- **`mcpp pack <target>` 可以把一个库打成「接口 + 预编译二进制」的包(#433)。**
11+
12+
闭源库、离线环境、以及「构建农场已经编过一遍了」这三种场景,过去都只能自己
13+
写脚本收集产物。现在:
14+
15+
```bash
16+
mcpp pack mathkit # 静态库包
17+
mcpp pack mathkit --target x86_64-linux-gnu \
18+
--target aarch64-linux-gnu # 一个包,两条腿
19+
```
20+
21+
**产出的是一个普通的 mcpp 包** —— 一份正常的 `mcpp.toml`,走 mcpp 早就有的
22+
「载荷自带 manifest」通路。**新增 manifest 段 0 个、键 0 个**:打什么由
23+
`[targets.<n>].kind` 决定(所以没有 `--lib`、没有 `--artifact`),发布哪些接口
24+
`[lib]` 约定 + 模块图决定,公开头是 `[build].include_dirs` 全量,
25+
每条腿的 ABI tag 与 digest 记在既有的 `[[runtime.artifacts]]` 上。
26+
一个**老版本 mcpp 照样能构建**这种包 —— 它只是不执行下面那两道闸门。
27+
28+
一个包可以同时带**两种接口**:`include/`(文本,`#include`,不编译)与
29+
`interface/`(模块,消费者编译它)。实测同一个包被「只 #include」/「只 import」/
30+
「两者都用」三种方式消费,静态与动态两种形态,六格全过。
31+
32+
发布哪些 `.cppm`**算出来的** —— lib root 的模块闭包,不是「所有 `.m.o`」。
33+
实现分区(`module M:secret;`)照样产 `.m.o`,按扩展名挑会**泄露闭源源码**;
34+
同一个闭包反过来决定归档里要删哪些对象,按 `.m.o` 删则会删掉真代码、
35+
三个平台全部链接失败。两条清单都会打印出来。
36+
37+
详见 `docs/12-binary-distribution.md``examples/05-lib-dist``examples/06-lib-consume`
38+
39+
- **消费预编译包时的两道闸门。** 都是不检查就会静默出错的:
40+
41+
**接口与二进制是否仍然配对。** 这条闸门存在是因为另一种结果被实测过:把随包
42+
接口里一个结构体的两个 `int` 成员互换 —— Itanium ABI 不 mangle 字段顺序 ——
43+
消费者**编译过、链接过、运行过、打印出交换后的错数据**,任何工具都没有一句诊断。
44+
45+
**二进制是否为这套工具链所编。** 失配时诊断会**列出包里确实有哪些 tag** ——
46+
一句「找不到」会让人去找一个就在自己硬盘上的包。
47+
48+
另外,在解开的分发包目录里直接 `mcpp build` 会被拒绝:那儿的 `interface/`
49+
是声明,定义在旁边的归档里,构建会产出一个几乎空的库然后报告成功。
50+
851
### 修复
952

53+
- **`[target.'<三元组>'.build]` 在没有 `--target` 时从不命中。**
54+
55+
同一个语句的两种拼写互相矛盾:`cfg(linux)` 在原生构建上命中,
56+
`[target.'x86_64-linux-gnu'.build]` 不命中。根因是 `matches()` 拿着原始的
57+
`--target` 字符串(原生构建下是空的)短路返回 false,而同一文件的
58+
`context_for()``cfg(...)` **回落到宿主三元组** —— 一个决定两处推导。
59+
`manifest/types.cppm` 的注释从写下起承诺的就是回落那一种。
60+
61+
**形状是最坏的那种**:CI`--target` 是绿的,开发者本机的 `mcpp build`
62+
静默丢掉那一段,失败在链接期出现、点的是符号而不是谓词。
63+
64+
修法是**删掉第二个答题者**:解析后的三元组进 `cfgpred::Ctx`,`matches()`
65+
只有一个来源。
66+
67+
- **`sources = []` 与不写 `sources` 逐字节等价。**
68+
69+
解析器在向量为空时一律填默认 glob,于是作者**没有任何写法**能表达
70+
「什么都不要编」。二进制分发需要这个:一个纯头文件的包不编译任何东西,
71+
`src/` 下任何遗留文件都会被扫进消费者的构建,并可能与预编译库里的符号
72+
重复定义。改成记录**键是否出现**(`BuildConfig::sourcesDeclared`),
73+
`XlingsConfig::subosDeclared` 同一个模式。
74+
75+
1076
- **卸载后的清扫会波及**别的版本**,而那可能正在被另一个进程解压。**
1177

1278
`sweep_parked_payloads` 原来把整个 family 目录扫一遍,把**任何**没有文件的

docs/02-pack-and-release.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,12 @@
1-
# 02 — Packaging for Release
1+
# 02 — Packaging an Application for Release
2+
3+
> This page is about bundling a **program**. To ship a *library* as interface +
4+
> prebuilt binaries, see [12 - Distributing a Prebuilt Library](12-binary-distribution.md).
5+
>
6+
> Which one `mcpp pack` does is decided by the target's `kind`, not by a flag:
7+
> `mcpp pack <name>` packs `[targets.<name>]`, and a `bin` becomes a bundle
8+
> while a `lib`/`shared` becomes a library package. With no name, mcpp picks
9+
> the only packable target.
210
311
> A default dynamically linked binary produced by `mcpp build` has a loader and
412
> RUNPATH tied to the build sandbox. It is a development artifact, not a

docs/05-mcpp-toml.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,12 @@ the package/feature boundary, not on an individual target.
153153

154154
### 2.3 `[build]` — Build Configuration
155155

156+
> **`sources = []` is not the same as omitting `sources`.** An absent key
157+
> selects the default glob; an explicitly empty list means *compile nothing*,
158+
> which is what a header-only distribution package needs to say. Until
159+
> mcpp 2026.8.17.2 the two were byte-identical, so there was no spelling for
160+
> "nothing" and any file left under `src/` was swept in.
161+
156162
```toml
157163
[build]
158164
sources = ["src/**/*.cppm", "src/**/*.cpp"] # Source globs (default: src/**/*.{cppm,cpp,cc,c,S,s,asm})
@@ -1598,6 +1604,17 @@ do`.
15981604
- Package-level knobs all converge into features; for sugar keys (such as `backend=`)
15991605
to enter the core syntax, they must satisfy: ① domain-neutral (a cross-ecosystem
16001606
general pattern) ② 1:1 desugaring with zero new parsing semantics.
1607+
- **A key that duplicates an answer another section already gives is not admitted.**
1608+
Two places to state one fact is two places that can disagree, and the failure
1609+
is silent — whichever reader loses the race is simply wrong. Library packaging
1610+
([12](12-binary-distribution.md)) is the worked example: it added **zero**
1611+
manifest keys, because what to pack is `[targets.<n>].kind`, which interface
1612+
to publish is `[lib]` plus the module graph, which headers are public is
1613+
`[build].include_dirs`, and the per-artifact evidence is `[[runtime.artifacts]]`.
1614+
- A field that describes what a *generated* package IS (rather than what a build
1615+
should DO) belongs on `[[runtime.artifacts]]` — see §2.11. `provenance`
1616+
beginning with `mcpp-pack` is what marks a directory as one, and mcpp refuses
1617+
to `build` inside it.
16011618
- See `.agents/docs/2026-06-04-manifest-schema-ownership.md` for the full field-ownership
16021619
table and the finalized decisions.
16031620

docs/12-binary-distribution.md

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
# 12 - Distributing a Prebuilt Library
2+
3+
**English** | [简体中文](zh/12-binary-distribution.md)
4+
5+
> Ship a library as **interface + prebuilt binaries** instead of as source.
6+
> This is the closed-source case, and the offline case, and the "our build farm
7+
> already compiled this once" case.
8+
>
9+
> [02 - Packaging & Release](02-pack-and-release.md) is the sibling: bundling an
10+
> *application*. [10 - Publishing a Library](10-publishing-a-library.md) is the
11+
> source route.
12+
13+
## The whole idea in one paragraph
14+
15+
`mcpp pack <target>` builds a library target and writes a directory that is an
16+
**ordinary mcpp package** — a normal `mcpp.toml`, the interface a consumer must
17+
compile, and the binaries it then links. Consumers use it exactly like any
18+
other dependency. There is no new manifest section, no new archive format, and
19+
no new resolution path.
20+
21+
```bash
22+
mcpp pack mathkit # a static library package
23+
mcpp pack mathkit-shared # a dynamic one (Linux/ELF today)
24+
mcpp pack mathkit --target x86_64-linux-gnu \
25+
--target aarch64-linux-gnu # one package, two legs
26+
```
27+
28+
## What decides what gets packed
29+
30+
`[targets.<name>].kind`, and nothing else:
31+
32+
| `kind` | `mcpp pack <name>` produces | `--mode` |
33+
|---|---|---|
34+
| `bin` | an application bundle (see [02](02-pack-and-release.md)) | the four depths |
35+
| `lib` | a **static library package** ||
36+
| `shared` | a **dynamic library package** ||
37+
38+
There is no `--lib` flag and no `--artifact static\|shared`. `kind` is already
39+
where mcpp records what an artifact is; a flag would be a second place to say
40+
it, and two places can disagree. A project that publishes both forms declares
41+
both targets — which it must do for `mcpp build` to produce both anyway:
42+
43+
```toml
44+
[targets.mathkit]
45+
kind = "lib"
46+
47+
[targets.mathkit-shared]
48+
kind = "shared"
49+
soname = "libmathkit.so.1"
50+
```
51+
52+
Run `mcpp pack` with no name and mcpp picks the only packable target, or tells
53+
you which ones it found.
54+
55+
## The two interface modes
56+
57+
A package can carry both, and a consumer may use either or both.
58+
59+
```
60+
mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/
61+
├── mcpp.toml
62+
├── include/ ← TEXT interface: #include, never compiled
63+
├── interface/ ← MODULE interface: the consumer compiles it
64+
└── lib/<triple>/ ← the artifacts
65+
```
66+
67+
| | `include/` | `interface/` |
68+
|---|---|---|
69+
| whose input is it | the preprocessor's | the **compiler's** |
70+
| does the consumer compile it | no | **yes**, to get a BMI |
71+
| what it constrains | the libc ABI | compiler, C++ stdlib, C++ level |
72+
| can you trim it | **no** — see below | **no**, it is computed |
73+
74+
`lib/` is keyed by **triple**, not by OS. MinGW and MSVC are both Windows and
75+
produce `libfoo.a` and `foo.lib` respectively.
76+
77+
### Why neither set can be trimmed
78+
79+
A **source** distribution of the same package puts every one of its
80+
`include_dirs` on its consumers' include path. If a binary package shipped a
81+
subset, the same library would have a different public surface depending on how
82+
it was delivered. "Which headers are public" is already answered by the layout:
83+
`include/` is public, `src/` is not. A private header under `include/` is a
84+
project-layout mistake, not a packaging option.
85+
86+
## Which `.cppm` files travel
87+
88+
The **module closure of the lib root**`src/<package-tail>.cppm` by
89+
convention, or `[lib].path`. Whatever that unit's purview imports, transitively,
90+
is published; everything else is not.
91+
92+
```
93+
src/mathkit.cppm export module mathkit; export import :api; → published
94+
src/api.cppm export module mathkit:api; → published
95+
src/secret.cppm module mathkit:secret; ← implementation partition
96+
src/impl.cpp module mathkit; → withheld
97+
```
98+
99+
`mcpp pack` prints both lists:
100+
101+
```
102+
Interface mathkit.cppm, api.cppm
103+
Withheld capi.c, impl.cpp, secret.cppm
104+
```
105+
106+
**Read the second one** if you are shipping closed source.
107+
108+
> **`.m.o` is not the rule.** An implementation partition produces a BMI and an
109+
> object exactly like an interface unit does. Selecting sources by "does it
110+
> produce a BMI" would publish `secret.cppm`.
111+
112+
If the published interface *does* import an implementation partition, a
113+
consumer cannot compile it without that source — so `mcpp pack` stops:
114+
115+
```
116+
error: the published interface imports mathkit:secret , which no unit in this
117+
build provides.
118+
```
119+
120+
Restructure so the interface does not reach it, or make it an `export module`
121+
partition and accept that its source is published.
122+
123+
## The compatibility tag
124+
125+
Every artifact records the toolchain it was built for:
126+
127+
```
128+
x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface
129+
x86_64-linux-gnu # an extern "C" interface only
130+
```
131+
132+
`<arch>-<os>-<env>` then, when the interface is C++, `<compiler><major>`,
133+
`<stdlib><major>`, `c++<level>`.
134+
135+
**A shorter tag is a real statement, not a missing one.** A library whose whole
136+
interface is `extern "C"` constrains the libc ABI and not the C++ one, so it
137+
publishes a triple and stops — and links into any compiler. Unnamed dimensions
138+
are don't-care, so one tag per triple instead of one per triple per compiler.
139+
Nothing to configure: the shape is the statement.
140+
141+
The `c++` level is compared as a **floor**, not for equality: building at a
142+
higher level is fine, lower is not.
143+
144+
## What a consumer's build checks
145+
146+
Two things, both of which fail silently without a check:
147+
148+
**The interface still matches its binaries.**
149+
150+
```
151+
error: acme.mathkit@0.1.0: 'interface' does not match what was packaged.
152+
recorded fnv1a:25b2cf2a79d71c40
153+
found fnv1a:fe404d5be85118ff
154+
```
155+
156+
This exists because the alternative was measured. Swap two `int` members of a
157+
struct in a shipped interface — the Itanium ABI does not mangle field order —
158+
and the consumer compiles, links, runs, and prints transposed data, with no
159+
diagnostic from any tool. A digest cannot stop a publisher from shipping a
160+
mismatched pair (only producing both in one command does that), but it does
161+
catch the pair coming apart afterwards.
162+
163+
**The binaries were built for this toolchain.**
164+
165+
```
166+
error: acme.mathkit@0.1.0: no prebuilt artifact matches this toolchain.
167+
your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23
168+
published tags :
169+
x86_64-linux-gnu-gcc15-libstdcxx15-c++23
170+
closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on:
171+
compiler needs gcc15, this build has gcc16
172+
stdlib needs libstdcxx15, this build has libstdcxx16
173+
```
174+
175+
The tags it *does* have are part of the message: "not found" would send you
176+
looking for a package already on your disk.
177+
178+
## Consuming one
179+
180+
Three spellings, one code path:
181+
182+
```toml
183+
# a directory (what you hand a colleague)
184+
mathkit = { path = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" }
185+
186+
# a private git repo
187+
mathkit = { git = "ssh://git@internal/mathkit-dist.git", tag = "v0.1.0" }
188+
189+
# an index entry — identical in shape to a source package's
190+
mathkit = "0.1.0"
191+
```
192+
193+
Nothing about the consumer's manifest says "this one is prebuilt".
194+
195+
### Building *inside* a package is refused
196+
197+
```
198+
error: … is a distribution package produced by `mcpp pack`, not a source tree.
199+
```
200+
201+
Its `interface/` holds declarations whose definitions are in the archive beside
202+
them. Building there compiles the declarations, produces a near-empty library
203+
and reports success.
204+
205+
## One package, several targets
206+
207+
`--target` is repeatable. The generated manifest gets one conditional block per
208+
leg, and the consumer's build picks its own:
209+
210+
```toml
211+
[target.'cfg(all(arch = "x86_64", os = "linux", env = "gnu"))'.build]
212+
ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"]
213+
214+
[target.'cfg(all(arch = "x86_64", os = "linux", env = "musl"))'.build]
215+
ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"]
216+
```
217+
218+
Because selection happens in the **consumer's** build, where the resolved
219+
target is known, a fat package cross-compiles correctly with no index-side or
220+
installer-side support at all.
221+
222+
> The blocks are `cfg(...)` and never a bare `[target.'<triple>']` key. Before
223+
> mcpp 2026.8.17.2 the bare form was inert without an explicit `--target`, so a
224+
> package using it would work in CI and silently drop its flags on a
225+
> developer's machine. mcpp generates the spelling that means the same thing on
226+
> every client.
227+
228+
## Dependencies
229+
230+
A static archive does **not** carry its dependencies' code, so the package
231+
records them and the consumer resolves them:
232+
233+
```toml
234+
[dependencies]
235+
"compat.zlib" = "1.3.2"
236+
```
237+
238+
`path` and `git` dependencies are dropped: they address the publisher's disk,
239+
and republishing one hands the consumer an address that means something else.
240+
If your library depends on one, either publish that dependency too or vendor it
241+
before packing.
242+
243+
## What older mcpp does with these packages
244+
245+
**It builds against them.** Every key in the generated manifest already
246+
existed, so an older client reads the package and links it. What it does not do
247+
is run the two checks above — it has no way to know that `provenance =
248+
"mcpp-pack …"` means anything.
249+
250+
That is a degradation, not a break, and it is the right direction. But it means
251+
**the gate protects new clients only**, which belongs in your release notes if
252+
you publish to a mixed audience.
253+
254+
## Current limits
255+
256+
| | status |
257+
|---|---|
258+
| `kind = "lib"` (static) | ✅ every target |
259+
| `kind = "shared"` on Linux/ELF ||
260+
| `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet |
261+
| `kind = "shared"` on `*-musl` | ❌ a musl target links statically |
262+
| shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact |
263+
| bundling dependencies into the package | ❌ declare them instead (above) |

docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
- [09 - Releasing mcpp](09-release.md)
1515
- [10 - Publishing a Library to mcpp-index](10-publishing-a-library.md)
1616
- [11 - Machine-Readable Output](11-machine-output.md)
17+
- [12 - Distributing a Prebuilt Library](12-binary-distribution.md)
1718

1819
## Specifications
1920

0 commit comments

Comments
 (0)