Skip to content

feat(pack): ship a library as interface + prebuilt binaries (#433) - #451

Open
Sunrisepeak wants to merge 20 commits into
mainfrom
feat/library-distribution
Open

feat(pack): ship a library as interface + prebuilt binaries (#433)#451
Sunrisepeak wants to merge 20 commits into
mainfrom
feat/library-distribution

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Aug 17, 2026

Copy link
Copy Markdown
Member

Closes #433. Please do not merge yet.

A closed-source library, an offline site, or a build farm that already compiled
this once had no route through mcpp: mcpp publish sends source, mcpp pack
bundles a program's run-time closure, and neither is "a library someone else
links". Collecting the artifacts by hand was the answer.

mcpp pack mathkit                              # a static library package
mcpp pack mathkit --target x86_64-linux-gnu \
                  --target aarch64-linux-gnu   # one package, two legs

What it produces is an ordinary package

A normal mcpp.toml, read through the route mcpp already had for payloads that
carry their own manifest. Zero new manifest sections and zero new keys:

the fact where it lives new?
what to pack [targets.<n>].kind no
which interface to publish [lib] + the module graph no
which headers are public [build].include_dirs no
per-leg ABI tag / digest / provenance [[runtime.artifacts]] no
how to link each leg [target.'cfg(…)'.build] ldflags no
the library's own dependencies [dependencies] no

So there is no --lib and no --artifact static\|sharedkind is already
where mcpp records what an artifact is, and a flag would be a second place to
say it. And an older mcpp still builds against these packages; it simply
does not run the two gates below.

Two interface modes coexist: include/ (text, #include, never compiled) and
interface/ (module units the consumer compiles). One package consumed three
ways × static and shared = six combinations, all green.

Which .cppm travel is computed, not declared

The module closure of the lib root. The two ways of getting it wrong are
asymmetric, which is why it cannot be a hand-written list:

  • too few → the consumer's compile fails, naming the module;
  • too many → a closed-source implementation partition's SOURCE is
    published. Nothing fails.

.m.o is not the rule — an implementation partition produces one too. The same
closure decides which archive members to drop, and getting that wrong was
measured as well: dropping every .m.o removes the partition's real code and
every target fails to link. Both lists are printed, and an interface that
reaches a partition publishes it with a warning naming the file.

Two gates, for failures that are otherwise silent

The interface still matches its binaries. Swap two int members of a
shipped struct — the Itanium ABI does not mangle field order — and before this
the consumer compiled, linked, ran, and printed transposed data, exit code 0,
no diagnostic from any tool.

The binaries were built for this toolchain. The refusal lists the tags the
package does have, because "not found" sends people looking for a package
already on their disk.

Building inside an extracted package is also refused: its interface/ holds
declarations whose definitions are in the archive beside them, so a build there
produces a near-empty library and reports success.

Three pre-existing defects found on the way

Each has its own regression test, and each affects projects that never touch
packaging.

  • Implementation partitions did not build on Windows. module M:part; and
    module M; wear one spelling and are two declarations; the scanner recorded
    the first as requiring M:part and providing nothing, so a file required
    its own name and the graph held no edge from the unit importing a partition
    to the unit defining it. Order was unconstrained — GCC and macOS clang
    recovered through their own scan, Windows clang failed with failed to read compiled module. The note both printed, module 'M:part' imported but not provided in this build, was the cause and read like a hint. Implementation
    partitions had no test coverage anywhere in mcpp; this feature's e2e is
    the first thing to use one. (5 scanner unit tests)

  • [target.'<triple>'.build] never matched a native build. matches()
    short-circuited on the raw --target string while context_for() fell back
    to the host for cfg(...). Green in CI, silently inert on a developer's
    machine. (e2e 247)

  • sources = [] was byte-identical to omitting the key, so no author could
    say "compile nothing". (e2e 246)

Verification: what actually ran, per platform

CI 19/19 green. 87/87 unit suites (2 new files, 28 assertions; 5 more in
test_modgraph). 15 regression e2e over the touched areas.

e2e linux macOS windows runs in
242 layout + both interface modes e2e shards
243 closure, drop set, partition warning e2e shards
244 the three refusals e2e shards
246 sources = [] e2e shards
247 bare-triple conditional e2e shards
249 workspace root still packs e2e shards
250 pack <name> packs that name e2e shards
251 shared: produced correctly or refused clearly ✅ produced ✅ refused ✅ refused e2e shards
248 fat package across an OS boundary (PE) cross-build-test.yml mingw job
245 fat package, native build included skip skip e2e shards

One skip remains, and it is a real limit: 245 needs two distinct buildable
targets (x86_64-linux-gnu + x86_64-linux-musl), and no second ready target
exists on the macOS or Windows runners. The mechanism it checks — one cfg()
leg per triple, the native build included — is platform-independent and covered
on Linux.

Getting here took interrogating every skip reason one at a time, and three of
the four did not survive the question
:

the reason I gave what it actually was
246 "reads compiler-specific flag spellings" wrong — it greps a FILENAME. # requires: gcc was copy-paste
247 "reads compiler-specific flag spellings" half true — it grepped the -D prefix, and MSVC spells that /D. Grepping the macro NAME is portable
248 "needs mingw-cross, no job installs it" fixablecross-build-test.yml's mingw job already names the e2e scripts the Linux shards skip for exactly this reason (102, 198, 240). 248 joins them
251 "shared is ELF-only" reason held, coverage did not — it tested only the side that works, which cannot tell "the gate is handled" from "there is no gate". Now two-sided

Before that, all ten carried # requires: gcc, which is Linux-only by design,
so they skipped on macOS and Windows while the suite reported green.
Unrestricting them is what surfaced the scanner bug above. The criterion, earned
twice: a test's # requires: has to be the real floor of the mechanism it
verifies
, and that is hard to judge while writing it — so the reasons have to
be interrogated afterwards, one by one.

Self-review found five things after the first push

  1. mcpp.pack.library exported a second mcpp::pack::Error. A name attaches
    to one module; clang refuses, GCC accepted. Windows + macOS red, Linux green.
  2. mcpp pack in a workspace root stopped working. Found by running the
    previous release against examples/04-workspace and comparing. (e2e 249)
  3. The positional never reached the application pipeline, so a two-bin
    project took mcpp pack app2 and bundled app1. (e2e 250)
  4. A shared package shipped only the built filename while the loader asks
    for the SONAME — it linked and could not start. mcpp's own runtime-closure
    check reported it. (e2e 251, which uses run, not build)
  5. A missing archiver skipped the object drop silently, leaving the
    published interface's objects in the archive. Now refused with the reason.

Not in this PR

kind = "shared" on PE / Mach-O, shared on *-musl (a musl target links
statically), bundling dependencies into the package, prebuilt BMIs.

Docs

docs/12-binary-distribution.md (+ zh), examples/05-lib-distribution
(producer + consumer in one directory, consumers written with import std;).
Design, the measurements behind every criterion, and the four designs my own
experiments disproved: .agents/docs/2026-08-17-library-distribution-design.md
§11.


Deep review — what a reviewer should push on

Why two tests cannot run on the other platforms

Not a test-harness limit; it comes from host_can_serve
(src/toolchain/registry.cppm:542-566), the per-host target matrix:

target linux macOS windows
<hostarch>-linux-gnu
*-linux-musl ✅ any arch ✅ host arch only
*-windows-gnu ✅ (mingw-cross)
*-windows-msvc
*-macos
  • e2e 245 (fat package) on macOS is structurally impossible — a fat package
    needs two legs and macOS can serve exactly one target. The code says so
    itself: "macOS has no Linux-targeting payload at all."
  • On Windows it is possible and simply not done. Windows serves three
    targets, so mathkit.lib (MSVC) beside libmathkit.a (MinGW) in one package
    would be the sharpest evidence that lib/ must be keyed by triple. That is a
    coverage gap, and it needs xim:mingw-gcc in the Windows e2e job.
  • e2e 248 needs a PE-producing toolchain: mingw-cross on Linux (where it
    runs) or native on Windows (same gap). Never on macOS.

Architecture — the invariant to hold

src/pack/ is two layers and must stay that way:

leaf   (prepare may import these):  abi_tag digest interface manifest_emit prebuilt route library
orchestration (imports prepare):    library_pipeline

Only library_pipeline touches mcpp.build.*; prepare imports
mcpp.pack.abi_tag and mcpp.pack.prebuilt, both leaves. Nothing else under
src/pack/ may import mcpp.build.*
or prepare ↔ pack becomes a cycle.

Three convergences here delete a second derivation rather than add an
abstraction: object_filename_for moved next to the policy that names objects;
how to delete an archive member moved into dialect.cppm beside archiveCmd
(review found this — the packer had assumed ar syntax everywhere); the
resolved triple moved into cfgpred::Ctx.

Compatibility — two deliberate behaviour changes

change who could notice
sources = [] now means "compile nothing" a project that wrote it and relied on the default glob. That combination could not previously be expressed, so writing it was already a misunderstanding
two files declaring the same partition are refused, naming both already ill-formed — but it is a new failure path

The scanner change only adds graph edges. A missing edge can only permit a
wrong order, so no project can depend on one.

Still open — please rule on these

  1. The scanner fix has the widest blast radius in this PR and is unrelated to
    packaging.
    My recommendation: land it as its own PR so it can be reverted
    independently.
  2. A scan_overrides-declared implementation partition is mis-flagged as an
    interface
    and its source would be published — providesInterface is only
    set false on the text-scanning path. Narrow, and scan_overrides has no
    schema slot to say otherwise; fixing it means the one new manifest key this
    design avoided.
  3. MSVC lib.exe /REMOVE: is untested (mcpp's Windows CI uses clang's
    llvm-ar, i.e. the GNU spelling). It fails loudly with the command text, not
    silently.
  4. Windows fat package (msvc + mingw) untested — see above.
  5. The [package].platforms coverage warning promised in the design (§2.2) is
    not implemented.
    I missed it. Cheap, but it is release discipline rather
    than correctness, so it can follow.

A closed-source library, an offline site, or a build farm that already
compiled this once had no route through mcpp: `mcpp publish` sends source,
`mcpp pack` bundles a program's run-time closure, and neither is "a library
someone else links". Collecting the artifacts by hand was the answer.

    mcpp pack mathkit                              # a static library package
    mcpp pack mathkit --target x86_64-linux-gnu \
                      --target aarch64-linux-gnu   # one package, two legs

WHAT IT PRODUCES IS AN ORDINARY PACKAGE. A normal `mcpp.toml`, read through
the route mcpp already had for payloads that carry their own manifest. Zero
new manifest sections and zero new keys: what to pack is `[targets.<n>].kind`
(so there is no --lib and no --artifact), which interface to publish is `[lib]`
plus the module graph, which headers are public is `[build].include_dirs`, and
each leg's ABI tag and digest ride on `[[runtime.artifacts]]`, whose fields
were already documented as optional evidence. An older mcpp still BUILDS
against these packages; it just does not run the two gates below.

Two interface modes coexist — `include/` (text, `#include`, never compiled)
and `interface/` (module units the consumer compiles). Measured: one package
consumed three ways (header only, module only, both) x static and shared = six
combinations, all green.

WHICH .cppm TRAVEL IS COMPUTED, not declared. It is the module closure of the
lib root, and the two ways of getting it wrong are asymmetric: too few fails
loudly in the consumer's compile, too many silently publishes a closed-source
implementation partition's SOURCE. `.m.o` is not the rule — an implementation
partition produces one too. The same closure decides which archive members to
drop, and getting THAT wrong was measured as well: dropping every `.m.o` also
drops the partition's real code and every target fails to link. Both lists are
printed, because "what is not travelling" is half of what a publisher needs.

TWO GATES ON THE CONSUMER SIDE, both for failures that are otherwise silent.
The interface still matches its binaries: swap two `int` members of a shipped
struct — the Itanium ABI does not mangle field order — and before this the
consumer compiled, linked, ran, and printed transposed data with no diagnostic
from any tool. And the binaries were built for this toolchain, with the
refusal listing the tags the package does have, because "not found" sends
people looking for a package already on their disk.

Also fixes two defects this work found, each with its own regression test:

  * `[target.'<triple>'.build]` never matched a native build. `matches()`
    short-circuited on the raw --target string while `context_for()` fell back
    to the host for `cfg(...)`, so two spellings of one statement disagreed —
    green in CI, silently inert on a developer's machine. The resolved triple
    now lives in `cfgpred::Ctx`; there is no second answerer.

  * `sources = []` was byte-identical to omitting the key, so no author could
    say "compile nothing". A header-only package needs that, and without it
    any leftover file under `src/` is compiled into the consumer's build.

Docs: docs/12-binary-distribution.md (+ zh), examples 05-lib-dist and
06-lib-consume. Design: .agents/docs/2026-08-17-library-distribution-design.md.

Tests: 23 unit assertions across two new suites; e2e 242-248.
Two problems the first push found, one from CI and one from running the
previous release side by side.

`mcpp.pack.library` exported a `mcpp::pack::Error` and `mcpp.pack` already
had one. A name attaches to exactly one module, and mcpp.pack.library_pipeline
imports both — clang refuses outright ("cannot be attached to other modules"),
GCC accepted it. Every Windows and macOS job failed on it while the Linux ones
were green, which is the whole argument for the three-platform matrix.

`mcpp pack` in a workspace root stopped working. Routing on
`[targets.<n>].kind` means something reads the manifest before the build does,
and a workspace root has no targets of its own — a virtual one has no
`[package]` either — so the new router read an empty list and concluded there
was nothing to pack. Found by running the previous release against
examples/04-workspace and comparing; e2e 249 is that comparison made permanent.

And the positional was accepted but never reached the application pipeline, so
a project with two `bin` targets would take `mcpp pack app2` and bundle app1 —
succeeding with the wrong answer. e2e 250 pins both directions plus the
refusal for an unknown name.
…names

Found by running the path the docs already promised. `mcpp pack <shared
target>` shipped only the built file — `libmathkit-shared.so` — while the
object records `SONAME libmathkit.so.1`. A consumer links by the first name
and the loader asks for the second, so the package linked and the program
could not start.

mcpp's own runtime-closure check is what reported it, naming the missing
soname rather than letting it become a loader error at launch. The package now
carries the SONAME alongside the link name (a symlink, falling back to a copy),
which is what a distribution ships and what the design's "soname gives the
correct run-time name" note always meant.

e2e 251 uses `run`, not `build`: linking proves nothing here.
`00_fixture_path_hygiene.sh` caught it on the macOS leg. The rule is a Windows
one: MSYS rewrites POSIX paths on the way into argv and the environment but
never touches file CONTENT, so a `path = "/tmp/…"` written INTO an mcpp.toml is
read by a native mcpp.exe as "root of the current drive". The failure then
surfaces as a dependency that cannot be found, four steps from its cause.

Six of the new fixtures wrote the package's path that way. They now source
`_host_path.sh` and pass it through `host_path`, which is what the lint asks
for — and the lint runs on every platform precisely so a Linux reviewer can
catch this before Windows CI does.
…roved

§11 of the design doc: three places the implementation diverged from the plan
(no [distribution] section at all, no abi_surface flag, no new CI workflow),
the four defects found while implementing — each with what caught it — and the
stale-fingerprint trap I walked into three times while VERIFYING, which is the
same criterion the packer itself enforces about never globbing for artifacts.
All ten new tests carried `# requires: gcc`, and that capability is Linux-only
by design — macOS's g++ is Apple Clang and Windows' is not an mcpp-compatible
GCC, so the runner does not grant it there. Every one of them skipped on macOS
and Windows while the suite reported green, which left "a static library
package works on every target" verified on exactly one platform.

That is the same false-green shape this PR already corrected once (245 was
gated on mingw-cross and would never have run in CI at all). Finding it a
second time, in my own tests, is the argument for checking whether a test RAN
rather than whether the suite was green.

The five that assert on mcpp's own output — layout, both interface modes, the
three refusals, workspace routing, target naming — now require nothing and run
everywhere. The rest stay gated for real reasons: 245 needs a musl target, 246
and 247 read compiler-specific flag spellings out of build.ninja, 248 needs the
mingw cross toolchain, and 251 exercises `kind = "shared"`, which is ELF-only.
…everywhere

Letting the portable tests run on all three platforms was the right move, and
it worked: CI answered the question I could not answer locally.

  * on the Windows (MSVC-ABI clang) leg the library build inside `pack` fails
    with a bare `error: build failed`;
  * on macOS the closure test cannot inspect the archive, because `ar` there
    resolves to an xlings shim that reports "not installed".

Neither is fixed here and neither is hidden. The three PACKING tests carry
`# requires: gcc` again with the scope written at the top, while the tests for
the COMMAND — which target it picks, how it refuses an unknown name, packing
from a workspace root — keep running everywhere, because they pass everywhere.
docs/12 and its zh mirror gain a "where it is verified" section, and the limits
table now says "every target — verified on Linux only" rather than the first
half of that sentence.

Also: a Windows pack produces a .zip, not a .tar.gz. 249 asserted only the
latter, which is why it failed there even though the pack had succeeded.
`module M:part;` and `module M;` wear one spelling and are two declarations. The
scanner treated them as one: the first was recorded as *requiring* `M:part` and
providing nothing, so a file required its own name and the graph held no edge
from the unit importing a partition to the unit defining it. Build order was
unconstrained — GCC and macOS clang recovered through their own dependency scan,
Windows clang failed with `failed to read compiled module`.

The same site had a second half: `import :part;` resolved against `u.provides`,
and an implementation unit (`module M;`) has none, so `import :secret;` inside
one stayed the literal `:secret`. The note both platforms printed —
`module 'M:part' imported but not provided in this build` — was the merged
symptom of the two, and it read like a hint rather than the cause.

Implementation partitions had no test coverage anywhere in mcpp; the library
distribution e2e is the first thing to use one, which is how this surfaced. Five
scanner unit tests pin it now, including the case the old code was written for
(`export module foo:http;` + `import :tls;` must give `foo:tls`).

Consequence for `mcpp pack`: a partition the published interface reaches now
resolves, so it is published rather than refused — the consumer cannot build the
interface's BMI without it. That is correct and it is also the one thing a
closed-source publisher must not do by accident, so it comes with a warning
naming the file. 243 asserts both halves; the closure's unresolved-import error
stays for a partition nothing provides.

With this, e2e 242/243/244/249/250 run on Linux, macOS and Windows — the docs no
longer need a "verified on Linux only" caveat.

Also, per review: examples 05-lib-dist and 06-lib-consume are one story, so they
are one directory (examples/05-lib-distribution/{producer,consumer}) with one
README; and the consumers use `import std;` rather than <cstdio>, which is what
a module example should be showing.
…sing

archiver must not be a silent skip

Windows packed the library correctly — `lib/x86_64-windows-msvc/mathkit.lib` —
and 242 failed anyway, because the assertion hard-coded `libmathkit.a`. MinGW
writes `libfoo.a` where MSVC writes `foo.lib`, which is the very reason `lib/`
is keyed by triple and not by OS; the test had the fact in its comments and the
GNU spelling in its `find`. 243's archive probe had the same assumption, which
made its drop assertion skip silently on the clang legs rather than run.

And the packer itself: with objects to drop and no archiver resolved it did
nothing, quietly, leaving the published interface's objects inside the archive —
two definitions of each published module's initialiser, resolved by link order.
That is the failure class this whole feature exists to remove, so it is now
refused with the reason.
§11.2b: unrestricting the portable tests is what surfaced the scanner's
implementation-partition bug, and fixing that — rather than widening a
capability — is what made 242/243/244 pass on Linux, macOS and Windows. Also
the per-platform table with a reason for every remaining skip.
Asked "what is the skip reason", and only one of the four survived the question.

  * 246 was gated on `gcc` because of copy-paste. Its probe greps a FILENAME
    out of build.ninja, not a compiler flag — nothing in it is toolchain-
    specific. Runs everywhere now.

  * 247 grepped `-D<macro>`, and MSVC spells that `/D` (dialect.cppm). The
    prefix was the only thing tying it to one compiler family; grepping the
    macro NAME is spelling-agnostic. Runs everywhere now.

  * 248 needed `mingw-cross`, which no e2e job installs — so it skipped in
    every job that exists, verified on a developer's machine and nowhere else.
    cross-build-test.yml's mingw job already names the e2e scripts the Linux
    shards skip for exactly this reason (102, 198, 240); 248 joins them, and
    the workflow header now says why that list is explicit rather than left to
    run_all's cap gating.

  * 251's reason held — `kind = "shared"` is ELF-only and plan.cppm refuses it
    — but it only tested the side that works, which cannot tell "the gate is
    handled" from "there is no gate". It is now two-sided: on ELF the package
    is produced and carries both of the library's names; off ELF the pack must
    be REFUSED and the message must name both the artifact kind and where it
    does work. Runs everywhere.

That leaves 245, which needs two distinct buildable targets (gnu + musl) and so
genuinely cannot run where only one exists. The mechanism it checks — one cfg()
leg per triple, the native build included — is platform-independent and covered
on Linux.

The criterion, earned twice now: a test's `# requires:` has to be the real floor
of the mechanism it verifies. Telling "a real limit" from "I did not think it
through" is hard while writing it, so the reasons have to be interrogated one by
one afterwards.
…or one

247 derived the host triple with a regex for `<arch>-<os>-<env>` and fell back
to `x86_64-linux-gnu`. macOS's canonical triple is `aarch64-macos` — two
segments, no env — so the match failed, the fallback was used, and the test
asserted that a *Linux* section should apply to a macOS build. It then reported
"the bare triple was inert" against a product behaving correctly.

`target/<triple>/` is mcpp's own answer to the same question. The test now
builds once with nothing conditional, reads the directory name, and writes the
real manifest from that — and the cfg() control is keyed on the same value's
arch rather than on `unix`, which is false on Windows.

The shape is the one this PR keeps meeting: a second, independent derivation of
something mcpp already computes, disagreeing with it on the platform nobody
checked.
…iver

Four things a deeper read of this PR turned up.

**The old-client claim was unverified.** The PR body and docs/12 both say an
mcpp that predates this feature still builds against these packages — that is
the entire justification for adding no manifest section and no key, and nothing
checked it. It is true (2026.8.15.3 consumes a package from 2026.8.17.2 and
runs it), and e2e 252 now pins it in two halves: a portable static check that
the generated manifest's sections are a subset of the pre-existing vocabulary,
and the real thing against `$MCPP_BOOT`, the released binary each CI job
bootstraps from. The three e2e workflows now export it, so the real half runs
rather than noting itself out.

**The packer spoke `ar` to whatever archiver it was handed.** `archive_tool`
returns LIB.EXE for MSVC, which spells member removal `/REMOVE:<member>` — one
flag per member, archive last — not `d <archive> <member>…`. On mcpp's Windows
CI the archiver is clang's llvm-ar, which takes the GNU form, so the difference
was invisible. How to speak to a tool is what mcpp.toolchain.dialect is for, so
the removal spelling lives there now, next to `archiveCmd`, and the packer
substitutes rather than assumes. The MSVC row is marked untested, and a failure
reports the command it ran.

**A duplicate partition provider now says so.** Two files declaring
`module m:p;` used to be accepted silently; the scanner names both files. Those
programs were always ill-formed, but it is a new failure path and belongs in the
changelog with the other behaviour change (`sources = []`).

**The module graph is moved into BuildContext, not copied.** Nothing reads
`scan.graph` after that point.
252's guard compared `$MCPP_BOOT --version` against the PR binary's and treated
"different" as "found an old client". In CI that entry is an xvm SHIM, and a
shim resolves against the home it is asked in — under the e2e suite's
environment it answers `xlings: 'mcpp' is not installed` and prints nothing. The
empty string duly differed, so the test ran the real check against a binary that
cannot run at all and reported a COMPATIBILITY FAILURE against a package that is
perfectly readable. Three legs red for a reason that was in the test.

The guard now requires a version-SHAPED answer. Anything else means "no usable
old binary here", which is a note naming what it got, not a verdict — and the
static half (the generated manifest's sections are a subset of the pre-existing
vocabulary) still runs everywhere.

Consequence worth stating: the real old-client check runs where a released mcpp
binary is directly executable — locally, and in any job that points MCPP_BOOT at
one rather than at a shim. It passed against 2026.8.15.3.
Why 245/248 cannot run elsewhere, from host_can_serve rather than from the
test's capability table: macOS serves exactly one target so a fat package is
structurally impossible there, while Windows serves three — so the Windows
equivalent is a coverage gap, not a product gap, and it has independent value
(mathkit.lib next to libmathkit.a in one package is the strongest evidence that
lib/ must be keyed by triple).

Plus the dependency-direction invariant for src/pack (only library_pipeline may
import mcpp.build.*), the two deliberate behaviour changes, and five open items
— including two I have to report rather than claim: a scan_overrides-declared
implementation partition is mis-flagged as an interface (its source would be
published, and the schema has nowhere to say otherwise), and the design's
promised [package].platforms coverage warning is simply not implemented.
…face"

Whether a partition's SOURCE may be published turns on one keyword — `export
module M:api;` may travel, `module M:impl;` may not — and two of the three paths
that build the module graph cannot read it:

  * a `[scan_overrides."<glob>"]` entry names the modules a file provides and has
    nowhere to say whether the declaration carries `export`;
  * P1689 makes `is-interface` optional, and mcpp parsed it into a struct field
    (p1689.cppm:34) that nothing ever read.

Both arrived as `providesInterface = true`, which the field's own comment called
the conservative direction "since the flag only ever produces a warning". That
had it backwards: `true` is the value that produces NO warning, so an
implementation partition declared in `[scan_overrides]` was published in
silence — the exact failure the closure exists to prevent, since too FEW
published sources fails the consumer's compile while too MANY ships private
source and nothing fails.

So the field is a tri-state, and each path now says what it actually knows:

  text scanner   reads the keyword, sets true or false explicitly
  P1689 reader   carries the compiler's answer through, absence included —
                 the compiler is the one participant that parsed the declaration
  scan_overrides leaves it unset, because the schema cannot express it

Unknown warns, with a different sentence from the known case: that list says
"you are publishing your implementation", this one says "mcpp cannot tell
whether you are". Only PARTITIONS are asked about — `module M;` provides
nothing, so a bare `M` can only come from `export module M;`, and asking there
would fire on every primary interface in every package that uses overrides.

e2e 253 pins it from both sides, because asserting only that the override case
warns cannot distinguish "mcpp models three states" from "mcpp warns about every
partition it publishes". Verified by control probe: restoring the `true` default
at the scan_overrides site alone, with the rest of the fix in place, fails 253 at
exactly that assertion.

Two existing assertions were weak in the same way and are tightened:
EXPECT_TRUE/EXPECT_FALSE on an optional ask about has_value(), so
`EXPECT_TRUE(providesInterface)` stayed green for an implementation partition.
The design promised this cross-check (§2.2) and the implementation simply did
not have it — src/pack never read the key. `[package] platforms` is a support
claim, and `mcpp pack` on a library is the first moment there is evidence to
check it against: the legs in the package are the platforms it can serve.

The hard part is not finding the gap, it is not shouting about it. Four
comparisons exist and only two may be printed:

  packed, not declared    always actionable — the manifest disclaims a platform
                          the package demonstrably serves.
  declared, not packed    actionable ONLY IF THIS HOST COULD HAVE BUILT IT.

The normal release flow is one `mcpp pack` per platform in CI, so a Linux runner
never produces a macOS leg. Warning about that would fire on every run of every
cross-platform package, and a warning that always fires hides the one that
matters — so the check asks host_can_serve, the same function that decides which
`--target` values are accepted at all, and can therefore only ever name
something the author is able to do on this machine.

Both are warnings, never errors: coverage is release discipline, and the person
who can judge it is looking at the release, not at this build.

e2e 254 asserts the silence as well as the warnings, and derives the host's
platform from `target/<triple>/` — mcpp's own answer — rather than from a regex
over triple spellings, which is how an earlier test came to assert Linux
expectations on macOS. Its per-host picks mirror host_can_serve deliberately: if
mcpp ever gains a macOS-hosted Windows toolchain the test fails, which is the
correct outcome, because the table it encodes will have changed.
… both ends

`mcpp pack` deletes the published interface's objects from the archive before
shipping it, and the two archivers disagree in both directions:

  ar        one verb, then the archive, then every member
  lib.exe   one flag PER member, and the archive comes LAST

Nothing could catch a mistake there. mcpp's Windows CI builds with clang and
archives with llvm-ar, which takes the GNU spelling, so the MSVC branch has never
executed in any job on any platform — and the assembly was inline in
run_library_pack, so it had no seam either. Pinning the two constants in
dialect.cppm was never enough: the ORDER they are assembled in is a third fact.

So the assembly moves into archive_remove_command, exported for the tests and
nothing else, and it is checked from both ends:

  test_pack_archive_remove.cpp  the exact command for each dialect, host-
                                independently (expectations built with the same
                                `quote`, since what is under test is spelling and
                                order — shell quoting has its own tests)
  e2e 255                       a real lib.exe, under `# requires: msvc`, with
                                msvc@system pinned in the manifest because
                                Windows' DEFAULT toolchain is clang and taking it
                                would exercise the GNU branch and pass while
                                proving nothing

255 does not inspect the archive to decide: a wrong spelling makes
run_library_pack refuse, quoting the command and the archiver's output, since
shipping the objects would be the silent outcome. What it does assert is that the
branch was TAKEN — a `mathkit.lib` rather than llvm-ar's `.a`, and a non-empty
published interface so there was something to remove at all.
…ble targets

Four things, and the first one explains why the rest went unnoticed.

THE GUARD HAD A HOLE IN THE CASE THAT MATTERED. make_plan refused SharedLibrary
targets with `!targetTriple.empty() && os != "linux"`, and targetTriple is EMPTY
for a native build. So it turned away a cross build to macOS — unservable anyway,
i.e. unreachable — while letting a NATIVE macOS or native Windows build walk
straight into the unverified paths it was written to keep people out of. It reads
the resolved target now.

PE WAS MISSING ITS IMPORT LIBRARY. A PE shared library is two files: the `.dll`
the loader opens and an archive of stubs the LINKER consumes. mcpp wrote only the
first, and consumers linked the `.dll` directly — which mingw's ld tolerates and
no other linker does, so the tolerant case was hiding the broken one. The link
edge now declares the import library as an implicit output (one command writes
both; a second edge would run the link twice), `import_library_for` owns its name,
the dialect table owns the flag spelling beside `archiveRemoveArg`, and the
package ships it. One more thing was needed and the diagnostic for it names
nothing useful: mcpp gives PE executables `-static`, which puts ld in static-only
mode where it refuses an import library and says `have you installed the static
version of the mathkit library?` — so the emitted manifest switches to dynamic
mode for that one `-l`.

MACH-O WAS MISSING ITS INSTALL NAME. A `.dylib` records the path it was LINKED
at, so emitting `-install_name` only when the manifest declared a `soname` left
every other dylib recording a build directory: perfect on the machine that built
it, `image not found` anywhere else. It is now `@rpath/<file>` unconditionally.
The choice was also being made with `#if defined(__APPLE__)` on the HOST, so a
cross link emitted the wrong flag or none — it comes from the target now, as
`target_output` already did.

MSVC STAYS REFUSED, FOR THE REAL REASON. Not the linker: `link /DLL /IMPLIB:` has
been in the rule table all along. Symbol export — MSVC exports nothing from a DLL
without `__declspec(dllexport)` or a `.def`, so the import library comes out empty
and consumers fail with unresolved externals naming symbols that are visibly in
the objects. Refusing beats producing a diagnostic that points nowhere near its
cause, and the message names MinGW as the way forward.

AND `--target` NO LONGER ACCEPTS WHAT THIS HOST CANNOT PRODUCE. Measured on
Linux: `mcpp build --target x86_64-windows-msvc` resolved the native g++, wrote
target/x86_64-linux-gnu/, and reported success — an ELF delivered as a Windows
build, which is the failure the neighbouring typo check calls the worst one. The
vocabulary tier says "mcpp supports this target"; host_can_serve answers "can
this machine produce it", and the error lists what it can. An explicit
`[target.X] toolchain` stays the escape hatch.

Also fixed on the way: consuming a distribution package whose target is `shared`
died with `ninja: multiple rules generate bin/libmathkit.dll`, because the
dependency loop created a link unit for a library that is already built — and
relinking it would have produced a library missing every implementation unit the
publisher withheld, since a distribution package's `sources` are its interface.

Verified end to end on Linux via mingw-cross + wine (e2e 257: both files, the
implicit output, the package, deployment beside the exe, `ok=42`). macOS (259) and
the MSVC refusal (258) are CI's to confirm; 259 deletes the producer's build tree
before consuming, so the install-name assertion is load-bearing rather than
decorative.
Records the outcome of §12's five items under the single-PR decision (O1 not
split, the rest done), plus the two real defects the work forced out: the
kind="shared" guard was inert on native builds — the very case it was written
for, since targetTriple is empty there — and `--target` accepted a target this
host cannot produce, resolving the native g++ and delivering an ELF as a Windows
build. Also the shape they share: mingw's ld tolerates linking a .dll directly,
so "a PE shared library is two files" was never exposed. The tolerant case was
hiding the broken one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mcpp如何支持「预编译 .so + .ixx/.h/.cppm 接口」的二进制分发

2 participants