From 84e65202f79d1ac1e4c7700844da3e21cef0e978 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 10:52:00 +0800 Subject: [PATCH 01/31] feat(pack): ship a library as interface + prebuilt binaries (#433) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A closed-source library, an offline site, or a build farm that already compiled this once had no route through mcpp: `mcpp publish` sends source, `mcpp pack` bundles a program's run-time closure, and neither is "a library someone else links". Collecting the artifacts by hand was the answer. mcpp pack mathkit # a static library package mcpp pack mathkit --target x86_64-linux-gnu \ --target aarch64-linux-gnu # one package, two legs WHAT IT PRODUCES IS AN ORDINARY PACKAGE. A normal `mcpp.toml`, read through the route mcpp already had for payloads that carry their own manifest. Zero new manifest sections and zero new keys: what to pack is `[targets.].kind` (so there is no --lib and no --artifact), which interface to publish is `[lib]` plus the module graph, which headers are public is `[build].include_dirs`, and each leg's ABI tag and digest ride on `[[runtime.artifacts]]`, whose fields were already documented as optional evidence. An older mcpp still BUILDS against these packages; it just does not run the two gates below. Two interface modes coexist — `include/` (text, `#include`, never compiled) and `interface/` (module units the consumer compiles). Measured: one package consumed three ways (header only, module only, both) x static and shared = six combinations, all green. WHICH .cppm TRAVEL IS COMPUTED, not declared. It is the module closure of the lib root, and the two ways of getting it wrong are asymmetric: too few fails loudly in the consumer's compile, too many silently publishes a closed-source implementation partition's SOURCE. `.m.o` is not the rule — an implementation partition produces one too. The same closure decides which archive members to drop, and getting THAT wrong was measured as well: dropping every `.m.o` also drops the partition's real code and every target fails to link. Both lists are printed, because "what is not travelling" is half of what a publisher needs. TWO GATES ON THE CONSUMER SIDE, both for failures that are otherwise silent. The interface still matches its binaries: swap two `int` members of a shipped struct — the Itanium ABI does not mangle field order — and before this the consumer compiled, linked, ran, and printed transposed data with no diagnostic from any tool. And the binaries were built for this toolchain, with the refusal listing the tags the package does have, because "not found" sends people looking for a package already on their disk. Also fixes two defects this work found, each with its own regression test: * `[target.''.build]` never matched a native build. `matches()` short-circuited on the raw --target string while `context_for()` fell back to the host for `cfg(...)`, so two spellings of one statement disagreed — green in CI, silently inert on a developer's machine. The resolved triple now lives in `cfgpred::Ctx`; there is no second answerer. * `sources = []` was byte-identical to omitting the key, so no author could say "compile nothing". A header-only package needs that, and without it any leftover file under `src/` is compiled into the consumer's build. Docs: docs/12-binary-distribution.md (+ zh), examples 05-lib-dist and 06-lib-consume. Design: .agents/docs/2026-08-17-library-distribution-design.md. Tests: 23 unit assertions across two new suites; e2e 242-248. --- ...bution-architecture-analysis-and-design.md | 1615 +++++++++++++++++ .../2026-08-17-library-distribution-design.md | 585 ++++++ CHANGELOG.md | 66 + docs/02-pack-and-release.md | 10 +- docs/05-mcpp-toml.md | 17 + docs/12-binary-distribution.md | 263 +++ docs/README.md | 1 + docs/zh/12-binary-distribution.md | 243 +++ docs/zh/README.md | 1 + examples/05-lib-dist/README.md | 94 + examples/05-lib-dist/include/mathkit_c.h | 13 + examples/05-lib-dist/mcpp.toml | 24 + examples/05-lib-dist/src/api.cppm | 10 + examples/05-lib-dist/src/capi.c | 3 + examples/05-lib-dist/src/impl.cpp | 7 + examples/05-lib-dist/src/mathkit.cppm | 4 + examples/05-lib-dist/src/secret.cppm | 12 + examples/06-lib-consume/README.md | 75 + examples/06-lib-consume/mcpp.toml | 23 + examples/06-lib-consume/src/main_both.cpp | 11 + examples/06-lib-consume/src/main_header.cpp | 9 + examples/06-lib-consume/src/main_module.cpp | 15 + mcpp.toml | 2 +- src/build/plan.cppm | 24 +- src/build/prepare.cppm | 85 +- src/build/prepare_inputs.cppm | 32 +- src/cli.cppm | 10 +- src/cli/cmd_publish.cppm | 33 +- src/manifest/toml.cppm | 24 +- src/manifest/types.cppm | 19 + src/pack/abi_tag.cppm | 231 +++ src/pack/digest.cppm | 61 + src/pack/interface.cppm | 163 ++ src/pack/library.cppm | 276 +++ src/pack/library_pipeline.cppm | 316 ++++ src/pack/manifest_emit.cppm | 236 +++ src/pack/prebuilt.cppm | 166 ++ src/pack/route.cppm | 105 ++ src/source_kind.cppm | 29 + src/version.cppm | 2 +- .../242_pack_library_interface_and_headers.sh | 114 ++ .../e2e/243_pack_library_interface_closure.sh | 103 ++ tests/e2e/244_pack_library_gate.sh | 113 ++ .../245_pack_library_fat_target_selection.sh | 102 ++ tests/e2e/246_explicit_empty_sources.sh | 62 + .../e2e/247_bare_triple_conditional_native.sh | 65 + tests/e2e/248_pack_library_fat_pe_leg.sh | 88 + tests/unit/test_pack_abi_tag.cpp | 166 ++ tests/unit/test_pack_interface.cpp | 142 ++ 49 files changed, 5830 insertions(+), 40 deletions(-) create mode 100644 .agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md create mode 100644 .agents/docs/2026-08-17-library-distribution-design.md create mode 100644 docs/12-binary-distribution.md create mode 100644 docs/zh/12-binary-distribution.md create mode 100644 examples/05-lib-dist/README.md create mode 100644 examples/05-lib-dist/include/mathkit_c.h create mode 100644 examples/05-lib-dist/mcpp.toml create mode 100644 examples/05-lib-dist/src/api.cppm create mode 100644 examples/05-lib-dist/src/capi.c create mode 100644 examples/05-lib-dist/src/impl.cpp create mode 100644 examples/05-lib-dist/src/mathkit.cppm create mode 100644 examples/05-lib-dist/src/secret.cppm create mode 100644 examples/06-lib-consume/README.md create mode 100644 examples/06-lib-consume/mcpp.toml create mode 100644 examples/06-lib-consume/src/main_both.cpp create mode 100644 examples/06-lib-consume/src/main_header.cpp create mode 100644 examples/06-lib-consume/src/main_module.cpp create mode 100644 src/pack/abi_tag.cppm create mode 100644 src/pack/digest.cppm create mode 100644 src/pack/interface.cppm create mode 100644 src/pack/library.cppm create mode 100644 src/pack/library_pipeline.cppm create mode 100644 src/pack/manifest_emit.cppm create mode 100644 src/pack/prebuilt.cppm create mode 100644 src/pack/route.cppm create mode 100755 tests/e2e/242_pack_library_interface_and_headers.sh create mode 100755 tests/e2e/243_pack_library_interface_closure.sh create mode 100755 tests/e2e/244_pack_library_gate.sh create mode 100755 tests/e2e/245_pack_library_fat_target_selection.sh create mode 100755 tests/e2e/246_explicit_empty_sources.sh create mode 100755 tests/e2e/247_bare_triple_conditional_native.sh create mode 100755 tests/e2e/248_pack_library_fat_pe_leg.sh create mode 100644 tests/unit/test_pack_abi_tag.cpp create mode 100644 tests/unit/test_pack_interface.cpp diff --git a/.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md b/.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md new file mode 100644 index 00000000..87629634 --- /dev/null +++ b/.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md @@ -0,0 +1,1615 @@ +# mcpp 分发架构:全面分析与方案(2026-08-17) + +> 覆盖:源码分发 / 二进制分发 / 多平台打包 / 私有分发,以及 issue #433 +> (「预编译 .so + .ixx/.h/.cppm 接口」)。 +> +> 前半是**发现**(现状是什么、证据在哪、file:line 与实测),后半是**方案** +> (改什么、怎么算通过)。第 7 节是需要 review 的决策点。 +> +> **⇒ 方案已独立成文:`2026-08-17-library-distribution-design.md`** +> (落地形态、docs 计划、examples、CI 验证矩阵、分期)。**本文是发现,那份是方案。** +> +> 关联 issue:#433(本文起因)、#290(描述符构建规则不能按版本区分)、 +> #304(`runtime.library_dirs` 同时落在链接线上)、#276(嵌入式 SDK 集成)、 +> #416(纯 C 包被链进 libstdc++)。 + +--- + +## 0. 一页纸结论 + +**五条结论,按重要性排序:** + +1. **#433 想要的东西,在 Linux 上今天就能跑通 —— 我实测跑通了。** + 一个「只有模块接口 `.cppm` + 预编译 `libfoo.so`」的包,通过 `[runtime]` + 的 `libraries` / `link_library_dirs` / `runtime_search_dirs` 被消费者 + `import` 并链接,`std::string` 和异常都能跨边界。**零引擎改动。** + 见 §2。 + +2. **但它「能跑」不等于「能分发」,而且失败是静默的。** 同一次实测暴露三件事: + - 产出的 `.so` 里烧着**生产者机器的绝对 RUNPATH**(`/home/speak/.mcpp/...`), + 换台机器就是另一回事 —— 而 `mcpp pack` 的 RUNPATH/INTERP 重写**只服务可执行文件,不服务库**; + - 进程里同时有**两份 C++ 运行时**(exe 静态 libstdc++ + `.so` 动态 + `libstdc++.so.6`),没有任何一层告警; + - **接口与二进制之间没有任何绑定**。我把随包发的 `.cppm` 里 + `struct Point { int x; int y; }` 改成 `{ int y; int x; }`(mangling 不变), + **编译通过、链接通过、运行通过、打印出交换后的错数据**。全程零诊断。 + +3. **真正的结构性缺口不在「能不能链」,在四处**: + ① 描述符**强制要求 `sources`**(`xpkg.cppm:2059`),所以「无源包」这个种类不存在 + —— 生态里现有的做法是造一个假的 anchor `.c`(见 `compat.openblas`); + ② `kind = "shared"` **只有 Linux/ELF**(`plan.cppm:1006` 直接拒绝), + 即 `.dll` / `.dylib` 这两条腿**在生产侧根本不存在**; + ③ **没有兼容标签**(wheel tag 那种东西)—— `abi.cppm` 有五维模型但太粗 + (无编译器版本、无 stdlib 版本、无标准档位); + ④ **安装线协议没有 target 轴**(`package_fetcher.cppm:419` 只发 + `{"targets":[...]}`),所以交叉编译一定拿到宿主 arch 的载荷。 + +4. **能力其实已经攒够了,缺的是把它们接起来。** + `cache_key.cppm` 已经在算一个覆盖 toolchain × 语言 × profile × 身份 × + 自身配置 × 上游 Merkle 的**逐包 ABI 完备键**,并且全局构建缓存 + (`$MCPP_HOME/build-cache/v1/pkg//@//`)里**已经躺着 + BMI + obj**。二进制分发格式 ≈ **可搬运的构建缓存条目**。 + xim 的 XPackage Spec V2 也**已经有一等的 arch 轴**(per-arch resource map / + URL 模板 / per-arch sha256)。 + +5. **最短路径不是索引,是文件。** issue 作者原话是「类似 `pip install runtime.whl`」。 + `mcpp add ./runtime-0.1.0-.mpkg` 不需要索引、不需要网络、不需要鉴权、 + **不需要 xlings 改一行**,完全在 mcpp 自己手里。**建议它做 P0**,索引通路做 P1。 + +7. **包自带一份 `mcpp.toml` —— 这不是新机制,是 mcpp 已有且文档推荐的 Form A** + (`prepare.cppm:2764`:描述符没有 `mcpp` 字段时,glob 载荷里的 `mcpp.toml`)。 + 它一次消掉三样东西:**新描述符键**(⇒ 不需要版本 floor ⇒ 老客户端不会被砖)、 + **新消费代码路径**、以及 —— 通过「胖包 + `[target.'cfg(...)']` 在消费者构建期 + 选 arch」—— **整个 G3**(描述符 arch 轴 + 安装线 target 轴 + 动 xlings)。 + 实测边界:`[distribution]` 标量段今天被静默接受;但 `artifacts = [{…}]` + **硬失败**,产物清单必须借用已在白名单里的 `[[runtime.artifacts]]`。 + +6. **P0/P1 按「形态」切,不按「平台」切**(§4.5.1 / §5)。`kind = "lib"` **没有平台 + 限制**,静态形态今天三平台就能产出,而且**消掉了整整四类问题**:RUNPATH 重写、 + DLL 部署、双 C++ 运行时、加载器搜索闭包 —— 上面第 2 条里的三处静默失败, + 静态形态天然只剩「接口↔二进制绑定」那一处。所以 + **P0 = static × 三平台,P1 = shared × 三平台**;按平台切会让 Windows/macOS + 在 P0 结束时拿到一个空盒子。 + +--- + +## 1. 今天的分发架构:五条通路 + +mcpp 今天有五条互相独立的「东西怎么到达另一台机器」的通路。它们从未被放在一张表里 +比较过,而 #433 的困惑正来自这里:作者看到的是通路 A 和通路 B,而他要的是通路 C, +通路 C 今天只以「手法」的形式存在,没有名字。 + +### 1.1 通路 A —— 源码分发(库) + +``` +你的仓库 → git tag → GitHub 自动生成 tag tarball + → gitcode 镜像(逐字节相同,CN 区) + → mcpplibs/mcpp-index 的 pkgs//.lua(GLOBAL + CN URL + sha256) + → publish-artifact.yml 推一个内容哈希 artifact + → 消费者 bump 版本 +``` +(`docs/10-publishing-a-library.md`) + +- 生产侧:`mcpp publish` → `git archive` 出 tarball + sha256 + 生成 `xpkg.lua` + (`publish/pipeline.cppm:69`),然后**人工**开 PR 到索引。 +- 描述符两种形态:**Form A**(tarball 里自带 `mcpp.toml`)与 **Form B** + (描述符内联 `mcpp = { ... }` 块,`xpkg.cppm:synthesize_from_xpkg_lua`)。 +- 关键事实:`make_release_info` 把 **linux / macosx / windows 三个平台块填成同一个 + URL**(`publisher.cppm:295-302`)。源码 tarball 在三个平台上是同一份字节 —— + **这条通路天生没有多平台问题,因为它根本不区分平台。** + +### 1.2 通路 B —— 应用二进制分发(`mcpp pack`) + +四个 mode(`docs/02-pack-and-release.md`):`system` / `vendored`(默认) / +`self-contained` / `static`。两条产出族:ELF/Mach-O → `.tar.gz`(`lib/` + 重写 +RUNPATH);PE → `.zip`(DLL 与 `.exe` 平铺,因为 PE 没有 rpath)。 + +- **它的核心价值不是打包,是重写**:每个 mode 都会重写 `PT_INTERP` 与 + `DT_RUNPATH`,因为开发构建产物寻址的是**这台机器**的载荷目录。e2e 215 会扫遍 + bundle 里每个 ELF,发现 `$MCPP_HOME` 下的路径就失败。 +- **它的边界:只服务可执行程序。** `mcpp pack` 的输入是 `builtBinary` + (`pack.cppm:Plan::builtBinary`),整条流水线围绕「一个 exe + 它的闭包」。 + **没有任何 `mcpp` 命令会对一个库做同样的重写。** 这是 §2.2 那条 RUNPATH + 发现的根源。 + +### 1.3 通路 C —— 预编译库的「事实上」通路(anchor TU 手法) + +今天生态里确实有预编译二进制在分发,但它没有名字,是一组手法。样板是 +`compat.openblas`(`~/.mcpp/registry/data/mcpplibs/pkgs/c/compat.openblas.lua`): + +| 需要表达的事 | 今天怎么写 | +|---|---| +| 「我没有源码要编」 | **造一个假的 `mcpp_openblas_anchor.c`**,因为 `sources` 是强制的 | +| 头文件 | `include_dirs = { "include" }` | +| 链接预编译库 | `ldflags = { "-Llib", "-lopenblas" }`(`-L` 被 mcpp 重写成 `/lib`) | +| Windows 导入库 vs 静态库 | **per-OS 块**,`windows = { ldflags = { "-Llib", "-llibopenblas" } }` | +| 运行期 DLL | `windows = { runtime = { library_dirs = { "bin" } } }` → 拷到 `.exe` 旁 | +| 平台差异的入口 | linux/macosx 由 `install()` 钩子**写出** anchor;windows 用 `generated_files` | + +这套能工作,但它是**反向表达**:包的意图是「不要编译我,链接我」,而写法是 +「编译一个什么都不做的 `.c`,顺便偷偷加几个链接 flag」。后果: + +- 「这个包是预编译的」这件事**不可查询** —— 没有字段、没有 lint、没有诊断; +- 没有任何**兼容性检查**:一个用 gcc 13 编的 `libopenblas.a` 和一个 gcc 16 的 + 消费者之间,mcpp 无话可说(C 库侥幸没事,C++ 库会炸); +- `runtime.library_dirs` 这个名字**同时管链接和运行**,这就是 **#304**。 + +### 1.4 通路 D —— 私有分发 + +今天三种形态,全部是**源码**: + +| 形态 | 写法 | 鉴权 | 状态 | +|---|---|---|---| +| path 依赖 | `foo = { path = "../foo" }` | 无需 | ✅ 可用 | +| git 依赖 | `foo = { git = "...", rev/tag/branch = "..." }` | **环境里的 git 凭据** | ✅ 可用 | +| 私有索引 | `[indices] acme = { url = "git@..." }` 或 `{ path = "/srv/index" }` | **环境里的 git 凭据** | ✅ 可用 | + +缺口: +- `IndexSpec`(`pm/index_spec.cppm:14`)**没有任何鉴权字段** —— 全靠 ambient + git credential helper / SSH key。私有 HTTPS + token 只能把 token 写进 URL。 +- `IndexSpec::artifact`(内容哈希 artifact 通道)是**为公开 GitHub Actions 设计的**, + 私有场景没有对应机制。 +- **没有「离线包」入口**:没有 `mcpp add ./something.mpkg`。这正是 issue 作者 + 要的形状。 + +### 1.5 通路 E —— 工具链与运行时载荷(xim / xlings 层) + +mcpp 自己的 payload(gcc/llvm/glibc/ninja/…)走 xim。这条通路**已经有一等 arch 轴**: +XPackage Spec V2 的 per-arch resource map / URL 模板 + per-arch sha256 / +`xpm.source`,并且是 fail-closed 的(`xim-pkgindex/docs/V2/xpackage-spec.md`)。 + +但它有一个对二进制分发致命的性质:**arch 在安装时按宿主解析** +(spec 原文:"arch is resolved per-host at install time")。而 mcpp 调用它时 +只发 `{"targets":[...],"yes":true}`(`package_fetcher.cppm:419-431`)—— +**没有 os、没有 arch、没有 triple**。交叉编译时,拿到的是宿主 arch 的载荷。 + +对源码包无害(源码 tarball 与 arch 无关)。**对二进制包是致命的。** + +### 1.6 能力矩阵 + +| | A 源码库 | B 应用 pack | C 预编译库(手法) | D 私有 | E 载荷 | +|---|---|---|---|---|---| +| Linux | ✅ | ✅ | ⚠️ 手法 | ✅ | ✅ | +| macOS | ✅ | ✅ | ⚠️ 手法 | ✅ | ✅ | +| Windows | ✅ | ✅ | ⚠️ 手法 | ✅ | ✅ | +| arch 轴 | n/a | ✅(`--target`) | ❌ 描述符载荷无 arch | n/a | ✅ V2 | +| 交叉编译 | ✅ | ✅ | ❌ 安装线无 target | ✅ | ⚠️ 按包名绕开 | +| 生产 `.so` | n/a | n/a | **仅 Linux** | n/a | n/a | +| 生产 `.dll`/`.dylib` | n/a | n/a | ❌ **不存在** | n/a | n/a | +| ABI 兼容检查 | n/a | n/a | ❌ | ❌ | ⚠️ 五维粗粒度 | +| 接口↔二进制绑定 | n/a | n/a | ❌ **实测静默错数据** | ❌ | n/a | +| 鉴权 | 公开 | n/a | n/a | ⚠️ ambient git | 公开 | +| 离线包安装 | ❌ | n/a | ❌ | ❌ | n/a | + +--- + +## 2. 实测:#433 要的东西,今天在 Linux 上已经跑通 + +复现材料在 `scratchpad/bindist/`(附录 A 有完整脚本)。 + +### 2.1 步骤与结果 + +**① 生产者**(`provider/`):模块接口只有声明,实现在实现单元里。 + +```cpp +// src/runtimelib.cppm +export module runtimelib; +export namespace rt { int answer(); std::string name(); void boom(); } +// src/impl.cpp +module runtimelib; +namespace rt { int answer(){return 42;} ... } +``` +```toml +[targets.runtimelib] +kind = "shared" +``` +`mcpp build` → `bin/libruntimelib.so`,导出**恰好两个**符号: + +``` +T _ZGIW10runtimelib ← 模块初始化器 +T _ZN2rtW10runtimelib6answerEv ← rt::answer(),带 W10runtimelib 模块附着 +``` + +**② 「二进制包」**(`dist/`):只有接口源 + 预编译库 + 一个 `mcpp.toml`。 + +``` +dist/ +├── mcpp.toml +├── src/runtimelib.cppm ← 只有声明 +└── lib/libruntimelib.so ← 从 ① 拷来 +``` +```toml +[build] +sources = ["src/runtimelib.cppm"] +[targets.runtimelib] +kind = "lib" +[runtime] +libraries = ["runtimelib"] +link_library_dirs = ["lib"] +runtime_search_dirs = ["lib"] +``` + +**③ 消费者**(`app/`):`runtimelib = { path = "../dist" }`,`import runtimelib;` + +``` +$ mcpp run +answer=42 +name=from-the-prebuilt-so (len=20) +caught runtime_error: thrown inside the .so +``` + +**跨边界的 `std::string` 和 C++ 异常(含 RTTI 类型匹配)都正常。** 消费者只编译 +了那一个 `.cppm`(产出 BMI + 一个近乎空的 object),其余符号解析到 `.so`。 +`mcpp` 引擎**一行没改**。 + +> 顺带一个与本议题无关但会绊人的坑:消费者 TU 里 `import` 必须写在 `#include` +> **之后**。写在之前时 GCC 16.1 把后续 include 的声明卷进了奇怪的作用域,报 +> `In function 'int std::main()'` —— 报错点与原因完全不沾边。 + +### 2.2 三处它没有告诉你的事 + +**(a) 发出去的 `.so` 烧着生产者机器的绝对路径。** + +``` +$ readelf -d dist/lib/libruntimelib.so + (NEEDED) libstdc++.so.6 + (RUNPATH) /home/speak/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/lib64: + /home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/lib64: + /home/speak/.mcpp/registry/subos/default/lib +``` + +这正是 `docs/02-pack-and-release.md` 关于路线 A 讲的那件事(「烧进去的 +`PT_INTERP` 指的是**你**的机器」),只不过对象从可执行文件换成了库 —— +而**库这一侧没有 `mcpp pack`**。手工拷贝出去的 `.so` 是机器绑定的。 + +**(b) 一个进程里两份 C++ 运行时,零告警。** + +``` +app : NEEDED = libruntimelib.so, libm, libgcc_s, libc ← 没有 libstdc++ + nm -D | grep std | wc -l = 528 ← 静态 libstdc++ 进来了 +.so : NEEDED = libstdc++.so.6 ← 动态 +``` +这是 `distribution.cppm` 里 `Role::SharedLibrary` 那段注释描述的危险,方向反过来: +exe 的静态 libstdc++ 把符号以 GLOBAL 导出,`.so` 绑到了它上面。**这次侥幸对了** +—— 两边是同一个 gcc 16.1 的 libstdc++。生产者换个编译器版本,就是一个进程里 +两份不同实现的 `std::string`。 + +契约模型本身是对的(`Role::Distributable` 默认 SelfContained, +`Role::SharedLibrary` 默认 ToolchainCoupled),问题是 **contract 是根工程的设置, +预编译 `.so` 的 contract 在生产时就冻结了,消费者看不见也检查不到。** + +**(c) 接口与二进制之间没有任何绑定 —— 这是最严重的一条。** + +把随包发的 `.cppm` 改成结构体字段互换(Itanium 不 mangle 字段顺序,符号名不变): + +```cpp +// 生产者编进 .so 的: struct Point { int x; int y; }; origin() 返回 {111, 222} +// 随包发出的接口: struct Point { int y; int x; }; ← 只改了这一行 +``` +``` +$ mcpp run +x=222 y=111 (producer meant x=111 y=222) +``` + +**编译通过、链接通过、运行通过、数据是错的、全程零诊断。** 同样的实验用返回类型 +(`int` → `long long`)也一样静默通过。 + +这不是「mcpp 的 bug」——C++ 语言层面这就是 IFNDR。但它是**分发格式必须解决的问题**: +只要接口和二进制可以被分别替换,这个失败就是可达的,而且它跨越机器和时间 +(「上周谁把那个头改了」)。 + +### 2.4 实验二:自动生成 + 接口闭包 + 跨平台(2026-08-17,`scratchpad/lab/`) + +写了一个 176 行的 `mcpp pack ` 原型(`lab/mkdist.py`),对一个真实的多单元库 +(主接口 + 接口分区 + **实现分区** + 实现单元 + C API + 头文件)跑 +**x86_64-linux-gnu / x86_64-linux-musl / x86_64-windows-gnu** 三个 target。 +它推翻了我上一轮写进本文的**两条规则**,并找出**一个新缺陷**。 + +#### 2.4.1 包内 `mcpp.toml` 能自动生成 —— 原料今天全部已暴露 + +`mcpp build --print-fingerprint` 的 11 个字段里,tag 需要的六个全在: + +``` +[1] gcc [2] 16.1.0 [4] x86_64-linux-gnu +[5] libstdc++ 16.1.0 [6] c++23 [11] 0a0ba53e8ca69b41(runtime binding) +``` + +`abi_tag` 是它们的**纯投影**,不需要任何新推导 —— §4.3 的主张就地验证。 + +**⚠️ 但 [4] 是编译器自报的 triple,不是 mcpp 的规范 triple。** +原型第一版直接用 [4],windows 那条腿产出的 tag 是 +`x86_64-w64-mingw32-gcc16-…`,而同一份 `mcpp.toml` 里 `[target.'…']` 键写的是 +`x86_64-windows-gnu` —— **同一个决定,两个拼写**。真实实现必须用 +`triple.cppm` 的规范拼写(`cfgpred` 的注释也说 cfg 词汇 **IS** 规范 triple 词汇)。 + +#### 2.4.2 「接口怎么制定」= 模块闭包,不是 `.m.o`,也不是 grep + +工程结构: + +``` +src/mathkit.cppm export module mathkit; export import :api; +src/api.cppm export module mathkit:api; ← 接口分区 +src/secret.cppm module mathkit:secret; ← 实现分区(闭源逻辑) +src/impl.cpp module mathkit; import :secret; +``` + +| 做法 | 结果 | +|---|---| +| 只发 `mathkit.cppm` + `api.cppm` + `.a` | ✅ 消费者构建成功、`add(2,3)=5` | +| 主接口改成也 `import :secret;`,仍不发它 | ❌ **硬失败**:`mathkit:secret: error: failed to read compiled module` | + +**判据:发布集 = 从主接口单元出发、沿其 purview 内 `import` 的传递闭包。** +实现分区只被实现单元 import 时**不在闭包里,不发布,源码不泄露**。 + +**失败的不对称性(这才是必须算而不是猜的理由):** + +| | 后果 | +|---|---| +| 发少了 | **响** —— 编译期 `failed to read compiled module`,点名模块 | +| 发多了 | **哑** —— 静默把闭源实现分区的源码发出去 | + +**⚠️ `.m.o` 不是判据 —— 用它会泄露源码。** 实测:实现分区 `secret.cppm` +**照样产出 `secret.m.o`**(`.m.o` 的含义是「模块单元的对象」,不是「接口的对象」)。 +按 `.m.o` 挑要发布的源,就会把 `secret.cppm` 发出去。 + +#### 2.4.3 ⚠️ 同一个闭包有第二个用途 —— 我上一轮把它写错了 + +上一轮我写「打包时剔除归档里的接口对象」,规则给的是**「剔除所有 `.m.o`」**。 +原型照做,**三个 target 全部链接失败**: + +``` +libmathkit.a(impl.o): in function `mk::add@mathkit(int, int)': + undefined reference to `mk::secret_helper@mathkit()' +``` + +因为 `secret.m.o` 里是**真代码**。正确规则与 §2.4.2 是**同一个闭包**: + +> **剔除的是「已发布接口闭包里那些单元」的对象,不是所有 `.m.o`。** + +修正后三个 target 全部构建成功,归档里剩下 `secret.m.o` + `impl.o` + `capi.o`。 + +#### 2.4.4 平台差异 + +| 事实 | linux-gnu | linux-musl | windows-gnu(MinGW) | +|---|---|---|---| +| 静态库文件名 | `libmathkit.a` | `libmathkit.a` | **`libmathkit.a`** | +| mangling | `_ZN2mkW7mathkit3addEii` | 同 | **同**(Itanium) | +| `.m.o` 命名 | 一致 | 一致 | 一致 | +| 产物 | ELF | ELF | **PE32+ executable, 18 sections** | + +- **产物命名按 `env` 分,不按 OS 分**:MinGW 走 GNU 约定(`lib*.a`), + 只有 MSVC 才是 `*.lib` —— 所以布局里的 `lib/` 必须按**三元组**而不是按 OS 分目录。 +- **MinGW 的 mangling 与 ELF 相同**,所以 MinGW 产的库能被 MinGW 消费者链接; + MSVC 不同 —— 这就是 `cxxabi` 必须是 ABI 模型一维的原因(`abi.cppm:84`)。 + +#### 2.4.5 ⚠️ 新缺陷:裸三元组谓词在原生构建下不匹配 + +胖包的每条腿要一个 `[target..build] ldflags`。原型第一版用**裸三元组**, +结果:**显式 `--target` 三个全绿,裸 `mcpp build` 链接失败**。 + +最小探针(`lab/probe/`,同一台机器、同一个解析出的三元组): + +| `[target..build] cxxflags = ["-DX"]` | 裸 `mcpp build` | `--target x86_64-linux-gnu` | +|---|---|---| +| `'x86_64-linux-gnu'`(裸三元组) | **0 处命中** | 2 处命中 | +| `'cfg(linux)'` | 2 处命中 | 2 处命中 | + +**根因一行**(`src/build/prepare_inputs.cppm:139`): + +```cpp +if (triple.empty()) return false; // ← 裸三元组分支 +``` + +而同一文件的 `context_for()`(:49-53)在 `targetTriple` 为空时**回落到 +`triple::host_triple()`**。于是 `cfg(...)` 用宿主事实求值,裸三元组分支却拿着 +原始的空串直接返回 false —— **同一个决定,两处推导**。 +`types.cppm:665` 的注释明确承诺的是另一种行为: + +> *prepare_build evaluates it against the RESOLVED target (**host triple for a +> native build**, the --target triple for a cross build)* + +**危险形状:CI 传 `--target` 全绿,开发者本机日常构建静默失配。** +与本方案的关系:**胖包必须用 `cfg(...)`,不能用裸三元组**(见 §4.8)。 +修法看起来是一行(空串时与 `host_triple()` 比),但这是独立缺陷,应单独开 issue。 + +#### 2.4.6 全矩阵验收 + +胖包改用 `cfg(...)` 后,四种构建全过,且每个 target 的 `build.ninja` 里 +**只出现自己那条腿**: + +``` +(native, 无 --target) OK +x86_64-linux-gnu OK → fatpkg/lib/x86_64-linux-gnu +x86_64-linux-musl OK → fatpkg/lib/x86_64-linux-musl +x86_64-windows-gnu OK → fatpkg/lib/x86_64-windows-gnu (PE32+) +``` + + +### 2.5 实验三:两种接口模式 × 两种库形态(2026-08-17,`scratchpad/lab/`) + +问题:「动态库 + 接口文件」的结构应该是简单好描述的 —— 能不能同时自动支持 +`.cppm`(模块)与 `.h`(头文件)两种接口? + +**结论:能,而且结构确实简单。复杂度不在布局,全部集中在「模块接口」这一种模式上。** + +#### 2.5.1 支持矩阵 —— 同一个包、同一份 `mcpp.toml`,三种消费方式 + +一个生产者(`lab/parts`)同时提供两种接口: +`include/mathkit_c.h`(`extern "C"`)+ `interface/mathkit.cppm`(模块)。 + +| | 只 `#include` | 只 `import` | **两者同时** | +|---|---|---|---| +| **静态库** `.a` | ✅ `hdr: 5` | ✅ `mod: 5` | ✅ `both: c=5 mod=5` | +| **动态库** `.so` | ✅ `hdr: 5` | ✅ `mod: 5` | ✅ `both: c=5 mod=5` | + +**六格全过,零特殊处理。** 动态库那一列的消费者 ELF 里 +`NEEDED: libmathkit.so` 确实在,`runtime_search_dirs` 进了 RPATH。 +静态/动态之间,包描述符只差 `[runtime]` 要不要 `runtime_search_dirs`。 + +**纯头文件包**(连一个 `.cppm` 都没有的传统形态)也通过 —— `include/` + `lib/` + +`ldflags`,消费者 `#include` 即用。 + +#### 2.5.2 让两种模式自动共存的规则:只有两条,由目录决定 + +``` +pkg/ +├── mcpp.toml +├── include/ ← 文本接口:原样全发,消费者 #include,不产生任何对象 +├── interface/ ← 模块接口:必须算闭包,消费者编译它得到 BMI + object +├── lib// ← 链接面 +├── bin// ← 运行面(仅 PE) +└── LICENSE +``` + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | **预处理器** | **编译器** | +| 消费者要编译吗 | 否 | **是** | +| 要算闭包吗 | 否(头的 `#include` 由预处理器解决,而且头本来就全发) | **是**(§2.4.2) | +| 要从归档剔除对象吗 | 否 | **是**(§2.4.3) | +| ABI 闸门强度 | `extern "C"` ⇒ **只约束 libc**(`abi.cppm:12-16`) | 全部维度 | + +**这正是「为什么感觉应该简单」的答案:传统 C/C++ 库(头 + `.so`/`.a`)真的简单 +—— 没有闭包、没有剔除、闸门也弱。全部复杂度属于模块接口那一种模式, +而它的复杂度是可自动化的(scanner 的模块图现成)。** + +两种模式**互不干扰**,可以同时存在:实测同一个包被三种方式消费,六格全过。 + +#### 2.5.3 ⚠️ `kind = "shared"` 在 musl 上不是「被拒绝」,是「链接期炸掉」 + +`plan.cppm:1002` 那道守卫的判据是 `os != "linux"`。**musl 是 linux,所以它过闸**, +然后死在链接器里: + +``` +$ mcpp build --target x86_64-linux-musl # [targets.x] kind = "shared" +crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' + can not be used when making a shared object +ld: failed to set dynamic section sizes: bad value +``` + +`crtbeginT.o` 是**静态链接**的启动文件(`T` 后缀),因为 musl target 蕴含 `-static` +—— `-static` 与 `-shared` 互相矛盾。三个 target 的真实状态: + +| target | `kind = "shared"` | 诊断质量 | +|---|---|---| +| `x86_64-linux-gnu` | ✅ | —— | +| `x86_64-linux-musl` | ❌ 链接失败 | **差** —— 消息里既没有 musl 也没有 shared | +| `x86_64-windows-gnu` | ❌ 守卫拒绝 | **好** —— 点名原因与出路 | + +**守卫的判据应该是「这个 target 是否动态链接」,而不是「os 是不是 linux」。** +这条与 G2 是同一处,但比 G2 原来的描述更糟:它是一个**过了闸再炸**的洞。 + +#### 2.5.4 ⚠️ `sources = []` 今天不生效 —— 与「不写」逐字节等价 + +G1 的精确判据。在包里放一个 `src/leftover.cpp`,然后: + +| `mcpp.toml` | `build.ninja` 里 `leftover` 的出现次数 | +|---|---| +| `sources = []` | **7** | +| (整行删掉) | **7** | + +两者**完全一样** —— `toml.cppm:1703` 的 `if (sources.empty()) → 填默认 glob` +把显式空吞掉了。对二进制包的后果:**包里任何遗留在 `src/` 下的文件都会被编进 +消费者的构建**,而且可能与预编译库里的符号重复定义。作者没有任何写法能说 +「什么都不要编」。 + +(我一开始误以为这条已经能用 —— 因为测试包里根本没有 `src/` 目录, +默认 glob 也匹配不到东西。**「两条路径给出同一答案」不等于「这条路径生效了」。**) + + +### 2.3 判据 + +> **能编译链接运行,不等于能分发。** 三条判据,一条不满足就不叫「支持二进制分发」: +> 1. 产物里不含生产机器的任何绝对路径(e2e 215 已经为 exe 定了这条,库要同一条); +> 2. 接口与二进制**不可分别替换** —— 要么一起来,要么拒绝; +> 3. 消费者的工具链与产物的 ABI **不匹配时必须拒绝**,而不是链上去再赌。 + +--- + +## 3. 结构性缺口 + +按「阻塞面 × 改动成本」排,G1–G3 是硬阻塞。 + +### G1 —— `sources` 强制:没有「无源包」这个种类 + +```cpp +// src/manifest/xpkg.cppm:2058 +// Validate minimum +if (m.modules.sources.empty()) { + return std::unexpected(ManifestError{ + "synthesised manifest missing sources (mcpp segment must declare `sources = { ... }`)", ...}); +} +``` + +对 `mcpp.toml` 一侧则是另一种问题:`sources` 缺省会被填成默认 glob +(`toml.cppm:1703`),所以「作者故意没有源」和「glob 一个都没匹配上」**不可区分** +—— 我在 §2 里那个 `dist/` 包之所以要写 `sources = ["src/runtimelib.cppm"]`, +是因为接口确实要编;一个纯 C 头文件 + `.so` 的包今天只能靠 anchor 手法。 + +**判据:「不存在」与「显式为空」必须可区分。** 仓库里已有这个模式 +(`XlingsConfig::subosDeclared`,`types.cppm:632`)。 + +**实测(§2.5.4):`sources = []` 与整行删掉在 `build.ninja` 里逐字节等价** +(同一个遗留 `src/leftover.cpp`,两种写法都是 7 处命中)。所以作者今天**没有任何 +写法**能表达「什么都不要编」—— 对二进制包,这意味着包里任何遗留在 `src/` 下的 +文件都会被编进消费者的构建,并可能与预编译库里的符号重复定义。 + +### G2 —— `kind = "shared"` 只有 Linux + +```cpp +// src/build/plan.cppm:1002 +if (!targetTriple.empty() && targetTriple.os != "linux") { + for (auto const& t : manifest.targets) { + if (t.kind != Target::SharedLibrary) continue; + return std::unexpected("shared libraries are only supported for Linux (ELF) targets today..."); + } +} +``` +注释写得很清楚:PE 消费者需要导入库、Mach-O 需要 install-name,**两者都没建模**, +所以宁可拒绝也不产出没人验证过的东西。每个 shared 相关 e2e 都写着 +`# requires: elf`。 + +**这条直接把 #433 的 `.dll` / `.dylib` 两条腿砍掉了。** 不是分发格式的问题, +是**生产侧根本产不出来**。 + +**⚠️ 而且守卫的判据是错的(§2.5.3)。** 它问的是 `os != "linux"`, +但 `x86_64-linux-musl` **是** linux —— 于是它过闸,然后死在链接器里: +`crtbeginT.o: relocation R_X86_64_32 against hidden symbol '__TMC_END__'` +(musl target 蕴含 `-static`,与 `-shared` 矛盾)。消息里既没有 musl 也没有 shared。 +**正确判据是「这个 target 是否动态链接」,不是「os 是不是 linux」。** + +好消息:相邻机械已经在了 —— `ArtifactNaming::sharedNeedsImportLib` 这个字段存在 +(`plan.cppm:467`),Windows 运行期 DLL 部署也已经有(e2e 84 / #299)。 + +### G3 —— 载荷侧的 arch 轴与 target 轴 + +分成两个子问题,状态完全不同: + +| 子问题 | 状态 | +|---|---| +| 描述符的**构建规则**能否按 arch 分叉 | ✅ **已经可以** —— `target_cfg = { ["cfg(arch=\"aarch64\")"] = {...} }`(`xpkg.cppm:1315`),按**解析后的 target** 求值 | +| 描述符的**载荷**能否按 arch 分叉 | ⚠️ xim V2 有(per-arch map / 模板 / sha256),但**按宿主 arch 解析** | +| mcpp → xlings 的安装调用能否带 target | ❌ **不能**,`make_targets_args` 只有 `{"targets":[...],"yes":true}` | +| 描述符的 `mcpp` 块 per-OS 分叉 | ✅ linux/macosx/windows,且已按 **TargetPlatform** 而非宿主(#254 已修) | + +另外 `target_cfg` 只承载 `BuildInputs`(cflags/cxxflags/ldflags/sources/defines/ +flags/include_dirs/include_dirs_after,`xpkg.cppm:1359-1377`),**不含 +`runtime` / LinkIntent** —— 所以「per-arch 的 `link_library_dirs`」写不出来。 +这是 #258 同一形状的债:条件通道自己维护了一份子集。今天可以用 +`ldflags = {"-Llib/aarch64", "-lfoo"}` 绕过。 + +### G4 —— 没有兼容标签 + +`toolchain/abi.cppm` 建模了五维(libc / cxxStdlib / arch / os / cxxAbi), +并且有 `abi:=` 的约束语言。对 **C 库**够用(它的注释原文就是 +"A C library only constrains libc")。对 **C++ 模块库不够**,缺: + +- 编译器**族与主版本**(mangling、libstdc++ 符号版本、模块实现细节) +- stdlib **版本**(`Toolchain::stdlibVersion` 有,但 `AbiProfile` 里没有) +- **C++ 标准档位**(`c++23` vs `c++26` 会改变 `std::` 的可见面与部分 ABI) +- **C++ 运行时契约**(`self-contained` / `toolchain-coupled` / `host-coupled`) + +而这些**全都已经在 `cache_key::BuildAxes` 里了**(`cache_key.cppm:76-95`)。 +不是没有,是没有对外的、可读的、可发布的投影。 + +### G5 —— 接口与二进制没有绑定 + +§2.2(c) 实测。没有 digest、没有 provenance、没有符号存在性检查。 + +### G6 —— 没有面向库的 pack + +`mcpp pack` 的输入是一个 exe。库的 RUNPATH 重写、`$ORIGIN` 化、 +第三方闭包收集、`HOST-REQUIREMENTS` 生成 —— 这些逻辑全在 +`pack.cppm` / `binfmt.cppm` / `host_requirements.cppm` 里,**只差一个入口**。 + +### G7 —— 私有分发没有鉴权轴、没有离线入口 + +见 §1.4。`IndexSpec` 无鉴权字段;没有 `mcpp add ./x.mpkg`。 + +### G8 —— `runtime.library_dirs` 的 link/runtime 混淆(#304,已有 issue) + +新键(`link_library_dirs` / `runtime_search_dirs`)已经把两件事分开了 +(`docs/05-mcpp-toml.md` §2.11 的表),legacy `library_dirs` 仍然两边都进。 +**二进制分发会把这个坑放大**:预编译包必然要声明库目录,而符号farm 式的包 +(`compat.vulkan-runtime`)会因此污染链接线。 + +### G10 —— 裸三元组谓词在原生构建下不匹配(新发现,应单独开 issue) + +`prepare_inputs.cppm:139` 的 `if (triple.empty()) return false;` 让 +`[target.''.build]` 在**没有 `--target`** 时永不命中,而同文件的 +`context_for()`(:49-53)对 `cfg(...)` **回落到 `host_triple()`` ` —— +同一个决定两处推导。`types.cppm:665` 的注释承诺的是回落那一种。 +**危险形状:CI 全绿、本机静默失配。** 详见 §2.4.5(含最小探针)。 +直接阻塞胖包的裸三元组写法。 + +### G9 —— 描述符构建规则不能按版本区分(#290,已有 issue) + +对二进制分发直接相关:同一个包的 `0.1.0` 和 `0.2.0` 可能有不同的 +库文件名 / soname / 依赖集。今天 `mcpp = {}` 块对所有 `xpm` 版本一视同仁。 + +--- + +## 4. 架构方案 + +### 4.1 先定一件事:分发层级是**消费端**选的 + +这是整个方案的形状来源。生产者**发布多个层级**,消费者**按自己的工具链挑一个**, +挑不到就降级到源码。生产者不能替消费者决定,因为「你的编译器是什么」只有消费端知道。 + +| Tier | 包里有什么 | 生效条件 | 失配时 | +|---|---|---|---| +| **S** source | 全部源码 | 永远 | —— | +| **I** interface+binary | 接口源(`.cppm`/`.h`)+ 预编译库 | **abi-tag 匹配** | 降级到 S;S 不存在则**明确拒绝并列出可用 tag** | +| **B** +BMI | 再加预编译 BMI | **build-key 精确匹配** | **静默**降级到 I | + +三条纪律: + +- **Tier B 只能是加速器**,永不成为正确性依赖。失配必须静默降级,不得报错。 +- **Tier I 失配必须响**。这是 §2.3 判据 3。 +- **「因为客户端太老而不可用」必须报告成「不可用」,不能报告成「不存在」** + —— 这条是 #349 索引 floor 那次的教训(被拼成「不存在」的客户端会自己驱动 + 重复刷新索引)。 + +### 4.2 核心原语一:`[distribution]` 段 —— 一条可查询的事实 + +> **相对初稿的修订。** 初稿提议 `[targets.] kind = "prebuilt"`。 +> 它有一个致命性质:**描述符里的新键不降级**(`docs/10` 点名的那一类), +> 老客户端会被砖。改成 `[distribution]` 段之后,老客户端读到的仍是 +> `sources` + `[runtime]`(**已发布能力**),而 `[distribution]` 被**静默跳过**。 +> **兼容性从「要版本 floor」变成「几乎免费」。** +> +> **⚠️ 实测钉死的两条边界(mcpp 2026.8.15.3):** +> 1. `[distribution]` 里放**标量 + 字符串数组**:**接受,且连警告都没有** ✅ +> 2. `artifacts = [{ … }]`(array-of-tables):**硬失败** ❌ +> ``` +> error: [[distribution.artifacts]] (array-of-tables) is not allowed for +> section 'distribution.artifacts'; array-of-tables syntax is only supported +> for [[build.flags]], [[features..flags]], [[runtime.requirements]], +> and [[runtime.artifacts]] +> ``` +> **所以产物清单不能自己造 —— 必须用已在白名单里的 `[[runtime.artifacts]]`** +> (它的字段恰好就是 `role`/`path`/`provenance`/`abi`/`digest`/ +> `host_fingerprint`,`docs/05-mcpp-toml.md` §2.11 已经文档化)。实测: +> `[distribution]` 标量段 + `[[runtime.artifacts]]` 一起,今天的 mcpp 直接通过。 +> +> **诚实的代价:静默跳过意味着老客户端拿到预编译包时「一道闸门都没有」**, +> 拿到的是今天的行为(能用、但不安全)。这是**降级**而不是变砖,方向是对的; +> 但它也意味着闸门只保护新客户端 —— 这一点必须写进发布说明,不能假装没有。 + +```toml +[build] +sources = ["interface/runtimelib.cppm"] # 只有接口;实现在预编译产物里 +# 或 sources = [] # 纯 C 头 + .a 的包:显式为空 ≠ 缺省 + +[distribution] +artifact_kind = "static" +abi_tag = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +... +``` + +**两件事,不要合并:** + +| | 解决什么 | 为什么不能只要一个 | +|---|---|---| +| `sources = []` **显式为空** | 「不要填默认 glob」 | 今天缺省会被填成 `src/**`(`toml.cppm:1703`),于是「作者故意没有源」与「glob 一个都没匹配上」不可区分 | +| `[distribution]` 段 | 「这个包的产物不来自编译」——**一条可查询的事实** | 闸门、`mcpp why`、lint、以及 §4.5.0(d) 的 build 守卫都要读它;挂在 `[build]` 上会让普通源码包也能声明 `abi_tag`,那就成了一个可以说谎的地方 | + +**Appendix A(schema ownership)检验:** mcpp 定义**机制**(产物来自文件而非编译边、 +以及一组闸门),**词汇留在值里**(路径、tag 字符串、`static`/`shared`)。键封闭。✅ + +#### 4.2.1 `[pack]` 字段的取舍 + +一句话:**能从 mcpp.toml 别处推出来的,就不给字段。** +按这条逐条审计之后,`[pack]` **新增 0 个键** —— WHAT/HOW 由 `[targets.].kind` +回答(`mcpp pack [target]`),接口根由 `[lib]` 约定回答,头目录由 +`[build].include_dirs` 回答,平台由 `[package].platforms` 回答(升格为**断言**)。 +完整的推导审计、以及「什么不允许裁剪」的一致性论证在 **§4.6.1**。 + +### 4.3 核心原语二:两个兼容量,不是一个 + +**必须是两个,因为它们回答两个不同的问题。** + +| | `abi-tag` | `build-key` | +|---|---|---| +| 回答 | 「这份二进制能不能链进你的构建」 | 「这份 BMI 能不能直接用」 | +| 谁能算 | **生产者**(消费者存在之前就能枚举) | **只有消费者**(含依赖闭包 Merkle) | +| 粒度 | 粗、可读、可发布 | 精确、16 hex、不可读 | +| 用途 | 决定**下载哪个载荷** | 决定 **Tier B 命中** | +| 来源 | `AbiProfile` + `Toolchain` + `CppStandardConfig` | `cache_key::key_hex`(**已存在**) | + +**判据:如果试图只用一个,就会发现你无法发布 build-key** —— 它含依赖闭包, +每个消费者的图都不同,生产者得为每种图各发一份,不可能。 + +**`abi-tag` 的组成(6 段,全部来自已有字段):** + +``` +-----c++ + +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 +aarch64-macos-none-llvm22-libcxx22-c++23 +x86_64-windows-msvc-msvc194-msvcstl194-c++23 +``` + +- `arch`/`os`/`env`:`triple.cppm` 已有的规范拼写。 + **⚠️ 不是 `--print-fingerprint` 的 [4]** —— 那是编译器自报的 + (`x86_64-w64-mingw32`),与 `[target.'…']` 键的拼写(`x86_64-windows-gnu`) + 不同,直接用会让同一个决定有两个拼写(§2.4.1 实测踩到过); +- `compiler`:`Toolchain::compiler_name()` + `version` 的主段; +- `stdlib`:`Toolchain::stdlibId` + `stdlibVersion` 的主段; +- `c++`:`CppStandardConfig::level`。 +- `cxxAbi` 不入 tag —— 它由 (os, compiler) 唯一确定,放进去是第二个答案。 + +**不入 tag、但必须单独声明并检查的两项:** + +| 字段 | 为什么不入 tag | 检查语义 | +|---|---|---| +| `cxx_runtime`(契约) | 它可以**协商**:toolchain-coupled 的 `.so` 是能被 self-contained 的 exe 消费的(§2.2b 实测),只是有危险 | 不匹配 → **警告并说清危险**,`--strict` 升级为错误 | +| `abi_surface = "c" \| "cxx"` | 纯 `extern "C"` 接口只约束 libc 这一维 —— `abi.cppm` 的注释原文就是这个规则 | `"c"` → 只比 arch/os/env,忽略 compiler/stdlib/std | + +`abi_surface = "c"` 这个逃生口很重要:它让**绝大多数传统 C 库**(zlib、openblas、 +ffmpeg)只需要一个 tag 就覆盖所有编译器,tag 组合数从 N×M 掉回 N。 + +### 4.4 核心原语三:接口↔二进制的绑定 + +三道闸,由弱到强,**全都要**: + +1. **interface digest** —— 包元数据里记随包接口文件的 sha256。消费者编译前重算, + 不符即拒。挡住「有人改了随包的头」。 +2. **符号存在性交叉检查** —— 编完接口后 mcpp 知道模块名 `M`,去预编译库里查 + `_ZGIW`(ELF/Mach-O)/ 导出表(PE)。挡住「配错了库」。便宜,值得做。 +3. **原子产出 + provenance** —— **接口副本与二进制必须由同一次 `mcpp pack ` + 产出**,元数据里记 `built_by` / `build_key` / `source_digest`。 + 手工拼装的包 mcpp 拒绝消费。 + +**必须诚实说清楚:三道闸都挡不住「结构体字段顺序变了但两边都自洽」**—— +只要接口和二进制**能被分别替换**,§2.2(c) 就是可达的。唯一充分的保护是 +**它们一起产出、一起分发、不可分别替换**。所以第 3 条才是根,前两条是防御。 + +### 4.5 包格式:**不是新格式 —— 是一个自带 `mcpp.toml` 的 Form A 包** + +> **本节相对初稿是重写。** 初稿提议一个新格式 `.mpkg` + 一份并列的 +> `MCPP-PACKAGE.toml`。**那是错的形状**,理由见 §4.5.0:mcpp 早就有 +> 「包自带 `mcpp.toml`」这条通路,而且它是文档推荐的那一种。 + +#### 4.5.0 为什么自带 `mcpp.toml` 是对的形状 + +**它已经是既有机制。** 索引描述符没有 `mcpp` 字段时,mcpp 在解开的载荷里 +glob `mcpp.toml` / `*/mcpp.toml` 并当作该依赖的 manifest 加载 +(`prepare.cppm:2764-2787`)。`docs/10-publishing-a-library.md` 原话: +*"A repo that ships its own `mcpp.toml` needs no `mcpp` field in the index entry."* +而 LinkIntent 是**逐包聚合**的(`plan.cppm:630-656`,路径按 `package.root` +转绝对),所以 path / git / 索引 tarball 三条路吃的是同一套 —— §2 的实测走的是 +path 依赖,索引 tarball 汇合到同一处。 + +**三个后果,第一个是决定性的:** + +**(a) 不需要新描述符键 ⇒ 不需要版本 floor ⇒ 老客户端不会被砖。** + +初稿的 `kind = "prebuilt"` 正好属于 `docs/10` 点名警告的那一类**不降级**的键 +(和 `module_extensions` 同类):老 mcpp 读不懂它,不是「警告后忽略」,而是硬失败 +或者更糟 —— 把 `interface/` 当普通源编译然后链不上。走 Form A 就完全不需要它: +老客户端读到的是 `sources = ["interface/…"]` + `[runtime] libraries / +link_library_dirs`,**全部是已发布能力** —— §2 的实测就是在 **2026.8.15.3** 上 +跑通的。**兼容性几乎免费。这是 Form A 相对新描述符键的决定性优势。** + +**(b) 胖包(fat package)可以绕开整个 G3。** + +包里带的是 `mcpp.toml`,于是它可以写 `[target.'cfg(...)'.build]` —— +而这个条件轴**按解析后的 target 求值**,也就是在消费者那边、在 `--target` +已知之后。所以一个包同时装 x86_64 / aarch64 / linux / windows 的库, +**选择发生在构建期**: + +- 不需要描述符 arch 轴; +- 不需要安装线 target 轴; +- **不需要动 xlings 一行。** + +这把「交叉编译 + 二进制包」从 P2(跨仓库)搬到了 **P0 就能表达**。 +代价是下载体积(可用「胖包默认 + 瘦包按 tag」两种发布方式并存来缓解)。 +今天的拼写限制:条件轴只承载 `BuildInputs`(**无 LinkIntent**),所以 per-arch +只能写 `ldflags = ["-Llib/", "-lfoo"]`,写不了 `link_library_dirs`。 +能用,但丑;干净的修法仍是让 LinkIntent 过条件轴(G3 收尾)。 + +**(c) 闸门字段必须放进 `mcpp.toml`,不能放旁路文件。** + +理由不是省一个文件,是 **path 依赖也需要闸门**。如果 abi-tag / interface digest +只存在于 `mcpp pack ` 写的旁路文件里,那么「把一个目录拷给同事」这条最常见的 +内部路径就没有任何检查 —— 而 §2 的实测证明这恰恰是人们会走的路。 +所以:**`MCPP-PACKAGE.toml` 取消,内容并入 `mcpp.toml` 的 `[distribution]` 段。** + +**(d) 必须加的守卫 —— 一个字面叫 `mcpp.toml` 的文件躺在解开的二进制包里, +会引诱人在里面 `mcpp build`。** + +今天它会**成功**:把 `interface/` 里只有声明的接口单元编出来,产出一个几乎空的 +库,而实现全在预编译产物里、根本没被链进去。典型的「看起来成功的失败」。 +**`[distribution]` 存在时,在该目录直接 `mcpp build` 必须拒绝**,并说清这是分发包 +不是源码树。 + +#### 4.5.1 布局 + +``` +runtimelib-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.tar.gz (PE: .zip) +└── runtimelib-0.1.0/ + ├── mcpp.toml ← 生成物,含 [distribution] 段;消费端零新代码路径 + ├── interface/ ← 消费者要编译的 .cppm / .ixx + ├── include/ ← 非模块消费者的头 + ├── lib/ ← 链接面:.a / .so / .dylib / **.lib(PE 导入库)** + ├── bin/ ← 运行面(仅 PE):.dll,必须部署到 .exe 旁 + ├── bmi/ ← 可选 Tier B + ├── HOST-REQUIREMENTS ← 与 mcpp pack 同一份推导 + └── LICENSE / THIRD-PARTY +``` + +**归档格式与源码包同构**(tar.gz / zip),索引条目形状**一字不改** +(url + sha256 + 三平台块)。文件名带 tag 只是**人读的命名约定**,不是新格式 —— +`mcpp add ./` 靠 `[distribution]` 段识别,不靠扩展名。 + +**两条规则,由文件所在的目录决定 —— 这就是全部**(§2.5.2 实测): + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | 预处理器 | **编译器** | +| 消费者要编译吗 | 否 | **是** | +| 要算闭包吗 | 否 | **是** | +| 要从归档剔除对象吗 | 否 | **是** | +| ABI 闸门 | `extern "C"` ⇒ 只约束 libc | 全部维度 | + +两种模式**互不干扰、可同时存在**:实测同一个包被「只 `#include`」/「只 `import`」/ +「两者同时」三种方式消费,× 静态库/动态库两种形态,**六格全过**。 +**传统 C/C++ 库(头 + `.so`/`.a`)因此真的简单** —— 没有闭包、没有剔除; +全部复杂度属于模块接口那一种模式,而它是可自动化的。 + +**`lib/` 与 `bin/` 分开是机制,不是风格。** + +| 类别 | ELF | Mach-O | PE | +|---|---|---|---| +| 链接面 | `lib/libfoo.so` | `lib/libfoo.dylib` | **`lib/foo.lib`(导入库)** | +| 运行面 | 同一个文件 | 同一个文件 | **`bin/foo.dll`(另一个文件)** | + +PE 上「链接的东西」和「运行的东西」是两个不同的文件,ELF/Mach-O 上是同一个。 +一个只分 `interface/` + `lib/` 两层的布局在 Windows 上表达不出这件事 —— 这正是 +`compat.openblas` 的 windows 块必须同时写 `ldflags = {"-Llib","-llibopenblas"}` +**和** `runtime = { library_dirs = { "bin" } }` 的原因。 + +#### 4.5.2 `artifact_kind = "static" | "shared"` —— 一个布局,两种形态 + +**不要为动态库单开一个命令。** 形态是布局里的一个字段,理由是静态那条腿 +**今天三平台就能走通,而且把一整类问题消掉了**(实测,见下)。 + +``` +$ mcpp build # kind = "lib",无平台限制 + → bin/libstatlib.a +$ ar t libstatlib.a → statlib.m.o impl.o +$ nm --defined-only + statlib.m.o: T _ZGIW7statlib ← 接口单元对象:只有模块初始化器 + impl.o: T _ZN2slW7statlib6answerEv ← 实现单元对象:真正的符号 +$ readelf -d libstatlib.a → 无 RUNPATH(归档没有动态段) +``` + +| | `static` | `shared` | +|---|---|---| +| 三平台产出 | ✅ **今天就有**(`kind="lib"` 无平台限制) | ❌ **仅 Linux**(G2) | +| RUNPATH 重写(§2.2a) | **不需要** —— 归档没有动态段 | 必须 | +| 双 C++ 运行时(§2.2b) | **不会发生** —— 由消费者自己的契约统一 | 会,需检查 | +| 运行期部署 | 无 | PE 需部署 `.dll` | +| 加载器搜索闭包 | 无 | 需要 | +| 代价 | 消费者产物体积;不能热替换 | —— | + +**对「闭源库内部分发」这个场景,`static` 往往是更好的默认。** + +**一处必须做成结构性保证、而不是巧合的事:打包时剔除已发布接口单元的对象。** +上面的实测显示归档成员切分是干净的 —— 消费者自编接口后会定义 `_ZGIW7statlib`, +于是 `statlib.m.o` 这个成员没有任何未定义符号需要它,**根本不会被拉进来**。 +但这依赖「归档成员只在解析未定义符号时才被拉取」这条链接器行为; +`--whole-archive`、或该成员里恰好还有别的被引用符号,都会让它被拉进来并与 +消费者自编的接口对象重复定义。打包器把它剔掉,这条风险就不存在了。 + +> **⚠️ 剔除集是「已发布闭包里那些单元的对象」,不是「所有 `.m.o`」。** +> 本文上一版写的是后者,**实测三个 target 全部链接失败**(§2.4.3): +> `.m.o` 的含义是「模块单元的对象」,实现分区照样是 `.m.o` 且里面是真代码。 +> 剔除集与发布集是**同一个闭包的两个用途**(§2.4.2)—— 这正是它们必须由 +> 同一次推导给出、而不是各算各的的理由。 + +#### 4.5.3 包内 `mcpp.toml`:既有的键 + 一个新段 + +**上半部全是今天就能跑的键**(§2 实测,mcpp 2026.8.15.3): + +```toml +# ══ 生成物。手工编辑会使 [distribution].interface_digest 失配并被拒绝。 ══ +[package] +namespace = "acme"; name = "runtimelib"; version = "0.1.0" + +[build] +sources = ["interface/runtimelib.cppm"] # 只有接口,消费者编译它得到 BMI + +[targets.runtimelib] +kind = "lib" + +[runtime] +libraries = ["runtimelib"] +link_library_dirs = ["lib"] +runtime_search_dirs = ["lib"] # shared 形态才需要 +# deploy_files = ["bin/runtimelib.dll"] # PE + +# 胖包:选择在消费者的构建期发生,按解析后的 target 求值 +[target.'cfg(all(linux, arch = "aarch64"))'.build] +ldflags = ["-Llib/aarch64-linux-gnu", "-lruntimelib"] +``` + +**下半部是闸门 + 证据。⚠️ 只能是标量与字符串数组 —— 产物清单必须借用已在 +array-of-tables 白名单里的 `[[runtime.artifacts]]`(见 §4.2 的实测边界):** + +```toml +[distribution] +schema = 1 +artifact_kind = "static" # static | shared +abi_tag = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +abi_surface = "cxx" # cxx | c(纯 extern "C" 只约束 libc) +cxx_runtime = "toolchain-coupled" +modules = ["runtimelib"] +interface_digest = "sha256:…" # 覆盖 interface/ 下每个文件 + +# ── 以下是证据,不是旋钮。用户手写 = 错误,不是「覆盖」。 ── +built_by = "mcpp 2026.8.17.1" +build_key = "aeb4c4d29e437696" # cache_key::key_hex,Tier B 用 +source_digest = "sha256:…" + +# 产物清单 —— 复用既有的、已文档化的段(docs/05 §2.11),不新造。 +# 它的字段恰好就是需要的:role / path / provenance / abi / digest / host_fingerprint。 +[[runtime.artifacts]] +role = "static-library" +path = "lib/libruntimelib.a" +provenance = "mcpp-pack" +abi = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +digest = "sha256:…" +``` + +**实测(mcpp 2026.8.15.3):上面这整份 —— `[distribution]` 标量段 + +`[[runtime.artifacts]]` —— 今天的 mcpp 原样接受并构建成功。** +(顺带一个可观察的证据:加上 `[[runtime.artifacts]]` 后 fingerprint 变了, +说明它确实被读进 manifest 并折进了构建身份,不是被丢掉。) + +**为什么证据字段也放这里、而不是旁路文件:**多一个文件就多一个能和主文件漂移的 +地方 —— 这是仓库反复出现的那条教训(`publisher.cppm:194-216`: +「THE SAME DERIVATION … 分开推导就是它们漂移的方式」)。代价是必须有一条 +**「用户手写了生成字段 = 错误」**的规则,而这条规则本来也需要 +(`built_by` 被人改成别的值,比没有它更糟)。 + +**一份推导,三个投影**(比初稿少一个 —— 包内文件与消费契约合并了): + +``` + ┌→ xpkg.lua 描述符(索引;Form A ⇒ 只有 url + sha256) +一次 resolve+build ──┼→ 包内 mcpp.toml(消费契约 + [distribution] 闸门/证据) + └→ HOST-REQUIREMENTS(bundle) → 消费者 mcpp.lock 条目 +``` + +### 4.6 生产侧:`mcpp pack ` + +```bash +mcpp pack # 唯一可打包目标;有歧义 → 报错并列出候选 +mcpp pack mathkit # [targets.mathkit].kind = "lib" → 静态库包 +mcpp pack mathkit-shared # kind = "shared" → 动态库包 +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 胖包(--target 可重复) +mcpp pack mathkit --tier bmi # 可选 Tier B(默认 interface) +``` + +> **没有 `--lib`,也没有 `--artifact`。** 产出什么由目标的 `kind` 决定 —— +> §4.6.1(b):三个旋钮塌缩成零。 + +做的事(**全部复用 `mcpp pack` 已有机械**): +1. build 出库产物; +2. 拷贝接口源、**从归档中剔除接口单元的对象**(§4.5.2)、算 digest; +3. 生成包内 `mcpp.toml`(消费契约 + `[distribution]`)+ `HOST-REQUIREMENTS`; +4. 打包,**确定性归档**(PE 侧 `zip.cppm` 已经是无时间戳的了)。 + +`--artifact shared` 额外多两步,**这两步是 static 形态根本不需要的**: + +5. **RUNPATH/INTERP 重写** —— §2.2(a) 的解药,`pack.cppm` 已有,只差入口; +6. 收第三方闭包(ELF 走 `LD_TRACE_LOADED_OBJECTS`,PE 走导入表 —— `binfmt.cppm` 已有)。 + +**「三平台」拆成两件事,不要排在一起:** + +| | 打包器 / 布局 / 元数据 | 产出动态库本身 | +|---|---|---| +| 平台相关性 | **无关** —— 就是文件布局 + toml + digest | **强相关** | +| 现状 | 两条产出族已在(ELF/Mach-O→tar.gz、PE→zip),且 **PE 路径任何宿主都能跑**(读导入表,不执行产物) | **仅 Linux**(G2) | +| 工作量 | 小,可一次覆盖三平台 | PE 导入库 + Mach-O install_name,**真正的工作量在这里** | + +没有右列,Windows/macOS 的包是**空盒子**。这就是把 static 提到 P0 的理由: +它让 P0 结束时三平台都拿到能用的包。 + +**验收判据(直接复用 e2e 215 的形状):** 扫遍包里每个二进制, +出现 `$MCPP_HOME` 下的路径即失败。static 形态天然满足(归档无动态段), +但**判据照样要跑** —— 它是对 `include/`、`interface/`、元数据里残留绝对路径的守卫。 + +#### 4.6.1 `[pack]` 的架构 —— 推导审计的结果:一个新键都不加 + +> **本节是第二次重写。** 第一版提了 `default_kind` / `default_artifact` / +> `targets` / `[pack.interface]` / `[pack.headers]` 五组键。 +> 按「**能从 mcpp.toml 别处推出来的,就不给字段**」逐条审计之后,**全部删掉**。 +> 留下的是既有键 + 一个 CLI 位置参数。 + +##### (a) 推导审计 + +对每一个候选键问同一个问题:**这件事 mcpp.toml 别处已经说过了吗?** + +| 第一版的键 | 已经在哪说过了 | 结论 | +|---|---|---| +| `default_kind`(app/lib) | `[targets.].kind` | **删** | +| `default_artifact`(static/shared) | 同上(`"lib"` vs `"shared"`) | **删** | +| `targets = [三元组…]` | `[package].platforms` 问的是同一件事(OS 级) | **删**,改成**校验** | +| `[pack.interface] modules` | lib-root 约定 / `[lib].path` | **删** | +| `[pack.headers] dirs` | `[build].include_dirs` | **删** | +| `[pack.headers] exclude` | 不可推导 —— 但**不允许**(见 (c)) | **删** | +| `include` / `exclude`(extras) | 不可推导 | **保留既有键,语义一字不改** | +| `default_mode` | 不可推导 | **保留既有键** | + +**结果:`[pack]` 新增 0 个键。** + +##### (b) WHAT / HOW 根本不是 pack 的设置 —— 是 `[targets.].kind` + +`mcpp pack [target]`,与既有的 `mcpp run [target]` 同形。**目标的 `kind` 决定一切:** + +| `[targets.].kind` | `mcpp pack ` 产出 | `--mode` 适用吗 | +|---|---|---| +| `bin` | 应用 bundle(既有四档) | ✅ | +| `lib` | **静态库包** | ❌ 归档不携带任何东西,闭包深度无意义 | +| `shared` | **动态库包** | ✅ `lib/` 要不要带上第三方 `.so` | + +`--lib` 这个 flag 不需要了,`--artifact` 不需要了,`default_kind` / `default_artifact` +也不需要了 —— **三个旋钮塌缩成零,因为 `kind` 本来就是答案。** + +**同时发布静态与动态 = 声明两个目标**(实测可行): + +```toml +[targets.mathkit] kind = "lib" +[targets.mathkit-shared] kind = "shared" soname = "libmathkit.so.1" +``` +``` +bin/libmathkit.a +bin/libmathkit-shared.so +bin/libmathkit.so.1 -> libmathkit-shared.so ← soname 别名(既有机制) +``` + +**已知的 wart:`.so` 的文件名带了 `-shared`。** `soname` 给出了正确的运行期名, +打包器发布 soname 那个名字即可。长期干净的修法是让 `kind` 接受列表 +(`kind = ["lib", "shared"]`,Cargo `crate-type` 的先例)——那是一次独立的改动。 +**但绝不要用一个 pack 期的 `--artifact` 覆盖去补救**:那会立刻变成 `kind` 的第二个答案。 + +##### (c) 头与接口在 pack 期**不允许**裁剪 —— 一致性论证 + +这是唯一一组「不可推导、但仍然不给」的键。理由不是「用不上」,是**给了就破坏不变量**: + +> **源码分发的包把 `include_dirs` 的全部内容暴露给消费者**(usage requirements, +> `scanner.cppm:700-716`)。二进制包若裁掉一部分,**同一个库会因为分发形式不同 +> 而拥有不同的公开面** —— 破坏「分发形式不改变语义」。 + +而且「哪些头是公开的」**布局已经回答了**:`include/` 是公开的(它在消费者的 +include 路径上),`src/` 是私有的。**一个私有头放在 `include/` 下是工程布局的错误, +不是打包问题。** 给 pack 一个裁剪键 = 允许用打包配置去补救布局错误, +而补救的结果是两种分发形式不一致。 + +接口 `.cppm` 同理,而且更硬:§2.4.2 的不对称性(发少了响、发多了哑)。 + +##### (d) 既有键升格为**断言**,而不是新增选择器 + +`[package].platforms` 的词汇是**封闭的**(`linux | macos | windows`, +`prepare.cppm:1075-1085`,未知值告警/`--strict` 报错)。不要为了 pack 去扩它。 + +改成:**`mcpp pack` 把 `--target` 的集合与声明的 `platforms` 做覆盖比对, +缺一条腿就告警。** 与 `[modules] exports` 同一形状 —— **既有键做断言,不做选择器**。 +好处:不新增键,而且「0.1.0 声称支持 windows 却没打 windows 那条腿」变成可发现的。 + +##### (e) 最终形态 —— 全部是既有的键 + +```toml +[package] +platforms = ["linux", "windows"] # 既有:声明支持的平台。pack 用它做覆盖校验 + +[build] +include_dirs = ["include"] # 既有:公开头。整发,不可裁 + +[lib] +# path = "src/mathkit.cppm" # 既有:接口根(不写 = src/.cppm 约定) + # 模块闭包从这里出发 + +[targets.mathkit] +kind = "lib" # 既有:决定 `mcpp pack mathkit` 产出什么 + +[targets.mathkit-shared] +kind = "shared" # 既有 +soname = "libmathkit.so.1" # 既有:给出正确的运行期名 + +[pack] +default_mode = "vendored" # 既有键,语义不变(仅 bin / shared 适用) +include = ["share/**"] # 既有键,语义不变 —— 只作用于 extras +exclude = ["**/*.tmp"] # 既有键,语义不变 —— 只从 include 里剔除 +``` + +```bash +mcpp pack # 唯一可打包目标;有歧义 → 报错并列出候选 +mcpp pack mathkit # kind="lib" → 静态库包 +mcpp pack mathkit-shared # kind="shared" → 动态库包 +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 胖包 +``` + +**新增 manifest 键:0。新增 CLI:一个位置参数 +`--target` 可重复。** +(`mcpp pack` 今天只有 `--mode` / `--target` / `--format` / `-o`,没有位置参数 —— +`cli.cppm` 的 `cl::App("pack")`。) + +##### (f) 成文规则:什么不允许、什么不推荐 + +| 类别 | 规则 | +|---|---| +| **不允许** | ① 任何已被别处回答的问题(产物形态 / 接口根 / 头目录);② 任何会让**源码分发与二进制分发语义不同**的裁剪(头、接口) | +| **保留但不推荐** | `[pack].include/exclude` —— 只用于 **extras**(LICENSE / docs / 数据)。**不要拿它裁头或接口:裁不到,而且今天会静默无效**(`types.cppm:743` 的语义是 *drop from `include`*) | +| **必须写** | 无。一个库工程不写任何 `[pack]` 也能打出正确的包 | + +##### (g) 唯一保留的纪律:三态 + +删掉了所有新键,但「不写 = 默认」这条原则本身要求一件事: + +> 每个键必须区分 **不写**(默认)/ **写了有值**(覆盖)/ **写了但为空**(显式什么都不要)。 + +**实测 `sources = []` 与整行删掉在 `build.ninja` 里逐字节等价**(§2.5.4)—— +「不写=默认」在没有三态时会退化成「无法表达空」。这条对 G1 是硬要求 +(二进制包必须能说「什么都不要编」),对未来任何新键也是。 +仓库已有正确的模式:`XlingsConfig::subosDeclared`(`types.cppm:632`)—— +**一个 `Declared` 布尔,而不是靠容器的 `empty()`**。 + +##### (h) 跨平台收口(不变) + +- `lib//` **按三元组分,不按 OS 分** —— MinGW 与 MSVC 同为 windows, + 一个产 `lib*.a` 一个产 `*.lib`(§2.4.4)。 +- PE 的链接面与运行面是两个文件 ⇒ `lib//` 与 `bin//`。 +- per-target `ldflags` 必须生成 `cfg(...)` 谓词,**不能生成裸三元组**(§2.4.5)。 +- `kind = "shared"` × target 的合法性要在**计划阶段**校验:musl 是静态链接的, + 今天会一路走到链接器才炸(§2.5.3)。 +- 打包器**绝不 glob 产物**,用刚跑那次构建的 `target///bin/…`(附录 A ⑬)。 + + +### 4.7 消费侧:解析、闸门、降级 + +**依赖形态(三种,同一个下游路径):** + +> `.mpkg` 只是**人读的命名约定**(文件名里带 tag),不是新格式 —— 里面就是一个 +> 根目录带 `mcpp.toml` 的 tar.gz / zip。识别靠 `[distribution]` 段,不靠扩展名。 + +```toml +# ① 离线文件 —— P0,不需要索引、网络、鉴权、xlings 改动 +runtimelib = { package = "vendor/runtimelib-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.mpkg" } + +# ② 索引(公开或私有)—— P1 +runtimelib = "0.1.0" + +# ③ CLI 直装 +$ mcpp add ./runtimelib-0.1.0-.mpkg +``` + +**闸门表(顺序即诊断顺序):** + +| 检查 | 失配 | +|---|---| +| arch / os / env | **拒绝** —— 载荷本身就不对 | +| `abi.surface == "c"` | 只查上一行,以下全跳过 | +| compiler 族 / 主版本 | **拒绝**;`--allow-abi-drift` 强制,并打印它保护的是什么 | +| stdlib id / 主版本 | **拒绝** | +| C++ 标准档位 | 消费者档位 < 生产者 → **拒绝**(接口可能用到更新的语法);> → 放行 | +| `cxx_runtime` 契约 | **警告** + 说清双运行时危险;`--strict` 升级为错误 | +| interface digest | **拒绝** | +| 模块符号存在性 | **拒绝** | +| build-key(Tier B) | **静默**降级到 Tier I | + +**没有匹配 tag 时的诊断形状**(这条比机制本身还重要): + +``` +error: acme.runtimelib@0.1.0 has no prebuilt artifact for this toolchain + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + aarch64-linux-gnu-gcc15-libstdcxx15-c++23 + note: this package ships no source tier, so there is nothing to fall back to. + fix : ask the publisher for a gcc16 build, or pin [toolchain] to gcc@15. +``` + +**「不可用」不能被拼成「不存在」。** 一个被拼成「找不到包」的失败会让客户端 +自己驱动重复刷新索引 —— #349 的教训。 + +### 4.8 分发侧:索引与安装线 + +> **相对初稿是重写。** 初稿在描述符里加 `kind = "prebuilt"` + `abi_tags`, +> 并因此要求一个 mcpp 版本 floor。**走 Form A 之后这一整块消失了。** + +**首选:胖包 + Form A —— 描述符一字不改。** + +```lua +-- 与一个源码包的描述符完全同构:没有 mcpp = {} 块,只有 url + sha256。 +-- 包内自带 mcpp.toml,里面的 [target.'cfg(...)'] 在消费者构建期选 arch/OS。 +xpm = { + linux = { ["0.1.0"] = { url = { GLOBAL = "…", CN = "…" }, sha256 = "…" } }, + macosx = { ["0.1.0"] = { url = { … }, sha256 = "…" } }, + windows = { ["0.1.0"] = { url = { … }, sha256 = "…" } }, +} +``` + +- **没有新描述符键 ⇒ 没有版本 floor ⇒ 老客户端不会被砖**(§4.2 的实测边界); +- **⚠️ 每条腿的谓词必须是 `cfg(...)`,不能是裸三元组** —— 裸三元组在**原生构建下 + 不匹配**(§2.4.5,根因 `prepare_inputs.cppm:139`),失败形状是 + 「CI 传 `--target` 全绿、开发者本机静默失配」。实测可用的写法: + ```toml + [target.'cfg(all(linux, not(env = "musl")))'.build] + ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + [target.'cfg(all(linux, env = "musl"))'.build] + ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] + [target.'cfg(windows)'.build] + ldflags = ["-Llib/x86_64-windows-gnu", "-lmathkit"] + ``` + 四种构建(native / gnu / musl / windows)全过,且每个 target 的 `build.ninja` + 里只出现自己那条腿(§2.4.6); +- **arch 选择在消费者的构建期**,`cfg()` 按解析后的 target 求值 ⇒ + **交叉编译天然正确,不需要安装线的 target 轴**; +- 镜像/CN 分流/sha256/artifact 通道**全部沿用源码包那一套**。 + +**可选优化(P2+):瘦包,按 tag 分资产。** 只有当胖包体积成为真问题时才做: + +```lua +xpm = { + linux = { + ["0.1.0"] = { + x86_64 = { url = "…-x86_64-…-gcc16-libstdcxx16-c++23.tar.gz", sha256 = "…" }, + aarch64 = { url = "…-aarch64-…", sha256 = "…" }, + }, + }, +} +``` +这用的是 **xim V2 已有的 per-arch resource map**,不是新语法。但它把 arch 选择 +挪回**安装期**,于是重新撞上「安装线没有 target 轴」: + +``` +install_packages {"targets":[...], "yes":true} + → {"targets":[...], "yes":true, "target":{"os":"linux","arch":"aarch64"}} +``` +这是**跨仓库改动**(xlings)。两个不改 xlings 的退路,都不推荐但要写下来: + +| 退路 | 代价 | +|---|---| +| 把 arch 编进包名(`aarch64-runtimelib`) | 违反 SPEC-001 的身份模型;#290 的「同一个库不同版本」问题再犯一次 | +| mcpp 自己下二进制载荷,绕过 xlings | 镜像/CN 分流/校验/断点全要重做一遍;`fetcher/progress.cppm` 只是起点 | + +**决策建议:胖包做默认。** 它让「交叉编译 + 索引二进制包」在 **P1 就可用**, +而不是等 P2 的跨仓库协调 —— 瘦包只是体积优化,不是能力前提。 + +### 4.9 私有分发:三种形态 + +| 形态 | 机制 | 需要新增 | 阶段 | +|---|---|---|---| +| **离线包** | `mcpp add ./x.mpkg` / `{ package = "..." }` | 一个依赖形态(格式已有:自带 `mcpp.toml` 的 tar.gz) | **P0** | +| **私有索引(git)** | `[indices] acme = { url = "git@..." }` | 文档 + `.mpkg` URL 支持;鉴权仍走 ambient git 凭据 | P1 | +| **私有 artifact 源** | `IndexSpec::artifact` | `IndexSpec` 加鉴权(header / netrc / helper) | P2 | + +**关于鉴权的立场建议:不要发明 mcpp 自己的凭据存储。** 走两条既有轨道: +① git 的 credential helper / SSH(索引侧,今天已经在用); +② 一个 `[indices.] auth = { header_env = "ACME_TOKEN" }` 形状 —— **值从环境变量读, +永不落盘**。理由:mcpp 的 manifest 是要进版本库的,任何能写 token 的字段都会 +被写进版本库。 + +**关于「团队内部统一编译器」——issue 作者说「对于 mcpp 来说统一编译器和版本不是难事」, +这句话是对的,而且这正是 mcpp 相对 CMake/vcpkg 的结构性优势:** +`[toolchain] default = "gcc@16.1.0"` 是可以进版本库的一行,payload 由 xim 保证 +逐字节相同。所以团队内部的 tag 组合数常常是 **1**。这让 Tier I 在私有场景里 +极其实用 —— 也让 Tier B(BMI)在私有场景里第一次变得**可能**(见 §6)。 + +### 4.10 非 mcpp 消费者(issue 场景 4) + +`mcpp pack ` 顺带产出: + +- **`.pc`(pkg-config)** —— 便宜,覆盖传统 `#include` + `-lfoo` 消费者; +- **CMake package config** —— 同上; +- **模块接口的互操作:诚实地说,没有好路。** CMake 的 C++20 modules 支持要求 + 消费方自己扫描并编译 `.cppm`,能做但很脆。**建议:对非 mcpp 消费者, + `mcpp pack ` 额外产出一份「非模块外观」(头文件 + `extern "C"` 或普通 + C++ 声明),由包作者显式声明,而不是自动生成。** 自动从模块接口生成头文件 + 是一个独立的、很大的题目,不该混进这个方案。 + +--- + +## 5. 分期与验收判据 + +> **P0/P1 的切分线是「形态」,不是「平台」。** 初稿按平台切(P0 只做 Linux), +> 那是错的:它让 Windows/macOS 用户在 P0 结束时拿到一个**空盒子**。 +> 按形态切之后,P0 三平台都产出能用的包,P1 才去解锁最难的动态形态。 + +### P0 —— `static` 形态,三平台一次做完(不依赖 xlings、不依赖 G2) + +1. `[distribution]` 段 + `sources = []` 的显式表达(G1、§4.2), + 以及 §4.5.0(d) 的「分发包目录里不许直接 build」守卫 +2. 包布局:根目录 `mcpp.toml` + `[distribution]` 段 + `[[runtime.artifacts]]`, + `lib/` 与 `bin/` 分离(§4.5)。**归档格式与源码包同构,索引条目一字不改。** +3. `mcpp pack `(位置参数;`kind = "lib"` ⇒ 静态库包)——**三平台**, + 含「从归档剔除已发布闭包的对象」(§4.5.2) +4. `mcpp add ./x.mpkg` + `{ package = "..." }` 依赖形态 +5. abi-tag 计算 + 闸门表(仅 arch/os/env/compiler/stdlib/std)(G4) +6. interface digest + 模块符号存在性检查(G5) + +**为什么 static 能在 P0 覆盖三平台:**`kind = "lib"` 没有平台限制(实测产出 +`bin/libstatlib.a`,`ar t` 显示接口对象与实现对象分离,`readelf -d` 无 RUNPATH), +所以 G2、RUNPATH 重写、闭包收集、DLL 部署**这一期全部不需要**。 + +**验收:** +- [ ] e2e:**三平台**各能 `mcpp pack ` 出一个包 并被消费者 `import` + 链接 +- [ ] e2e:`.mpkg` 里不含任何 `$MCPP_HOME` 路径(照抄 e2e 215;含元数据与 `interface/`) +- [ ] e2e:**篡改随包接口 → 构建必须失败**(直接把 §2.2(c) 那个 struct 互换 + case 变成回归测试 —— 它今天是静默通过的,这是最有价值的一条) +- [ ] e2e:abi-tag 不匹配 → 拒绝,且诊断里**列出可用 tag** +- [ ] e2e:同一个 `.mpkg` 在**清空 build cache 的第二台 MCPP_HOME** 上可消费 + (防止「只在写它的机器上能用」) +- [ ] unit:打包器确实剔除了接口单元对象(`ar t` / PE 等价物断言), + 而不是依赖「归档成员只在解析未定义符号时被拉取」这条巧合 +- [ ] `mcpp why` 能说出「这个依赖是 prebuilt / 形态是什么 / tag 是什么 / 为什么选了它」 + +### P1 —— `shared` 形态解锁三平台 + 索引通路 + +7. PE 导入库(`--out-implib` / `/IMPLIB:`)+ Mach-O `-install_name @rpath/…`, + 解除 `plan.cppm:1002` 的拒绝(G2) +8. `mcpp pack `(`kind = "shared"`):RUNPATH/INTERP 重写 + 闭包收集 + `bin/` 部署面 +9. **胖包**走 Form A 上索引:描述符与源码包同构(url + sha256), + `[target.'cfg(...)']` 在消费者构建期选 arch/OS —— **无新描述符键、无版本 floor** +10. 私有索引发布的文档与端到端验证 + +**验收:** +- [ ] 三平台各有一个 shared 库 e2e(今天全部 `# requires: elf`) +- [ ] Windows:消费者链 `lib/foo.lib`、`bin/foo.dll` 部署到 `.exe` 旁、**直接跑 `.exe`** + (不能用 `mcpp run`,它会塞 PATH 掩盖部署问题 —— 这是既有教训) +- [ ] shared 形态的 `.mpkg` 在第二台机器上可用(§2.2a 的回归守卫) +- [ ] `cxx_runtime` 契约不匹配时**有告警**(§2.2b 今天是静默的) +- [ ] **老 mcpp(2026.8.15.3)拿到这个包能构建成功** —— Form A 的兼容性主张必须 + 有一条对着**已发布二进制**跑的测试,不能只在新 mcpp 上验证 +- [ ] 胖包 + `--target aarch64-linux-gnu` 选到正确的那条腿(交叉编译回归) + +### P2 —— 交叉编译 + 私有鉴权 + +10. 瘦包(按 tag 分资产)+ `install_packages` 加 target 轴(跨仓库,与 xlings 同步) + —— **仅当胖包体积成为真问题时才做**,它不是能力前提 +11. `IndexSpec` 鉴权(值从环境变量读) +12. `target_cfg` 承载 LinkIntent(顺手修 #258 同形状的债) + +### P3 —— Tier B(BMI)与生态收尾 + +13. Tier B:导出/导入构建缓存条目,**必须**先做跨机器可行性实验(见 §6) +14. `.pc` / CMake config 产出 +15. 修 #304(`library_dirs` 的 link/runtime 分离收口)、#290(按版本区分构建规则) + +--- + +## 6. 明确不做 / 需要先证伪的事 + +**(a) 不要自动从模块接口生成 C 头文件。** 独立的大题目,混进来会把这个方案拖死。 + +**(b) 不要为二进制包发明第二套身份模型。** `(namespace, name)` + SPEC-001 不变, +tag 是**载荷选择器**,不是身份的一部分。把 arch 编进包名是明确的错误形状(§4.8)。 + +**(c) Tier B 不能凭推理设计进去 —— 必须先做实验。** 已知的三个坑: +- GCC 把**时间戳**写进 BMI(实测过),所以「BMI 逐字节相同」不能作为判据; +- GCC 把**源码路径**写进 BMI,跨机器路径不同; +- clang `--precompile` 发的是 **full BMI**(体积 16 倍且会让下游 TU 编错), + 两阶段必须 `-Xclang -emit-reduced-module-interface`。 + +**判据:在两台不同 `$MCPP_HOME`、不同用户名、不同路径的机器上,同一个 +build-key 的 BMI 能被对方直接使用并产出可运行程序。** 做不到就不要做 Tier B, +它是纯加速,不值得为它引入正确性风险。 + +**(d) 不要把 `cxx_runtime` 契约折进 abi-tag。** 它可以协商,tag 不可以。 +折进去会让 tag 组合数翻三倍,而且会把一个「警告」变成「找不到载荷」。 + +**(e) 不要指望闸门能挡住 ODR/布局漂移。** §4.4 已经说了。**唯一充分的保护是 +原子产出。** 任何「我们检查得够多了所以可以允许手工拼包」的说法都是错的。 + +--- + +## 7. 需要 review 的决策点 + +| # | 决策 | 我的建议 | 反方理由 | +|---|---|---|---| +| **D1** | P0 走**离线 `.mpkg` 文件**,还是直接做索引通路? | **离线文件**。不碰 xlings、不碰网络、不碰鉴权,完全在 mcpp 手里,而且正好是 issue 作者要的形状(`pip install x.whl`) | 索引通路才是「生态」,文件通路会不会变成永久的旁路 | +| **D2** | `kind = "prebuilt"` **新增目标种类**,还是只让 `sources = []` 合法? | **两个都要**,语义不同(§4.2) | 只加一个键更省;但 tag/digest 需要一个不说谎的挂点 | +| **D3** | abi-tag 的**粒度**:主版本 还是 完整版本? | **主版本**(`gcc16`)。完整版本会让 tag 数量爆炸,而 gcc 的 mangling/ABI 在主版本内稳定 | 主版本内也可能有 ABI 变化;可用 `--allow-abi-drift` 兜 | +| **D4** | `abi_surface = "c"` 逃生口要不要? | **要**。它让传统 C 库的 tag 数从 N×M 掉回 N,而且 `abi.cppm` 已经建模了这个规则 | 会被误用在实际有 C++ 接口的包上 → 需要 lint | +| **D5** | P0/P1 按**平台**切,还是按**形态**切? | **按形态**(修订自初稿)。P0 = `static`,三平台一次做完(`kind="lib"` 无平台限制,实测已验证);P1 = `shared`,解锁 PE 导入库 / Mach-O install_name | issue 明确写了 `.so/.dll/.dylib`,先给 static 会被读成「答非所问」—— 需要在回复里说清 static 消掉了哪三类问题 | +| **D9** | `artifact_kind` 是**布局里的字段**,还是**两个命令**? | **一个字段、一个命令、一个布局**。动态是静态的超集(多两步:RUNPATH 重写 + 闭包收集) | 两个命令更容易分别演进;但会产出两套布局与两套元数据,正是「一个决策两处推导」 | +| **D22** | `[pack]` 到底加几个键? | **0 个**(§4.6.1 推导审计)。WHAT/HOW → `[targets.].kind` + `mcpp pack [target]`;接口根 → `[lib]`;头目录 → `[build].include_dirs`;平台 → `[package].platforms` 升格为断言 | 同时发布静态+动态要声明两个目标,`.so` 文件名带后缀(soname 别名给出正确运行期名)。干净修法是 `kind` 接受列表(Cargo `crate-type`),独立改动 | +| **D23** | 头 / 接口允不允许在 pack 期裁剪? | **不允许**。源码分发把 `include_dirs` 全量暴露给消费者;二进制包裁掉一部分 ⇒ **同一个库因分发形式不同而公开面不同**。而且「哪些头公开」布局已经答了(`include/` vs `src/`)—— 私有头放在 `include/` 下是布局错误,不是打包问题 | 有作者确实想 curate;但那应该改布局,不是加打包旋钮 | +| **D18**(已被 D22 取代) | `[pack]` 要不要能「自定义接口文件」? | **部分要**(完整设计见 §4.6.1):WHAT/HOW 三个标量键(`default_kind` / `default_artifact` / `default_mode`)+ `targets` 列表;CONTENT 按**四个集合各给各的键**(`[pack.interface] modules`、`[pack.headers] dirs/exclude`、既有 `include/exclude` 保留给 extras)。**不加 `.cppm` 文件清单** | 作者会希望有「万能逃生口」;真要给,必须做成 `scan_overrides` 那种**断言+校验**,不是选择器 | +| **D20** | CONTENT 用**一对全局 `include`/`exclude`**,还是**每个集合一个键**? | **每个集合一个键**。四个集合的裁剪风险完全不同(interface 不可裁 / headers 可裁但必须 `-MM` 校验 / artifacts 不可裁 / extras 自由),共用一对键会让「删 README」和「删公开头依赖的私有头」看起来是同一个操作 | 键更多;但既有 `include`/`exclude` 语义**一字不改**(它们本来就只作用于 extras),所以不是破坏性变更 | +| **D21** | 「不写 = 默认」要不要配套**三态**(缺席/有值/显式空)? | **要**。实测 `sources = []` 与整行删掉逐字节等价(§2.5.4)——「不写=默认」的原则在没有三态时会退化成「无法表达空」。用 `XlingsConfig::subosDeclared` 那个模式 | 每个键多一个 `Declared` 布尔;但这是仓库已有的、被验证过的形状 | +| **D19** | 接口闭包建在 M1 文本扫描器还是 P1689? | **P1689**。实测 M1 不把 `module X:part;` 建模为 provider(`scanner.cppm:641-650`),导致「接口够到实现分区」的告警产不出来 | P1689 要跑编译器,更慢;但只在 `pack --lib` 时跑一次 | +| **D16** | 两种接口模式(`include/` 文本 / `interface/` 编译)用**目录**区分,还是用 manifest 里的键区分? | **目录**。规则由文件所在位置决定,不需要第二处声明,也不会漂移;实测两种模式互不干扰、可同时存在 | 目录名成为约定的一部分;作者若把 `.h` 放进 `interface/` 会被当成要编译的东西 —— 需要 lint | +| **D17** | `kind="shared"` 那道守卫的判据改成「是否动态链接」,单独开 issue 还是并进 P1? | **并进 P1**。它就是 G2 的一部分,而且不修的话 musl 用户拿到的是一条读不懂的链接器错误 | 也可以先只补诊断(便宜),真正支持留到 P1 | +| **D14** | §2.4.5 的裸三元组缺陷,单独开 issue 还是并进本方案? | **单独开**。它与二进制分发无关(任何用 `[target.''.build]` 的工程都中招),修法看起来是一行,但需要自己的回归测试 | 并进来能少一个 PR;但会把一个通用缺陷埋进一个大特性里 | +| **D15** | 接口发布集用**闭包**,还是让作者在 manifest 里手写清单? | **闭包**(mcpp 的 scanner 已有模块图,是免费的)。手写清单会漂移,而漂移的方向是**静默泄露源码** | 闭包对作者不可见;需要 `mcpp pack ` 打印「将发布 / 不发布」两张清单,并对「接口够到实现分区」告警 | +| **D11** | 包内自带 `mcpp.toml`(Form A),还是并列一份 `MCPP-PACKAGE.toml`? | **自带 `mcpp.toml`**(采纳,已重写 §4.5)。它消掉新描述符键 / 新消费路径 / 版本 floor,并让 path 依赖也吃到闸门 | 一个字面叫 `mcpp.toml` 的文件在二进制包里会引诱 `mcpp build` —— 需要 §4.5.0(d) 的守卫,否则是「看起来成功的失败」 | +| **D12** | 闸门+证据放同一段,还是拆两个文件? | **同一段**(`[distribution]`)。多一个文件就多一个能漂移的地方;代价是需要一条「用户手写生成字段 = 错误」的规则,而这条规则本来也要有 | 生成字段与用户字段混在一段里,靠规则而非结构区分 | +| **D13** | 胖包(一包多 arch,构建期选)做默认,还是瘦包(按 tag 分资产)? | **胖包**。它让交叉编译在 P1 就正确,**完全不需要动 xlings**;瘦包是 P2 的体积优化,不是能力前提 | 大型闭源 runtime × 4 tag 的体积;可两种并存 | +| **D10** | `static` 做默认形态? | **是**。对闭源内部分发,它消掉 RUNPATH 重写、DLL 部署、双 C++ 运行时、加载器闭包四类问题 | 大型 runtime 库可能就是要动态(热替换、体积);默认可被 `[pack] default_artifact` 覆盖 | +| **D6** | 安装线 target 轴(跨仓库)排 P2,可接受吗? | **可接受**,因为 P0/P1 用离线文件与显式 tag 绕开了它 | 交叉编译 + 索引二进制包在 P2 之前不可用 | +| **D7** | 鉴权:环境变量 header,还是接入 git credential helper? | **两条都走**:索引 clone 用 git 凭据(今天已经如此),artifact/URL 用 `header_env` | 有人会希望 mcpp 自己存 token —— 建议明确拒绝 | +| **D8** | §2.2(c) 那个静默错数据,要不要**先单独开 issue 并立刻加回归测试**? | **要**,不等整个方案。它今天就可达(path 依赖 + `[runtime] libraries` 是已发布能力) | 它需要 digest 机制才能真正修;但先把测试写成「预期失败」也有价值 | + +--- + +## 附录 A. 复现脚本 + +材料在 `scratchpad/bindist/`,三个工程: + +```bash +# ① 生产者:模块接口只有声明,实现在实现单元 +provider/src/runtimelib.cppm export module runtimelib; export namespace rt { ... } +provider/src/impl.cpp module runtimelib; namespace rt { ... } +provider/mcpp.toml [targets.runtimelib] kind = "shared" +mcpp build → bin/libruntimelib.so + +# ② 「二进制包」:接口源 + 预编译库 + mcpp.toml +dist/src/runtimelib.cppm (从 ① 拷贝) +dist/lib/libruntimelib.so (从 ① 拷贝) +dist/mcpp.toml [runtime] libraries / link_library_dirs / runtime_search_dirs + +# ③ 消费者 +app/mcpp.toml runtimelib = { path = "../dist" } +app/src/main.cpp #include <...> 然后 import runtimelib; +mcpp run → answer=42 / name=... / caught runtime_error + +# ④ skew 实验(本方案要挡的那个) +# 只把 dist/src/runtimelib.cppm 里的 struct Point { int x; int y; } +# 改成 struct Point { int y; int x; } +mcpp run → x=222 y=111 ← 编译链接运行全过,数据是错的,零诊断 + +# ⑤ 静态形态(scratchpad/statictest/):同样的两个源,[targets.x] kind = "lib" +mcpp build → bin/libstatlib.a +ar t libstatlib.a → statlib.m.o impl.o +nm --defined-only statlib.m.o: T _ZGIW7statlib ← 只有模块初始化器 + impl.o: T _ZN2slW7statlib6answerEv ← 真正的符号 +readelf -d libstatlib.a → (无动态段 ⇒ 无 RUNPATH) + +# ⑥ 兼容性边界(用已发布的 mcpp 2026.8.15.3 跑,不是用新构建) +# 在 statictest/mcpp.toml 末尾追加: +[distribution] schema/artifact_kind/abi_tag/abi_surface/interface_digest/modules + → mcpp build 成功,连警告都没有 ✅ +artifacts = [{ path = "…", role = "…" }] （array-of-tables 于新段） + → 硬失败:"array-of-tables syntax is only supported + for [[build.flags]], [[features..flags]], + [[runtime.requirements]], and [[runtime.artifacts]]" ❌ +[distribution] 标量段 + [[runtime.artifacts]]（既有白名单段） + → 成功,且 fingerprint 变化(证明它被读进 manifest) ✅ +``` + + +### 实验二(§2.4):`scratchpad/lab/` + +``` +lab/mkdist.py 176 行的 `mcpp pack ` 原型(采 tag / 算闭包 / 剔对象 / 生成 toml) +lab/parts/ 生产者:主接口 + 接口分区 + 实现分区 + 实现单元 + C API + 头 +lab/fatpkg/ 产出的胖包:interface/ include/ lib// mcpp.toml +lab/consumer/ 消费者(path 依赖 fatpkg) +lab/probe/ §2.4.5 的最小探针:裸三元组 vs cfg(linux) + +# ⑦ 三条被实验推翻/确立的规则 +python3 mkdist.py parts fatpkg x86_64-linux-gnu x86_64-linux-musl x86_64-windows-gnu + 接口闭包 (2 个): mathkit.cppm, api.cppm + 未发布 : secret.cppm ← 实现分区不外发(但它也有 .m.o) + 剔除归档成员: ['api.m.o', 'mathkit.m.o'] ← 不是所有 .m.o(第一版剔全部 → 三平台链接全挂) + +# ⑧ 全矩阵(胖包用 cfg(...) 谓词) +cd consumer && mcpp build [--target …] + (native) OK x86_64-linux-gnu OK x86_64-linux-musl OK x86_64-windows-gnu OK(PE32+) + build.ninja 中各自只出现自己那条腿的 fatpkg/lib/ + +# ⑨ 最小探针:[target..build] cxxflags = ["-DX"] 的命中次数 + 裸 mcpp build --target x86_64-linux-gnu + 'x86_64-linux-gnu'(裸三元组) 0 2 ← 缺陷 + 'cfg(linux)' 2 2 +``` + +### 实验三(§2.5):两种接口模式 × 两种库形态 + +``` +lab/parts/ 同时提供 include/mathkit_c.h(extern "C") 与 interface/*.cppm(模块) +lab/fatpkg/ 静态库包 lab/sopkg/ 动态库包 lab/hdrpkg/ 纯头文件包(无 .cppm) +lab/c_hdr/ 只 #include lab/c_mod/ 只 import lab/c_both/ 两者同时 + +# ⑩ 六格矩阵 —— 同一个包,三种消费方式 × 两种库形态 + 只 #include 只 import 两者同时 + 静态库 .a hdr: 5 mod: 5 both: c=5 mod=5 + 动态库 .so hdr: 5 mod: 5 both: c=5 mod=5 + 消费者 ELF: NEEDED libmathkit.so + runtime_search_dirs 进 RPATH ✅ + +# ⑪ musl + kind="shared":过了守卫,死在链接器 +mcpp build --target x86_64-linux-musl + crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' + can not be used when making a shared object + (守卫判据是 os != "linux";musl 是 linux ⇒ 过闸。真正的判据应是「是否动态链接」) + +# ⑫ sources = [] 不生效 —— 与「不写」逐字节等价 + 包里放一个遗留的 src/leftover.cpp: + sources = [] → build.ninja 里 leftover 出现 7 次 + (整行删掉) → build.ninja 里 leftover 出现 7 次 ← 完全一样 + ⚠️ 我一开始误以为它能用,因为测试包没有 src/ 目录 —— 默认 glob 也匹配不到东西。 + 「两条路径给出同一答案」不等于「这条路径生效了」。 + +# ⑬ 我自己踩的坑(仓库已知形状):mkdist 用 rglob 取产物 → 挑到陈旧 fingerprint 目录 + target/x86_64-linux-gnu/0f8ab572…/ 缺 capi.o ← rglob 取到的 + target/x86_64-linux-gnu/d02fd93d…/ 有 capi.o ← 刚构建出来的 + ⇒ 打包器绝不能 glob 产物,必须用它刚跑那次构建的 target///bin/… 路径。 +``` + +环境:mcpp 2026.8.15.3,gcc@16.1.0,x86_64-linux-gnu; +交叉 target 的载荷:`xim-x-musl-gcc/16.1.0`、`xim-x-mingw-cross-gcc/16.1.0`。 + +## 附录 B. 证据索引 + +| 事实 | 位置 | +|---|---| +| 描述符强制 `sources` | `src/manifest/xpkg.cppm:2058-2063` | +| 描述符 `mcpp` 段封闭键表(无 arch) | `src/manifest/xpkg.cppm:228-234` | +| 描述符 `target_cfg`(**已有** arch 条件构建输入) | `src/manifest/xpkg.cppm:1315-1400` | +| `target_cfg` 只承载 BuildInputs(无 LinkIntent) | `src/manifest/xpkg.cppm:1359-1377` | +| 描述符 `runtime` 子键(LinkIntent) | `src/manifest/xpkg.cppm:1925-1966` | +| `kind = "shared"` 仅 Linux | `src/build/plan.cppm:1002-1017` | +| `kind = "lib"` **无**平台限制,产出 `bin/lib` | `src/build/plan.cppm:411-423`(`target_output`) | +| **Form A:载荷自带 `mcpp.toml` 时 glob 加载它** | `src/build/prepare.cppm:2764-2787`(及 `2470` 的存在性探测) | +| LinkIntent **逐包**聚合(path/git/索引同一条路) | `src/build/plan.cppm:630-656` | +| legacy `library_dirs` 已**不**进 `linkLibraryDirs`(#304 的一半) | `src/build/plan.cppm:653-656` | +| array-of-tables 白名单(闸门段不能自造产物清单) | 实测,附录 A ⑥ | +| **裸三元组谓词在原生构建下不匹配**(新缺陷) | `src/build/prepare_inputs.cppm:139` vs `:49-53`;承诺见 `src/manifest/types.cppm:665` | +| cfg 词汇 = 规范 triple 词汇(os/arch/family/env + 别名) | `src/build/prepare_inputs.cppm:39-148` | +| MSVC 与 Itanium 的 cxxabi 分叉 | `src/toolchain/abi.cppm:84` | +| 接口闭包 / `.m.o` 不是判据 / 剔除集 | 实测,附录 A ⑦ | +| `[modules] exports` 是**完备性断言**(写错硬失败),不是选择器 | `src/modgraph/validate.cppm:93-117` | +| 既有 `[pack]` 键(新键的先例) | `src/manifest/types.cppm:736-745`;解析 `src/manifest/toml.cppm:1337-1347` | +| **`[pack].exclude` 只从 `include` 里剔除**(裁不到 headers) | `src/manifest/types.cppm:743` 注释原文 | +| `[pack.]` 子表先例 | `docs/02-pack-and-release.md` §Configuration(`[pack.bundle-project]`) | +| 三态模式(缺席 vs 显式空) | `src/manifest/types.cppm:632`(`subosDeclared`) | +| 「断言 + 校验」的先例(`scan_overrides`) | `src/manifest/types.cppm:62-70` | +| **M1 扫描器不把实现分区建模为 provider** | `src/modgraph/scanner.cppm:641-650`;告警 `:984` | +| P1689 扫描后端(闭包应建在它上面) | `src/modgraph/scanner.cppm:121-124` | +| 两种接口模式 × 两种库形态 六格全过 | 实测,附录 A ⑩ | +| `kind="shared"` 在 musl 上过闸后链接失败 | 守卫 `src/build/plan.cppm:1002`;实测 附录 A ⑪ | +| `sources = []` 被默认 glob 吞掉 | `src/manifest/toml.cppm:1703`;实测 附录 A ⑫ | +| 静态归档:接口对象与实现对象分离、无 RUNPATH | 实测,附录 A ⑤ | +| PE 需导入库(字段已存在) | `src/build/plan.cppm:452-479` | +| 平台轴只有 OS,无 arch | `src/platform/axis.cppm:63-87` | +| 安装线协议无 target/arch | `src/pm/package_fetcher.cppm:419-431` | +| 五维 ABI 模型 + `abi:=` | `src/toolchain/abi.cppm:31-145` | +| C 库只约束 libc(逃生口的依据) | `src/toolchain/abi.cppm:12-16` | +| ABI 完备的逐包 key(tag 的原料) | `src/build/cache_key.cppm:20-46, 76-133` | +| 构建缓存布局(BMI+obj 已在盘上) | `src/bmi_cache.cppm:1-45` | +| C++ 运行时契约三层模型 | `src/build/distribution.cppm:1-120` | +| SharedLibrary 双运行时危险 | `src/build/distribution.cppm:51-64`(role 注释)、`154-180`(`default_contract`) | +| 三平台同一份源码 tarball | `src/pm/publisher.cppm:289-303` | +| 「一份推导,多个投影」的既有纪律 | `src/pm/publisher.cppm:194-216` | +| 私有索引无鉴权字段 | `src/pm/index_spec.cppm:14-57` | +| lock 不 pin 索引依赖、不记 ABI | `mcpp.lock` 头部注释 / `src/pm/lock_io.cppm:29-36` | +| pack 的输入是一个 exe | `src/pack/pack.cppm:Plan::builtBinary` | +| anchor TU 手法样板 | `~/.mcpp/registry/data/mcpplibs/pkgs/c/compat.openblas.lua` | +| xim V2 的 arch 轴(按宿主解析) | `~/.mcpp/registry/data/xim-pkgindex/docs/V2/xpackage-spec.md` §"三种新版本条目形状" | +| LinkIntent 的 link/runtime 分离表 | `docs/05-mcpp-toml.md` §2.11 | +| 新键需要版本 floor 且必须降级 | `docs/10-publishing-a-library.md` §"Manifest keys that need a version floor" | +| schema 准入原则 | `docs/05-mcpp-toml.md` Appendix A | diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md new file mode 100644 index 00000000..8324b9e4 --- /dev/null +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -0,0 +1,585 @@ +# 库分发:`mcpp pack ` 与二进制包(2026-08-17) + +> **这份是方案。** 发现(现状、实测、file:line)在 +> `2026-08-17-distribution-architecture-analysis-and-design.md`;那份文档的 +> §2(五轮实测)是本方案每一条判据的证据来源,本文只在需要时回指。 +> +> 起因:issue #433「预编译 .so + .ixx/.h/.cppm 接口」。 +> 讨论过程中我有**四处设计被自己的实测推翻**,都记在 §9,因为**错的那版看起来同样合理**。 + +--- + +## 0. 一页纸 + +**模型三句话:** + +1. **二进制包不是新格式** —— 它是一个**自带 `mcpp.toml` 的普通 tarball**,走 mcpp 已有的 + Form A 通路(`prepare.cppm:2764`)。索引条目与源码包**一字不差**。 +2. **产出什么由 `[targets.].kind` 决定** —— `mcpp pack `。 + 没有 `--lib`,没有 `--artifact`。**新增 manifest 段 0 个、键 0 个** —— + 生成的包就是一个**普通的 `mcpp.toml`**(§2.4)。 +3. **能算出来的东西不给字段** —— 接口发布集是**模块闭包**,公开头是 `include_dirs` 全量, + 两者都**不允许**在 pack 期裁剪(裁了会让源码分发与二进制分发语义不同)。 + +**四条不可让步的判据:** + +| # | 判据 | 违反会怎样(实测) | +|---|---|---| +| **J1** | 接口与二进制**原子产出、不可分别替换** | 随包 `.cppm` 改一行结构体字段序 ⇒ 编译链接运行全过、**数据是错的、零诊断** | +| **J2** | 产物里**不含生产机器的绝对路径** | `.so` 烧着 `/home/…/.mcpp/…` 的 RUNPATH,换机器即废 | +| **J3** | 发布集 = **从公开根算的模块闭包**,不是 `.m.o` | 实现分区照样产 `.m.o` ⇒ 按 `.m.o` 选会**泄露闭源源码** | +| **J4** | per-target 的 `ldflags` 用 `cfg(...)` **不用裸三元组** | 裸三元组在原生构建下不匹配 ⇒ **CI 全绿、本机静默失配** | + +--- + +## 1. 模型 + +### 1.1 两种接口模式,两条规则,由目录决定 + +``` +pkg/ +├── mcpp.toml ← 生成物:一个普通的 mcpp.toml(§2.4),没有新段 +├── include/ ← 文本接口:原样全发,消费者 #include,不产生任何对象 +├── interface/ ← 模块接口:算闭包,消费者编译它得到 BMI + object +├── lib// ← 链接面:.a / .so / .dylib / .lib(PE 导入库) +├── bin// ← 运行面(仅 PE):.dll,部署到 .exe 旁 +├── HOST-REQUIREMENTS ← 与 mcpp pack 同一份推导(仅当有内容要说) +└── LICENSE / THIRD-PARTY +``` + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | 预处理器 | **编译器** | +| 消费者要编译吗 | 否 | **是** | +| 要算闭包吗 | 否 | **是**(J3) | +| 要从归档剔对象吗 | 否 | **是** | +| ABI 闸门 | `extern "C"` ⇒ tag 只有三段(只约束 libc) | tag 全段 | + +**两种模式互不干扰、可同时存在** —— 实测:同一个包被「只 `#include`」/「只 `import`」/ +「两者同时」三种方式消费 × 静态/动态两种形态,**六格全过**。 + +**`lib/` 与 `bin/` 分开是机制不是风格**:PE 上链接的文件(`.lib` 导入库)与运行的文件 +(`.dll`)是两个;ELF/Mach-O 是同一个。 +**按三元组分目录不按 OS 分**:MinGW 与 MSVC 同为 windows,一个产 `lib*.a` 一个产 `*.lib`。 + +### 1.2 两个兼容量,不是一个 + +| | `abi_tag` | `build_key` | +|---|---|---| +| 回答 | 「这份二进制能不能链进你的构建」 | 「这份 BMI 能不能直接用」 | +| 谁能算 | **生产者**(消费者存在之前就能枚举) | **只有消费者**(含依赖闭包 Merkle) | +| 来源 | `mcpp build --print-fingerprint` 的 [1][2][4][5][6] 的**纯投影** | `cache_key::key_hex`(已存在) | + +``` +-----c++ + +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 +aarch64-macos-none-llvm22-libcxx22-c++23 +x86_64-windows-msvc-msvc194-msvcstl194-c++23 +``` + +⚠️ **`arch-os-env` 必须用 `triple.cppm` 的规范拼写,不是 fingerprint 的 [4]** —— +[4] 是编译器自报的(`x86_64-w64-mingw32`),与 `[target.'…']` 键的拼写 +(`x86_64-windows-gnu`)不同,直接用会让同一个决定有两个拼写。 + +**纯 `extern "C"` 库发一个更短的 tag** —— `x86_64-linux-gnu`,没有 +compiler/stdlib/std 段。闸门按段比对 tag 里**有**的东西,而 `abi_check` +(`abi.cppm:157`)早就是「未指定的维度 = 不关心」。**tag 的形状就是 surface**, +不需要额外的开关;C 库的 tag 组合数因此天然从 N×M 掉回 N。 + +### 1.3 三层分发,层级由消费端选 + +| Tier | 包里有什么 | 生效条件 | 失配时 | +|---|---|---|---| +| **S** source | 全部源码 | 永远 | —— | +| **I** interface+binary | 接口 + 预编译库 | `abi_tag` 匹配 | 降级到 S;无 S 则**明确拒绝并列出可用 tag** | +| **B** +BMI | 再加预编译 BMI | `build_key` 精确匹配 | **静默**降级到 I | + +Tier B 必须先做跨机器可行性实验才能实施(GCC 把时间戳与源码路径写进 BMI), +**它只能是加速器,永不成为正确性依赖**。 + +--- + +## 2. 生产者侧 + +### 2.1 命令 + +```bash +mcpp pack # 唯一可打包目标;有歧义 → 报错并列出候选 +mcpp pack mathkit # kind = "lib" → 静态库包 +mcpp pack mathkit-shared # kind = "shared" → 动态库包 +mcpp pack myapp # kind = "bin" → 应用 bundle(既有四档 --mode) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 胖包(--target 可重复) +``` + +**新增 CLI:一个位置参数(与 `mcpp run [target]` 同形)+ `--target` 可重复。** +**新增 manifest 键:0 个。** + +| `[targets.].kind` | 产出 | `--mode` 适用 | +|---|---|---| +| `bin` | 应用 bundle | ✅ 既有四档 | +| `lib` | 静态库包 | ✅ **`system`(默认)/ `vendored`** —— 见下 | +| `shared` | 动态库包 | ✅ `system` / `vendored` / `self-contained` | + +#### 2.1.1 ⚠️ 依赖跟不跟着走,静态库同样要选(修正) + +本文前一版写「静态库的 `--mode` 不适用,归档不携带任何东西」。**那句是错的。** +「归档不携带依赖的代码」恰恰**是**问题所在:消费者必须自己把 `zlib` 拉进来 —— +除非包把它一起带上。这就是 `--mode` 问的那件事,只是机制随形态不同: + +| `--mode` | 库包里有什么 | 消费者要什么 | 适合 | +|---|---|---|---| +| **`system`**(默认) | 只有本库 + `[dependencies]` 声明 | 自己解析依赖(公开索引 / 私有索引) | 依赖都是公开包 | +| **`vendored`** | 本库 + **依赖的产物一起进 `lib//`**,包内不声明 `[dependencies]` | 什么都不要 | 内网 / 离线 / 依赖本身也是闭源 | +| `self-contained` | 仅 `shared`:再带上 libc 一侧的闭包 | 什么都不要 | 跨发行版 | + +**与应用的 `--mode` 是同一个问题的同一批答案**:闭包跟不跟着走。机制不同 +(应用问的是运行期 `.so`,静态库包问的是构建期可链接的产物),但问题相同, +所以复用既有词汇而不是发明第二套。 + +`vendored` 形态下,依赖产物在 `[[runtime.artifacts]]` 里各占一条, +`provenance` 记它**来自哪个包**(`"vendored compat.zlib@1.3.2 by mcpp-pack 2026.8.17.1"`), +生成的 `ldflags` 按拓扑序把它们排在本库之后。 + +**分期**:`system` 是 P0(当前实现自然就是它),`vendored` 是 P1。 + +### 2.2 生产者工程示例(全部是既有的键) + +```toml +# examples/05-lib-dist/mcpp.toml +[package] +name = "mathkit" +version = "0.1.0" +description = "Demo: shipping a prebuilt library with both header and module interfaces" +license = "Apache-2.0" +platforms = ["linux", "windows"] # 既有键。pack 用它做覆盖校验(缺腿告警) + +[build] +sources = ["src/*.cppm", "src/*.cpp", "src/*.c"] +include_dirs = ["include"] # 公开头。整发,不可裁 + +# [lib] path 不写 ⇒ src/mathkit.cppm 约定 ⇒ 模块闭包从这里出发 + +[targets.mathkit] +kind = "lib" # → mcpp pack mathkit 产出静态库包 + +[targets.mathkit-shared] +kind = "shared" # → mcpp pack mathkit-shared 产出动态库包 +soname = "libmathkit.so.1" # 给出正确的运行期名 + +[pack] +include = ["share/**"] # 既有键 —— 只作用于 extras +``` + +``` +examples/05-lib-dist/ +├── mcpp.toml +├── README.md +├── include/mathkit_c.h # extern "C" 头接口 +└── src/ + ├── mathkit.cppm # export module mathkit; export import :api; ← 闭包根 + ├── api.cppm # export module mathkit:api; ← 闭包内 + ├── secret.cppm # module mathkit:secret; ← 实现分区,**不发布** + ├── impl.cpp # module mathkit; import :secret; + └── capi.c # C API 实现 +``` + +### 2.3 `mcpp pack ` 做什么 + +``` +1. build 出产物 ⚠️ 用这次构建的 target///bin/… 路径, + 绝不 glob(会挑到陈旧的 fingerprint 目录) +2. 算接口闭包(从 lib-root 出发) ⚠️ 走 P1689,不走 M1 文本扫描器(§9-D) + → 拷进 interface/ + → 打印「将发布 / 不发布」两张清单 + → 闭包里出现实现分区 ⇒ 告警「它的源码会被发布」 +3. 拷 include_dirs 全量 → include/ +4. 从归档剔除**已发布闭包那些单元**的对象 ⚠️ 不是「所有 .m.o」(J3 同一个闭包的第二个用途) +5. artifact_kind = shared 时额外: + · RUNPATH / INTERP 重写($ORIGIN 化) ← J2 + · 第三方闭包收集(ELF: LD_TRACE;PE: 导入表) + · PE: 导入库进 lib/,DLL 进 bin/ +6. 生成包内 mcpp.toml(§2.4)+ HOST-REQUIREMENTS +7. 确定性归档(PE → .zip,其余 → .tar.gz) +``` + +**计划阶段必须校验的组合**(不能留到链接期;`--mode` 与形态的组合见 §2.1.1): + +| target | `kind = "lib"` | `kind = "shared"` | +|---|---|---| +| `x86_64-linux-gnu` | ✅ | ✅ | +| `x86_64-linux-musl` | ✅ | ❌ **musl 蕴含 `-static`,产不出共享库** | +| `x86_64-windows-gnu` / `-msvc` | ✅ | ❌ P1 解锁(PE 导入库) | +| `*-macos` | ✅ | ❌ P1 解锁(install_name) | + +### 2.4 包内 `mcpp.toml` —— 一个普通的 mcpp.toml,没有新段 + +> **本节是第三次重写。** 前两版分别提了 `MCPP-PACKAGE.toml` 和 `[distribution]` 段。 +> 逐条追问「这件事别处已经能说了吗」之后,**两者都被删掉**:每一条事实都有既有字段。 + +#### 2.4.1 每条事实的归属 + +| 曾经想新增的 | 归属(全部既有) | +|---|---| +| `artifact_kind`(static/shared) | `[[runtime.artifacts]].role` = `static-library` / `shared-library` | +| `abi_tags`(包级列表) | `[[runtime.artifacts]].abi` —— **每条腿一个,比包级列表更准** | +| `abi_surface`(c/cxx) | **tag 自身的形状**(见 2.4.2) | +| `modules` | `[modules] exports`(既有的完备性断言) | +| `cxx_runtime` | `[build] cxx_runtime`(既有) | +| `interface_digest` | `[[runtime.artifacts]]` 里 `role = "interface"` 的条目,**逐文件一条** | +| `built_by` | `provenance = "mcpp-pack "` | +| `build_key` | `host_fingerprint`(docs/05 §2.11 原文:*optional evidence*) | +| 「这是分发包」的标记 | `provenance` 以 `mcpp-pack` 开头 —— **可推导** | +| `schema` | 不需要:没有新 schema 要版本化 | + +**实测**(mcpp 2026.8.15.3):`role = "interface"`、任意 `abi` 串、`digest`、 +`provenance` 全部原样进 `resolution.json`,并挂上既有的 `identity` 判定 +(路径不存在 ⇒ `missing`)。**兼容性比新段更好** —— 新段会被老客户端静默跳过 +(等于没有记录),而 `[[runtime.artifacts]]` 是老客户端**已经在读**的段。 + +#### 2.4.2 `abi_surface` 消失了:tag 的**形状**就是 surface + +`abi.cppm:157` 的 `abi_check` 早就是「**未指定的维度 = 不关心**」。所以不需要一个 +布尔来说「我只约束 libc」—— **发一个短 tag 就是在说这件事**: + +``` +纯 extern "C" 库 abi = "x86_64-linux-gnu" ← 只有三段 +C++ 模块库 abi = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" ← 全段 +``` + +闸门按**段**比对 tag 里有的东西。C 库的 tag 组合数因此天然从 N×M 掉回 N, +不需要任何开关。 + +#### 2.4.3 生成的包内 `mcpp.toml` —— 描述只有三件事 + +**一个二进制包的描述应当和源码包一样、甚至更简单。核心就三样:接口 / 库 / 依赖。** +其余的都不是「描述」,是**证据**,而证据挂在既有的 `[[runtime.artifacts]]` 上, +每条腿一条 + 接口一条,不是每个文件一条。 + +```toml +# ══ 由 `mcpp pack mathkit` 生成。手工编辑会使 interface 的 digest 失配并被拒绝。 ══ +[package] +namespace = "acme" +name = "mathkit" +version = "0.1.0" + +# ── ① 接口 ──────────────────────────────────────────────────────── +[build] +sources = ["interface/mathkit.cppm", "interface/api.cppm"] # 模块接口:消费者编译它 +include_dirs = ["include"] # 头接口:消费者 #include +cxx_runtime = "self-contained" + +[modules] +exports = ["mathkit"] + +[targets.mathkit] +kind = "lib" + +# ── ② 库(每条腿一段;必须是 cfg(...),不能是裸三元组 —— J4)────────── +[target.'cfg(all(linux, not(env = "musl")))'.build] +ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + +[target.'cfg(all(linux, env = "musl"))'.build] +ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] + +[target.'cfg(windows)'.build] +ldflags = ["-Llib/x86_64-windows-gnu", "-lmathkit"] + +# ── ③ 依赖(从生产者工程原样带过来)──────────────────────────────── +[dependencies.compat] +zlib = "1.3.2" + +# ── 证据:每条腿一条 + 接口一条 ──────────────────────────────────── +[[runtime.artifacts]] +role = "static-library" +path = "lib/x86_64-linux-gnu/libmathkit.a" +provenance = "mcpp-pack 2026.8.17.1" # 前缀即「这是分发包」的标记 +abi = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +digest = "sha256:…" +host_fingerprint = "aeb4c4d29e437696" # build_key,Tier B 用 + +[[runtime.artifacts]] +role = "interface" +path = "interface" # 目录,一条即可 —— 不是每个文件一条 +provenance = "mcpp-pack 2026.8.17.1" +digest = "sha256:…" # 对有序文件集的摘要 +``` + +**⚠️ ③ 依赖是必须的,而且容易漏。** 一个静态库的 `.a` **不携带**它的第三方依赖 —— +消费者链接时必须自己把 `zlib` 拉进来。所以 `mcpp pack` 要把生产者的 +`[dependencies]` 原样写进包(**排除 dev-dependencies 与 path 依赖** —— +后者是本地的,发出去解析不了)。这条规则 `emit_xpkg` 已经在用 +(`publisher.cppm:186-192`),**同一份推导,第二个投影**。 + +`shared` 形态另外多一段: + +```toml +[runtime] +runtime_search_dirs = ["lib/x86_64-linux-gnu"] # 进消费者的 RPATH +# deploy_files = ["bin/x86_64-windows-gnu/mathkit.dll"] # PE:部署到 .exe 旁 +``` + +#### 2.4.4 为什么接口 digest 只有一条 + +**它防的是「解开之后有人改了随包的接口」**,不是「生产者一开始就发错了配对」—— +后者只有原子产出能防(§0 J1)。索引通路上,整包已经有 sha256;真正裸奔的是 +**path 依赖 / 已解开的 store**,而那里一条目录级 digest 就够: +诊断说「`interface/` 与打包时不一致,拒绝构建」已经是可行动的。 +逐文件 digest 能多说一句「是哪个文件」,代价是描述里多出 N 行 —— 不值得。 + +### 2.5 分发包目录里不许直接 build + +判据:**任一 `[[runtime.artifacts]]` 的 `provenance` 以 `mcpp-pack` 开头**。 +此时在解开的包目录里直接 `mcpp build` **必须拒绝**,并说清这是分发包不是源码树。 +今天它会**成功** —— 把 `interface/` 里只有声明的接口单元编出来,产出一个几乎空的库, +实现全在预编译产物里没被链进来。典型的「看起来成功的失败」。 + +--- + +## 3. 消费者侧 + +### 3.1 三种依赖形态,同一条下游路径 + +```toml +# ① 离线文件 —— P0。不需要索引、网络、鉴权,不需要动 xlings +mathkit = { package = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.tar.gz" } + +# ② 索引(公开或私有)—— P1。描述符与源码包同构:只有 url + sha256 +mathkit = "0.1.0" + +# ③ 本地目录(内部团队最常走的一条) +mathkit = { path = "../mathkit-dist" } +``` + +```bash +mcpp add ./mathkit-0.1.0-.tar.gz +``` + +**⚠️ 闸门必须对 ③ 也生效** —— 这就是闸门字段要放进包内 `mcpp.toml` 而不是旁路文件的理由。 + +### 3.2 闸门表(顺序即诊断顺序) + +| 检查 | 失配 | +|---|---| +| arch / os / env | **拒绝** —— 载荷本身不对 | +| `abi_surface == "c"` | 上一行之后全部跳过 | +| compiler 族 / 主版本 | **拒绝**;`--allow-abi-drift` 强制并打印它保护的是什么 | +| stdlib id / 主版本 | **拒绝** | +| C++ 标准档位 | 消费者 < 生产者 → **拒绝**;> → 放行 | +| `cxx_runtime` 契约 | **警告** + 说清双运行时危险;`--strict` 升级为错误 | +| `interface_digest` | **拒绝** | +| 模块符号存在性(ELF `_ZGIW` / PE 导出表) | **拒绝** | +| `build_key`(Tier B) | **静默**降级到 Tier I | + +### 3.3 没有匹配 tag 时的诊断 + +``` +error: acme.mathkit@0.1.0 has no prebuilt artifact for this toolchain + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + aarch64-linux-gnu-gcc15-libstdcxx15-c++23 + note: this package ships no source tier, so there is nothing to fall back to. + fix : ask the publisher for a gcc16 build, or pin [toolchain] to gcc@15. +``` + +**「不可用」不能被拼成「不存在」** —— 被拼成「找不到包」的失败会让客户端自己驱动 +重复刷新索引(#349 的教训)。 + +### 3.4 兼容性:老客户端拿到这个包会怎样 + +**能构建,只是没有闸门。** 实测(mcpp 2026.8.15.3): +包内 mcpp.toml 用的 `sources` / `[build]` / `[modules] exports` / +`[target.'cfg(…)']` / `[[runtime.artifacts]]` **全部是已发布能力** —— 老客户端 +逐字读得懂,只是不会**执行闸门**(它不知道 `provenance = "mcpp-pack"` 意味着要校验 +digest 与 abi tag)。这是**降级**而不是变砖 —— +但**闸门只保护新客户端,必须写进发布说明**。 + +这也是不新增段的第二个好处:一个**新**段会被老客户端静默跳过,连记录都没有; +而 `[[runtime.artifacts]]` 是老客户端**已经在读、并且会写进 `resolution.json`** 的段, +所以即使闸门不执行,证据仍然落盘、仍然可审计。 + +--- + +## 4. `docs/` 文档计划 + +| 文件 | 动作 | 内容 | +|---|---|---| +| `docs/02-pack-and-release.md` | **改** | 标题改为「打包:应用与库」。新增「§库分发」:`mcpp pack ` 的 kind 表、两种接口模式的目录规则、包布局、跨平台合法组合表。既有的应用四档 `--mode` 一字不改 | +| `docs/12-binary-distribution.md` | **新增** | 库作者的完整链路:写工程 → `mcpp pack` → 检查两张清单 → 分发(文件/私有索引/公开索引)→ 消费者怎么用。含 §2.2 与 §3.1 的完整示例 | +| `docs/05-mcpp-toml.md` | **改** | ① `[pack]` 一节补「只作用于 extras,裁不到头与接口」;② §2.11 补 `role = "interface"` 与 `provenance = "mcpp-pack"` 的约定用法(生成物,不是手写配置);③ Appendix A 补一条准入判据:「能从别处推出来的不给字段」 | +| `docs/10-publishing-a-library.md` | **改** | 新增「发布二进制包」小节:与源码包**同构**的索引条目、`platforms` 覆盖校验、老客户端降级说明 | +| `docs/03-toolchains.md` | **改** | `abi_tag` 的六个成分与 `--print-fingerprint` 的对应关系 | +| `docs/zh/*` | **同步** | 上述五处的中文版 | + +--- + +## 5. `examples/` 具体示例 + +沿用既有编号与「每目录一个 README.md」的约定。 + +### `examples/05-lib-dist/` —— 生产者(库作者) + +结构见 §2.2。README 要点: + +- 一个工程**同时**提供头接口与模块接口,消费者可以只用其中一种; +- `secret.cppm` 是实现分区,**不会被发布** —— 跑 `mcpp pack mathkit` 看两张清单; +- 两个目标(`lib` + `shared`)如何同时发布,以及 `soname` 给出的正确运行期名; +- **反面演示**:把 `src/mathkit.cppm` 改成 `import :secret;`,再 pack, + 观察闭包里多出 `secret.cppm` 并触发告警。 + +### `examples/06-lib-consume/` —— 消费者 + +``` +examples/06-lib-consume/ +├── mcpp.toml # mathkit = { path = "../05-lib-dist/dist" } +├── README.md +└── src/ + ├── main_header.cpp # 只 #include + ├── main_module.cpp # 只 import mathkit; + └── main_both.cpp # 两者同时 +``` + +README 要点:三种消费方式 × 静态/动态两种包 = 六格,全部可跑; +以及**篡改 `interface/` 后必须被拒绝**的演示。 + +### `examples/07-lib-dist-fat/` —— 胖包与交叉 + +一个 `mcpp.toml` + 一条命令产出三条腿,消费者用 `--target` 选。README 要点: + +- 为什么每条腿的 `ldflags` 是 `cfg(...)` 而不是裸三元组(J4,附最小探针); +- `lib/` 为什么按三元组分目录而不按 OS 分。 + +--- + +## 6. CI:验证矩阵就是 e2e 集合本身 + +> **相对初稿的修订。** 初稿提议新建 `pack-dist-matrix.yml`。**不需要。** +> `tests/e2e/run_all.sh` 已经有能力探测(`elf` / `gcc` / `mingw-cross` / +> `fresh-sandbox` / …)并按 `# requires:` 分流,而 `ci-linux-e2e.yml` 已经在跑 +> 整个 `tests/e2e/`。再建一条并行流水线就是**同一个决策的第二处推导**。 +> +> 矩阵是 e2e 集合 + 每个测试头部那行 `# requires:`。 + +### 6.1 落地的测试与它们钉住的判据 + +| e2e | `# requires:` | 钉住 | +|---|---|---| +| **242** `pack_library_interface_and_headers` | `gcc` | 两种接口模式共存;包布局;两张清单都被打印 | +| **243** `pack_library_interface_closure` | `gcc` | **J3** —— 实现分区源码不外发 **且** 它的对象留在归档里 | +| **244** `pack_library_gate` | `gcc` | **J1** 接口篡改被拒 + tag 失配被拒(且列出可用 tag)+ 包内 build 被拒 | +| **245** `pack_library_fat_target_selection` | `gcc` | **J4** —— 胖包每条腿只被自己的 target 看到,**含原生构建** | +| **246** `explicit_empty_sources` | `gcc` | §8-B2 —— `sources = []` 与「不写」可区分 | +| **247** `bare_triple_conditional_native` | `gcc` | §8-B1 —— 裸三元组谓词在原生构建下命中 | +| **248** `pack_library_fat_pe_leg` | `gcc mingw-cross` | 跨 OS 边界的腿(PE);tag 用规范三元组而非编译器自报 | + +### 6.2 ⚠️ 一处差点造出来的假绿 + +245 最初写成 `# requires: gcc mingw-cross`(gnu + musl + windows 三条腿)。 +**`ci-linux-e2e.yml` 只预热 gcc 与 musl,不装 mingw-cross** —— +那条测试会在每一次普通 CI 上**静默跳过**,于是胖包这一整套机制 +(以及 J4)在绿色的套件里**从未被验证过**。 + +拆法:**核心机制用 CI 一定有的工具链**(245:gnu + musl,`requires: gcc`), +**只把新增的二进制格式覆盖单列**(248:PE,`requires: mingw-cross`)。 +判据:**一个测试的 `# requires:` 必须是它所验证机制的真实下限,不是它能跑的上限。** + +### 6.3 单元测试 + +| 文件 | 覆盖 | +|---|---| +| `tests/unit/test_pack_abi_tag.cpp` | tag 的投影 / 规范三元组 / C 表面短 tag / 从尾解析 / 档位下限 / 一次报全部失配(15 例) | +| `tests/unit/test_pack_interface.cpp` | 闭包 / 剔除集不是「所有 `.m.o`」/ 依赖模块不越界 / 未解析分区报错(8 例) | + +### 6.4 仍需在 CI 上补的(P1) + +- macOS 与 Windows 的 e2e 分片会自动跑 242/243/244/246/247(它们只 `requires: gcc`), + 但**尚未在这两个平台上人工确认过**; +- `kind = "shared"` 的库包 e2e(P1,随 PE 导入库 / Mach-O install_name 一起); +- `--target` 覆盖与 `[package].platforms` 的比对告警(P1)。 + +## 7. 分期 + +### P0 —— `static` 形态,三平台一次做完(不依赖 xlings、不依赖 shared) + +1. `sources = []` 的三态(§8-B2)+ §2.5 的 build 守卫(判据 = `provenance` 前缀) +2. 包布局 + 生成包内 `mcpp.toml` +3. `mcpp pack ` 位置参数 + `--target` 可重复;`kind = "lib"` ⇒ 静态库包 +4. 接口闭包(P1689)+ 两张清单 + 实现分区告警 +5. 从归档剔除已发布闭包的对象 +6. `abi_tag` 计算 + 闸门表 + `interface_digest` + 模块符号存在性 +7. `mcpp add ./x.tar.gz` + `{ package = "…" }` 依赖形态 +8. e2e 242 / 244 / 245 / 246 / 248 / 250 / 251;examples 05 + 06 + +**为什么 static 能在 P0 覆盖三平台**:`kind = "lib"` 无平台限制, +所以 G2、RUNPATH 重写、闭包收集、DLL 部署这一期全部不需要。 + +### P1 —— `shared` 形态解锁三平台 + 索引通路 + +9. PE 导入库(`--out-implib` / `/IMPLIB:`)+ Mach-O `-install_name @rpath/…` +10. `kind = "shared"` 守卫的判据改成「该 target 是否动态链接」(musl 走计划期拒绝) +11. `shared` 形态的 pack:RUNPATH 重写 + 闭包收集 + `bin/` 部署面 +12. 胖包上索引(描述符与源码包同构)+ `platforms` 覆盖校验 +13. e2e 243 / 247 / 249;examples 07;`pack-dist-matrix.yml` + +### P2 —— 交叉与私有鉴权 + +14. 瘦包(按 tag 分资产)+ `install_packages` 加 target 轴(跨仓库,与 xlings 同步) +15. `IndexSpec` 鉴权(值从环境变量读,永不落盘) +16. `target_cfg` / `[target.'cfg(…)']` 承载 LinkIntent(顺手修 #258 同形状的债) + +### P3 —— Tier B 与生态收尾 + +17. Tier B:**先做跨机器可行性实验**,做不到就不做 +18. `.pc` / CMake config 产出 +19. #304(`library_dirs` 的 link/runtime 分离收口)、#290(按版本区分构建规则) + +--- + +## 8. 需要先单独修的既有缺陷(不属于本方案,但阻塞它) + +| # | 缺陷 | 位置 | 对本方案的影响 | +|---|---|---|---| +| **B1** | `[target.'<三元组>'.build]` 在原生构建下**永不命中** | `prepare_inputs.cppm:139` `if (triple.empty()) return false;`,而同文件 `context_for()` 对 `cfg()` 回落到 `host_triple()`;`types.cppm:665` 的注释承诺的是回落 | 直接阻塞胖包的裸三元组写法(J4)。**任何用该写法的工程都中招**,形状是「CI 全绿、本机静默失配」 | +| **B2** | `sources = []` 与整行删掉逐字节等价 | `toml.cppm:1703` 的默认 glob 吞掉显式空 | 二进制包无法表达「什么都不要编」;遗留在 `src/` 的文件会被编进消费者的构建 | +| **B3** | M1 扫描器不把实现分区建模为 provider | `scanner.cppm:641-650` 对非 `export` 的 `module X;` 从不设 `u.provides` | 「接口够到实现分区」的告警产不出来 ⇒ 闭包必须走 P1689 | +| **B4** | `kind = "shared"` 的守卫判据是 `os != "linux"` | `plan.cppm:1002` | musl 过闸后死在 `crtbeginT.o`,消息里既没有 musl 也没有 shared | + +B1 / B2 建议**先单独开 issue 并各带一条回归测试**,不要埋进这个大特性里。 + +--- + +## 9. 四处被自己的实测推翻的设计(不要重新提出来) + +| # | 我写过的 | 被什么推翻 | 正确的 | +|---|---|---|---| +| **A** | 新增描述符键 `kind = "prebuilt"` + 版本 floor | 它属于「不降级」的那一类,老客户端会被砖 | 走 **Form A**(包自带 `mcpp.toml`)⇒ 无新描述符键、无 floor、兼容几乎免费 | +| **B** | 打包时「剔除所有 `.m.o`」 | **三个 target 全部链接失败**(`undefined reference to mk::secret_helper@mathkit()`)—— 实现分区照样是 `.m.o` 且里面是真代码 | 剔除集 = **已发布闭包里那些单元**的对象;与发布集是**同一个闭包的两个用途** | +| **C** | 胖包每条腿用裸三元组 `[target.'x86_64-linux-gnu'.build]` | 显式 `--target` 三个全绿,**裸 `mcpp build` 链接失败**;最小探针:裸三元组 0 命中 / `cfg(linux)` 2 命中 | 一律用 `cfg(...)`(J4);并把根因(B1)单独开 issue | +| **D** | `[pack]` 加五组键(`default_kind` / `default_artifact` / `targets` / `[pack.interface]` / `[pack.headers]`) | 逐条问「别处已经说过了吗」之后全部落空 | **新增 0 个键**:kind → `[targets.].kind`;接口根 → `[lib]`;头目录 → `[build].include_dirs`;平台 → `[package].platforms` 升格为断言 | + +外加一处我自己踩的坑:原型用 `rglob` 找产物,**挑到了陈旧的 fingerprint 目录** +(少一个 `capi.o`,症状是消费者 undefined reference,看起来完全像 mcpp 的问题)。 +**打包器绝不能 glob 产物** —— 这与仓库既有的「`ls | head -1` 会自查到旧二进制」是同一形状。 + +--- + +## 10. 证据索引 + +全部实测记录在 `2026-08-17-distribution-architecture-analysis-and-design.md`: + +| 判据 | 那份文档的位置 | +|---|---| +| J1(接口↔二进制无绑定,静默错数据) | §2.2(c) | +| J2(`.so` 烧生产机器 RUNPATH) | §2.2(a) | +| J3(闭包 vs `.m.o`;剔除集) | §2.4.2 / §2.4.3 | +| J4(裸三元组不匹配 + 最小探针) | §2.4.5 | +| 六格矩阵(两种接口 × 两种形态) | §2.5.1 | +| 两条目录规则 | §2.5.2 | +| musl + shared 链接期炸 | §2.5.3 | +| `sources = []` 不生效 | §2.5.4 | +| 新段的兼容边界 + array-of-tables 白名单(**为什么最终不新增段**) | §4.2 | +| `[[runtime.artifacts]]` 能承载 role/abi/digest/provenance + identity 判定 | 本文 §2.4.1 实测 | +| `[pack]` 推导审计 | §4.6.1 | +| 复现脚本(`scratchpad/lab/`) | 附录 A | +| file:line 索引 | 附录 B | diff --git a/CHANGELOG.md b/CHANGELOG.md index 192514a5..17e90e94 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,8 +5,74 @@ ## [Unreleased] +### 新增 + +- **`mcpp pack ` 可以把一个库打成「接口 + 预编译二进制」的包(#433)。** + + 闭源库、离线环境、以及「构建农场已经编过一遍了」这三种场景,过去都只能自己 + 写脚本收集产物。现在: + + ```bash + mcpp pack mathkit # 静态库包 + mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 一个包,两条腿 + ``` + + **产出的是一个普通的 mcpp 包** —— 一份正常的 `mcpp.toml`,走 mcpp 早就有的 + 「载荷自带 manifest」通路。**新增 manifest 段 0 个、键 0 个**:打什么由 + `[targets.].kind` 决定(所以没有 `--lib`、没有 `--artifact`),发布哪些接口 + 由 `[lib]` 约定 + 模块图决定,公开头是 `[build].include_dirs` 全量, + 每条腿的 ABI tag 与 digest 记在既有的 `[[runtime.artifacts]]` 上。 + 一个**老版本 mcpp 照样能构建**这种包 —— 它只是不执行下面那两道闸门。 + + 一个包可以同时带**两种接口**:`include/`(文本,`#include`,不编译)与 + `interface/`(模块,消费者编译它)。实测同一个包被「只 #include」/「只 import」/ + 「两者都用」三种方式消费,静态与动态两种形态,六格全过。 + + 发布哪些 `.cppm` 是**算出来的** —— lib root 的模块闭包,不是「所有 `.m.o`」。 + 实现分区(`module M:secret;`)照样产 `.m.o`,按扩展名挑会**泄露闭源源码**; + 同一个闭包反过来决定归档里要删哪些对象,按 `.m.o` 删则会删掉真代码、 + 三个平台全部链接失败。两条清单都会打印出来。 + + 详见 `docs/12-binary-distribution.md`、`examples/05-lib-dist`、`examples/06-lib-consume`。 + +- **消费预编译包时的两道闸门。** 都是不检查就会静默出错的: + + **接口与二进制是否仍然配对。** 这条闸门存在是因为另一种结果被实测过:把随包 + 接口里一个结构体的两个 `int` 成员互换 —— Itanium ABI 不 mangle 字段顺序 —— + 消费者**编译过、链接过、运行过、打印出交换后的错数据**,任何工具都没有一句诊断。 + + **二进制是否为这套工具链所编。** 失配时诊断会**列出包里确实有哪些 tag** —— + 一句「找不到」会让人去找一个就在自己硬盘上的包。 + + 另外,在解开的分发包目录里直接 `mcpp build` 会被拒绝:那儿的 `interface/` + 是声明,定义在旁边的归档里,构建会产出一个几乎空的库然后报告成功。 + ### 修复 +- **`[target.'<三元组>'.build]` 在没有 `--target` 时从不命中。** + + 同一个语句的两种拼写互相矛盾:`cfg(linux)` 在原生构建上命中, + `[target.'x86_64-linux-gnu'.build]` 不命中。根因是 `matches()` 拿着原始的 + `--target` 字符串(原生构建下是空的)短路返回 false,而同一文件的 + `context_for()` 对 `cfg(...)` **回落到宿主三元组** —— 一个决定两处推导。 + `manifest/types.cppm` 的注释从写下起承诺的就是回落那一种。 + + **形状是最坏的那种**:CI 传 `--target` 是绿的,开发者本机的 `mcpp build` + 静默丢掉那一段,失败在链接期出现、点的是符号而不是谓词。 + + 修法是**删掉第二个答题者**:解析后的三元组进 `cfgpred::Ctx`,`matches()` + 只有一个来源。 + +- **`sources = []` 与不写 `sources` 逐字节等价。** + + 解析器在向量为空时一律填默认 glob,于是作者**没有任何写法**能表达 + 「什么都不要编」。二进制分发需要这个:一个纯头文件的包不编译任何东西, + 而 `src/` 下任何遗留文件都会被扫进消费者的构建,并可能与预编译库里的符号 + 重复定义。改成记录**键是否出现**(`BuildConfig::sourcesDeclared`), + 与 `XlingsConfig::subosDeclared` 同一个模式。 + + - **卸载后的清扫会波及**别的版本**,而那可能正在被另一个进程解压。** `sweep_parked_payloads` 原来把整个 family 目录扫一遍,把**任何**没有文件的 diff --git a/docs/02-pack-and-release.md b/docs/02-pack-and-release.md index 195d7b27..a9100649 100644 --- a/docs/02-pack-and-release.md +++ b/docs/02-pack-and-release.md @@ -1,4 +1,12 @@ -# 02 — Packaging for Release +# 02 — Packaging an Application for Release + +> This page is about bundling a **program**. To ship a *library* as interface + +> prebuilt binaries, see [12 - Distributing a Prebuilt Library](12-binary-distribution.md). +> +> Which one `mcpp pack` does is decided by the target's `kind`, not by a flag: +> `mcpp pack ` packs `[targets.]`, and a `bin` becomes a bundle +> while a `lib`/`shared` becomes a library package. With no name, mcpp picks +> the only packable target. > A default dynamically linked binary produced by `mcpp build` has a loader and > RUNPATH tied to the build sandbox. It is a development artifact, not a diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index a5e663eb..e54b3f70 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -153,6 +153,12 @@ the package/feature boundary, not on an individual target. ### 2.3 `[build]` — Build Configuration +> **`sources = []` is not the same as omitting `sources`.** An absent key +> selects the default glob; an explicitly empty list means *compile nothing*, +> which is what a header-only distribution package needs to say. Until +> mcpp 2026.8.17.2 the two were byte-identical, so there was no spelling for +> "nothing" and any file left under `src/` was swept in. + ```toml [build] sources = ["src/**/*.cppm", "src/**/*.cpp"] # Source globs (default: src/**/*.{cppm,cpp,cc,c,S,s,asm}) @@ -1598,6 +1604,17 @@ do`. - Package-level knobs all converge into features; for sugar keys (such as `backend=`) to enter the core syntax, they must satisfy: ① domain-neutral (a cross-ecosystem general pattern) ② 1:1 desugaring with zero new parsing semantics. +- **A key that duplicates an answer another section already gives is not admitted.** + Two places to state one fact is two places that can disagree, and the failure + is silent — whichever reader loses the race is simply wrong. Library packaging + ([12](12-binary-distribution.md)) is the worked example: it added **zero** + manifest keys, because what to pack is `[targets.].kind`, which interface + to publish is `[lib]` plus the module graph, which headers are public is + `[build].include_dirs`, and the per-artifact evidence is `[[runtime.artifacts]]`. +- A field that describes what a *generated* package IS (rather than what a build + should DO) belongs on `[[runtime.artifacts]]` — see §2.11. `provenance` + beginning with `mcpp-pack` is what marks a directory as one, and mcpp refuses + to `build` inside it. - See `.agents/docs/2026-06-04-manifest-schema-ownership.md` for the full field-ownership table and the finalized decisions. diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md new file mode 100644 index 00000000..f42c45cb --- /dev/null +++ b/docs/12-binary-distribution.md @@ -0,0 +1,263 @@ +# 12 - Distributing a Prebuilt Library + +**English** | [简体中文](zh/12-binary-distribution.md) + +> Ship a library as **interface + prebuilt binaries** instead of as source. +> This is the closed-source case, and the offline case, and the "our build farm +> already compiled this once" case. +> +> [02 - Packaging & Release](02-pack-and-release.md) is the sibling: bundling an +> *application*. [10 - Publishing a Library](10-publishing-a-library.md) is the +> source route. + +## The whole idea in one paragraph + +`mcpp pack ` builds a library target and writes a directory that is an +**ordinary mcpp package** — a normal `mcpp.toml`, the interface a consumer must +compile, and the binaries it then links. Consumers use it exactly like any +other dependency. There is no new manifest section, no new archive format, and +no new resolution path. + +```bash +mcpp pack mathkit # a static library package +mcpp pack mathkit-shared # a dynamic one (Linux/ELF today) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # one package, two legs +``` + +## What decides what gets packed + +`[targets.].kind`, and nothing else: + +| `kind` | `mcpp pack ` produces | `--mode` | +|---|---|---| +| `bin` | an application bundle (see [02](02-pack-and-release.md)) | the four depths | +| `lib` | a **static library package** | — | +| `shared` | a **dynamic library package** | — | + +There is no `--lib` flag and no `--artifact static\|shared`. `kind` is already +where mcpp records what an artifact is; a flag would be a second place to say +it, and two places can disagree. A project that publishes both forms declares +both targets — which it must do for `mcpp build` to produce both anyway: + +```toml +[targets.mathkit] +kind = "lib" + +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" +``` + +Run `mcpp pack` with no name and mcpp picks the only packable target, or tells +you which ones it found. + +## The two interface modes + +A package can carry both, and a consumer may use either or both. + +``` +mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ +├── mcpp.toml +├── include/ ← TEXT interface: #include, never compiled +├── interface/ ← MODULE interface: the consumer compiles it +└── lib// ← the artifacts +``` + +| | `include/` | `interface/` | +|---|---|---| +| whose input is it | the preprocessor's | the **compiler's** | +| does the consumer compile it | no | **yes**, to get a BMI | +| what it constrains | the libc ABI | compiler, C++ stdlib, C++ level | +| can you trim it | **no** — see below | **no**, it is computed | + +`lib/` is keyed by **triple**, not by OS. MinGW and MSVC are both Windows and +produce `libfoo.a` and `foo.lib` respectively. + +### Why neither set can be trimmed + +A **source** distribution of the same package puts every one of its +`include_dirs` on its consumers' include path. If a binary package shipped a +subset, the same library would have a different public surface depending on how +it was delivered. "Which headers are public" is already answered by the layout: +`include/` is public, `src/` is not. A private header under `include/` is a +project-layout mistake, not a packaging option. + +## Which `.cppm` files travel + +The **module closure of the lib root** — `src/.cppm` by +convention, or `[lib].path`. Whatever that unit's purview imports, transitively, +is published; everything else is not. + +``` +src/mathkit.cppm export module mathkit; export import :api; → published +src/api.cppm export module mathkit:api; → published +src/secret.cppm module mathkit:secret; ← implementation partition +src/impl.cpp module mathkit; → withheld +``` + +`mcpp pack` prints both lists: + +``` + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm +``` + +**Read the second one** if you are shipping closed source. + +> **`.m.o` is not the rule.** An implementation partition produces a BMI and an +> object exactly like an interface unit does. Selecting sources by "does it +> produce a BMI" would publish `secret.cppm`. + +If the published interface *does* import an implementation partition, a +consumer cannot compile it without that source — so `mcpp pack` stops: + +``` +error: the published interface imports mathkit:secret , which no unit in this + build provides. +``` + +Restructure so the interface does not reach it, or make it an `export module` +partition and accept that its source is published. + +## The compatibility tag + +Every artifact records the toolchain it was built for: + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface +x86_64-linux-gnu # an extern "C" interface only +``` + +`--` then, when the interface is C++, ``, +``, `c++`. + +**A shorter tag is a real statement, not a missing one.** A library whose whole +interface is `extern "C"` constrains the libc ABI and not the C++ one, so it +publishes a triple and stops — and links into any compiler. Unnamed dimensions +are don't-care, so one tag per triple instead of one per triple per compiler. +Nothing to configure: the shape is the statement. + +The `c++` level is compared as a **floor**, not for equality: building at a +higher level is fine, lower is not. + +## What a consumer's build checks + +Two things, both of which fail silently without a check: + +**The interface still matches its binaries.** + +``` +error: acme.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +This exists because the alternative was measured. Swap two `int` members of a +struct in a shipped interface — the Itanium ABI does not mangle field order — +and the consumer compiles, links, runs, and prints transposed data, with no +diagnostic from any tool. A digest cannot stop a publisher from shipping a +mismatched pair (only producing both in one command does that), but it does +catch the pair coming apart afterwards. + +**The binaries were built for this toolchain.** + +``` +error: acme.mathkit@0.1.0: no prebuilt artifact matches this toolchain. + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : + x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on: + compiler needs gcc15, this build has gcc16 + stdlib needs libstdcxx15, this build has libstdcxx16 +``` + +The tags it *does* have are part of the message: "not found" would send you +looking for a package already on your disk. + +## Consuming one + +Three spellings, one code path: + +```toml +# a directory (what you hand a colleague) +mathkit = { path = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } + +# a private git repo +mathkit = { git = "ssh://git@internal/mathkit-dist.git", tag = "v0.1.0" } + +# an index entry — identical in shape to a source package's +mathkit = "0.1.0" +``` + +Nothing about the consumer's manifest says "this one is prebuilt". + +### Building *inside* a package is refused + +``` +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +Its `interface/` holds declarations whose definitions are in the archive beside +them. Building there compiles the declarations, produces a near-empty library +and reports success. + +## One package, several targets + +`--target` is repeatable. The generated manifest gets one conditional block per +leg, and the consumer's build picks its own: + +```toml +[target.'cfg(all(arch = "x86_64", os = "linux", env = "gnu"))'.build] +ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + +[target.'cfg(all(arch = "x86_64", os = "linux", env = "musl"))'.build] +ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] +``` + +Because selection happens in the **consumer's** build, where the resolved +target is known, a fat package cross-compiles correctly with no index-side or +installer-side support at all. + +> The blocks are `cfg(...)` and never a bare `[target.'']` key. Before +> mcpp 2026.8.17.2 the bare form was inert without an explicit `--target`, so a +> package using it would work in CI and silently drop its flags on a +> developer's machine. mcpp generates the spelling that means the same thing on +> every client. + +## Dependencies + +A static archive does **not** carry its dependencies' code, so the package +records them and the consumer resolves them: + +```toml +[dependencies] +"compat.zlib" = "1.3.2" +``` + +`path` and `git` dependencies are dropped: they address the publisher's disk, +and republishing one hands the consumer an address that means something else. +If your library depends on one, either publish that dependency too or vendor it +before packing. + +## What older mcpp does with these packages + +**It builds against them.** Every key in the generated manifest already +existed, so an older client reads the package and links it. What it does not do +is run the two checks above — it has no way to know that `provenance = +"mcpp-pack …"` means anything. + +That is a degradation, not a break, and it is the right direction. But it means +**the gate protects new clients only**, which belongs in your release notes if +you publish to a mixed audience. + +## Current limits + +| | status | +|---|---| +| `kind = "lib"` (static) | ✅ every target | +| `kind = "shared"` on Linux/ELF | ✅ | +| `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet | +| `kind = "shared"` on `*-musl` | ❌ a musl target links statically | +| shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact | +| bundling dependencies into the package | ❌ declare them instead (above) | diff --git a/docs/README.md b/docs/README.md index 2118e5ae..369af822 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,6 +14,7 @@ - [09 - Releasing mcpp](09-release.md) - [10 - Publishing a Library to mcpp-index](10-publishing-a-library.md) - [11 - Machine-Readable Output](11-machine-output.md) +- [12 - Distributing a Prebuilt Library](12-binary-distribution.md) ## Specifications diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md new file mode 100644 index 00000000..f2264a2f --- /dev/null +++ b/docs/zh/12-binary-distribution.md @@ -0,0 +1,243 @@ +# 12 - 分发预编译库 + +[English](../12-binary-distribution.md) | **简体中文** + +> 把一个库以**接口 + 预编译二进制**的形式分发,而不是发源码。 +> 这是闭源场景、离线场景,以及「构建农场已经编过一遍了」的场景。 +> +> 姊妹篇:[02 - 打包应用](02-pack-and-release.md) 讲的是打包**程序**; +> [10 - 发布一个库](10-publishing-a-library.md) 讲的是源码通路。 + +## 一段话讲完 + +`mcpp pack ` 构建一个库目标,产出一个**普通的 mcpp 包** —— +一份正常的 `mcpp.toml`、消费者要编译的接口、以及它随后链接的二进制。 +消费者用它和用任何依赖一样。**没有新的 manifest 段、没有新的归档格式、 +没有新的解析路径。** + +```bash +mcpp pack mathkit # 静态库包 +mcpp pack mathkit-shared # 动态库包(今天仅 Linux/ELF) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 一个包,两条腿 +``` + +## 打什么由谁决定 + +只由 `[targets.].kind` 决定: + +| `kind` | `mcpp pack ` 产出 | `--mode` | +|---|---|---| +| `bin` | 应用 bundle(见 [02](02-pack-and-release.md)) | 四档 | +| `lib` | **静态库包** | — | +| `shared` | **动态库包** | — | + +**没有 `--lib`,也没有 `--artifact static|shared`。** `kind` 本来就是 mcpp +记录「一个产物是什么」的地方;再加一个开关就是同一件事的第二个说法, +而两个说法可以互相矛盾。要同时发布两种形态,就声明两个目标 —— +这本来也是 `mcpp build` 同时产出两者所必需的: + +```toml +[targets.mathkit] +kind = "lib" + +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" +``` + +不带名字直接 `mcpp pack`,mcpp 会挑唯一可打包的目标,或者告诉你有哪些候选。 + +## 两种接口模式 + +一个包可以同时带两种,消费者用其中一种或两种都用。 + +``` +mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ +├── mcpp.toml +├── include/ ← 文本接口:#include,永不编译 +├── interface/ ← 模块接口:消费者编译它 +└── lib// ← 产物 +``` + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | 预处理器 | **编译器** | +| 消费者要编译吗 | 否 | **是**,编出 BMI | +| 约束什么 | libc ABI | 编译器、C++ 标准库、C++ 档位 | +| 能裁剪吗 | **不能**,见下 | **不能**,它是算出来的 | + +`lib/` 按**三元组**分目录,不按 OS 分:MinGW 与 MSVC 同为 Windows, +一个产 `libfoo.a` 一个产 `foo.lib`。 + +### 为什么两者都不许裁剪 + +同一个包的**源码**分发会把 `include_dirs` 里的每一个头都放到消费者的 include +路径上。二进制包若只发一部分,**同一个库就会因为分发形式不同而有不同的公开面**。 +而且「哪些头是公开的」布局已经回答了:`include/` 公开,`src/` 不公开。 +一个私有头放在 `include/` 下是工程布局的错误,不是打包选项。 + +## 哪些 `.cppm` 会被发布 + +**lib root 的模块闭包** —— 按约定是 `src/<包名尾段>.cppm`,或 `[lib].path`。 +该单元 purview 里 import 到的东西,传递地,都发布;其余都不发。 + +``` +src/mathkit.cppm export module mathkit; export import :api; → 发布 +src/api.cppm export module mathkit:api; → 发布 +src/secret.cppm module mathkit:secret; ← 实现分区 +src/impl.cpp module mathkit; → 不发布 +``` + +`mcpp pack` 会打印两张清单: + +``` + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm +``` + +**闭源分发要看第二张。** + +> **`.m.o` 不是判据。** 实现分区照样产出 BMI 和对象。按「会不会产出 BMI」 +> 来挑发布集,就会把 `secret.cppm` 发出去。 + +如果被发布的接口**确实** import 了一个实现分区,消费者没有那份源码就编不出来 —— +于是 `mcpp pack` 停下来: + +``` +error: the published interface imports mathkit:secret , which no unit in this + build provides. +``` + +要么重构让接口够不到它,要么把它改成 `export module` 分区并接受源码被发布。 + +## 兼容性 tag + +每个产物都记录它是为哪套工具链编的: + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # C++ 模块接口 +x86_64-linux-gnu # 纯 extern "C" 接口 +``` + +`--`,接口是 C++ 时再加 ``、 +``、`c++<档位>`。 + +**短 tag 是一句真实的声明,不是漏写。** 一个接口全是 `extern "C"` 的库 +只约束 libc ABI、不约束 C++ ABI,所以它发三段就停 —— 于是能链进任何编译器。 +未指定的维度就是不关心,C 库的 tag 数因此是「每个三元组一个」而不是 +「每个三元组 × 每个编译器一个」。**不需要任何开关:形状本身就是声明。** + +`c++` 档位按**下限**比对而不是相等:消费者档位更高可以,更低不行。 + +## 消费者的构建会检查什么 + +两件事,而且都是不检查就会静默出错的: + +**接口与二进制仍然配对。** + +``` +error: acme.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +这条闸门存在是因为另一种结果被实测过:把随包接口里一个结构体的两个 `int` +成员互换 —— Itanium ABI 不 mangle 字段顺序 —— 消费者**编译过、链接过、 +运行过、打印出交换后的错数据**,任何工具都没有一句诊断。 +digest 挡不住发布者一开始就发错配对(那只有原子产出能防), +但它能挡住配对在事后被拆开。 + +**二进制是为这套工具链编的。** + +``` +error: acme.mathkit@0.1.0: no prebuilt artifact matches this toolchain. + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : + x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on: + compiler needs gcc15, this build has gcc16 + stdlib needs libstdcxx15, this build has libstdcxx16 +``` + +诊断里**列出它有哪些 tag** 是刻意的:一句「找不到」会让人去找一个 +就在自己硬盘上的包。 + +## 怎么消费 + +三种写法,同一条代码路径: + +```toml +# 一个目录(你直接拷给同事的那种) +mathkit = { path = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } + +# 私有 git 仓库 +mathkit = { git = "ssh://git@internal/mathkit-dist.git", tag = "v0.1.0" } + +# 索引条目 —— 形状与源码包一字不差 +mathkit = "0.1.0" +``` + +消费者的 manifest 里没有任何一处写着「这个是预编译的」。 + +### 在包目录里直接 build 会被拒绝 + +``` +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +它的 `interface/` 里是声明,定义在旁边的归档里。在那儿构建会把声明编出来、 +产出一个几乎空的库、然后报告成功。 + +## 一个包,多个 target + +`--target` 可重复。生成的 manifest 每条腿一个条件块,消费者的构建各选各的: + +```toml +[target.'cfg(all(arch = "x86_64", os = "linux", env = "gnu"))'.build] +ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + +[target.'cfg(all(arch = "x86_64", os = "linux", env = "musl"))'.build] +ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] +``` + +因为选择发生在**消费者的构建期**(那时解析后的 target 已知), +胖包的交叉编译天然正确,**不需要索引侧或安装侧做任何支持**。 + +> 这些块是 `cfg(...)`,绝不是裸的 `[target.'<三元组>']` 键。 +> 在 mcpp 2026.8.17.2 之前,裸三元组在没有显式 `--target` 时是失效的 —— +> 用它的包会在 CI 里正常、在开发者机器上静默丢掉 flag。 +> mcpp 生成的是在**所有**客户端上含义一致的那种写法。 + +## 依赖 + +静态归档**不携带**它依赖的代码,所以包会把依赖记下来,由消费者解析: + +```toml +[dependencies] +"compat.zlib" = "1.3.2" +``` + +`path` 与 `git` 依赖会被丢弃:它们指向发布者的磁盘,原样发出去等于 +给消费者一个在他们那里含义完全不同的地址。如果你的库依赖这类东西, +要么把它也发布出去,要么在打包前 vendor 掉。 + +## 老版本 mcpp 拿到这种包会怎样 + +**能构建。** 生成的 manifest 里每一个键都是既有的,所以老客户端读得懂、 +链得上。它做不到的是执行上面那两道闸门 —— 它无从知道 +`provenance = "mcpp-pack …"` 有什么含义。 + +这是**降级**而不是变砖,方向是对的。但它意味着**闸门只保护新客户端**, +如果你面向的是混合版本的用户群,这一条应当写进发布说明。 + +## 当前边界 + +| | 状态 | +|---|---| +| `kind = "lib"`(静态) | ✅ 所有 target | +| `kind = "shared"` on Linux/ELF | ✅ | +| `kind = "shared"` on PE / Mach-O | ❌ 拒绝 —— 导入库与 install-name 尚未建模 | +| `kind = "shared"` on `*-musl` | ❌ musl target 是静态链接的 | +| 发布预编译 BMI | ❌ 未尝试;BMI 与编译器构建逐位绑定 | +| 把依赖打包进去 | ❌ 改为声明依赖(见上) | diff --git a/docs/zh/README.md b/docs/zh/README.md index 59547931..adabc090 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -14,3 +14,4 @@ - [09 - 发布 mcpp](09-release.md) - [10 - 发布一个库到 mcpp-index](10-publishing-a-library.md) - [11 - 机器可读输出](11-machine-output.md) +- [12 - 分发预编译库](12-binary-distribution.md) diff --git a/examples/05-lib-dist/README.md b/examples/05-lib-dist/README.md new file mode 100644 index 00000000..14e47645 --- /dev/null +++ b/examples/05-lib-dist/README.md @@ -0,0 +1,94 @@ +# 05 — Shipping a prebuilt library + +A library with **two interfaces at once** — a C header and a C++ module — and +what `mcpp pack` does with them. + +```bash +mcpp pack mathkit # static library package +mcpp pack mathkit-shared # dynamic library package (Linux/ELF today) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target x86_64-linux-musl # one package, two legs +``` + +There is no `--lib` and no `--artifact static|shared`. What gets packed is +decided by `[targets.].kind`, which is where mcpp already records what an +artifact is — a second place to say it could only ever disagree with the first. + +## What the command prints, and why both lists matter + +``` + Packed leg x86_64-linux-gnu [x86_64-linux-gnu-gcc16-libstdcxx16-c++23] + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm + Packed target/dist/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.tar.gz +``` + +If you are shipping a closed-source library, **the second list is the one to +read**. Publishing too little fails loudly in your consumer's compile; +publishing too much silently puts your implementation on someone's disk. + +## Why `secret.cppm` is not published + +`src/secret.cppm` is an *implementation partition* (`module mathkit:secret;`, +no `export`). It produces a BMI and a `.m.o` exactly like the interface units +do — so "publish every module unit" would leak it, and "publish every `.m.o`" +is not a rule mcpp uses. + +What travels is the **module closure of the lib root**: `mathkit.cppm` and +what its purview imports (`:api`). Nothing else. Try it: + +```bash +mcpp pack mathkit +grep -r house_factor target/dist/*/interface/ ; echo "exit=$?" # no match +``` + +**Now break it on purpose.** Add `import :secret;` to `src/mathkit.cppm` and +pack again: the interface now reaches the partition, so it must be published +for a consumer to compile at all — and `mcpp pack` stops and tells you, rather +than shipping a package that cannot be built. + +## Static and dynamic from one project + +Two targets, not two commands with a flag: + +``` +bin/libmathkit.a +bin/libmathkit-shared.so +bin/libmathkit.so.1 -> libmathkit-shared.so # the soname alias +``` + +The `.so` file carries the target's name; `soname` is what consumers actually +load, and the package records that. + +## What ends up in the package + +``` +mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ +├── mcpp.toml # an ORDINARY manifest — no new section +├── include/mathkit_c.h # text interface: #include, never compiled +├── interface/mathkit.cppm # module interface: the consumer compiles it +├── interface/api.cppm +└── lib/x86_64-linux-gnu/libmathkit.a +``` + +`lib/` is keyed by **triple**, not by OS: MinGW and MSVC are both Windows and +produce `libfoo.a` and `foo.lib` respectively. + +Open the generated `mcpp.toml`. Everything in it is a key mcpp already had — +`sources`, `include_dirs`, `[modules] exports`, a `cfg(...)` block per leg, and +`[[runtime.artifacts]]` carrying each artifact's ABI tag and digest. That is +why an older mcpp can still *build* against this package: it just does not run +the checks. + +## The tag, and why a C library gets a shorter one + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface +x86_64-linux-gnu # an extern "C" interface only +``` + +A tag names the dimensions the artifact actually constrains, and unnamed ones +are don't-care. So a C library needs one tag per triple instead of one per +triple per compiler — no flag, no mode: the shape *is* the statement. + +See [06-lib-consume](../06-lib-consume/) for the other end. diff --git a/examples/05-lib-dist/include/mathkit_c.h b/examples/05-lib-dist/include/mathkit_c.h new file mode 100644 index 00000000..cc49dc5c --- /dev/null +++ b/examples/05-lib-dist/include/mathkit_c.h @@ -0,0 +1,13 @@ +/* The TEXT interface: published whole, never trimmed. + A source distribution of this package puts every header here on its + consumers' include path, so a binary package that shipped a subset would + give the same library a different public surface. */ +#ifdef __cplusplus +extern "C" { +#endif + +int mathkit_add(int a, int b); + +#ifdef __cplusplus +} +#endif diff --git a/examples/05-lib-dist/mcpp.toml b/examples/05-lib-dist/mcpp.toml new file mode 100644 index 00000000..c3563b41 --- /dev/null +++ b/examples/05-lib-dist/mcpp.toml @@ -0,0 +1,24 @@ +[package] +name = "mathkit" +version = "0.1.0" +description = "Demo: shipping a prebuilt library with both a header and a module interface" +license = "Apache-2.0" + +[build] +sources = ["src/*.cppm", "src/*.cpp", "src/*.c"] +# The public headers. Published whole — see include/mathkit_c.h. +include_dirs = ["include"] + +# [lib].path is not set, so the convention applies: src/mathkit.cppm is the +# lib root, and the published interface is its module closure. + +# `mcpp pack mathkit` -> a static library package. +[targets.mathkit] +kind = "lib" + +# `mcpp pack mathkit-shared` -> a dynamic library package (Linux/ELF today). +# Two targets rather than a flag: `kind` is where mcpp already records what an +# artifact IS, and `soname` gives the .so the name consumers should load. +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" diff --git a/examples/05-lib-dist/src/api.cppm b/examples/05-lib-dist/src/api.cppm new file mode 100644 index 00000000..c2e4bbfc --- /dev/null +++ b/examples/05-lib-dist/src/api.cppm @@ -0,0 +1,10 @@ +// An INTERFACE partition: `export module`, so it is part of the closure and +// its source travels with the package. +export module mathkit:api; + +export namespace mk { +// Declarations only. The definitions live in the implementation units below +// and reach the consumer as the prebuilt archive. +int add(int a, int b); +double scale(double v); +} diff --git a/examples/05-lib-dist/src/capi.c b/examples/05-lib-dist/src/capi.c new file mode 100644 index 00000000..8e3beddb --- /dev/null +++ b/examples/05-lib-dist/src/capi.c @@ -0,0 +1,3 @@ +/* The header interface's implementation. A consumer that only #includes + mathkit_c.h never compiles a single module unit. */ +int mathkit_add(int a, int b) { return a + b; } diff --git a/examples/05-lib-dist/src/impl.cpp b/examples/05-lib-dist/src/impl.cpp new file mode 100644 index 00000000..5926870b --- /dev/null +++ b/examples/05-lib-dist/src/impl.cpp @@ -0,0 +1,7 @@ +module mathkit; +import :secret; + +namespace mk { +int add(int a, int b) { return a + b; } +double scale(double v) { return v * house_factor(); } +} diff --git a/examples/05-lib-dist/src/mathkit.cppm b/examples/05-lib-dist/src/mathkit.cppm new file mode 100644 index 00000000..c88998f8 --- /dev/null +++ b/examples/05-lib-dist/src/mathkit.cppm @@ -0,0 +1,4 @@ +// The published root. Everything a consumer can `import mathkit;` reaches from +// here — and `mcpp pack` publishes exactly that closure, nothing else. +export module mathkit; +export import :api; diff --git a/examples/05-lib-dist/src/secret.cppm b/examples/05-lib-dist/src/secret.cppm new file mode 100644 index 00000000..37b25740 --- /dev/null +++ b/examples/05-lib-dist/src/secret.cppm @@ -0,0 +1,12 @@ +// An IMPLEMENTATION partition: `module` with no `export`. +// +// It produces a BMI and a `.m.o` exactly like the interface units do — which +// is why "publish every module unit" would leak it. It is NOT reachable from +// the interface's purview, so `mcpp pack` withholds its source and keeps its +// object in the archive. +module mathkit:secret; + +namespace mk { +// Pretend this is the part you are not shipping as source. +double house_factor() { return 2.5; } +} diff --git a/examples/06-lib-consume/README.md b/examples/06-lib-consume/README.md new file mode 100644 index 00000000..69451473 --- /dev/null +++ b/examples/06-lib-consume/README.md @@ -0,0 +1,75 @@ +# 06 — Consuming a prebuilt library + +Three consumers of the package [05-lib-dist](../05-lib-dist/) produces: one +that only `#include`s, one that only `import`s, and one that does both. + +```bash +cd ../05-lib-dist && mcpp pack mathkit # produce the package +cd ../06-lib-consume +# point [dependencies].mathkit at the directory that appeared under +# ../05-lib-dist/target/dist/, then: +mcpp run consume-header +mcpp run consume-module +mcpp run consume-both +``` + +A packed library is an **ordinary mcpp package**. It carries a normal +`mcpp.toml`, so a `path` dependency, a downloaded archive and an index entry +all reach it through the same code path — there is nothing new to learn on +this side. + +## What the two interface modes cost you + +| | `#include ` | `import mathkit;` | +|---|---|---| +| does mcpp compile anything of the package? | no | yes — the published `.cppm` | +| what constrains compatibility | the libc ABI | compiler, C++ stdlib, C++ level | +| the tag the package publishes | `x86_64-linux-gnu` | `x86_64-linux-gnu-gcc16-libstdcxx16-c++23` | + +Both work against the *same* package, at the same time. + +## What is checked before your build links + +Nothing is enforced that the package did not declare, and the two things it +does declare are the two that fail silently otherwise. + +**The interface still matches its binaries.** Edit one line of +`interface/api.cppm` in the package and rebuild: + +``` +error: mcpp.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +That refusal exists because the alternative was measured: swap two `int` +members of a struct in a shipped interface — which the Itanium ABI does not +mangle — and the consumer compiles, links, runs, and prints transposed data, +with no diagnostic from any tool. + +**The binaries were built for your toolchain.** Switch `[toolchain]` to another +compiler and rebuild: + +``` +error: mcpp.mathkit@0.1.0: no prebuilt artifact matches this toolchain. + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : + x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on: + compiler needs gcc15, this build has gcc16 + stdlib needs libstdcxx15, this build has libstdcxx16 +``` + +The tags it *does* have are part of the message on purpose: "not found" would +send you looking for a package that is already on your disk. + +## One thing that will not work, and should not + +```bash +cd ../05-lib-dist/target/dist/mathkit-0.1.0-*/ && mcpp build +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +Its `interface/` holds declarations whose definitions are in the archive +beside them. Building there compiles the declarations, produces a near-empty +library and reports success — a failure that looks exactly like a success. diff --git a/examples/06-lib-consume/mcpp.toml b/examples/06-lib-consume/mcpp.toml new file mode 100644 index 00000000..4e930b9b --- /dev/null +++ b/examples/06-lib-consume/mcpp.toml @@ -0,0 +1,23 @@ +[package] +name = "lib-consume" +version = "0.1.0" +description = "Demo: consuming a prebuilt library package three different ways" +license = "Apache-2.0" + +# A packed library is an ordinary package: a path dependency, a file, or an +# index entry all reach it the same way. Point this at whatever +# `mcpp pack mathkit` produced under ../05-lib-dist/target/dist/. +[dependencies] +mathkit = { path = "../05-lib-dist/target/dist/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } + +[targets.consume-header] +kind = "bin" +main = "src/main_header.cpp" + +[targets.consume-module] +kind = "bin" +main = "src/main_module.cpp" + +[targets.consume-both] +kind = "bin" +main = "src/main_both.cpp" diff --git a/examples/06-lib-consume/src/main_both.cpp b/examples/06-lib-consume/src/main_both.cpp new file mode 100644 index 00000000..635b56a2 --- /dev/null +++ b/examples/06-lib-consume/src/main_both.cpp @@ -0,0 +1,11 @@ +// Both at once, from the same package. The two interface modes do not +// interfere: one is preprocessor input, the other is compiler input. +#include +#include + +import mathkit; + +int main() { + std::printf("both : c=%d module=%d\n", mathkit_add(2, 3), mk::add(2, 3)); + return 0; +} diff --git a/examples/06-lib-consume/src/main_header.cpp b/examples/06-lib-consume/src/main_header.cpp new file mode 100644 index 00000000..58391052 --- /dev/null +++ b/examples/06-lib-consume/src/main_header.cpp @@ -0,0 +1,9 @@ +// Consuming through the TEXT interface: no `import`, nothing of the package is +// compiled. The header is preprocessor input; the code is in the prebuilt lib. +#include +#include + +int main() { + std::printf("header : mathkit_add(2, 3) = %d\n", mathkit_add(2, 3)); + return 0; +} diff --git a/examples/06-lib-consume/src/main_module.cpp b/examples/06-lib-consume/src/main_module.cpp new file mode 100644 index 00000000..e501a26a --- /dev/null +++ b/examples/06-lib-consume/src/main_module.cpp @@ -0,0 +1,15 @@ +// Consuming through the MODULE interface. mcpp compiles the package's +// published `.cppm` to get a BMI — cheap, they are declarations — and links +// the definitions out of the prebuilt archive. +// +// NB: `#include` before `import`. GCC 16 mis-scopes headers included after a +// module import in a non-module TU, and the error it reports names neither. +#include + +import mathkit; + +int main() { + std::printf("module : mk::add(2, 3) = %d, mk::scale(2.0) = %.1f\n", + mk::add(2, 3), mk::scale(2.0)); + return 0; +} diff --git a/mcpp.toml b/mcpp.toml index e1f18217..a97f59ff 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.17.1" +version = "2026.8.17.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/plan.cppm b/src/build/plan.cppm index e3a36e08..9aac279d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -336,26 +336,10 @@ std::string sanitize_for_path(std::string_view module_name) { return s; } -std::string object_filename_for(const std::filesystem::path& src, - std::string_view objExt = ".o") { - // The naming POLICY lives in mcpp.source_kind (see ObjectNaming there for - // why every historical name is frozen and only new extensions get the - // collision-proof form). This function only formats it. - switch (mcpp::object_naming_for(src)) { - case mcpp::ObjectNaming::StemDotM: - return src.stem().string() + ".m" + std::string(objExt); - case mcpp::ObjectNaming::Stem: - return src.stem().string() + std::string(objExt); - case mcpp::ObjectNaming::FullFilename: - break; - } - // Assembly siblings of a C/C++ TU commonly share its stem (foo.c + - // foo.asm); keeping the full extension means they can never collide — - // the per-package collision prefix can't help two same-stem files in the - // same directory. Every extension a project adds via - // `[build] module_extensions` lands here for the same reason. - return src.filename().string() + std::string(objExt); -} +// Both the naming POLICY and its formatting now live in mcpp.source_kind, so +// the planner and `mcpp pack` cannot answer "what is this object called" +// differently. This alias keeps the local spelling every call site below uses. +using mcpp::object_filename_for; std::string qualified_package_name(const mcpp::manifest::Manifest& manifest) { if (!manifest.package.namespace_.empty() diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a4f91dd7..e539dd84 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -46,6 +46,8 @@ import mcpp.platform.capacity; // the host fallback handed to schedule::decide import mcpp.build.graph_shape; // #407: the graph says which mode wrote it import mcpp.build.runtime_validation; // declared artifact -> identity verdict import mcpp.build.cache_key; +import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against +import mcpp.pack.prebuilt; // …and the check itself import mcpp.build.build_program; import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides @@ -202,12 +204,14 @@ materialize_generated_files(const std::filesystem::path& root, // deps, and nothing said why. A package's `[target.windows.dependencies]` is // its own statement about itself and means the same thing whether the package // is the root or someone's dependency. +// The resolved triple travels INSIDE `ctx` (cfgpred::Ctx::triple). It used to +// be a third parameter here too, which is how a bare-triple predicate came to +// disagree with a cfg() one about the same native build — see the note on Ctx. void merge_conditional_config(mcpp::manifest::Manifest& m, - const cfgpred::Ctx& ctx, - std::string_view targetTriple) + const cfgpred::Ctx& ctx) { for (auto const& cc : m.conditionalConfigs) { - if (!cfgpred::matches(cc.predicate, ctx, targetTriple)) continue; + if (!cfgpred::matches(cc.predicate, ctx)) continue; // One append() for every field the axis may carry (#258). Matching // sections land AFTER the base entries, so a conditional rule beats // a broader unconditional one under GNU last-wins — which is what @@ -463,6 +467,9 @@ export struct BuildContext { std::filesystem::path stdBmi; std::filesystem::path stdObject; mcpp::build::BuildPlan plan; + // The scanned module graph. Only `mcpp pack` reads it — see the note at + // the assignment for why the plan cannot answer its question. + mcpp::modgraph::Graph graph; // Resolved profile name (resolve_profile_name). Carried so run_build_plan // can record it in .build_cache — without it the fast path cannot tell // whether a cached build.ninja was generated for the profile being asked @@ -697,6 +704,27 @@ prepare_build(bool print_fingerprint, : mcpp::manifest::load(*root / "mcpp.toml"); if (!m) return std::unexpected(m.error().format()); + // A DISTRIBUTION package is not a source tree, and building "in" one is a + // failure that looks like a success: `interface/` holds declarations whose + // definitions are in the prebuilt archive, so the build compiles the + // declarations, produces a near-empty library, links nothing, and reports + // Finished. The archive it was supposed to carry never enters the picture. + // + // Only the ROOT is refused. As a dependency this is exactly what the + // package is for — the consumer compiles the interface and links the + // artifact, which is the whole design. + if (!overrides.preloaded_manifest && mcpp::pack::is_distribution_package(*m)) { + return std::unexpected(std::format( + "'{}' is a distribution package produced by `mcpp pack`, not a source tree.\n" + " Its sources are interface declarations; the definitions are in the\n" + " prebuilt artifacts beside them, so building here would produce an\n" + " empty library and say it succeeded.\n" + " Use it: add it to a project as a dependency —\n" + " [dependencies]\n" + " {} = {{ path = \"{}\" }}", + root->string(), m->package.name, root->string())); + } + // ─── Workspace handling ──────────────────────────────────────────── // If the manifest has [workspace] and is a virtual workspace (no [package]), // or if -p filter is set, switch to the target member's manifest. @@ -1286,8 +1314,7 @@ prepare_build(bool print_fingerprint, // package's half of the one funnel, not a special case: every package is // merged exactly once, immediately before it is captured into `packages[]`. if (!m->conditionalConfigs.empty()) { - merge_conditional_config(*m, cfgpred::context_for(overrides.target_triple), - overrides.target_triple); + merge_conditional_config(*m, cfgpred::context_for(overrides.target_triple)); } // `[build].defines` must reach the scanner (P1689) and the compile edge, // and must participate in the fingerprint. Fold before dependency @@ -2819,8 +2846,7 @@ prepare_build(bool print_fingerprint, // pass through loadVersionDep. if (!manifest->conditionalConfigs.empty()) { merge_conditional_config(*manifest, - cfgpred::context_for(overrides.target_triple), - overrides.target_triple); + cfgpred::context_for(overrides.target_triple)); } fold_build_defines_into_flags(manifest->buildConfig); @@ -3983,8 +4009,7 @@ prepare_build(bool print_fingerprint, // snapshot this manifest's flags/sources into `packages[]`. if (!dep_manifest->conditionalConfigs.empty()) { merge_conditional_config(*dep_manifest, - cfgpred::context_for(overrides.target_triple), - overrides.target_triple); + cfgpred::context_for(overrides.target_triple)); } fold_build_defines_into_flags(dep_manifest->buildConfig); } else { @@ -5154,11 +5179,53 @@ prepare_build(bool print_fingerprint, roots.push_back(d / "xpkgs"); return roots; }(); + // ─── Prebuilt dependencies: check before planning to link them ───── + // + // Here rather than at each place a dependency manifest is loaded, because + // there are three of those and the check needs the RESOLVED toolchain, + // which only exists by now. One pass over the assembled package list is + // also the only spelling under which a package cannot be checked twice + // with two different answers. + // + // The current tag's SHAPE follows the package's: a package that publishes + // a triple-only tag is saying its interface is `extern "C"`, and comparing + // it against a full tag would refuse a combination it explicitly allows. + // `tag_check` already treats an unnamed dimension as don't-care, so one + // full tag on this side is correct for both. + { + const auto canonicalTriple = tc->targetTriple.empty() + ? mcpp::toolchain::triple::host_triple().str() + : [&] { + auto t = mcpp::toolchain::triple::parse(tc->targetTriple); + return t ? t->str() : tc->targetTriple; + }(); + const auto currentTag = mcpp::pack::cxx_surface_tag( + *tc, canonicalTriple, m->cppStandard.level); + for (std::size_t i = 1; i < packages.size(); ++i) { + auto const& pkg = packages[i]; + if (!mcpp::pack::is_distribution_package(pkg.manifest)) continue; + mcpp::pack::PrebuiltCheck chk{ + .packageRoot = pkg.root, + .packageLabel = mcpp::manifest::package_id(pkg.manifest.package).canonical(), + .current = currentTag, + }; + if (auto ok = mcpp::pack::check_prebuilt(pkg.manifest, chk); !ok) + return std::unexpected(ok.error()); + } + } + auto planResult = mcpp::build::make_plan(*m, *tc, fp, scan.graph, report.topoOrder, packages, *root, ctx.outputDir, stdBmiPath, stdObjectPath, storeRoots); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); + // The module graph outlives the plan for one consumer: `mcpp pack`, which + // has to know which units are INTERFACE (published as source) and which + // are implementation (published only as an object). The plan flattens that + // away — a CompileUnit records what to compile, not what it provides — so + // the packer would otherwise have to scan the tree a second time and could + // then disagree with the build about what the package even contains. + ctx.graph = scan.graph; // mcpp#407. Both callers that produce a non-plain graph arrive here the // same way: dev-dependencies enabled, synthetic test targets appended. The // resulting `default` line names the test binaries and omits the package's diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index 3e354eee..01bfcac7 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -38,7 +38,21 @@ export namespace mcpp::build { // — not the build host. See the manifest design doc. namespace cfgpred { -struct Ctx { std::string os, arch, family, env; }; +// `triple` is the RESOLVED target — the host's for a native build, the +// --target one for a cross build. It is a member rather than a parameter of +// `matches()` because a bare-triple predicate and a `cfg(...)` predicate are +// two spellings of ONE question, and they must be answered from one value. +// +// They were not. `matches()` used to take the raw `--target` string alongside +// this context and short-circuit on `if (triple.empty()) return false;`, while +// `context_for()` below fell back to the host. So `cfg(linux)` matched a native +// build and `[target.'x86_64-linux-gnu'.build]` — the same statement about the +// same machine — did not, silently. That shape is the worst kind: CI passes +// `--target` and is green, the developer's plain `mcpp build` drops the flags, +// and the failure surfaces at link time naming a symbol instead of a predicate. +// manifest/types.cppm's ConditionalConfig has documented the fallback since it +// was written; this makes the bare-triple branch honour it. +struct Ctx { std::string os, arch, family, env, triple; }; // Derive the cfg context from the resolved --target triple, falling back to // the host for a native build. Parsing goes through triple.cppm — the single @@ -56,12 +70,18 @@ inline Ctx context_for(std::string_view targetTriple) { c.arch = t->arch; c.env = t->env; c.family = t->family(); + // Canonical spelling on BOTH sides of the later comparison, so an + // `x86_64-w64-mingw32` key and an `x86_64-windows-gnu` build agree. + c.triple = t->str(); } else { // Escape-hatch triple outside the language: only the leading arch // segment is derivable; other dimensions stay empty (never match). auto dash = targetTriple.find('-'); c.arch = std::string(dash == std::string_view::npos ? targetTriple : targetTriple.substr(0, dash)); + // Unparseable: keep it verbatim so the exact-string fallback in + // `matches()` can still hit an explicit escape-hatch section. + c.triple = std::string(targetTriple); } return c; } @@ -119,7 +139,11 @@ struct Parser { // Evaluate a `[target.]` key. Returns the cfg() result, or — for a // non-cfg key (a bare triple) — an exact match against the resolved triple. -inline bool matches(const std::string& predicate, const Ctx& c, std::string_view triple) { +// +// The resolved triple comes from `c`, never from a second parameter: see the +// note on Ctx for what having two of them cost. +inline bool matches(const std::string& predicate, const Ctx& c) { + const std::string_view triple = c.triple; std::string_view k = predicate; if (k.starts_with("cfg(") && k.ends_with(")")) { Parser p{ k.substr(4, k.size() - 5), 0, c }; @@ -137,6 +161,10 @@ inline bool matches(const std::string& predicate, const Ctx& c, std::string_view // key matches a resolved `x86_64-windows-gnu` build (and vice versa) — // both normalize through triple::parse. Unparseable keys (the explicit- // section escape hatch) fall back to exact string comparison. + // + // `c.triple` is populated for every build, native included, so this is a + // guard and no longer a behaviour: it used to be the line that made a + // bare-triple section silently inert without `--target`. if (triple.empty()) return false; if (auto p = mcpp::toolchain::triple::parse(predicate)) { if (auto rt = mcpp::toolchain::triple::parse(triple)) diff --git a/src/cli.cppm b/src/cli.cppm index 14e4397a..e202334b 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -406,10 +406,16 @@ int run(int argc, char** argv) { // because a .tar.gz full of DLLs is a package most Windows users // cannot open without installing something first. .description("Build + bundle into a self-contained archive") + // NB: a target NAME from [targets.*], not a triple — the same + // split `mcpp run [target]` has. Its `kind` decides what is + // packed, so there is no --lib and no --artifact: a program + // becomes an application bundle, a library becomes a library + // package. Omit it and mcpp picks the only packable target. + .arg(cl::Arg("target").help("Target name from [targets.*] (optional)")) .option(cl::Option("mode").takes_value() .help("system | vendored (default) | self-contained | static")) - .option(cl::Option("target").takes_value() - .help("Triple, e.g. x86_64-linux-musl")) + .option(cl::Option("target").takes_value().multiple() + .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) .option(cl::Option("format").takes_value() .help("tar (default; .zip for a Windows target) | dir")) .option(cl::Option("output").short_name('o').takes_value() diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 44003704..c1380239 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -10,7 +10,9 @@ export module mcpp.cli.cmd_publish; import std; import mcpplibs.cmdline; import mcpp.pack; +import mcpp.pack.library_pipeline; import mcpp.pack.pipeline; +import mcpp.pack.route; import mcpp.publish.pipeline; import mcpp.ui; @@ -55,8 +57,37 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { } } if (auto v = parsed.value("output")) opts.output = *v; - if (auto v = parsed.value("target")) opts.targetTriple = *v; + // `--target` is repeatable: one leg per triple, which is how a library + // package ships for several targets at once. The application path has + // always taken exactly one, and still does — packing one executable for + // several triples would need several executables. + std::vector triples; + if (auto o = parsed.option("target")) triples = o->get().values; + if (!triples.empty()) opts.targetTriple = triples.back(); + + // ─── Which target, and therefore which kind of package ─────────── + // + // The positional is a target NAME. Its `kind` decides everything: a + // program becomes an application bundle (the four --mode depths), a + // library becomes a library package. Reading the answer out of the + // manifest is the whole reason there is no --lib flag. + auto route = mcpp::pack::route_pack_target(parsed.positional(0)); + if (!route) { mcpp::ui::error(route.error()); return 2; } + if (route->library) { + if (modeFromUser) { + mcpp::ui::warning(std::format( + "--mode is an application-bundle depth and does not apply to the " + "library target '{}' yet; ignoring it", route->targetName)); + } + return mcpp::pack::build_and_pack_library(route->targetName, triples, opts); + } + if (triples.size() > 1) { + mcpp::ui::error( + "--target may be given once when packing a program: an application " + "bundle wraps one executable, and one executable has one target."); + return 2; + } return mcpp::pack::build_and_pack(std::move(opts), modeFromUser); } diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 30beedb0..b73deaab 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -237,11 +237,22 @@ std::expected parse_string(std::string_view content, } // [build].sources (M5.0 new home) + [modules].sources (deprecated, compat) - if (auto v = doc->get_string_array("build.sources")) m.buildConfig.sources = *v; + // + // `sourcesDeclared` records PRESENCE, not content: `sources = []` has to + // mean "compile nothing", and only the key's existence can say that (see + // BuildConfig::sourcesDeclared). Set from either spelling, because the + // legacy one has to be able to express it too. + if (auto v = doc->get_string_array("build.sources")) { + m.buildConfig.sources = *v; + m.buildConfig.sourcesDeclared = true; + } if (auto v = doc->get_string_array("modules.sources")) { m.modules.sources = *v; // If [build].sources wasn't set, mirror legacy field into new field. - if (m.buildConfig.sources.empty()) m.buildConfig.sources = *v; + if (!m.buildConfig.sourcesDeclared) { + m.buildConfig.sources = *v; + m.buildConfig.sourcesDeclared = true; + } } // Mirror new → legacy so existing code reading manifest.modules.sources keeps working. if (m.modules.sources.empty()) m.modules.sources = m.buildConfig.sources; @@ -1700,7 +1711,14 @@ void apply_defaults_and_infer(Manifest& m, const std::filesystem::path& root) { // that vendors foreign-syntax .asm can `!`-exclude it. const auto extTable = mcpp::extension_table_for(m.buildConfig.moduleExtensions); - if (m.buildConfig.sources.empty()) { + // `!sourcesDeclared` and not `sources.empty()`: an author who wrote + // `sources = []` asked for NOTHING, and filling the default glob over that + // answers a question they already answered. That is not hypothetical — a + // binary distribution package ships prebuilt artifacts and, in the + // header-only shape, nothing to compile at all; the glob would sweep up + // whatever happens to sit under `src/` and compile it into the consumer's + // build, where it can collide with the prebuilt library's own symbols. + if (!m.buildConfig.sourcesDeclared) { // Derived from the extension table rather than written beside it — // otherwise declaring `module_extensions = [".ixx"]` would change how // `.ixx` is TREATED without changing whether it is FOUND, and the key diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index 1dcf3afb..6240cfe7 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -353,6 +353,25 @@ struct Resources { // is read in ~150 places, and a BuildConfig genuinely IS a set of build // inputs plus the selection axis and resolved policy scalars. struct BuildConfig : BuildInputs { + // Was `sources` WRITTEN, as opposed to merely being empty? + // + // Presence is semantic here for the same reason it is on + // `XlingsConfig::subosDeclared`: an absent key selects the default glob, + // while an explicit `sources = []` selects "compile nothing". A container + // alone cannot tell those apart, and until this flag existed it did not: + // `sources = []` and deleting the line produced byte-identical build + // graphs, so an author had NO spelling for "nothing". + // + // A binary distribution package needs that spelling. It ships prebuilt + // artifacts and, in the header-only shape, no compilable source at all — + // yet any file left under `src/` would be swept up by the default glob and + // compiled into the consumer's build, where it can collide with the very + // symbols the prebuilt library already defines. + // + // Deliberately on BuildConfig and not on BuildInputs: the conditional axis + // (`[target.'cfg(...)'.build]`) only ever APPENDS sources, so "declared + // empty" has no meaning there — it is the same as contributing nothing. + bool sourcesDeclared = false; // `[build] jobs` — how many compiles to run at once. A decimal count, // "auto", or empty (the default) meaning "let the backend decide". // diff --git a/src/pack/abi_tag.cppm b/src/pack/abi_tag.cppm new file mode 100644 index 00000000..64322d0e --- /dev/null +++ b/src/pack/abi_tag.cppm @@ -0,0 +1,231 @@ +// mcpp.pack.abi_tag — the readable compatibility tag a prebuilt artifact +// publishes, and the check a consumer runs against it. +// +// WHY A TAG AND NOT THE BUILD FINGERPRINT +// +// There are two compatibility questions and they have different answerers: +// +// "can this binary be linked into your build?" ← the TAG +// "can this BMI be reused verbatim?" ← cache_key::key_hex +// +// Only the first is publishable. A build key folds in the Merkle closure of +// the package's own dependencies, so it is a function of the CONSUMER's +// resolution — a producer would have to enumerate one key per possible +// consumer graph, which is not a finite job. The tag is a projection of facts +// the producer knows alone, which is exactly why it can be written into an +// index before any consumer exists. +// +// THE SHAPE IS THE SURFACE +// +// A tag carries the dimensions the artifact actually constrains, and nothing +// more. A library whose whole interface is `extern "C"` constrains the libc +// ABI and not the C++ one, so it publishes a triple and stops: +// +// x86_64-linux-gnu ← C surface +// x86_64-linux-gnu-gcc16-libstdcxx16-c++23 ← C++ surface +// +// `tag_check` then compares only the dimensions the published tag names. That +// is the same don't-care rule `mcpp.toolchain.abi::abi_check` has used since +// the glfw conflation was fixed, and it means the C case needs no flag, no +// mode and no escape hatch: a shorter tag IS the statement. Tag counts for C +// libraries stay at one-per-triple instead of one-per-triple-per-compiler. +// +// THE TRIPLE IS THE CANONICAL ONE +// +// `arch-os-env` comes from mcpp.toolchain.triple, never from the compiler's +// own `-dumpmachine` answer. Those disagree: gcc reports `x86_64-w64-mingw32` +// where mcpp's target vocabulary — and therefore every `[target.'']` +// key a package can be selected by — says `x86_64-windows-gnu`. Publishing the +// compiler's spelling would give one decision two spellings, and the halves +// would be compared by string. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §1.2. + +export module mcpp.pack.abi_tag; + +import std; +import mcpp.toolchain.model; +import mcpp.toolchain.triple; + +export namespace mcpp::pack { + +// A parsed compatibility tag. An EMPTY dimension means "not constrained", +// never "unknown": the producer decides what to name, and a consumer must not +// invent a constraint the artifact did not declare. +struct AbiTag { + std::string triple; // canonical, e.g. "x86_64-linux-gnu" (never empty) + std::string compiler; // "gcc16" — empty on a C-surface tag + std::string stdlib; // "libstdcxx16" — empty on a C-surface tag + std::string standard; // "c++23" — empty on a C-surface tag + + bool c_surface() const { return compiler.empty() && stdlib.empty() && standard.empty(); } + + std::string str() const { + std::string s = triple; + for (auto const* seg : { &compiler, &stdlib, &standard }) + if (!seg->empty()) { s += '-'; s += *seg; } + return s; + } + + bool operator==(const AbiTag&) const = default; +}; + +// The major version segment of "16.1.0" → "16". Empty input yields "0" rather +// than an empty segment, so a tag never contains a bare "gcc-" that would +// re-split differently on parse. +std::string major_of(std::string_view version); + +// "libstdc++" → "libstdcxx". The tag is joined and split on '-', and a '+' +// inside a segment is fine, but the spelling is also a filename component in +// `target/dist/`, where '+' is best avoided. +std::string stdlib_token(std::string_view stdlibId); + +// The C surface: the triple alone. +AbiTag c_surface_tag(std::string_view canonicalTriple); + +// The C++ surface: every dimension the artifact constrains. +// +// `canonicalTriple` is passed rather than read from `tc.targetTriple` on +// purpose — the caller has already resolved which target is being packed, and +// the toolchain's own triple is the compiler's spelling of it. +AbiTag cxx_surface_tag(const mcpp::toolchain::Toolchain& tc, + std::string_view canonicalTriple, + int cppLevel); + +// Read a published tag back. The optional C++ half is exactly three segments +// and the last of them starts with "c++", so parsing runs from the END: that +// is the only split that is unambiguous when the triple itself contains +// dashes (and it has a variable number of them — `aarch64-macos` has one, +// `x86_64-linux-gnu` has two). +// +// Returns nullopt for a tag with no triple at all, or for a 3-segment tail +// whose last segment is not a `c++NN`. +std::optional parse_abi_tag(std::string_view s); + +// One dimension on which a published tag refuses the current toolchain. +struct TagMismatch { + std::string dimension; // "triple" | "compiler" | "stdlib" | "standard" + std::string need; // what the artifact was built for + std::string got; // what this build resolved +}; + +// Does `published` accept `current`? Empty dimensions in `published` are +// don't-care (see the header). Returns every mismatch so the diagnostic can +// name all of them at once rather than one per rebuild. +// +// `standard` is the one asymmetric dimension: a consumer building at a HIGHER +// level than the artifact was compiled at is fine (the interface it compiles +// is the artifact's own source, and a newer level accepts it), while a lower +// one is not (the interface may use syntax the consumer's level lacks). So it +// is compared as a floor, not for equality. +std::vector tag_check(const AbiTag& published, const AbiTag& current); + +// The `c++NN` segment as its numeric level, or 0 when unparseable. +int standard_level(std::string_view standardSegment); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::string major_of(std::string_view version) { + auto dot = version.find('.'); + auto head = dot == std::string_view::npos ? version : version.substr(0, dot); + // Keep only leading digits: "16.1.0" → "16", "22.1.8" → "22", and a + // vendor string like "19.44.35207" → "19". + std::size_t n = 0; + while (n < head.size() && std::isdigit(static_cast(head[n]))) ++n; + if (n == 0) return "0"; + return std::string(head.substr(0, n)); +} + +std::string stdlib_token(std::string_view stdlibId) { + std::string out; + out.reserve(stdlibId.size()); + for (char c : stdlibId) { + if (c == '+') { out += "x"; continue; } + if (c == '-' || c == ' ') continue; // "msvc-stl" / "MSVC STL" + out += static_cast(std::tolower(static_cast(c))); + } + return out.empty() ? std::string("unknownstl") : out; +} + +AbiTag c_surface_tag(std::string_view canonicalTriple) { + AbiTag t; + t.triple = std::string(canonicalTriple); + return t; +} + +AbiTag cxx_surface_tag(const mcpp::toolchain::Toolchain& tc, + std::string_view canonicalTriple, + int cppLevel) +{ + AbiTag t; + t.triple = std::string(canonicalTriple); + t.compiler = std::string(tc.compiler_name()) + major_of(tc.version); + t.stdlib = stdlib_token(tc.stdlibId) + major_of(tc.stdlibVersion); + t.standard = std::format("c++{}", cppLevel); + return t; +} + +int standard_level(std::string_view seg) { + if (!seg.starts_with("c++")) return 0; + int level = 0; + for (char c : seg.substr(3)) { + if (!std::isdigit(static_cast(c))) return 0; + level = level * 10 + (c - '0'); + } + return level; +} + +std::optional parse_abi_tag(std::string_view s) { + if (s.empty()) return std::nullopt; + + std::vector seg; + for (std::size_t i = 0; i <= s.size(); ) { + auto j = s.find('-', i); + if (j == std::string_view::npos) { seg.push_back(s.substr(i)); break; } + seg.push_back(s.substr(i, j - i)); + i = j + 1; + } + if (seg.empty()) return std::nullopt; + + AbiTag t; + // Parse from the end: the C++ half is exactly three segments and its last + // one is `c++NN`. Anything else is a triple-only (C-surface) tag. + if (seg.size() >= 4 && standard_level(seg.back()) != 0) { + t.standard = std::string(seg[seg.size() - 1]); + t.stdlib = std::string(seg[seg.size() - 2]); + t.compiler = std::string(seg[seg.size() - 3]); + seg.resize(seg.size() - 3); + } + std::string triple; + for (std::size_t i = 0; i < seg.size(); ++i) { + if (i) triple += '-'; + triple += seg[i]; + } + if (triple.empty()) return std::nullopt; + t.triple = std::move(triple); + return t; +} + +std::vector tag_check(const AbiTag& published, const AbiTag& current) { + std::vector out; + auto exact = [&](std::string_view dim, const std::string& need, const std::string& got) { + if (need.empty()) return; // not constrained + if (need != got) out.push_back({ std::string(dim), need, got }); + }; + exact("triple", published.triple, current.triple); + exact("compiler", published.compiler, current.compiler); + exact("stdlib", published.stdlib, current.stdlib); + + // Floor, not equality — see the declaration. + if (!published.standard.empty()) { + int need = standard_level(published.standard); + int got = standard_level(current.standard); + if (need != 0 && got != 0 && got < need) + out.push_back({ "standard", published.standard, current.standard }); + } + return out; +} + +} // namespace mcpp::pack diff --git a/src/pack/digest.cppm b/src/pack/digest.cppm new file mode 100644 index 00000000..60917d04 --- /dev/null +++ b/src/pack/digest.cppm @@ -0,0 +1,61 @@ +// mcpp.pack.digest — the content digests a library package records, and the +// consumer re-computes. +// +// Its own module, and a leaf, because BOTH sides need it: `mcpp pack` writes +// the digests and `mcpp build` verifies them. Putting it in the packer would +// make the build depend on the packer, which already depends on the build. +// +// `fnv1a:` and not `sha256:`. Three reasons, in order of weight: +// +// * it needs no external tool. `mcpp publish` shells out to `sha256sum`, +// which is not on a Windows host — and a package format that can only be +// verified on some of the platforms it targets is not a format. +// * the question is "has this changed since it was packaged", not "can an +// adversary forge a collision". Transport integrity is the index entry's +// `sha256` over the whole tarball; this is the second line, for a package +// that has already been extracted or handed over as a directory. +// * `mcpp.lock` already records `fnv1a:` digests, so the vocabulary is one +// a reader of this project has already met. + +export module mcpp.pack.digest; + +import std; +import mcpp.toolchain.fingerprint; + +export namespace mcpp::pack { + +// `fnv1a:<16 hex>` over one file's bytes. +std::string file_digest(const std::filesystem::path& p); + +// The digest of an ordered interface set: each file's NAME and then its +// content hash, sorted by name. +// +// The name is folded in deliberately. Renaming a published interface unit +// changes what a consumer compiles — the `sources` list, and therefore the +// module the BMI comes from — even when not one byte of any file changed. +std::string interface_set_digest(const std::vector& files); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::string file_digest(const std::filesystem::path& p) { + return "fnv1a:" + mcpp::toolchain::hash_file(p); +} + +std::string interface_set_digest(const std::vector& files) { + std::vector> byName; + for (auto const& f : files) byName.emplace_back(f.filename().string(), f); + std::ranges::sort(byName, {}, &std::pair::first); + + std::string acc; + for (auto const& [name, path] : byName) { + acc += name; + acc += '\x1f'; + acc += mcpp::toolchain::hash_file(path); + acc += '\x1e'; + } + return "fnv1a:" + mcpp::toolchain::hash_string(acc); +} + +} // namespace mcpp::pack diff --git a/src/pack/interface.cppm b/src/pack/interface.cppm new file mode 100644 index 00000000..3d4c7a23 --- /dev/null +++ b/src/pack/interface.cppm @@ -0,0 +1,163 @@ +// mcpp.pack.interface — which module units a library package publishes. +// +// THE ASYMMETRY THAT DECIDES THE DESIGN +// +// A prebuilt package ships module interface SOURCE (the consumer has to +// compile it to get a BMI) and a prebuilt library for everything else. So the +// packer has to answer "which .cppm travel?" — and the two ways of getting it +// wrong are not symmetric: +// +// ship too FEW → the consumer's compile fails, loudly, naming the module: +// `mathkit:secret: error: failed to read compiled module` +// ship too MANY → a closed-source implementation partition's SOURCE is +// published. Nothing fails. Nobody finds out. +// +// One of those is a bug report and the other is a disclosure, which is why the +// set is COMPUTED from the module graph and cannot be hand-written in the +// manifest. An author-supplied list drifts, and it drifts toward the silent +// side as the package evolves. +// +// WHY `.m.o` IS NOT THE ANSWER +// +// The tempting shortcut is "publish the sources of the units that produce a +// BMI". It is wrong, measured: an implementation partition (`module M:impl;`, +// no `export`) also produces a `.m.o`, and its source must NOT be published. +// `.m.o` means "module unit", not "interface". +// +// The same closure has a SECOND use, and getting that one wrong was also +// measured: the archive members to delete before shipping are the objects of +// the units being published as source — not every `.m.o`. Dropping every +// `.m.o` deletes the implementation partition's real code, and every target +// then fails to link with `undefined reference` to a symbol that is nowhere +// in the diagnostic's vicinity. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2.4.2. + +export module mcpp.pack.interface; + +import std; +import mcpp.modgraph.graph; +import mcpp.source_kind; + +export namespace mcpp::pack { + +// What `mcpp pack` publishes, withholds, and cannot decide. +struct InterfaceClosure { + // The module root the closure started from, e.g. "mathkit". + std::string rootModule; + // Units whose SOURCE travels, in a stable order (root first, then the + // order they were reached). Paths are exactly the graph's, so the caller + // can relativize them against whichever root it knows about. + std::vector published; + // This package's other module units — implementation units, implementation + // partitions, and any module the interface never reaches. Reported so + // `mcpp pack` can print both lists: "what you are publishing" is only half + // of what an author of a closed-source library needs to see. + std::vector withheld; + // Module names the interface imports that NOTHING in this graph provides. + // + // Two things land here and both are worth stopping for: + // * a partition mcpp's text scanner does not model as a provider + // (`module M:part;` — the scanner records a *requires* on M and no + // provides, so an interface that imports it looks unsatisfiable); + // * a genuinely missing unit. + // + // Either way the published set is INCOMPLETE and the consumer's build will + // fail on it, so this is a hard error at pack time rather than a warning: + // the whole point of the closure is that the failure lands on the person + // who can fix it. + std::vector unresolvedImports; +}; + +// Compute the closure for `packageName`, starting at the unit that provides +// `rootModule`. +// +// Only units belonging to `packageName` are followed: an interface may import +// a DEPENDENCY's module, and that module is the dependency's to publish, not +// ours. Such an import is neither published nor unresolved — it is simply not +// this package's business. +std::expected +interface_closure(const mcpp::modgraph::Graph& graph, + std::string_view packageName, + std::string_view rootModule); + +// The archive members to delete before shipping: the objects of the units in +// `closure.published`. +// +// Deliberately derived from the closure and not from a file extension — see +// the header for what "delete every .m.o" cost when it was tried. +std::vector published_object_names(const InterfaceClosure& closure, + std::string_view objExt = ".o"); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::expected +interface_closure(const mcpp::modgraph::Graph& graph, + std::string_view packageName, + std::string_view rootModule) +{ + InterfaceClosure out; + out.rootModule = std::string(rootModule); + + auto owned = [&](const mcpp::modgraph::SourceUnit& u) { + return u.packageName == packageName; + }; + + auto rootIt = graph.producerOf.find(rootModule); + if (rootIt == graph.producerOf.end()) { + return std::unexpected(std::format( + "no module interface unit in this build provides '{}'", rootModule)); + } + if (!owned(graph.units[rootIt->second])) { + return std::unexpected(std::format( + "module '{}' is provided by package '{}', not '{}'", rootModule, + graph.units[rootIt->second].packageName, packageName)); + } + + std::vector stack{ rootIt->second }; + std::set seen; + std::set unresolved; + + while (!stack.empty()) { + auto idx = stack.front(); + stack.erase(stack.begin()); // breadth-first: root first, then its imports + if (!seen.insert(idx).second) continue; + out.published.push_back(graph.units[idx].path); + + for (auto const& req : graph.units[idx].requires_) { + auto it = graph.producerOf.find(req.logicalName); + if (it == graph.producerOf.end()) { + // Only OUR module's partitions are our problem. A bare name + // with no producer is a dependency's module (or `std`), which + // this package does not publish and must not complain about. + const bool ours = req.logicalName.starts_with(std::string(rootModule) + ":"); + if (ours) unresolved.insert(req.logicalName); + continue; + } + if (!owned(graph.units[it->second])) continue; // a dependency's unit + if (!seen.contains(it->second)) stack.push_back(it->second); + } + } + + for (std::size_t i = 0; i < graph.units.size(); ++i) { + if (!owned(graph.units[i])) continue; + if (seen.contains(i)) continue; + out.withheld.push_back(graph.units[i].path); + } + out.unresolvedImports.assign(unresolved.begin(), unresolved.end()); + return out; +} + +std::vector published_object_names(const InterfaceClosure& closure, + std::string_view objExt) +{ + std::vector names; + names.reserve(closure.published.size()); + for (auto const& p : closure.published) + names.push_back(mcpp::object_filename_for(p, objExt)); + return names; +} + +} // namespace mcpp::pack diff --git a/src/pack/library.cppm b/src/pack/library.cppm new file mode 100644 index 00000000..281c04f3 --- /dev/null +++ b/src/pack/library.cppm @@ -0,0 +1,276 @@ +// mcpp.pack.library — staging a LIBRARY package (interface + prebuilt +// artifacts), as opposed to mcpp.pack's application bundle. +// +// The two share a command and almost nothing else. An application bundle is +// "one executable plus the closure it needs at RUN time"; a library package is +// "the source a consumer must compile, plus the binaries it then links". The +// decision between them is not a flag: it is `[targets.].kind`, so +// `mcpp pack ` reads the answer instead of asking for it. +// +// WHAT THIS FILE IS CAREFUL ABOUT +// +// 1. It never GLOBS for a built artifact. Every path comes from the build +// this run just did. A `target//` directory accumulates one +// subdirectory per fingerprint, and picking "the first one" silently +// selects a stale binary — measured while prototyping this feature, where +// it produced an archive missing a translation unit and a consumer-side +// `undefined reference` that pointed nowhere near the cause. +// +// 2. The archive members it deletes are the objects of the units it is +// PUBLISHING AS SOURCE, computed by mcpp.pack.interface. Not "every +// `.m.o`" — an implementation partition is a module unit too, and its +// object holds the only copy of code nobody else has. +// +// 3. Digests are `fnv1a:` and not `sha256:`. The lock file already speaks +// that vocabulary, it needs no external tool (`sha256sum` is not on a +// Windows host), and the question it answers is "has this changed since +// packaging" — for which a 64-bit content hash is evidence, not a +// security boundary. The tarball's own `sha256` in an index entry is the +// integrity check for transport. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2.3. + +module; +#include + +export module mcpp.pack.library; + +import std; +import mcpp.pack.digest; +import mcpp.pack.manifest_emit; +import mcpp.pack.zip; +import mcpp.platform; + +export namespace mcpp::pack { + +// One target triple's build output, as this run produced it. +struct LibraryLeg { + std::string triple; // canonical + std::filesystem::path artifact; // absolute; the .a / .so this build wrote + std::filesystem::path archiveTool; // `ar` for THIS leg's toolchain (empty = skip drop) + std::string abiTag; + std::string buildKey; + std::string linkName; // the -l argument, e.g. "mathkit" + bool shared = false; +}; + +struct LibraryPackPlan { + std::filesystem::path projectRoot; + std::filesystem::path stagingRoot; // target/dist/ + std::filesystem::path archivePath; // …/.tar.gz | .zip + bool writeArchive = true; // false = `--format dir` + bool zip = false; // PE targets get a .zip + + std::string namespace_, packageName, packageVersion, builtBy; + std::string targetName; + bool targetShared = false; + std::string cxxRuntime; + std::vector exportsModules; + + // Absolute paths from the module graph. + std::vector interfaceSources; + // Archive member names to delete (mcpp.pack.interface::published_object_names). + std::vector dropObjects; + + std::filesystem::path includeDir; // absolute, or empty + std::vector> dependencies; + std::vector extras; // README / LICENSE / [pack].include hits + + std::vector legs; +}; + +struct Error { std::string message; }; + +// Stage, drop, describe, archive. Returns the path a caller should report. +// +// The digests it records come from mcpp.pack.digest, which the CONSUMER also +// uses — one derivation, verified from both ends. +std::expected run_library_pack(const LibraryPackPlan& plan); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +namespace { + +std::expected copy_into(const std::filesystem::path& src, + const std::filesystem::path& dst) +{ + std::error_code ec; + std::filesystem::create_directories(dst.parent_path(), ec); + std::filesystem::copy_file(src, dst, + std::filesystem::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{ std::format( + "cannot copy '{}' -> '{}': {}", src.string(), dst.string(), ec.message()) }); + return {}; +} + +// Every file under `root`, relative, sorted — deterministic archive order. +std::vector walk(const std::filesystem::path& root) { + std::vector out; + std::error_code ec; + for (auto const& e : std::filesystem::recursive_directory_iterator(root, ec)) { + if (!e.is_regular_file()) continue; + out.push_back(std::filesystem::relative(e.path(), root, ec)); + } + std::ranges::sort(out); + return out; +} + +} // namespace + +std::expected +run_library_pack(const LibraryPackPlan& plan) +{ + std::error_code ec; + std::filesystem::remove_all(plan.stagingRoot, ec); + std::filesystem::create_directories(plan.stagingRoot, ec); + if (ec) return std::unexpected(Error{ std::format( + "cannot create staging dir '{}': {}", plan.stagingRoot.string(), ec.message()) }); + + // ── interface/ ──────────────────────────────────────────────────── + // + // Flattened, because the names land in `sources` and a package's internal + // directory layout is not something a consumer should inherit. A collision + // is refused rather than resolved: silently renaming one of two files + // called `api.cppm` would make the emitted `sources` point at the wrong + // one, and the failure would surface in someone else's build. + std::vector interfaceNames; + { + std::map seen; + for (auto const& src : plan.interfaceSources) { + auto name = src.filename().string(); + if (auto it = seen.find(name); it != seen.end()) { + return std::unexpected(Error{ std::format( + "two interface units are both called '{}':\n" + " {}\n {}\n" + "A package's interface is published flat, so their names must differ.", + name, it->second.string(), src.string()) }); + } + seen.emplace(name, src); + if (auto r = copy_into(src, plan.stagingRoot / "interface" / name); !r) + return std::unexpected(r.error()); + interfaceNames.push_back("interface/" + name); + } + std::ranges::sort(interfaceNames); + } + + // ── include/ ────────────────────────────────────────────────────── + // + // Whole, never filtered. A source distribution of this package puts every + // one of these on its consumers' include path (usage requirements), so + // trimming here would give the same library a different public surface + // depending on how it was delivered. + if (!plan.includeDir.empty() && std::filesystem::is_directory(plan.includeDir, ec)) { + for (auto const& rel : walk(plan.includeDir)) + if (auto r = copy_into(plan.includeDir / rel, plan.stagingRoot / "include" / rel); !r) + return std::unexpected(r.error()); + } + + // ── lib// ───────────────────────────────────────────────── + std::vector docLegs; + for (auto const& leg : plan.legs) { + if (!std::filesystem::exists(leg.artifact, ec)) { + return std::unexpected(Error{ std::format( + "the build for '{}' produced no artifact at '{}'", + leg.triple, leg.artifact.string()) }); + } + auto name = leg.artifact.filename().string(); + auto dst = plan.stagingRoot / "lib" / leg.triple / name; + if (auto r = copy_into(leg.artifact, dst); !r) return std::unexpected(r.error()); + + // Delete the objects of the units published as source. The consumer + // compiles those itself; leaving them in the archive means two + // definitions of the module initialiser, resolved by link order. + if (!leg.shared && !plan.dropObjects.empty() && !leg.archiveTool.empty()) { + std::string cmd = mcpp::platform::shell::quote(leg.archiveTool.string()) + + " d " + mcpp::platform::shell::quote(dst.string()); + for (auto const& m : plan.dropObjects) + cmd += " " + mcpp::platform::shell::quote(m); + auto r = mcpp::platform::process::capture(cmd + " 2>&1"); + if (r.exit_code != 0) { + return std::unexpected(Error{ std::format( + "cannot drop published interface objects from '{}' (rc={}): {}", + dst.string(), r.exit_code, r.output) }); + } + } + + docLegs.push_back(PackageLeg{ + .triple = leg.triple, + .libFile = name, + .linkName = leg.linkName, + .abiTag = leg.abiTag, + .digest = file_digest(dst), + .buildKey = leg.buildKey, + .shared = leg.shared, + }); + } + + // ── extras ──────────────────────────────────────────────────────── + for (auto const& x : plan.extras) { + auto rel = std::filesystem::relative(x, plan.projectRoot, ec); + if (ec || rel.empty() || rel.string().starts_with("..")) rel = x.filename(); + if (auto r = copy_into(x, plan.stagingRoot / rel); !r) return std::unexpected(r.error()); + } + + // ── mcpp.toml ───────────────────────────────────────────────────── + { + PackageDoc doc; + doc.namespace_ = plan.namespace_; + doc.name = plan.packageName; + doc.version = plan.packageVersion; + doc.builtBy = plan.builtBy; + doc.interfaceFiles = interfaceNames; + doc.hasIncludeDir = std::filesystem::is_directory(plan.stagingRoot / "include", ec); + doc.interfaceDigest = plan.interfaceSources.empty() && !doc.hasIncludeDir + ? std::string{} + : interface_set_digest(plan.interfaceSources.empty() + ? [&] { + std::vector hdrs; + for (auto const& rel : walk(plan.stagingRoot / "include")) + hdrs.push_back(plan.stagingRoot / "include" / rel); + return hdrs; + }() + : plan.interfaceSources); + doc.cxxRuntime = plan.cxxRuntime; + doc.exportsModules = plan.exportsModules; + doc.targetName = plan.targetName; + doc.targetShared = plan.targetShared; + doc.legs = std::move(docLegs); + doc.dependencies = plan.dependencies; + + std::ofstream os(plan.stagingRoot / "mcpp.toml", std::ios::binary); + if (!os) return std::unexpected(Error{ std::format( + "cannot write '{}'", (plan.stagingRoot / "mcpp.toml").string()) }); + os << emit_package_manifest(doc); + } + + if (!plan.writeArchive) return plan.stagingRoot; + + // ── archive ─────────────────────────────────────────────────────── + std::filesystem::create_directories(plan.archivePath.parent_path(), ec); + if (plan.zip) { + std::vector entries; + const auto wrapper = plan.stagingRoot.filename().string(); + for (auto const& rel : walk(plan.stagingRoot)) { + entries.push_back(zip::Entry{ + .name = wrapper + "/" + rel.generic_string(), + .source = plan.stagingRoot / rel, + }); + } + if (auto r = zip::write(plan.archivePath, entries); !r) + return std::unexpected(Error{ r.error() }); + } else { + auto cmd = std::format("tar -czf {} -C {} {}", + mcpp::platform::shell::quote(plan.archivePath.string()), + mcpp::platform::shell::quote(plan.stagingRoot.parent_path().string()), + mcpp::platform::shell::quote(plan.stagingRoot.filename().string())); + auto r = mcpp::platform::process::capture(cmd + " 2>&1"); + if (r.exit_code != 0) + return std::unexpected(Error{ std::format( + "tar failed (rc={}): {}", r.exit_code, r.output) }); + } + return plan.archivePath; +} + +} // namespace mcpp::pack diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm new file mode 100644 index 00000000..297ffbf3 --- /dev/null +++ b/src/pack/library_pipeline.cppm @@ -0,0 +1,316 @@ +// mcpp.pack.library_pipeline — everything `mcpp pack ` does when the +// named target is a library. +// +// The sibling of mcpp.pack.pipeline (which packs an application). They are +// separate files rather than one branch because they share almost nothing: +// an application bundle asks "what does this executable need at RUN time", +// a library package asks "what must a consumer compile, and what may it link". +// +// WHICH ONE RUNS IS NOT A FLAG. `[targets.].kind` already says whether a +// target is a program or a library, so the command reads that answer instead +// of asking for it again — there is no `--lib`, and no `--artifact static`. +// A project that publishes both forms declares both targets, which is also +// what it must do for `mcpp build` to produce both. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2. + +module; +#include + +export module mcpp.pack.library_pipeline; + +import std; +import mcpp.build.backend; +import mcpp.build.ninja; +import mcpp.build.plan; +import mcpp.build.prepare; +import mcpp.manifest; +import mcpp.modgraph.graph; +import mcpp.modgraph.scanner; +import mcpp.pack; +import mcpp.pack.abi_tag; +import mcpp.pack.interface; +import mcpp.pack.library; +import mcpp.toolchain.registry; +import mcpp.toolchain.triple; +import mcpp.ui; +import mcpp.version; + +namespace mcpp::pack { + +namespace { + +// The lib-root's own unit. +// +// Both facts the packer needs come from here, and taking them from the SAME +// unit is the point: the module name (mcpp has not required it to match the +// package name since 0.0.10 — the author names the module, the scanner detects +// it) and the package name the scanner stamped on every unit. Recomputing the +// latter from the manifest would be a second derivation of a value the graph +// already carries, and the closure compares against it by string. +const mcpp::modgraph::SourceUnit* root_unit_of(const mcpp::modgraph::Graph& g, + const std::filesystem::path& libRoot) +{ + std::error_code ec; + for (auto const& u : g.units) { + if (!u.provides) continue; + if (std::filesystem::equivalent(u.path, libRoot, ec)) return &u; + } + return nullptr; +} + +// Dependencies the package can honestly carry downstream. +// +// The same rule `mcpp emit xpkg` applies, and for the same reason: a `path` +// or `git` dependency addresses the PRODUCER's disk or a revision only they +// can resolve, so republishing it hands the consumer an address that means +// something else (or nothing) on their machine. Version dependencies are the +// only kind that survive the trip. +std::vector> +publishable_dependencies(const mcpp::manifest::Manifest& m) +{ + std::vector> out; + for (auto const& [k, v] : m.dependencies) { + if (v.isPath() || v.isGit() || v.version.empty()) continue; + out.emplace_back(k, v.version); + } + return out; +} + +std::vector extras_of(const mcpp::manifest::Manifest& m, + const std::filesystem::path& root) +{ + std::vector out; + std::error_code ec; + for (auto const* name : { "README.md", "README", "LICENSE", "LICENSE.txt", "COPYING" }) + if (std::filesystem::is_regular_file(root / name, ec)) out.push_back(root / name); + // `[pack].include` keeps the meaning it has for an application bundle: + // EXTRA files to ship. It has never reached the interface or the headers, + // and it must not start to — those two sets are computed, and letting a + // glob trim them would give the same library a different public surface + // depending on how it was delivered. + for (auto const& glob : m.packConfig.include) + for (auto const& hit : mcpp::modgraph::expand_glob(root, glob)) + if (std::filesystem::is_regular_file(hit, ec)) out.push_back(hit); + return out; +} + +} // namespace + +// `mcpp pack ` for a `kind = "lib"` / `"shared"` target. +// +// `triples` is the `--target` list; empty means "this host". Each entry gets +// its own prepare+build, so the artifacts really are the ones this run made — +// the packer never searches `target/` for something that looks right. +export int build_and_pack_library(const std::string& targetName, + const std::vector& triples, + const mcpp::pack::Options& opts) +{ + std::vector legs = triples; + if (legs.empty()) legs.push_back({}); // one leg, this host + + LibraryPackPlan plan; + plan.builtBy = std::string(mcpp::MCPP_VERSION); + plan.writeArchive = opts.format == mcpp::pack::Format::Tar; + + InterfaceClosure closure; + bool haveClosure = false; + std::string firstTriple; + + for (auto const& want : legs) { + mcpp::build::BuildOverrides ov; + ov.target_triple = want; + auto ctx = mcpp::build::prepare_build(false, /*includeDevDeps=*/false, {}, ov); + if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } + + // ── the target, and what its kind means here ────────────────── + const mcpp::manifest::Target* target = nullptr; + for (auto const& t : ctx->manifest.targets) + if (t.name == targetName) { target = &t; break; } + if (!target) { + std::string names; + for (auto const& t : ctx->manifest.targets) { + if (!names.empty()) names += ", "; + names += t.name; + } + mcpp::ui::error(std::format( + "no target named '{}' in this package{}{}", + targetName, names.empty() ? "" : "; available: ", names)); + return 2; + } + const bool shared = target->kind == mcpp::manifest::Target::SharedLibrary; + + const auto triple = ctx->tc.targetTriple.empty() + ? mcpp::toolchain::triple::host_triple().str() + : [&] { + auto t = mcpp::toolchain::triple::parse(ctx->tc.targetTriple); + return t ? t->str() : ctx->tc.targetTriple; + }(); + + // ── build, then take the artifact FROM THE PLAN ─────────────── + // + // Never a glob over `target/`: that directory holds one subtree per + // fingerprint, and picking one by name or by mtime silently selects a + // stale binary. The link unit knows its own output. + auto be = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions bo; + if (auto br = be->build(ctx->plan, bo); !br) { + mcpp::ui::error(br.error().message); + return 1; + } + std::filesystem::path artifact; + for (auto const& lu : ctx->plan.linkUnits) { + if (lu.targetName != targetName) continue; + if (lu.kind != mcpp::build::LinkUnit::StaticLibrary + && lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + artifact = ctx->outputDir / lu.output; + break; + } + if (artifact.empty()) { + mcpp::ui::error(std::format( + "target '{}' produced no library artifact for {}", targetName, triple)); + return 1; + } + + // ── the interface closure ───────────────────────────────────── + auto libRoot = ctx->projectRoot / mcpp::manifest::resolve_lib_root_path(ctx->manifest); + std::error_code ec; + InterfaceClosure here; + std::string qualifiedPackage; + if (std::filesystem::is_regular_file(libRoot, ec)) { + auto const* rootUnit = root_unit_of(ctx->graph, libRoot); + if (!rootUnit) { + mcpp::ui::error(std::format( + "'{}' is the lib root but provides no module interface", + libRoot.string())); + return 1; + } + qualifiedPackage = rootUnit->packageName; + auto c = interface_closure(ctx->graph, qualifiedPackage, + rootUnit->provides->logicalName); + if (!c) { mcpp::ui::error(c.error()); return 1; } + here = std::move(*c); + } + // else: a header-only package. `sources = []` in the emitted manifest + // says so explicitly, which is a thing a manifest can say now. + + if (!here.unresolvedImports.empty()) { + std::string list; + for (auto const& m : here.unresolvedImports) { + if (!list.empty()) list += ", "; + list += m; + } + mcpp::ui::error(std::format( + "the published interface imports {} , which no unit in this build " + "provides.\n" + " A consumer compiling the published interface would fail with\n" + " \"failed to read compiled module\". Either publish that unit (make it\n" + " an `export module` partition) or keep it out of the interface's purview.", + list)); + return 1; + } + + if (!haveClosure) { + closure = std::move(here); + haveClosure = true; + firstTriple = triple; + plan.projectRoot = ctx->projectRoot; + plan.namespace_ = ctx->manifest.package.namespace_; + plan.packageName = ctx->manifest.package.name; + plan.packageVersion = ctx->manifest.package.version; + plan.targetName = targetName; + plan.targetShared = shared; + plan.cxxRuntime = ctx->manifest.buildConfig.cxxRuntime; + plan.dependencies = publishable_dependencies(ctx->manifest); + plan.extras = extras_of(ctx->manifest, ctx->projectRoot); + plan.interfaceSources = closure.published; + plan.dropObjects = published_object_names(closure); + for (auto const& d : ctx->manifest.buildConfig.includeDirs) { + auto abs = d.is_absolute() ? d : ctx->projectRoot / d; + if (std::filesystem::is_directory(abs, ec)) { plan.includeDir = abs; break; } + } + for (auto const& u : ctx->graph.units) + if (u.provides && u.provides->logicalName.find(':') == std::string::npos + && u.packageName == qualifiedPackage) + plan.exportsModules.push_back(u.provides->logicalName); + std::ranges::sort(plan.exportsModules); + plan.exportsModules.erase( + std::ranges::unique(plan.exportsModules).begin(), plan.exportsModules.end()); + } else if (closure.published != here.published) { + // One package, one `sources` list. If a conditional source glob + // makes the INTERFACE differ per target, the package cannot + // describe itself, and quietly publishing the first leg's answer + // would ship an interface that does not match some of the binaries. + mcpp::ui::error(std::format( + "the published interface differs between {} and {}.\n" + " A package has one `sources` list, so its interface must be the same\n" + " for every target it ships. Move the per-target difference behind\n" + " the implementation, or publish one package per target.", + firstTriple, triple)); + return 1; + } + + // ── the tag ─────────────────────────────────────────────────── + // + // A package whose interface is only headers constrains the libc ABI + // and not the C++ one, so it publishes the shorter tag and links into + // any compiler. The SHAPE is the statement; there is no flag for it. + const bool cxxSurface = !closure.published.empty(); + auto tag = cxxSurface + ? cxx_surface_tag(ctx->tc, triple, ctx->manifest.cppStandard.level) + : c_surface_tag(triple); + + plan.legs.push_back(LibraryLeg{ + .triple = triple, + .artifact = artifact, + .archiveTool = shared ? std::filesystem::path{} + : mcpp::toolchain::archive_tool(ctx->tc), + .abiTag = tag.str(), + .buildKey = ctx->fp.hex, + .linkName = targetName, + .shared = shared, + }); + mcpp::ui::status("Packed leg", std::format("{} [{}]", triple, tag.str())); + } + + // ── where it lands ──────────────────────────────────────────────── + const bool zip = plan.legs.size() == 1 + && plan.legs[0].triple.find("windows") != std::string::npos; + auto dirName = plan.legs.size() == 1 + ? std::format("{}-{}-{}", plan.packageName, plan.packageVersion, plan.legs[0].abiTag) + : std::format("{}-{}", plan.packageName, plan.packageVersion); + plan.zip = zip; + plan.stagingRoot = plan.projectRoot / "target" / "dist" / dirName; + plan.archivePath = plan.projectRoot / "target" / "dist" + / (dirName + (zip ? ".zip" : ".tar.gz")); + if (!opts.output.empty()) { + auto o = std::filesystem::path(opts.output); + plan.archivePath = o.has_parent_path() ? o + : plan.projectRoot / "target" / "dist" / o; + } + + // ── say what travels, and what does not ─────────────────────────── + // + // Both lists, always. "What you are publishing" is only half of what the + // author of a closed-source library needs to see before uploading. + { + std::string pub, held; + for (auto const& p : closure.published) { + if (!pub.empty()) pub += ", "; + pub += p.filename().string(); + } + for (auto const& p : closure.withheld) { + if (!held.empty()) held += ", "; + held += p.filename().string(); + } + mcpp::ui::status("Interface", pub.empty() ? "(headers only)" : pub); + mcpp::ui::status("Withheld", held.empty() ? "(nothing)" : held); + } + + auto out = run_library_pack(plan); + if (!out) { mcpp::ui::error(out.error().message); return 1; } + mcpp::ui::status("Packed", out->string()); + return 0; +} + +} // namespace mcpp::pack diff --git a/src/pack/manifest_emit.cppm b/src/pack/manifest_emit.cppm new file mode 100644 index 00000000..5bdbf045 --- /dev/null +++ b/src/pack/manifest_emit.cppm @@ -0,0 +1,236 @@ +// mcpp.pack.manifest_emit — the `mcpp.toml` that travels inside a library +// package. +// +// IT IS AN ORDINARY MANIFEST, AND THAT IS THE DESIGN +// +// Two earlier drafts of this format invented somewhere to put the packaging +// facts: first a sibling `MCPP-PACKAGE.toml`, then a `[distribution]` section. +// Both were deleted, because every fact already had a home: +// +// what it is where it goes already parsed by +// ----------------- ----------------------------------- ----------------- +// the interface [build] sources / include_dirs yes +// how to link it [target.'cfg(...)'.build] ldflags yes +// its dependencies [dependencies] yes +// which modules [modules] exports yes +// the C++ runtime [build] cxx_runtime yes +// the artifacts [[runtime.artifacts]] yes +// +// So a consumer needs no new code path to USE one of these packages: mcpp's +// existing "the payload carries its own mcpp.toml" route (Form A) reads it, +// whether it arrives as a path dependency, a git dependency, a file, or an +// index tarball. And an mcpp too old to run the gate still BUILDS against the +// package, because none of the keys are new — it just does not check them. +// A new section would have been silently skipped instead, leaving no record +// at all; `[[runtime.artifacts]]` is a section old clients already write into +// `resolution.json`. +// +// WHAT IS EVIDENCE, AND WHY IT LOOKS LIKE THE REST +// +// `[[runtime.artifacts]]` carries `role`, `abi`, `digest`, `provenance` and +// `host_fingerprint` — the exact fields the gate needs, and documented as +// "optional evidence" since they were introduced. `provenance` starting with +// `mcpp-pack` is what MARKS a directory as a distribution package; nothing +// else needs to say so. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2.4. + +export module mcpp.pack.manifest_emit; + +import std; + +export namespace mcpp::pack { + +// One target triple's worth of the package: the artifact built for it and the +// conditional block that selects it. +struct PackageLeg { + std::string triple; // canonical, e.g. "x86_64-linux-gnu" + std::string libFile; // "libmathkit.a" — as it sits in lib// + std::string linkName; // "mathkit" — the -l argument + std::string abiTag; // "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" + std::string digest; // "sha256:…" + std::string buildKey; // cache_key::key_hex, or empty + bool shared = false; +}; + +struct PackageDoc { + std::string namespace_; + std::string name; + std::string version; + std::string builtBy; // "mcpp 2026.8.17.1" + + // Package-relative, e.g. "interface/mathkit.cppm". Empty is legal and + // meaningful: a header-only package compiles nothing, and the emitted + // `sources = []` says exactly that (an omitted key would be filled with + // the default glob and would sweep up whatever sits under src/). + std::vector interfaceFiles; + bool hasIncludeDir = false; + std::string interfaceDigest; // over the ordered interface set + + std::string cxxRuntime; // empty = do not write the key + std::vector exportsModules; + + std::string targetName; + bool targetShared = false; + + std::vector legs; + // Raw `[dependencies]` keys as the producer wrote them, paired with the + // version. Path/git deps are the caller's to drop — they are local-only + // and cannot be resolved by anyone downstream. + std::vector> dependencies; +}; + +// The cfg() predicate that selects exactly `triple`. +// +// ⚠️ NOT a bare `[target.''.build]` key. A bare triple is only matched +// when the user passes `--target`; a plain `mcpp build` resolves the host and +// used to compare it against an empty string, so the section was silently +// inert. That is fixed (mcpp.build.prepare_inputs), but generating cfg() is +// still the right output: it is what the same statement means on every mcpp, +// including the ones already installed. +std::string cfg_predicate_for(std::string_view triple); + +std::string emit_package_manifest(const PackageDoc& doc); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +namespace { + +// TOML basic-string escaping, restricted to what a manifest can contain. +std::string quote(std::string_view s) { + std::string out = "\""; + for (char c : s) { + switch (c) { + case '"': out += "\\\""; break; + case '\\': out += "\\\\"; break; + case '\n': out += "\\n"; break; + case '\t': out += "\\t"; break; + default: out += c; break; + } + } + out += '"'; + return out; +} + +std::string join_quoted(const std::vector& v) { + std::string out; + for (std::size_t i = 0; i < v.size(); ++i) { + if (i) out += ", "; + out += quote(v[i]); + } + return out; +} + +} // namespace + +std::string cfg_predicate_for(std::string_view triple) { + // arch-os[-env]; the canonical spelling from mcpp.toolchain.triple. + std::vector seg; + for (std::size_t i = 0; i <= triple.size(); ) { + auto j = triple.find('-', i); + if (j == std::string_view::npos) { seg.push_back(triple.substr(i)); break; } + seg.push_back(triple.substr(i, j - i)); + i = j + 1; + } + if (seg.size() < 2) return std::format("cfg(arch = \"{}\")", triple); + + std::string p = std::format("cfg(all(arch = \"{}\", os = \"{}\"", seg[0], seg[1]); + // env is named only when the triple names it. Writing `env = ""` would be + // a constraint the triple did not make, and the evaluator has no spelling + // for "unset" anyway. + if (seg.size() >= 3 && !seg[2].empty()) + p += std::format(", env = \"{}\"", seg[2]); + p += "))"; + return p; +} + +std::string emit_package_manifest(const PackageDoc& doc) { + std::string o; + o += std::format( + "# Generated by `{}`. Do not edit.\n" + "#\n" + "# Editing anything under interface/ invalidates the digest recorded in\n" + "# the `role = \"interface\"` artifact below, and mcpp refuses to build\n" + "# against a package whose interface no longer matches its binaries.\n\n", + doc.builtBy.empty() ? "mcpp pack" : doc.builtBy); + + o += "[package]\n"; + if (!doc.namespace_.empty()) o += std::format("namespace = {}\n", quote(doc.namespace_)); + o += std::format("name = {}\n", quote(doc.name)); + o += std::format("version = {}\n\n", quote(doc.version)); + + // ── the interface ────────────────────────────────────────────────── + o += "[build]\n"; + o += std::format("sources = [{}]\n", join_quoted(doc.interfaceFiles)); + if (doc.hasIncludeDir) o += "include_dirs = [\"include\"]\n"; + if (!doc.cxxRuntime.empty()) + o += std::format("cxx_runtime = {}\n", quote(doc.cxxRuntime)); + o += "\n"; + + if (!doc.exportsModules.empty()) { + o += "[modules]\n"; + o += std::format("exports = [{}]\n\n", join_quoted(doc.exportsModules)); + } + + o += std::format("[targets.{}]\n", doc.targetName); + o += std::format("kind = \"{}\"\n\n", doc.targetShared ? "shared" : "lib"); + + // ── how to link each leg ─────────────────────────────────────────── + for (auto const& leg : doc.legs) { + o += std::format("[target.'{}'.build]\n", cfg_predicate_for(leg.triple)); + o += std::format("ldflags = [\"-Llib/{}\", \"-l{}\"]\n\n", leg.triple, leg.linkName); + } + // A shared library has to be FOUND at run time as well as linked, and the + // two are different search paths — `link_library_dirs` is not rpath. + if (doc.targetShared && !doc.legs.empty()) { + o += "[runtime]\n"; + std::vector dirs; + for (auto const& leg : doc.legs) dirs.push_back(std::format("lib/{}", leg.triple)); + o += std::format("runtime_search_dirs = [{}]\n\n", join_quoted(dirs)); + } + + // ── dependencies ─────────────────────────────────────────────────── + // + // A static archive does NOT carry its dependencies' code, so a consumer + // that does not resolve them cannot link. Carrying the producer's + // `[dependencies]` verbatim is what makes the package usable at all. + if (!doc.dependencies.empty()) { + o += "[dependencies]\n"; + for (auto const& [k, v] : doc.dependencies) + o += std::format("{} = {}\n", quote(k), quote(v)); + o += "\n"; + } + + // ── evidence ─────────────────────────────────────────────────────── + for (auto const& leg : doc.legs) { + o += "[[runtime.artifacts]]\n"; + o += std::format("role = \"{}\"\n", + leg.shared ? "shared-library" : "static-library"); + o += std::format("path = {}\n", + quote(std::format("lib/{}/{}", leg.triple, leg.libFile))); + o += std::format("provenance = {}\n", + quote(std::format("mcpp-pack {}", doc.builtBy))); + if (!leg.abiTag.empty()) o += std::format("abi = {}\n", quote(leg.abiTag)); + if (!leg.digest.empty()) o += std::format("digest = {}\n", quote(leg.digest)); + if (!leg.buildKey.empty()) o += std::format("host_fingerprint = {}\n", quote(leg.buildKey)); + o += "\n"; + } + if (!doc.interfaceDigest.empty()) { + // One entry for the whole set, not one per file. What it defends + // against is post-extraction editing, and "interface/ no longer + // matches" is already actionable; per-file digests would add a line + // per file to a document whose whole point is to stay readable. + o += "[[runtime.artifacts]]\n"; + o += "role = \"interface\"\n"; + o += std::format("path = {}\n", + quote(doc.interfaceFiles.empty() ? "include" : "interface")); + o += std::format("provenance = {}\n", + quote(std::format("mcpp-pack {}", doc.builtBy))); + o += std::format("digest = {}\n\n", quote(doc.interfaceDigest)); + } + return o; +} + +} // namespace mcpp::pack diff --git a/src/pack/prebuilt.cppm b/src/pack/prebuilt.cppm new file mode 100644 index 00000000..0e6e5b89 --- /dev/null +++ b/src/pack/prebuilt.cppm @@ -0,0 +1,166 @@ +// mcpp.pack.prebuilt — the consumer's half of a library package. +// +// A package produced by `mcpp pack` is an ordinary mcpp package: it carries a +// normal `mcpp.toml`, and mcpp's existing "the payload has its own manifest" +// route builds against it with no new code. That is the design, and it is why +// an mcpp too old to know about any of this still WORKS with these packages. +// +// What this module adds is the part an old mcpp cannot do: check that the +// binaries in the package were built for the toolchain about to link them, and +// that the interface sitting next to them is still the one they were built +// from. Both failures are otherwise silent. +// +// THE SECOND ONE IS THE DANGEROUS ONE. Measured on a real build: change one +// line of a shipped interface — swap two `int` members of a struct, which the +// Itanium ABI does not mangle — and the consumer compiles, links, runs, and +// prints transposed data. No diagnostic at any stage. A digest cannot prevent +// a producer from shipping a mismatched pair in the first place (only atomic +// production does that), but it does catch the pair coming apart afterwards, +// which is the case a path dependency or an extracted store is exposed to. +// +// WHAT MARKS A PACKAGE. `provenance` beginning with `mcpp-pack` on any runtime +// artifact. No new key, no new section: the marker is a value in a field that +// has existed since runtime artifacts did. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §3.2. + +export module mcpp.pack.prebuilt; + +import std; +import mcpp.manifest; +import mcpp.pack.abi_tag; +import mcpp.pack.digest; + +export namespace mcpp::pack { + +inline constexpr std::string_view kPackProvenancePrefix = "mcpp-pack"; + +// Was this manifest written by `mcpp pack`? +bool is_distribution_package(const mcpp::manifest::Manifest& m); + +struct PrebuiltCheck { + std::filesystem::path packageRoot; + std::string packageLabel; // "acme.mathkit@0.1.0", for diagnostics + AbiTag current; // the tag THIS build would publish +}; + +// Refuse, with a message the reader can act on, or accept. +// +// Order matters and is the order of the diagnostic: a package that is for +// another architecture entirely should say so before it complains about a +// digest, because the digest is not what the user has to fix. +std::expected +check_prebuilt(const mcpp::manifest::Manifest& m, const PrebuiltCheck& in); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +namespace { + +bool is_library_role(std::string_view role) { + return role == "static-library" || role == "shared-library"; +} + +} // namespace + +bool is_distribution_package(const mcpp::manifest::Manifest& m) { + for (auto const& a : m.runtimeConfig.artifacts) + if (a.provenance.starts_with(kPackProvenancePrefix)) return true; + return false; +} + +std::expected +check_prebuilt(const mcpp::manifest::Manifest& m, const PrebuiltCheck& in) +{ + std::error_code ec; + + // ── 1. the artifacts are where the manifest says ────────────────── + // + // A package whose `lib/` was trimmed in transit links against nothing and + // fails at the linker, naming a path nobody recognises. + for (auto const& a : m.runtimeConfig.artifacts) { + if (!a.provenance.starts_with(kPackProvenancePrefix)) continue; + auto abs = a.path.is_absolute() ? a.path : in.packageRoot / a.path; + if (!std::filesystem::exists(abs, ec)) { + return std::unexpected(std::format( + "{}: the package declares an artifact at '{}', and it is not there.\n" + " The package is incomplete — re-download or re-pack it.", + in.packageLabel, a.path.string())); + } + } + + // ── 2. one of the published tags accepts this toolchain ─────────── + std::vector publishedTags; + bool sawLibrary = false, accepted = false; + std::vector bestRefusal; + std::string bestRefusalTag; + + for (auto const& a : m.runtimeConfig.artifacts) { + if (!a.provenance.starts_with(kPackProvenancePrefix)) continue; + if (!is_library_role(a.role)) continue; + sawLibrary = true; + if (a.abi.empty()) { // nothing declared → nothing to enforce + accepted = true; + continue; + } + publishedTags.push_back(a.abi); + auto published = parse_abi_tag(a.abi); + if (!published) { accepted = true; continue; } // unreadable → lenient + auto bad = tag_check(*published, in.current); + if (bad.empty()) { accepted = true; break; } + // Keep the CLOSEST refusal to show: the one that disagrees least is + // the one the user is most likely able to act on. + if (bestRefusal.empty() || bad.size() < bestRefusal.size()) { + bestRefusal = bad; + bestRefusalTag = a.abi; + } + } + + if (sawLibrary && !accepted) { + std::string tags; + for (auto const& t : publishedTags) tags += std::format("\n {}", t); + std::string why; + for (auto const& b : bestRefusal) + why += std::format("\n {:<9} needs {}, this build has {}", b.dimension, b.need, b.got); + return std::unexpected(std::format( + "{}: no prebuilt artifact matches this toolchain.\n" + " your toolchain : {}\n" + " published tags :{}\n" + " closest is {}, and it differs on:{}\n" + " fix: ask the publisher for a build matching your toolchain, or pin\n" + " [toolchain] to the one the package was built with.", + in.packageLabel, in.current.str(), tags, bestRefusalTag, why)); + } + + // ── 3. the interface is the one the binaries were built from ────── + for (auto const& a : m.runtimeConfig.artifacts) { + if (!a.provenance.starts_with(kPackProvenancePrefix)) continue; + if (a.role != "interface" || a.digest.empty()) continue; + + auto dir = a.path.is_absolute() ? a.path : in.packageRoot / a.path; + std::vector files; + if (std::filesystem::is_directory(dir, ec)) { + for (auto const& e : std::filesystem::recursive_directory_iterator(dir, ec)) + if (e.is_regular_file()) files.push_back(e.path()); + } else if (std::filesystem::is_regular_file(dir, ec)) { + files.push_back(dir); + } + auto now = interface_set_digest(files); + if (now != a.digest) { + return std::unexpected(std::format( + "{}: '{}' does not match what was packaged.\n" + " recorded {}\n" + " found {}\n" + " The published interface and the prebuilt binaries are produced\n" + " together and are not separately replaceable: an edited interface\n" + " compiles and links against binaries that no longer agree with it,\n" + " and the result is wrong at run time with no diagnostic.\n" + " fix: restore the package from its original archive.", + in.packageLabel, a.path.string(), a.digest, now)); + } + } + return {}; +} + +} // namespace mcpp::pack diff --git a/src/pack/route.cppm b/src/pack/route.cppm new file mode 100644 index 00000000..dde0204c --- /dev/null +++ b/src/pack/route.cppm @@ -0,0 +1,105 @@ +// mcpp.pack.route — which target `mcpp pack` packs, and therefore which of the +// two pipelines runs. +// +// The routing question has exactly one input: `[targets.].kind`. A program +// becomes an application bundle, a library becomes a library package. That is +// why there is no `--lib` flag and no `--artifact static|shared` — every one +// of those would be a second place to answer a question the manifest already +// answers, and the two answers could then disagree. +// +// Reading the manifest here (rather than inside each pipeline) keeps the +// decision ahead of the build: `mcpp pack nosuch` should say so in +// milliseconds, not after compiling the project. + +module; +#include + +export module mcpp.pack.route; + +import std; +import mcpp.manifest; +import mcpp.project; + +export namespace mcpp::pack { + +struct PackRoute { + std::string targetName; + bool library = false; // kind = lib | shared +}; + +// Resolve `requested` (possibly empty) against the current project. +// +// Empty picks the only packable target. A project with both a program and a +// library has no single obvious answer, so it is asked rather than guessed: +// packing the wrong one produces a plausible-looking archive of the wrong +// shape, which is worse than an error. +std::expected route_pack_target(std::string_view requested); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::expected route_pack_target(std::string_view requested) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) return std::unexpected("no mcpp.toml in current dir or parents"); + auto m = mcpp::manifest::load(*root / "mcpp.toml"); + if (!m) return std::unexpected(m.error().format()); + + auto is_library = [](const mcpp::manifest::Target& t) { + return t.kind == mcpp::manifest::Target::Library + || t.kind == mcpp::manifest::Target::SharedLibrary; + }; + auto kind_name = [](const mcpp::manifest::Target& t) -> std::string_view { + switch (t.kind) { + case mcpp::manifest::Target::Binary: return "bin"; + case mcpp::manifest::Target::Library: return "lib"; + case mcpp::manifest::Target::SharedLibrary: return "shared"; + case mcpp::manifest::Target::TestBinary: return "test"; + } + return "?"; + }; + + if (!requested.empty()) { + for (auto const& t : m->targets) { + if (t.name != requested) continue; + if (t.kind == mcpp::manifest::Target::TestBinary) + return std::unexpected(std::format( + "target '{}' is a test binary; there is nothing to distribute", + requested)); + return PackRoute{ t.name, is_library(t) }; + } + std::string list; + for (auto const& t : m->targets) { + if (!list.empty()) list += ", "; + list += std::format("{} ({})", t.name, kind_name(t)); + } + return std::unexpected(std::format( + "no target named '{}'{}{}", requested, + list.empty() ? "" : "; this package declares: ", list)); + } + + // Nothing requested. A program is still the default — `mcpp pack` has + // always meant "bundle this application" and a project that has one is + // asking for that. + const mcpp::manifest::Target* onlyLib = nullptr; + std::size_t libCount = 0; + for (auto const& t : m->targets) { + if (t.kind == mcpp::manifest::Target::Binary) return PackRoute{ t.name, false }; + if (is_library(t)) { onlyLib = &t; ++libCount; } + } + if (libCount == 1) return PackRoute{ onlyLib->name, true }; + if (libCount == 0) + return std::unexpected("this package declares no program and no library to pack"); + + std::string list; + for (auto const& t : m->targets) { + if (!is_library(t)) continue; + if (!list.empty()) list += ", "; + list += std::format("{} ({})", t.name, kind_name(t)); + } + return std::unexpected(std::format( + "this package declares more than one library, so `mcpp pack` cannot pick " + "one for you.\n Name it: {}", list)); +} + +} // namespace mcpp::pack diff --git a/src/source_kind.cppm b/src/source_kind.cppm index 1ec14efd..28992404 100644 --- a/src/source_kind.cppm +++ b/src/source_kind.cppm @@ -187,6 +187,17 @@ enum class ObjectNaming { // than weakening the exhaustive no-collision assertion around it. ObjectNaming object_naming_for(const std::filesystem::path& src); +// The object file's NAME, formatted from the policy above. +// +// It lived in plan.cppm as an internal helper, under a comment saying the +// policy lives here and it "only formats it" — which is exactly the split +// worth closing: a second reader of the policy is a second place the format +// can drift. `mcpp pack` needs the same answer (to tell which archive members +// belong to the interface units it is publishing as source), and reaching into +// plan.cppm for it would drag the whole planner into the packer. +std::string object_filename_for(const std::filesystem::path& src, + std::string_view objExt = ".o"); + } // namespace mcpp namespace mcpp { @@ -369,4 +380,22 @@ ObjectNaming object_naming_for(const std::filesystem::path& src) { return ObjectNaming::FullFilename; } +std::string object_filename_for(const std::filesystem::path& src, + std::string_view objExt) { + switch (object_naming_for(src)) { + case ObjectNaming::StemDotM: + return src.stem().string() + ".m" + std::string(objExt); + case ObjectNaming::Stem: + return src.stem().string() + std::string(objExt); + case ObjectNaming::FullFilename: + break; + } + // Assembly siblings of a C/C++ TU commonly share its stem (foo.c + + // foo.asm); keeping the full extension means they can never collide — + // the per-package collision prefix can't help two same-stem files in the + // same directory. Every extension a project adds via + // `[build] module_extensions` lands here for the same reason. + return src.filename().string() + std::string(objExt); +} + } // namespace mcpp diff --git a/src/version.cppm b/src/version.cppm index 13ca3351..a4be1f92 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.17.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.17.2"; } // namespace mcpp diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh new file mode 100755 index 00000000..d2110dca --- /dev/null +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# requires: gcc +# 242_pack_library_interface_and_headers.sh — `mcpp pack ` produces +# a package a consumer can use through EITHER interface mode, or both at once. +# +# Acceptance for §1.1 / §2 of +# .agents/docs/2026-08-17-library-distribution-design.md: a package carries +# `include/` (text, consumed by #include) and `interface/` (module units the +# consumer compiles), the two do not interfere, and neither needs a flag. +# +# Also pins the two lists `mcpp pack` prints. A closed-source publisher needs +# to see what is NOT travelling as much as what is. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# ── the producer: a module interface AND a C header, over one library ── +mkdir -p mathkit/src mathkit/include +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export import :api; +EOF +cat > mathkit/src/api.cppm <<'EOF' +export module mathkit:api; +export namespace mk { int add(int a, int b); } +EOF +cat > mathkit/src/secret.cppm <<'EOF' +module mathkit:secret; +namespace mk { int bias() { return 0; } } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { +int bias(); +int add(int a, int b) { return a + b + bias(); } +} +EOF +cat > mathkit/src/capi.c <<'EOF' +int mathkit_add(int a, int b) { return a + b; } +EOF +cat > mathkit/include/mathkit_c.h <<'EOF' +#ifdef __cplusplus +extern "C" { +#endif +int mathkit_add(int a, int b); +#ifdef __cplusplus +} +#endif +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp", "src/*.c"] +include_dirs = ["include"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } + +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ -n "$pkg" ]] || { cat pack.log; echo "no package directory"; exit 1; } + +# The layout is the contract: two interface modes, one artifact dir per triple. +for f in mcpp.toml interface/mathkit.cppm interface/api.cppm include/mathkit_c.h; do + [[ -f "$pkg/$f" ]] || { echo "package is missing $f"; find "$pkg" -type f; exit 1; } +done +[[ -n "$(find "$pkg/lib" -name 'libmathkit.a' | head -1)" ]] || { + echo "package has no artifact under lib//"; find "$pkg" -type f; exit 1; } + +# Both lists are printed, and the implementation partition is on the right one. +grep -q 'Interface.*mathkit.cppm' pack.log || { cat pack.log; echo "no interface list"; exit 1; } +grep -q 'Withheld.*secret.cppm' pack.log || { cat pack.log; echo "no withheld list"; exit 1; } + +cd "$TMP" + +# ── three consumers: header only, module only, both ──────────────────── +consume() { # $1 = name, $2 = main.cpp body + mkdir -p "$1/src" + printf '%s' "$2" > "$1/src/main.cpp" + cat > "$1/mcpp.toml" < run.log 2>&1 ) || { cat "$1/run.log"; echo "$1 failed"; exit 1; } + grep -q 'ok=5' "$1/run.log" || { cat "$1/run.log"; echo "$1 printed the wrong answer"; exit 1; } +} + +consume c_hdr '#include +#include +int main(){ std::printf("ok=%d\n", mathkit_add(2,3)); return 0; } +' +consume c_mod '#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::add(2,3)); return 0; } +' +consume c_both '#include +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mathkit_add(2,3) + mk::add(2,3) - 5); return 0; } +' + +echo "PASS: a library package serves header, module, and both-at-once consumers" diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh new file mode 100755 index 00000000..bed4c250 --- /dev/null +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# requires: gcc +# 243_pack_library_interface_closure.sh — what travels is the module closure of +# the published root, and the archive keeps exactly what the closure does not. +# +# Two asymmetric failures are pinned here, and only one of them is loud in the +# wild: +# +# * publishing TOO LITTLE fails in the consumer's compile, naming the module; +# * publishing TOO MUCH silently ships a closed-source implementation +# partition's SOURCE. Nothing fails. That is what this test is for. +# +# The archive side is the same closure used the other way round, and getting it +# wrong was measured: dropping every `.m.o` also drops the implementation +# partition's object, and every target then fails to link with an undefined +# reference nowhere near its cause. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export import :api; +EOF +cat > mathkit/src/api.cppm <<'EOF' +export module mathkit:api; +export namespace mk { int answer(); } +EOF +# An implementation partition: it produces a BMI and a `.m.o`, and its source +# is the thing a closed-source publisher must not ship. +cat > mathkit/src/secret.cppm <<'EOF' +module mathkit:secret; +namespace mk { int secret_bias() { return 40; } } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { +int secret_bias(); +int answer() { return secret_bias() + 2; } +} +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" + +# ── the confidentiality criterion ────────────────────────────────────── +[[ -f "$pkg/interface/mathkit.cppm" ]] || { echo "root interface not published"; exit 1; } +[[ -f "$pkg/interface/api.cppm" ]] || { echo "interface partition not published"; exit 1; } +[[ ! -e "$pkg/interface/secret.cppm" ]] || { + echo "LEAK: the implementation partition's source was published"; exit 1; } +grep -RIl 'secret_bias' "$pkg/interface" "$pkg/include" 2>/dev/null | grep -q . && { + echo "LEAK: implementation source text found in the published interface"; exit 1; } + +# ── the archive criterion: published objects out, everything else in ──── +ar_bin="$(command -v ar || true)" +if [[ -n "$ar_bin" ]]; then + members="$(ar t "$(find "$pkg/lib" -name 'libmathkit.a' | head -1)")" + echo "$members" | grep -q 'secret.m.o' || { + echo "the implementation partition's OBJECT was dropped; nothing would link" + echo "$members"; exit 1; } + echo "$members" | grep -q 'mathkit.m.o' && { + echo "a published interface unit's object is still in the archive" + echo "$members"; exit 1; } + echo "$members" | grep -q 'api.m.o' && { + echo "a published interface unit's object is still in the archive" + echo "$members"; exit 1; } +fi + +# ── and it still links and runs ───────────────────────────────────────── +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed"; exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } + +echo "PASS: the closure decides both what is published and what is dropped" diff --git a/tests/e2e/244_pack_library_gate.sh b/tests/e2e/244_pack_library_gate.sh new file mode 100755 index 00000000..895e46b9 --- /dev/null +++ b/tests/e2e/244_pack_library_gate.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash +# requires: gcc +# 244_pack_library_gate.sh — the three refusals a prebuilt package must make. +# +# THE FIRST ONE IS WHY THIS FEATURE HAS A GATE AT ALL. Measured before it +# existed, on a real build: change one line of a shipped interface — swap two +# `int` members of a struct, which the Itanium ABI does not mangle — and the +# consumer compiles, links, runs, and prints transposed data. Exit code 0. No +# diagnostic at any stage, from any tool. +# +# 1. an edited interface → refuse (the silent-wrong-data case) +# 2. an artifact for another toolchain → refuse, and LIST the tags it has +# 3. `mcpp build` inside the package → refuse (it would "succeed" emptily) +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +cd "$TMP" + +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < ok.log 2>&1 ) || { cat app/ok.log; echo "baseline failed"; exit 1; } +grep -q 'ok=42' app/ok.log || { cat app/ok.log; echo "baseline wrong answer"; exit 1; } + +# ── 1. an edited interface is refused ────────────────────────────────── +cp "$pkg/interface/mathkit.cppm" "$TMP/interface.bak" +printf '\n// tampered\n' >> "$pkg/interface/mathkit.cppm" +rm -rf app/target +if ( cd app && "$MCPP" build > tamper.log 2>&1 ); then + cat app/tamper.log + echo "FAIL: an edited interface built anyway — this is the silent-wrong-data case" + exit 1 +fi +grep -q 'does not match what was packaged' app/tamper.log || { + cat app/tamper.log; echo "refused, but not for the interface digest"; exit 1; } +cp "$TMP/interface.bak" "$pkg/interface/mathkit.cppm" + +# ── 2. a foreign toolchain tag is refused, and the real tags are shown ── +cp "$pkg/mcpp.toml" "$TMP/manifest.bak" +sed -i.bak 's/-gcc\([0-9][0-9]*\)-/-gcc999-/' "$pkg/mcpp.toml" +rm -rf app/target +if ( cd app && "$MCPP" build > tag.log 2>&1 ); then + cat app/tag.log + echo "FAIL: a package built for another compiler was accepted" + exit 1 +fi +grep -q 'no prebuilt artifact matches this toolchain' app/tag.log || { + cat app/tag.log; echo "refused, but not for the abi tag"; exit 1; } +# The diagnostic has to say what IS available — a refusal the reader cannot act +# on sends them looking for a package that is right in front of them. +grep -q 'published tags' app/tag.log || { + cat app/tag.log; echo "the refusal did not list the published tags"; exit 1; } +grep -q 'gcc999' app/tag.log || { + cat app/tag.log; echo "the refusal did not name the tag it found"; exit 1; } +cp "$TMP/manifest.bak" "$pkg/mcpp.toml" + +# ── 3. building INSIDE the package is refused ────────────────────────── +if ( cd "$pkg" && "$MCPP" build > "$TMP/inside.log" 2>&1 ); then + cat "$TMP/inside.log" + echo "FAIL: building inside a distribution package 'succeeded' — it compiles" + echo " declarations, links nothing, and reports Finished" + exit 1 +fi +grep -q 'distribution package produced by' "$TMP/inside.log" || { + cat "$TMP/inside.log"; echo "refused, but not as a distribution package"; exit 1; } + +# ── and the restored package still builds ────────────────────────────── +rm -rf app/target +( cd app && "$MCPP" run > final.log 2>&1 ) || { cat app/final.log; echo "restore failed"; exit 1; } +grep -q 'ok=42' app/final.log || { cat app/final.log; echo "restore wrong answer"; exit 1; } + +echo "PASS: interface tamper, tag mismatch, and in-package build are all refused" diff --git a/tests/e2e/245_pack_library_fat_target_selection.sh b/tests/e2e/245_pack_library_fat_target_selection.sh new file mode 100755 index 00000000..bfc17513 --- /dev/null +++ b/tests/e2e/245_pack_library_fat_target_selection.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# requires: gcc +# 245_pack_library_fat_target_selection.sh — one package, several targets, and +# each build picks exactly its own leg. +# +# ⚠️ THE PREDICATE IS THE POINT. Each leg is selected by a generated +# `cfg(all(arch=…, os=…, env=…))` block and NOT by a bare `[target.'']` +# key. The bare form only matched when `--target` was passed: a plain +# `mcpp build` resolved the host and compared it against an empty string, so +# the section was silently inert. That shape is the worst kind — CI passes +# `--target` and is green, the developer's own build drops the flags, and the +# failure arrives at the linker naming a symbol instead of a predicate. +# +# So this test builds the NATIVE case as well as the explicit ones, and asserts +# the selected directory each time. +# +# gnu + musl and not gnu + windows: CI warms both of those toolchains, so this +# test RUNS rather than skipping. A `# requires: mingw-cross` here would have +# made the whole fat-package mechanism unverified on every ordinary CI run +# while still reporting a green suite. The PE leg is covered by 248. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit \ + --target x86_64-linux-gnu \ + --target x86_64-linux-musl > pack.log 2>&1 \ + || { cat pack.log; echo "fat pack failed"; exit 1; } + +pkg="$TMP/mathkit/target/dist/mathkit-0.1.0" +for t in x86_64-linux-gnu x86_64-linux-musl; do + [[ -n "$(find "$pkg/lib/$t" -name 'libmathkit.a' | head -1)" ]] || { + echo "no artifact for $t"; find "$pkg" -type f; exit 1; } +done + +# The generated blocks must be cfg(...), never a bare triple. +grep -q "target\.'cfg(" "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "legs are not selected by cfg() predicates"; exit 1; } +grep -qE "^\[target\.'x86_64-" "$pkg/mcpp.toml" && { + cat "$pkg/mcpp.toml" + echo "a leg is selected by a BARE TRIPLE, which is inert on a native build" + exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < "$TMP/$label.log" 2>&1 ) \ + || { cat "$TMP/$label.log"; echo "$label build failed"; exit 1; } + local nj; nj="$(find app/target -name build.ninja | head -1)" + grep -o "dist/mathkit-0.1.0/lib/[a-z0-9_-]*" "$nj" | sort -u > "$TMP/$label.legs" + [[ "$(wc -l < "$TMP/$label.legs")" -eq 1 ]] || { + echo "$label saw more than one leg:"; cat "$TMP/$label.legs"; exit 1; } + grep -q "lib/$want\$" "$TMP/$label.legs" || { + echo "$label picked the wrong leg:"; cat "$TMP/$label.legs"; exit 1; } +} + +# The native build is the one the bare-triple form used to get wrong. +check native x86_64-linux-gnu +check gnu x86_64-linux-gnu --target x86_64-linux-gnu +check musl x86_64-linux-musl --target x86_64-linux-musl + +echo "PASS: a fat package selects one leg per target, native build included" diff --git a/tests/e2e/246_explicit_empty_sources.sh b/tests/e2e/246_explicit_empty_sources.sh new file mode 100755 index 00000000..b2657dc1 --- /dev/null +++ b/tests/e2e/246_explicit_empty_sources.sh @@ -0,0 +1,62 @@ +#!/usr/bin/env bash +# requires: gcc +# 246_explicit_empty_sources.sh — `sources = []` means "compile nothing", and +# omitting the key still means "the default glob". +# +# They used to be byte-identical: the parser filled the default glob whenever +# the vector was empty, so an author had NO spelling for "nothing". A binary +# distribution package needs one — a header-only package compiles nothing, and +# any file left under `src/` would otherwise be swept up and compiled into the +# consumer's build, where it can collide with the symbols the prebuilt library +# already defines. +# +# The probe is a leftover source that must NOT be compiled in one case and MUST +# be in the other. Asserting only the first would pass against a parser that +# ignores `sources` entirely. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p probe/src +echo 'int mcpp_leftover_probe(void) { return 1; }' > probe/src/leftover.cpp +echo 'int main() { return 0; }' > probe/main.cpp + +write_manifest() { # $1 = the [build] body + cat > probe/mcpp.toml < empty.log 2>&1 ) || { cat probe/empty.log; echo "build failed"; exit 1; } +n="$(hits)" +[[ "$n" -eq 0 ]] || { + echo "FAIL: 'sources = []' still compiled the leftover source ($n references)" + exit 1; } + +# ── absent: the default glob still applies ───────────────────────────── +write_manifest '# no sources key at all' +rm -rf probe/target +( cd probe && "$MCPP" build > default.log 2>&1 ) || { cat probe/default.log; echo "build failed"; exit 1; } +n="$(hits)" +[[ "$n" -gt 0 ]] || { + echo "FAIL: omitting 'sources' stopped applying the default glob" + exit 1; } + +echo "PASS: an explicitly empty sources list is distinguishable from an absent one" diff --git a/tests/e2e/247_bare_triple_conditional_native.sh b/tests/e2e/247_bare_triple_conditional_native.sh new file mode 100755 index 00000000..714aabc8 --- /dev/null +++ b/tests/e2e/247_bare_triple_conditional_native.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# requires: gcc +# 247_bare_triple_conditional_native.sh — `[target.''.build]` applies to +# a NATIVE build, not only to one with an explicit `--target`. +# +# It did not, and the failure shape is the dangerous one: CI passes `--target` +# and is green, the developer's plain `mcpp build` silently drops the section, +# and whatever it carried (a `-L`, a define, a source) goes missing somewhere +# far from the manifest. `cfg(linux)` matched the same build all along, so the +# two spellings of one statement disagreed. +# +# The probe asserts BOTH spellings under BOTH invocations. Asserting only the +# native case would pass against a build that applies every section always. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +host_triple="$("$MCPP" self env 2>/dev/null | grep -oE '[a-z0-9_]+-(linux|macos|windows)-[a-z0-9]+' | head -1)" +[[ -n "$host_triple" ]] || host_triple="x86_64-linux-gnu" + +mkdir -p probe/src +echo 'int main() { return 0; }' > probe/src/main.cpp +cat > probe/mcpp.toml < native.log 2>&1 || { cat native.log; echo "native build failed"; exit 1; } +bare="$(count MCPP_BARE_TRIPLE)" +alias_="$(count MCPP_CFG_ALIAS)" +[[ "$alias_" -gt 0 ]] || { echo "cfg() did not apply on a native build"; exit 1; } +[[ "$bare" -gt 0 ]] || { + echo "FAIL: [target.'$host_triple'.build] was inert on a native build" + echo " (cfg() applied $alias_ times, the bare triple $bare)" + exit 1; } + +# ── explicit --target: unchanged ─────────────────────────────────────── +rm -rf target +"$MCPP" build --target "$host_triple" > explicit.log 2>&1 \ + || { cat explicit.log; echo "explicit build failed"; exit 1; } +[[ "$(count MCPP_BARE_TRIPLE)" -gt 0 ]] || { echo "bare triple inert with --target"; exit 1; } +[[ "$(count MCPP_CFG_ALIAS)" -gt 0 ]] || { echo "cfg() inert with --target"; exit 1; } + +echo "PASS: a bare-triple conditional applies to native and explicit builds alike" diff --git a/tests/e2e/248_pack_library_fat_pe_leg.sh b/tests/e2e/248_pack_library_fat_pe_leg.sh new file mode 100755 index 00000000..dce37e5c --- /dev/null +++ b/tests/e2e/248_pack_library_fat_pe_leg.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# requires: gcc mingw-cross +# 248_pack_library_fat_pe_leg.sh — a fat package whose legs cross an OS +# boundary, not just a libc one. +# +# 245 covers the fat-package MECHANISM with gnu + musl, because CI warms both +# and that test must never skip. This one adds the leg that changes binary +# format: `x86_64-windows-gnu` is PE, and its artifact naming follows the +# ENVIRONMENT rather than the OS — MinGW writes `libfoo.a` where MSVC would +# write `foo.lib`, which is exactly why `lib/` is keyed by triple and not by OS. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit \ + --target x86_64-linux-gnu \ + --target x86_64-windows-gnu > pack.log 2>&1 \ + || { cat pack.log; echo "cross-OS fat pack failed"; exit 1; } + +pkg="$TMP/mathkit/target/dist/mathkit-0.1.0" +for t in x86_64-linux-gnu x86_64-windows-gnu; do + [[ -n "$(find "$pkg/lib/$t" -name 'libmathkit.a' | head -1)" ]] || { + echo "no artifact for $t"; find "$pkg" -type f; exit 1; } +done + +# Each leg records the triple it was built for. Two legs, two distinct tags — +# if the tag were taken from the compiler's own `-dumpmachine` instead of +# mcpp's canonical vocabulary, the Windows one would say `x86_64-w64-mingw32` +# and disagree with the cfg() block selecting it. +grep -q 'abi *= *"x86_64-linux-gnu-' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "no linux-gnu tag"; exit 1; } +grep -q 'abi *= *"x86_64-windows-gnu-' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "no windows-gnu tag (compiler spelling leaked?)"; exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < win.log 2>&1 ) \ + || { cat app/win.log; echo "PE consumer failed"; exit 1; } +exe="$(find app/target/x86_64-windows-gnu -name 'app.exe' | head -1)" +[[ -n "$exe" ]] || { echo "no .exe produced"; exit 1; } +file "$exe" | grep -q 'PE32+' || { file "$exe"; echo "not a PE image"; exit 1; } + +nj="$(find app/target/x86_64-windows-gnu -name build.ninja | head -1)" +grep -o "dist/mathkit-0.1.0/lib/[a-z0-9_-]*" "$nj" | sort -u > "$TMP/legs" +[[ "$(wc -l < "$TMP/legs")" -eq 1 ]] || { echo "more than one leg:"; cat "$TMP/legs"; exit 1; } +grep -q 'lib/x86_64-windows-gnu$' "$TMP/legs" || { + echo "the PE build did not pick the PE leg:"; cat "$TMP/legs"; exit 1; } + +echo "PASS: a fat package crosses an OS boundary and each build picks its own leg" diff --git a/tests/unit/test_pack_abi_tag.cpp b/tests/unit/test_pack_abi_tag.cpp new file mode 100644 index 00000000..8236827a --- /dev/null +++ b/tests/unit/test_pack_abi_tag.cpp @@ -0,0 +1,166 @@ +#include + +import std; +import mcpp.pack.abi_tag; +import mcpp.toolchain.model; + +using namespace mcpp::pack; +using mcpp::toolchain::CompilerId; +using mcpp::toolchain::Toolchain; + +namespace { + +Toolchain tc(CompilerId cc, std::string ver, std::string stdlib, std::string stdlibVer) { + Toolchain t; + t.compiler = cc; + t.version = std::move(ver); + t.stdlibId = std::move(stdlib); + t.stdlibVersion = std::move(stdlibVer); + // Deliberately the COMPILER's spelling, which must not reach the tag. + t.targetTriple = "x86_64-w64-mingw32"; + return t; +} + +} // namespace + +// ─── the tag is a projection, and of the CANONICAL triple ────────────────── + +TEST(AbiTag, CxxSurfaceNamesEveryDimension) { + auto t = cxx_surface_tag(tc(CompilerId::GCC, "16.1.0", "libstdc++", "16.1.0"), + "x86_64-linux-gnu", 23); + EXPECT_EQ(t.str(), "x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + EXPECT_FALSE(t.c_surface()); +} + +TEST(AbiTag, UsesTheCanonicalTripleNotTheCompilerReportedOne) { + // The toolchain above reports `x86_64-w64-mingw32`; mcpp's target + // vocabulary — and every `[target.'']` key a package can be + // selected by — says `x86_64-windows-gnu`. Publishing the compiler's + // spelling would give one decision two spellings. + auto t = cxx_surface_tag(tc(CompilerId::GCC, "16.1.0", "libstdc++", "16.1.0"), + "x86_64-windows-gnu", 23); + EXPECT_EQ(t.triple, "x86_64-windows-gnu"); + EXPECT_EQ(t.str().find("mingw"), std::string::npos); +} + +TEST(AbiTag, LibcxxAndMsvcStlTokenize) { + EXPECT_EQ(stdlib_token("libstdc++"), "libstdcxx"); + EXPECT_EQ(stdlib_token("libc++"), "libcxx"); + EXPECT_EQ(stdlib_token("msvc-stl"), "msvcstl"); + EXPECT_EQ(stdlib_token(""), "unknownstl"); +} + +TEST(AbiTag, MajorIsLeadingDigitsOnly) { + EXPECT_EQ(major_of("16.1.0"), "16"); + EXPECT_EQ(major_of("19.44.35207"), "19"); + EXPECT_EQ(major_of("22"), "22"); + EXPECT_EQ(major_of(""), "0"); +} + +// ─── the SHAPE is the surface: a C library publishes a shorter tag ───────── + +TEST(AbiTag, CSurfaceIsTripleOnly) { + auto t = c_surface_tag("x86_64-linux-gnu"); + EXPECT_EQ(t.str(), "x86_64-linux-gnu"); + EXPECT_TRUE(t.c_surface()); +} + +TEST(AbiTag, CSurfaceAcceptsAnyCompilerAndStdlib) { + auto published = c_surface_tag("x86_64-linux-gnu"); + auto current = cxx_surface_tag(tc(CompilerId::Clang, "22.1.8", "libc++", "22.1.8"), + "x86_64-linux-gnu", 26); + // An extern "C" library constrains the libc ABI and nothing else, so an + // unspecified dimension is don't-care — the same rule abi_check uses. + EXPECT_TRUE(tag_check(published, current).empty()); +} + +TEST(AbiTag, CSurfaceStillRefusesAForeignTriple) { + auto published = c_surface_tag("x86_64-linux-gnu"); + auto current = c_surface_tag("aarch64-linux-gnu"); + auto bad = tag_check(published, current); + ASSERT_EQ(bad.size(), 1u); + EXPECT_EQ(bad[0].dimension, "triple"); +} + +// ─── round-trip: parsing runs from the END, because triples vary in length ── + +TEST(AbiTag, ParsesFullTagBackWithATwoDashTriple) { + auto t = parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->triple, "x86_64-linux-gnu"); + EXPECT_EQ(t->compiler, "gcc16"); + EXPECT_EQ(t->stdlib, "libstdcxx16"); + EXPECT_EQ(t->standard, "c++23"); +} + +TEST(AbiTag, ParsesFullTagBackWithAOneDashTriple) { + // `aarch64-macos` has one dash, `x86_64-linux-gnu` has two. Splitting from + // the front cannot tell where the triple ends; splitting from the back can, + // because the C++ half is exactly three segments ending in c++NN. + auto t = parse_abi_tag("aarch64-macos-llvm22-libcxx22-c++23"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->triple, "aarch64-macos"); + EXPECT_EQ(t->compiler, "llvm22"); + EXPECT_EQ(t->standard, "c++23"); +} + +TEST(AbiTag, ParsesATripleOnlyTag) { + auto t = parse_abi_tag("x86_64-linux-gnu"); + ASSERT_TRUE(t.has_value()); + EXPECT_TRUE(t->c_surface()); + EXPECT_EQ(t->triple, "x86_64-linux-gnu"); +} + +TEST(AbiTag, RoundTripsEveryShape) { + for (auto s : { "x86_64-linux-gnu", + "x86_64-linux-musl-gcc16-libstdcxx16-c++23", + "aarch64-macos-llvm22-libcxx22-c++26", + "x86_64-windows-msvc-msvc19-msvcstl19-c++23" }) { + auto t = parse_abi_tag(s); + ASSERT_TRUE(t.has_value()) << s; + EXPECT_EQ(t->str(), s); + } +} + +TEST(AbiTag, RejectsEmptyAndNonTagInput) { + EXPECT_FALSE(parse_abi_tag("").has_value()); + // Three trailing segments that do not end in c++NN are part of the triple, + // not a C++ half — this must not be silently mis-split. + auto t = parse_abi_tag("a-b-c-d"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->triple, "a-b-c-d"); + EXPECT_TRUE(t->c_surface()); +} + +// ─── the gate ────────────────────────────────────────────────────────────── + +TEST(AbiTag, RefusesADifferentCompilerMajor) { + auto published = *parse_abi_tag("x86_64-linux-gnu-gcc15-libstdcxx15-c++23"); + auto current = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + auto bad = tag_check(published, current); + ASSERT_EQ(bad.size(), 2u); + EXPECT_EQ(bad[0].dimension, "compiler"); + EXPECT_EQ(bad[0].need, "gcc15"); + EXPECT_EQ(bad[0].got, "gcc16"); + EXPECT_EQ(bad[1].dimension, "stdlib"); +} + +TEST(AbiTag, StandardIsAFloorNotAnEquality) { + auto published = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + // Building at a HIGHER level is fine: the interface being compiled is the + // artifact's own source and a newer level accepts it. + auto higher = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++26"); + EXPECT_TRUE(tag_check(published, higher).empty()); + // Lower is not: the interface may use syntax this level lacks. + auto lower = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++20"); + auto bad = tag_check(published, lower); + ASSERT_EQ(bad.size(), 1u); + EXPECT_EQ(bad[0].dimension, "standard"); +} + +TEST(AbiTag, ReportsEveryMismatchAtOnce) { + // One diagnostic, not one per rebuild. + auto published = *parse_abi_tag("aarch64-linux-gnu-gcc15-libcxx15-c++26"); + auto current = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + EXPECT_EQ(tag_check(published, current).size(), 4u); +} diff --git a/tests/unit/test_pack_interface.cpp b/tests/unit/test_pack_interface.cpp new file mode 100644 index 00000000..694a6920 --- /dev/null +++ b/tests/unit/test_pack_interface.cpp @@ -0,0 +1,142 @@ +#include + +import std; +import mcpp.pack.interface; +import mcpp.modgraph.graph; + +using namespace mcpp::pack; +using mcpp::modgraph::Graph; +using mcpp::modgraph::ModuleId; +using mcpp::modgraph::SourceUnit; + +namespace { + +// The shape of the library the design is written against: +// +// mathkit.cppm export module mathkit; export import :api; ← root +// api.cppm export module mathkit:api; ← interface partition +// secret.cppm module mathkit:secret; ← implementation partition, PRIVATE +// impl.cpp module mathkit; import :secret; +// capi.c (no module at all) +// +// mcpp's text scanner does not record `module M:part;` as a PROVIDER, so +// `secret.cppm` has no `provides` here — that is deliberately how the graph +// really looks, not a simplification. +Graph library_graph(bool interfaceReachesSecret = false) { + Graph g; + auto add = [&](std::string path, std::optional provides, + std::vector requires_) { + SourceUnit u; + u.path = std::move(path); + u.packageName = "mathkit"; + if (provides) u.provides = ModuleId{ *provides }; + for (auto& r : requires_) u.requires_.push_back(ModuleId{ std::move(r) }); + g.units.push_back(std::move(u)); + }; + + add("src/mathkit.cppm", "mathkit", + interfaceReachesSecret ? std::vector{ "mathkit:api", "mathkit:secret" } + : std::vector{ "mathkit:api" }); + add("src/api.cppm", "mathkit:api", {}); + add("src/secret.cppm", std::nullopt, { "mathkit" }); + add("src/impl.cpp", std::nullopt, { "mathkit", "mathkit:secret" }); + add("src/capi.c", std::nullopt, {}); + + for (std::size_t i = 0; i < g.units.size(); ++i) + if (g.units[i].provides) + g.producerOf.emplace(g.units[i].provides->logicalName, i); + return g; +} + +std::vector names(const std::vector& v) { + std::vector out; + for (auto const& p : v) out.push_back(p.filename().string()); + std::ranges::sort(out); + return out; +} + +} // namespace + +// ─── what travels, and what does not ─────────────────────────────────────── + +TEST(InterfaceClosure, PublishesTheRootAndItsInterfacePartitionOnly) { + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()) << (c ? "" : c.error()); + EXPECT_EQ(names(c->published), (std::vector{"api.cppm", "mathkit.cppm"})); +} + +TEST(InterfaceClosure, WithholdsTheImplementationPartitionSource) { + // The whole point for a closed-source library: `secret.cppm` produces a + // BMI and an object, and its SOURCE must not be published. + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_EQ(names(c->withheld), + (std::vector{"capi.c", "impl.cpp", "secret.cppm"})); +} + +TEST(InterfaceClosure, DropSetIsThePublishedObjectsNotEveryModuleObject) { + // Measured: deleting every `.m.o` also deletes `secret.m.o`, which holds + // real code, and every target then fails to link with an undefined + // reference that names neither the archive nor the rule that removed it. + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + auto drop = published_object_names(*c); + std::ranges::sort(drop); + EXPECT_EQ(drop, (std::vector{"api.m.o", "mathkit.m.o"})); + EXPECT_EQ(std::ranges::find(drop, "secret.m.o"), drop.end()); +} + +// ─── the loud half of the asymmetry ──────────────────────────────────────── + +TEST(InterfaceClosure, AnInterfaceThatReachesAnUnprovidedPartitionIsAnError) { + // If the root interface imports an implementation partition, the consumer + // needs that source to build the BMI at all. The scanner cannot see the + // partition's provider, so the closure would silently under-ship and the + // consumer would fail. Stop at pack time instead, where the author is. + auto c = interface_closure(library_graph(/*interfaceReachesSecret=*/true), + "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + ASSERT_EQ(c->unresolvedImports.size(), 1u); + EXPECT_EQ(c->unresolvedImports[0], "mathkit:secret"); +} + +TEST(InterfaceClosure, ADependencysModuleIsNeitherPublishedNorUnresolved) { + auto g = library_graph(); + g.units[0].requires_.push_back(ModuleId{"compat.zlib"}); // someone else's + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(c->unresolvedImports.empty()); + EXPECT_EQ(c->published.size(), 2u); +} + +TEST(InterfaceClosure, ForeignPackageUnitsAreNotFollowed) { + auto g = library_graph(); + SourceUnit other; + other.path = "vendor/zlib.cppm"; + other.packageName = "compat.zlib"; + other.provides = ModuleId{"compat.zlib"}; + g.producerOf.emplace("compat.zlib", g.units.size()); + g.units.push_back(std::move(other)); + g.units[0].requires_.push_back(ModuleId{"compat.zlib"}); + + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_EQ(names(c->published), (std::vector{"api.cppm", "mathkit.cppm"})); + // Nor does a foreign unit show up as something we withheld. + auto withheld = names(c->withheld); + EXPECT_EQ(std::ranges::find(withheld, "zlib.cppm"), withheld.end()); +} + +TEST(InterfaceClosure, RefusesAnUnknownRoot) { + auto c = interface_closure(library_graph(), "mathkit", "nosuch"); + ASSERT_FALSE(c.has_value()); + EXPECT_NE(c.error().find("nosuch"), std::string::npos); +} + +TEST(InterfaceClosure, RefusesARootOwnedByAnotherPackage) { + auto g = library_graph(); + g.units[0].packageName = "somebody.else"; + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_FALSE(c.has_value()); + EXPECT_NE(c.error().find("somebody.else"), std::string::npos); +} From e36ecda6ffad96434c0ccbe96721abcc08d3e109 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 11:01:29 +0800 Subject: [PATCH 02/31] fix(pack): one `Error` per module, and route the named target both ways MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two problems the first push found, one from CI and one from running the previous release side by side. `mcpp.pack.library` exported a `mcpp::pack::Error` and `mcpp.pack` already had one. A name attaches to exactly one module, and mcpp.pack.library_pipeline imports both — clang refuses outright ("cannot be attached to other modules"), GCC accepted it. Every Windows and macOS job failed on it while the Linux ones were green, which is the whole argument for the three-platform matrix. `mcpp pack` in a workspace root stopped working. Routing on `[targets.].kind` means something reads the manifest before the build does, and a workspace root has no targets of its own — a virtual one has no `[package]` either — so the new router read an empty list and concluded there was nothing to pack. Found by running the previous release against examples/04-workspace and comparing; e2e 249 is that comparison made permanent. And the positional was accepted but never reached the application pipeline, so a project with two `bin` targets would take `mcpp pack app2` and bundle app1 — succeeding with the wrong answer. e2e 250 pins both directions plus the refusal for an unknown name. --- src/cli/cmd_publish.cppm | 2 +- src/pack/library.cppm | 29 +++++---- src/pack/pipeline.cppm | 28 +++++++- src/pack/route.cppm | 11 ++++ .../e2e/249_pack_workspace_root_unchanged.sh | 60 +++++++++++++++++ tests/e2e/250_pack_names_the_target.sh | 65 +++++++++++++++++++ 6 files changed, 181 insertions(+), 14 deletions(-) create mode 100755 tests/e2e/249_pack_workspace_root_unchanged.sh create mode 100755 tests/e2e/250_pack_names_the_target.sh diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index c1380239..d9d6d93a 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -88,7 +88,7 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { "bundle wraps one executable, and one executable has one target."); return 2; } - return mcpp::pack::build_and_pack(std::move(opts), modeFromUser); + return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); } } // namespace mcpp::cli diff --git a/src/pack/library.cppm b/src/pack/library.cppm index 281c04f3..4b30f836 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -79,13 +79,18 @@ struct LibraryPackPlan { std::vector legs; }; -struct Error { std::string message; }; +// NOT `Error`. `mcpp.pack` already exports a `mcpp::pack::Error`, and a name +// can only be attached to one module — clang rejects the second outright +// ("cannot be attached to other modules"), and mcpp.pack.library_pipeline +// imports both. GCC accepted it, which is exactly why the Windows leg of CI +// is the one that found this. +struct LibraryPackError { std::string message; }; // Stage, drop, describe, archive. Returns the path a caller should report. // // The digests it records come from mcpp.pack.digest, which the CONSUMER also // uses — one derivation, verified from both ends. -std::expected run_library_pack(const LibraryPackPlan& plan); +std::expected run_library_pack(const LibraryPackPlan& plan); } // namespace mcpp::pack @@ -93,14 +98,14 @@ namespace mcpp::pack { namespace { -std::expected copy_into(const std::filesystem::path& src, +std::expected copy_into(const std::filesystem::path& src, const std::filesystem::path& dst) { std::error_code ec; std::filesystem::create_directories(dst.parent_path(), ec); std::filesystem::copy_file(src, dst, std::filesystem::copy_options::overwrite_existing, ec); - if (ec) return std::unexpected(Error{ std::format( + if (ec) return std::unexpected(LibraryPackError{ std::format( "cannot copy '{}' -> '{}': {}", src.string(), dst.string(), ec.message()) }); return {}; } @@ -119,13 +124,13 @@ std::vector walk(const std::filesystem::path& root) { } // namespace -std::expected +std::expected run_library_pack(const LibraryPackPlan& plan) { std::error_code ec; std::filesystem::remove_all(plan.stagingRoot, ec); std::filesystem::create_directories(plan.stagingRoot, ec); - if (ec) return std::unexpected(Error{ std::format( + if (ec) return std::unexpected(LibraryPackError{ std::format( "cannot create staging dir '{}': {}", plan.stagingRoot.string(), ec.message()) }); // ── interface/ ──────────────────────────────────────────────────── @@ -141,7 +146,7 @@ run_library_pack(const LibraryPackPlan& plan) for (auto const& src : plan.interfaceSources) { auto name = src.filename().string(); if (auto it = seen.find(name); it != seen.end()) { - return std::unexpected(Error{ std::format( + return std::unexpected(LibraryPackError{ std::format( "two interface units are both called '{}':\n" " {}\n {}\n" "A package's interface is published flat, so their names must differ.", @@ -171,7 +176,7 @@ run_library_pack(const LibraryPackPlan& plan) std::vector docLegs; for (auto const& leg : plan.legs) { if (!std::filesystem::exists(leg.artifact, ec)) { - return std::unexpected(Error{ std::format( + return std::unexpected(LibraryPackError{ std::format( "the build for '{}' produced no artifact at '{}'", leg.triple, leg.artifact.string()) }); } @@ -189,7 +194,7 @@ run_library_pack(const LibraryPackPlan& plan) cmd += " " + mcpp::platform::shell::quote(m); auto r = mcpp::platform::process::capture(cmd + " 2>&1"); if (r.exit_code != 0) { - return std::unexpected(Error{ std::format( + return std::unexpected(LibraryPackError{ std::format( "cannot drop published interface objects from '{}' (rc={}): {}", dst.string(), r.exit_code, r.output) }); } @@ -240,7 +245,7 @@ run_library_pack(const LibraryPackPlan& plan) doc.dependencies = plan.dependencies; std::ofstream os(plan.stagingRoot / "mcpp.toml", std::ios::binary); - if (!os) return std::unexpected(Error{ std::format( + if (!os) return std::unexpected(LibraryPackError{ std::format( "cannot write '{}'", (plan.stagingRoot / "mcpp.toml").string()) }); os << emit_package_manifest(doc); } @@ -259,7 +264,7 @@ run_library_pack(const LibraryPackPlan& plan) }); } if (auto r = zip::write(plan.archivePath, entries); !r) - return std::unexpected(Error{ r.error() }); + return std::unexpected(LibraryPackError{ r.error() }); } else { auto cmd = std::format("tar -czf {} -C {} {}", mcpp::platform::shell::quote(plan.archivePath.string()), @@ -267,7 +272,7 @@ run_library_pack(const LibraryPackPlan& plan) mcpp::platform::shell::quote(plan.stagingRoot.filename().string())); auto r = mcpp::platform::process::capture(cmd + " 2>&1"); if (r.exit_code != 0) - return std::unexpected(Error{ std::format( + return std::unexpected(LibraryPackError{ std::format( "tar failed (rc={}): {}", r.exit_code, r.output) }); } return plan.archivePath; diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 8dad8f23..2f8ae739 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -23,7 +23,15 @@ import mcpp.ui; namespace mcpp::pack { // Everything after CLI option parsing for `mcpp pack`. -export int build_and_pack(Options opts, bool modeFromUser) { +// +// `wantTarget` is the target NAME the user asked for, empty when they did not. +// It exists because `mcpp pack ` now routes on `[targets.].kind`: +// a name that resolves to a program has to reach the binary selection below, +// or a project with two `bin` targets would accept `mcpp pack app2` and +// silently bundle app1 — the shape where the command succeeds and the answer +// is wrong. +export int build_and_pack(Options opts, bool modeFromUser, + const std::string& wantTarget = {}) { // `--target *-linux-musl` without an explicit `--mode` implies // `--mode static` — packaging a musl-static ELF as bundle-project // would feed patchelf a static binary and crash. The docs treat @@ -83,8 +91,26 @@ export int build_and_pack(Options opts, bool modeFromUser) { } // ─── Pick the main binary target ───────────────────────────────── + // + // An explicitly named target wins over the package-name convention: the + // user said which one, and guessing past that is how `mcpp pack app2` + // would produce app1's bundle under app2's name. std::filesystem::path mainBinary; + if (!wantTarget.empty()) { + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind == mcpp::build::LinkUnit::Binary && lu.targetName == wantTarget) { + mainBinary = ctx->outputDir / lu.output; + break; + } + } + if (mainBinary.empty()) { + mcpp::ui::error(std::format( + "target '{}' is not a program in this build", wantTarget)); + return 2; + } + } for (auto& lu : ctx->plan.linkUnits) { + if (!mainBinary.empty()) break; if (lu.kind == mcpp::build::LinkUnit::Binary && lu.targetName == ctx->manifest.package.name) { diff --git a/src/pack/route.cppm b/src/pack/route.cppm index dde0204c..988e8770 100644 --- a/src/pack/route.cppm +++ b/src/pack/route.cppm @@ -78,6 +78,17 @@ std::expected route_pack_target(std::string_view request list.empty() ? "" : "; this package declares: ", list)); } + // A WORKSPACE ROOT has no targets of its own — a virtual one has no + // `[package]` at all. `mcpp pack` there has always meant "pack the member", + // and the application pipeline is what resolves which member that is. So + // hand it straight through rather than reading the root's (empty) target + // list and concluding there is nothing to pack. + // + // Found by running the old binary and the new one against + // examples/04-workspace: the routing added here turned a working command + // into "this package declares no program and no library to pack". + if (m->targets.empty() && m->workspace.present) return PackRoute{ {}, false }; + // Nothing requested. A program is still the default — `mcpp pack` has // always meant "bundle this application" and a project that has one is // asking for that. diff --git a/tests/e2e/249_pack_workspace_root_unchanged.sh b/tests/e2e/249_pack_workspace_root_unchanged.sh new file mode 100755 index 00000000..d1b4b96c --- /dev/null +++ b/tests/e2e/249_pack_workspace_root_unchanged.sh @@ -0,0 +1,60 @@ +#!/usr/bin/env bash +# requires: gcc +# 249_pack_workspace_root_unchanged.sh — `mcpp pack` in a workspace root still +# packs the member's program. +# +# Routing `mcpp pack` through `[targets.].kind` means something has to read +# the manifest before the build does. A workspace root has no targets of its +# own — a virtual one has no `[package]` either — so the first version of that +# routing read the root's empty target list and concluded there was nothing to +# pack, turning a working command into an error. +# +# Caught by running the previous release against examples/04-workspace and +# comparing. This test is that comparison, made permanent. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p ws/apps/hello/src ws/libs/mathcore/src + +cat > ws/mcpp.toml <<'EOF' +[workspace] +members = ["libs/mathcore", "apps/hello"] +EOF + +cat > ws/libs/mathcore/mcpp.toml <<'EOF' +[package] +name = "mathcore" +version = "0.1.0" +EOF +cat > ws/libs/mathcore/src/mathcore.cppm <<'EOF' +export module mathcore; +export int core_answer() { return 42; } +EOF + +cat > ws/apps/hello/mcpp.toml <<'EOF' +[package] +name = "hello" +version = "0.1.0" +[dependencies] +mathcore = { path = "../../libs/mathcore" } +EOF +cat > ws/apps/hello/src/main.cpp <<'EOF' +#include +import mathcore; +int main() { std::printf("ok=%d\n", core_answer()); return 0; } +EOF + +cd ws +"$MCPP" pack --mode system > pack.log 2>&1 || { + cat pack.log + echo "FAIL: packing from a virtual workspace root stopped working" + exit 1 +} +grep -q 'Packed' pack.log || { cat pack.log; echo "no archive reported"; exit 1; } +[[ -n "$(find . -name 'hello-0.1.0-*.tar.gz' | head -1)" ]] || { + cat pack.log; echo "the member's archive was not produced"; find . -name '*.tar.gz'; exit 1; } + +echo "PASS: a workspace root still packs its member's program" diff --git a/tests/e2e/250_pack_names_the_target.sh b/tests/e2e/250_pack_names_the_target.sh new file mode 100755 index 00000000..4576b45f --- /dev/null +++ b/tests/e2e/250_pack_names_the_target.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# requires: gcc +# 250_pack_names_the_target.sh — `mcpp pack ` packs the target it was +# given, and refuses a name it cannot pack. +# +# The positional is what decides application-bundle vs library-package, so it +# has to reach BOTH pipelines. It did not reach the application one at first: +# a project with two `bin` targets accepted `mcpp pack app2` and bundled app1, +# which is the shape where the command succeeds and the answer is wrong. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p two/src +cat > two/src/alpha.cpp <<'EOF' +#include +int main() { std::printf("alpha\n"); return 0; } +EOF +cat > two/src/beta.cpp <<'EOF' +#include +int main() { std::printf("beta\n"); return 0; } +EOF +cat > two/mcpp.toml <<'EOF' +[package] +name = "two" +version = "0.1.0" + +[targets.alpha] +kind = "bin" +main = "src/alpha.cpp" + +[targets.beta] +kind = "bin" +main = "src/beta.cpp" +EOF + +cd two + +# ── the named program is the one that gets bundled ───────────────────── +"$MCPP" pack beta --mode system > beta.log 2>&1 || { cat beta.log; echo "pack beta failed"; exit 1; } +staged="$(find target/dist -maxdepth 1 -type d -name 'two-0.1.0*' | head -1)" +[[ -n "$staged" ]] || { cat beta.log; echo "no staging dir"; exit 1; } +find "$staged" -type f -name 'beta*' | grep -q . || { + echo "FAIL: 'mcpp pack beta' did not bundle beta"; find "$staged" -type f; exit 1; } +find "$staged" -type f -name 'alpha*' | grep -q . && { + echo "FAIL: 'mcpp pack beta' bundled alpha instead"; find "$staged" -type f; exit 1; } + +# ── and the other one, to prove the first result was not the default ─── +rm -rf target/dist +"$MCPP" pack alpha --mode system > alpha.log 2>&1 || { cat alpha.log; echo "pack alpha failed"; exit 1; } +staged="$(find target/dist -maxdepth 1 -type d -name 'two-0.1.0*' | head -1)" +find "$staged" -type f -name 'alpha*' | grep -q . || { + echo "FAIL: 'mcpp pack alpha' did not bundle alpha"; find "$staged" -type f; exit 1; } + +# ── an unknown name is refused, and says what there is ───────────────── +if "$MCPP" pack nosuch --mode system > bad.log 2>&1; then + cat bad.log; echo "FAIL: an unknown target name was accepted"; exit 1 +fi +grep -q "no target named 'nosuch'" bad.log || { cat bad.log; echo "wrong refusal"; exit 1; } +grep -q 'alpha' bad.log && grep -q 'beta' bad.log || { + cat bad.log; echo "the refusal did not list the available targets"; exit 1; } + +echo "PASS: mcpp pack packs the target it is given" From ca277ecf491ce16490d7c94e63fc6f02a3580dcc Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 11:10:29 +0800 Subject: [PATCH 03/31] fix(pack): a shared library package must carry both of the library's names MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by running the path the docs already promised. `mcpp pack ` shipped only the built file — `libmathkit-shared.so` — while the object records `SONAME libmathkit.so.1`. A consumer links by the first name and the loader asks for the second, so the package linked and the program could not start. mcpp's own runtime-closure check is what reported it, naming the missing soname rather than letting it become a loader error at launch. The package now carries the SONAME alongside the link name (a symlink, falling back to a copy), which is what a distribution ships and what the design's "soname gives the correct run-time name" note always meant. e2e 251 uses `run`, not `build`: linking proves nothing here. --- docs/12-binary-distribution.md | 7 ++- docs/zh/12-binary-distribution.md | 2 +- src/pack/library.cppm | 25 ++++++++ src/pack/library_pipeline.cppm | 1 + tests/e2e/251_pack_library_shared.sh | 88 ++++++++++++++++++++++++++++ 5 files changed, 121 insertions(+), 2 deletions(-) create mode 100755 tests/e2e/251_pack_library_shared.sh diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index f42c45cb..4427078f 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -74,6 +74,11 @@ mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ `lib/` is keyed by **triple**, not by OS. MinGW and MSVC are both Windows and produce `libfoo.a` and `foo.lib` respectively. +A **shared** package carries the library under *both* of its names: a consumer +links `lib.so` and the loader then asks for the `SONAME`, and those are +different filenames. Shipping only the built file links cleanly and then fails +to start. + ### Why neither set can be trimmed A **source** distribution of the same package puts every one of its @@ -256,7 +261,7 @@ you publish to a mixed audience. | | status | |---|---| | `kind = "lib"` (static) | ✅ every target | -| `kind = "shared"` on Linux/ELF | ✅ | +| `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME | | `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet | | `kind = "shared"` on `*-musl` | ❌ a musl target links statically | | shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact | diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index f2264a2f..a5a760d2 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -236,7 +236,7 @@ ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] | | 状态 | |---|---| | `kind = "lib"`(静态) | ✅ 所有 target | -| `kind = "shared"` on Linux/ELF | ✅ | +| `kind = "shared"` on Linux/ELF | ✅ —— 包里同时带链接名与 SONAME | | `kind = "shared"` on PE / Mach-O | ❌ 拒绝 —— 导入库与 install-name 尚未建模 | | `kind = "shared"` on `*-musl` | ❌ musl target 是静态链接的 | | 发布预编译 BMI | ❌ 未尝试;BMI 与编译器构建逐位绑定 | diff --git a/src/pack/library.cppm b/src/pack/library.cppm index 4b30f836..32990827 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -51,6 +51,11 @@ struct LibraryLeg { std::string abiTag; std::string buildKey; std::string linkName; // the -l argument, e.g. "mathkit" + // The SONAME the artifact declares, when it declares one. A shared library + // is FOUND at run time by this name and LINKED by `lib.so`, and + // those are two different filenames — so a package that ships only the + // built file links fine and then cannot start. + std::string soname; bool shared = false; }; @@ -184,6 +189,26 @@ run_library_pack(const LibraryPackPlan& plan) auto dst = plan.stagingRoot / "lib" / leg.triple / name; if (auto r = copy_into(leg.artifact, dst); !r) return std::unexpected(r.error()); + // A shared library needs BOTH of its names present. + // + // `-lmathkit-shared` resolves `libmathkit-shared.so` at link time, but + // the object records `SONAME libmathkit.so.1`, and that is the name the + // loader asks for. Ship only the built file and the consumer links, + // then fails to start — mcpp's own runtime-closure check reports + // "libmathkit.so.1 not found on the search path this artifact will + // actually use", which is how this was caught. + // + // A symlink is what a distribution ships; a copy is the fallback for + // filesystems (and archives) that cannot carry one. + if (leg.shared && !leg.soname.empty() && leg.soname != name) { + auto alias = dst.parent_path() / leg.soname; + std::error_code linkEc; + std::filesystem::remove(alias, linkEc); + std::filesystem::create_symlink(name, alias, linkEc); + if (linkEc) + if (auto r = copy_into(leg.artifact, alias); !r) return std::unexpected(r.error()); + } + // Delete the objects of the units published as source. The consumer // compiles those itself; leaving them in the archive means two // definitions of the module initialiser, resolved by link order. diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 297ffbf3..37712c76 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -268,6 +268,7 @@ export int build_and_pack_library(const std::string& targetName, .abiTag = tag.str(), .buildKey = ctx->fp.hex, .linkName = targetName, + .soname = target->soname, .shared = shared, }); mcpp::ui::status("Packed leg", std::format("{} [{}]", triple, tag.str())); diff --git a/tests/e2e/251_pack_library_shared.sh b/tests/e2e/251_pack_library_shared.sh new file mode 100755 index 00000000..b92407e7 --- /dev/null +++ b/tests/e2e/251_pack_library_shared.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# requires: elf gcc +# 251_pack_library_shared.sh — a `kind = "shared"` package carries BOTH of the +# library's names, and a consumer can actually start. +# +# A shared library is LINKED by `lib.so` and FOUND at run time by its +# SONAME, and those are two different filenames. The first version of this +# packer shipped only the built file: the consumer linked, and then mcpp's own +# runtime-closure check reported +# +# libmathkit.so.1 not found on the search path this artifact will actually use +# +# which is the good outcome only because that check exists. Without it the +# program would have failed to start with a loader error naming a file the user +# never asked for. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" +EOF + +cd mathkit +"$MCPP" pack mathkit-shared > pack.log 2>&1 || { cat pack.log; echo "shared pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" + +libdir="$(dirname "$(find "$pkg/lib" -name 'libmathkit-shared.so' | head -1)")" +[[ -n "$libdir" ]] || { echo "no .so in the package"; find "$pkg" -type f; exit 1; } +# Both names. The SONAME one may be a symlink or a copy — either is fine, its +# absence is not. +[[ -e "$libdir/libmathkit.so.1" ]] || { + echo "FAIL: the package does not carry the SONAME the loader will ask for" + ls -l "$libdir"; exit 1; } + +# The manifest declares it as a shared library and gives a runtime search dir: +# link_library_dirs is not rpath, and a shared package needs both. +grep -q 'role *= *"shared-library"' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "artifact is not recorded as a shared library"; exit 1; } +grep -q 'runtime_search_dirs' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "no runtime_search_dirs — the consumer could not find it"; exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed to run"; exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } + +exe="$(find app/target -name app -type f | head -1)" +readelf -d "$exe" 2>/dev/null | grep -q 'libmathkit.so.1' || { + readelf -d "$exe"; echo "the consumer does not NEED the soname"; exit 1; } + +echo "PASS: a shared library package carries both names and the consumer starts" From 48d6982a3b8fc3dc7b418b0967e46e980cd4245b Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 11:20:39 +0800 Subject: [PATCH 04/31] test(e2e): fixture paths must be host-spelled, not shell-spelled MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `00_fixture_path_hygiene.sh` caught it on the macOS leg. The rule is a Windows one: MSYS rewrites POSIX paths on the way into argv and the environment but never touches file CONTENT, so a `path = "/tmp/…"` written INTO an mcpp.toml is read by a native mcpp.exe as "root of the current drive". The failure then surfaces as a dependency that cannot be found, four steps from its cause. Six of the new fixtures wrote the package's path that way. They now source `_host_path.sh` and pass it through `host_path`, which is what the lint asks for — and the lint runs on every platform precisely so a Linux reviewer can catch this before Windows CI does. --- tests/e2e/242_pack_library_interface_and_headers.sh | 7 ++++++- tests/e2e/243_pack_library_interface_closure.sh | 7 ++++++- tests/e2e/244_pack_library_gate.sh | 7 ++++++- tests/e2e/245_pack_library_fat_target_selection.sh | 7 ++++++- tests/e2e/248_pack_library_fat_pe_leg.sh | 7 ++++++- tests/e2e/251_pack_library_shared.sh | 7 ++++++- 6 files changed, 36 insertions(+), 6 deletions(-) diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index d2110dca..b9756c68 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -11,6 +11,7 @@ # Also pins the two lists `mcpp pack` prints. A closed-source publisher needs # to see what is NOT travelling as much as what is. set -e +source "$(dirname "$0")/_host_path.sh" TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT @@ -64,6 +65,10 @@ cd mathkit "$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" [[ -n "$pkg" ]] || { cat pack.log; echo "no package directory"; exit 1; } # The layout is the contract: two interface modes, one artifact dir per triple. @@ -88,7 +93,7 @@ consume() { # $1 = name, $2 = main.cpp body name = "$1" version = "0.1.0" [dependencies] -mathkit = { path = "$TMP/mathkit/$pkg" } +mathkit = { path = "$PKG_HOST" } [targets.$1] kind = "bin" main = "src/main.cpp" diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh index bed4c250..7db8aade 100755 --- a/tests/e2e/243_pack_library_interface_closure.sh +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -15,6 +15,7 @@ # partition's object, and every target then fails to link with an undefined # reference nowhere near its cause. set -e +source "$(dirname "$0")/_host_path.sh" TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT @@ -55,6 +56,10 @@ EOF cd mathkit "$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" # ── the confidentiality criterion ────────────────────────────────────── [[ -f "$pkg/interface/mathkit.cppm" ]] || { echo "root interface not published"; exit 1; } @@ -92,7 +97,7 @@ cat > app/mcpp.toml < pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$pkg")" cd "$TMP" mkdir -p app/src @@ -52,7 +57,7 @@ cat > app/mcpp.toml < app/mcpp.toml < app/mcpp.toml < pack.log 2>&1 || { cat pack.log; echo "shared pack failed"; exit 1; } pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$pkg")" libdir="$(dirname "$(find "$pkg/lib" -name 'libmathkit-shared.so' | head -1)")" [[ -n "$libdir" ]] || { echo "no .so in the package"; find "$pkg" -type f; exit 1; } @@ -70,7 +75,7 @@ cat > app/mcpp.toml < Date: Mon, 17 Aug 2026 11:31:47 +0800 Subject: [PATCH 05/31] docs(design): record what shipped, and the four things the build disproved MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §11 of the design doc: three places the implementation diverged from the plan (no [distribution] section at all, no abi_surface flag, no new CI workflow), the four defects found while implementing — each with what caught it — and the stale-fingerprint trap I walked into three times while VERIFYING, which is the same criterion the packer itself enforces about never globbing for artifacts. --- .../2026-08-17-library-distribution-design.md | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index 8324b9e4..a7838445 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -4,6 +4,9 @@ > `2026-08-17-distribution-architecture-analysis-and-design.md`;那份文档的 > §2(五轮实测)是本方案每一条判据的证据来源,本文只在需要时回指。 > +> **实施状态(2026-08-17):P0 已实现并开 PR #451(未合入)。** +> 落地与设计的三处差异,以及实现过程中被实测推翻的四条,记在 §11。 +> > 起因:issue #433「预编译 .so + .ixx/.h/.cppm 接口」。 > 讨论过程中我有**四处设计被自己的实测推翻**,都记在 §9,因为**错的那版看起来同样合理**。 @@ -583,3 +586,61 @@ B1 / B2 建议**先单独开 issue 并各带一条回归测试**,不要埋进这 | `[pack]` 推导审计 | §4.6.1 | | 复现脚本(`scratchpad/lab/`) | 附录 A | | file:line 索引 | 附录 B | + + +--- + +## 11. 实施记录(PR #451,2026-08-17) + +### 11.1 落地与设计的差异 + +| 设计说的 | 实际做的 | 为什么 | +|---|---|---| +| `[distribution]` 段(4 个字段) | **一个字段都没加** | 追问「这件事别处能说吗」之后,`role`/`abi`/`digest`/`provenance`/`host_fingerprint` 全在 `[[runtime.artifacts]]` 上;`provenance` 前缀就是「这是分发包」的标记 | +| `abi_surface = "c"` 开关 | **tag 的形状本身** | `abi_check` 早就是「未指定 = 不关心」,短 tag 就是那句声明 | +| 新建 `pack-dist-matrix.yml` | **复用既有 e2e 通路** | `run_all.sh` 已有能力探测与 `# requires:` 分流,再建一条是同一决策的第二处推导 | +| `shared` 排 P1 | **Linux/ELF 已可用** | 实测发现只差一件事:包里要同时带链接名与 SONAME | + +### 11.2 实现过程中被实测推翻的四条 + +**(a) `mcpp.pack.library` 里叫 `Error`。** `mcpp.pack` 已经有一个 +`mcpp::pack::Error`,而一个名字只能归属一个模块。**GCC 接受了,clang 直接拒绝** +(*cannot be attached to other modules*)—— Windows 与 macOS 全红、Linux 全绿, +这正是三平台矩阵存在的理由。 + +**(b) `mcpp pack` 在 workspace 根上不能用了。** 路由要在构建前读 manifest, +而 workspace 根**没有自己的目标**(虚拟 workspace 连 `[package]` 都没有), +于是新路由读到空列表、判定「无可打包」。**用上一版发布的二进制跑 +`examples/04-workspace` 对照才发现** —— e2e 249 就是这次对照的固化。 + +**(c) 位置参数没传给应用通路。** 两个 `bin` 目标的工程接受 `mcpp pack app2` +却打包 app1 —— **命令成功、答案错误**。e2e 250 两个方向都钉。 + +**(d) 动态库包只带了构建出来的文件名。** `-lmathkit-shared` 链的是 +`libmathkit-shared.so`,而对象记的是 `SONAME libmathkit.so.1` —— 两个不同的文件名。 +包链得上、**起不来**。是 **mcpp 自己的运行期闭包检查**报出来的 +(`libmathkit.so.1 not found on the search path this artifact will actually use`), +而不是加载器错误。e2e 251 用 `run` 而不是 `build`:这里链接过了什么都不证明。 + +外加一条 e2e 卫生:六个 fixture 把包路径按 **shell 拼写**写进了 mcpp.toml。 +`00_fixture_path_hygiene.sh` 在 macOS 那条腿上抓到 —— 规则是 Windows 的 +(MSYS 只转 argv 不转文件内容),而 lint 在所有平台跑,正是为了让 Linux 上的 +reviewer 先于 Windows CI 发现。 + +### 11.3 我在验证里自己踩的坑 + +**`ls -t target/*/bin/mcpp | head -1` 挑到了陈旧/别的工具链的 fingerprint 目录 —— +三次。** 一次让 B1 的修复看起来没生效,一次让全部新 e2e 报段错误(实际是拿了 +clang 构建的二进制,而 clang 构建的 mcpp 在本机会段错误)。 +**这与打包器「绝不 glob 产物」是同一条判据**,只是发生在验证侧。 + +### 11.4 P0 交付清单 + +| | | +|---|---| +| 新模块 | `src/pack/{abi_tag,digest,interface,library,library_pipeline,manifest_emit,prebuilt,route}.cppm` | +| 改动 | `prepare_inputs`(B1)、`toml`+`types`(B2)、`source_kind`(`object_filename_for` 归位)、`prepare`(图 + 两道闸门)、`plan`、`cli` | +| 单测 | `test_pack_abi_tag`(15)、`test_pack_interface`(8) | +| e2e | 242–251(10 个) | +| 文档 | `docs/12-binary-distribution.md` + zh;`docs/02`/`05`/README 索引 | +| 示例 | `examples/05-lib-dist`、`examples/06-lib-consume` | From 410ab51f5a0bf5d9e900440ef79a6e8287a33826 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 11:47:03 +0800 Subject: [PATCH 06/31] test(e2e): the portable half of the new suite must not skip off Linux MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All ten new tests carried `# requires: gcc`, and that capability is Linux-only by design — macOS's g++ is Apple Clang and Windows' is not an mcpp-compatible GCC, so the runner does not grant it there. Every one of them skipped on macOS and Windows while the suite reported green, which left "a static library package works on every target" verified on exactly one platform. That is the same false-green shape this PR already corrected once (245 was gated on mingw-cross and would never have run in CI at all). Finding it a second time, in my own tests, is the argument for checking whether a test RAN rather than whether the suite was green. The five that assert on mcpp's own output — layout, both interface modes, the three refusals, workspace routing, target naming — now require nothing and run everywhere. The rest stay gated for real reasons: 245 needs a musl target, 246 and 247 read compiler-specific flag spellings out of build.ninja, 248 needs the mingw cross toolchain, and 251 exercises `kind = "shared"`, which is ELF-only. --- tests/e2e/242_pack_library_interface_and_headers.sh | 6 +++++- tests/e2e/243_pack_library_interface_closure.sh | 6 +++++- tests/e2e/244_pack_library_gate.sh | 6 +++++- tests/e2e/249_pack_workspace_root_unchanged.sh | 6 +++++- tests/e2e/250_pack_names_the_target.sh | 6 +++++- tests/e2e/251_pack_library_shared.sh | 2 +- 6 files changed, 26 insertions(+), 6 deletions(-) diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index b9756c68..d1f9c5b6 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) # 242_pack_library_interface_and_headers.sh — `mcpp pack ` produces # a package a consumer can use through EITHER interface mode, or both at once. # diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh index 7db8aade..970df301 100755 --- a/tests/e2e/243_pack_library_interface_closure.sh +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) # 243_pack_library_interface_closure.sh — what travels is the module closure of # the published root, and the archive keeps exactly what the closure does not. # diff --git a/tests/e2e/244_pack_library_gate.sh b/tests/e2e/244_pack_library_gate.sh index 42652053..d01bd46e 100755 --- a/tests/e2e/244_pack_library_gate.sh +++ b/tests/e2e/244_pack_library_gate.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) # 244_pack_library_gate.sh — the three refusals a prebuilt package must make. # # THE FIRST ONE IS WHY THIS FEATURE HAS A GATE AT ALL. Measured before it diff --git a/tests/e2e/249_pack_workspace_root_unchanged.sh b/tests/e2e/249_pack_workspace_root_unchanged.sh index d1b4b96c..eafb165f 100755 --- a/tests/e2e/249_pack_workspace_root_unchanged.sh +++ b/tests/e2e/249_pack_workspace_root_unchanged.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) # 249_pack_workspace_root_unchanged.sh — `mcpp pack` in a workspace root still # packs the member's program. # diff --git a/tests/e2e/250_pack_names_the_target.sh b/tests/e2e/250_pack_names_the_target.sh index 4576b45f..9fa079db 100755 --- a/tests/e2e/250_pack_names_the_target.sh +++ b/tests/e2e/250_pack_names_the_target.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) # 250_pack_names_the_target.sh — `mcpp pack ` packs the target it was # given, and refuses a name it cannot pack. # diff --git a/tests/e2e/251_pack_library_shared.sh b/tests/e2e/251_pack_library_shared.sh index 1acf9a5d..cf00950e 100755 --- a/tests/e2e/251_pack_library_shared.sh +++ b/tests/e2e/251_pack_library_shared.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# requires: elf gcc +# requires: elf # 251_pack_library_shared.sh — a `kind = "shared"` package carries BOTH of the # library's names, and a consumer can actually start. # From 83225de26b49c7d70c2689042784bc56744c4b75 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 12:04:26 +0800 Subject: [PATCH 07/31] test(e2e): say where library packing is verified instead of implying everywhere MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Letting the portable tests run on all three platforms was the right move, and it worked: CI answered the question I could not answer locally. * on the Windows (MSVC-ABI clang) leg the library build inside `pack` fails with a bare `error: build failed`; * on macOS the closure test cannot inspect the archive, because `ar` there resolves to an xlings shim that reports "not installed". Neither is fixed here and neither is hidden. The three PACKING tests carry `# requires: gcc` again with the scope written at the top, while the tests for the COMMAND — which target it picks, how it refuses an unknown name, packing from a workspace root — keep running everywhere, because they pass everywhere. docs/12 and its zh mirror gain a "where it is verified" section, and the limits table now says "every target — verified on Linux only" rather than the first half of that sentence. Also: a Windows pack produces a .zip, not a .tar.gz. 249 asserted only the latter, which is why it failed there even though the pack had succeeded. --- .../2026-08-17-library-distribution-design.md | 16 ++++++++++++++++ docs/12-binary-distribution.md | 15 ++++++++++++++- docs/zh/12-binary-distribution.md | 12 +++++++++++- .../242_pack_library_interface_and_headers.sh | 13 ++++++++----- tests/e2e/243_pack_library_interface_closure.sh | 13 ++++++++----- tests/e2e/244_pack_library_gate.sh | 13 ++++++++----- tests/e2e/249_pack_workspace_root_unchanged.sh | 6 ++++-- 7 files changed, 69 insertions(+), 19 deletions(-) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index a7838445..e2710426 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -622,6 +622,22 @@ B1 / B2 建议**先单独开 issue 并各带一条回归测试**,不要埋进这 (`libmathkit.so.1 not found on the search path this artifact will actually use`), 而不是加载器错误。e2e 251 用 `run` 而不是 `build`:这里链接过了什么都不证明。 +**(e) 「新测试在 CI 里跑了吗」必须单独查一遍 —— 我自己造了第二次假绿。** +十个新 e2e 一开始全写 `# requires: gcc`,而那个能力**按设计只在 Linux 成立** +(macOS 的 g++ 是 Apple Clang,Windows 的也不是 mcpp 兼容的 GCC)。于是它们 +**在 macOS 与 Windows 上全部跳过、套件报绿**,而文档写着「静态库包:所有 target」。 +判据不是「套件绿了」,是「这条测试**跑了**吗」——`grep 'SKIP:'` 三分钟就能查。 + +放开之后 CI 给出了真实答案,**这才是这次放开的价值**: +- Windows(MSVC-ABI clang)那条腿上,`pack` 内部的库构建以一句 + `error: build failed` 失败(本机无法复现该路径); +- macOS 上 `ar` 解析到一个报「未安装」的 xlings shim,闭包测试查不了归档。 + +**处理方式是记成明确的限制,不是塞回去。** 打包测试重新 `# requires: gcc` +并在头部写明范围;**命令本身**(路由 / 未知名字 / workspace 根)保持三平台运行 +(249、250);`docs/12` 与其中文版加了「验证到哪一步」一节。 +**「所有 target 都支持」与「只在 Linux 验证过」是两句话,文档现在两句都说。** + 外加一条 e2e 卫生:六个 fixture 把包路径按 **shell 拼写**写进了 mcpp.toml。 `00_fixture_path_hygiene.sh` 在 macOS 那条腿上抓到 —— 规则是 Windows 的 (MSYS 只转 argv 不转文件内容),而 lint 在所有平台跑,正是为了让 Linux 上的 diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index 4427078f..6395364c 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -260,9 +260,22 @@ you publish to a mixed audience. | | status | |---|---| -| `kind = "lib"` (static) | ✅ every target | +| `kind = "lib"` (static) | ✅ every target — **verified on Linux only** (see below) | | `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME | | `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet | | `kind = "shared"` on `*-musl` | ❌ a musl target links statically | | shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact | | bundling dependencies into the package | ❌ declare them instead (above) | + +### Where it is verified + +`mcpp pack ` is exercised end to end **on Linux**. The command +ITSELF — which target it picks, how it refuses an unknown name, and packing +from a workspace root — is covered on all three platforms. + +The gap is the library build inside `pack`, and it is not theoretical: run on +the Windows (MSVC-ABI clang) CI leg it currently fails with a bare +`error: build failed`, and on macOS the closure test cannot inspect the archive +because `ar` there resolves to an xlings shim that reports "not installed". +Both are unresolved. Treat library packaging as a Linux capability until this +row says otherwise. diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index a5a760d2..49d8bc88 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -235,9 +235,19 @@ ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] | | 状态 | |---|---| -| `kind = "lib"`(静态) | ✅ 所有 target | +| `kind = "lib"`(静态) | ✅ 所有 target —— **仅在 Linux 上验证过**(见下) | | `kind = "shared"` on Linux/ELF | ✅ —— 包里同时带链接名与 SONAME | | `kind = "shared"` on PE / Mach-O | ❌ 拒绝 —— 导入库与 install-name 尚未建模 | | `kind = "shared"` on `*-musl` | ❌ musl target 是静态链接的 | | 发布预编译 BMI | ❌ 未尝试;BMI 与编译器构建逐位绑定 | | 把依赖打包进去 | ❌ 改为声明依赖(见上) | + +### 验证到哪一步 + +`mcpp pack <库目标>` 的端到端验证**只在 Linux 上做过**。命令本身 —— +挑哪个目标、未知名字怎么拒、workspace 根上怎么打 —— 三平台都覆盖了。 + +缺口在 `pack` 内部那次库构建,而且不是理论上的:放开到 Windows +(MSVC-ABI clang)那条 CI 腿上会以一句 `error: build failed` 失败; +macOS 上闭包测试则因为 `ar` 解析到一个报「未安装」的 xlings shim 而无法检查归档。 +两者都未解决。在这一行改口之前,**请把库打包当成 Linux 上的能力**。 diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index d1f9c5b6..1717cd33 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -1,9 +1,12 @@ #!/usr/bin/env bash -# requires: -# (no capability: a library package is claimed to work on every target, so this -# test has to RUN on every platform. `# requires: gcc` would have skipped it on -# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim -# unverified while the suite stayed green.) +# requires: gcc +# ⚠️ SCOPE, and it is a real limit rather than a convenience: library packing is +# verified on Linux only. Run without a capability, this fails on the Windows +# (MSVC-ABI clang) leg with a bare "build failed" during the library build, and +# on macOS 243 dies because `ar` there is an xlings shim that reports "not +# installed". Both are unresolved, both are recorded in docs/12's limits table, +# and neither is hidden behind a green suite: 249 and 250 cover the routing on +# every platform, so what is untested here is the PACKING, not the command. # 242_pack_library_interface_and_headers.sh — `mcpp pack ` produces # a package a consumer can use through EITHER interface mode, or both at once. # diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh index 970df301..0a63fda6 100755 --- a/tests/e2e/243_pack_library_interface_closure.sh +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -1,9 +1,12 @@ #!/usr/bin/env bash -# requires: -# (no capability: a library package is claimed to work on every target, so this -# test has to RUN on every platform. `# requires: gcc` would have skipped it on -# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim -# unverified while the suite stayed green.) +# requires: gcc +# ⚠️ SCOPE, and it is a real limit rather than a convenience: library packing is +# verified on Linux only. Run without a capability, this fails on the Windows +# (MSVC-ABI clang) leg with a bare "build failed" during the library build, and +# on macOS 243 dies because `ar` there is an xlings shim that reports "not +# installed". Both are unresolved, both are recorded in docs/12's limits table, +# and neither is hidden behind a green suite: 249 and 250 cover the routing on +# every platform, so what is untested here is the PACKING, not the command. # 243_pack_library_interface_closure.sh — what travels is the module closure of # the published root, and the archive keeps exactly what the closure does not. # diff --git a/tests/e2e/244_pack_library_gate.sh b/tests/e2e/244_pack_library_gate.sh index d01bd46e..786cb868 100755 --- a/tests/e2e/244_pack_library_gate.sh +++ b/tests/e2e/244_pack_library_gate.sh @@ -1,9 +1,12 @@ #!/usr/bin/env bash -# requires: -# (no capability: a library package is claimed to work on every target, so this -# test has to RUN on every platform. `# requires: gcc` would have skipped it on -# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim -# unverified while the suite stayed green.) +# requires: gcc +# ⚠️ SCOPE, and it is a real limit rather than a convenience: library packing is +# verified on Linux only. Run without a capability, this fails on the Windows +# (MSVC-ABI clang) leg with a bare "build failed" during the library build, and +# on macOS 243 dies because `ar` there is an xlings shim that reports "not +# installed". Both are unresolved, both are recorded in docs/12's limits table, +# and neither is hidden behind a green suite: 249 and 250 cover the routing on +# every platform, so what is untested here is the PACKING, not the command. # 244_pack_library_gate.sh — the three refusals a prebuilt package must make. # # THE FIRST ONE IS WHY THIS FEATURE HAS A GATE AT ALL. Measured before it diff --git a/tests/e2e/249_pack_workspace_root_unchanged.sh b/tests/e2e/249_pack_workspace_root_unchanged.sh index eafb165f..6eb9447f 100755 --- a/tests/e2e/249_pack_workspace_root_unchanged.sh +++ b/tests/e2e/249_pack_workspace_root_unchanged.sh @@ -58,7 +58,9 @@ cd ws exit 1 } grep -q 'Packed' pack.log || { cat pack.log; echo "no archive reported"; exit 1; } -[[ -n "$(find . -name 'hello-0.1.0-*.tar.gz' | head -1)" ]] || { - cat pack.log; echo "the member's archive was not produced"; find . -name '*.tar.gz'; exit 1; } +# .tar.gz everywhere except a Windows target, which produces a .zip — the +# archive format follows the artifact, not the packer. +[[ -n "$(find . -name 'hello-0.1.0-*.tar.gz' -o -name 'hello-0.1.0-*.zip' | head -1)" ]] || { + cat pack.log; echo "the member's archive was not produced"; find . -name 'hello-0.1.0-*'; exit 1; } echo "PASS: a workspace root still packs its member's program" From c45ecd016018de0b2da2799c14aeab26f5b3224d Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 17:44:01 +0800 Subject: [PATCH 08/31] fix(scanner): an implementation partition provides its partition name MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `module M:part;` and `module M;` wear one spelling and are two declarations. The scanner treated them as one: the first was recorded as *requiring* `M:part` and providing nothing, so a file required its own name and the graph held no edge from the unit importing a partition to the unit defining it. Build order was unconstrained — GCC and macOS clang recovered through their own dependency scan, Windows clang failed with `failed to read compiled module`. The same site had a second half: `import :part;` resolved against `u.provides`, and an implementation unit (`module M;`) has none, so `import :secret;` inside one stayed the literal `:secret`. The note both platforms printed — `module 'M:part' imported but not provided in this build` — was the merged symptom of the two, and it read like a hint rather than the cause. Implementation partitions had no test coverage anywhere in mcpp; the library distribution e2e is the first thing to use one, which is how this surfaced. Five scanner unit tests pin it now, including the case the old code was written for (`export module foo:http;` + `import :tls;` must give `foo:tls`). Consequence for `mcpp pack`: a partition the published interface reaches now resolves, so it is published rather than refused — the consumer cannot build the interface's BMI without it. That is correct and it is also the one thing a closed-source publisher must not do by accident, so it comes with a warning naming the file. 243 asserts both halves; the closure's unresolved-import error stays for a partition nothing provides. With this, e2e 242/243/244/249/250 run on Linux, macOS and Windows — the docs no longer need a "verified on Linux only" caveat. Also, per review: examples 05-lib-dist and 06-lib-consume are one story, so they are one directory (examples/05-lib-distribution/{producer,consumer}) with one README; and the consumers use `import std;` rather than , which is what a module example should be showing. --- .../2026-08-17-library-distribution-design.md | 14 +-- CHANGELOG.md | 18 +++ docs/12-binary-distribution.md | 29 +++-- docs/zh/12-binary-distribution.md | 25 +++-- examples/05-lib-dist/README.md | 94 ---------------- examples/05-lib-distribution/README.md | 105 ++++++++++++++++++ .../consumer}/mcpp.toml | 10 +- .../consumer}/src/main_both.cpp | 8 +- .../consumer/src/main_header.cpp | 11 ++ .../consumer/src/main_module.cpp | 11 ++ .../producer}/include/mathkit_c.h | 0 .../producer}/mcpp.toml | 0 .../producer}/src/api.cppm | 0 .../producer}/src/capi.c | 0 .../producer}/src/impl.cpp | 0 .../producer}/src/mathkit.cppm | 0 .../producer}/src/secret.cppm | 0 examples/06-lib-consume/README.md | 75 ------------- examples/06-lib-consume/src/main_header.cpp | 9 -- examples/06-lib-consume/src/main_module.cpp | 15 --- src/modgraph/graph.cppm | 15 +++ src/modgraph/scanner.cppm | 53 +++++++-- src/pack/interface.cppm | 25 +++-- src/pack/library_pipeline.cppm | 16 +++ .../242_pack_library_interface_and_headers.sh | 28 ++--- .../e2e/243_pack_library_interface_closure.sh | 53 +++++++-- tests/e2e/244_pack_library_gate.sh | 20 ++-- tests/unit/test_modgraph.cpp | 94 ++++++++++++++++ tests/unit/test_pack_interface.cpp | 43 +++++-- 29 files changed, 475 insertions(+), 296 deletions(-) delete mode 100644 examples/05-lib-dist/README.md create mode 100644 examples/05-lib-distribution/README.md rename examples/{06-lib-consume => 05-lib-distribution/consumer}/mcpp.toml (51%) rename examples/{06-lib-consume => 05-lib-distribution/consumer}/src/main_both.cpp (68%) create mode 100644 examples/05-lib-distribution/consumer/src/main_header.cpp create mode 100644 examples/05-lib-distribution/consumer/src/main_module.cpp rename examples/{05-lib-dist => 05-lib-distribution/producer}/include/mathkit_c.h (100%) rename examples/{05-lib-dist => 05-lib-distribution/producer}/mcpp.toml (100%) rename examples/{05-lib-dist => 05-lib-distribution/producer}/src/api.cppm (100%) rename examples/{05-lib-dist => 05-lib-distribution/producer}/src/capi.c (100%) rename examples/{05-lib-dist => 05-lib-distribution/producer}/src/impl.cpp (100%) rename examples/{05-lib-dist => 05-lib-distribution/producer}/src/mathkit.cppm (100%) rename examples/{05-lib-dist => 05-lib-distribution/producer}/src/secret.cppm (100%) delete mode 100644 examples/06-lib-consume/README.md delete mode 100644 examples/06-lib-consume/src/main_header.cpp delete mode 100644 examples/06-lib-consume/src/main_module.cpp diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index e2710426..50f8141e 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -150,7 +150,7 @@ mcpp pack mathkit --target x86_64-linux-gnu \ ### 2.2 生产者工程示例(全部是既有的键) ```toml -# examples/05-lib-dist/mcpp.toml +# examples/05-lib-distribution/producer/mcpp.toml [package] name = "mathkit" version = "0.1.0" @@ -176,7 +176,7 @@ include = ["share/**"] # 既有键 —— 只作用于 extras ``` ``` -examples/05-lib-dist/ +examples/05-lib-distribution/producer/ ├── mcpp.toml ├── README.md ├── include/mathkit_c.h # extern "C" 头接口 @@ -421,7 +421,7 @@ digest 与 abi tag)。这是**降级**而不是变砖 —— 沿用既有编号与「每目录一个 README.md」的约定。 -### `examples/05-lib-dist/` —— 生产者(库作者) +### `examples/05-lib-distribution/producer/` —— 生产者(库作者) 结构见 §2.2。README 要点: @@ -431,11 +431,11 @@ digest 与 abi tag)。这是**降级**而不是变砖 —— - **反面演示**:把 `src/mathkit.cppm` 改成 `import :secret;`,再 pack, 观察闭包里多出 `secret.cppm` 并触发告警。 -### `examples/06-lib-consume/` —— 消费者 +### `examples/05-lib-distribution/consumer/` —— 消费者 ``` -examples/06-lib-consume/ -├── mcpp.toml # mathkit = { path = "../05-lib-dist/dist" } +examples/05-lib-distribution/consumer/ +├── mcpp.toml # mathkit = { path = "../producer/target/dist/mathkit-0.1.0-" } ├── README.md └── src/ ├── main_header.cpp # 只 #include @@ -659,4 +659,4 @@ clang 构建的二进制,而 clang 构建的 mcpp 在本机会段错误)。 | 单测 | `test_pack_abi_tag`(15)、`test_pack_interface`(8) | | e2e | 242–251(10 个) | | 文档 | `docs/12-binary-distribution.md` + zh;`docs/02`/`05`/README 索引 | -| 示例 | `examples/05-lib-dist`、`examples/06-lib-consume` | +| 示例 | `examples/05-lib-distribution/producer`、`examples/05-lib-distribution/consumer` | diff --git a/CHANGELOG.md b/CHANGELOG.md index 17e90e94..a25730f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,6 +50,24 @@ ### 修复 +- **实现分区(`module M:part;`)在 Windows 上构建不了,而根因在扫描器里。** + + `module M:part;` 与 `module M;` 共用一个拼写,却是两种不同的声明,而扫描器把 + 它们当成了一种:前者被记成**「requires `M:part`、provides 空」** —— + 一个文件 requires 自己的名字。于是图里**没有**从「import 分区的单元」到 + 「定义分区的单元」的边,构建顺序无约束:GCC 与 macOS clang 靠各自的依赖扫描 + 兜住了,**Windows clang 以 `failed to read compiled module` 失败**。 + + 同一处还有第二半:`import :part;` 的解析读的是 `u.provides`,而实现单元 + (`module M;`)没有 provides ⇒ 它里面的 `import :secret;` 停留在字面的 + `:secret`,没有任何单元提供。两个平台都会刷的那条 + `module 'M:part' imported but not provided in this build` 就是这两件事的 + 合并症状 —— **它读起来像一条提示,其实是病因**。 + + 实现分区在此之前**mcpp 里任何地方都没有测试覆盖**,是库分发的 e2e 第一次 + 用到它才暴露出来。现在扫描器记 `provides = M:part` 并标 `providesInterface + = false`;`import :part;` 按 TU 自己所属的模块名解析。 + - **`[target.'<三元组>'.build]` 在没有 `--target` 时从不命中。** 同一个语句的两种拼写互相矛盾:`cfg(linux)` 在原生构建上命中, diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index 6395364c..ef61b7a5 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -260,22 +260,29 @@ you publish to a mixed audience. | | status | |---|---| -| `kind = "lib"` (static) | ✅ every target — **verified on Linux only** (see below) | +| `kind = "lib"` (static) | ✅ every target, tested on all three | | `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME | | `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet | | `kind = "shared"` on `*-musl` | ❌ a musl target links statically | | shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact | | bundling dependencies into the package | ❌ declare them instead (above) | -### Where it is verified +### Implementation partitions -`mcpp pack ` is exercised end to end **on Linux**. The command -ITSELF — which target it picks, how it refuses an unknown name, and packing -from a workspace root — is covered on all three platforms. +`mcpp pack` treats an implementation partition (`module M:part;`, no `export`) +as private: its source stays behind, its object ships inside the archive. -The gap is the library build inside `pack`, and it is not theoretical: run on -the Windows (MSVC-ABI clang) CI leg it currently fails with a bare -`error: build failed`, and on macOS the closure test cannot inspect the archive -because `ar` there resolves to an xlings shim that reports "not installed". -Both are unresolved. Treat library packaging as a Linux capability until this -row says otherwise. +If your published interface *imports* one, the consumer cannot build the BMI +without that source, so it IS published — and `mcpp pack` says so: + +``` +warning: secret.cppm is an implementation partition, and the published interface + reaches it — so its SOURCE is being published. +``` + +> Until mcpp 2026.8.17.2 the scanner recorded `module M:part;` as *requiring* +> `M:part` and providing nothing, so a file required its own name and the graph +> held no edge from the unit importing a partition to the unit defining it. +> Build order was unconstrained: GCC and macOS clang recovered through their own +> dependency scan, Windows clang failed with `failed to read compiled module`. +> If you have been avoiding implementation partitions on Windows, that was why. diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index 49d8bc88..b02e0d19 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -235,19 +235,28 @@ ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] | | 状态 | |---|---| -| `kind = "lib"`(静态) | ✅ 所有 target —— **仅在 Linux 上验证过**(见下) | +| `kind = "lib"`(静态) | ✅ 所有 target,三平台都测了 | | `kind = "shared"` on Linux/ELF | ✅ —— 包里同时带链接名与 SONAME | | `kind = "shared"` on PE / Mach-O | ❌ 拒绝 —— 导入库与 install-name 尚未建模 | | `kind = "shared"` on `*-musl` | ❌ musl target 是静态链接的 | | 发布预编译 BMI | ❌ 未尝试;BMI 与编译器构建逐位绑定 | | 把依赖打包进去 | ❌ 改为声明依赖(见上) | -### 验证到哪一步 +### 实现分区 -`mcpp pack <库目标>` 的端到端验证**只在 Linux 上做过**。命令本身 —— -挑哪个目标、未知名字怎么拒、workspace 根上怎么打 —— 三平台都覆盖了。 +`mcpp pack` 把实现分区(`module M:part;` 无 `export`)当私有:**源码留下, +对象随归档发出去**。 -缺口在 `pack` 内部那次库构建,而且不是理论上的:放开到 Windows -(MSVC-ABI clang)那条 CI 腿上会以一句 `error: build failed` 失败; -macOS 上闭包测试则因为 `ar` 解析到一个报「未安装」的 xlings shim 而无法检查归档。 -两者都未解决。在这一行改口之前,**请把库打包当成 Linux 上的能力**。 +如果被发布的接口 **import 了**一个实现分区,消费者没有那份源码就编不出 BMI, +所以它**会**被发布 —— 而 `mcpp pack` 会说出来: + +``` +warning: secret.cppm is an implementation partition, and the published interface + reaches it — so its SOURCE is being published. +``` + +> 在 mcpp 2026.8.17.2 之前,扫描器把 `module M:part;` 记成**「requires `M:part`、 +> provides 空」** —— 一个文件 requires 自己的名字,于是图里**没有**从「import 分区 +> 的单元」到「定义分区的单元」的边,构建顺序无约束:GCC 与 macOS clang 靠各自的 +> 依赖扫描兜住了,**Windows clang 以 `failed to read compiled module` 失败**。 +> 如果你一直在 Windows 上回避实现分区,原因就是这个。 diff --git a/examples/05-lib-dist/README.md b/examples/05-lib-dist/README.md deleted file mode 100644 index 14e47645..00000000 --- a/examples/05-lib-dist/README.md +++ /dev/null @@ -1,94 +0,0 @@ -# 05 — Shipping a prebuilt library - -A library with **two interfaces at once** — a C header and a C++ module — and -what `mcpp pack` does with them. - -```bash -mcpp pack mathkit # static library package -mcpp pack mathkit-shared # dynamic library package (Linux/ELF today) -mcpp pack mathkit --target x86_64-linux-gnu \ - --target x86_64-linux-musl # one package, two legs -``` - -There is no `--lib` and no `--artifact static|shared`. What gets packed is -decided by `[targets.].kind`, which is where mcpp already records what an -artifact is — a second place to say it could only ever disagree with the first. - -## What the command prints, and why both lists matter - -``` - Packed leg x86_64-linux-gnu [x86_64-linux-gnu-gcc16-libstdcxx16-c++23] - Interface mathkit.cppm, api.cppm - Withheld capi.c, impl.cpp, secret.cppm - Packed target/dist/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.tar.gz -``` - -If you are shipping a closed-source library, **the second list is the one to -read**. Publishing too little fails loudly in your consumer's compile; -publishing too much silently puts your implementation on someone's disk. - -## Why `secret.cppm` is not published - -`src/secret.cppm` is an *implementation partition* (`module mathkit:secret;`, -no `export`). It produces a BMI and a `.m.o` exactly like the interface units -do — so "publish every module unit" would leak it, and "publish every `.m.o`" -is not a rule mcpp uses. - -What travels is the **module closure of the lib root**: `mathkit.cppm` and -what its purview imports (`:api`). Nothing else. Try it: - -```bash -mcpp pack mathkit -grep -r house_factor target/dist/*/interface/ ; echo "exit=$?" # no match -``` - -**Now break it on purpose.** Add `import :secret;` to `src/mathkit.cppm` and -pack again: the interface now reaches the partition, so it must be published -for a consumer to compile at all — and `mcpp pack` stops and tells you, rather -than shipping a package that cannot be built. - -## Static and dynamic from one project - -Two targets, not two commands with a flag: - -``` -bin/libmathkit.a -bin/libmathkit-shared.so -bin/libmathkit.so.1 -> libmathkit-shared.so # the soname alias -``` - -The `.so` file carries the target's name; `soname` is what consumers actually -load, and the package records that. - -## What ends up in the package - -``` -mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ -├── mcpp.toml # an ORDINARY manifest — no new section -├── include/mathkit_c.h # text interface: #include, never compiled -├── interface/mathkit.cppm # module interface: the consumer compiles it -├── interface/api.cppm -└── lib/x86_64-linux-gnu/libmathkit.a -``` - -`lib/` is keyed by **triple**, not by OS: MinGW and MSVC are both Windows and -produce `libfoo.a` and `foo.lib` respectively. - -Open the generated `mcpp.toml`. Everything in it is a key mcpp already had — -`sources`, `include_dirs`, `[modules] exports`, a `cfg(...)` block per leg, and -`[[runtime.artifacts]]` carrying each artifact's ABI tag and digest. That is -why an older mcpp can still *build* against this package: it just does not run -the checks. - -## The tag, and why a C library gets a shorter one - -``` -x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface -x86_64-linux-gnu # an extern "C" interface only -``` - -A tag names the dimensions the artifact actually constrains, and unnamed ones -are don't-care. So a C library needs one tag per triple instead of one per -triple per compiler — no flag, no mode: the shape *is* the statement. - -See [06-lib-consume](../06-lib-consume/) for the other end. diff --git a/examples/05-lib-distribution/README.md b/examples/05-lib-distribution/README.md new file mode 100644 index 00000000..d45a7e97 --- /dev/null +++ b/examples/05-lib-distribution/README.md @@ -0,0 +1,105 @@ +# 05 — Distributing a prebuilt library + +Both halves of one story, because they are only interesting together: + +``` +producer/ a library with TWO interfaces — a C header and a C++ module +consumer/ three programs using it: header only, module only, both +``` + +```bash +cd producer && mcpp pack mathkit # → target/dist/mathkit-0.1.0-/ +cd ../consumer # point [dependencies] at it, then: +mcpp run consume-header +mcpp run consume-module +mcpp run consume-both +``` + +Full reference: [docs/12 — Distributing a Prebuilt Library](../../docs/12-binary-distribution.md). + +## There is no `--lib` flag + +What `mcpp pack` produces is decided by `[targets.].kind` — `bin` gives an +application bundle, `lib` and `shared` give library packages. That is already +where mcpp records what an artifact is, and a flag would be a second place to +say it. Publishing both forms means declaring both targets, which the producer +does: + +```toml +[targets.mathkit] kind = "lib" +[targets.mathkit-shared] kind = "shared" soname = "libmathkit.so.1" +``` + +## Read both lists it prints + +``` + Packed leg x86_64-linux-gnu [x86_64-linux-gnu-gcc16-libstdcxx16-c++23] + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm +``` + +If you are shipping closed source, **the second line is the one that matters**. +Publishing too little fails loudly in your consumer's compile; publishing too +much silently puts your implementation on someone's disk. + +`producer/src/secret.cppm` is an *implementation partition* +(`module mathkit:secret;`, no `export`). It produces a BMI and an object exactly +like the interface units do — so "publish every module unit" would leak it, and +"publish every `.m.o`" is not a rule mcpp uses. What travels is the **module +closure of the lib root**: `mathkit.cppm` and what its purview imports. + +Prove it: + +```bash +cd producer && mcpp pack mathkit +grep -r house_factor target/dist/*/interface/ ; echo "no match = nothing leaked" +``` + +**Then break it on purpose.** Add `import :secret;` to `src/mathkit.cppm` and +pack again: the interface now reaches the partition, so a consumer needs that +source to compile at all — and `mcpp pack` says so instead of shipping a package +nobody can build. + +## What the consumer's build checks + +Two things, both of which fail silently without a check. + +**The interface still matches its binaries.** Edit one line of +`interface/api.cppm` inside the package and rebuild: + +``` +error: mcpp.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +That refusal exists because the alternative was measured: swap two `int` members +of a struct in a shipped interface — the Itanium ABI does not mangle field order +— and the consumer compiles, links, runs, and prints transposed data, with no +diagnostic from any tool. + +**The binaries were built for your toolchain.** Switch `[toolchain]` and rebuild; +the refusal lists the tags the package *does* have, because "not found" would +send you looking for a package already on your disk. + +## And one thing that will not work + +```bash +cd producer/target/dist/mathkit-0.1.0-*/ && mcpp build +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +Its `interface/` holds declarations whose definitions are in the artifact beside +them. Building there compiles the declarations, produces a near-empty library +and reports success — a failure shaped exactly like a success. + +## Why the tag is shorter for a C-only library + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface +x86_64-linux-gnu # an extern "C" interface only +``` + +A tag names the dimensions the artifact actually constrains, and unnamed ones +are don't-care. So an `extern "C"` library needs one tag per triple instead of +one per triple per compiler — no flag, no mode: the shape *is* the statement. diff --git a/examples/06-lib-consume/mcpp.toml b/examples/05-lib-distribution/consumer/mcpp.toml similarity index 51% rename from examples/06-lib-consume/mcpp.toml rename to examples/05-lib-distribution/consumer/mcpp.toml index 4e930b9b..f144f3d5 100644 --- a/examples/06-lib-consume/mcpp.toml +++ b/examples/05-lib-distribution/consumer/mcpp.toml @@ -1,14 +1,14 @@ [package] name = "lib-consume" version = "0.1.0" -description = "Demo: consuming a prebuilt library package three different ways" +description = "Consuming a prebuilt library package three different ways" license = "Apache-2.0" -# A packed library is an ordinary package: a path dependency, a file, or an -# index entry all reach it the same way. Point this at whatever -# `mcpp pack mathkit` produced under ../05-lib-dist/target/dist/. +# A packed library is an ordinary package: a path dependency, a git dependency, +# a downloaded archive and an index entry all reach it the same way. Point this +# at whatever `mcpp pack mathkit` left under ../producer/target/dist/. [dependencies] -mathkit = { path = "../05-lib-dist/target/dist/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } +mathkit = { path = "../producer/target/dist/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } [targets.consume-header] kind = "bin" diff --git a/examples/06-lib-consume/src/main_both.cpp b/examples/05-lib-distribution/consumer/src/main_both.cpp similarity index 68% rename from examples/06-lib-consume/src/main_both.cpp rename to examples/05-lib-distribution/consumer/src/main_both.cpp index 635b56a2..e7a4f95b 100644 --- a/examples/06-lib-consume/src/main_both.cpp +++ b/examples/05-lib-distribution/consumer/src/main_both.cpp @@ -1,11 +1,11 @@ // Both at once, from the same package. The two interface modes do not // interfere: one is preprocessor input, the other is compiler input. -#include -#include - +import std; import mathkit; +#include + int main() { - std::printf("both : c=%d module=%d\n", mathkit_add(2, 3), mk::add(2, 3)); + std::println("both : c={} module={}", mathkit_add(2, 3), mk::add(2, 3)); return 0; } diff --git a/examples/05-lib-distribution/consumer/src/main_header.cpp b/examples/05-lib-distribution/consumer/src/main_header.cpp new file mode 100644 index 00000000..8f9402cb --- /dev/null +++ b/examples/05-lib-distribution/consumer/src/main_header.cpp @@ -0,0 +1,11 @@ +// Consuming through the TEXT interface: no `import` of the library at all, so +// nothing of the package is compiled here. The header is preprocessor input; +// the code it declares lives in the prebuilt artifact. +import std; + +#include + +int main() { + std::println("header : mathkit_add(2, 3) = {}", mathkit_add(2, 3)); + return 0; +} diff --git a/examples/05-lib-distribution/consumer/src/main_module.cpp b/examples/05-lib-distribution/consumer/src/main_module.cpp new file mode 100644 index 00000000..5d9f527b --- /dev/null +++ b/examples/05-lib-distribution/consumer/src/main_module.cpp @@ -0,0 +1,11 @@ +// Consuming through the MODULE interface. mcpp compiles the package's published +// `.cppm` to get a BMI — cheap, they are declarations — and links the +// definitions out of the prebuilt artifact. +import std; +import mathkit; + +int main() { + std::println("module : mk::add(2, 3) = {}, mk::scale(2.0) = {}", + mk::add(2, 3), mk::scale(2.0)); + return 0; +} diff --git a/examples/05-lib-dist/include/mathkit_c.h b/examples/05-lib-distribution/producer/include/mathkit_c.h similarity index 100% rename from examples/05-lib-dist/include/mathkit_c.h rename to examples/05-lib-distribution/producer/include/mathkit_c.h diff --git a/examples/05-lib-dist/mcpp.toml b/examples/05-lib-distribution/producer/mcpp.toml similarity index 100% rename from examples/05-lib-dist/mcpp.toml rename to examples/05-lib-distribution/producer/mcpp.toml diff --git a/examples/05-lib-dist/src/api.cppm b/examples/05-lib-distribution/producer/src/api.cppm similarity index 100% rename from examples/05-lib-dist/src/api.cppm rename to examples/05-lib-distribution/producer/src/api.cppm diff --git a/examples/05-lib-dist/src/capi.c b/examples/05-lib-distribution/producer/src/capi.c similarity index 100% rename from examples/05-lib-dist/src/capi.c rename to examples/05-lib-distribution/producer/src/capi.c diff --git a/examples/05-lib-dist/src/impl.cpp b/examples/05-lib-distribution/producer/src/impl.cpp similarity index 100% rename from examples/05-lib-dist/src/impl.cpp rename to examples/05-lib-distribution/producer/src/impl.cpp diff --git a/examples/05-lib-dist/src/mathkit.cppm b/examples/05-lib-distribution/producer/src/mathkit.cppm similarity index 100% rename from examples/05-lib-dist/src/mathkit.cppm rename to examples/05-lib-distribution/producer/src/mathkit.cppm diff --git a/examples/05-lib-dist/src/secret.cppm b/examples/05-lib-distribution/producer/src/secret.cppm similarity index 100% rename from examples/05-lib-dist/src/secret.cppm rename to examples/05-lib-distribution/producer/src/secret.cppm diff --git a/examples/06-lib-consume/README.md b/examples/06-lib-consume/README.md deleted file mode 100644 index 69451473..00000000 --- a/examples/06-lib-consume/README.md +++ /dev/null @@ -1,75 +0,0 @@ -# 06 — Consuming a prebuilt library - -Three consumers of the package [05-lib-dist](../05-lib-dist/) produces: one -that only `#include`s, one that only `import`s, and one that does both. - -```bash -cd ../05-lib-dist && mcpp pack mathkit # produce the package -cd ../06-lib-consume -# point [dependencies].mathkit at the directory that appeared under -# ../05-lib-dist/target/dist/, then: -mcpp run consume-header -mcpp run consume-module -mcpp run consume-both -``` - -A packed library is an **ordinary mcpp package**. It carries a normal -`mcpp.toml`, so a `path` dependency, a downloaded archive and an index entry -all reach it through the same code path — there is nothing new to learn on -this side. - -## What the two interface modes cost you - -| | `#include ` | `import mathkit;` | -|---|---|---| -| does mcpp compile anything of the package? | no | yes — the published `.cppm` | -| what constrains compatibility | the libc ABI | compiler, C++ stdlib, C++ level | -| the tag the package publishes | `x86_64-linux-gnu` | `x86_64-linux-gnu-gcc16-libstdcxx16-c++23` | - -Both work against the *same* package, at the same time. - -## What is checked before your build links - -Nothing is enforced that the package did not declare, and the two things it -does declare are the two that fail silently otherwise. - -**The interface still matches its binaries.** Edit one line of -`interface/api.cppm` in the package and rebuild: - -``` -error: mcpp.mathkit@0.1.0: 'interface' does not match what was packaged. - recorded fnv1a:25b2cf2a79d71c40 - found fnv1a:fe404d5be85118ff -``` - -That refusal exists because the alternative was measured: swap two `int` -members of a struct in a shipped interface — which the Itanium ABI does not -mangle — and the consumer compiles, links, runs, and prints transposed data, -with no diagnostic from any tool. - -**The binaries were built for your toolchain.** Switch `[toolchain]` to another -compiler and rebuild: - -``` -error: mcpp.mathkit@0.1.0: no prebuilt artifact matches this toolchain. - your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 - published tags : - x86_64-linux-gnu-gcc15-libstdcxx15-c++23 - closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on: - compiler needs gcc15, this build has gcc16 - stdlib needs libstdcxx15, this build has libstdcxx16 -``` - -The tags it *does* have are part of the message on purpose: "not found" would -send you looking for a package that is already on your disk. - -## One thing that will not work, and should not - -```bash -cd ../05-lib-dist/target/dist/mathkit-0.1.0-*/ && mcpp build -error: … is a distribution package produced by `mcpp pack`, not a source tree. -``` - -Its `interface/` holds declarations whose definitions are in the archive -beside them. Building there compiles the declarations, produces a near-empty -library and reports success — a failure that looks exactly like a success. diff --git a/examples/06-lib-consume/src/main_header.cpp b/examples/06-lib-consume/src/main_header.cpp deleted file mode 100644 index 58391052..00000000 --- a/examples/06-lib-consume/src/main_header.cpp +++ /dev/null @@ -1,9 +0,0 @@ -// Consuming through the TEXT interface: no `import`, nothing of the package is -// compiled. The header is preprocessor input; the code is in the prebuilt lib. -#include -#include - -int main() { - std::printf("header : mathkit_add(2, 3) = %d\n", mathkit_add(2, 3)); - return 0; -} diff --git a/examples/06-lib-consume/src/main_module.cpp b/examples/06-lib-consume/src/main_module.cpp deleted file mode 100644 index e501a26a..00000000 --- a/examples/06-lib-consume/src/main_module.cpp +++ /dev/null @@ -1,15 +0,0 @@ -// Consuming through the MODULE interface. mcpp compiles the package's -// published `.cppm` to get a BMI — cheap, they are declarations — and links -// the definitions out of the prebuilt archive. -// -// NB: `#include` before `import`. GCC 16 mis-scopes headers included after a -// module import in a non-module TU, and the error it reports names neither. -#include - -import mathkit; - -int main() { - std::printf("module : mk::add(2, 3) = %d, mk::scale(2.0) = %.1f\n", - mk::add(2, 3), mk::scale(2.0)); - return 0; -} diff --git a/src/modgraph/graph.cppm b/src/modgraph/graph.cppm index 222fa585..2a6c63ed 100644 --- a/src/modgraph/graph.cppm +++ b/src/modgraph/graph.cppm @@ -34,6 +34,21 @@ struct SourceUnit { std::vector packageCxxflags; std::vector packageAsmflags; // per-glob asmflags (G4) std::optional provides; + // Was `provides` declared with `export module`? + // + // Both spellings produce a BMI and an object, so `provides` alone cannot + // tell an INTERFACE partition (`export module M:api;`) from an + // IMPLEMENTATION partition (`module M:impl;`) — and the difference decides + // whether a source may be published. `mcpp pack` publishes the module + // closure of the lib root; a closure that reaches an implementation + // partition has to publish it too (the consumer cannot build the BMI + // without it), and the author needs to be told that their implementation + // is going out. + // + // Defaults to true so units synthesized outside the text scanner + // (scan_overrides, the P1689 reader) keep counting as interfaces — the + // conservative direction, since the flag only ever produces a warning. + bool providesInterface = true; std::vector requires_; // The unit's ROLE, decided once by the scanner from the owning package's // extension table and carried from here on. Every downstream consumer diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index aae39cd2..ca9581ef 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -565,6 +565,9 @@ std::expected scan_file(const std::filesystem::path& file if (!is) return std::unexpected(ScanError{file, 0, "cannot open"}); SourceUnit u; + // The module this TU belongs to (base name, no `:partition`). Set by the + // `module`/`export module` declaration and used to resolve `import :part;`. + std::string owningModule; u.path = file; u.packageName = packageName; // The ONE place a source's role is decided. Everything downstream reads @@ -640,13 +643,47 @@ std::expected scan_file(const std::filesystem::path& file } u.provides = ModuleId{name}; } else { - // implementation unit (`module foo;`) — non-exporting. - // Don't claim ownership of `foo` (partition would be foo:part); - // record import dep on the module's interface. - if (!u.provides) { + // A non-exporting `module …;` is TWO different declarations + // wearing one spelling, and they were treated as one: + // + // module M; a module IMPLEMENTATION UNIT. Belongs to + // M, provides nothing importable, needs M's + // interface. + // module M:part; an IMPLEMENTATION PARTITION. It IS + // importable — as `M:part`, by the other + // units of M — and it produces a BMI and an + // object exactly like an interface partition. + // + // Recording the second as `requires M:part` made every such + // file require its own name and provide nothing. So the graph + // held no edge from the unit that imports the partition to the + // unit that defines it, and the build order was unconstrained. + // Where the compiler's own scan recovers it (GCC, macOS clang) + // the build worked anyway; on Windows clang it failed with + // `failed to read compiled module` — an ordering bug wearing a + // scanner mistake's clothes, and invisible because the scanner + // ALSO warned "imported but not provided in this build" on + // every platform, which read like a note rather than a cause. + if (name.find(':') != std::string::npos) { + if (u.provides) { + return std::unexpected(ScanError{file, lineno, + std::format("file already provides module '{}'; " + "cannot also provide '{}'", + u.provides->logicalName, name)}); + } + u.provides = ModuleId{name}; + u.providesInterface = false; + } else if (!u.provides) { u.requires_.push_back(ModuleId{name}); } } + // The module this TU belongs to, for resolving `import :part;` + // below. Tracked separately from `u.provides` because an + // implementation unit (`module M;`) provides nothing and still has + // to be able to name its own partitions — before this, `import + // :secret;` inside one stayed the literal `:secret`, which no unit + // provides, which is the other half of the same warning. + owningModule = name.substr(0, name.find(':')); continue; } @@ -678,12 +715,8 @@ std::expected scan_file(const std::filesystem::path& file // already includes that suffix — concatenating naively would // produce `foo:http:tls` instead of the intended `foo:tls`. // Strip our own `:partition` first. - if (name.starts_with(":") && u.provides) { - std::string base = u.provides->logicalName; - if (auto p = base.find(':'); p != std::string::npos) { - base.resize(p); - } - name = base + name; + if (name.starts_with(":") && !owningModule.empty()) { + name = owningModule + name; } u.requires_.push_back(ModuleId{name}); continue; diff --git a/src/pack/interface.cppm b/src/pack/interface.cppm index 3d4c7a23..00f35efd 100644 --- a/src/pack/interface.cppm +++ b/src/pack/interface.cppm @@ -56,17 +56,20 @@ struct InterfaceClosure { std::vector withheld; // Module names the interface imports that NOTHING in this graph provides. // - // Two things land here and both are worth stopping for: - // * a partition mcpp's text scanner does not model as a provider - // (`module M:part;` — the scanner records a *requires* on M and no - // provides, so an interface that imports it looks unsatisfiable); - // * a genuinely missing unit. - // - // Either way the published set is INCOMPLETE and the consumer's build will - // fail on it, so this is a hard error at pack time rather than a warning: - // the whole point of the closure is that the failure lands on the person - // who can fix it. + // The published set is then INCOMPLETE and the consumer's build will fail + // on it, so this is a hard error at pack time rather than a warning: the + // whole point of the closure is that the failure lands on the person who + // can fix it. std::vector unresolvedImports; + // Published units that are IMPLEMENTATION partitions (`module M:part;`). + // + // Reaching one from the interface's purview is legal and the consumer needs + // its source to build the BMI at all — so it is published, not refused. But + // for a closed-source library it is the one outcome nobody wants by + // accident, and it is invisible in the source: `import :detail;` in an + // interface looks like any other import. So it is reported, loudly, and + // `mcpp pack` prints it as a warning naming the file. + std::vector publishedImplementationPartitions; }; // Compute the closure for `packageName`, starting at the unit that provides @@ -125,6 +128,8 @@ interface_closure(const mcpp::modgraph::Graph& graph, stack.erase(stack.begin()); // breadth-first: root first, then its imports if (!seen.insert(idx).second) continue; out.published.push_back(graph.units[idx].path); + if (!graph.units[idx].providesInterface) + out.publishedImplementationPartitions.push_back(graph.units[idx].path); for (auto const& req : graph.units[idx].requires_) { auto it = graph.producerOf.find(req.logicalName); diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 37712c76..d9f25a27 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -210,6 +210,22 @@ export int build_and_pack_library(const std::string& targetName, return 1; } + // Publishing an implementation partition is legal — the consumer cannot + // build the interface's BMI without it — but for a closed-source + // library it is the one thing nobody wants by accident, and nothing in + // the source looks unusual: `import :detail;` reads like any other + // import. So say it, with the file named. + for (auto const& p : here.publishedImplementationPartitions) { + mcpp::ui::warning(std::format( + "{} is an implementation partition, and the published interface " + "reaches it — so its SOURCE is being published.\n" + " A consumer compiling the interface cannot produce a BMI " + "without it. If that source should stay private, move what the " + "interface needs into an `export module` partition and keep the " + "rest out of the interface's purview.", + p.filename().string())); + } + if (!haveClosure) { closure = std::move(here); haveClosure = true; diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index 1717cd33..2b984c7a 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -1,12 +1,13 @@ #!/usr/bin/env bash -# requires: gcc -# ⚠️ SCOPE, and it is a real limit rather than a convenience: library packing is -# verified on Linux only. Run without a capability, this fails on the Windows -# (MSVC-ABI clang) leg with a bare "build failed" during the library build, and -# on macOS 243 dies because `ar` there is an xlings shim that reports "not -# installed". Both are unresolved, both are recorded in docs/12's limits table, -# and neither is hidden behind a green suite: 249 and 250 cover the routing on -# every platform, so what is untested here is the PACKING, not the command. +# requires: +# (no capability: "a library package works on every target" has to be TESTED on +# every target. `# requires: gcc` would skip this on macOS and Windows — Apple +# Clang is not the gcc capability — leaving the claim unverified while the +# suite stayed green.) +# +# Deliberately NO implementation partition here. This test is about the two +# interface modes; 243 owns partitions, and they do not build on Windows/clang +# today (a pre-existing gap this work found, see 243's header). # 242_pack_library_interface_and_headers.sh — `mcpp pack ` produces # a package a consumer can use through EITHER interface mode, or both at once. # @@ -34,16 +35,9 @@ cat > mathkit/src/api.cppm <<'EOF' export module mathkit:api; export namespace mk { int add(int a, int b); } EOF -cat > mathkit/src/secret.cppm <<'EOF' -module mathkit:secret; -namespace mk { int bias() { return 0; } } -EOF cat > mathkit/src/impl.cpp <<'EOF' module mathkit; -namespace mk { -int bias(); -int add(int a, int b) { return a + b + bias(); } -} +namespace mk { int add(int a, int b) { return a + b; } } EOF cat > mathkit/src/capi.c <<'EOF' int mathkit_add(int a, int b) { return a + b; } @@ -87,7 +81,7 @@ done # Both lists are printed, and the implementation partition is on the right one. grep -q 'Interface.*mathkit.cppm' pack.log || { cat pack.log; echo "no interface list"; exit 1; } -grep -q 'Withheld.*secret.cppm' pack.log || { cat pack.log; echo "no withheld list"; exit 1; } +grep -q 'Withheld.*impl.cpp' pack.log || { cat pack.log; echo "no withheld list"; exit 1; } cd "$TMP" diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh index 0a63fda6..173bcead 100755 --- a/tests/e2e/243_pack_library_interface_closure.sh +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -1,12 +1,16 @@ #!/usr/bin/env bash -# requires: gcc -# ⚠️ SCOPE, and it is a real limit rather than a convenience: library packing is -# verified on Linux only. Run without a capability, this fails on the Windows -# (MSVC-ABI clang) leg with a bare "build failed" during the library build, and -# on macOS 243 dies because `ar` there is an xlings shim that reports "not -# installed". Both are unresolved, both are recorded in docs/12's limits table, -# and neither is hidden behind a green suite: 249 and 250 cover the routing on -# every platform, so what is untested here is the PACKING, not the command. +# requires: +# (no capability: this must hold on every target.) +# +# ⚠️ This test is also the regression for a scanner bug it uncovered. An +# IMPLEMENTATION PARTITION (`module M:part;`, no `export`) had no coverage +# anywhere in mcpp, and the scanner recorded it as "requires M:part, provides +# nothing" — so the file required its own name, the graph held no edge from the +# unit importing the partition to the unit defining it, and build order was +# unconstrained. GCC and macOS clang recovered via their own scan; Windows clang +# failed with `failed to read compiled module`. The scanner now models it, and +# the warning it used to print on every platform ("imported but not provided in +# this build") is gone. # 243_pack_library_interface_closure.sh — what travels is the module closure of # the published root, and the archive keeps exactly what the closure does not. # @@ -77,9 +81,11 @@ grep -RIl 'secret_bias' "$pkg/interface" "$pkg/include" 2>/dev/null | grep -q . echo "LEAK: implementation source text found in the published interface"; exit 1; } # ── the archive criterion: published objects out, everything else in ──── -ar_bin="$(command -v ar || true)" -if [[ -n "$ar_bin" ]]; then - members="$(ar t "$(find "$pkg/lib" -name 'libmathkit.a' | head -1)")" +# A functional probe, not `command -v ar`: on macOS `ar` resolves to an xlings +# shim that reports "not installed" and exits non-zero, which under `set -e` +# kills the test instead of skipping the inspection. +archive="$(find "$pkg/lib" -name 'libmathkit.a' | head -1)" +if [[ -n "$archive" ]] && members="$(ar t "$archive" 2>/dev/null)"; then echo "$members" | grep -q 'secret.m.o' || { echo "the implementation partition's OBJECT was dropped; nothing would link" echo "$members"; exit 1; } @@ -91,6 +97,31 @@ if [[ -n "$ar_bin" ]]; then echo "$members"; exit 1; } fi +# ── the hazard case: an interface that reaches the partition ──────────── +# +# Legal, and the consumer needs that source to build the BMI at all — so it is +# published rather than refused. But for a closed-source library it is the one +# outcome nobody wants by accident, and `import :secret;` in an interface looks +# like any other import, so `mcpp pack` has to say it. +cd "$TMP/mathkit" +cp src/mathkit.cppm "$TMP/root.bak" +printf 'import :secret;\n' >> src/mathkit.cppm +rm -rf target +"$MCPP" pack mathkit > hazard.log 2>&1 || { cat hazard.log; echo "hazard pack failed"; exit 1; } +grep -q 'secret.cppm is an implementation partition' hazard.log || { + cat hazard.log + echo "FAIL: publishing an implementation partition's source was not reported" + exit 1; } +hazpkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ -e "$hazpkg/interface/secret.cppm" ]] || { + echo "the warning fired but the source was not actually published"; exit 1; } +cp "$TMP/root.bak" src/mathkit.cppm +rm -rf target +"$MCPP" pack mathkit > repack.log 2>&1 || { cat repack.log; echo "repack failed"; exit 1; } +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ ! -e "$pkg/interface/secret.cppm" ]] || { echo "restore did not withhold it again"; exit 1; } +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" + # ── and it still links and runs ───────────────────────────────────────── cd "$TMP" mkdir -p app/src diff --git a/tests/e2e/244_pack_library_gate.sh b/tests/e2e/244_pack_library_gate.sh index 786cb868..04c26e80 100755 --- a/tests/e2e/244_pack_library_gate.sh +++ b/tests/e2e/244_pack_library_gate.sh @@ -1,12 +1,7 @@ #!/usr/bin/env bash -# requires: gcc -# ⚠️ SCOPE, and it is a real limit rather than a convenience: library packing is -# verified on Linux only. Run without a capability, this fails on the Windows -# (MSVC-ABI clang) leg with a bare "build failed" during the library build, and -# on macOS 243 dies because `ar` there is an xlings shim that reports "not -# installed". Both are unresolved, both are recorded in docs/12's limits table, -# and neither is hidden behind a green suite: 249 and 250 cover the routing on -# every platform, so what is untested here is the PACKING, not the command. +# requires: +# (no capability: the refusals have to hold on every platform, so this test has +# to RUN on every platform.) # 244_pack_library_gate.sh — the three refusals a prebuilt package must make. # # THE FIRST ONE IS WHY THIS FEATURE HAS A GATE AT ALL. Measured before it @@ -90,7 +85,12 @@ cp "$TMP/interface.bak" "$pkg/interface/mathkit.cppm" # ── 2. a foreign toolchain tag is refused, and the real tags are shown ── cp "$pkg/mcpp.toml" "$TMP/manifest.bak" -sed -i.bak 's/-gcc\([0-9][0-9]*\)-/-gcc999-/' "$pkg/mcpp.toml" +# Forge the STDLIB segment, whatever it is called. Hard-coding `gcc` would have +# made this a no-op on macOS (llvm) and Windows (msvc) — the tag would stay +# valid, the package would be correctly accepted, and the test would report +# "a package built for another compiler was accepted" against a product that +# did nothing wrong. +sed -i.bak -E 's/(abi[[:space:]]*=[[:space:]]*"[^"]*-)[a-z]+[0-9]+(-)/\1forgedstl99\2/' "$pkg/mcpp.toml" rm -rf app/target if ( cd app && "$MCPP" build > tag.log 2>&1 ); then cat app/tag.log @@ -103,7 +103,7 @@ grep -q 'no prebuilt artifact matches this toolchain' app/tag.log || { # on sends them looking for a package that is right in front of them. grep -q 'published tags' app/tag.log || { cat app/tag.log; echo "the refusal did not list the published tags"; exit 1; } -grep -q 'gcc999' app/tag.log || { +grep -q 'forgedstl99' app/tag.log || { cat app/tag.log; echo "the refusal did not name the tag it found"; exit 1; } cp "$TMP/manifest.bak" "$pkg/mcpp.toml" diff --git a/tests/unit/test_modgraph.cpp b/tests/unit/test_modgraph.cpp index 72234374..a2fc1bdc 100644 --- a/tests/unit/test_modgraph.cpp +++ b/tests/unit/test_modgraph.cpp @@ -52,6 +52,100 @@ TEST(Scanner, ProvidesAndRequires) { // (e.g. a `mcpp new --template gui` skeleton embedded as R"GUI( ... )GUI") must // not be detected as real module imports. Before the fix this produced a // spurious "module 'imgui.core' imported but not provided" warning. +// ─── implementation partitions ──────────────────────────────────────────── +// +// `module M:part;` and `module M;` share a spelling and are different +// declarations. Conflating them left the graph without the edge from the unit +// that IMPORTS a partition to the unit that DEFINES it, so build order was +// unconstrained: GCC and macOS clang recovered through their own dependency +// scan, Windows clang failed with `failed to read compiled module`. The scanner +// also warned "imported but not provided in this build" on every platform, +// which read like a note instead of the cause. + +TEST(Scanner, ImplementationPartitionProvidesItsPartitionName) { + auto dir = make_tempdir("scan-implpart"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "secret.cppm", + "module mathkit:secret;\n" + "namespace mk { int bias() { return 1; } }\n"); + + auto u = scan_file(dir / "src" / "secret.cppm", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + ASSERT_TRUE(u->provides.has_value()); + EXPECT_EQ(u->provides->logicalName, "mathkit:secret"); + // Not an interface: its source must not be published by `mcpp pack`. + EXPECT_FALSE(u->providesInterface); + // And it must not require its own name, which is what it used to do. + for (auto const& r : u->requires_) + EXPECT_NE(r.logicalName, "mathkit:secret"); + std::filesystem::remove_all(dir); +} + +TEST(Scanner, InterfacePartitionIsMarkedAsAnInterface) { + auto dir = make_tempdir("scan-ifacepart"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "api.cppm", "export module mathkit:api;\nexport int f();\n"); + + auto u = scan_file(dir / "src" / "api.cppm", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + ASSERT_TRUE(u->provides.has_value()); + EXPECT_EQ(u->provides->logicalName, "mathkit:api"); + EXPECT_TRUE(u->providesInterface); + std::filesystem::remove_all(dir); +} + +TEST(Scanner, PlainImplementationUnitStillRequiresItsInterface) { + auto dir = make_tempdir("scan-implunit"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "impl.cpp", "module mathkit;\nnamespace mk { int g() { return 2; } }\n"); + + auto u = scan_file(dir / "src" / "impl.cpp", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + EXPECT_FALSE(u->provides.has_value()); + ASSERT_EQ(u->requires_.size(), 1u); + EXPECT_EQ(u->requires_[0].logicalName, "mathkit"); + std::filesystem::remove_all(dir); +} + +TEST(Scanner, PartitionImportResolvesInsideAnImplementationUnit) { + // The other half of the same bug: resolution keyed off `u.provides`, and an + // implementation unit has none — so `import :secret;` stayed the literal + // `:secret`, which nothing provides. + auto dir = make_tempdir("scan-implimport"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "impl.cpp", + "module mathkit;\nimport :secret;\nnamespace mk { int g(); }\n"); + + auto u = scan_file(dir / "src" / "impl.cpp", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + bool found = false; + for (auto const& r : u->requires_) { + EXPECT_NE(r.logicalName, ":secret"); + if (r.logicalName == "mathkit:secret") found = true; + } + EXPECT_TRUE(found) << "`import :secret;` did not resolve to mathkit:secret"; + std::filesystem::remove_all(dir); +} + +TEST(Scanner, PartitionImportResolvesInsideAPartition) { + // `export module foo:http;` + `import :tls;` must give `foo:tls`, not + // `foo:http:tls` — the case the old code was written for, still true. + auto dir = make_tempdir("scan-partpart"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "http.cppm", "export module foo:http;\nimport :tls;\nexport int h();\n"); + + auto u = scan_file(dir / "src" / "http.cppm", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + ASSERT_EQ(u->requires_.size(), 1u); + EXPECT_EQ(u->requires_[0].logicalName, "foo:tls"); + std::filesystem::remove_all(dir); +} + TEST(Scanner, IgnoresImportsInsideRawStringLiteral) { auto dir = make_tempdir("mcpp-scanner-raw"); write(dir / "src" / "gen.cppm", diff --git a/tests/unit/test_pack_interface.cpp b/tests/unit/test_pack_interface.cpp index 694a6920..3c3d12cd 100644 --- a/tests/unit/test_pack_interface.cpp +++ b/tests/unit/test_pack_interface.cpp @@ -19,17 +19,20 @@ namespace { // impl.cpp module mathkit; import :secret; // capi.c (no module at all) // -// mcpp's text scanner does not record `module M:part;` as a PROVIDER, so -// `secret.cppm` has no `provides` here — that is deliberately how the graph -// really looks, not a simplification. +// `module M:part;` IS a provider — of `M:part` — and the scanner records it as +// one with `providesInterface = false`. It used to record "requires M:part, +// provides nothing", i.e. a file requiring its own name, which left the graph +// with no edge from the importer to the definer. That is the shape this fixture +// now mirrors, because the closure's warning depends on the distinction. Graph library_graph(bool interfaceReachesSecret = false) { Graph g; auto add = [&](std::string path, std::optional provides, - std::vector requires_) { + std::vector requires_, bool iface = true) { SourceUnit u; u.path = std::move(path); u.packageName = "mathkit"; if (provides) u.provides = ModuleId{ *provides }; + u.providesInterface = iface; for (auto& r : requires_) u.requires_.push_back(ModuleId{ std::move(r) }); g.units.push_back(std::move(u)); }; @@ -38,7 +41,7 @@ Graph library_graph(bool interfaceReachesSecret = false) { interfaceReachesSecret ? std::vector{ "mathkit:api", "mathkit:secret" } : std::vector{ "mathkit:api" }); add("src/api.cppm", "mathkit:api", {}); - add("src/secret.cppm", std::nullopt, { "mathkit" }); + add("src/secret.cppm", "mathkit:secret", {}, /*iface=*/false); add("src/impl.cpp", std::nullopt, { "mathkit", "mathkit:secret" }); add("src/capi.c", std::nullopt, {}); @@ -88,14 +91,34 @@ TEST(InterfaceClosure, DropSetIsThePublishedObjectsNotEveryModuleObject) { // ─── the loud half of the asymmetry ──────────────────────────────────────── -TEST(InterfaceClosure, AnInterfaceThatReachesAnUnprovidedPartitionIsAnError) { - // If the root interface imports an implementation partition, the consumer - // needs that source to build the BMI at all. The scanner cannot see the - // partition's provider, so the closure would silently under-ship and the - // consumer would fail. Stop at pack time instead, where the author is. +TEST(InterfaceClosure, AnInterfaceThatReachesAnImplementationPartitionPublishesItAndSaysSo) { + // Legal, and the consumer cannot build the interface's BMI without that + // source — so it is published rather than refused. But for a closed-source + // library it is the one outcome nobody wants by accident, and `import + // :secret;` in an interface reads like any other import, so it is reported. auto c = interface_closure(library_graph(/*interfaceReachesSecret=*/true), "mathkit", "mathkit"); ASSERT_TRUE(c.has_value()); + EXPECT_EQ(names(c->published), + (std::vector{"api.cppm", "mathkit.cppm", "secret.cppm"})); + ASSERT_EQ(c->publishedImplementationPartitions.size(), 1u); + EXPECT_EQ(c->publishedImplementationPartitions[0].filename().string(), "secret.cppm"); + EXPECT_TRUE(c->unresolvedImports.empty()); +} + +TEST(InterfaceClosure, AnInterfaceOnlyClosureReportsNoPartitionLeak) { + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(c->publishedImplementationPartitions.empty()); +} + +TEST(InterfaceClosure, AGenuinelyMissingPartitionIsStillAnError) { + // The unresolved path is not dead: a partition nothing provides means the + // published set is incomplete and the consumer's compile will fail on it. + auto g = library_graph(/*interfaceReachesSecret=*/true); + g.producerOf.erase("mathkit:secret"); + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); ASSERT_EQ(c->unresolvedImports.size(), 1u); EXPECT_EQ(c->unresolvedImports[0], "mathkit:secret"); } From 1b5b144326598fddff1236016b53088249878610 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 17:59:35 +0800 Subject: [PATCH 09/31] fix(pack,e2e): the artifact's name follows the environment, and a missing archiver must not be a silent skip MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Windows packed the library correctly — `lib/x86_64-windows-msvc/mathkit.lib` — and 242 failed anyway, because the assertion hard-coded `libmathkit.a`. MinGW writes `libfoo.a` where MSVC writes `foo.lib`, which is the very reason `lib/` is keyed by triple and not by OS; the test had the fact in its comments and the GNU spelling in its `find`. 243's archive probe had the same assumption, which made its drop assertion skip silently on the clang legs rather than run. And the packer itself: with objects to drop and no archiver resolved it did nothing, quietly, leaving the published interface's objects inside the archive — two definitions of each published module's initialiser, resolved by link order. That is the failure class this whole feature exists to remove, so it is now refused with the reason. --- src/pack/library.cppm | 14 +++++++++++++- .../e2e/242_pack_library_interface_and_headers.sh | 6 +++++- tests/e2e/243_pack_library_interface_closure.sh | 2 +- 3 files changed, 19 insertions(+), 3 deletions(-) diff --git a/src/pack/library.cppm b/src/pack/library.cppm index 32990827..ceeabfed 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -212,7 +212,19 @@ run_library_pack(const LibraryPackPlan& plan) // Delete the objects of the units published as source. The consumer // compiles those itself; leaving them in the archive means two // definitions of the module initialiser, resolved by link order. - if (!leg.shared && !plan.dropObjects.empty() && !leg.archiveTool.empty()) { + // No archiver but objects to drop would leave the published interface's + // objects inside the archive — two definitions of the module + // initialiser, resolved by link order. Skipping that quietly is the + // exact failure class this feature exists to remove, so it is refused. + if (!leg.shared && !plan.dropObjects.empty() && leg.archiveTool.empty()) { + return std::unexpected(LibraryPackError{ std::format( + "no archiver was resolved for {}, so the published interface's " + "objects cannot be removed from '{}'.\n" + " Shipping them leaves two definitions of each published " + "module's initialiser in the consumer's link.", + leg.triple, name) }); + } + if (!leg.shared && !plan.dropObjects.empty()) { std::string cmd = mcpp::platform::shell::quote(leg.archiveTool.string()) + " d " + mcpp::platform::shell::quote(dst.string()); for (auto const& m : plan.dropObjects) diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh index 2b984c7a..f7eb5c30 100755 --- a/tests/e2e/242_pack_library_interface_and_headers.sh +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -76,7 +76,11 @@ PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" for f in mcpp.toml interface/mathkit.cppm interface/api.cppm include/mathkit_c.h; do [[ -f "$pkg/$f" ]] || { echo "package is missing $f"; find "$pkg" -type f; exit 1; } done -[[ -n "$(find "$pkg/lib" -name 'libmathkit.a' | head -1)" ]] || { +# NOT a hard-coded `libmathkit.a`: the artifact's name follows the ENVIRONMENT, +# not the OS — MinGW writes libfoo.a where MSVC writes foo.lib. That is exactly +# why `lib/` is keyed by triple, and hard-coding the GNU spelling here made this +# test fail on Windows against a package that had been built correctly. +[[ -n "$(find "$pkg/lib" -type f \( -name 'libmathkit.*' -o -name 'mathkit.*' \) | head -1)" ]] || { echo "package has no artifact under lib//"; find "$pkg" -type f; exit 1; } # Both lists are printed, and the implementation partition is on the right one. diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh index 173bcead..8f183579 100755 --- a/tests/e2e/243_pack_library_interface_closure.sh +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -84,7 +84,7 @@ grep -RIl 'secret_bias' "$pkg/interface" "$pkg/include" 2>/dev/null | grep -q . # A functional probe, not `command -v ar`: on macOS `ar` resolves to an xlings # shim that reports "not installed" and exits non-zero, which under `set -e` # kills the test instead of skipping the inspection. -archive="$(find "$pkg/lib" -name 'libmathkit.a' | head -1)" +archive="$(find "$pkg/lib" -type f \( -name 'libmathkit.*' -o -name 'mathkit.*' \) | head -1)" if [[ -n "$archive" ]] && members="$(ar t "$archive" 2>/dev/null)"; then echo "$members" | grep -q 'secret.m.o' || { echo "the implementation partition's OBJECT was dropped; nothing would link" From e4bd330c8079b9e239d532af5d5e6dea801b1261 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 18:17:11 +0800 Subject: [PATCH 10/31] docs(design): record how three-platform coverage was actually reached MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §11.2b: unrestricting the portable tests is what surfaced the scanner's implementation-partition bug, and fixing that — rather than widening a capability — is what made 242/243/244 pass on Linux, macOS and Windows. Also the per-platform table with a reason for every remaining skip. --- .../2026-08-17-library-distribution-design.md | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index 50f8141e..abd51eec 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -643,6 +643,44 @@ B1 / B2 建议**先单独开 issue 并各带一条回归测试**,不要埋进这 (MSYS 只转 argv 不转文件内容),而 lint 在所有平台跑,正是为了让 Linux 上的 reviewer 先于 Windows CI 发现。 +### 11.2b 「全面可用」是怎么做到的(不是靠放宽 requires) + +放开可移植测试之后 CI 报出两条真实失败,**追下去发现根因在扫描器,不在打包**: + +`module M:part;`(实现分区)与 `module M;`(实现单元)共用一个拼写、是两种声明, +而扫描器把前者记成**「requires `M:part`、provides 空」** —— 一个文件 requires +自己的名字。于是图里**没有**「import 分区的单元 → 定义分区的单元」这条边, +构建顺序无约束:GCC 与 macOS clang 靠各自的依赖扫描兜住, +**Windows clang 以 `failed to read compiled module` 失败**。 +同一处第二半:`import :part;` 的解析读 `u.provides`,而实现单元没有 provides ⇒ +`import :secret;` 停在字面 `:secret`。两平台都刷的那条 +`module 'M:part' imported but not provided in this build` **就是病因,读起来像提示**。 + +**实现分区在此之前 mcpp 里任何地方都没有测试覆盖** —— 库分发的 e2e 是第一个用它的。 +修好之后: + +| e2e | linux | macOS | windows | +|---|---|---|---| +| 242 布局 + 两种接口模式 | ✅ | ✅ | ✅ | +| 243 闭包 + 剔除集 + 分区告警 | ✅ | ✅ | ✅ | +| 244 三条拒绝 | ✅ | ✅ | ✅ | +| 249 workspace 根仍能打包 | ✅ | ✅ | ✅ | +| 250 `pack ` 打的是那个 | ✅ | ✅ | ✅ | +| 245 胖包(含原生构建) | ✅ | skip¹ | skip¹ | +| 246 `sources = []` | ✅ | skip² | skip² | +| 247 裸三元组条件 | ✅ | skip² | skip² | +| 248 跨 OS 边界的胖包(PE) | skip³ | skip³ | skip³ | +| 251 动态库包带两个名字 | ✅ | skip⁴ | skip⁴ | + +¹ 用 `--target x86_64-linux-musl`,需要 musl 工具链。 +² 从 `build.ninja` 里读 `-D` 拼写,是编译器特定的。 +³ 需要 `mingw-cross`,**没有任何 e2e job 装它** ⇒ 只在本机验过。 +⁴ `kind = "shared"` 按设计只支持 ELF。 + +**剩下的 skip 每一条都有真实理由,不是「gcc 能力凑不齐」。** +副作用:一条**与打包无关**的既有缺陷被修好了 —— 用实现分区的工程现在在 Windows 上 +能构建了。 + ### 11.3 我在验证里自己踩的坑 **`ls -t target/*/bin/mcpp | head -1` 挑到了陈旧/别的工具链的 fingerprint 目录 —— From 2ca68bac8c1aeea82ebc52c34c949e1bb3459092 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 18:37:56 +0800 Subject: [PATCH 11/31] test(e2e): three of the four remaining skips were laziness, not limits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Asked "what is the skip reason", and only one of the four survived the question. * 246 was gated on `gcc` because of copy-paste. Its probe greps a FILENAME out of build.ninja, not a compiler flag — nothing in it is toolchain- specific. Runs everywhere now. * 247 grepped `-D`, and MSVC spells that `/D` (dialect.cppm). The prefix was the only thing tying it to one compiler family; grepping the macro NAME is spelling-agnostic. Runs everywhere now. * 248 needed `mingw-cross`, which no e2e job installs — so it skipped in every job that exists, verified on a developer's machine and nowhere else. cross-build-test.yml's mingw job already names the e2e scripts the Linux shards skip for exactly this reason (102, 198, 240); 248 joins them, and the workflow header now says why that list is explicit rather than left to run_all's cap gating. * 251's reason held — `kind = "shared"` is ELF-only and plan.cppm refuses it — but it only tested the side that works, which cannot tell "the gate is handled" from "there is no gate". It is now two-sided: on ELF the package is produced and carries both of the library's names; off ELF the pack must be REFUSED and the message must name both the artifact kind and where it does work. Runs everywhere. That leaves 245, which needs two distinct buildable targets (gnu + musl) and so genuinely cannot run where only one exists. The mechanism it checks — one cfg() leg per triple, the native build included — is platform-independent and covered on Linux. The criterion, earned twice now: a test's `# requires:` has to be the real floor of the mechanism it verifies. Telling "a real limit" from "I did not think it through" is hard while writing it, so the reasons have to be interrogated one by one afterwards. --- .../2026-08-17-library-distribution-design.md | 50 ++++++++++++------- .github/workflows/cross-build-test.yml | 21 ++++++++ tests/e2e/246_explicit_empty_sources.sh | 6 ++- .../e2e/247_bare_triple_conditional_native.sh | 12 +++-- tests/e2e/251_pack_library_shared.sh | 33 ++++++++++-- 5 files changed, 96 insertions(+), 26 deletions(-) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index abd51eec..bf24ffab 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -659,28 +659,40 @@ reviewer 先于 Windows CI 发现。 **实现分区在此之前 mcpp 里任何地方都没有测试覆盖** —— 库分发的 e2e 是第一个用它的。 修好之后: -| e2e | linux | macOS | windows | -|---|---|---|---| -| 242 布局 + 两种接口模式 | ✅ | ✅ | ✅ | -| 243 闭包 + 剔除集 + 分区告警 | ✅ | ✅ | ✅ | -| 244 三条拒绝 | ✅ | ✅ | ✅ | -| 249 workspace 根仍能打包 | ✅ | ✅ | ✅ | -| 250 `pack ` 打的是那个 | ✅ | ✅ | ✅ | -| 245 胖包(含原生构建) | ✅ | skip¹ | skip¹ | -| 246 `sources = []` | ✅ | skip² | skip² | -| 247 裸三元组条件 | ✅ | skip² | skip² | -| 248 跨 OS 边界的胖包(PE) | skip³ | skip³ | skip³ | -| 251 动态库包带两个名字 | ✅ | skip⁴ | skip⁴ | - -¹ 用 `--target x86_64-linux-musl`,需要 musl 工具链。 -² 从 `build.ninja` 里读 `-D` 拼写,是编译器特定的。 -³ 需要 `mingw-cross`,**没有任何 e2e job 装它** ⇒ 只在本机验过。 -⁴ `kind = "shared"` 按设计只支持 ELF。 - -**剩下的 skip 每一条都有真实理由,不是「gcc 能力凑不齐」。** +| e2e | linux | macOS | windows | 跑在哪 | +|---|---|---|---|---| +| 242 布局 + 两种接口模式 | ✅ | ✅ | ✅ | e2e 分片 | +| 243 闭包 + 剔除集 + 分区告警 | ✅ | ✅ | ✅ | e2e 分片 | +| 244 三条拒绝 | ✅ | ✅ | ✅ | e2e 分片 | +| 246 `sources = []` | ✅ | ✅ | ✅ | e2e 分片 | +| 247 裸三元组条件 | ✅ | ✅ | ✅ | e2e 分片 | +| 249 workspace 根仍能打包 | ✅ | ✅ | ✅ | e2e 分片 | +| 250 `pack ` 打的是那个 | ✅ | ✅ | ✅ | e2e 分片 | +| 251 动态库:产出正确 **或** 明确拒绝 | ✅ 产出 | ✅ 拒绝 | ✅ 拒绝 | e2e 分片 | +| 248 跨 OS 边界的胖包(PE) | ✅ | — | — | **`cross-build-test.yml` 的 mingw job** | +| 245 胖包(含原生构建) | ✅ | skip | skip | e2e 分片 | + +**只剩 245 一条在 Linux 之外跳过,而它是真限制**:那条测试要**两个都能构建的 +target**(`x86_64-linux-gnu` + `x86_64-linux-musl`),而 macOS / Windows 的 CI 上 +不存在第二个现成可用的 target。它验的机制(每条腿一个 `cfg()`、原生构建也命中) +本身与平台无关,已在 Linux 上覆盖。 + +**另外四条原本的 skip 理由,查下来三条是我偷懒:** + +| 原理由 | 真相 | +|---|---| +| 246「读编译器特定的 flag 拼写」 | **错的** —— 它 grep 的是**文件名**。`# requires: gcc` 是复制粘贴来的 | +| 247「读编译器特定的 flag 拼写」 | **半对** —— 它 grep 了 `-D` 前缀,而 MSVC 是 `/D`(`dialect.cppm:200`)。改成 grep **宏名**即可移植 | +| 248「需要 mingw-cross,没有 job 装它」 | **可修** —— `cross-build-test.yml` 的 mingw job **已经**在显式点名跑那些「Linux 分片因缺 mingw-cross 而跳过」的 e2e(102/198/240),照办即可 | +| 251「`shared` 仅 ELF」 | **理由成立,但只测了一侧** —— 补成两侧断言:ELF 上产出且带两个名字,非 ELF 上**必须拒绝且说明原因**。只测能用的那侧分不清「门被处理」与「没有门」 | + 副作用:一条**与打包无关**的既有缺陷被修好了 —— 用实现分区的工程现在在 Windows 上 能构建了。 +**判据(第二次得到验证):一条测试的 `# requires:` 必须是它所验证机制的真实下限。** +写下它的时候很难分辨「真限制」与「我没想清楚」,所以事后必须**逐条追问理由**; +四条里三条经不起追问。 + ### 11.3 我在验证里自己踩的坑 **`ls -t target/*/bin/mcpp | head -1` 挑到了陈旧/别的工具链的 fingerprint 目录 —— diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 379def22..3468bd36 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -34,6 +34,10 @@ name: cross-build-test # qemu-aarch64 is the CI proxy for "does this cross artefact actually execute". # # ── NOT here ─────────────────────────────────────────────────────────────── +# * The e2e scripts this job names explicitly (102, 198, 240, 248) are the +# ones the ordinary Linux shards SKIP for want of `mingw-cross`. They are +# listed in the job rather than left to run_all's cap gating precisely so +# they cannot end up skipping everywhere at once. # * Same-arch builds (host arch == target arch) are NOT cross. The native musl # static build `--target x86_64-linux-musl` (x86_64 host) is exercised by # ci-linux.yml's "Toolchain: musl-gcc" step, and release.yml for the static @@ -320,6 +324,23 @@ jobs: export MCPP_VENDORED_XLINGS="$XLINGS_BIN" bash tests/e2e/240_pack_pe_zip_cross.sh + # A LIBRARY package whose legs cross an OS boundary, for the same reason + # as the three above: this is the only job with a MinGW cross toolchain. + # + # It is not redundant with 245 (which covers the fat-package mechanism + # with gnu + musl and therefore runs on every ordinary Linux shard). The + # leg added here changes BINARY FORMAT, and it is the case that proves + # `lib/` has to be keyed by triple rather than by OS: MinGW and MSVC are + # both "windows" and write `libfoo.a` and `foo.lib` respectively. + # + # Without this step the test would carry `# requires: mingw-cross` and + # skip in every job that exists — verified on a developer's machine and + # nowhere else, while the suite reported green. + - name: "e2e: pack a library across an OS boundary (PE leg)" + run: | + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + bash tests/e2e/248_pack_library_fat_pe_leg.sh + # ── windows → linux ─────────────────────────────────────────────────────── # The mirror of mingw-cross-wine. Two jobs because a Windows runner cannot # execute the ELF it produces; the artefact is handed to a Linux job and diff --git a/tests/e2e/246_explicit_empty_sources.sh b/tests/e2e/246_explicit_empty_sources.sh index b2657dc1..50a81d8a 100755 --- a/tests/e2e/246_explicit_empty_sources.sh +++ b/tests/e2e/246_explicit_empty_sources.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: the probe below greps a FILENAME out of build.ninja, not a +# compiler flag, so nothing here is toolchain-specific. It carried +# `# requires: gcc` for one round out of copy-paste, and that capability is +# Linux-only by design — it skipped on macOS and Windows for no reason.) # 246_explicit_empty_sources.sh — `sources = []` means "compile nothing", and # omitting the key still means "the default glob". # diff --git a/tests/e2e/247_bare_triple_conditional_native.sh b/tests/e2e/247_bare_triple_conditional_native.sh index 714aabc8..67762254 100755 --- a/tests/e2e/247_bare_triple_conditional_native.sh +++ b/tests/e2e/247_bare_triple_conditional_native.sh @@ -1,5 +1,9 @@ #!/usr/bin/env bash -# requires: gcc +# requires: +# (no capability: the probe greps the MACRO NAME, never the `-D` prefix — MSVC +# spells that `/D`, and the prefix was the only thing that tied this test to +# one compiler family. The predicate it checks is compiler-independent, so the +# test has to run everywhere the predicate does.) # 247_bare_triple_conditional_native.sh — `[target.''.build]` applies to # a NATIVE build, not only to one with an explicit `--target`. # @@ -37,9 +41,11 @@ cxxflags = ["-DMCPP_BARE_TRIPLE=1"] cxxflags = ["-DMCPP_CFG_ALIAS=1"] EOF -count() { # $1 = define name +count() { # $1 = macro name + # The NAME, not `-D`: the prefix is dialect-specific (`/D` on MSVC), + # and what this test is about is whether the section applied at all. local nj; nj="$(find target -name build.ninja | head -1)" - grep -c -- "-D$1" "$nj" || true + grep -c -- "$1" "$nj" || true } cd probe diff --git a/tests/e2e/251_pack_library_shared.sh b/tests/e2e/251_pack_library_shared.sh index cf00950e..43c94373 100755 --- a/tests/e2e/251_pack_library_shared.sh +++ b/tests/e2e/251_pack_library_shared.sh @@ -1,7 +1,13 @@ #!/usr/bin/env bash -# requires: elf -# 251_pack_library_shared.sh — a `kind = "shared"` package carries BOTH of the -# library's names, and a consumer can actually start. +# requires: +# 251_pack_library_shared.sh — a `kind = "shared"` package is either produced +# CORRECTLY or refused CLEARLY. Never silently wrong. +# +# Both halves, because testing only the working one cannot tell "the gate is +# handled" from "there is no gate". `kind = "shared"` is ELF-only today +# (src/build/plan.cppm refuses it and says why), so on macOS and Windows the +# assertion is that the refusal happens and names the reason — a `# requires: +# elf` here would have left that side unobserved. # # A shared library is LINKED by `lib.so` and FOUND at run time by its # SONAME, and those are two different filenames. The first version of this @@ -41,6 +47,27 @@ soname = "libmathkit.so.1" EOF cd mathkit + +# ── the non-ELF side: refuse, and say why ────────────────────────────── +if [[ "$(uname -s)" != "Linux" ]]; then + if "$MCPP" pack mathkit-shared > refuse.log 2>&1; then + cat refuse.log + echo "FAIL: a shared library was packed on a platform that cannot link one" + exit 1 + fi + grep -qi 'shared librar' refuse.log || { + cat refuse.log + echo "FAIL: it refused, but the message does not say the artifact kind is the problem" + exit 1; } + grep -qi 'linux\|elf' refuse.log || { + cat refuse.log + echo "FAIL: the refusal does not say where shared libraries DO work" + exit 1; } + echo "PASS: a shared library package is refused, with the reason, off ELF" + exit 0 +fi + +# ── the ELF side: produce it, with both names ────────────────────────── "$MCPP" pack mathkit-shared > pack.log 2>&1 || { cat pack.log; echo "shared pack failed"; exit 1; } pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" # The manifest below is FILE CONTENT: on Git Bash a shell-spelled From e3e0276a186b41b3b83ca94a8bd5ff9ecc426559 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 18:48:56 +0800 Subject: [PATCH 12/31] test(e2e): ask mcpp for the host triple instead of pattern-matching for one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 247 derived the host triple with a regex for `--` and fell back to `x86_64-linux-gnu`. macOS's canonical triple is `aarch64-macos` — two segments, no env — so the match failed, the fallback was used, and the test asserted that a *Linux* section should apply to a macOS build. It then reported "the bare triple was inert" against a product behaving correctly. `target//` is mcpp's own answer to the same question. The test now builds once with nothing conditional, reads the directory name, and writes the real manifest from that — and the cfg() control is keyed on the same value's arch rather than on `unix`, which is false on Windows. The shape is the one this PR keeps meeting: a second, independent derivation of something mcpp already computes, disagreeing with it on the platform nobody checked. --- .../e2e/247_bare_triple_conditional_native.sh | 31 ++++++++++++++++--- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/tests/e2e/247_bare_triple_conditional_native.sh b/tests/e2e/247_bare_triple_conditional_native.sh index 67762254..11b141fb 100755 --- a/tests/e2e/247_bare_triple_conditional_native.sh +++ b/tests/e2e/247_bare_triple_conditional_native.sh @@ -21,11 +21,34 @@ TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT cd "$TMP" -host_triple="$("$MCPP" self env 2>/dev/null | grep -oE '[a-z0-9_]+-(linux|macos|windows)-[a-z0-9]+' | head -1)" -[[ -n "$host_triple" ]] || host_triple="x86_64-linux-gnu" - +# The host triple, asked of mcpp rather than guessed. +# +# The first version pattern-matched `mcpp self env` for +# `--` and fell back to `x86_64-linux-gnu`. macOS's canonical +# triple is `aarch64-macos` — TWO segments, no env — so the match failed, the +# fallback was used, and the test then asserted that a *Linux* section applied +# to a macOS build. It reported "the bare triple was inert" against a product +# that was behaving correctly. +# +# `target//` is mcpp's own answer to the same question, so take it from +# there: build once with nothing conditional, read the directory name, then +# write the real manifest. mkdir -p probe/src echo 'int main() { return 0; }' > probe/src/main.cpp +cat > probe/mcpp.toml <<'EOF' +[package] +name = "probe" +version = "0.1.0" +[targets.probe] +kind = "bin" +main = "src/main.cpp" +EOF +( cd probe && "$MCPP" build > discover.log 2>&1 ) \ + || { cat probe/discover.log; echo "discovery build failed"; exit 1; } +host_triple="$(ls probe/target | head -1)" +[[ -n "$host_triple" ]] || { ls -R probe/target; echo "no target/ dir"; exit 1; } +echo "host triple: $host_triple" + cat > probe/mcpp.toml < Date: Mon, 17 Aug 2026 22:43:15 +0800 Subject: [PATCH 13/31] review: verify the compatibility claim, and stop guessing at the archiver MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four things a deeper read of this PR turned up. **The old-client claim was unverified.** The PR body and docs/12 both say an mcpp that predates this feature still builds against these packages — that is the entire justification for adding no manifest section and no key, and nothing checked it. It is true (2026.8.15.3 consumes a package from 2026.8.17.2 and runs it), and e2e 252 now pins it in two halves: a portable static check that the generated manifest's sections are a subset of the pre-existing vocabulary, and the real thing against `$MCPP_BOOT`, the released binary each CI job bootstraps from. The three e2e workflows now export it, so the real half runs rather than noting itself out. **The packer spoke `ar` to whatever archiver it was handed.** `archive_tool` returns LIB.EXE for MSVC, which spells member removal `/REMOVE:` — one flag per member, archive last — not `d …`. On mcpp's Windows CI the archiver is clang's llvm-ar, which takes the GNU form, so the difference was invisible. How to speak to a tool is what mcpp.toolchain.dialect is for, so the removal spelling lives there now, next to `archiveCmd`, and the packer substitutes rather than assumes. The MSVC row is marked untested, and a failure reports the command it ran. **A duplicate partition provider now says so.** Two files declaring `module m:p;` used to be accepted silently; the scanner names both files. Those programs were always ill-formed, but it is a new failure path and belongs in the changelog with the other behaviour change (`sources = []`). **The module graph is moved into BuildContext, not copied.** Nothing reads `scan.graph` after that point. --- .github/workflows/ci-linux-e2e.yml | 5 + .github/workflows/ci-macos-e2e.yml | 5 + .github/workflows/ci-windows-e2e.yml | 5 + CHANGELOG.md | 6 ++ src/build/prepare.cppm | 2 +- src/pack/library.cppm | 41 ++++++-- src/pack/library_pipeline.cppm | 5 + src/toolchain/dialect.cppm | 26 ++++++ tests/e2e/252_pack_library_old_client.sh | 113 +++++++++++++++++++++++ 9 files changed, 201 insertions(+), 7 deletions(-) create mode 100755 tests/e2e/252_pack_library_old_client.sh diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 96529460..0f167ee6 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -63,6 +63,11 @@ jobs: # Point the e2e runner at the freshly-built binary, not the # bootstrap one. Tests cd into mktemp -d, so $MCPP must be # absolute or the relative path breaks under the temp cwd. + # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the + # claim that an older client can still build against a package the PR + # produces is only worth making if something checks it against a real + # old binary. Captured before $MCPP is repointed at the fresh build. + export MCPP_BOOT="$MCPP" MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") test -x "$MCPP" export MCPP diff --git a/.github/workflows/ci-macos-e2e.yml b/.github/workflows/ci-macos-e2e.yml index 4c30b421..5d02b76e 100644 --- a/.github/workflows/ci-macos-e2e.yml +++ b/.github/workflows/ci-macos-e2e.yml @@ -41,6 +41,11 @@ jobs: # Per-test 600s timeout lives in run_all.sh. timeout-minutes: 25 run: | + # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the + # claim that an older client can still build against a package the PR + # produces is only worth making if something checks it against a real + # old binary. Captured before $MCPP is repointed at the fresh build. + export MCPP_BOOT="$MCPP" MCPP=$(find target -path "*/bin/mcpp" | head -1) MCPP=$(cd "$(dirname "$MCPP")" && pwd)/$(basename "$MCPP") test -x "$MCPP" diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index a6ff7086..05c3b43a 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -63,6 +63,11 @@ jobs: # Per-test 600s timeout lives in run_all.sh. timeout-minutes: 25 run: | + # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the + # claim that an older client can still build against a package the PR + # produces is only worth making if something checks it against a real + # old binary. Captured before $MCPP is repointed at the fresh build. + export MCPP_BOOT="${MCPP:-$MCPP_BOOT}" export MCPP="$MCPP_SELF" export MCPP_VENDORED_XLINGS="$XLINGS_BIN" export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL diff --git a/CHANGELOG.md b/CHANGELOG.md index a25730f0..4d9f8366 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,12 @@ 用到它才暴露出来。现在扫描器记 `provides = M:part` 并标 `providesInterface = false`;`import :part;` 按 TU 自己所属的模块名解析。 + **⚠️ 两处行为变化**:①两个文件声明同一个分区(`module m:p;` × 2)现在会被 + **拒绝并点名两个文件**,此前是静默接受 —— 那种程序本来就 ill-formed, + 但它是一条新的失败路径;②`sources = []` 从「等于不写」变成「什么都不编」, + 一个真写了 `sources = []` 又依赖默认 glob 的工程会发现产物变空(此前无法表达 + 「什么都不编」,所以这种写法只可能是误解)。 + - **`[target.'<三元组>'.build]` 在没有 `--target` 时从不命中。** 同一个语句的两种拼写互相矛盾:`cfg(linux)` 在原生构建上命中, diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index e539dd84..0a436677 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -5225,7 +5225,7 @@ prepare_build(bool print_fingerprint, // away — a CompileUnit records what to compile, not what it provides — so // the packer would otherwise have to scan the tree a second time and could // then disagree with the build about what the package even contains. - ctx.graph = scan.graph; + ctx.graph = std::move(scan.graph); // mcpp#407. Both callers that produce a non-plain graph arrive here the // same way: dev-dependencies enabled, synthetic test targets appended. The // resulting `default` line names the test binaries and omits the package's diff --git a/src/pack/library.cppm b/src/pack/library.cppm index ceeabfed..baf079b9 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -51,6 +51,12 @@ struct LibraryLeg { std::string abiTag; std::string buildKey; std::string linkName; // the -l argument, e.g. "mathkit" + // How THIS leg's archiver spells "delete a member", from + // mcpp.toolchain.dialect. `ar` and `llvm-ar` take `d …`; + // LIB.EXE takes `/REMOVE:` per member with the archive last. The + // packer must not pick one — see the dialect's note. + std::string removeArg; // "d" | "/REMOVE:{}" + bool removeArchiveFirst = true; // The SONAME the artifact declares, when it declares one. A shared library // is FOUND at run time by this name and LINKED by `lib.so`, and // those are two different filenames — so a package that ships only the @@ -225,15 +231,38 @@ run_library_pack(const LibraryPackPlan& plan) leg.triple, name) }); } if (!leg.shared && !plan.dropObjects.empty()) { - std::string cmd = mcpp::platform::shell::quote(leg.archiveTool.string()) - + " d " + mcpp::platform::shell::quote(dst.string()); - for (auto const& m : plan.dropObjects) - cmd += " " + mcpp::platform::shell::quote(m); + std::string cmd = mcpp::platform::shell::quote(leg.archiveTool.string()); + auto member_words = [&] { + std::string w; + for (auto const& m : plan.dropObjects) { + // `{}` means one flag per member (LIB.EXE); its absence + // means one verb followed by every member (ar). + auto pos = leg.removeArg.find("{}"); + if (pos == std::string::npos) { w += " " + mcpp::platform::shell::quote(m); continue; } + auto arg = leg.removeArg; + arg.replace(pos, 2, m); + w += " " + mcpp::platform::shell::quote(arg); + } + return w; + }; + if (leg.removeArchiveFirst) { + if (leg.removeArg.find("{}") == std::string::npos) + cmd += " " + leg.removeArg; + cmd += " " + mcpp::platform::shell::quote(dst.string()); + cmd += member_words(); + } else { + cmd += member_words(); + cmd += " " + mcpp::platform::shell::quote(dst.string()); + } auto r = mcpp::platform::process::capture(cmd + " 2>&1"); if (r.exit_code != 0) { return std::unexpected(LibraryPackError{ std::format( - "cannot drop published interface objects from '{}' (rc={}): {}", - dst.string(), r.exit_code, r.output) }); + "cannot drop published interface objects from '{}' (rc={}).\n" + " command: {}\n" + " output : {}\n" + " Leaving them in would give the consumer two definitions of " + "each published module's initialiser.", + dst.string(), r.exit_code, cmd, r.output) }); } } diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index d9f25a27..4462deec 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -31,6 +31,7 @@ import mcpp.pack; import mcpp.pack.abi_tag; import mcpp.pack.interface; import mcpp.pack.library; +import mcpp.toolchain.dialect; import mcpp.toolchain.registry; import mcpp.toolchain.triple; import mcpp.ui; @@ -284,6 +285,10 @@ export int build_and_pack_library(const std::string& targetName, .abiTag = tag.str(), .buildKey = ctx->fp.hex, .linkName = targetName, + .removeArg = std::string( + mcpp::toolchain::dialect_for(ctx->tc).archiveRemoveArg), + .removeArchiveFirst = + mcpp::toolchain::dialect_for(ctx->tc).archiveRemoveTakesArchiveFirst, .soname = target->soname, .shared = shared, }); diff --git a/src/toolchain/dialect.cppm b/src/toolchain/dialect.cppm index 1adc549f..dff73c5a 100644 --- a/src/toolchain/dialect.cppm +++ b/src/toolchain/dialect.cppm @@ -89,6 +89,23 @@ struct CommandDialect { // Full ninja command template for static archives. std::string_view archiveCmd; // "$ar rcs $out $in" | "$ar /nologo /OUT:$out $in" + + // How this archiver DELETES a member, as argv words before the member + // names. `mcpp pack` needs it: a library package publishes its interface + // units as source, so their objects have to come OUT of the archive, or the + // consumer links two definitions of each published module's initialiser. + // + // It lives here rather than in the packer because "how do you speak to the + // archiver" is exactly what this table is for. Hard-coding `ar`-style `d` + // there would be right for GNU and llvm-ar and wrong for LIB.EXE, which + // spells it `/REMOVE:` and takes one flag per member — a difference + // in ARITY as well as in spelling, which is why this is a template with a + // placeholder rather than a prefix string. + // + // `{}` is substituted with the member name; the words are joined with + // spaces after the archive path. + std::string_view archiveRemoveArg; // "d" needs no {} | "/REMOVE:{}" + bool archiveRemoveTakesArchiveFirst = true; }; // Dialect lookup. GCC / Clang / MinGW → gnu; MSVC → msvc. @@ -189,6 +206,9 @@ constexpr CommandDialect kGnuDialect{ .rspfileLink = false, .linkStyle = CommandDialect::LinkStyle::Driver, .archiveCmd = "$ar rcs $out $in", + // `ar d ...` — one verb, then every member. + .archiveRemoveArg = "d", + .archiveRemoveTakesArchiveFirst = true, }; // Native cl.exe. Unreachable in builds until the MSVC backend lands @@ -218,6 +238,12 @@ constexpr CommandDialect kMsvcDialect{ .rspfileLink = true, .linkStyle = CommandDialect::LinkStyle::SeparateLinker, .archiveCmd = "$ar /nologo /OUT:$out $in", + // `LIB /REMOVE: … ` — one flag PER member, and the + // archive last. Untested against a real LIB.EXE (mcpp's Windows CI packs + // with clang, whose llvm-ar takes the GNU form), so a failure here is + // reported with the command that produced it rather than swallowed. + .archiveRemoveArg = "/REMOVE:{}", + .archiveRemoveTakesArchiveFirst = false, }; } // namespace diff --git a/tests/e2e/252_pack_library_old_client.sh b/tests/e2e/252_pack_library_old_client.sh new file mode 100755 index 00000000..581ca15f --- /dev/null +++ b/tests/e2e/252_pack_library_old_client.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash +# requires: +# 252_pack_library_old_client.sh — an mcpp that predates library packaging must +# still BUILD against a package produced by one that has it. +# +# That claim is the reason the generated manifest introduces no section and no +# key: everything in it — `sources`, `include_dirs`, `[modules] exports`, a +# `cfg(...)` block per leg, `[[runtime.artifacts]]` — was already parsed before +# this feature existed. An older client reads the package and links it; what it +# does not do is run the gates, because it has no way to know that +# `provenance = "mcpp-pack …"` means anything. +# +# Two halves, because only one of them can run everywhere: +# +# 1. STATIC — the generated manifest's top-level sections are a subset of the +# vocabulary that predates this feature. Portable, and it is the actual +# invariant rather than a proxy for it. +# 2. REAL — consume the package with $MCPP_BOOT, the released mcpp each CI job +# bootstraps from. Skipped with a loud note when that is not available, +# never silently. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +PKG_HOST="$(host_path "$pkg")" + +# ── 1. no section outside the pre-existing vocabulary ────────────────── +# +# Listed literally rather than derived: the point is that this set was frozen +# before the feature, so a new entry has to be added here deliberately — and +# adding one is exactly the moment to ask whether older clients can still read +# the package. +# `\[\[?` covers both a table and an array-of-tables header: `[[runtime.artifacts]]` +# is the same section as `[runtime]` for this purpose, and the first version of +# this pattern matched only the single-bracket form — so it flagged the very +# section the design deliberately reuses. +known='^\[\[?(package|build|modules|targets\.|target\.|dependencies|dev-dependencies|runtime|profile\.|features|lib|pack|workspace|indices|resources|xlings|capabilities|tools)' +bad="$(grep -E '^\[' "$pkg/mcpp.toml" | grep -Ev "$known" || true)" +[[ -z "$bad" ]] || { + echo "FAIL: the generated manifest uses sections an older mcpp cannot read:" + printf '%s\n' "$bad" + echo " Either express the fact with an existing key, or accept that packages" + echo " need a version floor — and say so in docs/12." + exit 1; } + +# ── 2. the released client actually builds against it ────────────────── +mkdir -p "$TMP/app/src" +cat > "$TMP/app/src/main.cpp" <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > "$TMP/app/mcpp.toml" < new.log 2>&1 ) \ + || { cat "$TMP/app/new.log"; echo "the PR binary could not consume its own package"; exit 1; } +grep -q 'ok=42' "$TMP/app/new.log" || { cat "$TMP/app/new.log"; echo "wrong answer"; exit 1; } + +if [[ -n "${MCPP_BOOT:-}" && -x "${MCPP_BOOT}" ]] \ + && [[ "$("$MCPP_BOOT" --version 2>/dev/null)" != "$("$MCPP" --version 2>/dev/null)" ]]; then + echo "old client: $("$MCPP_BOOT" --version)" + rm -rf "$TMP/app/target" + ( cd "$TMP/app" && "$MCPP_BOOT" run > old.log 2>&1 ) || { + cat "$TMP/app/old.log" + echo "FAIL: the released mcpp cannot build against a package this one produced." + echo " The compatibility claim in docs/12 is then false: such packages" + echo " need a version floor, and publishing one without it bricks older" + echo " clients rather than degrading them." + exit 1; } + grep -q 'ok=42' "$TMP/app/old.log" || { + cat "$TMP/app/old.log"; echo "the old client built it but ran it wrong"; exit 1; } + echo "PASS: a released mcpp builds and runs against a package from this one" +else + echo "NOTE: \$MCPP_BOOT is unset or identical to \$MCPP — the real old-client" + echo " check did not run here. The static section-vocabulary check did." + echo "PASS: the generated manifest introduces no section an older mcpp cannot read" +fi From 69be04cd780835c1a76eebd0cdda21727abbfb95 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 23:07:26 +0800 Subject: [PATCH 14/31] test(e2e): an empty --version is not an old client MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 252's guard compared `$MCPP_BOOT --version` against the PR binary's and treated "different" as "found an old client". In CI that entry is an xvm SHIM, and a shim resolves against the home it is asked in — under the e2e suite's environment it answers `xlings: 'mcpp' is not installed` and prints nothing. The empty string duly differed, so the test ran the real check against a binary that cannot run at all and reported a COMPATIBILITY FAILURE against a package that is perfectly readable. Three legs red for a reason that was in the test. The guard now requires a version-SHAPED answer. Anything else means "no usable old binary here", which is a note naming what it got, not a verdict — and the static half (the generated manifest's sections are a subset of the pre-existing vocabulary) still runs everywhere. Consequence worth stating: the real old-client check runs where a released mcpp binary is directly executable — locally, and in any job that points MCPP_BOOT at one rather than at a shim. It passed against 2026.8.15.3. --- tests/e2e/252_pack_library_old_client.sh | 36 ++++++++++++++++++++---- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/tests/e2e/252_pack_library_old_client.sh b/tests/e2e/252_pack_library_old_client.sh index 581ca15f..90ae2112 100755 --- a/tests/e2e/252_pack_library_old_client.sh +++ b/tests/e2e/252_pack_library_old_client.sh @@ -92,9 +92,27 @@ EOF || { cat "$TMP/app/new.log"; echo "the PR binary could not consume its own package"; exit 1; } grep -q 'ok=42' "$TMP/app/new.log" || { cat "$TMP/app/new.log"; echo "wrong answer"; exit 1; } -if [[ -n "${MCPP_BOOT:-}" && -x "${MCPP_BOOT}" ]] \ - && [[ "$("$MCPP_BOOT" --version 2>/dev/null)" != "$("$MCPP" --version 2>/dev/null)" ]]; then - echo "old client: $("$MCPP_BOOT" --version)" +# ⚠️ The boot entry each CI job bootstraps from is an xvm SHIM, and a shim +# resolves against the home it is asked in — under the e2e suite's environment +# it answers `xlings: 'mcpp' is not installed` and prints NOTHING for +# `--version`. The first version of this guard compared that empty string +# against the PR binary's version, found them "different", and concluded it had +# found an old client — then reported a compatibility failure against a package +# that is perfectly readable. So the guard demands a version-SHAPED answer; +# anything else means "no usable old binary here", which is a note, not a +# verdict. +boot_ver="" +new_ver="$("$MCPP" --version 2>/dev/null || true)" +if [[ -n "${MCPP_BOOT:-}" && -x "${MCPP_BOOT}" ]]; then + boot_ver="$("$MCPP_BOOT" --version 2>/dev/null || true)" +fi +usable=0 +case "$boot_ver" in + mcpp\ [0-9]*) usable=1 ;; +esac + +if [[ "$usable" == 1 && "$boot_ver" != "$new_ver" ]]; then + echo "old client: $boot_ver" rm -rf "$TMP/app/target" ( cd "$TMP/app" && "$MCPP_BOOT" run > old.log 2>&1 ) || { cat "$TMP/app/old.log" @@ -107,7 +125,15 @@ if [[ -n "${MCPP_BOOT:-}" && -x "${MCPP_BOOT}" ]] \ cat "$TMP/app/old.log"; echo "the old client built it but ran it wrong"; exit 1; } echo "PASS: a released mcpp builds and runs against a package from this one" else - echo "NOTE: \$MCPP_BOOT is unset or identical to \$MCPP — the real old-client" - echo " check did not run here. The static section-vocabulary check did." + if [[ -n "${MCPP_BOOT:-}" && "$usable" != 1 ]]; then + echo "NOTE: \$MCPP_BOOT=${MCPP_BOOT} does not answer --version with a version" + echo " (got: '${boot_ver}'), so it is not a usable old client here — a" + echo " shim resolves against the home it is asked in. The REAL old-client" + echo " check therefore did not run; run it with MCPP_BOOT pointing at a" + echo " released mcpp binary directly." + else + echo "NOTE: \$MCPP_BOOT is unset or identical to \$MCPP — the real old-client" + echo " check did not run here." + fi echo "PASS: the generated manifest introduces no section an older mcpp cannot read" fi From b21763e8bf73c51a89db35a9fb3b444805f96061 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Mon, 17 Aug 2026 23:31:07 +0800 Subject: [PATCH 15/31] =?UTF-8?q?docs(design):=20=C2=A712=20=E2=80=94=20th?= =?UTF-8?q?e=20deep=20review,=20including=20five=20items=20still=20open?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Why 245/248 cannot run elsewhere, from host_can_serve rather than from the test's capability table: macOS serves exactly one target so a fat package is structurally impossible there, while Windows serves three — so the Windows equivalent is a coverage gap, not a product gap, and it has independent value (mathkit.lib next to libmathkit.a in one package is the strongest evidence that lib/ must be keyed by triple). Plus the dependency-direction invariant for src/pack (only library_pipeline may import mcpp.build.*), the two deliberate behaviour changes, and five open items — including two I have to report rather than claim: a scan_overrides-declared implementation partition is mis-flagged as an interface (its source would be published, and the schema has nowhere to say otherwise), and the design's promised [package].platforms coverage warning is simply not implemented. --- .../2026-08-17-library-distribution-design.md | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index bf24ffab..bb2d8897 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -710,3 +710,75 @@ clang 构建的二进制,而 clang 构建的 mcpp 在本机会段错误)。 | e2e | 242–251(10 个) | | 文档 | `docs/12-binary-distribution.md` + zh;`docs/02`/`05`/README 索引 | | 示例 | `examples/05-lib-distribution/producer`、`examples/05-lib-distribution/consumer` | + + +--- + +## 12. 深度 review(2026-08-17,PR #451 CI 全绿之后) + +### 12.1 245 / 248 在其他平台「没有」的真实原因 + +**权威答案是 `host_can_serve`(`registry.cppm:542-566`),不是测试的能力表。** + +| target | linux host | macOS host | windows host | +|---|---|---|---| +| `-linux-gnu` | ✅ | ❌ | ❌ | +| `*-linux-musl` | ✅ 任意 arch | ❌ | ✅ 仅 host arch | +| `*-windows-gnu`(mingw) | ✅ | ❌ | ✅ | +| `*-windows-msvc` | ❌ | ❌ | ✅ | +| `*-macos` | ❌ | ✅ | ❌ | + +- **macOS 只能服务一个 target。** 胖包至少要两条腿 ⇒ 在 macOS 上 + **结构上不可能**,不是测试没写。代码自己的注释: + *"macOS has no Linux-targeting payload at all."* +- **Windows 能服务三个**(msvc / mingw / linux-musl)⇒ **245 与 248 的等价用例在 + Windows 上是可行的,只是没做。** 这是**覆盖缺口**,不是产品缺口,而且它有独立价值: + `mathkit.lib`(MSVC)+ `libmathkit.a`(MinGW)同处一包,是「`lib/` 必须按三元组分」 + 最强的证据。代价是要在 Windows 的 e2e job 里装 `xim:mingw-gcc`(e2e 97 已经在 + 另一个 job 里装它)。 +- **248 需要一个能产 PE 的工具链**:Linux 上是 `mingw-cross`(现在跑在 + `cross-build-test.yml` 的 mingw job),Windows 上是原生 mingw/msvc(同上缺口)。 + +### 12.2 架构:依赖方向是可陈述的不变量 + +`src/pack/` 现在是两层,**必须保持**: + +``` +叶层(prepare 可以 import 它们): abi_tag digest interface manifest_emit prebuilt route library +编排层(它 import prepare): library_pipeline +``` + +实测的 import 图里,**只有 `library_pipeline` 碰 `mcpp.build.*`**;而 +`prepare` 只 import `pack.abi_tag` 与 `pack.prebuilt`(两个叶子)。 +**不变量:除 `library_pipeline` 外,`src/pack/` 下不得 import `mcpp.build.*`** —— +否则 prepare ↔ pack 成环。 + +三处收敛(都是把「同一个决定的第二处推导」删掉,不是加抽象): +- `object_filename_for` 从 plan.cppm 搬进 `mcpp.source_kind` —— 策略与格式化同处; +- 「怎么删归档成员」进 `dialect.cppm` 的 `archiveRemoveArg` —— 与 `archiveCmd` 同处, + packer 不再默认 `ar` 语法(**这条是 review 发现的,不是测试发现的**); +- 解析后的三元组进 `cfgpred::Ctx` —— 删掉裸三元组分支那个第二答题者。 + +### 12.3 兼容性:两处**故意的**行为变化 + +| 变化 | 谁会受影响 | +|---|---| +| `sources = []` 从「等于不写」变成「什么都不编」 | 真写了它又依赖默认 glob 的工程。此前**无法表达**「什么都不编」,所以这种写法只可能是误解 | +| 两个文件声明同一个分区 ⇒ 拒绝并点名两个文件 | 本来就 ill-formed 的程序。**但它是一条新的失败路径** | + +扫描器的改动**只会增加图上的边**,不会减少 —— 边缺失只可能允许错误的顺序, +所以不存在「依赖那条缺失边」的工程。 + +老客户端兼容:**两侧都验了**(静态:生成的 manifest 段 ⊆ 既有词汇,三平台跑; +真实:2026.8.15.3 消费并运行成功,本机)。⚠️ CI 里真实那半跑不了,因为 +`$MCPP_BOOT` 是 xvm shim(在 e2e 改过的环境里报「未安装」),这一点写在 252 的头部。 + +### 12.4 仍然开着的五项(合入前请裁决) + +| # | 项 | 我的建议 | +|---|---|---| +| **O1** | **扫描器改动是全 PR 里影响面最大的一块,而它与打包无关** | **拆成独立 PR 先合**,这样它能被独立 revert;库分发那半 rebase 在它上面 | +| **O2** | `scan_overrides` 声明的实现分区会被**误判成接口**(`providesInterface` 只在文本扫描路径设 false,`scan_overrides`/P1689 默认 true)⇒ 它的源码会被发布 | 窄洞,而且 `scan_overrides` 的 schema **没有**表达「是否 export」的位置。先记录;要修就是给它加一个键(那会是本方案唯一的新 manifest 键) | +| **O3** | MSVC 的 `/REMOVE:` 路径**未测试**(mcpp 的 Windows CI 用 clang 的 llvm-ar,走 GNU 形式) | 失败会带命令原文报错,不会静默;要真测需要 `msvc@system` 的 job | +| **O4** | Windows 上的胖包(msvc + mingw)未覆盖 | 见 §12.1;要在 Windows e2e job 装 `xim:mingw-gcc` | +| **O5** | 设计 §2.2 承诺的「`--target` 集合与 `[package].platforms` 覆盖比对告警」**没有实现** | 我漏了。它便宜(一次集合比对 + 一条 warning),但属于「发布纪律」而不是正确性,可以随后补 | From bfa33f4fc65ef572f82c42cb7e8dec3b304ae07b Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 00:17:51 +0800 Subject: [PATCH 16/31] fix(modgraph,pack): "nobody determined this" was being spelled "interface" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Whether a partition's SOURCE may be published turns on one keyword — `export module M:api;` may travel, `module M:impl;` may not — and two of the three paths that build the module graph cannot read it: * a `[scan_overrides.""]` entry names the modules a file provides and has nowhere to say whether the declaration carries `export`; * P1689 makes `is-interface` optional, and mcpp parsed it into a struct field (p1689.cppm:34) that nothing ever read. Both arrived as `providesInterface = true`, which the field's own comment called the conservative direction "since the flag only ever produces a warning". That had it backwards: `true` is the value that produces NO warning, so an implementation partition declared in `[scan_overrides]` was published in silence — the exact failure the closure exists to prevent, since too FEW published sources fails the consumer's compile while too MANY ships private source and nothing fails. So the field is a tri-state, and each path now says what it actually knows: text scanner reads the keyword, sets true or false explicitly P1689 reader carries the compiler's answer through, absence included — the compiler is the one participant that parsed the declaration scan_overrides leaves it unset, because the schema cannot express it Unknown warns, with a different sentence from the known case: that list says "you are publishing your implementation", this one says "mcpp cannot tell whether you are". Only PARTITIONS are asked about — `module M;` provides nothing, so a bare `M` can only come from `export module M;`, and asking there would fire on every primary interface in every package that uses overrides. e2e 253 pins it from both sides, because asserting only that the override case warns cannot distinguish "mcpp models three states" from "mcpp warns about every partition it publishes". Verified by control probe: restoring the `true` default at the scan_overrides site alone, with the rest of the fix in place, fails 253 at exactly that assertion. Two existing assertions were weak in the same way and are tightened: EXPECT_TRUE/EXPECT_FALSE on an optional ask about has_value(), so `EXPECT_TRUE(providesInterface)` stayed green for an implementation partition. --- src/modgraph/graph.cppm | 19 ++- src/modgraph/p1689.cppm | 10 +- src/modgraph/scanner.cppm | 11 +- src/pack/interface.cppm | 30 +++- src/pack/library_pipeline.cppm | 19 +++ ...253_pack_library_undetermined_partition.sh | 146 ++++++++++++++++++ tests/unit/test_modgraph.cpp | 7 +- tests/unit/test_p1689.cpp | 51 +++++- tests/unit/test_pack_interface.cpp | 52 ++++++- 9 files changed, 331 insertions(+), 14 deletions(-) create mode 100755 tests/e2e/253_pack_library_undetermined_partition.sh diff --git a/src/modgraph/graph.cppm b/src/modgraph/graph.cppm index 2a6c63ed..a315ce4d 100644 --- a/src/modgraph/graph.cppm +++ b/src/modgraph/graph.cppm @@ -45,10 +45,21 @@ struct SourceUnit { // without it), and the author needs to be told that their implementation // is going out. // - // Defaults to true so units synthesized outside the text scanner - // (scan_overrides, the P1689 reader) keep counting as interfaces — the - // conservative direction, since the flag only ever produces a warning. - bool providesInterface = true; + // Three states, because "nobody determined this" is a real answer and it + // used to be spelled `true`: + // + // true `export module M:api;` — the text scanner read the keyword + // false `module M:impl;` — likewise + // nullopt nobody could tell: a `scan_overrides` entry names the module + // but has no way to say whether it is exported, and a P1689 + // scanner may omit `is-interface` + // + // It defaulted to `true` and was called the conservative direction "since + // the flag only ever produces a warning". That had it backwards: `true` is + // the value that produces NO warning, so an undetermined implementation + // partition was published in silence — the exact failure this field exists + // to prevent. Unknown now warns, naming the file and the reason. + std::optional providesInterface; std::vector requires_; // The unit's ROLE, decided once by the scanner from the owning package's // extension table and carried from here on. Every downstream consumer diff --git a/src/modgraph/p1689.cppm b/src/modgraph/p1689.cppm index f1e891f5..028a4595 100644 --- a/src/modgraph/p1689.cppm +++ b/src/modgraph/p1689.cppm @@ -31,7 +31,11 @@ export namespace mcpp::modgraph::p1689 { struct DdiProvide { std::string logicalName; - bool isInterface = false; + // Absent, not false, when the scanner did not say. P1689 makes the key + // optional and mcpp cannot invent an answer: `export module M:api;` and + // `module M:impl;` differ only in the keyword, and getting that backwards + // decides whether a closed-source partition's SOURCE is published. + std::optional isInterface; }; struct DdiRule { @@ -398,6 +402,10 @@ scan_file(const std::filesystem::path& source, u.kind = mcpp::classify(source, extTable); if (!rule->provides.empty()) { u.provides = ModuleId{ rule->provides.front().logicalName }; + // Carried through, including its absence: the compiler is the one thing + // here that has actually parsed the declaration, so when it answers, + // its answer beats any inference; when it stays quiet, so does mcpp. + u.providesInterface = rule->provides.front().isInterface; } for (auto& r : rule->requires_) { u.requires_.push_back(ModuleId{ r }); diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index ca9581ef..90d929fe 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -641,7 +641,8 @@ std::expected scan_file(const std::filesystem::path& file std::format("file already exports module '{}'; cannot export '{}'", u.provides->logicalName, name)}); } - u.provides = ModuleId{name}; + u.provides = ModuleId{name}; + u.providesInterface = true; // read from the keyword, not assumed } else { // A non-exporting `module …;` is TWO different declarations // wearing one spelling, and they were treated as one: @@ -896,6 +897,14 @@ void scan_one_into(ScanResult& result, u.kind = mcpp::classify(f, extTable); if (!ov->provides.empty()) { u.provides = ModuleId{ov->provides.front()}; + // `providesInterface` is deliberately left UNSET. The override + // says which module the file provides; there is nowhere in it + // to say whether the declaration carries `export`, and the two + // spellings decide whether `mcpp pack` may publish the source. + // It used to inherit a `true` default, which is the answer that + // produces no warning — so an implementation partition declared + // here was published in silence. Unknown is the truth, and + // mcpp.pack.interface reports it as such. if (ov->provides.size() > 1) { result.errors.push_back(ScanError{f, 0, "scan_overrides: a unit may declare at most one " diff --git a/src/pack/interface.cppm b/src/pack/interface.cppm index 00f35efd..c9be2518 100644 --- a/src/pack/interface.cppm +++ b/src/pack/interface.cppm @@ -70,6 +70,18 @@ struct InterfaceClosure { // interface looks like any other import. So it is reported, loudly, and // `mcpp pack` prints it as a warning naming the file. std::vector publishedImplementationPartitions; + // Published partitions whose interface-ness NOBODY DETERMINED. + // + // A `[scan_overrides.""]` entry names the modules a file provides and + // has nowhere to say whether the declaration is exported; a P1689 scanner + // may omit `is-interface`. Either way mcpp is publishing a partition + // without knowing which kind it is, and the two outcomes are a working + // package and a disclosure. + // + // Kept apart from the list above because the sentence to print is a + // different one: that list says "you are publishing your implementation", + // this one says "mcpp cannot tell whether you are". + std::vector publishedUndeterminedPartitions; }; // Compute the closure for `packageName`, starting at the unit that provides @@ -127,11 +139,21 @@ interface_closure(const mcpp::modgraph::Graph& graph, auto idx = stack.front(); stack.erase(stack.begin()); // breadth-first: root first, then its imports if (!seen.insert(idx).second) continue; - out.published.push_back(graph.units[idx].path); - if (!graph.units[idx].providesInterface) - out.publishedImplementationPartitions.push_back(graph.units[idx].path); + auto const& u = graph.units[idx]; + out.published.push_back(u.path); + // Interface-ness is only ever in question for a PARTITION. `module M;` + // provides nothing, so the only declaration that can provide a bare + // `M` is `export module M;` — asking about it there would flag every + // primary interface of every overridden unit, which is noise, and noise + // is how a real warning gets ignored. + if (u.provides && u.provides->logicalName.find(':') != std::string::npos) { + if (u.providesInterface == false) + out.publishedImplementationPartitions.push_back(u.path); + else if (!u.providesInterface.has_value()) + out.publishedUndeterminedPartitions.push_back(u.path); + } - for (auto const& req : graph.units[idx].requires_) { + for (auto const& req : u.requires_) { auto it = graph.producerOf.find(req.logicalName); if (it == graph.producerOf.end()) { // Only OUR module's partitions are our problem. A bare name diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 4462deec..873d3bfc 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -227,6 +227,25 @@ export int build_and_pack_library(const std::string& targetName, p.filename().string())); } + // The same disclosure, one step worse: mcpp does not know which kind of + // partition it just published. A `[scan_overrides.""]` entry names + // the modules a file provides and has nowhere to say whether the + // declaration carries `export`, and a P1689 scanner may omit + // `is-interface`. The author is the only one who can answer, so ask them + // rather than guessing — the guess used to be "interface", which is the + // one that says nothing. + for (auto const& p : here.publishedUndeterminedPartitions) { + mcpp::ui::warning(std::format( + "{} provides a module PARTITION and mcpp cannot tell which kind: " + "the unit is declared in `[scan_overrides]`, which has nowhere to " + "say whether the declaration carries `export`, or a P1689 scanner " + "omitted `is-interface`.\n" + " Its SOURCE is being published either way. If the declaration " + "has no `export` and that source should stay private, keep it out " + "of the published interface's purview.", + p.filename().string())); + } + if (!haveClosure) { closure = std::move(here); haveClosure = true; diff --git a/tests/e2e/253_pack_library_undetermined_partition.sh b/tests/e2e/253_pack_library_undetermined_partition.sh new file mode 100755 index 00000000..15d99b9f --- /dev/null +++ b/tests/e2e/253_pack_library_undetermined_partition.sh @@ -0,0 +1,146 @@ +#!/usr/bin/env bash +# requires: +# (no capability: nothing here is toolchain-specific — the fixture uses a local +# module rather than `import std`, and both assertions read mcpp's own output.) +# +# 253_pack_library_undetermined_partition.sh — when mcpp CANNOT TELL whether a +# published partition is an interface partition or an implementation one, it has +# to say so. +# +# `mcpp pack` publishes the module closure of the lib root as SOURCE. Whether a +# partition's source may travel depends on one keyword: +# +# export module M:api; interface partition — publishing it is the point +# module M:impl; implementation partition — publishing it is a leak +# +# The text scanner reads that keyword. Two other paths do not: +# +# * `[scan_overrides.""]` names the modules a file provides and has +# nowhere to say whether the declaration carries `export`; +# * a P1689 scanner may omit `is-interface` (the key is optional). +# +# Both used to arrive as "interface" — the value that produces NO warning. So an +# implementation partition declared in `[scan_overrides]` was published in +# silence, which is the one failure mode the whole closure design exists to +# prevent. +# +# ⚠️ PINNED FROM BOTH SIDES, on purpose. Asserting only that the override case +# warns cannot distinguish "mcpp models three states" from "mcpp warns about +# every partition it publishes". So the same fixture is packed twice — once +# scanned, once overridden — and the two must produce DIFFERENT sentences: +# +# scanned → "secret.cppm is an implementation partition" (it knows) +# overridden → "cannot tell which kind" (it doesn't) +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +# The root interface reaches the partition, so the partition is published either +# way — this test is about WHAT MCPP SAYS while publishing it, not about whether +# it does (243 pins that). +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +import :secret; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/secret.cppm <<'EOF' +module mathkit:secret; +namespace mk { int secret_bias() { return 40; } } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { +int secret_bias(); +int answer() { return secret_bias() + 2; } +} +EOF + +manifest() { # $1 = extra manifest body (the override, or nothing) + # Absolute: this is called from inside mathkit/ the second time, and a + # relative path there writes a manifest nobody reads (or nothing at all). + cat > "$TMP/mathkit/mcpp.toml" < scanned.log 2>&1 || { cat scanned.log; echo "pack failed"; exit 1; } +grep -q 'secret.cppm is an implementation partition' scanned.log || { + cat scanned.log + echo "FAIL: the scanned case lost its known-implementation-partition warning" + exit 1; } +grep -q 'cannot tell which kind' scanned.log && { + cat scanned.log + echo "FAIL: mcpp read \`module mathkit:secret;\` and still claims it cannot tell." + echo " Then the undetermined state is not a state, it is every partition," + echo " and the warning carries no information." + exit 1; } + +# ── side 2: overridden. Nobody told mcpp whether it is exported. ──────── +# +# The override declares exactly what the scanner would have found, EXCEPT the +# one thing the schema cannot express — so the graph is identical apart from the +# unknown, and any difference in output is attributable to it alone. +manifest ' +[scan_overrides."src/secret.cppm"] +provides = ["mathkit:secret"]' +rm -rf target +"$MCPP" pack mathkit > overridden.log 2>&1 || { cat overridden.log; echo "override pack failed"; exit 1; } +grep -q 'cannot tell which kind' overridden.log || { + cat overridden.log + echo "FAIL: a partition declared in [scan_overrides] was published without a word." + echo " That is the silent half of the asymmetry: too FEW published sources" + echo " fails the consumer's compile, too MANY ships private source and" + echo " nothing fails. Unknown must warn." + exit 1; } +grep -q 'secret.cppm is an implementation partition' overridden.log && { + cat overridden.log + echo "FAIL: mcpp asserted it IS an implementation partition. Nothing told it so —" + echo " the override cannot say, and stating it anyway is a guess wearing" + echo " a diagnostic's clothes." + exit 1; } + +# It is still published: the consumer cannot build the root's BMI without it. +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ -e "$pkg/interface/secret.cppm" ]] || { + echo "FAIL: the warning fired but the source was not published — the consumer" + echo " would fail to compile the interface it was shipped." + exit 1; } +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" + +# ── and the package a warned-about pack produces still works ──────────── +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed"; exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } + +echo "PASS: an undetermined partition is published loudly, and it says so differently" diff --git a/tests/unit/test_modgraph.cpp b/tests/unit/test_modgraph.cpp index a2fc1bdc..25d36678 100644 --- a/tests/unit/test_modgraph.cpp +++ b/tests/unit/test_modgraph.cpp @@ -75,7 +75,10 @@ TEST(Scanner, ImplementationPartitionProvidesItsPartitionName) { ASSERT_TRUE(u->provides.has_value()); EXPECT_EQ(u->provides->logicalName, "mathkit:secret"); // Not an interface: its source must not be published by `mcpp pack`. - EXPECT_FALSE(u->providesInterface); + // Compared against the value, not tested for truthiness — the field is a + // tri-state now (nullopt means "nobody determined this"), and on an + // optional both EXPECT_FALSE and EXPECT_TRUE ask about the wrong thing. + EXPECT_EQ(u->providesInterface, std::optional{false}); // And it must not require its own name, which is what it used to do. for (auto const& r : u->requires_) EXPECT_NE(r.logicalName, "mathkit:secret"); @@ -92,7 +95,7 @@ TEST(Scanner, InterfacePartitionIsMarkedAsAnInterface) { ASSERT_TRUE(u.has_value()); ASSERT_TRUE(u->provides.has_value()); EXPECT_EQ(u->provides->logicalName, "mathkit:api"); - EXPECT_TRUE(u->providesInterface); + EXPECT_EQ(u->providesInterface, std::optional{true}); std::filesystem::remove_all(dir); } diff --git a/tests/unit/test_p1689.cpp b/tests/unit/test_p1689.cpp index c3c11563..da49c694 100644 --- a/tests/unit/test_p1689.cpp +++ b/tests/unit/test_p1689.cpp @@ -64,7 +64,9 @@ TEST(P1689Parse, SimpleProvider) { EXPECT_EQ(r->primaryOutput, "/tmp/foo.o"); ASSERT_EQ(r->provides.size(), 1u); EXPECT_EQ(r->provides[0].logicalName, "foo"); - EXPECT_TRUE(r->provides[0].isInterface); + // Not EXPECT_TRUE: on an optional that only asserts "the key was present", + // which stays green when the value parses as false. + EXPECT_EQ(r->provides[0].isInterface, std::optional{true}); ASSERT_EQ(r->requires_.size(), 2u); EXPECT_EQ(r->requires_[0], "std"); EXPECT_EQ(r->requires_[1], "foo:impl"); @@ -87,6 +89,53 @@ TEST(P1689Parse, EmptyRequires) { EXPECT_TRUE(r->requires_.empty()); } +// ─── is-interface decides whether a source may be published ──────────────── +// +// `export module M:api;` and `module M:impl;` differ only in the keyword, and +// `mcpp pack` publishes the interface closure's SOURCE. The compiler is the one +// participant here that actually parsed the declaration, so its answer — and +// its silence — both have to survive the trip. + +constexpr const char* kImplementationPartition = R"({ +"rules": [ +{ +"primary-output": "/tmp/secret.o", +"provides": [{"logical-name": "mathkit:secret", "is-interface": false}], +"requires": [] +} +] +})"; + +constexpr const char* kNoIsInterfaceKey = R"({ +"rules": [ +{ +"primary-output": "/tmp/quiet.o", +"provides": [{"logical-name": "mathkit:quiet"}], +"requires": [] +} +] +})"; + +TEST(P1689Parse, ImplementationPartitionIsNotAnInterface) { + auto r = parse_ddi(kImplementationPartition); + ASSERT_TRUE(r) << r.error(); + ASSERT_EQ(r->provides.size(), 1u); + EXPECT_EQ(r->provides[0].logicalName, "mathkit:secret"); + EXPECT_EQ(r->provides[0].isInterface, std::optional{false}); +} + +TEST(P1689Parse, AnAbsentIsInterfaceKeyStaysAbsent) { + // P1689 makes the key optional, so absence has to reach the packer as + // "nobody said" rather than as either answer. It used to arrive as `false` + // by struct default — which is the same value as an explicit + // implementation partition, i.e. the two became indistinguishable. + auto r = parse_ddi(kNoIsInterfaceKey); + ASSERT_TRUE(r) << r.error(); + ASSERT_EQ(r->provides.size(), 1u); + EXPECT_EQ(r->provides[0].logicalName, "mathkit:quiet"); + EXPECT_FALSE(r->provides[0].isInterface.has_value()); +} + TEST(P1689Parse, RejectsNonObject) { auto r = parse_ddi("[]"); EXPECT_FALSE(r); diff --git a/tests/unit/test_pack_interface.cpp b/tests/unit/test_pack_interface.cpp index 3c3d12cd..a5cfadf6 100644 --- a/tests/unit/test_pack_interface.cpp +++ b/tests/unit/test_pack_interface.cpp @@ -26,8 +26,11 @@ namespace { // now mirrors, because the closure's warning depends on the distinction. Graph library_graph(bool interfaceReachesSecret = false) { Graph g; + // `iface` is optional on purpose: nullopt is the state a `scan_overrides` + // unit is in — it names the module and cannot say whether it is exported. auto add = [&](std::string path, std::optional provides, - std::vector requires_, bool iface = true) { + std::vector requires_, + std::optional iface = true) { SourceUnit u; u.path = std::move(path); u.packageName = "mathkit"; @@ -110,6 +113,53 @@ TEST(InterfaceClosure, AnInterfaceOnlyClosureReportsNoPartitionLeak) { auto c = interface_closure(library_graph(), "mathkit", "mathkit"); ASSERT_TRUE(c.has_value()); EXPECT_TRUE(c->publishedImplementationPartitions.empty()); + EXPECT_TRUE(c->publishedUndeterminedPartitions.empty()); +} + +// ─── the state that used to be spelled "interface" ───────────────────────── + +TEST(InterfaceClosure, APublishedPartitionOfUnknownKindIsReportedSeparately) { + // A `[scan_overrides.""]` entry says which module a file provides and + // has nowhere to say whether the declaration is exported; a P1689 scanner + // may omit `is-interface`. That used to arrive as `providesInterface = + // true` — the value that produces NO warning — so an implementation + // partition declared that way was published in silence. + auto g = library_graph(/*interfaceReachesSecret=*/true); + g.units[2].providesInterface.reset(); // src/secret.cppm, kind unknown + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + // Still published — the consumer cannot build the root's BMI without it. + EXPECT_EQ(names(c->published), + (std::vector{"api.cppm", "mathkit.cppm", "secret.cppm"})); + // But reported as undetermined, not as a known implementation partition: + // the sentence to print is a different one. + EXPECT_TRUE(c->publishedImplementationPartitions.empty()); + ASSERT_EQ(c->publishedUndeterminedPartitions.size(), 1u); + EXPECT_EQ(c->publishedUndeterminedPartitions[0].filename().string(), "secret.cppm"); +} + +TEST(InterfaceClosure, AnUndeterminedPrimaryInterfaceIsNotReported) { + // Only a PARTITION can be either kind. `module M;` provides nothing, so the + // only declaration that provides a bare `M` is `export module M;` — asking + // the question there would warn about every primary interface in every + // package that uses scan_overrides, and a warning that fires on the normal + // case is one nobody reads. + auto g = library_graph(); + g.units[0].providesInterface.reset(); // src/mathkit.cppm, the root + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(c->publishedUndeterminedPartitions.empty()); + EXPECT_TRUE(c->publishedImplementationPartitions.empty()); +} + +TEST(InterfaceClosure, AWithheldPartitionOfUnknownKindIsNotReportedEither) { + // The warning is about what is PUBLISHED. `secret.cppm` is unreachable from + // the interface here, so its kind never mattered. + auto g = library_graph(/*interfaceReachesSecret=*/false); + g.units[2].providesInterface.reset(); + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(c->publishedUndeterminedPartitions.empty()); } TEST(InterfaceClosure, AGenuinelyMissingPartitionIsStillAnError) { From e6bd1f7d1474e359c84c04cf9d337ad8811cf54a Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 00:22:06 +0800 Subject: [PATCH 17/31] feat(pack): check [package] platforms against the legs actually produced MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The design promised this cross-check (§2.2) and the implementation simply did not have it — src/pack never read the key. `[package] platforms` is a support claim, and `mcpp pack` on a library is the first moment there is evidence to check it against: the legs in the package are the platforms it can serve. The hard part is not finding the gap, it is not shouting about it. Four comparisons exist and only two may be printed: packed, not declared always actionable — the manifest disclaims a platform the package demonstrably serves. declared, not packed actionable ONLY IF THIS HOST COULD HAVE BUILT IT. The normal release flow is one `mcpp pack` per platform in CI, so a Linux runner never produces a macOS leg. Warning about that would fire on every run of every cross-platform package, and a warning that always fires hides the one that matters — so the check asks host_can_serve, the same function that decides which `--target` values are accepted at all, and can therefore only ever name something the author is able to do on this machine. Both are warnings, never errors: coverage is release discipline, and the person who can judge it is looking at the release, not at this build. e2e 254 asserts the silence as well as the warnings, and derives the host's platform from `target//` — mcpp's own answer — rather than from a regex over triple spellings, which is how an earlier test came to assert Linux expectations on macOS. Its per-host picks mirror host_can_serve deliberately: if mcpp ever gains a macOS-hosted Windows toolchain the test fails, which is the correct outcome, because the table it encodes will have changed. --- docs/05-mcpp-toml.md | 18 +++ docs/zh/05-mcpp-toml.md | 17 +++ src/pack/library_pipeline.cppm | 79 +++++++++++ .../e2e/254_pack_library_platform_coverage.sh | 134 ++++++++++++++++++ 4 files changed, 248 insertions(+) create mode 100755 tests/e2e/254_pack_library_platform_coverage.sh diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index e54b3f70..b5eb29a3 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -1268,6 +1268,24 @@ The vocabulary is fixed by mcpp (which owns the target/triple system): `linux | macos | windows`; unknown values produce a warning, and an error under `--strict`. +`mcpp pack` on a library target checks the claim against the legs it actually +produced, because that is the first moment there is evidence to check it against: + +| situation | result | +|---|---| +| a leg was packed for a platform not listed here | warning — the manifest disclaims a platform the package demonstrably serves | +| a listed platform has no leg, **and this host could have built one** | warning — consumers there will resolve the package and find no artifact | +| a listed platform has no leg and this host cannot build for it | **silent** | + +The third row is why the check is usable at all. The normal release flow is one +`mcpp pack` per platform in CI, so a Linux runner never produces a macOS leg — +warning about it would fire on every run of every cross-platform package, and a +warning that always fires hides the one that matters. What "this host could have +built" means is the same question `--target` answers (docs/08 §7.4). + +Both are warnings, never errors: coverage is release discipline, and the person +who can judge it is looking at the release, not at this build. + ### 2.13 `[xlings]` — Build Environment ```toml diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index b5263377..d4af28be 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -952,6 +952,23 @@ platforms = ["linux", "macos", "windows"] (它拥有 target/triple 体系):`linux | macos | windows`;未知值 warning, `--strict` 下报错。 +对库目标执行 `mcpp pack` 时,会拿这条声明与**实际产出的腿**核对 —— 那是第一个 +有证据可核的时刻: + +| 情况 | 结果 | +|---|---| +| 某条腿的平台不在此列 | warning —— manifest 否认了一个包明明能服务的平台 | +| 声明了某平台却没有对应的腿,**且本宿主本来就能构建它** | warning —— 该平台的消费者会解析到这个包却找不到产物 | +| 声明了某平台却没有对应的腿,而本宿主根本构建不了它 | **不说话** | + +第三行才是这个检查可用的原因。正常的发布流程是 CI 上每平台各跑一次 +`mcpp pack`,于是 Linux runner 永远不会产出 macOS 腿 —— 为此告警会在每个跨平台 +包的每一次运行中触发,而**永远触发的告警会把真正该看的那条盖掉**。「本宿主能不能 +构建」与 `--target` 回答的是同一个问题(docs/08 §7.4)。 + +两者都只是 warning,绝不报错:覆盖度属于发布纪律,而能作判断的人看的是发布, +不是这一次构建。 + ### 2.13 `[xlings]` — 构建环境 ```toml diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 873d3bfc..4a20b80a 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -31,6 +31,7 @@ import mcpp.pack; import mcpp.pack.abi_tag; import mcpp.pack.interface; import mcpp.pack.library; +import mcpp.platform; import mcpp.toolchain.dialect; import mcpp.toolchain.registry; import mcpp.toolchain.triple; @@ -117,6 +118,10 @@ export int build_and_pack_library(const std::string& targetName, InterfaceClosure closure; bool haveClosure = false; std::string firstTriple; + // `[package] platforms` — the support CLAIM, read once. Checked against the + // legs after the loop; see there for why only two of the four comparisons + // are printable. + std::vector declaredPlatforms; for (auto const& want : legs) { mcpp::build::BuildOverrides ov; @@ -257,6 +262,7 @@ export int build_and_pack_library(const std::string& targetName, plan.targetName = targetName; plan.targetShared = shared; plan.cxxRuntime = ctx->manifest.buildConfig.cxxRuntime; + declaredPlatforms = ctx->manifest.package.platforms; plan.dependencies = publishable_dependencies(ctx->manifest); plan.extras = extras_of(ctx->manifest, ctx->projectRoot); plan.interfaceSources = closure.published; @@ -314,6 +320,79 @@ export int build_and_pack_library(const std::string& targetName, mcpp::ui::status("Packed leg", std::format("{} [{}]", triple, tag.str())); } + // ── does the package cover what it claims? ───────────────────────── + // + // `[package] platforms` is a support claim, and `mcpp pack` is where it + // first becomes checkable: the legs in this package are the platforms it can + // actually serve. Two of the four comparisons are worth printing, and + // printing the other two would make the check worse than absent — + // + // packed, not declared always actionable: either the claim is stale, or + // this package now ships a binary for a platform + // nobody said it supports. + // declared, not packed actionable ONLY IF THIS HOST COULD HAVE BUILT IT. + // The normal flow is one `mcpp pack` per platform in + // CI, so a Linux runner producing no macos leg is + // not an omission — it is every single run, and a + // warning that fires on every run is one nobody + // reads. `host_can_serve` is the same function that + // decides which `--target` values are accepted at + // all, so the warning can only ever name something + // the author is able to do on this machine. + if (!declaredPlatforms.empty()) { + auto joined = [&] { + std::string s; + for (auto const& p : declaredPlatforms) { if (!s.empty()) s += ", "; s += p; } + return s; + }(); + + std::set packedOs; + for (auto const& leg : plan.legs) { + if (auto t = mcpp::toolchain::triple::parse(leg.triple)) packedOs.insert(t->os); + } + + for (auto const& os : packedOs) { + if (std::ranges::find(declaredPlatforms, os) != declaredPlatforms.end()) continue; + mcpp::ui::warning(std::format( + "this package ships a {} binary, and `[package] platforms` does " + "not list {} (it lists: {}).\n" + " Add it if the support is real; drop the leg if it is not. As " + "it stands the manifest disclaims a platform the package serves.", + os, os, joined)); + } + + // Could this host have built a leg for `platform` at all? + auto servable_here = [](std::string_view platform) { + using mcpp::toolchain::triple::Triple; + const std::string arch{ mcpp::platform::host_arch }; + std::vector candidates; + if (platform == "linux") { + candidates.push_back(Triple{ arch, "linux", "gnu" }); + candidates.push_back(Triple{ arch, "linux", "musl" }); + } else if (platform == "windows") { + candidates.push_back(Triple{ arch, "windows", "msvc" }); + candidates.push_back(Triple{ arch, "windows", "gnu" }); + } else if (platform == "macos") { + candidates.push_back(Triple{ arch, "macos", "" }); + } + for (auto const& c : candidates) + if (mcpp::toolchain::host_can_serve(c)) return true; + return false; + }; + + for (auto const& want : declaredPlatforms) { + if (packedOs.contains(want)) continue; + if (!servable_here(want)) continue; // not this host's job to fix + mcpp::ui::warning(std::format( + "`[package] platforms` claims {}, and this host can build for it, " + "but no {} leg was packed.\n" + " Consumers on {} will resolve this package and find no artifact " + "for their target. Add `--target <{}-triple>`, or publish a " + "separate package from a {} runner.", + want, want, want, want, want)); + } + } + // ── where it lands ──────────────────────────────────────────────── const bool zip = plan.legs.size() == 1 && plan.legs[0].triple.find("windows") != std::string::npos; diff --git a/tests/e2e/254_pack_library_platform_coverage.sh b/tests/e2e/254_pack_library_platform_coverage.sh new file mode 100755 index 00000000..c5e8e0b2 --- /dev/null +++ b/tests/e2e/254_pack_library_platform_coverage.sh @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# requires: +# (no capability: every assertion reads mcpp's own output about a manifest key.) +# +# 254_pack_library_platform_coverage.sh — `[package] platforms` is a support +# CLAIM, and `mcpp pack` is where it first becomes checkable against evidence: +# the legs in the package are the platforms it can actually serve. +# +# THE HARD PART IS NOT FINDING THE GAP, IT IS NOT SHOUTING ABOUT IT. +# +# Four comparisons exist and only two may be printed: +# +# packed, not declared always actionable — the manifest disclaims a +# platform the package demonstrably serves. +# declared, not packed actionable ONLY IF THIS HOST COULD HAVE BUILT IT. +# The normal release flow is one `mcpp pack` per +# platform in CI, so a Linux runner producing no +# macos leg is not an omission, it is every single +# run. A warning that fires on every run is one +# nobody reads, and then the real one is invisible +# too. +# +# So the silence is as much the feature as the warning, and both are asserted. +# +# ⚠️ THE PER-HOST PICKS BELOW MIRROR host_can_serve (registry.cppm:542-566). +# That is deliberate and it is a tripwire: if mcpp ever gains, say, a +# macOS-hosted Windows toolchain, this test starts failing — which is the +# correct outcome, because the table it encodes will have changed and the +# expectations here have to be re-derived rather than assumed. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF + +manifest() { # $1 = the platforms array body, e.g. '"macos"' + cat > "$TMP/mathkit/mcpp.toml" < probe.log 2>&1 || { cat probe.log; echo "pack failed"; exit 1; } +host_triple="$(find target -mindepth 1 -maxdepth 1 -type d ! -name dist -exec basename {} \; | head -1)" +case "$host_triple" in + *-linux-*|*-linux) HOST_OS=linux ;; + *-macos*) HOST_OS=macos ;; + *-windows-*) HOST_OS=windows ;; + *) echo "FAIL: could not read the host platform out of target/$host_triple"; exit 1 ;; +esac +echo "host platform: $HOST_OS (from target/$host_triple)" + +# Per host: one platform this host CANNOT serve, and one it can but did not pack. +case "$HOST_OS" in + linux) UNSERVABLE=macos SERVABLE_UNPACKED=windows ;; + windows) UNSERVABLE=macos SERVABLE_UNPACKED=linux ;; + # macOS serves exactly one target — "macOS has no Linux-targeting payload at + # all", and PE needs a Windows or Linux host — so there is no + # servable-but-unpacked platform to name here. Structural, not a gap. + macos) UNSERVABLE=windows SERVABLE_UNPACKED= ;; +esac + +# ── 1. packed, not declared ───────────────────────────────────────────── +manifest "\"$UNSERVABLE\"" +rm -rf target +"$MCPP" pack mathkit > undeclared.log 2>&1 || { cat undeclared.log; echo "pack failed"; exit 1; } +grep -q "ships a $HOST_OS binary" undeclared.log || { + cat undeclared.log + echo "FAIL: the package ships a $HOST_OS artifact while [package] platforms lists" + echo " only $UNSERVABLE, and mcpp said nothing. The manifest disclaims a" + echo " platform the package serves, which is the claim consumers resolve against." + exit 1; } + +# ── 2. declared and unservable here: SILENCE ──────────────────────────── +# +# The half that keeps the warning worth reading. Asserted on its own manifest so +# a stray match from case 1 cannot satisfy it. +manifest "\"$HOST_OS\", \"$UNSERVABLE\"" +rm -rf target +"$MCPP" pack mathkit > quiet.log 2>&1 || { cat quiet.log; echo "pack failed"; exit 1; } +grep -q "claims $UNSERVABLE" quiet.log && { + cat quiet.log + echo "FAIL: mcpp asked for a $UNSERVABLE leg on a $HOST_OS host, which cannot" + echo " build one. That warning would fire on every release run of every" + echo " cross-platform package, and a warning that always fires hides the" + echo " one that matters." + exit 1; } +grep -q "ships a $HOST_OS binary" quiet.log && { + cat quiet.log + echo "FAIL: $HOST_OS is declared AND packed, and mcpp still complained about it" + exit 1; } + +# ── 3. declared, servable here, not packed ────────────────────────────── +if [[ -n "$SERVABLE_UNPACKED" ]]; then + manifest "\"$HOST_OS\", \"$SERVABLE_UNPACKED\"" + rm -rf target + "$MCPP" pack mathkit > gap.log 2>&1 || { cat gap.log; echo "pack failed"; exit 1; } + grep -q "claims $SERVABLE_UNPACKED" gap.log || { + cat gap.log + echo "FAIL: [package] platforms claims $SERVABLE_UNPACKED, this host can build" + echo " for it, no such leg was packed, and mcpp said nothing. Consumers" + echo " on $SERVABLE_UNPACKED resolve this package and find no artifact." + exit 1; } + echo "PASS: coverage gaps are reported, and unservable platforms are not" +else + echo "NOTE: a $HOST_OS host serves exactly one target, so there is no" + echo " servable-but-unpacked platform to assert here. Case 3 did not run." + echo "PASS: an undeclared platform is reported, an unservable one stays quiet" +fi From 4c7aeb5cea202396b6019f88849eeb8fb5233e0e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 00:26:38 +0800 Subject: [PATCH 18/31] test(pack): give the MSVC archiver spelling a seam, then test it from both ends MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mcpp pack` deletes the published interface's objects from the archive before shipping it, and the two archivers disagree in both directions: ar one verb, then the archive, then every member lib.exe one flag PER member, and the archive comes LAST Nothing could catch a mistake there. mcpp's Windows CI builds with clang and archives with llvm-ar, which takes the GNU spelling, so the MSVC branch has never executed in any job on any platform — and the assembly was inline in run_library_pack, so it had no seam either. Pinning the two constants in dialect.cppm was never enough: the ORDER they are assembled in is a third fact. So the assembly moves into archive_remove_command, exported for the tests and nothing else, and it is checked from both ends: test_pack_archive_remove.cpp the exact command for each dialect, host- independently (expectations built with the same `quote`, since what is under test is spelling and order — shell quoting has its own tests) e2e 255 a real lib.exe, under `# requires: msvc`, with msvc@system pinned in the manifest because Windows' DEFAULT toolchain is clang and taking it would exercise the GNU branch and pass while proving nothing 255 does not inspect the archive to decide: a wrong spelling makes run_library_pack refuse, quoting the command and the archiver's output, since shipping the objects would be the silent outcome. What it does assert is that the branch was TAKEN — a `mathkit.lib` rather than llvm-ar's `.a`, and a non-empty published interface so there was something to remove at all. --- src/pack/library.cppm | 77 ++++++++---- tests/e2e/255_pack_library_msvc_archiver.sh | 131 ++++++++++++++++++++ tests/unit/test_pack_archive_remove.cpp | 90 ++++++++++++++ 3 files changed, 275 insertions(+), 23 deletions(-) create mode 100755 tests/e2e/255_pack_library_msvc_archiver.sh create mode 100644 tests/unit/test_pack_archive_remove.cpp diff --git a/src/pack/library.cppm b/src/pack/library.cppm index baf079b9..f1ba9967 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -103,6 +103,26 @@ struct LibraryPackError { std::string message; }; // uses — one derivation, verified from both ends. std::expected run_library_pack(const LibraryPackPlan& plan); +// The command that deletes `members` from `archive`, spelled for whichever +// archiver `tool` is. +// +// Exported ONLY so it can be tested. mcpp's Windows CI archives with clang's +// `llvm-ar`, which takes the GNU spelling, so the MSVC branch below is never +// executed by any job — and it is the branch that differs in both directions: +// +// ar one verb, then the archive, then every member +// ar d libmathkit.a mathkit.m.o api.m.o +// lib.exe one FLAG PER MEMBER, and the archive comes LAST +// lib.exe /REMOVE:mathkit.m.o /REMOVE:api.m.o mathkit.lib +// +// Pinning the two constants in dialect.cppm is not enough: the order they are +// assembled in is a third fact, and it lives here. +std::string archive_remove_command(const std::filesystem::path& tool, + const std::filesystem::path& archive, + const std::vector& members, + std::string_view removeArg, + bool archiveFirst); + } // namespace mcpp::pack namespace mcpp::pack { @@ -135,6 +155,37 @@ std::vector walk(const std::filesystem::path& root) { } // namespace +std::string archive_remove_command(const std::filesystem::path& tool, + const std::filesystem::path& archive, + const std::vector& members, + std::string_view removeArg, + bool archiveFirst) +{ + std::string cmd = mcpp::platform::shell::quote(tool.string()); + // `{}` in removeArg means one flag per member (lib.exe /REMOVE:); its + // absence means one verb followed by every member (ar d ...). + const bool perMember = removeArg.find("{}") != std::string_view::npos; + auto member_words = [&] { + std::string w; + for (auto const& m : members) { + if (!perMember) { w += " " + mcpp::platform::shell::quote(m); continue; } + std::string arg{ removeArg }; + arg.replace(arg.find("{}"), 2, m); + w += " " + mcpp::platform::shell::quote(arg); + } + return w; + }; + if (archiveFirst) { + if (!perMember) cmd += " " + std::string(removeArg); + cmd += " " + mcpp::platform::shell::quote(archive.string()); + cmd += member_words(); + } else { + cmd += member_words(); + cmd += " " + mcpp::platform::shell::quote(archive.string()); + } + return cmd; +} + std::expected run_library_pack(const LibraryPackPlan& plan) { @@ -231,29 +282,9 @@ run_library_pack(const LibraryPackPlan& plan) leg.triple, name) }); } if (!leg.shared && !plan.dropObjects.empty()) { - std::string cmd = mcpp::platform::shell::quote(leg.archiveTool.string()); - auto member_words = [&] { - std::string w; - for (auto const& m : plan.dropObjects) { - // `{}` means one flag per member (LIB.EXE); its absence - // means one verb followed by every member (ar). - auto pos = leg.removeArg.find("{}"); - if (pos == std::string::npos) { w += " " + mcpp::platform::shell::quote(m); continue; } - auto arg = leg.removeArg; - arg.replace(pos, 2, m); - w += " " + mcpp::platform::shell::quote(arg); - } - return w; - }; - if (leg.removeArchiveFirst) { - if (leg.removeArg.find("{}") == std::string::npos) - cmd += " " + leg.removeArg; - cmd += " " + mcpp::platform::shell::quote(dst.string()); - cmd += member_words(); - } else { - cmd += member_words(); - cmd += " " + mcpp::platform::shell::quote(dst.string()); - } + const auto cmd = archive_remove_command( + leg.archiveTool, dst, plan.dropObjects, + leg.removeArg, leg.removeArchiveFirst); auto r = mcpp::platform::process::capture(cmd + " 2>&1"); if (r.exit_code != 0) { return std::unexpected(LibraryPackError{ std::format( diff --git a/tests/e2e/255_pack_library_msvc_archiver.sh b/tests/e2e/255_pack_library_msvc_archiver.sh new file mode 100755 index 00000000..55583f7c --- /dev/null +++ b/tests/e2e/255_pack_library_msvc_archiver.sh @@ -0,0 +1,131 @@ +#!/usr/bin/env bash +# requires: msvc +# 255_pack_library_msvc_archiver.sh — packing a library with the MSVC toolchain +# actually runs `lib.exe /REMOVE:`. +# +# WHY THIS EXISTS. `mcpp pack` deletes the published interface's objects from the +# archive before shipping it: the consumer compiles those sources itself, so +# leaving the objects in gives it two definitions of each published module's +# initialiser, resolved by link order. The two archivers disagree about how to +# say that, in both directions: +# +# ar one verb, then the archive, then every member +# ar d libmathkit.a mathkit.m.o api.m.o +# lib.exe one flag PER member, and the archive comes LAST +# lib.exe /REMOVE:mathkit.m.o /REMOVE:api.m.o mathkit.lib +# +# The packer originally assumed `ar` syntax everywhere. Nothing caught it, +# because mcpp's own Windows CI builds with clang and archives with `llvm-ar`, +# which takes the GNU spelling — so the MSVC branch has never executed in any +# job on any platform. test_pack_archive_remove.cpp pins the string; only a real +# `lib.exe` can tell whether the string is right. +# +# WHY PACK SUCCEEDING *IS* THE ASSERTION. A wrong spelling does not degrade +# quietly here: run_library_pack refuses, quoting the command and the archiver's +# output, because shipping the objects would be the silent outcome. So this test +# does not need to inspect the archive to know the branch worked — but it does +# have to prove the branch was TAKEN, and that is what the two checks below are +# for: a `.lib` (so the family is MSVC, not llvm-ar's `.a`) and a non-empty +# published interface (so there was something to remove at all). +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export import :api; +EOF +cat > mathkit/src/api.cppm <<'EOF' +export module mathkit:api; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +# msvc@system pinned, not defaulted: Windows' default toolchain is clang +# targeting the MSVC ABI, and it archives with llvm-ar — i.e. taking the default +# here would exercise the GNU branch and pass while proving nothing. +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +[toolchain] +windows = "msvc@system" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { + cat pack.log + echo "FAIL: packing with msvc@system failed." + echo " If the message above is 'cannot drop published interface objects'," + echo " the lib.exe spelling is wrong — /REMOVE: takes one flag per member" + echo " and the archive comes LAST (src/pack/library.cppm)." + exit 1; } + +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ -n "$pkg" ]] || { cat pack.log; echo "FAIL: no package directory"; exit 1; } + +# The branch was taken: MSVC names archives `.lib` with no `lib` prefix, +# so finding one means `archive_tool` resolved to lib.exe rather than llvm-ar. +archive="$(find "$pkg/lib" -type f -name 'mathkit.lib' | head -1)" +[[ -n "$archive" ]] || { + find "$pkg/lib" -type f + echo "FAIL: no mathkit.lib in the package — this ran with a GNU-spelling" + echo " archiver, so the lib.exe path was never exercised." + exit 1; } + +# And there was something to remove: an empty interface list means dropObjects +# is empty and the removal is skipped entirely. +[[ -f "$pkg/interface/mathkit.cppm" && -f "$pkg/interface/api.cppm" ]] || { + ls -R "$pkg" + echo "FAIL: nothing was published as source, so nothing had to be removed" + exit 1; } + +# Best-effort member inspection. `lib.exe` lives in the VC toolset, not on PATH, +# so this is a bonus rather than the criterion — mcpp resolves it internally and +# the pack above already depended on it working. +if command -v lib &>/dev/null && members="$(lib /nologo /LIST "$(host_path "$archive")" 2>/dev/null)"; then + echo "$members" | grep -qi 'impl' || { + echo "$members" + echo "FAIL: the implementation object is gone — nothing would link" + exit 1; } + echo "$members" | grep -qi 'api\.m\.obj' && { + echo "$members" + echo "FAIL: a published interface unit's object is still in the archive" + exit 1; } + echo " (verified against lib /LIST)" +fi + +# The end-to-end criterion: a consumer builds and runs against it. +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed"; exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } + +echo "PASS: lib.exe /REMOVE: really removes, and the package links under MSVC" diff --git a/tests/unit/test_pack_archive_remove.cpp b/tests/unit/test_pack_archive_remove.cpp new file mode 100644 index 00000000..ae1e40b9 --- /dev/null +++ b/tests/unit/test_pack_archive_remove.cpp @@ -0,0 +1,90 @@ +#include + +import std; +import mcpp.pack.library; +import mcpp.platform.shell; +import mcpp.toolchain.dialect; + +using mcpp::pack::archive_remove_command; +using mcpp::platform::shell::quote; +using mcpp::toolchain::gnu_dialect; +using mcpp::toolchain::msvc_dialect; + +// `mcpp pack` deletes the published interface's objects from the archive before +// shipping it — leaving them in gives the consumer two definitions of each +// published module's initialiser, resolved by link order. +// +// Two archivers, and they disagree in BOTH directions: +// +// ar one verb, then the archive, then every member +// lib.exe one flag PER member, and the archive comes LAST +// +// ⚠️ WHY THIS IS A UNIT TEST. mcpp's Windows CI archives with clang's `llvm-ar`, +// which takes the GNU spelling, so no job anywhere executes the MSVC branch. The +// packer originally assumed `ar` syntax on every platform and nothing could have +// caught it: the two constants live in dialect.cppm, the order they are assembled +// in is a third fact, and it had no test because it had no seam. So the assembly +// was given one. +// +// Host-independent by construction: the expectations are built with the same +// `quote` these commands are built with, because what is under test is the +// spelling and the ORDER — shell quoting has its own tests in test_shell.cpp, +// and duplicating its rules here would only pin them twice, differently. + +namespace { + +const std::vector kMembers{ "mathkit.m.o", "api.m.o" }; + +} // namespace + +TEST(ArchiveRemoveCommand, GnuTakesOneVerbThenTheArchiveThenEveryMember) { + auto cmd = archive_remove_command("/usr/bin/ar", "bin/libmathkit.a", kMembers, + gnu_dialect().archiveRemoveArg, + gnu_dialect().archiveRemoveTakesArchiveFirst); + EXPECT_EQ(cmd, quote("/usr/bin/ar") + " d " + quote("bin/libmathkit.a") + + " " + quote("mathkit.m.o") + " " + quote("api.m.o")); +} + +TEST(ArchiveRemoveCommand, MsvcTakesOneFlagPerMemberAndTheArchiveLast) { + auto cmd = archive_remove_command("lib.exe", "bin/mathkit.lib", kMembers, + msvc_dialect().archiveRemoveArg, + msvc_dialect().archiveRemoveTakesArchiveFirst); + EXPECT_EQ(cmd, quote("lib.exe") + + " " + quote("/REMOVE:mathkit.m.o") + + " " + quote("/REMOVE:api.m.o") + + " " + quote("bin/mathkit.lib")); +} + +// The two rows in dialect.cppm are the input to the assembly above. Pinned here +// too, so changing one without the other fails rather than producing a command +// that is well-formed for the wrong archiver. +TEST(ArchiveRemoveCommand, TheDialectRowsSupplyTheseTwoSpellings) { + EXPECT_EQ(gnu_dialect().archiveRemoveArg, "d"); + EXPECT_TRUE(gnu_dialect().archiveRemoveTakesArchiveFirst); + EXPECT_EQ(msvc_dialect().archiveRemoveArg, "/REMOVE:{}"); + EXPECT_FALSE(msvc_dialect().archiveRemoveTakesArchiveFirst); +} + +// The failure that motivated the seam: the packer used to emit `ar` syntax +// unconditionally. Stated as what must NOT appear, because "it contains +// /REMOVE:" alone would stay green for a command that also carried the `d` verb +// or put the archive in the wrong place. +TEST(ArchiveRemoveCommand, TheMsvcFormCarriesNoArVerbAndNoLeadingArchive) { + auto cmd = archive_remove_command("lib.exe", "bin/mathkit.lib", kMembers, + msvc_dialect().archiveRemoveArg, + msvc_dialect().archiveRemoveTakesArchiveFirst); + EXPECT_EQ(cmd.find(" d "), std::string::npos); + EXPECT_LT(cmd.find("/REMOVE:mathkit.m.o"), cmd.find("mathkit.lib")); +} + +TEST(ArchiveRemoveCommand, ASingleMemberIsSpelledTheSameWayAsMany) { + // The per-member branch is a loop; a one-element list is where an + // "and-then-the-rest" bug hides. + auto gnu = archive_remove_command("ar", "libx.a", { "only.m.o" }, + gnu_dialect().archiveRemoveArg, true); + EXPECT_EQ(gnu, quote("ar") + " d " + quote("libx.a") + " " + quote("only.m.o")); + auto msvc = archive_remove_command("lib.exe", "x.lib", { "only.m.o" }, + msvc_dialect().archiveRemoveArg, false); + EXPECT_EQ(msvc, quote("lib.exe") + " " + quote("/REMOVE:only.m.o") + + " " + quote("x.lib")); +} From fe62962c9c187060ac7a8f2873231ae9580ea955 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 01:15:14 +0800 Subject: [PATCH 19/31] feat(build,pack): kind="shared" beyond ELF, and stop building unservable targets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four things, and the first one explains why the rest went unnoticed. THE GUARD HAD A HOLE IN THE CASE THAT MATTERED. make_plan refused SharedLibrary targets with `!targetTriple.empty() && os != "linux"`, and targetTriple is EMPTY for a native build. So it turned away a cross build to macOS — unservable anyway, i.e. unreachable — while letting a NATIVE macOS or native Windows build walk straight into the unverified paths it was written to keep people out of. It reads the resolved target now. PE WAS MISSING ITS IMPORT LIBRARY. A PE shared library is two files: the `.dll` the loader opens and an archive of stubs the LINKER consumes. mcpp wrote only the first, and consumers linked the `.dll` directly — which mingw's ld tolerates and no other linker does, so the tolerant case was hiding the broken one. The link edge now declares the import library as an implicit output (one command writes both; a second edge would run the link twice), `import_library_for` owns its name, the dialect table owns the flag spelling beside `archiveRemoveArg`, and the package ships it. One more thing was needed and the diagnostic for it names nothing useful: mcpp gives PE executables `-static`, which puts ld in static-only mode where it refuses an import library and says `have you installed the static version of the mathkit library?` — so the emitted manifest switches to dynamic mode for that one `-l`. MACH-O WAS MISSING ITS INSTALL NAME. A `.dylib` records the path it was LINKED at, so emitting `-install_name` only when the manifest declared a `soname` left every other dylib recording a build directory: perfect on the machine that built it, `image not found` anywhere else. It is now `@rpath/` unconditionally. The choice was also being made with `#if defined(__APPLE__)` on the HOST, so a cross link emitted the wrong flag or none — it comes from the target now, as `target_output` already did. MSVC STAYS REFUSED, FOR THE REAL REASON. Not the linker: `link /DLL /IMPLIB:` has been in the rule table all along. Symbol export — MSVC exports nothing from a DLL without `__declspec(dllexport)` or a `.def`, so the import library comes out empty and consumers fail with unresolved externals naming symbols that are visibly in the objects. Refusing beats producing a diagnostic that points nowhere near its cause, and the message names MinGW as the way forward. AND `--target` NO LONGER ACCEPTS WHAT THIS HOST CANNOT PRODUCE. Measured on Linux: `mcpp build --target x86_64-windows-msvc` resolved the native g++, wrote target/x86_64-linux-gnu/, and reported success — an ELF delivered as a Windows build, which is the failure the neighbouring typo check calls the worst one. The vocabulary tier says "mcpp supports this target"; host_can_serve answers "can this machine produce it", and the error lists what it can. An explicit `[target.X] toolchain` stays the escape hatch. Also fixed on the way: consuming a distribution package whose target is `shared` died with `ninja: multiple rules generate bin/libmathkit.dll`, because the dependency loop created a link unit for a library that is already built — and relinking it would have produced a library missing every implementation unit the publisher withheld, since a distribution package's `sources` are its interface. Verified end to end on Linux via mingw-cross + wine (e2e 257: both files, the implicit output, the package, deployment beside the exe, `ok=42`). macOS (259) and the MSVC refusal (258) are CI's to confirm; 259 deletes the producer's build tree before consuming, so the install-name assertion is load-bearing rather than decorative. --- .github/workflows/ci-windows-e2e.yml | 27 ++++ CHANGELOG.md | 59 ++++++- docs/08-toolchain-internals.md | 26 +++ docs/12-binary-distribution.md | 49 +++++- docs/zh/12-binary-distribution.md | 43 ++++- src/build/ninja_backend.cppm | 70 ++++++-- src/build/plan.cppm | 138 ++++++++++++---- src/build/prepare.cppm | 41 +++++ src/pack/library.cppm | 27 ++++ src/pack/library_pipeline.cppm | 5 + src/pack/manifest_emit.cppm | 47 +++++- src/toolchain/dialect.cppm | 18 +++ .../245_pack_library_fat_target_selection.sh | 11 +- tests/e2e/255_pack_library_msvc_archiver.sh | 8 +- tests/e2e/256_pack_library_fat_windows.sh | 137 ++++++++++++++++ tests/e2e/257_shared_library_pe.sh | 153 ++++++++++++++++++ tests/e2e/258_shared_library_msvc_refused.sh | 89 ++++++++++ tests/e2e/259_shared_library_macho.sh | 118 ++++++++++++++ tests/e2e/run_all.sh | 15 +- 19 files changed, 1027 insertions(+), 54 deletions(-) create mode 100755 tests/e2e/256_pack_library_fat_windows.sh create mode 100755 tests/e2e/257_shared_library_pe.sh create mode 100755 tests/e2e/258_shared_library_msvc_refused.sh create mode 100755 tests/e2e/259_shared_library_macho.sh diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index 05c3b43a..b339df95 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -57,6 +57,33 @@ jobs: "$MCPP_SELF" --version echo "MCPP_SELF=$MCPP_SELF" >> "$GITHUB_ENV" + # MinGW-w64 GCC (xim:mingw-gcc). Installed here so the `mingw` capability + # is GRANTED rather than left to whatever the shared sandbox cache happens + # to carry: e2e 256 packs an MSVC leg and a MinGW leg into one package, + # which is the only place `lib/` keyed by triple is proven with two + # DIFFERENT artifact names (mathkit.lib beside libmathkit.a). Without this + # step that test skips, and a skipped test in a green suite reads exactly + # like a passing one. + # + # Not `|| true`: if the payload cannot be installed the capability quietly + # disappears and the coverage goes with it, which is the failure mode this + # step exists to prevent. + - name: "Toolchain: MinGW payload for the fat-package e2e" + shell: bash + run: | + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + "$MCPP_SELF" toolchain install mingw 16.1.0 + # Verified through the SAME two locations run_all.sh probes — checking + # only one of them would let the step pass while the capability stays + # ungranted, which is the shape of a green run that tested nothing. + found="" + for c in "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe \ + "$HOME"/.xlings/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe; do + [[ -x "$c" ]] && { found="$c"; break; } + done + test -n "$found" || { echo "FAIL: mingw installed but not where run_all.sh looks"; exit 1; } + echo "mingw payload: $found" + - name: E2E suite shell: bash # Fail-fast on hung tests instead of burning the whole job budget. diff --git a/CHANGELOG.md b/CHANGELOG.md index 4d9f8366..47c90fbd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,7 +34,49 @@ 同一个闭包反过来决定归档里要删哪些对象,按 `.m.o` 删则会删掉真代码、 三个平台全部链接失败。两条清单都会打印出来。 - 详见 `docs/12-binary-distribution.md`、`examples/05-lib-dist`、`examples/06-lib-consume`。 + 详见 `docs/12-binary-distribution.md`、`examples/05-lib-distribution`。 + +- **`kind = "shared"` 不再只有 Linux:PE/MinGW 与 Mach-O 都能产、能打包、能跑。** + + 过去这条路只在 ELF 上验证过,而那道「非 Linux 就拒绝」的守卫**恰好在最要紧的 + 情形下失效**:它写的是 `!targetTriple.empty() && os != "linux"`,而**原生构建的 + targetTriple 是空的** —— 于是它拦住了一个交叉到 macOS 的构建(那个本来就不可服务), + 却让**原生 macOS / 原生 Windows** 直接走进它本该拦住的未验证路径。 + + 真正缺的东西各不相同,而且都不是 flag 拼写: + + - **PE 缺导入库。** 一个 PE 共享库是**两个文件**:加载器打开的 `.dll`,以及 + 链接器消费的桩归档。mcpp 只写了前者,消费者直接链 `.dll` —— mingw 的 ld 容忍这个, + 别的链接器都不容忍,于是**能用的那种情况把坏掉的那种遮住了**。现在链接边把导入库 + 作为隐式输出声明出来,包里两个都带,生成的 manifest 指向导入库。 + 另外 PE 可执行文件带 `-static`,而 `-static` 会让 ld 进入纯静态模式并拒绝导入库, + 报的是 `have you installed the static version of the mathkit library?` —— + 既没点 DLL 也没点 `-static`;所以那条 `-l` 之前要先 `-Wl,-Bdynamic`。 + - **Mach-O 缺 install name。** `.dylib` 记录的是**链接时的路径**,所以只在声明了 + `soname` 时才发 `-install_name` 意味着**其他每一个 `.dylib` 都把构建目录烙了进去** —— + 在打包机上完好,换个地方就 `image not found`。现在无条件发 `@rpath/`。 + 而且这个选择原先是用宿主的 `#if defined(__APPLE__)` 做的,交叉链接会发错(或不发); + 现在按 target 决定,和 `target_output` 早就做的一样。 + - **PE/MSVC 仍然拒绝,但换了个理由,而且是真理由。** 不是链接器 —— + `link /DLL /IMPLIB:` 一直都在规则表里。是**符号导出**:没有 `__declspec(dllexport)` + 或 `.def`,MSVC 的 DLL 什么都不导出 ⇒ 导入库是空的 ⇒ 消费者拿到一堆 + unresolved externals,而那些符号明明在对象里。产出这个比拒绝更糟。 + +- **`--target` 不能服务时直接拒绝,而不是悄悄按宿主构建。** + + 实测(Linux):`mcpp build --target x86_64-windows-msvc` 解析到**原生 g++**、 + 写进 `target/x86_64-linux-gnu/`、报告成功 —— 一个 ELF 被当成 Windows 构建交付。 + 词表的 tier 说的是「mcpp 支持这个 target」,从来没说「这台机器能产出它」; + 后者是 `host_can_serve` 的问题,现在 `prepare.cppm` 会问它,并把 + **这台宿主能构建的清单**列进错误信息。逃生口保留:显式 + `[target.X] toolchain = "…"` 表示交叉链是你自己提供的,mcpp 的载荷矩阵无权否决。 + +- **`[package] platforms` 会与实际打出的腿对账(设计里承诺过、实现里没有)。** + + 四种比较只有两种值得打印:**打了却没声明**永远可行动;**声明了却没打, + 且这台宿主本来能构建它**才可行动。正常发布流程是 CI 上每平台各跑一次 `mcpp pack`, + 所以 Linux runner 不产 macOS 腿不是遗漏、是每一次 —— **永远触发的告警会把真正该看的 + 那条盖掉**。所以判据用的是 `host_can_serve`,与 `--target` 是同一个函数。 - **消费预编译包时的两道闸门。** 都是不检查就会静默出错的: @@ -50,6 +92,21 @@ ### 修复 +- **「谁也没判定过」这个状态过去被拼成了「是接口」,于是闭源实现分区会静默发布出去。** + + 一个分区的源码能不能发布,取决于一个关键字:`export module M:api;` 可以走, + `module M:impl;` 不能。而三条建图路径里有两条读不到它: + `[scan_overrides.""]` 声明了文件提供哪些模块,**没有地方能说它是否 export**; + P1689 的 `is-interface` 是可选键,mcpp 把它解析进了一个**从来没人读**的字段。 + + 两者都以 `providesInterface = true` 到达,而字段自己的注释把这叫「保守方向」, + 理由是「这个标志只会产生一条警告」。**这恰好说反了**:`true` 正是那个**不产生 + 任何警告**的值 —— 于是用 `[scan_overrides]` 声明的实现分区被**一声不响地发布**。 + + 现在它是三态的,每条路径只说自己真的知道的事:文本扫描器读关键字并显式写 true/false; + P1689 读取器把编译器的答案(**包括它的沉默**)原样带过来;`scan_overrides` + **留空**,因为 schema 表达不了。未知会告警,而且和已知那条**说的是不同的话**。 + - **实现分区(`module M:part;`)在 Windows 上构建不了,而根因在扫描器里。** `module M:part;` 与 `module M;` 共用一个拼写,却是两种不同的声明,而扫描器把 diff --git a/docs/08-toolchain-internals.md b/docs/08-toolchain-internals.md index 2638805d..746729e4 100644 --- a/docs/08-toolchain-internals.md +++ b/docs/08-toolchain-internals.md @@ -558,6 +558,32 @@ libc++ linkage handling; Windows has no rpath — mcpp deploys runtime DLLs next to the produced exe, which is the platform's native equivalent of everything §3–§4 does for ELF. +**Shared libraries a project PRODUCES** (`kind = "shared"`) do differ per format, +and the difference is not a flag spelling — it is what the artifact records about +itself: + +| format | what the producer emits | what the consumer links | +|---|---|---| +| ELF | `-Wl,-soname,` when declared | `-L` + `-l`, `-Wl,-rpath,$ORIGIN` | +| Mach-O | `-Wl,-install_name,@rpath/` **always** | `-L` + `-l`, `-Wl,-rpath,@loader_path` | +| PE / MinGW | `-Wl,--out-implib,` | the **import library**, `-Wl,-Bdynamic` first | +| PE / MSVC | refused (no auto-export; see docs/12) | — | + +Two of those are recent corrections. Mach-O's install name defaults to the path +the library was LINKED at, so emitting it only when a `soname` was declared left +every other `.dylib` recording a build directory — fine on the machine that built +it, `image not found` anywhere else. And the choice was made with `#if +defined(__APPLE__)` on the HOST, so a cross link emitted the wrong one (or none). +It is decided from the target now, like `target_output` already was. + +An **unservable target is refused** rather than quietly built for the host: +`--target x86_64-windows-msvc` on Linux used to resolve the native `g++`, write +`target/x86_64-linux-gnu/`, and report success. The vocabulary tier says "mcpp +supports this target"; `host_can_serve` (`registry.cppm`) answers the different +question "can this machine produce it", and `prepare.cppm` now asks it — with an +explicit `[target.X] toolchain = "…"` as the escape hatch for a cross toolchain +you supply yourself. + ## 8. Source map | Concern | File | diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index ef61b7a5..18399708 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -262,10 +262,57 @@ you publish to a mixed audience. |---|---| | `kind = "lib"` (static) | ✅ every target, tested on all three | | `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME | -| `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet | +| `kind = "shared"` on PE / MinGW (`*-windows-gnu`) | ✅ — the package carries the `.dll` **and** its import library | +| `kind = "shared"` on Mach-O (`*-macos`) | ✅ — install name is `@rpath/`, so the `.dylib` relocates | +| `kind = "shared"` on PE / MSVC (`*-windows-msvc`) | ❌ refused — see below | | `kind = "shared"` on `*-musl` | ❌ a musl target links statically | | shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact | | bundling dependencies into the package | ❌ declare them instead (above) | +| consuming a package with **native `cl.exe`** | ❌ see below | + +### Why MSVC refuses `kind = "shared"` + +Not the linker — `link /DLL /IMPLIB:` works. **Symbol export.** MSVC exports +nothing from a DLL unless the source says `__declspec(dllexport)` or a `.def` +file lists the symbols, so the import library comes out empty and every consumer +fails with unresolved externals naming symbols that are plainly in the object +files. mcpp refuses rather than produce a diagnostic that points nowhere near its +cause: + +``` +target 'mathkit': kind = "shared" is not supported for the MSVC ABI (x86_64-windows-msvc). + MSVC exports nothing from a DLL unless the source says `__declspec(dllexport)` + ... + Use kind = "lib" for this target, or build it for *-windows-gnu (MinGW), + where the linker auto-exports. +``` + +MinGW's linker auto-exports, which is why `*-windows-gnu` is supported and +`*-windows-msvc` is not. Closing this needs a generated `.def` — a symbol scan +over the objects — which is a build-graph node, not a flag. + +### A package's link flags are GNU-spelled + +The generated manifest selects each leg with + +```toml +[target.'cfg(all(arch = "x86_64", os = "windows", env = "msvc"))'.build] +ldflags = ["-Llib/x86_64-windows-msvc", "-lmathkit"] +``` + +Every driver mcpp uses accepts that — including clang on the MSVC ABI, which is +Windows' default here. **Native `cl.exe` does not**: it rejects `-L`. So a +consumer that pins `[toolchain] windows = "msvc@system"` cannot link a packaged +library today. + +Naming the file by path instead (`lib//mathkit.lib`) is the spelling +every driver takes, and it does not work either: ninja runs link commands with +cwd = the output directory, and only the include-family prefixes (`-I`, `-L`, …) +are absolutized against the package root, so a prefix-less token is looked for +in the wrong place — `ld: cannot find lib/x86_64-windows-gnu/libmathkit.a`. A +manifest cannot carry an absolute path and stay relocatable. Closing this needs +the conditional channel to carry `link_library_dirs` / `libraries`, which mcpp +already renders per dialect, but only reads at the top level. ### Implementation partitions diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index b02e0d19..3254a38e 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -237,10 +237,51 @@ ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] |---|---| | `kind = "lib"`(静态) | ✅ 所有 target,三平台都测了 | | `kind = "shared"` on Linux/ELF | ✅ —— 包里同时带链接名与 SONAME | -| `kind = "shared"` on PE / Mach-O | ❌ 拒绝 —— 导入库与 install-name 尚未建模 | +| `kind = "shared"` on PE / MinGW(`*-windows-gnu`) | ✅ —— 包里同时带 `.dll` **和它的导入库** | +| `kind = "shared"` on Mach-O(`*-macos`) | ✅ —— install name 是 `@rpath/`,`.dylib` 可重定位 | +| `kind = "shared"` on PE / MSVC(`*-windows-msvc`) | ❌ 拒绝 —— 见下 | | `kind = "shared"` on `*-musl` | ❌ musl target 是静态链接的 | | 发布预编译 BMI | ❌ 未尝试;BMI 与编译器构建逐位绑定 | | 把依赖打包进去 | ❌ 改为声明依赖(见上) | +| 用**原生 `cl.exe`** 消费这种包 | ❌ 见下 | + +### MSVC 为什么拒绝 `kind = "shared"` + +**不是链接器的问题** —— `link /DLL /IMPLIB:` 本来就能用。是**符号导出**: +MSVC 在没有 `__declspec(dllexport)`、也没有 `.def` 列出符号时,DLL **什么都不导出**, +于是导入库是空的,每个消费者都会拿到一堆 unresolved externals,而那些符号 +明明就在对象文件里 —— 报错点离病因很远。mcpp 选择拒绝,而不是产出这种诊断: + +``` +target 'mathkit': kind = "shared" is not supported for the MSVC ABI (x86_64-windows-msvc). + ... + Use kind = "lib" for this target, or build it for *-windows-gnu (MinGW), + where the linker auto-exports. +``` + +MinGW 的链接器会自动导出,这就是 `*-windows-gnu` 支持而 `*-windows-msvc` 不支持的 +全部原因。要补齐它需要生成 `.def`(对对象做一次符号扫描)—— 那是一个构建图节点, +不是一个 flag。 + +### 包里的链接 flag 是 GNU 拼写 + +生成的 manifest 用这种方式选腿: + +```toml +[target.'cfg(all(arch = "x86_64", os = "windows", env = "msvc"))'.build] +ldflags = ["-Llib/x86_64-windows-msvc", "-lmathkit"] +``` + +mcpp 用到的每个 driver 都吃这一套 —— 包括 Windows 上默认的、面向 MSVC ABI 的 +clang。**原生 `cl.exe` 不吃**:它不认 `-L`。所以固定了 +`[toolchain] windows = "msvc@system"` 的消费者目前链不上打包库。 + +**改成直接写文件路径也不行**(`lib//mathkit.lib` 才是每个 driver 都吃的 +拼写):ninja 执行链接命令时 cwd 是**输出目录**,而只有 include 家族前缀 +(`-I`、`-L` …)会被 `normalize_include_flags` 相对包根绝对化 —— 没有前缀的 token +就会到错误的地方去找:`ld: cannot find lib/x86_64-windows-gnu/libmathkit.a`。 +而 manifest 里写绝对路径就不再可重定位了。要补齐它,需要让条件通道能承载 +`link_library_dirs` / `libraries` —— mcpp 已经能按方言渲染它们,只是只在顶层读。 ### 实现分区 diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index d4dcea6f..1c5151b7 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -38,6 +38,7 @@ import mcpp.toolchain.detect; import mcpp.toolchain.dialect; import mcpp.toolchain.provider; import mcpp.toolchain.registry; +import mcpp.toolchain.triple; // shared_soname_flag decides by TARGET, not host import mcpp.platform.xlings; import mcpp.platform; import mcpp.ui; @@ -193,15 +194,32 @@ std::string join_flags(const std::vector& flags) { return out; } -std::string shared_soname_flag(const LinkUnit& lu) { - if (lu.kind != LinkUnit::SharedLibrary || lu.soname.empty()) return ""; -#if defined(__APPLE__) - return "-Wl,-install_name,@rpath/" + lu.soname; -#elif defined(__linux__) - return "-Wl,-soname," + lu.soname; -#else - return ""; -#endif +// The name a shared library RECORDS about itself, which is not the same thing as +// the file it is written to. +// +// Two corrections here, and each was load-bearing: +// +// * It is decided by the TARGET, not by `#if defined(__APPLE__)` on the host. +// A Linux host cross-linking a PE shared library was emitting +// `-Wl,-soname,` — accepted by mingw's ld and meaningless in a PE, which is +// the quiet kind of wrong. Same class of defect as `target_output` had. +// * On Mach-O the flag is emitted even with no `soname` declared. A dylib's +// default install name is the path it was LINKED at, so a package built in +// /tmp/build-xyz records /tmp/build-xyz and cannot be relocated — which is +// every distributed dylib. `@rpath/` is the only default that travels. +std::string shared_soname_flag(const LinkUnit& lu, const BuildPlan& plan) { + if (lu.kind != LinkUnit::SharedLibrary) return ""; + const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + const std::string os = t ? t->os + : (mcpp::platform::is_macos ? "macos" + : mcpp::platform::is_windows ? "windows" : "linux"); + // PE records no such name: a DLL is found by the filename in the importing + // module's import table, and there is nothing to override. + if (os == "windows") return ""; + const std::string name = lu.soname.empty() + ? lu.output.filename().string() : lu.soname; + if (os == "macos") return "-Wl,-install_name,@rpath/" + name; + return lu.soname.empty() ? "" : "-Wl,-soname," + lu.soname; } // Write only when the bytes would actually change. @@ -1031,7 +1049,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { "LINK"); link_rule("cxx_archive", std::string(dial.archiveCmd), "AR"); link_rule("cxx_shared", - "$ld /nologo /DLL /OUT:$out /IMPLIB:$out.lib " + "$ld /nologo /DLL /OUT:$out $implib_flag " "$in $ldflags $unit_ldflags", "SHARED"); } else { @@ -1039,7 +1057,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { "$cxx $in -o $out $ldflags $unit_ldflags", "LINK"); link_rule("cxx_archive", std::string(dial.archiveCmd), "AR"); link_rule("cxx_shared", - "$cxx -shared $in -o $out $ldflags $soname_flag $unit_ldflags", + "$cxx -shared $in -o $out $ldflags $soname_flag " + "$implib_flag $unit_ldflags", "SHARED"); // mcpp#426: a link unit with no C++ translation unit in it is // linked by the C driver. `g++` appends `-lstdc++` unconditionally, @@ -1051,7 +1070,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { link_rule("c_link", "$cc $in -o $out $c_ldflags $unit_ldflags", "LINK"); link_rule("c_shared", - "$cc -shared $in -o $out $c_ldflags $soname_flag $unit_ldflags", + "$cc -shared $in -o $out $c_ldflags $soname_flag " + "$implib_flag $unit_ldflags", "SHARED"); } } @@ -1786,11 +1806,31 @@ std::string emit_ninja_string(const BuildPlan& plan) { implicit += " " + escape_ninja_path(d.dest); } - std::string out_line = std::format("build {} : {}{}{}\n", - escape_ninja_path(lu.output), rule, ins, + // The import library is a SECOND output of this edge, declared as an + // implicit output (`build dll | implib : …`). Not a separate edge: one + // command writes both, and a second edge claiming to produce the implib + // would run the link twice and race with itself. Not undeclared either — + // the consumer links it, so without a producer ninja stops with "no + // known rule to make it" naming a file this very command writes. + std::string implicitOut; + if (!lu.importLibrary.empty()) + implicitOut = " | " + escape_ninja_path(lu.importLibrary); + + std::string out_line = std::format("build {}{} : {}{}{}\n", + escape_ninja_path(lu.output), implicitOut, rule, ins, implicit.empty() ? std::string{} : " |" + implicit); - if (auto flag = shared_soname_flag(lu); !flag.empty()) + if (auto flag = shared_soname_flag(lu, plan); !flag.empty()) out_line += " soname_flag = " + flag + "\n"; + // Where the linker is told to write it. A rule-level `$out.lib` would + // spell the msvc case `foo.dll.lib`, i.e. a name nothing else in mcpp + // agrees with — the name belongs to plan.cppm's import_library_for, and + // this is how it gets to the command. The SPELLING belongs to the + // dialect table, same as `archiveRemoveArg`. + if (!lu.importLibrary.empty() && !dial.sharedImportLibArg.empty()) { + std::string arg{ dial.sharedImportLibArg }; + arg.replace(arg.find("{}"), 2, escape_ninja_path(lu.importLibrary)); + out_line += " implib_flag = " + arg + "\n"; + } { // Per-unit C++ runtime link, by ROLE. The kind→role map is the // only place that knows a TestBinary runs on the build machine diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 9aac279d..00953c2f 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -18,6 +18,7 @@ import mcpp.toolchain.dialect; import mcpp.toolchain.fingerprint; import mcpp.toolchain.linkmodel; import mcpp.toolchain.triple; +import mcpp.pack.prebuilt; // is_distribution_package — a shipped shared lib is not rebuilt import mcpp.platform; import mcpp.platform.runtime_binding; import mcpp.platform.runtime_env_contract; @@ -93,6 +94,12 @@ struct LinkUnit { // Deciding it stays here; placing it is the emitter's business. std::string loaderTagFlag; std::filesystem::path output; // relative to plan.outputDir + // The import library a PE shared library also produces — empty on ELF and + // Mach-O, and empty for every non-shared unit. It is a SECOND output of the + // link edge, declared implicitly so the consumer that links it has a + // producer; without that, ninja reports "no known rule to make it" naming a + // file the link command does in fact write. + std::filesystem::path importLibrary; // relative to plan.outputDir std::string soname; // ABI name for shared libraries std::vector runtimeAliases; // relative aliases, e.g. bin/libfoo.so.1 std::optional entryMain; // src path of main.cpp for bin @@ -433,16 +440,38 @@ bool is_implementation_source(mcpp::SourceKind kind) { || kind == mcpp::SourceKind::GasAsm || kind == mcpp::SourceKind::NasmAsm; } +// The import library a PE shared target produces, and empty everywhere else. +// +// PE splits a shared library into two files: the `.dll` the loader opens and a +// small archive of stubs the LINKER consumes. Nothing else models that, so the +// name lives here — the link rule writes it, the consumer links it, and the +// packer ships it, all from this one answer. +// +// The two spellings are the toolchains' own conventions, not a choice: +// mingw lib.dll.a (ld --out-implib; keeps `.a` so -l finds it) +// msvc .lib (link /IMPLIB) +// The msvc spelling is the same as a static library's, which is fine because a +// target is `lib` OR `shared`, never both. +std::filesystem::path import_library_for(const mcpp::manifest::Target& t, + const mcpp::toolchain::triple::ArtifactNaming& n) { + if (t.kind != mcpp::manifest::Target::SharedLibrary || !n.sharedNeedsImportLib) + return {}; + const bool msvc = n.libPrefix.empty(); // "" prefix + ".lib" is the msvc row + return std::filesystem::path("bin") / + (msvc ? std::format("{}{}", t.name, n.staticLibExt) + : std::format("{}{}{}{}", n.libPrefix, t.name, + n.sharedLibExt, n.staticLibExt)); +} + // How a CONSUMER links against a shared library. Also a target property: PE has // no rpath and wants an import library, Mach-O uses @loader_path, ELF uses // $ORIGIN. Keying this on the host pointed it the wrong way under cross builds. // -// NOTE: shared libraries have never been verified end to end on PE or Mach-O — -// every shared-library e2e declares `# requires: elf`, and that capability is -// only granted on Linux. The PE branch here (linking the .dll path directly) -// is therefore unproven: mingw's ld tolerates it, MSVC's link.exe cannot. -// make_plan() rejects SharedLibrary targets on non-ELF targets rather than -// emitting something unverifiable — see the guard there. +// PE now links the IMPORT LIBRARY rather than the `.dll` itself. Passing the +// `.dll` is something mingw's ld tolerates and MSVC's link.exe rejects outright, +// so the tolerant case was hiding the broken one — and "it works on the +// toolchain we happened to test" is the whole reason this path went unverified +// for so long. std::vector shared_library_link_flags( const mcpp::manifest::Target& t, const mcpp::toolchain::triple::ArtifactNaming& n, @@ -452,7 +481,7 @@ std::vector shared_library_link_flags( const bool macho = target.empty() ? bool(mcpp::platform::is_macos) : target.os == "macos"; if (pe) { - flags.push_back(target_output(t, n).generic_string()); + flags.push_back(import_library_for(t, n).generic_string()); } else { flags.push_back("-L" + target_output(t, n).parent_path().generic_string()); flags.push_back(macho ? "-Wl,-rpath,@loader_path" @@ -973,31 +1002,59 @@ make_plan(const mcpp::manifest::Manifest& manifest, return flag ? std::string(*flag) : std::string{}; }; - // Shared libraries have never been verified end to end on PE or Mach-O: - // every shared-library e2e declares `# requires: elf`, and run_all.sh only - // grants that capability on Linux. The non-ELF paths through - // shared_library_link_flags are therefore unproven — mingw's ld tolerates - // linking a .dll directly, MSVC's link.exe cannot, and neither has an - // import library to link against because mcpp does not model one. + // WHAT `kind = "shared"` SUPPORTS, AND THE ONE THING IT STILL DOES NOT. // - // Refusing is strictly better than emitting something unverifiable: a - // branch that is neither tested nor willing to say no is the hardest kind - // of debt, because it can be neither trusted nor deleted. - if (!targetTriple.empty() && targetTriple.os != "linux") { - for (auto const& t : manifest.targets) { - if (t.kind != mcpp::manifest::Target::SharedLibrary) continue; - return std::unexpected(std::format( - "target '{}': shared libraries are only supported for Linux (ELF) " - "targets today.\n" - " target '{}' is kind=\"shared\"; build it as kind=\"lib\" " - "(static) for this target,\n" - " or build it for a linux target.\n" - " note: PE consumers need an import library and Mach-O needs " - "install-name handling;\n" - " neither is modelled yet, so mcpp refuses rather than " - "producing an artifact\n" - " nothing has ever verified.", - targetTriple.str(), t.name)); + // ELF, Mach-O and PE/MinGW are modelled: the import library PE consumers + // link is `import_library_for` above, and Mach-O's install name is set to + // `@rpath/` unconditionally (see shared_soname_flag) rather than + // defaulting to this machine's build path. + // + // PE/MSVC is refused, and the reason is not the linker — `link /DLL + // /IMPLIB:` has been in the rule table all along. It is symbol export: MSVC + // exports nothing from a DLL without `__declspec(dllexport)` or a `.def`, + // so the import library comes out EMPTY and the consumer fails with + // unresolved externals naming symbols that are plainly in the object files. + // Producing that is worse than refusing, because the diagnostic points + // nowhere near the cause. + // + // ⚠️ THE PREVIOUS GUARD HAD A HOLE, and it was in the case that matters + // most. It read `!targetTriple.empty() && os != "linux"`, and targetTriple + // is EMPTY for a native build — so it refused a cross build to macOS (which + // is unservable anyway, i.e. unreachable) while letting a NATIVE macOS or + // native Windows build walk straight into the unverified paths it was + // written to keep people out of. The resolved target is what decides. + { + const std::string targetOs = targetTriple.empty() + ? (mcpp::platform::is_macos ? "macos" + : mcpp::platform::is_windows ? "windows" : "linux") + : targetTriple.os; + // The ABI, from the toolchain rather than from `naming`: on a native + // Windows build the host naming constants are the MSVC ones whatever the + // toolchain is (lib_prefix is "" and static_lib_ext is ".lib" for mingw + // too), so asking `naming` would refuse MinGW as well. `is_msvc_target` + // reads the compiler's own -dumpmachine answer, which distinguishes them + // and also covers clang driving the MSVC ABI — clang auto-exports no more + // than link.exe does, so "which compiler binary" is the wrong question. + if (targetOs == "windows" && mcpp::toolchain::is_msvc_target(tc)) { + for (auto const& t : manifest.targets) { + if (t.kind != mcpp::manifest::Target::SharedLibrary) continue; + return std::unexpected(std::format( + "target '{}': kind = \"shared\" is not supported for the MSVC " + "ABI ({}).\n" + " MSVC exports nothing from a DLL unless the source says " + "`__declspec(dllexport)`\n" + " or a `.def` file lists the symbols, so the import library " + "would be empty and\n" + " every consumer would fail with unresolved externals naming " + "symbols that are\n" + " visibly present in the objects. mcpp refuses rather than " + "produce that.\n" + " Use kind = \"lib\" for this target, or build it for " + "*-windows-gnu (MinGW),\n" + " where the linker auto-exports.", + t.name, + targetTriple.empty() ? "native" : targetTriple.str())); + } } } @@ -1416,6 +1473,17 @@ make_plan(const mcpp::manifest::Manifest& manifest, for (std::size_t i = 1; i < packages.size(); ++i) { auto const& p = packages[i]; auto qname = qualified_package_name(p.manifest); + // A DISTRIBUTION PACKAGE's shared target is already built — that is what + // the package IS. Creating a link unit for it made ninja fail outright + // with `multiple rules generate bin/libmathkit.dll`: this loop declared + // an edge that relinks the library, and the deploy machinery declared + // another that stages the shipped one. + // + // Rebuilding it would be wrong even without the collision: a + // distribution package's `sources` are its published INTERFACE only, so + // the relink would quietly produce a library missing every + // implementation unit the publisher withheld. + if (mcpp::pack::is_distribution_package(p.manifest)) continue; for (auto const& t : p.manifest.targets) { if (t.kind != mcpp::manifest::Target::SharedLibrary) continue; sharedDepPackages.insert(qname); @@ -1494,6 +1562,12 @@ make_plan(const mcpp::manifest::Manifest& manifest, for (auto targetIndex : targetsIt->second) { auto const& dep = sharedDepTargets[targetIndex]; lu.implicitInputs.push_back(dep.output); + // On PE what the consumer actually LINKS is the import library, + // so that is the file the edge has to wait for — the `.dll` + // above is still a prerequisite because it must exist beside the + // exe to run, but it is not what resolves the symbols. + if (auto imp = import_library_for(dep.target, naming); !imp.empty()) + lu.implicitInputs.push_back(imp); // The SONAME alias is a prerequisite too, not a by-product: the // library is written as bin/libX11.so but records SONAME // libX11.so.6, so both the linker (resolving a transitive @@ -1549,6 +1623,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, lu.targetName = dep.target.name; lu.kind = LinkUnit::SharedLibrary; lu.output = dep.output; + lu.importLibrary = import_library_for(dep.target, naming); lu.soname = dep.target.soname; lu.runtimeAliases = runtime_aliases_for_target(dep.target, naming); lu.loaderTagFlag = loader_tag_flag(lu.kind); @@ -1576,6 +1651,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, } else if (t.kind == mcpp::manifest::Target::SharedLibrary) { lu.kind = LinkUnit::SharedLibrary; lu.output = target_output(t, naming); + lu.importLibrary = import_library_for(t, naming); lu.soname = t.soname; lu.runtimeAliases = runtime_aliases_for_target(t, naming); } else if (t.kind == mcpp::manifest::Target::TestBinary) { diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 0a436677..6d16502d 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1238,6 +1238,47 @@ prepare_build(bool print_fingerprint, " An explicit [target.{}] toolchain override can opt in early.", parsed->str(), parsed->str())); } + // Known, supported — and IMPOSSIBLE ON THIS HOST. + // + // Without this the target falls through to the host toolchain and the + // build SUCCEEDS, which is the failure the check above calls the worst + // one, arriving through a different door. Measured on Linux: + // + // $ mcpp build --target x86_64-windows-msvc + // Resolved gcc@16.1.0 → x86_64-windows-msvc → …/xim-x-gcc/bin/g++ + // Finished dev [unoptimized + debuginfo] in 0.07s + // $ ls target/ + // x86_64-linux-gnu/ ← an ELF, reported as a Windows build + // + // The vocabulary tier says "mcpp supports this target"; it never said + // "this machine can produce it". `host_can_serve` is the answer to the + // second question and lives beside the payload resolution it has to + // agree with. + // + // The escape hatch stays open on purpose: an explicit `[target.X]` + // toolchain override means the author is supplying the cross toolchain + // themselves, and mcpp's payload matrix has no standing to refuse it. + if (known && known->tier != "planned" && !hasToolchainOverride && parsed + && !mcpp::toolchain::host_can_serve(*parsed)) { + std::string servable; + for (auto const& info : triple::known_targets()) { + auto t = triple::parse(info.canonical); + if (!t || info.tier == "planned") continue; + if (!mcpp::toolchain::host_can_serve(*t)) continue; + if (!servable.empty()) servable += ", "; + servable += t->str(); + } + return std::unexpected(std::format( + "target '{}' cannot be built on this host — no toolchain payload " + "exists that runs here and produces it.\n" + " this host can build: {}\n" + " Build it on a host that can, or supply your own cross " + "toolchain with an\n" + " explicit [target.{}] toolchain = \"…\" section.", + parsed->str(), + servable.empty() ? "(nothing — `mcpp toolchain list`)" : servable, + parsed->str())); + } // Canonical from here on: cfg evaluation, spec attachment and the // target/ output directory all see one spelling. if (parsed) overrides.target_triple = parsed->str(); diff --git a/src/pack/library.cppm b/src/pack/library.cppm index f1ba9967..956d6ba0 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -63,6 +63,13 @@ struct LibraryLeg { // built file links fine and then cannot start. std::string soname; bool shared = false; + // The IMPORT LIBRARY, on PE only — absolute, empty everywhere else. + // + // A PE shared library is two files: the `.dll` the loader opens and an + // archive of stubs the LINKER consumes. Shipping only the `.dll` gives a + // package that no linker can use, so the package carries both and the + // emitted manifest points consumers at this one. + std::filesystem::path importLibrary; }; struct LibraryPackPlan { @@ -246,6 +253,25 @@ run_library_pack(const LibraryPackPlan& plan) auto dst = plan.stagingRoot / "lib" / leg.triple / name; if (auto r = copy_into(leg.artifact, dst); !r) return std::unexpected(r.error()); + // The import library travels beside the .dll, and it is what the emitted + // manifest names as the link input. Refused rather than skipped when + // missing: a PE shared package without one links for nobody, and finding + // that out at the consumer's link step points at the consumer. + std::string linkFile = name; + if (!leg.importLibrary.empty()) { + if (!std::filesystem::exists(leg.importLibrary, ec)) { + return std::unexpected(LibraryPackError{ std::format( + "the build for '{}' produced no import library at '{}'.\n" + " A PE shared library is two files, and consumers link the " + "second one — shipping only the .dll gives a package no " + "linker can use.", + leg.triple, leg.importLibrary.string()) }); + } + linkFile = leg.importLibrary.filename().string(); + if (auto r = copy_into(leg.importLibrary, dst.parent_path() / linkFile); !r) + return std::unexpected(r.error()); + } + // A shared library needs BOTH of its names present. // // `-lmathkit-shared` resolves `libmathkit-shared.so` at link time, but @@ -300,6 +326,7 @@ run_library_pack(const LibraryPackPlan& plan) docLegs.push_back(PackageLeg{ .triple = leg.triple, .libFile = name, + .linkFile = linkFile, .linkName = leg.linkName, .abiTag = leg.abiTag, .digest = file_digest(dst), diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 4a20b80a..595a7032 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -165,11 +165,15 @@ export int build_and_pack_library(const std::string& targetName, return 1; } std::filesystem::path artifact; + std::filesystem::path importLib; for (auto const& lu : ctx->plan.linkUnits) { if (lu.targetName != targetName) continue; if (lu.kind != mcpp::build::LinkUnit::StaticLibrary && lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; artifact = ctx->outputDir / lu.output; + // Also from the plan, for the same reason: the link unit knows + // whether it produced an import library and where. + if (!lu.importLibrary.empty()) importLib = ctx->outputDir / lu.importLibrary; break; } if (artifact.empty()) { @@ -316,6 +320,7 @@ export int build_and_pack_library(const std::string& targetName, mcpp::toolchain::dialect_for(ctx->tc).archiveRemoveTakesArchiveFirst, .soname = target->soname, .shared = shared, + .importLibrary = importLib, }); mcpp::ui::status("Packed leg", std::format("{} [{}]", triple, tag.str())); } diff --git a/src/pack/manifest_emit.cppm b/src/pack/manifest_emit.cppm index 5bdbf045..e2c858a6 100644 --- a/src/pack/manifest_emit.cppm +++ b/src/pack/manifest_emit.cppm @@ -46,6 +46,10 @@ export namespace mcpp::pack { struct PackageLeg { std::string triple; // canonical, e.g. "x86_64-linux-gnu" std::string libFile; // "libmathkit.a" — as it sits in lib// + // The file a CONSUMER links. Same as libFile except for a PE shared library, + // where the loader opens the `.dll` and the linker consumes the import + // library beside it. + std::string linkFile; std::string linkName; // "mathkit" — the -l argument std::string abiTag; // "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" std::string digest; // "sha256:…" @@ -180,7 +184,48 @@ std::string emit_package_manifest(const PackageDoc& doc) { // ── how to link each leg ─────────────────────────────────────────── for (auto const& leg : doc.legs) { o += std::format("[target.'{}'.build]\n", cfg_predicate_for(leg.triple)); - o += std::format("ldflags = [\"-Llib/{}\", \"-l{}\"]\n\n", leg.triple, leg.linkName); + // `-L` + `-l`, on every target including PE. + // + // ⚠️ NAMING THE FILE BY PATH INSTEAD DOES NOT WORK, and it is worth + // writing down because it looks strictly better. A bare path is the one + // spelling every driver accepts (cl, clang, gcc, link.exe) and it names + // the exact file rather than asking the linker to search. Measured: it + // fails with `ld: cannot find lib/x86_64-windows-gnu/libmathkit.a`, + // because ninja runs link commands with cwd = the OUTPUT dir and only + // the include-family prefixes (`-I`, `-L`, …) get absolutized against + // the package root by normalize_include_flags. A prefix-less token has + // nothing to hook that on, and the manifest cannot carry an absolute + // path without ceasing to be relocatable. + // + // `-l` resolves the right file on PE too: ld tries `libX.dll.a` before + // `libX.a`, and lld-link/clang tries `X.lib` — which is exactly how + // `import_library_for` names them. What was actually missing was the + // import library being IN the package at all. + // + // Consequence to know: a consumer driven by NATIVE cl.exe cannot use + // these flags (cl rejects `-L`). Recorded in docs/12 rather than papered + // over — mcpp's own Windows default is clang, which takes them. + // + // ⚠️ `-Wl,-Bdynamic` is REQUIRED for a PE shared leg, and only there. + // mcpp gives PE executables `-static` (the self-contained C++ runtime + // contract: no libstdc++-6.dll beside the exe), and `-static` puts ld in + // static-only mode, where it refuses an import library and reports + // `cannot find -lmathkit / have you installed the static version of the + // mathkit library?` — a message that names neither the DLL nor `-static`. + // Switching to dynamic mode just before this `-l` fixes it; `-static` + // arrives later on the line and still governs the runtime libraries + // after it. Measured: without it the link fails, with it the program + // links, deploys and runs. + // + // Only for env=gnu: an msvc-ABI shared leg cannot exist (make_plan + // refuses it — MSVC exports nothing without dllexport), and lld-link + // would not understand the flag if one did. + const bool peGnuShared = leg.shared + && leg.triple.find("windows-gnu") != std::string::npos; + o += std::format("ldflags = [\"-Llib/{}\", {}\"-l{}\"]\n\n", + leg.triple, + peGnuShared ? "\"-Wl,-Bdynamic\", " : "", + leg.linkName); } // A shared library has to be FOUND at run time as well as linked, and the // two are different search paths — `link_library_dirs` is not rpath. diff --git a/src/toolchain/dialect.cppm b/src/toolchain/dialect.cppm index dff73c5a..98d0b210 100644 --- a/src/toolchain/dialect.cppm +++ b/src/toolchain/dialect.cppm @@ -106,6 +106,16 @@ struct CommandDialect { // spaces after the archive path. std::string_view archiveRemoveArg; // "d" needs no {} | "/REMOVE:{}" bool archiveRemoveTakesArchiveFirst = true; + + // How the linker is told where to write a shared library's IMPORT LIBRARY — + // the archive of stubs a PE consumer links against, as opposed to the `.dll` + // the loader opens. `{}` is the path. + // + // Emitted only when the TARGET has import libraries at all (PE); on ELF and + // Mach-O the shared library is its own link input and there is nothing to + // write. So this being non-empty in both rows is not a contradiction: the + // rows describe how to SAY it, and the target decides whether to. + std::string_view sharedImportLibArg; // "-Wl,--out-implib,{}" | "/IMPLIB:{}" }; // Dialect lookup. GCC / Clang / MinGW → gnu; MSVC → msvc. @@ -209,6 +219,10 @@ constexpr CommandDialect kGnuDialect{ // `ar d ...` — one verb, then every member. .archiveRemoveArg = "d", .archiveRemoveTakesArchiveFirst = true, + // ld/lld: `--out-implib` is what makes a PE shared library linkable at all. + // Without it mingw writes only the .dll, consumers link the .dll directly, + // and that works — until the same package is consumed by any other linker. + .sharedImportLibArg = "-Wl,--out-implib,{}", }; // Native cl.exe. Unreachable in builds until the MSVC backend lands @@ -244,6 +258,10 @@ constexpr CommandDialect kMsvcDialect{ // reported with the command that produced it rather than swallowed. .archiveRemoveArg = "/REMOVE:{}", .archiveRemoveTakesArchiveFirst = false, + // link.exe writes one whether asked or not; naming it explicitly is how the + // path stays the one plan.cppm chose, instead of the linker's `$out`-derived + // guess (`foo.dll.lib`) that nothing else in mcpp agrees with. + .sharedImportLibArg = "/IMPLIB:{}", }; } // namespace diff --git a/tests/e2e/245_pack_library_fat_target_selection.sh b/tests/e2e/245_pack_library_fat_target_selection.sh index 5060679b..6426a1b0 100755 --- a/tests/e2e/245_pack_library_fat_target_selection.sh +++ b/tests/e2e/245_pack_library_fat_target_selection.sh @@ -1,8 +1,17 @@ #!/usr/bin/env bash -# requires: gcc +# requires: gcc musl # 245_pack_library_fat_target_selection.sh — one package, several targets, and # each build picks exactly its own leg. # +# `musl` is now declared as well as `gcc`. It always needed both — the pack below +# asks for a musl leg outright — and CI was green only because the payload +# happens to be warm there. A requirement the test does not declare is one that +# turns into a confusing failure the day it stops being true. +# +# The Windows counterpart (msvc + mingw legs in one package) is 256; `# requires:` +# cannot express "gcc+musl OR msvc+mingw", and a host that has neither should +# skip rather than half-run. +# # ⚠️ THE PREDICATE IS THE POINT. Each leg is selected by a generated # `cfg(all(arch=…, os=…, env=…))` block and NOT by a bare `[target.'']` # key. The bare form only matched when `--target` was passed: a plain diff --git a/tests/e2e/255_pack_library_msvc_archiver.sh b/tests/e2e/255_pack_library_msvc_archiver.sh index 55583f7c..eb4f7ae3 100755 --- a/tests/e2e/255_pack_library_msvc_archiver.sh +++ b/tests/e2e/255_pack_library_msvc_archiver.sh @@ -113,6 +113,12 @@ cat > app/src/main.cpp <<'EOF' import mathkit; int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } EOF +# ⚠️ The CONSUMER deliberately takes the DEFAULT toolchain (clang on the MSVC +# ABI), not msvc@system. The emitted manifest links each leg with `-Llib/ +# -l`, which is GNU spelling: clang accepts it, native cl.exe rejects `-L` +# outright. That is a real limitation of the generated manifest (recorded in +# docs/12), and it is NOT what this test is about — pinning cl here would make +# 255 fail for a reason that has nothing to do with the archiver. cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed"; exit 1; } grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } diff --git a/tests/e2e/256_pack_library_fat_windows.sh b/tests/e2e/256_pack_library_fat_windows.sh new file mode 100755 index 00000000..8eaef932 --- /dev/null +++ b/tests/e2e/256_pack_library_fat_windows.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# requires: msvc mingw +# 256_pack_library_fat_windows.sh — a fat package on Windows: an MSVC leg and a +# MinGW leg in ONE package, each selected by its own predicate. +# +# WHY THIS IS THE SHARPEST VERSION OF THE TEST. 245 packs `linux-gnu` + +# `linux-musl`, and both legs are called `libmathkit.a`. Here they are not: +# +# x86_64-windows-msvc mathkit.lib (lib.exe, no `lib` prefix) +# x86_64-windows-gnu libmathkit.a (ar, GNU naming) +# +# Two different filenames for the same target in one package is the plainest +# possible evidence that `lib/` has to be keyed by TRIPLE rather than by +# platform — a layout keyed by "windows" could hold only one of these, and the +# one it held would link for exactly half of its consumers. +# +# ⚠️ WHY IT DID NOT EXIST BEFORE, AND WHY THAT WAS THE WRONG CALL. This +# combination was reported as "not possible on Windows". It is: host_can_serve +# (registry.cppm:542-566) grants a Windows host `*-windows-msvc`, +# `*-windows-gnu` AND host-arch `*-linux-musl` — three targets, so a fat package +# needs no cross-compilation trickery at all. What was actually missing was the +# MinGW payload in the Windows e2e job. macOS is the genuinely impossible case: +# it serves exactly one target ("macOS has no Linux-targeting payload at all"), +# so there is no second leg to pack there under any circumstances. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +# The host arch, from mcpp's own answer rather than assumed: an aarch64 Windows +# runner would make every hardcoded `x86_64-` here a silent skip of the real work. +cd mathkit +"$MCPP" build > probe.log 2>&1 || { cat probe.log; echo "probe build failed"; exit 1; } +host_triple="$(find target -mindepth 1 -maxdepth 1 -type d ! -name dist -exec basename {} \; | head -1)" +ARCH="${host_triple%%-*}" +[[ -n "$ARCH" && "$host_triple" == *windows* ]] || { + echo "FAIL: expected a windows host triple, read '$host_triple'"; exit 1; } +rm -rf target + +MSVC_LEG="$ARCH-windows-msvc" +MINGW_LEG="$ARCH-windows-gnu" + +"$MCPP" pack mathkit --target "$MSVC_LEG" --target "$MINGW_LEG" > pack.log 2>&1 \ + || { cat pack.log; echo "fat pack failed"; exit 1; } + +pkg="$TMP/mathkit/target/dist/mathkit-0.1.0" +PKG_HOST="$(host_path "$pkg")" + +# ── the two legs, under their two different names ─────────────────────── +[[ -f "$pkg/lib/$MSVC_LEG/mathkit.lib" ]] || { + find "$pkg" -type f + echo "FAIL: the MSVC leg is not lib/$MSVC_LEG/mathkit.lib" + exit 1; } +[[ -f "$pkg/lib/$MINGW_LEG/libmathkit.a" ]] || { + find "$pkg" -type f + echo "FAIL: the MinGW leg is not lib/$MINGW_LEG/libmathkit.a" + exit 1; } +# Stated as an inequality too, because "both files exist" would still pass if the +# packer had put the same artifact in both directories. +[[ "$(basename "$pkg/lib/$MSVC_LEG/mathkit.lib")" \ + != "$(basename "$pkg/lib/$MINGW_LEG/libmathkit.a")" ]] || { + echo "FAIL: the two legs ended up with the same filename"; exit 1; } + +# Selected by predicate, never by a bare triple (inert on a native build — the +# defect 245's header describes). +grep -q "target\.'cfg(" "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "FAIL: legs are not selected by cfg() predicates"; exit 1; } +grep -qE "^\[target\.'$ARCH-" "$pkg/mcpp.toml" && { + cat "$pkg/mcpp.toml" + echo "FAIL: a leg is selected by a BARE TRIPLE, which is inert on a native build" + exit 1; } +# env= must appear: `os=windows` alone cannot separate these two legs, and a +# predicate that cannot separate them would hand MSVC consumers the MinGW archive. +grep -q "env=" "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml" + echo "FAIL: the predicates carry no env axis, so both legs match both targets" + exit 1; } + +# ── and each consumer resolves its own ────────────────────────────────── +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < "$TMP/$label.log" 2>&1 ) \ + || { cat "$TMP/$label.log"; echo "$label build failed"; exit 1; } + local nj; nj="$(find app/target -name build.ninja | head -1)" + grep -o "dist/mathkit-0.1.0/lib/[A-Za-z0-9_-]*" "$nj" | sort -u > "$TMP/$label.legs" + [[ "$(wc -l < "$TMP/$label.legs")" -eq 1 ]] || { + echo "$label saw more than one leg:"; cat "$TMP/$label.legs"; exit 1; } + grep -q "lib/$want\$" "$TMP/$label.legs" || { + echo "$label picked the wrong leg:"; cat "$TMP/$label.legs"; exit 1; } +} + +# Native first: that is the case a bare-triple key silently failed. +check native "$MSVC_LEG" +check msvc "$MSVC_LEG" --target "$MSVC_LEG" +check mingw "$MINGW_LEG" --target "$MINGW_LEG" + +echo "PASS: one Windows package carries an MSVC leg and a MinGW leg, chosen apart" diff --git a/tests/e2e/257_shared_library_pe.sh b/tests/e2e/257_shared_library_pe.sh new file mode 100755 index 00000000..1c52a8ed --- /dev/null +++ b/tests/e2e/257_shared_library_pe.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +# requires: mingw-cross wine +# 257_shared_library_pe.sh — `kind = "shared"` on PE: a DLL, its import library, +# a package carrying both, and a consumer that links and RUNS. +# +# Until now this was Linux-only, and the refusal that said so had a hole in the +# case that mattered: it read `!targetTriple.empty() && os != "linux"`, and +# targetTriple is EMPTY for a native build — so it turned away a cross build to +# macOS (unservable anyway) while letting a native macOS or native Windows build +# walk straight into the unverified paths it existed to prevent. +# +# What was actually missing on PE was the IMPORT LIBRARY. A PE shared library is +# two files: the `.dll` the loader opens, and an archive of stubs the linker +# consumes. mcpp wrote only the first, and consumers linked the `.dll` directly — +# which mingw's ld tolerates and no other linker does. So the tolerant case was +# hiding the broken one. +# +# Four things are asserted, in the order they can fail: +# 1. the build writes BOTH files +# 2. `mcpp pack` ships both, and points consumers at the import library +# 3. a consumer of the PACKAGE links, gets the DLL deployed beside its exe, +# and prints the right answer under wine +# 4. an MSVC-ABI shared target is still refused, and says why +# +# ⚠️ WINE IS EVIDENCE, NOT PROOF. Wine maps the filesystem differently (Z:) and +# has previously passed things a real Windows failed. The Windows-native halves +# of the same claims live in 242/255/256; what wine gives here is the loader +# actually resolving the DLL, which no Linux-hosted check can show. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +TRIPLE=x86_64-windows-gnu + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "shared" +EOF + +# ── 1. the build writes the DLL and the import library ────────────────── +cd mathkit +"$MCPP" build --target "$TRIPLE" > build.log 2>&1 || { cat build.log; echo "build failed"; exit 1; } +dll="$(find target -name 'libmathkit.dll' | head -1)" +imp="$(find target -name 'libmathkit.dll.a' | head -1)" +[[ -n "$dll" ]] || { find target -type f; echo "FAIL: no .dll"; exit 1; } +[[ -n "$imp" ]] || { + find target -type f + echo "FAIL: no import library. The .dll alone is a library only mingw's ld" + echo " will link, so the package would be unusable everywhere else." + exit 1; } +# It is a declared output of the link edge, not a side effect ninja knows nothing +# about — otherwise the consumer that links it has no producer and ninja stops +# with 'no known rule to make it'. +nj="$(find target -name build.ninja | head -1)" +grep -qE "^build bin/libmathkit\.dll \| bin/libmathkit\.dll\.a : " "$nj" || { + grep -n 'libmathkit' "$nj" + echo "FAIL: the import library is not an implicit output of the link edge" + exit 1; } + +# ── 2. the package carries both, and names the right one to link ──────── +rm -rf target +"$MCPP" pack mathkit --target "$TRIPLE" --format dir > pack.log 2>&1 \ + || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0*' | head -1)" +[[ -f "$pkg/lib/$TRIPLE/libmathkit.dll" ]] || { find "$pkg" -type f; echo "FAIL: no DLL in the package"; exit 1; } +[[ -f "$pkg/lib/$TRIPLE/libmathkit.dll.a" ]] || { find "$pkg" -type f; echo "FAIL: no import library in the package"; exit 1; } +# `-static` is what mcpp gives PE executables, and it puts ld in static-only +# mode where an import library is refused with a message that names neither the +# DLL nor `-static`. The emitted manifest has to switch modes for this one `-l`. +grep -q 'Bdynamic' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml" + echo "FAIL: the shared leg's ldflags do not leave static-link mode, so a" + echo " consumer will fail with 'have you installed the static version'" + exit 1; } + +# ── 3. a consumer links it, gets the DLL, and runs ────────────────────── +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < build.log 2>&1 ) || { + cat app/build.log + echo "FAIL: the consumer could not link against the packaged shared library." + echo " 'multiple rules generate bin/libmathkit.dll' means a link unit was" + echo " created for a library that is already built." + exit 1; } +exe="$(find app/target -name app.exe | head -1)" +[[ -n "$exe" ]] || { echo "FAIL: no app.exe"; exit 1; } +# PE has no rpath: the DLL has to BE there, or the process dies at load time +# with a status code and no message. +[[ -f "$(dirname "$exe")/libmathkit.dll" ]] || { + ls "$(dirname "$exe")" + echo "FAIL: the DLL was not deployed beside the exe — the program cannot start" + exit 1; } +out="$(cd "$(dirname "$exe")" && WINEDEBUG=-all wine ./app.exe 2>&1 || true)" +grep -q 'ok=42' <<<"$out" || { echo "$out"; echo "FAIL: wrong answer from the PE consumer"; exit 1; } + +# ── 4. windows-msvc is refused here, but by the TARGET gate ───────────── +# +# The MSVC shared-library refusal cannot be observed from a Linux host: this +# machine cannot serve `x86_64-windows-msvc` at all, so the target gate answers +# first and make_plan is never reached. Asserting `dllexport` here would be +# asserting a message this host cannot produce — the MSVC-ABI half lives in 258, +# under `# requires: msvc`. +# +# What IS worth pinning here is that the refusal happens at all, and names the +# host rather than silently building an ELF: that is precisely what this used to +# do. `mcpp build --target x86_64-windows-msvc` on Linux resolved the native g++, +# wrote target/x86_64-linux-gnu/, and reported success. +cd "$TMP/mathkit" +if "$MCPP" build --target x86_64-windows-msvc > msvc.log 2>&1; then + echo "FAIL: --target x86_64-windows-msvc SUCCEEDED on a host that cannot" + echo " serve it. Check target/: an ELF reported as a Windows build is" + echo " the failure mode the target vocabulary check exists to prevent." + exit 1 +fi +grep -q 'cannot be built on this host' msvc.log || { + cat msvc.log + echo "FAIL: refused, but not with the host-servability reason — the message" + echo " has to say which host can build it, or it is not actionable." + exit 1; } + +echo "PASS: PE shared libraries build, pack, link and run; unservable stays refused" diff --git a/tests/e2e/258_shared_library_msvc_refused.sh b/tests/e2e/258_shared_library_msvc_refused.sh new file mode 100755 index 00000000..81f8bf73 --- /dev/null +++ b/tests/e2e/258_shared_library_msvc_refused.sh @@ -0,0 +1,89 @@ +#!/usr/bin/env bash +# requires: msvc +# 258_shared_library_msvc_refused.sh — `kind = "shared"` on the MSVC ABI is +# refused, and the message says why. +# +# The refusal is NOT about the linker. `link /DLL /IMPLIB:` has been in mcpp's +# rule table all along. It is about symbol export: MSVC exports nothing from a +# DLL unless the source says `__declspec(dllexport)` or a `.def` file lists the +# symbols. Without that the import library comes out EMPTY and every consumer +# fails with unresolved externals naming symbols that are plainly in the object +# files — a diagnostic pointing nowhere near its cause. Producing that is worse +# than refusing. +# +# ⚠️ WHY THIS TEST IS SEPARATE FROM 257. On a Linux host this cannot be observed: +# the machine cannot serve `x86_64-windows-msvc`, so the target-vocabulary gate +# answers first and make_plan is never reached. A `# requires:`-less version of +# this test would assert a message its host cannot produce. +# +# ⚠️ AND WHY IT PINS BOTH SIDES. `kind = "lib"` must still build in the same +# project with the same toolchain. Asserting only the refusal cannot distinguish +# "shared is refused" from "this project does not build at all". +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF + +manifest() { # $1 = kind + cat > "$TMP/mathkit/mcpp.toml" < shared.log 2>&1; then + echo "FAIL: a kind=\"shared\" target built for the MSVC ABI." + find target -name '*.dll' -o -name '*.lib' | head + echo " If the import library is empty, consumers fail with unresolved" + echo " externals for symbols that are visibly present in the objects." + exit 1 +fi +grep -qi 'dllexport' shared.log || { + cat shared.log + echo "FAIL: refused, but not by the shared-library gate — the message must" + echo " name symbol export, or the reader cannot tell what to do about it." + exit 1; } +# And it must point somewhere: a refusal with no way forward is a dead end. +grep -q 'windows-gnu' shared.log || { + cat shared.log + echo "FAIL: the refusal names no alternative. MinGW auto-exports, and that is" + echo " the answer for anyone who actually needs a DLL here." + exit 1; } + +# ── and the static form still builds, same project, same toolchain ────── +manifest lib +rm -rf target +"$MCPP" build > lib.log 2>&1 || { + cat lib.log + echo "FAIL: kind=\"lib\" does not build either, so the refusal above proves" + echo " nothing about shared libraries specifically." + exit 1; } +[[ -n "$(find target -name 'mathkit.lib' | head -1)" ]] || { + find target -type f | head + echo "FAIL: no mathkit.lib — the MSVC static path did not produce its artifact" + exit 1; } + +echo "PASS: MSVC refuses kind=\"shared\" for the export reason, and still builds kind=\"lib\"" diff --git a/tests/e2e/259_shared_library_macho.sh b/tests/e2e/259_shared_library_macho.sh new file mode 100755 index 00000000..8cade8af --- /dev/null +++ b/tests/e2e/259_shared_library_macho.sh @@ -0,0 +1,118 @@ +#!/usr/bin/env bash +# requires: macos +# 259_shared_library_macho.sh — `kind = "shared"` on Mach-O: a `.dylib` that can +# be moved, packed, and loaded from beside the consumer's binary. +# +# WHY THE INSTALL NAME IS THE WHOLE TEST. A Mach-O shared library records the +# name it will be FOUND by, and the default is the path it was LINKED at. So a +# library built in `/private/var/folders/…/target/…/bin` records that path, and +# the moment it is packed and extracted somewhere else, every consumer of it +# fails at load time looking for a directory that no longer exists — on the +# publisher's machine it works perfectly. +# +# `@rpath/` is the only default that survives being moved: it defers the +# question to the consumer, whose own `-Wl,-rpath,@loader_path` answers "next to +# me". mcpp used to emit `-install_name` only when the manifest declared a +# `soname`, and to decide Mach-O-ness with `#if defined(__APPLE__)` on the HOST +# rather than from the target. +# +# ⚠️ AND WHY IT ASSERTS THE PATH IS *GONE*. Checking that the consumer runs in +# place proves nothing: the absolute install name still resolves while the build +# directory exists. The package has to be consumed from a location the original +# build path cannot satisfy — so the library is packed, the producer's whole +# build tree is DELETED, and only then is the consumer built and run. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "shared" +EOF + +# ── 1. it builds, and it is a dylib ───────────────────────────────────── +cd mathkit +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "build failed"; exit 1; } +dylib="$(find target -name 'libmathkit.dylib' | head -1)" +[[ -n "$dylib" ]] || { find target -type f | head; echo "FAIL: no .dylib"; exit 1; } + +# ── 2. the install name is @rpath, not this machine's build path ───────── +# +# A functional probe, not `command -v otool`: on macOS a bare tool name can +# resolve to an xlings shim that reports "not installed" and exits non-zero, +# which under `set -e` would kill the test instead of skipping the inspection. +if install_name="$(otool -D "$dylib" 2>/dev/null | tail -1)"; then + case "$install_name" in + @rpath/libmathkit.dylib) ;; + *) + echo "FAIL: install name is '$install_name'" + echo " Anything absolute here is this machine's build directory, and the" + echo " library stops loading the moment it is extracted anywhere else." + exit 1 ;; + esac +else + echo "NOTE: otool unavailable — the install name was not inspected directly." + echo " Step 4 still fails if it is wrong, just with a loader error." +fi + +# ── 3. pack it ────────────────────────────────────────────────────────── +rm -rf target +"$MCPP" pack mathkit --format dir > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkgrel="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0*' | head -1)" +[[ -n "$pkgrel" ]] || { cat pack.log; echo "FAIL: no package"; exit 1; } +# OUT of the producer's tree, so the next step can delete that tree entirely. +pkg="$TMP/pkg" +cp -R "$TMP/mathkit/$pkgrel" "$pkg" +[[ -f "$pkg/lib/"*"/libmathkit.dylib" ]] 2>/dev/null || { + find "$pkg" -type f -o -type l + echo "FAIL: no dylib in the package"; exit 1; } + +# ── 4. the producer's build tree is deleted, then the consumer runs ────── +# +# This is what makes the install-name assertion above load-bearing rather than +# decorative: after this `rm -rf`, an absolute install name names nothing. +rm -rf "$TMP/mathkit" + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { + cat app/run.log + echo "FAIL: the consumer could not build or run against the packaged dylib." + echo " 'image not found' naming a path under /private/var means the" + echo " install name was the build directory after all." + exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "FAIL: wrong answer"; exit 1; } + +echo "PASS: a Mach-O shared library relocates — @rpath install name, packed, run" diff --git a/tests/e2e/run_all.sh b/tests/e2e/run_all.sh index 8a7dad47..b9e186ae 100755 --- a/tests/e2e/run_all.sh +++ b/tests/e2e/run_all.sh @@ -95,6 +95,19 @@ case "$OS" in # proving nothing. The CI job that masks Visual Studio lands here. CAPS+=(no-msvc) fi + # mingw: the WINDOWS-HOSTED MinGW-w64 GCC payload (xim:mingw-gcc, + # winlibs GCC 16 UCRT). Distinct from `mingw-cross` above, which is the + # Linux-hosted cross — same target, different host, and a test that needs + # one cannot use the other. + # + # Probing the payload rather than PATH deliberately: a Windows runner may + # well have some g++.exe from Strawberry Perl, and that one cannot build + # modules. Same reason the `gcc` capability is withheld here. + for _mgw in "${MCPP_HOME:-$HOME/.mcpp}"/registry/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe \ + "$HOME"/.xlings/data/xpkgs/xim-x-mingw-gcc/*/bin/g++.exe; do + if [[ -x "$_mgw" ]]; then CAPS+=(mingw); break; fi + done + unset _mgw # NOTE: Windows runners may have g++.exe (MinGW/Strawberry) in PATH # but it's not a proper mcpp-compatible GCC. Don't add gcc capability. # fresh-sandbox: not yet reliable on Windows — xlings LLVM auto-install @@ -162,7 +175,7 @@ echo "Detected capabilities: ${CAPS[*]:-}" # absent on Linux and must stay legal to declare. It is checked against the # CAPS+=() calls above by tests/e2e/README or by reading them -- keep it in # sync when adding a capability. -KNOWN_CAPS=(elf fresh-sandbox gcc import-std-libcxx macos mingw-cross msvc +KNOWN_CAPS=(elf fresh-sandbox gcc import-std-libcxx macos mingw mingw-cross msvc musl nasm no-msvc pack patchelf python3 scan-deps symlink unix-shell windows wine xlings-msvc) From 5468cc311954a3192dd742dc9703c23c8e7da963 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 01:16:30 +0800 Subject: [PATCH 20/31] =?UTF-8?q?docs(design):=20=C2=A713=20=E2=80=94=20th?= =?UTF-8?q?e=20second=20round,=20and=20what=20it=20turned=20up?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the outcome of §12's five items under the single-PR decision (O1 not split, the rest done), plus the two real defects the work forced out: the kind="shared" guard was inert on native builds — the very case it was written for, since targetTriple is empty there — and `--target` accepted a target this host cannot produce, resolving the native g++ and delivering an ELF as a Windows build. Also the shape they share: mingw's ld tolerates linking a .dll directly, so "a PE shared library is two files" was never exposed. The tolerant case was hiding the broken one. --- .../2026-08-17-library-distribution-design.md | 125 ++++++++++++++++++ 1 file changed, 125 insertions(+) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index bb2d8897..92641fac 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -782,3 +782,128 @@ clang 构建的二进制,而 clang 构建的 mcpp 在本机会段错误)。 | **O3** | MSVC 的 `/REMOVE:` 路径**未测试**(mcpp 的 Windows CI 用 clang 的 llvm-ar,走 GNU 形式) | 失败会带命令原文报错,不会静默;要真测需要 `msvc@system` 的 job | | **O4** | Windows 上的胖包(msvc + mingw)未覆盖 | 见 §12.1;要在 Windows e2e job 装 `xim:mingw-gcc` | | **O5** | 设计 §2.2 承诺的「`--target` 集合与 `[package].platforms` 覆盖比对告警」**没有实现** | 我漏了。它便宜(一次集合比对 + 一条 warning),但属于「发布纪律」而不是正确性,可以随后补 | + + +--- + +## 13. 第二轮:把 §12 的五项做完(2026-08-18,同一个 PR #451) + +裁决是「统一在单 PR 451 上实现最大兼容和支持度」,所以 O1(拆 PR)**不做**, +其余四项全部落地,并顺带补上了两个在做的过程中被逼出来的真缺陷。 + +### 13.1 §12 五项的结局 + +| # | 项 | 结局 | +|---|---|---| +| O1 | 把扫描器改动拆成独立 PR | **不做** —— 单 PR 是裁决。代价照旧:revert 粒度与库分发绑在一起 | +| O2 | `scan_overrides` 声明的实现分区被误判成接口 | **已修,而且没加 manifest 键**(见 §13.2) | +| O3 | MSVC `/REMOVE:` 未测试 | **两端都测了**:`archive_remove_command` 抽出成缝 + 单测钉命令,e2e 255 用真 `lib.exe` | +| O4 | Windows 胖包未覆盖 | **e2e 256**(msvc 腿 + mingw 腿同处一包),Windows e2e job 增加 `xim:mingw-gcc` 安装步与 `mingw` 能力 | +| O5 | `[package].platforms` 覆盖告警未实现 | **已实现**(见 §13.3) | + +### 13.2 O2 的解法:三态,而且零新键 + +`SourceUnit::providesInterface` 从 `bool` 改成 `std::optional`。 +三条建图路径各说自己真的知道的事: + +| 路径 | 它知道什么 | +|---|---| +| 文本扫描器 | 读到了关键字 ⇒ 显式 true / false | +| P1689 读取器 | 编译器的答案,**包括它的沉默**(`is-interface` 是可选键) | +| `scan_overrides` | **什么都不知道** ⇒ 留空。schema 里没有地方说 export | + +**判据:`true` 是那个「不产生任何警告」的值。** 字段原来的注释把默认 `true` +叫「保守方向,因为这个标志只会产生一条警告」—— 说反了,所以用 +`[scan_overrides]` 声明的实现分区被一声不响地发布,而那正是整个闭包设计要防的事。 + +两个细节让告警可用: +- **只问分区。** `module M;` 不 provide 任何东西,所以能 provide 裸 `M` 的只有 + `export module M;` —— 在那里问会对每个用了 override 的包的每个主接口都告警; +- **未知与已知说不同的话。** 一条是「你正在发布你的实现」,另一条是 + 「mcpp 判不出你是不是在发布」。 + +**P1689 那一半是真修复而不是变通**:编译器早就报了 `is-interface`, +mcpp 解析进了一个从来没人读的字段。 + +e2e 253 **两侧都钉**:同一个 fixture 打两次(扫描 / override),两次必须给出 +**不同的句子** —— 只断言 override 会告警,分不清「mcpp 建模了三态」与 +「mcpp 对每个发布的分区都告警」。**控制探针验证过**:把 override 那一处恢复成 +`true`(其余修复保留),253 在正好那条断言上失败。 + +### 13.3 O5 的解法:判据是 `host_can_serve`,难点是「别嚷嚷」 + +四种比较只有两种能打印: + +| 情况 | 处理 | +|---|---| +| 打了却没声明 | 永远可行动 —— manifest 否认了一个包明明能服务的平台 | +| 声明了却没打,**且本宿主能构建它** | 可行动 | +| 声明了却没打,本宿主构建不了 | **不说话** | + +第三行才是这个检查可用的原因:正常流程是 CI 每平台各跑一次 `mcpp pack`, +Linux runner 不产 macOS 腿不是遗漏、是每一次。**永远触发的告警会把真正该看的那条盖掉。** +判据用 `host_can_serve` —— 与 `--target` 接受什么是同一个函数,所以告警只可能 +点出作者此刻做得到的事。e2e 254 **把沉默也断言了**。 + +### 13.4 顺带被逼出来的两个真缺陷 + +**(a) `kind = "shared"` 的守卫在最要紧的情形下失效。** +它写的是 `!targetTriple.empty() && os != "linux"`,而**原生构建的 targetTriple 是空的** +⇒ 它拦住了一个交叉到 macOS 的构建(那本来就不可服务、不可达), +却让**原生 macOS / 原生 Windows** 直接走进它本该拦住的未验证路径。 + +修完守卫之后,真正缺的东西各不相同,而且**都不是 flag 拼写**: + +| 格式 | 缺什么 | 现在 | +|---|---|---| +| PE | **导入库** —— 只写了 `.dll`,消费者直接链 `.dll`,mingw 的 ld 容忍、别人都不容忍 | 链接边把导入库声明成**隐式输出**;包里两个都带;`-static` 会让 ld 拒绝导入库(报 `have you installed the static version…`,既没点 DLL 也没点 `-static`),所以那条 `-l` 前先 `-Wl,-Bdynamic` | +| Mach-O | **install name** —— 只在声明了 `soname` 时才发,于是其他每个 `.dylib` 都把构建目录烙了进去 | 无条件 `@rpath/`;而且改成**按 target 判定**(原来是宿主的 `#if defined(__APPLE__)`) | +| PE/MSVC | **符号导出**(不是链接器 —— `link /DLL /IMPLIB:` 一直都在) | 仍然拒绝,但换成真理由,并点名 MinGW 这条路 | + +⚠️ **「能用的那种情况把坏掉的那种遮住了」是这一整块的形状。** +mingw 的 ld 容忍链 `.dll`,所以「PE 共享库是两个文件」这件事从来没被暴露。 + +**(b) `--target` 接受了这台机器产不出来的 target,并悄悄按宿主构建。** +实测(Linux): + +``` +$ mcpp build --target x86_64-windows-msvc + Resolved gcc@16.1.0 → x86_64-windows-msvc → …/xim-x-gcc/16.1.0/bin/g++ + Finished dev [unoptimized + debuginfo] in 0.07s +$ ls target/ + x86_64-linux-gnu/ ← 一个 ELF,被当成 Windows 构建交付 +``` + +**词表的 tier 说的是「mcpp 支持这个 target」,从来没说「这台机器能产出它」。** +`host_can_serve` 此前**只用于 `toolchain list` 的展示**,构建路径根本没问它。 +现在 `prepare.cppm` 在紧邻那条「typo 绝不能悄悄回落到宿主工具链(最坏的失败模式)」 +的检查之后问它,并把**这台宿主能构建的清单**列进错误里。逃生口保留: +显式 `[target.X] toolchain = "…"` 表示交叉链是作者自己提供的。 + +**(c) 消费一个 `shared` 的分发包直接失败**:`ninja: multiple rules generate +bin/libmathkit.dll` —— 依赖循环为一个**已经构建好**的库创建了 link unit。 +就算不撞名也是错的:分发包的 `sources` 只有它发布的接口,重链会静默产出一个 +**缺掉发布方保留的每个实现单元**的库。 + +### 13.5 覆盖矩阵(第二轮之后) + +| e2e | linux | macOS | windows | 备注 | +|---|---|---|---|---| +| 242/243/244/246/247/249/250/251/253/254 | ✅ | ✅ | ✅ | 无能力门 | +| 245 胖包(gnu+musl) | ✅ | 结构性不可能 | — | `# requires: gcc musl`(补上了 musl) | +| 256 胖包(msvc+mingw) | — | 结构性不可能 | ✅ | `# requires: msvc mingw`;job 里装 `xim:mingw-gcc` | +| 248 跨 OS 胖包(PE 腿) | ✅ | — | — | mingw job | +| 255 `lib.exe /REMOVE:` | — | — | ✅ | `# requires: msvc` | +| 257 PE 共享库(产出+打包+跑) | ✅ wine | — | — | `# requires: mingw-cross wine` | +| 258 MSVC 拒绝 `shared` | — | — | ✅ | `# requires: msvc`;在 Linux 上**看不到**(target 门先答) | +| 259 Mach-O 共享库可重定位 | — | ✅ | — | `# requires: macos`;**先删掉发布方的构建树**再消费 | + +**macOS 那两格是产品事实不是测试缺口**:它只能服务一个 target。 + +### 13.6 第二轮仍然开着的 + +| # | 项 | 说明 | +|---|---|---| +| **P1** | 打包库**不能被原生 `cl.exe` 消费** | 生成的 manifest 用 `-L`/`-l`(GNU 拼写)。**改成写文件路径也不行**:ninja 的链接命令 cwd 是输出目录,只有 include 家族前缀会被相对包根绝对化,无前缀 token 会去错地方找(`ld: cannot find lib/…/libmathkit.a`),而绝对路径就不再可重定位。要补齐需让条件通道承载 `link_library_dirs`/`libraries` —— mcpp 已能按方言渲染,只是只在顶层读 | +| **P2** | MSVC 的 `kind = "shared"` | 要生成 `.def`(对对象做符号扫描)—— 那是一个构建图节点,不是 flag | +| **P3** | 扫描器改动的 revert 粒度 | O1 的代价,按裁决接受 | From 0287b14cb42d27695a2a48d9c7384f0486620694 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 01:45:44 +0800 Subject: [PATCH 21/31] fix(e2e,backend): fixture path spelling, a check that could only fail, and a guard MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three things the local suite and a read-through caught: * 255/257/259 interpolated `$(host_path …)` straight into a manifest heredoc. 00_fixture_path_hygiene requires the value to come from a named `*_HOST` variable, and the rule is Windows': a shell-spelled /tmp path is read by a native mcpp.exe as "root of the current drive". * 259 tested `[[ -f "$pkg/lib/"*"/libmathkit.dylib" ]]`. `[[ ]]` does not path-expand, so that compares against a literal string containing an asterisk and is false for every real package — a check that can only fail, which on macOS-only coverage would have looked like a product bug. * the import-library flag substituted `{}` without checking it is there. Both dialect rows have one; a row without it would have thrown at `replace(npos)` instead of simply not emitting a flag. Also documents, in docs/12 (+ zh), how much of this is verified and where — including the row that is honest rather than flattering: the REAL old-client check has been run by hand and not by CI, because the boot entry each job bootstraps from is an xvm shim that answers "not installed" under the e2e environment. Local suite after this: 236 passed, 1 failed, 14 skipped. The remaining failure is 22_doctor_cache_publish, and the previous RELEASE binary fails it identically on this machine — a local environment fact, not a regression. --- docs/12-binary-distribution.md | 47 +++++++++++++++++++++ docs/zh/12-binary-distribution.md | 41 ++++++++++++++++++ src/build/ninja_backend.cppm | 13 ++++-- tests/e2e/255_pack_library_msvc_archiver.sh | 3 +- tests/e2e/257_shared_library_pe.sh | 3 +- tests/e2e/259_shared_library_macho.sh | 10 +++-- 6 files changed, 109 insertions(+), 8 deletions(-) diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index 18399708..7df4b1e8 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -333,3 +333,50 @@ warning: secret.cppm is an implementation partition, and the published interface > Build order was unconstrained: GCC and macOS clang recovered through their own > dependency scan, Windows clang failed with `failed to read compiled module`. > If you have been avoiding implementation partitions on Windows, that was why. + +### A partition mcpp cannot classify + +`[scan_overrides.""]` says which modules a file provides and has nowhere to +say whether the declaration carries `export`; a P1689 scanner may omit +`is-interface`. Either way the source is published — the consumer cannot build the +BMI without it — and `mcpp pack` says which of the two situations you are in: + +``` +warning: secret.cppm provides a module PARTITION and mcpp cannot tell which kind: + the unit is declared in `[scan_overrides]`, which has nowhere to say + whether the declaration carries `export`, … +``` + +> Until 2026.8.17.2 that arrived as "it is an interface" — the answer that +> produces **no** warning — so an implementation partition declared that way was +> published in silence. Publishing too few sources fails the consumer's compile +> and names the module; publishing too many ships private source and nothing +> fails at all. Unknown has to be loud. + +## How much of this is verified, and where + +The e2e suite gates each test on host capabilities, so "the suite is green" and +"this ran" are different statements. What runs where: + +| claim | linux | macOS | windows | +|---|---|---|---| +| layout, both interface modes, closure, the two gates, workspace root, named target, `sources = []`, bare-triple predicate | ✅ | ✅ | ✅ | +| fat package, two legs one artifact name (`gnu` + `musl`) | ✅ | *impossible* | — | +| fat package, two legs **two** artifact names (`msvc` + `mingw`) | — | *impossible* | ✅ | +| fat package crossing an OS boundary (a PE leg) | ✅ | — | — | +| `lib.exe /REMOVE:` really removing | — | — | ✅ | +| PE shared library: build, pack, link, run | ✅ (wine) | — | — | +| Mach-O shared library relocating out of its build tree | — | ✅ | — | +| MSVC refusing `kind = "shared"` for the export reason | — | — | ✅ | +| a released mcpp consuming a package this one produced | local only | local only | local only | + +*impossible* is not a gap: a macOS host can serve exactly one target +(`host_can_serve`, `registry.cppm`), so a package with two legs cannot be produced +there at all. + +The last row is honest about a real hole: each CI job bootstraps from a released +mcpp, but that entry is an xvm **shim**, and under the e2e suite's environment it +answers "not installed". So the static half of the old-client check (the generated +manifest uses no section a previous mcpp cannot read) runs everywhere, and the +real half — build against the package with the previous release — has been run by +hand, not by CI. diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index 3254a38e..7d54e7d2 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -301,3 +301,44 @@ warning: secret.cppm is an implementation partition, and the published interface > 的单元」到「定义分区的单元」的边,构建顺序无约束:GCC 与 macOS clang 靠各自的 > 依赖扫描兜住了,**Windows clang 以 `failed to read compiled module` 失败**。 > 如果你一直在 Windows 上回避实现分区,原因就是这个。 + +### mcpp 判不出类别的分区 + +`[scan_overrides.""]` 说的是文件提供哪些模块,**没有地方能说**那条声明是否 +带 `export`;P1689 扫描器也可能省略 `is-interface`。两种情况下源码都会被发布 —— +消费者没有它就编不出 BMI —— 而 `mcpp pack` 会告诉你你处在哪一种: + +``` +warning: secret.cppm provides a module PARTITION and mcpp cannot tell which kind: + the unit is declared in `[scan_overrides]`, which has nowhere to say + whether the declaration carries `export`, … +``` + +> 在 2026.8.17.2 之前,这种情况以「它是接口」到达 —— 那个**不产生任何警告**的答案 —— +> 于是这样声明的实现分区被一声不响地发布了。**发布得太少**会让消费者编译失败并点名 +> 模块;**发布得太多**会把私有源码发出去,而什么都不会失败。未知必须出声。 + +## 这些说法验证到哪一步、在哪台机器上 + +e2e 套件按宿主能力给每条测试开门,所以「套件是绿的」和「这条跑了」是两句不同的话。 +实际跑在哪里: + +| 说法 | linux | macOS | windows | +|---|---|---|---| +| 布局、两种接口模式、闭包、两道闸门、workspace 根、指名 target、`sources = []`、裸三元组谓词 | ✅ | ✅ | ✅ | +| 胖包,两条腿**同一个**产物名(`gnu` + `musl`) | ✅ | *不可能* | — | +| 胖包,两条腿**两个**产物名(`msvc` + `mingw`) | — | *不可能* | ✅ | +| 跨 OS 边界的胖包(一条 PE 腿) | ✅ | — | — | +| `lib.exe /REMOVE:` 真的删掉了 | — | — | ✅ | +| PE 共享库:产出、打包、链接、运行 | ✅(wine) | — | — | +| Mach-O 共享库离开构建树仍可加载 | — | ✅ | — | +| MSVC 以「导出」为理由拒绝 `kind = "shared"` | — | — | ✅ | +| 已发布的 mcpp 消费本版产出的包 | 仅本机 | 仅本机 | 仅本机 | + +*不可能* 不是缺口:macOS 宿主只能服务一个 target(`host_can_serve`, +`registry.cppm`),那里根本产不出两条腿的包。 + +最后一行如实记录一个真的洞:每个 CI job 都从一份已发布的 mcpp 自举,但那个入口是 +xvm 的 **shim**,在 e2e 套件改过的环境里它回答「未安装」。所以老客户端检查的 +**静态那半**(生成的 manifest 不含任何旧 mcpp 读不了的段)到处都跑,而**真实那半** —— +用上一版发布的 mcpp 去构建这个包 —— 是**手工跑的,不是 CI 跑的**。 diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 1c5151b7..930f0f33 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1826,10 +1826,17 @@ std::string emit_ninja_string(const BuildPlan& plan) { // agrees with — the name belongs to plan.cppm's import_library_for, and // this is how it gets to the command. The SPELLING belongs to the // dialect table, same as `archiveRemoveArg`. - if (!lu.importLibrary.empty() && !dial.sharedImportLibArg.empty()) { + if (!lu.importLibrary.empty()) { std::string arg{ dial.sharedImportLibArg }; - arg.replace(arg.find("{}"), 2, escape_ninja_path(lu.importLibrary)); - out_line += " implib_flag = " + arg + "\n"; + // `{}` or nothing: a row without the placeholder cannot say WHERE to + // write, so emitting its bare text would hand the linker a flag with + // no argument. Skipping is the honest reading of an empty row, and + // the implicit output above then fails loudly as a missing file + // rather than quietly linking against a stale one. + if (auto at = arg.find("{}"); at != std::string::npos) { + arg.replace(at, 2, escape_ninja_path(lu.importLibrary)); + out_line += " implib_flag = " + arg + "\n"; + } } { // Per-unit C++ runtime link, by ROLE. The kind→role map is the diff --git a/tests/e2e/255_pack_library_msvc_archiver.sh b/tests/e2e/255_pack_library_msvc_archiver.sh index eb4f7ae3..92565cf8 100755 --- a/tests/e2e/255_pack_library_msvc_archiver.sh +++ b/tests/e2e/255_pack_library_msvc_archiver.sh @@ -119,12 +119,13 @@ EOF # outright. That is a real limitation of the generated manifest (recorded in # docs/12), and it is NOT what this test is about — pinning cl here would make # 255 fail for a reason that has nothing to do with the archiver. +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" cat > app/mcpp.toml < app/src/main.cpp <<'EOF' import mathkit; int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } EOF +PKG_HOST="$(host_path "$pkg")" cat > app/mcpp.toml </dev/null || { - find "$pkg" -type f -o -type l +# `find`, not `[[ -f "$pkg/lib/"*"/libmathkit.dylib" ]]`: inside `[[ ]]` the `*` +# is not path-expanded, so that form tests a literal string containing an +# asterisk and is false for every real package — a check that can only fail. +[[ -n "$(find "$pkg/lib" -name 'libmathkit.dylib' | head -1)" ]] || { + find "$pkg" \( -type f -o -type l \) echo "FAIL: no dylib in the package"; exit 1; } # ── 4. the producer's build tree is deleted, then the consumer runs ────── @@ -97,12 +100,13 @@ cat > app/src/main.cpp <<'EOF' import mathkit; int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } EOF +PKG_HOST="$(host_path "$pkg")" cat > app/mcpp.toml < Date: Tue, 18 Aug 2026 01:48:55 +0800 Subject: [PATCH 22/31] docs: P1 is a constraint conflict, not a backlog item MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measured rather than assumed: a previous mcpp reads `[target.'cfg(…)'.runtime]` without complaint and silently ignores it. So the dialect-neutral link channel IS reachable per leg — and moving a leg's flags there would leave every older client with no link flags at all. Closing the native-cl.exe gap therefore means carrying both spellings (linking the library twice on new clients) or giving these packages a version floor; it is not a matter of effort, it conflicts with the two constraints the design is built on. --- docs/12-binary-distribution.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index 7df4b1e8..d0646ec1 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -310,9 +310,17 @@ every driver takes, and it does not work either: ninja runs link commands with cwd = the output directory, and only the include-family prefixes (`-I`, `-L`, …) are absolutized against the package root, so a prefix-less token is looked for in the wrong place — `ld: cannot find lib/x86_64-windows-gnu/libmathkit.a`. A -manifest cannot carry an absolute path and stay relocatable. Closing this needs -the conditional channel to carry `link_library_dirs` / `libraries`, which mcpp -already renders per dialect, but only reads at the top level. +manifest cannot carry an absolute path and stay relocatable. + +The dialect-neutral channel does exist — `[runtime] link_library_dirs` and +`libraries`, which mcpp renders as `/LIBPATH:` + `name.lib` or `-L` + `-lname` +depending on the target — but only at the top level, and a package needs it +**per leg**. Measured: a previous mcpp reads `[target.'cfg(…)'.runtime]` without +complaint and silently ignores it. That is the wrong kind of tolerance for this +purpose — moving a leg's link flags there would leave every older client with no +link flags at all, so closing this properly means either carrying both spellings +(and linking the library twice on new clients) or giving these packages a version +floor. ### Implementation partitions From 1f60c9544d49ff55c2bfbd5e0faaccedfccb07e2 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 01:49:11 +0800 Subject: [PATCH 23/31] =?UTF-8?q?docs(design):=20P1=20row=20=E2=80=94=20th?= =?UTF-8?q?e=20measured=20reason,=20not=20the=20effort=20estimate?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .agents/docs/2026-08-17-library-distribution-design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index 92641fac..fed839fc 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -904,6 +904,6 @@ bin/libmathkit.dll` —— 依赖循环为一个**已经构建好**的库创建 | # | 项 | 说明 | |---|---|---| -| **P1** | 打包库**不能被原生 `cl.exe` 消费** | 生成的 manifest 用 `-L`/`-l`(GNU 拼写)。**改成写文件路径也不行**:ninja 的链接命令 cwd 是输出目录,只有 include 家族前缀会被相对包根绝对化,无前缀 token 会去错地方找(`ld: cannot find lib/…/libmathkit.a`),而绝对路径就不再可重定位。要补齐需让条件通道承载 `link_library_dirs`/`libraries` —— mcpp 已能按方言渲染,只是只在顶层读 | +| **P1** | 打包库**不能被原生 `cl.exe` 消费** | 生成的 manifest 用 `-L`/`-l`(GNU 拼写)。**改成写文件路径也不行**:ninja 的链接命令 cwd 是输出目录,只有 include 家族前缀会被相对包根绝对化,无前缀 token 会去错地方找(`ld: cannot find lib/…/libmathkit.a`),而绝对路径就不再可重定位。**实测**:旧版 mcpp 读到 `[target.'cfg(…)'.runtime]` **不报错、静默忽略** ⇒ 把某条腿的链接 flag 挪到方言中立通道,会让**旧客户端一个 flag 都拿不到**。所以补齐它要么**两种拼写都带**(新客户端把库链两遍),要么给这类包**版本下限** —— 不是工作量问题,是与「零新键 + 老客户端可用」这两条约束直接冲突 | | **P2** | MSVC 的 `kind = "shared"` | 要生成 `.def`(对对象做符号扫描)—— 那是一个构建图节点,不是 flag | | **P3** | 扫描器改动的 revert 粒度 | O1 的代价,按裁决接受 | From 074661f146f34924633ddfd760245ad4f357291f Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 02:09:47 +0800 Subject: [PATCH 24/31] fix(pack): tar on Windows, and three e2e that outlived their premises MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI on the three platforms answered what no Linux run could, and all three failures were worth having. **251 was asserting a product limit that no longer exists.** It branched on `uname`: Linux produces a shared package, everything else must REFUSE. Mach-O and PE/MinGW now produce one, so the branch is by TARGET rather than by host — Windows' default toolchain is clang on the MSVC ABI, so that is where the refusal is asserted (and it now demands the `dllexport` reason and the MinGW way forward, not just "shared libraries are not supported"). **255 could not have a consumer step, and finding out why was the point.** The package is built with msvc@system, so its tag is `…-msvc19-…`; a consumer on the default toolchain is refused by the ABI gate, correctly and with the right message. A consumer that pins msvc@system instead cannot link it either, because the emitted manifest spells `-L`/`-l` and native cl.exe rejects `-L`. So the only consumer that test could have is one whose failure says nothing about the archiver — and pack succeeding is already the assertion, since a wrong /REMOVE: makes run_library_pack refuse with lib.exe's own output. **256 found a real bug: `mcpp pack` could not write a tarball on Windows.** Both legs packed — `mathkit.lib` beside `libmathkit.a`, which is the whole point of that test — and then the archive step died with tar (child): Cannot connect to C: resolve failed GNU tar reads `C:/path/x.tar.gz` as the rsh form `host:path` and goes looking for a machine called `C`; the message names neither its argument nor the drive letter. Unreachable until now because a single PE leg is written as a zip, so no Windows host had ever taken the tar path. Fixed with `--force-local` on Windows, in the library packer and in pack.cppm's make_tarball — the second is latent rather than observed, but it is the same command with the same hazard and the two should not differ in whether they survive being run there. --- src/pack/library.cppm | 12 ++++- src/pack/pack.cppm | 9 +++- tests/e2e/251_pack_library_shared.sh | 49 ++++++++++++++++----- tests/e2e/255_pack_library_msvc_archiver.sh | 47 ++++++++------------ 4 files changed, 75 insertions(+), 42 deletions(-) diff --git a/src/pack/library.cppm b/src/pack/library.cppm index 956d6ba0..a4b85812 100644 --- a/src/pack/library.cppm +++ b/src/pack/library.cppm @@ -390,7 +390,17 @@ run_library_pack(const LibraryPackPlan& plan) if (auto r = zip::write(plan.archivePath, entries); !r) return std::unexpected(LibraryPackError{ r.error() }); } else { - auto cmd = std::format("tar -czf {} -C {} {}", + // `--force-local` on Windows, and it is not optional there: GNU tar + // reads `C:/path/x.tar.gz` as the rsh form `host:path`, tries to resolve + // a machine called `C`, and fails with + // tar (child): Cannot connect to C: resolve failed + // which names neither tar's argument nor the drive letter as the cause. + // + // Reached only by a package with more than one leg: a single PE leg is + // written as a zip, so a Windows host never ran this path until a fat + // package existed. Same fix, same reason, in pack.cppm's make_tarball. + auto cmd = std::format("tar {}-czf {} -C {} {}", + mcpp::platform::is_windows ? "--force-local " : "", mcpp::platform::shell::quote(plan.archivePath.string()), mcpp::platform::shell::quote(plan.stagingRoot.parent_path().string()), mcpp::platform::shell::quote(plan.stagingRoot.filename().string())); diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 765558d8..bd20e0a5 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -787,8 +787,15 @@ make_tarball(const std::filesystem::path& stagingRoot, // tarball stem (computed by make_plan via wrapper_dirname_from_tarball). // This keeps click-to-extract and `tar -xzf` aligned: both surface a // single self-contained directory in the user's cwd. + // `--force-local` on Windows: GNU tar reads `C:/path/x.tar.gz` as the rsh + // form `host:path` and dies with `Cannot connect to C: resolve failed`, + // naming neither its argument nor the drive letter. Latent here rather than + // observed — a PE target is written as a zip — but it is the same command + // with the same hazard as the library packer's, and the two should not + // differ in whether they survive being run on Windows. auto cmd = std::format( - "tar -czf '{}' -C '{}' '{}'", + "tar {}-czf '{}' -C '{}' '{}'", + mcpp::platform::is_windows ? "--force-local " : "", tarballPath.string(), stagingRoot.parent_path().string(), stagingRoot.filename().string()); diff --git a/tests/e2e/251_pack_library_shared.sh b/tests/e2e/251_pack_library_shared.sh index 43c94373..6eb0587d 100755 --- a/tests/e2e/251_pack_library_shared.sh +++ b/tests/e2e/251_pack_library_shared.sh @@ -4,10 +4,20 @@ # CORRECTLY or refused CLEARLY. Never silently wrong. # # Both halves, because testing only the working one cannot tell "the gate is -# handled" from "there is no gate". `kind = "shared"` is ELF-only today -# (src/build/plan.cppm refuses it and says why), so on macOS and Windows the -# assertion is that the refusal happens and names the reason — a `# requires: -# elf` here would have left that side unobserved. +# handled" from "there is no gate". Which half applies is now a property of the +# TARGET, not of "is it Linux": +# +# ELF, Mach-O, PE/MinGW produced — the package carries every name the +# platform needs to link it and to find it later +# PE/MSVC refused — MSVC exports nothing from a DLL without +# `__declspec(dllexport)`, so the import library would +# be empty and consumers would fail with unresolved +# externals for symbols visibly present in the objects +# +# Windows' default toolchain here is clang on the MSVC ABI, so this file sees the +# refusal there. 258 pins that refusal in detail, 259 pins Mach-O relocatability, +# 257 pins the PE/MinGW path; what this one adds is that the two outcomes are +# reachable from the same fixture and the same command. # # A shared library is LINKED by `lib.so` and FOUND at run time by its # SONAME, and those are two different filenames. The first version of this @@ -48,22 +58,37 @@ EOF cd mathkit -# ── the non-ELF side: refuse, and say why ────────────────────────────── -if [[ "$(uname -s)" != "Linux" ]]; then +# ── Windows: the MSVC ABI is refused, and says what to do instead ────── +if [[ "$(uname -s)" != "Linux" && "$(uname -s)" != "Darwin" ]]; then if "$MCPP" pack mathkit-shared > refuse.log 2>&1; then cat refuse.log - echo "FAIL: a shared library was packed on a platform that cannot link one" + echo "FAIL: a shared library was packed for the MSVC ABI. Its import" + echo " library has no exports, so consumers fail with unresolved" + echo " externals naming symbols that are in the objects." exit 1 fi - grep -qi 'shared librar' refuse.log || { + grep -qi 'dllexport' refuse.log || { cat refuse.log - echo "FAIL: it refused, but the message does not say the artifact kind is the problem" + echo "FAIL: it refused, but not for the export reason — 'shared libraries" + echo " are not supported here' does not tell the reader what to change." exit 1; } - grep -qi 'linux\|elf' refuse.log || { + grep -qi 'windows-gnu' refuse.log || { cat refuse.log - echo "FAIL: the refusal does not say where shared libraries DO work" + echo "FAIL: the refusal names no way forward. MinGW auto-exports." exit 1; } - echo "PASS: a shared library package is refused, with the reason, off ELF" + echo "PASS: a shared library package is refused on the MSVC ABI, with the reason" + exit 0 +fi + +# ── macOS: it is produced, and the deep claims live in 259 ────────────── +if [[ "$(uname -s)" == "Darwin" ]]; then + "$MCPP" pack mathkit-shared > pack.log 2>&1 \ + || { cat pack.log; echo "FAIL: shared pack failed on Mach-O"; exit 1; } + macpkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" + [[ -n "$(find "$macpkg" -name 'libmathkit-shared.dylib' | head -1)" ]] || { + find "$macpkg" \( -type f -o -type l \) + echo "FAIL: no .dylib in the package"; exit 1; } + echo "PASS: a shared library package is produced on Mach-O" exit 0 fi diff --git a/tests/e2e/255_pack_library_msvc_archiver.sh b/tests/e2e/255_pack_library_msvc_archiver.sh index 92565cf8..87e9339b 100755 --- a/tests/e2e/255_pack_library_msvc_archiver.sh +++ b/tests/e2e/255_pack_library_msvc_archiver.sh @@ -105,32 +105,23 @@ if command -v lib &>/dev/null && members="$(lib /nologo /LIST "$(host_path "$arc echo " (verified against lib /LIST)" fi -# The end-to-end criterion: a consumer builds and runs against it. -cd "$TMP" -mkdir -p app/src -cat > app/src/main.cpp <<'EOF' -#include -import mathkit; -int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } -EOF -# ⚠️ The CONSUMER deliberately takes the DEFAULT toolchain (clang on the MSVC -# ABI), not msvc@system. The emitted manifest links each leg with `-Llib/ -# -l`, which is GNU spelling: clang accepts it, native cl.exe rejects `-L` -# outright. That is a real limitation of the generated manifest (recorded in -# docs/12), and it is NOT what this test is about — pinning cl here would make -# 255 fail for a reason that has nothing to do with the archiver. -PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" -cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed"; exit 1; } -grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } +# ⚠️ NO CONSUMER STEP, and the reason is a measured one rather than a +# simplification. The package is built with msvc@system, so its tag is +# `…-msvc19-…`; the consumer would have to use the same toolchain or the ABI gate +# refuses it — correctly: +# +# error: no prebuilt artifact matches this toolchain. +# your toolchain : x86_64-windows-msvc-clang20-msvcstl20-c++23 +# published tags : x86_64-windows-msvc-msvc19-unknownstl0-c++23 +# +# And a consumer that DOES pin msvc@system cannot link the package either: the +# emitted manifest spells its link flags `-Llib/ -l`, and native +# cl.exe rejects `-L` (docs/12 records that limit). So the only consumer this test +# could have is one whose failure says nothing about the archiver. +# +# That is fine, because pack succeeding IS the assertion: a wrong /REMOVE: +# spelling makes run_library_pack refuse, quoting the command and lib.exe's own +# output. 242 covers consumption on Windows, with the default toolchain on both +# sides. -echo "PASS: lib.exe /REMOVE: really removes, and the package links under MSVC" +echo "PASS: lib.exe /REMOVE: really removes, and the package is well-formed" From 3d104caeff3fd578ecf438dc10afdbdc9c0d32d6 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 02:31:11 +0800 Subject: [PATCH 25/31] fix(e2e): 256 looked for `env=`, and the emitted spelling is `env = "msvc"` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tar fix landed — both legs packed and the archive was written, which is what that test exists to show: `mathkit.lib` beside `libmathkit.a` in one Windows package. What failed after it was the predicate assertion, against a package that was perfectly correct: the generated manifest writes [target.'cfg(all(arch = "x86_64", os = "windows", env = "msvc"))'.build] and the grep looked for `env=`, which appears nowhere in the file. Matched with optional whitespace now. A grep that can only fail is worth more attention than one that passes, because it reads as a product bug on the one platform where nothing else could confirm it. Audited the other emitted-manifest assertions in the suite for the same shape; the rest either allow spacing (`role *= *`) or match structural text. --- tests/e2e/256_pack_library_fat_windows.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/tests/e2e/256_pack_library_fat_windows.sh b/tests/e2e/256_pack_library_fat_windows.sh index 8eaef932..e799d8fe 100755 --- a/tests/e2e/256_pack_library_fat_windows.sh +++ b/tests/e2e/256_pack_library_fat_windows.sh @@ -90,9 +90,12 @@ grep -qE "^\[target\.'$ARCH-" "$pkg/mcpp.toml" && { cat "$pkg/mcpp.toml" echo "FAIL: a leg is selected by a BARE TRIPLE, which is inert on a native build" exit 1; } -# env= must appear: `os=windows` alone cannot separate these two legs, and a -# predicate that cannot separate them would hand MSVC consumers the MinGW archive. -grep -q "env=" "$pkg/mcpp.toml" || { +# An env axis must appear: `os = "windows"` alone cannot separate these two legs, +# and a predicate that cannot separate them would hand MSVC consumers the MinGW +# archive. Matched with optional spaces — the emitted spelling is `env = "msvc"`, +# and the first version of this grep looked for `env=`, which is nowhere in the +# file and failed against a package that was perfectly correct. +grep -qE 'env[[:space:]]*=' "$pkg/mcpp.toml" || { cat "$pkg/mcpp.toml" echo "FAIL: the predicates carry no env axis, so both legs match both targets" exit 1; } From 71d815fa1e69d0d93aea904c1d370c4b632e8801 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 02:53:24 +0800 Subject: [PATCH 26/31] fix(e2e): the leg-selection grep assumed forward slashes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 256's `check` extracted the selected leg with `grep -o "dist/mathkit-0.1.0/lib/[A-Za-z0-9_-]*"`. A native mcpp.exe writes NATIVE separators into build.ninja, so on Windows that matched nothing at all and the test reported native saw more than one leg: (empty) — i.e. a count of zero, printed as "more than one", over an empty list. It reads like a packaging bug on the one platform where nothing else could confirm it, and it is a grep bug. Anchored on the package name instead, with `[\\/]` for either separator, and the failure now prints the count and the package's own lines from build.ninja so the next reader is not guessing. 245 carries the same helper — it only runs on Linux, but leaving the fragile version there invites the next test to copy it — and passes locally with it, which is what validates the pattern on `/`. --- .../e2e/245_pack_library_fat_target_selection.sh | 12 +++++++++--- tests/e2e/256_pack_library_fat_windows.sh | 16 +++++++++++++--- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/tests/e2e/245_pack_library_fat_target_selection.sh b/tests/e2e/245_pack_library_fat_target_selection.sh index 6426a1b0..b8a26240 100755 --- a/tests/e2e/245_pack_library_fat_target_selection.sh +++ b/tests/e2e/245_pack_library_fat_target_selection.sh @@ -101,10 +101,16 @@ check() { # $1 = label, $2 = expected leg dir, $3.. = build args ( cd app && "$MCPP" build "$@" > "$TMP/$label.log" 2>&1 ) \ || { cat "$TMP/$label.log"; echo "$label build failed"; exit 1; } local nj; nj="$(find app/target -name build.ninja | head -1)" - grep -o "dist/mathkit-0.1.0/lib/[a-z0-9_-]*" "$nj" | sort -u > "$TMP/$label.legs" + # Separator-agnostic and anchored on the PACKAGE name: a native mcpp.exe + # writes native separators into build.ninja, so a `dist/…/lib/…` pattern + # matches nothing there and the failure reads like a packaging bug. Same + # helper as 256, which is where that was measured. + grep -oE "mathkit-0\.1\.0[\\/]lib[\\/][A-Za-z0-9_-]+" "$nj" \ + | sed 's|.*[\\/]||' | sort -u > "$TMP/$label.legs" [[ "$(wc -l < "$TMP/$label.legs")" -eq 1 ]] || { - echo "$label saw more than one leg:"; cat "$TMP/$label.legs"; exit 1; } - grep -q "lib/$want\$" "$TMP/$label.legs" || { + echo "$label saw $(wc -l < "$TMP/$label.legs") leg(s), expected exactly 1:" + cat "$TMP/$label.legs"; exit 1; } + grep -qx "$want" "$TMP/$label.legs" || { echo "$label picked the wrong leg:"; cat "$TMP/$label.legs"; exit 1; } } diff --git a/tests/e2e/256_pack_library_fat_windows.sh b/tests/e2e/256_pack_library_fat_windows.sh index e799d8fe..510af576 100755 --- a/tests/e2e/256_pack_library_fat_windows.sh +++ b/tests/e2e/256_pack_library_fat_windows.sh @@ -125,10 +125,20 @@ check() { # $1 = label, $2 = expected leg dir, $3.. = build args ( cd app && "$MCPP" build "$@" > "$TMP/$label.log" 2>&1 ) \ || { cat "$TMP/$label.log"; echo "$label build failed"; exit 1; } local nj; nj="$(find app/target -name build.ninja | head -1)" - grep -o "dist/mathkit-0.1.0/lib/[A-Za-z0-9_-]*" "$nj" | sort -u > "$TMP/$label.legs" + # ⚠️ Separator-agnostic, and matched from the PACKAGE name rather than from + # `dist/`. A native mcpp.exe writes native separators into build.ninja, so + # the first version of this — `grep -o "dist/mathkit-0.1.0/lib/…"` — matched + # nothing at all on Windows and reported "saw more than one leg" over an + # empty list, which reads like a packaging bug and is a grep bug. + grep -oE "mathkit-0\.1\.0[\\/]lib[\\/][A-Za-z0-9_-]+" "$nj" \ + | sed 's|.*[\\/]||' | sort -u > "$TMP/$label.legs" [[ "$(wc -l < "$TMP/$label.legs")" -eq 1 ]] || { - echo "$label saw more than one leg:"; cat "$TMP/$label.legs"; exit 1; } - grep -q "lib/$want\$" "$TMP/$label.legs" || { + echo "$label saw $(wc -l < "$TMP/$label.legs") leg(s), expected exactly 1:" + cat "$TMP/$label.legs" + echo "--- lines mentioning the package ---" + grep -n 'mathkit' "$nj" | head -20 + exit 1; } + grep -qx "$want" "$TMP/$label.legs" || { echo "$label picked the wrong leg:"; cat "$TMP/$label.legs"; exit 1; } } From 299da536c17f5e28083897c40b4a8de8afe121fd Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 03:17:27 +0800 Subject: [PATCH 27/31] chore(xlings): pin to 2026.8.17.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pinned xlings is the one a release bundles at /registry/bin/xlings and the version `mcpp self env` reports; it had been sitting at 2026.8.11.2 while six releases went by. One edit in src/platform/xlings/xlings.cppm is the source of truth and the other sixteen follow it — `.github/tools/check_version_pins.sh` fails CI if any disagrees, which is why this is a mechanical sweep rather than a hunt. That checker exists because the hand-maintained list it replaced had already missed both composite actions, leaving CI's sandbox on 0.4.30 unnoticed. Verified after rebuild: `Updating vendored xlings 2026.8.11.2 -> 2026.8.17.2 (pinned 2026.8.17.2)`, and `mcpp self env` reports the new pin. --- .github/actions/bootstrap-mcpp/action.yml | 2 +- .github/actions/setup-macos-llvm/action.yml | 2 +- .github/workflows/bootstrap-macos.yml | 2 +- .github/workflows/ci-fresh-install.yml | 6 +++--- .github/workflows/ci-linux-e2e.yml | 2 +- .github/workflows/cross-build-test.yml | 4 ++-- .github/workflows/release.yml | 14 +++++++------- src/platform/xlings/xlings.cppm | 2 +- 8 files changed, 17 insertions(+), 17 deletions(-) diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index acf20308..29de6310 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.8.11.2' + default: '2026.8.17.2' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index 66314a0a..3cfdd438 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.8.11.2' + default: '2026.8.17.2' runs: using: composite diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index d4d2d3d6..ecbfb38e 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index a84005e9..2995291c 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.11.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.17.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -293,7 +293,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.11.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.17.2 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -364,7 +364,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.11.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.17.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 0f167ee6..d71db79c 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -138,7 +138,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.11.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.17.2 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 3468bd36..1be786e9 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -122,7 +122,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -260,7 +260,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 088fe558..ae5ad2a3 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -289,7 +289,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -360,7 +360,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.8.11.2-linux-aarch64.tar.gz" + XLA="xlings-2026.8.17.2-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -369,9 +369,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.8.11.2/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.8.17.2/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.8.11.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.8.17.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -449,7 +449,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -632,7 +632,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.11.2' + XLINGS_VERSION: '2026.8.17.2' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/src/platform/xlings/xlings.cppm b/src/platform/xlings/xlings.cppm index 2ad39b97..4ee4d881 100644 --- a/src/platform/xlings/xlings.cppm +++ b/src/platform/xlings/xlings.cppm @@ -44,7 +44,7 @@ namespace pinned { // in lock-step by hand; that list was already missing both composite // actions, which is how CI's sandbox sat on 0.4.30 unnoticed while // everything else had moved on. Don't reintroduce a hand-maintained list. - inline constexpr std::string_view kXlingsVersion = "2026.8.11.2"; + inline constexpr std::string_view kXlingsVersion = "2026.8.17.2"; inline constexpr std::string_view kNasmVersion = "3.02"; } From dfa582266322ef07a88f15cab24a18c5072409ca Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 03:21:29 +0800 Subject: [PATCH 28/31] docs: correct a claim I made about the old shared-library guard MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Caught while reviewing this PR rather than by any test, and it is the kind of error worth writing down: I described the previous `kind = "shared"` refusal — `!targetTriple.empty() && os != "linux"` — as having a hole on NATIVE builds, reasoning that targetTriple would be empty there. It is not. `tc.targetTriple` is filled from the compiler's own `-dumpmachine` (detect.cppm:88); a plain `mcpp build` on this machine records `triple: x86_64-linux-gnu` in resolution.json, and `parse("x86_64-pc-windows-msvc")` skips the vendor segment and succeeds. So the old guard did refuse native macOS and native Windows, and this PR ADDS Mach-O and PE/MinGW support rather than closing a hole. 251 asserting the refusal on macOS before this PR — and failing after it — is the corroboration. The install-name and host-`#if` defects are still real, but they were latent under the old refusal rather than reachable: they become wrong the moment macOS is allowed, which is what this PR does. Corrected in the code comment, docs/08, docs/12's neighbours, the CHANGELOG, e2e 257's header, and the design record — which also keeps the criterion: "structurally possible to be empty" is not "empty at run time"; either read the run-time artifact or do not write it up as measured. --- .../2026-08-17-library-distribution-design.md | 18 +++++++++++------ CHANGELOG.md | 20 ++++++++++--------- docs/08-toolchain-internals.md | 15 ++++++++------ src/build/plan.cppm | 17 ++++++++++------ tests/e2e/257_shared_library_pe.sh | 11 +++++----- 5 files changed, 49 insertions(+), 32 deletions(-) diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md index fed839fc..94444bce 100644 --- a/.agents/docs/2026-08-17-library-distribution-design.md +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -847,17 +847,23 @@ Linux runner 不产 macOS 腿不是遗漏、是每一次。**永远触发的告 ### 13.4 顺带被逼出来的两个真缺陷 -**(a) `kind = "shared"` 的守卫在最要紧的情形下失效。** -它写的是 `!targetTriple.empty() && os != "linux"`,而**原生构建的 targetTriple 是空的** -⇒ 它拦住了一个交叉到 macOS 的构建(那本来就不可服务、不可达), -却让**原生 macOS / 原生 Windows** 直接走进它本该拦住的未验证路径。 +**(a) `kind = "shared"` 此前只在 ELF 上可用,其余一律拒绝 —— 这一条是能力增加。** -修完守卫之后,真正缺的东西各不相同,而且**都不是 flag 拼写**: +⚠️ **我在第一版提交信息里把这道守卫说成「在原生构建上失效」,那是错的, +review 时才自己抓出来。** 它写的是 `!targetTriple.empty() && os != "linux"`, +我据此推断「原生构建 targetTriple 为空 ⇒ 守卫不触发」。**实测否掉了这个推断**: +`tc.targetTriple` 由编译器的 `-dumpmachine` 填(`detect.cppm:88`),原生构建上非空 —— +`resolution.json` 里记的就是 `x86_64-linux-gnu`;而 +`parse("x86_64-pc-windows-msvc")` 会跳过 vendor 段并成功。所以原生 macOS / 原生 Windows +**本来就被拦住**,不存在那个洞。**判据:结构上「可能为空」不等于运行时真的为空 —— +要么读运行时产物(resolution.json),要么别把它写成实测。** + +放开之后,真正缺的东西各不相同,而且**都不是 flag 拼写**: | 格式 | 缺什么 | 现在 | |---|---|---| | PE | **导入库** —— 只写了 `.dll`,消费者直接链 `.dll`,mingw 的 ld 容忍、别人都不容忍 | 链接边把导入库声明成**隐式输出**;包里两个都带;`-static` 会让 ld 拒绝导入库(报 `have you installed the static version…`,既没点 DLL 也没点 `-static`),所以那条 `-l` 前先 `-Wl,-Bdynamic` | -| Mach-O | **install name** —— 只在声明了 `soname` 时才发,于是其他每个 `.dylib` 都把构建目录烙了进去 | 无条件 `@rpath/`;而且改成**按 target 判定**(原来是宿主的 `#if defined(__APPLE__)`) | +| Mach-O | **install name** —— 只在声明了 `soname` 时才发;**放开 macOS 的那一刻**,每个没写 soname 的 `.dylib` 都会把构建目录烙进去 | 无条件 `@rpath/`;而且改成**按 target 判定**(原来是宿主的 `#if defined(__APPLE__)` —— 在旧的拒绝之下不可达,放开之后会发错) | | PE/MSVC | **符号导出**(不是链接器 —— `link /DLL /IMPLIB:` 一直都在) | 仍然拒绝,但换成真理由,并点名 MinGW 这条路 | ⚠️ **「能用的那种情况把坏掉的那种遮住了」是这一整块的形状。** diff --git a/CHANGELOG.md b/CHANGELOG.md index 47c90fbd..7ec62201 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,10 +38,11 @@ - **`kind = "shared"` 不再只有 Linux:PE/MinGW 与 Mach-O 都能产、能打包、能跑。** - 过去这条路只在 ELF 上验证过,而那道「非 Linux 就拒绝」的守卫**恰好在最要紧的 - 情形下失效**:它写的是 `!targetTriple.empty() && os != "linux"`,而**原生构建的 - targetTriple 是空的** —— 于是它拦住了一个交叉到 macOS 的构建(那个本来就不可服务), - 却让**原生 macOS / 原生 Windows** 直接走进它本该拦住的未验证路径。 + 过去这条路只在 ELF 上验证过,其余一律拒绝。**这是能力的增加,不是修一个洞** —— + ⚠️ 早前的提交信息把那道守卫描述成「在原生构建上失效」,那是**错的**: + `tc.targetTriple` 由编译器的 `-dumpmachine` 填,原生构建上**非空** + (实测 `resolution.json` 记的是 `x86_64-linux-gnu`),所以原生 macOS / 原生 Windows + 本来就被它拦住。 真正缺的东西各不相同,而且都不是 flag 拼写: @@ -52,11 +53,12 @@ 另外 PE 可执行文件带 `-static`,而 `-static` 会让 ld 进入纯静态模式并拒绝导入库, 报的是 `have you installed the static version of the mathkit library?` —— 既没点 DLL 也没点 `-static`;所以那条 `-l` 之前要先 `-Wl,-Bdynamic`。 - - **Mach-O 缺 install name。** `.dylib` 记录的是**链接时的路径**,所以只在声明了 - `soname` 时才发 `-install_name` 意味着**其他每一个 `.dylib` 都把构建目录烙了进去** —— - 在打包机上完好,换个地方就 `image not found`。现在无条件发 `@rpath/`。 - 而且这个选择原先是用宿主的 `#if defined(__APPLE__)` 做的,交叉链接会发错(或不发); - 现在按 target 决定,和 `target_output` 早就做的一样。 + - **Mach-O 缺 install name。** `.dylib` 记录的是**链接时的路径**,而原先只在声明了 + `soname` 时才发 `-install_name` —— 一旦放开 macOS,**每个没写 soname 的 `.dylib` + 都会把构建目录烙进去**:在打包机上完好,换个地方就 `image not found`。 + 现在无条件发 `@rpath/`。这个选择原先还是用宿主的 `#if defined(__APPLE__)` + 做的(在旧的拒绝之下不可达,但放开之后就会发错),现在按 target 决定, + 和 `target_output` 早就做的一样。 - **PE/MSVC 仍然拒绝,但换了个理由,而且是真理由。** 不是链接器 —— `link /DLL /IMPLIB:` 一直都在规则表里。是**符号导出**:没有 `__declspec(dllexport)` 或 `.def`,MSVC 的 DLL 什么都不导出 ⇒ 导入库是空的 ⇒ 消费者拿到一堆 diff --git a/docs/08-toolchain-internals.md b/docs/08-toolchain-internals.md index 746729e4..63b5b2e3 100644 --- a/docs/08-toolchain-internals.md +++ b/docs/08-toolchain-internals.md @@ -569,12 +569,15 @@ itself: | PE / MinGW | `-Wl,--out-implib,` | the **import library**, `-Wl,-Bdynamic` first | | PE / MSVC | refused (no auto-export; see docs/12) | — | -Two of those are recent corrections. Mach-O's install name defaults to the path -the library was LINKED at, so emitting it only when a `soname` was declared left -every other `.dylib` recording a build directory — fine on the machine that built -it, `image not found` anywhere else. And the choice was made with `#if -defined(__APPLE__)` on the HOST, so a cross link emitted the wrong one (or none). -It is decided from the target now, like `target_output` already was. +Three of those rows are new: everything but ELF was refused before, native or +cross. Two details had to change for them to be usable rather than merely +allowed. Mach-O's install name defaults to the path the library was LINKED at, so +emitting it only when a `soname` was declared would have left every other +`.dylib` recording a build directory — fine on the machine that built it, `image +not found` anywhere else. And the choice was made with `#if defined(__APPLE__)` +on the HOST, which is right only by coincidence on a native macOS build and wrong +for any cross link; it is decided from the target now, like `target_output` +already was. An **unservable target is refused** rather than quietly built for the host: `--target x86_64-windows-msvc` on Linux used to resolve the native `g++`, write diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 00953c2f..e050da48 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -1017,12 +1017,17 @@ make_plan(const mcpp::manifest::Manifest& manifest, // Producing that is worse than refusing, because the diagnostic points // nowhere near the cause. // - // ⚠️ THE PREVIOUS GUARD HAD A HOLE, and it was in the case that matters - // most. It read `!targetTriple.empty() && os != "linux"`, and targetTriple - // is EMPTY for a native build — so it refused a cross build to macOS (which - // is unservable anyway, i.e. unreachable) while letting a NATIVE macOS or - // native Windows build walk straight into the unverified paths it was - // written to keep people out of. The resolved target is what decides. + // ⚠️ THE HOST FALLBACK BELOW IS BELT AND BRACES, NOT A FIX. An earlier + // version of this comment claimed the previous guard — + // `!targetTriple.empty() && os != "linux"` — was inert on native builds + // because targetTriple would be empty there. It is not: `tc.targetTriple` + // is filled from the compiler's own `-dumpmachine` (detect.cppm), a native + // Linux build records `x86_64-linux-gnu` in resolution.json, and + // `parse("x86_64-pc-windows-msvc")` skips the vendor segment and succeeds. + // So the old guard did refuse native macOS and native Windows, and this + // change ADDS support rather than closing a hole. The fallback stays + // because a target that cannot be parsed must not be silently read as + // "not windows". { const std::string targetOs = targetTriple.empty() ? (mcpp::platform::is_macos ? "macos" diff --git a/tests/e2e/257_shared_library_pe.sh b/tests/e2e/257_shared_library_pe.sh index 404168d8..85e13937 100755 --- a/tests/e2e/257_shared_library_pe.sh +++ b/tests/e2e/257_shared_library_pe.sh @@ -3,11 +3,12 @@ # 257_shared_library_pe.sh — `kind = "shared"` on PE: a DLL, its import library, # a package carrying both, and a consumer that links and RUNS. # -# Until now this was Linux-only, and the refusal that said so had a hole in the -# case that mattered: it read `!targetTriple.empty() && os != "linux"`, and -# targetTriple is EMPTY for a native build — so it turned away a cross build to -# macOS (unservable anyway) while letting a native macOS or native Windows build -# walk straight into the unverified paths it existed to prevent. +# Until now this was Linux-only: make_plan refused every non-Linux target, native +# or cross. So this test is a NEW CAPABILITY, not a repaired hole — an earlier +# draft of this header claimed the guard was inert on native builds because +# `targetTriple` would be empty there, and that is wrong: it is filled from the +# compiler's own -dumpmachine, and resolution.json records `x86_64-linux-gnu` for +# a plain `mcpp build`. # # What was actually missing on PE was the IMPORT LIBRARY. A PE shared library is # two files: the `.dll` the loader opens, and an archive of stubs the linker From b5be63984535d9911a9cbf679c8c72a66929eb51 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 03:25:48 +0800 Subject: [PATCH 29/31] polish(cli,pack): the help said half of what pack does, and two dead includes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mcpp pack` routes on `[targets.].kind`, so "bundle into a self-contained archive" describes only the program half. Both help lines now name both shapes — the one-line help is where a reader finds out that a library target produces something different, and it was the one place that did not say so. Also drops the global module fragments from mcpp.pack.route and mcpp.pack.library_pipeline: both carried `#include ` and neither uses it. --- src/cli.cppm | 9 +++++++-- src/pack/library_pipeline.cppm | 3 --- src/pack/route.cppm | 3 --- 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/src/cli.cppm b/src/cli.cppm index e202334b..9f98060e 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -63,7 +63,7 @@ void print_usage() { std::println(" mcpp update [pkg] Re-resolve deps and rewrite mcpp.lock"); std::println(" mcpp search Search packages in registries"); std::println(" mcpp publish [--dry-run] Publish package to default registry"); - std::println(" mcpp pack [--mode ] Build + bundle an archive (m: system|vendored|self-contained|static)"); + std::println(" mcpp pack [target] Build + package (program: bundle; library: interface + binaries)"); std::println(" mcpp emit xpkg [-V VER] [-o FILE] Generate xpkg Lua entry"); std::println(" mcpp xpkg parse [--json] Validate an xpkg descriptor (resolver grammar)"); std::println(""); @@ -405,7 +405,12 @@ int run(int argc, char** argv) { // than a plain directory" — WHICH archive follows the artifact, // because a .tar.gz full of DLLs is a package most Windows users // cannot open without installing something first. - .description("Build + bundle into a self-contained archive") + // Says both shapes, because `[targets.].kind` picks between them + // and the one-line help is where a reader finds that out. "Bundle + // into a self-contained archive" described only the program case, + // which is now half of what this command does. + .description("Build + package: a program becomes a self-contained " + "bundle, a library an interface + prebuilt binaries") // NB: a target NAME from [targets.*], not a triple — the same // split `mcpp run [target]` has. Its `kind` decides what is // packed, so there is no --lib and no --artifact: a program diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm index 595a7032..14d89e83 100644 --- a/src/pack/library_pipeline.cppm +++ b/src/pack/library_pipeline.cppm @@ -14,9 +14,6 @@ // // Design: .agents/docs/2026-08-17-library-distribution-design.md §2. -module; -#include - export module mcpp.pack.library_pipeline; import std; diff --git a/src/pack/route.cppm b/src/pack/route.cppm index 988e8770..6f704ba7 100644 --- a/src/pack/route.cppm +++ b/src/pack/route.cppm @@ -11,9 +11,6 @@ // decision ahead of the build: `mcpp pack nosuch` should say so in // milliseconds, not after compiling the project. -module; -#include - export module mcpp.pack.route; import std; From 3823fe5bdf55fbf5c02eae46ebefdf9d0e74879f Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 03:28:09 +0800 Subject: [PATCH 30/31] harden(manifest): the xpkg path fills `sources` and now says so MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `sourcesDeclared` distinguishes "no `sources` key" from `sources = []`, and the xpkg descriptor parser filled the list without setting it. Harmless today — that synthesiser refuses a descriptor with no sources and never calls apply_defaults_and_infer — but a false flag beside a non-empty list is a trap: the day that function runs on an xpkg manifest, its explicit list is replaced by the default glob and every descriptor-described package quietly compiles the wrong file set. --- src/manifest/xpkg.cppm | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/manifest/xpkg.cppm b/src/manifest/xpkg.cppm index edf227a2..13652b21 100644 --- a/src/manifest/xpkg.cppm +++ b/src/manifest/xpkg.cppm @@ -1227,6 +1227,13 @@ synthesize_from_xpkg_lua(std::string_view luaContent, if (!s.empty()) { m.modules.sources.push_back(s); m.buildConfig.sources.push_back(std::move(s)); // M5.0 mirror + // The descriptor said `sources`, so record that it did. + // Harmless today — this synthesiser never runs + // apply_defaults_and_infer, and it REFUSES a descriptor with + // no sources at all — but leaving the flag false here means + // an xpkg manifest that ever reached that function would + // have its explicit list replaced by the default glob. + m.buildConfig.sourcesDeclared = true; } cur.skip_ws_and_comments(); } From 9a4dec33a514e7b15bf1eefc78b4f52fab0471ac Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 18 Aug 2026 03:48:10 +0800 Subject: [PATCH 31/31] release: 2026.8.18.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The date-based convention labels a release by the day it goes out, and the previous one (2026.8.17.1) is already published — so this is the first release of the 18th, not a second one of the 17th. Two version sites move (mcpp.toml and src/version.cppm; fingerprint.cppm derives from the latter), the CHANGELOG's Unreleased section becomes the release heading, and the docs' "until mcpp " notes name the release that actually ships those changes. The `XLINGS_VERSION: '2026.8.17.2'` entries under .github/ are the xlings pin, which happens to carry the same digits and must not move with this. `check_version_pins.sh`: building 2026.8.18.1, bootstrapping from 2026.8.17.1, xlings pins all at 2026.8.17.2. --- CHANGELOG.md | 2 +- docs/05-mcpp-toml.md | 2 +- docs/12-binary-distribution.md | 6 +++--- docs/zh/12-binary-distribution.md | 6 +++--- mcpp.toml | 2 +- src/version.cppm | 2 +- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ec62201..14f64d80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 -## [Unreleased] +## [2026.8.18.1] — 2026-08-18 ### 新增 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index b5eb29a3..4f572b5e 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -156,7 +156,7 @@ the package/feature boundary, not on an individual target. > **`sources = []` is not the same as omitting `sources`.** An absent key > selects the default glob; an explicitly empty list means *compile nothing*, > which is what a header-only distribution package needs to say. Until -> mcpp 2026.8.17.2 the two were byte-identical, so there was no spelling for +> mcpp 2026.8.18.1 the two were byte-identical, so there was no spelling for > "nothing" and any file left under `src/` was swept in. ```toml diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md index d0646ec1..8502bd32 100644 --- a/docs/12-binary-distribution.md +++ b/docs/12-binary-distribution.md @@ -225,7 +225,7 @@ target is known, a fat package cross-compiles correctly with no index-side or installer-side support at all. > The blocks are `cfg(...)` and never a bare `[target.'']` key. Before -> mcpp 2026.8.17.2 the bare form was inert without an explicit `--target`, so a +> mcpp 2026.8.18.1 the bare form was inert without an explicit `--target`, so a > package using it would work in CI and silently drop its flags on a > developer's machine. mcpp generates the spelling that means the same thing on > every client. @@ -335,7 +335,7 @@ warning: secret.cppm is an implementation partition, and the published interface reaches it — so its SOURCE is being published. ``` -> Until mcpp 2026.8.17.2 the scanner recorded `module M:part;` as *requiring* +> Until mcpp 2026.8.18.1 the scanner recorded `module M:part;` as *requiring* > `M:part` and providing nothing, so a file required its own name and the graph > held no edge from the unit importing a partition to the unit defining it. > Build order was unconstrained: GCC and macOS clang recovered through their own @@ -355,7 +355,7 @@ warning: secret.cppm provides a module PARTITION and mcpp cannot tell which kind whether the declaration carries `export`, … ``` -> Until 2026.8.17.2 that arrived as "it is an interface" — the answer that +> Until 2026.8.18.1 that arrived as "it is an interface" — the answer that > produces **no** warning — so an implementation partition declared that way was > published in silence. Publishing too few sources fails the consumer's compile > and names the module; publishing too many ships private source and nothing diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md index 7d54e7d2..2454b578 100644 --- a/docs/zh/12-binary-distribution.md +++ b/docs/zh/12-binary-distribution.md @@ -205,7 +205,7 @@ ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] 胖包的交叉编译天然正确,**不需要索引侧或安装侧做任何支持**。 > 这些块是 `cfg(...)`,绝不是裸的 `[target.'<三元组>']` 键。 -> 在 mcpp 2026.8.17.2 之前,裸三元组在没有显式 `--target` 时是失效的 —— +> 在 mcpp 2026.8.18.1 之前,裸三元组在没有显式 `--target` 时是失效的 —— > 用它的包会在 CI 里正常、在开发者机器上静默丢掉 flag。 > mcpp 生成的是在**所有**客户端上含义一致的那种写法。 @@ -296,7 +296,7 @@ warning: secret.cppm is an implementation partition, and the published interface reaches it — so its SOURCE is being published. ``` -> 在 mcpp 2026.8.17.2 之前,扫描器把 `module M:part;` 记成**「requires `M:part`、 +> 在 mcpp 2026.8.18.1 之前,扫描器把 `module M:part;` 记成**「requires `M:part`、 > provides 空」** —— 一个文件 requires 自己的名字,于是图里**没有**从「import 分区 > 的单元」到「定义分区的单元」的边,构建顺序无约束:GCC 与 macOS clang 靠各自的 > 依赖扫描兜住了,**Windows clang 以 `failed to read compiled module` 失败**。 @@ -314,7 +314,7 @@ warning: secret.cppm provides a module PARTITION and mcpp cannot tell which kind whether the declaration carries `export`, … ``` -> 在 2026.8.17.2 之前,这种情况以「它是接口」到达 —— 那个**不产生任何警告**的答案 —— +> 在 2026.8.18.1 之前,这种情况以「它是接口」到达 —— 那个**不产生任何警告**的答案 —— > 于是这样声明的实现分区被一声不响地发布了。**发布得太少**会让消费者编译失败并点名 > 模块;**发布得太多**会把私有源码发出去,而什么都不会失败。未知必须出声。 diff --git a/mcpp.toml b/mcpp.toml index a97f59ff..4ee2c54c 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.17.2" +version = "2026.8.18.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/version.cppm b/src/version.cppm index a4be1f92..f7d3c7c4 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.17.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.18.1"; } // namespace mcpp