diff --git a/.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md b/.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md new file mode 100644 index 00000000..87629634 --- /dev/null +++ b/.agents/docs/2026-08-17-distribution-architecture-analysis-and-design.md @@ -0,0 +1,1615 @@ +# mcpp 分发架构:全面分析与方案(2026-08-17) + +> 覆盖:源码分发 / 二进制分发 / 多平台打包 / 私有分发,以及 issue #433 +> (「预编译 .so + .ixx/.h/.cppm 接口」)。 +> +> 前半是**发现**(现状是什么、证据在哪、file:line 与实测),后半是**方案** +> (改什么、怎么算通过)。第 7 节是需要 review 的决策点。 +> +> **⇒ 方案已独立成文:`2026-08-17-library-distribution-design.md`** +> (落地形态、docs 计划、examples、CI 验证矩阵、分期)。**本文是发现,那份是方案。** +> +> 关联 issue:#433(本文起因)、#290(描述符构建规则不能按版本区分)、 +> #304(`runtime.library_dirs` 同时落在链接线上)、#276(嵌入式 SDK 集成)、 +> #416(纯 C 包被链进 libstdc++)。 + +--- + +## 0. 一页纸结论 + +**五条结论,按重要性排序:** + +1. **#433 想要的东西,在 Linux 上今天就能跑通 —— 我实测跑通了。** + 一个「只有模块接口 `.cppm` + 预编译 `libfoo.so`」的包,通过 `[runtime]` + 的 `libraries` / `link_library_dirs` / `runtime_search_dirs` 被消费者 + `import` 并链接,`std::string` 和异常都能跨边界。**零引擎改动。** + 见 §2。 + +2. **但它「能跑」不等于「能分发」,而且失败是静默的。** 同一次实测暴露三件事: + - 产出的 `.so` 里烧着**生产者机器的绝对 RUNPATH**(`/home/speak/.mcpp/...`), + 换台机器就是另一回事 —— 而 `mcpp pack` 的 RUNPATH/INTERP 重写**只服务可执行文件,不服务库**; + - 进程里同时有**两份 C++ 运行时**(exe 静态 libstdc++ + `.so` 动态 + `libstdc++.so.6`),没有任何一层告警; + - **接口与二进制之间没有任何绑定**。我把随包发的 `.cppm` 里 + `struct Point { int x; int y; }` 改成 `{ int y; int x; }`(mangling 不变), + **编译通过、链接通过、运行通过、打印出交换后的错数据**。全程零诊断。 + +3. **真正的结构性缺口不在「能不能链」,在四处**: + ① 描述符**强制要求 `sources`**(`xpkg.cppm:2059`),所以「无源包」这个种类不存在 + —— 生态里现有的做法是造一个假的 anchor `.c`(见 `compat.openblas`); + ② `kind = "shared"` **只有 Linux/ELF**(`plan.cppm:1006` 直接拒绝), + 即 `.dll` / `.dylib` 这两条腿**在生产侧根本不存在**; + ③ **没有兼容标签**(wheel tag 那种东西)—— `abi.cppm` 有五维模型但太粗 + (无编译器版本、无 stdlib 版本、无标准档位); + ④ **安装线协议没有 target 轴**(`package_fetcher.cppm:419` 只发 + `{"targets":[...]}`),所以交叉编译一定拿到宿主 arch 的载荷。 + +4. **能力其实已经攒够了,缺的是把它们接起来。** + `cache_key.cppm` 已经在算一个覆盖 toolchain × 语言 × profile × 身份 × + 自身配置 × 上游 Merkle 的**逐包 ABI 完备键**,并且全局构建缓存 + (`$MCPP_HOME/build-cache/v1/pkg//@//`)里**已经躺着 + BMI + obj**。二进制分发格式 ≈ **可搬运的构建缓存条目**。 + xim 的 XPackage Spec V2 也**已经有一等的 arch 轴**(per-arch resource map / + URL 模板 / per-arch sha256)。 + +5. **最短路径不是索引,是文件。** issue 作者原话是「类似 `pip install runtime.whl`」。 + `mcpp add ./runtime-0.1.0-.mpkg` 不需要索引、不需要网络、不需要鉴权、 + **不需要 xlings 改一行**,完全在 mcpp 自己手里。**建议它做 P0**,索引通路做 P1。 + +7. **包自带一份 `mcpp.toml` —— 这不是新机制,是 mcpp 已有且文档推荐的 Form A** + (`prepare.cppm:2764`:描述符没有 `mcpp` 字段时,glob 载荷里的 `mcpp.toml`)。 + 它一次消掉三样东西:**新描述符键**(⇒ 不需要版本 floor ⇒ 老客户端不会被砖)、 + **新消费代码路径**、以及 —— 通过「胖包 + `[target.'cfg(...)']` 在消费者构建期 + 选 arch」—— **整个 G3**(描述符 arch 轴 + 安装线 target 轴 + 动 xlings)。 + 实测边界:`[distribution]` 标量段今天被静默接受;但 `artifacts = [{…}]` + **硬失败**,产物清单必须借用已在白名单里的 `[[runtime.artifacts]]`。 + +6. **P0/P1 按「形态」切,不按「平台」切**(§4.5.1 / §5)。`kind = "lib"` **没有平台 + 限制**,静态形态今天三平台就能产出,而且**消掉了整整四类问题**:RUNPATH 重写、 + DLL 部署、双 C++ 运行时、加载器搜索闭包 —— 上面第 2 条里的三处静默失败, + 静态形态天然只剩「接口↔二进制绑定」那一处。所以 + **P0 = static × 三平台,P1 = shared × 三平台**;按平台切会让 Windows/macOS + 在 P0 结束时拿到一个空盒子。 + +--- + +## 1. 今天的分发架构:五条通路 + +mcpp 今天有五条互相独立的「东西怎么到达另一台机器」的通路。它们从未被放在一张表里 +比较过,而 #433 的困惑正来自这里:作者看到的是通路 A 和通路 B,而他要的是通路 C, +通路 C 今天只以「手法」的形式存在,没有名字。 + +### 1.1 通路 A —— 源码分发(库) + +``` +你的仓库 → git tag → GitHub 自动生成 tag tarball + → gitcode 镜像(逐字节相同,CN 区) + → mcpplibs/mcpp-index 的 pkgs//.lua(GLOBAL + CN URL + sha256) + → publish-artifact.yml 推一个内容哈希 artifact + → 消费者 bump 版本 +``` +(`docs/10-publishing-a-library.md`) + +- 生产侧:`mcpp publish` → `git archive` 出 tarball + sha256 + 生成 `xpkg.lua` + (`publish/pipeline.cppm:69`),然后**人工**开 PR 到索引。 +- 描述符两种形态:**Form A**(tarball 里自带 `mcpp.toml`)与 **Form B** + (描述符内联 `mcpp = { ... }` 块,`xpkg.cppm:synthesize_from_xpkg_lua`)。 +- 关键事实:`make_release_info` 把 **linux / macosx / windows 三个平台块填成同一个 + URL**(`publisher.cppm:295-302`)。源码 tarball 在三个平台上是同一份字节 —— + **这条通路天生没有多平台问题,因为它根本不区分平台。** + +### 1.2 通路 B —— 应用二进制分发(`mcpp pack`) + +四个 mode(`docs/02-pack-and-release.md`):`system` / `vendored`(默认) / +`self-contained` / `static`。两条产出族:ELF/Mach-O → `.tar.gz`(`lib/` + 重写 +RUNPATH);PE → `.zip`(DLL 与 `.exe` 平铺,因为 PE 没有 rpath)。 + +- **它的核心价值不是打包,是重写**:每个 mode 都会重写 `PT_INTERP` 与 + `DT_RUNPATH`,因为开发构建产物寻址的是**这台机器**的载荷目录。e2e 215 会扫遍 + bundle 里每个 ELF,发现 `$MCPP_HOME` 下的路径就失败。 +- **它的边界:只服务可执行程序。** `mcpp pack` 的输入是 `builtBinary` + (`pack.cppm:Plan::builtBinary`),整条流水线围绕「一个 exe + 它的闭包」。 + **没有任何 `mcpp` 命令会对一个库做同样的重写。** 这是 §2.2 那条 RUNPATH + 发现的根源。 + +### 1.3 通路 C —— 预编译库的「事实上」通路(anchor TU 手法) + +今天生态里确实有预编译二进制在分发,但它没有名字,是一组手法。样板是 +`compat.openblas`(`~/.mcpp/registry/data/mcpplibs/pkgs/c/compat.openblas.lua`): + +| 需要表达的事 | 今天怎么写 | +|---|---| +| 「我没有源码要编」 | **造一个假的 `mcpp_openblas_anchor.c`**,因为 `sources` 是强制的 | +| 头文件 | `include_dirs = { "include" }` | +| 链接预编译库 | `ldflags = { "-Llib", "-lopenblas" }`(`-L` 被 mcpp 重写成 `/lib`) | +| Windows 导入库 vs 静态库 | **per-OS 块**,`windows = { ldflags = { "-Llib", "-llibopenblas" } }` | +| 运行期 DLL | `windows = { runtime = { library_dirs = { "bin" } } }` → 拷到 `.exe` 旁 | +| 平台差异的入口 | linux/macosx 由 `install()` 钩子**写出** anchor;windows 用 `generated_files` | + +这套能工作,但它是**反向表达**:包的意图是「不要编译我,链接我」,而写法是 +「编译一个什么都不做的 `.c`,顺便偷偷加几个链接 flag」。后果: + +- 「这个包是预编译的」这件事**不可查询** —— 没有字段、没有 lint、没有诊断; +- 没有任何**兼容性检查**:一个用 gcc 13 编的 `libopenblas.a` 和一个 gcc 16 的 + 消费者之间,mcpp 无话可说(C 库侥幸没事,C++ 库会炸); +- `runtime.library_dirs` 这个名字**同时管链接和运行**,这就是 **#304**。 + +### 1.4 通路 D —— 私有分发 + +今天三种形态,全部是**源码**: + +| 形态 | 写法 | 鉴权 | 状态 | +|---|---|---|---| +| path 依赖 | `foo = { path = "../foo" }` | 无需 | ✅ 可用 | +| git 依赖 | `foo = { git = "...", rev/tag/branch = "..." }` | **环境里的 git 凭据** | ✅ 可用 | +| 私有索引 | `[indices] acme = { url = "git@..." }` 或 `{ path = "/srv/index" }` | **环境里的 git 凭据** | ✅ 可用 | + +缺口: +- `IndexSpec`(`pm/index_spec.cppm:14`)**没有任何鉴权字段** —— 全靠 ambient + git credential helper / SSH key。私有 HTTPS + token 只能把 token 写进 URL。 +- `IndexSpec::artifact`(内容哈希 artifact 通道)是**为公开 GitHub Actions 设计的**, + 私有场景没有对应机制。 +- **没有「离线包」入口**:没有 `mcpp add ./something.mpkg`。这正是 issue 作者 + 要的形状。 + +### 1.5 通路 E —— 工具链与运行时载荷(xim / xlings 层) + +mcpp 自己的 payload(gcc/llvm/glibc/ninja/…)走 xim。这条通路**已经有一等 arch 轴**: +XPackage Spec V2 的 per-arch resource map / URL 模板 + per-arch sha256 / +`xpm.source`,并且是 fail-closed 的(`xim-pkgindex/docs/V2/xpackage-spec.md`)。 + +但它有一个对二进制分发致命的性质:**arch 在安装时按宿主解析** +(spec 原文:"arch is resolved per-host at install time")。而 mcpp 调用它时 +只发 `{"targets":[...],"yes":true}`(`package_fetcher.cppm:419-431`)—— +**没有 os、没有 arch、没有 triple**。交叉编译时,拿到的是宿主 arch 的载荷。 + +对源码包无害(源码 tarball 与 arch 无关)。**对二进制包是致命的。** + +### 1.6 能力矩阵 + +| | A 源码库 | B 应用 pack | C 预编译库(手法) | D 私有 | E 载荷 | +|---|---|---|---|---|---| +| Linux | ✅ | ✅ | ⚠️ 手法 | ✅ | ✅ | +| macOS | ✅ | ✅ | ⚠️ 手法 | ✅ | ✅ | +| Windows | ✅ | ✅ | ⚠️ 手法 | ✅ | ✅ | +| arch 轴 | n/a | ✅(`--target`) | ❌ 描述符载荷无 arch | n/a | ✅ V2 | +| 交叉编译 | ✅ | ✅ | ❌ 安装线无 target | ✅ | ⚠️ 按包名绕开 | +| 生产 `.so` | n/a | n/a | **仅 Linux** | n/a | n/a | +| 生产 `.dll`/`.dylib` | n/a | n/a | ❌ **不存在** | n/a | n/a | +| ABI 兼容检查 | n/a | n/a | ❌ | ❌ | ⚠️ 五维粗粒度 | +| 接口↔二进制绑定 | n/a | n/a | ❌ **实测静默错数据** | ❌ | n/a | +| 鉴权 | 公开 | n/a | n/a | ⚠️ ambient git | 公开 | +| 离线包安装 | ❌ | n/a | ❌ | ❌ | n/a | + +--- + +## 2. 实测:#433 要的东西,今天在 Linux 上已经跑通 + +复现材料在 `scratchpad/bindist/`(附录 A 有完整脚本)。 + +### 2.1 步骤与结果 + +**① 生产者**(`provider/`):模块接口只有声明,实现在实现单元里。 + +```cpp +// src/runtimelib.cppm +export module runtimelib; +export namespace rt { int answer(); std::string name(); void boom(); } +// src/impl.cpp +module runtimelib; +namespace rt { int answer(){return 42;} ... } +``` +```toml +[targets.runtimelib] +kind = "shared" +``` +`mcpp build` → `bin/libruntimelib.so`,导出**恰好两个**符号: + +``` +T _ZGIW10runtimelib ← 模块初始化器 +T _ZN2rtW10runtimelib6answerEv ← rt::answer(),带 W10runtimelib 模块附着 +``` + +**② 「二进制包」**(`dist/`):只有接口源 + 预编译库 + 一个 `mcpp.toml`。 + +``` +dist/ +├── mcpp.toml +├── src/runtimelib.cppm ← 只有声明 +└── lib/libruntimelib.so ← 从 ① 拷来 +``` +```toml +[build] +sources = ["src/runtimelib.cppm"] +[targets.runtimelib] +kind = "lib" +[runtime] +libraries = ["runtimelib"] +link_library_dirs = ["lib"] +runtime_search_dirs = ["lib"] +``` + +**③ 消费者**(`app/`):`runtimelib = { path = "../dist" }`,`import runtimelib;` + +``` +$ mcpp run +answer=42 +name=from-the-prebuilt-so (len=20) +caught runtime_error: thrown inside the .so +``` + +**跨边界的 `std::string` 和 C++ 异常(含 RTTI 类型匹配)都正常。** 消费者只编译 +了那一个 `.cppm`(产出 BMI + 一个近乎空的 object),其余符号解析到 `.so`。 +`mcpp` 引擎**一行没改**。 + +> 顺带一个与本议题无关但会绊人的坑:消费者 TU 里 `import` 必须写在 `#include` +> **之后**。写在之前时 GCC 16.1 把后续 include 的声明卷进了奇怪的作用域,报 +> `In function 'int std::main()'` —— 报错点与原因完全不沾边。 + +### 2.2 三处它没有告诉你的事 + +**(a) 发出去的 `.so` 烧着生产者机器的绝对路径。** + +``` +$ readelf -d dist/lib/libruntimelib.so + (NEEDED) libstdc++.so.6 + (RUNPATH) /home/speak/.mcpp/registry/data/xpkgs/xim-x-glibc/2.44/lib64: + /home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/lib64: + /home/speak/.mcpp/registry/subos/default/lib +``` + +这正是 `docs/02-pack-and-release.md` 关于路线 A 讲的那件事(「烧进去的 +`PT_INTERP` 指的是**你**的机器」),只不过对象从可执行文件换成了库 —— +而**库这一侧没有 `mcpp pack`**。手工拷贝出去的 `.so` 是机器绑定的。 + +**(b) 一个进程里两份 C++ 运行时,零告警。** + +``` +app : NEEDED = libruntimelib.so, libm, libgcc_s, libc ← 没有 libstdc++ + nm -D | grep std | wc -l = 528 ← 静态 libstdc++ 进来了 +.so : NEEDED = libstdc++.so.6 ← 动态 +``` +这是 `distribution.cppm` 里 `Role::SharedLibrary` 那段注释描述的危险,方向反过来: +exe 的静态 libstdc++ 把符号以 GLOBAL 导出,`.so` 绑到了它上面。**这次侥幸对了** +—— 两边是同一个 gcc 16.1 的 libstdc++。生产者换个编译器版本,就是一个进程里 +两份不同实现的 `std::string`。 + +契约模型本身是对的(`Role::Distributable` 默认 SelfContained, +`Role::SharedLibrary` 默认 ToolchainCoupled),问题是 **contract 是根工程的设置, +预编译 `.so` 的 contract 在生产时就冻结了,消费者看不见也检查不到。** + +**(c) 接口与二进制之间没有任何绑定 —— 这是最严重的一条。** + +把随包发的 `.cppm` 改成结构体字段互换(Itanium 不 mangle 字段顺序,符号名不变): + +```cpp +// 生产者编进 .so 的: struct Point { int x; int y; }; origin() 返回 {111, 222} +// 随包发出的接口: struct Point { int y; int x; }; ← 只改了这一行 +``` +``` +$ mcpp run +x=222 y=111 (producer meant x=111 y=222) +``` + +**编译通过、链接通过、运行通过、数据是错的、全程零诊断。** 同样的实验用返回类型 +(`int` → `long long`)也一样静默通过。 + +这不是「mcpp 的 bug」——C++ 语言层面这就是 IFNDR。但它是**分发格式必须解决的问题**: +只要接口和二进制可以被分别替换,这个失败就是可达的,而且它跨越机器和时间 +(「上周谁把那个头改了」)。 + +### 2.4 实验二:自动生成 + 接口闭包 + 跨平台(2026-08-17,`scratchpad/lab/`) + +写了一个 176 行的 `mcpp pack ` 原型(`lab/mkdist.py`),对一个真实的多单元库 +(主接口 + 接口分区 + **实现分区** + 实现单元 + C API + 头文件)跑 +**x86_64-linux-gnu / x86_64-linux-musl / x86_64-windows-gnu** 三个 target。 +它推翻了我上一轮写进本文的**两条规则**,并找出**一个新缺陷**。 + +#### 2.4.1 包内 `mcpp.toml` 能自动生成 —— 原料今天全部已暴露 + +`mcpp build --print-fingerprint` 的 11 个字段里,tag 需要的六个全在: + +``` +[1] gcc [2] 16.1.0 [4] x86_64-linux-gnu +[5] libstdc++ 16.1.0 [6] c++23 [11] 0a0ba53e8ca69b41(runtime binding) +``` + +`abi_tag` 是它们的**纯投影**,不需要任何新推导 —— §4.3 的主张就地验证。 + +**⚠️ 但 [4] 是编译器自报的 triple,不是 mcpp 的规范 triple。** +原型第一版直接用 [4],windows 那条腿产出的 tag 是 +`x86_64-w64-mingw32-gcc16-…`,而同一份 `mcpp.toml` 里 `[target.'…']` 键写的是 +`x86_64-windows-gnu` —— **同一个决定,两个拼写**。真实实现必须用 +`triple.cppm` 的规范拼写(`cfgpred` 的注释也说 cfg 词汇 **IS** 规范 triple 词汇)。 + +#### 2.4.2 「接口怎么制定」= 模块闭包,不是 `.m.o`,也不是 grep + +工程结构: + +``` +src/mathkit.cppm export module mathkit; export import :api; +src/api.cppm export module mathkit:api; ← 接口分区 +src/secret.cppm module mathkit:secret; ← 实现分区(闭源逻辑) +src/impl.cpp module mathkit; import :secret; +``` + +| 做法 | 结果 | +|---|---| +| 只发 `mathkit.cppm` + `api.cppm` + `.a` | ✅ 消费者构建成功、`add(2,3)=5` | +| 主接口改成也 `import :secret;`,仍不发它 | ❌ **硬失败**:`mathkit:secret: error: failed to read compiled module` | + +**判据:发布集 = 从主接口单元出发、沿其 purview 内 `import` 的传递闭包。** +实现分区只被实现单元 import 时**不在闭包里,不发布,源码不泄露**。 + +**失败的不对称性(这才是必须算而不是猜的理由):** + +| | 后果 | +|---|---| +| 发少了 | **响** —— 编译期 `failed to read compiled module`,点名模块 | +| 发多了 | **哑** —— 静默把闭源实现分区的源码发出去 | + +**⚠️ `.m.o` 不是判据 —— 用它会泄露源码。** 实测:实现分区 `secret.cppm` +**照样产出 `secret.m.o`**(`.m.o` 的含义是「模块单元的对象」,不是「接口的对象」)。 +按 `.m.o` 挑要发布的源,就会把 `secret.cppm` 发出去。 + +#### 2.4.3 ⚠️ 同一个闭包有第二个用途 —— 我上一轮把它写错了 + +上一轮我写「打包时剔除归档里的接口对象」,规则给的是**「剔除所有 `.m.o`」**。 +原型照做,**三个 target 全部链接失败**: + +``` +libmathkit.a(impl.o): in function `mk::add@mathkit(int, int)': + undefined reference to `mk::secret_helper@mathkit()' +``` + +因为 `secret.m.o` 里是**真代码**。正确规则与 §2.4.2 是**同一个闭包**: + +> **剔除的是「已发布接口闭包里那些单元」的对象,不是所有 `.m.o`。** + +修正后三个 target 全部构建成功,归档里剩下 `secret.m.o` + `impl.o` + `capi.o`。 + +#### 2.4.4 平台差异 + +| 事实 | linux-gnu | linux-musl | windows-gnu(MinGW) | +|---|---|---|---| +| 静态库文件名 | `libmathkit.a` | `libmathkit.a` | **`libmathkit.a`** | +| mangling | `_ZN2mkW7mathkit3addEii` | 同 | **同**(Itanium) | +| `.m.o` 命名 | 一致 | 一致 | 一致 | +| 产物 | ELF | ELF | **PE32+ executable, 18 sections** | + +- **产物命名按 `env` 分,不按 OS 分**:MinGW 走 GNU 约定(`lib*.a`), + 只有 MSVC 才是 `*.lib` —— 所以布局里的 `lib/` 必须按**三元组**而不是按 OS 分目录。 +- **MinGW 的 mangling 与 ELF 相同**,所以 MinGW 产的库能被 MinGW 消费者链接; + MSVC 不同 —— 这就是 `cxxabi` 必须是 ABI 模型一维的原因(`abi.cppm:84`)。 + +#### 2.4.5 ⚠️ 新缺陷:裸三元组谓词在原生构建下不匹配 + +胖包的每条腿要一个 `[target..build] ldflags`。原型第一版用**裸三元组**, +结果:**显式 `--target` 三个全绿,裸 `mcpp build` 链接失败**。 + +最小探针(`lab/probe/`,同一台机器、同一个解析出的三元组): + +| `[target..build] cxxflags = ["-DX"]` | 裸 `mcpp build` | `--target x86_64-linux-gnu` | +|---|---|---| +| `'x86_64-linux-gnu'`(裸三元组) | **0 处命中** | 2 处命中 | +| `'cfg(linux)'` | 2 处命中 | 2 处命中 | + +**根因一行**(`src/build/prepare_inputs.cppm:139`): + +```cpp +if (triple.empty()) return false; // ← 裸三元组分支 +``` + +而同一文件的 `context_for()`(:49-53)在 `targetTriple` 为空时**回落到 +`triple::host_triple()`**。于是 `cfg(...)` 用宿主事实求值,裸三元组分支却拿着 +原始的空串直接返回 false —— **同一个决定,两处推导**。 +`types.cppm:665` 的注释明确承诺的是另一种行为: + +> *prepare_build evaluates it against the RESOLVED target (**host triple for a +> native build**, the --target triple for a cross build)* + +**危险形状:CI 传 `--target` 全绿,开发者本机日常构建静默失配。** +与本方案的关系:**胖包必须用 `cfg(...)`,不能用裸三元组**(见 §4.8)。 +修法看起来是一行(空串时与 `host_triple()` 比),但这是独立缺陷,应单独开 issue。 + +#### 2.4.6 全矩阵验收 + +胖包改用 `cfg(...)` 后,四种构建全过,且每个 target 的 `build.ninja` 里 +**只出现自己那条腿**: + +``` +(native, 无 --target) OK +x86_64-linux-gnu OK → fatpkg/lib/x86_64-linux-gnu +x86_64-linux-musl OK → fatpkg/lib/x86_64-linux-musl +x86_64-windows-gnu OK → fatpkg/lib/x86_64-windows-gnu (PE32+) +``` + + +### 2.5 实验三:两种接口模式 × 两种库形态(2026-08-17,`scratchpad/lab/`) + +问题:「动态库 + 接口文件」的结构应该是简单好描述的 —— 能不能同时自动支持 +`.cppm`(模块)与 `.h`(头文件)两种接口? + +**结论:能,而且结构确实简单。复杂度不在布局,全部集中在「模块接口」这一种模式上。** + +#### 2.5.1 支持矩阵 —— 同一个包、同一份 `mcpp.toml`,三种消费方式 + +一个生产者(`lab/parts`)同时提供两种接口: +`include/mathkit_c.h`(`extern "C"`)+ `interface/mathkit.cppm`(模块)。 + +| | 只 `#include` | 只 `import` | **两者同时** | +|---|---|---|---| +| **静态库** `.a` | ✅ `hdr: 5` | ✅ `mod: 5` | ✅ `both: c=5 mod=5` | +| **动态库** `.so` | ✅ `hdr: 5` | ✅ `mod: 5` | ✅ `both: c=5 mod=5` | + +**六格全过,零特殊处理。** 动态库那一列的消费者 ELF 里 +`NEEDED: libmathkit.so` 确实在,`runtime_search_dirs` 进了 RPATH。 +静态/动态之间,包描述符只差 `[runtime]` 要不要 `runtime_search_dirs`。 + +**纯头文件包**(连一个 `.cppm` 都没有的传统形态)也通过 —— `include/` + `lib/` + +`ldflags`,消费者 `#include` 即用。 + +#### 2.5.2 让两种模式自动共存的规则:只有两条,由目录决定 + +``` +pkg/ +├── mcpp.toml +├── include/ ← 文本接口:原样全发,消费者 #include,不产生任何对象 +├── interface/ ← 模块接口:必须算闭包,消费者编译它得到 BMI + object +├── lib// ← 链接面 +├── bin// ← 运行面(仅 PE) +└── LICENSE +``` + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | **预处理器** | **编译器** | +| 消费者要编译吗 | 否 | **是** | +| 要算闭包吗 | 否(头的 `#include` 由预处理器解决,而且头本来就全发) | **是**(§2.4.2) | +| 要从归档剔除对象吗 | 否 | **是**(§2.4.3) | +| ABI 闸门强度 | `extern "C"` ⇒ **只约束 libc**(`abi.cppm:12-16`) | 全部维度 | + +**这正是「为什么感觉应该简单」的答案:传统 C/C++ 库(头 + `.so`/`.a`)真的简单 +—— 没有闭包、没有剔除、闸门也弱。全部复杂度属于模块接口那一种模式, +而它的复杂度是可自动化的(scanner 的模块图现成)。** + +两种模式**互不干扰**,可以同时存在:实测同一个包被三种方式消费,六格全过。 + +#### 2.5.3 ⚠️ `kind = "shared"` 在 musl 上不是「被拒绝」,是「链接期炸掉」 + +`plan.cppm:1002` 那道守卫的判据是 `os != "linux"`。**musl 是 linux,所以它过闸**, +然后死在链接器里: + +``` +$ mcpp build --target x86_64-linux-musl # [targets.x] kind = "shared" +crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' + can not be used when making a shared object +ld: failed to set dynamic section sizes: bad value +``` + +`crtbeginT.o` 是**静态链接**的启动文件(`T` 后缀),因为 musl target 蕴含 `-static` +—— `-static` 与 `-shared` 互相矛盾。三个 target 的真实状态: + +| target | `kind = "shared"` | 诊断质量 | +|---|---|---| +| `x86_64-linux-gnu` | ✅ | —— | +| `x86_64-linux-musl` | ❌ 链接失败 | **差** —— 消息里既没有 musl 也没有 shared | +| `x86_64-windows-gnu` | ❌ 守卫拒绝 | **好** —— 点名原因与出路 | + +**守卫的判据应该是「这个 target 是否动态链接」,而不是「os 是不是 linux」。** +这条与 G2 是同一处,但比 G2 原来的描述更糟:它是一个**过了闸再炸**的洞。 + +#### 2.5.4 ⚠️ `sources = []` 今天不生效 —— 与「不写」逐字节等价 + +G1 的精确判据。在包里放一个 `src/leftover.cpp`,然后: + +| `mcpp.toml` | `build.ninja` 里 `leftover` 的出现次数 | +|---|---| +| `sources = []` | **7** | +| (整行删掉) | **7** | + +两者**完全一样** —— `toml.cppm:1703` 的 `if (sources.empty()) → 填默认 glob` +把显式空吞掉了。对二进制包的后果:**包里任何遗留在 `src/` 下的文件都会被编进 +消费者的构建**,而且可能与预编译库里的符号重复定义。作者没有任何写法能说 +「什么都不要编」。 + +(我一开始误以为这条已经能用 —— 因为测试包里根本没有 `src/` 目录, +默认 glob 也匹配不到东西。**「两条路径给出同一答案」不等于「这条路径生效了」。**) + + +### 2.3 判据 + +> **能编译链接运行,不等于能分发。** 三条判据,一条不满足就不叫「支持二进制分发」: +> 1. 产物里不含生产机器的任何绝对路径(e2e 215 已经为 exe 定了这条,库要同一条); +> 2. 接口与二进制**不可分别替换** —— 要么一起来,要么拒绝; +> 3. 消费者的工具链与产物的 ABI **不匹配时必须拒绝**,而不是链上去再赌。 + +--- + +## 3. 结构性缺口 + +按「阻塞面 × 改动成本」排,G1–G3 是硬阻塞。 + +### G1 —— `sources` 强制:没有「无源包」这个种类 + +```cpp +// src/manifest/xpkg.cppm:2058 +// Validate minimum +if (m.modules.sources.empty()) { + return std::unexpected(ManifestError{ + "synthesised manifest missing sources (mcpp segment must declare `sources = { ... }`)", ...}); +} +``` + +对 `mcpp.toml` 一侧则是另一种问题:`sources` 缺省会被填成默认 glob +(`toml.cppm:1703`),所以「作者故意没有源」和「glob 一个都没匹配上」**不可区分** +—— 我在 §2 里那个 `dist/` 包之所以要写 `sources = ["src/runtimelib.cppm"]`, +是因为接口确实要编;一个纯 C 头文件 + `.so` 的包今天只能靠 anchor 手法。 + +**判据:「不存在」与「显式为空」必须可区分。** 仓库里已有这个模式 +(`XlingsConfig::subosDeclared`,`types.cppm:632`)。 + +**实测(§2.5.4):`sources = []` 与整行删掉在 `build.ninja` 里逐字节等价** +(同一个遗留 `src/leftover.cpp`,两种写法都是 7 处命中)。所以作者今天**没有任何 +写法**能表达「什么都不要编」—— 对二进制包,这意味着包里任何遗留在 `src/` 下的 +文件都会被编进消费者的构建,并可能与预编译库里的符号重复定义。 + +### G2 —— `kind = "shared"` 只有 Linux + +```cpp +// src/build/plan.cppm:1002 +if (!targetTriple.empty() && targetTriple.os != "linux") { + for (auto const& t : manifest.targets) { + if (t.kind != Target::SharedLibrary) continue; + return std::unexpected("shared libraries are only supported for Linux (ELF) targets today..."); + } +} +``` +注释写得很清楚:PE 消费者需要导入库、Mach-O 需要 install-name,**两者都没建模**, +所以宁可拒绝也不产出没人验证过的东西。每个 shared 相关 e2e 都写着 +`# requires: elf`。 + +**这条直接把 #433 的 `.dll` / `.dylib` 两条腿砍掉了。** 不是分发格式的问题, +是**生产侧根本产不出来**。 + +**⚠️ 而且守卫的判据是错的(§2.5.3)。** 它问的是 `os != "linux"`, +但 `x86_64-linux-musl` **是** linux —— 于是它过闸,然后死在链接器里: +`crtbeginT.o: relocation R_X86_64_32 against hidden symbol '__TMC_END__'` +(musl target 蕴含 `-static`,与 `-shared` 矛盾)。消息里既没有 musl 也没有 shared。 +**正确判据是「这个 target 是否动态链接」,不是「os 是不是 linux」。** + +好消息:相邻机械已经在了 —— `ArtifactNaming::sharedNeedsImportLib` 这个字段存在 +(`plan.cppm:467`),Windows 运行期 DLL 部署也已经有(e2e 84 / #299)。 + +### G3 —— 载荷侧的 arch 轴与 target 轴 + +分成两个子问题,状态完全不同: + +| 子问题 | 状态 | +|---|---| +| 描述符的**构建规则**能否按 arch 分叉 | ✅ **已经可以** —— `target_cfg = { ["cfg(arch=\"aarch64\")"] = {...} }`(`xpkg.cppm:1315`),按**解析后的 target** 求值 | +| 描述符的**载荷**能否按 arch 分叉 | ⚠️ xim V2 有(per-arch map / 模板 / sha256),但**按宿主 arch 解析** | +| mcpp → xlings 的安装调用能否带 target | ❌ **不能**,`make_targets_args` 只有 `{"targets":[...],"yes":true}` | +| 描述符的 `mcpp` 块 per-OS 分叉 | ✅ linux/macosx/windows,且已按 **TargetPlatform** 而非宿主(#254 已修) | + +另外 `target_cfg` 只承载 `BuildInputs`(cflags/cxxflags/ldflags/sources/defines/ +flags/include_dirs/include_dirs_after,`xpkg.cppm:1359-1377`),**不含 +`runtime` / LinkIntent** —— 所以「per-arch 的 `link_library_dirs`」写不出来。 +这是 #258 同一形状的债:条件通道自己维护了一份子集。今天可以用 +`ldflags = {"-Llib/aarch64", "-lfoo"}` 绕过。 + +### G4 —— 没有兼容标签 + +`toolchain/abi.cppm` 建模了五维(libc / cxxStdlib / arch / os / cxxAbi), +并且有 `abi:=` 的约束语言。对 **C 库**够用(它的注释原文就是 +"A C library only constrains libc")。对 **C++ 模块库不够**,缺: + +- 编译器**族与主版本**(mangling、libstdc++ 符号版本、模块实现细节) +- stdlib **版本**(`Toolchain::stdlibVersion` 有,但 `AbiProfile` 里没有) +- **C++ 标准档位**(`c++23` vs `c++26` 会改变 `std::` 的可见面与部分 ABI) +- **C++ 运行时契约**(`self-contained` / `toolchain-coupled` / `host-coupled`) + +而这些**全都已经在 `cache_key::BuildAxes` 里了**(`cache_key.cppm:76-95`)。 +不是没有,是没有对外的、可读的、可发布的投影。 + +### G5 —— 接口与二进制没有绑定 + +§2.2(c) 实测。没有 digest、没有 provenance、没有符号存在性检查。 + +### G6 —— 没有面向库的 pack + +`mcpp pack` 的输入是一个 exe。库的 RUNPATH 重写、`$ORIGIN` 化、 +第三方闭包收集、`HOST-REQUIREMENTS` 生成 —— 这些逻辑全在 +`pack.cppm` / `binfmt.cppm` / `host_requirements.cppm` 里,**只差一个入口**。 + +### G7 —— 私有分发没有鉴权轴、没有离线入口 + +见 §1.4。`IndexSpec` 无鉴权字段;没有 `mcpp add ./x.mpkg`。 + +### G8 —— `runtime.library_dirs` 的 link/runtime 混淆(#304,已有 issue) + +新键(`link_library_dirs` / `runtime_search_dirs`)已经把两件事分开了 +(`docs/05-mcpp-toml.md` §2.11 的表),legacy `library_dirs` 仍然两边都进。 +**二进制分发会把这个坑放大**:预编译包必然要声明库目录,而符号farm 式的包 +(`compat.vulkan-runtime`)会因此污染链接线。 + +### G10 —— 裸三元组谓词在原生构建下不匹配(新发现,应单独开 issue) + +`prepare_inputs.cppm:139` 的 `if (triple.empty()) return false;` 让 +`[target.''.build]` 在**没有 `--target`** 时永不命中,而同文件的 +`context_for()`(:49-53)对 `cfg(...)` **回落到 `host_triple()`` ` —— +同一个决定两处推导。`types.cppm:665` 的注释承诺的是回落那一种。 +**危险形状:CI 全绿、本机静默失配。** 详见 §2.4.5(含最小探针)。 +直接阻塞胖包的裸三元组写法。 + +### G9 —— 描述符构建规则不能按版本区分(#290,已有 issue) + +对二进制分发直接相关:同一个包的 `0.1.0` 和 `0.2.0` 可能有不同的 +库文件名 / soname / 依赖集。今天 `mcpp = {}` 块对所有 `xpm` 版本一视同仁。 + +--- + +## 4. 架构方案 + +### 4.1 先定一件事:分发层级是**消费端**选的 + +这是整个方案的形状来源。生产者**发布多个层级**,消费者**按自己的工具链挑一个**, +挑不到就降级到源码。生产者不能替消费者决定,因为「你的编译器是什么」只有消费端知道。 + +| Tier | 包里有什么 | 生效条件 | 失配时 | +|---|---|---|---| +| **S** source | 全部源码 | 永远 | —— | +| **I** interface+binary | 接口源(`.cppm`/`.h`)+ 预编译库 | **abi-tag 匹配** | 降级到 S;S 不存在则**明确拒绝并列出可用 tag** | +| **B** +BMI | 再加预编译 BMI | **build-key 精确匹配** | **静默**降级到 I | + +三条纪律: + +- **Tier B 只能是加速器**,永不成为正确性依赖。失配必须静默降级,不得报错。 +- **Tier I 失配必须响**。这是 §2.3 判据 3。 +- **「因为客户端太老而不可用」必须报告成「不可用」,不能报告成「不存在」** + —— 这条是 #349 索引 floor 那次的教训(被拼成「不存在」的客户端会自己驱动 + 重复刷新索引)。 + +### 4.2 核心原语一:`[distribution]` 段 —— 一条可查询的事实 + +> **相对初稿的修订。** 初稿提议 `[targets.] kind = "prebuilt"`。 +> 它有一个致命性质:**描述符里的新键不降级**(`docs/10` 点名的那一类), +> 老客户端会被砖。改成 `[distribution]` 段之后,老客户端读到的仍是 +> `sources` + `[runtime]`(**已发布能力**),而 `[distribution]` 被**静默跳过**。 +> **兼容性从「要版本 floor」变成「几乎免费」。** +> +> **⚠️ 实测钉死的两条边界(mcpp 2026.8.15.3):** +> 1. `[distribution]` 里放**标量 + 字符串数组**:**接受,且连警告都没有** ✅ +> 2. `artifacts = [{ … }]`(array-of-tables):**硬失败** ❌ +> ``` +> error: [[distribution.artifacts]] (array-of-tables) is not allowed for +> section 'distribution.artifacts'; array-of-tables syntax is only supported +> for [[build.flags]], [[features..flags]], [[runtime.requirements]], +> and [[runtime.artifacts]] +> ``` +> **所以产物清单不能自己造 —— 必须用已在白名单里的 `[[runtime.artifacts]]`** +> (它的字段恰好就是 `role`/`path`/`provenance`/`abi`/`digest`/ +> `host_fingerprint`,`docs/05-mcpp-toml.md` §2.11 已经文档化)。实测: +> `[distribution]` 标量段 + `[[runtime.artifacts]]` 一起,今天的 mcpp 直接通过。 +> +> **诚实的代价:静默跳过意味着老客户端拿到预编译包时「一道闸门都没有」**, +> 拿到的是今天的行为(能用、但不安全)。这是**降级**而不是变砖,方向是对的; +> 但它也意味着闸门只保护新客户端 —— 这一点必须写进发布说明,不能假装没有。 + +```toml +[build] +sources = ["interface/runtimelib.cppm"] # 只有接口;实现在预编译产物里 +# 或 sources = [] # 纯 C 头 + .a 的包:显式为空 ≠ 缺省 + +[distribution] +artifact_kind = "static" +abi_tag = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +... +``` + +**两件事,不要合并:** + +| | 解决什么 | 为什么不能只要一个 | +|---|---|---| +| `sources = []` **显式为空** | 「不要填默认 glob」 | 今天缺省会被填成 `src/**`(`toml.cppm:1703`),于是「作者故意没有源」与「glob 一个都没匹配上」不可区分 | +| `[distribution]` 段 | 「这个包的产物不来自编译」——**一条可查询的事实** | 闸门、`mcpp why`、lint、以及 §4.5.0(d) 的 build 守卫都要读它;挂在 `[build]` 上会让普通源码包也能声明 `abi_tag`,那就成了一个可以说谎的地方 | + +**Appendix A(schema ownership)检验:** mcpp 定义**机制**(产物来自文件而非编译边、 +以及一组闸门),**词汇留在值里**(路径、tag 字符串、`static`/`shared`)。键封闭。✅ + +#### 4.2.1 `[pack]` 字段的取舍 + +一句话:**能从 mcpp.toml 别处推出来的,就不给字段。** +按这条逐条审计之后,`[pack]` **新增 0 个键** —— WHAT/HOW 由 `[targets.].kind` +回答(`mcpp pack [target]`),接口根由 `[lib]` 约定回答,头目录由 +`[build].include_dirs` 回答,平台由 `[package].platforms` 回答(升格为**断言**)。 +完整的推导审计、以及「什么不允许裁剪」的一致性论证在 **§4.6.1**。 + +### 4.3 核心原语二:两个兼容量,不是一个 + +**必须是两个,因为它们回答两个不同的问题。** + +| | `abi-tag` | `build-key` | +|---|---|---| +| 回答 | 「这份二进制能不能链进你的构建」 | 「这份 BMI 能不能直接用」 | +| 谁能算 | **生产者**(消费者存在之前就能枚举) | **只有消费者**(含依赖闭包 Merkle) | +| 粒度 | 粗、可读、可发布 | 精确、16 hex、不可读 | +| 用途 | 决定**下载哪个载荷** | 决定 **Tier B 命中** | +| 来源 | `AbiProfile` + `Toolchain` + `CppStandardConfig` | `cache_key::key_hex`(**已存在**) | + +**判据:如果试图只用一个,就会发现你无法发布 build-key** —— 它含依赖闭包, +每个消费者的图都不同,生产者得为每种图各发一份,不可能。 + +**`abi-tag` 的组成(6 段,全部来自已有字段):** + +``` +-----c++ + +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 +aarch64-macos-none-llvm22-libcxx22-c++23 +x86_64-windows-msvc-msvc194-msvcstl194-c++23 +``` + +- `arch`/`os`/`env`:`triple.cppm` 已有的规范拼写。 + **⚠️ 不是 `--print-fingerprint` 的 [4]** —— 那是编译器自报的 + (`x86_64-w64-mingw32`),与 `[target.'…']` 键的拼写(`x86_64-windows-gnu`) + 不同,直接用会让同一个决定有两个拼写(§2.4.1 实测踩到过); +- `compiler`:`Toolchain::compiler_name()` + `version` 的主段; +- `stdlib`:`Toolchain::stdlibId` + `stdlibVersion` 的主段; +- `c++`:`CppStandardConfig::level`。 +- `cxxAbi` 不入 tag —— 它由 (os, compiler) 唯一确定,放进去是第二个答案。 + +**不入 tag、但必须单独声明并检查的两项:** + +| 字段 | 为什么不入 tag | 检查语义 | +|---|---|---| +| `cxx_runtime`(契约) | 它可以**协商**:toolchain-coupled 的 `.so` 是能被 self-contained 的 exe 消费的(§2.2b 实测),只是有危险 | 不匹配 → **警告并说清危险**,`--strict` 升级为错误 | +| `abi_surface = "c" \| "cxx"` | 纯 `extern "C"` 接口只约束 libc 这一维 —— `abi.cppm` 的注释原文就是这个规则 | `"c"` → 只比 arch/os/env,忽略 compiler/stdlib/std | + +`abi_surface = "c"` 这个逃生口很重要:它让**绝大多数传统 C 库**(zlib、openblas、 +ffmpeg)只需要一个 tag 就覆盖所有编译器,tag 组合数从 N×M 掉回 N。 + +### 4.4 核心原语三:接口↔二进制的绑定 + +三道闸,由弱到强,**全都要**: + +1. **interface digest** —— 包元数据里记随包接口文件的 sha256。消费者编译前重算, + 不符即拒。挡住「有人改了随包的头」。 +2. **符号存在性交叉检查** —— 编完接口后 mcpp 知道模块名 `M`,去预编译库里查 + `_ZGIW`(ELF/Mach-O)/ 导出表(PE)。挡住「配错了库」。便宜,值得做。 +3. **原子产出 + provenance** —— **接口副本与二进制必须由同一次 `mcpp pack ` + 产出**,元数据里记 `built_by` / `build_key` / `source_digest`。 + 手工拼装的包 mcpp 拒绝消费。 + +**必须诚实说清楚:三道闸都挡不住「结构体字段顺序变了但两边都自洽」**—— +只要接口和二进制**能被分别替换**,§2.2(c) 就是可达的。唯一充分的保护是 +**它们一起产出、一起分发、不可分别替换**。所以第 3 条才是根,前两条是防御。 + +### 4.5 包格式:**不是新格式 —— 是一个自带 `mcpp.toml` 的 Form A 包** + +> **本节相对初稿是重写。** 初稿提议一个新格式 `.mpkg` + 一份并列的 +> `MCPP-PACKAGE.toml`。**那是错的形状**,理由见 §4.5.0:mcpp 早就有 +> 「包自带 `mcpp.toml`」这条通路,而且它是文档推荐的那一种。 + +#### 4.5.0 为什么自带 `mcpp.toml` 是对的形状 + +**它已经是既有机制。** 索引描述符没有 `mcpp` 字段时,mcpp 在解开的载荷里 +glob `mcpp.toml` / `*/mcpp.toml` 并当作该依赖的 manifest 加载 +(`prepare.cppm:2764-2787`)。`docs/10-publishing-a-library.md` 原话: +*"A repo that ships its own `mcpp.toml` needs no `mcpp` field in the index entry."* +而 LinkIntent 是**逐包聚合**的(`plan.cppm:630-656`,路径按 `package.root` +转绝对),所以 path / git / 索引 tarball 三条路吃的是同一套 —— §2 的实测走的是 +path 依赖,索引 tarball 汇合到同一处。 + +**三个后果,第一个是决定性的:** + +**(a) 不需要新描述符键 ⇒ 不需要版本 floor ⇒ 老客户端不会被砖。** + +初稿的 `kind = "prebuilt"` 正好属于 `docs/10` 点名警告的那一类**不降级**的键 +(和 `module_extensions` 同类):老 mcpp 读不懂它,不是「警告后忽略」,而是硬失败 +或者更糟 —— 把 `interface/` 当普通源编译然后链不上。走 Form A 就完全不需要它: +老客户端读到的是 `sources = ["interface/…"]` + `[runtime] libraries / +link_library_dirs`,**全部是已发布能力** —— §2 的实测就是在 **2026.8.15.3** 上 +跑通的。**兼容性几乎免费。这是 Form A 相对新描述符键的决定性优势。** + +**(b) 胖包(fat package)可以绕开整个 G3。** + +包里带的是 `mcpp.toml`,于是它可以写 `[target.'cfg(...)'.build]` —— +而这个条件轴**按解析后的 target 求值**,也就是在消费者那边、在 `--target` +已知之后。所以一个包同时装 x86_64 / aarch64 / linux / windows 的库, +**选择发生在构建期**: + +- 不需要描述符 arch 轴; +- 不需要安装线 target 轴; +- **不需要动 xlings 一行。** + +这把「交叉编译 + 二进制包」从 P2(跨仓库)搬到了 **P0 就能表达**。 +代价是下载体积(可用「胖包默认 + 瘦包按 tag」两种发布方式并存来缓解)。 +今天的拼写限制:条件轴只承载 `BuildInputs`(**无 LinkIntent**),所以 per-arch +只能写 `ldflags = ["-Llib/", "-lfoo"]`,写不了 `link_library_dirs`。 +能用,但丑;干净的修法仍是让 LinkIntent 过条件轴(G3 收尾)。 + +**(c) 闸门字段必须放进 `mcpp.toml`,不能放旁路文件。** + +理由不是省一个文件,是 **path 依赖也需要闸门**。如果 abi-tag / interface digest +只存在于 `mcpp pack ` 写的旁路文件里,那么「把一个目录拷给同事」这条最常见的 +内部路径就没有任何检查 —— 而 §2 的实测证明这恰恰是人们会走的路。 +所以:**`MCPP-PACKAGE.toml` 取消,内容并入 `mcpp.toml` 的 `[distribution]` 段。** + +**(d) 必须加的守卫 —— 一个字面叫 `mcpp.toml` 的文件躺在解开的二进制包里, +会引诱人在里面 `mcpp build`。** + +今天它会**成功**:把 `interface/` 里只有声明的接口单元编出来,产出一个几乎空的 +库,而实现全在预编译产物里、根本没被链进去。典型的「看起来成功的失败」。 +**`[distribution]` 存在时,在该目录直接 `mcpp build` 必须拒绝**,并说清这是分发包 +不是源码树。 + +#### 4.5.1 布局 + +``` +runtimelib-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.tar.gz (PE: .zip) +└── runtimelib-0.1.0/ + ├── mcpp.toml ← 生成物,含 [distribution] 段;消费端零新代码路径 + ├── interface/ ← 消费者要编译的 .cppm / .ixx + ├── include/ ← 非模块消费者的头 + ├── lib/ ← 链接面:.a / .so / .dylib / **.lib(PE 导入库)** + ├── bin/ ← 运行面(仅 PE):.dll,必须部署到 .exe 旁 + ├── bmi/ ← 可选 Tier B + ├── HOST-REQUIREMENTS ← 与 mcpp pack 同一份推导 + └── LICENSE / THIRD-PARTY +``` + +**归档格式与源码包同构**(tar.gz / zip),索引条目形状**一字不改** +(url + sha256 + 三平台块)。文件名带 tag 只是**人读的命名约定**,不是新格式 —— +`mcpp add ./` 靠 `[distribution]` 段识别,不靠扩展名。 + +**两条规则,由文件所在的目录决定 —— 这就是全部**(§2.5.2 实测): + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | 预处理器 | **编译器** | +| 消费者要编译吗 | 否 | **是** | +| 要算闭包吗 | 否 | **是** | +| 要从归档剔除对象吗 | 否 | **是** | +| ABI 闸门 | `extern "C"` ⇒ 只约束 libc | 全部维度 | + +两种模式**互不干扰、可同时存在**:实测同一个包被「只 `#include`」/「只 `import`」/ +「两者同时」三种方式消费,× 静态库/动态库两种形态,**六格全过**。 +**传统 C/C++ 库(头 + `.so`/`.a`)因此真的简单** —— 没有闭包、没有剔除; +全部复杂度属于模块接口那一种模式,而它是可自动化的。 + +**`lib/` 与 `bin/` 分开是机制,不是风格。** + +| 类别 | ELF | Mach-O | PE | +|---|---|---|---| +| 链接面 | `lib/libfoo.so` | `lib/libfoo.dylib` | **`lib/foo.lib`(导入库)** | +| 运行面 | 同一个文件 | 同一个文件 | **`bin/foo.dll`(另一个文件)** | + +PE 上「链接的东西」和「运行的东西」是两个不同的文件,ELF/Mach-O 上是同一个。 +一个只分 `interface/` + `lib/` 两层的布局在 Windows 上表达不出这件事 —— 这正是 +`compat.openblas` 的 windows 块必须同时写 `ldflags = {"-Llib","-llibopenblas"}` +**和** `runtime = { library_dirs = { "bin" } }` 的原因。 + +#### 4.5.2 `artifact_kind = "static" | "shared"` —— 一个布局,两种形态 + +**不要为动态库单开一个命令。** 形态是布局里的一个字段,理由是静态那条腿 +**今天三平台就能走通,而且把一整类问题消掉了**(实测,见下)。 + +``` +$ mcpp build # kind = "lib",无平台限制 + → bin/libstatlib.a +$ ar t libstatlib.a → statlib.m.o impl.o +$ nm --defined-only + statlib.m.o: T _ZGIW7statlib ← 接口单元对象:只有模块初始化器 + impl.o: T _ZN2slW7statlib6answerEv ← 实现单元对象:真正的符号 +$ readelf -d libstatlib.a → 无 RUNPATH(归档没有动态段) +``` + +| | `static` | `shared` | +|---|---|---| +| 三平台产出 | ✅ **今天就有**(`kind="lib"` 无平台限制) | ❌ **仅 Linux**(G2) | +| RUNPATH 重写(§2.2a) | **不需要** —— 归档没有动态段 | 必须 | +| 双 C++ 运行时(§2.2b) | **不会发生** —— 由消费者自己的契约统一 | 会,需检查 | +| 运行期部署 | 无 | PE 需部署 `.dll` | +| 加载器搜索闭包 | 无 | 需要 | +| 代价 | 消费者产物体积;不能热替换 | —— | + +**对「闭源库内部分发」这个场景,`static` 往往是更好的默认。** + +**一处必须做成结构性保证、而不是巧合的事:打包时剔除已发布接口单元的对象。** +上面的实测显示归档成员切分是干净的 —— 消费者自编接口后会定义 `_ZGIW7statlib`, +于是 `statlib.m.o` 这个成员没有任何未定义符号需要它,**根本不会被拉进来**。 +但这依赖「归档成员只在解析未定义符号时才被拉取」这条链接器行为; +`--whole-archive`、或该成员里恰好还有别的被引用符号,都会让它被拉进来并与 +消费者自编的接口对象重复定义。打包器把它剔掉,这条风险就不存在了。 + +> **⚠️ 剔除集是「已发布闭包里那些单元的对象」,不是「所有 `.m.o`」。** +> 本文上一版写的是后者,**实测三个 target 全部链接失败**(§2.4.3): +> `.m.o` 的含义是「模块单元的对象」,实现分区照样是 `.m.o` 且里面是真代码。 +> 剔除集与发布集是**同一个闭包的两个用途**(§2.4.2)—— 这正是它们必须由 +> 同一次推导给出、而不是各算各的的理由。 + +#### 4.5.3 包内 `mcpp.toml`:既有的键 + 一个新段 + +**上半部全是今天就能跑的键**(§2 实测,mcpp 2026.8.15.3): + +```toml +# ══ 生成物。手工编辑会使 [distribution].interface_digest 失配并被拒绝。 ══ +[package] +namespace = "acme"; name = "runtimelib"; version = "0.1.0" + +[build] +sources = ["interface/runtimelib.cppm"] # 只有接口,消费者编译它得到 BMI + +[targets.runtimelib] +kind = "lib" + +[runtime] +libraries = ["runtimelib"] +link_library_dirs = ["lib"] +runtime_search_dirs = ["lib"] # shared 形态才需要 +# deploy_files = ["bin/runtimelib.dll"] # PE + +# 胖包:选择在消费者的构建期发生,按解析后的 target 求值 +[target.'cfg(all(linux, arch = "aarch64"))'.build] +ldflags = ["-Llib/aarch64-linux-gnu", "-lruntimelib"] +``` + +**下半部是闸门 + 证据。⚠️ 只能是标量与字符串数组 —— 产物清单必须借用已在 +array-of-tables 白名单里的 `[[runtime.artifacts]]`(见 §4.2 的实测边界):** + +```toml +[distribution] +schema = 1 +artifact_kind = "static" # static | shared +abi_tag = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +abi_surface = "cxx" # cxx | c(纯 extern "C" 只约束 libc) +cxx_runtime = "toolchain-coupled" +modules = ["runtimelib"] +interface_digest = "sha256:…" # 覆盖 interface/ 下每个文件 + +# ── 以下是证据,不是旋钮。用户手写 = 错误,不是「覆盖」。 ── +built_by = "mcpp 2026.8.17.1" +build_key = "aeb4c4d29e437696" # cache_key::key_hex,Tier B 用 +source_digest = "sha256:…" + +# 产物清单 —— 复用既有的、已文档化的段(docs/05 §2.11),不新造。 +# 它的字段恰好就是需要的:role / path / provenance / abi / digest / host_fingerprint。 +[[runtime.artifacts]] +role = "static-library" +path = "lib/libruntimelib.a" +provenance = "mcpp-pack" +abi = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +digest = "sha256:…" +``` + +**实测(mcpp 2026.8.15.3):上面这整份 —— `[distribution]` 标量段 + +`[[runtime.artifacts]]` —— 今天的 mcpp 原样接受并构建成功。** +(顺带一个可观察的证据:加上 `[[runtime.artifacts]]` 后 fingerprint 变了, +说明它确实被读进 manifest 并折进了构建身份,不是被丢掉。) + +**为什么证据字段也放这里、而不是旁路文件:**多一个文件就多一个能和主文件漂移的 +地方 —— 这是仓库反复出现的那条教训(`publisher.cppm:194-216`: +「THE SAME DERIVATION … 分开推导就是它们漂移的方式」)。代价是必须有一条 +**「用户手写了生成字段 = 错误」**的规则,而这条规则本来也需要 +(`built_by` 被人改成别的值,比没有它更糟)。 + +**一份推导,三个投影**(比初稿少一个 —— 包内文件与消费契约合并了): + +``` + ┌→ xpkg.lua 描述符(索引;Form A ⇒ 只有 url + sha256) +一次 resolve+build ──┼→ 包内 mcpp.toml(消费契约 + [distribution] 闸门/证据) + └→ HOST-REQUIREMENTS(bundle) → 消费者 mcpp.lock 条目 +``` + +### 4.6 生产侧:`mcpp pack ` + +```bash +mcpp pack # 唯一可打包目标;有歧义 → 报错并列出候选 +mcpp pack mathkit # [targets.mathkit].kind = "lib" → 静态库包 +mcpp pack mathkit-shared # kind = "shared" → 动态库包 +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 胖包(--target 可重复) +mcpp pack mathkit --tier bmi # 可选 Tier B(默认 interface) +``` + +> **没有 `--lib`,也没有 `--artifact`。** 产出什么由目标的 `kind` 决定 —— +> §4.6.1(b):三个旋钮塌缩成零。 + +做的事(**全部复用 `mcpp pack` 已有机械**): +1. build 出库产物; +2. 拷贝接口源、**从归档中剔除接口单元的对象**(§4.5.2)、算 digest; +3. 生成包内 `mcpp.toml`(消费契约 + `[distribution]`)+ `HOST-REQUIREMENTS`; +4. 打包,**确定性归档**(PE 侧 `zip.cppm` 已经是无时间戳的了)。 + +`--artifact shared` 额外多两步,**这两步是 static 形态根本不需要的**: + +5. **RUNPATH/INTERP 重写** —— §2.2(a) 的解药,`pack.cppm` 已有,只差入口; +6. 收第三方闭包(ELF 走 `LD_TRACE_LOADED_OBJECTS`,PE 走导入表 —— `binfmt.cppm` 已有)。 + +**「三平台」拆成两件事,不要排在一起:** + +| | 打包器 / 布局 / 元数据 | 产出动态库本身 | +|---|---|---| +| 平台相关性 | **无关** —— 就是文件布局 + toml + digest | **强相关** | +| 现状 | 两条产出族已在(ELF/Mach-O→tar.gz、PE→zip),且 **PE 路径任何宿主都能跑**(读导入表,不执行产物) | **仅 Linux**(G2) | +| 工作量 | 小,可一次覆盖三平台 | PE 导入库 + Mach-O install_name,**真正的工作量在这里** | + +没有右列,Windows/macOS 的包是**空盒子**。这就是把 static 提到 P0 的理由: +它让 P0 结束时三平台都拿到能用的包。 + +**验收判据(直接复用 e2e 215 的形状):** 扫遍包里每个二进制, +出现 `$MCPP_HOME` 下的路径即失败。static 形态天然满足(归档无动态段), +但**判据照样要跑** —— 它是对 `include/`、`interface/`、元数据里残留绝对路径的守卫。 + +#### 4.6.1 `[pack]` 的架构 —— 推导审计的结果:一个新键都不加 + +> **本节是第二次重写。** 第一版提了 `default_kind` / `default_artifact` / +> `targets` / `[pack.interface]` / `[pack.headers]` 五组键。 +> 按「**能从 mcpp.toml 别处推出来的,就不给字段**」逐条审计之后,**全部删掉**。 +> 留下的是既有键 + 一个 CLI 位置参数。 + +##### (a) 推导审计 + +对每一个候选键问同一个问题:**这件事 mcpp.toml 别处已经说过了吗?** + +| 第一版的键 | 已经在哪说过了 | 结论 | +|---|---|---| +| `default_kind`(app/lib) | `[targets.].kind` | **删** | +| `default_artifact`(static/shared) | 同上(`"lib"` vs `"shared"`) | **删** | +| `targets = [三元组…]` | `[package].platforms` 问的是同一件事(OS 级) | **删**,改成**校验** | +| `[pack.interface] modules` | lib-root 约定 / `[lib].path` | **删** | +| `[pack.headers] dirs` | `[build].include_dirs` | **删** | +| `[pack.headers] exclude` | 不可推导 —— 但**不允许**(见 (c)) | **删** | +| `include` / `exclude`(extras) | 不可推导 | **保留既有键,语义一字不改** | +| `default_mode` | 不可推导 | **保留既有键** | + +**结果:`[pack]` 新增 0 个键。** + +##### (b) WHAT / HOW 根本不是 pack 的设置 —— 是 `[targets.].kind` + +`mcpp pack [target]`,与既有的 `mcpp run [target]` 同形。**目标的 `kind` 决定一切:** + +| `[targets.].kind` | `mcpp pack ` 产出 | `--mode` 适用吗 | +|---|---|---| +| `bin` | 应用 bundle(既有四档) | ✅ | +| `lib` | **静态库包** | ❌ 归档不携带任何东西,闭包深度无意义 | +| `shared` | **动态库包** | ✅ `lib/` 要不要带上第三方 `.so` | + +`--lib` 这个 flag 不需要了,`--artifact` 不需要了,`default_kind` / `default_artifact` +也不需要了 —— **三个旋钮塌缩成零,因为 `kind` 本来就是答案。** + +**同时发布静态与动态 = 声明两个目标**(实测可行): + +```toml +[targets.mathkit] kind = "lib" +[targets.mathkit-shared] kind = "shared" soname = "libmathkit.so.1" +``` +``` +bin/libmathkit.a +bin/libmathkit-shared.so +bin/libmathkit.so.1 -> libmathkit-shared.so ← soname 别名(既有机制) +``` + +**已知的 wart:`.so` 的文件名带了 `-shared`。** `soname` 给出了正确的运行期名, +打包器发布 soname 那个名字即可。长期干净的修法是让 `kind` 接受列表 +(`kind = ["lib", "shared"]`,Cargo `crate-type` 的先例)——那是一次独立的改动。 +**但绝不要用一个 pack 期的 `--artifact` 覆盖去补救**:那会立刻变成 `kind` 的第二个答案。 + +##### (c) 头与接口在 pack 期**不允许**裁剪 —— 一致性论证 + +这是唯一一组「不可推导、但仍然不给」的键。理由不是「用不上」,是**给了就破坏不变量**: + +> **源码分发的包把 `include_dirs` 的全部内容暴露给消费者**(usage requirements, +> `scanner.cppm:700-716`)。二进制包若裁掉一部分,**同一个库会因为分发形式不同 +> 而拥有不同的公开面** —— 破坏「分发形式不改变语义」。 + +而且「哪些头是公开的」**布局已经回答了**:`include/` 是公开的(它在消费者的 +include 路径上),`src/` 是私有的。**一个私有头放在 `include/` 下是工程布局的错误, +不是打包问题。** 给 pack 一个裁剪键 = 允许用打包配置去补救布局错误, +而补救的结果是两种分发形式不一致。 + +接口 `.cppm` 同理,而且更硬:§2.4.2 的不对称性(发少了响、发多了哑)。 + +##### (d) 既有键升格为**断言**,而不是新增选择器 + +`[package].platforms` 的词汇是**封闭的**(`linux | macos | windows`, +`prepare.cppm:1075-1085`,未知值告警/`--strict` 报错)。不要为了 pack 去扩它。 + +改成:**`mcpp pack` 把 `--target` 的集合与声明的 `platforms` 做覆盖比对, +缺一条腿就告警。** 与 `[modules] exports` 同一形状 —— **既有键做断言,不做选择器**。 +好处:不新增键,而且「0.1.0 声称支持 windows 却没打 windows 那条腿」变成可发现的。 + +##### (e) 最终形态 —— 全部是既有的键 + +```toml +[package] +platforms = ["linux", "windows"] # 既有:声明支持的平台。pack 用它做覆盖校验 + +[build] +include_dirs = ["include"] # 既有:公开头。整发,不可裁 + +[lib] +# path = "src/mathkit.cppm" # 既有:接口根(不写 = src/.cppm 约定) + # 模块闭包从这里出发 + +[targets.mathkit] +kind = "lib" # 既有:决定 `mcpp pack mathkit` 产出什么 + +[targets.mathkit-shared] +kind = "shared" # 既有 +soname = "libmathkit.so.1" # 既有:给出正确的运行期名 + +[pack] +default_mode = "vendored" # 既有键,语义不变(仅 bin / shared 适用) +include = ["share/**"] # 既有键,语义不变 —— 只作用于 extras +exclude = ["**/*.tmp"] # 既有键,语义不变 —— 只从 include 里剔除 +``` + +```bash +mcpp pack # 唯一可打包目标;有歧义 → 报错并列出候选 +mcpp pack mathkit # kind="lib" → 静态库包 +mcpp pack mathkit-shared # kind="shared" → 动态库包 +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 胖包 +``` + +**新增 manifest 键:0。新增 CLI:一个位置参数 +`--target` 可重复。** +(`mcpp pack` 今天只有 `--mode` / `--target` / `--format` / `-o`,没有位置参数 —— +`cli.cppm` 的 `cl::App("pack")`。) + +##### (f) 成文规则:什么不允许、什么不推荐 + +| 类别 | 规则 | +|---|---| +| **不允许** | ① 任何已被别处回答的问题(产物形态 / 接口根 / 头目录);② 任何会让**源码分发与二进制分发语义不同**的裁剪(头、接口) | +| **保留但不推荐** | `[pack].include/exclude` —— 只用于 **extras**(LICENSE / docs / 数据)。**不要拿它裁头或接口:裁不到,而且今天会静默无效**(`types.cppm:743` 的语义是 *drop from `include`*) | +| **必须写** | 无。一个库工程不写任何 `[pack]` 也能打出正确的包 | + +##### (g) 唯一保留的纪律:三态 + +删掉了所有新键,但「不写 = 默认」这条原则本身要求一件事: + +> 每个键必须区分 **不写**(默认)/ **写了有值**(覆盖)/ **写了但为空**(显式什么都不要)。 + +**实测 `sources = []` 与整行删掉在 `build.ninja` 里逐字节等价**(§2.5.4)—— +「不写=默认」在没有三态时会退化成「无法表达空」。这条对 G1 是硬要求 +(二进制包必须能说「什么都不要编」),对未来任何新键也是。 +仓库已有正确的模式:`XlingsConfig::subosDeclared`(`types.cppm:632`)—— +**一个 `Declared` 布尔,而不是靠容器的 `empty()`**。 + +##### (h) 跨平台收口(不变) + +- `lib//` **按三元组分,不按 OS 分** —— MinGW 与 MSVC 同为 windows, + 一个产 `lib*.a` 一个产 `*.lib`(§2.4.4)。 +- PE 的链接面与运行面是两个文件 ⇒ `lib//` 与 `bin//`。 +- per-target `ldflags` 必须生成 `cfg(...)` 谓词,**不能生成裸三元组**(§2.4.5)。 +- `kind = "shared"` × target 的合法性要在**计划阶段**校验:musl 是静态链接的, + 今天会一路走到链接器才炸(§2.5.3)。 +- 打包器**绝不 glob 产物**,用刚跑那次构建的 `target///bin/…`(附录 A ⑬)。 + + +### 4.7 消费侧:解析、闸门、降级 + +**依赖形态(三种,同一个下游路径):** + +> `.mpkg` 只是**人读的命名约定**(文件名里带 tag),不是新格式 —— 里面就是一个 +> 根目录带 `mcpp.toml` 的 tar.gz / zip。识别靠 `[distribution]` 段,不靠扩展名。 + +```toml +# ① 离线文件 —— P0,不需要索引、网络、鉴权、xlings 改动 +runtimelib = { package = "vendor/runtimelib-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.mpkg" } + +# ② 索引(公开或私有)—— P1 +runtimelib = "0.1.0" + +# ③ CLI 直装 +$ mcpp add ./runtimelib-0.1.0-.mpkg +``` + +**闸门表(顺序即诊断顺序):** + +| 检查 | 失配 | +|---|---| +| arch / os / env | **拒绝** —— 载荷本身就不对 | +| `abi.surface == "c"` | 只查上一行,以下全跳过 | +| compiler 族 / 主版本 | **拒绝**;`--allow-abi-drift` 强制,并打印它保护的是什么 | +| stdlib id / 主版本 | **拒绝** | +| C++ 标准档位 | 消费者档位 < 生产者 → **拒绝**(接口可能用到更新的语法);> → 放行 | +| `cxx_runtime` 契约 | **警告** + 说清双运行时危险;`--strict` 升级为错误 | +| interface digest | **拒绝** | +| 模块符号存在性 | **拒绝** | +| build-key(Tier B) | **静默**降级到 Tier I | + +**没有匹配 tag 时的诊断形状**(这条比机制本身还重要): + +``` +error: acme.runtimelib@0.1.0 has no prebuilt artifact for this toolchain + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + aarch64-linux-gnu-gcc15-libstdcxx15-c++23 + note: this package ships no source tier, so there is nothing to fall back to. + fix : ask the publisher for a gcc16 build, or pin [toolchain] to gcc@15. +``` + +**「不可用」不能被拼成「不存在」。** 一个被拼成「找不到包」的失败会让客户端 +自己驱动重复刷新索引 —— #349 的教训。 + +### 4.8 分发侧:索引与安装线 + +> **相对初稿是重写。** 初稿在描述符里加 `kind = "prebuilt"` + `abi_tags`, +> 并因此要求一个 mcpp 版本 floor。**走 Form A 之后这一整块消失了。** + +**首选:胖包 + Form A —— 描述符一字不改。** + +```lua +-- 与一个源码包的描述符完全同构:没有 mcpp = {} 块,只有 url + sha256。 +-- 包内自带 mcpp.toml,里面的 [target.'cfg(...)'] 在消费者构建期选 arch/OS。 +xpm = { + linux = { ["0.1.0"] = { url = { GLOBAL = "…", CN = "…" }, sha256 = "…" } }, + macosx = { ["0.1.0"] = { url = { … }, sha256 = "…" } }, + windows = { ["0.1.0"] = { url = { … }, sha256 = "…" } }, +} +``` + +- **没有新描述符键 ⇒ 没有版本 floor ⇒ 老客户端不会被砖**(§4.2 的实测边界); +- **⚠️ 每条腿的谓词必须是 `cfg(...)`,不能是裸三元组** —— 裸三元组在**原生构建下 + 不匹配**(§2.4.5,根因 `prepare_inputs.cppm:139`),失败形状是 + 「CI 传 `--target` 全绿、开发者本机静默失配」。实测可用的写法: + ```toml + [target.'cfg(all(linux, not(env = "musl")))'.build] + ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + [target.'cfg(all(linux, env = "musl"))'.build] + ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] + [target.'cfg(windows)'.build] + ldflags = ["-Llib/x86_64-windows-gnu", "-lmathkit"] + ``` + 四种构建(native / gnu / musl / windows)全过,且每个 target 的 `build.ninja` + 里只出现自己那条腿(§2.4.6); +- **arch 选择在消费者的构建期**,`cfg()` 按解析后的 target 求值 ⇒ + **交叉编译天然正确,不需要安装线的 target 轴**; +- 镜像/CN 分流/sha256/artifact 通道**全部沿用源码包那一套**。 + +**可选优化(P2+):瘦包,按 tag 分资产。** 只有当胖包体积成为真问题时才做: + +```lua +xpm = { + linux = { + ["0.1.0"] = { + x86_64 = { url = "…-x86_64-…-gcc16-libstdcxx16-c++23.tar.gz", sha256 = "…" }, + aarch64 = { url = "…-aarch64-…", sha256 = "…" }, + }, + }, +} +``` +这用的是 **xim V2 已有的 per-arch resource map**,不是新语法。但它把 arch 选择 +挪回**安装期**,于是重新撞上「安装线没有 target 轴」: + +``` +install_packages {"targets":[...], "yes":true} + → {"targets":[...], "yes":true, "target":{"os":"linux","arch":"aarch64"}} +``` +这是**跨仓库改动**(xlings)。两个不改 xlings 的退路,都不推荐但要写下来: + +| 退路 | 代价 | +|---|---| +| 把 arch 编进包名(`aarch64-runtimelib`) | 违反 SPEC-001 的身份模型;#290 的「同一个库不同版本」问题再犯一次 | +| mcpp 自己下二进制载荷,绕过 xlings | 镜像/CN 分流/校验/断点全要重做一遍;`fetcher/progress.cppm` 只是起点 | + +**决策建议:胖包做默认。** 它让「交叉编译 + 索引二进制包」在 **P1 就可用**, +而不是等 P2 的跨仓库协调 —— 瘦包只是体积优化,不是能力前提。 + +### 4.9 私有分发:三种形态 + +| 形态 | 机制 | 需要新增 | 阶段 | +|---|---|---|---| +| **离线包** | `mcpp add ./x.mpkg` / `{ package = "..." }` | 一个依赖形态(格式已有:自带 `mcpp.toml` 的 tar.gz) | **P0** | +| **私有索引(git)** | `[indices] acme = { url = "git@..." }` | 文档 + `.mpkg` URL 支持;鉴权仍走 ambient git 凭据 | P1 | +| **私有 artifact 源** | `IndexSpec::artifact` | `IndexSpec` 加鉴权(header / netrc / helper) | P2 | + +**关于鉴权的立场建议:不要发明 mcpp 自己的凭据存储。** 走两条既有轨道: +① git 的 credential helper / SSH(索引侧,今天已经在用); +② 一个 `[indices.] auth = { header_env = "ACME_TOKEN" }` 形状 —— **值从环境变量读, +永不落盘**。理由:mcpp 的 manifest 是要进版本库的,任何能写 token 的字段都会 +被写进版本库。 + +**关于「团队内部统一编译器」——issue 作者说「对于 mcpp 来说统一编译器和版本不是难事」, +这句话是对的,而且这正是 mcpp 相对 CMake/vcpkg 的结构性优势:** +`[toolchain] default = "gcc@16.1.0"` 是可以进版本库的一行,payload 由 xim 保证 +逐字节相同。所以团队内部的 tag 组合数常常是 **1**。这让 Tier I 在私有场景里 +极其实用 —— 也让 Tier B(BMI)在私有场景里第一次变得**可能**(见 §6)。 + +### 4.10 非 mcpp 消费者(issue 场景 4) + +`mcpp pack ` 顺带产出: + +- **`.pc`(pkg-config)** —— 便宜,覆盖传统 `#include` + `-lfoo` 消费者; +- **CMake package config** —— 同上; +- **模块接口的互操作:诚实地说,没有好路。** CMake 的 C++20 modules 支持要求 + 消费方自己扫描并编译 `.cppm`,能做但很脆。**建议:对非 mcpp 消费者, + `mcpp pack ` 额外产出一份「非模块外观」(头文件 + `extern "C"` 或普通 + C++ 声明),由包作者显式声明,而不是自动生成。** 自动从模块接口生成头文件 + 是一个独立的、很大的题目,不该混进这个方案。 + +--- + +## 5. 分期与验收判据 + +> **P0/P1 的切分线是「形态」,不是「平台」。** 初稿按平台切(P0 只做 Linux), +> 那是错的:它让 Windows/macOS 用户在 P0 结束时拿到一个**空盒子**。 +> 按形态切之后,P0 三平台都产出能用的包,P1 才去解锁最难的动态形态。 + +### P0 —— `static` 形态,三平台一次做完(不依赖 xlings、不依赖 G2) + +1. `[distribution]` 段 + `sources = []` 的显式表达(G1、§4.2), + 以及 §4.5.0(d) 的「分发包目录里不许直接 build」守卫 +2. 包布局:根目录 `mcpp.toml` + `[distribution]` 段 + `[[runtime.artifacts]]`, + `lib/` 与 `bin/` 分离(§4.5)。**归档格式与源码包同构,索引条目一字不改。** +3. `mcpp pack `(位置参数;`kind = "lib"` ⇒ 静态库包)——**三平台**, + 含「从归档剔除已发布闭包的对象」(§4.5.2) +4. `mcpp add ./x.mpkg` + `{ package = "..." }` 依赖形态 +5. abi-tag 计算 + 闸门表(仅 arch/os/env/compiler/stdlib/std)(G4) +6. interface digest + 模块符号存在性检查(G5) + +**为什么 static 能在 P0 覆盖三平台:**`kind = "lib"` 没有平台限制(实测产出 +`bin/libstatlib.a`,`ar t` 显示接口对象与实现对象分离,`readelf -d` 无 RUNPATH), +所以 G2、RUNPATH 重写、闭包收集、DLL 部署**这一期全部不需要**。 + +**验收:** +- [ ] e2e:**三平台**各能 `mcpp pack ` 出一个包 并被消费者 `import` + 链接 +- [ ] e2e:`.mpkg` 里不含任何 `$MCPP_HOME` 路径(照抄 e2e 215;含元数据与 `interface/`) +- [ ] e2e:**篡改随包接口 → 构建必须失败**(直接把 §2.2(c) 那个 struct 互换 + case 变成回归测试 —— 它今天是静默通过的,这是最有价值的一条) +- [ ] e2e:abi-tag 不匹配 → 拒绝,且诊断里**列出可用 tag** +- [ ] e2e:同一个 `.mpkg` 在**清空 build cache 的第二台 MCPP_HOME** 上可消费 + (防止「只在写它的机器上能用」) +- [ ] unit:打包器确实剔除了接口单元对象(`ar t` / PE 等价物断言), + 而不是依赖「归档成员只在解析未定义符号时被拉取」这条巧合 +- [ ] `mcpp why` 能说出「这个依赖是 prebuilt / 形态是什么 / tag 是什么 / 为什么选了它」 + +### P1 —— `shared` 形态解锁三平台 + 索引通路 + +7. PE 导入库(`--out-implib` / `/IMPLIB:`)+ Mach-O `-install_name @rpath/…`, + 解除 `plan.cppm:1002` 的拒绝(G2) +8. `mcpp pack `(`kind = "shared"`):RUNPATH/INTERP 重写 + 闭包收集 + `bin/` 部署面 +9. **胖包**走 Form A 上索引:描述符与源码包同构(url + sha256), + `[target.'cfg(...)']` 在消费者构建期选 arch/OS —— **无新描述符键、无版本 floor** +10. 私有索引发布的文档与端到端验证 + +**验收:** +- [ ] 三平台各有一个 shared 库 e2e(今天全部 `# requires: elf`) +- [ ] Windows:消费者链 `lib/foo.lib`、`bin/foo.dll` 部署到 `.exe` 旁、**直接跑 `.exe`** + (不能用 `mcpp run`,它会塞 PATH 掩盖部署问题 —— 这是既有教训) +- [ ] shared 形态的 `.mpkg` 在第二台机器上可用(§2.2a 的回归守卫) +- [ ] `cxx_runtime` 契约不匹配时**有告警**(§2.2b 今天是静默的) +- [ ] **老 mcpp(2026.8.15.3)拿到这个包能构建成功** —— Form A 的兼容性主张必须 + 有一条对着**已发布二进制**跑的测试,不能只在新 mcpp 上验证 +- [ ] 胖包 + `--target aarch64-linux-gnu` 选到正确的那条腿(交叉编译回归) + +### P2 —— 交叉编译 + 私有鉴权 + +10. 瘦包(按 tag 分资产)+ `install_packages` 加 target 轴(跨仓库,与 xlings 同步) + —— **仅当胖包体积成为真问题时才做**,它不是能力前提 +11. `IndexSpec` 鉴权(值从环境变量读) +12. `target_cfg` 承载 LinkIntent(顺手修 #258 同形状的债) + +### P3 —— Tier B(BMI)与生态收尾 + +13. Tier B:导出/导入构建缓存条目,**必须**先做跨机器可行性实验(见 §6) +14. `.pc` / CMake config 产出 +15. 修 #304(`library_dirs` 的 link/runtime 分离收口)、#290(按版本区分构建规则) + +--- + +## 6. 明确不做 / 需要先证伪的事 + +**(a) 不要自动从模块接口生成 C 头文件。** 独立的大题目,混进来会把这个方案拖死。 + +**(b) 不要为二进制包发明第二套身份模型。** `(namespace, name)` + SPEC-001 不变, +tag 是**载荷选择器**,不是身份的一部分。把 arch 编进包名是明确的错误形状(§4.8)。 + +**(c) Tier B 不能凭推理设计进去 —— 必须先做实验。** 已知的三个坑: +- GCC 把**时间戳**写进 BMI(实测过),所以「BMI 逐字节相同」不能作为判据; +- GCC 把**源码路径**写进 BMI,跨机器路径不同; +- clang `--precompile` 发的是 **full BMI**(体积 16 倍且会让下游 TU 编错), + 两阶段必须 `-Xclang -emit-reduced-module-interface`。 + +**判据:在两台不同 `$MCPP_HOME`、不同用户名、不同路径的机器上,同一个 +build-key 的 BMI 能被对方直接使用并产出可运行程序。** 做不到就不要做 Tier B, +它是纯加速,不值得为它引入正确性风险。 + +**(d) 不要把 `cxx_runtime` 契约折进 abi-tag。** 它可以协商,tag 不可以。 +折进去会让 tag 组合数翻三倍,而且会把一个「警告」变成「找不到载荷」。 + +**(e) 不要指望闸门能挡住 ODR/布局漂移。** §4.4 已经说了。**唯一充分的保护是 +原子产出。** 任何「我们检查得够多了所以可以允许手工拼包」的说法都是错的。 + +--- + +## 7. 需要 review 的决策点 + +| # | 决策 | 我的建议 | 反方理由 | +|---|---|---|---| +| **D1** | P0 走**离线 `.mpkg` 文件**,还是直接做索引通路? | **离线文件**。不碰 xlings、不碰网络、不碰鉴权,完全在 mcpp 手里,而且正好是 issue 作者要的形状(`pip install x.whl`) | 索引通路才是「生态」,文件通路会不会变成永久的旁路 | +| **D2** | `kind = "prebuilt"` **新增目标种类**,还是只让 `sources = []` 合法? | **两个都要**,语义不同(§4.2) | 只加一个键更省;但 tag/digest 需要一个不说谎的挂点 | +| **D3** | abi-tag 的**粒度**:主版本 还是 完整版本? | **主版本**(`gcc16`)。完整版本会让 tag 数量爆炸,而 gcc 的 mangling/ABI 在主版本内稳定 | 主版本内也可能有 ABI 变化;可用 `--allow-abi-drift` 兜 | +| **D4** | `abi_surface = "c"` 逃生口要不要? | **要**。它让传统 C 库的 tag 数从 N×M 掉回 N,而且 `abi.cppm` 已经建模了这个规则 | 会被误用在实际有 C++ 接口的包上 → 需要 lint | +| **D5** | P0/P1 按**平台**切,还是按**形态**切? | **按形态**(修订自初稿)。P0 = `static`,三平台一次做完(`kind="lib"` 无平台限制,实测已验证);P1 = `shared`,解锁 PE 导入库 / Mach-O install_name | issue 明确写了 `.so/.dll/.dylib`,先给 static 会被读成「答非所问」—— 需要在回复里说清 static 消掉了哪三类问题 | +| **D9** | `artifact_kind` 是**布局里的字段**,还是**两个命令**? | **一个字段、一个命令、一个布局**。动态是静态的超集(多两步:RUNPATH 重写 + 闭包收集) | 两个命令更容易分别演进;但会产出两套布局与两套元数据,正是「一个决策两处推导」 | +| **D22** | `[pack]` 到底加几个键? | **0 个**(§4.6.1 推导审计)。WHAT/HOW → `[targets.].kind` + `mcpp pack [target]`;接口根 → `[lib]`;头目录 → `[build].include_dirs`;平台 → `[package].platforms` 升格为断言 | 同时发布静态+动态要声明两个目标,`.so` 文件名带后缀(soname 别名给出正确运行期名)。干净修法是 `kind` 接受列表(Cargo `crate-type`),独立改动 | +| **D23** | 头 / 接口允不允许在 pack 期裁剪? | **不允许**。源码分发把 `include_dirs` 全量暴露给消费者;二进制包裁掉一部分 ⇒ **同一个库因分发形式不同而公开面不同**。而且「哪些头公开」布局已经答了(`include/` vs `src/`)—— 私有头放在 `include/` 下是布局错误,不是打包问题 | 有作者确实想 curate;但那应该改布局,不是加打包旋钮 | +| **D18**(已被 D22 取代) | `[pack]` 要不要能「自定义接口文件」? | **部分要**(完整设计见 §4.6.1):WHAT/HOW 三个标量键(`default_kind` / `default_artifact` / `default_mode`)+ `targets` 列表;CONTENT 按**四个集合各给各的键**(`[pack.interface] modules`、`[pack.headers] dirs/exclude`、既有 `include/exclude` 保留给 extras)。**不加 `.cppm` 文件清单** | 作者会希望有「万能逃生口」;真要给,必须做成 `scan_overrides` 那种**断言+校验**,不是选择器 | +| **D20** | CONTENT 用**一对全局 `include`/`exclude`**,还是**每个集合一个键**? | **每个集合一个键**。四个集合的裁剪风险完全不同(interface 不可裁 / headers 可裁但必须 `-MM` 校验 / artifacts 不可裁 / extras 自由),共用一对键会让「删 README」和「删公开头依赖的私有头」看起来是同一个操作 | 键更多;但既有 `include`/`exclude` 语义**一字不改**(它们本来就只作用于 extras),所以不是破坏性变更 | +| **D21** | 「不写 = 默认」要不要配套**三态**(缺席/有值/显式空)? | **要**。实测 `sources = []` 与整行删掉逐字节等价(§2.5.4)——「不写=默认」的原则在没有三态时会退化成「无法表达空」。用 `XlingsConfig::subosDeclared` 那个模式 | 每个键多一个 `Declared` 布尔;但这是仓库已有的、被验证过的形状 | +| **D19** | 接口闭包建在 M1 文本扫描器还是 P1689? | **P1689**。实测 M1 不把 `module X:part;` 建模为 provider(`scanner.cppm:641-650`),导致「接口够到实现分区」的告警产不出来 | P1689 要跑编译器,更慢;但只在 `pack --lib` 时跑一次 | +| **D16** | 两种接口模式(`include/` 文本 / `interface/` 编译)用**目录**区分,还是用 manifest 里的键区分? | **目录**。规则由文件所在位置决定,不需要第二处声明,也不会漂移;实测两种模式互不干扰、可同时存在 | 目录名成为约定的一部分;作者若把 `.h` 放进 `interface/` 会被当成要编译的东西 —— 需要 lint | +| **D17** | `kind="shared"` 那道守卫的判据改成「是否动态链接」,单独开 issue 还是并进 P1? | **并进 P1**。它就是 G2 的一部分,而且不修的话 musl 用户拿到的是一条读不懂的链接器错误 | 也可以先只补诊断(便宜),真正支持留到 P1 | +| **D14** | §2.4.5 的裸三元组缺陷,单独开 issue 还是并进本方案? | **单独开**。它与二进制分发无关(任何用 `[target.''.build]` 的工程都中招),修法看起来是一行,但需要自己的回归测试 | 并进来能少一个 PR;但会把一个通用缺陷埋进一个大特性里 | +| **D15** | 接口发布集用**闭包**,还是让作者在 manifest 里手写清单? | **闭包**(mcpp 的 scanner 已有模块图,是免费的)。手写清单会漂移,而漂移的方向是**静默泄露源码** | 闭包对作者不可见;需要 `mcpp pack ` 打印「将发布 / 不发布」两张清单,并对「接口够到实现分区」告警 | +| **D11** | 包内自带 `mcpp.toml`(Form A),还是并列一份 `MCPP-PACKAGE.toml`? | **自带 `mcpp.toml`**(采纳,已重写 §4.5)。它消掉新描述符键 / 新消费路径 / 版本 floor,并让 path 依赖也吃到闸门 | 一个字面叫 `mcpp.toml` 的文件在二进制包里会引诱 `mcpp build` —— 需要 §4.5.0(d) 的守卫,否则是「看起来成功的失败」 | +| **D12** | 闸门+证据放同一段,还是拆两个文件? | **同一段**(`[distribution]`)。多一个文件就多一个能漂移的地方;代价是需要一条「用户手写生成字段 = 错误」的规则,而这条规则本来也要有 | 生成字段与用户字段混在一段里,靠规则而非结构区分 | +| **D13** | 胖包(一包多 arch,构建期选)做默认,还是瘦包(按 tag 分资产)? | **胖包**。它让交叉编译在 P1 就正确,**完全不需要动 xlings**;瘦包是 P2 的体积优化,不是能力前提 | 大型闭源 runtime × 4 tag 的体积;可两种并存 | +| **D10** | `static` 做默认形态? | **是**。对闭源内部分发,它消掉 RUNPATH 重写、DLL 部署、双 C++ 运行时、加载器闭包四类问题 | 大型 runtime 库可能就是要动态(热替换、体积);默认可被 `[pack] default_artifact` 覆盖 | +| **D6** | 安装线 target 轴(跨仓库)排 P2,可接受吗? | **可接受**,因为 P0/P1 用离线文件与显式 tag 绕开了它 | 交叉编译 + 索引二进制包在 P2 之前不可用 | +| **D7** | 鉴权:环境变量 header,还是接入 git credential helper? | **两条都走**:索引 clone 用 git 凭据(今天已经如此),artifact/URL 用 `header_env` | 有人会希望 mcpp 自己存 token —— 建议明确拒绝 | +| **D8** | §2.2(c) 那个静默错数据,要不要**先单独开 issue 并立刻加回归测试**? | **要**,不等整个方案。它今天就可达(path 依赖 + `[runtime] libraries` 是已发布能力) | 它需要 digest 机制才能真正修;但先把测试写成「预期失败」也有价值 | + +--- + +## 附录 A. 复现脚本 + +材料在 `scratchpad/bindist/`,三个工程: + +```bash +# ① 生产者:模块接口只有声明,实现在实现单元 +provider/src/runtimelib.cppm export module runtimelib; export namespace rt { ... } +provider/src/impl.cpp module runtimelib; namespace rt { ... } +provider/mcpp.toml [targets.runtimelib] kind = "shared" +mcpp build → bin/libruntimelib.so + +# ② 「二进制包」:接口源 + 预编译库 + mcpp.toml +dist/src/runtimelib.cppm (从 ① 拷贝) +dist/lib/libruntimelib.so (从 ① 拷贝) +dist/mcpp.toml [runtime] libraries / link_library_dirs / runtime_search_dirs + +# ③ 消费者 +app/mcpp.toml runtimelib = { path = "../dist" } +app/src/main.cpp #include <...> 然后 import runtimelib; +mcpp run → answer=42 / name=... / caught runtime_error + +# ④ skew 实验(本方案要挡的那个) +# 只把 dist/src/runtimelib.cppm 里的 struct Point { int x; int y; } +# 改成 struct Point { int y; int x; } +mcpp run → x=222 y=111 ← 编译链接运行全过,数据是错的,零诊断 + +# ⑤ 静态形态(scratchpad/statictest/):同样的两个源,[targets.x] kind = "lib" +mcpp build → bin/libstatlib.a +ar t libstatlib.a → statlib.m.o impl.o +nm --defined-only statlib.m.o: T _ZGIW7statlib ← 只有模块初始化器 + impl.o: T _ZN2slW7statlib6answerEv ← 真正的符号 +readelf -d libstatlib.a → (无动态段 ⇒ 无 RUNPATH) + +# ⑥ 兼容性边界(用已发布的 mcpp 2026.8.15.3 跑,不是用新构建) +# 在 statictest/mcpp.toml 末尾追加: +[distribution] schema/artifact_kind/abi_tag/abi_surface/interface_digest/modules + → mcpp build 成功,连警告都没有 ✅ +artifacts = [{ path = "…", role = "…" }] （array-of-tables 于新段） + → 硬失败:"array-of-tables syntax is only supported + for [[build.flags]], [[features..flags]], + [[runtime.requirements]], and [[runtime.artifacts]]" ❌ +[distribution] 标量段 + [[runtime.artifacts]]（既有白名单段） + → 成功,且 fingerprint 变化(证明它被读进 manifest) ✅ +``` + + +### 实验二(§2.4):`scratchpad/lab/` + +``` +lab/mkdist.py 176 行的 `mcpp pack ` 原型(采 tag / 算闭包 / 剔对象 / 生成 toml) +lab/parts/ 生产者:主接口 + 接口分区 + 实现分区 + 实现单元 + C API + 头 +lab/fatpkg/ 产出的胖包:interface/ include/ lib// mcpp.toml +lab/consumer/ 消费者(path 依赖 fatpkg) +lab/probe/ §2.4.5 的最小探针:裸三元组 vs cfg(linux) + +# ⑦ 三条被实验推翻/确立的规则 +python3 mkdist.py parts fatpkg x86_64-linux-gnu x86_64-linux-musl x86_64-windows-gnu + 接口闭包 (2 个): mathkit.cppm, api.cppm + 未发布 : secret.cppm ← 实现分区不外发(但它也有 .m.o) + 剔除归档成员: ['api.m.o', 'mathkit.m.o'] ← 不是所有 .m.o(第一版剔全部 → 三平台链接全挂) + +# ⑧ 全矩阵(胖包用 cfg(...) 谓词) +cd consumer && mcpp build [--target …] + (native) OK x86_64-linux-gnu OK x86_64-linux-musl OK x86_64-windows-gnu OK(PE32+) + build.ninja 中各自只出现自己那条腿的 fatpkg/lib/ + +# ⑨ 最小探针:[target..build] cxxflags = ["-DX"] 的命中次数 + 裸 mcpp build --target x86_64-linux-gnu + 'x86_64-linux-gnu'(裸三元组) 0 2 ← 缺陷 + 'cfg(linux)' 2 2 +``` + +### 实验三(§2.5):两种接口模式 × 两种库形态 + +``` +lab/parts/ 同时提供 include/mathkit_c.h(extern "C") 与 interface/*.cppm(模块) +lab/fatpkg/ 静态库包 lab/sopkg/ 动态库包 lab/hdrpkg/ 纯头文件包(无 .cppm) +lab/c_hdr/ 只 #include lab/c_mod/ 只 import lab/c_both/ 两者同时 + +# ⑩ 六格矩阵 —— 同一个包,三种消费方式 × 两种库形态 + 只 #include 只 import 两者同时 + 静态库 .a hdr: 5 mod: 5 both: c=5 mod=5 + 动态库 .so hdr: 5 mod: 5 both: c=5 mod=5 + 消费者 ELF: NEEDED libmathkit.so + runtime_search_dirs 进 RPATH ✅ + +# ⑪ musl + kind="shared":过了守卫,死在链接器 +mcpp build --target x86_64-linux-musl + crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' + can not be used when making a shared object + (守卫判据是 os != "linux";musl 是 linux ⇒ 过闸。真正的判据应是「是否动态链接」) + +# ⑫ sources = [] 不生效 —— 与「不写」逐字节等价 + 包里放一个遗留的 src/leftover.cpp: + sources = [] → build.ninja 里 leftover 出现 7 次 + (整行删掉) → build.ninja 里 leftover 出现 7 次 ← 完全一样 + ⚠️ 我一开始误以为它能用,因为测试包没有 src/ 目录 —— 默认 glob 也匹配不到东西。 + 「两条路径给出同一答案」不等于「这条路径生效了」。 + +# ⑬ 我自己踩的坑(仓库已知形状):mkdist 用 rglob 取产物 → 挑到陈旧 fingerprint 目录 + target/x86_64-linux-gnu/0f8ab572…/ 缺 capi.o ← rglob 取到的 + target/x86_64-linux-gnu/d02fd93d…/ 有 capi.o ← 刚构建出来的 + ⇒ 打包器绝不能 glob 产物,必须用它刚跑那次构建的 target///bin/… 路径。 +``` + +环境:mcpp 2026.8.15.3,gcc@16.1.0,x86_64-linux-gnu; +交叉 target 的载荷:`xim-x-musl-gcc/16.1.0`、`xim-x-mingw-cross-gcc/16.1.0`。 + +## 附录 B. 证据索引 + +| 事实 | 位置 | +|---|---| +| 描述符强制 `sources` | `src/manifest/xpkg.cppm:2058-2063` | +| 描述符 `mcpp` 段封闭键表(无 arch) | `src/manifest/xpkg.cppm:228-234` | +| 描述符 `target_cfg`(**已有** arch 条件构建输入) | `src/manifest/xpkg.cppm:1315-1400` | +| `target_cfg` 只承载 BuildInputs(无 LinkIntent) | `src/manifest/xpkg.cppm:1359-1377` | +| 描述符 `runtime` 子键(LinkIntent) | `src/manifest/xpkg.cppm:1925-1966` | +| `kind = "shared"` 仅 Linux | `src/build/plan.cppm:1002-1017` | +| `kind = "lib"` **无**平台限制,产出 `bin/lib` | `src/build/plan.cppm:411-423`(`target_output`) | +| **Form A:载荷自带 `mcpp.toml` 时 glob 加载它** | `src/build/prepare.cppm:2764-2787`(及 `2470` 的存在性探测) | +| LinkIntent **逐包**聚合(path/git/索引同一条路) | `src/build/plan.cppm:630-656` | +| legacy `library_dirs` 已**不**进 `linkLibraryDirs`(#304 的一半) | `src/build/plan.cppm:653-656` | +| array-of-tables 白名单(闸门段不能自造产物清单) | 实测,附录 A ⑥ | +| **裸三元组谓词在原生构建下不匹配**(新缺陷) | `src/build/prepare_inputs.cppm:139` vs `:49-53`;承诺见 `src/manifest/types.cppm:665` | +| cfg 词汇 = 规范 triple 词汇(os/arch/family/env + 别名) | `src/build/prepare_inputs.cppm:39-148` | +| MSVC 与 Itanium 的 cxxabi 分叉 | `src/toolchain/abi.cppm:84` | +| 接口闭包 / `.m.o` 不是判据 / 剔除集 | 实测,附录 A ⑦ | +| `[modules] exports` 是**完备性断言**(写错硬失败),不是选择器 | `src/modgraph/validate.cppm:93-117` | +| 既有 `[pack]` 键(新键的先例) | `src/manifest/types.cppm:736-745`;解析 `src/manifest/toml.cppm:1337-1347` | +| **`[pack].exclude` 只从 `include` 里剔除**(裁不到 headers) | `src/manifest/types.cppm:743` 注释原文 | +| `[pack.]` 子表先例 | `docs/02-pack-and-release.md` §Configuration(`[pack.bundle-project]`) | +| 三态模式(缺席 vs 显式空) | `src/manifest/types.cppm:632`(`subosDeclared`) | +| 「断言 + 校验」的先例(`scan_overrides`) | `src/manifest/types.cppm:62-70` | +| **M1 扫描器不把实现分区建模为 provider** | `src/modgraph/scanner.cppm:641-650`;告警 `:984` | +| P1689 扫描后端(闭包应建在它上面) | `src/modgraph/scanner.cppm:121-124` | +| 两种接口模式 × 两种库形态 六格全过 | 实测,附录 A ⑩ | +| `kind="shared"` 在 musl 上过闸后链接失败 | 守卫 `src/build/plan.cppm:1002`;实测 附录 A ⑪ | +| `sources = []` 被默认 glob 吞掉 | `src/manifest/toml.cppm:1703`;实测 附录 A ⑫ | +| 静态归档:接口对象与实现对象分离、无 RUNPATH | 实测,附录 A ⑤ | +| PE 需导入库(字段已存在) | `src/build/plan.cppm:452-479` | +| 平台轴只有 OS,无 arch | `src/platform/axis.cppm:63-87` | +| 安装线协议无 target/arch | `src/pm/package_fetcher.cppm:419-431` | +| 五维 ABI 模型 + `abi:=` | `src/toolchain/abi.cppm:31-145` | +| C 库只约束 libc(逃生口的依据) | `src/toolchain/abi.cppm:12-16` | +| ABI 完备的逐包 key(tag 的原料) | `src/build/cache_key.cppm:20-46, 76-133` | +| 构建缓存布局(BMI+obj 已在盘上) | `src/bmi_cache.cppm:1-45` | +| C++ 运行时契约三层模型 | `src/build/distribution.cppm:1-120` | +| SharedLibrary 双运行时危险 | `src/build/distribution.cppm:51-64`(role 注释)、`154-180`(`default_contract`) | +| 三平台同一份源码 tarball | `src/pm/publisher.cppm:289-303` | +| 「一份推导,多个投影」的既有纪律 | `src/pm/publisher.cppm:194-216` | +| 私有索引无鉴权字段 | `src/pm/index_spec.cppm:14-57` | +| lock 不 pin 索引依赖、不记 ABI | `mcpp.lock` 头部注释 / `src/pm/lock_io.cppm:29-36` | +| pack 的输入是一个 exe | `src/pack/pack.cppm:Plan::builtBinary` | +| anchor TU 手法样板 | `~/.mcpp/registry/data/mcpplibs/pkgs/c/compat.openblas.lua` | +| xim V2 的 arch 轴(按宿主解析) | `~/.mcpp/registry/data/xim-pkgindex/docs/V2/xpackage-spec.md` §"三种新版本条目形状" | +| LinkIntent 的 link/runtime 分离表 | `docs/05-mcpp-toml.md` §2.11 | +| 新键需要版本 floor 且必须降级 | `docs/10-publishing-a-library.md` §"Manifest keys that need a version floor" | +| schema 准入原则 | `docs/05-mcpp-toml.md` Appendix A | diff --git a/.agents/docs/2026-08-17-library-distribution-design.md b/.agents/docs/2026-08-17-library-distribution-design.md new file mode 100644 index 00000000..bb2d8897 --- /dev/null +++ b/.agents/docs/2026-08-17-library-distribution-design.md @@ -0,0 +1,784 @@ +# 库分发:`mcpp pack ` 与二进制包(2026-08-17) + +> **这份是方案。** 发现(现状、实测、file:line)在 +> `2026-08-17-distribution-architecture-analysis-and-design.md`;那份文档的 +> §2(五轮实测)是本方案每一条判据的证据来源,本文只在需要时回指。 +> +> **实施状态(2026-08-17):P0 已实现并开 PR #451(未合入)。** +> 落地与设计的三处差异,以及实现过程中被实测推翻的四条,记在 §11。 +> +> 起因:issue #433「预编译 .so + .ixx/.h/.cppm 接口」。 +> 讨论过程中我有**四处设计被自己的实测推翻**,都记在 §9,因为**错的那版看起来同样合理**。 + +--- + +## 0. 一页纸 + +**模型三句话:** + +1. **二进制包不是新格式** —— 它是一个**自带 `mcpp.toml` 的普通 tarball**,走 mcpp 已有的 + Form A 通路(`prepare.cppm:2764`)。索引条目与源码包**一字不差**。 +2. **产出什么由 `[targets.].kind` 决定** —— `mcpp pack `。 + 没有 `--lib`,没有 `--artifact`。**新增 manifest 段 0 个、键 0 个** —— + 生成的包就是一个**普通的 `mcpp.toml`**(§2.4)。 +3. **能算出来的东西不给字段** —— 接口发布集是**模块闭包**,公开头是 `include_dirs` 全量, + 两者都**不允许**在 pack 期裁剪(裁了会让源码分发与二进制分发语义不同)。 + +**四条不可让步的判据:** + +| # | 判据 | 违反会怎样(实测) | +|---|---|---| +| **J1** | 接口与二进制**原子产出、不可分别替换** | 随包 `.cppm` 改一行结构体字段序 ⇒ 编译链接运行全过、**数据是错的、零诊断** | +| **J2** | 产物里**不含生产机器的绝对路径** | `.so` 烧着 `/home/…/.mcpp/…` 的 RUNPATH,换机器即废 | +| **J3** | 发布集 = **从公开根算的模块闭包**,不是 `.m.o` | 实现分区照样产 `.m.o` ⇒ 按 `.m.o` 选会**泄露闭源源码** | +| **J4** | per-target 的 `ldflags` 用 `cfg(...)` **不用裸三元组** | 裸三元组在原生构建下不匹配 ⇒ **CI 全绿、本机静默失配** | + +--- + +## 1. 模型 + +### 1.1 两种接口模式,两条规则,由目录决定 + +``` +pkg/ +├── mcpp.toml ← 生成物:一个普通的 mcpp.toml(§2.4),没有新段 +├── include/ ← 文本接口:原样全发,消费者 #include,不产生任何对象 +├── interface/ ← 模块接口:算闭包,消费者编译它得到 BMI + object +├── lib// ← 链接面:.a / .so / .dylib / .lib(PE 导入库) +├── bin// ← 运行面(仅 PE):.dll,部署到 .exe 旁 +├── HOST-REQUIREMENTS ← 与 mcpp pack 同一份推导(仅当有内容要说) +└── LICENSE / THIRD-PARTY +``` + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | 预处理器 | **编译器** | +| 消费者要编译吗 | 否 | **是** | +| 要算闭包吗 | 否 | **是**(J3) | +| 要从归档剔对象吗 | 否 | **是** | +| ABI 闸门 | `extern "C"` ⇒ tag 只有三段(只约束 libc) | tag 全段 | + +**两种模式互不干扰、可同时存在** —— 实测:同一个包被「只 `#include`」/「只 `import`」/ +「两者同时」三种方式消费 × 静态/动态两种形态,**六格全过**。 + +**`lib/` 与 `bin/` 分开是机制不是风格**:PE 上链接的文件(`.lib` 导入库)与运行的文件 +(`.dll`)是两个;ELF/Mach-O 是同一个。 +**按三元组分目录不按 OS 分**:MinGW 与 MSVC 同为 windows,一个产 `lib*.a` 一个产 `*.lib`。 + +### 1.2 两个兼容量,不是一个 + +| | `abi_tag` | `build_key` | +|---|---|---| +| 回答 | 「这份二进制能不能链进你的构建」 | 「这份 BMI 能不能直接用」 | +| 谁能算 | **生产者**(消费者存在之前就能枚举) | **只有消费者**(含依赖闭包 Merkle) | +| 来源 | `mcpp build --print-fingerprint` 的 [1][2][4][5][6] 的**纯投影** | `cache_key::key_hex`(已存在) | + +``` +-----c++ + +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 +aarch64-macos-none-llvm22-libcxx22-c++23 +x86_64-windows-msvc-msvc194-msvcstl194-c++23 +``` + +⚠️ **`arch-os-env` 必须用 `triple.cppm` 的规范拼写,不是 fingerprint 的 [4]** —— +[4] 是编译器自报的(`x86_64-w64-mingw32`),与 `[target.'…']` 键的拼写 +(`x86_64-windows-gnu`)不同,直接用会让同一个决定有两个拼写。 + +**纯 `extern "C"` 库发一个更短的 tag** —— `x86_64-linux-gnu`,没有 +compiler/stdlib/std 段。闸门按段比对 tag 里**有**的东西,而 `abi_check` +(`abi.cppm:157`)早就是「未指定的维度 = 不关心」。**tag 的形状就是 surface**, +不需要额外的开关;C 库的 tag 组合数因此天然从 N×M 掉回 N。 + +### 1.3 三层分发,层级由消费端选 + +| Tier | 包里有什么 | 生效条件 | 失配时 | +|---|---|---|---| +| **S** source | 全部源码 | 永远 | —— | +| **I** interface+binary | 接口 + 预编译库 | `abi_tag` 匹配 | 降级到 S;无 S 则**明确拒绝并列出可用 tag** | +| **B** +BMI | 再加预编译 BMI | `build_key` 精确匹配 | **静默**降级到 I | + +Tier B 必须先做跨机器可行性实验才能实施(GCC 把时间戳与源码路径写进 BMI), +**它只能是加速器,永不成为正确性依赖**。 + +--- + +## 2. 生产者侧 + +### 2.1 命令 + +```bash +mcpp pack # 唯一可打包目标;有歧义 → 报错并列出候选 +mcpp pack mathkit # kind = "lib" → 静态库包 +mcpp pack mathkit-shared # kind = "shared" → 动态库包 +mcpp pack myapp # kind = "bin" → 应用 bundle(既有四档 --mode) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 胖包(--target 可重复) +``` + +**新增 CLI:一个位置参数(与 `mcpp run [target]` 同形)+ `--target` 可重复。** +**新增 manifest 键:0 个。** + +| `[targets.].kind` | 产出 | `--mode` 适用 | +|---|---|---| +| `bin` | 应用 bundle | ✅ 既有四档 | +| `lib` | 静态库包 | ✅ **`system`(默认)/ `vendored`** —— 见下 | +| `shared` | 动态库包 | ✅ `system` / `vendored` / `self-contained` | + +#### 2.1.1 ⚠️ 依赖跟不跟着走,静态库同样要选(修正) + +本文前一版写「静态库的 `--mode` 不适用,归档不携带任何东西」。**那句是错的。** +「归档不携带依赖的代码」恰恰**是**问题所在:消费者必须自己把 `zlib` 拉进来 —— +除非包把它一起带上。这就是 `--mode` 问的那件事,只是机制随形态不同: + +| `--mode` | 库包里有什么 | 消费者要什么 | 适合 | +|---|---|---|---| +| **`system`**(默认) | 只有本库 + `[dependencies]` 声明 | 自己解析依赖(公开索引 / 私有索引) | 依赖都是公开包 | +| **`vendored`** | 本库 + **依赖的产物一起进 `lib//`**,包内不声明 `[dependencies]` | 什么都不要 | 内网 / 离线 / 依赖本身也是闭源 | +| `self-contained` | 仅 `shared`:再带上 libc 一侧的闭包 | 什么都不要 | 跨发行版 | + +**与应用的 `--mode` 是同一个问题的同一批答案**:闭包跟不跟着走。机制不同 +(应用问的是运行期 `.so`,静态库包问的是构建期可链接的产物),但问题相同, +所以复用既有词汇而不是发明第二套。 + +`vendored` 形态下,依赖产物在 `[[runtime.artifacts]]` 里各占一条, +`provenance` 记它**来自哪个包**(`"vendored compat.zlib@1.3.2 by mcpp-pack 2026.8.17.1"`), +生成的 `ldflags` 按拓扑序把它们排在本库之后。 + +**分期**:`system` 是 P0(当前实现自然就是它),`vendored` 是 P1。 + +### 2.2 生产者工程示例(全部是既有的键) + +```toml +# examples/05-lib-distribution/producer/mcpp.toml +[package] +name = "mathkit" +version = "0.1.0" +description = "Demo: shipping a prebuilt library with both header and module interfaces" +license = "Apache-2.0" +platforms = ["linux", "windows"] # 既有键。pack 用它做覆盖校验(缺腿告警) + +[build] +sources = ["src/*.cppm", "src/*.cpp", "src/*.c"] +include_dirs = ["include"] # 公开头。整发,不可裁 + +# [lib] path 不写 ⇒ src/mathkit.cppm 约定 ⇒ 模块闭包从这里出发 + +[targets.mathkit] +kind = "lib" # → mcpp pack mathkit 产出静态库包 + +[targets.mathkit-shared] +kind = "shared" # → mcpp pack mathkit-shared 产出动态库包 +soname = "libmathkit.so.1" # 给出正确的运行期名 + +[pack] +include = ["share/**"] # 既有键 —— 只作用于 extras +``` + +``` +examples/05-lib-distribution/producer/ +├── mcpp.toml +├── README.md +├── include/mathkit_c.h # extern "C" 头接口 +└── src/ + ├── mathkit.cppm # export module mathkit; export import :api; ← 闭包根 + ├── api.cppm # export module mathkit:api; ← 闭包内 + ├── secret.cppm # module mathkit:secret; ← 实现分区,**不发布** + ├── impl.cpp # module mathkit; import :secret; + └── capi.c # C API 实现 +``` + +### 2.3 `mcpp pack ` 做什么 + +``` +1. build 出产物 ⚠️ 用这次构建的 target///bin/… 路径, + 绝不 glob(会挑到陈旧的 fingerprint 目录) +2. 算接口闭包(从 lib-root 出发) ⚠️ 走 P1689,不走 M1 文本扫描器(§9-D) + → 拷进 interface/ + → 打印「将发布 / 不发布」两张清单 + → 闭包里出现实现分区 ⇒ 告警「它的源码会被发布」 +3. 拷 include_dirs 全量 → include/ +4. 从归档剔除**已发布闭包那些单元**的对象 ⚠️ 不是「所有 .m.o」(J3 同一个闭包的第二个用途) +5. artifact_kind = shared 时额外: + · RUNPATH / INTERP 重写($ORIGIN 化) ← J2 + · 第三方闭包收集(ELF: LD_TRACE;PE: 导入表) + · PE: 导入库进 lib/,DLL 进 bin/ +6. 生成包内 mcpp.toml(§2.4)+ HOST-REQUIREMENTS +7. 确定性归档(PE → .zip,其余 → .tar.gz) +``` + +**计划阶段必须校验的组合**(不能留到链接期;`--mode` 与形态的组合见 §2.1.1): + +| target | `kind = "lib"` | `kind = "shared"` | +|---|---|---| +| `x86_64-linux-gnu` | ✅ | ✅ | +| `x86_64-linux-musl` | ✅ | ❌ **musl 蕴含 `-static`,产不出共享库** | +| `x86_64-windows-gnu` / `-msvc` | ✅ | ❌ P1 解锁(PE 导入库) | +| `*-macos` | ✅ | ❌ P1 解锁(install_name) | + +### 2.4 包内 `mcpp.toml` —— 一个普通的 mcpp.toml,没有新段 + +> **本节是第三次重写。** 前两版分别提了 `MCPP-PACKAGE.toml` 和 `[distribution]` 段。 +> 逐条追问「这件事别处已经能说了吗」之后,**两者都被删掉**:每一条事实都有既有字段。 + +#### 2.4.1 每条事实的归属 + +| 曾经想新增的 | 归属(全部既有) | +|---|---| +| `artifact_kind`(static/shared) | `[[runtime.artifacts]].role` = `static-library` / `shared-library` | +| `abi_tags`(包级列表) | `[[runtime.artifacts]].abi` —— **每条腿一个,比包级列表更准** | +| `abi_surface`(c/cxx) | **tag 自身的形状**(见 2.4.2) | +| `modules` | `[modules] exports`(既有的完备性断言) | +| `cxx_runtime` | `[build] cxx_runtime`(既有) | +| `interface_digest` | `[[runtime.artifacts]]` 里 `role = "interface"` 的条目,**逐文件一条** | +| `built_by` | `provenance = "mcpp-pack "` | +| `build_key` | `host_fingerprint`(docs/05 §2.11 原文:*optional evidence*) | +| 「这是分发包」的标记 | `provenance` 以 `mcpp-pack` 开头 —— **可推导** | +| `schema` | 不需要:没有新 schema 要版本化 | + +**实测**(mcpp 2026.8.15.3):`role = "interface"`、任意 `abi` 串、`digest`、 +`provenance` 全部原样进 `resolution.json`,并挂上既有的 `identity` 判定 +(路径不存在 ⇒ `missing`)。**兼容性比新段更好** —— 新段会被老客户端静默跳过 +(等于没有记录),而 `[[runtime.artifacts]]` 是老客户端**已经在读**的段。 + +#### 2.4.2 `abi_surface` 消失了:tag 的**形状**就是 surface + +`abi.cppm:157` 的 `abi_check` 早就是「**未指定的维度 = 不关心**」。所以不需要一个 +布尔来说「我只约束 libc」—— **发一个短 tag 就是在说这件事**: + +``` +纯 extern "C" 库 abi = "x86_64-linux-gnu" ← 只有三段 +C++ 模块库 abi = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" ← 全段 +``` + +闸门按**段**比对 tag 里有的东西。C 库的 tag 组合数因此天然从 N×M 掉回 N, +不需要任何开关。 + +#### 2.4.3 生成的包内 `mcpp.toml` —— 描述只有三件事 + +**一个二进制包的描述应当和源码包一样、甚至更简单。核心就三样:接口 / 库 / 依赖。** +其余的都不是「描述」,是**证据**,而证据挂在既有的 `[[runtime.artifacts]]` 上, +每条腿一条 + 接口一条,不是每个文件一条。 + +```toml +# ══ 由 `mcpp pack mathkit` 生成。手工编辑会使 interface 的 digest 失配并被拒绝。 ══ +[package] +namespace = "acme" +name = "mathkit" +version = "0.1.0" + +# ── ① 接口 ──────────────────────────────────────────────────────── +[build] +sources = ["interface/mathkit.cppm", "interface/api.cppm"] # 模块接口:消费者编译它 +include_dirs = ["include"] # 头接口:消费者 #include +cxx_runtime = "self-contained" + +[modules] +exports = ["mathkit"] + +[targets.mathkit] +kind = "lib" + +# ── ② 库(每条腿一段;必须是 cfg(...),不能是裸三元组 —— J4)────────── +[target.'cfg(all(linux, not(env = "musl")))'.build] +ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + +[target.'cfg(all(linux, env = "musl"))'.build] +ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] + +[target.'cfg(windows)'.build] +ldflags = ["-Llib/x86_64-windows-gnu", "-lmathkit"] + +# ── ③ 依赖(从生产者工程原样带过来)──────────────────────────────── +[dependencies.compat] +zlib = "1.3.2" + +# ── 证据:每条腿一条 + 接口一条 ──────────────────────────────────── +[[runtime.artifacts]] +role = "static-library" +path = "lib/x86_64-linux-gnu/libmathkit.a" +provenance = "mcpp-pack 2026.8.17.1" # 前缀即「这是分发包」的标记 +abi = "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" +digest = "sha256:…" +host_fingerprint = "aeb4c4d29e437696" # build_key,Tier B 用 + +[[runtime.artifacts]] +role = "interface" +path = "interface" # 目录,一条即可 —— 不是每个文件一条 +provenance = "mcpp-pack 2026.8.17.1" +digest = "sha256:…" # 对有序文件集的摘要 +``` + +**⚠️ ③ 依赖是必须的,而且容易漏。** 一个静态库的 `.a` **不携带**它的第三方依赖 —— +消费者链接时必须自己把 `zlib` 拉进来。所以 `mcpp pack` 要把生产者的 +`[dependencies]` 原样写进包(**排除 dev-dependencies 与 path 依赖** —— +后者是本地的,发出去解析不了)。这条规则 `emit_xpkg` 已经在用 +(`publisher.cppm:186-192`),**同一份推导,第二个投影**。 + +`shared` 形态另外多一段: + +```toml +[runtime] +runtime_search_dirs = ["lib/x86_64-linux-gnu"] # 进消费者的 RPATH +# deploy_files = ["bin/x86_64-windows-gnu/mathkit.dll"] # PE:部署到 .exe 旁 +``` + +#### 2.4.4 为什么接口 digest 只有一条 + +**它防的是「解开之后有人改了随包的接口」**,不是「生产者一开始就发错了配对」—— +后者只有原子产出能防(§0 J1)。索引通路上,整包已经有 sha256;真正裸奔的是 +**path 依赖 / 已解开的 store**,而那里一条目录级 digest 就够: +诊断说「`interface/` 与打包时不一致,拒绝构建」已经是可行动的。 +逐文件 digest 能多说一句「是哪个文件」,代价是描述里多出 N 行 —— 不值得。 + +### 2.5 分发包目录里不许直接 build + +判据:**任一 `[[runtime.artifacts]]` 的 `provenance` 以 `mcpp-pack` 开头**。 +此时在解开的包目录里直接 `mcpp build` **必须拒绝**,并说清这是分发包不是源码树。 +今天它会**成功** —— 把 `interface/` 里只有声明的接口单元编出来,产出一个几乎空的库, +实现全在预编译产物里没被链进来。典型的「看起来成功的失败」。 + +--- + +## 3. 消费者侧 + +### 3.1 三种依赖形态,同一条下游路径 + +```toml +# ① 离线文件 —— P0。不需要索引、网络、鉴权,不需要动 xlings +mathkit = { package = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23.tar.gz" } + +# ② 索引(公开或私有)—— P1。描述符与源码包同构:只有 url + sha256 +mathkit = "0.1.0" + +# ③ 本地目录(内部团队最常走的一条) +mathkit = { path = "../mathkit-dist" } +``` + +```bash +mcpp add ./mathkit-0.1.0-.tar.gz +``` + +**⚠️ 闸门必须对 ③ 也生效** —— 这就是闸门字段要放进包内 `mcpp.toml` 而不是旁路文件的理由。 + +### 3.2 闸门表(顺序即诊断顺序) + +| 检查 | 失配 | +|---|---| +| arch / os / env | **拒绝** —— 载荷本身不对 | +| `abi_surface == "c"` | 上一行之后全部跳过 | +| compiler 族 / 主版本 | **拒绝**;`--allow-abi-drift` 强制并打印它保护的是什么 | +| stdlib id / 主版本 | **拒绝** | +| C++ 标准档位 | 消费者 < 生产者 → **拒绝**;> → 放行 | +| `cxx_runtime` 契约 | **警告** + 说清双运行时危险;`--strict` 升级为错误 | +| `interface_digest` | **拒绝** | +| 模块符号存在性(ELF `_ZGIW` / PE 导出表) | **拒绝** | +| `build_key`(Tier B) | **静默**降级到 Tier I | + +### 3.3 没有匹配 tag 时的诊断 + +``` +error: acme.mathkit@0.1.0 has no prebuilt artifact for this toolchain + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + aarch64-linux-gnu-gcc15-libstdcxx15-c++23 + note: this package ships no source tier, so there is nothing to fall back to. + fix : ask the publisher for a gcc16 build, or pin [toolchain] to gcc@15. +``` + +**「不可用」不能被拼成「不存在」** —— 被拼成「找不到包」的失败会让客户端自己驱动 +重复刷新索引(#349 的教训)。 + +### 3.4 兼容性:老客户端拿到这个包会怎样 + +**能构建,只是没有闸门。** 实测(mcpp 2026.8.15.3): +包内 mcpp.toml 用的 `sources` / `[build]` / `[modules] exports` / +`[target.'cfg(…)']` / `[[runtime.artifacts]]` **全部是已发布能力** —— 老客户端 +逐字读得懂,只是不会**执行闸门**(它不知道 `provenance = "mcpp-pack"` 意味着要校验 +digest 与 abi tag)。这是**降级**而不是变砖 —— +但**闸门只保护新客户端,必须写进发布说明**。 + +这也是不新增段的第二个好处:一个**新**段会被老客户端静默跳过,连记录都没有; +而 `[[runtime.artifacts]]` 是老客户端**已经在读、并且会写进 `resolution.json`** 的段, +所以即使闸门不执行,证据仍然落盘、仍然可审计。 + +--- + +## 4. `docs/` 文档计划 + +| 文件 | 动作 | 内容 | +|---|---|---| +| `docs/02-pack-and-release.md` | **改** | 标题改为「打包:应用与库」。新增「§库分发」:`mcpp pack ` 的 kind 表、两种接口模式的目录规则、包布局、跨平台合法组合表。既有的应用四档 `--mode` 一字不改 | +| `docs/12-binary-distribution.md` | **新增** | 库作者的完整链路:写工程 → `mcpp pack` → 检查两张清单 → 分发(文件/私有索引/公开索引)→ 消费者怎么用。含 §2.2 与 §3.1 的完整示例 | +| `docs/05-mcpp-toml.md` | **改** | ① `[pack]` 一节补「只作用于 extras,裁不到头与接口」;② §2.11 补 `role = "interface"` 与 `provenance = "mcpp-pack"` 的约定用法(生成物,不是手写配置);③ Appendix A 补一条准入判据:「能从别处推出来的不给字段」 | +| `docs/10-publishing-a-library.md` | **改** | 新增「发布二进制包」小节:与源码包**同构**的索引条目、`platforms` 覆盖校验、老客户端降级说明 | +| `docs/03-toolchains.md` | **改** | `abi_tag` 的六个成分与 `--print-fingerprint` 的对应关系 | +| `docs/zh/*` | **同步** | 上述五处的中文版 | + +--- + +## 5. `examples/` 具体示例 + +沿用既有编号与「每目录一个 README.md」的约定。 + +### `examples/05-lib-distribution/producer/` —— 生产者(库作者) + +结构见 §2.2。README 要点: + +- 一个工程**同时**提供头接口与模块接口,消费者可以只用其中一种; +- `secret.cppm` 是实现分区,**不会被发布** —— 跑 `mcpp pack mathkit` 看两张清单; +- 两个目标(`lib` + `shared`)如何同时发布,以及 `soname` 给出的正确运行期名; +- **反面演示**:把 `src/mathkit.cppm` 改成 `import :secret;`,再 pack, + 观察闭包里多出 `secret.cppm` 并触发告警。 + +### `examples/05-lib-distribution/consumer/` —— 消费者 + +``` +examples/05-lib-distribution/consumer/ +├── mcpp.toml # mathkit = { path = "../producer/target/dist/mathkit-0.1.0-" } +├── README.md +└── src/ + ├── main_header.cpp # 只 #include + ├── main_module.cpp # 只 import mathkit; + └── main_both.cpp # 两者同时 +``` + +README 要点:三种消费方式 × 静态/动态两种包 = 六格,全部可跑; +以及**篡改 `interface/` 后必须被拒绝**的演示。 + +### `examples/07-lib-dist-fat/` —— 胖包与交叉 + +一个 `mcpp.toml` + 一条命令产出三条腿,消费者用 `--target` 选。README 要点: + +- 为什么每条腿的 `ldflags` 是 `cfg(...)` 而不是裸三元组(J4,附最小探针); +- `lib/` 为什么按三元组分目录而不按 OS 分。 + +--- + +## 6. CI:验证矩阵就是 e2e 集合本身 + +> **相对初稿的修订。** 初稿提议新建 `pack-dist-matrix.yml`。**不需要。** +> `tests/e2e/run_all.sh` 已经有能力探测(`elf` / `gcc` / `mingw-cross` / +> `fresh-sandbox` / …)并按 `# requires:` 分流,而 `ci-linux-e2e.yml` 已经在跑 +> 整个 `tests/e2e/`。再建一条并行流水线就是**同一个决策的第二处推导**。 +> +> 矩阵是 e2e 集合 + 每个测试头部那行 `# requires:`。 + +### 6.1 落地的测试与它们钉住的判据 + +| e2e | `# requires:` | 钉住 | +|---|---|---| +| **242** `pack_library_interface_and_headers` | `gcc` | 两种接口模式共存;包布局;两张清单都被打印 | +| **243** `pack_library_interface_closure` | `gcc` | **J3** —— 实现分区源码不外发 **且** 它的对象留在归档里 | +| **244** `pack_library_gate` | `gcc` | **J1** 接口篡改被拒 + tag 失配被拒(且列出可用 tag)+ 包内 build 被拒 | +| **245** `pack_library_fat_target_selection` | `gcc` | **J4** —— 胖包每条腿只被自己的 target 看到,**含原生构建** | +| **246** `explicit_empty_sources` | `gcc` | §8-B2 —— `sources = []` 与「不写」可区分 | +| **247** `bare_triple_conditional_native` | `gcc` | §8-B1 —— 裸三元组谓词在原生构建下命中 | +| **248** `pack_library_fat_pe_leg` | `gcc mingw-cross` | 跨 OS 边界的腿(PE);tag 用规范三元组而非编译器自报 | + +### 6.2 ⚠️ 一处差点造出来的假绿 + +245 最初写成 `# requires: gcc mingw-cross`(gnu + musl + windows 三条腿)。 +**`ci-linux-e2e.yml` 只预热 gcc 与 musl,不装 mingw-cross** —— +那条测试会在每一次普通 CI 上**静默跳过**,于是胖包这一整套机制 +(以及 J4)在绿色的套件里**从未被验证过**。 + +拆法:**核心机制用 CI 一定有的工具链**(245:gnu + musl,`requires: gcc`), +**只把新增的二进制格式覆盖单列**(248:PE,`requires: mingw-cross`)。 +判据:**一个测试的 `# requires:` 必须是它所验证机制的真实下限,不是它能跑的上限。** + +### 6.3 单元测试 + +| 文件 | 覆盖 | +|---|---| +| `tests/unit/test_pack_abi_tag.cpp` | tag 的投影 / 规范三元组 / C 表面短 tag / 从尾解析 / 档位下限 / 一次报全部失配(15 例) | +| `tests/unit/test_pack_interface.cpp` | 闭包 / 剔除集不是「所有 `.m.o`」/ 依赖模块不越界 / 未解析分区报错(8 例) | + +### 6.4 仍需在 CI 上补的(P1) + +- macOS 与 Windows 的 e2e 分片会自动跑 242/243/244/246/247(它们只 `requires: gcc`), + 但**尚未在这两个平台上人工确认过**; +- `kind = "shared"` 的库包 e2e(P1,随 PE 导入库 / Mach-O install_name 一起); +- `--target` 覆盖与 `[package].platforms` 的比对告警(P1)。 + +## 7. 分期 + +### P0 —— `static` 形态,三平台一次做完(不依赖 xlings、不依赖 shared) + +1. `sources = []` 的三态(§8-B2)+ §2.5 的 build 守卫(判据 = `provenance` 前缀) +2. 包布局 + 生成包内 `mcpp.toml` +3. `mcpp pack ` 位置参数 + `--target` 可重复;`kind = "lib"` ⇒ 静态库包 +4. 接口闭包(P1689)+ 两张清单 + 实现分区告警 +5. 从归档剔除已发布闭包的对象 +6. `abi_tag` 计算 + 闸门表 + `interface_digest` + 模块符号存在性 +7. `mcpp add ./x.tar.gz` + `{ package = "…" }` 依赖形态 +8. e2e 242 / 244 / 245 / 246 / 248 / 250 / 251;examples 05 + 06 + +**为什么 static 能在 P0 覆盖三平台**:`kind = "lib"` 无平台限制, +所以 G2、RUNPATH 重写、闭包收集、DLL 部署这一期全部不需要。 + +### P1 —— `shared` 形态解锁三平台 + 索引通路 + +9. PE 导入库(`--out-implib` / `/IMPLIB:`)+ Mach-O `-install_name @rpath/…` +10. `kind = "shared"` 守卫的判据改成「该 target 是否动态链接」(musl 走计划期拒绝) +11. `shared` 形态的 pack:RUNPATH 重写 + 闭包收集 + `bin/` 部署面 +12. 胖包上索引(描述符与源码包同构)+ `platforms` 覆盖校验 +13. e2e 243 / 247 / 249;examples 07;`pack-dist-matrix.yml` + +### P2 —— 交叉与私有鉴权 + +14. 瘦包(按 tag 分资产)+ `install_packages` 加 target 轴(跨仓库,与 xlings 同步) +15. `IndexSpec` 鉴权(值从环境变量读,永不落盘) +16. `target_cfg` / `[target.'cfg(…)']` 承载 LinkIntent(顺手修 #258 同形状的债) + +### P3 —— Tier B 与生态收尾 + +17. Tier B:**先做跨机器可行性实验**,做不到就不做 +18. `.pc` / CMake config 产出 +19. #304(`library_dirs` 的 link/runtime 分离收口)、#290(按版本区分构建规则) + +--- + +## 8. 需要先单独修的既有缺陷(不属于本方案,但阻塞它) + +| # | 缺陷 | 位置 | 对本方案的影响 | +|---|---|---|---| +| **B1** | `[target.'<三元组>'.build]` 在原生构建下**永不命中** | `prepare_inputs.cppm:139` `if (triple.empty()) return false;`,而同文件 `context_for()` 对 `cfg()` 回落到 `host_triple()`;`types.cppm:665` 的注释承诺的是回落 | 直接阻塞胖包的裸三元组写法(J4)。**任何用该写法的工程都中招**,形状是「CI 全绿、本机静默失配」 | +| **B2** | `sources = []` 与整行删掉逐字节等价 | `toml.cppm:1703` 的默认 glob 吞掉显式空 | 二进制包无法表达「什么都不要编」;遗留在 `src/` 的文件会被编进消费者的构建 | +| **B3** | M1 扫描器不把实现分区建模为 provider | `scanner.cppm:641-650` 对非 `export` 的 `module X;` 从不设 `u.provides` | 「接口够到实现分区」的告警产不出来 ⇒ 闭包必须走 P1689 | +| **B4** | `kind = "shared"` 的守卫判据是 `os != "linux"` | `plan.cppm:1002` | musl 过闸后死在 `crtbeginT.o`,消息里既没有 musl 也没有 shared | + +B1 / B2 建议**先单独开 issue 并各带一条回归测试**,不要埋进这个大特性里。 + +--- + +## 9. 四处被自己的实测推翻的设计(不要重新提出来) + +| # | 我写过的 | 被什么推翻 | 正确的 | +|---|---|---|---| +| **A** | 新增描述符键 `kind = "prebuilt"` + 版本 floor | 它属于「不降级」的那一类,老客户端会被砖 | 走 **Form A**(包自带 `mcpp.toml`)⇒ 无新描述符键、无 floor、兼容几乎免费 | +| **B** | 打包时「剔除所有 `.m.o`」 | **三个 target 全部链接失败**(`undefined reference to mk::secret_helper@mathkit()`)—— 实现分区照样是 `.m.o` 且里面是真代码 | 剔除集 = **已发布闭包里那些单元**的对象;与发布集是**同一个闭包的两个用途** | +| **C** | 胖包每条腿用裸三元组 `[target.'x86_64-linux-gnu'.build]` | 显式 `--target` 三个全绿,**裸 `mcpp build` 链接失败**;最小探针:裸三元组 0 命中 / `cfg(linux)` 2 命中 | 一律用 `cfg(...)`(J4);并把根因(B1)单独开 issue | +| **D** | `[pack]` 加五组键(`default_kind` / `default_artifact` / `targets` / `[pack.interface]` / `[pack.headers]`) | 逐条问「别处已经说过了吗」之后全部落空 | **新增 0 个键**:kind → `[targets.].kind`;接口根 → `[lib]`;头目录 → `[build].include_dirs`;平台 → `[package].platforms` 升格为断言 | + +外加一处我自己踩的坑:原型用 `rglob` 找产物,**挑到了陈旧的 fingerprint 目录** +(少一个 `capi.o`,症状是消费者 undefined reference,看起来完全像 mcpp 的问题)。 +**打包器绝不能 glob 产物** —— 这与仓库既有的「`ls | head -1` 会自查到旧二进制」是同一形状。 + +--- + +## 10. 证据索引 + +全部实测记录在 `2026-08-17-distribution-architecture-analysis-and-design.md`: + +| 判据 | 那份文档的位置 | +|---|---| +| J1(接口↔二进制无绑定,静默错数据) | §2.2(c) | +| J2(`.so` 烧生产机器 RUNPATH) | §2.2(a) | +| J3(闭包 vs `.m.o`;剔除集) | §2.4.2 / §2.4.3 | +| J4(裸三元组不匹配 + 最小探针) | §2.4.5 | +| 六格矩阵(两种接口 × 两种形态) | §2.5.1 | +| 两条目录规则 | §2.5.2 | +| musl + shared 链接期炸 | §2.5.3 | +| `sources = []` 不生效 | §2.5.4 | +| 新段的兼容边界 + array-of-tables 白名单(**为什么最终不新增段**) | §4.2 | +| `[[runtime.artifacts]]` 能承载 role/abi/digest/provenance + identity 判定 | 本文 §2.4.1 实测 | +| `[pack]` 推导审计 | §4.6.1 | +| 复现脚本(`scratchpad/lab/`) | 附录 A | +| file:line 索引 | 附录 B | + + +--- + +## 11. 实施记录(PR #451,2026-08-17) + +### 11.1 落地与设计的差异 + +| 设计说的 | 实际做的 | 为什么 | +|---|---|---| +| `[distribution]` 段(4 个字段) | **一个字段都没加** | 追问「这件事别处能说吗」之后,`role`/`abi`/`digest`/`provenance`/`host_fingerprint` 全在 `[[runtime.artifacts]]` 上;`provenance` 前缀就是「这是分发包」的标记 | +| `abi_surface = "c"` 开关 | **tag 的形状本身** | `abi_check` 早就是「未指定 = 不关心」,短 tag 就是那句声明 | +| 新建 `pack-dist-matrix.yml` | **复用既有 e2e 通路** | `run_all.sh` 已有能力探测与 `# requires:` 分流,再建一条是同一决策的第二处推导 | +| `shared` 排 P1 | **Linux/ELF 已可用** | 实测发现只差一件事:包里要同时带链接名与 SONAME | + +### 11.2 实现过程中被实测推翻的四条 + +**(a) `mcpp.pack.library` 里叫 `Error`。** `mcpp.pack` 已经有一个 +`mcpp::pack::Error`,而一个名字只能归属一个模块。**GCC 接受了,clang 直接拒绝** +(*cannot be attached to other modules*)—— Windows 与 macOS 全红、Linux 全绿, +这正是三平台矩阵存在的理由。 + +**(b) `mcpp pack` 在 workspace 根上不能用了。** 路由要在构建前读 manifest, +而 workspace 根**没有自己的目标**(虚拟 workspace 连 `[package]` 都没有), +于是新路由读到空列表、判定「无可打包」。**用上一版发布的二进制跑 +`examples/04-workspace` 对照才发现** —— e2e 249 就是这次对照的固化。 + +**(c) 位置参数没传给应用通路。** 两个 `bin` 目标的工程接受 `mcpp pack app2` +却打包 app1 —— **命令成功、答案错误**。e2e 250 两个方向都钉。 + +**(d) 动态库包只带了构建出来的文件名。** `-lmathkit-shared` 链的是 +`libmathkit-shared.so`,而对象记的是 `SONAME libmathkit.so.1` —— 两个不同的文件名。 +包链得上、**起不来**。是 **mcpp 自己的运行期闭包检查**报出来的 +(`libmathkit.so.1 not found on the search path this artifact will actually use`), +而不是加载器错误。e2e 251 用 `run` 而不是 `build`:这里链接过了什么都不证明。 + +**(e) 「新测试在 CI 里跑了吗」必须单独查一遍 —— 我自己造了第二次假绿。** +十个新 e2e 一开始全写 `# requires: gcc`,而那个能力**按设计只在 Linux 成立** +(macOS 的 g++ 是 Apple Clang,Windows 的也不是 mcpp 兼容的 GCC)。于是它们 +**在 macOS 与 Windows 上全部跳过、套件报绿**,而文档写着「静态库包:所有 target」。 +判据不是「套件绿了」,是「这条测试**跑了**吗」——`grep 'SKIP:'` 三分钟就能查。 + +放开之后 CI 给出了真实答案,**这才是这次放开的价值**: +- Windows(MSVC-ABI clang)那条腿上,`pack` 内部的库构建以一句 + `error: build failed` 失败(本机无法复现该路径); +- macOS 上 `ar` 解析到一个报「未安装」的 xlings shim,闭包测试查不了归档。 + +**处理方式是记成明确的限制,不是塞回去。** 打包测试重新 `# requires: gcc` +并在头部写明范围;**命令本身**(路由 / 未知名字 / workspace 根)保持三平台运行 +(249、250);`docs/12` 与其中文版加了「验证到哪一步」一节。 +**「所有 target 都支持」与「只在 Linux 验证过」是两句话,文档现在两句都说。** + +外加一条 e2e 卫生:六个 fixture 把包路径按 **shell 拼写**写进了 mcpp.toml。 +`00_fixture_path_hygiene.sh` 在 macOS 那条腿上抓到 —— 规则是 Windows 的 +(MSYS 只转 argv 不转文件内容),而 lint 在所有平台跑,正是为了让 Linux 上的 +reviewer 先于 Windows CI 发现。 + +### 11.2b 「全面可用」是怎么做到的(不是靠放宽 requires) + +放开可移植测试之后 CI 报出两条真实失败,**追下去发现根因在扫描器,不在打包**: + +`module M:part;`(实现分区)与 `module M;`(实现单元)共用一个拼写、是两种声明, +而扫描器把前者记成**「requires `M:part`、provides 空」** —— 一个文件 requires +自己的名字。于是图里**没有**「import 分区的单元 → 定义分区的单元」这条边, +构建顺序无约束:GCC 与 macOS clang 靠各自的依赖扫描兜住, +**Windows clang 以 `failed to read compiled module` 失败**。 +同一处第二半:`import :part;` 的解析读 `u.provides`,而实现单元没有 provides ⇒ +`import :secret;` 停在字面 `:secret`。两平台都刷的那条 +`module 'M:part' imported but not provided in this build` **就是病因,读起来像提示**。 + +**实现分区在此之前 mcpp 里任何地方都没有测试覆盖** —— 库分发的 e2e 是第一个用它的。 +修好之后: + +| e2e | linux | macOS | windows | 跑在哪 | +|---|---|---|---|---| +| 242 布局 + 两种接口模式 | ✅ | ✅ | ✅ | e2e 分片 | +| 243 闭包 + 剔除集 + 分区告警 | ✅ | ✅ | ✅ | e2e 分片 | +| 244 三条拒绝 | ✅ | ✅ | ✅ | e2e 分片 | +| 246 `sources = []` | ✅ | ✅ | ✅ | e2e 分片 | +| 247 裸三元组条件 | ✅ | ✅ | ✅ | e2e 分片 | +| 249 workspace 根仍能打包 | ✅ | ✅ | ✅ | e2e 分片 | +| 250 `pack ` 打的是那个 | ✅ | ✅ | ✅ | e2e 分片 | +| 251 动态库:产出正确 **或** 明确拒绝 | ✅ 产出 | ✅ 拒绝 | ✅ 拒绝 | e2e 分片 | +| 248 跨 OS 边界的胖包(PE) | ✅ | — | — | **`cross-build-test.yml` 的 mingw job** | +| 245 胖包(含原生构建) | ✅ | skip | skip | e2e 分片 | + +**只剩 245 一条在 Linux 之外跳过,而它是真限制**:那条测试要**两个都能构建的 +target**(`x86_64-linux-gnu` + `x86_64-linux-musl`),而 macOS / Windows 的 CI 上 +不存在第二个现成可用的 target。它验的机制(每条腿一个 `cfg()`、原生构建也命中) +本身与平台无关,已在 Linux 上覆盖。 + +**另外四条原本的 skip 理由,查下来三条是我偷懒:** + +| 原理由 | 真相 | +|---|---| +| 246「读编译器特定的 flag 拼写」 | **错的** —— 它 grep 的是**文件名**。`# requires: gcc` 是复制粘贴来的 | +| 247「读编译器特定的 flag 拼写」 | **半对** —— 它 grep 了 `-D` 前缀,而 MSVC 是 `/D`(`dialect.cppm:200`)。改成 grep **宏名**即可移植 | +| 248「需要 mingw-cross,没有 job 装它」 | **可修** —— `cross-build-test.yml` 的 mingw job **已经**在显式点名跑那些「Linux 分片因缺 mingw-cross 而跳过」的 e2e(102/198/240),照办即可 | +| 251「`shared` 仅 ELF」 | **理由成立,但只测了一侧** —— 补成两侧断言:ELF 上产出且带两个名字,非 ELF 上**必须拒绝且说明原因**。只测能用的那侧分不清「门被处理」与「没有门」 | + +副作用:一条**与打包无关**的既有缺陷被修好了 —— 用实现分区的工程现在在 Windows 上 +能构建了。 + +**判据(第二次得到验证):一条测试的 `# requires:` 必须是它所验证机制的真实下限。** +写下它的时候很难分辨「真限制」与「我没想清楚」,所以事后必须**逐条追问理由**; +四条里三条经不起追问。 + +### 11.3 我在验证里自己踩的坑 + +**`ls -t target/*/bin/mcpp | head -1` 挑到了陈旧/别的工具链的 fingerprint 目录 —— +三次。** 一次让 B1 的修复看起来没生效,一次让全部新 e2e 报段错误(实际是拿了 +clang 构建的二进制,而 clang 构建的 mcpp 在本机会段错误)。 +**这与打包器「绝不 glob 产物」是同一条判据**,只是发生在验证侧。 + +### 11.4 P0 交付清单 + +| | | +|---|---| +| 新模块 | `src/pack/{abi_tag,digest,interface,library,library_pipeline,manifest_emit,prebuilt,route}.cppm` | +| 改动 | `prepare_inputs`(B1)、`toml`+`types`(B2)、`source_kind`(`object_filename_for` 归位)、`prepare`(图 + 两道闸门)、`plan`、`cli` | +| 单测 | `test_pack_abi_tag`(15)、`test_pack_interface`(8) | +| e2e | 242–251(10 个) | +| 文档 | `docs/12-binary-distribution.md` + zh;`docs/02`/`05`/README 索引 | +| 示例 | `examples/05-lib-distribution/producer`、`examples/05-lib-distribution/consumer` | + + +--- + +## 12. 深度 review(2026-08-17,PR #451 CI 全绿之后) + +### 12.1 245 / 248 在其他平台「没有」的真实原因 + +**权威答案是 `host_can_serve`(`registry.cppm:542-566`),不是测试的能力表。** + +| target | linux host | macOS host | windows host | +|---|---|---|---| +| `-linux-gnu` | ✅ | ❌ | ❌ | +| `*-linux-musl` | ✅ 任意 arch | ❌ | ✅ 仅 host arch | +| `*-windows-gnu`(mingw) | ✅ | ❌ | ✅ | +| `*-windows-msvc` | ❌ | ❌ | ✅ | +| `*-macos` | ❌ | ✅ | ❌ | + +- **macOS 只能服务一个 target。** 胖包至少要两条腿 ⇒ 在 macOS 上 + **结构上不可能**,不是测试没写。代码自己的注释: + *"macOS has no Linux-targeting payload at all."* +- **Windows 能服务三个**(msvc / mingw / linux-musl)⇒ **245 与 248 的等价用例在 + Windows 上是可行的,只是没做。** 这是**覆盖缺口**,不是产品缺口,而且它有独立价值: + `mathkit.lib`(MSVC)+ `libmathkit.a`(MinGW)同处一包,是「`lib/` 必须按三元组分」 + 最强的证据。代价是要在 Windows 的 e2e job 里装 `xim:mingw-gcc`(e2e 97 已经在 + 另一个 job 里装它)。 +- **248 需要一个能产 PE 的工具链**:Linux 上是 `mingw-cross`(现在跑在 + `cross-build-test.yml` 的 mingw job),Windows 上是原生 mingw/msvc(同上缺口)。 + +### 12.2 架构:依赖方向是可陈述的不变量 + +`src/pack/` 现在是两层,**必须保持**: + +``` +叶层(prepare 可以 import 它们): abi_tag digest interface manifest_emit prebuilt route library +编排层(它 import prepare): library_pipeline +``` + +实测的 import 图里,**只有 `library_pipeline` 碰 `mcpp.build.*`**;而 +`prepare` 只 import `pack.abi_tag` 与 `pack.prebuilt`(两个叶子)。 +**不变量:除 `library_pipeline` 外,`src/pack/` 下不得 import `mcpp.build.*`** —— +否则 prepare ↔ pack 成环。 + +三处收敛(都是把「同一个决定的第二处推导」删掉,不是加抽象): +- `object_filename_for` 从 plan.cppm 搬进 `mcpp.source_kind` —— 策略与格式化同处; +- 「怎么删归档成员」进 `dialect.cppm` 的 `archiveRemoveArg` —— 与 `archiveCmd` 同处, + packer 不再默认 `ar` 语法(**这条是 review 发现的,不是测试发现的**); +- 解析后的三元组进 `cfgpred::Ctx` —— 删掉裸三元组分支那个第二答题者。 + +### 12.3 兼容性:两处**故意的**行为变化 + +| 变化 | 谁会受影响 | +|---|---| +| `sources = []` 从「等于不写」变成「什么都不编」 | 真写了它又依赖默认 glob 的工程。此前**无法表达**「什么都不编」,所以这种写法只可能是误解 | +| 两个文件声明同一个分区 ⇒ 拒绝并点名两个文件 | 本来就 ill-formed 的程序。**但它是一条新的失败路径** | + +扫描器的改动**只会增加图上的边**,不会减少 —— 边缺失只可能允许错误的顺序, +所以不存在「依赖那条缺失边」的工程。 + +老客户端兼容:**两侧都验了**(静态:生成的 manifest 段 ⊆ 既有词汇,三平台跑; +真实:2026.8.15.3 消费并运行成功,本机)。⚠️ CI 里真实那半跑不了,因为 +`$MCPP_BOOT` 是 xvm shim(在 e2e 改过的环境里报「未安装」),这一点写在 252 的头部。 + +### 12.4 仍然开着的五项(合入前请裁决) + +| # | 项 | 我的建议 | +|---|---|---| +| **O1** | **扫描器改动是全 PR 里影响面最大的一块,而它与打包无关** | **拆成独立 PR 先合**,这样它能被独立 revert;库分发那半 rebase 在它上面 | +| **O2** | `scan_overrides` 声明的实现分区会被**误判成接口**(`providesInterface` 只在文本扫描路径设 false,`scan_overrides`/P1689 默认 true)⇒ 它的源码会被发布 | 窄洞,而且 `scan_overrides` 的 schema **没有**表达「是否 export」的位置。先记录;要修就是给它加一个键(那会是本方案唯一的新 manifest 键) | +| **O3** | MSVC 的 `/REMOVE:` 路径**未测试**(mcpp 的 Windows CI 用 clang 的 llvm-ar,走 GNU 形式) | 失败会带命令原文报错,不会静默;要真测需要 `msvc@system` 的 job | +| **O4** | Windows 上的胖包(msvc + mingw)未覆盖 | 见 §12.1;要在 Windows e2e job 装 `xim:mingw-gcc` | +| **O5** | 设计 §2.2 承诺的「`--target` 集合与 `[package].platforms` 覆盖比对告警」**没有实现** | 我漏了。它便宜(一次集合比对 + 一条 warning),但属于「发布纪律」而不是正确性,可以随后补 | diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 96529460..0f167ee6 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -63,6 +63,11 @@ jobs: # Point the e2e runner at the freshly-built binary, not the # bootstrap one. Tests cd into mktemp -d, so $MCPP must be # absolute or the relative path breaks under the temp cwd. + # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the + # claim that an older client can still build against a package the PR + # produces is only worth making if something checks it against a real + # old binary. Captured before $MCPP is repointed at the fresh build. + export MCPP_BOOT="$MCPP" MCPP=$(realpath "$(find target -type f -name mcpp -printf '%T@ %p\n' | sort -rn | head -1 | cut -d' ' -f2)") test -x "$MCPP" export MCPP diff --git a/.github/workflows/ci-macos-e2e.yml b/.github/workflows/ci-macos-e2e.yml index 4c30b421..5d02b76e 100644 --- a/.github/workflows/ci-macos-e2e.yml +++ b/.github/workflows/ci-macos-e2e.yml @@ -41,6 +41,11 @@ jobs: # Per-test 600s timeout lives in run_all.sh. timeout-minutes: 25 run: | + # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the + # claim that an older client can still build against a package the PR + # produces is only worth making if something checks it against a real + # old binary. Captured before $MCPP is repointed at the fresh build. + export MCPP_BOOT="$MCPP" MCPP=$(find target -path "*/bin/mcpp" | head -1) MCPP=$(cd "$(dirname "$MCPP")" && pwd)/$(basename "$MCPP") test -x "$MCPP" diff --git a/.github/workflows/ci-windows-e2e.yml b/.github/workflows/ci-windows-e2e.yml index a6ff7086..05c3b43a 100644 --- a/.github/workflows/ci-windows-e2e.yml +++ b/.github/workflows/ci-windows-e2e.yml @@ -63,6 +63,11 @@ jobs: # Per-test 600s timeout lives in run_all.sh. timeout-minutes: 25 run: | + # The RELEASED mcpp this job bootstrapped from, kept for e2e 252: the + # claim that an older client can still build against a package the PR + # produces is only worth making if something checks it against a real + # old binary. Captured before $MCPP is repointed at the fresh build. + export MCPP_BOOT="${MCPP:-$MCPP_BOOT}" export MCPP="$MCPP_SELF" export MCPP_VENDORED_XLINGS="$XLINGS_BIN" export MCPP_E2E_TOOLCHAIN_MIRROR=GLOBAL diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 379def22..3468bd36 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -34,6 +34,10 @@ name: cross-build-test # qemu-aarch64 is the CI proxy for "does this cross artefact actually execute". # # ── NOT here ─────────────────────────────────────────────────────────────── +# * The e2e scripts this job names explicitly (102, 198, 240, 248) are the +# ones the ordinary Linux shards SKIP for want of `mingw-cross`. They are +# listed in the job rather than left to run_all's cap gating precisely so +# they cannot end up skipping everywhere at once. # * Same-arch builds (host arch == target arch) are NOT cross. The native musl # static build `--target x86_64-linux-musl` (x86_64 host) is exercised by # ci-linux.yml's "Toolchain: musl-gcc" step, and release.yml for the static @@ -320,6 +324,23 @@ jobs: export MCPP_VENDORED_XLINGS="$XLINGS_BIN" bash tests/e2e/240_pack_pe_zip_cross.sh + # A LIBRARY package whose legs cross an OS boundary, for the same reason + # as the three above: this is the only job with a MinGW cross toolchain. + # + # It is not redundant with 245 (which covers the fat-package mechanism + # with gnu + musl and therefore runs on every ordinary Linux shard). The + # leg added here changes BINARY FORMAT, and it is the case that proves + # `lib/` has to be keyed by triple rather than by OS: MinGW and MSVC are + # both "windows" and write `libfoo.a` and `foo.lib` respectively. + # + # Without this step the test would carry `# requires: mingw-cross` and + # skip in every job that exists — verified on a developer's machine and + # nowhere else, while the suite reported green. + - name: "e2e: pack a library across an OS boundary (PE leg)" + run: | + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + bash tests/e2e/248_pack_library_fat_pe_leg.sh + # ── windows → linux ─────────────────────────────────────────────────────── # The mirror of mingw-cross-wine. Two jobs because a Windows runner cannot # execute the ELF it produces; the artefact is handed to a Linux job and diff --git a/CHANGELOG.md b/CHANGELOG.md index 192514a5..4d9f8366 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,8 +5,98 @@ ## [Unreleased] +### 新增 + +- **`mcpp pack ` 可以把一个库打成「接口 + 预编译二进制」的包(#433)。** + + 闭源库、离线环境、以及「构建农场已经编过一遍了」这三种场景,过去都只能自己 + 写脚本收集产物。现在: + + ```bash + mcpp pack mathkit # 静态库包 + mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 一个包,两条腿 + ``` + + **产出的是一个普通的 mcpp 包** —— 一份正常的 `mcpp.toml`,走 mcpp 早就有的 + 「载荷自带 manifest」通路。**新增 manifest 段 0 个、键 0 个**:打什么由 + `[targets.].kind` 决定(所以没有 `--lib`、没有 `--artifact`),发布哪些接口 + 由 `[lib]` 约定 + 模块图决定,公开头是 `[build].include_dirs` 全量, + 每条腿的 ABI tag 与 digest 记在既有的 `[[runtime.artifacts]]` 上。 + 一个**老版本 mcpp 照样能构建**这种包 —— 它只是不执行下面那两道闸门。 + + 一个包可以同时带**两种接口**:`include/`(文本,`#include`,不编译)与 + `interface/`(模块,消费者编译它)。实测同一个包被「只 #include」/「只 import」/ + 「两者都用」三种方式消费,静态与动态两种形态,六格全过。 + + 发布哪些 `.cppm` 是**算出来的** —— lib root 的模块闭包,不是「所有 `.m.o`」。 + 实现分区(`module M:secret;`)照样产 `.m.o`,按扩展名挑会**泄露闭源源码**; + 同一个闭包反过来决定归档里要删哪些对象,按 `.m.o` 删则会删掉真代码、 + 三个平台全部链接失败。两条清单都会打印出来。 + + 详见 `docs/12-binary-distribution.md`、`examples/05-lib-dist`、`examples/06-lib-consume`。 + +- **消费预编译包时的两道闸门。** 都是不检查就会静默出错的: + + **接口与二进制是否仍然配对。** 这条闸门存在是因为另一种结果被实测过:把随包 + 接口里一个结构体的两个 `int` 成员互换 —— Itanium ABI 不 mangle 字段顺序 —— + 消费者**编译过、链接过、运行过、打印出交换后的错数据**,任何工具都没有一句诊断。 + + **二进制是否为这套工具链所编。** 失配时诊断会**列出包里确实有哪些 tag** —— + 一句「找不到」会让人去找一个就在自己硬盘上的包。 + + 另外,在解开的分发包目录里直接 `mcpp build` 会被拒绝:那儿的 `interface/` + 是声明,定义在旁边的归档里,构建会产出一个几乎空的库然后报告成功。 + ### 修复 +- **实现分区(`module M:part;`)在 Windows 上构建不了,而根因在扫描器里。** + + `module M:part;` 与 `module M;` 共用一个拼写,却是两种不同的声明,而扫描器把 + 它们当成了一种:前者被记成**「requires `M:part`、provides 空」** —— + 一个文件 requires 自己的名字。于是图里**没有**从「import 分区的单元」到 + 「定义分区的单元」的边,构建顺序无约束:GCC 与 macOS clang 靠各自的依赖扫描 + 兜住了,**Windows clang 以 `failed to read compiled module` 失败**。 + + 同一处还有第二半:`import :part;` 的解析读的是 `u.provides`,而实现单元 + (`module M;`)没有 provides ⇒ 它里面的 `import :secret;` 停留在字面的 + `:secret`,没有任何单元提供。两个平台都会刷的那条 + `module 'M:part' imported but not provided in this build` 就是这两件事的 + 合并症状 —— **它读起来像一条提示,其实是病因**。 + + 实现分区在此之前**mcpp 里任何地方都没有测试覆盖**,是库分发的 e2e 第一次 + 用到它才暴露出来。现在扫描器记 `provides = M:part` 并标 `providesInterface + = false`;`import :part;` 按 TU 自己所属的模块名解析。 + + **⚠️ 两处行为变化**:①两个文件声明同一个分区(`module m:p;` × 2)现在会被 + **拒绝并点名两个文件**,此前是静默接受 —— 那种程序本来就 ill-formed, + 但它是一条新的失败路径;②`sources = []` 从「等于不写」变成「什么都不编」, + 一个真写了 `sources = []` 又依赖默认 glob 的工程会发现产物变空(此前无法表达 + 「什么都不编」,所以这种写法只可能是误解)。 + +- **`[target.'<三元组>'.build]` 在没有 `--target` 时从不命中。** + + 同一个语句的两种拼写互相矛盾:`cfg(linux)` 在原生构建上命中, + `[target.'x86_64-linux-gnu'.build]` 不命中。根因是 `matches()` 拿着原始的 + `--target` 字符串(原生构建下是空的)短路返回 false,而同一文件的 + `context_for()` 对 `cfg(...)` **回落到宿主三元组** —— 一个决定两处推导。 + `manifest/types.cppm` 的注释从写下起承诺的就是回落那一种。 + + **形状是最坏的那种**:CI 传 `--target` 是绿的,开发者本机的 `mcpp build` + 静默丢掉那一段,失败在链接期出现、点的是符号而不是谓词。 + + 修法是**删掉第二个答题者**:解析后的三元组进 `cfgpred::Ctx`,`matches()` + 只有一个来源。 + +- **`sources = []` 与不写 `sources` 逐字节等价。** + + 解析器在向量为空时一律填默认 glob,于是作者**没有任何写法**能表达 + 「什么都不要编」。二进制分发需要这个:一个纯头文件的包不编译任何东西, + 而 `src/` 下任何遗留文件都会被扫进消费者的构建,并可能与预编译库里的符号 + 重复定义。改成记录**键是否出现**(`BuildConfig::sourcesDeclared`), + 与 `XlingsConfig::subosDeclared` 同一个模式。 + + - **卸载后的清扫会波及**别的版本**,而那可能正在被另一个进程解压。** `sweep_parked_payloads` 原来把整个 family 目录扫一遍,把**任何**没有文件的 diff --git a/docs/02-pack-and-release.md b/docs/02-pack-and-release.md index 195d7b27..a9100649 100644 --- a/docs/02-pack-and-release.md +++ b/docs/02-pack-and-release.md @@ -1,4 +1,12 @@ -# 02 — Packaging for Release +# 02 — Packaging an Application for Release + +> This page is about bundling a **program**. To ship a *library* as interface + +> prebuilt binaries, see [12 - Distributing a Prebuilt Library](12-binary-distribution.md). +> +> Which one `mcpp pack` does is decided by the target's `kind`, not by a flag: +> `mcpp pack ` packs `[targets.]`, and a `bin` becomes a bundle +> while a `lib`/`shared` becomes a library package. With no name, mcpp picks +> the only packable target. > A default dynamically linked binary produced by `mcpp build` has a loader and > RUNPATH tied to the build sandbox. It is a development artifact, not a diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index a5e663eb..e54b3f70 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -153,6 +153,12 @@ the package/feature boundary, not on an individual target. ### 2.3 `[build]` — Build Configuration +> **`sources = []` is not the same as omitting `sources`.** An absent key +> selects the default glob; an explicitly empty list means *compile nothing*, +> which is what a header-only distribution package needs to say. Until +> mcpp 2026.8.17.2 the two were byte-identical, so there was no spelling for +> "nothing" and any file left under `src/` was swept in. + ```toml [build] sources = ["src/**/*.cppm", "src/**/*.cpp"] # Source globs (default: src/**/*.{cppm,cpp,cc,c,S,s,asm}) @@ -1598,6 +1604,17 @@ do`. - Package-level knobs all converge into features; for sugar keys (such as `backend=`) to enter the core syntax, they must satisfy: ① domain-neutral (a cross-ecosystem general pattern) ② 1:1 desugaring with zero new parsing semantics. +- **A key that duplicates an answer another section already gives is not admitted.** + Two places to state one fact is two places that can disagree, and the failure + is silent — whichever reader loses the race is simply wrong. Library packaging + ([12](12-binary-distribution.md)) is the worked example: it added **zero** + manifest keys, because what to pack is `[targets.].kind`, which interface + to publish is `[lib]` plus the module graph, which headers are public is + `[build].include_dirs`, and the per-artifact evidence is `[[runtime.artifacts]]`. +- A field that describes what a *generated* package IS (rather than what a build + should DO) belongs on `[[runtime.artifacts]]` — see §2.11. `provenance` + beginning with `mcpp-pack` is what marks a directory as one, and mcpp refuses + to `build` inside it. - See `.agents/docs/2026-06-04-manifest-schema-ownership.md` for the full field-ownership table and the finalized decisions. diff --git a/docs/12-binary-distribution.md b/docs/12-binary-distribution.md new file mode 100644 index 00000000..ef61b7a5 --- /dev/null +++ b/docs/12-binary-distribution.md @@ -0,0 +1,288 @@ +# 12 - Distributing a Prebuilt Library + +**English** | [简体中文](zh/12-binary-distribution.md) + +> Ship a library as **interface + prebuilt binaries** instead of as source. +> This is the closed-source case, and the offline case, and the "our build farm +> already compiled this once" case. +> +> [02 - Packaging & Release](02-pack-and-release.md) is the sibling: bundling an +> *application*. [10 - Publishing a Library](10-publishing-a-library.md) is the +> source route. + +## The whole idea in one paragraph + +`mcpp pack ` builds a library target and writes a directory that is an +**ordinary mcpp package** — a normal `mcpp.toml`, the interface a consumer must +compile, and the binaries it then links. Consumers use it exactly like any +other dependency. There is no new manifest section, no new archive format, and +no new resolution path. + +```bash +mcpp pack mathkit # a static library package +mcpp pack mathkit-shared # a dynamic one (Linux/ELF today) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # one package, two legs +``` + +## What decides what gets packed + +`[targets.].kind`, and nothing else: + +| `kind` | `mcpp pack ` produces | `--mode` | +|---|---|---| +| `bin` | an application bundle (see [02](02-pack-and-release.md)) | the four depths | +| `lib` | a **static library package** | — | +| `shared` | a **dynamic library package** | — | + +There is no `--lib` flag and no `--artifact static\|shared`. `kind` is already +where mcpp records what an artifact is; a flag would be a second place to say +it, and two places can disagree. A project that publishes both forms declares +both targets — which it must do for `mcpp build` to produce both anyway: + +```toml +[targets.mathkit] +kind = "lib" + +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" +``` + +Run `mcpp pack` with no name and mcpp picks the only packable target, or tells +you which ones it found. + +## The two interface modes + +A package can carry both, and a consumer may use either or both. + +``` +mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ +├── mcpp.toml +├── include/ ← TEXT interface: #include, never compiled +├── interface/ ← MODULE interface: the consumer compiles it +└── lib// ← the artifacts +``` + +| | `include/` | `interface/` | +|---|---|---| +| whose input is it | the preprocessor's | the **compiler's** | +| does the consumer compile it | no | **yes**, to get a BMI | +| what it constrains | the libc ABI | compiler, C++ stdlib, C++ level | +| can you trim it | **no** — see below | **no**, it is computed | + +`lib/` is keyed by **triple**, not by OS. MinGW and MSVC are both Windows and +produce `libfoo.a` and `foo.lib` respectively. + +A **shared** package carries the library under *both* of its names: a consumer +links `lib.so` and the loader then asks for the `SONAME`, and those are +different filenames. Shipping only the built file links cleanly and then fails +to start. + +### Why neither set can be trimmed + +A **source** distribution of the same package puts every one of its +`include_dirs` on its consumers' include path. If a binary package shipped a +subset, the same library would have a different public surface depending on how +it was delivered. "Which headers are public" is already answered by the layout: +`include/` is public, `src/` is not. A private header under `include/` is a +project-layout mistake, not a packaging option. + +## Which `.cppm` files travel + +The **module closure of the lib root** — `src/.cppm` by +convention, or `[lib].path`. Whatever that unit's purview imports, transitively, +is published; everything else is not. + +``` +src/mathkit.cppm export module mathkit; export import :api; → published +src/api.cppm export module mathkit:api; → published +src/secret.cppm module mathkit:secret; ← implementation partition +src/impl.cpp module mathkit; → withheld +``` + +`mcpp pack` prints both lists: + +``` + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm +``` + +**Read the second one** if you are shipping closed source. + +> **`.m.o` is not the rule.** An implementation partition produces a BMI and an +> object exactly like an interface unit does. Selecting sources by "does it +> produce a BMI" would publish `secret.cppm`. + +If the published interface *does* import an implementation partition, a +consumer cannot compile it without that source — so `mcpp pack` stops: + +``` +error: the published interface imports mathkit:secret , which no unit in this + build provides. +``` + +Restructure so the interface does not reach it, or make it an `export module` +partition and accept that its source is published. + +## The compatibility tag + +Every artifact records the toolchain it was built for: + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface +x86_64-linux-gnu # an extern "C" interface only +``` + +`--` then, when the interface is C++, ``, +``, `c++`. + +**A shorter tag is a real statement, not a missing one.** A library whose whole +interface is `extern "C"` constrains the libc ABI and not the C++ one, so it +publishes a triple and stops — and links into any compiler. Unnamed dimensions +are don't-care, so one tag per triple instead of one per triple per compiler. +Nothing to configure: the shape is the statement. + +The `c++` level is compared as a **floor**, not for equality: building at a +higher level is fine, lower is not. + +## What a consumer's build checks + +Two things, both of which fail silently without a check: + +**The interface still matches its binaries.** + +``` +error: acme.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +This exists because the alternative was measured. Swap two `int` members of a +struct in a shipped interface — the Itanium ABI does not mangle field order — +and the consumer compiles, links, runs, and prints transposed data, with no +diagnostic from any tool. A digest cannot stop a publisher from shipping a +mismatched pair (only producing both in one command does that), but it does +catch the pair coming apart afterwards. + +**The binaries were built for this toolchain.** + +``` +error: acme.mathkit@0.1.0: no prebuilt artifact matches this toolchain. + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : + x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on: + compiler needs gcc15, this build has gcc16 + stdlib needs libstdcxx15, this build has libstdcxx16 +``` + +The tags it *does* have are part of the message: "not found" would send you +looking for a package already on your disk. + +## Consuming one + +Three spellings, one code path: + +```toml +# a directory (what you hand a colleague) +mathkit = { path = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } + +# a private git repo +mathkit = { git = "ssh://git@internal/mathkit-dist.git", tag = "v0.1.0" } + +# an index entry — identical in shape to a source package's +mathkit = "0.1.0" +``` + +Nothing about the consumer's manifest says "this one is prebuilt". + +### Building *inside* a package is refused + +``` +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +Its `interface/` holds declarations whose definitions are in the archive beside +them. Building there compiles the declarations, produces a near-empty library +and reports success. + +## One package, several targets + +`--target` is repeatable. The generated manifest gets one conditional block per +leg, and the consumer's build picks its own: + +```toml +[target.'cfg(all(arch = "x86_64", os = "linux", env = "gnu"))'.build] +ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + +[target.'cfg(all(arch = "x86_64", os = "linux", env = "musl"))'.build] +ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] +``` + +Because selection happens in the **consumer's** build, where the resolved +target is known, a fat package cross-compiles correctly with no index-side or +installer-side support at all. + +> The blocks are `cfg(...)` and never a bare `[target.'']` key. Before +> mcpp 2026.8.17.2 the bare form was inert without an explicit `--target`, so a +> package using it would work in CI and silently drop its flags on a +> developer's machine. mcpp generates the spelling that means the same thing on +> every client. + +## Dependencies + +A static archive does **not** carry its dependencies' code, so the package +records them and the consumer resolves them: + +```toml +[dependencies] +"compat.zlib" = "1.3.2" +``` + +`path` and `git` dependencies are dropped: they address the publisher's disk, +and republishing one hands the consumer an address that means something else. +If your library depends on one, either publish that dependency too or vendor it +before packing. + +## What older mcpp does with these packages + +**It builds against them.** Every key in the generated manifest already +existed, so an older client reads the package and links it. What it does not do +is run the two checks above — it has no way to know that `provenance = +"mcpp-pack …"` means anything. + +That is a degradation, not a break, and it is the right direction. But it means +**the gate protects new clients only**, which belongs in your release notes if +you publish to a mixed audience. + +## Current limits + +| | status | +|---|---| +| `kind = "lib"` (static) | ✅ every target, tested on all three | +| `kind = "shared"` on Linux/ELF | ✅ — the package carries both the link name and the SONAME | +| `kind = "shared"` on PE / Mach-O | ❌ refused — import libraries and install-names are not modelled yet | +| `kind = "shared"` on `*-musl` | ❌ a musl target links statically | +| shipping prebuilt BMIs | ❌ not attempted; BMIs are compiler-build-exact | +| bundling dependencies into the package | ❌ declare them instead (above) | + +### Implementation partitions + +`mcpp pack` treats an implementation partition (`module M:part;`, no `export`) +as private: its source stays behind, its object ships inside the archive. + +If your published interface *imports* one, the consumer cannot build the BMI +without that source, so it IS published — and `mcpp pack` says so: + +``` +warning: secret.cppm is an implementation partition, and the published interface + reaches it — so its SOURCE is being published. +``` + +> Until mcpp 2026.8.17.2 the scanner recorded `module M:part;` as *requiring* +> `M:part` and providing nothing, so a file required its own name and the graph +> held no edge from the unit importing a partition to the unit defining it. +> Build order was unconstrained: GCC and macOS clang recovered through their own +> dependency scan, Windows clang failed with `failed to read compiled module`. +> If you have been avoiding implementation partitions on Windows, that was why. diff --git a/docs/README.md b/docs/README.md index 2118e5ae..369af822 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,6 +14,7 @@ - [09 - Releasing mcpp](09-release.md) - [10 - Publishing a Library to mcpp-index](10-publishing-a-library.md) - [11 - Machine-Readable Output](11-machine-output.md) +- [12 - Distributing a Prebuilt Library](12-binary-distribution.md) ## Specifications diff --git a/docs/zh/12-binary-distribution.md b/docs/zh/12-binary-distribution.md new file mode 100644 index 00000000..b02e0d19 --- /dev/null +++ b/docs/zh/12-binary-distribution.md @@ -0,0 +1,262 @@ +# 12 - 分发预编译库 + +[English](../12-binary-distribution.md) | **简体中文** + +> 把一个库以**接口 + 预编译二进制**的形式分发,而不是发源码。 +> 这是闭源场景、离线场景,以及「构建农场已经编过一遍了」的场景。 +> +> 姊妹篇:[02 - 打包应用](02-pack-and-release.md) 讲的是打包**程序**; +> [10 - 发布一个库](10-publishing-a-library.md) 讲的是源码通路。 + +## 一段话讲完 + +`mcpp pack ` 构建一个库目标,产出一个**普通的 mcpp 包** —— +一份正常的 `mcpp.toml`、消费者要编译的接口、以及它随后链接的二进制。 +消费者用它和用任何依赖一样。**没有新的 manifest 段、没有新的归档格式、 +没有新的解析路径。** + +```bash +mcpp pack mathkit # 静态库包 +mcpp pack mathkit-shared # 动态库包(今天仅 Linux/ELF) +mcpp pack mathkit --target x86_64-linux-gnu \ + --target aarch64-linux-gnu # 一个包,两条腿 +``` + +## 打什么由谁决定 + +只由 `[targets.].kind` 决定: + +| `kind` | `mcpp pack ` 产出 | `--mode` | +|---|---|---| +| `bin` | 应用 bundle(见 [02](02-pack-and-release.md)) | 四档 | +| `lib` | **静态库包** | — | +| `shared` | **动态库包** | — | + +**没有 `--lib`,也没有 `--artifact static|shared`。** `kind` 本来就是 mcpp +记录「一个产物是什么」的地方;再加一个开关就是同一件事的第二个说法, +而两个说法可以互相矛盾。要同时发布两种形态,就声明两个目标 —— +这本来也是 `mcpp build` 同时产出两者所必需的: + +```toml +[targets.mathkit] +kind = "lib" + +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" +``` + +不带名字直接 `mcpp pack`,mcpp 会挑唯一可打包的目标,或者告诉你有哪些候选。 + +## 两种接口模式 + +一个包可以同时带两种,消费者用其中一种或两种都用。 + +``` +mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23/ +├── mcpp.toml +├── include/ ← 文本接口:#include,永不编译 +├── interface/ ← 模块接口:消费者编译它 +└── lib// ← 产物 +``` + +| | `include/` | `interface/` | +|---|---|---| +| 是谁的输入 | 预处理器 | **编译器** | +| 消费者要编译吗 | 否 | **是**,编出 BMI | +| 约束什么 | libc ABI | 编译器、C++ 标准库、C++ 档位 | +| 能裁剪吗 | **不能**,见下 | **不能**,它是算出来的 | + +`lib/` 按**三元组**分目录,不按 OS 分:MinGW 与 MSVC 同为 Windows, +一个产 `libfoo.a` 一个产 `foo.lib`。 + +### 为什么两者都不许裁剪 + +同一个包的**源码**分发会把 `include_dirs` 里的每一个头都放到消费者的 include +路径上。二进制包若只发一部分,**同一个库就会因为分发形式不同而有不同的公开面**。 +而且「哪些头是公开的」布局已经回答了:`include/` 公开,`src/` 不公开。 +一个私有头放在 `include/` 下是工程布局的错误,不是打包选项。 + +## 哪些 `.cppm` 会被发布 + +**lib root 的模块闭包** —— 按约定是 `src/<包名尾段>.cppm`,或 `[lib].path`。 +该单元 purview 里 import 到的东西,传递地,都发布;其余都不发。 + +``` +src/mathkit.cppm export module mathkit; export import :api; → 发布 +src/api.cppm export module mathkit:api; → 发布 +src/secret.cppm module mathkit:secret; ← 实现分区 +src/impl.cpp module mathkit; → 不发布 +``` + +`mcpp pack` 会打印两张清单: + +``` + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm +``` + +**闭源分发要看第二张。** + +> **`.m.o` 不是判据。** 实现分区照样产出 BMI 和对象。按「会不会产出 BMI」 +> 来挑发布集,就会把 `secret.cppm` 发出去。 + +如果被发布的接口**确实** import 了一个实现分区,消费者没有那份源码就编不出来 —— +于是 `mcpp pack` 停下来: + +``` +error: the published interface imports mathkit:secret , which no unit in this + build provides. +``` + +要么重构让接口够不到它,要么把它改成 `export module` 分区并接受源码被发布。 + +## 兼容性 tag + +每个产物都记录它是为哪套工具链编的: + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # C++ 模块接口 +x86_64-linux-gnu # 纯 extern "C" 接口 +``` + +`--`,接口是 C++ 时再加 ``、 +``、`c++<档位>`。 + +**短 tag 是一句真实的声明,不是漏写。** 一个接口全是 `extern "C"` 的库 +只约束 libc ABI、不约束 C++ ABI,所以它发三段就停 —— 于是能链进任何编译器。 +未指定的维度就是不关心,C 库的 tag 数因此是「每个三元组一个」而不是 +「每个三元组 × 每个编译器一个」。**不需要任何开关:形状本身就是声明。** + +`c++` 档位按**下限**比对而不是相等:消费者档位更高可以,更低不行。 + +## 消费者的构建会检查什么 + +两件事,而且都是不检查就会静默出错的: + +**接口与二进制仍然配对。** + +``` +error: acme.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +这条闸门存在是因为另一种结果被实测过:把随包接口里一个结构体的两个 `int` +成员互换 —— Itanium ABI 不 mangle 字段顺序 —— 消费者**编译过、链接过、 +运行过、打印出交换后的错数据**,任何工具都没有一句诊断。 +digest 挡不住发布者一开始就发错配对(那只有原子产出能防), +但它能挡住配对在事后被拆开。 + +**二进制是为这套工具链编的。** + +``` +error: acme.mathkit@0.1.0: no prebuilt artifact matches this toolchain. + your toolchain : x86_64-linux-gnu-gcc16-libstdcxx16-c++23 + published tags : + x86_64-linux-gnu-gcc15-libstdcxx15-c++23 + closest is x86_64-linux-gnu-gcc15-libstdcxx15-c++23, and it differs on: + compiler needs gcc15, this build has gcc16 + stdlib needs libstdcxx15, this build has libstdcxx16 +``` + +诊断里**列出它有哪些 tag** 是刻意的:一句「找不到」会让人去找一个 +就在自己硬盘上的包。 + +## 怎么消费 + +三种写法,同一条代码路径: + +```toml +# 一个目录(你直接拷给同事的那种) +mathkit = { path = "vendor/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } + +# 私有 git 仓库 +mathkit = { git = "ssh://git@internal/mathkit-dist.git", tag = "v0.1.0" } + +# 索引条目 —— 形状与源码包一字不差 +mathkit = "0.1.0" +``` + +消费者的 manifest 里没有任何一处写着「这个是预编译的」。 + +### 在包目录里直接 build 会被拒绝 + +``` +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +它的 `interface/` 里是声明,定义在旁边的归档里。在那儿构建会把声明编出来、 +产出一个几乎空的库、然后报告成功。 + +## 一个包,多个 target + +`--target` 可重复。生成的 manifest 每条腿一个条件块,消费者的构建各选各的: + +```toml +[target.'cfg(all(arch = "x86_64", os = "linux", env = "gnu"))'.build] +ldflags = ["-Llib/x86_64-linux-gnu", "-lmathkit"] + +[target.'cfg(all(arch = "x86_64", os = "linux", env = "musl"))'.build] +ldflags = ["-Llib/x86_64-linux-musl", "-lmathkit"] +``` + +因为选择发生在**消费者的构建期**(那时解析后的 target 已知), +胖包的交叉编译天然正确,**不需要索引侧或安装侧做任何支持**。 + +> 这些块是 `cfg(...)`,绝不是裸的 `[target.'<三元组>']` 键。 +> 在 mcpp 2026.8.17.2 之前,裸三元组在没有显式 `--target` 时是失效的 —— +> 用它的包会在 CI 里正常、在开发者机器上静默丢掉 flag。 +> mcpp 生成的是在**所有**客户端上含义一致的那种写法。 + +## 依赖 + +静态归档**不携带**它依赖的代码,所以包会把依赖记下来,由消费者解析: + +```toml +[dependencies] +"compat.zlib" = "1.3.2" +``` + +`path` 与 `git` 依赖会被丢弃:它们指向发布者的磁盘,原样发出去等于 +给消费者一个在他们那里含义完全不同的地址。如果你的库依赖这类东西, +要么把它也发布出去,要么在打包前 vendor 掉。 + +## 老版本 mcpp 拿到这种包会怎样 + +**能构建。** 生成的 manifest 里每一个键都是既有的,所以老客户端读得懂、 +链得上。它做不到的是执行上面那两道闸门 —— 它无从知道 +`provenance = "mcpp-pack …"` 有什么含义。 + +这是**降级**而不是变砖,方向是对的。但它意味着**闸门只保护新客户端**, +如果你面向的是混合版本的用户群,这一条应当写进发布说明。 + +## 当前边界 + +| | 状态 | +|---|---| +| `kind = "lib"`(静态) | ✅ 所有 target,三平台都测了 | +| `kind = "shared"` on Linux/ELF | ✅ —— 包里同时带链接名与 SONAME | +| `kind = "shared"` on PE / Mach-O | ❌ 拒绝 —— 导入库与 install-name 尚未建模 | +| `kind = "shared"` on `*-musl` | ❌ musl target 是静态链接的 | +| 发布预编译 BMI | ❌ 未尝试;BMI 与编译器构建逐位绑定 | +| 把依赖打包进去 | ❌ 改为声明依赖(见上) | + +### 实现分区 + +`mcpp pack` 把实现分区(`module M:part;` 无 `export`)当私有:**源码留下, +对象随归档发出去**。 + +如果被发布的接口 **import 了**一个实现分区,消费者没有那份源码就编不出 BMI, +所以它**会**被发布 —— 而 `mcpp pack` 会说出来: + +``` +warning: secret.cppm is an implementation partition, and the published interface + reaches it — so its SOURCE is being published. +``` + +> 在 mcpp 2026.8.17.2 之前,扫描器把 `module M:part;` 记成**「requires `M:part`、 +> provides 空」** —— 一个文件 requires 自己的名字,于是图里**没有**从「import 分区 +> 的单元」到「定义分区的单元」的边,构建顺序无约束:GCC 与 macOS clang 靠各自的 +> 依赖扫描兜住了,**Windows clang 以 `failed to read compiled module` 失败**。 +> 如果你一直在 Windows 上回避实现分区,原因就是这个。 diff --git a/docs/zh/README.md b/docs/zh/README.md index 59547931..adabc090 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -14,3 +14,4 @@ - [09 - 发布 mcpp](09-release.md) - [10 - 发布一个库到 mcpp-index](10-publishing-a-library.md) - [11 - 机器可读输出](11-machine-output.md) +- [12 - 分发预编译库](12-binary-distribution.md) diff --git a/examples/05-lib-distribution/README.md b/examples/05-lib-distribution/README.md new file mode 100644 index 00000000..d45a7e97 --- /dev/null +++ b/examples/05-lib-distribution/README.md @@ -0,0 +1,105 @@ +# 05 — Distributing a prebuilt library + +Both halves of one story, because they are only interesting together: + +``` +producer/ a library with TWO interfaces — a C header and a C++ module +consumer/ three programs using it: header only, module only, both +``` + +```bash +cd producer && mcpp pack mathkit # → target/dist/mathkit-0.1.0-/ +cd ../consumer # point [dependencies] at it, then: +mcpp run consume-header +mcpp run consume-module +mcpp run consume-both +``` + +Full reference: [docs/12 — Distributing a Prebuilt Library](../../docs/12-binary-distribution.md). + +## There is no `--lib` flag + +What `mcpp pack` produces is decided by `[targets.].kind` — `bin` gives an +application bundle, `lib` and `shared` give library packages. That is already +where mcpp records what an artifact is, and a flag would be a second place to +say it. Publishing both forms means declaring both targets, which the producer +does: + +```toml +[targets.mathkit] kind = "lib" +[targets.mathkit-shared] kind = "shared" soname = "libmathkit.so.1" +``` + +## Read both lists it prints + +``` + Packed leg x86_64-linux-gnu [x86_64-linux-gnu-gcc16-libstdcxx16-c++23] + Interface mathkit.cppm, api.cppm + Withheld capi.c, impl.cpp, secret.cppm +``` + +If you are shipping closed source, **the second line is the one that matters**. +Publishing too little fails loudly in your consumer's compile; publishing too +much silently puts your implementation on someone's disk. + +`producer/src/secret.cppm` is an *implementation partition* +(`module mathkit:secret;`, no `export`). It produces a BMI and an object exactly +like the interface units do — so "publish every module unit" would leak it, and +"publish every `.m.o`" is not a rule mcpp uses. What travels is the **module +closure of the lib root**: `mathkit.cppm` and what its purview imports. + +Prove it: + +```bash +cd producer && mcpp pack mathkit +grep -r house_factor target/dist/*/interface/ ; echo "no match = nothing leaked" +``` + +**Then break it on purpose.** Add `import :secret;` to `src/mathkit.cppm` and +pack again: the interface now reaches the partition, so a consumer needs that +source to compile at all — and `mcpp pack` says so instead of shipping a package +nobody can build. + +## What the consumer's build checks + +Two things, both of which fail silently without a check. + +**The interface still matches its binaries.** Edit one line of +`interface/api.cppm` inside the package and rebuild: + +``` +error: mcpp.mathkit@0.1.0: 'interface' does not match what was packaged. + recorded fnv1a:25b2cf2a79d71c40 + found fnv1a:fe404d5be85118ff +``` + +That refusal exists because the alternative was measured: swap two `int` members +of a struct in a shipped interface — the Itanium ABI does not mangle field order +— and the consumer compiles, links, runs, and prints transposed data, with no +diagnostic from any tool. + +**The binaries were built for your toolchain.** Switch `[toolchain]` and rebuild; +the refusal lists the tags the package *does* have, because "not found" would +send you looking for a package already on your disk. + +## And one thing that will not work + +```bash +cd producer/target/dist/mathkit-0.1.0-*/ && mcpp build +error: … is a distribution package produced by `mcpp pack`, not a source tree. +``` + +Its `interface/` holds declarations whose definitions are in the artifact beside +them. Building there compiles the declarations, produces a near-empty library +and reports success — a failure shaped exactly like a success. + +## Why the tag is shorter for a C-only library + +``` +x86_64-linux-gnu-gcc16-libstdcxx16-c++23 # a C++ module interface +x86_64-linux-gnu # an extern "C" interface only +``` + +A tag names the dimensions the artifact actually constrains, and unnamed ones +are don't-care. So an `extern "C"` library needs one tag per triple instead of +one per triple per compiler — no flag, no mode: the shape *is* the statement. diff --git a/examples/05-lib-distribution/consumer/mcpp.toml b/examples/05-lib-distribution/consumer/mcpp.toml new file mode 100644 index 00000000..f144f3d5 --- /dev/null +++ b/examples/05-lib-distribution/consumer/mcpp.toml @@ -0,0 +1,23 @@ +[package] +name = "lib-consume" +version = "0.1.0" +description = "Consuming a prebuilt library package three different ways" +license = "Apache-2.0" + +# A packed library is an ordinary package: a path dependency, a git dependency, +# a downloaded archive and an index entry all reach it the same way. Point this +# at whatever `mcpp pack mathkit` left under ../producer/target/dist/. +[dependencies] +mathkit = { path = "../producer/target/dist/mathkit-0.1.0-x86_64-linux-gnu-gcc16-libstdcxx16-c++23" } + +[targets.consume-header] +kind = "bin" +main = "src/main_header.cpp" + +[targets.consume-module] +kind = "bin" +main = "src/main_module.cpp" + +[targets.consume-both] +kind = "bin" +main = "src/main_both.cpp" diff --git a/examples/05-lib-distribution/consumer/src/main_both.cpp b/examples/05-lib-distribution/consumer/src/main_both.cpp new file mode 100644 index 00000000..e7a4f95b --- /dev/null +++ b/examples/05-lib-distribution/consumer/src/main_both.cpp @@ -0,0 +1,11 @@ +// Both at once, from the same package. The two interface modes do not +// interfere: one is preprocessor input, the other is compiler input. +import std; +import mathkit; + +#include + +int main() { + std::println("both : c={} module={}", mathkit_add(2, 3), mk::add(2, 3)); + return 0; +} diff --git a/examples/05-lib-distribution/consumer/src/main_header.cpp b/examples/05-lib-distribution/consumer/src/main_header.cpp new file mode 100644 index 00000000..8f9402cb --- /dev/null +++ b/examples/05-lib-distribution/consumer/src/main_header.cpp @@ -0,0 +1,11 @@ +// Consuming through the TEXT interface: no `import` of the library at all, so +// nothing of the package is compiled here. The header is preprocessor input; +// the code it declares lives in the prebuilt artifact. +import std; + +#include + +int main() { + std::println("header : mathkit_add(2, 3) = {}", mathkit_add(2, 3)); + return 0; +} diff --git a/examples/05-lib-distribution/consumer/src/main_module.cpp b/examples/05-lib-distribution/consumer/src/main_module.cpp new file mode 100644 index 00000000..5d9f527b --- /dev/null +++ b/examples/05-lib-distribution/consumer/src/main_module.cpp @@ -0,0 +1,11 @@ +// Consuming through the MODULE interface. mcpp compiles the package's published +// `.cppm` to get a BMI — cheap, they are declarations — and links the +// definitions out of the prebuilt artifact. +import std; +import mathkit; + +int main() { + std::println("module : mk::add(2, 3) = {}, mk::scale(2.0) = {}", + mk::add(2, 3), mk::scale(2.0)); + return 0; +} diff --git a/examples/05-lib-distribution/producer/include/mathkit_c.h b/examples/05-lib-distribution/producer/include/mathkit_c.h new file mode 100644 index 00000000..cc49dc5c --- /dev/null +++ b/examples/05-lib-distribution/producer/include/mathkit_c.h @@ -0,0 +1,13 @@ +/* The TEXT interface: published whole, never trimmed. + A source distribution of this package puts every header here on its + consumers' include path, so a binary package that shipped a subset would + give the same library a different public surface. */ +#ifdef __cplusplus +extern "C" { +#endif + +int mathkit_add(int a, int b); + +#ifdef __cplusplus +} +#endif diff --git a/examples/05-lib-distribution/producer/mcpp.toml b/examples/05-lib-distribution/producer/mcpp.toml new file mode 100644 index 00000000..c3563b41 --- /dev/null +++ b/examples/05-lib-distribution/producer/mcpp.toml @@ -0,0 +1,24 @@ +[package] +name = "mathkit" +version = "0.1.0" +description = "Demo: shipping a prebuilt library with both a header and a module interface" +license = "Apache-2.0" + +[build] +sources = ["src/*.cppm", "src/*.cpp", "src/*.c"] +# The public headers. Published whole — see include/mathkit_c.h. +include_dirs = ["include"] + +# [lib].path is not set, so the convention applies: src/mathkit.cppm is the +# lib root, and the published interface is its module closure. + +# `mcpp pack mathkit` -> a static library package. +[targets.mathkit] +kind = "lib" + +# `mcpp pack mathkit-shared` -> a dynamic library package (Linux/ELF today). +# Two targets rather than a flag: `kind` is where mcpp already records what an +# artifact IS, and `soname` gives the .so the name consumers should load. +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" diff --git a/examples/05-lib-distribution/producer/src/api.cppm b/examples/05-lib-distribution/producer/src/api.cppm new file mode 100644 index 00000000..c2e4bbfc --- /dev/null +++ b/examples/05-lib-distribution/producer/src/api.cppm @@ -0,0 +1,10 @@ +// An INTERFACE partition: `export module`, so it is part of the closure and +// its source travels with the package. +export module mathkit:api; + +export namespace mk { +// Declarations only. The definitions live in the implementation units below +// and reach the consumer as the prebuilt archive. +int add(int a, int b); +double scale(double v); +} diff --git a/examples/05-lib-distribution/producer/src/capi.c b/examples/05-lib-distribution/producer/src/capi.c new file mode 100644 index 00000000..8e3beddb --- /dev/null +++ b/examples/05-lib-distribution/producer/src/capi.c @@ -0,0 +1,3 @@ +/* The header interface's implementation. A consumer that only #includes + mathkit_c.h never compiles a single module unit. */ +int mathkit_add(int a, int b) { return a + b; } diff --git a/examples/05-lib-distribution/producer/src/impl.cpp b/examples/05-lib-distribution/producer/src/impl.cpp new file mode 100644 index 00000000..5926870b --- /dev/null +++ b/examples/05-lib-distribution/producer/src/impl.cpp @@ -0,0 +1,7 @@ +module mathkit; +import :secret; + +namespace mk { +int add(int a, int b) { return a + b; } +double scale(double v) { return v * house_factor(); } +} diff --git a/examples/05-lib-distribution/producer/src/mathkit.cppm b/examples/05-lib-distribution/producer/src/mathkit.cppm new file mode 100644 index 00000000..c88998f8 --- /dev/null +++ b/examples/05-lib-distribution/producer/src/mathkit.cppm @@ -0,0 +1,4 @@ +// The published root. Everything a consumer can `import mathkit;` reaches from +// here — and `mcpp pack` publishes exactly that closure, nothing else. +export module mathkit; +export import :api; diff --git a/examples/05-lib-distribution/producer/src/secret.cppm b/examples/05-lib-distribution/producer/src/secret.cppm new file mode 100644 index 00000000..37b25740 --- /dev/null +++ b/examples/05-lib-distribution/producer/src/secret.cppm @@ -0,0 +1,12 @@ +// An IMPLEMENTATION partition: `module` with no `export`. +// +// It produces a BMI and a `.m.o` exactly like the interface units do — which +// is why "publish every module unit" would leak it. It is NOT reachable from +// the interface's purview, so `mcpp pack` withholds its source and keeps its +// object in the archive. +module mathkit:secret; + +namespace mk { +// Pretend this is the part you are not shipping as source. +double house_factor() { return 2.5; } +} diff --git a/mcpp.toml b/mcpp.toml index e1f18217..a97f59ff 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.17.1" +version = "2026.8.17.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/plan.cppm b/src/build/plan.cppm index e3a36e08..9aac279d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -336,26 +336,10 @@ std::string sanitize_for_path(std::string_view module_name) { return s; } -std::string object_filename_for(const std::filesystem::path& src, - std::string_view objExt = ".o") { - // The naming POLICY lives in mcpp.source_kind (see ObjectNaming there for - // why every historical name is frozen and only new extensions get the - // collision-proof form). This function only formats it. - switch (mcpp::object_naming_for(src)) { - case mcpp::ObjectNaming::StemDotM: - return src.stem().string() + ".m" + std::string(objExt); - case mcpp::ObjectNaming::Stem: - return src.stem().string() + std::string(objExt); - case mcpp::ObjectNaming::FullFilename: - break; - } - // Assembly siblings of a C/C++ TU commonly share its stem (foo.c + - // foo.asm); keeping the full extension means they can never collide — - // the per-package collision prefix can't help two same-stem files in the - // same directory. Every extension a project adds via - // `[build] module_extensions` lands here for the same reason. - return src.filename().string() + std::string(objExt); -} +// Both the naming POLICY and its formatting now live in mcpp.source_kind, so +// the planner and `mcpp pack` cannot answer "what is this object called" +// differently. This alias keeps the local spelling every call site below uses. +using mcpp::object_filename_for; std::string qualified_package_name(const mcpp::manifest::Manifest& manifest) { if (!manifest.package.namespace_.empty() diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a4f91dd7..0a436677 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -46,6 +46,8 @@ import mcpp.platform.capacity; // the host fallback handed to schedule::decide import mcpp.build.graph_shape; // #407: the graph says which mode wrote it import mcpp.build.runtime_validation; // declared artifact -> identity verdict import mcpp.build.cache_key; +import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against +import mcpp.pack.prebuilt; // …and the check itself import mcpp.build.build_program; import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides @@ -202,12 +204,14 @@ materialize_generated_files(const std::filesystem::path& root, // deps, and nothing said why. A package's `[target.windows.dependencies]` is // its own statement about itself and means the same thing whether the package // is the root or someone's dependency. +// The resolved triple travels INSIDE `ctx` (cfgpred::Ctx::triple). It used to +// be a third parameter here too, which is how a bare-triple predicate came to +// disagree with a cfg() one about the same native build — see the note on Ctx. void merge_conditional_config(mcpp::manifest::Manifest& m, - const cfgpred::Ctx& ctx, - std::string_view targetTriple) + const cfgpred::Ctx& ctx) { for (auto const& cc : m.conditionalConfigs) { - if (!cfgpred::matches(cc.predicate, ctx, targetTriple)) continue; + if (!cfgpred::matches(cc.predicate, ctx)) continue; // One append() for every field the axis may carry (#258). Matching // sections land AFTER the base entries, so a conditional rule beats // a broader unconditional one under GNU last-wins — which is what @@ -463,6 +467,9 @@ export struct BuildContext { std::filesystem::path stdBmi; std::filesystem::path stdObject; mcpp::build::BuildPlan plan; + // The scanned module graph. Only `mcpp pack` reads it — see the note at + // the assignment for why the plan cannot answer its question. + mcpp::modgraph::Graph graph; // Resolved profile name (resolve_profile_name). Carried so run_build_plan // can record it in .build_cache — without it the fast path cannot tell // whether a cached build.ninja was generated for the profile being asked @@ -697,6 +704,27 @@ prepare_build(bool print_fingerprint, : mcpp::manifest::load(*root / "mcpp.toml"); if (!m) return std::unexpected(m.error().format()); + // A DISTRIBUTION package is not a source tree, and building "in" one is a + // failure that looks like a success: `interface/` holds declarations whose + // definitions are in the prebuilt archive, so the build compiles the + // declarations, produces a near-empty library, links nothing, and reports + // Finished. The archive it was supposed to carry never enters the picture. + // + // Only the ROOT is refused. As a dependency this is exactly what the + // package is for — the consumer compiles the interface and links the + // artifact, which is the whole design. + if (!overrides.preloaded_manifest && mcpp::pack::is_distribution_package(*m)) { + return std::unexpected(std::format( + "'{}' is a distribution package produced by `mcpp pack`, not a source tree.\n" + " Its sources are interface declarations; the definitions are in the\n" + " prebuilt artifacts beside them, so building here would produce an\n" + " empty library and say it succeeded.\n" + " Use it: add it to a project as a dependency —\n" + " [dependencies]\n" + " {} = {{ path = \"{}\" }}", + root->string(), m->package.name, root->string())); + } + // ─── Workspace handling ──────────────────────────────────────────── // If the manifest has [workspace] and is a virtual workspace (no [package]), // or if -p filter is set, switch to the target member's manifest. @@ -1286,8 +1314,7 @@ prepare_build(bool print_fingerprint, // package's half of the one funnel, not a special case: every package is // merged exactly once, immediately before it is captured into `packages[]`. if (!m->conditionalConfigs.empty()) { - merge_conditional_config(*m, cfgpred::context_for(overrides.target_triple), - overrides.target_triple); + merge_conditional_config(*m, cfgpred::context_for(overrides.target_triple)); } // `[build].defines` must reach the scanner (P1689) and the compile edge, // and must participate in the fingerprint. Fold before dependency @@ -2819,8 +2846,7 @@ prepare_build(bool print_fingerprint, // pass through loadVersionDep. if (!manifest->conditionalConfigs.empty()) { merge_conditional_config(*manifest, - cfgpred::context_for(overrides.target_triple), - overrides.target_triple); + cfgpred::context_for(overrides.target_triple)); } fold_build_defines_into_flags(manifest->buildConfig); @@ -3983,8 +4009,7 @@ prepare_build(bool print_fingerprint, // snapshot this manifest's flags/sources into `packages[]`. if (!dep_manifest->conditionalConfigs.empty()) { merge_conditional_config(*dep_manifest, - cfgpred::context_for(overrides.target_triple), - overrides.target_triple); + cfgpred::context_for(overrides.target_triple)); } fold_build_defines_into_flags(dep_manifest->buildConfig); } else { @@ -5154,11 +5179,53 @@ prepare_build(bool print_fingerprint, roots.push_back(d / "xpkgs"); return roots; }(); + // ─── Prebuilt dependencies: check before planning to link them ───── + // + // Here rather than at each place a dependency manifest is loaded, because + // there are three of those and the check needs the RESOLVED toolchain, + // which only exists by now. One pass over the assembled package list is + // also the only spelling under which a package cannot be checked twice + // with two different answers. + // + // The current tag's SHAPE follows the package's: a package that publishes + // a triple-only tag is saying its interface is `extern "C"`, and comparing + // it against a full tag would refuse a combination it explicitly allows. + // `tag_check` already treats an unnamed dimension as don't-care, so one + // full tag on this side is correct for both. + { + const auto canonicalTriple = tc->targetTriple.empty() + ? mcpp::toolchain::triple::host_triple().str() + : [&] { + auto t = mcpp::toolchain::triple::parse(tc->targetTriple); + return t ? t->str() : tc->targetTriple; + }(); + const auto currentTag = mcpp::pack::cxx_surface_tag( + *tc, canonicalTriple, m->cppStandard.level); + for (std::size_t i = 1; i < packages.size(); ++i) { + auto const& pkg = packages[i]; + if (!mcpp::pack::is_distribution_package(pkg.manifest)) continue; + mcpp::pack::PrebuiltCheck chk{ + .packageRoot = pkg.root, + .packageLabel = mcpp::manifest::package_id(pkg.manifest.package).canonical(), + .current = currentTag, + }; + if (auto ok = mcpp::pack::check_prebuilt(pkg.manifest, chk); !ok) + return std::unexpected(ok.error()); + } + } + auto planResult = mcpp::build::make_plan(*m, *tc, fp, scan.graph, report.topoOrder, packages, *root, ctx.outputDir, stdBmiPath, stdObjectPath, storeRoots); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); + // The module graph outlives the plan for one consumer: `mcpp pack`, which + // has to know which units are INTERFACE (published as source) and which + // are implementation (published only as an object). The plan flattens that + // away — a CompileUnit records what to compile, not what it provides — so + // the packer would otherwise have to scan the tree a second time and could + // then disagree with the build about what the package even contains. + ctx.graph = std::move(scan.graph); // mcpp#407. Both callers that produce a non-plain graph arrive here the // same way: dev-dependencies enabled, synthetic test targets appended. The // resulting `default` line names the test binaries and omits the package's diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index 3e354eee..01bfcac7 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -38,7 +38,21 @@ export namespace mcpp::build { // — not the build host. See the manifest design doc. namespace cfgpred { -struct Ctx { std::string os, arch, family, env; }; +// `triple` is the RESOLVED target — the host's for a native build, the +// --target one for a cross build. It is a member rather than a parameter of +// `matches()` because a bare-triple predicate and a `cfg(...)` predicate are +// two spellings of ONE question, and they must be answered from one value. +// +// They were not. `matches()` used to take the raw `--target` string alongside +// this context and short-circuit on `if (triple.empty()) return false;`, while +// `context_for()` below fell back to the host. So `cfg(linux)` matched a native +// build and `[target.'x86_64-linux-gnu'.build]` — the same statement about the +// same machine — did not, silently. That shape is the worst kind: CI passes +// `--target` and is green, the developer's plain `mcpp build` drops the flags, +// and the failure surfaces at link time naming a symbol instead of a predicate. +// manifest/types.cppm's ConditionalConfig has documented the fallback since it +// was written; this makes the bare-triple branch honour it. +struct Ctx { std::string os, arch, family, env, triple; }; // Derive the cfg context from the resolved --target triple, falling back to // the host for a native build. Parsing goes through triple.cppm — the single @@ -56,12 +70,18 @@ inline Ctx context_for(std::string_view targetTriple) { c.arch = t->arch; c.env = t->env; c.family = t->family(); + // Canonical spelling on BOTH sides of the later comparison, so an + // `x86_64-w64-mingw32` key and an `x86_64-windows-gnu` build agree. + c.triple = t->str(); } else { // Escape-hatch triple outside the language: only the leading arch // segment is derivable; other dimensions stay empty (never match). auto dash = targetTriple.find('-'); c.arch = std::string(dash == std::string_view::npos ? targetTriple : targetTriple.substr(0, dash)); + // Unparseable: keep it verbatim so the exact-string fallback in + // `matches()` can still hit an explicit escape-hatch section. + c.triple = std::string(targetTriple); } return c; } @@ -119,7 +139,11 @@ struct Parser { // Evaluate a `[target.]` key. Returns the cfg() result, or — for a // non-cfg key (a bare triple) — an exact match against the resolved triple. -inline bool matches(const std::string& predicate, const Ctx& c, std::string_view triple) { +// +// The resolved triple comes from `c`, never from a second parameter: see the +// note on Ctx for what having two of them cost. +inline bool matches(const std::string& predicate, const Ctx& c) { + const std::string_view triple = c.triple; std::string_view k = predicate; if (k.starts_with("cfg(") && k.ends_with(")")) { Parser p{ k.substr(4, k.size() - 5), 0, c }; @@ -137,6 +161,10 @@ inline bool matches(const std::string& predicate, const Ctx& c, std::string_view // key matches a resolved `x86_64-windows-gnu` build (and vice versa) — // both normalize through triple::parse. Unparseable keys (the explicit- // section escape hatch) fall back to exact string comparison. + // + // `c.triple` is populated for every build, native included, so this is a + // guard and no longer a behaviour: it used to be the line that made a + // bare-triple section silently inert without `--target`. if (triple.empty()) return false; if (auto p = mcpp::toolchain::triple::parse(predicate)) { if (auto rt = mcpp::toolchain::triple::parse(triple)) diff --git a/src/cli.cppm b/src/cli.cppm index 14e4397a..e202334b 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -406,10 +406,16 @@ int run(int argc, char** argv) { // because a .tar.gz full of DLLs is a package most Windows users // cannot open without installing something first. .description("Build + bundle into a self-contained archive") + // NB: a target NAME from [targets.*], not a triple — the same + // split `mcpp run [target]` has. Its `kind` decides what is + // packed, so there is no --lib and no --artifact: a program + // becomes an application bundle, a library becomes a library + // package. Omit it and mcpp picks the only packable target. + .arg(cl::Arg("target").help("Target name from [targets.*] (optional)")) .option(cl::Option("mode").takes_value() .help("system | vendored (default) | self-contained | static")) - .option(cl::Option("target").takes_value() - .help("Triple, e.g. x86_64-linux-musl")) + .option(cl::Option("target").takes_value().multiple() + .help("Triple, e.g. x86_64-linux-musl (repeatable: one leg per triple)")) .option(cl::Option("format").takes_value() .help("tar (default; .zip for a Windows target) | dir")) .option(cl::Option("output").short_name('o').takes_value() diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 44003704..d9d6d93a 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -10,7 +10,9 @@ export module mcpp.cli.cmd_publish; import std; import mcpplibs.cmdline; import mcpp.pack; +import mcpp.pack.library_pipeline; import mcpp.pack.pipeline; +import mcpp.pack.route; import mcpp.publish.pipeline; import mcpp.ui; @@ -55,9 +57,38 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { } } if (auto v = parsed.value("output")) opts.output = *v; - if (auto v = parsed.value("target")) opts.targetTriple = *v; - return mcpp::pack::build_and_pack(std::move(opts), modeFromUser); + // `--target` is repeatable: one leg per triple, which is how a library + // package ships for several targets at once. The application path has + // always taken exactly one, and still does — packing one executable for + // several triples would need several executables. + std::vector triples; + if (auto o = parsed.option("target")) triples = o->get().values; + if (!triples.empty()) opts.targetTriple = triples.back(); + + // ─── Which target, and therefore which kind of package ─────────── + // + // The positional is a target NAME. Its `kind` decides everything: a + // program becomes an application bundle (the four --mode depths), a + // library becomes a library package. Reading the answer out of the + // manifest is the whole reason there is no --lib flag. + auto route = mcpp::pack::route_pack_target(parsed.positional(0)); + if (!route) { mcpp::ui::error(route.error()); return 2; } + if (route->library) { + if (modeFromUser) { + mcpp::ui::warning(std::format( + "--mode is an application-bundle depth and does not apply to the " + "library target '{}' yet; ignoring it", route->targetName)); + } + return mcpp::pack::build_and_pack_library(route->targetName, triples, opts); + } + if (triples.size() > 1) { + mcpp::ui::error( + "--target may be given once when packing a program: an application " + "bundle wraps one executable, and one executable has one target."); + return 2; + } + return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, route->targetName); } } // namespace mcpp::cli diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 30beedb0..b73deaab 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -237,11 +237,22 @@ std::expected parse_string(std::string_view content, } // [build].sources (M5.0 new home) + [modules].sources (deprecated, compat) - if (auto v = doc->get_string_array("build.sources")) m.buildConfig.sources = *v; + // + // `sourcesDeclared` records PRESENCE, not content: `sources = []` has to + // mean "compile nothing", and only the key's existence can say that (see + // BuildConfig::sourcesDeclared). Set from either spelling, because the + // legacy one has to be able to express it too. + if (auto v = doc->get_string_array("build.sources")) { + m.buildConfig.sources = *v; + m.buildConfig.sourcesDeclared = true; + } if (auto v = doc->get_string_array("modules.sources")) { m.modules.sources = *v; // If [build].sources wasn't set, mirror legacy field into new field. - if (m.buildConfig.sources.empty()) m.buildConfig.sources = *v; + if (!m.buildConfig.sourcesDeclared) { + m.buildConfig.sources = *v; + m.buildConfig.sourcesDeclared = true; + } } // Mirror new → legacy so existing code reading manifest.modules.sources keeps working. if (m.modules.sources.empty()) m.modules.sources = m.buildConfig.sources; @@ -1700,7 +1711,14 @@ void apply_defaults_and_infer(Manifest& m, const std::filesystem::path& root) { // that vendors foreign-syntax .asm can `!`-exclude it. const auto extTable = mcpp::extension_table_for(m.buildConfig.moduleExtensions); - if (m.buildConfig.sources.empty()) { + // `!sourcesDeclared` and not `sources.empty()`: an author who wrote + // `sources = []` asked for NOTHING, and filling the default glob over that + // answers a question they already answered. That is not hypothetical — a + // binary distribution package ships prebuilt artifacts and, in the + // header-only shape, nothing to compile at all; the glob would sweep up + // whatever happens to sit under `src/` and compile it into the consumer's + // build, where it can collide with the prebuilt library's own symbols. + if (!m.buildConfig.sourcesDeclared) { // Derived from the extension table rather than written beside it — // otherwise declaring `module_extensions = [".ixx"]` would change how // `.ixx` is TREATED without changing whether it is FOUND, and the key diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index 1dcf3afb..6240cfe7 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -353,6 +353,25 @@ struct Resources { // is read in ~150 places, and a BuildConfig genuinely IS a set of build // inputs plus the selection axis and resolved policy scalars. struct BuildConfig : BuildInputs { + // Was `sources` WRITTEN, as opposed to merely being empty? + // + // Presence is semantic here for the same reason it is on + // `XlingsConfig::subosDeclared`: an absent key selects the default glob, + // while an explicit `sources = []` selects "compile nothing". A container + // alone cannot tell those apart, and until this flag existed it did not: + // `sources = []` and deleting the line produced byte-identical build + // graphs, so an author had NO spelling for "nothing". + // + // A binary distribution package needs that spelling. It ships prebuilt + // artifacts and, in the header-only shape, no compilable source at all — + // yet any file left under `src/` would be swept up by the default glob and + // compiled into the consumer's build, where it can collide with the very + // symbols the prebuilt library already defines. + // + // Deliberately on BuildConfig and not on BuildInputs: the conditional axis + // (`[target.'cfg(...)'.build]`) only ever APPENDS sources, so "declared + // empty" has no meaning there — it is the same as contributing nothing. + bool sourcesDeclared = false; // `[build] jobs` — how many compiles to run at once. A decimal count, // "auto", or empty (the default) meaning "let the backend decide". // diff --git a/src/modgraph/graph.cppm b/src/modgraph/graph.cppm index 222fa585..2a6c63ed 100644 --- a/src/modgraph/graph.cppm +++ b/src/modgraph/graph.cppm @@ -34,6 +34,21 @@ struct SourceUnit { std::vector packageCxxflags; std::vector packageAsmflags; // per-glob asmflags (G4) std::optional provides; + // Was `provides` declared with `export module`? + // + // Both spellings produce a BMI and an object, so `provides` alone cannot + // tell an INTERFACE partition (`export module M:api;`) from an + // IMPLEMENTATION partition (`module M:impl;`) — and the difference decides + // whether a source may be published. `mcpp pack` publishes the module + // closure of the lib root; a closure that reaches an implementation + // partition has to publish it too (the consumer cannot build the BMI + // without it), and the author needs to be told that their implementation + // is going out. + // + // Defaults to true so units synthesized outside the text scanner + // (scan_overrides, the P1689 reader) keep counting as interfaces — the + // conservative direction, since the flag only ever produces a warning. + bool providesInterface = true; std::vector requires_; // The unit's ROLE, decided once by the scanner from the owning package's // extension table and carried from here on. Every downstream consumer diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index aae39cd2..ca9581ef 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -565,6 +565,9 @@ std::expected scan_file(const std::filesystem::path& file if (!is) return std::unexpected(ScanError{file, 0, "cannot open"}); SourceUnit u; + // The module this TU belongs to (base name, no `:partition`). Set by the + // `module`/`export module` declaration and used to resolve `import :part;`. + std::string owningModule; u.path = file; u.packageName = packageName; // The ONE place a source's role is decided. Everything downstream reads @@ -640,13 +643,47 @@ std::expected scan_file(const std::filesystem::path& file } u.provides = ModuleId{name}; } else { - // implementation unit (`module foo;`) — non-exporting. - // Don't claim ownership of `foo` (partition would be foo:part); - // record import dep on the module's interface. - if (!u.provides) { + // A non-exporting `module …;` is TWO different declarations + // wearing one spelling, and they were treated as one: + // + // module M; a module IMPLEMENTATION UNIT. Belongs to + // M, provides nothing importable, needs M's + // interface. + // module M:part; an IMPLEMENTATION PARTITION. It IS + // importable — as `M:part`, by the other + // units of M — and it produces a BMI and an + // object exactly like an interface partition. + // + // Recording the second as `requires M:part` made every such + // file require its own name and provide nothing. So the graph + // held no edge from the unit that imports the partition to the + // unit that defines it, and the build order was unconstrained. + // Where the compiler's own scan recovers it (GCC, macOS clang) + // the build worked anyway; on Windows clang it failed with + // `failed to read compiled module` — an ordering bug wearing a + // scanner mistake's clothes, and invisible because the scanner + // ALSO warned "imported but not provided in this build" on + // every platform, which read like a note rather than a cause. + if (name.find(':') != std::string::npos) { + if (u.provides) { + return std::unexpected(ScanError{file, lineno, + std::format("file already provides module '{}'; " + "cannot also provide '{}'", + u.provides->logicalName, name)}); + } + u.provides = ModuleId{name}; + u.providesInterface = false; + } else if (!u.provides) { u.requires_.push_back(ModuleId{name}); } } + // The module this TU belongs to, for resolving `import :part;` + // below. Tracked separately from `u.provides` because an + // implementation unit (`module M;`) provides nothing and still has + // to be able to name its own partitions — before this, `import + // :secret;` inside one stayed the literal `:secret`, which no unit + // provides, which is the other half of the same warning. + owningModule = name.substr(0, name.find(':')); continue; } @@ -678,12 +715,8 @@ std::expected scan_file(const std::filesystem::path& file // already includes that suffix — concatenating naively would // produce `foo:http:tls` instead of the intended `foo:tls`. // Strip our own `:partition` first. - if (name.starts_with(":") && u.provides) { - std::string base = u.provides->logicalName; - if (auto p = base.find(':'); p != std::string::npos) { - base.resize(p); - } - name = base + name; + if (name.starts_with(":") && !owningModule.empty()) { + name = owningModule + name; } u.requires_.push_back(ModuleId{name}); continue; diff --git a/src/pack/abi_tag.cppm b/src/pack/abi_tag.cppm new file mode 100644 index 00000000..64322d0e --- /dev/null +++ b/src/pack/abi_tag.cppm @@ -0,0 +1,231 @@ +// mcpp.pack.abi_tag — the readable compatibility tag a prebuilt artifact +// publishes, and the check a consumer runs against it. +// +// WHY A TAG AND NOT THE BUILD FINGERPRINT +// +// There are two compatibility questions and they have different answerers: +// +// "can this binary be linked into your build?" ← the TAG +// "can this BMI be reused verbatim?" ← cache_key::key_hex +// +// Only the first is publishable. A build key folds in the Merkle closure of +// the package's own dependencies, so it is a function of the CONSUMER's +// resolution — a producer would have to enumerate one key per possible +// consumer graph, which is not a finite job. The tag is a projection of facts +// the producer knows alone, which is exactly why it can be written into an +// index before any consumer exists. +// +// THE SHAPE IS THE SURFACE +// +// A tag carries the dimensions the artifact actually constrains, and nothing +// more. A library whose whole interface is `extern "C"` constrains the libc +// ABI and not the C++ one, so it publishes a triple and stops: +// +// x86_64-linux-gnu ← C surface +// x86_64-linux-gnu-gcc16-libstdcxx16-c++23 ← C++ surface +// +// `tag_check` then compares only the dimensions the published tag names. That +// is the same don't-care rule `mcpp.toolchain.abi::abi_check` has used since +// the glfw conflation was fixed, and it means the C case needs no flag, no +// mode and no escape hatch: a shorter tag IS the statement. Tag counts for C +// libraries stay at one-per-triple instead of one-per-triple-per-compiler. +// +// THE TRIPLE IS THE CANONICAL ONE +// +// `arch-os-env` comes from mcpp.toolchain.triple, never from the compiler's +// own `-dumpmachine` answer. Those disagree: gcc reports `x86_64-w64-mingw32` +// where mcpp's target vocabulary — and therefore every `[target.'']` +// key a package can be selected by — says `x86_64-windows-gnu`. Publishing the +// compiler's spelling would give one decision two spellings, and the halves +// would be compared by string. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §1.2. + +export module mcpp.pack.abi_tag; + +import std; +import mcpp.toolchain.model; +import mcpp.toolchain.triple; + +export namespace mcpp::pack { + +// A parsed compatibility tag. An EMPTY dimension means "not constrained", +// never "unknown": the producer decides what to name, and a consumer must not +// invent a constraint the artifact did not declare. +struct AbiTag { + std::string triple; // canonical, e.g. "x86_64-linux-gnu" (never empty) + std::string compiler; // "gcc16" — empty on a C-surface tag + std::string stdlib; // "libstdcxx16" — empty on a C-surface tag + std::string standard; // "c++23" — empty on a C-surface tag + + bool c_surface() const { return compiler.empty() && stdlib.empty() && standard.empty(); } + + std::string str() const { + std::string s = triple; + for (auto const* seg : { &compiler, &stdlib, &standard }) + if (!seg->empty()) { s += '-'; s += *seg; } + return s; + } + + bool operator==(const AbiTag&) const = default; +}; + +// The major version segment of "16.1.0" → "16". Empty input yields "0" rather +// than an empty segment, so a tag never contains a bare "gcc-" that would +// re-split differently on parse. +std::string major_of(std::string_view version); + +// "libstdc++" → "libstdcxx". The tag is joined and split on '-', and a '+' +// inside a segment is fine, but the spelling is also a filename component in +// `target/dist/`, where '+' is best avoided. +std::string stdlib_token(std::string_view stdlibId); + +// The C surface: the triple alone. +AbiTag c_surface_tag(std::string_view canonicalTriple); + +// The C++ surface: every dimension the artifact constrains. +// +// `canonicalTriple` is passed rather than read from `tc.targetTriple` on +// purpose — the caller has already resolved which target is being packed, and +// the toolchain's own triple is the compiler's spelling of it. +AbiTag cxx_surface_tag(const mcpp::toolchain::Toolchain& tc, + std::string_view canonicalTriple, + int cppLevel); + +// Read a published tag back. The optional C++ half is exactly three segments +// and the last of them starts with "c++", so parsing runs from the END: that +// is the only split that is unambiguous when the triple itself contains +// dashes (and it has a variable number of them — `aarch64-macos` has one, +// `x86_64-linux-gnu` has two). +// +// Returns nullopt for a tag with no triple at all, or for a 3-segment tail +// whose last segment is not a `c++NN`. +std::optional parse_abi_tag(std::string_view s); + +// One dimension on which a published tag refuses the current toolchain. +struct TagMismatch { + std::string dimension; // "triple" | "compiler" | "stdlib" | "standard" + std::string need; // what the artifact was built for + std::string got; // what this build resolved +}; + +// Does `published` accept `current`? Empty dimensions in `published` are +// don't-care (see the header). Returns every mismatch so the diagnostic can +// name all of them at once rather than one per rebuild. +// +// `standard` is the one asymmetric dimension: a consumer building at a HIGHER +// level than the artifact was compiled at is fine (the interface it compiles +// is the artifact's own source, and a newer level accepts it), while a lower +// one is not (the interface may use syntax the consumer's level lacks). So it +// is compared as a floor, not for equality. +std::vector tag_check(const AbiTag& published, const AbiTag& current); + +// The `c++NN` segment as its numeric level, or 0 when unparseable. +int standard_level(std::string_view standardSegment); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::string major_of(std::string_view version) { + auto dot = version.find('.'); + auto head = dot == std::string_view::npos ? version : version.substr(0, dot); + // Keep only leading digits: "16.1.0" → "16", "22.1.8" → "22", and a + // vendor string like "19.44.35207" → "19". + std::size_t n = 0; + while (n < head.size() && std::isdigit(static_cast(head[n]))) ++n; + if (n == 0) return "0"; + return std::string(head.substr(0, n)); +} + +std::string stdlib_token(std::string_view stdlibId) { + std::string out; + out.reserve(stdlibId.size()); + for (char c : stdlibId) { + if (c == '+') { out += "x"; continue; } + if (c == '-' || c == ' ') continue; // "msvc-stl" / "MSVC STL" + out += static_cast(std::tolower(static_cast(c))); + } + return out.empty() ? std::string("unknownstl") : out; +} + +AbiTag c_surface_tag(std::string_view canonicalTriple) { + AbiTag t; + t.triple = std::string(canonicalTriple); + return t; +} + +AbiTag cxx_surface_tag(const mcpp::toolchain::Toolchain& tc, + std::string_view canonicalTriple, + int cppLevel) +{ + AbiTag t; + t.triple = std::string(canonicalTriple); + t.compiler = std::string(tc.compiler_name()) + major_of(tc.version); + t.stdlib = stdlib_token(tc.stdlibId) + major_of(tc.stdlibVersion); + t.standard = std::format("c++{}", cppLevel); + return t; +} + +int standard_level(std::string_view seg) { + if (!seg.starts_with("c++")) return 0; + int level = 0; + for (char c : seg.substr(3)) { + if (!std::isdigit(static_cast(c))) return 0; + level = level * 10 + (c - '0'); + } + return level; +} + +std::optional parse_abi_tag(std::string_view s) { + if (s.empty()) return std::nullopt; + + std::vector seg; + for (std::size_t i = 0; i <= s.size(); ) { + auto j = s.find('-', i); + if (j == std::string_view::npos) { seg.push_back(s.substr(i)); break; } + seg.push_back(s.substr(i, j - i)); + i = j + 1; + } + if (seg.empty()) return std::nullopt; + + AbiTag t; + // Parse from the end: the C++ half is exactly three segments and its last + // one is `c++NN`. Anything else is a triple-only (C-surface) tag. + if (seg.size() >= 4 && standard_level(seg.back()) != 0) { + t.standard = std::string(seg[seg.size() - 1]); + t.stdlib = std::string(seg[seg.size() - 2]); + t.compiler = std::string(seg[seg.size() - 3]); + seg.resize(seg.size() - 3); + } + std::string triple; + for (std::size_t i = 0; i < seg.size(); ++i) { + if (i) triple += '-'; + triple += seg[i]; + } + if (triple.empty()) return std::nullopt; + t.triple = std::move(triple); + return t; +} + +std::vector tag_check(const AbiTag& published, const AbiTag& current) { + std::vector out; + auto exact = [&](std::string_view dim, const std::string& need, const std::string& got) { + if (need.empty()) return; // not constrained + if (need != got) out.push_back({ std::string(dim), need, got }); + }; + exact("triple", published.triple, current.triple); + exact("compiler", published.compiler, current.compiler); + exact("stdlib", published.stdlib, current.stdlib); + + // Floor, not equality — see the declaration. + if (!published.standard.empty()) { + int need = standard_level(published.standard); + int got = standard_level(current.standard); + if (need != 0 && got != 0 && got < need) + out.push_back({ "standard", published.standard, current.standard }); + } + return out; +} + +} // namespace mcpp::pack diff --git a/src/pack/digest.cppm b/src/pack/digest.cppm new file mode 100644 index 00000000..60917d04 --- /dev/null +++ b/src/pack/digest.cppm @@ -0,0 +1,61 @@ +// mcpp.pack.digest — the content digests a library package records, and the +// consumer re-computes. +// +// Its own module, and a leaf, because BOTH sides need it: `mcpp pack` writes +// the digests and `mcpp build` verifies them. Putting it in the packer would +// make the build depend on the packer, which already depends on the build. +// +// `fnv1a:` and not `sha256:`. Three reasons, in order of weight: +// +// * it needs no external tool. `mcpp publish` shells out to `sha256sum`, +// which is not on a Windows host — and a package format that can only be +// verified on some of the platforms it targets is not a format. +// * the question is "has this changed since it was packaged", not "can an +// adversary forge a collision". Transport integrity is the index entry's +// `sha256` over the whole tarball; this is the second line, for a package +// that has already been extracted or handed over as a directory. +// * `mcpp.lock` already records `fnv1a:` digests, so the vocabulary is one +// a reader of this project has already met. + +export module mcpp.pack.digest; + +import std; +import mcpp.toolchain.fingerprint; + +export namespace mcpp::pack { + +// `fnv1a:<16 hex>` over one file's bytes. +std::string file_digest(const std::filesystem::path& p); + +// The digest of an ordered interface set: each file's NAME and then its +// content hash, sorted by name. +// +// The name is folded in deliberately. Renaming a published interface unit +// changes what a consumer compiles — the `sources` list, and therefore the +// module the BMI comes from — even when not one byte of any file changed. +std::string interface_set_digest(const std::vector& files); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::string file_digest(const std::filesystem::path& p) { + return "fnv1a:" + mcpp::toolchain::hash_file(p); +} + +std::string interface_set_digest(const std::vector& files) { + std::vector> byName; + for (auto const& f : files) byName.emplace_back(f.filename().string(), f); + std::ranges::sort(byName, {}, &std::pair::first); + + std::string acc; + for (auto const& [name, path] : byName) { + acc += name; + acc += '\x1f'; + acc += mcpp::toolchain::hash_file(path); + acc += '\x1e'; + } + return "fnv1a:" + mcpp::toolchain::hash_string(acc); +} + +} // namespace mcpp::pack diff --git a/src/pack/interface.cppm b/src/pack/interface.cppm new file mode 100644 index 00000000..00f35efd --- /dev/null +++ b/src/pack/interface.cppm @@ -0,0 +1,168 @@ +// mcpp.pack.interface — which module units a library package publishes. +// +// THE ASYMMETRY THAT DECIDES THE DESIGN +// +// A prebuilt package ships module interface SOURCE (the consumer has to +// compile it to get a BMI) and a prebuilt library for everything else. So the +// packer has to answer "which .cppm travel?" — and the two ways of getting it +// wrong are not symmetric: +// +// ship too FEW → the consumer's compile fails, loudly, naming the module: +// `mathkit:secret: error: failed to read compiled module` +// ship too MANY → a closed-source implementation partition's SOURCE is +// published. Nothing fails. Nobody finds out. +// +// One of those is a bug report and the other is a disclosure, which is why the +// set is COMPUTED from the module graph and cannot be hand-written in the +// manifest. An author-supplied list drifts, and it drifts toward the silent +// side as the package evolves. +// +// WHY `.m.o` IS NOT THE ANSWER +// +// The tempting shortcut is "publish the sources of the units that produce a +// BMI". It is wrong, measured: an implementation partition (`module M:impl;`, +// no `export`) also produces a `.m.o`, and its source must NOT be published. +// `.m.o` means "module unit", not "interface". +// +// The same closure has a SECOND use, and getting that one wrong was also +// measured: the archive members to delete before shipping are the objects of +// the units being published as source — not every `.m.o`. Dropping every +// `.m.o` deletes the implementation partition's real code, and every target +// then fails to link with `undefined reference` to a symbol that is nowhere +// in the diagnostic's vicinity. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2.4.2. + +export module mcpp.pack.interface; + +import std; +import mcpp.modgraph.graph; +import mcpp.source_kind; + +export namespace mcpp::pack { + +// What `mcpp pack` publishes, withholds, and cannot decide. +struct InterfaceClosure { + // The module root the closure started from, e.g. "mathkit". + std::string rootModule; + // Units whose SOURCE travels, in a stable order (root first, then the + // order they were reached). Paths are exactly the graph's, so the caller + // can relativize them against whichever root it knows about. + std::vector published; + // This package's other module units — implementation units, implementation + // partitions, and any module the interface never reaches. Reported so + // `mcpp pack` can print both lists: "what you are publishing" is only half + // of what an author of a closed-source library needs to see. + std::vector withheld; + // Module names the interface imports that NOTHING in this graph provides. + // + // The published set is then INCOMPLETE and the consumer's build will fail + // on it, so this is a hard error at pack time rather than a warning: the + // whole point of the closure is that the failure lands on the person who + // can fix it. + std::vector unresolvedImports; + // Published units that are IMPLEMENTATION partitions (`module M:part;`). + // + // Reaching one from the interface's purview is legal and the consumer needs + // its source to build the BMI at all — so it is published, not refused. But + // for a closed-source library it is the one outcome nobody wants by + // accident, and it is invisible in the source: `import :detail;` in an + // interface looks like any other import. So it is reported, loudly, and + // `mcpp pack` prints it as a warning naming the file. + std::vector publishedImplementationPartitions; +}; + +// Compute the closure for `packageName`, starting at the unit that provides +// `rootModule`. +// +// Only units belonging to `packageName` are followed: an interface may import +// a DEPENDENCY's module, and that module is the dependency's to publish, not +// ours. Such an import is neither published nor unresolved — it is simply not +// this package's business. +std::expected +interface_closure(const mcpp::modgraph::Graph& graph, + std::string_view packageName, + std::string_view rootModule); + +// The archive members to delete before shipping: the objects of the units in +// `closure.published`. +// +// Deliberately derived from the closure and not from a file extension — see +// the header for what "delete every .m.o" cost when it was tried. +std::vector published_object_names(const InterfaceClosure& closure, + std::string_view objExt = ".o"); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::expected +interface_closure(const mcpp::modgraph::Graph& graph, + std::string_view packageName, + std::string_view rootModule) +{ + InterfaceClosure out; + out.rootModule = std::string(rootModule); + + auto owned = [&](const mcpp::modgraph::SourceUnit& u) { + return u.packageName == packageName; + }; + + auto rootIt = graph.producerOf.find(rootModule); + if (rootIt == graph.producerOf.end()) { + return std::unexpected(std::format( + "no module interface unit in this build provides '{}'", rootModule)); + } + if (!owned(graph.units[rootIt->second])) { + return std::unexpected(std::format( + "module '{}' is provided by package '{}', not '{}'", rootModule, + graph.units[rootIt->second].packageName, packageName)); + } + + std::vector stack{ rootIt->second }; + std::set seen; + std::set unresolved; + + while (!stack.empty()) { + auto idx = stack.front(); + stack.erase(stack.begin()); // breadth-first: root first, then its imports + if (!seen.insert(idx).second) continue; + out.published.push_back(graph.units[idx].path); + if (!graph.units[idx].providesInterface) + out.publishedImplementationPartitions.push_back(graph.units[idx].path); + + for (auto const& req : graph.units[idx].requires_) { + auto it = graph.producerOf.find(req.logicalName); + if (it == graph.producerOf.end()) { + // Only OUR module's partitions are our problem. A bare name + // with no producer is a dependency's module (or `std`), which + // this package does not publish and must not complain about. + const bool ours = req.logicalName.starts_with(std::string(rootModule) + ":"); + if (ours) unresolved.insert(req.logicalName); + continue; + } + if (!owned(graph.units[it->second])) continue; // a dependency's unit + if (!seen.contains(it->second)) stack.push_back(it->second); + } + } + + for (std::size_t i = 0; i < graph.units.size(); ++i) { + if (!owned(graph.units[i])) continue; + if (seen.contains(i)) continue; + out.withheld.push_back(graph.units[i].path); + } + out.unresolvedImports.assign(unresolved.begin(), unresolved.end()); + return out; +} + +std::vector published_object_names(const InterfaceClosure& closure, + std::string_view objExt) +{ + std::vector names; + names.reserve(closure.published.size()); + for (auto const& p : closure.published) + names.push_back(mcpp::object_filename_for(p, objExt)); + return names; +} + +} // namespace mcpp::pack diff --git a/src/pack/library.cppm b/src/pack/library.cppm new file mode 100644 index 00000000..baf079b9 --- /dev/null +++ b/src/pack/library.cppm @@ -0,0 +1,347 @@ +// mcpp.pack.library — staging a LIBRARY package (interface + prebuilt +// artifacts), as opposed to mcpp.pack's application bundle. +// +// The two share a command and almost nothing else. An application bundle is +// "one executable plus the closure it needs at RUN time"; a library package is +// "the source a consumer must compile, plus the binaries it then links". The +// decision between them is not a flag: it is `[targets.].kind`, so +// `mcpp pack ` reads the answer instead of asking for it. +// +// WHAT THIS FILE IS CAREFUL ABOUT +// +// 1. It never GLOBS for a built artifact. Every path comes from the build +// this run just did. A `target//` directory accumulates one +// subdirectory per fingerprint, and picking "the first one" silently +// selects a stale binary — measured while prototyping this feature, where +// it produced an archive missing a translation unit and a consumer-side +// `undefined reference` that pointed nowhere near the cause. +// +// 2. The archive members it deletes are the objects of the units it is +// PUBLISHING AS SOURCE, computed by mcpp.pack.interface. Not "every +// `.m.o`" — an implementation partition is a module unit too, and its +// object holds the only copy of code nobody else has. +// +// 3. Digests are `fnv1a:` and not `sha256:`. The lock file already speaks +// that vocabulary, it needs no external tool (`sha256sum` is not on a +// Windows host), and the question it answers is "has this changed since +// packaging" — for which a 64-bit content hash is evidence, not a +// security boundary. The tarball's own `sha256` in an index entry is the +// integrity check for transport. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2.3. + +module; +#include + +export module mcpp.pack.library; + +import std; +import mcpp.pack.digest; +import mcpp.pack.manifest_emit; +import mcpp.pack.zip; +import mcpp.platform; + +export namespace mcpp::pack { + +// One target triple's build output, as this run produced it. +struct LibraryLeg { + std::string triple; // canonical + std::filesystem::path artifact; // absolute; the .a / .so this build wrote + std::filesystem::path archiveTool; // `ar` for THIS leg's toolchain (empty = skip drop) + std::string abiTag; + std::string buildKey; + std::string linkName; // the -l argument, e.g. "mathkit" + // How THIS leg's archiver spells "delete a member", from + // mcpp.toolchain.dialect. `ar` and `llvm-ar` take `d …`; + // LIB.EXE takes `/REMOVE:` per member with the archive last. The + // packer must not pick one — see the dialect's note. + std::string removeArg; // "d" | "/REMOVE:{}" + bool removeArchiveFirst = true; + // The SONAME the artifact declares, when it declares one. A shared library + // is FOUND at run time by this name and LINKED by `lib.so`, and + // those are two different filenames — so a package that ships only the + // built file links fine and then cannot start. + std::string soname; + bool shared = false; +}; + +struct LibraryPackPlan { + std::filesystem::path projectRoot; + std::filesystem::path stagingRoot; // target/dist/ + std::filesystem::path archivePath; // …/.tar.gz | .zip + bool writeArchive = true; // false = `--format dir` + bool zip = false; // PE targets get a .zip + + std::string namespace_, packageName, packageVersion, builtBy; + std::string targetName; + bool targetShared = false; + std::string cxxRuntime; + std::vector exportsModules; + + // Absolute paths from the module graph. + std::vector interfaceSources; + // Archive member names to delete (mcpp.pack.interface::published_object_names). + std::vector dropObjects; + + std::filesystem::path includeDir; // absolute, or empty + std::vector> dependencies; + std::vector extras; // README / LICENSE / [pack].include hits + + std::vector legs; +}; + +// NOT `Error`. `mcpp.pack` already exports a `mcpp::pack::Error`, and a name +// can only be attached to one module — clang rejects the second outright +// ("cannot be attached to other modules"), and mcpp.pack.library_pipeline +// imports both. GCC accepted it, which is exactly why the Windows leg of CI +// is the one that found this. +struct LibraryPackError { std::string message; }; + +// Stage, drop, describe, archive. Returns the path a caller should report. +// +// The digests it records come from mcpp.pack.digest, which the CONSUMER also +// uses — one derivation, verified from both ends. +std::expected run_library_pack(const LibraryPackPlan& plan); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +namespace { + +std::expected copy_into(const std::filesystem::path& src, + const std::filesystem::path& dst) +{ + std::error_code ec; + std::filesystem::create_directories(dst.parent_path(), ec); + std::filesystem::copy_file(src, dst, + std::filesystem::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(LibraryPackError{ std::format( + "cannot copy '{}' -> '{}': {}", src.string(), dst.string(), ec.message()) }); + return {}; +} + +// Every file under `root`, relative, sorted — deterministic archive order. +std::vector walk(const std::filesystem::path& root) { + std::vector out; + std::error_code ec; + for (auto const& e : std::filesystem::recursive_directory_iterator(root, ec)) { + if (!e.is_regular_file()) continue; + out.push_back(std::filesystem::relative(e.path(), root, ec)); + } + std::ranges::sort(out); + return out; +} + +} // namespace + +std::expected +run_library_pack(const LibraryPackPlan& plan) +{ + std::error_code ec; + std::filesystem::remove_all(plan.stagingRoot, ec); + std::filesystem::create_directories(plan.stagingRoot, ec); + if (ec) return std::unexpected(LibraryPackError{ std::format( + "cannot create staging dir '{}': {}", plan.stagingRoot.string(), ec.message()) }); + + // ── interface/ ──────────────────────────────────────────────────── + // + // Flattened, because the names land in `sources` and a package's internal + // directory layout is not something a consumer should inherit. A collision + // is refused rather than resolved: silently renaming one of two files + // called `api.cppm` would make the emitted `sources` point at the wrong + // one, and the failure would surface in someone else's build. + std::vector interfaceNames; + { + std::map seen; + for (auto const& src : plan.interfaceSources) { + auto name = src.filename().string(); + if (auto it = seen.find(name); it != seen.end()) { + return std::unexpected(LibraryPackError{ std::format( + "two interface units are both called '{}':\n" + " {}\n {}\n" + "A package's interface is published flat, so their names must differ.", + name, it->second.string(), src.string()) }); + } + seen.emplace(name, src); + if (auto r = copy_into(src, plan.stagingRoot / "interface" / name); !r) + return std::unexpected(r.error()); + interfaceNames.push_back("interface/" + name); + } + std::ranges::sort(interfaceNames); + } + + // ── include/ ────────────────────────────────────────────────────── + // + // Whole, never filtered. A source distribution of this package puts every + // one of these on its consumers' include path (usage requirements), so + // trimming here would give the same library a different public surface + // depending on how it was delivered. + if (!plan.includeDir.empty() && std::filesystem::is_directory(plan.includeDir, ec)) { + for (auto const& rel : walk(plan.includeDir)) + if (auto r = copy_into(plan.includeDir / rel, plan.stagingRoot / "include" / rel); !r) + return std::unexpected(r.error()); + } + + // ── lib// ───────────────────────────────────────────────── + std::vector docLegs; + for (auto const& leg : plan.legs) { + if (!std::filesystem::exists(leg.artifact, ec)) { + return std::unexpected(LibraryPackError{ std::format( + "the build for '{}' produced no artifact at '{}'", + leg.triple, leg.artifact.string()) }); + } + auto name = leg.artifact.filename().string(); + auto dst = plan.stagingRoot / "lib" / leg.triple / name; + if (auto r = copy_into(leg.artifact, dst); !r) return std::unexpected(r.error()); + + // A shared library needs BOTH of its names present. + // + // `-lmathkit-shared` resolves `libmathkit-shared.so` at link time, but + // the object records `SONAME libmathkit.so.1`, and that is the name the + // loader asks for. Ship only the built file and the consumer links, + // then fails to start — mcpp's own runtime-closure check reports + // "libmathkit.so.1 not found on the search path this artifact will + // actually use", which is how this was caught. + // + // A symlink is what a distribution ships; a copy is the fallback for + // filesystems (and archives) that cannot carry one. + if (leg.shared && !leg.soname.empty() && leg.soname != name) { + auto alias = dst.parent_path() / leg.soname; + std::error_code linkEc; + std::filesystem::remove(alias, linkEc); + std::filesystem::create_symlink(name, alias, linkEc); + if (linkEc) + if (auto r = copy_into(leg.artifact, alias); !r) return std::unexpected(r.error()); + } + + // Delete the objects of the units published as source. The consumer + // compiles those itself; leaving them in the archive means two + // definitions of the module initialiser, resolved by link order. + // No archiver but objects to drop would leave the published interface's + // objects inside the archive — two definitions of the module + // initialiser, resolved by link order. Skipping that quietly is the + // exact failure class this feature exists to remove, so it is refused. + if (!leg.shared && !plan.dropObjects.empty() && leg.archiveTool.empty()) { + return std::unexpected(LibraryPackError{ std::format( + "no archiver was resolved for {}, so the published interface's " + "objects cannot be removed from '{}'.\n" + " Shipping them leaves two definitions of each published " + "module's initialiser in the consumer's link.", + leg.triple, name) }); + } + if (!leg.shared && !plan.dropObjects.empty()) { + std::string cmd = mcpp::platform::shell::quote(leg.archiveTool.string()); + auto member_words = [&] { + std::string w; + for (auto const& m : plan.dropObjects) { + // `{}` means one flag per member (LIB.EXE); its absence + // means one verb followed by every member (ar). + auto pos = leg.removeArg.find("{}"); + if (pos == std::string::npos) { w += " " + mcpp::platform::shell::quote(m); continue; } + auto arg = leg.removeArg; + arg.replace(pos, 2, m); + w += " " + mcpp::platform::shell::quote(arg); + } + return w; + }; + if (leg.removeArchiveFirst) { + if (leg.removeArg.find("{}") == std::string::npos) + cmd += " " + leg.removeArg; + cmd += " " + mcpp::platform::shell::quote(dst.string()); + cmd += member_words(); + } else { + cmd += member_words(); + cmd += " " + mcpp::platform::shell::quote(dst.string()); + } + auto r = mcpp::platform::process::capture(cmd + " 2>&1"); + if (r.exit_code != 0) { + return std::unexpected(LibraryPackError{ std::format( + "cannot drop published interface objects from '{}' (rc={}).\n" + " command: {}\n" + " output : {}\n" + " Leaving them in would give the consumer two definitions of " + "each published module's initialiser.", + dst.string(), r.exit_code, cmd, r.output) }); + } + } + + docLegs.push_back(PackageLeg{ + .triple = leg.triple, + .libFile = name, + .linkName = leg.linkName, + .abiTag = leg.abiTag, + .digest = file_digest(dst), + .buildKey = leg.buildKey, + .shared = leg.shared, + }); + } + + // ── extras ──────────────────────────────────────────────────────── + for (auto const& x : plan.extras) { + auto rel = std::filesystem::relative(x, plan.projectRoot, ec); + if (ec || rel.empty() || rel.string().starts_with("..")) rel = x.filename(); + if (auto r = copy_into(x, plan.stagingRoot / rel); !r) return std::unexpected(r.error()); + } + + // ── mcpp.toml ───────────────────────────────────────────────────── + { + PackageDoc doc; + doc.namespace_ = plan.namespace_; + doc.name = plan.packageName; + doc.version = plan.packageVersion; + doc.builtBy = plan.builtBy; + doc.interfaceFiles = interfaceNames; + doc.hasIncludeDir = std::filesystem::is_directory(plan.stagingRoot / "include", ec); + doc.interfaceDigest = plan.interfaceSources.empty() && !doc.hasIncludeDir + ? std::string{} + : interface_set_digest(plan.interfaceSources.empty() + ? [&] { + std::vector hdrs; + for (auto const& rel : walk(plan.stagingRoot / "include")) + hdrs.push_back(plan.stagingRoot / "include" / rel); + return hdrs; + }() + : plan.interfaceSources); + doc.cxxRuntime = plan.cxxRuntime; + doc.exportsModules = plan.exportsModules; + doc.targetName = plan.targetName; + doc.targetShared = plan.targetShared; + doc.legs = std::move(docLegs); + doc.dependencies = plan.dependencies; + + std::ofstream os(plan.stagingRoot / "mcpp.toml", std::ios::binary); + if (!os) return std::unexpected(LibraryPackError{ std::format( + "cannot write '{}'", (plan.stagingRoot / "mcpp.toml").string()) }); + os << emit_package_manifest(doc); + } + + if (!plan.writeArchive) return plan.stagingRoot; + + // ── archive ─────────────────────────────────────────────────────── + std::filesystem::create_directories(plan.archivePath.parent_path(), ec); + if (plan.zip) { + std::vector entries; + const auto wrapper = plan.stagingRoot.filename().string(); + for (auto const& rel : walk(plan.stagingRoot)) { + entries.push_back(zip::Entry{ + .name = wrapper + "/" + rel.generic_string(), + .source = plan.stagingRoot / rel, + }); + } + if (auto r = zip::write(plan.archivePath, entries); !r) + return std::unexpected(LibraryPackError{ r.error() }); + } else { + auto cmd = std::format("tar -czf {} -C {} {}", + mcpp::platform::shell::quote(plan.archivePath.string()), + mcpp::platform::shell::quote(plan.stagingRoot.parent_path().string()), + mcpp::platform::shell::quote(plan.stagingRoot.filename().string())); + auto r = mcpp::platform::process::capture(cmd + " 2>&1"); + if (r.exit_code != 0) + return std::unexpected(LibraryPackError{ std::format( + "tar failed (rc={}): {}", r.exit_code, r.output) }); + } + return plan.archivePath; +} + +} // namespace mcpp::pack diff --git a/src/pack/library_pipeline.cppm b/src/pack/library_pipeline.cppm new file mode 100644 index 00000000..4462deec --- /dev/null +++ b/src/pack/library_pipeline.cppm @@ -0,0 +1,338 @@ +// mcpp.pack.library_pipeline — everything `mcpp pack ` does when the +// named target is a library. +// +// The sibling of mcpp.pack.pipeline (which packs an application). They are +// separate files rather than one branch because they share almost nothing: +// an application bundle asks "what does this executable need at RUN time", +// a library package asks "what must a consumer compile, and what may it link". +// +// WHICH ONE RUNS IS NOT A FLAG. `[targets.].kind` already says whether a +// target is a program or a library, so the command reads that answer instead +// of asking for it again — there is no `--lib`, and no `--artifact static`. +// A project that publishes both forms declares both targets, which is also +// what it must do for `mcpp build` to produce both. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2. + +module; +#include + +export module mcpp.pack.library_pipeline; + +import std; +import mcpp.build.backend; +import mcpp.build.ninja; +import mcpp.build.plan; +import mcpp.build.prepare; +import mcpp.manifest; +import mcpp.modgraph.graph; +import mcpp.modgraph.scanner; +import mcpp.pack; +import mcpp.pack.abi_tag; +import mcpp.pack.interface; +import mcpp.pack.library; +import mcpp.toolchain.dialect; +import mcpp.toolchain.registry; +import mcpp.toolchain.triple; +import mcpp.ui; +import mcpp.version; + +namespace mcpp::pack { + +namespace { + +// The lib-root's own unit. +// +// Both facts the packer needs come from here, and taking them from the SAME +// unit is the point: the module name (mcpp has not required it to match the +// package name since 0.0.10 — the author names the module, the scanner detects +// it) and the package name the scanner stamped on every unit. Recomputing the +// latter from the manifest would be a second derivation of a value the graph +// already carries, and the closure compares against it by string. +const mcpp::modgraph::SourceUnit* root_unit_of(const mcpp::modgraph::Graph& g, + const std::filesystem::path& libRoot) +{ + std::error_code ec; + for (auto const& u : g.units) { + if (!u.provides) continue; + if (std::filesystem::equivalent(u.path, libRoot, ec)) return &u; + } + return nullptr; +} + +// Dependencies the package can honestly carry downstream. +// +// The same rule `mcpp emit xpkg` applies, and for the same reason: a `path` +// or `git` dependency addresses the PRODUCER's disk or a revision only they +// can resolve, so republishing it hands the consumer an address that means +// something else (or nothing) on their machine. Version dependencies are the +// only kind that survive the trip. +std::vector> +publishable_dependencies(const mcpp::manifest::Manifest& m) +{ + std::vector> out; + for (auto const& [k, v] : m.dependencies) { + if (v.isPath() || v.isGit() || v.version.empty()) continue; + out.emplace_back(k, v.version); + } + return out; +} + +std::vector extras_of(const mcpp::manifest::Manifest& m, + const std::filesystem::path& root) +{ + std::vector out; + std::error_code ec; + for (auto const* name : { "README.md", "README", "LICENSE", "LICENSE.txt", "COPYING" }) + if (std::filesystem::is_regular_file(root / name, ec)) out.push_back(root / name); + // `[pack].include` keeps the meaning it has for an application bundle: + // EXTRA files to ship. It has never reached the interface or the headers, + // and it must not start to — those two sets are computed, and letting a + // glob trim them would give the same library a different public surface + // depending on how it was delivered. + for (auto const& glob : m.packConfig.include) + for (auto const& hit : mcpp::modgraph::expand_glob(root, glob)) + if (std::filesystem::is_regular_file(hit, ec)) out.push_back(hit); + return out; +} + +} // namespace + +// `mcpp pack ` for a `kind = "lib"` / `"shared"` target. +// +// `triples` is the `--target` list; empty means "this host". Each entry gets +// its own prepare+build, so the artifacts really are the ones this run made — +// the packer never searches `target/` for something that looks right. +export int build_and_pack_library(const std::string& targetName, + const std::vector& triples, + const mcpp::pack::Options& opts) +{ + std::vector legs = triples; + if (legs.empty()) legs.push_back({}); // one leg, this host + + LibraryPackPlan plan; + plan.builtBy = std::string(mcpp::MCPP_VERSION); + plan.writeArchive = opts.format == mcpp::pack::Format::Tar; + + InterfaceClosure closure; + bool haveClosure = false; + std::string firstTriple; + + for (auto const& want : legs) { + mcpp::build::BuildOverrides ov; + ov.target_triple = want; + auto ctx = mcpp::build::prepare_build(false, /*includeDevDeps=*/false, {}, ov); + if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } + + // ── the target, and what its kind means here ────────────────── + const mcpp::manifest::Target* target = nullptr; + for (auto const& t : ctx->manifest.targets) + if (t.name == targetName) { target = &t; break; } + if (!target) { + std::string names; + for (auto const& t : ctx->manifest.targets) { + if (!names.empty()) names += ", "; + names += t.name; + } + mcpp::ui::error(std::format( + "no target named '{}' in this package{}{}", + targetName, names.empty() ? "" : "; available: ", names)); + return 2; + } + const bool shared = target->kind == mcpp::manifest::Target::SharedLibrary; + + const auto triple = ctx->tc.targetTriple.empty() + ? mcpp::toolchain::triple::host_triple().str() + : [&] { + auto t = mcpp::toolchain::triple::parse(ctx->tc.targetTriple); + return t ? t->str() : ctx->tc.targetTriple; + }(); + + // ── build, then take the artifact FROM THE PLAN ─────────────── + // + // Never a glob over `target/`: that directory holds one subtree per + // fingerprint, and picking one by name or by mtime silently selects a + // stale binary. The link unit knows its own output. + auto be = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions bo; + if (auto br = be->build(ctx->plan, bo); !br) { + mcpp::ui::error(br.error().message); + return 1; + } + std::filesystem::path artifact; + for (auto const& lu : ctx->plan.linkUnits) { + if (lu.targetName != targetName) continue; + if (lu.kind != mcpp::build::LinkUnit::StaticLibrary + && lu.kind != mcpp::build::LinkUnit::SharedLibrary) continue; + artifact = ctx->outputDir / lu.output; + break; + } + if (artifact.empty()) { + mcpp::ui::error(std::format( + "target '{}' produced no library artifact for {}", targetName, triple)); + return 1; + } + + // ── the interface closure ───────────────────────────────────── + auto libRoot = ctx->projectRoot / mcpp::manifest::resolve_lib_root_path(ctx->manifest); + std::error_code ec; + InterfaceClosure here; + std::string qualifiedPackage; + if (std::filesystem::is_regular_file(libRoot, ec)) { + auto const* rootUnit = root_unit_of(ctx->graph, libRoot); + if (!rootUnit) { + mcpp::ui::error(std::format( + "'{}' is the lib root but provides no module interface", + libRoot.string())); + return 1; + } + qualifiedPackage = rootUnit->packageName; + auto c = interface_closure(ctx->graph, qualifiedPackage, + rootUnit->provides->logicalName); + if (!c) { mcpp::ui::error(c.error()); return 1; } + here = std::move(*c); + } + // else: a header-only package. `sources = []` in the emitted manifest + // says so explicitly, which is a thing a manifest can say now. + + if (!here.unresolvedImports.empty()) { + std::string list; + for (auto const& m : here.unresolvedImports) { + if (!list.empty()) list += ", "; + list += m; + } + mcpp::ui::error(std::format( + "the published interface imports {} , which no unit in this build " + "provides.\n" + " A consumer compiling the published interface would fail with\n" + " \"failed to read compiled module\". Either publish that unit (make it\n" + " an `export module` partition) or keep it out of the interface's purview.", + list)); + return 1; + } + + // Publishing an implementation partition is legal — the consumer cannot + // build the interface's BMI without it — but for a closed-source + // library it is the one thing nobody wants by accident, and nothing in + // the source looks unusual: `import :detail;` reads like any other + // import. So say it, with the file named. + for (auto const& p : here.publishedImplementationPartitions) { + mcpp::ui::warning(std::format( + "{} is an implementation partition, and the published interface " + "reaches it — so its SOURCE is being published.\n" + " A consumer compiling the interface cannot produce a BMI " + "without it. If that source should stay private, move what the " + "interface needs into an `export module` partition and keep the " + "rest out of the interface's purview.", + p.filename().string())); + } + + if (!haveClosure) { + closure = std::move(here); + haveClosure = true; + firstTriple = triple; + plan.projectRoot = ctx->projectRoot; + plan.namespace_ = ctx->manifest.package.namespace_; + plan.packageName = ctx->manifest.package.name; + plan.packageVersion = ctx->manifest.package.version; + plan.targetName = targetName; + plan.targetShared = shared; + plan.cxxRuntime = ctx->manifest.buildConfig.cxxRuntime; + plan.dependencies = publishable_dependencies(ctx->manifest); + plan.extras = extras_of(ctx->manifest, ctx->projectRoot); + plan.interfaceSources = closure.published; + plan.dropObjects = published_object_names(closure); + for (auto const& d : ctx->manifest.buildConfig.includeDirs) { + auto abs = d.is_absolute() ? d : ctx->projectRoot / d; + if (std::filesystem::is_directory(abs, ec)) { plan.includeDir = abs; break; } + } + for (auto const& u : ctx->graph.units) + if (u.provides && u.provides->logicalName.find(':') == std::string::npos + && u.packageName == qualifiedPackage) + plan.exportsModules.push_back(u.provides->logicalName); + std::ranges::sort(plan.exportsModules); + plan.exportsModules.erase( + std::ranges::unique(plan.exportsModules).begin(), plan.exportsModules.end()); + } else if (closure.published != here.published) { + // One package, one `sources` list. If a conditional source glob + // makes the INTERFACE differ per target, the package cannot + // describe itself, and quietly publishing the first leg's answer + // would ship an interface that does not match some of the binaries. + mcpp::ui::error(std::format( + "the published interface differs between {} and {}.\n" + " A package has one `sources` list, so its interface must be the same\n" + " for every target it ships. Move the per-target difference behind\n" + " the implementation, or publish one package per target.", + firstTriple, triple)); + return 1; + } + + // ── the tag ─────────────────────────────────────────────────── + // + // A package whose interface is only headers constrains the libc ABI + // and not the C++ one, so it publishes the shorter tag and links into + // any compiler. The SHAPE is the statement; there is no flag for it. + const bool cxxSurface = !closure.published.empty(); + auto tag = cxxSurface + ? cxx_surface_tag(ctx->tc, triple, ctx->manifest.cppStandard.level) + : c_surface_tag(triple); + + plan.legs.push_back(LibraryLeg{ + .triple = triple, + .artifact = artifact, + .archiveTool = shared ? std::filesystem::path{} + : mcpp::toolchain::archive_tool(ctx->tc), + .abiTag = tag.str(), + .buildKey = ctx->fp.hex, + .linkName = targetName, + .removeArg = std::string( + mcpp::toolchain::dialect_for(ctx->tc).archiveRemoveArg), + .removeArchiveFirst = + mcpp::toolchain::dialect_for(ctx->tc).archiveRemoveTakesArchiveFirst, + .soname = target->soname, + .shared = shared, + }); + mcpp::ui::status("Packed leg", std::format("{} [{}]", triple, tag.str())); + } + + // ── where it lands ──────────────────────────────────────────────── + const bool zip = plan.legs.size() == 1 + && plan.legs[0].triple.find("windows") != std::string::npos; + auto dirName = plan.legs.size() == 1 + ? std::format("{}-{}-{}", plan.packageName, plan.packageVersion, plan.legs[0].abiTag) + : std::format("{}-{}", plan.packageName, plan.packageVersion); + plan.zip = zip; + plan.stagingRoot = plan.projectRoot / "target" / "dist" / dirName; + plan.archivePath = plan.projectRoot / "target" / "dist" + / (dirName + (zip ? ".zip" : ".tar.gz")); + if (!opts.output.empty()) { + auto o = std::filesystem::path(opts.output); + plan.archivePath = o.has_parent_path() ? o + : plan.projectRoot / "target" / "dist" / o; + } + + // ── say what travels, and what does not ─────────────────────────── + // + // Both lists, always. "What you are publishing" is only half of what the + // author of a closed-source library needs to see before uploading. + { + std::string pub, held; + for (auto const& p : closure.published) { + if (!pub.empty()) pub += ", "; + pub += p.filename().string(); + } + for (auto const& p : closure.withheld) { + if (!held.empty()) held += ", "; + held += p.filename().string(); + } + mcpp::ui::status("Interface", pub.empty() ? "(headers only)" : pub); + mcpp::ui::status("Withheld", held.empty() ? "(nothing)" : held); + } + + auto out = run_library_pack(plan); + if (!out) { mcpp::ui::error(out.error().message); return 1; } + mcpp::ui::status("Packed", out->string()); + return 0; +} + +} // namespace mcpp::pack diff --git a/src/pack/manifest_emit.cppm b/src/pack/manifest_emit.cppm new file mode 100644 index 00000000..5bdbf045 --- /dev/null +++ b/src/pack/manifest_emit.cppm @@ -0,0 +1,236 @@ +// mcpp.pack.manifest_emit — the `mcpp.toml` that travels inside a library +// package. +// +// IT IS AN ORDINARY MANIFEST, AND THAT IS THE DESIGN +// +// Two earlier drafts of this format invented somewhere to put the packaging +// facts: first a sibling `MCPP-PACKAGE.toml`, then a `[distribution]` section. +// Both were deleted, because every fact already had a home: +// +// what it is where it goes already parsed by +// ----------------- ----------------------------------- ----------------- +// the interface [build] sources / include_dirs yes +// how to link it [target.'cfg(...)'.build] ldflags yes +// its dependencies [dependencies] yes +// which modules [modules] exports yes +// the C++ runtime [build] cxx_runtime yes +// the artifacts [[runtime.artifacts]] yes +// +// So a consumer needs no new code path to USE one of these packages: mcpp's +// existing "the payload carries its own mcpp.toml" route (Form A) reads it, +// whether it arrives as a path dependency, a git dependency, a file, or an +// index tarball. And an mcpp too old to run the gate still BUILDS against the +// package, because none of the keys are new — it just does not check them. +// A new section would have been silently skipped instead, leaving no record +// at all; `[[runtime.artifacts]]` is a section old clients already write into +// `resolution.json`. +// +// WHAT IS EVIDENCE, AND WHY IT LOOKS LIKE THE REST +// +// `[[runtime.artifacts]]` carries `role`, `abi`, `digest`, `provenance` and +// `host_fingerprint` — the exact fields the gate needs, and documented as +// "optional evidence" since they were introduced. `provenance` starting with +// `mcpp-pack` is what MARKS a directory as a distribution package; nothing +// else needs to say so. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §2.4. + +export module mcpp.pack.manifest_emit; + +import std; + +export namespace mcpp::pack { + +// One target triple's worth of the package: the artifact built for it and the +// conditional block that selects it. +struct PackageLeg { + std::string triple; // canonical, e.g. "x86_64-linux-gnu" + std::string libFile; // "libmathkit.a" — as it sits in lib// + std::string linkName; // "mathkit" — the -l argument + std::string abiTag; // "x86_64-linux-gnu-gcc16-libstdcxx16-c++23" + std::string digest; // "sha256:…" + std::string buildKey; // cache_key::key_hex, or empty + bool shared = false; +}; + +struct PackageDoc { + std::string namespace_; + std::string name; + std::string version; + std::string builtBy; // "mcpp 2026.8.17.1" + + // Package-relative, e.g. "interface/mathkit.cppm". Empty is legal and + // meaningful: a header-only package compiles nothing, and the emitted + // `sources = []` says exactly that (an omitted key would be filled with + // the default glob and would sweep up whatever sits under src/). + std::vector interfaceFiles; + bool hasIncludeDir = false; + std::string interfaceDigest; // over the ordered interface set + + std::string cxxRuntime; // empty = do not write the key + std::vector exportsModules; + + std::string targetName; + bool targetShared = false; + + std::vector legs; + // Raw `[dependencies]` keys as the producer wrote them, paired with the + // version. Path/git deps are the caller's to drop — they are local-only + // and cannot be resolved by anyone downstream. + std::vector> dependencies; +}; + +// The cfg() predicate that selects exactly `triple`. +// +// ⚠️ NOT a bare `[target.''.build]` key. A bare triple is only matched +// when the user passes `--target`; a plain `mcpp build` resolves the host and +// used to compare it against an empty string, so the section was silently +// inert. That is fixed (mcpp.build.prepare_inputs), but generating cfg() is +// still the right output: it is what the same statement means on every mcpp, +// including the ones already installed. +std::string cfg_predicate_for(std::string_view triple); + +std::string emit_package_manifest(const PackageDoc& doc); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +namespace { + +// TOML basic-string escaping, restricted to what a manifest can contain. +std::string quote(std::string_view s) { + std::string out = "\""; + for (char c : s) { + switch (c) { + case '"': out += "\\\""; break; + case '\\': out += "\\\\"; break; + case '\n': out += "\\n"; break; + case '\t': out += "\\t"; break; + default: out += c; break; + } + } + out += '"'; + return out; +} + +std::string join_quoted(const std::vector& v) { + std::string out; + for (std::size_t i = 0; i < v.size(); ++i) { + if (i) out += ", "; + out += quote(v[i]); + } + return out; +} + +} // namespace + +std::string cfg_predicate_for(std::string_view triple) { + // arch-os[-env]; the canonical spelling from mcpp.toolchain.triple. + std::vector seg; + for (std::size_t i = 0; i <= triple.size(); ) { + auto j = triple.find('-', i); + if (j == std::string_view::npos) { seg.push_back(triple.substr(i)); break; } + seg.push_back(triple.substr(i, j - i)); + i = j + 1; + } + if (seg.size() < 2) return std::format("cfg(arch = \"{}\")", triple); + + std::string p = std::format("cfg(all(arch = \"{}\", os = \"{}\"", seg[0], seg[1]); + // env is named only when the triple names it. Writing `env = ""` would be + // a constraint the triple did not make, and the evaluator has no spelling + // for "unset" anyway. + if (seg.size() >= 3 && !seg[2].empty()) + p += std::format(", env = \"{}\"", seg[2]); + p += "))"; + return p; +} + +std::string emit_package_manifest(const PackageDoc& doc) { + std::string o; + o += std::format( + "# Generated by `{}`. Do not edit.\n" + "#\n" + "# Editing anything under interface/ invalidates the digest recorded in\n" + "# the `role = \"interface\"` artifact below, and mcpp refuses to build\n" + "# against a package whose interface no longer matches its binaries.\n\n", + doc.builtBy.empty() ? "mcpp pack" : doc.builtBy); + + o += "[package]\n"; + if (!doc.namespace_.empty()) o += std::format("namespace = {}\n", quote(doc.namespace_)); + o += std::format("name = {}\n", quote(doc.name)); + o += std::format("version = {}\n\n", quote(doc.version)); + + // ── the interface ────────────────────────────────────────────────── + o += "[build]\n"; + o += std::format("sources = [{}]\n", join_quoted(doc.interfaceFiles)); + if (doc.hasIncludeDir) o += "include_dirs = [\"include\"]\n"; + if (!doc.cxxRuntime.empty()) + o += std::format("cxx_runtime = {}\n", quote(doc.cxxRuntime)); + o += "\n"; + + if (!doc.exportsModules.empty()) { + o += "[modules]\n"; + o += std::format("exports = [{}]\n\n", join_quoted(doc.exportsModules)); + } + + o += std::format("[targets.{}]\n", doc.targetName); + o += std::format("kind = \"{}\"\n\n", doc.targetShared ? "shared" : "lib"); + + // ── how to link each leg ─────────────────────────────────────────── + for (auto const& leg : doc.legs) { + o += std::format("[target.'{}'.build]\n", cfg_predicate_for(leg.triple)); + o += std::format("ldflags = [\"-Llib/{}\", \"-l{}\"]\n\n", leg.triple, leg.linkName); + } + // A shared library has to be FOUND at run time as well as linked, and the + // two are different search paths — `link_library_dirs` is not rpath. + if (doc.targetShared && !doc.legs.empty()) { + o += "[runtime]\n"; + std::vector dirs; + for (auto const& leg : doc.legs) dirs.push_back(std::format("lib/{}", leg.triple)); + o += std::format("runtime_search_dirs = [{}]\n\n", join_quoted(dirs)); + } + + // ── dependencies ─────────────────────────────────────────────────── + // + // A static archive does NOT carry its dependencies' code, so a consumer + // that does not resolve them cannot link. Carrying the producer's + // `[dependencies]` verbatim is what makes the package usable at all. + if (!doc.dependencies.empty()) { + o += "[dependencies]\n"; + for (auto const& [k, v] : doc.dependencies) + o += std::format("{} = {}\n", quote(k), quote(v)); + o += "\n"; + } + + // ── evidence ─────────────────────────────────────────────────────── + for (auto const& leg : doc.legs) { + o += "[[runtime.artifacts]]\n"; + o += std::format("role = \"{}\"\n", + leg.shared ? "shared-library" : "static-library"); + o += std::format("path = {}\n", + quote(std::format("lib/{}/{}", leg.triple, leg.libFile))); + o += std::format("provenance = {}\n", + quote(std::format("mcpp-pack {}", doc.builtBy))); + if (!leg.abiTag.empty()) o += std::format("abi = {}\n", quote(leg.abiTag)); + if (!leg.digest.empty()) o += std::format("digest = {}\n", quote(leg.digest)); + if (!leg.buildKey.empty()) o += std::format("host_fingerprint = {}\n", quote(leg.buildKey)); + o += "\n"; + } + if (!doc.interfaceDigest.empty()) { + // One entry for the whole set, not one per file. What it defends + // against is post-extraction editing, and "interface/ no longer + // matches" is already actionable; per-file digests would add a line + // per file to a document whose whole point is to stay readable. + o += "[[runtime.artifacts]]\n"; + o += "role = \"interface\"\n"; + o += std::format("path = {}\n", + quote(doc.interfaceFiles.empty() ? "include" : "interface")); + o += std::format("provenance = {}\n", + quote(std::format("mcpp-pack {}", doc.builtBy))); + o += std::format("digest = {}\n\n", quote(doc.interfaceDigest)); + } + return o; +} + +} // namespace mcpp::pack diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 8dad8f23..2f8ae739 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -23,7 +23,15 @@ import mcpp.ui; namespace mcpp::pack { // Everything after CLI option parsing for `mcpp pack`. -export int build_and_pack(Options opts, bool modeFromUser) { +// +// `wantTarget` is the target NAME the user asked for, empty when they did not. +// It exists because `mcpp pack ` now routes on `[targets.].kind`: +// a name that resolves to a program has to reach the binary selection below, +// or a project with two `bin` targets would accept `mcpp pack app2` and +// silently bundle app1 — the shape where the command succeeds and the answer +// is wrong. +export int build_and_pack(Options opts, bool modeFromUser, + const std::string& wantTarget = {}) { // `--target *-linux-musl` without an explicit `--mode` implies // `--mode static` — packaging a musl-static ELF as bundle-project // would feed patchelf a static binary and crash. The docs treat @@ -83,8 +91,26 @@ export int build_and_pack(Options opts, bool modeFromUser) { } // ─── Pick the main binary target ───────────────────────────────── + // + // An explicitly named target wins over the package-name convention: the + // user said which one, and guessing past that is how `mcpp pack app2` + // would produce app1's bundle under app2's name. std::filesystem::path mainBinary; + if (!wantTarget.empty()) { + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind == mcpp::build::LinkUnit::Binary && lu.targetName == wantTarget) { + mainBinary = ctx->outputDir / lu.output; + break; + } + } + if (mainBinary.empty()) { + mcpp::ui::error(std::format( + "target '{}' is not a program in this build", wantTarget)); + return 2; + } + } for (auto& lu : ctx->plan.linkUnits) { + if (!mainBinary.empty()) break; if (lu.kind == mcpp::build::LinkUnit::Binary && lu.targetName == ctx->manifest.package.name) { diff --git a/src/pack/prebuilt.cppm b/src/pack/prebuilt.cppm new file mode 100644 index 00000000..0e6e5b89 --- /dev/null +++ b/src/pack/prebuilt.cppm @@ -0,0 +1,166 @@ +// mcpp.pack.prebuilt — the consumer's half of a library package. +// +// A package produced by `mcpp pack` is an ordinary mcpp package: it carries a +// normal `mcpp.toml`, and mcpp's existing "the payload has its own manifest" +// route builds against it with no new code. That is the design, and it is why +// an mcpp too old to know about any of this still WORKS with these packages. +// +// What this module adds is the part an old mcpp cannot do: check that the +// binaries in the package were built for the toolchain about to link them, and +// that the interface sitting next to them is still the one they were built +// from. Both failures are otherwise silent. +// +// THE SECOND ONE IS THE DANGEROUS ONE. Measured on a real build: change one +// line of a shipped interface — swap two `int` members of a struct, which the +// Itanium ABI does not mangle — and the consumer compiles, links, runs, and +// prints transposed data. No diagnostic at any stage. A digest cannot prevent +// a producer from shipping a mismatched pair in the first place (only atomic +// production does that), but it does catch the pair coming apart afterwards, +// which is the case a path dependency or an extracted store is exposed to. +// +// WHAT MARKS A PACKAGE. `provenance` beginning with `mcpp-pack` on any runtime +// artifact. No new key, no new section: the marker is a value in a field that +// has existed since runtime artifacts did. +// +// Design: .agents/docs/2026-08-17-library-distribution-design.md §3.2. + +export module mcpp.pack.prebuilt; + +import std; +import mcpp.manifest; +import mcpp.pack.abi_tag; +import mcpp.pack.digest; + +export namespace mcpp::pack { + +inline constexpr std::string_view kPackProvenancePrefix = "mcpp-pack"; + +// Was this manifest written by `mcpp pack`? +bool is_distribution_package(const mcpp::manifest::Manifest& m); + +struct PrebuiltCheck { + std::filesystem::path packageRoot; + std::string packageLabel; // "acme.mathkit@0.1.0", for diagnostics + AbiTag current; // the tag THIS build would publish +}; + +// Refuse, with a message the reader can act on, or accept. +// +// Order matters and is the order of the diagnostic: a package that is for +// another architecture entirely should say so before it complains about a +// digest, because the digest is not what the user has to fix. +std::expected +check_prebuilt(const mcpp::manifest::Manifest& m, const PrebuiltCheck& in); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +namespace { + +bool is_library_role(std::string_view role) { + return role == "static-library" || role == "shared-library"; +} + +} // namespace + +bool is_distribution_package(const mcpp::manifest::Manifest& m) { + for (auto const& a : m.runtimeConfig.artifacts) + if (a.provenance.starts_with(kPackProvenancePrefix)) return true; + return false; +} + +std::expected +check_prebuilt(const mcpp::manifest::Manifest& m, const PrebuiltCheck& in) +{ + std::error_code ec; + + // ── 1. the artifacts are where the manifest says ────────────────── + // + // A package whose `lib/` was trimmed in transit links against nothing and + // fails at the linker, naming a path nobody recognises. + for (auto const& a : m.runtimeConfig.artifacts) { + if (!a.provenance.starts_with(kPackProvenancePrefix)) continue; + auto abs = a.path.is_absolute() ? a.path : in.packageRoot / a.path; + if (!std::filesystem::exists(abs, ec)) { + return std::unexpected(std::format( + "{}: the package declares an artifact at '{}', and it is not there.\n" + " The package is incomplete — re-download or re-pack it.", + in.packageLabel, a.path.string())); + } + } + + // ── 2. one of the published tags accepts this toolchain ─────────── + std::vector publishedTags; + bool sawLibrary = false, accepted = false; + std::vector bestRefusal; + std::string bestRefusalTag; + + for (auto const& a : m.runtimeConfig.artifacts) { + if (!a.provenance.starts_with(kPackProvenancePrefix)) continue; + if (!is_library_role(a.role)) continue; + sawLibrary = true; + if (a.abi.empty()) { // nothing declared → nothing to enforce + accepted = true; + continue; + } + publishedTags.push_back(a.abi); + auto published = parse_abi_tag(a.abi); + if (!published) { accepted = true; continue; } // unreadable → lenient + auto bad = tag_check(*published, in.current); + if (bad.empty()) { accepted = true; break; } + // Keep the CLOSEST refusal to show: the one that disagrees least is + // the one the user is most likely able to act on. + if (bestRefusal.empty() || bad.size() < bestRefusal.size()) { + bestRefusal = bad; + bestRefusalTag = a.abi; + } + } + + if (sawLibrary && !accepted) { + std::string tags; + for (auto const& t : publishedTags) tags += std::format("\n {}", t); + std::string why; + for (auto const& b : bestRefusal) + why += std::format("\n {:<9} needs {}, this build has {}", b.dimension, b.need, b.got); + return std::unexpected(std::format( + "{}: no prebuilt artifact matches this toolchain.\n" + " your toolchain : {}\n" + " published tags :{}\n" + " closest is {}, and it differs on:{}\n" + " fix: ask the publisher for a build matching your toolchain, or pin\n" + " [toolchain] to the one the package was built with.", + in.packageLabel, in.current.str(), tags, bestRefusalTag, why)); + } + + // ── 3. the interface is the one the binaries were built from ────── + for (auto const& a : m.runtimeConfig.artifacts) { + if (!a.provenance.starts_with(kPackProvenancePrefix)) continue; + if (a.role != "interface" || a.digest.empty()) continue; + + auto dir = a.path.is_absolute() ? a.path : in.packageRoot / a.path; + std::vector files; + if (std::filesystem::is_directory(dir, ec)) { + for (auto const& e : std::filesystem::recursive_directory_iterator(dir, ec)) + if (e.is_regular_file()) files.push_back(e.path()); + } else if (std::filesystem::is_regular_file(dir, ec)) { + files.push_back(dir); + } + auto now = interface_set_digest(files); + if (now != a.digest) { + return std::unexpected(std::format( + "{}: '{}' does not match what was packaged.\n" + " recorded {}\n" + " found {}\n" + " The published interface and the prebuilt binaries are produced\n" + " together and are not separately replaceable: an edited interface\n" + " compiles and links against binaries that no longer agree with it,\n" + " and the result is wrong at run time with no diagnostic.\n" + " fix: restore the package from its original archive.", + in.packageLabel, a.path.string(), a.digest, now)); + } + } + return {}; +} + +} // namespace mcpp::pack diff --git a/src/pack/route.cppm b/src/pack/route.cppm new file mode 100644 index 00000000..988e8770 --- /dev/null +++ b/src/pack/route.cppm @@ -0,0 +1,116 @@ +// mcpp.pack.route — which target `mcpp pack` packs, and therefore which of the +// two pipelines runs. +// +// The routing question has exactly one input: `[targets.].kind`. A program +// becomes an application bundle, a library becomes a library package. That is +// why there is no `--lib` flag and no `--artifact static|shared` — every one +// of those would be a second place to answer a question the manifest already +// answers, and the two answers could then disagree. +// +// Reading the manifest here (rather than inside each pipeline) keeps the +// decision ahead of the build: `mcpp pack nosuch` should say so in +// milliseconds, not after compiling the project. + +module; +#include + +export module mcpp.pack.route; + +import std; +import mcpp.manifest; +import mcpp.project; + +export namespace mcpp::pack { + +struct PackRoute { + std::string targetName; + bool library = false; // kind = lib | shared +}; + +// Resolve `requested` (possibly empty) against the current project. +// +// Empty picks the only packable target. A project with both a program and a +// library has no single obvious answer, so it is asked rather than guessed: +// packing the wrong one produces a plausible-looking archive of the wrong +// shape, which is worse than an error. +std::expected route_pack_target(std::string_view requested); + +} // namespace mcpp::pack + +namespace mcpp::pack { + +std::expected route_pack_target(std::string_view requested) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) return std::unexpected("no mcpp.toml in current dir or parents"); + auto m = mcpp::manifest::load(*root / "mcpp.toml"); + if (!m) return std::unexpected(m.error().format()); + + auto is_library = [](const mcpp::manifest::Target& t) { + return t.kind == mcpp::manifest::Target::Library + || t.kind == mcpp::manifest::Target::SharedLibrary; + }; + auto kind_name = [](const mcpp::manifest::Target& t) -> std::string_view { + switch (t.kind) { + case mcpp::manifest::Target::Binary: return "bin"; + case mcpp::manifest::Target::Library: return "lib"; + case mcpp::manifest::Target::SharedLibrary: return "shared"; + case mcpp::manifest::Target::TestBinary: return "test"; + } + return "?"; + }; + + if (!requested.empty()) { + for (auto const& t : m->targets) { + if (t.name != requested) continue; + if (t.kind == mcpp::manifest::Target::TestBinary) + return std::unexpected(std::format( + "target '{}' is a test binary; there is nothing to distribute", + requested)); + return PackRoute{ t.name, is_library(t) }; + } + std::string list; + for (auto const& t : m->targets) { + if (!list.empty()) list += ", "; + list += std::format("{} ({})", t.name, kind_name(t)); + } + return std::unexpected(std::format( + "no target named '{}'{}{}", requested, + list.empty() ? "" : "; this package declares: ", list)); + } + + // A WORKSPACE ROOT has no targets of its own — a virtual one has no + // `[package]` at all. `mcpp pack` there has always meant "pack the member", + // and the application pipeline is what resolves which member that is. So + // hand it straight through rather than reading the root's (empty) target + // list and concluding there is nothing to pack. + // + // Found by running the old binary and the new one against + // examples/04-workspace: the routing added here turned a working command + // into "this package declares no program and no library to pack". + if (m->targets.empty() && m->workspace.present) return PackRoute{ {}, false }; + + // Nothing requested. A program is still the default — `mcpp pack` has + // always meant "bundle this application" and a project that has one is + // asking for that. + const mcpp::manifest::Target* onlyLib = nullptr; + std::size_t libCount = 0; + for (auto const& t : m->targets) { + if (t.kind == mcpp::manifest::Target::Binary) return PackRoute{ t.name, false }; + if (is_library(t)) { onlyLib = &t; ++libCount; } + } + if (libCount == 1) return PackRoute{ onlyLib->name, true }; + if (libCount == 0) + return std::unexpected("this package declares no program and no library to pack"); + + std::string list; + for (auto const& t : m->targets) { + if (!is_library(t)) continue; + if (!list.empty()) list += ", "; + list += std::format("{} ({})", t.name, kind_name(t)); + } + return std::unexpected(std::format( + "this package declares more than one library, so `mcpp pack` cannot pick " + "one for you.\n Name it: {}", list)); +} + +} // namespace mcpp::pack diff --git a/src/source_kind.cppm b/src/source_kind.cppm index 1ec14efd..28992404 100644 --- a/src/source_kind.cppm +++ b/src/source_kind.cppm @@ -187,6 +187,17 @@ enum class ObjectNaming { // than weakening the exhaustive no-collision assertion around it. ObjectNaming object_naming_for(const std::filesystem::path& src); +// The object file's NAME, formatted from the policy above. +// +// It lived in plan.cppm as an internal helper, under a comment saying the +// policy lives here and it "only formats it" — which is exactly the split +// worth closing: a second reader of the policy is a second place the format +// can drift. `mcpp pack` needs the same answer (to tell which archive members +// belong to the interface units it is publishing as source), and reaching into +// plan.cppm for it would drag the whole planner into the packer. +std::string object_filename_for(const std::filesystem::path& src, + std::string_view objExt = ".o"); + } // namespace mcpp namespace mcpp { @@ -369,4 +380,22 @@ ObjectNaming object_naming_for(const std::filesystem::path& src) { return ObjectNaming::FullFilename; } +std::string object_filename_for(const std::filesystem::path& src, + std::string_view objExt) { + switch (object_naming_for(src)) { + case ObjectNaming::StemDotM: + return src.stem().string() + ".m" + std::string(objExt); + case ObjectNaming::Stem: + return src.stem().string() + std::string(objExt); + case ObjectNaming::FullFilename: + break; + } + // Assembly siblings of a C/C++ TU commonly share its stem (foo.c + + // foo.asm); keeping the full extension means they can never collide — + // the per-package collision prefix can't help two same-stem files in the + // same directory. Every extension a project adds via + // `[build] module_extensions` lands here for the same reason. + return src.filename().string() + std::string(objExt); +} + } // namespace mcpp diff --git a/src/toolchain/dialect.cppm b/src/toolchain/dialect.cppm index 1adc549f..dff73c5a 100644 --- a/src/toolchain/dialect.cppm +++ b/src/toolchain/dialect.cppm @@ -89,6 +89,23 @@ struct CommandDialect { // Full ninja command template for static archives. std::string_view archiveCmd; // "$ar rcs $out $in" | "$ar /nologo /OUT:$out $in" + + // How this archiver DELETES a member, as argv words before the member + // names. `mcpp pack` needs it: a library package publishes its interface + // units as source, so their objects have to come OUT of the archive, or the + // consumer links two definitions of each published module's initialiser. + // + // It lives here rather than in the packer because "how do you speak to the + // archiver" is exactly what this table is for. Hard-coding `ar`-style `d` + // there would be right for GNU and llvm-ar and wrong for LIB.EXE, which + // spells it `/REMOVE:` and takes one flag per member — a difference + // in ARITY as well as in spelling, which is why this is a template with a + // placeholder rather than a prefix string. + // + // `{}` is substituted with the member name; the words are joined with + // spaces after the archive path. + std::string_view archiveRemoveArg; // "d" needs no {} | "/REMOVE:{}" + bool archiveRemoveTakesArchiveFirst = true; }; // Dialect lookup. GCC / Clang / MinGW → gnu; MSVC → msvc. @@ -189,6 +206,9 @@ constexpr CommandDialect kGnuDialect{ .rspfileLink = false, .linkStyle = CommandDialect::LinkStyle::Driver, .archiveCmd = "$ar rcs $out $in", + // `ar d ...` — one verb, then every member. + .archiveRemoveArg = "d", + .archiveRemoveTakesArchiveFirst = true, }; // Native cl.exe. Unreachable in builds until the MSVC backend lands @@ -218,6 +238,12 @@ constexpr CommandDialect kMsvcDialect{ .rspfileLink = true, .linkStyle = CommandDialect::LinkStyle::SeparateLinker, .archiveCmd = "$ar /nologo /OUT:$out $in", + // `LIB /REMOVE: … ` — one flag PER member, and the + // archive last. Untested against a real LIB.EXE (mcpp's Windows CI packs + // with clang, whose llvm-ar takes the GNU form), so a failure here is + // reported with the command that produced it rather than swallowed. + .archiveRemoveArg = "/REMOVE:{}", + .archiveRemoveTakesArchiveFirst = false, }; } // namespace diff --git a/src/version.cppm b/src/version.cppm index 13ca3351..a4be1f92 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.17.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.17.2"; } // namespace mcpp diff --git a/tests/e2e/242_pack_library_interface_and_headers.sh b/tests/e2e/242_pack_library_interface_and_headers.sh new file mode 100755 index 00000000..f7eb5c30 --- /dev/null +++ b/tests/e2e/242_pack_library_interface_and_headers.sh @@ -0,0 +1,124 @@ +#!/usr/bin/env bash +# requires: +# (no capability: "a library package works on every target" has to be TESTED on +# every target. `# requires: gcc` would skip this on macOS and Windows — Apple +# Clang is not the gcc capability — leaving the claim unverified while the +# suite stayed green.) +# +# Deliberately NO implementation partition here. This test is about the two +# interface modes; 243 owns partitions, and they do not build on Windows/clang +# today (a pre-existing gap this work found, see 243's header). +# 242_pack_library_interface_and_headers.sh — `mcpp pack ` produces +# a package a consumer can use through EITHER interface mode, or both at once. +# +# Acceptance for §1.1 / §2 of +# .agents/docs/2026-08-17-library-distribution-design.md: a package carries +# `include/` (text, consumed by #include) and `interface/` (module units the +# consumer compiles), the two do not interfere, and neither needs a flag. +# +# Also pins the two lists `mcpp pack` prints. A closed-source publisher needs +# to see what is NOT travelling as much as what is. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# ── the producer: a module interface AND a C header, over one library ── +mkdir -p mathkit/src mathkit/include +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export import :api; +EOF +cat > mathkit/src/api.cppm <<'EOF' +export module mathkit:api; +export namespace mk { int add(int a, int b); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int add(int a, int b) { return a + b; } } +EOF +cat > mathkit/src/capi.c <<'EOF' +int mathkit_add(int a, int b) { return a + b; } +EOF +cat > mathkit/include/mathkit_c.h <<'EOF' +#ifdef __cplusplus +extern "C" { +#endif +int mathkit_add(int a, int b); +#ifdef __cplusplus +} +#endif +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp", "src/*.c"] +include_dirs = ["include"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } + +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" +[[ -n "$pkg" ]] || { cat pack.log; echo "no package directory"; exit 1; } + +# The layout is the contract: two interface modes, one artifact dir per triple. +for f in mcpp.toml interface/mathkit.cppm interface/api.cppm include/mathkit_c.h; do + [[ -f "$pkg/$f" ]] || { echo "package is missing $f"; find "$pkg" -type f; exit 1; } +done +# NOT a hard-coded `libmathkit.a`: the artifact's name follows the ENVIRONMENT, +# not the OS — MinGW writes libfoo.a where MSVC writes foo.lib. That is exactly +# why `lib/` is keyed by triple, and hard-coding the GNU spelling here made this +# test fail on Windows against a package that had been built correctly. +[[ -n "$(find "$pkg/lib" -type f \( -name 'libmathkit.*' -o -name 'mathkit.*' \) | head -1)" ]] || { + echo "package has no artifact under lib//"; find "$pkg" -type f; exit 1; } + +# Both lists are printed, and the implementation partition is on the right one. +grep -q 'Interface.*mathkit.cppm' pack.log || { cat pack.log; echo "no interface list"; exit 1; } +grep -q 'Withheld.*impl.cpp' pack.log || { cat pack.log; echo "no withheld list"; exit 1; } + +cd "$TMP" + +# ── three consumers: header only, module only, both ──────────────────── +consume() { # $1 = name, $2 = main.cpp body + mkdir -p "$1/src" + printf '%s' "$2" > "$1/src/main.cpp" + cat > "$1/mcpp.toml" < run.log 2>&1 ) || { cat "$1/run.log"; echo "$1 failed"; exit 1; } + grep -q 'ok=5' "$1/run.log" || { cat "$1/run.log"; echo "$1 printed the wrong answer"; exit 1; } +} + +consume c_hdr '#include +#include +int main(){ std::printf("ok=%d\n", mathkit_add(2,3)); return 0; } +' +consume c_mod '#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::add(2,3)); return 0; } +' +consume c_both '#include +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mathkit_add(2,3) + mk::add(2,3) - 5); return 0; } +' + +echo "PASS: a library package serves header, module, and both-at-once consumers" diff --git a/tests/e2e/243_pack_library_interface_closure.sh b/tests/e2e/243_pack_library_interface_closure.sh new file mode 100755 index 00000000..8f183579 --- /dev/null +++ b/tests/e2e/243_pack_library_interface_closure.sh @@ -0,0 +1,146 @@ +#!/usr/bin/env bash +# requires: +# (no capability: this must hold on every target.) +# +# ⚠️ This test is also the regression for a scanner bug it uncovered. An +# IMPLEMENTATION PARTITION (`module M:part;`, no `export`) had no coverage +# anywhere in mcpp, and the scanner recorded it as "requires M:part, provides +# nothing" — so the file required its own name, the graph held no edge from the +# unit importing the partition to the unit defining it, and build order was +# unconstrained. GCC and macOS clang recovered via their own scan; Windows clang +# failed with `failed to read compiled module`. The scanner now models it, and +# the warning it used to print on every platform ("imported but not provided in +# this build") is gone. +# 243_pack_library_interface_closure.sh — what travels is the module closure of +# the published root, and the archive keeps exactly what the closure does not. +# +# Two asymmetric failures are pinned here, and only one of them is loud in the +# wild: +# +# * publishing TOO LITTLE fails in the consumer's compile, naming the module; +# * publishing TOO MUCH silently ships a closed-source implementation +# partition's SOURCE. Nothing fails. That is what this test is for. +# +# The archive side is the same closure used the other way round, and getting it +# wrong was measured: dropping every `.m.o` also drops the implementation +# partition's object, and every target then fails to link with an undefined +# reference nowhere near its cause. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export import :api; +EOF +cat > mathkit/src/api.cppm <<'EOF' +export module mathkit:api; +export namespace mk { int answer(); } +EOF +# An implementation partition: it produces a BMI and a `.m.o`, and its source +# is the thing a closed-source publisher must not ship. +cat > mathkit/src/secret.cppm <<'EOF' +module mathkit:secret; +namespace mk { int secret_bias() { return 40; } } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { +int secret_bias(); +int answer() { return secret_bias() + 2; } +} +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" + +# ── the confidentiality criterion ────────────────────────────────────── +[[ -f "$pkg/interface/mathkit.cppm" ]] || { echo "root interface not published"; exit 1; } +[[ -f "$pkg/interface/api.cppm" ]] || { echo "interface partition not published"; exit 1; } +[[ ! -e "$pkg/interface/secret.cppm" ]] || { + echo "LEAK: the implementation partition's source was published"; exit 1; } +grep -RIl 'secret_bias' "$pkg/interface" "$pkg/include" 2>/dev/null | grep -q . && { + echo "LEAK: implementation source text found in the published interface"; exit 1; } + +# ── the archive criterion: published objects out, everything else in ──── +# A functional probe, not `command -v ar`: on macOS `ar` resolves to an xlings +# shim that reports "not installed" and exits non-zero, which under `set -e` +# kills the test instead of skipping the inspection. +archive="$(find "$pkg/lib" -type f \( -name 'libmathkit.*' -o -name 'mathkit.*' \) | head -1)" +if [[ -n "$archive" ]] && members="$(ar t "$archive" 2>/dev/null)"; then + echo "$members" | grep -q 'secret.m.o' || { + echo "the implementation partition's OBJECT was dropped; nothing would link" + echo "$members"; exit 1; } + echo "$members" | grep -q 'mathkit.m.o' && { + echo "a published interface unit's object is still in the archive" + echo "$members"; exit 1; } + echo "$members" | grep -q 'api.m.o' && { + echo "a published interface unit's object is still in the archive" + echo "$members"; exit 1; } +fi + +# ── the hazard case: an interface that reaches the partition ──────────── +# +# Legal, and the consumer needs that source to build the BMI at all — so it is +# published rather than refused. But for a closed-source library it is the one +# outcome nobody wants by accident, and `import :secret;` in an interface looks +# like any other import, so `mcpp pack` has to say it. +cd "$TMP/mathkit" +cp src/mathkit.cppm "$TMP/root.bak" +printf 'import :secret;\n' >> src/mathkit.cppm +rm -rf target +"$MCPP" pack mathkit > hazard.log 2>&1 || { cat hazard.log; echo "hazard pack failed"; exit 1; } +grep -q 'secret.cppm is an implementation partition' hazard.log || { + cat hazard.log + echo "FAIL: publishing an implementation partition's source was not reported" + exit 1; } +hazpkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ -e "$hazpkg/interface/secret.cppm" ]] || { + echo "the warning fired but the source was not actually published"; exit 1; } +cp "$TMP/root.bak" src/mathkit.cppm +rm -rf target +"$MCPP" pack mathkit > repack.log 2>&1 || { cat repack.log; echo "repack failed"; exit 1; } +pkg="$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +[[ ! -e "$pkg/interface/secret.cppm" ]] || { echo "restore did not withhold it again"; exit 1; } +PKG_HOST="$(host_path "$TMP/mathkit/$pkg")" + +# ── and it still links and runs ───────────────────────────────────────── +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed"; exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } + +echo "PASS: the closure decides both what is published and what is dropped" diff --git a/tests/e2e/244_pack_library_gate.sh b/tests/e2e/244_pack_library_gate.sh new file mode 100755 index 00000000..04c26e80 --- /dev/null +++ b/tests/e2e/244_pack_library_gate.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# requires: +# (no capability: the refusals have to hold on every platform, so this test has +# to RUN on every platform.) +# 244_pack_library_gate.sh — the three refusals a prebuilt package must make. +# +# THE FIRST ONE IS WHY THIS FEATURE HAS A GATE AT ALL. Measured before it +# existed, on a real build: change one line of a shipped interface — swap two +# `int` members of a struct, which the Itanium ABI does not mangle — and the +# consumer compiles, links, runs, and prints transposed data. Exit code 0. No +# diagnostic at any stage, from any tool. +# +# 1. an edited interface → refuse (the silent-wrong-data case) +# 2. an artifact for another toolchain → refuse, and LIST the tags it has +# 3. `mcpp build` inside the package → refuse (it would "succeed" emptily) +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$pkg")" +cd "$TMP" + +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < ok.log 2>&1 ) || { cat app/ok.log; echo "baseline failed"; exit 1; } +grep -q 'ok=42' app/ok.log || { cat app/ok.log; echo "baseline wrong answer"; exit 1; } + +# ── 1. an edited interface is refused ────────────────────────────────── +cp "$pkg/interface/mathkit.cppm" "$TMP/interface.bak" +printf '\n// tampered\n' >> "$pkg/interface/mathkit.cppm" +rm -rf app/target +if ( cd app && "$MCPP" build > tamper.log 2>&1 ); then + cat app/tamper.log + echo "FAIL: an edited interface built anyway — this is the silent-wrong-data case" + exit 1 +fi +grep -q 'does not match what was packaged' app/tamper.log || { + cat app/tamper.log; echo "refused, but not for the interface digest"; exit 1; } +cp "$TMP/interface.bak" "$pkg/interface/mathkit.cppm" + +# ── 2. a foreign toolchain tag is refused, and the real tags are shown ── +cp "$pkg/mcpp.toml" "$TMP/manifest.bak" +# Forge the STDLIB segment, whatever it is called. Hard-coding `gcc` would have +# made this a no-op on macOS (llvm) and Windows (msvc) — the tag would stay +# valid, the package would be correctly accepted, and the test would report +# "a package built for another compiler was accepted" against a product that +# did nothing wrong. +sed -i.bak -E 's/(abi[[:space:]]*=[[:space:]]*"[^"]*-)[a-z]+[0-9]+(-)/\1forgedstl99\2/' "$pkg/mcpp.toml" +rm -rf app/target +if ( cd app && "$MCPP" build > tag.log 2>&1 ); then + cat app/tag.log + echo "FAIL: a package built for another compiler was accepted" + exit 1 +fi +grep -q 'no prebuilt artifact matches this toolchain' app/tag.log || { + cat app/tag.log; echo "refused, but not for the abi tag"; exit 1; } +# The diagnostic has to say what IS available — a refusal the reader cannot act +# on sends them looking for a package that is right in front of them. +grep -q 'published tags' app/tag.log || { + cat app/tag.log; echo "the refusal did not list the published tags"; exit 1; } +grep -q 'forgedstl99' app/tag.log || { + cat app/tag.log; echo "the refusal did not name the tag it found"; exit 1; } +cp "$TMP/manifest.bak" "$pkg/mcpp.toml" + +# ── 3. building INSIDE the package is refused ────────────────────────── +if ( cd "$pkg" && "$MCPP" build > "$TMP/inside.log" 2>&1 ); then + cat "$TMP/inside.log" + echo "FAIL: building inside a distribution package 'succeeded' — it compiles" + echo " declarations, links nothing, and reports Finished" + exit 1 +fi +grep -q 'distribution package produced by' "$TMP/inside.log" || { + cat "$TMP/inside.log"; echo "refused, but not as a distribution package"; exit 1; } + +# ── and the restored package still builds ────────────────────────────── +rm -rf app/target +( cd app && "$MCPP" run > final.log 2>&1 ) || { cat app/final.log; echo "restore failed"; exit 1; } +grep -q 'ok=42' app/final.log || { cat app/final.log; echo "restore wrong answer"; exit 1; } + +echo "PASS: interface tamper, tag mismatch, and in-package build are all refused" diff --git a/tests/e2e/245_pack_library_fat_target_selection.sh b/tests/e2e/245_pack_library_fat_target_selection.sh new file mode 100755 index 00000000..5060679b --- /dev/null +++ b/tests/e2e/245_pack_library_fat_target_selection.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# requires: gcc +# 245_pack_library_fat_target_selection.sh — one package, several targets, and +# each build picks exactly its own leg. +# +# ⚠️ THE PREDICATE IS THE POINT. Each leg is selected by a generated +# `cfg(all(arch=…, os=…, env=…))` block and NOT by a bare `[target.'']` +# key. The bare form only matched when `--target` was passed: a plain +# `mcpp build` resolved the host and compared it against an empty string, so +# the section was silently inert. That shape is the worst kind — CI passes +# `--target` and is green, the developer's own build drops the flags, and the +# failure arrives at the linker naming a symbol instead of a predicate. +# +# So this test builds the NATIVE case as well as the explicit ones, and asserts +# the selected directory each time. +# +# gnu + musl and not gnu + windows: CI warms both of those toolchains, so this +# test RUNS rather than skipping. A `# requires: mingw-cross` here would have +# made the whole fat-package mechanism unverified on every ordinary CI run +# while still reporting a green suite. The PE leg is covered by 248. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit \ + --target x86_64-linux-gnu \ + --target x86_64-linux-musl > pack.log 2>&1 \ + || { cat pack.log; echo "fat pack failed"; exit 1; } + +pkg="$TMP/mathkit/target/dist/mathkit-0.1.0" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$pkg")" +for t in x86_64-linux-gnu x86_64-linux-musl; do + [[ -n "$(find "$pkg/lib/$t" -name 'libmathkit.a' | head -1)" ]] || { + echo "no artifact for $t"; find "$pkg" -type f; exit 1; } +done + +# The generated blocks must be cfg(...), never a bare triple. +grep -q "target\.'cfg(" "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "legs are not selected by cfg() predicates"; exit 1; } +grep -qE "^\[target\.'x86_64-" "$pkg/mcpp.toml" && { + cat "$pkg/mcpp.toml" + echo "a leg is selected by a BARE TRIPLE, which is inert on a native build" + exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < "$TMP/$label.log" 2>&1 ) \ + || { cat "$TMP/$label.log"; echo "$label build failed"; exit 1; } + local nj; nj="$(find app/target -name build.ninja | head -1)" + grep -o "dist/mathkit-0.1.0/lib/[a-z0-9_-]*" "$nj" | sort -u > "$TMP/$label.legs" + [[ "$(wc -l < "$TMP/$label.legs")" -eq 1 ]] || { + echo "$label saw more than one leg:"; cat "$TMP/$label.legs"; exit 1; } + grep -q "lib/$want\$" "$TMP/$label.legs" || { + echo "$label picked the wrong leg:"; cat "$TMP/$label.legs"; exit 1; } +} + +# The native build is the one the bare-triple form used to get wrong. +check native x86_64-linux-gnu +check gnu x86_64-linux-gnu --target x86_64-linux-gnu +check musl x86_64-linux-musl --target x86_64-linux-musl + +echo "PASS: a fat package selects one leg per target, native build included" diff --git a/tests/e2e/246_explicit_empty_sources.sh b/tests/e2e/246_explicit_empty_sources.sh new file mode 100755 index 00000000..50a81d8a --- /dev/null +++ b/tests/e2e/246_explicit_empty_sources.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# requires: +# (no capability: the probe below greps a FILENAME out of build.ninja, not a +# compiler flag, so nothing here is toolchain-specific. It carried +# `# requires: gcc` for one round out of copy-paste, and that capability is +# Linux-only by design — it skipped on macOS and Windows for no reason.) +# 246_explicit_empty_sources.sh — `sources = []` means "compile nothing", and +# omitting the key still means "the default glob". +# +# They used to be byte-identical: the parser filled the default glob whenever +# the vector was empty, so an author had NO spelling for "nothing". A binary +# distribution package needs one — a header-only package compiles nothing, and +# any file left under `src/` would otherwise be swept up and compiled into the +# consumer's build, where it can collide with the symbols the prebuilt library +# already defines. +# +# The probe is a leftover source that must NOT be compiled in one case and MUST +# be in the other. Asserting only the first would pass against a parser that +# ignores `sources` entirely. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p probe/src +echo 'int mcpp_leftover_probe(void) { return 1; }' > probe/src/leftover.cpp +echo 'int main() { return 0; }' > probe/main.cpp + +write_manifest() { # $1 = the [build] body + cat > probe/mcpp.toml < empty.log 2>&1 ) || { cat probe/empty.log; echo "build failed"; exit 1; } +n="$(hits)" +[[ "$n" -eq 0 ]] || { + echo "FAIL: 'sources = []' still compiled the leftover source ($n references)" + exit 1; } + +# ── absent: the default glob still applies ───────────────────────────── +write_manifest '# no sources key at all' +rm -rf probe/target +( cd probe && "$MCPP" build > default.log 2>&1 ) || { cat probe/default.log; echo "build failed"; exit 1; } +n="$(hits)" +[[ "$n" -gt 0 ]] || { + echo "FAIL: omitting 'sources' stopped applying the default glob" + exit 1; } + +echo "PASS: an explicitly empty sources list is distinguishable from an absent one" diff --git a/tests/e2e/247_bare_triple_conditional_native.sh b/tests/e2e/247_bare_triple_conditional_native.sh new file mode 100755 index 00000000..11b141fb --- /dev/null +++ b/tests/e2e/247_bare_triple_conditional_native.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# requires: +# (no capability: the probe greps the MACRO NAME, never the `-D` prefix — MSVC +# spells that `/D`, and the prefix was the only thing that tied this test to +# one compiler family. The predicate it checks is compiler-independent, so the +# test has to run everywhere the predicate does.) +# 247_bare_triple_conditional_native.sh — `[target.''.build]` applies to +# a NATIVE build, not only to one with an explicit `--target`. +# +# It did not, and the failure shape is the dangerous one: CI passes `--target` +# and is green, the developer's plain `mcpp build` silently drops the section, +# and whatever it carried (a `-L`, a define, a source) goes missing somewhere +# far from the manifest. `cfg(linux)` matched the same build all along, so the +# two spellings of one statement disagreed. +# +# The probe asserts BOTH spellings under BOTH invocations. Asserting only the +# native case would pass against a build that applies every section always. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# The host triple, asked of mcpp rather than guessed. +# +# The first version pattern-matched `mcpp self env` for +# `--` and fell back to `x86_64-linux-gnu`. macOS's canonical +# triple is `aarch64-macos` — TWO segments, no env — so the match failed, the +# fallback was used, and the test then asserted that a *Linux* section applied +# to a macOS build. It reported "the bare triple was inert" against a product +# that was behaving correctly. +# +# `target//` is mcpp's own answer to the same question, so take it from +# there: build once with nothing conditional, read the directory name, then +# write the real manifest. +mkdir -p probe/src +echo 'int main() { return 0; }' > probe/src/main.cpp +cat > probe/mcpp.toml <<'EOF' +[package] +name = "probe" +version = "0.1.0" +[targets.probe] +kind = "bin" +main = "src/main.cpp" +EOF +( cd probe && "$MCPP" build > discover.log 2>&1 ) \ + || { cat probe/discover.log; echo "discovery build failed"; exit 1; } +host_triple="$(ls probe/target | head -1)" +[[ -n "$host_triple" ]] || { ls -R probe/target; echo "no target/ dir"; exit 1; } +echo "host triple: $host_triple" + +cat > probe/mcpp.toml <`: the prefix is dialect-specific (`/D` on MSVC), + # and what this test is about is whether the section applied at all. + local nj; nj="$(find target -name build.ninja | head -1)" + grep -c -- "$1" "$nj" || true +} + +cd probe + +# ── native: no --target ──────────────────────────────────────────────── +rm -rf target +"$MCPP" build > native.log 2>&1 || { cat native.log; echo "native build failed"; exit 1; } +bare="$(count MCPP_BARE_TRIPLE)" +alias_="$(count MCPP_CFG_ALIAS)" +[[ "$alias_" -gt 0 ]] || { echo "cfg() did not apply on a native build"; exit 1; } +[[ "$bare" -gt 0 ]] || { + echo "FAIL: [target.'$host_triple'.build] was inert on a native build" + echo " (cfg() applied $alias_ times, the bare triple $bare)" + exit 1; } + +# ── explicit --target: unchanged ─────────────────────────────────────── +rm -rf target +"$MCPP" build --target "$host_triple" > explicit.log 2>&1 \ + || { cat explicit.log; echo "explicit build failed"; exit 1; } +[[ "$(count MCPP_BARE_TRIPLE)" -gt 0 ]] || { echo "bare triple inert with --target"; exit 1; } +[[ "$(count MCPP_CFG_ALIAS)" -gt 0 ]] || { echo "cfg() inert with --target"; exit 1; } + +echo "PASS: a bare-triple conditional applies to native and explicit builds alike" diff --git a/tests/e2e/248_pack_library_fat_pe_leg.sh b/tests/e2e/248_pack_library_fat_pe_leg.sh new file mode 100755 index 00000000..d66e83a5 --- /dev/null +++ b/tests/e2e/248_pack_library_fat_pe_leg.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# requires: gcc mingw-cross +# 248_pack_library_fat_pe_leg.sh — a fat package whose legs cross an OS +# boundary, not just a libc one. +# +# 245 covers the fat-package MECHANISM with gnu + musl, because CI warms both +# and that test must never skip. This one adds the leg that changes binary +# format: `x86_64-windows-gnu` is PE, and its artifact naming follows the +# ENVIRONMENT rather than the OS — MinGW writes `libfoo.a` where MSVC would +# write `foo.lib`, which is exactly why `lib/` is keyed by triple and not by OS. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit \ + --target x86_64-linux-gnu \ + --target x86_64-windows-gnu > pack.log 2>&1 \ + || { cat pack.log; echo "cross-OS fat pack failed"; exit 1; } + +pkg="$TMP/mathkit/target/dist/mathkit-0.1.0" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$pkg")" +for t in x86_64-linux-gnu x86_64-windows-gnu; do + [[ -n "$(find "$pkg/lib/$t" -name 'libmathkit.a' | head -1)" ]] || { + echo "no artifact for $t"; find "$pkg" -type f; exit 1; } +done + +# Each leg records the triple it was built for. Two legs, two distinct tags — +# if the tag were taken from the compiler's own `-dumpmachine` instead of +# mcpp's canonical vocabulary, the Windows one would say `x86_64-w64-mingw32` +# and disagree with the cfg() block selecting it. +grep -q 'abi *= *"x86_64-linux-gnu-' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "no linux-gnu tag"; exit 1; } +grep -q 'abi *= *"x86_64-windows-gnu-' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "no windows-gnu tag (compiler spelling leaked?)"; exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < win.log 2>&1 ) \ + || { cat app/win.log; echo "PE consumer failed"; exit 1; } +exe="$(find app/target/x86_64-windows-gnu -name 'app.exe' | head -1)" +[[ -n "$exe" ]] || { echo "no .exe produced"; exit 1; } +file "$exe" | grep -q 'PE32+' || { file "$exe"; echo "not a PE image"; exit 1; } + +nj="$(find app/target/x86_64-windows-gnu -name build.ninja | head -1)" +grep -o "dist/mathkit-0.1.0/lib/[a-z0-9_-]*" "$nj" | sort -u > "$TMP/legs" +[[ "$(wc -l < "$TMP/legs")" -eq 1 ]] || { echo "more than one leg:"; cat "$TMP/legs"; exit 1; } +grep -q 'lib/x86_64-windows-gnu$' "$TMP/legs" || { + echo "the PE build did not pick the PE leg:"; cat "$TMP/legs"; exit 1; } + +echo "PASS: a fat package crosses an OS boundary and each build picks its own leg" diff --git a/tests/e2e/249_pack_workspace_root_unchanged.sh b/tests/e2e/249_pack_workspace_root_unchanged.sh new file mode 100755 index 00000000..6eb9447f --- /dev/null +++ b/tests/e2e/249_pack_workspace_root_unchanged.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) +# 249_pack_workspace_root_unchanged.sh — `mcpp pack` in a workspace root still +# packs the member's program. +# +# Routing `mcpp pack` through `[targets.].kind` means something has to read +# the manifest before the build does. A workspace root has no targets of its +# own — a virtual one has no `[package]` either — so the first version of that +# routing read the root's empty target list and concluded there was nothing to +# pack, turning a working command into an error. +# +# Caught by running the previous release against examples/04-workspace and +# comparing. This test is that comparison, made permanent. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p ws/apps/hello/src ws/libs/mathcore/src + +cat > ws/mcpp.toml <<'EOF' +[workspace] +members = ["libs/mathcore", "apps/hello"] +EOF + +cat > ws/libs/mathcore/mcpp.toml <<'EOF' +[package] +name = "mathcore" +version = "0.1.0" +EOF +cat > ws/libs/mathcore/src/mathcore.cppm <<'EOF' +export module mathcore; +export int core_answer() { return 42; } +EOF + +cat > ws/apps/hello/mcpp.toml <<'EOF' +[package] +name = "hello" +version = "0.1.0" +[dependencies] +mathcore = { path = "../../libs/mathcore" } +EOF +cat > ws/apps/hello/src/main.cpp <<'EOF' +#include +import mathcore; +int main() { std::printf("ok=%d\n", core_answer()); return 0; } +EOF + +cd ws +"$MCPP" pack --mode system > pack.log 2>&1 || { + cat pack.log + echo "FAIL: packing from a virtual workspace root stopped working" + exit 1 +} +grep -q 'Packed' pack.log || { cat pack.log; echo "no archive reported"; exit 1; } +# .tar.gz everywhere except a Windows target, which produces a .zip — the +# archive format follows the artifact, not the packer. +[[ -n "$(find . -name 'hello-0.1.0-*.tar.gz' -o -name 'hello-0.1.0-*.zip' | head -1)" ]] || { + cat pack.log; echo "the member's archive was not produced"; find . -name 'hello-0.1.0-*'; exit 1; } + +echo "PASS: a workspace root still packs its member's program" diff --git a/tests/e2e/250_pack_names_the_target.sh b/tests/e2e/250_pack_names_the_target.sh new file mode 100755 index 00000000..9fa079db --- /dev/null +++ b/tests/e2e/250_pack_names_the_target.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +# requires: +# (no capability: a library package is claimed to work on every target, so this +# test has to RUN on every platform. `# requires: gcc` would have skipped it on +# macOS and Windows — Apple Clang is not the gcc capability — leaving the claim +# unverified while the suite stayed green.) +# 250_pack_names_the_target.sh — `mcpp pack ` packs the target it was +# given, and refuses a name it cannot pack. +# +# The positional is what decides application-bundle vs library-package, so it +# has to reach BOTH pipelines. It did not reach the application one at first: +# a project with two `bin` targets accepted `mcpp pack app2` and bundled app1, +# which is the shape where the command succeeds and the answer is wrong. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p two/src +cat > two/src/alpha.cpp <<'EOF' +#include +int main() { std::printf("alpha\n"); return 0; } +EOF +cat > two/src/beta.cpp <<'EOF' +#include +int main() { std::printf("beta\n"); return 0; } +EOF +cat > two/mcpp.toml <<'EOF' +[package] +name = "two" +version = "0.1.0" + +[targets.alpha] +kind = "bin" +main = "src/alpha.cpp" + +[targets.beta] +kind = "bin" +main = "src/beta.cpp" +EOF + +cd two + +# ── the named program is the one that gets bundled ───────────────────── +"$MCPP" pack beta --mode system > beta.log 2>&1 || { cat beta.log; echo "pack beta failed"; exit 1; } +staged="$(find target/dist -maxdepth 1 -type d -name 'two-0.1.0*' | head -1)" +[[ -n "$staged" ]] || { cat beta.log; echo "no staging dir"; exit 1; } +find "$staged" -type f -name 'beta*' | grep -q . || { + echo "FAIL: 'mcpp pack beta' did not bundle beta"; find "$staged" -type f; exit 1; } +find "$staged" -type f -name 'alpha*' | grep -q . && { + echo "FAIL: 'mcpp pack beta' bundled alpha instead"; find "$staged" -type f; exit 1; } + +# ── and the other one, to prove the first result was not the default ─── +rm -rf target/dist +"$MCPP" pack alpha --mode system > alpha.log 2>&1 || { cat alpha.log; echo "pack alpha failed"; exit 1; } +staged="$(find target/dist -maxdepth 1 -type d -name 'two-0.1.0*' | head -1)" +find "$staged" -type f -name 'alpha*' | grep -q . || { + echo "FAIL: 'mcpp pack alpha' did not bundle alpha"; find "$staged" -type f; exit 1; } + +# ── an unknown name is refused, and says what there is ───────────────── +if "$MCPP" pack nosuch --mode system > bad.log 2>&1; then + cat bad.log; echo "FAIL: an unknown target name was accepted"; exit 1 +fi +grep -q "no target named 'nosuch'" bad.log || { cat bad.log; echo "wrong refusal"; exit 1; } +grep -q 'alpha' bad.log && grep -q 'beta' bad.log || { + cat bad.log; echo "the refusal did not list the available targets"; exit 1; } + +echo "PASS: mcpp pack packs the target it is given" diff --git a/tests/e2e/251_pack_library_shared.sh b/tests/e2e/251_pack_library_shared.sh new file mode 100755 index 00000000..43c94373 --- /dev/null +++ b/tests/e2e/251_pack_library_shared.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# requires: +# 251_pack_library_shared.sh — a `kind = "shared"` package is either produced +# CORRECTLY or refused CLEARLY. Never silently wrong. +# +# Both halves, because testing only the working one cannot tell "the gate is +# handled" from "there is no gate". `kind = "shared"` is ELF-only today +# (src/build/plan.cppm refuses it and says why), so on macOS and Windows the +# assertion is that the refusal happens and names the reason — a `# requires: +# elf` here would have left that side unobserved. +# +# A shared library is LINKED by `lib.so` and FOUND at run time by its +# SONAME, and those are two different filenames. The first version of this +# packer shipped only the built file: the consumer linked, and then mcpp's own +# runtime-closure check reported +# +# libmathkit.so.1 not found on the search path this artifact will actually use +# +# which is the good outcome only because that check exists. Without it the +# program would have failed to start with a loader error naming a file the user +# never asked for. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit-shared] +kind = "shared" +soname = "libmathkit.so.1" +EOF + +cd mathkit + +# ── the non-ELF side: refuse, and say why ────────────────────────────── +if [[ "$(uname -s)" != "Linux" ]]; then + if "$MCPP" pack mathkit-shared > refuse.log 2>&1; then + cat refuse.log + echo "FAIL: a shared library was packed on a platform that cannot link one" + exit 1 + fi + grep -qi 'shared librar' refuse.log || { + cat refuse.log + echo "FAIL: it refused, but the message does not say the artifact kind is the problem" + exit 1; } + grep -qi 'linux\|elf' refuse.log || { + cat refuse.log + echo "FAIL: the refusal does not say where shared libraries DO work" + exit 1; } + echo "PASS: a shared library package is refused, with the reason, off ELF" + exit 0 +fi + +# ── the ELF side: produce it, with both names ────────────────────────── +"$MCPP" pack mathkit-shared > pack.log 2>&1 || { cat pack.log; echo "shared pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +# The manifest below is FILE CONTENT: on Git Bash a shell-spelled +# /tmp/... path is read by a native mcpp.exe as "root of the current +# drive". host_path is the conversion (tests/e2e/_host_path.sh). +PKG_HOST="$(host_path "$pkg")" + +libdir="$(dirname "$(find "$pkg/lib" -name 'libmathkit-shared.so' | head -1)")" +[[ -n "$libdir" ]] || { echo "no .so in the package"; find "$pkg" -type f; exit 1; } +# Both names. The SONAME one may be a symlink or a copy — either is fine, its +# absence is not. +[[ -e "$libdir/libmathkit.so.1" ]] || { + echo "FAIL: the package does not carry the SONAME the loader will ask for" + ls -l "$libdir"; exit 1; } + +# The manifest declares it as a shared library and gives a runtime search dir: +# link_library_dirs is not rpath, and a shared package needs both. +grep -q 'role *= *"shared-library"' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "artifact is not recorded as a shared library"; exit 1; } +grep -q 'runtime_search_dirs' "$pkg/mcpp.toml" || { + cat "$pkg/mcpp.toml"; echo "no runtime_search_dirs — the consumer could not find it"; exit 1; } + +cd "$TMP" +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > app/mcpp.toml < run.log 2>&1 ) || { cat app/run.log; echo "consumer failed to run"; exit 1; } +grep -q 'ok=42' app/run.log || { cat app/run.log; echo "wrong answer"; exit 1; } + +exe="$(find app/target -name app -type f | head -1)" +readelf -d "$exe" 2>/dev/null | grep -q 'libmathkit.so.1' || { + readelf -d "$exe"; echo "the consumer does not NEED the soname"; exit 1; } + +echo "PASS: a shared library package carries both names and the consumer starts" diff --git a/tests/e2e/252_pack_library_old_client.sh b/tests/e2e/252_pack_library_old_client.sh new file mode 100755 index 00000000..90ae2112 --- /dev/null +++ b/tests/e2e/252_pack_library_old_client.sh @@ -0,0 +1,139 @@ +#!/usr/bin/env bash +# requires: +# 252_pack_library_old_client.sh — an mcpp that predates library packaging must +# still BUILD against a package produced by one that has it. +# +# That claim is the reason the generated manifest introduces no section and no +# key: everything in it — `sources`, `include_dirs`, `[modules] exports`, a +# `cfg(...)` block per leg, `[[runtime.artifacts]]` — was already parsed before +# this feature existed. An older client reads the package and links it; what it +# does not do is run the gates, because it has no way to know that +# `provenance = "mcpp-pack …"` means anything. +# +# Two halves, because only one of them can run everywhere: +# +# 1. STATIC — the generated manifest's top-level sections are a subset of the +# vocabulary that predates this feature. Portable, and it is the actual +# invariant rather than a proxy for it. +# 2. REAL — consume the package with $MCPP_BOOT, the released mcpp each CI job +# bootstraps from. Skipped with a loud note when that is not available, +# never silently. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mathkit/src +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export namespace mk { int answer(); } +EOF +cat > mathkit/src/impl.cpp <<'EOF' +module mathkit; +namespace mk { int answer() { return 42; } } +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "lib" +EOF + +cd mathkit +"$MCPP" pack mathkit > pack.log 2>&1 || { cat pack.log; echo "pack failed"; exit 1; } +pkg="$TMP/mathkit/$(find target/dist -maxdepth 1 -type d -name 'mathkit-0.1.0-*' | head -1)" +PKG_HOST="$(host_path "$pkg")" + +# ── 1. no section outside the pre-existing vocabulary ────────────────── +# +# Listed literally rather than derived: the point is that this set was frozen +# before the feature, so a new entry has to be added here deliberately — and +# adding one is exactly the moment to ask whether older clients can still read +# the package. +# `\[\[?` covers both a table and an array-of-tables header: `[[runtime.artifacts]]` +# is the same section as `[runtime]` for this purpose, and the first version of +# this pattern matched only the single-bracket form — so it flagged the very +# section the design deliberately reuses. +known='^\[\[?(package|build|modules|targets\.|target\.|dependencies|dev-dependencies|runtime|profile\.|features|lib|pack|workspace|indices|resources|xlings|capabilities|tools)' +bad="$(grep -E '^\[' "$pkg/mcpp.toml" | grep -Ev "$known" || true)" +[[ -z "$bad" ]] || { + echo "FAIL: the generated manifest uses sections an older mcpp cannot read:" + printf '%s\n' "$bad" + echo " Either express the fact with an existing key, or accept that packages" + echo " need a version floor — and say so in docs/12." + exit 1; } + +# ── 2. the released client actually builds against it ────────────────── +mkdir -p "$TMP/app/src" +cat > "$TMP/app/src/main.cpp" <<'EOF' +#include +import mathkit; +int main(){ std::printf("ok=%d\n", mk::answer()); return 0; } +EOF +cat > "$TMP/app/mcpp.toml" < new.log 2>&1 ) \ + || { cat "$TMP/app/new.log"; echo "the PR binary could not consume its own package"; exit 1; } +grep -q 'ok=42' "$TMP/app/new.log" || { cat "$TMP/app/new.log"; echo "wrong answer"; exit 1; } + +# ⚠️ The boot entry each CI job bootstraps from is an xvm SHIM, and a shim +# resolves against the home it is asked in — under the e2e suite's environment +# it answers `xlings: 'mcpp' is not installed` and prints NOTHING for +# `--version`. The first version of this guard compared that empty string +# against the PR binary's version, found them "different", and concluded it had +# found an old client — then reported a compatibility failure against a package +# that is perfectly readable. So the guard demands a version-SHAPED answer; +# anything else means "no usable old binary here", which is a note, not a +# verdict. +boot_ver="" +new_ver="$("$MCPP" --version 2>/dev/null || true)" +if [[ -n "${MCPP_BOOT:-}" && -x "${MCPP_BOOT}" ]]; then + boot_ver="$("$MCPP_BOOT" --version 2>/dev/null || true)" +fi +usable=0 +case "$boot_ver" in + mcpp\ [0-9]*) usable=1 ;; +esac + +if [[ "$usable" == 1 && "$boot_ver" != "$new_ver" ]]; then + echo "old client: $boot_ver" + rm -rf "$TMP/app/target" + ( cd "$TMP/app" && "$MCPP_BOOT" run > old.log 2>&1 ) || { + cat "$TMP/app/old.log" + echo "FAIL: the released mcpp cannot build against a package this one produced." + echo " The compatibility claim in docs/12 is then false: such packages" + echo " need a version floor, and publishing one without it bricks older" + echo " clients rather than degrading them." + exit 1; } + grep -q 'ok=42' "$TMP/app/old.log" || { + cat "$TMP/app/old.log"; echo "the old client built it but ran it wrong"; exit 1; } + echo "PASS: a released mcpp builds and runs against a package from this one" +else + if [[ -n "${MCPP_BOOT:-}" && "$usable" != 1 ]]; then + echo "NOTE: \$MCPP_BOOT=${MCPP_BOOT} does not answer --version with a version" + echo " (got: '${boot_ver}'), so it is not a usable old client here — a" + echo " shim resolves against the home it is asked in. The REAL old-client" + echo " check therefore did not run; run it with MCPP_BOOT pointing at a" + echo " released mcpp binary directly." + else + echo "NOTE: \$MCPP_BOOT is unset or identical to \$MCPP — the real old-client" + echo " check did not run here." + fi + echo "PASS: the generated manifest introduces no section an older mcpp cannot read" +fi diff --git a/tests/unit/test_modgraph.cpp b/tests/unit/test_modgraph.cpp index 72234374..a2fc1bdc 100644 --- a/tests/unit/test_modgraph.cpp +++ b/tests/unit/test_modgraph.cpp @@ -52,6 +52,100 @@ TEST(Scanner, ProvidesAndRequires) { // (e.g. a `mcpp new --template gui` skeleton embedded as R"GUI( ... )GUI") must // not be detected as real module imports. Before the fix this produced a // spurious "module 'imgui.core' imported but not provided" warning. +// ─── implementation partitions ──────────────────────────────────────────── +// +// `module M:part;` and `module M;` share a spelling and are different +// declarations. Conflating them left the graph without the edge from the unit +// that IMPORTS a partition to the unit that DEFINES it, so build order was +// unconstrained: GCC and macOS clang recovered through their own dependency +// scan, Windows clang failed with `failed to read compiled module`. The scanner +// also warned "imported but not provided in this build" on every platform, +// which read like a note instead of the cause. + +TEST(Scanner, ImplementationPartitionProvidesItsPartitionName) { + auto dir = make_tempdir("scan-implpart"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "secret.cppm", + "module mathkit:secret;\n" + "namespace mk { int bias() { return 1; } }\n"); + + auto u = scan_file(dir / "src" / "secret.cppm", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + ASSERT_TRUE(u->provides.has_value()); + EXPECT_EQ(u->provides->logicalName, "mathkit:secret"); + // Not an interface: its source must not be published by `mcpp pack`. + EXPECT_FALSE(u->providesInterface); + // And it must not require its own name, which is what it used to do. + for (auto const& r : u->requires_) + EXPECT_NE(r.logicalName, "mathkit:secret"); + std::filesystem::remove_all(dir); +} + +TEST(Scanner, InterfacePartitionIsMarkedAsAnInterface) { + auto dir = make_tempdir("scan-ifacepart"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "api.cppm", "export module mathkit:api;\nexport int f();\n"); + + auto u = scan_file(dir / "src" / "api.cppm", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + ASSERT_TRUE(u->provides.has_value()); + EXPECT_EQ(u->provides->logicalName, "mathkit:api"); + EXPECT_TRUE(u->providesInterface); + std::filesystem::remove_all(dir); +} + +TEST(Scanner, PlainImplementationUnitStillRequiresItsInterface) { + auto dir = make_tempdir("scan-implunit"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "impl.cpp", "module mathkit;\nnamespace mk { int g() { return 2; } }\n"); + + auto u = scan_file(dir / "src" / "impl.cpp", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + EXPECT_FALSE(u->provides.has_value()); + ASSERT_EQ(u->requires_.size(), 1u); + EXPECT_EQ(u->requires_[0].logicalName, "mathkit"); + std::filesystem::remove_all(dir); +} + +TEST(Scanner, PartitionImportResolvesInsideAnImplementationUnit) { + // The other half of the same bug: resolution keyed off `u.provides`, and an + // implementation unit has none — so `import :secret;` stayed the literal + // `:secret`, which nothing provides. + auto dir = make_tempdir("scan-implimport"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "impl.cpp", + "module mathkit;\nimport :secret;\nnamespace mk { int g(); }\n"); + + auto u = scan_file(dir / "src" / "impl.cpp", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + bool found = false; + for (auto const& r : u->requires_) { + EXPECT_NE(r.logicalName, ":secret"); + if (r.logicalName == "mathkit:secret") found = true; + } + EXPECT_TRUE(found) << "`import :secret;` did not resolve to mathkit:secret"; + std::filesystem::remove_all(dir); +} + +TEST(Scanner, PartitionImportResolvesInsideAPartition) { + // `export module foo:http;` + `import :tls;` must give `foo:tls`, not + // `foo:http:tls` — the case the old code was written for, still true. + auto dir = make_tempdir("scan-partpart"); + std::filesystem::create_directories(dir / "src"); + write(dir / "src" / "http.cppm", "export module foo:http;\nimport :tls;\nexport int h();\n"); + + auto u = scan_file(dir / "src" / "http.cppm", "pkg", + mcpp::builtin_extension_table()); + ASSERT_TRUE(u.has_value()); + ASSERT_EQ(u->requires_.size(), 1u); + EXPECT_EQ(u->requires_[0].logicalName, "foo:tls"); + std::filesystem::remove_all(dir); +} + TEST(Scanner, IgnoresImportsInsideRawStringLiteral) { auto dir = make_tempdir("mcpp-scanner-raw"); write(dir / "src" / "gen.cppm", diff --git a/tests/unit/test_pack_abi_tag.cpp b/tests/unit/test_pack_abi_tag.cpp new file mode 100644 index 00000000..8236827a --- /dev/null +++ b/tests/unit/test_pack_abi_tag.cpp @@ -0,0 +1,166 @@ +#include + +import std; +import mcpp.pack.abi_tag; +import mcpp.toolchain.model; + +using namespace mcpp::pack; +using mcpp::toolchain::CompilerId; +using mcpp::toolchain::Toolchain; + +namespace { + +Toolchain tc(CompilerId cc, std::string ver, std::string stdlib, std::string stdlibVer) { + Toolchain t; + t.compiler = cc; + t.version = std::move(ver); + t.stdlibId = std::move(stdlib); + t.stdlibVersion = std::move(stdlibVer); + // Deliberately the COMPILER's spelling, which must not reach the tag. + t.targetTriple = "x86_64-w64-mingw32"; + return t; +} + +} // namespace + +// ─── the tag is a projection, and of the CANONICAL triple ────────────────── + +TEST(AbiTag, CxxSurfaceNamesEveryDimension) { + auto t = cxx_surface_tag(tc(CompilerId::GCC, "16.1.0", "libstdc++", "16.1.0"), + "x86_64-linux-gnu", 23); + EXPECT_EQ(t.str(), "x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + EXPECT_FALSE(t.c_surface()); +} + +TEST(AbiTag, UsesTheCanonicalTripleNotTheCompilerReportedOne) { + // The toolchain above reports `x86_64-w64-mingw32`; mcpp's target + // vocabulary — and every `[target.'']` key a package can be + // selected by — says `x86_64-windows-gnu`. Publishing the compiler's + // spelling would give one decision two spellings. + auto t = cxx_surface_tag(tc(CompilerId::GCC, "16.1.0", "libstdc++", "16.1.0"), + "x86_64-windows-gnu", 23); + EXPECT_EQ(t.triple, "x86_64-windows-gnu"); + EXPECT_EQ(t.str().find("mingw"), std::string::npos); +} + +TEST(AbiTag, LibcxxAndMsvcStlTokenize) { + EXPECT_EQ(stdlib_token("libstdc++"), "libstdcxx"); + EXPECT_EQ(stdlib_token("libc++"), "libcxx"); + EXPECT_EQ(stdlib_token("msvc-stl"), "msvcstl"); + EXPECT_EQ(stdlib_token(""), "unknownstl"); +} + +TEST(AbiTag, MajorIsLeadingDigitsOnly) { + EXPECT_EQ(major_of("16.1.0"), "16"); + EXPECT_EQ(major_of("19.44.35207"), "19"); + EXPECT_EQ(major_of("22"), "22"); + EXPECT_EQ(major_of(""), "0"); +} + +// ─── the SHAPE is the surface: a C library publishes a shorter tag ───────── + +TEST(AbiTag, CSurfaceIsTripleOnly) { + auto t = c_surface_tag("x86_64-linux-gnu"); + EXPECT_EQ(t.str(), "x86_64-linux-gnu"); + EXPECT_TRUE(t.c_surface()); +} + +TEST(AbiTag, CSurfaceAcceptsAnyCompilerAndStdlib) { + auto published = c_surface_tag("x86_64-linux-gnu"); + auto current = cxx_surface_tag(tc(CompilerId::Clang, "22.1.8", "libc++", "22.1.8"), + "x86_64-linux-gnu", 26); + // An extern "C" library constrains the libc ABI and nothing else, so an + // unspecified dimension is don't-care — the same rule abi_check uses. + EXPECT_TRUE(tag_check(published, current).empty()); +} + +TEST(AbiTag, CSurfaceStillRefusesAForeignTriple) { + auto published = c_surface_tag("x86_64-linux-gnu"); + auto current = c_surface_tag("aarch64-linux-gnu"); + auto bad = tag_check(published, current); + ASSERT_EQ(bad.size(), 1u); + EXPECT_EQ(bad[0].dimension, "triple"); +} + +// ─── round-trip: parsing runs from the END, because triples vary in length ── + +TEST(AbiTag, ParsesFullTagBackWithATwoDashTriple) { + auto t = parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->triple, "x86_64-linux-gnu"); + EXPECT_EQ(t->compiler, "gcc16"); + EXPECT_EQ(t->stdlib, "libstdcxx16"); + EXPECT_EQ(t->standard, "c++23"); +} + +TEST(AbiTag, ParsesFullTagBackWithAOneDashTriple) { + // `aarch64-macos` has one dash, `x86_64-linux-gnu` has two. Splitting from + // the front cannot tell where the triple ends; splitting from the back can, + // because the C++ half is exactly three segments ending in c++NN. + auto t = parse_abi_tag("aarch64-macos-llvm22-libcxx22-c++23"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->triple, "aarch64-macos"); + EXPECT_EQ(t->compiler, "llvm22"); + EXPECT_EQ(t->standard, "c++23"); +} + +TEST(AbiTag, ParsesATripleOnlyTag) { + auto t = parse_abi_tag("x86_64-linux-gnu"); + ASSERT_TRUE(t.has_value()); + EXPECT_TRUE(t->c_surface()); + EXPECT_EQ(t->triple, "x86_64-linux-gnu"); +} + +TEST(AbiTag, RoundTripsEveryShape) { + for (auto s : { "x86_64-linux-gnu", + "x86_64-linux-musl-gcc16-libstdcxx16-c++23", + "aarch64-macos-llvm22-libcxx22-c++26", + "x86_64-windows-msvc-msvc19-msvcstl19-c++23" }) { + auto t = parse_abi_tag(s); + ASSERT_TRUE(t.has_value()) << s; + EXPECT_EQ(t->str(), s); + } +} + +TEST(AbiTag, RejectsEmptyAndNonTagInput) { + EXPECT_FALSE(parse_abi_tag("").has_value()); + // Three trailing segments that do not end in c++NN are part of the triple, + // not a C++ half — this must not be silently mis-split. + auto t = parse_abi_tag("a-b-c-d"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->triple, "a-b-c-d"); + EXPECT_TRUE(t->c_surface()); +} + +// ─── the gate ────────────────────────────────────────────────────────────── + +TEST(AbiTag, RefusesADifferentCompilerMajor) { + auto published = *parse_abi_tag("x86_64-linux-gnu-gcc15-libstdcxx15-c++23"); + auto current = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + auto bad = tag_check(published, current); + ASSERT_EQ(bad.size(), 2u); + EXPECT_EQ(bad[0].dimension, "compiler"); + EXPECT_EQ(bad[0].need, "gcc15"); + EXPECT_EQ(bad[0].got, "gcc16"); + EXPECT_EQ(bad[1].dimension, "stdlib"); +} + +TEST(AbiTag, StandardIsAFloorNotAnEquality) { + auto published = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + // Building at a HIGHER level is fine: the interface being compiled is the + // artifact's own source and a newer level accepts it. + auto higher = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++26"); + EXPECT_TRUE(tag_check(published, higher).empty()); + // Lower is not: the interface may use syntax this level lacks. + auto lower = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++20"); + auto bad = tag_check(published, lower); + ASSERT_EQ(bad.size(), 1u); + EXPECT_EQ(bad[0].dimension, "standard"); +} + +TEST(AbiTag, ReportsEveryMismatchAtOnce) { + // One diagnostic, not one per rebuild. + auto published = *parse_abi_tag("aarch64-linux-gnu-gcc15-libcxx15-c++26"); + auto current = *parse_abi_tag("x86_64-linux-gnu-gcc16-libstdcxx16-c++23"); + EXPECT_EQ(tag_check(published, current).size(), 4u); +} diff --git a/tests/unit/test_pack_interface.cpp b/tests/unit/test_pack_interface.cpp new file mode 100644 index 00000000..3c3d12cd --- /dev/null +++ b/tests/unit/test_pack_interface.cpp @@ -0,0 +1,165 @@ +#include + +import std; +import mcpp.pack.interface; +import mcpp.modgraph.graph; + +using namespace mcpp::pack; +using mcpp::modgraph::Graph; +using mcpp::modgraph::ModuleId; +using mcpp::modgraph::SourceUnit; + +namespace { + +// The shape of the library the design is written against: +// +// mathkit.cppm export module mathkit; export import :api; ← root +// api.cppm export module mathkit:api; ← interface partition +// secret.cppm module mathkit:secret; ← implementation partition, PRIVATE +// impl.cpp module mathkit; import :secret; +// capi.c (no module at all) +// +// `module M:part;` IS a provider — of `M:part` — and the scanner records it as +// one with `providesInterface = false`. It used to record "requires M:part, +// provides nothing", i.e. a file requiring its own name, which left the graph +// with no edge from the importer to the definer. That is the shape this fixture +// now mirrors, because the closure's warning depends on the distinction. +Graph library_graph(bool interfaceReachesSecret = false) { + Graph g; + auto add = [&](std::string path, std::optional provides, + std::vector requires_, bool iface = true) { + SourceUnit u; + u.path = std::move(path); + u.packageName = "mathkit"; + if (provides) u.provides = ModuleId{ *provides }; + u.providesInterface = iface; + for (auto& r : requires_) u.requires_.push_back(ModuleId{ std::move(r) }); + g.units.push_back(std::move(u)); + }; + + add("src/mathkit.cppm", "mathkit", + interfaceReachesSecret ? std::vector{ "mathkit:api", "mathkit:secret" } + : std::vector{ "mathkit:api" }); + add("src/api.cppm", "mathkit:api", {}); + add("src/secret.cppm", "mathkit:secret", {}, /*iface=*/false); + add("src/impl.cpp", std::nullopt, { "mathkit", "mathkit:secret" }); + add("src/capi.c", std::nullopt, {}); + + for (std::size_t i = 0; i < g.units.size(); ++i) + if (g.units[i].provides) + g.producerOf.emplace(g.units[i].provides->logicalName, i); + return g; +} + +std::vector names(const std::vector& v) { + std::vector out; + for (auto const& p : v) out.push_back(p.filename().string()); + std::ranges::sort(out); + return out; +} + +} // namespace + +// ─── what travels, and what does not ─────────────────────────────────────── + +TEST(InterfaceClosure, PublishesTheRootAndItsInterfacePartitionOnly) { + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()) << (c ? "" : c.error()); + EXPECT_EQ(names(c->published), (std::vector{"api.cppm", "mathkit.cppm"})); +} + +TEST(InterfaceClosure, WithholdsTheImplementationPartitionSource) { + // The whole point for a closed-source library: `secret.cppm` produces a + // BMI and an object, and its SOURCE must not be published. + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_EQ(names(c->withheld), + (std::vector{"capi.c", "impl.cpp", "secret.cppm"})); +} + +TEST(InterfaceClosure, DropSetIsThePublishedObjectsNotEveryModuleObject) { + // Measured: deleting every `.m.o` also deletes `secret.m.o`, which holds + // real code, and every target then fails to link with an undefined + // reference that names neither the archive nor the rule that removed it. + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + auto drop = published_object_names(*c); + std::ranges::sort(drop); + EXPECT_EQ(drop, (std::vector{"api.m.o", "mathkit.m.o"})); + EXPECT_EQ(std::ranges::find(drop, "secret.m.o"), drop.end()); +} + +// ─── the loud half of the asymmetry ──────────────────────────────────────── + +TEST(InterfaceClosure, AnInterfaceThatReachesAnImplementationPartitionPublishesItAndSaysSo) { + // Legal, and the consumer cannot build the interface's BMI without that + // source — so it is published rather than refused. But for a closed-source + // library it is the one outcome nobody wants by accident, and `import + // :secret;` in an interface reads like any other import, so it is reported. + auto c = interface_closure(library_graph(/*interfaceReachesSecret=*/true), + "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_EQ(names(c->published), + (std::vector{"api.cppm", "mathkit.cppm", "secret.cppm"})); + ASSERT_EQ(c->publishedImplementationPartitions.size(), 1u); + EXPECT_EQ(c->publishedImplementationPartitions[0].filename().string(), "secret.cppm"); + EXPECT_TRUE(c->unresolvedImports.empty()); +} + +TEST(InterfaceClosure, AnInterfaceOnlyClosureReportsNoPartitionLeak) { + auto c = interface_closure(library_graph(), "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(c->publishedImplementationPartitions.empty()); +} + +TEST(InterfaceClosure, AGenuinelyMissingPartitionIsStillAnError) { + // The unresolved path is not dead: a partition nothing provides means the + // published set is incomplete and the consumer's compile will fail on it. + auto g = library_graph(/*interfaceReachesSecret=*/true); + g.producerOf.erase("mathkit:secret"); + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + ASSERT_EQ(c->unresolvedImports.size(), 1u); + EXPECT_EQ(c->unresolvedImports[0], "mathkit:secret"); +} + +TEST(InterfaceClosure, ADependencysModuleIsNeitherPublishedNorUnresolved) { + auto g = library_graph(); + g.units[0].requires_.push_back(ModuleId{"compat.zlib"}); // someone else's + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(c->unresolvedImports.empty()); + EXPECT_EQ(c->published.size(), 2u); +} + +TEST(InterfaceClosure, ForeignPackageUnitsAreNotFollowed) { + auto g = library_graph(); + SourceUnit other; + other.path = "vendor/zlib.cppm"; + other.packageName = "compat.zlib"; + other.provides = ModuleId{"compat.zlib"}; + g.producerOf.emplace("compat.zlib", g.units.size()); + g.units.push_back(std::move(other)); + g.units[0].requires_.push_back(ModuleId{"compat.zlib"}); + + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_TRUE(c.has_value()); + EXPECT_EQ(names(c->published), (std::vector{"api.cppm", "mathkit.cppm"})); + // Nor does a foreign unit show up as something we withheld. + auto withheld = names(c->withheld); + EXPECT_EQ(std::ranges::find(withheld, "zlib.cppm"), withheld.end()); +} + +TEST(InterfaceClosure, RefusesAnUnknownRoot) { + auto c = interface_closure(library_graph(), "mathkit", "nosuch"); + ASSERT_FALSE(c.has_value()); + EXPECT_NE(c.error().find("nosuch"), std::string::npos); +} + +TEST(InterfaceClosure, RefusesARootOwnedByAnotherPackage) { + auto g = library_graph(); + g.units[0].packageName = "somebody.else"; + auto c = interface_closure(g, "mathkit", "mathkit"); + ASSERT_FALSE(c.has_value()); + EXPECT_NE(c.error().find("somebody.else"), std::string::npos); +}