diff --git a/.agents/docs/2026-08-23-target-side-resolution-architecture.md b/.agents/docs/2026-08-23-target-side-resolution-architecture.md new file mode 100644 index 00000000..c5e0e839 --- /dev/null +++ b/.agents/docs/2026-08-23-target-side-resolution-architecture.md @@ -0,0 +1,1310 @@ +# 目标侧解析:预构建体系与构建期体系的统一架构 + +2026-08-23。本文提出一项架构改动:把「目标侧从哪里来」从散落各处的**推导**,改为管线中一次 +显式的**解析**,并让预构建体系与构建期体系在同一个模型下按层共存。 + +## 0. 本文的证据边界 + +本文区分三类陈述并逐条标注: + +* **实测** —— 本次会话跑出过输出的; +* **读码** —— 从 `src/` 的表与分支读出的,未单独跑验证; +* **推断** —— 由前两类推出的判断,可能被后续测量推翻。 + +⚠️ 这个区分在本次讨论中挽回过两处错误。其一,笔者据行号断言「拒绝先于依赖解析」,而行号 +不是执行顺序;改用探针实测才确认。其二,笔者曾引用 `prepare.cppm:5796` 的注释作为「mcpp +已有先例」,而该注释与其机制**同属本分支**(`5875a53`,2026-08-22),是循环论证。凡未标注 +实测的结论,应当按可能有错来读。 + +--- + +## 1. 问题:一件事,三处推导,一处已测缺陷 + +一次构建必须回答:**目标侧(平台、C 库、C++ 运行时)从哪里来。** 今天这件事在三个地方各被 +推导一次,判据互不相同(读码): + +| 位置 | 判据 | +|---|---| +| `prepare.cppm:1404` `openkalTargetSide` | 工具链族名是否为 `openkal-llvm`(字符串比较) | +| `flags.cppm:555` `graphTargetSide` | `targetCxxRuntime && !crossTargetFlag.empty()` | +| `flags.cppm` → `distribution` `graphCxxRuntime` | `targetCxxRuntime` | + +**实测缺陷**:纯 C 程序在 openkal 栈上交叉到 macOS 时失败: + +``` +ld64.lld: error: …/xim-x-llvm/22.1.8/lib/x86_64-unknown-linux-gnu/libc++.so: + unhandled file type +``` + +因果链(读码 + 实测):纯 C 的依赖图里没有 `hosted-standard-library` 能力 ⇒ +`targetCxxRuntime` 为假 ⇒ 分发契约的自足短路不触发 ⇒ 契约落到 `host-coupled` 并加 +`-lc++` ⇒ 解析到载荷中**宿主**的 `libc++.so`。 + +⭐ 门(判据一)放行了这次构建,而链接行(判据二)没有相应替换。**同一件事的两处推导给出了 +不同答案**,而两者都不是被声明的。 + +--- + +## 2. 根因:两个体系的事实可知时刻相反 + +mcpp 今天服务两种目标侧供给方式,而它们的**事实可知时刻是相反的**: + +| | **预构建体系** | **构建期体系** | +|---|---|---| +| 目标侧是什么 | 一个目录(编译器载荷 / xpkg sysroot) | 一组包 | +| 谁决定目标可达 | `host_can_serve` —— 硬编码的闭表 | 图里有没有实现 —— 开集 | +| 目标集合 | `kKnownTargets`,封闭 | 加一个后端就多一批 | +| **事实何时可知** | **依赖解析之前** | **依赖解析之后** | +| 组合怎么选 | 版本号(一个载荷=一种组合) | `cfg(os)` 依赖 + features | + +**实测**:目标可达性的拒绝发生在依赖解析之前。探针为一个依赖指向不存在目录、同时构建一个 +本宿主无载荷的目标的工程: + +``` +$ mcpp build --target aarch64-macos # 依赖指向不存在的目录 +error: target 'aarch64-macos' cannot be built on this host … ← 坏依赖从未走到 + +$ mcpp build # 同一工程,本机 + Resolving toolchain ← 工具链先 +error: path dependency 'definitely-not-here' … has no mcpp.toml ← 依赖后 +``` + +⇒ ⭐⭐ **管线只有一条,而两个体系的事实在这条管线上的可知时刻相反。** 于是第 1 节那三处 +推导全部在做同一个动作:**在预构建体系的时刻,猜构建期体系的答案。** 三个猜法不一致是必然 +的,因为被猜的东西在那个时刻还不存在。 + +⇒ 由此得到本文的核心主张:**修法不是找一个更好的猜法,而是不猜。** + +--- + +## 3. 已测边界:编译器白送什么,什么必须由体系提供 + +「任意宿主交叉到任意目标」这件事有多少是编译器本来就给的,是划分体系边界的前提。**实测**, +同一个 clang 二进制(`xim-x-llvm/22.1.8`): + +| `--target=` | 产物 | +|---|---| +| `x86_64-unknown-linux-gnu` | ELF 64-bit x86-64 | +| `arm64-apple-macos14.0` | Mach-O 64-bit arm64 | +| `aarch64-macos` | Mach-O 64-bit arm64 | +| `x86_64-w64-windows-gnu` | COFF amd64 | +| `riscv64-none-elf` | ELF 64-bit RISC-V | + +⇒ **代码生成与目标格式与具体目标无关**:格式由三元组的 OS 决定,后端全部编在同一个二进制 +里。链接器同理 —— 载荷带着 `ld.lld` / `ld64.lld` / `lld-link` 三个前端。 + +而两件事编译器**不给**: + +**其一,平台元数据。实测** mcpp 自己的拼法与 LLVM 规范拼法产出不同: + +``` +aarch64-macos → MinVersion { Version: 10.4 } ← 无 Platform 字段 +arm64-apple-macos14.0 → MinVersion { Platform: macos, Version: 14.0 } +``` + +格式对、架构对、平台元数据错(arm64 macOS 不存在 10.4)。⚠️ 这是「看起来对、实际错」的 +形态,也是 `Triple::llvm_triple()` 必须翻译而非透传的原因。 + +**其二,系统侧。实测**未指定 `-fuse-ld=lld` 时链接落到宿主 binutils: + +``` +ld: unrecognised emulation mode: llvm +``` + +⇒ 边界因此可以画在确定的位置: + +| | 谁提供 | 与具体目标有关 | +|---|---|---| +| 代码生成 + 目标格式 | 编译器(一个二进制全包) | 否 | +| 链接器 | 编译器载荷(lld 三前端) | 否 | +| ── 分界线 ── | | | +| 平台元数据(部署版本、ABI) | **mcpp 的目标表 + `llvm_triple()`** | 是 | +| 系统侧(头 / C 库 / 运行时) | **预构建体系 或 构建期体系** | 是 | + +⚠️ 由此可知,「构建期体系」不需要证明交叉可行 —— 前两行本来就成立。它要解决的只有第四行。 + +--- + +## 4. 反例:两个体系按层混用 + +**实测**,生态中已存在与 openkal 无关的构建期供给: + +| 包 | 从图供给什么 | +|---|---| +| `std-freestanding` | C++ 标准库的 freestanding 子集(作为模块) | +| `std-freestanding-nolibc` | 五个函数四个头的 C 表面,给没有 C 库的目标 | +| `std-freestanding-alloc-libc` | 分配器,`provides = ["freestanding-allocator"]`,转发给目标的 C 库 | +| `std-freestanding-alloc-kal` | 同一能力的另一实现,转发给 openkal | + +其中 `std-freestanding` + picolibc 的组合是一次**按层混用**: + +| 层 | 来源 | +|---|---| +| C 库 | 预构建 —— `xim:picolibc-riscv@1.8.12`(目标表的 sysroot 列) | +| C++ 标准库 | 构建期 —— `std-freestanding` | +| 分配器 | 构建期 —— `alloc-libc`,而它转发给上面那个预构建的 C 库 | + +⇒ ⭐⭐ **一次构建可以同时属于两个体系,按层分开。** + +**这条反例有两个后果。**其一,任何单一布尔值都不足以表达目标侧来源 —— 它说不出「C 库是 +预构建的、C++ 标准库是图供给的」。其二,构建期体系**不应以 openkal 命名**:openkal 是它 +今天最大的实例,不是唯一实例,以实例命名机制会在下一个实例出现时被迫分裂。 + +⚠️ 附带更正一处:`provides` 机制并非本分支首创。`freestanding-allocator` 是更早的独立 +用户(feature 的 `requires` 匹配包级 `provides`)。本分支新增的只是 `hosted-standard-library` +这一个名字。 + +--- + +## 5. 模型:按层的来源解析 + +### 5.1 三层,而非两层 + +目标侧不是两层而是三层。⭐ 中间那一层在传统栈上是**隐式**的 —— C 库直接发 syscall 或调 +Win32,没有名字;而 openkal 的全部意义正是**把它显式化并命名**。openkal 的自述即 +「**a portable kernel ABI specification**」。 + +| 层 | 对应三元组的位 | 传统栈 | openkal 栈 | +|---|---|---|---| +| **kernel-abi** | **os** | 隐式(linux syscalls / win32 / darwin) | ⭐ **显式:openkal** | +| **c-abi** | **env** | glibc / musl / ucrt / libSystem | musl(经 openkal) | +| **c++** | 无对应位 | libstdc++ / libc++ / MSVC STL | libc++ 或 freestanding 子集 | + +⚠️ 每层是**接口**与**实现**两件事,输出必须分开显示:`openkal@0.5.1` 是接口, +`openkal-macos@0.3.1` 是实现。传统栈上两者常常是同一个物件,这本身是信息。 + +### 5.2 结构 + +```cpp +enum class Origin { Payload, Xpkg, Graph, None }; + +struct Layer { + Origin origin; + std::string interface; // 接口:openkal / linux / win32 / darwin / musl / glibc / libc++ + std::string impl; // 实现:包名@版本,或载荷内的 xpkg 引用 + bool subset = false; // C++ 层:是否为 freestanding 子集 +}; + +struct TargetSide { + std::string llvmTriple; // ⭐ 目标表的翻译结果,§3 实测其差异承重 + Layer kernelAbi; // ← triple 的 os 位 + Layer cAbi; // ← triple 的 env 位 + Layer cxx; // 三元组无对应位 +}; +``` + +| 场景 | `kernelAbi` | `cAbi` | `cxx` | +|---|---|---|---| +| 传统本机 / 载荷交叉 | `Payload` | `Payload` | `Payload` | +| 裸机 + picolibc | `None` | `Xpkg{picolibc}` | `None` | +| **裸机 + picolibc + std-freestanding** | `None` | **`Xpkg{picolibc}`** | **`Graph{std-freestanding}`** | +| 零 libc | `None` | `None` | `None` | +| openkal C++ | `Graph{openkal-*}` | `Graph{openkal-musl}` | `Graph{openkal-llvm-runtime}` | +| **openkal 纯 C** | `Graph{openkal-*}` | `Graph{openkal-musl}` | **`None`** | +| 直接用 openkal | `Graph{openkal-*}` | **`None`** | `None` | + +⭐ 第 1 节那条实测缺陷对应第六行;第三行是第 4 节的混用反例;第七行是「只要那 48 个接口」的 +程序。三者在这个模型里**天然可表达**,不需要任何例外分支。 + +⭐⭐ **第二行与第五行的对照是本模型最有价值的一处**:同一个 `riscv64-none-elf`,picolibc 路线 +`kernelAbi = None`(裸机没有内核),openkal 路线 `kernelAbi = Graph{openkal-opensbi}`。⇒ 同一份 +源码之所以能落到裸机,正是因为那里**仍然有一个被命名的内核接口**。这件事今日在任何输出里都 +看不见。 + +### 5.3 命名 + +构建期体系即 `Origin::Graph`。openkal 是它的一个实例,`std-freestanding` 是另一个,Zig 的 +运行时模型是同一件事的外部先例。⚠️ 层名(`kernel-abi` / `c-abi` / `c++`)与来源名 +(`payload` / `xpkg` / `graph` / 无)**互相正交**,不得合并 —— 合并正是 `openkal-llvm` 那个 +族名的错误。 + +--- + +## 6. 管线重排 + +``` +今天 重构后 +1. 读清单 1. 读清单 +2. 解析工具链 2. 解析工具链(只决定编译器) +3. ⚠️ 可达性门 ← 构建期事实不存在 3. 解析依赖图 +4. 解析依赖图 4. ⭐ 目标侧解析 → TargetSide +5. 建能力表 5. 可达性门 ← 读 TargetSide +6. 算 flags(再猜一遍) 6. 算 flags ← 读 TargetSide +``` + +⇒ 门与 flags 读**同一个已解析的值**。三处推导消失 —— 不是被统一,是不再需要。 + +**对预构建体系工程的影响**:同样的报错,只是发生在依赖解析之后。这类工程本来就没有目标侧 +依赖,解析很便宜。 + +**对构建期体系工程的影响**:天然正确,不需要例外。且报错质量提升 —— 从「这台机器造不出这 +个目标」变为 openkal 6.1 条款的 `undefined symbol: kal_*`,后者点名缺失的接口。 + +--- + +## 7. 目标表的权威范围收缩(而非降级) + +第 3 节的实测表明,目标表承载的平台元数据是图无法供给的。因此目标表继续是权威,但权威范围 +需要写清: + +| 目标表回答 | 状态 | +|---|---| +| 目标名如何翻译成 LLVM 三元组、部署版本是多少 | **永远权威** | +| 该目标默认用哪个预构建 sysroot | 权威(预构建体系的默认值) | +| ~~该目标可不可达~~ | **不再权威** —— 可达性是依赖图的函数 | + +⚠️ 最后一行是「普通依赖会让可达目标变多」的必然结论。一个包让一个目标从不可达变为可达,这 +在多数包管理器中不成立,而它正是构建期体系的定义性性质。 + +--- + +## 8. 分阶段落地 + +**本轮范围**:阶段 0–3 全部,加 §10.1 记录的 `linkage` 诊断缺口;模块化(§14)与之并行推进。 + +| 阶段 | 内容 | 验收 | +|---|---|---| +| **0** | 测本机 openkal 构建今天实际链接的是什么 | ✅ **已完成,见 §10.1** —— 推断被推翻,本机路径正确 | +| **1** | 三处推导 → 一个字段(消费端统一,生产端暂由族名填充) | 纯 C 的 e2e 先红后绿;既有 CI 全绿 | +| **2** | 门移到解析之后;`TargetSide` 改由图解析产出;删 `Family::OpenkalLlvm`、`family_serves_every_target`、`openkalTargetSide`、`graphTargetSide`;清除 §10.1 的惰性 rpath 残留 | 例外分支为零;报错变为 `kal_*` | +| **3** | `Origin` 四值 + 按层结构;`sysroot` 四种来源统一;§13 的非法组合被拒并给出理由;构建输出打印解析结果 | §4 的混用组合可表达 | +| **3b** | `[build] linkage = "dynamic"` 在目标侧来自图时发出诊断而非静默失效 | 见 §15 场景 9 | + +阶段 1 的消费端改动在阶段 2、3 中全部保留,不是返工。 + +⚠️ **阶段 1 的单一字段与 §4 的结论并不矛盾,但其范围必须写清。** §4 证明单一布尔值表达不了 +按层混用;而阶段 1 的字段**只替换第 1 节那三处推导所在的 hosted 路径**,`std-freestanding` + +picolibc 一类走的是 freestanding 分支(`isFreestandingTarget` 提前短路),阶段 1 不触及。 + +⇒ 阶段 1 **不增加**表达力,只消除同一问题上的三处分歧;表达力在阶段 3 才提升。把阶段 1 当作 +终点会重犯本文所批评的错误 —— 用一个不足以表达事实的量去承载一个多层的事实。 + +**阶段 3 的可解释性产出**: + +``` + Resolved llvm@22.1.8 → aarch64-macos + Target aarch64-macos → arm64-apple-macos14.0 + kernel-abi openkal@0.5.1 (openkal-macos@0.3.1, graph) + c-abi musl (openkal-musl@0.3.1, graph) + c++ libc++ (openkal-llvm-runtime@0.1.0, graph) +``` + +⭐ 打印的是**实际解析出来的事实**,而非清单里一句可能过期的声明。这是本设计选择「不引入新 +清单字段」的理由:可见性由输出承担,而输出不会过期。 + +--- + +## 9. 被否决的方案及其理由 + +本设计在讨论中经历四个被否决的形态,记录理由以免重复。 + +| 方案 | 形态 | 否决理由 | +|---|---|---| +| **A 包声明**(原形态) | `openkal-musl` 加 `provides = ["target-system"]` | 单一名字表达不了三层;且能力名是自由字符串、零校验,在其旁再加一个是放大问题 —— ⚠️ **见下方说明,该否决理由已被 §17 消解** | +| **B 工程声明** | `[build] sysroot = "openkal-musl"` | **实测**:`openkal-musl` 不在用户的 `[dependencies]` 里(是 `openkal-llvm-runtime` 的传递依赖)⇒ 要求用户陈述别人的事实,且供给者换 C 库时用户那行即失效 | +| **C 目标定义包** | 包可以向 `kKnownTargets` 加行 | 解的是另一个问题(目标词表可扩展),与本问题正交,可后置 | +| **D 独立工具链** | `openkal-llvm` / `openkal-gcc` 各为一套 | 名字命名的是 2×N×M 空间里的一个点,只表达「运行时是 llvm」一维;C 库与平台后端两维无法表达 | + +⚠️⚠️ **关于方案 A 的否决必须说清,否则本文自相矛盾:§17 采用的正是「包声明能力」这个机制。** + +被否决的是它**当时的形态**,而非机制本身。当时的两条理由,如今各自有了答案: + +| 当时的理由 | §17 的答案 | +|---|---| +| 单一名字 `target-system` 表达不了目标侧 | §5 证明有**三层**,§17.2 给出三个名字 | +| 能力名自由、零校验,拼错则静默回退 | §17.3 的保留前缀:`mcpp:*` 是封闭集合,拼错即报错 | + +⇒ 记录此事的目的不是修饰,而是留下判断依据:**一个机制被否决时,要分清否决的是机制还是它 +当时的形状。** 本文在讨论中曾把二者混为一谈,并因此多绕了两轮。 + +另有两个被否决的实现手段: + +* **切分 llvm 载荷**(`llvm-core` / `llvm-host-runtime`)。**实测**载荷构成为 `bin/` 756M、 + `lib/clang/` 112M、`include/c++/` 16M、`lib//` 9M ⇒ 切出宿主专属部分只省 25M + (2.8%)。而「宿主目标侧」实际不在文件里,在 `bin/clang++.cfg` 的 12 行绝对路径中, + `--no-default-config` 一个 flag 即整份关闭。⇒ 收益是第二道防线,不是修复;单独立项。 +* **给包加种类标签**(bsp / sysroot / library)。`openkal-opensbi` 同时是平台实现与板级 + 布局供给者,任一标签都是错的。⭐ 而**实测** `riscv-virt-rt` 不声明任何种类,其板级功能 + 经 `build.mcpp` 的 `link_script` / `runner` 正常工作 —— mcpp 从不需要知道「BSP」是什么。 + +--- + +## 10. 风险与未决 + +| | | +|---|---| +| ✅ **阶段 0 已完成,推断被推翻** | 见 §10.1 | +| ⚠️ `targetCxxRuntime` 的其他流向 | 该值还流入 freestanding flag 计算(`-ffreestanding`、`-fasynchronous-unwind-tables`、`model.cppm:410` 的 `-fdwarf-exceptions` / `-femulated-tls` / visibility)。这些调用点的时机需确认在解析之后 | +| ⚠️ 能力名零校验(§17.3 给出方案) | `provides` 是自由字符串,拼错一个字母则七处行为静默回退而构建「成功」。阶段 2 应加已知名集合与未知名警告 | +| ⚠️ `hosted-standard-library` 零测试 | **实测** `tests/` 全树无该字符串,而它带动七处行为。阶段 1 补测试 | +| ⚠️ `sysroot = ""` 的文档偏差 | 注释与错误信息称其为「no C library at all」,而 `same-source` 写着 `sysroot = ""` 且经 `openkal-llvm-runtime → openkal-musl` **有** C 库。字段实际选中的是「不放预构建目录」。此偏差需一并修正 | +| 门后移的代价 | 不可达目标将先解析完依赖才报错。换来的是报错说对了对象 | + +### 10.1 阶段 0 的测量结果:推断被推翻(实测) + +笔者曾**推断**本机 openkal 构建处于「openkal 的 libc++ + 载荷的 glibc」混合态,理由是 +`crossTargetFlag` 为空使 `graphTargetSide` 为假。该推断**错误**。 + +探针为一个只依赖 `openkal-llvm-runtime`、工具链为 `openkal-llvm@22.1.8` 的本机工程,源码 +使用 `import std`、异常与排序。**实测**: + +| 观测 | 值 | +|---|---| +| 产物 | ELF 64-bit,**statically linked** | +| `ldd` | not a dynamic executable | +| `kal_*` 定义 | **48**(openkal 接口全集) | +| 未定义的 glibc 符号 | **0** | +| C 库身份 | musl 特征符号 7 处 | +| 运行 | `sorted: 2 4 7` / `caught: 42` / `unwound: true` / `import std over openkal: ok`,退出码 0 | + +⇒ **本机路径今天是正确的,没有混合态。** 阶段 1 因此不必为该路径设计迁移。 + +⚠️ 但测量发现一处**惰性**残留。链接线上有载荷侧的贡献: + +``` +unit_ldflags = -nostdlib++ -Wl,-rpath,/home/speak/.mcpp/registry/subos/default/lib … +``` + +而**实测**该 rpath **没有落进产物**(`readelf -d` 报「There is no dynamic section」)。原因是 +openkal 构建的产物必然静态 —— 图里的 C 库与平台实现都是从源码建出的静态库。 + +⚠️ 追加**实测**,含一处对笔者自己的更正。初次记录称「`[build] linkage = "dynamic"` 无效且不 +发声」,**该记录有误**:`linkage` 根本不是 `[build]` 的键,mcpp 对它有明确诊断 —— + +``` +warning: [build] has unsupported key 'linkage' (ignored). Supported keys: … +``` + +—— 而笔者当时只读了输出的最后三行,没有看见它。⚠️ **这与本文批评的缺陷是同一种**:一次不 +完整的观察被当作了一个事实。 + +重测后的真实情形:该指令写在 `[target.]` 下时确实**静默无效** —— + +``` +[target.x86_64-linux-gnu] +linkage = "dynamic" → 产物仍为 statically linked,零诊断 +``` + +⇒ 本轮为此补一条诊断,措辞不点名具体的表。 + +⇒ 结论:载荷贡献存在于链接线而不存在于产物,今日无害;但它是「按宿主决定、而事实属于目标」 +的又一处残留,应在阶段 2 随例外分支一并清除。 + +--- + +## 11. 判据 + +本设计的验收不是「CI 绿」,而是下列各条可被单独证否: + +1. 纯 C 程序在 openkal 栈上交叉到 macOS / Linux 成功链接(**已达成,实测**:`aarch64-macos` → + Mach-O 64-bit arm64、`x86_64-linux-gnu` → ELF)。Windows 目标另需一个 builtins 供给者, + 见 §16.2 形态 3 的实测记录 —— 带上它之后同样成功(PE32+),而那属于打包粒度,不属本设计; +2. `std-freestanding` + picolibc 的按层混用在模型中可表达且构建不变; +3. `grep -r "openkal" src/` 在阶段 2 后不再出现于任何**行为分支**中(注释除外); +4. 构建输出打印的目标侧来源与实际链接线一致(阶段 3); +5. 生态八个仓库除 `same-source` 一行外零改动; +6. §13 的非法组合(`cxx.origin == Payload` 而 `cAbi.origin != Payload`)在解析阶段被拒绝, + 且报错陈述理由(「该 C++ 运行时是为载荷的 C 库 configure 的」),而非在链接期表现为一条 + 指向错误目录的 `-L`; +7. **本机 openkal 构建的 §10.1 六项观测在改动后逐项不变** —— 该路径今日正确,任何改动都不得 + 使其退化。这一条是回归判据,不是改进判据; +8. 旧拼法 `[toolchain] default = "openkal-llvm@22.1.8"` 在新引擎上构建成功,且产物与新拼法 + 一致(§15.4 的无感升级路径); +9. `[build] linkage = "dynamic"` 在目标侧来自图时发出诊断,且产物仍为静态(§15 场景 9)。 + +--- + +## 12. 生态影响 + +**实测**:全生态只有一处写了 `openkal-llvm@` 工具链 —— `openkal-llvm-runtime/examples/same-source/mcpp.toml:58`。阶段 2 后该行变为 `llvm@22.1.8`。 + +其余七个仓库(openkal、openkal-musl、openkal-linux、openkal-macos、openkal-windows、 +openkal-opensbi、openkal-uefi)**零改动**。 + +**实测**的依赖分层与本模型一致: + +``` +C++ 程序 → openkal-llvm-runtime → openkal-musl → openkal +纯 C 程序 ─────────────────────→ openkal-musl → openkal +``` + +而平台后端由 `openkal-musl` 的 `[target.'cfg(os = "…")'.dependencies]` 按目标选中,今日 +已有五个:linux、macos、windows、opensbi、uefi。⇒ 组合数为 2×N×M 而包数为 2+N+M,组合由 +依赖图承担,不需要任何新机制。 + +--- + +## 13. 两个体系的可组合性 + +**结论:应当可组合,而且已经在组合 —— 这不是一项待做的设计,是一项待承认的事实。** + +第 4 节的 `std-freestanding` + picolibc 即为实例。因此问题不是「要不要允许」,而是「哪些组合 +成立」。按 §5 的模型枚举: + +三层各有四种来源,组合空间为 4³。逐格枚举既冗长又无必要,因为⭐**约束只有一条,而且它在每 +一道缝上是同一条**: + +> ⭐⭐ **每一层的实现,必须是为它下面那一层 configure 过的。** + +这不是实现限制,而是性质。载荷里的 libc++ 是针对**载荷的 C 库** configure 出来的,它的 +`__config_site` 记录了那次 configure 的结论(线程 API、`_LIBCPP_HAS_*` 一族)。把它放到另一个 +C 库之上,不是「可能不兼容」,是**它从来没有为这件事被配置过**。 + +⚠️ 本生态已三次踩到这条性质的同一面: + +* `chrono.cpp` 的 `__has_include` 是关于机器的提问,而包缺 `-nostdinc`; +* MSVC STL 的头在裸机目标上**从未被读到**,而症状看起来像编不过; +* 第 1 节那条 `unhandled file type` —— 载荷的 libc++ 落到了图供给的 C 库之上。 + +⇒ **判据是「那份实现有没有为这个目标 configure 过」,不是「语言是否相同」。** + +### 13.1 该原则在两道缝上的具体形态 + +| 缝 | 约束 | 后果 | +|---|---|---| +| **c-abi ↔ c++** | `cxx.origin == Payload` 蕴含 `cAbi.origin == Payload` | 载荷的 libc++ 只配载荷的 C 库 | +| **kernel-abi ↔ c-abi** | 图供给的 C 库要求其下有对应的 kernel-abi 实现 | `openkal-musl` 调 `kal_*`,必须有一个 openkal 实现 | + +⚠️ 第二条不等于「C 库必须有 kernel-abi」:picolibc 直接面向裸机,`kernelAbi = None` 是**正确** +的组合(§5 第二行)。约束是**为下层 configure 过**,不是**下层非空**。 + +### 13.2 成立的组合(按现存与可预见实例) + +| 形态 | `kernelAbi` | `cAbi` | `cxx` | 证据 | +|---|---|---|---|---| +| 传统本机 / 载荷交叉 | `Payload` | `Payload` | `Payload` | 实测 | +| openkal C++(五个平台) | `Graph` | `Graph` | `Graph` | 实测 | +| **openkal 纯 C** | `Graph` | `Graph` | `None` | ⚠️ **今日为红** | +| 直接用 openkal | `Graph` | `None` | `None` | 读码 | +| freestanding C++ over openkal | `Graph` | `None` | `Graph`(子集) | 读码 | +| 裸机 + picolibc | `None` | `Xpkg` | `None` | 实测 | +| **裸机 + picolibc + 子集** | `None` | `Xpkg` | `Graph`(子集) | **实测** | +| 零 libc | `None` | `None` | `None` | 实测 | + +⇒ 模型应在**解析阶段**拒绝违反 §13 原则的组合并陈述理由,而不是让它在链接期变成一条指向错误 +目录的 `-L`。 + +⭐ 而 §5.2 的解析器结构使**默认路径根本构造不出**违规组合(见 §17 伪代码的末支):`cxx` 只在 +`cAbi.origin == Payload` 时才落到 `Payload`。⇒ 该诊断只在**显式覆盖**路径上需要 —— 用户手写 +`cxx_runtime = "toolchain-coupled"` 而 `cAbi` 来自图。 + +--- + +## 14. 模块化:按 xlings 的方式拆分 + +### 14.1 现状(实测) + +| 目录 | 模块数 | 行数 | +|---|---|---| +| `build/` | 28 | **23167** | +| `toolchain/` | 20 | 8245 | +| `manifest/` | 4 | 5586 | +| `pm/` | 16 | 5414 | +| `pack/` | 15 | 5127 | +| `platform/` | 15 | 3985 | +| `modgraph/` | 5 | 2026 | +| 其余八个目录 | 27 | 5925 | + +单文件前三:`build/prepare.cppm` **7220 行**、`build/ninja_backend.cppm` 2553 行、 +`manifest/toml.cppm` 2266 行。⇒ 复杂度集中在 `build/`,而其中一个文件占该目录的三成。 + +### 14.2 xlings 的做法(实测) + +`openxlings/xlings` 的 `modules/` 下有 cancellation、i18n、json、platform、sha256、theme、 +tinyhttps 七个目录,而 `modules/platform` 的内容是 **`mcpp.toml` + `src/`** —— 即 **xlings +把自身拆成了 mcpp 包**。 + +⭐ mcpp 已经在消费同一形态的产物:`mcpp.toml` 里 `mcpplibs.cmdline = "0.0.1"`。而 mcpplibs +下已提取的候选有 `cmdline`、`libxpkg`、`primitives`、`tinyhttps`、`xfilesystem`。 + +⇒ **该模式在两个方向上都已建立**,本节只是把它应用到 mcpp 自身尚未拆分的部分。 + +### 14.3 可提取与不可提取 + +判据是**是否依赖 mcpp 的构建状态**(plan、toolchain、依赖图)。 + +| 部分 | 行数 | 判断 | +|---|---|---| +| `platform/` | 3985 | ✅ 可提取 —— 纯 OS 抽象;⭐ xlings 已有同名模块,存在共用可能 | +| `modgraph/` | 2026 | ✅ 可提取 —— C++ 模块依赖图,自足且可独立测 | +| `manifest/{toml,types}` | 3475 | ✅ 可提取 —— 清单语法与类型,不需要构建状态 | +| `pack/` | 5127 | ✅ 可提取 —— 分发格式,输入是产物不是计划 | +| `bmi_cache/`、`fetcher/`、`publish/` | 1050 | ✅ 可提取 | +| **`build/`、`toolchain/`** | 31412 | ❌ **不可提取** —— 这里正是耦合本身 | + +⚠️ 不可提取的部分恰是最大的部分。**拆分不解决 `build/` 的复杂度**,它只把可分离的 15000 余行 +移出视野,使 `build/` 的耦合暴露得更清楚。这一点应当明说,以免把拆分当成复杂度的解法。 + +### 14.4 ⚠️ 自举约束 + +被提取的包由**上一个已发布的 mcpp** 构建。因此提取出的模块**只能使用发布版 mcpp 已有的能力**。 + +本会话已在同一约束上失败过一次:`build.mcpp` 中调用 `mcpp::compiler()` 使三个仓库 CI 全红 +(`no member named 'compiler'`),而为它准备的兜底 `mcpp::toolchain_dir()` **同样不在**那个版本 +里 —— 兜底与被兜的一起失败。⇒ 提取任何模块前,须先确认其所需的清单键与构建能力在发布版中存在。 + +### 14.5 ⭐ 与本文架构的交点 + +§5 的目标侧解析是一个**纯函数**: + +``` +(清单, 依赖图, 目标表) → TargetSide +``` + +无 I/O、无全局状态、无 ninja。⇒ 它是一条天然的模块边界,而把它作为独立模块的直接收益是 +**可独立测试** —— §10 记录 `hosted-standard-library` 今日零测试覆盖却带动七处行为,原因正是 +该判断埋在 7220 行的 `prepare.cppm` 里,除跑一次完整构建外无从断言。 + +⇒ **建议:阶段 3 将目标侧解析实现为独立模块 `src/targetside/`(暂不外提为包),其单元测试即 +§11 判据 2 与 4 的载体。** 待其稳定且发布版 mcpp 具备所需能力后,再考虑外提。 + +### 14.6 建议顺序 + +模块化与本文的架构改动**正交**,不应互相阻塞: + +| | 内容 | 依赖关系 | +|---|---|---| +| M1 | `src/targetside/` 作为独立模块落地 | 属阶段 3,随架构走 | +| M2 | `platform/` 外提为包,评估与 xlings 同名模块共用 | 独立;先做自举能力核对 | +| M3 | `modgraph/`、`manifest/{toml,types}` 外提 | 独立 | +| M4 | `pack/` 外提 | 独立;⚠️ 其 e2e 只在 Linux 具备 pack 能力 | +| — | `build/` 的耦合 | ⚠️ 拆分**不解决**,需单独立项 | + +--- + +## 14A. 发布闭环:进入 mcpp-index + +⚠️ **实现完成、CI 全绿、PR 合入,都不等于使用者能用上。** 一个使用者写下的是 + +```toml +[dependencies] +openkal-llvm-runtime = "0.1.0" +``` + +而这一行经索引解析。若索引里没有该版本,上述整套设计对他不存在。 + +### 14A.1 今日的索引差距(实测) + +| 包 | 分支版本 | 索引里最新 | 状态 | +|---|---|---|---| +| `openkal` | 0.6.0 | 0.5.2 | ⚠️ 落后一版 | +| `openarch` | 0.7.0 | 0.6.0 | ⚠️ 落后一版 | +| **`openkal-llvm-runtime`** | 0.1.0 | **不在索引** | ⚠️⚠️ **整个包缺席** | +| `openkal-{linux,macos,windows,opensbi,uefi}` | 与索引一致 | — | 需随能力声明升版 | +| `openkal-musl` | 0.3.1 | 0.3.1 | 同上 | +| `std-freestanding` | 0.5.0 | 0.5.0 | 同上 | + +⭐ **`openkal-llvm-runtime` 不在索引**这一条尤其要紧:它是形态 2 的唯一入口。没有它,「一份 +源码四种格式」这件事只能靠 git 引用完成,而 git 引用不是受支持的分发路径。 + +### 14A.2 本设计使全部包都需要重新发布 + +§17.2 要求八个包各加一行 `provides = ["mcpp:…"]`。清单内容改变 ⇒ 版本必须递增 ⇒ 索引必须 +新增该版本条目。**没有例外**:索引服务的是发布的 tarball,而未发布的 tarball 里没有那一行, +新引擎在其上解析出的目标侧仍然是 `payload`。 + +⇒ 因此发布闭环包含,按顺序: + +1. 各包升版并合入(§8 的合入顺序); +2. 打 tag、发 release; +3. 本地 gtc 立即补 CN 镜像资源(⚠️ 否则 CN 侧解析落空); +4. 在 `mcpp-index/pkgs/<首字母>/<包名>.lua` 增加版本条目,含 `GLOBAL` / `CN` 两个 URL 与 + `sha256`; +5. **判据是索引 main 的 latest 指向它**,而不是「发布成功」。 + +### 14A.3 判据 + +⚠️ 最终验证必须**通过索引**进行,而不是通过 git 分支引用: + +```toml +[dependencies] +openkal-llvm-runtime = "0.1.0" # 索引解析,不是 { git = "…" } +``` + +§16 的八种形态逐一构建并运行,全部经此路径。⇒ 这才是「使用者能用上」的证明;git 引用能跑只 +证明代码正确,不证明分发正确。 + +--- + +## 15. 开发者使用侧:场景与变化 + +本节从写清单、跑命令、读输出的角度陈述本设计的全部可见后果。**判据是:除一行之外,现有工程 +不需要任何修改。** + +### 15.1 变化概览 + +| 使用者 | 清单要改吗 | 命令要改吗 | 看得到的差别 | +|---|---|---|---| +| 传统本机 / 载荷交叉 | 否 | 否 | 多一行目标侧来源;不可达目标的报错更晚更准 | +| 裸机 + picolibc | 否 | 否 | 多一行目标侧来源 | +| openkal C++ | **一行**(`openkal-llvm@` → `llvm@`) | 否 | 同上 | +| **openkal 纯 C** | 否 | 否 | ⭐ **从构建失败变为成功** | +| 生态包作者 | 否(七个仓库零改动) | 否 | — | +| 开发板 / BSP 作者 | 否 | 否 | — | + +### 15.2 场景 + +**场景 1 — 传统本机构建。** 无变化。 + +``` + Resolving toolchain + Resolved gcc@16.1.0 → …/xim-x-gcc/16.1.0/bin/g++ ++ Target x86_64-linux-gnu ++ kernel-abi linux (xim-x-linux-headers@5.11.1, payload) ++ c-abi glibc (xim-x-glibc@2.44, payload) ++ c++ libstdc++ (xim-x-gcc@16.1.0, payload) + Compiling my-app v0.1.0 (.) + Finished dev [unoptimized + debuginfo] in 0.31s +``` + +新增的四行是本设计唯一对这类工程可见的改动。它陈述的是**解析出来的事实**,不是清单里的声明。 + +**场景 2 — 载荷交叉(`--target x86_64-linux-musl`)。** 无变化,同样多四行: + +``` ++ Target x86_64-linux-musl ++ kernel-abi linux (xim-x-linux-headers@5.11.1, payload) ++ c-abi musl (payload) ++ c++ libstdc++ (xim-x-gcc@16.1.0, payload) +``` + +⭐ 与形态 2 的 openkal 输出并排,差别一眼可见:此处 `c-abi = musl (payload)` 是**载荷里预建 +好的** musl;openkal 那边是 `musl (openkal-musl@0.3.1, graph)` —— **同一个接口,两种来源**。 + +**场景 3 — openkal 交叉 C++。清单改一行:** + +```diff + [toolchain] +-default = "openkal-llvm@22.1.8" ++default = "llvm@22.1.8" +``` + +⭐ 这一行是本设计对整个生态**唯一**的清单改动(§12:全生态只有 `same-source` 写了它)。改动后: + +``` + Resolved llvm@22.1.8 → aarch64-macos ++ Target aarch64-macos → arm64-apple-macos14.0 ++ kernel-abi openkal@0.5.1 (openkal-macos@0.3.1, graph) ++ c-abi musl (openkal-musl@0.3.1, graph) ++ c++ libc++ (openkal-llvm-runtime@0.1.0, graph) +``` + +⇒ 编译器与目标侧**在输出里分开**,因为它们本来就是两件事。开发者不再需要知道 `openkal-llvm` +这个只在 mcpp 内部有意义的名字。 + +**场景 4 — openkal 纯 C。⭐ 从失败变为成功。** 清单不变: + +```toml +[package] +name = "sensor-fw" +version = "0.1.0" + +[dependencies] +openkal-musl = { git = "https://github.com/mcpplibs/openkal-musl" } +``` + +```diff + $ mcpp build --target aarch64-macos +-ld64.lld: error: …/xim-x-llvm/22.1.8/lib/x86_64-unknown-linux-gnu/libc++.so: +- unhandled file type ++ Target aarch64-macos → arm64-apple-macos14.0 ++ kernel-abi openkal@0.5.1 (openkal-macos@0.3.1, graph) ++ c-abi musl (openkal-musl@0.3.1, graph) ++ c++ — ++ Finished dev [unoptimized + debuginfo] in 4.12s +``` + +⚠️ 注意 `c++ = —` 是**正确**的:一个 C 程序不需要 C++ 运行时。今日之所以失败,正是因为「没有 +C++ 运行时」被当成了「目标侧不来自图」。 + +**场景 5 — 裸机 + picolibc。** 无变化: + +```toml +[target.riscv64-none-elf] +runner = ["qemu-system-riscv64", "-machine", "virt", "-nographic", "-bios", "default", "-kernel"] +``` +``` ++ Target riscv64-none-elf ++ kernel-abi — ++ c-abi picolibc-riscv (xim:picolibc-riscv@1.8.12, prebuilt) ++ c++ — +``` + +**场景 6 — 裸机 + picolibc + std-freestanding(按层混用)。** 无变化,而**来源现在说得清**: + +``` ++ Target riscv64-none-elf ++ kernel-abi — ++ c-abi picolibc-riscv (xim:picolibc-riscv@1.8.12, prebuilt) ← 预构建 ++ c++ freestanding subset (std-freestanding@0.5.0, graph) ← 构建期 +``` + +⭐ 中间两行来源不同,是 §13 可组合性的直接体现:一次构建同时属于两个体系,而输出讲清楚了。 + +**场景 7 — 非法组合被拒(新诊断)。** 一个工程若把载荷的 C++ 运行时用在图供给的 C 库之上: + +``` +error: the toolchain payload's C++ runtime cannot be used with a C library from + the dependency graph. + c-abi musl (openkal-musl@0.3.1, graph) + c++ libc++ (xim-x-llvm@22.1.8, payload) ← 冲突 + The payload's libc++ was configured against the payload's C library — its + `__config_site` records that configuration. It was never configured for + this one. + Supply a C++ runtime from the graph (e.g. `openkal-llvm-runtime`), or use + the payload for both. +``` + +⇒ 今日这一格**没有诊断**,它表现为链接期一条指向错误目录的 `-L`(§1)。 + +**场景 8 — 不可达目标:报错更晚,但说对了对象。** + +```diff + $ mcpp build --target aarch64-macos # Linux 宿主,无 openkal 依赖 +-error: target 'aarch64-macos' cannot be built on this host — no toolchain +- payload exists that runs here and produces it. +- this host can build: x86_64-linux-gnu, … ++ Resolving dependencies ++error: target 'aarch64-macos' cannot be built on this host. ++ No toolchain payload here produces it, and nothing in the dependency ++ graph supplies its system side. ++ this host can build with the payload alone: x86_64-linux-gnu, … ++ To build it anyway, depend on a package that implements the target's ++ system (see the openkal packages), or supply `[target.aarch64-macos] ++ toolchain = "…"`. +``` + +⚠️ **代价**:依赖先解析完才报错。对这类工程(无目标侧依赖)解析很便宜。**收益**:旧文案断言 +「这台机器造不出来」,而加一个依赖就能造出来 —— 旧文案说的是错的。 + +**场景 9 — `linkage = "dynamic"` 不再静默失效(新诊断)。** + +```toml +[build] +linkage = "dynamic" +``` +```diff + $ mcpp build # openkal 栈 +- Finished dev [unoptimized + debuginfo] in 0.12s ← 产物仍是静态,零提示 ++warning: `[build] linkage = "dynamic"` has no effect when the target's system ++ comes from the dependency graph: openkal-musl and the platform ++ implementation are static archives built from source, and there is no ++ shared object to link against. ++ Finished dev [unoptimized + debuginfo] in 0.12s +``` + +⇒ 是 warning 不是 error:静态是**正确**的结果,只是与所写的不同。**实测**该指令今日无效且无 +任何输出(§10.1)。 + +**场景 10 — 新开发板。** 无变化,仍是一个包: + +```toml +[dependencies] +my-board-bsp = { git = "…" } # build.mcpp 供给 link_script + runner +``` + +⭐ mcpp 从不知道「BSP」是什么概念(§9)。本设计不引入板级字段,新板成本仍为**零 mcpp 改动**。 + +### 15.3 不变的东西(同等重要) + +| | 状态 | +|---|---| +| `[toolchain] default` 的写法与值域 | 不变(`openkal-llvm` 之外) | +| `[target.X] sysroot` 的值域 | 不变 —— 仍是 xpkg 引用或 `""`;⚠️ 只修其**注释与错误文案**(§10) | +| `[target.X] runner` / `cxx_runtime` / `linkage` | 不变 | +| `build.mcpp` 的全部指令 | 不变 | +| `provides` 的写法 | 不变;阶段 2 增加未知名警告 | +| 依赖声明、features、`cfg(os)` 依赖 | 不变 | +| 锁文件、缓存布局 | 不变(族名不落盘,落盘的是字符串) | + +⚠️ **本设计不新增任何清单字段。** 这是有意的:§8 已论证可见性由**输出**承担,因为输出打印的是 +解析出来的事实,而清单里的一行会在供给者变化时过期。 + +### 15.4 升级路径 + +对已有工程: + +1. 什么都不做 —— 除 `openkal-llvm@` 那一处外,全部工程行为不变; +2. 写了 `openkal-llvm@` 的工程:该拼法在阶段 2 后**继续被接受**(解析为 `llvm` + 图供给),并 + 发出一条建议改写的提示,而不是报错。⇒ **无感升级**,不设截止版本。 + +⚠️ 该兼容路径需要一条测试:旧拼法在新引擎上构建成功且行为与新拼法一致。列为 §11 判据的补充。 + +--- + +## 16. 项目形态:开发者如何选择自己的目标侧 + +### 16.1 ⭐ 控制面只有三个旋钮 + +| 旋钮 | 控制什么 | 值域 | +|---|---|---| +| **`[dependencies]`** | **目标侧从哪来** —— 主旋钮 | 依赖哪些包 | +| `[toolchain] default` | 用哪个编译器 | `llvm@…` / `gcc@…` / `msvc@…` | +| `[target.X] sysroot` | 覆盖预构建目录 | xpkg 引用 / `""` / 缺省(用目标表) | + +⭐⭐ **依赖列表就是控制面。** 这是本设计从使用者视角的中心主张:开发者不通过一个开关声明 +「我要用 openkal」,而是**依赖了 openkal 的包,于是目标侧就来自它们**。没有第四个旋钮,也 +不需要。 + +### 16.2 八种项目形态 + +以下清单均为可直接使用的形态;引用形式沿用生态今日的实际写法(`openkal` 已在注册表, +闭包分支上的包用 git 引用)。 + +--- + +**形态 1 —— 默认:什么都不写** + +```toml +[package] +name = "app" +version = "0.1.0" +``` + +三个宿主上的解析结果。⭐ **每一行的形状相同:`接口 (实现, 来源)`。** + +``` +── Linux ──────────────────────────────────────────────────────────── + Target x86_64-linux-gnu + kernel-abi linux (xim-x-linux-headers@5.11.1, payload) + c-abi glibc (xim-x-glibc@2.44, payload) + c++ libc++ (xim-x-llvm@22.1.8, payload) + +── macOS ──────────────────────────────────────────────────────────── + Target aarch64-macos → arm64-apple-macos14.0 + kernel-abi darwin (macOS SDK, payload) + c-abi libSystem (macOS SDK, payload) ⚠️ 与上一行同一个物件 + c++ libc++ (macOS SDK, payload) + +── Windows ────────────────────────────────────────────────────────── + Target x86_64-windows-msvc + kernel-abi win32 (Windows SDK, payload) + c-abi ucrt (Windows SDK, payload) + c++ MSVC STL (msvc@14.4x, payload) +``` + +⚠️ macOS 的 `kernel-abi` 与 `c-abi` 指向同一个物件(`libSystem` 兼两层),这本身是信息 —— +它解释了为何 macOS 上没有「换一个 C 库」这个动作,而 Linux 上有。 + +| | | +|---|---| +| 得到 | 完整标准库、`import std`、异常、RTTI | +| 可交叉到 | 载荷支持的目标(`mcpp toolchain list` 可查) | +| 不可交叉到 | 本宿主无载荷的目标(如 Linux 上的 macOS) | + +--- + +**形态 2 —— openkal C++:一份源码,四种格式** ⭐ + +```toml +[package] +name = "portable-app" +version = "0.1.0" + +[dependencies] +openkal-llvm-runtime = { git = "https://github.com/mcpplibs/openkal-llvm-runtime" } +``` + +⭐ **同一份清单、同一份源码,四个 `--target` 的解析结果。三行里只有中间的实现名在变。** + +``` +── mcpp build ─────────────────────────────────────────────────────── + Target x86_64-linux-gnu + kernel-abi openkal@0.5.1 (openkal-linux@0.5.1, graph) + c-abi musl (openkal-musl@0.3.1, graph) + c++ libc++ (openkal-llvm-runtime@0.1.0, graph) + +── mcpp build --target aarch64-macos ──────────────────────────────── + Target aarch64-macos → arm64-apple-macos14.0 + kernel-abi openkal@0.5.1 (openkal-macos@0.3.1, graph) + c-abi musl (openkal-musl@0.3.1, graph) + c++ libc++ (openkal-llvm-runtime@0.1.0, graph) + +── mcpp build --target x86_64-windows-gnu ─────────────────────────── + Target x86_64-windows-gnu → x86_64-w64-windows-gnu + kernel-abi openkal@0.5.1 (openkal-windows@0.1.1, graph) + c-abi musl (openkal-musl@0.3.1, graph) ⭐ Windows 上的 musl + c++ libc++ (openkal-llvm-runtime@0.1.0, graph) + +── mcpp build --target riscv64-none-elf ───────────────────────────── + Target riscv64-none-elf + kernel-abi openkal@0.5.1 (openkal-opensbi@0.1.0, graph) ⭐ 裸机上仍有内核接口 + c-abi musl (openkal-musl@0.3.1, graph) + c++ libc++ (openkal-llvm-runtime@0.1.0, graph) +``` + +⭐⭐ 两处标注是本设计使 openkal 的价值第一次可见的地方: + +* **Windows 上的 musl** —— 传统 Windows 栈没有这个选项。musl 之所以能在那里,是因为它调 + `kal_*` 而不是 syscall,由 `openkal-windows` 在 Win32 上实现; +* **裸机上的 `kernel-abi`** —— 与形态 6(picolibc)对照,后者该行为 `—`。同一份源码之所以能 + 落到裸机,正是因为那里仍有一个被命名的内核接口。 + +| | | +|---|---| +| 得到 | `import std`、异常、RTTI —— **实测**在裸机上亦成立(`unwound: true`) | +| 可交叉到 | ⭐ **ELF / Mach-O / PE / 裸机**,从任意宿主 | +| 代价 | 首次构建需从源码建 libc++ 一族;之后走依赖缓存 | +| 平台后端 | 由 `openkal-musl` 按 `cfg(os)` 自动选中,今日五个 | + +⇒ **这是「基于 openkal 的应用」的默认形态。** 程序名此一个包,其余整条栈由图解析。 + +--- + +**形态 3 —— openkal 纯 C** + +```toml +[dependencies] +openkal-musl = { git = "https://github.com/mcpplibs/openkal-musl" } +``` + +| | | +|---|---| +| 得到 | 完整 C 标准库,可移植到全部 openkal 平台 | +| 失去 | C++ 标准库(可写 C++ 但不能 `import std`) | + +⚠️ **实测(2026-08-24):这一形态在 Windows 上缺 compiler-rt builtins。** + +``` +ld.lld: error: undefined symbol: __mulxc3 (musl 的 src/complex/cpowl.c) + undefined symbol: __mulsc3 / __muldc3 +``` + +builtins 是**编译器的支持库**,既不属于 kernel-abi、也不属于 c-abi 或 c++-abi。载荷里带着宿主 +那一份(`lib/clang/22/lib/…`),所以 Linux 目标(宿主即目标)与 macOS 目标(未触及这些符号) +都链接成功,而 Windows 目标触及了它们且载荷没有那一份。 + +⇒ 今日的供给者是 `openkal-llvm-runtime`,它把 compiler-rt 与 libc++ 建在同一个包里。**实测** +纯 C 程序依赖它即成功: + +```toml +[dependencies] +openkal-llvm-runtime = { git = "…" } # 为 builtins 而非为 C++ +``` +``` +Target x86_64-windows-gnu → x86_64-w64-windows-gnu + → PE32+ executable (console) x86-64 +``` + +⚠️ 这是**打包粒度**的缺口,不是本设计的缺陷:一个 C 程序为了三个符号带上整份 libc++。 +⇒ 后续应把 builtins 拆成独立包(`openkal-llvm-builtins`),届时形态 3 在三个平台上各只需 +一个依赖。**本轮不做**,因为它是新包 + 新索引条目,与目标侧解析正交。 + +``` + Target aarch64-macos → arm64-apple-macos14.0 + kernel-abi openkal@0.5.1 (openkal-macos@0.3.1, graph) + c-abi musl (openkal-musl@0.3.1, graph) + c++ — ⭐ C 程序不需要它 +``` + +⚠️ **本形态今日构建失败**(§1),本设计的判据 1 即为它。 + +--- + +**形态 4 —— 直接用 openkal:只要那 48 个接口** + +```toml +[dependencies] +openkal-linux = { git = "…/openkal-linux", features = ["standalone"] } +std-freestanding-nolibc = "^0.2.0" # ⭐ 见下 + +[target.x86_64-linux-gnu] +sysroot = "" +``` + +⭐ **实测(2026-08-24)必须补两处,而两处都是本形态的性质:** + +其一,`std-freestanding-nolibc`。openkal-linux 自身要用 `memset`,而这一层之下没有 C 库: + +``` +ld.lld: error: undefined symbol: memset +``` + +这正是该包存在的理由 ——「a freestanding C++ library still needs 五个函数四个头」。 + +其二,mcpp 侧的链接行。驱动被指向一个 hosted 三元组时会自带 crt 启动件与动态加载器,而本形态 +两者都不该有: + +``` +/usr/lib/gcc/x86_64-linux-gnu/13/crtbeginS.o (outside the sandbox) +/lib64/ld-linux-x86-64.so.2 (outside the sandbox) +``` + +⇒ 本轮实现:`c-abi` 缺席时链接行加 `-nostdlib -static`。理由不是策略而是性质 —— **没有 C 库 +就没有它的启动件,也没有属于这个程序的解释器。** + +实测结果: + +``` + Compiling openkal-linux / std-freestanding-nolibc v0.2.0 + Finished dev +产物: ELF 64-bit LSB executable, x86-64 运行: raw openkal exit=0 +``` + +| | | +|---|---| +| 得到 | 48 个 `kal_*`;`standalone` 表示这个实现即程序环境的全部 | +| 失去 | C 标准库 —— 没有 `printf`,有 `kal_stream_write` | +| 适用 | 内核、加载器、要求最小面积的程序 | + +``` + Target x86_64-linux-gnu + kernel-abi openkal@0.5.1 (openkal-linux@0.5.1, graph, standalone) + c-abi — + c++ — +``` + +⭐ 三行里只有一行有内容 —— 这正是「只要那 48 个接口」在输出里的样子。 + +--- + +**形态 5 —— freestanding C++ 建在 openkal 之上,不要 C 库** + +```toml +[dependencies] +openkal-linux = { git = "…/openkal-linux", features = ["standalone"] } +std-freestanding = { git = "…/std-freestanding", features = ["alloc-kal", "nolibc"] } +``` + +| | | +|---|---| +| 得到 | C++ 标准库的 freestanding 子集(`import mcpplibs.std.freestanding`);分配器经 `alloc-kal` 转发到 openkal | +| 失去 | 完整标准库;`nolibc` 下**实测** 103 个头中 94 个可编 | +| 适用 | 要 C++ 抽象而不要 C 库的裸机 / 内核程序 | + +``` + Target x86_64-linux-gnu + kernel-abi openkal@0.5.1 (openkal-linux@0.5.1, graph, standalone) + c-abi — + c++ freestanding subset (std-freestanding@0.5.0, graph) + ↳ allocator: openkal (std-freestanding-alloc-kal@0.1.0) +``` + +⚠️ `c++` 行显示 `subset` 而非 `libc++`,因为 `[package] std-module` 未声明 ⇒ `import std` 不 +可用,可用的是 `import mcpplibs.std.freestanding`。⭐ 这个区分**不需要第二个能力名**,它由 +清单里那个键本身承载(§17.2)。 + +⭐ `alloc-kal` 与 `alloc-libc` 是同一能力 `freestanding-allocator` 的两个实现 —— **换一个 +feature 就换掉分配器的下层**,这是构建期体系内部的组合。 + +--- + +**形态 6 —— 裸机 + picolibc(传统路线)** + +```toml +[target.riscv64-none-elf] +runner = ["qemu-system-riscv64", "-machine", "virt", "-nographic", "-bios", "default", "-kernel"] +``` + +| | | +|---|---| +| 得到 | picolibc 的 C 库 | +| 清单要写的 | **零依赖** —— C 库是目标的性质,由目标表给出 | + +``` + Target riscv64-none-elf + kernel-abi — ⭐ 裸机没有内核 + c-abi picolibc-riscv (xim:picolibc-riscv@1.8.12, prebuilt) + c++ — +``` + +⭐⭐ **把这一段与形态 2 的 `riscv64-none-elf` 并排看**,是本文档最要紧的一处对照:同一个目标, +picolibc 路线 `kernel-abi = —`,openkal 路线 `kernel-abi = openkal (openkal-opensbi)`。⇒ 后者 +之所以能跑同一份写给 Linux 的源码,原因就在这一行。 + +--- + +**形态 7 —— 裸机 + picolibc + freestanding 子集(按层混用)** ⭐ + +```toml +[dependencies] +std-freestanding = { git = "…/std-freestanding", features = ["alloc-libc"] } + +[target.riscv64-none-elf] +runner = [...] +``` + +| | | +|---|---| +| 说明 | ⭐ **一次构建同时属于两个体系**;`alloc-libc` 把分配器转发给那个预构建的 C 库 | + +``` + Target riscv64-none-elf + kernel-abi — + c-abi picolibc-riscv (xim:picolibc-riscv@1.8.12, prebuilt) ← 预构建 + c++ freestanding subset (std-freestanding@0.5.0, graph) ← 构建期 + ↳ allocator: picolibc (std-freestanding-alloc-libc@0.1.0) +``` + +⭐ 中间两行来源不同,**两个体系在同一次构建里并存**,而输出把它讲清楚了。这在清单里读不出来。 + +⇒ 这是 §13 可组合性的现存实例,也是「单一布尔表达不了目标侧」的证据。 + +--- + +**形态 8 —— 零 libc** + +```toml +[target.x86_64-none-elf] +sysroot = "" +``` + +| | | +|---|---| +| 得到 | 只有编译器内建与自己写的代码 | +| 说明 | `aarch64-none-elf` / `x86_64-none-elf` 两行目标表的 sysroot 列本就为空,**不写也落在此层** | + +``` + Target x86_64-none-elf + kernel-abi — + c-abi — + c++ — +``` + +⚠️ `sysroot = ""` 的语义是「**不放预构建目录**」,不是「这个程序没有 C 库」—— 形态 2 在裸机 +目标上也写它,而它经 openkal-musl **有** C 库。本设计一并修正该字段的注释与错误文案(§10)。 + +### 16.3 选择表 + +| 我要什么 | 形态 | 关键一行 | +|---|---|---| +| 就在本机跑,别麻烦 | 1 | (什么都不写) | +| 一份源码发到 Linux/macOS/Windows | **2** | `openkal-llvm-runtime` | +| 同上但只写 C | 3 | `openkal-musl` | +| 写内核 / 加载器,要最小面积 | 4 | `openkal-linux` + `standalone` | +| 内核里也想要 C++ 抽象 | 5 | `+ std-freestanding` + `alloc-kal` | +| RISC-V 裸机,用现成 C 库 | 6 | (目标表给 picolibc) | +| 同上但想要 C++ 子集 | 7 | `+ std-freestanding` + `alloc-libc` | +| 什么都不要 | 8 | `sysroot = ""` | + +⚠️ **形态之间没有开关,只有依赖。** 从形态 3 升到形态 2 是把 `openkal-musl` 换成 +`openkal-llvm-runtime`;从形态 6 升到形态 7 是加一个依赖。⇒ 不存在「切换到 openkal 模式」 +这样一个动作。 + +### 16.4 如何确认自己选中了什么 + +⭐ 这正是 §15 那一行输出的用途: + +``` +$ mcpp build --target aarch64-macos + Resolving toolchain + Resolved llvm@22.1.8 → aarch64-macos + Target aarch64-macos → arm64-apple-macos14.0 + kernel-abi openkal@0.5.1 (openkal-macos@0.3.1, graph) + c-abi musl (openkal-musl@0.3.1, graph) + c++ libc++ (openkal-llvm-runtime@0.1.0, graph) +``` + +⇒ 不需要读清单去推断,也不需要读文档 —— **构建过程报告它实际解析出的结果**。清单里可能写着 +一个已经过期的意图,而这三行不会。 + +⚠️ 而在本设计之前,这个问题**没有答案**:今日三处推导互不一致(§1),即使读遍清单也无法确定 +链接线上会出现什么。这是本设计最直接的使用侧收益。 + +--- + +## 17. 识别机制:mcpp 如何区分这些形态 + +### 17.1 ⭐⭐ 硬编码层名,不硬编码实现 + +| mcpp 里 | 内容 | 性质 | +|---|---|---| +| **硬编码** | 三个**层名**:`kernel-abi` / `c-abi` / `c++-abi` | 封闭集合,可校验,**不含任何产品名** | +| **绝不硬编码** | `openkal` / `openkal-musl` / `openkal-linux` / `picolibc` / … | 实现,mcpp 一个都不认识 | + +⇒ 这条分界是本设计能称为架构而非补丁的原因。今日 `prepare.cppm:1409` 里那句 +`fam == "openkal-llvm"` 违反的正是它 —— 一个产品名被写进了引擎。 + +⚠️ 层名之所以可以硬编码,是因为**层是有限的且由 C/C++ 的构建模型决定**(内核接口、C 库、 +C++ 运行时),不随生态增长;实现之所以不可以,是因为它们**正是要增长的东西**(§12:2×N×M)。 + +### 17.2 三个能力名,不是四个 + +| 层 | 能力名 | 今天 | 要加 | +|---|---|---|---| +| kernel-abi | `kernel-abi` | 无 | 五个平台后端 | +| c-abi | `c-abi` | 无 | `openkal-musl` | +| c++-abi | `c++-abi` | `hosted-standard-library`(改名) | `std-freestanding` | + +⭐ **hosted 与 freestanding 不需要各占一个能力名。** 二者的区别已由清单里的另一个键承载 —— +**实测**全生态只有 `openkal-llvm-runtime` 声明 `[package] std-module`,而 `prepare.cppm:5641` +正是读它: + +``` +声明 c++-abi + 声明 std-module → 完整标准库,`import std` 可用 +声明 c++-abi + 无 std-module → freestanding 子集(如 import mcpplibs.std.freestanding) +``` + +⇒ 能力名回答「**这一层有没有人供给**」,`std-module` 回答「**供给到什么程度**」。两个问题, +两个键,不需要把答案编进名字。 + +⚠️ `hosted-standard-library` → `c++-abi` 是改名。代价最低的时机就是现在:**实测**它只在一个 +包里出现(`openkal-llvm-runtime/mcpp.toml:18`),且尚未发布。 + +### 17.3 ⚠️ 校验:保留前缀,而非封闭整个 `provides` + +`provides` 今日**零校验**(`toml.cppm:458` 直接读进数组)。但它同时服务两类用途: + +| 用途 | 例 | 谁消费 | +|---|---|---| +| 目标侧层名 | `kernel-abi` | **mcpp 引擎** | +| 包之间的能力 | `freestanding-allocator` | 特性系统(包与包相互匹配) | + +⇒ 把整个 `provides` 变成封闭集合会**打断第二类**(`freestanding-allocator` 会被拒)。因此: + +```toml +provides = ["mcpp:kernel-abi"] # mcpp 拥有的名字空间:封闭集合,拼错即报错 +provides = ["freestanding-allocator"] # 包之间的能力:自由,mcpp 不过问 +``` + +⇒ 任何 `mcpp:*` 名字若不在封闭集合内即为**错误**而非静默忽略;其余名字照旧自由。这解决了 +§10 记录的「拼错一个字母则七处行为静默回退」。 + +⚠️ 前缀写法需要定:`mcpp:kernel-abi` 与 xpkg 引用 `xim:picolibc-riscv` 的形状一致,这是选它的 +理由;若不采用前缀,则需另一种把两类名字分开的办法。 + +### 17.4 解析器伪代码 + +```cpp +TargetSide resolve(const Manifest& root, const DepGraph& g, + const TargetRow& row, const Triple& t) +{ + TargetSide ts; + ts.llvmTriple = t.llvm_triple(row.macosVersion); // ⭐ §3:翻译本身承重 + + // ── kernel-abi ← triple 的 os 位 ──────────────────────────── + if (auto* p = g.provider_of("mcpp:kernel-abi")) + ts.kernelAbi = { Graph, p->declared_interface(), p->id() }; + else if (t.is_freestanding()) + ts.kernelAbi = { None }; // 裸机没有内核 + else + ts.kernelAbi = { Payload, t.os, payload.system_ref() }; + + // ── c-abi ← triple 的 env 位 ──────────────────────────────── + if (auto* p = g.provider_of("mcpp:c-abi")) + ts.cAbi = { Graph, p->declared_interface(), p->id() }; + else if (root.sysrootDeclared && root.sysroot->empty()) + ts.cAbi = { None }; + else if (auto x = root.sysroot.value_or(row.sysroot); !x.empty()) + ts.cAbi = { Xpkg, xpkg_interface(x), x }; + else + ts.cAbi = { Payload, t.env_or("glibc"), payload.libc_ref() }; + + // ── c++-abi(三元组无对应位)───────────────────────────────── + if (auto* p = g.provider_of("mcpp:c++-abi")) + ts.cxx = { Graph, p->declared_interface(), p->id(), + .subset = p->manifest.stdModule.empty() }; // ⭐ §17.2 + else if (ts.cAbi.origin == Payload) // ⭐⭐ §13 的原则 + ts.cxx = { Payload, payload.cxx_interface(), payload.cxx_ref() }; + else + ts.cxx = { None }; + + return ts; +} +``` + +⭐⭐ 倒数第二支即 §13 那条原则的**结构化形态**:载荷的 C++ 运行时只在 C 库也来自载荷时才被 +选中。⇒ **默认路径构造不出违规组合**,该诊断因此只在显式覆盖路径上需要。 + +⚠️ `declared_interface()` 是包声明的接口名(`openkal` / `musl` / `libc++`),供输出使用。它 +不参与任何判断 —— **mcpp 不认识这些名字,只转述它们**。这是 §17.1 那条分界在代码里的落点。 diff --git a/.github/workflows/openkal-cross.yml b/.github/workflows/openkal-cross.yml new file mode 100644 index 00000000..bfaf5f21 --- /dev/null +++ b/.github/workflows/openkal-cross.yml @@ -0,0 +1,224 @@ +name: openkal cross-build (3 hosts × 3 targets) + +# ⭐⭐ WHAT THIS WORKFLOW ASSERTS, AND WHY IT IS A MATRIX RATHER THAN A ROW. +# +# `cross-build-test.yml` verifies the crosses served by a PAYLOAD: a toolchain +# whose driver has exactly one target. There the host and the target are joined +# — `x86_64-w64-mingw32-g++` is the Windows cross and nothing else — so one row +# per supported combination is the honest shape. +# +# openkal changes the shape of the question. The target side — the C library, +# the C++ runtime, the platform's own implementation — is a set of PACKAGES in +# the dependency graph, and the compiler is an ordinary retargetable clang. The +# claim that follows is that N hosts × N targets collapses to N implementations +# plus one tool: **the machine doing the building stops being a variable.** +# +# ⚠️ THAT IS A CLAIM, AND CLAIMS OF THIS SHAPE HAVE BEEN WRONG IN THIS +# REPOSITORY. Reaching PE from a Linux host needed four separate repairs, and +# adding the other two hosts found seven more — every one of them a decision +# that had been keyed on which machine was building rather than on which machine +# the output was for: +# +# the link line's three host-shaped branches, only one of which carried +# `--target=`; the `std` module command's Windows branch, which dropped the +# package's own include paths; `cd X && …` not changing the drive in cmd.exe; +# the artefact-format test matching LLVM's `apple` rather than mcpp's `macos`; +# the C++ runtime contract naming a library to link when one was already in +# the objects; `-nostdinc` missing so a host SDK header could be found; and +# `-lgcc` naming GCC's runtime on a link whose compiler is clang. +# +# None of those was visible from one host. So the matrix is the test. +# +# ── The shape ────────────────────────────────────────────────────────────── +# +# THREE build jobs, one per host, each producing THREE artefacts — nine builds. +# THREE run jobs, one per system, each executing the artefact FOR that system +# produced by ALL THREE hosts. +# +# build on Linux build on macOS build on Windows +# run Linux ✓ ✓ ✓ +# run macOS ✓ ✓ ✓ +# run Windows ✓ ✓ ✓ +# +# ⭐ The diagonal is an ordinary native build. The six off-diagonal cells are +# the claim, and they are what a single-host workflow cannot reach. +# +# ⚠️ THE RUN JOBS INSTALL NOTHING — not mcpp, not a compiler, not a C runtime. +# A program above openkal carries its C library, its C++ runtime and its +# unwinder; what remains is the operating system it was built for. If a +# toolchain step is ever added to one of them because "the program needs it", +# that is the finding rather than the fix. +# +# ⚠️ AND THE ASSERTION IS ON THE OUTPUT, NOT THE EXIT STATUS. The program prints +# four lines, and `unwound: true` is the one a link cannot fake: it says a +# destructor ran while an exception was being carried out of a frame, which +# means the unwinder found this image's own frame descriptions. + +on: + pull_request: + workflow_dispatch: + +concurrency: + group: ci-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +env: + # ⚠️ No mcpp or xlings version here. `bootstrap-mcpp` owns both, and a second + # statement of them is a second thing to keep in step — the pin check + # (.github/tools/check_version_pins.sh) enforces the ones that exist and would + # not know about a copy in this file. + XLINGS_NON_INTERACTIVE: '1' + # The branch of the openkal packages this change is verified against. They + # move together with it; when they are on `main` this becomes `main`. + OPENKAL_BRANCH: feat/openkal-closure + +jobs: + build: + name: build 3 targets on ${{ matrix.host }} + runs-on: ${{ matrix.runner }} + timeout-minutes: 120 + strategy: + fail-fast: false + matrix: + include: + - { host: linux, runner: ubuntu-24.04 } + - { host: macos, runner: macos-14 } + - { host: windows, runner: windows-2022 } + defaults: + run: + shell: bash + steps: + - uses: actions/checkout@v4 + + # ⚠️ THE REPOSITORY'S OWN BOOTSTRAP, NOT A SECOND ONE. + # + # This job first wrote its own: fetch xlings, then + # `xlings install mcpp@`. It failed on the very first run: + # + # xlings: version '2026.8.17.1' not found for 'mcpp' + # available: 2026.8.19.4 + # + # `.xlings.json` at this repository's root pins the mcpp that BUILDS mcpp, + # and that pin does not move when mcpp is released — so it names a version + # the index no longer carries, and a bare install inside the checkout + # obeys the pin rather than the argument. `bootstrap-mcpp` already knows + # this (it runs `install_pinned_mcpp.sh`), works on all three systems, and + # shares the cache lineage every other job lands on. + # + # ⇒ Two bootstraps would be two things to keep correct, and the second one + # was wrong within a day of being written. + - uses: ./.github/actions/bootstrap-mcpp + + # ⭐ THE mcpp UNDER TEST. Everything after this uses the binary this step + # produces; the bootstrapped one above is only what compiles it. + - name: Build the mcpp in this pull request + run: | + set -euo pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + "$XLINGS_BIN" config --mirror GLOBAL 2>/dev/null || true + "$MCPP" self config --mirror GLOBAL 2>/dev/null || true + # ⚠️ `--dev` and not `--release`. What is under test is a set of + # decisions about compile and link flags; an optimisation level + # changes none of them and a release self-build is most of the budget + # of a job on a two-core runner. + "$MCPP" build --dev + BUILT=$(find target -type f \( -name 'mcpp' -o -name 'mcpp.exe' \) | head -1) + [ -n "$BUILT" ] || { echo "::error::mcpp did not build"; exit 1; } + BUILT=$(cd "$(dirname "$BUILT")" && pwd)/$(basename "$BUILT") + echo "MCPP_UNDER_TEST=$BUILT" >> "$GITHUB_ENV" + "$BUILT" --version + + - name: Select the toolchain the openkal packages ask for + run: | + set -euo pipefail + # ⚠️ Install, then select. `toolchain default` names one and does not + # fetch it. + "$MCPP_UNDER_TEST" self config --mirror GLOBAL 2>/dev/null || true + "$MCPP_UNDER_TEST" toolchain install llvm 22.1.8 + "$MCPP_UNDER_TEST" toolchain default 'llvm@22.1.8' + + - name: The program — one source, three targets + run: | + set -euo pipefail + git clone --quiet --depth 1 -b "$OPENKAL_BRANCH" \ + https://github.com/mcpplibs/openkal-llvm-runtime "$RUNNER_TEMP/okl" + cd "$RUNNER_TEMP/okl/examples/same-source" + mkdir -p "$RUNNER_TEMP/out" + # ⚠️ The three HOSTED targets. Bare metal is verified by + # `openkal-llvm-runtime`'s own CI under qemu; it has no runner here to + # execute on, and a build-only cell in a workflow whose point is + # running would be the weaker claim. + for t in x86_64-linux-gnu aarch64-macos x86_64-windows-gnu; do + rm -rf target + "$MCPP_UNDER_TEST" build --target "$t" + a=$(find target -type f \( -name 'openkal-same-source' -o -name 'openkal-same-source.exe' \) | head -1) + [ -n "$a" ] || { echo "::error::$t produced no artefact on ${{ matrix.host }}"; exit 1; } + case "$t" in + x86_64-windows-gnu) cp "$a" "$RUNNER_TEMP/out/windows.exe" ;; + aarch64-macos) cp "$a" "$RUNNER_TEMP/out/macos" ;; + *) cp "$a" "$RUNNER_TEMP/out/linux" ;; + esac + echo "${{ matrix.host }} → $t : $(ls -l "$a" | awk '{print $5}') bytes" + done + + - uses: actions/upload-artifact@v4 + with: + name: openkal-built-on-${{ matrix.host }} + path: ${{ runner.temp }}/out/ + if-no-files-found: error + + run: + name: run 3 builds on ${{ matrix.system }} + needs: build + runs-on: ${{ matrix.runner }} + timeout-minutes: 15 + strategy: + fail-fast: false + matrix: + include: + - { system: linux, runner: ubuntu-24.04, file: linux } + - { system: macos, runner: macos-14, file: macos } + - { system: windows, runner: windows-2022, file: windows.exe } + defaults: + run: + shell: bash + steps: + # ⚠️ NO checkout AND NO toolchain. This job is the claim: a program built + # above openkal needs the operating system it was built for and nothing + # else. Anything installed here would weaken what a pass means. + - uses: actions/download-artifact@v4 + with: { pattern: openkal-built-on-*, path: art } + + - name: The same program, from all three build hosts + run: | + set -euo pipefail + fail=0 + for host in linux macos windows; do + bin="art/openkal-built-on-$host/${{ matrix.file }}" + echo "──────── built on $host, running on ${{ matrix.system }} ────────" + if [ ! -f "$bin" ]; then + echo "::error::$bin is missing"; fail=1; continue + fi + # ⚠️ The executable bit does not survive an artefact upload. + chmod +x "$bin" || true + # ⚠️ arm64 macOS refuses an unsigned image, so the signature is + # asserted before the run: a failure here is "the linker did not + # ad-hoc sign it", which is a different repair from "it crashed". + if [ "${{ matrix.system }}" = "macos" ]; then + codesign -dv "$bin" 2>&1 | grep -q 'adhoc\|Signature' \ + || { echo "::error::built on $host: no code signature"; fail=1; continue; } + fi + if ! "./$bin" > out.log 2>&1; then + echo "::error::built on $host: it did not run"; cat out.log; fail=1; continue + fi + cat out.log + ok=1 + grep -q 'sorted: 2 4 7' out.log || ok=0 + grep -q 'caught: 42' out.log || ok=0 + # ⭐ The line a link cannot fake. + grep -q 'unwound: true' out.log || ok=0 + grep -q 'import std over openkal: ok' out.log || ok=0 + [ "$ok" = 1 ] || { echo "::error::built on $host: wrong output"; fail=1; } + done + [ "$fail" = 0 ] || exit 1 + echo "three builds, one system, same four lines" diff --git a/.gitignore b/.gitignore index 5f31875a..116bb5f7 100644 --- a/.gitignore +++ b/.gitignore @@ -43,3 +43,9 @@ bench/bench-report.json # --project mode writes the measured build's stdout/stderr next to the project bench-child.log .mcpp.toml.bench-backup + +# ⚠️ An empty file a shell redirection left behind. It was committed in #436 +# and lived on `main` for a week; the name reads like a variable that was meant +# to be expanded (`> $binDir`) and was not. Listed so the same slip is caught +# next time rather than reviewed again. +binDir diff --git a/binDir b/binDir deleted file mode 100644 index e69de29b..00000000 diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index f1819ff8..887e86d8 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -43,6 +43,21 @@ struct BuildProgramEnv { // hostprogram::toolchain_dir / sysroot_dir for why declaring was wrong. std::string toolchainDir; std::string targetSysroot; + // ⭐⭐ WHICH COMPILER RESOLVED — "gcc" | "clang" | "msvc" | "". + // + // A package should never have to guess this, and until this field existed + // the only way to was to look at `toolchainDir` and recognise a directory + // name. The question is real and recurring: the routines a compiler emits + // calls to and no C library defines live in `libgcc.a` under one and in + // compiler-rt under another, and the tool that turns a `.def` into an + // import library is `dlltool` under one and `llvm-dlltool` under another. + // + // ⚠️ Measured 2026-08-22, both on the same day and both from the same + // missing answer: `openkal-musl` naming `-lgcc` on a link whose compiler was + // clang (`unable to find library -lgcc`), and `openkal-windows` running + // `llvm-dlltool` under a GCC toolchain (`sh: 1: llvm-dlltool: not found`). + // Each package had made the assumption its author's toolchain made true. + std::string compilerId; // Three more answers a board-support package would otherwise hardcode. // // ⚠️ THE COUPLING THESE REMOVE IS INVISIBLE IN A MANIFEST. `riscv-virt-rt` @@ -302,6 +317,31 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv std::vector> e; auto hostT = mcpp::toolchain::triple::host_triple().str(); e.emplace_back("MCPP_TARGET", env.targetTriple.empty() ? hostT : env.targetTriple); + // ⭐⭐ THE SAME VALUE UNFILLED — EMPTY WHEN NOBODY NAMED A TARGET. + // + // `MCPP_TARGET` above answers "which machine is this for", and filling it + // in with the host is right for that question. It cannot answer a different + // one that a platform package has to ask: **was this build POINTED at a + // target**, or is it an ordinary native build? + // + // The two are not the same even when the triples are equal. `mcpp build + // --target aarch64-macos` on an arm64 Mac names the same machine the host + // is, and yet it is the graph that supplies the target side — so this tool + // puts no system SDK on the link, and the package that knows the system is + // the only thing that can name one. A native build on the same machine gets + // the SDK and needs nothing from the package. + // + // ⚠️ Measured 2026-08-23, `openkal-macos` trying to decide this from what + // was available. From the host: right for the cross, wrong for + // `--target aarch64-macos` ON a Mac (`library not found for -lSystem`). + // From `MCPP_TARGET`: right for the cross, wrong for the native build, + // because it is never empty (`undefined symbol: wcslen`, `strtoul`, … — + // the package's three-name stub had shadowed the vendor's complete one). + // + // ⭐ An older mcpp sets neither, and that is the correct answer for it: + // it has no graph-supplied target side, so the system is always on the + // link and a package should supply nothing. + e.emplace_back("MCPP_TARGET_REQUESTED", env.targetTriple); // Convenience splits of the resolved target (Cargo CARGO_CFG_TARGET_* // parity): parsed ONCE here through the canonical triple parser so every // build.mcpp stops hand-splitting MCPP_TARGET. MCPP_TARGET_ENV is "" when @@ -322,6 +362,7 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv // absent variable would make the answer depend on whatever the parent // process happened to export. e.emplace_back("MCPP_TOOLCHAIN_DIR", env.toolchainDir); + e.emplace_back("MCPP_COMPILER", env.compilerId); e.emplace_back("MCPP_TARGET_SYSROOT", env.targetSysroot); e.emplace_back("MCPP_TARGET_BUILTINS_LIB", env.targetBuiltinsLib); e.emplace_back("MCPP_TARGET_LIBC_PROFILE", env.targetLibcProfile); diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index 2d65bb5f..076a0c87 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -30,6 +30,7 @@ export module mcpp.build.distribution; import std; +import mcpp.toolchain.triple; export namespace mcpp::build::dist { @@ -102,6 +103,45 @@ enum class Contract { // priority-ordered initializer section, PE has no rpath, ELF has both. enum class Format { Elf, MachO, Pe }; +// ⭐⭐ WHICH FORMAT A TARGET PRODUCES, ASKED OF THE TARGET. +// +// `hostFallback` is what a triple outside the vocabulary falls back to, and it +// is a parameter rather than a compile-time constant so that this function can +// be examined without being the machine it is about. +// +// ⚠️ THIS USED TO BE A LAMBDA INSIDE A FIFTEEN-HUNDRED-LINE FUNCTION, AND THAT +// IS WHY IT HAD NO TEST. It tested the triple for the substrings `apple` and +// `darwin`, which are LLVM's words; mcpp's canonical form is `aarch64-macos` +// and contains neither, so the test fell through to a question about the HOST +// and produced opposite errors on opposite hosts: +// +// Linux host, macOS target → an ELF contract for a Mach-O +// macOS host, Linux target → a Mach-O contract for an ELF, which is +// `ld.lld: error: unable to find library +// -load_hidden` plus the host's own libc++.a on +// an ELF link line +// +// Both were found by running three hosts against three targets. Either would +// have been found by four lines of assertion, once this was a function. +// +// The substring tests remain as a fallback for a triple the vocabulary cannot +// parse — the `[target.X]` escape hatch — where a spelling is all there is. +Format format_for(std::string_view targetTriple, Format hostFallback) { + if (auto parsed = mcpp::toolchain::triple::parse(targetTriple)) { + if (parsed->is_pe()) return Format::Pe; + if (parsed->os == "macos") return Format::MachO; + if (parsed->os == "linux" + || parsed->os == "none") return Format::Elf; + } + if (targetTriple.find("windows") != std::string_view::npos + || targetTriple.find("mingw") != std::string_view::npos) + return Format::Pe; + if (targetTriple.find("apple") != std::string_view::npos + || targetTriple.find("darwin") != std::string_view::npos) + return Format::MachO; + return hostFallback; +} + std::string_view to_string(Contract c) { switch (c) { case Contract::SelfContained: return "self-contained"; @@ -256,6 +296,27 @@ struct MechanismInput { // (measured 2026-08-19). A target-side C++ runtime, if one is wanted, is // an ordinary package — the same way the libc is. bool freestanding = false; + // ⭐⭐ THE HOSTED FORM OF THE LINE ABOVE: a package in the graph supplies + // the C++ runtime, built for this target, and its objects are already on + // the link line. + // + // The table below has three answers and all of them name a runtime to LINK + // — the system's, the toolchain's, or a static form of one. Each is right + // when the runtime is something the artifact has to be JOINED to, and each + // is wrong here, where it is already inside. The archives it would find are + // the host's, which is the same defect the `freestanding` flag above + // exists for; the difference is only that this target has an OS. + // + // ⚠️ Measured 2026-08-23, cross-building for `aarch64-macos` over openkal + // right after the format decision was corrected to key on the target — the + // wrong format had been masking this: + // + // ld64.lld: error: library not found for -lc++ + // + // ⇒ Not "pick openkal's here". openkal's IS the objects; there is no + // library to name, and the honest flag is the one that stops the driver + // from adding its own. + bool graphCxxRuntime = false; }; struct Mechanism { @@ -351,7 +412,7 @@ Mechanism resolve(const MechanismInput& in) { // ELF here, and every ELF cell below reaches for the toolchain's HOST // archives. One of them silently produced a link line with // x86-64 libc++.a on a riscv64 link. - if (in.freestanding) { + if (in.freestanding || in.graphCxxRuntime) { m.effective = Contract::SelfContained; m.unitFlags = " -nostdlib++"; return m; diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 8b8ca8ec..75b1809f 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -513,7 +513,49 @@ CompileFlags compute_flags(const BuildPlan& plan) { std::string compile_toolchain_flags; std::string link_toolchain_flags; std::string link_toolchain_flags_c; // same, minus C++ runtime selection + // ⭐⭐ THE TRIPLE ON THE LINK LINE TOO, AND FOR A DIFFERENT REASON THAN ON + // THE COMPILE LINE. + // + // Compiling without it produces objects for the wrong machine. LINKING + // without it produces the wrong LINKER: `-fuse-ld=lld` names a family, and + // the clang driver picks the flavour from the target — `ld.lld` for ELF, + // `ld64.lld` for Mach-O, `lld-link` for PE. With no target it picks the + // host's. + // + // ⚠️ Measured 2026-08-23, cross-linking for macOS from Linux, after the + // objects were already correct Mach-O: + // + // ld.lld: error: obj/main.o: unknown file type + // + // — the ELF linker, handed Mach-O objects, describing them accurately and + // saying nothing about why it was the one running. + const std::string crossTarget = plan.toolchain.crossTargetFlag.empty() + ? std::string{} + : " " + plan.toolchain.crossTargetFlag; const bool isClangWithCfg = dm.hasCfg; + + // THE TARGET SIDE COMES FROM THE DEPENDENCY GRAPH, READ RATHER THAN + // DERIVED. + // + // This used to be `targetCxxRuntime && !crossTargetFlag.empty()`, and the + // twenty lines that stood here argued why neither condition could be + // dropped. The argument was sound about the two conditions and wrong about + // the question: both are proxies measured before the dependency graph + // exists, and a proxy cannot see a case it was not written for. + // + // The case it could not see was a C program. `targetCxxRuntime` says a + // package supplies a C++ RUNTIME, and a C program has none while its + // system still comes from the graph. So the gate in prepare admitted the + // build, this predicate rejected it, the payload's own libc++ stayed on the + // link line, and a macOS cross ended in: + // + // ld64.lld: error: …/lib/x86_64-unknown-linux-gnu/libc++.so: + // unhandled file type + // + // `mcpp.targetside` answers the question directly, after resolution, for + // every layer separately. Reading it here means this site and the gate + // cannot disagree, because there is nothing left to disagree about. + const bool graphTargetSide = plan.targetSide.system_from_graph(); // LLVM root of a clang-with-cfg toolchain — used by the macOS link // path below to locate libc++.a/libc++abi.a for staticStdlib. std::filesystem::path llvmRootForStdlib; @@ -567,18 +609,47 @@ CompileFlags compute_flags(const BuildPlan& plan) { llvmRootForStdlib = dm.llvmRoot; // Linker flags that cfg normally provides. The payload C-runtime // flags (-B/-L/loader) are appended via payload_ld below. - link_toolchain_flags = " --no-default-config"; + link_toolchain_flags = crossTarget + " --no-default-config"; + if (!crossTarget.empty()) { + // ⭐⭐ THE TARGET SIDE COMES FROM THE GRAPH, SO THE HOST'S MODEL + // CONTRIBUTES NOTHING — THE SAME REPLACEMENT `stdModuleFlags` + // ALREADY MAKES ON THE COMPILE SIDE. + // + // `lm.link_flags()` describes the C library THIS MACHINE has and + // `kLinkDriverFlags` selects the C++ runtime THE PAYLOAD ships. + // Both are right for a native link and both are wrong here: the C + // library, the C++ runtime and the platform are packages, and the + // package that knows a format states its own link line (openkal-musl + // carries `-nostdlib` plus that format's entry symbol). + // + // ⚠️ Measured 2026-08-23, after the correct linker was finally + // being chosen: + // + // ld64.lld: error: unknown argument '--as-needed' + // ld64.lld: error: unknown argument + // '--dynamic-linker=…/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2' + // + // — this host's glibc loader, handed to a Mach-O linker. Each + // message is accurate and none of them names the cause. + // + // `-fuse-ld=lld` stays because it names a FAMILY and the driver + // picks the flavour from the target; that is the one part of the + // selection that is still ours to make. + link_toolchain_flags += " -fuse-ld=lld"; + link_toolchain_flags_c = link_toolchain_flags; + } else { if (lm.mode == mcpp::toolchain::CLibMode::Sysroot) link_toolchain_flags += lm.link_flags(ninjaEsc); link_toolchain_flags_c = link_toolchain_flags + std::string(mcpp::toolchain::ClangDriverModel::kLinkDriverFlagsC); link_toolchain_flags += mcpp::toolchain::ClangDriverModel::kLinkDriverFlags; + } f.sysroot = link_toolchain_flags; } else if (lm.mode != mcpp::toolchain::CLibMode::None) { // GCC (or Clang without cfg): --sysroot from probe, or the payload // headers + C runtime (-B for crt discovery, -L for -lc/-lm). - link_toolchain_flags = lm.link_flags(ninjaEsc); + link_toolchain_flags = crossTarget + lm.link_flags(ninjaEsc); link_toolchain_flags_c = link_toolchain_flags; // nothing C++-only here f.sysroot = link_toolchain_flags; } @@ -833,31 +904,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { // // Target-keyed, not host-keyed: a Linux-hosted MinGW cross build // produces a PE and must take the PE answer. - const dist::Format format = [&] { - if (isMingwTc) return dist::Format::Pe; - // The TARGET's own word, when it has one. `isMingwTc` was the only - // cross case this knew about, so every other question about the - // output format was answered by asking the HOST — which is right - // whenever they agree and unaskable in a test that does not run on - // the platform it is about. A triple that names its OS is a fact; - // the host is a stand-in for one. - // - // Only ADDS answers: a triple that says neither falls through to - // exactly the previous derivation, so no existing build changes. - const auto& t = plan.toolchain.targetTriple; - if (t.find("windows") != std::string::npos - || t.find("mingw") != std::string::npos) - return dist::Format::Pe; - if (t.find("apple") != std::string::npos - || t.find("darwin") != std::string::npos) - return dist::Format::MachO; - if constexpr (mcpp::platform::needs_explicit_libcxx) - return dist::Format::MachO; - else if constexpr (mcpp::platform::is_windows) - return dist::Format::Pe; - else - return dist::Format::Elf; - }(); + // The format the TARGET produces. `isMingwTc` is kept ahead of the + // table because it recognises a mingw toolchain by more than its + // triple; everything after it is `dist::format_for`, which is where + // the question is answered and where it is tested. + const dist::Format format = + isMingwTc ? dist::Format::Pe + : dist::format_for(plan.toolchain.targetTriple, + mcpp::platform::needs_explicit_libcxx + ? dist::Format::MachO + : mcpp::platform::is_windows + ? dist::Format::Pe + : dist::Format::Elf); // `static_stdlib` is a faithful alias of the two ends of the contract: // its documented meaning has always been exactly self-contained vs the @@ -947,6 +1005,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { // "incompatible with elf64lriscv". See MechanismInput::freestanding. if (auto ft = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple)) mi.freestanding = ft->is_freestanding(); + // AND THE HOSTED FORM OF THE SAME FACT. The target's system comes from + // the graph — so, exactly as on bare metal, every archive the table + // below would reach for is the HOST's. + // + // The condition is the SYSTEM's origin and not the C++ runtime's. It + // was the latter until this line, and that is precisely why a C + // program over the same packages kept the payload's libc++ on its link + // line: the table asked whether a C++ runtime came from the graph, a C + // program has none, and the answer "no" was read as "so the payload's + // is right". A program with no C++ runtime needs the driver stopped + // from adding one just as much as a program that brought its own. + mi.graphCxxRuntime = plan.targetSide.system_from_graph(); const bool wantsArchives = (base == dist::Contract::SelfContained @@ -1107,7 +1177,21 @@ CompileFlags compute_flags(const BuildPlan& plan) { // but the link line has a hard 128KiB ceiling (MAX_ARG_STRLEN) that real // workspaces already spend 43% of. std::string payload_ld; + // ⚠️ AND NOT WHEN THE TARGET SIDE COMES FROM THE GRAPH — the second half of + // the replacement made where `link_toolchain_flags` is built. `lm` describes + // THIS MACHINE's C runtime: `-B`/`-L` into the payload and this host's + // dynamic loader. Measured 2026-08-23, with the correct linker already + // running for a macOS cross: + // + // ld64.lld: error: unknown argument + // '--dynamic-linker=…/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2' + // + // ⚠️ The first replacement alone was not enough, and that is the finding: + // the C-runtime group reaches the link line through TWO channels, and a + // reader who fixed one saw the identical error and could reasonably + // conclude the fix had not worked. if (isClangWithCfg + && plan.toolchain.crossTargetFlag.empty() && lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) payload_ld = lm.link_flags(ninjaEsc); // GCC: replace the payload's patched `*link:` with the pristine one, so @@ -1139,7 +1223,13 @@ CompileFlags compute_flags(const BuildPlan& plan) { // Windows MinGW build (host≠target). No rpath/loader/payload model. Static // + libstdc++exp (std::print's __open_terminal/__write_to_terminal live in // libstdc++exp.a, not plain libstdc++). Self-contained binutils → no -B. - if (isMingwTc) { + // ⚠️ AND ONLY WHILE THE PAYLOAD IS THE TARGET, which is what the paragraph + // above assumes without saying so: `x86_64-w64-mingw32-g++` needs no + // `--target` because it HAS no other. When the target side comes from the + // dependency graph the compiler is an ordinary retargetable clang, and this + // branch is one of three shaped by the HOST rather than by the target — see + // the replacement below, which covers all three at once. + if (isMingwTc && !graphTargetSide) { // `-static` / `-static-libstdc++` now come from the contract table via // unit_ldflags (dist::Format::Pe) — the whole-link `-static` is what // "self-contained" means here, since the piecemeal recipe still leaves @@ -1203,7 +1293,22 @@ CompileFlags compute_flags(const BuildPlan& plan) { // // Native cl.exe (isMsvcDialect, returned above) keeps link.exe: there // the response file is ours, and 2026.8.5.3 already fixed it. - f.ld = std::format(" -fuse-ld=lld{}{}{}", link_intent_ld, + // ⚠️ `full_static` IS ON THIS LINE, AND IT WAS NOT. + // + // The two branches below both carry it; this one did not, and nothing + // showed because a Windows host's `-static` for an ELF target was + // arriving from the C++ runtime contract instead — which had chosen the + // PE cell, because the FORMAT question above was being answered by + // asking which machine was building. Correcting that answer removed the + // flag, and the artefact this job asserts about changed shape: + // + // mcpp-linux-musl: ELF 64-bit LSB executable, x86-64, … + // dynamically linked, interpreter /lib/ld-musl-x86_64.so.1 + // + // where every other host produces a static one. ⇒ Whole-program static + // linkage is a property of the TARGET (`target_supports_full_static` + // plus the manifest's `linkage`), so it belongs on every host's line. + f.ld = std::format("{} -fuse-ld=lld{}{}{}", full_static, link_intent_ld, user_ldflags, link_extra); f.ldC = f.ld; // no C++ runtime token on this line } else if constexpr (mcpp::platform::needs_explicit_libcxx) { @@ -1256,8 +1361,16 @@ CompileFlags compute_flags(const BuildPlan& plan) { // its own. Inject `-latomic` (under --as-needed) after runtime_dirs // so its -L entries are on the search path; self-guards on the lib // actually being present (see atomic_link_flag). - std::string atomic_ld = atomic_link_flag(plan.toolchain.linkRuntimeDirs, - !full_static.empty()); + // ⚠️ THE THIRD CHANNEL. `libatomic` is this HOST's, found by looking in + // the payload's directories, and `--push-state` / `--as-needed` are GNU + // ld spellings that a Mach-O or PE linker does not have. A target whose + // runtime comes from the graph gets its own answer to oversized + // `std::atomic` from that graph. + std::string atomic_ld = + plan.toolchain.crossTargetFlag.empty() + ? atomic_link_flag(plan.toolchain.linkRuntimeDirs, + !full_static.empty()) + : std::string{}; f.ld = std::format("{}{}{}{}{}{}{}{}{}", full_static, link_toolchain_flags, b_flag, runtime_dirs, link_intent_ld, atomic_ld, payload_ld, @@ -1271,6 +1384,130 @@ CompileFlags compute_flags(const BuildPlan& plan) { user_ldflags, link_extra); } + // ── The target side comes from the graph, so the HOST's link is wrong ── + // + // ⭐⭐ THE THREE BRANCHES ABOVE ARE SHAPED BY THIS MACHINE, NOT BY THE + // TARGET. `if constexpr (is_windows)` / `needs_explicit_libcxx` / else is a + // question about where mcpp itself was built, and each answer describes a + // link on that machine: an SDK path, a deployment target, a loader search + // path, this host's `libatomic`. Every one of them is right when the target + // is the host or is served by a payload, and wrong when the C library, the + // C++ runtime and the platform are packages in the dependency graph. + // + // ⚠️ ONLY THE THIRD BRANCH EVER CONSUMED `link_toolchain_flags`, WHICH IS + // WHERE `--target=` LIVES. So a cross build over openkal linked correctly + // from a Linux host and would have handed a Mach-O or an ELF to a linker + // told nothing about the target from a macOS or a Windows one. The measured + // shape of that failure is on record from the PE case, which reached it a + // different way: + // + // ld.lld: error: obj/…/types.m.o: unknown file type (× 30) + // + // ⇒ Applied LAST and by REPLACEMENT, exactly as the freestanding block + // below is and for the same reason it is: what came before is not merely + // unnecessary but wrong, and appending to it would leave the outcome + // depending on the driver's flag ordering rather than on a decision. + // + // ⚠️ AND IT REPLACES A SPECIAL CASE RATHER THAN ADDING ONE. The PE branch + // above carried its own copy of this for one format; this covers PE, Mach-O + // and ELF, on every host, with the predicate stated once. + // + // What survives, and why each one is not the host's: + // full_static the contract table's, keyed on the target's FORMAT + // link_toolchain_flags `--target=`, `--no-default-config`, `-fuse-ld=lld` + // link_intent_ld what the user asked to build (exe/shared/static) + // user_ldflags the manifest's own words + // link_extra `-flto` / `-s`, profile decisions + // + // What does not: `b_flag` (this host's binutils), `runtime_dirs` and the + // `-rpath` beside them (this host's payload directories — measured on a + // Mach-O link as `-Wl,-rpath,…/lib/x86_64-unknown-linux-gnu`, which ld64 + // accepts and writes into the image), `payload_ld`, `atomic_ld`. + if (!isFreestandingTarget && graphTargetSide) { + // ⚠️ ASSEMBLED HERE RATHER THAN TAKEN FROM `link_toolchain_flags`, + // BECAUSE THAT STRING IS ONLY POPULATED WHEN THE PAYLOAD HAS A CONFIG + // FILE (`isClangWithCfg`). The Linux payload ships one and the Windows + // payload does not, so on a Windows host the replacement emitted no + // `--target=` at all and clang chose its own default: + // + // lld-link: error: obj/mcpplibs_openkal-linux/src/env.o: + // unknown file type (× many) + // + // ELF objects handed to lld's MSVC driver. `--target=` is not a + // property of whether a config file exists; it is the whole content of + // "which machine is this for". + std::string graphLd = crossTarget; + if (isClangWithCfg) graphLd += " --no-default-config"; + // NO C LIBRARY MEANS NO C LIBRARY'S STARTUP FILES EITHER. + // + // A hosted target whose C-ABI layer is absent is a real shape, not an + // incomplete one: a program that calls the platform interface directly + // — a kernel, a loader, anything that wants the smallest surface it can + // have — depends on the platform implementation and nothing above it. + // + // The driver does not know that. Told to emit for a hosted triple it + // supplies `crt1.o`, `crti.o`, the gcc startup objects and a dynamic + // linker, all of them the HOST's, and the hermetic link check reports + // them one by one: + // + // /lib/x86_64-linux-gnu/crti.o (outside the sandbox) + // /usr/lib/gcc/x86_64-linux-gnu/13/crtbeginS.o (outside the sandbox) + // /lib64/ld-linux-x86-64.so.2 (outside the sandbox) + // + // Measured, and the report names the symptom rather than the cause: the + // payload is not missing, it is being asked for something this program + // does not have. The platform package supplies the entry point (its + // `standalone` feature says so); what the driver must be told is to + // stop supplying one of its own. + // + // `-static` for the same reason, and it is not a policy choice. A + // dynamic executable names an interpreter in its program headers and + // the loader resolves its imports at run time; with no C library there + // is nothing to resolve and no interpreter that belongs to this + // program. Left off, the driver writes the HOST's: + // + // /lib64/ld-linux-x86-64.so.2 (outside the sandbox) + // + // — the one line that survived after `-nostdlib` removed the startup + // objects, measured. + if (plan.targetSide.cAbi.absent()) graphLd += " -nostdlib -static"; + // Names a FAMILY; the driver picks the flavour from the target, which + // is the one part of the selection that is still ours to make. + // + // Only for a driver that has lld. A graph-supplied target side does not + // imply clang — the same packages compiled by gcc are the intended + // second consumer — and `-fuse-ld=lld` handed to a gcc that was not + // built with it fails at the link with a message about a missing + // linker rather than about the choice made here. + if (plan.toolchain.compiler == mcpp::toolchain::CompilerId::Clang) + graphLd += " -fuse-ld=lld"; + + f.ld = std::format("{}{}{}{}{}", full_static, graphLd, + link_intent_ld, user_ldflags, link_extra); + f.ldC = f.ld; // no C++ runtime token on this line + + // AND THE SECOND CHANNEL, WHICH THE REPLACEMENT ABOVE DOES NOT REACH. + // + // `ldRuntimeFallback` carries `-Wl,-rpath` into this host's subos + // library view, and it is appended per unit rather than through `f.ld`. + // Measured on a native openkal build, whose target side is entirely + // from the graph: + // + // unit_ldflags = -nostdlib++ -Wl,-rpath,…/registry/subos/default/lib + // + // It did no harm there, because a program built over these packages + // links statically and the tag never reaches the image — `readelf -d` + // reports no dynamic section at all. That is luck rather than design: + // the path names directories on the machine that built the artifact, + // and the moment one of these targets produces a dynamic image it is a + // load-time reference to a directory the target machine does not have. + // + // Cleared for the same reason the freestanding block below clears it: + // a search path belongs to whoever supplies the libraries, and here + // that is the dependency graph. + f.ldRuntimeFallback.clear(); + } + // ── Freestanding: the target has no OS, so most of the above is wrong ── // // Applied LAST and by REPLACEMENT rather than woven in above, for two @@ -1290,7 +1527,8 @@ CompileFlags compute_flags(const BuildPlan& plan) { if (isFreestandingTarget) { if (auto spec = mcpp::freestanding::resolve( *mcpp::toolchain::triple::parse(plan.toolchain.targetTriple))) { - const auto prefix = mcpp::freestanding::compile_prefix(*spec); + const auto prefix = mcpp::freestanding::compile_prefix( + *spec, plan.toolchain.targetCxxRuntime); f.cxx += prefix; f.cc += prefix; f.as += mcpp::freestanding::assemble_prefix(*spec); diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index a88cf23c..6e7d4dee 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -212,6 +212,15 @@ inline const char* out_dir() { return env_or("MCPP_OUT_DIR" // `[toolchain]` actually resolved. inline const char* toolchain_dir() { return env_or("MCPP_TOOLCHAIN_DIR"); } +// Which compiler resolved: "gcc", "clang", "msvc", or "" if none did. +// +// ⭐ Ask this rather than inferring it from `toolchain_dir()`. The two questions +// a package has actually needed it for are which runtime library holds the +// routines the compiler emits calls to, and which spelling of a binutils tool +// exists beside the driver — and both have a different right answer per family +// rather than per version or per payload. +inline const char* compiler() { return env_or("MCPP_COMPILER"); } + // Where the TARGET's C library lives, for targets that have one of their own // (today: bare metal). Same argument one line up: the libc is a property of // the target, mcpp resolves it from the target's own row, and a package that diff --git a/src/build/plan.cppm b/src/build/plan.cppm index e21a8779..a6f110ef 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -7,6 +7,7 @@ export module mcpp.build.plan; import std; import mcpp.build.graph_shape; +import mcpp.targetside; import mcpp.build.loader_contract; import mcpp.manifest; import mcpp.source_kind; @@ -154,6 +155,16 @@ struct BuildPlan { mcpp::manifest::Manifest manifest; mcpp::toolchain::Toolchain toolchain; mcpp::toolchain::Fingerprint fingerprint; + // Where the target's platform interface, C library and C++ runtime come + // from, resolved once in prepare after the dependency graph is known. + // + // It is a field of the plan rather than of the toolchain because it is not + // a property of the compiler: the same clang serves a build whose system + // comes from its own payload and one whose system is built from source by + // that same clang. Attaching it to the toolchain is what produced the + // `openkal-llvm` family name, which had to be invented to carry a fact + // about the dependency graph on an object that describes a compiler. + mcpp::targetside::TargetSide targetSide; // Which graph this plan will write into build.ninja. The fingerprint does // NOT cover dev-deps or test targets, so `mcpp build` and `mcpp test` // share an output directory and overwrite each other's graph; this is what diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index c6881e61..657191bd 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -16,6 +16,7 @@ export module mcpp.build.prepare; export import mcpp.build.prepare_inputs; import std; +import mcpp.targetside; import mcpp.diag; import mcpp.home; import mcpp.platform.axis; @@ -36,6 +37,7 @@ import mcpp.toolchain.msvc; import mcpp.toolchain.registry; import mcpp.toolchain.stdmod; import mcpp.freestanding.target; // the target sysroot layout (libdir) +import mcpp.freestanding.linkline; // the ISA profile, for the std module command import mcpp.toolchain.post_install; import mcpp.toolchain.abi; import mcpp.toolchain.triple; @@ -755,12 +757,57 @@ sysroot_override(const mcpp::manifest::Manifest& m, // shape this codebase keeps paying for. A board package that got the right // answer as a root project and a stale one as a dependency would fail only in // the consuming build, which is the harder direction to debug. +// ⚠️⚠️ A NETWORK STEP OF A BUILD, RETRIED — AND IT HAD NO RETRY AT ALL. +// +// A dependency resolved by `git` is fetched on every machine that has not +// cached it, and a transport that hiccups once failed the whole build: +// +// error: git clone of 'https://github.com/…' failed: +// Cloning into '/home/runner/.mcpp/git/63269d80b47f71e6'... +// +// — no message from git, which is what a connection that dies mid-transfer +// looks like. Measured twice on 2026-08-23: once in continuous integration and +// once locally as `TLS connect error: … unexpected eof while reading`. +// +// ⚠️ THREE ATTEMPTS, AND THE LAST FAILURE IS REPORTED UNCHANGED. A wrong URL +// and a missing branch fail exactly as a transient fault does, so this cannot +// tell them apart and does not try: a permanent failure costs three seconds and +// produces the message it always did. Hiding a real error behind a retry is the +// worse trade, which is why the count is small and the report is untouched. +// +// ⚠️ BOTH NETWORK STEPS, not one. The first version retried only the clone — +// and a probe with a nonexistent repository failed in ONE second, because the +// step that runs first is `git ls-remote` and it was still bare. A retry on +// half of a path is a retry that reports success at having been added. +// +// `between` runs after a failed attempt: the clone needs the partial directory +// removed, or git's next attempt fails with "already exists and is not an empty +// directory" — a second, different error that says nothing about the first. +mcpp::platform::process::RunResult run_with_network_retry( + std::string_view command, + const std::function& between = {}) { + mcpp::platform::process::RunResult r{}; + for (int attempt = 1; attempt <= 3; ++attempt) { + r = mcpp::platform::process::capture(command); + if (r.exit_code == 0) return r; + if (between) between(); + if (attempt < 3) + std::this_thread::sleep_for(std::chrono::seconds(attempt)); + } + return r; +} + void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::toolchain::Toolchain* tc) { e.toolchainDir = (tc && !tc->binaryPath.empty()) ? tc->binaryPath.parent_path().parent_path().string() : std::string{}; e.targetSysroot = tc ? tc->targetSysrootRoot.string() : std::string{}; + e.compilerId = !tc ? std::string{} + : tc->compiler == mcpp::toolchain::CompilerId::GCC ? "gcc" + : tc->compiler == mcpp::toolchain::CompilerId::Clang ? "clang" + : tc->compiler == mcpp::toolchain::CompilerId::MSVC ? "msvc" + : std::string{}; e.targetLibc = tc ? tc->targetSysrootPkg : std::string{}; if (!tc) return; @@ -800,6 +847,13 @@ prepare_build(bool print_fingerprint, bool includeDevDeps = false, std::vector extraTargets = {}, BuildOverrides overrides = {}) { + // A refusal decided early and released late. `host_can_serve` answers + // "does a payload on this machine produce this target", which is knowable + // before dependency resolution and is only half the question: a package in + // the graph can supply the target's system, and the graph is not known + // here. Held until it is, and released only if nothing supplies it. + std::string unservedTargetDiagnosis; + auto root = overrides.project_root.empty() ? mcpp::project::find_manifest_root(std::filesystem::current_path()) : std::optional(overrides.project_root); @@ -1337,6 +1391,7 @@ prepare_build(bool print_fingerprint, bool hasExplicitSection = it != m->targetOverrides.end(); bool hasToolchainOverride = hasExplicitSection && !it->second.toolchain.empty(); + const triple::TargetInfo* known = parsed ? triple::find_known_target(*parsed) : nullptr; @@ -1381,7 +1436,28 @@ prepare_build(bool print_fingerprint, // The escape hatch stays open on purpose: an explicit `[target.X]` // toolchain override means the author is supplying the cross toolchain // themselves, and mcpp's payload matrix has no standing to refuse it. - if (known && known->tier != "planned" && !hasToolchainOverride && parsed + // DIAGNOSED HERE, REPORTED LATER, AND THE DIFFERENCE IS THE POINT. + // + // Whether a payload on this machine produces this target is knowable + // now. Whether anything ELSE produces it is not: a dependency can + // supply the target's platform interface and C library, and the + // dependency graph does not exist yet at this line. Refusing here + // therefore answered a narrower question than the one it claimed — + // measured, a project that only had to add a dependency was told its + // machine could not build the target at all. + // + // The refusal is kept in full, because it is right whenever nothing + // supplies the target side, which remains the ordinary case. It is + // carried to where the graph is known and released there. Nothing + // between here and there consumes the answer: what follows is toolchain + // and dependency resolution, and a target no payload serves resolves to + // a driver that simply will not be asked to emit anything. + // + // The escape hatch stays open on purpose: an explicit `[target.X]` + // toolchain override means the author is supplying the cross toolchain + // themselves, and mcpp's payload matrix has no standing to refuse it. + if (known && known->tier != "planned" && !hasToolchainOverride + && parsed && !mcpp::toolchain::host_can_serve(*parsed)) { std::string servable; for (auto const& info : triple::known_targets()) { @@ -1391,16 +1467,20 @@ prepare_build(bool print_fingerprint, if (!servable.empty()) servable += ", "; servable += t->str(); } - return std::unexpected(std::format( - "target '{}' cannot be built on this host — no toolchain payload " - "exists that runs here and produces it.\n" - " this host can build: {}\n" - " Build it on a host that can, or supply your own cross " - "toolchain with an\n" - " explicit [target.{}] toolchain = \"…\" section.", + unservedTargetDiagnosis = std::format( + "target '{}' cannot be built on this host.\n" + " No toolchain payload here produces it, and nothing in " + "the dependency graph\n" + " supplies its system side.\n" + " this host can build with the payload alone: {}\n" + " To build it anyway, depend on a package that implements " + "the target's system\n" + " (its kernel interface and C library), or supply your own " + "cross toolchain with\n" + " an explicit [target.{}] toolchain = \"…\" section.", parsed->str(), servable.empty() ? "(nothing — `mcpp toolchain list`)" : servable, - parsed->str())); + parsed->str()); } // Canonical from here on: cfg evaluation, spec attachment and the // target/ output directory all see one spelling. @@ -1427,16 +1507,28 @@ prepare_build(bool print_fingerprint, // A convention, not an instruction: on the Windows-GNU first-run path // this is what turns the seeded target into `gcc@16.1.0`. // - // It must not fire when a REMEMBERED target would overrule a - // toolchain the user wrote down. Once the no-Visual-Studio fallback - // persists `default_target = x86_64-windows-gnu`, every later project - // inherits that target — and the pin attached to it would then - // silently replace an explicit `[toolchain] windows = "llvm@…"`, - // which is exactly the promise the fallback is built on ("mcpp - // revises its own defaults, never yours"). A target the user asked - // for (--target, or [build] target) still wins, as it always has. - const bool pinWouldOverruleUser = - targetFromGlobalDefault && tc_origin_is_user_explicit(tcOrigin); + // It must not fire when it would overrule a toolchain the user wrote + // down. The pin is mcpp's own default for a target row — `gcc@16.1.0` + // for Windows-GNU, because the mingw payload is what supplies that + // target's headers and C library — and an explicit `[toolchain]` line + // is not a default. This is the promise the no-Visual-Studio fallback + // is built on: mcpp revises its own defaults, never yours. + // + // HOW THE TARGET WAS NAMED IS NOT PART OF THE QUESTION, and it used to + // be. The guard read `targetFromGlobalDefault && user_explicit`, so a + // target given on the command line disabled it — and then the row's pin + // replaced a toolchain the project had stated. Measured 2026-08-23: + // `--target x86_64-windows-gnu` with an explicit `llvm@22.1.8` resolved + // `x86_64-w64-mingw32-g++`, and gcc cannot compile libc++'s std module. + // + // A project that means to use a different compiler for a pinned target + // is stating something about its own build, and a project whose target + // side comes from its dependency graph is the ordinary reason to do so: + // the payload the row names supplies headers and a C library that such + // a project does not use. The narrower reading of this guard was + // patched with an openkal-specific exception; stating the rule + // correctly removes the need for one. + const bool pinWouldOverruleUser = tc_origin_is_user_explicit(tcOrigin); if (known && !hasToolchainOverride && !known->pin.empty() && !pinWouldOverruleUser) { tcSpec = std::string(known->pin); @@ -1822,15 +1914,55 @@ prepare_build(bool print_fingerprint, // `tc.targetTriple`, so correcting it here corrects all of them at once — // which is the point of there being one field rather than five answers. // - // ⚠️ Scoped to freestanding on purpose. The hosted cross targets already - // resolve a per-target binary, and overwriting their probed triple would - // replace a measured fact with an assumed one for no gain. + // ⚠️ THIS USED TO BE SCOPED TO FREESTANDING, WITH THIS REASON: + // + // The hosted cross targets already resolve a per-target binary, and + // overwriting their probed triple would replace a measured fact with + // an assumed one for no gain. + // + // ⭐⭐ That was true while every hosted cross was served by a payload. It + // stops being true when the TARGET SIDE comes from the dependency graph: + // the C library, the C++ runtime and the platform's own implementation are + // then packages built from source, and the compiler is an ordinary clang — + // whose `-dumpmachine` answers the host, exactly as the paragraph above + // describes for freestanding. + // + // ⚠️ Measured 2026-08-23, with an explicit `[target.aarch64-macos] + // toolchain = "llvm@…"`. The manifest's cfg evaluation used the REQUESTED + // target, so the C library's aarch64 headers were on the command line; the + // toolchain's own triple was still the host's, so code generation was + // x86_64. Two answers to one question, in one command: + // + // okm_float_assert.c: the C library and the compiler disagree about + // LDBL_DIG ('33 == 18') 33 = aarch64 binary128, 18 = x87 + // + // ⇒ The condition is now the property the first paragraph of this comment + // already names: a RETARGETABLE driver has to be told. gcc is not one — a + // gcc payload IS its target — so the mingw and musl-gcc crosses keep + // answering from `-dumpmachine`, which for them remains a measured fact. if (!overrides.target_triple.empty()) { if (auto want = mcpp::toolchain::triple::parse(overrides.target_triple); - want && want->is_freestanding()) + want && (want->is_freestanding() + || tc->compiler == mcpp::toolchain::CompilerId::Clang)) { tc->targetTriple = want->str(); + // And the flag that says it to the driver — for a HOSTED target + // only. Freestanding already emits its own `--target`, together + // with the ISA flags that must accompany it + // (freestanding/target.cppm); a second one here would be the same + // decision in two places. + if (!want->is_freestanding() + && tc->compiler == mcpp::toolchain::CompilerId::Clang) { + tc->crossTargetFlag = + "--target=" + want->llvm_triple( + mcpp::platform::macos::deployment_target( + m->buildConfig.macosDeploymentTarget)); + } + } + if (auto want = mcpp::toolchain::triple::parse(overrides.target_triple); + want && want->is_freestanding()) + { // `import std` is structurally hosted, and turning it off is the // SAME fact as the line above, not a second policy: libc++'s // std.cppm is one module over the whole library, including the @@ -4183,7 +4315,9 @@ prepare_build(bool print_fingerprint, std::format("mcpp.lock records no commit for branch " "'{}'", spec.gitRev), "resolve"); - auto r = mcpp::platform::process::capture(std::format( + // The FIRST network step of a git dependency, and therefore + // the one a transient fault is most likely to meet. + auto r = run_with_network_retry(std::format( "git ls-remote {} {} 2>&1", mcpp::platform::shell::quote(spec.git), mcpp::platform::shell::quote( @@ -4265,7 +4399,11 @@ prepare_build(bool print_fingerprint, mcpp::platform::shell::quote(gitRoot.string()), mcpp::platform::shell::quote(gitRoot.string()), mcpp::platform::shell::quote(resolvedGitRev)); - auto r = mcpp::platform::process::capture(cloneCmd); + // See `run_with_network_retry` for why, and for what the + // callback is removing between attempts. + auto r = run_with_network_retry(cloneCmd, [&] { + std::filesystem::remove_all(gitRoot, ec); + }); if (r.exit_code != 0) { std::filesystem::remove_all(gitRoot, ec); return std::unexpected(std::format( @@ -5289,6 +5427,142 @@ prepare_build(bool print_fingerprint, } } + mcpp::targetside::TargetSide resolvedTargetSide; + + // ── THE TARGET SIDE, RESOLVED ONCE ─────────────────────────────────────── + // + // HERE AND NOT EARLIER, AND THAT IS THE WHOLE POINT. + // + // mcpp serves two ways of supplying a target's platform interface, C + // library and C++ runtime, and the moment each becomes knowable is + // opposite: a prebuilt directory is known before dependency resolution, a + // set of packages only after it. Until now three separate derivations ran + // at the earlier moment and guessed the later answer — the family name in + // this file, `graphTargetSide` in flags, `graphCxxRuntime` in the contract + // — and they disagreed on the case none of them was written for. Measured: + // + // ld64.lld: error: …/lib/x86_64-unknown-linux-gnu/libc++.so: + // unhandled file type + // + // for a pure C program crossed to macOS, whose graph supplies a C library + // and no C++ runtime at all. + // + // Placing the resolution after capability binding and before the root + // build.mcpp means every later consumer reads one value, and a build + // program can be told what was resolved rather than re-deriving it. + { + namespace tsd = mcpp::targetside; + + // Scan the graph once for each layer. A package declares the layer it + // supplies and, optionally, the interface name it answers to: + // + // provides = ["mcpp:kernel-abi=openkal"] + // + // The engine knows the three layer names and nothing about the + // implementations that fill them. `hosted-standard-library` is accepted + // for the C++ layer as the spelling that shipped before this one, so an + // existing package keeps working unchanged. + auto provider_of = [&](tsd::CapLayer want) + -> std::optional { + std::optional found; + for (auto const& pkg : packages) { + for (auto const& entry : pkg.manifest.provides) { + std::optional decl; + if (auto parsed = tsd::parse_capability(entry); parsed && *parsed) + decl = **parsed; + else if (entry == "hosted-standard-library") + decl = tsd::CapDecl{ tsd::CapLayer::CxxAbi, {} }; + if (!decl || decl->layer != want) continue; + + tsd::Provider p; + p.name = pkg.manifest.package.name; + p.version = pkg.manifest.package.version; + p.interfaceName = decl->interfaceName; + p.hasStdModule = !pkg.manifest.stdModule.empty(); + // A package may carry both spellings during the transition, + // and the array order is the author's, not a preference. + // The current spelling names the interface; the older one + // cannot, so taking whichever came first would report a + // package name where an interface name belongs. + if (!found || (found->interfaceName.empty() + && !p.interfaceName.empty())) + found = p; + } + } + return found; + }; + + tsd::Inputs in; + if (tc) { + if (auto tt = mcpp::toolchain::triple::parse(tc->targetTriple)) { + in.llvmTriple = tt->llvm_triple( + mcpp::platform::macos::deployment_target( + m->buildConfig.macosDeploymentTarget)); + in.targetOs = tt->os; + in.targetEnv = tt->env; + in.freestandingTarget = tt->is_freestanding(); + + // `sysroot = ""` and "no sysroot key" are different answers and + // must not be collapsed: the first says this project wants no + // prebuilt C library, the second says it did not say. + if (auto const* ovr = sysroot_override(*m, *tt); ovr && ovr->empty()) + in.sysrootDeclaredEmpty = true; + else + in.sysrootXpkg = mcpp::toolchain::triple::effective_sysroot( + *tt, sysroot_override(*m, *tt)); + } + in.payloadLibcRef = tc->targetSysrootPkg; + in.payloadCxxInterface = tc->stdlibId; + } + in.kernelAbi = provider_of(tsd::CapLayer::KernelAbi); + in.cAbi = provider_of(tsd::CapLayer::CAbi); + in.cxxAbi = provider_of(tsd::CapLayer::CxxAbi); + + resolvedTargetSide = tsd::resolve(in); + if (auto why = tsd::check_layering(resolvedTargetSide)) + return std::unexpected(*why); + + // The refusal held since toolchain resolution, released now that the + // other half of its question has an answer. A payload on this machine + // does not produce this target; if the graph does not supply the + // target's system either, then nothing does and the diagnosis stands. + if (!unservedTargetDiagnosis.empty() + && !resolvedTargetSide.system_from_graph()) + return std::unexpected(unservedTargetDiagnosis); + + // A request that cannot be honoured is said so rather than dropped. + // + // Measured 2026-08-23: `[build] linkage = "dynamic"` on a project whose + // system comes from the graph produced a statically linked artifact and + // printed nothing. The outcome is correct — the graph supplies its + // libraries as objects compiled into this build, and there is no shared + // object for a loader to resolve at run time — but a directive that has + // no effect and no diagnostic is indistinguishable from one that was + // never read. + if (resolvedTargetSide.system_from_graph() + && m->buildConfig.linkage == "dynamic") + mcpp::ui::warning( + "`linkage = \"dynamic\"` has no effect when the " + "target's system comes from the dependency graph: those " + "packages are compiled into this build as objects, and there " + "is no shared object to link against. The artifact is static."); + + // Reported, and reported HERE rather than recorded in a manifest field. + // + // A line a project writes states an intention, and it goes stale the + // moment the packages beneath it change — a program that names its C + // library by name is naming a transitive dependency it did not choose. + // This states the outcome, so it cannot be stale, and it answers a + // question that until now had no answer at all: reading every manifest + // in the graph did not tell anyone what would end up on the link line, + // because three places derived it separately and could disagree. + mcpp::ui::info("Target", tsd::format_report( + resolvedTargetSide, + resolvedTargetCanonical.empty() + ? (tc ? tc->targetTriple : std::string{}) + : resolvedTargetCanonical)); + } + // ── L3: ROOT build.mcpp (moved after dependency resolution, design §3.1 // item 4) ──────────────────────────────────────────────────────────────── // Runs HERE — after dep resolution + feature activation (so the contract @@ -5482,6 +5756,135 @@ prepare_build(bool print_fingerprint, } bool needsStdModule = graph_or_targets_import_std(scan.graph, *m, *root); + // A standard library that came from a PACKAGE brings its own module + // source, because the compiler cannot be asked for one it does not have. + // + // `-print-library-module-manifest-path' is the right question when the + // standard library is the compiler's own. It is the wrong question when + // the library was configured by a package for a target the compiler + // knows nothing about: the source exists, and the compiler has never + // heard of it. So the package says where it is, and what it needs --- + // its include path and its own __config_site, neither of which the + // compiler would find. + // + // Both are read only from a package that ALSO provides the capability + // below. A package that named a std module without supplying the + // library would be describing something it does not have. + for (auto& pkg : packages) { + if (pkg.manifest.stdModule.empty()) continue; + const auto& provs = pkg.manifest.provides; + if (std::find(provs.begin(), provs.end(), + std::string{"hosted-standard-library"}) == provs.end()) + continue; + auto src = pkg.root / pkg.manifest.stdModule; + if (!std::filesystem::exists(src)) { + return std::unexpected(std::format( + "package '{}' declares [package].std-module = '{}', and there " + "is no such file under '{}'", + pkg.manifest.package.name, pkg.manifest.stdModule, + pkg.root.string())); + } + tc->stdModuleSource = src; + // ⚠️ AND THE COMPAT MODULE, FROM THE SAME PACKAGE OR NOT AT ALL. + // + // `std.compat` is a second module over the SAME library. Leaving it + // pointing at the toolchain's copy does not fail where it is set — it + // fails later, in that copy's own headers, against a configuration that + // was never generated for this target. Measured on a macOS cross: + // + // error: std module precompile failed (rc=1): + // …/xim-x-llvm/22.1.8/share/libc++/v1/std.compat.cppm + // …/include/c++/v1/__config:13: '__config_site' file not found + // + // — which reads as a broken toolchain payload and says nothing about + // the two libraries having been mixed. A package that supplies one + // module supplies both, or the pair is not offered. + if (!pkg.manifest.stdCompatModule.empty()) { + auto csrc = pkg.root / pkg.manifest.stdCompatModule; + if (!std::filesystem::exists(csrc)) { + return std::unexpected(std::format( + "package '{}' declares [package].std-compat-module = '{}', " + "and there is no such file under '{}'", + pkg.manifest.package.name, pkg.manifest.stdCompatModule, + pkg.root.string())); + } + tc->stdCompatSource = csrc; + } else { + tc->stdCompatSource.clear(); + } + tc->targetCxxRuntime = true; + tc->hasImportStd = true; + tc->importStdMinLevel = 20; // libc++'s own floor; see clang.cppm + // The target, first. On a freestanding target that means the whole ISA + // profile --- `--target', `-march', `-mabi', `-mcmodel' --- because a + // module built without them disagrees with every unit that imports it, + // and clang reports that as an ABI mismatch naming a .pcm file rather + // than the flag that split them. On a hosted one it is the triple alone. + std::string flags; + if (auto fs = mcpp::toolchain::triple::parse(tc->targetTriple); + fs && fs->is_freestanding()) { + if (auto spec = mcpp::freestanding::resolve(*fs)) + flags += mcpp::freestanding::compile_prefix(*spec, true); + } else if (!tc->crossTargetFlag.empty()) { + // ⚠️ `crossTargetFlag` and not `targetTriple`. The triple is mcpp's + // vocabulary (`aarch64-macos`); the flag carries the spelling a + // compiler takes (`arm64-apple-macos14.0`). Measured: emitting the + // first produced `--target=aarch64-macos`, which clang accepts as a + // triple it has never heard of and then treats as a bare-metal + // aarch64 — the module and its importers would agree with each + // other and with nothing else. + flags += " " + tc->crossTargetFlag; + // ⚠️ AND THE SECOND CHANNEL. `hostflags.cppm` reaches every ordinary + // translation unit; this command is assembled here instead, so a + // `std.pcm` built with SEH would be imported by units built with + // DWARF. Same function, not a second copy of the decision. + for (auto& f : mcpp::toolchain::graph_runtime_compile_flags(*tc)) + flags += " " + f; + } + // Everything up to here says which machine the module is for; what + // follows says where its headers are. The codegen step needs only the + // first — see Toolchain::stdModuleTargetFlags. + tc->stdModuleTargetFlags = flags; + for (auto& f : pkg.manifest.stdModuleFlags) { + // A flag naming a path is relative to the package that named it, + // for the same reason the module source is. + auto candidate = pkg.root / f; + flags += " " + mcpp::xlings::shq( + std::filesystem::exists(candidate) ? candidate.string() : f); + } + // ⚠️ AND THE HEADERS THIS PACKAGE ITSELF IS BUILT AGAINST. + // + // The std module source is one of this package's translation units in + // every way that matters, and it reaches the C library's headers the + // same way the rest of them do --- through the requirements the packages + // BENEATH this one publish. A package cannot name those in its own + // manifest: they belong to its dependencies, and their paths are known + // only after resolution. + // + // Measured: without them the module compiles until libc++ includes + // , which is the C library's, and stops there. + // publicUsage rather than privateBuild: the module is compiled once and + // imported by consumers, so the headers it must see are the ones the + // package PUBLISHES, not the ones it happens to build itself against. + // The two differ, and the difference is not cosmetic --- a package's own + // build path carries directories that exist for its .cpp files and that + // shadow the library's headers when a module is compiled against them. + for (auto& d : pkg.publicUsage.includeDirs) + flags += " -isystem " + mcpp::xlings::shq(d.string()); + for (auto& d : pkg.publicUsage.includeDirsAfter) + flags += " -idirafter " + mcpp::xlings::shq(d.string()); + // And the definitions, for the same reason as the directories: a C + // library's headers show a different library depending on which feature + // macros are set, and the ones this package is built with are the ones + // its own translation units see. Measured: without them the module + // reaches musl's and stops on `clockid_t', a name that header + // declares only under the macro the package carries. + for (auto& f : pkg.publicUsage.cxxflags) + flags += " " + mcpp::xlings::shq(f); + tc->stdModuleFlags = flags; + break; + } + if (needsStdModule && !tc->hasImportStd) { // A freestanding target reaches here for a reason the generic message // gets wrong. Nothing is missing from the toolchain — libc++'s std @@ -5512,8 +5915,29 @@ prepare_build(bool print_fingerprint, // not fix it either: the request has to name a version the index // actually carries. Publishing a new std-freestanding means updating // this literal in the same change. + // ⚠️ THE QUESTION IS WHETHER A HOSTED STANDARD LIBRARY IS PRESENT, NOT + // WHETHER THE TARGET IS FREESTANDING. + // + // Those were the same question for as long as no one had built one for + // such a target, and they stopped being the same when someone did: + // `mcpplibs/openkal-llvm-runtime' configures libc++, libc++abi and + // libunwind for a machine with no operating system, and a program above + // it has the library this refusal says it cannot have. + // + // The refusal is kept, because it is right in every case where nothing + // supplies one --- which is still the ordinary case, and the advice + // below is still the advice. What changes is that a package can now say + // otherwise, and it says so the way every other capability is declared: + // + // provides = ["hosted-standard-library"] + // + // A capability rather than a triple, because the fact is a property of + // the graph and not of the target, and because dependency resolution is + // the earliest time at which it is known. + const bool hostedStdProvided = + capProviders.find("hosted-standard-library") != capProviders.end(); if (auto ft = mcpp::toolchain::triple::parse(tc->targetTriple); - ft && ft->is_freestanding()) + ft && ft->is_freestanding() && !hostedStdProvided) { return std::unexpected(std::format( "`import std;` is not available on '{}' — a freestanding target " @@ -5695,6 +6119,11 @@ prepare_build(bool print_fingerprint, stdBmiPath, stdObjectPath, storeRoots); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); + // Resolved far above, where the dependency graph first exists. It is + // attached here rather than threaded through `make_plan` because nothing + // that function does depends on it: the flag assembly that does reads the + // plan, and every reader of `compute_flags` runs after this line. + ctx.plan.targetSide = resolvedTargetSide; // The module graph outlives the plan for one consumer: `mcpp pack`, which // has to know which units are INTERFACE (published as source) and which // are implementation (published only as an object). The plan flattens that diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index 01bfcac7..2a9eae1c 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -288,6 +288,45 @@ std::string canonical_package_build_metadata( s += pkg.manifest.package.version; s += " source="; s += pkg.manifest.package.sourceProvenance; + // ⭐⭐ WHAT THIS PACKAGE IS BUILT WITH, AND NOT ONLY WHAT IT ASKS THE + // RUNTIME FOR. + // + // Only the root's compile inputs used to reach the fingerprint, through + // `canonical_compile_flags` on the root manifest. A DEPENDENCY's + // `[build] cflags` / `defines` / `sources` / per-glob flags reached + // nothing — so editing one left the fingerprint unchanged, the consumer + // kept the same output directory, and the fast path replayed a + // build.ninja generated before the edit. + // + // ⚠️ AND THE WAY THAT SHOWS IS THAT THE EDIT APPEARS TO HAVE HAD NO + // EFFECT. Measured 2026-08-23 on a path dependency: a flag added to + // `[build] cflags` was absent from the generated `unit_cflags` after a + // rebuild, absent after touching the sources, and present the moment + // `target/` was removed. The first two observations are what a reader + // uses to conclude the flag is being filtered, and one was concluded + // and written down before the third measurement was taken. + // + // The comment beside the root-flag tail merge in prepare.cppm has said + // "canonical_package_build_metadata folds packages[].manifest. + // buildConfig" since before this fix. It now does. + // + // packages[0] is the root, whose flags `canonical_compile_flags` + // already folds; serialising it twice is harmless and keeps this loop + // one rule rather than one rule and an exception. + s += ' '; + s += canonical_compile_flags(pkg.manifest); + for (auto const& src : pkg.manifest.buildConfig.sources) { + s += " src:"; + s += src; + } + for (auto const& dir : pkg.manifest.buildConfig.includeDirs) { + s += " inc:"; + s += dir.generic_string(); + } + for (auto const& dir : pkg.manifest.buildConfig.includeDirsAfter) { + s += " inca:"; + s += dir.generic_string(); + } auto const& runtime = pkg.manifest.runtimeConfig; for (auto const& requirement : runtime.requirements) { s += " runtime-need:"; diff --git a/src/freestanding/linkline.cppm b/src/freestanding/linkline.cppm index 54b31b52..b5c9fff6 100644 --- a/src/freestanding/linkline.cppm +++ b/src/freestanding/linkline.cppm @@ -43,10 +43,10 @@ export namespace mcpp::freestanding { // `-dumpmachine`) nothing about the driver's identity says which target is // wanted. Omit it and the build silently produces host objects — measured, and // the exact shape of the E1 defect this work exists to close. -inline std::string compile_prefix(const Spec& s) { +inline std::string compile_prefix(const Spec& s, bool targetCxxRuntime = false) { std::string out; out += " --target=" + std::string(s.triple); - for (auto const& f : compile_flags(s)) { out += ' '; out += f; } + for (auto const& f : compile_flags(s, targetCxxRuntime)) { out += ' '; out += f; } return out; } diff --git a/src/freestanding/target.cppm b/src/freestanding/target.cppm index 1611e250..88871473 100644 --- a/src/freestanding/target.cppm +++ b/src/freestanding/target.cppm @@ -210,7 +210,13 @@ inline std::optional resolve(std::string_view triple) { // what the compiler may assume about the library (no `main` special-casing, no // builtin-to-libcall rewrites it cannot back up), and that assumption has to // hold for every TU in the build, including a dependency's. -inline std::vector compile_flags(const Spec& s) { +// `targetCxxRuntime` — a package in the graph supplies a C++ runtime BUILT FOR +// THIS TARGET (libc++abi and an unwinder). The comment below predicted this +// case and named it as the point at which the exception/RTTI pair stops being +// unconditional; the caller answers it from the capability the graph declares, +// so the answer is the graph's rather than a guess about the target. +inline std::vector compile_flags(const Spec& s, + bool targetCxxRuntime = false) { std::vector out; out.emplace_back(std::string("-march=") + std::string(s.march)); out.emplace_back(std::string("-mabi=") + std::string(s.mabi)); @@ -220,7 +226,56 @@ inline std::vector compile_flags(const Spec& s) { // `-ffreestanding` so the ordering of this function's output stays a // function of the table rather than of the row. for (auto flag : s.extra) out.emplace_back(flag); - out.emplace_back("-ffreestanding"); + // ⭐ AND `-ffreestanding` ITSELF IS ONE OF THE THINGS THE GRAPH DECIDES. + // + // The paragraph above this function names what the flag changes: "no `main` + // special-casing, no builtin-to-libcall rewrites it cannot back up". Both + // are statements about whether a library is there — and when a package in + // the graph provides `hosted-standard-library` FOR THIS TARGET, one is. + // `hosted` is the language's own word for not-freestanding, so a provider + // of that capability is asserting exactly the condition this flag denies. + // + // ⚠️ Measured 2026-08-23, and the way it showed was not a diagnostic about + // the flag. A bare-metal program whose `main` was an ordinary C++ `int + // main()` failed to link with `undefined symbol: main`, while `nm` on its + // own object showed `_Z4mainv` — under `-ffreestanding` a C++ `main` is not + // the reserved entry point and is therefore mangled like any other + // function. The startup object referred to `main` and nothing defined it. + // + // The alternative was to make every such program write `extern "C" int + // main()`, which is a workaround for a claim the build was making on the + // program's behalf and that was no longer true. + if (!targetCxxRuntime) out.emplace_back("-ffreestanding"); + // ⭐⭐ UNWIND TABLES, WHICH THE COMPILER TURNS OFF FOR THIS KIND OF TARGET + // AND WHICH NOTHING IN THE BUILD OTHERWISE SAYS. + // + // On a hosted ELF target clang emits `.eh_frame` for every function by + // default. On a bare-metal ELF target it does not — the assumption being + // that nothing will ever unwind. When a C++ runtime IS present for the + // target that assumption is wrong, and the way it is wrong is specific: + // the tables appear for anything compiled with `-fexceptions` (libc++abi, + // libunwind's C++ half, the program) and are ABSENT for everything else, + // which on this stack means the C library and libunwind's own C sources. + // + // ⚠️ AND A PARTIAL SET OF TABLES DOES NOT DEGRADE — IT STOPS THE WALK. + // Measured 2026-08-23 on riscv64-none-elf, and the measurement is worth + // keeping because every intermediate reading pointed elsewhere: + // + // __unw_get_proc_info -> 0 start=80200148 end=8020068c lsda=80447190 + // __unw_step -> 0 (UNW_STEP_END) + // after step -> 8021d55e + // + // The frame WAS found, its personality data WAS found, the step DID compute + // a return address — and `step` still reported the end of the stack, + // because libunwind re-derives the info for the caller and the caller was + // `__libc_start_main`, a C function with no table. `_Unwind_RaiseException` + // lives in libunwind's own `UnwindLevel1.c` and has none either, so a throw + // ends at the first step with `terminating due to uncaught exception`. + // + // The asynchronous form rather than `-funwind-tables`: it is what a hosted + // ELF target already gets by default, and the rest of this stack was + // developed against that behaviour. + if (targetCxxRuntime) out.emplace_back("-fasynchronous-unwind-tables"); // ⚠️ No C++ standard library headers. Not a preference — the toolchain's // libc++ headers are built for the HOST: `#include ` resolves to // libc++'s wrapper, which opens `<__config_site>`, which is generated per @@ -233,6 +288,8 @@ inline std::vector compile_flags(const Spec& s) { // package-private by design (the supply-chain rule in // mcpp.build.directives), so a libc wrapper includes the target headers // privately and exports what it wants seen. + // Kept in both cases: a target-side C++ library reaches a consumer through + // its own include dirs and modules, never through the compiler's. out.emplace_back("-nostdinc++"); // ⚠️ Exceptions and RTTI off, and this belongs HERE — with the target — for // the same reason `-ffreestanding` does: it is a property every TU in the @@ -259,8 +316,18 @@ inline std::vector compile_flags(const Spec& s) { // Not a preference, then, but not permanent either: a board that ships a // target-built libc++abi and unwinder has a real case for turning these // back on, and that is the point at which this becomes a manifest key. - out.emplace_back("-fno-exceptions"); - out.emplace_back("-fno-rtti"); + // + // ⭐ AND THAT POINT HAS ARRIVED. A package that provides the capability + // `hosted-standard-library' for this target IS the board described above: + // it carries libc++abi and libunwind compiled for it. With one present, + // forcing these off is what breaks the build --- the runtime is compiled + // with exceptions because it IMPLEMENTS them, and a graph that disagrees + // with it reports the same `exception handling was enabled in precompiled + // file' the paragraph above quotes. + if (!targetCxxRuntime) { + out.emplace_back("-fno-exceptions"); + out.emplace_back("-fno-rtti"); + } return out; } diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 256c19dd..c6da8b60 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -3,6 +3,7 @@ export module mcpp.manifest.toml; import mcpp.manifest.types; +import mcpp.targetside; import std; import mcpp.source_kind; import mcpp.libs.toml; @@ -455,7 +456,32 @@ std::expected parse_string(std::string_view content, } // [package] provides — package-level capabilities (Feature System v2 S3). - if (auto v = doc->get_string_array("package.provides")) m.provides = *v; + // + // Two populations share this array, and only one of them is mcpp's. Names + // under the reserved `mcpp:` prefix are target-side layers the engine + // resolves and acts on, so they are a closed set and a misspelling is an + // error here. Every other name belongs to the packages themselves — the + // feature system matches `requires` against `provides` without the engine + // having an opinion — so those pass through untouched. + // + // Validating the whole array instead would reject `freestanding-allocator`, + // which already ships. Validating none of it is what shipped until now, and + // its cost is that a single wrong letter in a layer name disables the + // behaviour it was meant to select while the build still reports success. + if (auto v = doc->get_string_array("package.provides")) { + for (auto const& entry : *v) + if (auto cap = mcpp::targetside::parse_capability(entry); !cap) + return std::unexpected(error(origin, cap.error())); + m.provides = *v; + } + // [package] std-module / std-module-flags — see manifest::types. Relative to + // the package root, because that is what a package can state about itself; + // the absolute path is made where the package's root is known. + if (auto v = doc->get_string("package.std-module")) m.stdModule = *v; + if (auto v = doc->get_string("package.std-compat-module")) + m.stdCompatModule = *v; + if (auto v = doc->get_string_array("package.std-module-flags")) + m.stdModuleFlags = *v; // [capabilities] cap = "provider" — root-only provider pins. if (auto* caps = doc->get_table("capabilities"); caps && !caps->empty()) { @@ -1501,16 +1527,21 @@ std::expected parse_string(std::string_view content, triple, e.cxxRuntime))); } } - // The target's C library, overriding the target table's `sysroot` - // column. Accepted forms are an xpkg reference (`xim:newlib-arm@4.4`) - // and the empty string. + // WHICH PREBUILT C LIBRARY DIRECTORY THIS TARGET TAKES, overriding + // the target table's `sysroot` column. Accepted forms are an xpkg + // reference (`xim:newlib-arm@4.4`) and the empty string. + // + // The empty string is MEANINGFUL and must not be normalised away. + // What it selects is "no prebuilt directory", which is NOT the same + // statement as "this program has no C library" — a project whose C + // library is built from source by a package in its dependency graph + // writes it too, and has one. Reading it as the stronger claim is a + // mistake the wording here used to invite. // - // ⚠️ The empty string is MEANINGFUL and must not be normalised - // away: it selects the zero-libc tier. Written into an - // `std::optional`, so "the key is absent" (inherit the target row) - // stays distinguishable from "the key is present and empty" (no C - // library at all). Collapsing the two is how a kernel project would - // silently get picolibc back. + // Written into an `std::optional`, so "the key is absent" (inherit + // the target row) stays distinguishable from "the key is present and + // empty". Collapsing the two is how a kernel project would silently + // get picolibc back. if (auto it = body.find("sysroot"); it != body.end() && it->second.is_string()) { std::string s = it->second.as_string(); if (!s.empty() && s.find(':') == std::string::npos) { @@ -1518,7 +1549,8 @@ std::expected parse_string(std::string_view content, "[target.{}].sysroot = '{}' is not an xpkg reference; " "expected `:[@]` (e.g. " "\"xim:picolibc-riscv@1.8.12\"), or \"\" for a target " - "with no C library.", triple, s))); + "that takes no prebuilt C library directory.", + triple, s))); } e.sysroot = std::move(s); e.sysrootDeclared = true; diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index d4cac1a3..e082dc9b 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -904,6 +904,24 @@ struct Manifest { // one provider from the graph. See // .agents/docs/2026-06-29-feature-capability-model-design.md. std::vector provides; // package-level + // [package] std-module / std-module-flags — a package that IS a standard + // library says where its `std' module source is and what that source needs + // to compile. The build tool otherwise asks the COMPILER where std.cppm is + // (`-print-library-module-manifest-path'), which is right whenever the + // standard library is the compiler's own and wrong when it is a package's: + // that source was configured for a target the compiler knows nothing about, + // and its include path and its __config_site are the package's. + // + // Both are read only from a package that also provides the capability the + // std-module gate reads; a package that says one without the other is + // saying something about a library it does not supply. + std::string stdModule; // relative path + // The second module over the SAME library. A package that names + // `std-module` and not this one offers `import std;` and not + // `import std.compat;` — a complete answer, and better than silently + // pairing its own `std` with the toolchain's `std.compat`. + std::string stdCompatModule; + std::vector stdModuleFlags; std::map> featureProvides; // feature → caps std::map> featureRequires; // feature → caps // Feature System v2 Stage 2a — dependencies activated by a feature. A dep diff --git a/src/targetside/model.cppm b/src/targetside/model.cppm new file mode 100644 index 00000000..28062562 --- /dev/null +++ b/src/targetside/model.cppm @@ -0,0 +1,363 @@ +// mcpp.targetside — WHERE THE TARGET SIDE COMES FROM, RESOLVED ONCE. +// +// THE DEFECT THIS MODULE EXISTS TO REMOVE. +// +// A build must answer one question before it can emit a command line: where do +// the target's platform interface, C library and C++ runtime come from. Until +// this module, that question was answered in three places with three different +// criteria: +// +// prepare `openkalTargetSide` the toolchain family name is "openkal-llvm" +// flags `graphTargetSide` targetCxxRuntime && !crossTargetFlag.empty() +// dist `graphCxxRuntime` targetCxxRuntime +// +// Measured 2026-08-23, a pure C program crossed to macOS over the openkal +// stack: +// +// ld64.lld: error: …/xim-x-llvm/22.1.8/lib/x86_64-unknown-linux-gnu/libc++.so: +// unhandled file type +// +// The first criterion admitted the build; the second rejected it, because a C +// program has no C++ runtime in its graph. ⇒ The link line kept the payload's +// own libc++ and handed a Linux shared object to a Mach-O linker. +// +// The three did not disagree by accident. mcpp serves two ways of supplying +// a target side, and the moment at which each is KNOWABLE is opposite: +// +// prebuilt a directory (compiler payload / xpkg sysroot) known BEFORE +// dependency resolution +// composed a set of packages built from source known AFTER it +// +// All three criteria ran at the prebuilt moment and guessed the composed +// answer. Three guesses at a fact that does not yet exist do not agree. ⇒ The +// fix is not a better guess. It is to resolve once, after the graph is known, +// and to have every consumer read that one value. +// +// WHY THIS IS A SEPARATE MODULE WITH NO DEPENDENCIES ON THE PIPELINE. +// +// `resolve` below is a pure function of plain data. It performs no I/O, reads +// no global state and knows nothing of ninja, toolchains or manifests. That is +// deliberate: the capability that preceded it (`hosted-standard-library`) drove +// seven behaviours from inside a 7000-line translation unit and had, measured, +// ZERO test coverage — there was no way to assert it short of running a whole +// build. Everything here can be asserted from a table. +export module mcpp.targetside; + +import std; + +export namespace mcpp::targetside { + +// ── The four ways a layer can be supplied ──────────────────────────────────── +// +// `Xpkg` and `Payload` are both "prebuilt", and they are still distinct: a +// payload directory is chosen by the toolchain and an xpkg by the target table +// or the manifest. A consumer that only needs "is this prebuilt" asks +// `prebuilt()`; one that needs to name the thing needs to know which. +enum class Origin { Payload, Xpkg, Graph, None }; + +constexpr std::string_view origin_name(Origin o) { + switch (o) { + case Origin::Payload: return "payload"; + case Origin::Xpkg: return "prebuilt"; + case Origin::Graph: return "graph"; + case Origin::None: break; + } + return "none"; +} + +// ── One layer of the target side ───────────────────────────────────────────── +// +// INTERFACE AND IMPLEMENTATION ARE TWO FIELDS, AND THE DISTINCTION CARRIES +// THE POINT OF THE ECOSYSTEM. +// +// `openkal` is an interface; `openkal-macos`, `openkal-windows`, +// `openkal-opensbi` are implementations of it. Collapsing them would hide the +// fact that one source reaches four machines because four packages answer to +// one name. On a traditional stack the two are often the same object — macOS +// supplies its kernel interface and its C library as one library — and that +// sameness is itself worth showing. +struct Layer { + Origin origin = Origin::None; + std::string interfaceName; // openkal / linux / win32 / darwin / musl / glibc / libc++ + std::string impl; // package@version, or an xpkg reference + bool subset = false; // C++ layer: a freestanding subset, not the whole library + + bool absent() const { return origin == Origin::None; } + bool prebuilt() const { return origin == Origin::Payload || origin == Origin::Xpkg; } + bool fromGraph() const { return origin == Origin::Graph; } +}; + +// ── The resolved target side ───────────────────────────────────────────────── +// +// Three layers, and their correspondence to the triple is not decoration: +// +// kernelAbi ← the triple's OS field linux / macos / windows / none +// cAbi ← the triple's ENV field gnu / musl / (msvc) +// cxx ← no field of the triple because it sits above the ABI +// +// The middle layer is implicit on a traditional stack — a C library issues +// syscalls or calls Win32 directly, and nothing names the seam. openkal's whole +// contribution is to name it, which is why `kernelAbi` reads `—` for a picolibc +// bare-metal build and `openkal` for an openkal one ON THE SAME TARGET. +struct TargetSide { + // The triple the driver is actually given, which is NOT the one the user + // wrote. Measured: `--target=aarch64-macos` produces a Mach-O whose + // MinVersion load command carries no platform and version 10.4, while + // `arm64-apple-macos14.0` carries `macos 14.0`. Right format, right + // architecture, wrong platform metadata — so the translation is load + // bearing and belongs in the report. + std::string llvmTriple; + + Layer kernelAbi; + Layer cAbi; + Layer cxx; + + // The single question the five former derivation sites actually asked. + // + // It is about the SYSTEM, not about the C++ runtime. A C program over + // openkal has no C++ runtime and its target side still comes from the + // graph — that case is exactly the measured defect above. + bool system_from_graph() const { + return kernelAbi.fromGraph() || cAbi.fromGraph(); + } +}; + +// ── Capability grammar: mcpp:[=] ─────────────────────────── +// +// mcpp HARDCODES LAYER NAMES AND NEVER HARDCODES IMPLEMENTATIONS. +// +// The three layer names below are a closed set compiled into the engine. The +// implementations that fill them — openkal, musl, picolibc, and whatever comes +// next — appear nowhere in this file or any other. That line is what separates +// this design from the string comparison it replaces (`fam == "openkal-llvm"` +// in prepare), which put a product name inside the engine. +// +// Layer names may be hardcoded because the layers are fixed by the C and C++ +// build model and do not grow. Implementations may not, because growing is +// precisely what they do: the ecosystem's combinations are 2×N×M while its +// packages are 2+N+M. +enum class CapLayer { KernelAbi, CAbi, CxxAbi }; + +constexpr std::string_view cap_layer_name(CapLayer l) { + switch (l) { + case CapLayer::KernelAbi: return "kernel-abi"; + case CapLayer::CAbi: return "c-abi"; + case CapLayer::CxxAbi: return "c++-abi"; + } + return {}; +} + +struct CapDecl { + CapLayer layer; + std::string interfaceName; // the `=` part; empty when omitted +}; + +// Parse one entry of a package's `provides` array. +// +// Returns: +// * an error — the name is in mcpp's reserved namespace and is not a +// layer this engine knows; +// * an empty optional — the name is not in mcpp's namespace at all, and +// belongs to the feature system (`freestanding-allocator`); +// * a declaration — a layer this engine acts on. +// +// THE MIDDLE CASE IS WHY THE PREFIX EXISTS. `provides` serves two +// populations: layer names the engine consumes, and capabilities packages match +// among themselves. Making the whole array a closed set would reject the +// second; leaving it entirely open means a typo in the first silently disables +// behaviour and the build still reports success. The reserved prefix keeps a +// closed set where one is needed and an open one everywhere else. +inline std::expected, std::string> +parse_capability(std::string_view entry) { + constexpr std::string_view kPrefix = "mcpp:"; + if (!entry.starts_with(kPrefix)) return std::optional{}; + + auto body = entry.substr(kPrefix.size()); + std::string_view layer = body, iface; + if (auto eq = body.find('='); eq != std::string_view::npos) { + layer = body.substr(0, eq); + iface = body.substr(eq + 1); + } + + CapDecl d{}; + if (layer == "kernel-abi") d.layer = CapLayer::KernelAbi; + else if (layer == "c-abi") d.layer = CapLayer::CAbi; + else if (layer == "c++-abi") d.layer = CapLayer::CxxAbi; + else + return std::unexpected(std::format( + "`provides = [\"{}\"]` names no capability mcpp knows.\n" + " The `mcpp:` prefix is reserved for the target-side layers " + "this engine resolves, and there are three:\n" + " mcpp:kernel-abi[=] the platform interface a C library sits on\n" + " mcpp:c-abi[=] the C library\n" + " mcpp:c++-abi[=] the C++ runtime\n" + " A capability of your own needs no prefix; those are passed " + "through untouched.", entry)); + + if (!iface.empty()) d.interfaceName = std::string(iface); + return std::optional{d}; +} + +// ── Resolver input ─────────────────────────────────────────────────────────── +// +// Plain data, assembled by the caller after dependency resolution. Keeping the +// pipeline out of this module is what makes the table in the unit tests a +// complete specification of the behaviour. +struct Provider { + std::string name; + std::string version; + std::string interfaceName; // from `mcpp:=`; may be empty + // Whether this package declares `[package] std-module`, which is what + // distinguishes a whole standard library from a freestanding subset. The + // distinction needs no second capability name: the capability says a layer + // has a supplier, and this key says how far the supply goes. + bool hasStdModule = false; + + std::string id() const { + return version.empty() ? name : std::format("{}@{}", name, version); + } + std::string display_interface() const { + return interfaceName.empty() ? name : interfaceName; + } +}; + +struct Inputs { + std::string llvmTriple; + std::string targetOs; // mcpp's own OS field, for the payload interface name + std::string targetEnv; // mcpp's own ENV field ("musl", "gnu", …) + bool freestandingTarget = false; + + std::optional kernelAbi; + std::optional cAbi; + std::optional cxxAbi; + + // The C library the prebuilt systems would supply, already resolved by the + // caller from `[target.X].sysroot` over the target table's column. + bool sysrootDeclaredEmpty = false; // `sysroot = ""` — the zero-libc tier + std::string sysrootXpkg; // an xpkg reference, or empty + + // What the payload would contribute, for display only. + std::string payloadSystemRef; + std::string payloadLibcRef; + std::string payloadCxxRef; + std::string payloadCxxInterface; // "libc++" / "libstdc++" / "MSVC STL" +}; + +// An xpkg reference is `:[@]`; the interface a reader +// wants to see is the name, not the whole address. +inline std::string xpkg_interface(std::string_view ref) { + auto colon = ref.find(':'); + auto body = colon == std::string_view::npos ? ref : ref.substr(colon + 1); + auto at = body.find('@'); + return std::string(at == std::string_view::npos ? body : body.substr(0, at)); +} + +// ── The resolution ─────────────────────────────────────────────────────────── +inline TargetSide resolve(const Inputs& in) { + TargetSide ts; + ts.llvmTriple = in.llvmTriple; + + // kernel-abi ← the triple's OS field. + if (in.kernelAbi) + ts.kernelAbi = { Origin::Graph, in.kernelAbi->display_interface(), + in.kernelAbi->id(), false }; + else if (in.freestandingTarget) + // Not a gap. A bare machine has no kernel, and saying so is the + // information: the same target reads `openkal` when an implementation + // of a kernel interface is in the graph, which is why one source can + // reach it at all. + ts.kernelAbi = { Origin::None, {}, {}, false }; + else + ts.kernelAbi = { Origin::Payload, in.targetOs, in.payloadSystemRef, false }; + + // c-abi ← the triple's ENV field. + if (in.cAbi) + ts.cAbi = { Origin::Graph, in.cAbi->display_interface(), in.cAbi->id(), false }; + else if (in.sysrootDeclaredEmpty) + ts.cAbi = { Origin::None, {}, {}, false }; + else if (!in.sysrootXpkg.empty()) + ts.cAbi = { Origin::Xpkg, xpkg_interface(in.sysrootXpkg), in.sysrootXpkg, false }; + else if (in.freestandingTarget) + ts.cAbi = { Origin::None, {}, {}, false }; + else + ts.cAbi = { Origin::Payload, in.targetEnv.empty() ? "glibc" : in.targetEnv, + in.payloadLibcRef, false }; + + // c++ — no field of the triple, because it sits above the ABI. + if (in.cxxAbi) + ts.cxx = { Origin::Graph, in.cxxAbi->display_interface(), in.cxxAbi->id(), + !in.cxxAbi->hasStdModule }; + else if (ts.cAbi.origin == Origin::Payload) + // THE LAYERING RULE, AS STRUCTURE RATHER THAN AS A LATER CHECK. + // + // An implementation must have been configured for the layer beneath it. + // The payload's libc++ was configured against the payload's C library — + // its `__config_site` records that configuration — so it is eligible + // only when the C library is also the payload's. Writing the rule here + // means the default path CANNOT construct the combination that produced + // the measured `unhandled file type`; a diagnostic is then needed only + // where an author overrides the contract explicitly. + ts.cxx = { Origin::Payload, in.payloadCxxInterface, in.payloadCxxRef, false }; + else + ts.cxx = { Origin::None, {}, {}, false }; + + return ts; +} + +// The same rule stated for the explicit-override path, where the resolver's +// structure no longer guarantees it. +inline std::optional check_layering(const TargetSide& ts) { + if (ts.cxx.origin == Origin::Payload && ts.cAbi.origin != Origin::Payload + && ts.cAbi.origin != Origin::None) + return std::format( + "the toolchain payload's C++ runtime cannot be used with a C " + "library that does not come from the payload.\n" + " c-abi {} ({}, {})\n" + " c++ {} ({}, payload)\n" + " The payload's C++ runtime was configured against the " + "payload's C library, and its configuration is recorded in the " + "headers it ships. It was never configured for this one.\n" + " Supply a C++ runtime from the dependency graph, or take " + "both from the payload.", + ts.cAbi.interfaceName, ts.cAbi.impl, origin_name(ts.cAbi.origin), + ts.cxx.interfaceName, ts.cxx.impl); + return std::nullopt; +} + +// ── Report ─────────────────────────────────────────────────────────────────── +// +// The build prints what it RESOLVED, and that is why this design adds no +// manifest field for the same information. A line in a manifest states an +// intention that goes stale when the packages beneath it change; this states +// the outcome and cannot. +inline std::string format_report(const TargetSide& ts, std::string_view targetName) { + // Thirteen spaces so the layer names sit under the triple rather than under + // the status verb: the caller's status line right-aligns a verb in twelve + // columns and follows it with one space. + constexpr std::string_view kIndent = " "; + auto line = [&](std::string_view label, const Layer& l) { + if (l.absent()) + return std::format("{}{:<11} —\n", kIndent, label); + std::string suffix = l.subset ? ", subset" : ""; + if (l.impl.empty()) + return std::format("{}{:<11} {} ({}{})\n", kIndent, label, + l.interfaceName, origin_name(l.origin), suffix); + return std::format("{}{:<11} {:<14} ({}, {}{})\n", kIndent, label, + l.interfaceName, l.impl, origin_name(l.origin), suffix); + }; + + // The head carries no verb of its own: the caller supplies one through the + // status line's own padding, and the layer lines below are indented to sit + // under it. + std::string head = (ts.llvmTriple.empty() || ts.llvmTriple == targetName) + ? std::format("{}\n", targetName) + : std::format("{} → {}\n", targetName, ts.llvmTriple); + + auto body = line("kernel-abi", ts.kernelAbi) + + line("c-abi", ts.cAbi) + + line("c++", ts.cxx); + if (!body.empty() && body.back() == '\n') body.pop_back(); + return head + body; +} + +} // namespace mcpp::targetside diff --git a/src/toolchain/clang.cppm b/src/toolchain/clang.cppm index 3c7f6034..184a5f70 100644 --- a/src/toolchain/clang.cppm +++ b/src/toolchain/clang.cppm @@ -186,6 +186,28 @@ std::vector std_module_build_commands(const Toolchain& tc, std::string_view sysrootFlag, std::string_view cppStandardFlag) { auto relBmi = std::filesystem::relative(bmiPath, cacheDir).string(); + // ⚠️ A PACKAGE-PROVIDED std MODULE REPLACES THE TOOLCHAIN'S SYSROOT FLAGS + // RATHER THAN BEING APPENDED TO THEM. + // + // Those flags describe the standard library the COMPILER ships and the C + // library the HOST has, and they lead with `-isystem' — so appending to them + // puts the host's headers ahead of the package's, and no later flag can + // undo it. Measured: the module then compiles the host C library's + // and stops on names that library expects the host compiler to + // have supplied. + // + // The triple has to be restated for the same reason: it was in the flags + // being replaced, and without it the module is built for whatever machine + // is doing the building. + // The replacement is complete: whoever set stdModuleFlags stated the target + // as well, because the triple was in the string being replaced and a module + // built without it is built for whatever machine is doing the building. + if (!tc.stdModuleFlags.empty()) sysrootFlag = {}; + const std::string& extraFlags = tc.stdModuleFlags; + // ⚠️ The codegen step compiles a BMI, which already carries what the + // headers contributed; only the machine has to be restated. See + // Toolchain::stdModuleTargetFlags. + const std::string& codegenFlags = tc.stdModuleTargetFlags; #if defined(_WIN32) // Windows: use absolute paths, raw binary path as first token // (cmd.exe strips leading quotes), shq for args with spaces. @@ -197,47 +219,86 @@ std::vector std_module_build_commands(const Toolchain& tc, // and generates harmless warnings about #include in module purview and // the reserved 'std' module name — suppress both. std::string ixxFlags = (ext == ".ixx") - ? " -x c++-module -Wno-include-angled-in-module-purview -Wno-reserved-module-identifier" + ? " -x c++-module -Wno-include-angled-in-module-purview" : ""; + // ⚠️ AND THE RESERVED-NAME WARNING UNCONDITIONALLY, WHICH IS WHAT THE OTHER + // BRANCH DOES. + // + // `export module std;` is a reserved identifier and every standard library + // that ships one triggers the warning; the non-Windows command has carried + // the suppression since it was written. This branch tied it to `.ixx`, + // which was correct while the only `std` module a Windows host ever saw was + // the MSVC STL's — and stopped being correct when a package could supply + // its own. Measured 2026-08-23, a Windows host building the openkal + // runtime's `llvm-generated/std.cppm`: + // + // std.cppm:167:15: warning: 'std' is a reserved name for a module + // [-Wreserved-module-identifier] + // + // A warning that is correct, unavoidable, and printed on every build is + // noise of the kind that hides the next one. + ixxFlags += " -Wno-reserved-module-identifier"; + // ⚠️ `extraFlags` IS ON BOTH COMMANDS HERE, AND IT WAS ON NEITHER. + // + // This branch was written when a Windows host built for itself against the + // MSVC STL, and `stdModuleFlags` did not exist — so the omission was not + // visible: there was nothing to omit. It became a defect when a package + // could supply its own `std` module, because that string is where the + // package's own headers, `-nostdinc` and the target triple live. + // + // ⚠️ Measured 2026-08-23, a Windows host cross-building for + // `x86_64-linux-gnu` over openkal — the command it produced carried FIVE + // tokens: + // + // clang++.exe -std=c++23 --precompile "…/std.cppm" -o "…/std.pcm" + // …/llvm-generated/std.cppm:16:10: fatal error: '__config' file not found + // + // The same build from a Linux host had `--target=`, `--no-default-config`, + // `-nostdinc`, `-nostdinc++` and eight `-I`s. The error names a header, and + // the cause is a branch keyed on which machine is doing the building. return { std::format( - "{} {}{}{} " + "{} {}{}{}{} " "--precompile {} -o {}", tc.binaryPath.string(), cppStandardFlag, ixxFlags, sysrootFlag, + extraFlags, mcpp::xlings::shq(tc.stdModuleSource.string()), mcpp::xlings::shq(absBmi)), std::format( - "{} {}{} " + "{} {}{}{} " "{} -c -o {}", tc.binaryPath.string(), cppStandardFlag, sysrootFlag, + codegenFlags, mcpp::xlings::shq(absBmi), mcpp::xlings::shq((cacheDir / "std.o").string())) }; #else return { std::format( - "cd {} && {}{} {} -Wno-reserved-module-identifier{} " + "cd {} && {}{} {} -Wno-reserved-module-identifier{}{} " "--precompile {} -o {} 2>&1", mcpp::xlings::shq(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, + extraFlags, mcpp::xlings::shq(tc.stdModuleSource.string()), mcpp::xlings::shq(relBmi)), std::format( - "cd {} && {}{} {} -Wno-reserved-module-identifier{} " + "cd {} && {}{} {} -Wno-reserved-module-identifier{}{} " "{} -c -o std.o 2>&1", mcpp::xlings::shq(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, + codegenFlags, mcpp::xlings::shq(relBmi)) }; #endif @@ -278,31 +339,79 @@ std::vector std_compat_build_commands(const Toolchain& tc, { auto relBmi = std::filesystem::relative(bmiPath, cacheDir).string(); auto relStdBmi = std::filesystem::relative(stdBmiPath, cacheDir).string(); + // ⚠️ THE SAME REPLACEMENT THE `std` BUILDER MAKES, FOR THE SAME REASON. + // + // `std.compat` is a second module over the SAME library, and it therefore + // needs the same headers, the same target and the same configuration. This + // used to take `sysrootFlag` unconditionally while its sibling above + // replaced it — so a package-provided pair had one module built against its + // own libc++ and the other against the toolchain's. + // + // ⚠️ It does not fail where the two are chosen. Measured on a macOS cross: + // + // error: std module precompile failed (rc=1): + // …/openkal-llvm-runtime/llvm-generated/std.compat.cppm:16 + // …/xim-x-llvm/22.1.8/include/c++/v1/__config:13 + // fatal error: '__config_site' file not found + // + // The SOURCE named is the package's; the header it opened is the + // toolchain's, whose per-installation configuration was never generated for + // this target. Reading that message, the mixture is invisible. + if (!tc.stdModuleFlags.empty()) sysrootFlag = {}; + const std::string& extraFlags = tc.stdModuleFlags; + // Same split as the `std` builder above: the second command compiles a BMI + // and needs the machine restated, not the include paths. + const std::string& codegenFlags = tc.stdModuleTargetFlags; // std.compat depends on std, so we need -fmodule-file=std= // Note: the path after = must NOT be shell-quoted separately; the // entire -fmodule-file flag is a single token to the compiler. + // + // ⚠️⚠️ ABSOLUTE PATHS AND NO `cd`, AND ONE FORM RATHER THAN TWO. + // + // This used to be `cd && … pcm.cache/std.pcm …`. `cd X && …` + // DOES NOT CHANGE THE DRIVE in cmd.exe — the build cache lives under the + // user's profile and a checkout lives wherever the runner put it, so on CI + // those are `C:` and `D:`. The `cd` succeeds, the drive stays where it was, + // and every relative path resolves against the wrong root. Measured + // 2026-08-23, a Windows host cross-building for `x86_64-linux-gnu`: + // + // std.compat.cppm:84:8: fatal error: module file 'pcm.cache\std.pcm' + // not found: module file not found + // + // — and `std.pcm` had been built successfully one command earlier. + // + // ⚠️ The obvious repair was a `#if defined(_WIN32)` branch, which is what + // the `std` builder above has. It was written and then withdrawn: a branch + // that only compiles on one platform is a branch this machine cannot check, + // and every defect this session found in the host dimension had exactly + // that shape — code shaped by which machine was doing the building. Naming + // absolute paths is correct everywhere, so there is one form. + auto absBmi = (cacheDir / relBmi).string(); + auto absStdBmi = (cacheDir / relStdBmi).string(); + auto absObj = (cacheDir / "std.compat.o").string(); return { - std::format("cd {} && {}{} {} -Wno-reserved-module-identifier{} " + std::format("{}{} {} -Wno-reserved-module-identifier{}{} " "-fmodule-file=std={} " "--precompile {} -o {} 2>&1", - mcpp::xlings::shq(cacheDir.string()), mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, - relStdBmi, + extraFlags, + absStdBmi, mcpp::xlings::shq(tc.stdCompatSource.string()), - mcpp::xlings::shq(relBmi)), - std::format("cd {} && {}{} {} -Wno-reserved-module-identifier{} " + mcpp::xlings::shq(absBmi)), + std::format("{}{} {} -Wno-reserved-module-identifier{}{} " "-fmodule-file=std={} " - "{} -c -o std.compat.o 2>&1", - mcpp::xlings::shq(cacheDir.string()), + "{} -c -o {} 2>&1", mcpp::toolchain::compiler_env_prefix(tc), mcpp::xlings::shq(tc.binaryPath.string()), cppStandardFlag, sysrootFlag, - relStdBmi, - mcpp::xlings::shq(relBmi)) + codegenFlags, + absStdBmi, + mcpp::xlings::shq(absBmi), + mcpp::xlings::shq(absObj)) }; } diff --git a/src/toolchain/compat.cppm b/src/toolchain/compat.cppm index b044be5b..af0dbb24 100644 --- a/src/toolchain/compat.cppm +++ b/src/toolchain/compat.cppm @@ -35,7 +35,7 @@ export namespace mcpp::toolchain::compat { // A user/config spec token pair, normalized to the two-axis identity model. struct NormalizedSpec { - std::string family; // "gcc" | "llvm" | "msvc" + std::string family; // "gcc" | "llvm" | "msvc" | "openkal-llvm" std::string version; // numeric (possibly partial), or "system"; never "-musl"-suffixed triple::Triple target; // empty = host // Set when a legacy spelling was rewritten; `hint` is the one-line note. @@ -108,7 +108,12 @@ std::optional normalize_spec(std::string_view compilerIn, out.version = version; // ── canonical families pass through ───────────────────────────────────── - if (compiler == "gcc" || compiler == "llvm" || compiler == "msvc") { + // ⭐ `openkal-llvm` is a canonical family and NOT an alias for `llvm`. The + // two resolve to the same payload, and they answer differently about which + // targets are reachable — see Family::OpenkalLlvm. An alias would collapse + // that difference, which is the whole content of the name. + if (compiler == "gcc" || compiler == "llvm" || compiler == "msvc" + || compiler == "openkal-llvm") { out.family = std::string(compiler); if (muslVersionSuffix && compiler == "gcc") { out.target = host_musl_triple(); diff --git a/src/toolchain/hostflags.cppm b/src/toolchain/hostflags.cppm index 0aae5038..594c90eb 100644 --- a/src/toolchain/hostflags.cppm +++ b/src/toolchain/hostflags.cppm @@ -29,6 +29,7 @@ import mcpp.platform; import mcpp.toolchain.model; import mcpp.toolchain.linkmodel; import mcpp.toolchain.registry; +import mcpp.toolchain.triple; export namespace mcpp::toolchain { @@ -122,6 +123,49 @@ std::vector host_compile_tokens(const Toolchain& tc, const auto dm = resolve_clang_driver(tc); const auto lm = resolve_link_model(tc); + // ⭐⭐ THE TRIPLE, SAID OUT LOUD, WHEN NOTHING ELSE SAYS IT. + // + // Every hosted cross this build tool could do was served by a payload whose + // driver had exactly one target — `x86_64-w64-mingw32-g++` needs no + // `--target` because it has no choice. So nothing emitted one outside the + // freestanding path, and the assumption "the driver knows" was true. + // + // It stops being true the moment the TARGET SIDE comes from the dependency + // graph instead of from a payload. Then the compiler is an ordinary clang, + // which emits every format it was built with, and which will emit for THIS + // machine unless told otherwise. + // + // ⚠️ Measured 2026-08-23. A build for `aarch64-macos` with an explicit + // `[target.aarch64-macos] toolchain = "llvm@…"` resolved the whole graph, + // took the C library's aarch64 headers, and compiled with no `--target` — + // host code generation, target declarations. It was caught by an assertion + // the C library port wrote for precisely this situation: + // + // the C library and the compiler disagree about LDBL_DIG ('33 == 18') + // + // 33 is aarch64's binary128 and 18 is x87: two machines in one command. + // + // The decision itself is not made here — see Toolchain::crossTargetFlag, + // which is set where both the request and the compiler are known. This + // reads it. + if (!tc.crossTargetFlag.empty()) out.push_back(tc.crossTargetFlag); + + // ⭐⭐ AND WHAT A `throw` AND A `thread_local` COMPILE INTO, WHICH IS A + // PROPERTY OF THE GRAPH AND NOT OF ANY ONE PACKAGE — see + // `graph_runtime_compile_flags` for what and why. + // + // ⚠️ IT WAS DECLARED PER-PACKAGE, WHICH IS EXACTLY AS FAR AS IT REACHED. + // `openkal-llvm-runtime` set `-fdwarf-exceptions` in its own `[build]`, so + // its objects agreed with each other and nothing else did. Measured + // 2026-08-23 — every object compiled, and the link said: + // + // ld.lld: error: undefined symbol: __gxx_personality_seh0 + // + // referenced from the CONSUMER's `main.o`, which had a `try` block and no + // reason to know any of this. A user cannot be asked to write a flag whose + // necessity is a fact about their dependencies. + for (auto& f : graph_runtime_compile_flags(tc)) out.push_back(f); + const bool bypassCfg = dm.hasCfg && (opt.cfgBypass == HostFlagOptions::CfgBypass::Always || mcpp::platform::is_linux); @@ -133,7 +177,28 @@ std::vector host_compile_tokens(const Toolchain& tc, // returning early. const bool trustCfg = !bypassCfg && dm.hasCfg; - if (bypassCfg) { + // ⚠️ AND NOT WHEN THE TARGET SIDE COMES FROM THE GRAPH — the compile-side + // counterpart of the replacement `flags.cppm` makes on the link line. + // + // These tokens are the payload's: `-isystem /include/c++/v1` and + // the C library beside it. For an openkal target the C++ runtime and the C + // library are packages, and the payload's copies are built for the machine + // doing the building. + // + // ⚠️ `-nostdinc++` DOES NOT REMOVE THEM, which is what makes this its own + // fix rather than a flag. That option suppresses the DRIVER's own C++ + // search; a path put there explicitly with `-isystem` stays. Measured + // 2026-08-23, cross-compiling openkal-windows — a package that uses no C++ + // standard library at all — with `-nostdinc++` on the command line: + // + // winnt.h:16 → …/xim-x-llvm/…/include/c++/v1/ctype.h + // → __config:13 '__config_site' file not found + // + // mingw's own header asked for ``, and the payload's libc++ was + // still ahead of the sysroot that had just been pointed at the right place. + const bool graphSuppliesTarget = !tc.crossTargetFlag.empty(); + + if (bypassCfg && !graphSuppliesTarget) { for (auto& t : dm.compile_tokens(esc, opt.clangStdlibSelect)) out.push_back(t); } @@ -147,7 +212,8 @@ std::vector host_compile_tokens(const Toolchain& tc, if (mcpp::platform::is_macos && !opt.macosDeploymentTarget.empty()) out.push_back("-mmacosx-version-min=" + opt.macosDeploymentTarget); - if (!trustCfg && (bypassCfg || lm.mode != CLibMode::None)) + if (!trustCfg && !graphSuppliesTarget + && (bypassCfg || lm.mode != CLibMode::None)) for (auto& t : lm.compile_tokens(esc)) out.push_back(t); return out; diff --git a/src/toolchain/model.cppm b/src/toolchain/model.cppm index d9b0711f..d085f9b4 100644 --- a/src/toolchain/model.cppm +++ b/src/toolchain/model.cppm @@ -81,6 +81,64 @@ struct Toolchain { std::string stdlibVersion; std::filesystem::path stdModuleSource; // bits/std.cc / std.cppm std::filesystem::path stdCompatSource; // bits/std_compat.cc / std.compat.cppm + // Flags the std module source needs that the compiler cannot supply itself. + // + // Empty for every toolchain that ships its own standard library: there the + // module source is the compiler's, and the compiler finds its own headers. + // Non-empty when the source comes from a PACKAGE instead --- a standard + // library configured for a target the compiler knows nothing about --- and + // then the include path and the configuration are the package's, so they + // have to be carried here. + // + // They reach the cache key without anything further being done: the key is + // derived from the build COMMANDS, and these are part of them. + std::string stdModuleFlags; + // ⭐⭐ THE PART OF THE ABOVE THAT SAYS WHICH MACHINE, SEPARATED FROM THE + // PART THAT SAYS WHERE THE HEADERS ARE. + // + // `stdModuleFlags` is one string carrying two different facts: the target + // and its ABI-affecting options, and the include paths the module's SOURCE + // needs. Building the module has two steps, and only the first needs both — + // the second compiles a BMI, which already contains everything the headers + // contributed. + // + // ⚠️ Passing the whole string to the second step is not wrong, it is noisy, + // and the noise is the kind that hides things: + // + // clang++: warning: argument unused during compilation: '-nostdinc++' + // clang++: warning: argument unused during compilation: '-isystem …' + // (× 17, once per include directory) + // + // Seventeen warnings that are correct and mean nothing, in front of any + // warning that would mean something. ⚠️ They were present on every platform + // and visible on none: the non-Windows command ends in `2>&1` and mcpp + // discards a successful command's output, so the Windows leg — which has no + // redirection — is where they first appeared. + std::string stdModuleTargetFlags; + // A package in the graph supplies a C++ runtime built FOR THIS TARGET. + // Read by the freestanding flag table, which otherwise forces exceptions + // and run-time type information off for every unit — right when nothing can + // throw, and wrong when something can. + bool targetCxxRuntime = false; + + // ⭐⭐ THE `--target=` A RETARGETABLE DRIVER HAS TO BE GIVEN, OR EMPTY. + // + // Non-empty only when the user asked for a cross AND the resolved compiler + // is one binary that emits many targets (clang). For a native build, and + // for a cross served by a driver that has exactly one target of its own + // (`x86_64-w64-mingw32-g++`), this stays empty and nothing is added. + // + // ⚠️ IT CANNOT BE DERIVED FROM `targetTriple` BEING NON-EMPTY. A native + // build has a `targetTriple` too — the probed one — so a consumer that + // tested for non-empty would add `--target=` to every compile in + // every project. Measured: it does, and what it produces is not a + // diagnostic about targets but `/bin/sh: 1: Syntax error: word unexpected` + // out of the generated build file. + // + // So the fact is recorded where it is KNOWN — at target resolution, which + // is the only place that has both the request and the compiler — and read + // verbatim everywhere else. + std::string crossTargetFlag; std::filesystem::path sysroot; // -print-sysroot output (or empty) std::optional payloadPaths; // fine-grained sysroot from xpkgs // The TARGET's C library, for targets whose row in kKnownTargets names one @@ -162,6 +220,74 @@ bool is_musl_target(const Toolchain& tc); bool is_msvc_target(const Toolchain& tc); bool is_mingw_target(const Toolchain& tc); +// ⭐⭐ THE FLAGS A WHOLE GRAPH HAS TO AGREE ON WHEN THE RUNTIME COMES FROM IT. +// +// An ordinary flag is a package's business. These two are not: they change what +// a translation unit EMITS for constructs the language guarantees work across a +// program — a `throw` and a `thread_local`. Two objects that disagree link, and +// then the disagreement is the bug. +// +// Both become necessary from one fact, `Toolchain::targetCxxRuntime`: the C++ +// runtime, the unwinder and the C library are packages rather than the +// compiler's payload. The compiler's defaults for these are chosen for the +// platform's OWN runtime, which is exactly the thing that is not being used. +// +// -fdwarf-exceptions PE only. clang defaults to SEH there, whose personality +// (`__gxx_personality_seh0`) and `.pdata`/`.xdata` come +// from the operating system's unwinder. The graph brings +// libunwind, which reads `.eh_frame`. +// -femulated-tls PE and Mach-O. Both reach a `thread_local` through +// something the DYNAMIC LOADER bootstraps — `_tls_index` +// on PE, `_tlv_bootstrap` on Mach-O. A self-contained +// image has no loader to do it, so the access becomes an +// ordinary call into compiler-rt against a key the C +// library owns. +// -fvisibility=hidden Mach-O only. There, a symbol with DEFAULT visibility +// -fvisibility-inlines- and weak (linkonce_odr) linkage — which is what every +// hidden template instantiation and inline function is — is +// coalesced BY THE DYNAMIC LOADER, so the linker routes +// calls to it through a stub and a GOT slot the loader +// fills. That is how one definition wins across dylibs, +// and it is machinery a self-contained image has no use +// for. +// +// ⚠️⚠️ AND THE THIRD ONE WAS FOUND BY A PROGRAM THAT LINKED, WAS SIGNED, AND +// CRASHED ON THE REAL MACHINE — which is the whole argument for running the +// artefact rather than inspecting it. On an arm64 Mac: +// +// stop reason = EXC_BAD_ACCESS (code=1, address=0x0) +// frame #0: 0x0000000000000000 +// +// No output, no frames: the program jumped to address zero at its first +// indirect call. The image had 1238 `__stubs` entries and 1335 `__got` slots +// for THREE undefined symbols, and the stubs' names were its own — +// `std::vector::__init_with_size`, `operator new`, and a thousand more +// libc++ internals. `main`'s first statement constructs a `std::vector`. +// +// The package builds libc++ with `_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS`, +// which is correct for a static build and leaves every instantiation at default +// visibility. On ELF that is inert — a static link resolves weak definitions at +// LINK time and nothing survives to run time. On Mach-O it produces the +// coalescing machinery above. +// +// ⇒ Measured after adding the two flags: `__stubs` 0x3a08 → 0x6c, `__got` +// 0x29b8 → 0x58. Nine stubs and eleven slots, which is the size a program with +// three imports should have. +// +// ⚠️ ELF IS DELIBERATELY ABSENT FROM THE SECOND, and it is not an oversight: +// there a `thread_local` is a fixed offset from the thread pointer, which the C +// library establishes itself. Adding the flag would work and cost an indirection +// on every access — but it would also make ELF the only target whose thread +// locals are laid out differently from every OTHER build of the same target. +// +// ⚠️ AND THE REASON THIS IS A FUNCTION RATHER THAN TWO `if`s: the compile +// command is assembled in two places (`hostflags.cppm` for every ordinary unit, +// `prepare.cppm` for the `std` module), and a `std.pcm` built with SEH imported +// by units built with DWARF is a defect that neither file can see. "One fact, +// two channels" has produced an identical bug three times in this ecosystem; +// here the second channel is removed instead of being told to remember. +std::vector graph_runtime_compile_flags(const Toolchain& tc); + // Can the artifact we are building be fully statically linked (`-static`)? // // This is a property of the TARGET, not of the machine doing the build — @@ -279,6 +405,22 @@ bool is_mingw_target(const Toolchain& tc) { return tc.targetTriple.find("mingw32") != std::string::npos; } +std::vector graph_runtime_compile_flags(const Toolchain& tc) { + std::vector out; + if (!tc.targetCxxRuntime) return out; + auto t = triple::parse(tc.targetTriple); + if (!t) return out; + if (t->is_pe()) out.emplace_back("-fdwarf-exceptions"); + if (t->is_pe() || t->os == "macos") out.emplace_back("-femulated-tls"); + // ⭐⭐ MACH-O ONLY, AND THE REASON IS THAT WEAK-DEF IS A RUN-TIME MECHANISM + // THERE. See the note on this function for the measurement. + if (t->os == "macos") { + out.emplace_back("-fvisibility=hidden"); + out.emplace_back("-fvisibility-inlines-hidden"); + } + return out; +} + bool target_supports_full_static(std::string_view targetTriple, bool hostCapability) { // Empty triple means "build for this machine" — target IS host, so the // host answer is the correct one. This is the only case where the host diff --git a/src/toolchain/registry.cppm b/src/toolchain/registry.cppm index 23e2b67b..2843dc1c 100644 --- a/src/toolchain/registry.cppm +++ b/src/toolchain/registry.cppm @@ -28,13 +28,30 @@ import mcpp.toolchain.triple; export namespace mcpp::toolchain { -enum class Family { Gcc, Llvm, Msvc }; +// `OpenkalLlvm` IS A SPELLING, NOT A COMPILER, AND NO LONGER A DECISION. +// +// It named the same llvm payload as `Llvm` and existed to carry one fact: that +// a project's headers, C library, C++ runtime and platform implementation come +// from packages rather than from a payload beside the compiler. A toolchain +// family was the wrong object to carry it. That fact belongs to the dependency +// graph, is only knowable after the graph is resolved, and says nothing about +// which compiler is running — the same packages compiled by gcc are the +// intended second consumer, and expressing them through a family name would +// have required a second name for the same fact. +// +// `mcpp.targetside` resolves it per layer, from what packages declare, at the +// point where the graph exists. This member survives so that a manifest +// written against the older spelling still resolves; it behaves in every +// respect as `Llvm`, and nothing branches on it. +enum class Family { Gcc, Llvm, Msvc, OpenkalLlvm }; + inline std::string_view family_name(Family f) { switch (f) { case Family::Gcc: return "gcc"; case Family::Llvm: return "llvm"; case Family::Msvc: return "msvc"; + case Family::OpenkalLlvm: return "openkal-llvm"; } return "?"; } @@ -294,6 +311,7 @@ parse_toolchain_spec(std::string compilerArg, ToolchainSpec spec; if (norm->family == "llvm") spec.family = Family::Llvm; else if (norm->family == "msvc") spec.family = Family::Msvc; + else if (norm->family == "openkal-llvm") spec.family = Family::OpenkalLlvm; else spec.family = Family::Gcc; spec.version = std::move(norm->version); spec.target = std::move(norm->target); @@ -390,7 +408,11 @@ XimToolchainPackage to_xim_package(const ToolchainSpec& spec) { pkg.frontendCandidates = {"cl.exe"}; return pkg; } - if (spec.family == Family::Llvm) { + if (spec.family == Family::Llvm || spec.family == Family::OpenkalLlvm) { + // ⭐ THE SAME PAYLOAD. `openkal-llvm` downloads nothing of its own and + // installs nothing of its own — it is a statement about where the + // TARGET SIDE comes from, and the compiler is the llvm payload either + // way. A user who has one has both. pkg.ximName = mcpp::toolchain::llvm::package_name(); pkg.frontendCandidates = mcpp::toolchain::llvm::frontend_candidates(); return pkg; @@ -494,6 +516,8 @@ std::filesystem::path payload_frontend(const std::filesystem::path& payloadRoot, if (family == Family::Msvc) { // Same resolution the install and build paths use, so the three // cannot disagree about where an msvc payload keeps its compiler. + // (OpenkalLlvm falls through to the generic bin/-shaped resolution + // below, which is the llvm payload's shape.) if (auto inst = mcpp::toolchain::msvc::installation_at(payloadRoot, pkg.ximVersion)) return inst->clPath; @@ -601,6 +625,10 @@ std::vector available_toolchain_indexes() { { "gcc", Family::Gcc }, { "musl-gcc", Family::Gcc }, { mcpp::toolchain::llvm::package_name(), Family::Llvm }, + // The same package, listed a second time under the name that says what + // its targets are. Installing either installs both, which is accurate: + // there is one payload and two ways of asking it a question. + { mcpp::toolchain::llvm::package_name(), Family::OpenkalLlvm }, }; // The Windows-PE gcc payload is host-split at the distribution layer // (§4.3); each host lists the package it would actually install. diff --git a/src/toolchain/triple.cppm b/src/toolchain/triple.cppm index c159d1db..54a8def3 100644 --- a/src/toolchain/triple.cppm +++ b/src/toolchain/triple.cppm @@ -44,6 +44,57 @@ struct Triple { return s; } + // ⭐⭐ THE SPELLING A COMPILER TAKES, WHICH IS NOT THE SPELLING mcpp USES. + // + // `str()` is mcpp's vocabulary: short, unambiguous, and the thing a user + // types. LLVM's is a four-field form with a vendor, and on Apple platforms + // the architecture has a different name and the OS carries a version. + // + // ⚠️ THIS EXISTS BECAUSE CROSS-COMPILING USED TO MEAN SOMETHING NARROWER. + // Every hosted cross mcpp could do was served by a payload whose DRIVER was + // already specialised — `x86_64-w64-mingw32-g++` needs no `--target`, + // because it has only one. So nothing ever needed this function, and + // nothing emitted `--target=` outside the freestanding path. + // + // openkal changes the shape of the question. The target side — headers, + // C library, C++ runtime, the OS's own openkal implementation — is a set of + // PACKAGES in the dependency graph, built from source by whichever compiler + // is running. What remains for the compiler is code generation, and clang + // emits every format it was built with from one binary. There is no payload + // to specialise, so the triple has to be said out loud. + // + // ⚠️ Measured 2026-08-23, before this existed: a build for `aarch64-macos` + // resolved the whole graph, took musl's aarch64 headers, and compiled with + // NO `--target` at all — so the host's x86_64 code generation met aarch64 + // declarations. What caught it was the port's own assertion, which exists + // for exactly this: + // + // okm_float_assert.c: the C library and the compiler disagree about + // LDBL_DIG ('33 == 18') + // + // 33 is aarch64's binary128; 18 is x87. Two machines in one command line. + std::string llvm_triple(std::string_view macosVersion = {}) const { + if (empty()) return {}; + if (os == "macos") { + // Apple spells the 64-bit ARM architecture `arm64`, and the OS + // component carries the deployment target: `arm64-apple-macos14`. + // Without a version clang picks its own default, which is a + // decision belonging to the project rather than to the compiler. + const std::string a = (arch == "aarch64") ? "arm64" : arch; + std::string t = a + "-apple-macos"; + t += macosVersion.empty() ? std::string("14.0") + : std::string(macosVersion); + return t; + } + if (os == "windows") { + if (is_msvc_env()) return arch + "-pc-windows-msvc"; + return arch + "-w64-windows-gnu"; + } + if (os == "linux") return arch + "-unknown-linux-" + (env.empty() ? "gnu" : env); + if (os == "none") return str(); // freestanding: already LLVM's form + return str(); + } + bool is_musl() const { return env == "musl"; } bool is_msvc_env() const { return env == "msvc"; } bool is_windows_gnu() const { return os == "windows" && env == "gnu"; } diff --git a/tests/e2e/267_target_requested_is_empty_for_a_native_build.sh b/tests/e2e/267_target_requested_is_empty_for_a_native_build.sh new file mode 100755 index 00000000..8189e65f --- /dev/null +++ b/tests/e2e/267_target_requested_is_empty_for_a_native_build.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# requires: unix-shell +# `MCPP_TARGET_REQUESTED` — the value a build program needs and `MCPP_TARGET` +# cannot give it. +# +# ⭐⭐ WHY THE TWO VARIABLES ARE NOT THE SAME QUESTION. +# +# `MCPP_TARGET` answers "which machine is this for", and it is filled in with +# the host when nobody named a target — which is right for that question and +# makes it useless for a different one a platform package has to ask: **was this +# build POINTED at a target**, or is it an ordinary native build? +# +# The two differ even when the triples are equal. `mcpp build --target +# aarch64-macos` on an arm64 Mac names the very machine the host is, and yet it +# is the dependency graph that supplies the target side — so mcpp puts no system +# SDK on the link, and the package that knows the system is the only thing that +# can name one. A native build on the same machine gets the SDK and needs +# nothing from the package. +# +# ⚠️ Both readings of that question have already been measured wrong in +# `openkal-macos`: +# +# from the host → right for the cross, and `library not found for -lSystem` +# for `--target aarch64-macos` ON a Mac +# from MCPP_TARGET → right for the cross, and `undefined symbol: wcslen` for +# the native build, because it is never empty and the +# package's three-name stub shadowed the vendor's complete one +# +# ⇒ The assertions below are the contract those two attempts needed: EMPTY for a +# native build, and the requested triple otherwise. +# +# ⚠️ `"$MCPP"`, never a bare `mcpp`: the harness passes the binary under test, +# and a bare name resolves through PATH to whichever engine is installed. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p src +cat > mcpp.toml <<'EOF' +[package] +name = "target-requested-probe" +version = "0.1.0" +EOF +printf 'int main() { return 0; }\n' > src/main.cpp + +# ⚠️ The build program reports through a NON-ZERO exit, because mcpp prints what +# it captured only from a program that failed. A probe that returned zero would +# have its output discarded and this test would assert nothing. +cat > build.mcpp <<'EOF' +import mcpp; +import std; +int main() { + const char* t = std::getenv("MCPP_TARGET"); + const char* r = std::getenv("MCPP_TARGET_REQUESTED"); + std::cerr << "PROBE target=[" << (t ? t : "") << "] " + << "requested=[" << (r ? r : "") << "]\n"; + return 1; +} +EOF + +# ── A native build: nobody named a target ────────────────────────────────── +native=$("$MCPP" build 2>&1 || true) +echo "$native" | grep -q 'PROBE ' || { + echo "the build program did not report; mcpp prints a failing program's output" >&2 + echo "$native" >&2 + exit 1 +} +echo "$native" | grep -qE 'requested=\[\]' || { + echo "MCPP_TARGET_REQUESTED should be empty for a native build" >&2 + echo "$native" | grep 'PROBE ' >&2 + exit 1 +} +# ⚠️ And the positive control on the same line: `MCPP_TARGET` must be filled in. +# Without this the assertion above would also pass if mcpp had stopped setting +# any of them. +echo "$native" | grep -qE 'target=\[[a-z0-9_]+-[a-z0-9-]+\]' || { + echo "MCPP_TARGET should carry the host triple for a native build" >&2 + echo "$native" | grep 'PROBE ' >&2 + exit 1 +} + +# ── A build pointed at a target ──────────────────────────────────────────── +# +# ⚠️ THE HOST'S OWN TRIPLE, NAMED EXPLICITLY — which is the case that +# distinguishes the two variables rather than merely one that differs from the +# native run. `--target ` produces equal values for `MCPP_TARGET` and the +# host triple, and `MCPP_TARGET_REQUESTED` is non-empty because a target was +# named. A test that used a foreign triple would pass with a variable that +# merely echoed `MCPP_TARGET`. +# +# ⚠️ And it needs no payload. The first version of this test named +# `x86_64-linux-musl` on the grounds that every host could resolve it, which was +# an assumption rather than a measurement: +# +# error: target 'x86_64-linux-musl' cannot be built on this host — +# no toolchain payload exists that runs here and produces it +# +# on the macOS leg. The host's own triple is the one target every host has by +# construction. +host_triple=$(echo "$native" | sed -n 's/.*PROBE target=\[\([^]]*\)\].*/\1/p' | head -1) +[ -n "$host_triple" ] || { echo "could not read the host triple from the probe" >&2; exit 1; } +cross=$("$MCPP" build --target "$host_triple" 2>&1 || true) +echo "$cross" | grep -q 'PROBE ' || { + echo "the build program did not report on the cross build" >&2 + echo "$cross" >&2 + exit 1 +} +echo "$cross" | grep -qE "requested=\[$host_triple\]" || { + echo "MCPP_TARGET_REQUESTED should carry the triple that was named" >&2 + echo "$cross" | grep 'PROBE ' >&2 + exit 1 +} + +echo "MCPP_TARGET_REQUESTED distinguishes a native build from a named target" diff --git a/tests/e2e/268_target_side_resolution.sh b/tests/e2e/268_target_side_resolution.sh new file mode 100755 index 00000000..a537b30c --- /dev/null +++ b/tests/e2e/268_target_side_resolution.sh @@ -0,0 +1,164 @@ +#!/usr/bin/env bash +# requires: unix-shell +# Target-side resolution: which layer of a target comes from where. +# +# WHAT THIS COVERS THAT THE UNIT TESTS CANNOT. +# +# `mcpp.targetside` is a pure function and its table is asserted directly in +# tests/unit/test_targetside.cpp. What only a build can show is the wiring: that +# a package's `provides` line actually reaches the resolver, that the resolution +# reaches the report, and that a misspelling in mcpp's reserved namespace stops +# the build instead of quietly disabling the behaviour it was meant to select. +# +# The packages here are local and trivial on purpose. The real ecosystem +# exercise lives in .github/workflows/openkal-cross.yml, which builds nine +# artifacts on three hosts; what this file needs is the mechanism, not the +# stack, and a test that fetched an ecosystem over the network to assert a +# string in a report would be slower and no more conclusive. +# +# ⚠️ NO `sed -i` ANYWHERE, AND THAT IS NOT STYLE. BSD sed reads the argument +# after `-i` as a backup suffix, so an in-place edit written for GNU sed fails +# on macOS. Measured on the macOS leg of this suite, in this very file: +# +# sed: 1: "sys/mcpp.toml": unterminated substitute pattern +# +# The provider's manifest is therefore written from scratch for each case. +# +# `"$MCPP"`, never a bare `mcpp`: the harness passes the binary under test, and +# a bare name resolves through PATH to whichever engine is installed. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p sys/src +printf 'int fake_system_marker(void) { return 0; }\n' > sys/src/sys.c + +# Rewrite the provider's manifest with the capabilities given as arguments, and +# drop the output tree so the next build re-plans from it. +provider_declares() { + { + echo '[package]' + echo 'name = "fake-system"' + echo 'version = "2.0.0"' + if [ $# -gt 0 ]; then + printf 'provides = [' + sep="" + for cap in "$@"; do printf '%s"%s"' "$sep" "$cap"; sep=", "; done + printf ']\n' + fi + echo '' + echo '[targets.fake-system]' + echo 'kind = "lib"' + echo 'sources = ["src/*.c"]' + } > sys/mcpp.toml + rm -rf target +} + +mkdir -p src +cat > mcpp.toml <<'EOF' +[package] +name = "target-side-probe" +version = "0.1.0" + +[dependencies] +fake-system = { path = "sys" } +EOF +printf 'int main() { return 0; }\n' > src/main.cpp + +# ── A package that supplies two layers ────────────────────────────────────── +# +# One package standing in for the platform implementation and the C library +# both. The resolver reads a layer per capability, not a layer per package, so +# a single provider of two layers is a legitimate shape and a useful one to +# assert: it proves the two lookups are independent. +# +# ⚠️ `|| true`, AND THAT IS THE TEST'S SUBJECT RATHER THAN A CONCESSION. The +# resolution is reported during planning, before a single object is compiled, so +# what this file asserts is complete whether or not the link afterwards +# succeeds. Requiring a successful link would additionally require a working C +# runtime payload for the host, which is a different thing to test and one the +# rest of the suite already covers. +provider_declares "mcpp:kernel-abi=fakeos" "mcpp:c-abi=fakelibc" +out=$("$MCPP" build 2>&1 || true) + +echo "$out" | grep -q 'kernel-abi *fakeos' || { + echo "the kernel-abi layer must report the interface the package declared" >&2 + echo "$out" >&2; exit 1 +} +echo "$out" | grep -q 'c-abi *fakelibc' || { + echo "the c-abi layer must report the interface the package declared" >&2 + echo "$out" >&2; exit 1 +} +echo "$out" | grep -qE 'fake-system@2\.0\.0' || { + echo "the report must name the providing package and its version" >&2 + echo "$out" >&2; exit 1 +} +echo "$out" | grep -q 'graph' || { + echo "a layer supplied by a dependency must be reported as coming from the graph" >&2 + echo "$out" >&2; exit 1 +} + +# ⚠️ AND THE POSITIVE CONTROL ON THE SAME OUTPUT. Without this, the assertions +# above would also pass on an engine that printed the same three lines for +# every build regardless of what the graph contained. +echo "$out" | grep -q 'c++ *—' || { + echo "a project with no C++ runtime package must report that layer absent" >&2 + echo "$out" >&2; exit 1 +} + +# ── The same project without the declarations ─────────────────────────────── +# +# The control that makes the block above mean something: remove the capability +# line and every layer must fall back to the payload. +provider_declares +plain=$("$MCPP" build 2>&1 || true) +echo "$plain" | grep -q 'fakeos' && { + echo "a package that declares no capability must not fill a layer" >&2 + echo "$plain" >&2; exit 1 +} +echo "$plain" | grep -qE 'kernel-abi .*payload' || { + echo "with nothing in the graph, the layers come from the payload" >&2 + echo "$plain" >&2; exit 1 +} + +# ── A misspelling inside mcpp's reserved namespace is an error ────────────── +# +# This is the whole reason the prefix exists. An unvalidated capability array +# turns one wrong letter into a behaviour that silently does not happen, and +# the build still reports success. +provider_declares "mcpp:kernel_abi=fakeos" "mcpp:c-abi=fakelibc" +bad=$("$MCPP" build 2>&1 || true) +echo "$bad" | grep -q "names no capability mcpp knows" || { + echo "a misspelled capability in the mcpp: namespace must fail the build" >&2 + echo "$bad" >&2; exit 1 +} +echo "$bad" | grep -q 'mcpp:kernel-abi' || { + echo "the diagnostic must list the layer names that do exist" >&2 + echo "$bad" >&2; exit 1 +} +# And it must fail BEFORE anything is compiled: a manifest this engine cannot +# read is not a build that got far enough to have a link. +echo "$bad" | grep -q 'Compiling' && { + echo "the manifest must be rejected before compilation begins" >&2 + echo "$bad" >&2; exit 1 +} + +# ── A name outside the namespace is none of mcpp's business ──────────────── +# +# `provides` also carries capabilities packages match among themselves. Closing +# the whole array would reject those, and the freestanding allocator selection +# that already ships is one. +provider_declares "a-capability-mcpp-never-heard-of" "mcpp:c-abi=fakelibc" +free=$("$MCPP" build 2>&1 || true) +echo "$free" | grep -q "names no capability mcpp knows" && { + echo "a capability outside the mcpp: namespace must pass through untouched" >&2 + echo "$free" >&2; cat sys/mcpp.toml >&2; exit 1 +} +echo "$free" | grep -q 'c-abi *fakelibc' || { + echo "the layer beside the unknown name must still resolve" >&2 + echo "$free" >&2; exit 1 +} + +echo "target-side resolution reads the graph, reports it, and validates its own namespace" diff --git a/tests/e2e/269_openkal_llvm_spelling_still_resolves.sh b/tests/e2e/269_openkal_llvm_spelling_still_resolves.sh new file mode 100755 index 00000000..313947a8 --- /dev/null +++ b/tests/e2e/269_openkal_llvm_spelling_still_resolves.sh @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# requires: import-std-libcxx +# The older toolchain spelling keeps working, and keeps meaning the same thing. +# +# WHY THIS FILE EXISTS. +# +# `openkal-llvm` was a toolchain family that named the same payload as `llvm` +# and existed to carry one fact: that a project's C library, C++ runtime and +# platform implementation come from packages rather than from a payload beside +# the compiler. That fact now belongs to `mcpp.targetside`, is read from what +# packages declare, and is resolved after the dependency graph exists — so the +# family name carries nothing and nothing branches on it. +# +# What must not happen is that a manifest written against the older spelling +# stops building. There is no deprecation deadline here on purpose: the spelling +# costs one row in a name table, and an engine that refuses a manifest it used to +# accept has broken a project that did nothing wrong. +# +# The assertion is about the RESOLVED DRIVER rather than about a successful +# build, because the two spellings are supposed to be indistinguishable at that +# point and a build additionally depends on what the project contains. +# +# `"$MCPP"`, never a bare `mcpp`: the harness passes the binary under test. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +mkdir -p src +printf 'int main() { return 0; }\n' > src/main.cpp + +manifest() { + cat > mcpp.toml <&1 | sed -n 's/.*Resolved [^ ]* → \(.*\)$/\1/p' | head -1 +} + +new_spelling=$(driver_for "llvm@22.1.8") +old_spelling=$(driver_for "openkal-llvm@22.1.8") + +[ -n "$new_spelling" ] || { + echo "could not read the resolved driver for the current spelling" >&2 + manifest "llvm@22.1.8"; "$MCPP" build 2>&1 | head -20 >&2 + exit 1 +} + +[ "$new_spelling" = "$old_spelling" ] || { + echo "the two spellings must resolve to the same driver" >&2 + echo " llvm@22.1.8 → $new_spelling" >&2 + echo " openkal-llvm@22.1.8 → $old_spelling" >&2 + exit 1 +} + +# ⚠️ AND THE OLDER SPELLING MUST NOT STILL DECIDE ANYTHING. Both manifests here +# have an empty dependency graph, so both must report a target side supplied +# entirely by the payload. If the family name still carried the fact it used to, +# the second would report `graph` somewhere and the first would not. +manifest "openkal-llvm@22.1.8" +rm -rf target +old_report=$("$MCPP" build 2>&1 | grep -E 'kernel-abi|c-abi' || true) +echo "$old_report" | grep -q 'graph' && { + echo "the toolchain family name must no longer decide where the target side comes from" >&2 + echo "$old_report" >&2 + exit 1 +} +echo "$old_report" | grep -q 'payload' || { + echo "with an empty graph every layer comes from the payload" >&2 + echo "$old_report" >&2 + exit 1 +} + +echo "the openkal-llvm spelling resolves as llvm and decides nothing" diff --git a/tests/unit/test_distribution.cpp b/tests/unit/test_distribution.cpp index 2953f406..bae075a9 100644 --- a/tests/unit/test_distribution.cpp +++ b/tests/unit/test_distribution.cpp @@ -557,3 +557,116 @@ TEST(Distribution, RoleCountCoversEveryRole) { EXPECT_FALSE(dist::to_string(r).empty()); } } + +// ── The two short-circuits: when there is no C++ runtime to distribute WITH ── +// +// Every cell of the table above answers "how does this artefact carry its C++ +// runtime", and all of the answers name a runtime to LINK — the system's, the +// toolchain's, or a static form of one. Two situations make all of them wrong +// rather than merely unnecessary, and in both the archives the table would +// reach for belong to the HOST. + +TEST(Distribution, FreestandingCarriesNoRuntimeToDistribute) { + dist::MechanismInput in; + in.format = dist::Format::Elf; + in.stdlibId = "libc++"; + in.freestanding = true; + in.requested = dist::Contract::SelfContained; + in.role = dist::Role::Distributable; + // Deliberately present: the point is that they are NOT reached. + in.libcxxArchive = "/tc/lib/libc++.a"; + in.libcxxAbiArchive = "/tc/lib/libc++abi.a"; + auto m = dist::resolve(in); + EXPECT_EQ(m.effective, dist::Contract::SelfContained); + EXPECT_EQ(m.unitFlags, " -nostdlib++"); + EXPECT_EQ(m.unitFlags.find("libc++.a"), std::string::npos); + EXPECT_FALSE(m.degraded); +} + +// ⚠️ The hosted form of the same fact. A package in the graph has compiled a +// C++ runtime FOR THIS TARGET and its objects are already on the link line, so +// there is no library to name and nothing to look for. Measured before this +// existed: `ld64.lld: error: library not found for -lc++`. +TEST(Distribution, GraphSuppliedRuntimeCarriesNoLibraryToName) { + auto in = macos_input(); + in.graphCxxRuntime = true; + in.requested = dist::Contract::SelfContained; + in.role = dist::Role::Distributable; + auto m = dist::resolve(in); + EXPECT_EQ(m.effective, dist::Contract::SelfContained); + EXPECT_EQ(m.unitFlags, " -nostdlib++"); + EXPECT_EQ(m.unitFlags.find("-lc++"), std::string::npos); + EXPECT_EQ(m.unitFlags.find("load_hidden"), std::string::npos); +} + +// The same on every format, because the fact is about the graph and not about +// the object format. +TEST(Distribution, GraphSuppliedRuntimeIsFormatIndependent) { + for (auto fmt : {dist::Format::Elf, dist::Format::MachO, dist::Format::Pe}) { + dist::MechanismInput in; + in.format = fmt; + in.stdlibId = "libc++"; + in.graphCxxRuntime = true; + in.requested = dist::Contract::SelfContained; + in.role = dist::Role::Distributable; + auto m = dist::resolve(in); + EXPECT_EQ(m.unitFlags, " -nostdlib++") ; + EXPECT_EQ(m.effective, dist::Contract::SelfContained); + } +} + +// ⚠️ And it does not depend on the contract the project asked for: a +// host-coupled request cannot be honoured by naming the system's runtime when +// the graph's is already inside the artefact. +TEST(Distribution, GraphSuppliedRuntimeIgnoresTheRequestedContract) { + for (auto c : {dist::Contract::SelfContained, + dist::Contract::ToolchainCoupled, + dist::Contract::HostCoupled}) { + auto in = macos_input(); + in.graphCxxRuntime = true; + in.requested = c; + in.role = dist::Role::Distributable; + auto m = dist::resolve(in); + EXPECT_EQ(m.unitFlags, " -nostdlib++"); + } +} + +// ── format_for: which format a target produces ────────────────────────────── +// +// ⚠️ This was a lambda inside a fifteen-hundred-line function and therefore had +// no test, and what it got wrong was found by running three hosts against three +// targets. The assertions below are the ones that would have found it in a +// second: the canonical spellings mcpp itself uses contain neither `apple` nor +// `darwin`, and the answer must not depend on the fallback. + +TEST(Distribution, FormatIsTakenFromTheTargetAndNotTheFallback) { + // Every fallback, so that a target the vocabulary knows can never be + // decided by the machine doing the building. + for (auto fb : {dist::Format::Elf, dist::Format::MachO, dist::Format::Pe}) { + EXPECT_EQ(dist::format_for("aarch64-macos", fb), dist::Format::MachO); + EXPECT_EQ(dist::format_for("x86_64-macos", fb), dist::Format::MachO); + EXPECT_EQ(dist::format_for("x86_64-windows-gnu", fb), dist::Format::Pe); + EXPECT_EQ(dist::format_for("x86_64-linux-gnu", fb), dist::Format::Elf); + EXPECT_EQ(dist::format_for("aarch64-linux-musl", fb), dist::Format::Elf); + EXPECT_EQ(dist::format_for("riscv64-none-elf", fb), dist::Format::Elf); + } +} + +// The `[target.X]` escape hatch: a spelling the vocabulary cannot parse is all +// there is to go on, so LLVM's words are recognised there. +TEST(Distribution, FormatFallsBackToSpellingForAnUnparseableTriple) { + EXPECT_EQ(dist::format_for("arm64-apple-macos14.0", dist::Format::Elf), + dist::Format::MachO); + EXPECT_EQ(dist::format_for("x86_64-w64-mingw32", dist::Format::Elf), + dist::Format::Pe); + EXPECT_EQ(dist::format_for("x86_64-unknown-darwin", dist::Format::Elf), + dist::Format::MachO); +} + +// ⚠️ And only then the host. A triple that says nothing at all is the one case +// where the machine doing the building is the best available answer. +TEST(Distribution, FormatUsesTheFallbackOnlyWhenTheTripleSaysNothing) { + EXPECT_EQ(dist::format_for("", dist::Format::MachO), dist::Format::MachO); + EXPECT_EQ(dist::format_for("", dist::Format::Pe), dist::Format::Pe); + EXPECT_EQ(dist::format_for("nonsense", dist::Format::Elf), dist::Format::Elf); +} diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp index e42a34e2..e5ac8d54 100644 --- a/tests/unit/test_hostflags.cpp +++ b/tests/unit/test_hostflags.cpp @@ -216,3 +216,82 @@ TEST(HostFlags, DeploymentTargetOnlyOnMacos) { }); EXPECT_EQ(found, mcpp::platform::is_macos); } + +// ── graph_runtime_compile_flags: what a `throw` and a `thread_local` compile +// into, when the runtime comes from the dependency graph. +// +// These are not ordinary flags. They change what a translation unit EMITS for +// constructs the language guarantees work across a whole program, so two +// objects that disagree link and the disagreement is the defect. The function +// exists so that the decision is made once; these tests exist so that each of +// its four states is stated rather than inferred from a build. + +namespace { + +mcpp::toolchain::Toolchain graph_tc(std::string triple) { + mcpp::toolchain::Toolchain tc; + tc.compiler = CompilerId::Clang; + tc.targetTriple = std::move(triple); + tc.targetCxxRuntime = true; + return tc; +} + +bool has(const std::vector& v, std::string_view f) { + return std::ranges::find(v, f) != v.end(); +} + +} // namespace + +// PE: clang defaults to SEH there, whose personality routine and unwind data +// come from the operating system's unwinder. A graph that supplies its own C++ +// runtime supplies its own unwinder with it, and the two cannot be mixed inside +// one image. A `thread_local` on PE is reached through `_tls_index`, which the +// dynamic loader bootstraps and a self-contained image has no loader for. +TEST(GraphRuntimeFlags, PeTakesDwarfExceptionsAndEmulatedTls) { + auto f = mcpp::toolchain::graph_runtime_compile_flags(graph_tc("x86_64-windows-gnu")); + EXPECT_TRUE(has(f, "-fdwarf-exceptions")); + EXPECT_TRUE(has(f, "-femulated-tls")); +} + +// Mach-O: the exception mechanism is already DWARF, so only the thread-local +// one applies — `_tlv_bootstrap` is the loader-bootstrapped name there. The +// visibility pair is present because on this format a default-visibility weak +// definition is coalesced BY THE LOADER, which is machinery a self-contained +// image has no use for and which produced a jump to address zero when it was +// left in place. +TEST(GraphRuntimeFlags, MachOTakesEmulatedTlsAndHiddenVisibilityButNotDwarf) { + auto f = mcpp::toolchain::graph_runtime_compile_flags(graph_tc("aarch64-macos")); + EXPECT_FALSE(has(f, "-fdwarf-exceptions")); + EXPECT_TRUE(has(f, "-femulated-tls")); + EXPECT_TRUE(has(f, "-fvisibility=hidden")); + EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); +} + +// ⚠️ ELF takes NONE of them, and that is a decision rather than an omission. +// There a `thread_local` is a fixed offset from the thread pointer, which the +// C library establishes itself; adding the flag would work, cost an +// indirection on every access, and make ELF the only target whose thread +// locals are laid out differently from every other build of the same target. +TEST(GraphRuntimeFlags, ElfTakesNone) { + auto f = mcpp::toolchain::graph_runtime_compile_flags(graph_tc("x86_64-linux-gnu")); + EXPECT_TRUE(f.empty()); +} + +// ⚠️ And nothing at all when the runtime is NOT the graph's, whatever the +// target. The predicate is `targetCxxRuntime`; a native or payload-served build +// of the same triple must be untouched. +TEST(GraphRuntimeFlags, PayloadServedTargetTakesNoneEvenOnPe) { + mcpp::toolchain::Toolchain tc; + tc.compiler = CompilerId::Clang; + tc.targetTriple = "x86_64-windows-gnu"; + tc.targetCxxRuntime = false; + EXPECT_TRUE(mcpp::toolchain::graph_runtime_compile_flags(tc).empty()); +} + +// A triple outside the vocabulary yields nothing rather than a guess: the +// answer depends on the object format, and a spelling that cannot be parsed +// does not name one. +TEST(GraphRuntimeFlags, UnparseableTripleTakesNone) { + EXPECT_TRUE(mcpp::toolchain::graph_runtime_compile_flags( + graph_tc("not-a-triple-at-all")).empty()); +} diff --git a/tests/unit/test_targetside.cpp b/tests/unit/test_targetside.cpp new file mode 100644 index 00000000..52fa7d4c --- /dev/null +++ b/tests/unit/test_targetside.cpp @@ -0,0 +1,272 @@ +// Target-side resolution: which layer comes from where. +// +// These tests exist because the decision they cover used to be three +// independent derivations with three different criteria, and the bug was that +// they disagreed on a case none of them had been written for: a C program whose +// system comes from the dependency graph. So the assertions below are about the +// TABLE's totality — every combination the ecosystem can produce gets a row, +// including the ones that were unrepresentable before. +// +// The module under test is a pure function of plain data on purpose. The +// capability it replaces (`hosted-standard-library`) drove seven behaviours +// from inside a 7000-line translation unit and had zero test coverage, because +// asserting it required running a whole build. Nothing here runs a build. + +#include + +import std; +import mcpp.targetside; + +namespace ts = mcpp::targetside; + +namespace { + +// The traditional stack: a payload supplies every layer. +ts::Inputs payload_linux() { + ts::Inputs in; + in.llvmTriple = "x86_64-unknown-linux-gnu"; + in.targetOs = "linux"; + in.targetEnv = "gnu"; + in.payloadSystemRef = "xim-x-linux-headers@5.11.1"; + in.payloadLibcRef = "xim-x-glibc@2.44"; + in.payloadCxxRef = "xim-x-llvm@22.1.8"; + in.payloadCxxInterface = "libc++"; + return in; +} + +ts::Provider provider(std::string name, std::string version, + std::string iface, bool stdModule = false) { + ts::Provider p; + p.name = std::move(name); + p.version = std::move(version); + p.interfaceName = std::move(iface); + p.hasStdModule = stdModule; + return p; +} + +} // namespace + +// ── The capability grammar ─────────────────────────────────────────────────── + +TEST(TargetSideCapability, ThreeLayerNamesAreAccepted) { + for (auto [entry, layer] : std::initializer_list< + std::pair>{ + {"mcpp:kernel-abi", ts::CapLayer::KernelAbi}, + {"mcpp:c-abi", ts::CapLayer::CAbi}, + {"mcpp:c++-abi", ts::CapLayer::CxxAbi}}) { + auto r = ts::parse_capability(entry); + ASSERT_TRUE(r.has_value()) << entry; + ASSERT_TRUE(r->has_value()) << entry; + EXPECT_EQ((*r)->layer, layer) << entry; + EXPECT_TRUE((*r)->interfaceName.empty()) << entry; + } +} + +TEST(TargetSideCapability, InterfaceNameIsCarriedByTheDeclaration) { + auto r = ts::parse_capability("mcpp:kernel-abi=openkal"); + ASSERT_TRUE(r.has_value()); + ASSERT_TRUE(r->has_value()); + EXPECT_EQ((*r)->layer, ts::CapLayer::KernelAbi); + EXPECT_EQ((*r)->interfaceName, "openkal"); +} + +// The engine must not reject names it does not own. `provides` serves the +// feature system too, and a closed set over the whole array would break the +// allocator selection that already ships. +TEST(TargetSideCapability, NamesOutsideTheReservedPrefixPassThrough) { + for (std::string_view entry : {"freestanding-allocator", + "hosted-standard-library", + "anything-a-package-invents"}) { + auto r = ts::parse_capability(entry); + ASSERT_TRUE(r.has_value()) << entry; + EXPECT_FALSE(r->has_value()) << entry; + } +} + +// And the point of having a prefix at all: inside it, a typo is an error rather +// than a silently disabled behaviour. +TEST(TargetSideCapability, UnknownNameInsideTheReservedPrefixIsAnError) { + for (std::string_view entry : {"mcpp:kernel_abi", // underscore, not hyphen + "mcpp:cabi", + "mcpp:c++abi", + "mcpp:target-system"}) { + auto r = ts::parse_capability(entry); + EXPECT_FALSE(r.has_value()) << entry; + if (!r.has_value()) + EXPECT_NE(r.error().find("mcpp:kernel-abi"), std::string::npos) + << "the diagnostic must list the layers that do exist"; + } +} + +// ── The resolution table ───────────────────────────────────────────────────── + +TEST(TargetSideResolve, TraditionalStackTakesEveryLayerFromThePayload) { + auto r = ts::resolve(payload_linux()); + EXPECT_EQ(r.kernelAbi.origin, ts::Origin::Payload); + EXPECT_EQ(r.cAbi.origin, ts::Origin::Payload); + EXPECT_EQ(r.cxx.origin, ts::Origin::Payload); + EXPECT_EQ(r.kernelAbi.interfaceName, "linux"); + EXPECT_EQ(r.cAbi.interfaceName, "gnu"); + EXPECT_EQ(r.cxx.interfaceName, "libc++"); + EXPECT_FALSE(r.system_from_graph()); +} + +TEST(TargetSideResolve, OpenkalCxxTakesEveryLayerFromTheGraph) { + auto in = payload_linux(); + in.llvmTriple = "arm64-apple-macos14.0"; + in.targetOs = "macos"; + in.kernelAbi = provider("openkal-macos", "0.3.1", "openkal"); + in.cAbi = provider("openkal-musl", "0.3.1", "musl"); + in.cxxAbi = provider("openkal-llvm-runtime", "0.1.0", "libc++", /*stdModule=*/true); + + auto r = ts::resolve(in); + EXPECT_EQ(r.kernelAbi.origin, ts::Origin::Graph); + EXPECT_EQ(r.cAbi.origin, ts::Origin::Graph); + EXPECT_EQ(r.cxx.origin, ts::Origin::Graph); + EXPECT_EQ(r.kernelAbi.impl, "openkal-macos@0.3.1"); + EXPECT_EQ(r.cAbi.impl, "openkal-musl@0.3.1"); + EXPECT_FALSE(r.cxx.subset) << "a package that declares a std module supplies the whole library"; + EXPECT_TRUE(r.system_from_graph()); +} + +// THE CASE THAT WAS UNREPRESENTABLE, AND THE MEASURED DEFECT. +// +// A C program over openkal has a kernel ABI and a C library from the graph and +// no C++ runtime at all. Before this module the absence of the third layer was +// read as "the target side is not from the graph", the payload's own libc++ +// stayed on the link line, and a macOS cross ended in +// `libc++.so: unhandled file type`. +TEST(TargetSideResolve, PureCOverOpenkalStillHasItsSystemFromTheGraph) { + auto in = payload_linux(); + in.llvmTriple = "arm64-apple-macos14.0"; + in.targetOs = "macos"; + in.kernelAbi = provider("openkal-macos", "0.3.1", "openkal"); + in.cAbi = provider("openkal-musl", "0.3.1", "musl"); + // no cxxAbi provider + + auto r = ts::resolve(in); + EXPECT_EQ(r.cxx.origin, ts::Origin::None) + << "a C program needs no C++ runtime, and that is not a failure to find one"; + EXPECT_TRUE(r.system_from_graph()) + << "the system still comes from the graph, which is the whole defect"; + EXPECT_NE(r.cxx.origin, ts::Origin::Payload) + << "the payload's C++ runtime must not be selected over a graph C library"; +} + +// Only the platform implementation, and nothing above it. +TEST(TargetSideResolve, RawOpenkalHasOnlyAKernelAbi) { + auto in = payload_linux(); + in.kernelAbi = provider("openkal-linux", "0.5.1", "openkal"); + in.sysrootDeclaredEmpty = true; + + auto r = ts::resolve(in); + EXPECT_EQ(r.kernelAbi.origin, ts::Origin::Graph); + EXPECT_TRUE(r.cAbi.absent()); + EXPECT_TRUE(r.cxx.absent()); +} + +TEST(TargetSideResolve, BareMetalWithPicolibcHasNoKernelAbi) { + ts::Inputs in; + in.llvmTriple = "riscv64-none-elf"; + in.freestandingTarget = true; + in.sysrootXpkg = "xim:picolibc-riscv@1.8.12"; + + auto r = ts::resolve(in); + EXPECT_TRUE(r.kernelAbi.absent()) + << "a bare machine has no kernel, and saying so is the information"; + EXPECT_EQ(r.cAbi.origin, ts::Origin::Xpkg); + // The package name, not a prettier form of it. Stripping the `-riscv` + // suffix would mean the engine knows how this ecosystem names its + // packages, which is exactly the knowledge this design keeps out of it. + EXPECT_EQ(r.cAbi.interfaceName, "picolibc-riscv"); + EXPECT_EQ(r.cAbi.impl, "xim:picolibc-riscv@1.8.12"); + EXPECT_TRUE(r.cxx.absent()); + EXPECT_FALSE(r.system_from_graph()); +} + +// THE MIXED CASE. It already ships, and no single boolean can express it. +TEST(TargetSideResolve, PrebuiltCLibraryUnderAGraphSuppliedCxxSubset) { + ts::Inputs in; + in.llvmTriple = "riscv64-none-elf"; + in.freestandingTarget = true; + in.sysrootXpkg = "xim:picolibc-riscv@1.8.12"; + in.cxxAbi = provider("std-freestanding", "0.5.0", "freestanding subset"); + + auto r = ts::resolve(in); + EXPECT_EQ(r.cAbi.origin, ts::Origin::Xpkg) << "prebuilt"; + EXPECT_EQ(r.cxx.origin, ts::Origin::Graph) << "composed at build time"; + EXPECT_TRUE(r.cxx.subset) << "no std module declared, so the library is a subset"; +} + +TEST(TargetSideResolve, ZeroLibcTierHasNothingAtAll) { + ts::Inputs in; + in.llvmTriple = "x86_64-none-elf"; + in.freestandingTarget = true; + in.sysrootDeclaredEmpty = true; + + auto r = ts::resolve(in); + EXPECT_TRUE(r.kernelAbi.absent()); + EXPECT_TRUE(r.cAbi.absent()); + EXPECT_TRUE(r.cxx.absent()); +} + +// ── The layering rule ──────────────────────────────────────────────────────── +// +// An implementation must have been configured for the layer beneath it. The +// resolver enforces it by construction; this asserts that the construction +// really does, rather than that a later check catches it. +TEST(TargetSideResolve, PayloadCxxIsNotSelectedOverANonPayloadCLibrary) { + for (auto make : std::initializer_list>{ + [] { auto in = payload_linux(); in.sysrootXpkg = "xim:picolibc-riscv@1.8.12"; return in; }, + [] { auto in = payload_linux(); + in.cAbi = provider("openkal-musl", "0.3.1", "musl"); return in; }}) { + auto r = ts::resolve(make()); + EXPECT_NE(r.cxx.origin, ts::Origin::Payload); + EXPECT_EQ(ts::check_layering(r), std::nullopt) + << "the default path must not be able to construct the violation"; + } +} + +TEST(TargetSideResolve, TheLayeringRuleIsStatedForTheOverridePath) { + ts::TargetSide bad; + bad.cAbi = { ts::Origin::Graph, "musl", "openkal-musl@0.3.1", false }; + bad.cxx = { ts::Origin::Payload, "libc++", "xim-x-llvm@22.1.8", false }; + + auto why = ts::check_layering(bad); + ASSERT_TRUE(why.has_value()); + EXPECT_NE(why->find("never configured for this one"), std::string::npos) + << "the diagnostic must state the reason, not merely the combination"; +} + +// ── The report ─────────────────────────────────────────────────────────────── + +TEST(TargetSideReport, ShowsTheTranslatedTripleWhenItDiffers) { + auto in = payload_linux(); + in.llvmTriple = "arm64-apple-macos14.0"; + auto text = ts::format_report(ts::resolve(in), "aarch64-macos"); + EXPECT_NE(text.find("aarch64-macos → arm64-apple-macos14.0"), std::string::npos) + << "the translation is load bearing: the untranslated form emits a Mach-O " + "whose MinVersion carries no platform"; +} + +TEST(TargetSideReport, AbsentLayersReadAsADashRatherThanBeingOmitted) { + ts::Inputs in; + in.llvmTriple = "x86_64-none-elf"; + in.freestandingTarget = true; + in.sysrootDeclaredEmpty = true; + auto text = ts::format_report(ts::resolve(in), "x86_64-none-elf"); + EXPECT_NE(text.find("kernel-abi —"), std::string::npos); + EXPECT_NE(text.find("c-abi —"), std::string::npos); + EXPECT_NE(text.find("c++ —"), std::string::npos); +} + +TEST(TargetSideReport, NamesInterfaceAndImplementationSeparately) { + auto in = payload_linux(); + in.kernelAbi = provider("openkal-opensbi", "0.1.0", "openkal"); + auto text = ts::format_report(ts::resolve(in), "riscv64-none-elf"); + EXPECT_NE(text.find("openkal"), std::string::npos); + EXPECT_NE(text.find("openkal-opensbi@0.1.0"), std::string::npos) + << "four packages answer to one interface name; collapsing them would " + "hide why one source reaches four machines"; + EXPECT_NE(text.find("graph"), std::string::npos); +} diff --git a/tests/unit/test_toolchain_triple.cpp b/tests/unit/test_toolchain_triple.cpp index 6d4d9318..1143023d 100644 --- a/tests/unit/test_toolchain_triple.cpp +++ b/tests/unit/test_toolchain_triple.cpp @@ -256,3 +256,88 @@ TEST(Triple, EffectiveSysrootIsEmptyForHostedTargets) { ASSERT_TRUE(t.has_value()); EXPECT_EQ(effective_sysroot(*t, nullptr), ""); } + +// ── llvm_triple: the spelling a compiler takes, which is not the one mcpp uses + +// mcpp's canonical form and LLVM's four-field form differ in more than +// punctuation: on Apple platforms the architecture has a different name and the +// operating system carries a version. Every one of these was a defect the +// three-host matrix found rather than a test. + +TEST(Triple, LlvmTripleLinuxGnu) { + auto t = parse("x86_64-linux-gnu"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->llvm_triple(""), "x86_64-unknown-linux-gnu"); +} + +TEST(Triple, LlvmTripleLinuxMusl) { + auto t = parse("aarch64-linux-musl"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->llvm_triple(""), "aarch64-unknown-linux-musl"); +} + +TEST(Triple, LlvmTripleWindowsGnu) { + auto t = parse("x86_64-windows-gnu"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->llvm_triple(""), "x86_64-w64-windows-gnu"); +} + +// ⚠️ `aarch64` becomes `arm64` and the version is appended. A build that +// emitted mcpp's own spelling produced `--target=aarch64-macos`, which clang +// accepts as a triple it has never heard of and then treats as bare-metal +// aarch64 — the module and its importers then agree with each other and with +// nothing else. +TEST(Triple, LlvmTripleMacosRenamesArchAndCarriesVersion) { + auto t = parse("aarch64-macos"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->llvm_triple("14.0"), "arm64-apple-macos14.0"); + EXPECT_EQ(t->llvm_triple("15.2"), "arm64-apple-macos15.2"); +} + +TEST(Triple, LlvmTripleMacosX86KeepsArchName) { + auto t = parse("x86_64-macos"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->llvm_triple("14.0"), "x86_64-apple-macos14.0"); +} + +// A freestanding triple is already LLVM's own form, so it is returned as +// written rather than expanded into four fields. +TEST(Triple, LlvmTripleFreestandingIsUnchanged) { + auto t = parse("riscv64-none-elf"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->llvm_triple(""), "riscv64-none-elf"); +} + +// ── The object format each target has, asked of the triple rather than of the +// machine running the build. + +// ⚠️ The artefact-format decision used to test the triple for the substrings +// `apple` and `darwin`. Those are LLVM's words; mcpp's canonical form is +// `aarch64-macos`, which contains neither — so the test fell through to a +// question about the HOST, and produced opposite errors on opposite hosts: an +// ELF contract for a Mach-O when built on Linux, and a Mach-O contract for an +// ELF when built on macOS. +TEST(Triple, OsFieldIdentifiesMacosWithoutTheWordApple) { + auto t = parse("aarch64-macos"); + ASSERT_TRUE(t.has_value()); + EXPECT_EQ(t->os, "macos"); + EXPECT_FALSE(t->is_pe()); + EXPECT_FALSE(t->is_freestanding()); + EXPECT_EQ(t->str().find("apple"), std::string::npos); + EXPECT_EQ(t->str().find("darwin"), std::string::npos); +} + +TEST(Triple, OsFieldIdentifiesPe) { + auto t = parse("x86_64-windows-gnu"); + ASSERT_TRUE(t.has_value()); + EXPECT_TRUE(t->is_pe()); + EXPECT_EQ(t->os, "windows"); +} + +TEST(Triple, OsFieldIdentifiesFreestanding) { + auto t = parse("riscv64-none-elf"); + ASSERT_TRUE(t.has_value()); + EXPECT_TRUE(t->is_freestanding()); + EXPECT_EQ(t->os, "none"); + EXPECT_FALSE(t->is_pe()); +}