From 183c60ad2870270e3d0bcf830ef570153fb7e5cb Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 00:05:15 +0900 Subject: [PATCH 01/14] 2026.11.2.3: openkal-platform 0.21.1; design 2026-11-02 part 3 landed Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...art3-ok-image-performance-parity-design.md | 22 +++++++++++++++++-- CHANGELOG.md | 20 +++++++++++++++++ docs/04-mcpp-toml.md | 2 +- docs/10-pack-and-release.md | 2 +- docs/24-openkal-cross.md | 2 +- docs/35-platforms.md | 8 +++---- docs/50-machine-output.md | 2 +- docs/specs/packaging.md | 2 +- docs/zh/04-mcpp-toml.md | 2 +- docs/zh/10-pack-and-release.md | 2 +- docs/zh/24-openkal-cross.md | 2 +- docs/zh/35-platforms.md | 8 +++---- docs/zh/50-machine-output.md | 2 +- mcpp.toml | 2 +- modules/toolchain-model/src/triple.cppm | 4 ++-- modules/versioning/src/version.cppm | 2 +- src/pack/distribution.cppm | 2 +- 17 files changed, 62 insertions(+), 24 deletions(-) diff --git a/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md b/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md index 43246540c..f3f13eb2b 100644 --- a/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md +++ b/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md @@ -601,6 +601,7 @@ mcpp 在首次初始化时已把 Ninja 装进自己的 home（`ensure_ninja`，` | openkal-macos 0.19 | 文件映射在可执行内存可用时声明；处理器时间取自 `proc_pid_rusage`；`kal_alloc` 同 Linux。启动程序保持 `fork`：`vfork` 在 CI 的 macOS 上以 SIGSYS 结束子进程 | | openkal-windows 0.17 | 回答 `kal_err_not_supported`，不声明该位：保留区中拆出的部分只有在整个保留以占位方式建立时才能换成文件视图，区域会成为需逐个释放的片段表（7.1）。加载器照旧复制 | | openkal-image 0.3 | okld 从文件映射满足四个条件的段：不可写、文件大小等于内存大小、文件偏移与地址模 64 KiB 同余、独占所在粒度；其余段复制。每个重定位只写入可写段。okrun 在映像运行期间保持文件打开 | +| openkal-image 0.3.1（openkal-platform 0.21.1） | 记住的 okb 准入不再读取成员：okrun 对此前在同一文件上比对过的成员只按形状准入（`okld_okb_admit_layout`）。0.2、0.3 在此计算归档声明的 CRC-32，每次启动遍历整个成员，clangd 的 71 MB 映像约 0.1 s 处理器时间；warm-prime 冷阶段启动 176 个 worker，CI 上映像因此为原生的 2.4–3.5 倍（本地以 `.oke` 测量，不经过 okb，未显现） | §5.2 的单元按 worker 建立：每个 worker 以 `kal_process_job_enter` 形成自己的单元，以 `kal_process_job_limit` 的内存限额取代 `RLIMIT_AS`（openkal 不陈述地址空间上限）；worker 组的预算仍按 `kal_process_props` 的回答选择。 @@ -639,6 +640,9 @@ mcpp 在首次初始化时已把 Ninja 装进自己的 home（`ensure_ninja`，` | 0040 | 自有缓存中的模块在其发布处、凭自身的租约读取，不再为每个读者复制 | | 0041 | worker 结束即被看到：所有者监视以条件变量等待，worker 退出时通知；父进程查看运行中的 worker 前 50 次间隔 1 ms，此后 10 ms。冷阶段 176 个模块的就绪由 4.9 s 降为 3.4 s，worker 的中位时长由 20 ms 降为 9 ms | | okgen | 工具链文件变化时删除 `CMakeCache.txt` 与 `CMakeFiles` 后重新配置 | +| worker 的上限（0042） | 每个 worker 的地址空间份额，在系统能为进程设上限的地方由 worker 自设（`RLIMIT_AS`），在 macOS 上由 clangd 监视 worker 的物理占用（`proc_pid_rusage` 的 `ri_phys_footprint`），超过份额即结束它。macOS 的进程在 main 之前已映射数百 GiB（macOS 15 上 392 GiB）：低于此值的上限被拒绝（EINVAL），原生引擎的每个 worker 因此失败；高于此值的上限不约束任何分配（上限比已映射量高 1 GiB 时，2 GiB 的分配仍成功） | +| 已删除名字的目录 | APFS 对名字已删除、仍打开的目录报告原有的名字数（2），文件则为 0；macOS 上目录只有一个名字，删除它即无名字 | +| 一个根只打开一次 | 一个根的前几个模块同时构建，各自测定根目录并打开其账本时，macOS 上其中一个因 ENOENT 失败；现由其中一个测定并打开，其余等待并取用其结果 | ### 17.4 验收的形态与结果 @@ -646,8 +650,22 @@ mcpp 在首次初始化时已把 Ninja 装进自己的 home（`ensure_ninja`，` 与缺页在 Linux 上取 `/proc//stat` 的 `utime`、`stime`、`cutime`、`cstime` 与对应的缺页字段，其他系统为引擎自身的时间 加 worker 日志中各 worker 的时间。门禁在 Linux 与 macOS（两个引擎都走自有缓存与 worker）；Windows 原生走可移植路径，只记录。 - **L3（Linux x64）**：映像对原生，冷阶段就绪 1.04、CPU 1.08、缺页 1.12；已有缓存的阶段 1.04、1.09、1.12。 -- **L2**：Linux x64 的 hosted 与映像全部 ≤ 1.15。CI 的映像对比交替运行两侧各五次、逐行取最小值：同一侧连续运行时，共享 - 机器的一段干扰只落在一侧（macOS arm64 的 options/command 一次为 1.22，重跑为 0.85）。 +- **L2**：CI 的映像对比交替运行两侧各五次、逐行取最小值：同一侧连续运行时，共享机器的一段干扰只落在一侧（macOS arm64 的 + options/command 一次为 1.22，重跑为 0.85）。系统一侧以 `clang++` 编译（门禁）并以系统默认的 `c++` 编译（报告）：两侧的编译器 + 相同时，比值只反映运行时。Linux arm64 的 index/symbol 以 GCC 与 libstdc++ 为 220 ns，以 Clang 与同一个 libstdc++ 为 261 ns， + hosted 与映像均为 237 ns；比值 1.15–1.18 来自编译器而不是运行时。openkal-llvm-runtime 0.17.1 的 main CI（mcpp 2026.11.2.2）， + 映像对系统的运行时（门禁一侧；括号内为对系统默认编译器的报告值）： + + | 工作负载 | Linux x64 | Linux arm64 | macOS arm64 | Linux x64 hosted | + |---|---|---|---|---| + | options/command | 0.80（0.82） | 0.92（1.00） | 0.78（0.98） | 0.78（0.80） | + | stringmap/element | 0.70（0.70） | 0.66（0.67） | 0.89（1.07） | 0.64（0.65） | + | densemap/element | 1.00（0.98） | 0.98（0.90） | 0.86（0.92） | 0.98（0.96） | + | bitstream/field | 0.97（0.95） | 1.03（1.06） | 0.74（0.76） | 0.98（0.95） | + | index/symbol | 0.86（0.91） | 0.96（1.18） | 1.00（1.12） | 0.83（0.86） | + + macOS 上 `c++` 与 `clang++` 同为 Apple Clang，两列是同一个编译器的两次构建，其差（options/command 0.78 与 0.98）是共享机器 + 上交替运行之后仍余下的波动。Windows 没有系统的 C++ 编译器，只记录映像的数值。 - **L1（报告）**：Linux x64，对 glibc。hosted（GCC 编译）69 行中 5 行超过 1.15：线程私有数据 1.20、stat 1.16、clock 1.21（多一层 openkal 调用，约 2 ns）、4 线程的 open-close 1.16、memcmp/64/7 1.19。映像（Clang 编译）16 行超过，集中在字符串函数 （memchr/4096 1.46、memcpy/4096 1.34、strcmp/64 1.29），另有 clock、stat 与线程私有数据；同一份源码以 Clang 编成 hosted 时为 diff --git a/CHANGELOG.md b/CHANGELOG.md index ecffd20af..c8f5dcb7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,26 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.11.2.3] - 2026-11-02 + +The built-in openkal platform is 0.21.1; design 2026-11-02 part 3 landed. + +### Changed + +- **The built-in `-openkal` platform is `mcpplibs.openkal-platform@0.21.1`** + (docs/35): openkal-image 0.3.1, whose runner admits an okb member it has + compared before, on the same file, by its shape alone. 0.3.0 computed the + member's CRC-32 on every start: 0.1 s of processor time for a 71 MB image, + paid by each program an okb starts (a language server's 176 module workers + in a cold build). + +### Design + +- Design 2026-11-02 part 3 (ok image performance parity) landed: the engine's + revisions measured on macOS (a worker's ceiling watched, an open directory's + count of names on APFS, one opening of a cache root), L2 on every system and + L3 on every system. + ## [2026.11.2.2] - 2026-11-02 The built-in openkal platform is 0.21.0; an embedded dependency's link diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 23a16cf1d..53496424e 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1502,7 +1502,7 @@ the platform's own key names ([35 — Platforms](35-platforms.md)): ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.21.0" # or { path = "…" }, or "none" +platform = "openkal-platform@0.21.1" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 7aa1d7bf7..0e533bb72 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -952,7 +952,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.21.0", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.21.1", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index 028ffda6b..e0cee468a 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -111,7 +111,7 @@ practice. ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.0" +platform = "openkal-platform@0.21.1" ``` One table (2026.10.27.1+). It selects the openkal platform for the host triples, diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 5407b39cb..16e6dd5b6 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -113,9 +113,9 @@ The first of these that answers selects the target's platform: | Source | Spelling | |---|---| -| the command line | `--platform openkal-platform@0.21.0`, `--platform path:../openkal-platform`, `--platform none` | +| the command line | `--platform openkal-platform@0.21.1`, `--platform path:../openkal-platform`, `--platform none` | | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| the project | `[target.] platform = "openkal-platform@0.21.0"`, or `{ path = "…" }` | +| the project | `[target.] platform = "openkal-platform@0.21.1"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | | the target table | the platform column (`mcpp toolchain list`) | @@ -129,7 +129,7 @@ its own conventions. `[host] platform` selects a platform for them instead: ```toml [host] -platform = "openkal-platform@0.21.0" +platform = "openkal-platform@0.21.1" ``` ``` @@ -151,7 +151,7 @@ implementation: ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.0" +platform = "openkal-platform@0.21.1" ``` On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 63f6ce2e1..cda7586fc 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -527,7 +527,7 @@ installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. A target row also carries `platform` (2026.10.27.1+): the platform package the -target table selects for it (`mcpplibs.openkal-platform@0.21.0`), or `null`. +target table selects for it (`mcpplibs.openkal-platform@0.21.1`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index af9ed077c..9b5a75535 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -237,7 +237,7 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` "schema": 1, "name": "hello", "version": "0.1.0", - "platform": "openkal-platform@0.21.0", + "platform": "openkal-platform@0.21.1", "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 1e85705f4..71482d129 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1384,7 +1384,7 @@ runner 与 `run_under`。 ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.21.0" # or { path = "…" }, or "none" +platform = "openkal-platform@0.21.1" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 3d562c391..f79fa3ba2 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -799,7 +799,7 @@ Authenticode 可以签名它。Linux 与 macOS 经 shell 启动它：`sh pg-0.1. 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.21.0", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.21.1", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index dfe8f33d2..420bffd9e 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -93,7 +93,7 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.0" +platform = "openkal-platform@0.21.1" ``` 一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index fa9a154f2..8591f2c0b 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -98,9 +98,9 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 来源 | 写法 | |---|---| -| 命令行 | `--platform openkal-platform@0.21.0`、`--platform path:../openkal-platform`、`--platform none` | +| 命令行 | `--platform openkal-platform@0.21.1`、`--platform path:../openkal-platform`、`--platform none` | | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| 工程 | `[target.] platform = "openkal-platform@0.21.0"`,或 `{ path = "…" }` | +| 工程 | `[target.] platform = "openkal-platform@0.21.1"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | | 目标表 | 平台列(`mcpp toolchain list`) | @@ -112,7 +112,7 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 ```toml [host] -platform = "openkal-platform@0.21.0" +platform = "openkal-platform@0.21.1" ``` ``` @@ -130,7 +130,7 @@ platform = "openkal-platform@0.21.0" ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.0" +platform = "openkal-platform@0.21.1" ``` Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index fa1269f15..1941d304b 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -463,7 +463,7 @@ mcpp toolchain list --format json 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 -（`mcpplibs.openkal-platform@0.21.0`），或 `null`。 +（`mcpplibs.openkal-platform@0.21.1`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 diff --git a/mcpp.toml b/mcpp.toml index 01190acbb..dc2bf381d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.11.2.2" +version = "2026.11.2.3" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 7c717900d..4131c1dea 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -906,9 +906,9 @@ inline constexpr TargetInfo kKnownTargets[] = { // `openkal-platform` (SPEC-013), pinned here. The two architectures the // loader relocates. { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.21.0" }, + "mcpplibs.openkal-platform@0.21.1" }, { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.21.0" }, + "mcpplibs.openkal-platform@0.21.1" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index f248e7315..652d42fbe 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.11.2.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.11.2.3"; } // namespace mcpp diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 12b421016..487416a92 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1075,7 +1075,7 @@ std::string exe_launcher_manifest(std::string_view name, std::string_view versio "[package]\nname = \"{0}\"\nversion = \"{1}\"\n\n" "[targets.{0}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" "# 0.2.7: the release written against openkal 0.21, the openkal the\n" - "# platform of mcpp's openkal rows names (openkal-platform 0.21.0), which\n" + "# platform of mcpp's openkal rows names (openkal-platform 0.21.1), which\n" "# the launcher is built over.\n" "[dependencies.mcpplibs]\nos = \"0.2.7\"\n", name, version.empty() ? std::string_view("0.0.0") : version); From e95d35df9db45217713f325f497940e7e3d6454a Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 00:42:36 +0900 Subject: [PATCH 02/14] a download source that connects and sends nothing is left for the next one: mcpp's stall bound is longer than a source's Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 13 ++++++++++++- docs/specs/home-initialization.md | 5 +++-- src/provider/xlings/xlings.cpp | 4 ++-- src/provider/xlings/xlings.cppm | 15 +++++++++++++-- tests/unit/test_network.cpp | 22 ++++++++++++++++++++++ 5 files changed, 52 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c8f5dcb7b..0ba16644c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,8 @@ ## [2026.11.2.3] - 2026-11-02 -The built-in openkal platform is 0.21.1; design 2026-11-02 part 3 landed. +The built-in openkal platform is 0.21.1; a download source that sends nothing +is left for the next one; design 2026-11-02 part 3 landed. ### Changed @@ -17,6 +18,16 @@ The built-in openkal platform is 0.21.1; design 2026-11-02 part 3 landed. paid by each program an okb starts (a language server's 176 module workers in a cold build). +### Fixed + +- **A download source that connects and sends nothing no longer stops the + install** (docs/specs/home-initialization.md §5). xlings leaves such a source + after 30 seconds without a byte and tries the next; mcpp stopped the install + after 30 seconds without a byte or a new source, so the next source was never + reached (CI, 2026-11-02: a GitHub proxy that accepted the connection, + attempt 1). mcpp's own bound is now 45 seconds; a source's bounds, 8 seconds + to connect and 30 without a byte, are unchanged. + ### Design - Design 2026-11-02 part 3 (ok image performance parity) landed: the engine's diff --git a/docs/specs/home-initialization.md b/docs/specs/home-initialization.md index 9e3b440f3..7e58aae76 100644 --- a/docs/specs/home-initialization.md +++ b/docs/specs/home-initialization.md @@ -87,8 +87,9 @@ mirror 更改只写本地配置。配置读取、本地初始化、能力获取 状态查询要求 `read_only_queries` 能力；缺失时**必须**跳过查询，继续本地诊断， **禁止**为了诊断安装替代提供者。 -下载连接上限 8 秒、无字节进展上限 30 秒；已有更短限制保留。 -heartbeat 或重复字节值**禁止**更新下载进展时间。小型索引请求保留 120 秒总保护， +下载的每个来源连接上限 8 秒、无字节进展上限 30 秒，由提供者执行，超出即换下一个来源；已有更短限制保留。 +父进程在一个文件既无新字节也无新来源 45 秒时终止安装：这一上限**必须**长于来源的上限，否则连上而不发送 +字节的来源会在提供者换源的同一时刻终止安装。heartbeat 或重复字节值**禁止**更新下载进展时间。小型索引请求保留 120 秒总保护， 大型安装保留 3 小时最终进程保护。实际下载源和候选重试次数来自提供者事件， **禁止**将探测成功的候选伪报为实际下载来源。 diff --git a/src/provider/xlings/xlings.cpp b/src/provider/xlings/xlings.cpp index 9d7f3d4c9..b12386882 100644 --- a/src/provider/xlings/xlings.cpp +++ b/src/provider/xlings/xlings.cpp @@ -925,8 +925,8 @@ std::vector invocation_env(const Env& env) { {"XLINGS_HOME", env.home.string(), true}, {"XLINGS_PROJECT_DIR", env.projectDir.string(), !env.projectDir.empty()}, {"XLINGS_ACTIVE_SUBOS", "", false}, - {"XLINGS_DOWNLOAD_CONNECT_TIMEOUT", cap("XLINGS_DOWNLOAD_CONNECT_TIMEOUT", 8), true}, - {"XLINGS_DOWNLOAD_IDLE_TIMEOUT", cap("XLINGS_DOWNLOAD_IDLE_TIMEOUT", 30), true}, + {"XLINGS_DOWNLOAD_CONNECT_TIMEOUT", cap("XLINGS_DOWNLOAD_CONNECT_TIMEOUT", static_cast(kSourceConnectTimeout.count())), true}, + {"XLINGS_DOWNLOAD_IDLE_TIMEOUT", cap("XLINGS_DOWNLOAD_IDLE_TIMEOUT", static_cast(kSourceIdleTimeout.count())), true}, {"XLINGS_DOWNLOAD_LOW_SPEED", mcpp::invocation::get("XLINGS_DOWNLOAD_LOW_SPEED").value_or("off"), true}, }; } diff --git a/src/provider/xlings/xlings.cppm b/src/provider/xlings/xlings.cppm index c92bcadea..db36797be 100644 --- a/src/provider/xlings/xlings.cppm +++ b/src/provider/xlings/xlings.cppm @@ -478,6 +478,17 @@ using BootstrapProgress = mcpp::provider::BootstrapProgress; using BootstrapProgressCallback = mcpp::provider::BootstrapProgressCallback; +// A download's bounds (design 2026-10-19 §6.2, R5). xlings gives each source +// kSourceConnectTimeout to connect and kSourceIdleTimeout without a byte, then +// tries the next source; mcpp stops the install when a file has had neither a +// byte nor a new source for kDownloadStallTimeout. The parent's bound is the +// longer one: were they equal, a source that connects and then sends nothing +// would stop the install at the moment xlings gives the source up, and no +// install would reach its second source. +inline constexpr std::chrono::seconds kSourceConnectTimeout{8}; +inline constexpr std::chrono::seconds kSourceIdleTimeout{30}; +inline constexpr std::chrono::seconds kDownloadStallTimeout{kSourceIdleTimeout + std::chrono::seconds(15)}; + // Parent-owned byte progress protection, independent of provider heartbeats // and socket reads (DNS or a trickle of headers may produce neither body bytes // nor a socket timeout). Time is injectable for the state-machine tests. @@ -491,7 +502,7 @@ class DownloadWatchdog { std::map files_; std::chrono::milliseconds idle_; public: - explicit DownloadWatchdog(std::chrono::milliseconds idle = std::chrono::seconds(30)) : idle_(idle) {} + explicit DownloadWatchdog(std::chrono::milliseconds idle = kDownloadStallTimeout) : idle_(idle) {} void observe(const BootstrapProgress& progress, std::chrono::steady_clock::time_point now = std::chrono::steady_clock::now()) { for (const auto& f : progress.files) { @@ -518,7 +529,7 @@ class InstallDeadline { bool downloadSeen_ = false, localWork_ = false; public: explicit InstallDeadline(std::chrono::milliseconds preparation = std::chrono::seconds(120), - std::chrono::milliseconds idle = std::chrono::seconds(30), + std::chrono::milliseconds idle = kDownloadStallTimeout, std::chrono::steady_clock::time_point start = std::chrono::steady_clock::now()) : downloads_(idle), started_(start), preparation_(preparation) {} void observe(std::string_view line, diff --git a/tests/unit/test_network.cpp b/tests/unit/test_network.cpp index 46f677880..e5f398fd6 100644 --- a/tests/unit/test_network.cpp +++ b/tests/unit/test_network.cpp @@ -98,6 +98,28 @@ TEST(NetworkCheck, HeartbeatsCannotKeepAByteStalledDownloadAlive) { EXPECT_FALSE(watchdog.expired(start + hours(3))); } +// xlings leaves a source that sent no byte for kSourceIdleTimeout and reports +// the next one; mcpp's own bound is longer, so the next source is reached. +// With the two equal, a source that connected and then sent nothing stopped the +// install at attempt 1 (CI, 2026-11-02). +TEST(NetworkCheck, ASilentSourceIsLeftBeforeTheInstallStops) { + using namespace std::chrono; + namespace x = mcpp::provider::xlings; + static_assert(x::kSourceConnectTimeout < x::kSourceIdleTimeout); + static_assert(x::kSourceIdleTimeout < x::kDownloadStallTimeout); + x::DownloadWatchdog watchdog; + const auto start = steady_clock::now(); + const auto left = start + x::kSourceIdleTimeout + seconds(1); + mcpp::provider::BootstrapProgress p; + p.files.push_back({.name = "pkg", .started = true, .sourceAttempt = 1}); + watchdog.observe(p, start); + EXPECT_FALSE(watchdog.expired(left)); + p.files.front().sourceAttempt = 2; + watchdog.observe(p, left); + EXPECT_FALSE(watchdog.expired(left + x::kSourceIdleTimeout)); + EXPECT_TRUE(watchdog.expired(left + x::kDownloadStallTimeout)); +} + TEST(NetworkProcess, ProgressCancellationIsPolledWhileOutputContinues) { using namespace std::chrono; const auto start = steady_clock::now(); From c11d021389b42cc0ac68bb2d744e329d6a87aa68 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 02:42:21 +0900 Subject: [PATCH 03/14] a requirement a layer decides is checked once the layers are resolved Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 12 +++- docs/22-target-side.md | 10 +++ docs/specs/target-side.md | 5 +- docs/zh/22-target-side.md | 9 +++ src/build/prepare/config.cpp | 11 +++ src/build/prepare/state.cppm | 5 ++ src/build/prepare/target_side.cpp | 34 ++++++++++ ..._requirement_a_layer_decides_is_checked.sh | 67 +++++++++++++++++++ 8 files changed, 151 insertions(+), 2 deletions(-) create mode 100755 tests/e2e/1007_a_requirement_a_layer_decides_is_checked.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ba16644c..c42048836 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,8 @@ ## [2026.11.2.3] - 2026-11-02 The built-in openkal platform is 0.21.1; a download source that sends nothing -is left for the next one; design 2026-11-02 part 3 landed. +is left for the next one; a requirement a layer decides is checked; design +2026-11-02 part 3 landed. ### Changed @@ -20,6 +21,15 @@ is left for the next one; design 2026-11-02 part 3 landed. ### Fixed +- **A requirement a layer decides is checked** (docs/22, docs/specs/target-side.md + §2.2). `[target.'cfg(all(windows, compiler = "llvm"))'] requires = + ["mcpp:compiler-runtime=compiler-rt"]` names a layer in its selector, and + such a table's requirement was dropped without a word: the requirements were + gathered before any layer resolved. It is now checked once the layers are + resolved and before anything is compiled, with the refusal a `[package]` + requirement gets. openkal-musl 0.27.0 states this one: on PE under clang its + link names no runtime, and a graph without compiler-rt (openkal-llvm-runtime) + reached the link with `___chkstk_ms` undefined. - **A download source that connects and sends nothing no longer stops the install** (docs/specs/home-initialization.md §5). xlings leaves such a source after 30 seconds without a byte and tries the next; mcpp stopped the install diff --git a/docs/22-target-side.md b/docs/22-target-side.md index e8ac04fd4..30da96a3c 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -343,6 +343,16 @@ An engine older than 2026.10.26.1 refuses the table form; a package that uses it states the floor (`[build-dependencies.mcpp] version = ">= 2026.10.26.1"`, or `mcpp = ">= 2026.10.26.1"` in an index descriptor). +A selector may name a layer, and the requirement then holds where that layer +resolved as named. It is checked once the layers are resolved and before +anything is compiled (mcpp 2026.11.2.3; an older engine does not check it): + +```toml +# the compiler's runtime on PE under clang is the graph's (openkal-musl) +[target.'cfg(all(windows, compiler = "llvm"))'] +requires = ["mcpp:compiler-runtime=compiler-rt"] +``` + ### One C++ runtime per process (mcpp 2026.10.26.1+) Two Itanium C++ runtimes — libstdc++, and libc++ with libc++abi — define the diff --git a/docs/specs/target-side.md b/docs/specs/target-side.md index e6f57a160..cd523758f 100644 --- a/docs/specs/target-side.md +++ b/docs/specs/target-side.md @@ -80,7 +80,9 @@ mcpp:<层名>[=<实现名>] 清单**必须**以 `requires-private-layer` 报错 —— 一个映像中的第二个 C 库不是孤岛,而是两个 C 库。 `requires` **可以**写在 `[target.<选择器>]` 下。生效的需求是 `[package] requires` 与所有匹配 -选择器的 `requires` 的并集,合并规则与 `requires_abi` 相同。 +选择器的 `requires` 的并集,合并规则与 `requires_abi` 相同。选择器引用层(`compiler`、 +`compiler-runtime`、`kernel-abi`、`c-abi`、`c++-abi`)时,需求**必须**在层解析之后、任何编译开始 +之前按规则二检查;引擎**禁止**忽略这样的需求。 ### 2.3 标准库模块源 已实现 @@ -335,4 +337,5 @@ mcpp:<层名>[=<实现名>] | v1.0 | 2026-08-24 | 初版。五层闭集、`provides`/`requires` 语法、三条规则、报告与兼容性条款。 | | v1.1 | 2026-09-26 | 规则三增补:`c-abi` 来自图时,链接不搜索宿主库目录(ELF 已实现,mcpp#696)。 | | v1.3 | 2026-10-26 | 随 mcpp 2026.10.26.1:§2.2 增加 `requires` 的对象形式与 `visibility`、`[target.<选择器>] requires`;新增 §7 一个进程一套 C++ 运行时(U/I/H、孤岛条件、外来运行时按类型识别、依赖图供给的 libstdc++)(mcpp#838)。 | +| v1.4 | 2026-11-03 | 随 mcpp 2026.11.2.3:§2.2 选择器引用层的 `requires` 在层解析之后检查(此前未检查即被丢弃)。 | | v1.2 | 2026-10-23 | 随 mcpp 2026.10.24.1:§1.3 `prebuilt` 放宽为被点名、按内容识别的预制目录;§2.4 改为由测量回答、`provides` 声明或核对,标为已实现;规则一增加基座是一个供给者与 c-abi 独占律;规则二增加基座上的编译器运行库与可测量的下限;规则三增加可叠加层;新增 §6 目标侧执行(组合、运行器契约、物理边界)。 | diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 465570b79..2ae1b621b 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -296,6 +296,15 @@ requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }] （`[build-dependencies.mcpp] version = ">= 2026.10.26.1"`，或 index 描述文件中的 `mcpp = ">= 2026.10.26.1"`）。 +选择器可以引用层，此时需求在该层解析为所写的实现时成立。它在层解析之后、任何编译开始之前检查 +（mcpp 2026.11.2.3；更早的引擎不检查）： + +```toml +# PE 上 clang 的编译器运行库来自依赖图（openkal-musl） +[target.'cfg(all(windows, compiler = "llvm"))'] +requires = ["mcpp:compiler-runtime=compiler-rt"] +``` + ### 一个进程一套 C++ 运行时（mcpp 2026.10.26.1+） 两套 Itanium C++ 运行时 —— libstdc++，以及 libc++ 加 libc++abi —— 定义同样的 ABI diff --git a/src/build/prepare/config.cpp b/src/build/prepare/config.cpp index e0f60ba3b..4af60bac5 100644 --- a/src/build/prepare/config.cpp +++ b/src/build/prepare/config.cpp @@ -727,6 +727,17 @@ workspace_listing(const std::filesystem::path& memberDir, // [build] sections only"). Dependencies are excluded by construction — they are // already resolved by now — and a section that tries is reported rather than // silently ignored; see `warn_layer_predicate_dependencies`. +std::vector layer_conditional_requires(const mcpp::manifest::Manifest& m, + const cfgpred::Ctx& ctx) { + std::vector out; + for (auto const& cc : m.conditionalConfigs) { + if (!cfgpred::uses_layer(cc.predicate) || !cfgpred::matches(cc.predicate, ctx)) continue; + for (auto const& r : cc.requires_) + if (std::ranges::find(out, r) == out.end()) out.push_back(r); + } + return out; +} + bool merge_layer_conditional_config(mcpp::manifest::Manifest& m, const cfgpred::Ctx& ctx) { bool any = false; diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index e9cbacd11..555125f4f 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -997,6 +997,11 @@ workspace_listing(const std::filesystem::path& memberDir, const std::filesystem::path& bound); bool merge_layer_conditional_config(mcpp::manifest::Manifest& m, const cfgpred::Ctx& ctx); +// The `requires` of the layer-predicated tables `ctx` matches +// (`[target.'cfg(compiler = "llvm")'] requires = [...]`): answerable only once +// the layers are resolved, and checked then. +std::vector layer_conditional_requires(const mcpp::manifest::Manifest& m, + const cfgpred::Ctx& ctx); std::vector feature_closure(const mcpp::manifest::Manifest& pm, const std::vector& requested, bool seedDefault = true); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index ba7174d2a..4a7dedeaf 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -1376,6 +1376,40 @@ static std::expected step9_layer_conditional_config(PrepareSt layerCtx.kernelAbi = state.resolvedTargetSide.kernelAbi.interfaceName; layerCtx.cAbi = state.resolvedTargetSide.cAbi.interfaceName; layerCtx.cxxAbi = state.resolvedTargetSide.cxx.interfaceName; + // A REQUIREMENT A LAYER DECIDES IS CHECKED NOW, WHEN THE LAYERS ARE + // KNOWN. `[target.'cfg(all(windows, compiler = "llvm"))'] requires = + // ["mcpp:compiler-runtime=compiler-rt"]` states what one compiler needs + // beneath it; the requirements above were gathered before any layer + // resolved, and this table's was dropped without a word, so a graph + // without the layer reached the link (`___chkstk_ms' undefined) rather + // than this refusal naming the layer and its supplier. + { + namespace tsd = mcpp::targetside; + std::vector layerRequirements; + auto gather_from = [&](const mcpp::manifest::Manifest& pm) { + const auto id = pm.package.version.empty() + ? pm.package.name + : std::format("{}@{}", pm.package.name, pm.package.version); + for (auto const& entry : layer_conditional_requires(pm, layerCtx)) { + auto parsed = tsd::parse_capability(entry); + if (!parsed || !*parsed) continue; + tsd::Requirement r{ id, (*parsed)->layer, (*parsed)->interfaceName }; + if (std::ranges::none_of(layerRequirements, [&](auto const& q) { + return q.requiredBy == r.requiredBy && q.layer == r.layer + && q.interfaceName == r.interfaceName; })) + layerRequirements.push_back(std::move(r)); + } + }; + gather_from(*state.m); + for (auto const& pkg : state.packages) gather_from(pkg.manifest); + if (auto why = tsd::check_requirements( + state.resolvedTargetSide, layerRequirements, + tc_origin_is_user_explicit(state.tcOrigin) ? tc_origin_name(state.tcOrigin) + : std::string_view{})) { + refusal::record(refusal::Code::LayerRequirement); + return std::unexpected(*why); + } + } // The root manifest and its snapshots are both read by the plan, so a // contribution reaching the snapshots but not `*m` would reach some of // the root's commands and not others. diff --git a/tests/e2e/1007_a_requirement_a_layer_decides_is_checked.sh b/tests/e2e/1007_a_requirement_a_layer_decides_is_checked.sh new file mode 100755 index 000000000..72fd5f4fe --- /dev/null +++ b/tests/e2e/1007_a_requirement_a_layer_decides_is_checked.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 1007 -- a requirement stated in a table whose selector names a layer +# (`[target.'cfg(compiler = "llvm")'] requires = [...]`) is checked once the +# layers are resolved, as a requirement of `[package]` is. +# +# openkal-musl links its C library with `-nostdlib`, and on PE under clang the +# routines a compiler emits calls to are compiler-rt's, which the graph supplies +# (openkal-llvm-runtime). It states that as a requirement of the clang build +# only; the table's requirement was dropped without a word, and a graph without +# the layer reached the link with `___chkstk_ms` undefined. +# +# Criteria: A a dependency whose layer-selected table requires a layer this +# build does not resolve is refused before anything is compiled, the message +# naming the dependency, the layer and what it requires; B the same table under +# a selector this build does not match states nothing, and the build succeeds. +set -e + +MCPP="${MCPP:-mcpp}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +needs() { # needs + mkdir -p "$work/needs/src" + cat > "$work/needs/mcpp.toml" < "$work/needs/src/needs.cpp" +} + +mkdir -p "$work/app/src" +cat > "$work/app/mcpp.toml" < "$work/app/src/main.cpp" + +# A: a selector naming a layer, which every build matches. +needs 'cfg(not(compiler = "nonesuch"))' +if (cd "$work/app" && "$MCPP" build > "$work/a.log" 2>&1); then + cat "$work/a.log"; echo "FAIL: A: a requirement a layer decides was not checked"; exit 1 +fi +grep -q 'needs@0.1.0` requires the compiler-runtime to be `nonesuch`' "$work/a.log" \ + || { cat "$work/a.log"; echo "FAIL: A: the refusal does not name the dependency, the layer and what it requires"; exit 1; } +if grep -q 'Compiling' "$work/a.log"; then + cat "$work/a.log"; echo "FAIL: A: something was compiled before the refusal"; exit 1 +fi +echo "ok: A: a requirement a layer decides is checked when the layers are known" + +# B: the same table under a selector this build does not match. +needs 'cfg(compiler = "nonesuch")' +rm -rf "$work/app/target" +(cd "$work/app" && "$MCPP" build > "$work/b.log" 2>&1) \ + || { cat "$work/b.log"; echo "FAIL: B: a table this build does not match stated a requirement"; exit 1; } +echo "ok: B: a table this build does not match states nothing" From fd62a99755e1b862ef2465dccfd042312c0c1c55 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 08:37:16 +0900 Subject: [PATCH 04/14] a format a package provides is told the compression stated (MCPP_PACK_COMPRESS), and --compress reaches it Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 12 +++ docs/10-pack-and-release.md | 8 +- docs/30-build-mcpp.md | 1 + docs/zh/10-pack-and-release.md | 6 +- docs/zh/30-build-mcpp.md | 1 + src/build/build_program.cppm | 4 + src/build/hostprogram.cppm | 6 ++ src/build/prepare.cppm | 5 ++ src/build/prepare/state.cppm | 1 + src/cli/cmd_publish.cppm | 12 ++- src/pack/pipeline.cppm | 5 ++ ...provided_format_is_told_the_compression.sh | 79 +++++++++++++++++++ 12 files changed, 133 insertions(+), 7 deletions(-) create mode 100755 tests/e2e/1008_a_provided_format_is_told_the_compression.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index c42048836..0187d6add 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,18 @@ is left for the next one; a requirement a layer decides is checked; design paid by each program an okb starts (a language server's 176 module workers in a cold build). +### Added + +- **A format a package provides is told the compression stated** (docs/10, + docs/30). `--compress` and `[pack] compress` reach a provided format's + program as `MCPP_PACK_COMPRESS` (`mcpp::pack_compress()`), spelled as the + flag takes it, and the provider decides what its container does with it. + `--compress` was refused with every provided format, so openkal-platform's + `okb` could not be asked for its compressed form: its images stored + compressed under `gzip`, an okb kept small on disk that okrun expands into + its cache when it first runs it (design 2026-11-03 part 4 §16). A directory + is still refused. + ### Fixed - **A requirement a layer decides is checked** (docs/22, docs/specs/target-side.md diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 0e533bb72..1915c49ad 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -126,7 +126,8 @@ choices: | Combination | Remedy | |---|---| | `--pkg-format single-file` with `--mode system` | `--mode vendored` or `self-contained`: a single file carries what it runs | -| `--compress` with `--pkg-format dir` or a provided format | `--pkg-format tar`, `zip` or `single-file`: only a container is compressed | +| `--compress` with `--pkg-format dir` | `--pkg-format tar`, `zip`, `single-file` or a provided format: only a container is compressed | +| a compression a provided format does not write (`--pkg-format okb --compress zstd`) | the one it writes (`okb`: `gzip`, its images stored compressed, or `none`) | | `--compress zstd` or `xz` with `--pkg-format zip` | `--compress gzip` (DEFLATE) or `none`, or `--pkg-format tar` | | `--compress gzip` or `xz` with `--pkg-format single-file` | `--compress zstd[:]` or `none`: the launcher decompresses zstd | | `--systems` on anything but a self-contained tree of images or a single file of an image | `--mode self-contained`, or `--pkg-format single-file`; `[pack] systems` that does not fit is noted and left aside | @@ -869,7 +870,10 @@ input's. "okb"]`) builds once, stages one tree and writes each container from it, in the order named. A list holds the containers of one tree — `tar`, `zip` and a provider's format — never `dir` or `single-file`; it takes a directory for `-o`; -`--compress` applies to its engine containers. okb holds the images alone, so a +`--compress` applies to its engine containers and reaches a provider's format as +`MCPP_PACK_COMPRESS` (2026.11.2.3): openkal-platform's `okb` stores its images +compressed under `gzip` --- an okb kept small on disk, which okrun expands into +its cache when it first runs it --- and as they are otherwise. okb holds the images alone, so a self-contained tree whose only format is okb is refused (`--pkg-format tar,okb`). `mcpp run --pkg-format` takes one format. diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 95a61f1ad..722e7ede5 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -1356,6 +1356,7 @@ The running program receives the build context as `MCPP_*` variables | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | Where `mcpp pack` has already staged the closure, absolute; empty when this build is not packaging. Read it to decide the shape of the work; write `${mcpp.stage_dir}` into the action, so the path in the graph and the path the program read cannot disagree | | `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | `1` when the `mcpp pack` pass this program is part of strips, `0` under `--no-strip`; empty for every ordinary build. A member that stages libraries of its own follows it, so one switch governs every file in the package | | `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | Where `--debug-symbols` sends the separated `*.debug` files, absolute; empty when they are discarded or the build is not packing | +| `MCPP_PACK_COMPRESS` *(2026.11.2.3+)* | `mcpp::pack_compress()` | The compression the command (`--compress`) or `[pack] compress` stated, as the flag spells it (`gzip`, `zstd:19`, `none`); empty when neither states one or the build is not packing. A package format a package provides decides what its container does with it: openkal-platform's `okb` stores its images compressed under `gzip` | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | the device-kind sources (`.cu`, `.hip`, …) the package's effective `sources` match, package-root-relative, one per line; empty when there are none. The engine compiles none of them — the rule package this program imports turns each into an `mcpp::action`. Already narrowed: a `{ glob, accel }` entry the build does not cover contributes nothing, so `--no-accel` yields an empty list | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | a writable output directory owned by mcpp, one per configuration (target, profile, toolchain, accelerators, the package's features) and package, below the build's root: `target/.build-mcpp/out//` (2026.10.10.1+). A script outside the build reads it from `resolution.json`, `graph.packages[].outDir` | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | the package root (= CWD) | diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index f79fa3ba2..ae004a702 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -111,7 +111,8 @@ glibc 专有行为（NSS 查询、`dlopen` 宿主插件）时的首选。 | 组合 | 出路 | |---|---| | `--pkg-format single-file` 与 `--mode system` | `--mode vendored` 或 `self-contained`：单文件携带它所运行的东西 | -| `--compress` 与 `--pkg-format dir` 或提供者的格式 | `--pkg-format tar`、`zip` 或 `single-file`：只有容器被压缩 | +| `--compress` 与 `--pkg-format dir` | `--pkg-format tar`、`zip`、`single-file` 或提供者的格式：只有容器被压缩 | +| 提供者的格式不写的压缩（`--pkg-format okb --compress zstd`） | 它写的那一种（`okb`：`gzip`，映像压缩存放；或 `none`） | | `--compress zstd` 或 `xz` 与 `--pkg-format zip` | `--compress gzip`（DEFLATE）或 `none`，或 `--pkg-format tar` | | `--compress gzip` 或 `xz` 与 `--pkg-format single-file` | `--compress zstd[:]` 或 `none`：启动器解压 zstd | | 自带运行器的映像树与映像的单文件之外使用 `--systems` | `--mode self-contained`，或 `--pkg-format single-file`；不相容的 `[pack] systems` 写出提示后搁置 | @@ -741,7 +742,8 @@ Cygwin 的 shell 交给 `bin/.exe`，以该目录中的映像与自身的 **容器列表。** `--pkg-format tar,okb`（或 `[pack] format = ["tar", "okb"]`）构建一次、暂存一棵树，按命名的顺序 从它写出各容器。列表只含同一棵树的容器 —— `tar`、`zip` 与提供者的格式 —— 不含 `dir` 与 `single-file`；列表要求 -`-o` 为目录；`--compress` 作用于其中的引擎容器。okb 只含映像，因此格式只有 okb 的 self-contained 树被拒绝 +`-o` 为目录；`--compress` 作用于其中的引擎容器，并以 `MCPP_PACK_COMPRESS` 传给提供者的格式（2026.11.2.3）：openkal-platform 的 `okb` +在 `gzip` 下压缩存放其映像——磁盘上小的 okb，okrun 首次运行时展开到缓存——其余情况原样存放。okb 只含映像，因此格式只有 okb 的 self-contained 树被拒绝 （`--pkg-format tar,okb`）。`mcpp run --pkg-format` 只接受一个格式。 ### 单文件包 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 3d53cfa82..adc2d884f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -1158,6 +1158,7 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来，缓存 | `MCPP_PACK_STAGE_DIR` *(2026.9.11.1+)* | `mcpp::pack_stage_dir()` | `mcpp pack` 已经把闭包暂存到的位置，绝对路径；本次构建不在打包时为空。读它来判断这次要干的活是什么形状，而把 `${mcpp.stage_dir}` 写进 action —— 这样图里的路径与程序读到的路径不可能不一致 | | `MCPP_PACK_STRIP` *(2026.9.16.1+)* | `mcpp::pack_strip()` | 本程序所处的这次 `mcpp pack` 剥离时为 `1`，`--no-strip` 下为 `0`；任何普通构建下都为空。自己暂存库的成员跟随它，于是一个开关管住包里的每个文件 | | `MCPP_PACK_DEBUG_SYMBOLS_DIR` *(2026.9.16.1+)* | `mcpp::pack_debug_symbols_dir()` | `--debug-symbols` 放置分离出的 `*.debug` 文件的位置，绝对路径；丢弃调试信息或本次构建不在打包时为空 | +| `MCPP_PACK_COMPRESS` *(2026.11.2.3+)* | `mcpp::pack_compress()` | 命令（`--compress`）或 `[pack] compress` 陈述的压缩，拼写同该选项（`gzip`、`zstd:19`、`none`）；两者都未陈述或本次构建不在打包时为空。包提供的格式自行决定其容器如何处理它：openkal-platform 的 `okb` 在 `gzip` 下压缩存放其映像 | | `MCPP_DEVICE_SOURCES` *(2026.9.5.2+)* | `mcpp::device_sources()` | 本包有效 `sources` 匹配到的设备类源文件（`.cu`、`.hip`…），相对包根，一行一个；没有时为空串。引擎一个都不编译 —— 由本程序引入的规则包把每一个变成一条 `mcpp::action`。已经过收窄：构建未覆盖的 `{ glob, accel }` 条目贡献为空，因此 `--no-accel` 得到空列表 | | `MCPP_OUT_DIR` | `mcpp::out_dir()` | mcpp 提供的可写输出目录，按配置（目标、profile、工具链、加速器、该包的 features）与包各一个，位于构建的根之下：`target/.build-mcpp/out/<配置>/<包>`（2026.10.10.1+）。构建之外的脚本从 `resolution.json` 的 `graph.packages[].outDir` 读取它 | | `MCPP_MANIFEST_DIR` | `mcpp::manifest_dir()` | 包根（= CWD） | diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 76a5d4d14..0ed6db28e 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -251,6 +251,9 @@ struct BuildProgramEnv { // engine's. std::string packStrip; std::filesystem::path packDebugSymbolsDir; + // The compression the packaging pass stated (`--compress`, `[pack] + // compress`), as the flag spells it; empty when unstated or not packing. + std::string packCompress; // #647 E1: the graph document offered to the ROOT package's program, and // the digest of its text. Empty for a dependency's program. The digest // joins the re-run key (see `run_build_program`), so a change to what the @@ -736,6 +739,7 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_PACK_STAGE_DIR", env.packStageDir.string()); e.emplace_back("MCPP_PACK_STRIP", env.packStrip); e.emplace_back("MCPP_PACK_DEBUG_SYMBOLS_DIR", env.packDebugSymbolsDir.string()); + e.emplace_back("MCPP_PACK_COMPRESS", env.packCompress); e.emplace_back("MCPP_GRAPH_FILE", env.graphFile.string()); std::string csv; for (auto const& f : env.features) { diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 94c704dc3..439150d65 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -758,6 +758,12 @@ inline const char* pack_strip() { return env_or("MCPP_PACK_STR // when they are discarded or this build is not packing (#649 E5). inline const char* pack_debug_symbols_dir() { return env_or("MCPP_PACK_DEBUG_SYMBOLS_DIR"); } +// THE COMPRESSION THIS PACKAGING PASS STATED, as `--compress` spells it +// (`gzip`, `zstd:19`, `none`); "" when the command and `[pack] compress` state +// none, or this build is not packing (2026.11.2.3). A package format a package +// provides decides what its container does with it. +inline const char* pack_compress() { return env_or("MCPP_PACK_COMPRESS"); } + // THE RESOLVED DEPENDENCY GRAPH, as a JSON document; "" for a dependency's // program and under an older engine (#647 E1). // diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 4a4d3d7ca..a740e6480 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -976,6 +976,11 @@ export struct BuildOverrides { // files as it reaches the engine's. std::string strip; std::filesystem::path debugSymbolsDir; + // The compression the command or `[pack] compress` STATED, as + // `--compress` spells it (`gzip`, `zstd:19`, `none`); empty when + // nothing was stated. A format a package provides decides what its + // container does with it (an okb compresses its images under `gzip`). + std::string compress; }; // The packed members' stages, by qualified package name. One entry is what a // pack of one member sets, and then every program of the plan receives it, diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 555125f4f..d4949b9f5 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -457,6 +457,7 @@ struct PrepareState { env.packStageDir = stage->dir; env.packStrip = stage->strip; env.packDebugSymbolsDir = stage->debugSymbolsDir; + env.packCompress = stage->compress; } } // `--features` as the command gave it. A workspace plan hands the tokens to diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 3b1223ef7..6cb3925ef 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -791,9 +791,15 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, const bool listsEngine = std::ranges::any_of(formatList, [](std::string const& f) { return f == "tar" || f == "zip"; }); const bool listsZip = std::ranges::find(formatList, "zip") != formatList.end(); + // A FORMAT A PACKAGE PROVIDES decides what its container does with the + // compression stated, which reaches its program as MCPP_PACK_COMPRESS + // (2026.11.2.3): openkal-platform's `okb` stores its images compressed + // under gzip, and refuses what it cannot write. + const bool listsProvided = std::ranges::any_of(formatList, [](std::string const& f) { + return f != "tar" && f != "zip" && f != "dir" && f != "single-file" && f != "polyglot"; }); const bool writesNone = opts.formats.empty() - ? (opts.format == mcpp::pack::Format::Dir || opts.format == mcpp::pack::Format::Dispatched) - : !listsEngine; + ? opts.format == mcpp::pack::Format::Dir + : !listsEngine && !listsProvided; const bool zipped = opts.format == mcpp::pack::Format::Zip || (!opts.formats.empty() && listsZip); const bool fits = !writesNone && !(zipped && (alg == Algorithm::Zstd || alg == Algorithm::Xz)) @@ -803,7 +809,7 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, if (writesNone) { mcpp::ui::error(std::format( "--compress {} compresses a container, and --pkg-format {} writes none.\n" - " use: --pkg-format tar | zip | single-file", + " use: --pkg-format tar | zip | single-file, or a format a package provides", mcpp::pack::compress::name(alg), opts.format == mcpp::pack::Format::Dir ? std::string("dir") : opts.formatName)); return 2; diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 2919cbff6..e129d19f9 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -25,6 +25,7 @@ import mcpp.manifest; import mcpp.pack; import mcpp.pack.stage_tree; import mcpp.pack.strip; +import mcpp.pack.compress; import mcpp.runtime.launch_env; import mcpp.toolchain.model; import mcpp.toolchain.probe; @@ -1341,6 +1342,10 @@ std::optional dispatch_group(PackRun& run, GroupJob& g, mcpp::build::Ba // switch `--no-strip` and `--debug-symbols` set for this tree. stage.strip = m.plan->strip ? "1" : "0"; stage.debugSymbolsDir = m.plan->debugDir; + if (run.opts.compression.stated) { + stage.compress = std::string(mcpp::pack::compress::name(run.opts.compression.algorithm)); + if (run.opts.compression.level >= 0) stage.compress += std::format(":{}", run.opts.compression.level); + } g.ov.pack_stages[m.name] = std::move(stage); } }; diff --git a/tests/e2e/1008_a_provided_format_is_told_the_compression.sh b/tests/e2e/1008_a_provided_format_is_told_the_compression.sh new file mode 100755 index 000000000..60c1a3b17 --- /dev/null +++ b/tests/e2e/1008_a_provided_format_is_told_the_compression.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# requires: pack +# 1008 -- a format a package provides is told the compression the command or +# `[pack] compress` stated (MCPP_PACK_COMPRESS, `mcpp::pack_compress()`), and +# decides what its container does with it (2026.11.2.3). +# +# `--compress` was refused with every provided format ("writes none"), so a +# provider whose container can be compressed --- openkal-platform's okb, which +# stores its images compressed under `gzip` --- had no way to learn it. +# +# Legs: +# A. `--pkg-format probe --compress gzip:9`: the program reads `gzip:9`. +# B. `--pkg-format probe`: it reads nothing. +# C. `[pack] compress = "zstd"`: it reads `zstd`. +# D. `--pkg-format dir --compress gzip` is still refused: a directory writes +# no container. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p app/src +cat > app/mcpp.toml <<'EOF2' +[package] +name = "app" +version = "0.1.0" +standard = "c++20" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF2 +printf 'int main() { return 0; }\n' > app/src/main.cpp +cat > app/build.mcpp <<'EOF2' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("probe"); + if (std::string_view(mcpp::pack_format()) != "probe") return 0; + const std::string out = std::string(mcpp::out_dir()) + "/probe.txt"; + const std::string text = std::string("compress=") + mcpp::pack_compress(); + mcpp::action a; + a.id = "probe"; + a.role = "artifact"; + a.description = "probe"; + a.arg("/bin/sh").arg("-c").arg(("printf '%s' '" + text + "' > " + out).c_str()) + .input("${mcpp.target_file:app}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF2 +cd app + +# ── A ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --pkg-format probe --compress gzip:9 > a.log 2>&1 || fail "A: pack failed" a.log +probe=$(find target -name probe.txt | head -1) +[ "$(cat "$probe")" = "compress=gzip:9" ] || fail "A: the program read '$(cat "$probe" 2>/dev/null)'" a.log +echo "ok: A, a provided format is told the compression the command states" + +# ── B ─────────────────────────────────────────────────────────────────────── +"$MCPP" pack --pkg-format probe > b.log 2>&1 || fail "B: pack failed" b.log +[ "$(cat "$probe")" = "compress=" ] || fail "B: with none stated the program read '$(cat "$probe")'" b.log +echo "ok: B, and nothing when none is stated" + +# ── C ─────────────────────────────────────────────────────────────────────── +printf '\n[pack]\ncompress = "zstd"\n' >> mcpp.toml +"$MCPP" pack --pkg-format probe > c.log 2>&1 || fail "C: pack failed" c.log +[ "$(cat "$probe")" = "compress=zstd" ] || fail "C: under [pack] compress the program read '$(cat "$probe")'" c.log +echo "ok: C, and the one [pack] compress states" + +# ── D ─────────────────────────────────────────────────────────────────────── +if "$MCPP" pack --pkg-format dir --compress gzip > d.log 2>&1; then fail "D: a directory was compressed" d.log; fi +grep -q "writes none" d.log || fail "D: the refusal" d.log +echo "ok: D, a directory writes no container and is refused" From 6c1ab22351106dc0469a93ce2f85226711c7c101 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 09:52:29 +0900 Subject: [PATCH 05/14] mcpp cache gc asks the newest okrun it built to reclaim the runner's own cache (okrun cache gc) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 5 ++ docs/09-commands-by-scenario.md | 5 +- docs/zh/09-commands-by-scenario.md | 4 +- src/bmi_cache/maintenance.cppm | 59 ++++++++++++++++++++++ tests/e2e/1009_cache_gc_asks_the_runner.sh | 48 ++++++++++++++++++ 5 files changed, 119 insertions(+), 2 deletions(-) create mode 100755 tests/e2e/1009_cache_gc_asks_the_runner.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 0187d6add..db48d6918 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,6 +30,11 @@ is left for the next one; a requirement a layer decides is checked; design compressed under `gzip`, an okb kept small on disk that okrun expands into its cache when it first runs it (design 2026-11-03 part 4 §16). A directory is still refused. +- **`mcpp cache gc` reclaims the ok-image runner's cache too** (docs/09). It + asks the newest `okrun` among the host tools it built to run `okrun cache + gc`: the prepared copies and snapshots openkal-image 0.4 keeps for an image + in the user's runtime or cache directory, of another boot. The layout is the + runner's, and so is its reclaiming (design 2026-11-03 part 4 §16). ### Fixed diff --git a/docs/09-commands-by-scenario.md b/docs/09-commands-by-scenario.md index 4490a57ac..9d5f0cc7e 100644 --- a/docs/09-commands-by-scenario.md +++ b/docs/09-commands-by-scenario.md @@ -88,7 +88,10 @@ key kind size last used package package and host tool entries. A `std` BMI is shared by every project on the machine, and the implementation excludes it from size-driven eviction on the grounds that rebuilding one trades a lot of time for a little disk. `mcpp cache -clean --std` remains the explicit way to remove it. +clean --std` remains the explicit way to remove it. It then asks the newest +`okrun` among the host tools it built to reclaim the runner's own cache (`okrun +cache gc`): an ok image's prepared copies and snapshots for another boot +(openkal-image 0.4, 2026.11.2.3+). After a successful build, at most once a day, every entry -- package, host tool or `std` -- that no build has used for `[cache] unused_days` days (config.toml; diff --git a/docs/zh/09-commands-by-scenario.md b/docs/zh/09-commands-by-scenario.md index caa04c019..4d71fbd21 100644 --- a/docs/zh/09-commands-by-scenario.md +++ b/docs/zh/09-commands-by-scenario.md @@ -76,7 +76,9 @@ key kind size last used package `mcpp cache gc` 要求给出 `--max-size`、`--older-than` 或两者，驱逐包条目与 host 工具条目。一份 `std` BMI 被机器上每个工程共享，实现以"重建它是用大量时间 换少量磁盘"为由，把它排除在按体积驱逐之外。`mcpp cache clean --std` 仍是 -显式移除它的做法。 +显式移除它的做法。随后它请它构建过的 host 工具中最新的 `okrun` 回收运行器自己的 +缓存（`okrun cache gc`）：ok 映像为其他开机准备的副本与快照（openkal-image 0.4， +2026.11.2.3+）。 构建成功后，每天至多一次，移除 `[cache] unused_days` 天（config.toml；默认 30， `0` 关闭）内没有构建使用过的条目，包、host 工具与 `std` 均在其列；命中即记为使用。 diff --git a/src/bmi_cache/maintenance.cppm b/src/bmi_cache/maintenance.cppm index 68ee5b320..9019ba113 100644 --- a/src/bmi_cache/maintenance.cppm +++ b/src/bmi_cache/maintenance.cppm @@ -30,6 +30,7 @@ import mcpp.home; import mcpp.libs.json; import mcpp.libs.toml; import mcpp.ui; +import mcpplibs.os.process; namespace mcpp::bmi_cache { @@ -528,6 +529,63 @@ export int cache_prune(const std::string& v) { return 0; } +// THE RUNNER'S OWN CACHE (design 2026-11-03 part 4 §16): an ok image's +// prepared copies and snapshots for this boot, and the okb members it expanded, +// which okrun keeps beside the user's runtime or cache directory +// (openkal-image 0.4, `okrun cache`). Its layout is the runner's, so the runner +// reclaims it --- what another boot prepared, and snapshots over their bound --- +// and mcpp asks the newest okrun among the host tools it built. Where it built +// none there is nothing of the runner's that mcpp put on this machine. +std::filesystem::path newest_runner() { + auto parts = [](std::string_view v) { + std::vector out; + for (auto part : std::views::split(v, '.')) { + int n = 0; + std::from_chars(part.data(), part.data() + part.size(), n); + out.push_back(n); + } + return out; + }; + std::filesystem::path best; + std::vector bestVersion; + std::error_code ec; + for (auto& pkg : std::filesystem::directory_iterator(tool_root() / "mcpplibs", ec)) { + const auto name = pkg.path().filename().string(); + if (!name.starts_with("openkal-tools@")) continue; + const auto version = parts(std::string_view(name).substr(14)); + if (!best.empty() && version <= bestVersion) continue; + std::error_code ke; + for (auto& key : std::filesystem::directory_iterator(pkg.path(), ke)) { + for (const char* program : {"okrun", "okrun.exe"}) { + const auto candidate = key.path() / "bin" / program; + std::error_code fe; + if (!std::filesystem::is_regular_file(candidate, fe)) continue; + best = candidate; + bestVersion = version; + } + } + } + return best; +} + +void reclaim_runner_cache() { + const auto okrun = newest_runner(); + if (okrun.empty()) return; + namespace proc = mcpplibs::os::process; + proc::spawn_options o; + o.program = okrun.string(); + o.args = {"cache", "gc"}; + auto r = proc::run(std::move(o)); + if (!r || !r->status.success()) { + mcpp::ui::warning(std::format("`{} cache gc` did not reclaim the runner's cache{}", + okrun.string(), r ? ": " + r->err : std::string())); + return; + } + std::string said = r->out.empty() ? r->err : r->out; + while (!said.empty() && (said.back() == '\n' || said.back() == '\r')) said.pop_back(); + mcpp::ui::status("Reclaimed", said.empty() ? std::string("the ok-image runner's cache") : said); +} + // `mcpp cache gc [--max-size N] [--older-than N]` — real LRU, driven by // entry.json's `accessed` stamp. export int cache_gc(const std::string& maxSizeArg, const std::string& olderThanArg) { @@ -593,6 +651,7 @@ export int cache_gc(const std::string& maxSizeArg, const std::string& olderThanA // std BMIs sit right next to it. std::println("Collected {} entries, freed {} (package and tool entries now {})", removed, human_bytes(freed), human_bytes(live)); + reclaim_runner_cache(); if (maxSize && live > *maxSize) { // Say it rather than silently under-delivering: a size target that // cannot be met without evicting std BMIs is a real outcome, and a diff --git a/tests/e2e/1009_cache_gc_asks_the_runner.sh b/tests/e2e/1009_cache_gc_asks_the_runner.sh new file mode 100755 index 000000000..d6420c368 --- /dev/null +++ b/tests/e2e/1009_cache_gc_asks_the_runner.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 1009 -- `mcpp cache gc` asks the newest okrun among the host tools it built +# to reclaim the runner's own cache (`okrun cache gc`, openkal-image 0.4; +# design 2026-11-03 part 4 §16; 2026.11.2.3). +# +# The prepared copies and snapshots an ok image's runner keeps are laid out by +# the runner, so the runner reclaims them. Two openkal-tools entries stand in +# here: the older one must not be asked, the newer one is asked `cache gc` and +# what it says is reported. A home with no okrun asks nothing. +set -e +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME="$TMP/mcpp-home" +tools="$MCPP_HOME/build-cache/v1/tool/mcpplibs" + +# A home that built no okrun: gc collects as before and asks no runner. +mkdir -p "$MCPP_HOME/build-cache/v1/tool" +"$MCPP" cache gc --older-than 3650d > none.log 2>&1 || { cat none.log; exit 1; } +if grep -q 'Reclaimed' none.log; then + echo "FAIL: a home without okrun reported a runner's cache"; cat none.log; exit 1 +fi + +runner() { # runner + local bin="$tools/openkal-tools@$1/$2/bin" + mkdir -p "$bin" + cat > "$bin/okrun" < "$TMP/asked-$1" +echo "$3" >&2 +SH + chmod +x "$bin/okrun" +} +runner 0.3.2 aaaa "okrun: the older runner" +runner 0.4.1 bbbb "okrun: removed 7 bytes prepared for another boot" +runner 0.4.0 cccc "okrun: the middle runner" + +"$MCPP" cache gc --older-than 3650d > gc.log 2>&1 || { cat gc.log; exit 1; } +[[ "$(cat "$TMP/asked-0.4.1" 2>/dev/null)" == "cache gc" ]] || { + echo "FAIL: the newest okrun was not asked \`cache gc\`"; cat gc.log; exit 1 +} +[[ ! -e "$TMP/asked-0.3.2" && ! -e "$TMP/asked-0.4.0" ]] || { + echo "FAIL: an older okrun was asked"; cat gc.log; exit 1 +} +grep -q 'Reclaimed.*removed 7 bytes prepared for another boot' gc.log || { + echo "FAIL: what the runner reclaimed is not reported"; cat gc.log; exit 1 +} +echo "OK" From e2c3fa8d986a7ea0c6598eaac868524e35150f59 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 10:07:56 +0900 Subject: [PATCH 06/14] design 2026-11-03 part 4 (beyond native): the design and its implementation record; part 3 rev 5 (the null device of a started program) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...art3-ok-image-performance-parity-design.md | 14 + ...03-ecosystem-part4-beyond-native-design.md | 699 ++++++++++++++++++ 2 files changed, 713 insertions(+) create mode 100644 .agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md diff --git a/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md b/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md index f3f13eb2b..44c107dbe 100644 --- a/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md +++ b/.agents/docs/2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md @@ -26,6 +26,9 @@ openkal-llvm-runtime 0.16.15、openkal-image 0.1.9）。前文：[part 2](2026-1 不一致。第三轮自我评审 17 条（§15.3），待评审项见 §16.4；四项均采纳（§16.5），本文定稿，按 §12 实施。 - rev 4：实施中的两项决定（§16.6）与修订（§17）：冷阶段的差距来自装入时复制映像的全部段，openkal 0.21 增加从文件映射指令的 可选操作；L1 改为报告，门禁在 L2、L3。 +- rev 5：Windows 上的映像以 worker 构建模块时全部失败：worker 的标准输入、输出被重定向到 `/dev/null`，映像的 C 库找不到这个名字。 + 决定（§16.7）：空设备是环境的资源，openkal 0.22 在启动的程序的位置上提供它；`/dev/null` 这个名字与进程内的空设备属于 C 库 + （openkal-musl 0.27）。版本组见 §17.5。 ## 1. 问题与结论 @@ -590,6 +593,17 @@ mcpp 在首次初始化时已把 Ninja 装进自己的 home（`ensure_ninja`，` 2. L1 在 Linux x64 上 69 行中 hosted 仍有 5 行、映像 16 行超过 1.15（§17.4），多为纳秒级的字符串函数；L2、L3 达标。决定：L1 在 CI 中照常运行并报告比值，不作门禁；门禁在 L2、L3。 +### 16.7 实施中的待决项与决定（2026-11-03） + +1. 引擎自 0039 起按实测的能力选择路径，Windows 上的映像因此第一次以 worker 构建模块，每个 worker 的启动都以 `ENOENT` 失败 + （warm-prime win32 的映像会话全部未就绪；mcppls 的 VS Code、Neovim 与 clean machine 在 Windows 上的模块功能全部失败）。LLVM 的 + `ExecuteNoWait` 把空路径的重定向写成 `/dev/null`；映像的 C 库按 preopen 解析名字，Windows 的 preopen 是卷，没有 `/`。 + 决定：两层各做自己才能做的事。空设备是每个启动程序的环境都有的资源（Linux、macOS 为 `/dev/null`，Windows 为 `NUL`）； + 交给启动的程序时需要环境的句柄，C 库只能以调用者持续读空的通道近似它（调用者先结束时写入失败，每个字节经过管道，每个位置 + 占用一个任务，程序看到的是管道），mcpplibs.os 0.2.7 的 `stdio::null` 即是这样近似的。openkal 0.22 以启动的标志 + `KAL_SPAWN_NULL_IN/OUT/ERR` 在位置上给出空设备；openkal-musl 0.27 把 POSIX 的名字 `/dev/null` 解析为进程内的空描述符， + 并在 `posix_spawn` 中把它映射为这些标志。openkal 的改动在 CI 通过后另做一轮多角度评审：规范、ABI、设计意图与架构不被破坏。 + ## 17. 实现中的修订 ### 17.1 openkal 0.21 与三个实现 diff --git a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md new file mode 100644 index 000000000..4b1713cca --- /dev/null +++ b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md @@ -0,0 +1,699 @@ +--- +subject: openkal-performance +status: active +--- + +# 生态设计 part 4：超越原生——openkal 之上与之下的优化、可观测性与安全 + +日期：2026-11-03（rev 3，定稿）。基线：openkal 0.23 分支（`openkal-0.23`：`kal_exec_alloc_at`、placement note、 +`kal_fs_resolved_name`）、openkal-image 0.4.0 分支、openkal-musl 0.27.0 分支、mcppls-clangd #16（引擎 23.1.0-mcppls.9）。 +前文：[part 3](2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md) §16.7、§16.8、§17。 + +**修订记录** + +- rev 1：综合 2026-11-03 的讨论与测量。已定：放置之外的五个方向（A 最小授予、A' 不变目录、B 精确输入集、C 按触页顺序排布、 + D 初始化后快照）全部实现并入本轮；Windows 的文件页映射与 hosted 启动成本两项对等缺口并入本轮；调试、性能分析、可观测性与 + 安全同轮完成。分层原则（§3）：openkal ABI 是内核接口，实现层是内核，程序只经由 openkal 请求与使用能力；不以运行器拦截 + `kal_*` 调用的方式实现功能。 +- rev 2：§17 的四项按推荐定下（§17.1），本文定稿，按 §15 实施。 +- rev 3：资源偏好（§4.8、§16）：程序的资源偏好 balanced（默认，自适应）、performance（以空间换时间）、economy（以时间换空间）， + 由 openkal 表达、各层兑现；可用内存的查询。并入本轮。 + +--- + +## 1. 结论 + +1. 放置（part 3 §16.8）之后，clangd 映像一次启动的处理器时间与原生相同（0.77 ms 对 0.76–0.85 ms），内核态少 13%。 + 此前看到的 2 倍差距中，一半来自 0.4.0 的一处回退：加载器每次装入都执行 `okld_check`，新增的 placement 内容校验占用户态 + 指令的 57%；内容校验移到封装时的 `okld check` 后差距消失（§2.1）。 +2. 余下的差距不在处理器时间而在缺页（519 对 284）与 warm 阶段的文件系统查询。缺页的三分之一是静态初始化写满的 + `.data`/`.bss`，三分之一是两层分配器的堆（§2.2）；warm 阶段约 30 万次失败的查找来自 clang 驱动对宿主 GCC 安装的探测， + 原生与映像相同（§2.3）。 +3. 这些差距原生程序无法消除，映像体系可以：映像对环境的一切访问都经过 openkal，加载器、C 库与封装都在体系之内。 + 本文以 openkal 的能力（§4）、映像格式（§5）与各层的实现（§6–§9）给出 A、A'、B、C、D，并把调试、性能分析、可观测性 + （§10）与安全（§11）放在同一组能力上。 +4. 三项安全缺口与原生相比是退步，本轮补齐：重定位后的只读数据（RELRO）仍可写；放置使每个映像的地址全球固定；映像未以 + 栈保护等加固选项编译（§11）。 + +## 2. 测量 + +### 2.1 启动（本机 x86-64，`taskset -c 6`，P 核，`perf stat -r 100`，`clangd --version`） + +| | task-clock | 用户态周期 | 内核态周期 | 缺页 | +| --- | --- | --- | --- | --- | +| 原生（非 PIE） | 0.76–0.85 ms | 1.09 M | 3.00 M | 284 | +| 映像，放置于基址（0.4.0 修正后） | 0.77 ms | 1.53 M | 2.63 M | 532 | +| 映像，任意地址（`OKRUN_PLACE=0`） | 1.54 ms | 3.40 M | 4.97 M | 1 103 | +| 映像，放置于基址（修正前） | 1.52 ms | 4.90 M | 3.37 M | 594 | + +修正前的用户态指令为 1 740 万条，其中 57% 在 `okld_check`：`okld_load` 每次装入都执行完整的格式检查，0.4.0 在其中加入了 +对存储段逐字的校验（16.4 万条 RELR 的展开与 2 MB 的比较）。修正：`okld_check` 只检查 placement 描述映像，内容校验成为 +`okld_check_placement`，由封装时的 `okld check` 与 `okld bundle check` 调用。存储段与映像的指令同样可信，运行时的完整性由 +签名或 okb 的摘要负责。 + +### 2.2 缺页的构成（同一启动，`perf record -e page-faults -d`，519 次） + +| 位置 | 缺页 | 说明 | +| --- | --- | --- | +| 映像 `.data`/`.bss` | 179 | 静态初始化写入（LLVM 的 `cl::opt` 对象）；加载器另将 `.bss` 全部填零 | +| 堆与匿名映射 | 176 | okrun 的分配器、映像的 mimalloc、openkal-linux 的 `kal_alloc` | +| 映像指令（文件页） | 84 | | +| 映像只读数据（文件页） | 31 | | +| RELRO（存储段与复制的首尾块） | 28 | | +| okrun 与栈 | 21 | | + +hosted 形态的 okrun（openkal-musl，静态）单独启动 185 次缺页，glibc 动态链接的同一 okrun 为 73 次；task-clock 相同（0.18 ms）。 +其中 85 次来自 `okm_table_init` 对 1 024 个描述记录的逐个写入（340 KiB 的 `.bss`）。 + +### 2.3 warm 阶段的文件系统查询（`strace -f -c`，3 个会话，CI probe 的 L3 数据） + +| | 原生 | 映像（0.3.1） | +| --- | --- | --- | +| stat 类 | 35.3 万（失败 18.9 万） | 87.3 万（失败 19.1 万） | +| `openat` | 18.2 万（失败 10.5 万） | 18.8 万（失败 10.5 万） | +| `readlink` | 9.9 万 | — | + +映像多出的 50 万次 stat 来自 realpath 的逐段询问，0.23 的 `kal_fs_resolved_name` 已去掉。两者共有的约 30 万次失败查找落在 +`/usr/lib/gcc/x86_64-linux-gnu/{9,11,12,13}/include`、`/usr/include/c++`、`/usr/lib32`、`/etc` 等：编译命令以 +`--sysroot=~/.mcpp/registry/subos/default` 使用 mcpp 的 gcc 16，clang 驱动仍在每个 worker、每条命令上探测宿主的 GCC 安装。 +探测的结果还使 LSP 的结论依赖宿主装了什么，与构建不一致。 + +### 2.4 静态初始化的份额 + +映像自身的代码占 okrun 进程用户态周期的约 35%（约 0.11 ms），其中约三分之一是 musl 与 mimalloc 的初始化（描述表、逐项 +`getenv` 的选项解析），其余是 LLVM 的静态初始化（约 0.07 ms）。D 的收益因此主要在缺页与内存：初始化写满的 `.data`/`.bss` +与初始化期间的堆成为文件页，按需读入、在进程间共享（每个 worker 约 1.5 MB 的私有页）。 + +### 2.5 现有缺口 + +| 缺口 | 现状 | +| --- | --- | +| RELRO | 加载器重定位后不改变可写段的保护，GOT 与 `.data.rel.ro` 在运行时可写；原生的 `-z now` + RELRO 使其只读 | +| 地址 | 放置使一个映像的地址在所有机器上相同，等同非 PIE；原生发行版的程序多为 PIE + ASLR | +| 加固 | 平台的编译选项有栈冲突保护，无栈保护、无自动变量初始化、无 CET/BTI 标记、无 libc++ 加固 | +| 回溯 | 映像不保留帧指针；perf 的 DWARF 回溯止于映像内的叶子帧，okrun 没有崩溃报告 | +| Windows 文件页 | openkal-windows 不声明 `KAL_EXEC_PROP_FILE`，每次启动复制全部 37 MB 指令 | +| Windows stat | 一次成功的 `kal_fs_info`：`NtCreateFile`、四次 `NtQueryInformationFile`、`NtClose` | +| 观测 | okprof 计 `kal_*` 调用与计数器；没有采样、触页、阶段与跨进程的时间线；macOS、Windows 上系统的采样器不认识映像 | + +## 3. 分层与原则 + +1. **openkal ABI 是内核接口。** 实现层（openkal-linux、-macos、-windows）对上即内核：今天是进程内的库，将来可以是直接实现 + openkal 的内核。一项需要环境配合的能力表达为 openkal 的操作、标志或属性，由实现层完成；同一映像与同一组请求在两种内核上 + 含义相同。 +2. **映像格式是加载器的契约。** 加载器今天是 okrun/libokld，将来可以是内核的程序装入。放置、触页集合、快照以第 12 条的 + note 表达。 +3. **程序与 C 库内部的优化不需要下层。** 描述表的惰性初始化、realpath 等属于此类。 +4. **不以运行器拦截 `kal_*` 实现功能。** 记录、缓存、只读都是能力的属性，由实现层在每次调用中完成。okprof 对 `kal_*` 调用的 + 计数是观测 ABI 边界本身，属于边界的绑定者，保留。 +5. **系统调用合并的判据。** openkal 的边界今天是函数调用，将来可能是真实的系统调用；一个常见操作应一次跨越。合并只在减少 + 内核切换、句柄创建或往返时采用（§12）。 +6. **能力以属性位声明，缺席有确定的语义。** 每项新能力：实现声明则可用；未声明时请求被拒绝或作为不生效的提示，规范写明是哪 + 一种。 +7. **证明而不是假设。** 快照、放置缓存、记录等以可检查的条件成立；条件不成立时回到普通路径，而不是报错或冒险。 + +## 4. openkal 0.23 的增补（ABI） + +0.23 已有：`kal_exec_alloc_at`、`KAL_EXEC_PROP_PLACE`（11 条 48）、`kal_fs_resolved_name`、`KAL_FS_PROP_RESOLVED_NAME`（49）。 +本文增补如下，均进入 0.23.0，条文为 11 条 50–57。 + +### 4.1 exec 区域为零（陈述，无新名字，11 条 50） + +`kal_exec_alloc` 与 `kal_exec_alloc_at` 返回的区域，字节为零。三个实现都以新映射的页满足（`mmap` 匿名、`VirtualAlloc` 提交）。 +加载器据此不再写 `.bss`：未触及的 `.bss` 页不驻留，与原生相同。 + +### 4.2 只读封存：`kal_exec_seal_part`（安全，11 条 51） + +```c +/* Makes a part of a region read-only and not executable: a loader's + * read-only-after-relocation data once relocated. KAL_EXEC_SEAL_FOREVER asks + * that neither the protection nor the mapping change again for the program's + * life, where the environment can hold that (KAL_EXEC_PROP_SEAL_FOREVER); + * elsewhere the part is read-only as asked. Version 0.23. */ +int kal_exec_seal_part(void* region, kal_uintptr offset, kal_uintptr size, kal_uintptr flags); +#define KAL_EXEC_SEAL_FOREVER ((kal_uintptr)1u << 0) +#define KAL_EXEC_PROP_SEAL_FOREVER ((kal_uintptr)1u << 4) +``` + +| 实现 | 只读 | `FOREVER` | +| --- | --- | --- | +| Linux | `mprotect(PROT_READ)` | `mseal`（6.10 起；属性按内核在运行时判定） | +| macOS | `mprotect(PROT_READ)` | `mach_vm_protect(set_maximum=TRUE, VM_PROT_READ)` | +| Windows | `VirtualProtect(PAGE_READONLY)` | 不声明 | + +加载器对入口映像（不卸载）请求 `FOREVER`，对 `dlopen` 的映像只请求只读（卸载要释放区域）。 + +### 4.3 预取与 BTI：`KAL_EXEC_MAP_PREFETCH`、`KAL_EXEC_MAP_BTI`（C、安全，11 条 52） + +`kal_exec_map_part` 的 `how` 增加两位： + +```c +#define KAL_EXEC_MAP_PREFETCH ((kal_uintptr)1u << 1) /* the part's bytes are read in now */ +#define KAL_EXEC_MAP_BTI ((kal_uintptr)1u << 2) /* executed with branch-target enforcement */ +#define KAL_EXEC_PROP_BTI ((kal_uintptr)1u << 5) /* KAL_EXEC_MAP_BTI is enforced */ +``` + +`PREFETCH` 把逐页的缺页合并为一次请求：Linux `MAP_POPULATE`，macOS 映射后 `madvise(MADV_WILLNEED)`，Windows +`PrefetchVirtualMemory`。它是提示，不改变语义。`BTI` 只对带 `EXECUTE` 的部分有意义：Linux arm64 内核支持时以 `PROT_BTI` +映射并声明 `KAL_EXEC_PROP_BTI`，其余环境忽略该位。 + +### 4.4 能力衰减与隔离：`kal_fs_dir_attenuate`、`KAL_SPAWN_CONFINE`（A、A'、B、安全，11 条 53） + +```c +struct kal_dir_rights { + kal_u32 self_size; /* set by the caller (clause 5.3) */ + kal_u32 reserved; /* zero */ + kal_uintptr flags; /* KAL_DIR_* */ + struct kal_file record; /* KAL_DIR_RECORDED: where reached names are appended */ +}; +#define KAL_DIR_READ_ONLY ((kal_uintptr)1u << 0) /* nothing beneath is created, written, removed or renamed */ +#define KAL_DIR_UNCHANGING ((kal_uintptr)1u << 1) /* the caller states nothing beneath changes while the handle lives */ +#define KAL_DIR_RECORDED ((kal_uintptr)1u << 2) /* each name reached beneath is appended to `record' */ + +int kal_fs_dir_attenuate(struct kal_dir d, const struct kal_dir_rights* rights, struct kal_dir* out); + +#define KAL_FS_PROP_ATTENUATE ((kal_uintptr)1u << 12) /* kal_fs_dir_attenuate is answered */ +#define KAL_SPAWN_CONFINE ((kal_uintptr)1u << 5) /* the environment holds the started program + * to its grants and their rights */ +#define KAL_PROCESS_PROP_CONFINE ((kal_uintptr)1u << 14) /* KAL_SPAWN_CONFINE is honoured */ +``` + +- 派生的句柄只能更少：属性只增不减，`READ_ONLY` 不可撤销；由它打开的目录继承属性。 +- 经 `kal_spawn.grants` 授予时属性随之传递，启动的程序在 `kal_fs_props` 中读到 `KAL_FS_PROP_READ_ONLY`、 + `KAL_FS_PROP_UNCHANGING`（新位 13、14）。 +- **只读**由实现层在每个修改操作中强制（`kal_err_permission`）。 +- **隔离**是启动时的请求：带 `KAL_SPAWN_CONFINE` 启动的程序，由环境把它限制在授予的目录与各自的权限之内，即使它不经由 openkal + 访问文件；声明 `KAL_PROCESS_PROP_CONFINE` 的实现才接受该标志，其余实现拒绝（`kal_err_not_supported`），不静默降级。 + Linux 以 Landlock（5.13 起）在启动的程序执行之前设定规则；macOS 以 `sandbox_init_with_parameters` 的 SBPL 规则；Windows 不声明。 + 隔离由调用者请求而不是随授予自动生效：一个动态链接的原生程序需要授予之外的运行时目录，只有知道这一点的调用者才请求它。 + mcppls 启动引擎（静态的 okrun 与 payload 中的映像）时请求隔离：被恶意仓库利用的语言服务器只能触及项目、payload 与缓存， + 读不到 `~/.ssh`。 +- **不变**是授予者的陈述。实现可以从已读到的答案回答此后的查询：目录列表作答不存在的名字，已取得的属性作答 stat。缓存在 + 实现层，对上透明；原生 openkal 内核以同一陈述省去重新验证。 +- **记录**：每个经由该句柄（及其下打开的目录）到达的名字，以 `<种类><名字>\0` 追加到 `record`，种类为 `r`（为读打开）、 + `w`（写、创建、删除、改名）、`s`（询问，存在）、`a`（询问或打开，不存在）、`l`（列举）；名字相对于被衰减的目录。实现 + 在进程内去重并成批写入，程序结束时写 `e\0`；没有 `e` 的记录是不完整的，使用者按"未知"处理。多个进程向同一文件追加 + （`O_APPEND`、`FILE_APPEND_DATA`），每条不超过一次原子写。 + +### 4.5 带属性的列举：`kal_fs_list_next_info`（A'、合并，11 条 54） + +```c +/* kal_fs_list_next, and what the listing told of the entry: the fields + * `wanted' asks for that the environment gave with the entry are filled and + * stated in `info->present'; a field it did not give is asked of + * kal_fs_info. KAL_FS_PROP_LIST_INFO: the listing gives size, modified time + * and identity without a further enquiry. Version 0.23. */ +int kal_fs_list_next_info(struct kal_dir, kal_uintptr* iter, char* name_out, kal_uintptr name_cap, + kal_uintptr* name_len, kal_u32 wanted, struct kal_node_info* info); +#define KAL_FS_PROP_LIST_INFO ((kal_uintptr)1u << 15) +``` + +macOS 以 `getattrlistbulk`、Windows 以 `NtQueryDirectoryFile(FileIdBothDirectoryInformation)` 一次给出一批条目的全部属性； +Linux 的 `getdents64` 只有类型，`wanted` 要求其余字段时逐项 `statx`（不声明该位，但仍是一次 ABI 跨越）。 + +### 4.6 页面用量：`KAL_PINFO_FAULTS`（观测，11 条 55） + +`kal_program_info` 增加缺页的两个计数（`faults_minor`、`faults_major`，追加在结构末尾，`self_size` 判定）。Linux +`getrusage`/`/proc/self/stat`，macOS `task_events_info.faults`，Windows `PROCESS_MEMORY_COUNTERS.PageFaultCount`（只有总数， +`major` 不声明）。okprof 以它给出每个阶段的缺页。 + +### 4.7 故障报告与采样（调试、观测，11 条 56） + +`openkal.debug` 与 `openkal.prof` 至今由加载器提供给映像（12.3）。故障与采样需要环境的机制（信号或异常处理、 +处理器时间定时器、挂起线程），属于实现层：0.23 起三个实现也向本环境的程序提供这两个接口，okrun 即其中之一；okrun 向映像 +提供它们时，计数与阶段由自己记录，采样转交给它的实现（okrun 绑定给映像的是 `own_kal_*`，与实现导出的名字不冲突）。 + +```c +/* openkal.debug */ +#define KAL_DEBUG_PROP_FAULT_REPORT ((kal_uintptr)1u << 0) +kal_uintptr kal_debug_props(void); + +/* openkal.prof */ +#define KAL_PROF_PROP_SAMPLES ((kal_uintptr)1u << 0) +#define KAL_PROF_SAMPLE_DEPTH 64u +struct kal_prof_sample { + kal_u64 time_ns; /* the monotonic clock */ + kal_u64 context; /* kal_task_current() of the sampled context */ + kal_u32 depth; /* frames in `pc', innermost first */ + kal_u32 reserved; + kal_uintptr pc[KAL_PROF_SAMPLE_DEPTH]; +}; +kal_uintptr kal_prof_props(void); +int kal_prof_sample_begin(kal_u32 per_second); /* every context of this program */ +kal_intptr kal_prof_sample_take(struct kal_prof_sample* out, kal_uintptr count); /* drains; returns how many */ +int kal_prof_sample_end(void); +``` + +- **故障报告**：声明 `KAL_DEBUG_PROP_FAULT_REPORT` 的实现，以故障结束程序时（非法访问、非法指令、栈溢出）向程序的错误流写出 + 故障的种类与地址，以及帧指针链给出的每一帧：地址、持有它的文件与文件内偏移（环境映射文件之处）、该文件为 ok 映像 + （第 12 条）或程序自身时的符号名与偏移。写完后以环境的默认处理结束程序，core 照常产生。Linux/macOS 在备用栈上处理 + `SIGSEGV`/`SIGBUS`/`SIGILL`/`SIGFPE`（`/proc/self/maps`、`mach_vm_region` + `proc_regionfilename` 给出文件），Windows 以向量化 + 异常处理（`GetMappedFileNameW`）。原生内核对应于内核的故障报告。 +- **采样**：`kal_prof_sample_begin` 后，实现按频率采样本程序的全部上下文，记录 PC 与帧指针回溯；`take` 取走已采的样本， + `end` 停止。Linux 与 macOS 以整个程序的处理器时间定时器（`setitimer(ITIMER_PROF)`）与 `SIGPROF`：信号送到正在使用 + 处理器时间的上下文，其上下文中即有 PC 与帧指针（`perf_event_open` 受 `perf_event_paranoid` 限制，CI 与许多机器上不可用）； + Windows 以采样线程 `SuspendThread` + `GetThreadContext`。环满时新的样本不记录。符号化在采样之外（okprof 按映像与程序的符号表）， + 同一份火焰图在三个系统上得出，不依赖系统的采样器是否认识映像。 + +### 4.8 资源偏好与可用内存（11 条 57） + +```c +/* openkal.process: in kal_spawn.flags; none of the three inherits the caller's. */ +#define KAL_SPAWN_RESOURCES_BALANCED ((kal_uintptr)1u << 6) +#define KAL_SPAWN_RESOURCES_PERFORMANCE ((kal_uintptr)1u << 7) +#define KAL_SPAWN_RESOURCES_ECONOMY ((kal_uintptr)1u << 8) + +/* openkal.env: the preference this program was started with. */ +#define KAL_RESOURCES_BALANCED 0 +#define KAL_RESOURCES_PERFORMANCE 1 +#define KAL_RESOURCES_ECONOMY 2 +int kal_env_resources(void); + +/* openkal.memory: what the environment can give this program now. */ +struct kal_memory_status { + kal_u32 self_size; /* set by the caller (clause 5.3) */ + kal_u32 present; /* KAL_MSTATUS_* */ + kal_u64 total_bytes; /* the most this program may use: the machine's, or a limit set on it */ + kal_u64 available_bytes; /* what can be given without taking it from another program */ +}; +#define KAL_MSTATUS_TOTAL ((kal_u32)1u << 0) +#define KAL_MSTATUS_AVAILABLE ((kal_u32)1u << 1) +int kal_memory_info(struct kal_memory_status* out); +``` + +- 偏好是程序与环境之间的请求，与 `kal_task_set_priority` 同属一类（各系统的 QoS 类别）。启动时设定，被启动的程序继承； + 多于一位为 `kal_err_invalid`。实现以自己的方式传递（Linux、macOS 为 `KAL_RESOURCES` 环境变量，与 `KAL_PREOPENS` 相同； + 手工运行时用户也以它设定）。 +- 实现层在程序开始时按偏好调整环境的机制：economy 时 Linux `prctl(PR_SET_THP_DISABLE)`、Windows + `SetProcessInformation(ProcessMemoryPriority, MEMORY_PRIORITY_LOW)`；performance 时不限制。实现自己的缓存上限按偏好取值。 +- `kal_memory_info` 给出本程序可用的上限与当前可得：Linux `MemAvailable` 与所在 cgroup 的 `memory.max` 取小，macOS + `hw.memsize` 与 `host_statistics64`，Windows `GlobalMemoryStatusEx` 与作业对象的限额，emscripten 线性内存的上限与余量。 + 均衡档以它自适应（§16）。 + +### 4.9 一致性套件 + +| 条 | 观察 | +| --- | --- | +| 4.1 | 新区域每页的首字节为零 | +| 4.2 | 封存后读到原值；声明 `FOREVER` 的实现：再次 `kal_exec_publish_part` 该部分被拒绝 | +| 4.3 | 带 `PREFETCH` 的映射成功，内容与不带时相同 | +| 4.4 | 只读句柄上的创建、写打开、删除、改名为 `kal_err_permission`，读成功；属性只增不减；由其打开的目录继承；记录中出现 + `r`、`a`、`l`、`e` 且去重；`UNCHANGING` 的句柄上已列举目录中不存在的名字为 `kal_err_not_found` | +| 4.5 | 列举给出的字段与 `kal_fs_info` 相同；`present` 只含给出的位 | +| 4.6 | 触及 N 页后 `faults_minor` 至少增加 N/2（考虑预读与大页） | +| 4.7 | 能力字只含分配的位 | +| 4.8 | 未设定时为 balanced；以三个标志之一启动的副本读回该值；不带标志时继承；两个标志为 `kal_err_invalid`；`available_bytes` ≤ `total_bytes` | + +隔离（Landlock、SBPL）由各实现自己的测试观察：带 `KAL_SPAWN_CONFINE` 启动的子进程以原始系统调用越过 openkal 写只读授予中的文件、读授予之外的文件，均被环境拒绝；未声明该属性的实现拒绝该标志。 + +## 5. 映像格式（第 12 条，版本 1.1） + +| 类型 | note | 内容 | +| --- | --- | --- | +| 3 | placement（已有） | 封装时的基址与存储段；本机缓存不可用时的后备 | +| 4 | touch（C） | 启动触及的页：若干 `(偏移, 页数)`，以 64 KiB 为单位，按首次触及的顺序；加载器以 `KAL_EXEC_MAP_PREFETCH` 预取 | +| 5 | snapshot（D） | 只出现在本机缓存的文件中，不出现在发布的映像中（§6.4） | + +签名（类型 2）覆盖整个文件，不变。type 4 由封装写入，前于签名。 + +## 6. 加载器与运行器（openkal-image 0.4.0） + +### 6.1 已在 0.4.0 分支 + +`okld place`、placement note、加载器按基址放置、`okld_check_placement`（只在封装时）、`okrun` 的 `reserve_at`。 + +### 6.2 装入路径 + +1. 不写 `.bss`（§4.1）。 +2. 符号重定位之后，以 `kal_exec_seal_part` 封存 `PT_GNU_RELRO`：入口映像请求 `FOREVER`。映射自文件的 RELRO 块本来只读。 +3. 有 type 4 note 时，映射指令与只读数据的那些部分带 `KAL_EXEC_MAP_PREFETCH`。 +4. `okrun_resolve` 以完美散列查找（构建时由 SURFACE 生成），代替逐项比较。 + +### 6.3 本机放置缓存（安全与共享） + +参照 macOS dyld 共享缓存按开机滑动的做法，默认策略： + +- 每个用户、每次开机一个随机滑动 `S`（64 KiB 的倍数，在 [16 TiB, 32 TiB) 中）；映像的基址为 `P(digest) + S`（模 16 TiB）。 +- 缓存目录由启动者授予（mcppls 授予 `/okrun`），或取 `OKRUN_CACHE`、`XDG_RUNTIME_DIR`/`TMPDIR` 下以 0700 新建的 + `okrun-`。滑动记在目录中，附带创建时的墙上时间与单调时间：二者之差变化超过 1 s 即为新的一次开机（单调时钟随开机 + 归零），重新选择。 +- 当次开机第一个装入该映像的进程，以文件中的存储段为起点按 `S` 重新计算相对重定位（或从文件字节计算），写入 + `-.okseg`（先写临时文件，再改名），然后照常运行。此后的进程映射这个文件的页。 +- 缓存文件以内容摘要命名，创建时由创建者校验（`okld_check_placement` 的同一算法），读取时校验头部的映像摘要与基址。同一用户 + 的其他进程能改写缓存，但它们本来就能调试该用户的进程，不增加能力。 +- 缓存不可用（只读环境、没有私有目录）时使用封装时的基址；`OKRUN_PLACE=sealed` 固定使用它，`OKRUN_PLACE=0` 放到任意地址。 + +### 6.4 初始化后快照（D） + +快照与放置缓存在一起，由本机生成： + +1. **生成**：缓存中尚无快照时，运行器在当次开机第一次装入后，以 `okrun --seal-local` 另起一个进程生成，原进程照常运行。 +2. **封装运行**：在基址处装入映像；以一个只提供内存的 openkal 实现运行 C 库的第一阶段初始化与全部静态初始化： + - `kal_alloc`、`kal_region_*` 由紧接映像跨度之后的初始化区供给（封装进程在 `基址 + 跨度` 处预留）；映像本身不带这段地址， + 平时的装入不预留它，Windows 的提交额度不受影响； + - `kal_task_current` 给出固定的主上下文； + - 其余任何 `kal_*` 调用都使封装放弃。 +3. **证明**：初始化结束后扫描可写段与初始化区，若有字指向封装进程的栈或 `kal_*` 桩所在的地址段，封装放弃。符号重定位在 + 装入时重新施加，静态数据中对 `kal_*` 的引用因此总是正确。 +4. **写出**：可写段与初始化区已用部分中非零的 64 KiB 块，加 type 5 note（基址、块表、主上下文的线程记录位置、第二阶段入口 + `__okm_libc_start`）。 +5. **装入**：缓存有快照时，加载器在 `基址 + 跨度` 处只预留初始化区实际用到的部分，映射快照块（私有、写时复制），施加符号 + 重定位，不运行 `DT_INIT_ARRAY`，把初始化区的范围写入映像导出的 `__okm_snapshot_arena`，先调用第二阶段入口（C 库读取参数、 + 环境、preopen、标准流，把当前上下文绑定为快照中的主上下文，以 `kal_random_fill` 设定栈保护的守卫值），然后进入 `ok_main`。 + C 库的分配器不把初始化区内的地址交还环境（它们不是本进程 `kal_region_reserve` 的结果）。 +6. **语义差异**：mimalloc 的 `MIMALLOC_*` 环境变量在快照中已按无环境读取，不再生效；文档写明，需要它们时以 + `OKRUN_SNAPSHOT=0` 运行。 + +### 6.5 运行器的观测 + +| 模式（`OKRUN_PROF`） | 内容 | +| --- | --- | +| `calls`、`counters`（已有） | `kal_*` 调用与时间；程序的计数器与阶段 | +| `phases` | 准入、装入、重定位、初始化、`main`、退出各阶段的时间与缺页（§4.6） | +| `samples[=hz]` | §4.7 的采样，帧指针回溯，按映像符号化 | +| `touch` | 触页顺序：指令与只读数据先以无访问保护映射，首次访问时由运行器记录并恢复保护（Linux/macOS 信号、Windows 向量化异常），用于 C | +| `names` | 程序经由 `RECORDED` 授予到达的名字（§4.4）的汇总：按目录统计不存在的查找 | +| `perfmap`（已有） | Linux 的 `/tmp/perf-.map` | + +`okprof report --format=trace` 输出 Chrome/Perfetto 的时间线：每个进程（worker）一行，阶段、标记、采样与缺页在同一时间轴上。 +`okprof report --flame` 输出折叠栈。 + +### 6.6 调试 + +1. **默认可调试，零成本**：Linux、macOS 上指令与只读数据是映像文件的页，`/proc//maps` 与 core 的 `NT_FILE` 写明映像文件 + 与基址。`okgdb.py` 与 `oklldb.py`（随 openkal-tools 发布）读这些映射，以 `add-symbol-file -o `、 + `target modules load --slide` 载入符号；对 core 同样有效。 +2. **复制的映像**（Windows 在 §7.3 之前、任意地址）：GDB JIT 接口（已有，`--debugger`）。 +3. **调试信息分离**：平台以 `--build-id` 链接；`okld debug-split ` 写出 `.debug` 与精简后的映像，布局与 + debuginfod 相同（`.build-id/xx/rest.debug`），okgdb 按 build-id 查找。 +4. **故障报告**：§4.7，帧指针回溯，`kal_debug_symbolize` 命名。 + +## 7. 实现层（openkal-linux 0.24.0、openkal-macos 0.21.0、openkal-windows 0.19.0、openkal-emscripten 0.12.0） + +### 7.1 共同 + +`kal_exec_seal_part`、`PREFETCH`、`kal_fs_dir_attenuate`（属性随 `KAL_PREOPENS` 一类的授予传递，在每个修改操作与路径解析中 +检查）、`UNCHANGING` 缓存（目录列表与已取属性，以句柄与相对名字为键）、`RECORDED` 记录（去重集合、64 KiB 缓冲、退出时写 +`e`）、`kal_fs_list_next_info`、`KAL_PINFO_FAULTS`、故障报告、采样。 + +### 7.2 Linux + +`mseal`（`ENOSYS` 时不声明 `FOREVER`）、`MAP_POPULATE`、Landlock（ABI 版本按内核探测；规则：授予的层级按读写权限，另加 +`/proc`、`/dev/null` 与被启动的程序文件本身的执行权限）、`statx`、`SIGPROF` 采样。 + +### 7.3 Windows + +1. **文件页映射**（`KAL_EXEC_PROP_FILE`）：`kal_exec_alloc` 以 `VirtualAlloc2(MEM_RESERVE | MEM_RESERVE_PLACEHOLDER)` 保留占位， + `kal_exec_map_part` 拆分占位后以 `NtMapViewOfSectionEx(MEM_REPLACE_PLACEHOLDER)` 映射文件的节（以执行权限重开文件，节为 + `SEC_COMMIT` + `PAGE_EXECUTE_READ`/`PAGE_READONLY`），其余部分以 `VirtualAlloc2(MEM_REPLACE_PLACEHOLDER | MEM_COMMIT)` 提交； + `kal_exec_free` 逐段释放。需要 Windows 10 1803 起，函数动态解析，缺失时不声明该位。 +2. **按名字查属性**：`NtQueryInformationByName(FileStatInformation)` 一次给出文件 ID、时间、大小、属性、链接数（Windows 10 1709 + 起）；卷序列号取自基目录的句柄（缓存在句柄上）；Windows 11 24H2 的 `FileStatBasicInformation` 直接给出卷序列号。 + `KAL_FS_NO_RESOLVE` 与函数缺失时回到打开句柄的路径。 +3. **带属性的列举**：`NtQueryDirectoryFile(FileIdBothDirectoryInformation)`，声明 `KAL_FS_PROP_LIST_INFO`。 +4. `PrefetchVirtualMemory`、`VirtualProtect`、`PROCESS_MEMORY_COUNTERS`、向量化异常的故障报告、采样线程。 + +### 7.4 macOS + +`mach_vm_protect(set_maximum)`、`MADV_WILLNEED`、`getattrlistbulk`（声明 `LIST_INFO`）、SBPL 沙箱（`sandbox_init_with_parameters`， +不可用时不声明 `GRANT_CONFINED`）、`task_events_info`、采样线程。 + +### 7.5 emscripten + +只提供 `openkal.fs` 的新增：`kal_fs_dir_attenuate`（只读、记录在实现内；不变的缓存）、`kal_fs_list_next_info`（逐项 `fstatat`）； +`openkal.exec`、`openkal.process` 仍不提供。 + +## 8. C 库（openkal-musl 0.27.0） + +1. 描述表惰性初始化（已改）；realpath（已改）；`dlopen` 的 `reserve_at`（已改）。 +2. **两阶段初始化**：`__okm_libc_init` 拆为与环境无关的第一阶段（表、分配器、主上下文的线程记录）与第二阶段 + `__okm_libc_start`（参数、环境、preopen、标准流、栈保护守卫）。普通启动两阶段相继执行；快照装入只执行第二阶段。 +3. **初始化区**：封装运行中，mimalloc 的操作系统层经由封装者的 `kal_region_*` 自然得到初始化区的地址；快照装入后， + 落在 `__okm_snapshot_arena` 范围内的地址不交还环境，也不丢弃：这段内存是加载器的映射，不是本进程 `kal_region_reserve` + 的结果，规范对它没有释放操作；它的大小是初始化期间的堆（clangd 约 1–2 MB）。 +4. **栈保护**：`__stack_chk_guard` 为全局（`-mstack-protector-guard=global`），第二阶段以随机值设定；设定它的函数自身不受保护。 +5. mimalloc 选项：一次扫描环境，没有 `MIMALLOC_` 前缀的变量时直接返回。 +6. `posix_spawn` 传递的授予：父进程的 preopen 原样授予（含其属性）；`O_PATH` 等不变。 + +## 9. 平台与消费者 + +### 9.1 openkal-platform 0.23.0 + +| 组 | 内容 | +| --- | --- | +| build | 加 `-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer`、`-fstack-protector-strong -mstack-protector-guard=global`、`-ftrivial-auto-var-init=zero`、`-D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST`；x86_64 `-fcf-protection=full`，aarch64 `-mbranch-protection=standard` | +| link | `--build-id=sha1`；`-z pack-relative-relocs`（已有）；包声明的触页顺序文件（`--symbol-ordering-file`）| +| seal | `okld note`、`okld place`、`okld touch`（有触页数据时写 type 4）、`okld check` | + +BTI 与 CET 只发出标记；Linux arm64 上加载器以 `KAL_EXEC_MAP_BTI`（§4.3 同一 `how` 的位 2，内核支持时 `PROT_BTI`）映射指令， +其余环境不强制。帧指针与加固的处理器时间成本以 L2 门禁衡量（§13）。 + +### 9.2 mcpplibs.os 0.2.8 + +`spawn_options` 增加 `grants`：`{ name, path, rights }` 的列表（`rights` 为只读、不变、记录的组合与记录文件），映射到 +`kal_fs_dir_attenuate` 与 `kal_spawn.grants`；未给出时保持实现的默认授予。 + +### 9.3 mcppls 0.0.20 + +1. **最小授予**：启动引擎时授予项目根（第一项、读写）、编译数据库中出现的每个目录（`-I`、`-isystem`、`-iquote`、`--sysroot`、 + `-B`、`-resource-dir`、编译器的安装前缀；只读、不变）、引擎 payload（只读、不变）、模块缓存与临时目录（读写）、okrun 的 + 缓存目录（读写）。宿主的 `/usr` 只在编译命令使用宿主编译器时出现在其中。 +2. **精确输入集**：模块 worker 的授予带 `RECORDED`，记录写在模块载荷旁；复用载荷时检查记录中 `a` 的名字现在是否存在、`r` 的 + 文件是否变化，任一成立即重建。这覆盖了 clang 的 PCM 校验不覆盖的情形：搜索路径靠前的目录中新增的同名头文件。 +3. 引擎崩溃时，mcppls 的日志包含实现的故障报告。 + +### 9.4 引擎（mcppls-clangd 0.0.20） + +CI 增加触页训练：以 `OKRUN_PROF=touch` 运行 warm-prime 的冷阶段（Linux x64、arm64），合并各进程的首次触及顺序得到函数顺序 +文件，重新链接（目标文件复用），封装时写 type 4。顺序文件是 CI 的产物，不提交。 + +### 9.5 mcpp + +`docs/` 中 openkal 平台一节说明加固、帧指针、本机放置缓存与授予；`CHANGELOG`；内置平台钉住 0.23.0。 + +## 10. 调试、性能分析与可观测性（汇总） + +| 需求 | 手段 | 所在层 | +| --- | --- | --- | +| 断点、单步、变量 | okgdb/oklldb 从映射载入符号；复制的映像用 JIT 接口；分离的调试信息按 build-id | 工具 | +| 崩溃 | 故障报告（帧指针回溯 + 符号化）；core 加 okgdb | 实现 | +| 热点 | `samples`：三系统一致的采样与火焰图；Linux 另有 perf（文件页可符号化，帧指针可回溯） | 实现 + okprof | +| 启动 | `phases`：各阶段时间与缺页；`touch`：触页顺序 | 加载器 + okprof | +| ABI 边界 | `calls`：每个 `kal_*` 的次数与时间 | 绑定者 | +| 文件系统 | `names`：到达的名字、不存在的查找按目录汇总 | 实现（`RECORDED`） | +| 多进程 | `--format=trace`：一个时间线上的全部 worker | okprof | + +## 11. 安全（汇总） + +| 项 | 原生（发行版常见） | 本轮之后的映像 | +| --- | --- | --- | +| 地址随机化 | PIE + ASLR，每进程 | 每用户每次开机的滑动，同一次开机内共享（§6.3） | +| RELRO | 只读 | 只读，入口映像永久（§4.2） | +| W^X | 有 | 有（12.2，已有） | +| 栈保护 | `-fstack-protector-strong` | 同，全局守卫，第二阶段随机设定 | +| 自动变量初始化 | 部分发行版 | `-ftrivial-auto-var-init=zero` | +| 标准库加固 | `_GLIBCXX_ASSERTIONS`（部分） | libc++ 加固 fast 模式 | +| CET/BTI | 标记，部分强制 | 标记；Linux arm64 强制 BTI | +| 文件系统范围 | 用户的全部文件 | 授予的目录；只读由实现强制，请求隔离时由环境强制（§4.4） | +| 完整性 | 包管理器 | 映像与 okb 的签名（已有）；本机缓存由创建者校验 | + +## 12. 系统调用合并的判断 + +| 候选 | 判断 | +| --- | --- | +| 按名字查属性 | 已是一次 ABI 调用；Windows 实现由 6 次系统调用改为 1 次（§7.3） | +| 带属性的列举 | 采纳（§4.5）：macOS、Windows 一次给出一批；为原生内核保留一次跨越 | +| 预取 | 采纳（§4.3）：把启动时数十次缺页合并为一次请求 | +| 进程内作答 | 采纳：最小授予之外的名字由 C 库以无 preopen 作答；不变授予由实现缓存作答 | +| 通用批量提交（io_uring、IoRing） | 不采纳：LLVM 的文件访问同步逐个进行，调用者无法成批；io_uring 在容器的默认 seccomp 与部分发行版被禁用，IoRing 只支持读 | +| 整文件读取 | 不采纳：Linux 上内部仍是同样的系统调用，收益只在 Windows 的一次查询 | + +## 13. 验证与门禁 + +1. **一致性套件**：§4.9；各实现的本地一致性与 CI。 +2. **openkal-image CI**：放置缓存（第一个进程写、第二个进程映射、换滑动后重建）；快照（生成、装入、输出与普通启动相同、 + 证明失败的映像回到普通路径）；RELRO 封存后写入被拒绝；预取；`touch`、`samples`、`phases` 的输出；okgdb 从 core 载入符号。 +3. **openkal-musl CI**：两阶段初始化；初始化区；栈保护的守卫在第二阶段设定；授予属性经 `posix_spawn` 传递。 +4. **安全测试**：Linux 上以原始系统调用越过 openkal 写只读授予中的文件被拒绝（Landlock）；macOS 同（SBPL）；栈溢出被栈保护 + 终止并有故障报告。 +5. **L2（引擎 warm-prime 门禁）**：Linux x64/arm64、macOS、Windows 四个平台；加固与帧指针的成本计入；预期见 §14。 +6. **L3**：warm 阶段 `strace -c`：失败查找的数量与来源；缺页。 + +## 14. 预期 + +| 指标 | 现在（本机） | 预期 | +| --- | --- | --- | +| 一次启动的缺页 | 532（原生 284） | 约 250（快照 −150，`.bss` 惰性，预取合并指令页） | +| warm 阶段失败查找 | 约 30 万（原生同） | 约 0：最小授予之外进程内作答，授予之内由不变缓存作答 | +| 每个 worker 的私有页 | 约 1.5 MB 初始化数据 | 文件页，跨进程共享 | +| Windows 每次启动 | 复制 37 MB 指令 | 文件页 | +| Windows 一次 stat | 6 次系统调用 | 1 次 | +| 帧指针与加固的成本 | — | 处理器时间 ≤ 2%（L2 计入） | + +## 15. 交付 + +| 仓库 | 版本 | PR | +| --- | --- | --- | +| openkal | 0.23.0 | #61（同一 PR 增补） | +| openkal-linux / -macos / -windows / -emscripten | 0.24.0 / 0.21.0 / 0.19.0 / 0.12.0 | #48 / #39 / #43 / #14 | +| openkal-image | 0.4.0 | 新 PR（分支 `openkal-0.23`） | +| openkal-musl | 0.27.0 | 新 PR | +| openkal-llvm-runtime | 0.18.0 | 新 PR（加固与帧指针随平台） | +| openkal-platform | 0.23.0 | 新 PR | +| mcpplibs.os | 0.2.8 | 新 PR | +| mcppls-clangd | 0.0.20 | #16 | +| mcppls | 0.0.20 | #75 | +| mcpp | 2026.11.2.3 | #884 | + +依赖顺序与 part 3 的版本组级联相同：openkal 与实现 → 索引 → image → musl → llvm-runtime → platform → os → 引擎 → mcppls → +mcpp-ide → mcpp。openkal 的改动在 CI 通过后做多角度的规范评审（规范、ABI、设计意图、架构）。 + +## 16. 资源偏好：各层的兑现 + +默认为 balanced；performance 以空间换时间，economy 以时间换空间。每一项在三档下各有确定的取值，单项设置优先于档位； +balanced 按 `kal_memory_info` 与缓存目录的余量自适应：资源充足时取偏快的值，可得内存低于总量的 15% 或缓存目录余量低于 +1 GiB 时取 economy 的值。 + +| 兑换点 | 层 | performance | balanced | economy | 单项设置 | +| --- | --- | --- | --- | --- | --- | +| 本机放置缓存 | 加载器 | 保留 | 保留，按开机重建 | 不写，每次重定位 | `OKRUN_PLACE` | +| 初始化后快照 | 加载器、C 库 | 保留，上限 2 GiB | 上限 512 MiB，LRU | 不生成 | `OKRUN_SNAPSHOT` | +| 启动页预取 | 加载器 → 实现 | 触页集合与全部 RELRO | 触页集合 | 不预取 | `OKRUN_PREFETCH` | +| 不变目录缓存 | 实现 | 列表与属性 | 列表与属性，64 Ki 条目 | 只缓存不存在 | `KAL_FS_CACHE` | +| 分配器的保留 | C 库 | 归还延迟 1 s，堆区域用大页 | mimalloc 默认（10 ms） | 立即归还 | `MIMALLOC_*`（快照装入时除外，§17.1） | +| 透明大页 | 实现 → 环境 | 不限制 | 环境默认 | 关闭 | — | +| 内存优先级 | 实现 → 环境 | 正常 | 正常 | 低 | — | +| worker 并发 | mcppls | 核数 | 核数的一半 | 2 | `mcppls.workers` | +| 模块缓存 | mcppls | 4 GiB | 1 GiB | 256 MiB，载荷压缩 | `mcppls.moduleCache` | +| okb 形态 | 打包 | 存储 | 压缩传输，安装时展开为存储 | 压缩存放，首次运行展开到缓存 | mcpp.toml 的打包设置 | + +- **入口**：构建期的项（触页 note、顺序文件、okb 形态）在 mcpp.toml 的平台节；运行期由启动者请求，mcppls 以 + `mcppls.resources = "balanced" | "performance" | "economy"` 设定，并经 `KAL_SPAWN_RESOURCES_*` 传给引擎及其 worker。 +- **观测**：okprof 的 `phases` 报告每个兑换点的命中、占用与节省（时间、缺页、字节）；`okrun cache` 列出与回收本机缓存， + `mcpp cache gc` 一并回收。 +- **衡量**：L2 门禁以 balanced 运行；另跑 performance 与 economy 的对照，报告峰值 RSS、磁盘占用、处理器时间与墙钟时间， + 写入本文。 + +## 17. 待评审项 + +1. **默认放置策略**：本文以每用户每次开机的滑动与本机缓存为默认（安全），封装时的基址为后备。另一选择是以封装时的基址为默认、 + 滑动为可选（每次开机的第一次启动少一次重定位）。 +2. **快照的语义差异**：`MIMALLOC_*` 环境变量对快照装入不生效（§6.4.6）。 +3. **加固的默认**：libc++ 加固 fast 模式与 `-ftrivial-auto-var-init=zero` 对引擎的处理器时间影响以 L2 衡量；若超过 2%，是否 + 保留为默认。 +4. **授予的推导**：mcppls 从编译数据库推导授予（§9.3）。编译命令依赖宿主编译器的隐含路径时，授予包含该编译器的安装前缀。 + 是否另给用户一个追加授予的设置（`mcppls.grants`）。 + +### 17.1 决定（2026-11-03） + +1. 默认放置策略为每用户每次开机的滑动与本机缓存；封装时的基址为缓存不可用时的后备。 +2. 快照装入时 `MIMALLOC_*` 环境变量不生效，写入 openkal-image 与 openkal-musl 的文档；需要它们时以 `OKRUN_SNAPSHOT=0` 运行。 +3. libc++ 加固 fast 模式与 `-ftrivial-auto-var-init=zero` 为平台默认，成本写入 L2 报告；包可以在自己的清单中关闭。 +4. mcppls 提供 `mcppls.grants`：用户追加的目录（只读或读写），`"all"` 回到实现的默认授予。 + +## 18. 实现记录（2026-11-03） + +本节记录实现与 §4–§16 的差别及其理由，以及实现中发现并修正的缺陷。§4–§16 的正文保留为设计；凡与本节冲突，以本节为准。 + +### 18.1 快照（D，openkal-image 0.4，docs/snapshot.md） + +1. **生成**：运行器在当次开机第一次以本机放置启动映像、而缓存中没有 `-.oksnap` 时，独占创建 + `-.sealing`，以 `OKRUN_SNAPSHOT=start` 重新启动自身（同一参数与环境，标准流为空设备）；该进程以 + `OKRUN_SNAPSHOT=make` 再启动一个进程后立即结束，原进程只等待前者，生成快照的进程不比任何父进程活得久。标记文件记录结果 + （`made` 或 `refused: <原因>`），同一次开机不再尝试。`--seal-local` 未采用。 +2. **C 库的入口**：映像的 C 库导出记录 `__okrun_snapshot {arena, size, phase, reserved, context}` 与入口 + `__okrun_snapshot_start`，代替 §6.4 的 `__okm_snapshot_arena` 与 `__okm_libc_start`：名字属于运行器的协议而不属于某个 C 库。 + phase 1 为生成时初始化之后，C 库报告快照不能保存的东西（例如初始化代码设置的环境变量）；phase 2 为从快照启动，C 库取得 + 参数、环境、流、目录、随机数与栈保护守卫，并把快照中的上下文记为当前上下文。 +3. **生成时的绑定**（run/src/seal.cpp）：内存操作由初始化区供给；实现自身的属性（`*_props`、粒度、版本、接口）作为事实； + `kal_task_parallelism`、`kal_task_cores`、`kal_env_resources` 与按名读取的环境变量作为记录的回答；其余调用使生成放弃， + 标记写明调用名。事实与被问到的回答折叠为一个字，启动时与本次的值比较，不同则不使用快照（报告中的 runner 一项写明原因）。 + 被读取的环境变量以名字与值的摘要保存在快照文件中：LLVM 在静态初始化中读取 `LLVM_OVERRIDE_PRODUCER`，原设计的"其余调用即 + 放弃"会使 clangd 永远没有快照。 +4. **证明**：只检查两件事：每个导入槽仍是装入时写入的值（`okld_bound_imports`）；其余可写数据与初始化区中没有任何字等于绑定 + 的地址。原设计的栈地址检查被删除：短字符串的复制携带其未用字节，常含已返回栈帧的地址，产生误报；这些地址在任何启动之后都 + 指向已返回的帧，不构成快照特有的风险。 +5. **初始化区**：位于映像跨度之后、按 256 MiB 对齐的边界两侧；区域与 64 KiB 以上对齐的分配在边界之上按地址本身对齐 + （mimalloc 以地址掩码找区域元数据，需要地址对齐而不是偏移对齐），其余小分配在边界之下向下放置；上限 1 GiB。 +6. **装入**：只重新施加命名导入的重定位（`applying::imports`）；快照的块以 `KAL_EXEC_MAP_WRITE` 私有映射，不预读（预读快照块 + 的内核时间超过其节省，§18.4）。 +7. **测量**：clangd `--version`：从快照启动 311 次缺页，不用快照 442 次，处理器时间少 15%。 + +### 18.2 放置、触页与调试 + +1. **本机放置缓存的位置**：`OKRUN_CACHE`，否则 `$XDG_RUNTIME_DIR/okrun`（该目录每次开机由系统清空），否则用户缓存目录的 + `okrun/placed`；滑动与开机时刻记在其中的 `boot` 文件。§6.3 的 `TMPDIR` 下 `okrun-` 未采用：用户缓存目录已只属于该用户。 +2. **触页**：经由 ABI（`kal_prof_touch_*`）由实现记录，`okprof record --what touch` 取得，`okld touch` 写入 type 4 note， + `okprof report --order` 输出符号顺序文件；note 由生产者（引擎 CI 的训练步骤）写入，运行器不在用户机器上记录触页。 +3. **数据段**：存储段以 `KAL_EXEC_MAP_WRITE`（私有可写映射）映射文件的页，代替复制。 +4. **okprof 与调试脚本**：okprof 成为 openkal-tools 的第三个程序，列入平台的 `[platform.tools]`；`okgdb.py`、`oklldb.py` 随 + release-binaries 发布为一个归档。 + +### 18.3 实现层 + +1. **Windows（openkal-windows 0.19，§7.3）**：文件页映射以占位实现：区域是页文件节的视图，映射文件的部分拆分占位后换为文件 + 视图，其余部分以同一节在原偏移的视图保持内容；部分须按 64 KiB 对齐，否则 `kal_err_not_supported`。`KAL_EXEC_PROP_SEAL_FOREVER` + 不声明：视图的持有者总能改变保护。按名字查属性用 `NtQueryInformationByName`（24H2 的 `FileStatBasicInformation` 带卷序列号， + 1709 的 `FileStatInformation` 对单一分量的名字用基目录的卷序列号），记录不能回答时打开节点。`KAL_SPAWN_CONFINE` 被拒绝： + 该系统只能经由文件的安全描述符限制程序，而一次启动不改变它们。 +2. **实现中发现的缺陷**： + - openkal-macos 0.21.1：`ulock_wait` 读取被等待的字时不处理缺页，对从未写过的零页返回 `EFAULT`；`kal_time_sleep` 等待 + 一个静态零字，每次等待立即失败，睡眠成为整段时长的自旋（arm64 与 x86_64 运行机上睡 200 ms 耗费 127–200 ms 处理器时间）。 + 由 openkal-musl 的 posix 探针（`CLOCK_PROCESS_CPUTIME_ID` 跨 `nanosleep`）发现，原生探针确认。等待在 `EFAULT` 时读取该字 + 后重试；睡眠的字位于栈上。 + - openkal-macos 0.21.2：单调时钟粒度原为每次启动测得（41、42 ns 或更大），快照的事实字因此在两次启动间不同，macOS 上的 + 快照被间歇拒绝。粒度改取 `mach_timebase_info` 的计时单位向上取整。 + - openkal-musl 0.27.0：Windows 上以卷开头的名字与持有目录的名字逐字节比较，`c:\Users\...` 到达不了名为 `C:` 或 + `C:/Users/...` 的目录。卷下的名字改为不区分大小写、两种分隔符等价（`examples/image-names`）。 + +### 18.4 资源偏好（§16 的实现值） + +| 兑换点 | performance | balanced | economy | +| --- | --- | --- | --- | +| 本机放置缓存 | 写入并保留 | 写入并保留 | 在内存中计算，不写缓存 | +| 快照 | 生成，总量上限 2 GiB，LRU | 生成，上限 512 MiB，LRU | 使用已有的，不生成 | +| 预取（`OKRUN_PREFETCH=0\|touch\|all`） | 级别 2：触页集合与数据 | 级别 1：触页集合 | 0 | +| mimalloc 归还延迟 | 1 s | mimalloc 默认（10 ms） | 0 | +| mcppls worker（`engine.workers=auto`） | 硬件线程数，至多内存 GB 数的一半 | 原有公式：线程数减一，2–8，至多内存 GB 数的一半 | 2 | +| mcppls 缓存预算（`cache.maxBytes` 未设置时） | 8 GiB | 4 GiB（原默认） | 1 GiB | + +balanced 在可得内存低于总量 15% 或缓存目录所在设备余量低于 1 GiB 时由运行器按 economy 处理。与 §16 的差别： + +1. **balanced 保持原有取值**（worker 公式与 4 GiB 预算）。升级不改变未设置偏好的用户所得的行为；§16 中 balanced 的 1 GiB 与 + "核数的一半"会使现有用户的缓存与并发在升级后缩小。 +2. **economy 不压缩模块载荷**。clang 以映射读取 BMI，压缩的 BMI 在每次读取时都要展开为一份副本，这与缓存已有的读时副本重复， + 以时间换得的空间在读取期间又被占用；economy 改为较小的预算。 +3. **预取不覆盖快照块**：测得预读快照块与可写数据所费内核时间超过其节省；数据只在 performance 预读。 + +`mcpp cache gc` 调用最新的 `okrun cache gc` 回收运行器的缓存（其他开机的副本与快照）；`okrun cache [list|gc [--all]]` 列出与回收。 +压缩的 okb（okb 2，DEFLATE）由 `mcpp pack --pkg-format okb --compress gzip` 经 `MCPP_PACK_COMPRESS` 请求，运行器首次运行时展开到 +用户缓存的 `okrun/expanded`。 + +### 18.5 授予与精确输入集（§9.3） + +1. **授予（mcppls 0.0.20）**：ok 映像引擎得到项目根（可写，第一项）、mcppls 的缓存与数据库目录、临时目录与运行器的缓存（可写）、 + 引擎自身的目录（只读、不变）、包仓库与系统根目录（只读）、编译数据库提到的目录（只读），以及 `mcppls.grants` 追加的目录； + 被链接指向的目录以两种名字给出。数据库在引擎启动之后才推送，故启动时的集合覆盖工具链与头文件的常见位置；之后的数据库提到 + 未覆盖的目录时，引擎带着扩大的集合重启（重启原因 plan）。`mcppls.grants = ["all"]` 回到运行器的默认（整个文件系统）。 + `mcppls.engine.confine` 默认关闭；系统不能执行时（Windows、无 Landlock 的 Linux），引擎只由 openkal 限制并在日志中说明。 +2. **精确输入集（B，mcppls-clangd openkal 0003）**：模块 worker 通过 C 库的 `okm_preopen_attenuate`（openkal-musl 0.27.0）为 + 自己持有的每个目录请求 `KAL_DIR_RECORDED`，记录写在 worker 单元的 `inputs-`。worker 结束后，clangd 取完整记录中查找 + 未果的名字（`a`），去掉 worker 自身单元之下的名字，连同 worker 开始前的墙上时间写在 BMI 旁（`.inputs`），先于 BMI + 发布。复用 BMI 之前，若其中的名字现在存在则重建；目录最后修改早于 worker 开始的，其下没有新增名字，只对修改过的目录逐名 + 查询。与 §9.3 的差别：`r` 项不检查，因为读取过的文件就是 BMI 的输入，clang 对它们的大小、时间与内容的校验已覆盖；检查之后 + 在 recheck 间隔内不重复（与验证结论的缓存相同）。owned 缓存把 `payload.pcm.inputs` 计为代次的载荷。本地 Linux 上的 ok 映像： + 在搜索路径更靠前的目录新建同名头文件后，日志为 `Rebuilding module good: …/first/lib.h was absent when the module was built, + and is present now`；四个平台的验收各运行一次（`ci/module_inputs_canary.py --absent-names-only`）。 +3. **故障报告**：mcppls 把实现的故障报告（`openkal: fault: ...` 与 `openkal: #N ...`）作为崩溃上下文保存，不受日志限流。 + +### 18.6 交付（实际版本） + +| 仓库 | 版本 | +| --- | --- | +| openkal | 0.23.0 | +| openkal-linux / -macos / -windows / -emscripten | 0.24.0 / 0.21.2 / 0.19.0 / 0.12.0 | +| openkal-image（ld、run、tools、bind） | 0.4.1（0.4.0 之后以 openkal-macos 0.21.2 重建） | +| openkal-musl / openkal-llvm-runtime / openkal-platform | 0.27.0 / 0.18.0 / 0.23.0 | +| mcpplibs.os | 0.2.8 | +| mcppls-clangd / mcppls | 0.0.20 / 0.0.20 | +| mcpp | 2026.11.2.3 | From 17aa0bb52f2ca0201f485e51c2d607ecaeccfc23 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 10:52:26 +0900 Subject: [PATCH 07/14] the built-in openkal platform is openkal-platform 0.23.0 (openkal 0.23); an openkal set's Windows launcher over mcpplibs.os 0.2.8 Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 30 +++++++++++++++++-------- docs/04-mcpp-toml.md | 2 +- docs/10-pack-and-release.md | 2 +- docs/24-openkal-cross.md | 2 +- docs/35-platforms.md | 8 +++---- docs/50-machine-output.md | 2 +- docs/specs/packaging.md | 2 +- docs/zh/04-mcpp-toml.md | 2 +- docs/zh/10-pack-and-release.md | 2 +- docs/zh/24-openkal-cross.md | 2 +- docs/zh/35-platforms.md | 8 +++---- docs/zh/50-machine-output.md | 2 +- modules/toolchain-model/src/triple.cppm | 4 ++-- src/pack/distribution.cppm | 6 ++--- 14 files changed, 43 insertions(+), 31 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index db48d6918..50eb50abd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,18 +6,30 @@ ## [2026.11.2.3] - 2026-11-02 -The built-in openkal platform is 0.21.1; a download source that sends nothing -is left for the next one; a requirement a layer decides is checked; design -2026-11-02 part 3 landed. +The built-in openkal platform is 0.23.0 (openkal 0.23: an image at a base of +each boot, a snapshot of its initialisers, hardening); a format a package +provides is told the compression stated; `mcpp cache gc` reclaims the runner's +cache; a download source that sends nothing is left for the next one; a +requirement a layer decides is checked; design 2026-11-02 part 3 landed, and +design 2026-11-03 part 4. ### Changed -- **The built-in `-openkal` platform is `mcpplibs.openkal-platform@0.21.1`** - (docs/35): openkal-image 0.3.1, whose runner admits an okb member it has - compared before, on the same file, by its shape alone. 0.3.0 computed the - member's CRC-32 on every start: 0.1 s of processor time for a 71 MB image, - paid by each program an okb starts (a language server's 176 module workers - in a cold build). +- **The built-in `-openkal` platform is `mcpplibs.openkal-platform@0.23.0`** + (docs/35; design 2026-11-03 part 4), the version group of openkal 0.23: + openkal-image 0.4.1, openkal-musl 0.27.0, openkal-llvm-runtime 0.18.0. + An image is sealed at a base (`okld place`), and the runner moves it to a + base chosen once each boot, its data kept as a prepared copy; the first start + of a boot has a snapshot of the image's initialisers made, which later starts + map instead of running them (clangd `--version`: 311 page faults against + 442, 15 % less processor time). What a start spends follows the program's + resource preference. An image is built with frame pointers, a global stack + guard, zeroed locals, libc++'s fast hardening, CET or PAC/BTI marks and a + build-id; okprof is among the platform's tools. The Windows launcher `mcpp + pack` writes for an openkal set is built over mcpplibs.os 0.2.8. Against + 2026.11.2.2's 0.21.0, the runner also admits an okb member it has compared + before by its shape alone (openkal-image 0.3.1): 0.3.0 computed the + member's CRC-32 at every start, 0.1 s of processor time for a 71 MB image. ### Added diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 53496424e..cc955557f 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1502,7 +1502,7 @@ the platform's own key names ([35 — Platforms](35-platforms.md)): ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.21.1" # or { path = "…" }, or "none" +platform = "openkal-platform@0.23.0" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 1915c49ad..954f33674 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -956,7 +956,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.21.1", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.23.0", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index e0cee468a..aaaba029b 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -111,7 +111,7 @@ practice. ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.1" +platform = "openkal-platform@0.23.0" ``` One table (2026.10.27.1+). It selects the openkal platform for the host triples, diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 16e6dd5b6..62671a347 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -113,9 +113,9 @@ The first of these that answers selects the target's platform: | Source | Spelling | |---|---| -| the command line | `--platform openkal-platform@0.21.1`, `--platform path:../openkal-platform`, `--platform none` | +| the command line | `--platform openkal-platform@0.23.0`, `--platform path:../openkal-platform`, `--platform none` | | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| the project | `[target.] platform = "openkal-platform@0.21.1"`, or `{ path = "…" }` | +| the project | `[target.] platform = "openkal-platform@0.23.0"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | | the target table | the platform column (`mcpp toolchain list`) | @@ -129,7 +129,7 @@ its own conventions. `[host] platform` selects a platform for them instead: ```toml [host] -platform = "openkal-platform@0.21.1" +platform = "openkal-platform@0.23.0" ``` ``` @@ -151,7 +151,7 @@ implementation: ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.1" +platform = "openkal-platform@0.23.0" ``` On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index cda7586fc..9e81ff1ef 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -527,7 +527,7 @@ installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. A target row also carries `platform` (2026.10.27.1+): the platform package the -target table selects for it (`mcpplibs.openkal-platform@0.21.1`), or `null`. +target table selects for it (`mcpplibs.openkal-platform@0.23.0`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 9b5a75535..25ac4b038 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -237,7 +237,7 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` "schema": 1, "name": "hello", "version": "0.1.0", - "platform": "openkal-platform@0.21.1", + "platform": "openkal-platform@0.23.0", "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 71482d129..ff7a78f5d 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1384,7 +1384,7 @@ runner 与 `run_under`。 ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.21.1" # or { path = "…" }, or "none" +platform = "openkal-platform@0.23.0" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index ae004a702..7408925b4 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -801,7 +801,7 @@ Authenticode 可以签名它。Linux 与 macOS 经 shell 启动它：`sh pg-0.1. 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.21.1", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.23.0", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index 420bffd9e..d7d473f32 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -93,7 +93,7 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.1" +platform = "openkal-platform@0.23.0" ``` 一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 8591f2c0b..1acdf6814 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -98,9 +98,9 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 来源 | 写法 | |---|---| -| 命令行 | `--platform openkal-platform@0.21.1`、`--platform path:../openkal-platform`、`--platform none` | +| 命令行 | `--platform openkal-platform@0.23.0`、`--platform path:../openkal-platform`、`--platform none` | | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| 工程 | `[target.] platform = "openkal-platform@0.21.1"`,或 `{ path = "…" }` | +| 工程 | `[target.] platform = "openkal-platform@0.23.0"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | | 目标表 | 平台列(`mcpp toolchain list`) | @@ -112,7 +112,7 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 ```toml [host] -platform = "openkal-platform@0.21.1" +platform = "openkal-platform@0.23.0" ``` ``` @@ -130,7 +130,7 @@ platform = "openkal-platform@0.21.1" ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.21.1" +platform = "openkal-platform@0.23.0" ``` Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 1941d304b..e4ea23b9e 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -463,7 +463,7 @@ mcpp toolchain list --format json 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 -（`mcpplibs.openkal-platform@0.21.1`），或 `null`。 +（`mcpplibs.openkal-platform@0.23.0`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 4131c1dea..25a0ce112 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -906,9 +906,9 @@ inline constexpr TargetInfo kKnownTargets[] = { // `openkal-platform` (SPEC-013), pinned here. The two architectures the // loader relocates. { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.21.1" }, + "mcpplibs.openkal-platform@0.23.0" }, { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.21.1" }, + "mcpplibs.openkal-platform@0.23.0" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 487416a92..8a75b9b5e 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1074,10 +1074,10 @@ std::string exe_launcher_manifest(std::string_view name, std::string_view versio "# Generated by `mcpp pack`: the Windows launcher of {0} (design 2026-10-29 §4.3).\n" "[package]\nname = \"{0}\"\nversion = \"{1}\"\n\n" "[targets.{0}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" - "# 0.2.7: the release written against openkal 0.21, the openkal the\n" - "# platform of mcpp's openkal rows names (openkal-platform 0.21.1), which\n" + "# 0.2.8: the release written against openkal 0.23, the openkal the\n" + "# platform of mcpp's openkal rows names (openkal-platform 0.23.0), which\n" "# the launcher is built over.\n" - "[dependencies.mcpplibs]\nos = \"0.2.7\"\n", + "[dependencies.mcpplibs]\nos = \"0.2.8\"\n", name, version.empty() ? std::string_view("0.0.0") : version); } From 4206f8196d386df6b5a74110d9ce68a9be0a69fa Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 11:08:12 +0900 Subject: [PATCH 08/14] =?UTF-8?q?design=20part=204=20=C2=A718.6:=20aarch64?= =?UTF-8?q?=20pac-ret+bti,=20not=20standard?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .../2026-11-03-ecosystem-part4-beyond-native-design.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md index 4b1713cca..a8fe63e70 100644 --- a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md +++ b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md @@ -686,7 +686,13 @@ balanced 在可得内存低于总量 15% 或缓存目录所在设备余量低于 and is present now`；四个平台的验收各运行一次（`ci/module_inputs_canary.py --absent-names-only`）。 3. **故障报告**：mcppls 把实现的故障报告（`openkal: fault: ...` 与 `openkal: #N ...`）作为崩溃上下文保存，不受日志限流。 -### 18.6 交付（实际版本） +### 18.6 平台的编译选项 + +aarch64 用 `-mbranch-protection=pac-ret+bti`，不用 §9.1 的 `standard`：clang 23 的 `standard` 含受保护的控制栈 +（定义 `__ARM_FEATURE_GCS_DEFAULT`），libunwind 因此按 GCS 展开并包含 C 库的 ``，映像的构建在 aarch64 上失败； +GCS 还需要系统为程序启用该栈，映像没有这一环节。返回地址签名与 BTI 标记不变。 + +### 18.7 交付（实际版本） | 仓库 | 版本 | | --- | --- | From 30ffdceacaa08f162d3c2c3665384414faf4dca8 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 11:24:20 +0900 Subject: [PATCH 09/14] the std module is compiled with the platform's compile words, as every unit of the target is (a BMI without -fcf-protection was refused by its importers) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- CHANGELOG.md | 10 +++ docs/35-platforms.md | 2 +- docs/zh/35-platforms.md | 2 +- src/build/prepare/platform.cpp | 26 ++++++-- src/build/prepare/state.cppm | 4 ++ src/build/prepare/target_side.cpp | 14 ++++ ...he_std_module_takes_the_platforms_flags.sh | 66 +++++++++++++++++++ 7 files changed, 115 insertions(+), 9 deletions(-) create mode 100755 tests/e2e/1010_the_std_module_takes_the_platforms_flags.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 50eb50abd..42d9c4a34 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,6 +50,16 @@ design 2026-11-03 part 4. ### Fixed +- **The std module is compiled with the platform's compile words** (docs/35, + SPEC-013 §2.5): `[platform.build]` and its `cfg` tables reach the std + module's two commands and the key its BMI is kept under, as they reach every + other unit of the target, whichever library supplies the module. + openkal-platform 0.23.0 compiles every unit on x86_64 with + `-fcf-protection=full`, which clang records in a BMI, and a std module built + without it was refused by every importer ("Control-Flow Branch Protection + enabled was disabled in precompiled file"); its `-D_LIBCPP_HARDENING_MODE` + is the library's own code inside the module. e2e 1010 states the flag in a + platform of this machine's triple and imports std. - **A requirement a layer decides is checked** (docs/22, docs/specs/target-side.md §2.2). `[target.'cfg(all(windows, compiler = "llvm"))'] requires = ["mcpp:compiler-runtime=compiler-rt"]` names a layer in its selector, and diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 62671a347..416fbe335 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -68,7 +68,7 @@ for this machine and one for an ok image leave each other's entries in place. | the targets | `targets`, `toolchain`, `sysroot` | which triples it serves, the toolchain they use, a prebuilt target base | | the layers | `[platform.layers]` | the package for `c-abi`, `c++-abi`, `compiler-runtime`, `kernel-abi`, or `false` | | the tools | `[platform.tools]` | programs built for this machine, or found (`{ program = "" }`) | -| the compile | `[platform.build]`, `[platform.build.'cfg(…)']` | flags for every unit of the target | +| the compile | `[platform.build]`, `[platform.build.'cfg(…)']` | flags for every unit of the target, the std module included (2026.11.2.3) | | the link | `[platform.link]` | `driver`, `flags`, `start`, `script`, `inputs`, the names `exe` and `shared`, the `seal` steps | | the run | `runner`, `runners` | how `mcpp run` and `mcpp test` start an artifact, and named device actions | | the carrier | `carrier` | the program `mcpp pack` builds of a loaded artifact for a system: a self-contained tree's runner, a single file; a table or a list of tables, each serving the `-` pairs of its `hosts` | diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 1acdf6814..5d9e342d0 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -57,7 +57,7 @@ mcpp run --target x86_64-openkal | 目标 | `targets`、`toolchain`、`sysroot` | 服务哪些三元组、它们使用的工具链、预构建的目标基 | | 层 | `[platform.layers]` | `c-abi`、`c++-abi`、`compiler-runtime`、`kernel-abi` 的提供包,或 `false` | | 工具 | `[platform.tools]` | 为本机构建的程序,或查找到的程序(`{ program = "" }`) | -| 编译 | `[platform.build]`、`[platform.build.'cfg(…)']` | 目标每个翻译单元的标志 | +| 编译 | `[platform.build]`、`[platform.build.'cfg(…)']` | 目标每个翻译单元的标志，含 std 模块（2026.11.2.3） | | 链接 | `[platform.link]` | `driver`、`flags`、`start`、`script`、`inputs`、名字 `exe` 与 `shared`、封装步骤 `seal` | | 运行 | `runner`、`runners` | `mcpp run` 与 `mcpp test` 怎样启动产物,以及具名的设备动作 | | 载体 | `carrier` | `mcpp pack` 为依赖加载器的产物按系统构建的程序:self-contained 树的运行器、单文件;一个表或表的列表,每项服务其 `hosts` 中的 `-` | diff --git a/src/build/prepare/platform.cpp b/src/build/prepare/platform.cpp index 139e9f55a..abd5c0c02 100644 --- a/src/build/prepare/platform.cpp +++ b/src/build/prepare/platform.cpp @@ -550,13 +550,12 @@ std::vector split_words(std::string_view v) { } // namespace -void step13_platform_statement(PrepareState& state) { - auto& P = state.platform; +// ── The statement every reader uses: platform → build programs → this +// machine → the project → the environment (§4.5) ────────────────────────── +mp::Statement platform_statement(const PrepareState& state) { + auto const& P = state.platform; const auto t = target_of(state); const auto cfgCtx = platform_cfg_context(state); - - // ── The statement every reader uses: platform → build programs → this - // machine → the project → the environment (§4.5) ────────────────────── mp::Statement s = P.active ? state.platformData : mp::Statement{}; mp::Statement program = state.programPlatform; // The root's own program states them as well (applied into its manifest); @@ -595,8 +594,21 @@ void step13_platform_statement(PrepareState& state) { s.runner.set = !s.runner.cmd.empty(); } } - P.fields = std::move(s); - P.flags = mcpp::platform::evaluate_flags(P.fields, matcher(cfgCtx)); + return s; +} + +void step13_platform_statement(PrepareState& state) { + auto& P = state.platform; + P.fields = platform_statement(state); + P.flags = mcpp::platform::evaluate_flags(P.fields, matcher(platform_cfg_context(state))); +} + +// A unit's compile words the platform states (`[platform.build]` and its cfg +// tables), as they stand when they are asked for: the std module is compiled +// before the statement is final, and is a unit like any other. +std::vector platform_unit_cxxflags(const PrepareState& state) { + if (!state.platform.active) return {}; + return mcpp::platform::evaluate_flags(platform_statement(state), matcher(platform_cfg_context(state))).cxxflags; } std::expected step13_finish_platform(PrepareState& state, BuildContext& ctx) { diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index d4949b9f5..965eec8a6 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -1173,6 +1173,10 @@ std::expected step4b_check_platform_versions(PrepareState& st void step4b_platform_member_edges(PrepareState& state); void step5_platform_toolchain(PrepareState& state); void step13_platform_statement(PrepareState& state); +// The platform's statement and the compile words it states for a unit of C++, +// as they stand now (the std module's are asked for before step 13). +mcpp::manifest::platform::Statement platform_statement(const PrepareState& state); +std::vector platform_unit_cxxflags(const PrepareState& state); std::expected step13_finish_platform(PrepareState& state, BuildContext& ctx); // retarget.cpp: a retargetable driver told its target, sysroot and multilib. std::expected step2_retarget_for_retargetable_driver(PrepareState& state); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 4a7dedeaf..52de48bf1 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -11,6 +11,7 @@ import mcpp.targetside; import mcpp.diag; import mcpp.build.refusal; import mcpp.build.version_floor; +import mcpp.build.command_line; import mcpp.home; import mcpp.toolchain.platform; import mcpp.libs.json; @@ -2505,6 +2506,19 @@ static std::expected step9_rerun_input_prepare_dir(PrepareSta state.stdFlagAndDialect += ' '; state.stdFlagAndDialect += f; } + // AND THE PLATFORM'S CONVENTIONS (SPEC-013 §2.5), which come before every + // unit's own: the std module is a unit of the target side, whichever + // library supplies it. A flag of them that clang records in a BMI decides + // whether a unit may import it --- openkal-platform 0.23.0's + // `-fcf-protection=full` on x86_64, and a std module compiled without it was + // refused by every importer ("Control-Flow Branch Protection enabled was + // disabled in precompiled file"); `-D_LIBCPP_HARDENING_MODE` selects the + // library's own code inside the module. The words reach both of the + // module's commands and the key its BMI is kept under. + for (auto const& w : platform_unit_cxxflags(state)) { + state.stdFlagAndDialect += ' '; + state.stdFlagAndDialect += mcpp::build::command_line::quote(w); + } return {}; } diff --git a/tests/e2e/1010_the_std_module_takes_the_platforms_flags.sh b/tests/e2e/1010_the_std_module_takes_the_platforms_flags.sh new file mode 100755 index 000000000..9c5e44ae1 --- /dev/null +++ b/tests/e2e/1010_the_std_module_takes_the_platforms_flags.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# requires: llvm unix-shell +# 1010 -- the std module is compiled with the compile words the platform states +# for every unit (SPEC-013 §2.5; 2026.11.2.3). +# +# openkal-platform 0.23.0 compiles every unit on x86_64 with +# `-fcf-protection=full`, which clang records in a BMI and checks when a unit +# imports it. The std module was compiled without the platform's words, and +# every importer was refused: +# +# error: Control-Flow Branch Protection enabled was disabled in precompiled +# file '.../pcm.cache/std.pcm' but is currently enabled +# +# A platform for this machine's own triple states the flag here; a program +# that imports std builds and runs with it, and the module's command names it. +# The flag is the x86_64 one, so the test asks it of an x86_64 machine only. +set -e +case "$(uname -m)" in x86_64|amd64) ;; *) echo "SKIP: -fcf-protection is an x86_64 flag"; exit 0 ;; esac +source "$(dirname "$0")/_toolchain_env.sh" + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/plat" "$TMP/app/src" +cat > "$TMP/plat/mcpp.toml" <<'TOML' +[package] +name = "cet-platform" +version = "0.1.0" + +[platform] +targets = ["x86_64-linux-gnu"] + +[platform.build] +cxxflags = ["-fcf-protection=full"] +TOML + +cd "$TMP/app" +cat > mcpp.toml < src/main.cpp <<'CPP' +import std; +int main() { + std::vector v{1, 2, 3}; + std::println("std with the platform's words: {}", v.size()); + return 0; +} +CPP + +"$MCPP" build -v > build.log 2>&1 || fail "a program importing std did not build with the platform's flag" build.log +grep -q 'Control-Flow Branch Protection' build.log && fail "the std module was refused for the flag" build.log +grep -Eq 'std\.cppm.*-fcf-protection=full|-fcf-protection=full.*std\.cppm' build.log \ + || grep -q 'fcf-protection=full' target/*/*/build.ninja 2>/dev/null \ + || fail "the std module's command does not name the platform's flag" build.log +"$MCPP" run > run.log 2>&1 || fail "the program did not run" run.log +grep -q "std with the platform's words: 3" run.log || fail "the program's output" run.log +echo "OK" From 1b98cfce611f6720cbb4d9d7a630db54852fcd31 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 11:26:40 +0900 Subject: [PATCH 10/14] =?UTF-8?q?design=20part=204=20=C2=A718.6:=20the=20t?= =?UTF-8?q?wo=20defects=20the=20hardening=20found=20beneath=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- ...26-11-03-ecosystem-part4-beyond-native-design.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md index a8fe63e70..88f1eccbb 100644 --- a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md +++ b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md @@ -692,6 +692,17 @@ aarch64 用 `-mbranch-protection=pac-ret+bti`，不用 §9.1 的 `standard`：cl （定义 `__ARM_FEATURE_GCS_DEFAULT`），libunwind 因此按 GCS 展开并包含 C 库的 ``，映像的构建在 aarch64 上失败； GCS 还需要系统为程序启用该栈，映像没有这一环节。返回地址签名与 BTI 标记不变。 +加固选项在平台的 CI 中暴露了两处下层缺陷，各在其所属的层修正： + +1. **compiler-rt 的 outline atomics（openkal-llvm-runtime 0.18.1）**：生成的 125 个 `oa_*.S` 先按自身路径包含 compiler-rt 的 + `assembly.h`，`lse.S` 再以 `"assembly.h"` 查找，包含路径给出 libunwind 的同名头文件，它把 `GNU_PROPERTY` 重定义为数字； + 有分支保护时 compiler-rt 写出的属性 note 无法汇编。各文件在包含 `lse.S` 前声明 libunwind 头文件的守卫；目标文件带有映像 + 加载器强制 BTI 前所读的 `aarch64 feature: BTI, PAC`。 +2. **std 模块的编译选项（mcpp 2026.11.2.3）**：平台的 `[platform.build]` 选项原先到达每个翻译单元而不到达 std 模块；clang 把 + `-fcf-protection` 记入 BMI，x86_64 上每个 `import std` 的单元都被拒绝。平台的选项现在进入 std 模块的两条命令与其 BMI 的缓存 + 键，无论 std 模块由工具链还是图中的包提供（e2e 1010）。openkal-platform 0.23.0 因此声明 `mcpp >= 2026.11.2.3`，其 CI 在 mcpp + 发布前以发布分支验证（`MCPP_SOURCE_REF`）。 + ### 18.7 交付（实际版本） | 仓库 | 版本 | @@ -699,7 +710,7 @@ GCS 还需要系统为程序启用该栈，映像没有这一环节。返回地 | openkal | 0.23.0 | | openkal-linux / -macos / -windows / -emscripten | 0.24.0 / 0.21.2 / 0.19.0 / 0.12.0 | | openkal-image（ld、run、tools、bind） | 0.4.1（0.4.0 之后以 openkal-macos 0.21.2 重建） | -| openkal-musl / openkal-llvm-runtime / openkal-platform | 0.27.0 / 0.18.0 / 0.23.0 | +| openkal-musl / openkal-llvm-runtime / openkal-platform | 0.27.0 / 0.18.1 / 0.23.0 | | mcpplibs.os | 0.2.8 | | mcppls-clangd / mcppls | 0.0.20 / 0.0.20 | | mcpp | 2026.11.2.3 | From 2ffe8a975383a4996117e0300ce933295f108dbc Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 11:35:58 +0900 Subject: [PATCH 11/14] prepare: P9's build programs in build_programs.cpp, cut from target_side.cpp at that step's boundary (2,546 lines against the 2,500 the layout keeps) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- src/build/prepare/build_programs.cpp | 645 +++++++++++++++++++++++++++ src/build/prepare/state.cppm | 4 + src/build/prepare/target_side.cpp | 578 ------------------------ 3 files changed, 649 insertions(+), 578 deletions(-) create mode 100644 src/build/prepare/build_programs.cpp diff --git a/src/build/prepare/build_programs.cpp b/src/build/prepare/build_programs.cpp new file mode 100644 index 000000000..7d831d9bc --- /dev/null +++ b/src/build/prepare/build_programs.cpp @@ -0,0 +1,645 @@ +// build_programs.cpp -- P9's build programs: the root's `build.mcpp`, and +// each member's, run once the target side is resolved (cut from +// target_side.cpp at that step's boundary, as check_file_lengths.sh asks). + +module mcpp.build.prepare; +import :state; + +import mcpp.build.prepare_inputs; + +import std; +import mcpp.targetside; +import mcpp.diag; +import mcpp.build.refusal; +import mcpp.build.version_floor; +import mcpp.build.command_line; +import mcpp.home; +import mcpp.toolchain.platform; +import mcpp.libs.json; +import mcpp.log; +import mcpp.manifest; +import mcpp.source_kind; +import mcpp.modgraph.glob; +import mcpp.graph; +import mcpp.build.progress; // the members' programs wait in order (design 2026-09-29 §4.2) +import mcpp.build.schedule.policy; // resolve_jobs — how many programs compile at once (#748, B2) +import mcpp.modgraph.graph; +import mcpp.modgraph.scanner; +import mcpp.modgraph.validate; +import mcpp.toolchain.hostflags; // the compile-token producer the package std module reuses +import mcpp.toolchain.cenv; // [c-abi] declaration → compiler configuration (design 2026-09-18) +import mcpp.toolchain.cenv_probe; // [c-abi] declaration is checked, not trusted (design §3.2) +import mcpp.toolchain.predefines; // the macros this engine defines: contract and emission in one module +import mcpp.toolchain.cppfly; +import mcpp.toolchain.detect; +import mcpp.toolchain.dialect; +import mcpp.toolchain.fingerprint; +import mcpp.toolchain.registry; +import mcpp.toolchain.linkmodel; +// For `resolve_version_match` / `list_installed_versions`: a bare compiler +// family named by the dependency graph resolves to a concrete version through +// exactly the path `mcpp toolchain default ` uses. +import mcpp.toolchain.lifecycle; +import mcpp.toolchain.stdmod; +import mcpp.freestanding.target; // the target sysroot layout (libdir) +import mcpp.freestanding.linkline; // the ISA profile, for the std module command +import mcpp.toolchain.post_install; +import mcpp.toolchain.abi; +import mcpp.toolchain.triple; +import mcpp.build.linkage_form; // #519 — which form each dependency takes +import mcpp.build.plan; +import mcpp.build.flags; // compute_flags — the per-role contracts (#418) +import mcpp.build.graph_shape; // #407: the graph says which mode wrote it +import mcpp.pack.abi_tag; // the tag a prebuilt dependency is checked against +import mcpp.pack.prebuilt; // …and the check itself +import mcpp.pack.stage_tree; // where `${mcpp.stage_dir}` points, and its manifest +import mcpp.build.build_program; +import mcpp.build.backend; // BuildOptions for the tool sub-build +import mcpp.build.ninja; // make_ninja_backend — driving that sub-build +import mcpp.toolchain.post_install; +import mcpp.ui; +import mcpp.log; +import mcpp.wire; // Severity, for PlanNote (#699 item 2, E3) + +namespace mcpp::build { + +std::expected step9_root_build_program(PrepareState& state) { + // ── L3: ROOT build.mcpp (moved after dependency resolution, design §3.1 + // item 4) ──────────────────────────────────────────────────────────────── + // Runs HERE — after dep resolution + feature activation (so the contract + // env can expose MCPP_DEP__DIR exactly like the dep loop above does) + // and BEFORE the modgraph scan / flag canonicalization / fingerprint (so + // its generated=/source= sources and flag directives are fully visible). + // Ordering invariants preserved relative to the pre-move call site: + // materialize_generated_files (may produce build.mcpp itself) and the L1 + // cfg merge still run earlier — ONLY this call moved later. + // + // One wrinkle the old ordering hid: back then apply() mutated *m BEFORE + // `packages[0] = makePackageRoot(*root, *m)` snapshotted buildConfig into + // privateBuild/manifest — the copies the scan and per-TU flag assembly + // actually read. Now the snapshot (and root feature activation on it) + // already happened, so mirror the directive TAILS into packages[0] + // explicitly, the same way the dep loop does for its package. + // A package with no `build.mcpp` still runs one when a rule dependency + // described it: `run_build_program` writes that program into the build + // directory. This guard therefore asks the same question the function does, + // and a guard that asked only about the file left the synthesis unreachable + // -- the shaders went uncompiled and the refusal named the missing program + // rather than the guard. Measured. + // A workspace plan's virtual root has no program; its members' programs + // run in `step9_member_build_programs`, below. + if (state.m->package.virtualRoot) return {}; + if (std::filesystem::exists(*state.root / "build.mcpp") + || !state.m->buildConfig.ruleModules.empty()) { + auto host = state.host_tc_for_build_program(); + if (!host) return std::unexpected(host.error()); + mcpp::build::BuildProgramEnv bpEnv; + bpEnv.targetTriple = state.resolvedTargetCanonical; + // Everything the engine already knows and a build program would + // otherwise hardcode: the payload ROOT (not the driver), the target's + // C library, which compiler and which C++ standard library resolved, + // and the three answers that keep a board package from naming a + // toolchain. One call — see fill_target_build_env. + fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, state.cfg_opt ? &*state.cfg_opt : nullptr); + apply_graph_cxx_abi(bpEnv, state.packages); + bpEnv.toolsBin = state.projectSubosBin; + bpEnv.profile = state.effectiveProfile; + bpEnv.accel = state.resolvedAccel(); + fill_package_build_env(bpEnv, *state.m); + state.fillPackEnv(bpEnv, 0); + bpEnv.languageModules = state.m->language.modules; + bpEnv.ruleModules = state.m->buildConfig.ruleModules; + if (auto dit = state.deviceSourcesByPackage.find(state.root->string()); dit != state.deviceSourcesByPackage.end()) + bpEnv.deviceSources = dit->second; + // Set explicitly rather than relying on build_dir()'s root-relative + // default: under BuildOverrides::work_dir the package root is shared + // and may be read-only, and the default would write the compiled + // helper straight into it. Same value as the default when work_dir is + // unset, so an ordinary build is unchanged. + bpEnv.artifactsDir = build_program_dir(state, 0); + // Root mode keeps genBase empty: a relative `generated=` from the ROOT + // package resolves against the package root (the documented contract), + // not against OUT_DIR. + // Same expression as the pre-move call site (and same order), so the + // contract hash — and therefore the build.mcpp cache — is unchanged + // across the move for feature-identical builds. + bpEnv.features = feature_closure(*state.m, parse_feature_request(state.overrides.features)); + bpEnv.outDir = state.programOutDir(state.workRoot, *state.m, bpEnv.features); + // mcpp#241 (root): consumer index 0, same owner as the dep loop. + // + // AND THE LINK FORM OF EACH DEPENDENCY (#642 E2), to this program only. + // The root decides every dependency's form, and when this program runs + // every input of that decision is final: the requests are the root + // manifest's, and each dependency's own program has already run. A + // DEPENDENCY's program is not offered the forms. It runs in discovery + // order, before the programs of packages discovered after it, and those + // programs supply facts the answer depends on (a `-L` they add makes a + // package static-only), so the value it could be given would be a guess. + { + std::map rootLinkForms; + for (auto const& [idx, form] : state.dependencyLinkForms) + if (form.recorded) + rootLinkForms.emplace(idx, std::string( + mcpp::build::linkage_form::to_string(form.answer.linkage))); + state.fillDepDirs(bpEnv, 0, &rootLinkForms); + } + state.fillXpkgDirs(bpEnv, *state.m, 0); + // #355: the host tools the ROOT package requested (consumer index 0). + if (auto tit = state.toolEnvByConsumer.find(0u); tit != state.toolEnvByConsumer.end()) + bpEnv.toolPaths = tit->second; + // The program, as the build programs' graph compiled it (mcpp#797). + if (auto program = build_program_binary(state, 0); program) { + bpEnv.programBinary = program->path; + bpEnv.programRunner = program->runner; + bpEnv.programIdentity = program->identity; + bpEnv.programCompile = program->compile; + } else if (!state.overrides.plan_only) { + return std::unexpected(program.error()); + } else { + bpEnv.programError = program.error(); + } + // #647 E1: THE RESOLVED GRAPH, FOR THE ROOT'S PROGRAM ONLY. + // + // Every package, dependencies before the packages that request them + // (ties in discovery order), so a program that merges what libraries + // contribute can apply them in override order without a sort of its + // own. The root decides the graph, and every input of that decision is + // final here -- the same reason `dep_linkage` is offered to this + // program alone. A file, not variables: a graph with metadata does not + // fit an environment block (`MAX_ARG_STRLEN`, the Windows limit). + // + // ITS DIGEST JOINS THE RE-RUN KEY. Editing a dependency's + // `[package.metadata]` changes what this program would answer, so it + // must run again; editing that dependency's sources does not, and the + // document does not change. + { + std::vector order; + std::vector placed(state.packages.size(), false); + while (order.size() < state.packages.size()) { + std::size_t pick = state.packages.size(); + for (std::size_t i = 0; i < state.packages.size() && pick == state.packages.size(); ++i) { + if (placed[i]) continue; + bool ready = true; + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i + && r.dependencyPackageIndex < state.packages.size() + && !placed[r.dependencyPackageIndex]) { ready = false; break; } + if (ready) pick = i; + } + // A cycle leaves nothing ready; its first member in discovery + // order is taken so the document is still complete. + if (pick == state.packages.size()) + for (std::size_t i = 0; i < state.packages.size(); ++i) + if (!placed[i]) { pick = i; break; } + placed[pick] = true; + order.push_back(pick); + } + nlohmann::json doc; + doc["kind"] = "mcpp.graph"; + doc["version"] = 1; + nlohmann::json list = nlohmann::json::array(); + for (auto i : order) list.push_back(state.graph_package_entry(i, true, nullptr)); + doc["packages"] = std::move(list); + const auto text = doc.dump(2) + "\n"; + const auto graphPath = bpEnv.artifactsDir / "graph.json"; + std::error_code gec; + std::filesystem::create_directories(graphPath.parent_path(), gec); + const auto tmp = graphPath.string() + ".tmp"; + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << text; + } + std::filesystem::rename(tmp, graphPath, gec); + if (gec) + return std::unexpected(std::format( + "cannot write the graph document '{}': {}", + graphPath.string(), gec.message())); + bpEnv.graphFile = graphPath; + bpEnv.graphDigest = mcpp::toolchain::hash_string(text); + } + auto& bcRoot = state.m->buildConfig; + const auto mark = state.markDirectiveTail(*state.m); + const auto rldN = bcRoot.ldflags.size(), rsrcN = bcRoot.sources.size(), + rmodN = state.m->modules.sources.size(); + const auto ractN = bcRoot.actions.size(); + // #622 A4: how many `[runtime] deploy` entries existed before this + // program ran — the manifest-sourced ones, already in `packages[0]`'s + // snapshot. Anything past this index is a `mcpp::deploy()` residue + // that needs the same mirror the flag/source tails get below. + const auto rdeployN = state.m->runtimeConfig.linkIntent.deploy.size(); + // Same reason, one field wide: `mcpp::runtime_search_dir()` residue + // needs the same mirror `deploy` does, or `resolve_runtime_contract` + // (which reads `packages[0]`'s snapshot, not `*m`) never sees it. + const auto rsearchDirN = state.m->runtimeConfig.linkIntent.runtimeSearchDirs.size(); + // What the dependencies supplied as runners, before the root's program + // speaks. The root's emissions are appended to the same slots, so a + // name both supply becomes one argv joining the two (#634, §9 item 8, + // measured: `run-A.sh run-B.sh `). + const auto runnerBeforeRoot = bcRoot.runner; + const auto namedBeforeRoot = bcRoot.namedRunners; + // The root's program is the requested package's. + bpEnv.requested = !state.m->package.virtualRoot; + auto bp = run_answering_requests(state, *state.m, bpEnv, 0, + state.m->package.name, [&] { + return mcpp::build::run_build_program( + *state.m, *state.root, host->first, host->second, + state.m->cppStandard, bpEnv); + }); + if (!bp && !state.overrides.plan_only) { + return std::unexpected(bp.error()); + } + // #699 item 2 (E3): under `emit build-database` (`plan_only`), a + // failing root build program describes the package without its + // directives rather than costing the whole plan. Every mirror below + // reads what the program would have added to `*m`, so skipping + // straight past it (nothing runs on this path) is what "without its + // directives" means; a later failure that follows from the gap + // fails the member under the ordinary rule (E1). + if (!bp) { + state.planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + bp.error(), mcpp::wire::Severity::Error, + (*state.root / "build.mcpp").string()}); + // Named so the device-source check below (and anything else whose + // premise is this program's directives) can tell a package whose + // program failed apart from one that simply has no program. + state.programFailedPackages.insert(state.root->string()); + } + if (bp) { + // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. + // Two suppliers of one runner are refused naming both, and the + // manifest is the way to choose: a `[target.]` runner the + // project writes outranks every supplied one where the runner is + // looked up, so a name the manifest declares is not refused here. + { + const auto parsedTarget = state.tc + ? mcpp::toolchain::triple::parse(state.tc->targetTriple) + : std::nullopt; + const auto rowKey = parsedTarget ? parsedTarget->str() + : state.tc ? state.tc->targetTriple : std::string{}; + const auto* row = [&]() -> const mcpp::manifest::TargetEntry* { + if (parsedTarget) return find_target_entry(*state.m, *parsedTarget); + auto it = state.m->targetOverrides.find(rowKey); + return it == state.m->targetOverrides.end() ? nullptr : &it->second; + }(); + const auto manifestNames = [&](std::string_view name) { + if (!row) return false; + if (name.empty()) return !row->runner.empty(); + return row->namedRunners.contains(std::string(name)); + }; + if (!state.runnerProvider.empty() && !runnerBeforeRoot.empty() + && bcRoot.runner.size() > runnerBeforeRoot.size() + && !manifestNames({})) { + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply the runner for this target, and the two would be " + "joined into one argv.\n" + " Drop one of them, or state the runner in " + "[target.{}].runner.", + state.runnerProvider, rowKey)); + } + for (auto const& [name, nr] : bcRoot.namedRunners) { + auto before = namedBeforeRoot.find(name); + auto who = state.namedRunnerProvider.find(name); + if (before == namedBeforeRoot.end() || before->second.argv.empty() + || who == state.namedRunnerProvider.end() || who->second.empty()) + continue; + if (nr.argv.size() <= before->second.argv.size()) continue; + if (manifestNames(name)) continue; + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply a runner named '{}' for this target, and the two " + "would be joined into one argv.\n" + " Drop one of them, or state it in " + "[target.{}.runners].{}.", + who->second, name, rowKey, name)); + } + } + auto& pkg0 = state.packages[0]; + // Compile-visible tail → privateBuild: the shared fold (same owner + // as the dep loop; the root's TUs read privateBuild). + state.foldDirectiveTailIntoPrivateBuild(pkg0, *state.m, mark); + // Before the source residues are mirrored below: adopting an action's + // outputs APPENDS to bcRoot.sources, and those appends must be inside + // the tail that gets copied into the packages[0] snapshot the scan reads. + state.adoptActionOutputs(*state.m, *state.root, ractN); + // The root's build program has spoken; a floor it stated is checked + // now, with the facts every package (it included) established. + if (auto err = state.checkVersionFloors(); err) return std::unexpected(*err); + // Root residues — apply() mutated *m, but packages[0].manifest is a + // value-copy snapshot taken at makePackageRoot, so everything the + // scan/fingerprint read from the snapshot needs the tail mirrored: + // sources → the scan walks packages[0].manifest, not *m. + pkg0.manifest.buildConfig.sources.insert( + pkg0.manifest.buildConfig.sources.end(), + bcRoot.sources.begin() + rsrcN, bcRoot.sources.end()); + pkg0.manifest.modules.sources.insert( + pkg0.manifest.modules.sources.end(), + state.m->modules.sources.begin() + rmodN, state.m->modules.sources.end()); + // The units its actions generate, as they declared them (D6). + for (auto const& [path, unit] : state.m->modules.declaredUnits) + pkg0.manifest.modules.declaredUnits.insert_or_assign(path, unit); + // The snapshot's manifest is what later readers of the root's + // flags see -- mirror the flag/include tails, as the old + // pre-snapshot ordering implicitly did. + pkg0.manifest.buildConfig.cflags.insert( + pkg0.manifest.buildConfig.cflags.end(), + bcRoot.cflags.begin() + static_cast(mark.cflags), + bcRoot.cflags.end()); + pkg0.manifest.buildConfig.cxxflags.insert( + pkg0.manifest.buildConfig.cxxflags.end(), + bcRoot.cxxflags.begin() + static_cast(mark.cxxflags), + bcRoot.cxxflags.end()); + pkg0.manifest.buildConfig.includeDirs.insert( + pkg0.manifest.buildConfig.includeDirs.end(), + bcRoot.includeDirs.begin() + static_cast(mark.includeDirs), + bcRoot.includeDirs.end()); + pkg0.manifest.buildConfig.includeDirsAfter.insert( + pkg0.manifest.buildConfig.includeDirsAfter.end(), + bcRoot.includeDirsAfter.begin() + + static_cast(mark.includeDirsAfter), + bcRoot.includeDirsAfter.end()); + // Link flags → the final link reads *m (already applied); keep the + // linkUsage snapshot and fingerprint metadata equivalent too. + pkg0.linkUsage.ldflags.insert(pkg0.linkUsage.ldflags.end(), + bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); + pkg0.manifest.buildConfig.ldflags.insert( + pkg0.manifest.buildConfig.ldflags.end(), + bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); + // #622 A4: `mcpp::deploy()` residue → `packages[0].manifest`, the + // object `resolve_runtime_contract` (plan.cppm) actually reads. + // Without this mirror a directive-sourced deploy entry lands in `*m` + // and nowhere the planner looks — the same gap this block already + // closes for sources/flags, one more field wide. + pkg0.manifest.runtimeConfig.linkIntent.deploy.insert( + pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), + state.m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), + state.m->runtimeConfig.linkIntent.deploy.end()); + // `mcpp::runtime_search_dir()` residue → `packages[0].manifest`, the + // same object and the same reason as the `deploy` mirror above: without + // it a directive-sourced entry lands in `*m` and `resolve_runtime_contract` + // never looks there. + pkg0.manifest.runtimeConfig.linkIntent.runtimeSearchDirs.insert( + pkg0.manifest.runtimeConfig.linkIntent.runtimeSearchDirs.end(), + state.m->runtimeConfig.linkIntent.runtimeSearchDirs.begin() + + static_cast(rsearchDirN), + state.m->runtimeConfig.linkIntent.runtimeSearchDirs.end()); + } + } + return {}; +} + +// The graph document a build program reads (#647 E1): every package the +// program's own package reaches, itself included, dependencies before the +// packages that request them (ties in discovery order), with `root` marking +// the package whose program reads it. For a root that is the whole graph. +static std::expected, std::string> +write_graph_document(PrepareState& state, std::size_t subject, + const std::filesystem::path& artifactsDir) { + std::vector reached(state.packages.size(), subject == 0); + reached[subject] = true; + for (bool grew = true; grew;) { + grew = false; + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex < reached.size() && reached[r.consumerPackageIndex] + && r.dependencyPackageIndex < reached.size() + && !reached[r.dependencyPackageIndex]) { + reached[r.dependencyPackageIndex] = true; + grew = true; + } + } + std::vector order; + std::vector placed(state.packages.size(), false); + std::size_t wanted = 0; + for (bool b : reached) wanted += b ? 1 : 0; + while (order.size() < wanted) { + std::size_t pick = state.packages.size(); + for (std::size_t i = 0; i < state.packages.size() && pick == state.packages.size(); ++i) { + if (placed[i] || !reached[i]) continue; + bool ready = true; + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i + && r.dependencyPackageIndex < state.packages.size() + && reached[r.dependencyPackageIndex] + && !placed[r.dependencyPackageIndex]) { ready = false; break; } + if (ready) pick = i; + } + // A cycle leaves nothing ready; its first member in discovery + // order is taken so the document is still complete. + if (pick == state.packages.size()) + for (std::size_t i = 0; i < state.packages.size(); ++i) + if (!placed[i] && reached[i]) { pick = i; break; } + placed[pick] = true; + order.push_back(pick); + } + nlohmann::json doc; + doc["kind"] = "mcpp.graph"; + doc["version"] = 1; + // The requests a document lists are those made by its own packages. A + // request from outside the program's closure (another workspace member, a + // workspace plan's virtual root) is a fact about the plan, not about the + // package, and would make the document, and with it the program's cache + // key, depend on which members the command selected. + nlohmann::json list = nlohmann::json::array(); + for (auto i : order) { + auto entry = state.graph_package_entry(i, true, &reached); + entry["root"] = i == subject; + list.push_back(std::move(entry)); + } + doc["packages"] = std::move(list); + const auto text = doc.dump(2) + "\n"; + const auto graphPath = artifactsDir / "graph.json"; + std::error_code gec; + std::filesystem::create_directories(graphPath.parent_path(), gec); + const auto tmp = graphPath.string() + ".tmp"; + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << text; + } + std::filesystem::rename(tmp, graphPath, gec); + if (gec) + return std::unexpected(std::format( + "cannot write the graph document '{}': {}", + graphPath.string(), gec.message())); + return std::pair{graphPath, mcpp::toolchain::hash_string(text)}; +} + +// The build programs of a workspace plan's members (workspace design +// 2026-09-29 §15). A member is the project being developed, so its program +// runs where a root's does: after every dependency's program, with the link +// form of each dependency and the graph document of what it reaches, and with +// its own artifacts directory, `/target/.build-mcpp`, where it was +// when the member was the root of its own build. A workspace member reached as +// a dependency of another member runs here too, so its program sees the same +// environment in every selection. +std::expected step9_member_build_programs(PrepareState& state) { + if (!state.m->package.virtualRoot) return {}; + // Dependencies first: a member's program runs after the programs of the + // members it depends on, whose directives are then in place in the graph + // it reads. Discovery order, which the packages are numbered in, puts a + // member before its dependency when its key sorts first. + mcpp::graph::AdjacencyList deps(state.packages.size()); + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex < deps.size() && r.dependencyPackageIndex < deps.size() + && r.consumerPackageIndex != r.dependencyPackageIndex) + deps[r.consumerPackageIndex].push_back(r.dependencyPackageIndex); + // Mutual dev-dependencies between members are legal, so a cycle skips its + // closing edge and every other dependency still comes first. + std::vector roots(state.packages.size()); + std::iota(roots.begin(), roots.end(), std::size_t{0}); + const auto order = *mcpp::graph::depth_first_order(deps, roots, mcpp::graph::Cycles::Skip); + auto hasProgram = [&](std::size_t i) { + if (i == 0 || !state.isWorkspaceMemberPackage(i) || !state.compilesHere(i)) return false; + std::error_code bpEc; + return std::filesystem::exists(state.packages[i].root / "build.mcpp", bpEc) + || !state.packages[i].manifest.buildConfig.ruleModules.empty(); + }; + // Their compiles may overlap (below), and their runs follow this order, so + // every program not yet run is waiting for its turn; its line says so + // (build progress design 2026-09-29, §3.2), named as the program names + // itself. + for (auto const i : order) { + if (!hasProgram(i)) continue; + const auto& m = state.packages[i].manifest; + mcpp::build::progress::program_scheduled( + m.package.namespace_.empty() ? m.package.name + : m.package.namespace_ + "." + m.package.name, + state.packages[i].selectedMember); + } + // THE ENVIRONMENT A MEMBER'S PROGRAM IS GIVEN. Computed from the plan as it + // stands when it is asked, so the compile phase and the run each ask for it. + auto program_env = [&](std::size_t i) + -> std::expected { + auto& pkg = state.packages[i]; + mcpp::build::BuildProgramEnv bpEnv; + bpEnv.targetTriple = state.resolvedTargetCanonical; + fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, + state.cfg_opt ? &*state.cfg_opt : nullptr); + apply_graph_cxx_abi(bpEnv, state.packages); + bpEnv.toolsBin = state.projectSubosBin; + bpEnv.profile = state.effectiveProfile; + bpEnv.accel = state.resolvedAccel(); + fill_package_build_env(bpEnv, pkg.manifest); + // The stage of the member this program acts for: its own, or the one + // packed member that reaches it (`PrepareState::fillPackEnv`). + state.fillPackEnv(bpEnv, i); + bpEnv.requested = pkg.selectedMember; + bpEnv.languageModules = pkg.manifest.language.modules; + bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; + if (auto dit = state.deviceSourcesByPackage.find(pkg.root.string()); + dit != state.deviceSourcesByPackage.end()) + bpEnv.deviceSources = dit->second; + bpEnv.artifactsDir = build_program_dir(state, i); + if (i < state.activeFeaturesByPackage.size()) + bpEnv.features = state.activeFeaturesByPackage[i]; + // Beside the cache above, which is in the project under a plan's + // work directory too: `emit build-database` reuses the build's run. + bpEnv.outDir = state.programOutDir(*state.root, pkg.manifest, bpEnv.features); + { + std::map linkForms; + for (auto const& [idx, form] : state.dependencyLinkForms) + if (form.recorded) + linkForms.emplace(idx, std::string( + mcpp::build::linkage_form::to_string(form.answer.linkage))); + state.fillDepDirs(bpEnv, i, &linkForms); + } + state.fillXpkgDirs(bpEnv, pkg.manifest, i); + if (auto tit = state.toolEnvByConsumer.find(i); tit != state.toolEnvByConsumer.end()) + bpEnv.toolPaths = tit->second; + // The program, as the build programs' graph compiled it (mcpp#797). + if (auto program = build_program_binary(state, i); program) { + bpEnv.programBinary = program->path; + bpEnv.programRunner = program->runner; + bpEnv.programIdentity = program->identity; + bpEnv.programCompile = program->compile; + } else { + bpEnv.programError = program.error(); + } + auto graph = write_graph_document(state, i, bpEnv.artifactsDir); + if (!graph) return std::unexpected(graph.error()); + bpEnv.graphFile = graph->first; + bpEnv.graphDigest = graph->second; + return bpEnv; + }; + + // THE COMPILES WERE MADE TOGETHER: every program this plan runs is compiled + // in one build of the build programs' graph, before the dependencies' + // programs run (features.cpp, mcpp#797). The programs run here, one at a + // time, in the order above. + for (auto const i : order) { + if (!hasProgram(i)) continue; + auto& pkg = state.packages[i]; + auto host = state.host_tc_for_build_program(); + if (!host) return std::unexpected(host.error()); + auto envOf = program_env(i); + if (!envOf) return std::unexpected(envOf.error()); + auto& bpEnv = *envOf; + + auto& bc = pkg.manifest.buildConfig; + const auto mark = state.markDirectiveTail(pkg.manifest); + const auto actN = bc.actions.size(); + const auto runnerN = bc.runner.size(); + auto namedBefore = bc.namedRunners; + const bool exclusiveBefore = bc.runExclusive; + auto bp = run_answering_requests(state, pkg.manifest, bpEnv, i, + pkg.manifest.package.name, [&] { + return mcpp::build::run_build_program( + pkg.manifest, pkg.root, host->first, host->second, + pkg.manifest.cppStandard, bpEnv); + }); + if (!bp) { + if (!state.overrides.plan_only) + return std::unexpected(std::format( + "workspace member '{}': {}", pkg.manifest.package.name, bp.error())); + state.planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + std::format("workspace member '{}': {}", pkg.manifest.package.name, bp.error()), + mcpp::wire::Severity::Error, (pkg.root / "build.mcpp").string()}); + state.programFailedPackages.insert(pkg.root.string()); + continue; + } + state.foldDirectiveTailIntoPrivateBuild(pkg, pkg.manifest, mark); + state.adoptActionOutputs(pkg.manifest, pkg.root, actN); + // How the artifact is executed reaches the plan's root, where `mcpp + // run` reads it, by the rule the dependencies' runners follow: one + // supplier per runner. + auto& rootBc = state.m->buildConfig; + if (bc.runner.size() > runnerN) { + if (!rootBc.runner.empty() && !state.runnerProvider.empty()) + return std::unexpected(std::format( + "'{}' and '{}' both supply a runner for this target.\n" + " Drop one of them, or state the runner in " + "[target.].runner.", + state.runnerProvider, pkg.manifest.package.name)); + rootBc.runner.assign(bc.runner.begin() + static_cast(runnerN), + bc.runner.end()); + state.runnerProvider = pkg.manifest.package.name; + } + for (auto const& [name, nr] : bc.namedRunners) { + auto before = namedBefore.find(name); + const bool grew = before == namedBefore.end() + || nr.argv.size() > before->second.argv.size() + || (nr.longLived && !before->second.longLived); + if (!grew) continue; + auto& slot = rootBc.namedRunners[name]; + auto& who = state.namedRunnerProvider[name]; + if (!slot.argv.empty() && !who.empty()) + return std::unexpected(std::format( + "'{}' and '{}' both supply a runner named '{}' for this target.", + who, pkg.manifest.package.name, name)); + slot = nr; + who = pkg.manifest.package.name; + } + if (bc.runExclusive && !exclusiveBefore) rootBc.runExclusive = true; + // The platform's fields the member's program stated, as a dependency's + // program's are (SPEC-013 §4.1). + merge_program_platform(state, std::exchange(bc.programPlatform, {}), + pkg.manifest.package.name); + } + if (auto err = state.checkVersionFloors(); err) return std::unexpected(*err); + // What the programs published (an include directory, an interface + // define) reaches the members' consumers, as after the dependencies' + // programs. + state.computeUsageRequirements(); + return {}; +} + +} // namespace mcpp::build diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 965eec8a6..1a22e8c9a 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -1164,6 +1164,10 @@ PackageInstances package_instances(const PrepareState& state); // build-program tier of the build's platform statement, named after `who`. void merge_program_platform(PrepareState& state, mcpp::manifest::platform::Statement stated, std::string_view who); +// P9's build programs (prepare/build_programs.cpp): the root's, then each +// member's, once the target side is resolved. +std::expected step9_root_build_program(PrepareState& state); +std::expected step9_member_build_programs(PrepareState& state); // The platform through the phases (prepare/platform.cpp). mcpp::build::cfgpred::Ctx platform_cfg_context(const PrepareState& state); std::expected step1_select_platform(PrepareState& state); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 52de48bf1..1863885ed 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -1711,584 +1711,6 @@ static void step9_define_graph_package_entry_closure(PrepareState& state) { } -static std::expected step9_root_build_program(PrepareState& state) { - // ── L3: ROOT build.mcpp (moved after dependency resolution, design §3.1 - // item 4) ──────────────────────────────────────────────────────────────── - // Runs HERE — after dep resolution + feature activation (so the contract - // env can expose MCPP_DEP__DIR exactly like the dep loop above does) - // and BEFORE the modgraph scan / flag canonicalization / fingerprint (so - // its generated=/source= sources and flag directives are fully visible). - // Ordering invariants preserved relative to the pre-move call site: - // materialize_generated_files (may produce build.mcpp itself) and the L1 - // cfg merge still run earlier — ONLY this call moved later. - // - // One wrinkle the old ordering hid: back then apply() mutated *m BEFORE - // `packages[0] = makePackageRoot(*root, *m)` snapshotted buildConfig into - // privateBuild/manifest — the copies the scan and per-TU flag assembly - // actually read. Now the snapshot (and root feature activation on it) - // already happened, so mirror the directive TAILS into packages[0] - // explicitly, the same way the dep loop does for its package. - // A package with no `build.mcpp` still runs one when a rule dependency - // described it: `run_build_program` writes that program into the build - // directory. This guard therefore asks the same question the function does, - // and a guard that asked only about the file left the synthesis unreachable - // -- the shaders went uncompiled and the refusal named the missing program - // rather than the guard. Measured. - // A workspace plan's virtual root has no program; its members' programs - // run in `step9_member_build_programs`, below. - if (state.m->package.virtualRoot) return {}; - if (std::filesystem::exists(*state.root / "build.mcpp") - || !state.m->buildConfig.ruleModules.empty()) { - auto host = state.host_tc_for_build_program(); - if (!host) return std::unexpected(host.error()); - mcpp::build::BuildProgramEnv bpEnv; - bpEnv.targetTriple = state.resolvedTargetCanonical; - // Everything the engine already knows and a build program would - // otherwise hardcode: the payload ROOT (not the driver), the target's - // C library, which compiler and which C++ standard library resolved, - // and the three answers that keep a board package from naming a - // toolchain. One call — see fill_target_build_env. - fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, state.cfg_opt ? &*state.cfg_opt : nullptr); - apply_graph_cxx_abi(bpEnv, state.packages); - bpEnv.toolsBin = state.projectSubosBin; - bpEnv.profile = state.effectiveProfile; - bpEnv.accel = state.resolvedAccel(); - fill_package_build_env(bpEnv, *state.m); - state.fillPackEnv(bpEnv, 0); - bpEnv.languageModules = state.m->language.modules; - bpEnv.ruleModules = state.m->buildConfig.ruleModules; - if (auto dit = state.deviceSourcesByPackage.find(state.root->string()); dit != state.deviceSourcesByPackage.end()) - bpEnv.deviceSources = dit->second; - // Set explicitly rather than relying on build_dir()'s root-relative - // default: under BuildOverrides::work_dir the package root is shared - // and may be read-only, and the default would write the compiled - // helper straight into it. Same value as the default when work_dir is - // unset, so an ordinary build is unchanged. - bpEnv.artifactsDir = build_program_dir(state, 0); - // Root mode keeps genBase empty: a relative `generated=` from the ROOT - // package resolves against the package root (the documented contract), - // not against OUT_DIR. - // Same expression as the pre-move call site (and same order), so the - // contract hash — and therefore the build.mcpp cache — is unchanged - // across the move for feature-identical builds. - bpEnv.features = feature_closure(*state.m, parse_feature_request(state.overrides.features)); - bpEnv.outDir = state.programOutDir(state.workRoot, *state.m, bpEnv.features); - // mcpp#241 (root): consumer index 0, same owner as the dep loop. - // - // AND THE LINK FORM OF EACH DEPENDENCY (#642 E2), to this program only. - // The root decides every dependency's form, and when this program runs - // every input of that decision is final: the requests are the root - // manifest's, and each dependency's own program has already run. A - // DEPENDENCY's program is not offered the forms. It runs in discovery - // order, before the programs of packages discovered after it, and those - // programs supply facts the answer depends on (a `-L` they add makes a - // package static-only), so the value it could be given would be a guess. - { - std::map rootLinkForms; - for (auto const& [idx, form] : state.dependencyLinkForms) - if (form.recorded) - rootLinkForms.emplace(idx, std::string( - mcpp::build::linkage_form::to_string(form.answer.linkage))); - state.fillDepDirs(bpEnv, 0, &rootLinkForms); - } - state.fillXpkgDirs(bpEnv, *state.m, 0); - // #355: the host tools the ROOT package requested (consumer index 0). - if (auto tit = state.toolEnvByConsumer.find(0u); tit != state.toolEnvByConsumer.end()) - bpEnv.toolPaths = tit->second; - // The program, as the build programs' graph compiled it (mcpp#797). - if (auto program = build_program_binary(state, 0); program) { - bpEnv.programBinary = program->path; - bpEnv.programRunner = program->runner; - bpEnv.programIdentity = program->identity; - bpEnv.programCompile = program->compile; - } else if (!state.overrides.plan_only) { - return std::unexpected(program.error()); - } else { - bpEnv.programError = program.error(); - } - // #647 E1: THE RESOLVED GRAPH, FOR THE ROOT'S PROGRAM ONLY. - // - // Every package, dependencies before the packages that request them - // (ties in discovery order), so a program that merges what libraries - // contribute can apply them in override order without a sort of its - // own. The root decides the graph, and every input of that decision is - // final here -- the same reason `dep_linkage` is offered to this - // program alone. A file, not variables: a graph with metadata does not - // fit an environment block (`MAX_ARG_STRLEN`, the Windows limit). - // - // ITS DIGEST JOINS THE RE-RUN KEY. Editing a dependency's - // `[package.metadata]` changes what this program would answer, so it - // must run again; editing that dependency's sources does not, and the - // document does not change. - { - std::vector order; - std::vector placed(state.packages.size(), false); - while (order.size() < state.packages.size()) { - std::size_t pick = state.packages.size(); - for (std::size_t i = 0; i < state.packages.size() && pick == state.packages.size(); ++i) { - if (placed[i]) continue; - bool ready = true; - for (auto const& r : state.graphRequests) - if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i - && r.dependencyPackageIndex < state.packages.size() - && !placed[r.dependencyPackageIndex]) { ready = false; break; } - if (ready) pick = i; - } - // A cycle leaves nothing ready; its first member in discovery - // order is taken so the document is still complete. - if (pick == state.packages.size()) - for (std::size_t i = 0; i < state.packages.size(); ++i) - if (!placed[i]) { pick = i; break; } - placed[pick] = true; - order.push_back(pick); - } - nlohmann::json doc; - doc["kind"] = "mcpp.graph"; - doc["version"] = 1; - nlohmann::json list = nlohmann::json::array(); - for (auto i : order) list.push_back(state.graph_package_entry(i, true, nullptr)); - doc["packages"] = std::move(list); - const auto text = doc.dump(2) + "\n"; - const auto graphPath = bpEnv.artifactsDir / "graph.json"; - std::error_code gec; - std::filesystem::create_directories(graphPath.parent_path(), gec); - const auto tmp = graphPath.string() + ".tmp"; - { - std::ofstream out(tmp, std::ios::binary | std::ios::trunc); - out << text; - } - std::filesystem::rename(tmp, graphPath, gec); - if (gec) - return std::unexpected(std::format( - "cannot write the graph document '{}': {}", - graphPath.string(), gec.message())); - bpEnv.graphFile = graphPath; - bpEnv.graphDigest = mcpp::toolchain::hash_string(text); - } - auto& bcRoot = state.m->buildConfig; - const auto mark = state.markDirectiveTail(*state.m); - const auto rldN = bcRoot.ldflags.size(), rsrcN = bcRoot.sources.size(), - rmodN = state.m->modules.sources.size(); - const auto ractN = bcRoot.actions.size(); - // #622 A4: how many `[runtime] deploy` entries existed before this - // program ran — the manifest-sourced ones, already in `packages[0]`'s - // snapshot. Anything past this index is a `mcpp::deploy()` residue - // that needs the same mirror the flag/source tails get below. - const auto rdeployN = state.m->runtimeConfig.linkIntent.deploy.size(); - // Same reason, one field wide: `mcpp::runtime_search_dir()` residue - // needs the same mirror `deploy` does, or `resolve_runtime_contract` - // (which reads `packages[0]`'s snapshot, not `*m`) never sees it. - const auto rsearchDirN = state.m->runtimeConfig.linkIntent.runtimeSearchDirs.size(); - // What the dependencies supplied as runners, before the root's program - // speaks. The root's emissions are appended to the same slots, so a - // name both supply becomes one argv joining the two (#634, §9 item 8, - // measured: `run-A.sh run-B.sh `). - const auto runnerBeforeRoot = bcRoot.runner; - const auto namedBeforeRoot = bcRoot.namedRunners; - // The root's program is the requested package's. - bpEnv.requested = !state.m->package.virtualRoot; - auto bp = run_answering_requests(state, *state.m, bpEnv, 0, - state.m->package.name, [&] { - return mcpp::build::run_build_program( - *state.m, *state.root, host->first, host->second, - state.m->cppStandard, bpEnv); - }); - if (!bp && !state.overrides.plan_only) { - return std::unexpected(bp.error()); - } - // #699 item 2 (E3): under `emit build-database` (`plan_only`), a - // failing root build program describes the package without its - // directives rather than costing the whole plan. Every mirror below - // reads what the program would have added to `*m`, so skipping - // straight past it (nothing runs on this path) is what "without its - // directives" means; a later failure that follows from the gap - // fails the member under the ordinary rule (E1). - if (!bp) { - state.planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", - bp.error(), mcpp::wire::Severity::Error, - (*state.root / "build.mcpp").string()}); - // Named so the device-source check below (and anything else whose - // premise is this program's directives) can tell a package whose - // program failed apart from one that simply has no program. - state.programFailedPackages.insert(state.root->string()); - } - if (bp) { - // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. - // Two suppliers of one runner are refused naming both, and the - // manifest is the way to choose: a `[target.]` runner the - // project writes outranks every supplied one where the runner is - // looked up, so a name the manifest declares is not refused here. - { - const auto parsedTarget = state.tc - ? mcpp::toolchain::triple::parse(state.tc->targetTriple) - : std::nullopt; - const auto rowKey = parsedTarget ? parsedTarget->str() - : state.tc ? state.tc->targetTriple : std::string{}; - const auto* row = [&]() -> const mcpp::manifest::TargetEntry* { - if (parsedTarget) return find_target_entry(*state.m, *parsedTarget); - auto it = state.m->targetOverrides.find(rowKey); - return it == state.m->targetOverrides.end() ? nullptr : &it->second; - }(); - const auto manifestNames = [&](std::string_view name) { - if (!row) return false; - if (name.empty()) return !row->runner.empty(); - return row->namedRunners.contains(std::string(name)); - }; - if (!state.runnerProvider.empty() && !runnerBeforeRoot.empty() - && bcRoot.runner.size() > runnerBeforeRoot.size() - && !manifestNames({})) { - return std::unexpected(std::format( - "the dependency '{}' and this project's build program both " - "supply the runner for this target, and the two would be " - "joined into one argv.\n" - " Drop one of them, or state the runner in " - "[target.{}].runner.", - state.runnerProvider, rowKey)); - } - for (auto const& [name, nr] : bcRoot.namedRunners) { - auto before = namedBeforeRoot.find(name); - auto who = state.namedRunnerProvider.find(name); - if (before == namedBeforeRoot.end() || before->second.argv.empty() - || who == state.namedRunnerProvider.end() || who->second.empty()) - continue; - if (nr.argv.size() <= before->second.argv.size()) continue; - if (manifestNames(name)) continue; - return std::unexpected(std::format( - "the dependency '{}' and this project's build program both " - "supply a runner named '{}' for this target, and the two " - "would be joined into one argv.\n" - " Drop one of them, or state it in " - "[target.{}.runners].{}.", - who->second, name, rowKey, name)); - } - } - auto& pkg0 = state.packages[0]; - // Compile-visible tail → privateBuild: the shared fold (same owner - // as the dep loop; the root's TUs read privateBuild). - state.foldDirectiveTailIntoPrivateBuild(pkg0, *state.m, mark); - // Before the source residues are mirrored below: adopting an action's - // outputs APPENDS to bcRoot.sources, and those appends must be inside - // the tail that gets copied into the packages[0] snapshot the scan reads. - state.adoptActionOutputs(*state.m, *state.root, ractN); - // The root's build program has spoken; a floor it stated is checked - // now, with the facts every package (it included) established. - if (auto err = state.checkVersionFloors(); err) return std::unexpected(*err); - // Root residues — apply() mutated *m, but packages[0].manifest is a - // value-copy snapshot taken at makePackageRoot, so everything the - // scan/fingerprint read from the snapshot needs the tail mirrored: - // sources → the scan walks packages[0].manifest, not *m. - pkg0.manifest.buildConfig.sources.insert( - pkg0.manifest.buildConfig.sources.end(), - bcRoot.sources.begin() + rsrcN, bcRoot.sources.end()); - pkg0.manifest.modules.sources.insert( - pkg0.manifest.modules.sources.end(), - state.m->modules.sources.begin() + rmodN, state.m->modules.sources.end()); - // The units its actions generate, as they declared them (D6). - for (auto const& [path, unit] : state.m->modules.declaredUnits) - pkg0.manifest.modules.declaredUnits.insert_or_assign(path, unit); - // The snapshot's manifest is what later readers of the root's - // flags see -- mirror the flag/include tails, as the old - // pre-snapshot ordering implicitly did. - pkg0.manifest.buildConfig.cflags.insert( - pkg0.manifest.buildConfig.cflags.end(), - bcRoot.cflags.begin() + static_cast(mark.cflags), - bcRoot.cflags.end()); - pkg0.manifest.buildConfig.cxxflags.insert( - pkg0.manifest.buildConfig.cxxflags.end(), - bcRoot.cxxflags.begin() + static_cast(mark.cxxflags), - bcRoot.cxxflags.end()); - pkg0.manifest.buildConfig.includeDirs.insert( - pkg0.manifest.buildConfig.includeDirs.end(), - bcRoot.includeDirs.begin() + static_cast(mark.includeDirs), - bcRoot.includeDirs.end()); - pkg0.manifest.buildConfig.includeDirsAfter.insert( - pkg0.manifest.buildConfig.includeDirsAfter.end(), - bcRoot.includeDirsAfter.begin() - + static_cast(mark.includeDirsAfter), - bcRoot.includeDirsAfter.end()); - // Link flags → the final link reads *m (already applied); keep the - // linkUsage snapshot and fingerprint metadata equivalent too. - pkg0.linkUsage.ldflags.insert(pkg0.linkUsage.ldflags.end(), - bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); - pkg0.manifest.buildConfig.ldflags.insert( - pkg0.manifest.buildConfig.ldflags.end(), - bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); - // #622 A4: `mcpp::deploy()` residue → `packages[0].manifest`, the - // object `resolve_runtime_contract` (plan.cppm) actually reads. - // Without this mirror a directive-sourced deploy entry lands in `*m` - // and nowhere the planner looks — the same gap this block already - // closes for sources/flags, one more field wide. - pkg0.manifest.runtimeConfig.linkIntent.deploy.insert( - pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), - state.m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), - state.m->runtimeConfig.linkIntent.deploy.end()); - // `mcpp::runtime_search_dir()` residue → `packages[0].manifest`, the - // same object and the same reason as the `deploy` mirror above: without - // it a directive-sourced entry lands in `*m` and `resolve_runtime_contract` - // never looks there. - pkg0.manifest.runtimeConfig.linkIntent.runtimeSearchDirs.insert( - pkg0.manifest.runtimeConfig.linkIntent.runtimeSearchDirs.end(), - state.m->runtimeConfig.linkIntent.runtimeSearchDirs.begin() - + static_cast(rsearchDirN), - state.m->runtimeConfig.linkIntent.runtimeSearchDirs.end()); - } - } - return {}; -} - -// The graph document a build program reads (#647 E1): every package the -// program's own package reaches, itself included, dependencies before the -// packages that request them (ties in discovery order), with `root` marking -// the package whose program reads it. For a root that is the whole graph. -static std::expected, std::string> -write_graph_document(PrepareState& state, std::size_t subject, - const std::filesystem::path& artifactsDir) { - std::vector reached(state.packages.size(), subject == 0); - reached[subject] = true; - for (bool grew = true; grew;) { - grew = false; - for (auto const& r : state.graphRequests) - if (r.consumerPackageIndex < reached.size() && reached[r.consumerPackageIndex] - && r.dependencyPackageIndex < reached.size() - && !reached[r.dependencyPackageIndex]) { - reached[r.dependencyPackageIndex] = true; - grew = true; - } - } - std::vector order; - std::vector placed(state.packages.size(), false); - std::size_t wanted = 0; - for (bool b : reached) wanted += b ? 1 : 0; - while (order.size() < wanted) { - std::size_t pick = state.packages.size(); - for (std::size_t i = 0; i < state.packages.size() && pick == state.packages.size(); ++i) { - if (placed[i] || !reached[i]) continue; - bool ready = true; - for (auto const& r : state.graphRequests) - if (r.consumerPackageIndex == i && r.dependencyPackageIndex != i - && r.dependencyPackageIndex < state.packages.size() - && reached[r.dependencyPackageIndex] - && !placed[r.dependencyPackageIndex]) { ready = false; break; } - if (ready) pick = i; - } - // A cycle leaves nothing ready; its first member in discovery - // order is taken so the document is still complete. - if (pick == state.packages.size()) - for (std::size_t i = 0; i < state.packages.size(); ++i) - if (!placed[i] && reached[i]) { pick = i; break; } - placed[pick] = true; - order.push_back(pick); - } - nlohmann::json doc; - doc["kind"] = "mcpp.graph"; - doc["version"] = 1; - // The requests a document lists are those made by its own packages. A - // request from outside the program's closure (another workspace member, a - // workspace plan's virtual root) is a fact about the plan, not about the - // package, and would make the document, and with it the program's cache - // key, depend on which members the command selected. - nlohmann::json list = nlohmann::json::array(); - for (auto i : order) { - auto entry = state.graph_package_entry(i, true, &reached); - entry["root"] = i == subject; - list.push_back(std::move(entry)); - } - doc["packages"] = std::move(list); - const auto text = doc.dump(2) + "\n"; - const auto graphPath = artifactsDir / "graph.json"; - std::error_code gec; - std::filesystem::create_directories(graphPath.parent_path(), gec); - const auto tmp = graphPath.string() + ".tmp"; - { - std::ofstream out(tmp, std::ios::binary | std::ios::trunc); - out << text; - } - std::filesystem::rename(tmp, graphPath, gec); - if (gec) - return std::unexpected(std::format( - "cannot write the graph document '{}': {}", - graphPath.string(), gec.message())); - return std::pair{graphPath, mcpp::toolchain::hash_string(text)}; -} - -// The build programs of a workspace plan's members (workspace design -// 2026-09-29 §15). A member is the project being developed, so its program -// runs where a root's does: after every dependency's program, with the link -// form of each dependency and the graph document of what it reaches, and with -// its own artifacts directory, `/target/.build-mcpp`, where it was -// when the member was the root of its own build. A workspace member reached as -// a dependency of another member runs here too, so its program sees the same -// environment in every selection. -static std::expected step9_member_build_programs(PrepareState& state) { - if (!state.m->package.virtualRoot) return {}; - // Dependencies first: a member's program runs after the programs of the - // members it depends on, whose directives are then in place in the graph - // it reads. Discovery order, which the packages are numbered in, puts a - // member before its dependency when its key sorts first. - mcpp::graph::AdjacencyList deps(state.packages.size()); - for (auto const& r : state.graphRequests) - if (r.consumerPackageIndex < deps.size() && r.dependencyPackageIndex < deps.size() - && r.consumerPackageIndex != r.dependencyPackageIndex) - deps[r.consumerPackageIndex].push_back(r.dependencyPackageIndex); - // Mutual dev-dependencies between members are legal, so a cycle skips its - // closing edge and every other dependency still comes first. - std::vector roots(state.packages.size()); - std::iota(roots.begin(), roots.end(), std::size_t{0}); - const auto order = *mcpp::graph::depth_first_order(deps, roots, mcpp::graph::Cycles::Skip); - auto hasProgram = [&](std::size_t i) { - if (i == 0 || !state.isWorkspaceMemberPackage(i) || !state.compilesHere(i)) return false; - std::error_code bpEc; - return std::filesystem::exists(state.packages[i].root / "build.mcpp", bpEc) - || !state.packages[i].manifest.buildConfig.ruleModules.empty(); - }; - // Their compiles may overlap (below), and their runs follow this order, so - // every program not yet run is waiting for its turn; its line says so - // (build progress design 2026-09-29, §3.2), named as the program names - // itself. - for (auto const i : order) { - if (!hasProgram(i)) continue; - const auto& m = state.packages[i].manifest; - mcpp::build::progress::program_scheduled( - m.package.namespace_.empty() ? m.package.name - : m.package.namespace_ + "." + m.package.name, - state.packages[i].selectedMember); - } - // THE ENVIRONMENT A MEMBER'S PROGRAM IS GIVEN. Computed from the plan as it - // stands when it is asked, so the compile phase and the run each ask for it. - auto program_env = [&](std::size_t i) - -> std::expected { - auto& pkg = state.packages[i]; - mcpp::build::BuildProgramEnv bpEnv; - bpEnv.targetTriple = state.resolvedTargetCanonical; - fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, - state.cfg_opt ? &*state.cfg_opt : nullptr); - apply_graph_cxx_abi(bpEnv, state.packages); - bpEnv.toolsBin = state.projectSubosBin; - bpEnv.profile = state.effectiveProfile; - bpEnv.accel = state.resolvedAccel(); - fill_package_build_env(bpEnv, pkg.manifest); - // The stage of the member this program acts for: its own, or the one - // packed member that reaches it (`PrepareState::fillPackEnv`). - state.fillPackEnv(bpEnv, i); - bpEnv.requested = pkg.selectedMember; - bpEnv.languageModules = pkg.manifest.language.modules; - bpEnv.ruleModules = pkg.manifest.buildConfig.ruleModules; - if (auto dit = state.deviceSourcesByPackage.find(pkg.root.string()); - dit != state.deviceSourcesByPackage.end()) - bpEnv.deviceSources = dit->second; - bpEnv.artifactsDir = build_program_dir(state, i); - if (i < state.activeFeaturesByPackage.size()) - bpEnv.features = state.activeFeaturesByPackage[i]; - // Beside the cache above, which is in the project under a plan's - // work directory too: `emit build-database` reuses the build's run. - bpEnv.outDir = state.programOutDir(*state.root, pkg.manifest, bpEnv.features); - { - std::map linkForms; - for (auto const& [idx, form] : state.dependencyLinkForms) - if (form.recorded) - linkForms.emplace(idx, std::string( - mcpp::build::linkage_form::to_string(form.answer.linkage))); - state.fillDepDirs(bpEnv, i, &linkForms); - } - state.fillXpkgDirs(bpEnv, pkg.manifest, i); - if (auto tit = state.toolEnvByConsumer.find(i); tit != state.toolEnvByConsumer.end()) - bpEnv.toolPaths = tit->second; - // The program, as the build programs' graph compiled it (mcpp#797). - if (auto program = build_program_binary(state, i); program) { - bpEnv.programBinary = program->path; - bpEnv.programRunner = program->runner; - bpEnv.programIdentity = program->identity; - bpEnv.programCompile = program->compile; - } else { - bpEnv.programError = program.error(); - } - auto graph = write_graph_document(state, i, bpEnv.artifactsDir); - if (!graph) return std::unexpected(graph.error()); - bpEnv.graphFile = graph->first; - bpEnv.graphDigest = graph->second; - return bpEnv; - }; - - // THE COMPILES WERE MADE TOGETHER: every program this plan runs is compiled - // in one build of the build programs' graph, before the dependencies' - // programs run (features.cpp, mcpp#797). The programs run here, one at a - // time, in the order above. - for (auto const i : order) { - if (!hasProgram(i)) continue; - auto& pkg = state.packages[i]; - auto host = state.host_tc_for_build_program(); - if (!host) return std::unexpected(host.error()); - auto envOf = program_env(i); - if (!envOf) return std::unexpected(envOf.error()); - auto& bpEnv = *envOf; - - auto& bc = pkg.manifest.buildConfig; - const auto mark = state.markDirectiveTail(pkg.manifest); - const auto actN = bc.actions.size(); - const auto runnerN = bc.runner.size(); - auto namedBefore = bc.namedRunners; - const bool exclusiveBefore = bc.runExclusive; - auto bp = run_answering_requests(state, pkg.manifest, bpEnv, i, - pkg.manifest.package.name, [&] { - return mcpp::build::run_build_program( - pkg.manifest, pkg.root, host->first, host->second, - pkg.manifest.cppStandard, bpEnv); - }); - if (!bp) { - if (!state.overrides.plan_only) - return std::unexpected(std::format( - "workspace member '{}': {}", pkg.manifest.package.name, bp.error())); - state.planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", - std::format("workspace member '{}': {}", pkg.manifest.package.name, bp.error()), - mcpp::wire::Severity::Error, (pkg.root / "build.mcpp").string()}); - state.programFailedPackages.insert(pkg.root.string()); - continue; - } - state.foldDirectiveTailIntoPrivateBuild(pkg, pkg.manifest, mark); - state.adoptActionOutputs(pkg.manifest, pkg.root, actN); - // How the artifact is executed reaches the plan's root, where `mcpp - // run` reads it, by the rule the dependencies' runners follow: one - // supplier per runner. - auto& rootBc = state.m->buildConfig; - if (bc.runner.size() > runnerN) { - if (!rootBc.runner.empty() && !state.runnerProvider.empty()) - return std::unexpected(std::format( - "'{}' and '{}' both supply a runner for this target.\n" - " Drop one of them, or state the runner in " - "[target.].runner.", - state.runnerProvider, pkg.manifest.package.name)); - rootBc.runner.assign(bc.runner.begin() + static_cast(runnerN), - bc.runner.end()); - state.runnerProvider = pkg.manifest.package.name; - } - for (auto const& [name, nr] : bc.namedRunners) { - auto before = namedBefore.find(name); - const bool grew = before == namedBefore.end() - || nr.argv.size() > before->second.argv.size() - || (nr.longLived && !before->second.longLived); - if (!grew) continue; - auto& slot = rootBc.namedRunners[name]; - auto& who = state.namedRunnerProvider[name]; - if (!slot.argv.empty() && !who.empty()) - return std::unexpected(std::format( - "'{}' and '{}' both supply a runner named '{}' for this target.", - who, pkg.manifest.package.name, name)); - slot = nr; - who = pkg.manifest.package.name; - } - if (bc.runExclusive && !exclusiveBefore) rootBc.runExclusive = true; - // The platform's fields the member's program stated, as a dependency's - // program's are (SPEC-013 §4.1). - merge_program_platform(state, std::exchange(bc.programPlatform, {}), - pkg.manifest.package.name); - } - if (auto err = state.checkVersionFloors(); err) return std::unexpected(*err); - // What the programs published (an include directory, an interface - // define) reaches the members' consumers, as after the dependencies' - // programs. - state.computeUsageRequirements(); - return {}; -} static std::expected step9_device_sources_reach_an_action(PrepareState& state) { // ── Every device source must reach some action ───────────────────────── From 23af679f0885af867a4b8ed423315d180358b14c Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 11:40:14 +0900 Subject: [PATCH 12/14] agents index: design 2026-11-03 part 4 Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .agents/docs/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.agents/docs/README.md b/.agents/docs/README.md index c3a0cc659..14409ad17 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -345 records. +346 records. ## By subject @@ -90,6 +90,7 @@ Records that declare one. Everything else is listed by date below. ### openkal-performance +- [生态设计 part 4：超越原生——openkal 之上与之下的优化、可观测性与安全](2026-11-03-ecosystem-part4-beyond-native-design.md) — active - [生态设计 part 3：ok 映像的性能对等——引擎路径、openkal 运行时与观测](2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md) — active ### plan @@ -160,6 +161,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-11 +- [生态设计 part 4：超越原生——openkal 之上与之下的优化、可观测性与安全](2026-11-03-ecosystem-part4-beyond-native-design.md) — active - [生态设计 part 3：ok 映像的性能对等——引擎路径、openkal 运行时与观测](2026-11-02-ecosystem-part3-ok-image-performance-parity-design.md) — active ### 2026-10 From d847906f3d6eda0675f59f33b5d39faf1793e1d2 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 12:17:21 +0900 Subject: [PATCH 13/14] unit: the launcher's manifest names mcpplibs.os 0.2.8 Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- tests/unit/test_pack_distribution.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/unit/test_pack_distribution.cpp b/tests/unit/test_pack_distribution.cpp index 4e309d947..68998d82f 100644 --- a/tests/unit/test_pack_distribution.cpp +++ b/tests/unit/test_pack_distribution.cpp @@ -261,7 +261,7 @@ TEST(PackDistribution, TheWindowsLauncherIsAProgram) { EXPECT_NE(src.find("(\"windows-\" + arch)"), std::string::npos) << src; EXPECT_NE(src.find("os::process::exec"), std::string::npos) << src; const auto manifest = d::exe_launcher_manifest("hello", "1.0.0"); - EXPECT_NE(manifest.find("os = \"0.2.7\""), std::string::npos) << manifest; + EXPECT_NE(manifest.find("os = \"0.2.8\""), std::string::npos) << manifest; EXPECT_NE(manifest.find("[targets.hello]"), std::string::npos) << manifest; } From 20da2d0526f975375b56a97e18854e37f7296ebe Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Tue, 3 Nov 2026 12:52:18 +0900 Subject: [PATCH 14/14] the built-in openkal platform is openkal-platform 0.23.1 (openkal-macos 0.21.3: a hosted program on macOS is given the directories it was granted) Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com> --- .../2026-11-03-ecosystem-part4-beyond-native-design.md | 10 +++++++--- CHANGELOG.md | 7 ++++--- docs/04-mcpp-toml.md | 2 +- docs/10-pack-and-release.md | 2 +- docs/24-openkal-cross.md | 2 +- docs/35-platforms.md | 8 ++++---- docs/50-machine-output.md | 2 +- docs/specs/packaging.md | 2 +- docs/zh/04-mcpp-toml.md | 2 +- docs/zh/10-pack-and-release.md | 2 +- docs/zh/24-openkal-cross.md | 2 +- docs/zh/35-platforms.md | 8 ++++---- docs/zh/50-machine-output.md | 2 +- modules/toolchain-model/src/triple.cppm | 4 ++-- src/pack/distribution.cppm | 2 +- 15 files changed, 31 insertions(+), 26 deletions(-) diff --git a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md index 88f1eccbb..810a27c3c 100644 --- a/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md +++ b/.agents/docs/2026-11-03-ecosystem-part4-beyond-native-design.md @@ -643,6 +643,10 @@ balanced 按 `kal_memory_info` 与缓存目录的余量自适应：资源充足 后重试；睡眠的字位于栈上。 - openkal-macos 0.21.2：单调时钟粒度原为每次启动测得（41、42 ns 或更大），快照的事实字因此在两次启动间不同，macOS 上的 快照被间歇拒绝。粒度改取 `mach_timebase_info` 的计时单位向上取整。 + - openkal-macos 0.21.3：该格式按链接顺序运行初始化函数，平台的 hosted 安排把 C 库链接在实现之前，C 库的初始化函数读取 + 程序的目录时实现尚未记录环境，启动者授予的目录（及其权限）被实现的默认目录（整个文件系统、可写）取代（mcpplibs.os 的 + hosted macOS 测试：被授予只读目录的程序在其中创建了文件）。实现在尚未记录时从加载器的记录（`_NSGetArgc`、`_NSGetArgv`、 + `_NSGetEnviron`）取参数与环境；一致性测试从链接在实现之前的初始化函数提问。 - openkal-musl 0.27.0：Windows 上以卷开头的名字与持有目录的名字逐字节比较，`c:\Users\...` 到达不了名为 `C:` 或 `C:/Users/...` 的目录。卷下的名字改为不区分大小写、两种分隔符等价（`examples/image-names`）。 @@ -708,9 +712,9 @@ GCS 还需要系统为程序启用该栈，映像没有这一环节。返回地 | 仓库 | 版本 | | --- | --- | | openkal | 0.23.0 | -| openkal-linux / -macos / -windows / -emscripten | 0.24.0 / 0.21.2 / 0.19.0 / 0.12.0 | -| openkal-image（ld、run、tools、bind） | 0.4.1（0.4.0 之后以 openkal-macos 0.21.2 重建） | -| openkal-musl / openkal-llvm-runtime / openkal-platform | 0.27.0 / 0.18.1 / 0.23.0 | +| openkal-linux / -macos / -windows / -emscripten | 0.24.0 / 0.21.3 / 0.19.0 / 0.12.0 | +| openkal-image（ld、run、tools、bind） | 0.4.2（0.4.0 之后两次以新的 openkal-macos 重建） | +| openkal-musl / openkal-llvm-runtime / openkal-platform | 0.27.1 / 0.18.2 / 0.23.1 | | mcpplibs.os | 0.2.8 | | mcppls-clangd / mcppls | 0.0.20 / 0.0.20 | | mcpp | 2026.11.2.3 | diff --git a/CHANGELOG.md b/CHANGELOG.md index 42d9c4a34..dd7d54605 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ ## [2026.11.2.3] - 2026-11-02 -The built-in openkal platform is 0.23.0 (openkal 0.23: an image at a base of +The built-in openkal platform is 0.23.1 (openkal 0.23: an image at a base of each boot, a snapshot of its initialisers, hardening); a format a package provides is told the compression stated; `mcpp cache gc` reclaims the runner's cache; a download source that sends nothing is left for the next one; a @@ -15,9 +15,10 @@ design 2026-11-03 part 4. ### Changed -- **The built-in `-openkal` platform is `mcpplibs.openkal-platform@0.23.0`** +- **The built-in `-openkal` platform is `mcpplibs.openkal-platform@0.23.1`** (docs/35; design 2026-11-03 part 4), the version group of openkal 0.23: - openkal-image 0.4.1, openkal-musl 0.27.0, openkal-llvm-runtime 0.18.0. + openkal-image 0.4.2, openkal-musl 0.27.1, openkal-llvm-runtime 0.18.2, + openkal-macos 0.21.3. An image is sealed at a base (`okld place`), and the runner moves it to a base chosen once each boot, its data kept as a prepared copy; the first start of a boot has a snapshot of the image's initialisers made, which later starts diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index cc955557f..6530b0625 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1502,7 +1502,7 @@ the platform's own key names ([35 — Platforms](35-platforms.md)): ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.23.0" # or { path = "…" }, or "none" +platform = "openkal-platform@0.23.1" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 954f33674..805dea658 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -956,7 +956,7 @@ and sha256. A provided format, `--merge` and `mcpp emit xpkg` read it ([SPEC-014](specs/packaging.md) §4.5): ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.23.0", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.23.1", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/24-openkal-cross.md b/docs/24-openkal-cross.md index aaaba029b..2c3d18096 100644 --- a/docs/24-openkal-cross.md +++ b/docs/24-openkal-cross.md @@ -111,7 +111,7 @@ practice. ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.23.0" +platform = "openkal-platform@0.23.1" ``` One table (2026.10.27.1+). It selects the openkal platform for the host triples, diff --git a/docs/35-platforms.md b/docs/35-platforms.md index 416fbe335..b1930a4db 100644 --- a/docs/35-platforms.md +++ b/docs/35-platforms.md @@ -113,9 +113,9 @@ The first of these that answers selects the target's platform: | Source | Spelling | |---|---| -| the command line | `--platform openkal-platform@0.23.0`, `--platform path:../openkal-platform`, `--platform none` | +| the command line | `--platform openkal-platform@0.23.1`, `--platform path:../openkal-platform`, `--platform none` | | the environment | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| the project | `[target.] platform = "openkal-platform@0.23.0"`, or `{ path = "…" }` | +| the project | `[target.] platform = "openkal-platform@0.23.1"`, or `{ path = "…" }` | | this machine | the same key in `~/.mcpp/config.toml` | | the target table | the platform column (`mcpp toolchain list`) | @@ -129,7 +129,7 @@ its own conventions. `[host] platform` selects a platform for them instead: ```toml [host] -platform = "openkal-platform@0.23.0" +platform = "openkal-platform@0.23.1" ``` ``` @@ -151,7 +151,7 @@ implementation: ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.23.0" +platform = "openkal-platform@0.23.1" ``` On Windows the arrangement is the GNU ABI's row (`--target x86_64-windows-gnu`); diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 9e81ff1ef..e76cc8d38 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -527,7 +527,7 @@ installation, which is located on the machine rather than installed by mcpp. A target row is `{target, note, toolchain, pin, status, default}`, and `status` is one of `installed` / `available` / `via dependency graph` / `planned`. A target row also carries `platform` (2026.10.27.1+): the platform package the -target table selects for it (`mcpplibs.openkal-platform@0.23.0`), or `null`. +target table selects for it (`mcpplibs.openkal-platform@0.23.1`), or `null`. **`toolchain` and `pin` are not the same field twice.** `toolchain` is what the row is associated with — the installed payload on an installed row, the diff --git a/docs/specs/packaging.md b/docs/specs/packaging.md index 25ac4b038..d73f76e46 100644 --- a/docs/specs/packaging.md +++ b/docs/specs/packaging.md @@ -237,7 +237,7 @@ openkal 目标上 `static` 为一个 `.oke`、无 `.okso`,`shared` 为 `.oke` "schema": 1, "name": "hello", "version": "0.1.0", - "platform": "openkal-platform@0.23.0", + "platform": "openkal-platform@0.23.1", "linkage": "declared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index ff7a78f5d..676a6960d 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1384,7 +1384,7 @@ runner 与 `run_under`。 ```toml [target.'cfg(os = "openkal")'] -platform = "openkal-platform@0.23.0" # or { path = "…" }, or "none" +platform = "openkal-platform@0.23.1" # or { path = "…" }, or "none" [target.'cfg(platform = "openkal")'.layers] c-abi = { path = "../openkal-musl" } diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 7408925b4..be374259a 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -801,7 +801,7 @@ Authenticode 可以签名它。Linux 与 macOS 经 shell 启动它：`sh pg-0.1. 提供的格式、`--merge` 与 `mcpp emit xpkg` 读取它（[SPEC-014](../specs/packaging.md) §4.5）： ```json -{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.23.0", +{"schema": 1, "name": "hello", "version": "0.1.0", "platform": "openkal-platform@0.23.1", "linkage": "shared", "mode": "vendored", "legs": ["x86_64-openkal", "aarch64-openkal"], "systems": [], "launch": {"runner": ["okrun", "{}"], "entry": {"x86_64": "lib/x86_64/hello.oke", "aarch64": "lib/aarch64/hello.oke"}}, "files": [{"path": "lib/x86_64/hello.oke", "role": "image", "leg": "x86_64-openkal", "sha256": "…"}]} diff --git a/docs/zh/24-openkal-cross.md b/docs/zh/24-openkal-cross.md index d7d473f32..a1b7eac70 100644 --- a/docs/zh/24-openkal-cross.md +++ b/docs/zh/24-openkal-cross.md @@ -93,7 +93,7 @@ SDK，或者区分 `linux`/`windows`/`macos`。那些是 C 环境层或平台层 ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.23.0" +platform = "openkal-platform@0.23.1" ``` 一张表（2026.10.27.1+）。它为宿主三元组选择 openkal 平台，平台的宿主部分命名编译器与目标侧三个层的包； diff --git a/docs/zh/35-platforms.md b/docs/zh/35-platforms.md index 5d9e342d0..8c44f9871 100644 --- a/docs/zh/35-platforms.md +++ b/docs/zh/35-platforms.md @@ -98,9 +98,9 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 | 来源 | 写法 | |---|---| -| 命令行 | `--platform openkal-platform@0.23.0`、`--platform path:../openkal-platform`、`--platform none` | +| 命令行 | `--platform openkal-platform@0.23.1`、`--platform path:../openkal-platform`、`--platform none` | | 环境变量 | `MCPP_TARGET_X86_64_OPENKAL_PLATFORM=…` | -| 工程 | `[target.] platform = "openkal-platform@0.23.0"`,或 `{ path = "…" }` | +| 工程 | `[target.] platform = "openkal-platform@0.23.1"`,或 `{ path = "…" }` | | 本机 | `~/.mcpp/config.toml` 中的同一个键 | | 目标表 | 平台列(`mcpp toolchain list`) | @@ -112,7 +112,7 @@ C++ 运行时的包同时提供编译器运行时,因此 `"c++-abi" = false` 只 ```toml [host] -platform = "openkal-platform@0.23.0" +platform = "openkal-platform@0.23.1" ``` ``` @@ -130,7 +130,7 @@ platform = "openkal-platform@0.23.0" ```toml [target.'cfg(any(linux, macos, windows))'] -platform = "openkal-platform@0.23.0" +platform = "openkal-platform@0.23.1" ``` Windows 上这一安排对应 GNU ABI 的行(`--target x86_64-windows-gnu`);`[platform.hosted] targets` 之外的 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index e4ea23b9e..fd23fe4ba 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -463,7 +463,7 @@ mcpp toolchain list --format json 的形状是 `{target, note, toolchain, pin, status, default}`，`status` 取 `installed` / `available` / `via dependency graph` / `planned` 之一。 目标行另有 `platform`（2026.10.27.1+）：目标表为该目标选择的平台包 -（`mcpplibs.openkal-platform@0.23.0`），或 `null`。 +（`mcpplibs.openkal-platform@0.23.1`），或 `null`。 **`toolchain` 与 `pin` 不是把同一个字段写了两遍。** `toolchain` 表示这一行 关联到什么 —— 已装的行是装好的载荷，词表行是它自己的约定。`pin` 只承载目标 diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index 25a0ce112..a9ce00947 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -906,9 +906,9 @@ inline constexpr TargetInfo kKnownTargets[] = { // `openkal-platform` (SPEC-013), pinned here. The two architectures the // loader relocates. { "x86_64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.23.0" }, + "mcpplibs.openkal-platform@0.23.1" }, { "aarch64-openkal", "preview", "ok", "llvm@23.1.3","", true, - "mcpplibs.openkal-platform@0.23.0" }, + "mcpplibs.openkal-platform@0.23.1" }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/src/pack/distribution.cppm b/src/pack/distribution.cppm index 8a75b9b5e..78a05d846 100644 --- a/src/pack/distribution.cppm +++ b/src/pack/distribution.cppm @@ -1075,7 +1075,7 @@ std::string exe_launcher_manifest(std::string_view name, std::string_view versio "[package]\nname = \"{0}\"\nversion = \"{1}\"\n\n" "[targets.{0}]\nkind = \"bin\"\nmain = \"src/main.cpp\"\n\n" "# 0.2.8: the release written against openkal 0.23, the openkal the\n" - "# platform of mcpp's openkal rows names (openkal-platform 0.23.0), which\n" + "# platform of mcpp's openkal rows names (openkal-platform 0.23.1), which\n" "# the launcher is built over.\n" "[dependencies.mcpplibs]\nos = \"0.2.8\"\n", name, version.empty() ? std::string_view("0.0.0") : version);