docs: review of #455-#459, the openkal design, and the openkal 0.1 implementation plan - #461
Merged
Conversation
Covers what the five PRs did, the four releases they produced, and the ecosystem work alongside them (xim-pkgindex #651/#652/#653, mcpp-index #219/#220, two new mcpplibs repos), assessed on architecture, compatibility, simplicity, stability and cross-platform. Every claim carries its source — a PR number, a file, or the command that measured it — because the point of the document is to be checkable rather than summarised. Three of its assertions were re-verified against the tree while writing it. The uncomfortable half is deliberate: * five defects in this round were self-inflicted, two of them found only AFTER a release; * three test criteria were themselves wrong — green tests that were not testing the thing they named; * the largest architectural error (packages declaring the target's C library) was found by review, not by me, and I had written 'this cannot be done' about the std subset while my own research document had measured that it could.
…hip answers
Adds a scenario document — what a user types, what they see, and what they no
longer have to write — with the commands and outputs taken from a real run of
the released binary rather than sketched.
Folds two review questions into the analysis:
* `picolibc-riscv` / `qemu-riscv` carrying the target in the name follows
xim's existing rule, which is visible across the index:
aarch64-linux-musl-gcc, riscv64-linux-musl-gcc, mingw-cross-gcc, musl-gcc.
The NAME carries the target; the `archs` axis carries the host, which is
why `llvm` has no target in its name at all.
* picolibc belongs on the xim side, resolved at build time from the target's
row. It is not importable, it is chosen by the target rather than by a
dependency graph, and every other C library in the ecosystem — glibc, musl,
musl-cross-make — is already there; mcpp-index carries zero libc packages.
Moving it would put the libc back into the package graph, undoing #459.
Both answers came with a gap worth recording: `[target.X]` has toolchain,
linkage, runner and cxx_runtime but no `sysroot`, so a project cannot swap
picolibc for newlib today.
Written from the discussion that followed the #455-#459 review, and grounded in what is measurable today rather than sketched: every claim marked with a source was verified against the shipped payloads (llvm 22.1.8, gcc 16.1.0, picolibc 1.8.12) while writing. The load-bearing decisions: * Partitioning by RESOURCE KIND, not by which standard-library facility it lights up. The latter couples a kernel ABI to C++ and to today's library, and inverts the dependency — it is the same mistake POSIX made for C and WASIp1 made for POSIX, one generation further on. 'Lights up std::X' is demoted to the upward admission criterion, which is where it belongs. * An opaque one-word handle, because that is what makes openkal indifferent to sitting above or below libc. An int fd forces it below (Windows needs a table); a FILE* forces it above. Measured: four backends store their native thing with zero bridging. * fs and net dissolve. Naming goes to openkal.namespace, and what it hands back is the same stream resource a file, a socket or a UART gives you. Cleaner than 'everything is a file', because naming failure and I/O failure end up in different interfaces. * core is abort + stream + memory. Memory is core because a bump allocator over a static arena is an IMPLEMENTATION, not an emulation — the test being whether a fake would make callers silently wrong, which is true of a clock that does not advance but not of an allocator that can fail. Also records the caps-can-lie problem with four defences ordered by strength, led by making unsupported operations unrepresentable in the type system rather than false in a bool — the same conclusion K1/K2 reached for the MMU.
…n questions
Two things the design got wrong, both retracted with the reasoning that made
them look right at the time:
* `openkal.namespace` replacing fs and net. It violated this document's own
§5.1 rule (a stream whose caps are the union of file and socket operations
is precisely the 'present but useless' antipattern), it required every
backend to carry a URI parser — which is an emulation layer by the downward
admission criterion — and the WASIp2 precedent it cited was a misreading:
WASIp2 separates resource KINDS and shares only the stream type.
* Extending cfg() with capability predicates. That conclusion was about
openarch's AddressSpace; going through openkal interface by interface, core
has no semantic axis at all, and the triple already carries most of what
cfg(mmu) would have. The design is now a zero-engine-change proposal.
Also corrects the module wiring: `reexport` propagates DOWNSTREAM, so an
interface package cannot use it to reach a backend the consumer chose. The
backend reexports the interface instead, which is what that mechanism is for.
Six open questions the draft did not take a position on, led by one that is
measured rather than theoretical: picolibc's vfprintf references free, so
routing operator new to kal_alloc while printf keeps picolibc's malloc puts two
allocators on the same RAM. The spec has to require that kal_alloc be built
over a libc allocator where one exists, not beside it.
…ts config file are gone
The design's §4 rested on one measurement: `requires { mcpp::runner("x") }` is
a hard error when the name is absent. The measurement was right; the quantifier
in the conclusion was not. It is QUALIFIED names that cannot be probed —
unqualified lookup through ADL is dependent inside a template and evaluates to
false, exactly as wanted.
Verified on llvm 22.1.8 against real C++20 modules, not headers, with all three
behaviours holding at once:
* backend present → the concept is true and the call resolves to it
* backend absent → the concept is FALSE, so `if constexpr` degrades
* backend absent, called anyway → a compile error carrying the spec's own
wording, which is what 'build the diagnostic in' was asking for
⚠️ One trap worth the record: if the fallback overload returns the same type as
the real one, the concept is true even with no backend — a requires-expression
does not instantiate the body, so the static_assert never fires. The fallback
must return a distinct type. Measured, after writing it the other way first.
Consequences: the caps struct, the generated caps module and capabilities.toml
are all deleted. A backend's module interface IS its capability declaration, so
claim and implementation become the same artifact by construction and the whole
'caps can lie' problem shrinks from structural+behavioural to behavioural only.
Nothing leaves mcpp.toml — backend selection stays a conditional dependency.
…ers declare both Answers two review questions that turned out to be the same question. The draft had the application write `import openkal.uart;`. That is wrong — it pins the source to a backend, which is the one thing openkal exists to avoid. But it papered over a real constraint, now measured on mcpp 2026.8.19.4 with gcc 16.1.0: * a transitive dependency's module IS importable *⚠️ but ADL does NOT reach a module the translation unit did not import ('seek' was not declared in this scope) So the backend's declarations must live in the module the app imports, which forces the backend to own the well-known name `openkal.stream` while the interface package provides `openkal.abi.stream`. Verified end to end: the app writes one import, names no backend, and ADL resolves to the backend's seek.⚠️ And a trap worth the record: the interface module cannot be called `openkal.stream.abi` — the module graph reads the dots as hierarchy and ninja reports a self-cycle on openkal.stream.gcm. `openkal.abi.stream` is fine. On dependencies: two, not one. The backend alone would work, but declaring the contract is what lets the APPLICATION pin the contract version, turning a mismatch into a resolution error instead of a pile of signature errors at compile time. Same shape as embedded-hal plus a board crate.
…anically bounded
Naming: the interface package is `openkal`, not `openkal-abi` — openkal is the
specification, so saying it twice is noise. Two module names are still forced by
the language (§4.3), but the qualifier now lands only where implementers see it:
applications write `import openkal.stream;`, backend authors write
`export import openkal.decl.stream;`.
The backend owning the application-visible module name is the one real cost of
that shape, and it is a fragmentation risk: a backend could put non-standard
names into the standard module and applications would not notice. What bounds
it is that most of the surface is not the backend's to touch — ⓘ measured, a
backend redefining the interface's types is rejected outright:
error: redeclaring 'struct kal::io_result@openkal.decl.stream' in module
'openkal.stream' conflicts with import
so the only freedom left is adding overloads, and THAT is statically checkable:
conformance diffs the module's exported name set — and signatures, since an
`unsigned long` offset would still win ADL through a conversion — against the
spec list. Vendor extensions must live under a different module name, which
makes 'I used an extension' visible in the source.
Second review pass adds two findings: the cardinality that matters is one
implementation per INTERFACE rather than one backend per program (a program may
take stream from one provider and memory from another), and the fallback
overload is too greedy — unconstrained, it catches every kal type and tells a
socket it is not a seekable stream.
Two identities: a backend that works today, and the thing other implementers
copy. It does not move the D0 gate — that gate is whether a THIRD PARTY writes
a third backend — but it turns 'guess the shape and write the implementation'
into 'write the implementation'.
Writing it out surfaced three things the design document had not:
* core operations need no ADL at all. They are declared `extern "C"` by the
interface and defined by the backend; missing means a link error. The ADL
mechanism serves optional capabilities only, which makes the common path
simpler than the draft implied.
* short writes are a spec question nobody had asked. ::write(2) may write
less than requested, so openkal must choose: write-all-or-error (the loop
lives once, in the backend) or allow short writes (every caller writes the
loop — which is exactly where POSIX has tripped programs up for decades).
* ⭐ it independently confirms the fs/net decomposition. On Linux, seekability
is a property of the HANDLE, not of the backend — lseek succeeds on a file
and returns ESPIPE on a pipe. If openkal.stream had seek, the Linux backend
could not answer honestly: claiming it means always failing on pipes, which
is precisely the 'present but useless' antipattern. Because §2.3 puts seek
on openkal.fs's descriptor instead, the question does not arise.
That last point is the strongest argument for writing a complete reference at
all: it is the only way to find a decomposition error, and it finds it earlier
than a conformance suite would.
…eable with impl `openkal.impl.*` would be semantically backwards: that module belongs to the interface package and holds declarations — types, the extern "C" surface, the fallback overloads, the concepts. What an implementation provides is `openkal.<interface>` itself. The name has to be in the spec rather than left to taste, for three reasons that are all load-bearing: every backend must `export import` that exact name, so it is part of the contract; the guarantee that a backend cannot redefine the interface's types only holds while all backends import the SAME module; and conformance's exported-name diff needs to know which names came from the shared module.⚠️ The rationale has to ship with the rule. A spec reader will naturally reach for `openkal.stream.decl` — the dotted extension — and that one was measured to produce a ninja self-cycle on openkal.stream.gcm. A rule without its reason sends the first implementer straight into it. Also records a simplification that was considered and rejected: one `openkal` module holding every interface's declarations. It costs a naming level but breaks per-interface independent versioning, and drags task/fs declarations into a backend that only provides streams.
…ints at the shipped packages openkal 0.1 exists as two published packages: mcpplibs/openkal carries the specification and the modules that declare it, and mcpplibs/openkal-linux is the reference implementation, maintained as the worked example other implementations follow. Both are mirrored, and the mirrored archives were verified byte-identical. The plan document records the task dependencies, the criteria applied to each decision, and what verification established. Two results are worth separating from the rest. Writing a complete reference implementation confirmed the decomposition independently of the reasoning that produced it: on Linux, whether a stream can be repositioned is a property of the individual descriptor rather than of the implementation, so an openkal.stream that offered positioning could have been neither claimed honestly nor withheld usefully. A decomposition error of that kind is invisible in specification text and would have surfaced later. The exported-surface checker required by clause 9.3 was verified in both directions, and the negative direction mattered: an earlier version of it was vacuous, comparing a set of C++ symbols that inline functions never emit. The design document is now marked as the record of derivation, including withdrawn proposals and their reasons, while the specification records only conclusions. Where they disagree the specification governs.
Sunrisepeak
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three documents, produced in sequence and reviewed against each other.
Review of #455 to #459
What the five pull requests did, the four releases they produced, and the
ecosystem work alongside them, assessed on architecture, compatibility,
simplicity, stability and cross-platform coverage. Every claim carries its
source, and three assertions were re-verified against the tree while writing.
The uncomfortable half is deliberate: five defects in that round were
self-inflicted, two of them found only after a release; three test criteria were
themselves wrong; and the largest architectural error was found by review rather
than by me.
openkal design
The derivation of the specification, including the proposals that were withdrawn
and the reasons they looked correct at the time. Two were withdrawn during
review and a third after a measurement showed the conclusion's quantifier was
wrong: it is qualified names that cannot be probed for absence, not names in
general, and unqualified lookup through argument-dependent lookup evaluates to
false as required. That correction removed a capability record, a generated
module and a second configuration file from the design.
openkal implementation plan
The plan by which openkal 0.1 was implemented, the dependencies among its tasks,
the criteria applied, and the outcome. The specification and its declaration
modules are published as mcpplibs/openkal, and the reference implementation as
mcpplibs/openkal-linux; both are registered in mcpp-index and mirrored.
Writing the reference implementation confirmed the interface decomposition
independently of the reasoning that produced it, which is the argument for
writing one rather than a sketch.