From a67adaa11a71006d9d8786a37a5cebe3446f6ffb Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Tue, 1 Sep 2026 21:07:17 -0400 Subject: [PATCH 1/2] =?UTF-8?q?=E2=9C=A8=20Make=20``=20expand=20its?= =?UTF-8?q?=20approved=20program=20inline=20(#711)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An approved Plan now runs where the author wrote the element. `` without `as` completes authorship, tears the constrained authorship frame down, structurally admits the exact approved source, and expands those bytes at the site as an embedded text root. `` binds the same bytes and performs none of the program's effects, which is what keeps `xmd plan` source-only: its adapter captures. A trusted host says so with new host-only metadata on a declared Markdown component, `returnDisposition: { kind: "executable-source", sourceIdentity }`. It is admitted against the source — a declaration whose Markdown returns anything but a string is refused — has no frontmatter field, travels in the durable declared-component selection, and is compared whole and by absence on replay, so a record that carries none is never read as executable source. The projection is one expansion path in canonical core, after the invocation and its teardown. It refuses a root `returns` and validates a declared root props schema against the properties ambient at the site, both before the first effect; applies the source's own frontmatter metadata and top-level ``; and inherits the component selection, import authority, Workspace, error mode, checked-failure ledger, block counter, output owner, cancellation scope and journal. Its identities extend the authored element's, so a partial journal resumes inside the program without repeating a completed effect. It is its own flow: a stray `` is reported there rather than ending the caller's loop, and a `` is reserved. Ordinary value components are unchanged — a string-valued one still requires `as`, and a repository component cannot opt in through frontmatter. Claude-Session: https://claude.ai/code/session_01TNJwcFmnt3kYSn9gGsx9u7 --- architecture.md | 113 +++- packages/cli/src/documents/Plan.md | 8 +- packages/cli/src/plan-component.ts | 20 + packages/cli/src/syntax.ts | 8 + packages/cli/tests/plan-component.test.ts | 369 +++++++++- packages/cli/tests/syntax-cli.test.ts | 28 + packages/core/mod.ts | 1 + .../core/src/components/declared-markdown.ts | 77 +++ packages/core/src/components/select.ts | 3 + packages/core/src/document-validation.ts | 13 +- packages/core/src/execute.ts | 79 ++- packages/core/src/expand.ts | 155 ++++- packages/core/src/inspect.ts | 20 +- packages/core/src/invocation-rules.ts | 9 +- packages/core/src/types.ts | 22 + packages/core/tests/executable-source.test.ts | 637 ++++++++++++++++++ scripts/tests/cli-npm-bin.test.ts | 9 + scripts/tests/plan-component-compiled.test.ts | 9 + specs/executable-mdx-spec.md | 170 ++++- specs/plan-command-spec.md | 13 + specs/release-process-spec.md | 7 +- 21 files changed, 1704 insertions(+), 66 deletions(-) create mode 100644 packages/core/tests/executable-source.test.ts diff --git a/architecture.md b/architecture.md index b3d26989..ae8e5908 100644 --- a/architecture.md +++ b/architecture.md @@ -26,8 +26,10 @@ Existing documents and code get aligned to this section retroactively. | document execution | one evaluation of a root document initiated through `execute()`, producing one output stream and one completion result while reading and appending a durable journal; its ongoing effects belong to the Effection scope in which the evaluation runs | | workflow definition | what a workflow run is a run of: a versioned descriptor naming an immutable object — its format and object ID — together with the repository-relative path of the root document inside it, the exact canonical document target when one is selected, and the component bundle it is closed over when it declares one. A repository locator is not part of it, and it is distinct from every Repository created inside the run's Workspace | | workflow component bundle | the closed set of authored Markdown components a workflow root declares, each named, located by a canonical repository-relative path inside the pinned commit, and identified by that blob's object ID. It is both definition identity and execution authority: the names it declares resolve to its exact pinned sources and to nothing else | -| declared Markdown component | exact first-party Markdown a trusted host declares to one document execution, read once and held by the invocation — schemas copied, not referenced — before any installation runs, as immutable data on an `ExecutionInstallation`: the public name, the reported origin, the bytes, the SHA-256 of those bytes, the accepted forms, the contract parsed from them, and any private component closure only those bytes may write. The host claims the name rather than offering a default for it, so it resolves ahead of a repository file, a workflow component bundle and every registration; a second claim on one name is a configuration failure. It is a composition mechanism and not a policy loader: no caller-facing option selects one, adds one, or names its source | +| declared Markdown component | exact first-party Markdown a trusted host declares to one document execution, read once and held by the invocation — schemas copied, not referenced — before any installation runs, as immutable data on an `ExecutionInstallation`: the public name, the reported origin, the bytes, the SHA-256 of those bytes, the accepted forms, the contract parsed from them, any executable-source return disposition, and any private component closure only those bytes may write. The host claims the name rather than offering a default for it, so it resolves ahead of a repository file, a workflow component bundle and every registration; a second claim on one name is a configuration failure. It is a composition mechanism and not a policy loader: no caller-facing option selects one, adds one, or names its source | | private component closure | the components one declared Markdown component carries that only its own bytes may write. Each is minted like any other invocation-identity component and registered nowhere, so it resolves while canonical core is expanding that declaration's body and nowhere else — not from the caller's root, the content the caller projected through it, a sibling declaration, an imported component, middleware, or an implementation kept past teardown. Eligibility is the authored occurrence rather than the name: what an import may invoke is what canonical core produced inside that exact ask; an implementation the closure built is refused for every other name, copy, later site and later execution, because a run that has ended authorizes nothing; and a private name written anywhere else resolves to nothing before any other tier can answer. It is lexical availability, not authority: every private component still takes its operation from the invocation that carries it | +| executable-source return disposition | host-only metadata on a declared Markdown component saying that the string its source returns is Executable Markdown rather than an ordinary value: `{ kind: "executable-source", sourceIdentity }`, where the identity is what positions and diagnostics inside those returned bytes report. It is admitted against the source as well as against itself, so a declaration whose Markdown returns anything but a string is refused. There is no frontmatter field for it, so a repository component, a workflow bundle member, a registration, a function component and a document-authored component cannot request it; omitting it is ordinary value-component behavior. It travels with the declaration rather than with the bytes, is part of the durable declared-component selection, and is compared whole and by absence on replay | +| embedded text root | Executable Markdown expanded where a component carrying the executable-source return disposition was written, once that component's invocation has torn down. It is a root in three respects only — a root `returns` declaration is refused, a declared root props schema validates the properties ambient at the site, and its own frontmatter metadata and top-level `` selection apply — and everything else is the enclosing expansion's: component selection, import authority, Workspace, error mode, checked-failure ledger, block counter, output owner, cancellation scope and journal. It starts no lifecycle, imports no `__root__`, produces no root result and installs no profile, and its expansion and every effect take durable identities beneath the authored site. It is not a nested document execution | | retrieval metadata | replaceable, credential-free information about where a workflow definition can be fetched from now; it takes no part in run identity and is reauthorized by the host before use | | stop reason | why a workflow run or a document execution stopped: a categorical host code, or a reference to an already-filtered journal event | | run ID | an opaque stable public identifier generated by the host or selected by an authorized caller; it associates the run's durable records and effects, remains unchanged for the life of the run, and has no semantics beyond equality and lifecycle addressing | @@ -42,7 +44,7 @@ Existing documents and code get aligned to this section retroactively. | document target | an addressable static heading in a root document's own Markdown flow, named by the canonical path of heading labels that reaches it; selecting one executes the preamble, each ancestor's own content, and that heading's complete subtree | | Prompt | a person's original request, in ordinary natural language. `xmd plan` takes exactly one | | Plan | the executable program produced from a Prompt: an Executable Markdown document combining readable prose that expresses the Prompt's intent with the components that carry it out, each placed beside the prose describing what it does. It begins with one descriptive level-one heading. A Plan is what `xmd plan` approves and then delivers: printed to stdout by default, written to an `--output` file, and run only under `--run`. It is not a synonym for a workflow, a policy document or any executable Markdown file | -| plan command document | the one exact checked-in first-party Markdown value root `xmd plan` executes. It is the command's adapter and nothing else: it projects the request into ``, supplies the session, and returns the approved source. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | +| plan command document | the one exact checked-in first-party Markdown value root `xmd plan` executes. It is the command's adapter and nothing else: it projects the request into ``, supplies the session, and captures the approved source with `as` — which is what keeps `xmd plan` source-only, because a captured Plan runs nothing. It is not itself a Plan. Internal: no command-line option selects another one, and no repository component search can answer for it | | packaged `` Component | the one exact checked-in first-party Markdown value component that converts a Prompt into a Plan, `packages/cli/src/documents/Plan.md`, declared to every ordinary run as the public ``. It owns and implements the Plan authorship workflow — the Prompt wording, the draft and repair loops, the `` branches, human review, revision, approval, stopping, exhaustion and the final explanation turn — and returns the exact approved Plan source. Every Plan-producing turn in it states the complete Plan requirements for itself, so a replacement may add or correct a title rather than only carry one forward. Both surfaces expand these exact bytes under one origin and one digest; there is no generated TypeScript copy and no second Markdown implementation. Its four phase components are private to it, and it is not itself a Plan | | authorship profile | the trusted-host assembly the packaged `` Component runs its authored turns under, installed by its own `` inside the invocation that owns it rather than around an execution — which is what makes it the same ceiling whether `xmd plan` or an ordinary document asked: its fixed inputs, a constrained Agent provider, Elicitation, the fixed first-party components and the host-declared ``. It uses no repository component search and exposes no custom root, and the ceiling it establishes is not readable from the command line. Its working directory is one host-owned directory dedicated to the logical session, keyed by the digest of that name, created empty and required to be empty on the way in. An explicitly named session's directory is durable, because continuation derives the same session identity from it; an invocation-unique default session's is scope-owned, claimed before it is created, and exactly one cleanup is attempted after profile teardown and before admission on every ending — the leaf removed non-recursively when it is still the empty directory that was handed over, and left as found with the command failing terminally when it has gained content or vanished. Where those directories live is a host dependency no caller or document selects | | upgrade command document | the one exact checked-in first-party Markdown streaming text root `xmd upgrade` executes to select and install a published release. It owns the exact-tag grammar, release selection, semantic-version comparison, consent, the status, already-current and installation branches, and the wording of every refusal and report; its rendered body is the command's output rather than a value it returns. Internal: no command-line option selects another one, and no repository component search can answer for it | @@ -3527,10 +3529,21 @@ other name in the execution stays the ordinary open import it has always been. Declaring one asset does not take component substitution away from every document the host runs. +**A declaration may say what its return is.** `returnDisposition` is host-only +metadata and the one member that states something the bytes cannot state about +themselves. Its single value, `{ kind: "executable-source", sourceIdentity }`, +says the string those bytes return is Executable Markdown rather than a value: +an invocation that captures nothing has somewhere for that return to go, because +canonical core expands it at the authored site as an embedded text root. It is +admitted against the source as well as against itself, so a declaration whose +Markdown returns anything but a string is refused. No frontmatter field names it, +which is what keeps a repository file, a workflow bundle member, a registration, +a function component and a document-authored component from asking for it. + **Every declaration is read once, before any installation runs.** Name, origin, -source, digest, forms, prose and each private declaration are captured by the -invocation and held by it — factory bound, arrays and schemas copied — on the -same terms as the admissions and the bundle. A schema is copied rather than +source, digest, forms, prose, return disposition and each private declaration are +captured by the invocation and held by it — factory bound, arrays, schemas and +the disposition copied — on the same terms as the admissions and the bundle. A schema is copied rather than referenced because it is a whole object graph, and holding the caller's object would leave the contract mutable after capture. A host that hands a declaration over and then replaces a member of it, or mutates a schema it still holds, has @@ -3562,13 +3575,72 @@ and validation, so a repository file under a private name is described by none of them and runs nowhere. **The journal records the asset, not a lookup.** A declared import records -exactly `{ kind: "declared-markdown", origin, digest, content }`, and a private -one records exactly `{ kind: "declared-private", origin }`. A continuation reads -both as hostile data, verifies the recorded origin, digest and bytes against -what this run declares, and reads no file: a host that no longer declares the -name, or that declares different bytes under it, refuses rather than continuing -somebody else's Markdown. A recorded private import refuses unless the element -asking for it is inside the same declaration. +exactly `{ kind: "declared-markdown", origin, digest, content }`, with +`disposition` beside them when the host stated one, and a private one records +exactly `{ kind: "declared-private", origin }`. A continuation reads both as +hostile data, verifies the recorded origin, digest, bytes and disposition +against what this run declares, and reads no file: a host that no longer +declares the name, that declares different bytes under it, or that now says +something else about what its return is, refuses rather than continuing somebody +else's Markdown. The disposition is compared whole and by absence, because it is +part of what was selected rather than a description of it: a record carrying +none was continuing an ordinary value component and is never read as executable +source. A recorded private import refuses unless the element asking for it is +inside the same declaration. + +## The embedded text root boundary + +A component whose host declared its return to be executable source produces +source the engine expands **where the component was written**. `` invoked +without `as` is the one the `run` profile has. This is one expansion path in +canonical core, not a second execution. + +**The projection happens after teardown.** The component's invocation completes +and is dismantled first, so the source runs under the authority the authored site +already had rather than under anything the component installed for its own body — +for ``, the constrained authorship ceiling is gone before a byte of the +approved program can run. + +**Three things are the source's own, and each is decided before its first +effect.** A root `returns` declaration is refused, because source expanded where +it was written has nowhere to hand a value back to. A declared root props schema +validates the properties ambient at the site, which is what a caller can actually +offer. And its own frontmatter metadata and top-level `` selection apply, +so what reaches the surrounding document is what the program selected. + +**Everything else is the enclosing expansion's.** The component selection and +import authority, the Workspace, the error mode, the checked-failure ledger, the +block counter, the output owner, the cancellation scope and the journal are +already in force and are not rebuilt. The projection calls no `execute()`, +imports no `__root__`, enters no `Execution.document`, creates no second root +result or root `Close`, installs no profile, and writes to no output channel of +its own. Bindings the document made are readable inside the source and the +validated root props are what it reads under `props`; what the source binds +stays inside it. + +The authority is the authored site's rather than the declaration's, so the +private closure only those declared bytes may write is unreachable from the +program they produced — a Plan cannot write ``. And the source +expands under the producing component's own hide set, so a program a component +produced cannot re-enter the component that produced it. + +It is its own control flow as well. A `` written in the source belongs to +a `` written in the source, so a stray one is reported there rather than +silently ending the loop the element was written in, and a `` is +reserved because an embedded root owns no return frame. + +**Identities extend the authored element's.** Every effect the source performs is +durable beneath the site the component was written at, so two sites running one +program are two sets of effects and a replay re-enters the same expansion rather +than deriving a new one. A partial journal restores whatever produced the source — +for ``, the authorship and the exact approved bytes `` retained — +re-enters at the same expansion path, restores the effects that had completed, +and continues without repeating one. + +**`as` is what prevents all of it.** The same invocation written with `as` binds +the exact approved source and expands none of it, which is why `xmd plan` remains +source-only: its adapter captures, and `--run` is still the one path that starts +an execution over those bytes. ## The syntax catalog boundary @@ -3597,10 +3669,13 @@ happen. **A declared Markdown component is described from the same declaration.** The exact Markdown a host would declare to an execution is admitted here on execution's terms too, and each declared name contributes one complete entry -under built-in, reporting `declared-markdown` as its source kind and the -declared origin and digest as its origin. Its private closure contributes -nothing: those names are not syntax a document may write, so a catalog that -listed them would describe an environment that does not exist. +under built-in, reporting `declared-markdown` as its source kind, the declared +origin and digest as its origin, and the return disposition its host stated. That +last is part of the contract an author is reading — it says whether writing the +component without `as` expands what it returns — so both renderers carry it. Its +private closure contributes nothing: those names are not syntax a document may +write, so a catalog that listed them would describe an environment that does not +exist. **Inspection is observation, never authority.** Producing a catalog installs only the declarative registration layer selection needs. It does not enter @@ -3767,8 +3842,8 @@ Status is measured against main. | --- | --- | --- | | `xmd syntax` | describes every structural construct and every selected component the production `run` profile would let a document write in the contextual working directory, as deterministic Markdown or as version-1 JSON, from one catalog. Inspection only: it registers the run profile's declarations in a bounded scope and reads the filesystem for which files exist and, for a selected Markdown component, that file's frontmatter. It runs no body, imports no repository TypeScript module, installs no provider, mints no authority and writes no journal. An include it cannot enumerate — a selection-relevant symbolic link to a directory beneath it included — fails the whole request rather than printing a healthy subset | built on the #632 stack | | document validation | validates one supplied root projection and the recursive Markdown source closure normal component selection discovers, returning deterministic version-1 document diagnostics and `valid`, `invalid` or `not-statically-checkable` invocation outcomes without evaluating document code or installing operational host behavior | built on the #654 stack | -| `xmd plan` | turns one Prompt into a Plan and delivers it, by executing one root document — and, only under `--run`, a second — with a complete scope boundary between them. First the packaged plan command document, under the internal `` identity — an adapter that projects the request into `` and returns what comes back — and inside that, the packaged `` Component under the authorship profile its own `` installs: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — under `~/.xmd/plan/sessions` by default, keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and the Component's own private `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require every Plan to begin with one descriptive level-one title and to keep the Prompt's outcomes as readable steps with each component beside the step it performs, through repairs and revisions alike; that is an authorship and human-review requirement, and `` never enforces it. A tenth draft that still has problems may be stopped or explained: the explanation is one more ordinary turn in the same Session carrying only the final diagnostics, is inert text, reopens no draft limit, and ends the command. The host's instruction layer states only that an answer belongs to the message that asked for it, so which shape a turn wants stays in the document. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after every provider, Prompt task and Elicitation resource inside the authorship frame has torn down, the Component's own `` structurally admits the exact approved bytes, and after that execution ends the host validates the returned Plan again against the command line, resolves props for exactly those bytes, and then delivers the approved Plan where the caller asked: to stdout byte for byte by default, to an exclusively created `--output` path, and — only under `--run` — through the ordinary supplied-source path under the `` identity, one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A journal exists only when `--run` begins, and the flags that configure only a run are refused in preflight without it. The retired `prompt` spelling is not a command and is not absorbed by the default `run` grammar, which would read it as a document reference and execute a file of that name: an invocation whose exact first token is `prompt` is refused before any scan, selection or path lookup, establishing nothing, while `xmd run ./prompt` still executes a document legitimately called that | built on the #660 stack | -| `` | writes and reviews one Plan, from a Prompt an ordinary document wrote. `…` expands its paired body once with the capabilities the calling document already has — the complete untrimmed rendering is the Prompt, and it is never emitted separately — and binds the exact approved Plan source. It is the public name of the packaged `` Component: exact first-party Markdown declared to every ordinary run, so a repository `Plan.md`, a workflow bundle, a registration, `Component.importComponent` middleware and another loaded copy can none of them answer for it. Paired only, one optional non-empty `session` prop, and a required `as`; a body that renders to nothing fails before any catalog, directory, Session, turn, review or check exists. The Agent writes under exactly the `xmd plan` ceiling however broad the calling document's authority is, and a host that cannot establish it refuses before placement — including the `` child an `xmd test` document launches, which is the run profile and therefore resolves the same protected bytes rather than reporting a missing component. An authored `session` keeps the existing durable named-directory lifetime, so the same name at the same site reaches the same conversation next time, while an omitted one is site- and iteration-unique, replay-stable and handed back after teardown; sibling sites stay distinct even when they write one name, and the name never becomes a path. Every turn, answer, check, approval and admission belongs to the enclosing document's journal, so a continuation restores completed authorship instead of repeating it; there is no second journal. Complete authorship teardown precedes structural admission, which precedes the binding — and the admission is structure alone, so a Plan declaring properties a later run will supply is returned rather than refused. It prints no source, creates no file, and executes nothing it returns | built on the #660 stack; no delivery, custom root, policy prop or replacement selector exists (#536 owns constrained caller-authored policy) | +| `xmd plan` | turns one Prompt into a Plan and delivers it, by executing one root document — and, only under `--run`, a second — with a complete scope boundary between them. First the packaged plan command document, under the internal `` identity — an adapter that projects the request into `` and returns the bytes it captured, so the command delivers a Plan rather than carrying one out — and inside that, the packaged `` Component under the authorship profile its own `` installs: one enclosing Session, a host ceiling of one host-owned directory dedicated to that logical session — under `~/.xmd/plan/sessions` by default, keyed by the digest of the name, never the name, created empty and required to be empty before the provider exists or a session is materialized, refused rather than cleaned when it is not, durable when the caller named the session and handed back non-recursively after teardown when it did not — with no additional directories, no MCP servers, no native tools and a private strict denial no permission flag widens, no Files, command, service or network capability for that document, no repository component search, and the Component's own private `` whose closed assessment answers `valid: false` for a defect the draft authored and raises for a defect the command line authored. Its instructions require every Plan to begin with one descriptive level-one title and to keep the Prompt's outcomes as readable steps with each component beside the step it performs, through repairs and revisions alike; that is an authorship and human-review requirement, and `` never enforces it. A tenth draft that still has problems may be stopped or explained: the explanation is one more ordinary turn in the same Session carrying only the final diagnostics, is inert text, reopens no draft limit, and ends the command. The host's instruction layer states only that an answer belongs to the message that asked for it, so which shape a turn wants stays in the document. Authorship sits outside durability: it runs on an invocation-owned in-memory stream that is never journaled, persisted, reused or replayed. Then, only after every provider, Prompt task and Elicitation resource inside the authorship frame has torn down, the Component's own `` structurally admits the exact approved bytes, and after that execution ends the host validates the returned Plan again against the command line, resolves props for exactly those bytes, and then delivers the approved Plan where the caller asked: to stdout byte for byte by default, to an exclusively created `--output` path, and — only under `--run` — through the ordinary supplied-source path under the `` identity, one ordinary document with its own Agent provider, its own journal and ordinary run output, result and failure behavior. A journal exists only when `--run` begins, and the flags that configure only a run are refused in preflight without it. The retired `prompt` spelling is not a command and is not absorbed by the default `run` grammar, which would read it as a document reference and execute a file of that name: an invocation whose exact first token is `prompt` is refused before any scan, selection or path lookup, establishing nothing, while `xmd run ./prompt` still executes a document legitimately called that | built on the #660 stack | +| `` | writes, reviews and carries out one Plan, from a Prompt an ordinary document wrote. `…` expands its paired body once with the capabilities the calling document already has — the complete untrimmed rendering is the Prompt, and it is never emitted separately — and then expands the exact approved Plan source where the element was written, as an embedded text root under the `` identity. Writing `as` binds those same bytes instead and performs none of the program's effects. It is the public name of the packaged `` Component: exact first-party Markdown declared to every ordinary run, so a repository `Plan.md`, a workflow bundle, a registration, `Component.importComponent` middleware and another loaded copy can none of them answer for it. Paired only, one optional non-empty `session` prop, and an optional `as`; a body that renders to nothing fails before any catalog, directory, Session, turn, review or check exists. The Agent writes under exactly the `xmd plan` ceiling however broad the calling document's authority is, and a host that cannot establish it refuses before placement — including the `` child an `xmd test` document launches, which is the run profile and therefore resolves the same protected bytes rather than reporting a missing component. An authored `session` keeps the existing durable named-directory lifetime, so the same name at the same site reaches the same conversation next time, while an omitted one is site- and iteration-unique, replay-stable and handed back after teardown; sibling sites stay distinct even when they write one name, and the name never becomes a path. Every turn, answer, check, approval and admission belongs to the enclosing document's journal, so a continuation restores completed authorship instead of repeating it; there is no second journal. Complete authorship teardown precedes structural admission, which precedes the expansion or the binding — and the admission is structure alone, so a Plan declaring properties a later run will supply is admitted rather than refused; the expansion then validates those properties against the ones ambient at the site and refuses a root `returns` declaration, both before the program's first effect. Stopping, repeated rejection and exhaustion, a failed turn, a structural refusal and cancellation each leave the calling document with no program effect and no binding. It prints no source and creates no file: what a reader sees without `as` is the program's own selected output, and with `as`, nothing at all | built on the #660 stack; no delivery, custom root, policy prop or replacement selector exists (#536 owns constrained caller-authored policy) | | `` / `` | chooses one branch by comparing a value with `===`. `……` decides its whole case structure from source before evaluating anything, then evaluates the selector once and each non-default matcher at most once in source order, expands the first `===` match — or the final default, or nothing — inline and transparently, and appends no journal event | built on the #692 stack | | `xmd upgrade` | replaces the standalone binary that ran it with a published release, by executing one root document: the packaged upgrade command document, under the internal `` identity, with an empty component search path and no Files, Process, Service, command, Fetch, Agent, Elicitation, workflow or repository capability. That document is an **ordinary streaming text root** — it declares no `returns` and uses neither `` nor `` — so its rendered body is the command's output: each root segment reaches the reader as it completes, and a branch the command did not take contributes no prose, no phase call and no result. Its durable events go to one invocation-local in-memory stream, or to the file `--journal` named and the CLI exclusively created; neither is ever read back, and neither grants any resume or retry authority. Markdown owns the whole of the policy — the exact-tag grammar, which release is selected, semantic-version comparison through the npm `semver` package, which consent an install needs, the status, already-current and installation branches, and the wording of every refusal and every report. A compiled macOS or Linux binary whose platform the release publishes for is the only host that declares the four phases that policy may reach, ``, ``, `` and ``, and it declares them to canonical execution rather than through any contextual Api, middleware, repository lookup, ordinary `xmd run` profile or public syntax catalog; every other entrypoint states its provenance and no authority at all, so an npm, Bun, Deno-source or compiled Windows invocation has no phase to reach and stops at its own refusal before release lookup or any filesystem change. That host alone owns the private half: the exact `process.execPath` spelling it will replace and never a link it resolved, one non-blocking exclusive advisory lock on a stable sidecar beside that file, the bounded anonymous GitHub reads under a scope-bound abort signal, the downloaded bytes, the digest, the staged candidate it runs for its version, and one same-directory rename. Opaque identity is the boundary between the two halves — a release identity per admitted release, then one candidate advancing `downloaded → verified → committed` exactly once, with one installation attempt per invocation — so the document chooses among the releases it was shown and can name no other release, target, asset or destination, skip verification or replay a phase. Before the rename every failure and cancellation leaves the installed file byte-identical; after it the candidate is authoritative and no cleanup restores the old bytes | built on the #659 stack | | `` / `printErrors(fn)` | prints failures | built on main | @@ -3785,7 +3860,7 @@ Status is measured against main. | document-aware `xmd run … --help` | describes what one document declares and every target it addresses, each as a full document reference with the description its section states, by inspection alone | built on the #463 stack | | targeted `xmd run` | reads a file argument as a document reference and executes the one exact target its selector resolved to, replacing the selector before execution rereads the file | built on the #412 stack | | targeted workflow definition | the V1 workflow definition optionally carries the exact canonical document target, which takes part in definition identity and in compatible reuse | built on the #412 stack; the workflow CLI does not supply one yet | -| declared Markdown component | a trusted host declares exact first-party Markdown to one execution as immutable data on an `ExecutionInstallation`: name, origin, source, its SHA-256, the accepted forms, an optional statement of the props and return that must agree with the parsed source, and an optional private component closure. Admission parses the bytes and refuses a mismatched digest or schema, a non-canonical form, a name that is not a component name or is structural, a duplicate, a reserved-registration collision and a private name a registration also claims. Resolution places it in the protected tier with reserved registrations, above the workflow component bundle, repository files and every registered default. Live import and retained history are held to the declared origin, digest and bytes, private names resolve only while canonical core expands the declaring bytes' own body — by the authored occurrence rather than by the name, so an answer kept from a legitimate private import authorizes no later site, no alias, no copy of the definition, no invocation that is over and no later execution — including one that declares no Markdown at all — while a private name written anywhere else resolves to nothing before the bundle, the repository or a registration can answer for it — and `xmd syntax` and document validation describe the declared contract from the same declaration without describing the closure. Closure is per name: only the declared component and its private closure become canonical imports, and every other name in the execution stays the ordinary open import middleware may still answer | built on the #660 stack; no public component uses it yet (#660 PR 2) | +| declared Markdown component | a trusted host declares exact first-party Markdown to one execution as immutable data on an `ExecutionInstallation`: name, origin, source, its SHA-256, the accepted forms, an optional statement of the props and return that must agree with the parsed source, an optional executable-source return disposition, and an optional private component closure. Admission parses the bytes and refuses a mismatched digest or schema, a non-canonical form, a name that is not a component name or is structural, a duplicate, a reserved-registration collision and a private name a registration also claims. Resolution places it in the protected tier with reserved registrations, above the workflow component bundle, repository files and every registered default. Live import and retained history are held to the declared origin, digest and bytes, private names resolve only while canonical core expands the declaring bytes' own body — by the authored occurrence rather than by the name, so an answer kept from a legitimate private import authorizes no later site, no alias, no copy of the definition, no invocation that is over and no later execution — including one that declares no Markdown at all — while a private name written anywhere else resolves to nothing before the bundle, the repository or a registration can answer for it — and `xmd syntax` and document validation describe the declared contract from the same declaration without describing the closure. Closure is per name: only the declared component and its private closure become canonical imports, and every other name in the execution stays the ordinary open import middleware may still answer | built on the #660 stack; `` is the one public component that uses it | | workflow component bundle | a workflow root declares a closed set of authored Markdown components; the V1 workflow definition optionally carries them as one array sorted by component name, each entry holding the name, its canonical repository-relative path inside the pinned commit and that blob's object ID, and an absent member identifies a run closed over no components — so a definition retained before the member existed reads unchanged. `start` and `resume` read every component from the definition's own pinned commit; the array takes part in definition identity and is compared as part of the same V1 descriptor in compatible reuse; and canonical core resolves those names and holds both live import and retained history to that exact bundle | built on the #301 stack; the full adversarial implementation loop and its scheduling remain unbuilt (#300), and generated XMD admits no bundled Markdown component (#369) | | `workflowInstallation()` / `getWorkflowRun()` | associates one document execution with a workflow run, through an `ExecutionInstallation` the trusted host passes to `executeInstalled()` | built on the #366 stack | | `retainedWorkflowInstallation()` | associates one document execution with a run storage already created, requiring exact journal agreement | built on the #366 stack | diff --git a/packages/cli/src/documents/Plan.md b/packages/cli/src/documents/Plan.md index 7a1c41e9..4f57859f 100644 --- a/packages/cli/src/documents/Plan.md +++ b/packages/cli/src/documents/Plan.md @@ -7,9 +7,11 @@ props: returns: type: string description: >- - Create an XMD program from a Prompt. `Ask for the user's - age.` expands its content normally to form the complete Prompt. -as: Required. The exact approved Plan source after teardown and structural admission. + Create an XMD program from a prompt. `Ask for the user's age.` + expands the approved plan. +as: >- + Optional. It captures the exact approved Plan source, after teardown and + structural admission, without expanding any of it. context: The complete Prompt, expanded once before authorship. --- diff --git a/packages/cli/src/plan-component.ts b/packages/cli/src/plan-component.ts index ff65ecfd..644d3b2a 100644 --- a/packages/cli/src/plan-component.ts +++ b/packages/cli/src/plan-component.ts @@ -91,6 +91,24 @@ export const PLAN_COMPONENT = "Plan"; */ export const PLAN_IDENTITY = ""; +/** + * What this host says the Component's return is. + * + * `` produces approved XMD program source rather than ordinary text, so a + * site that captures nothing has somewhere for that return to go: canonical + * core expands those exact bytes where the author wrote the element, under the + * identity above. A site that writes `as` binds the same bytes and runs none of + * them. + * + * It travels with the declaration and nowhere else. No frontmatter field names + * it, so a repository `Plan.md`, a workflow bundle member and a registration + * cannot ask for it, and neither can the Plan a run approves. + */ +const PLAN_DISPOSITION = { + kind: "executable-source", + sourceIdentity: PLAN_IDENTITY, +} as const; + /** Which surface reached the Component. Sealed: it is never a public prop. */ export type PlanSurface = "command" | "component"; @@ -236,6 +254,7 @@ export function* planComponentDeclaration( // empty rendering rather than on the spelling, which is the same answer for // a body that rendered to nothing. forms: ["paired"], + returnDisposition: PLAN_DISPOSITION, privates: [ planInputs(assembly), planAuthorship(assembly), @@ -270,6 +289,7 @@ export function* planComponentDescription(): Operation` written in an ordinary document. * * The same packaged Component `xmd plan` runs, reached the other way: a document - * writes ``, its body renders the Prompt, and the approved source arrives - * under `as`. What this tier is about is everything that differs from the - * command — the caller's Prompt, the caller's journal, the caller's authority — - * and everything that must not: the ceiling the Agent writes under, the order - * the phases run in, and the exact bytes that come back. + * writes ``, its body renders the Prompt, and the approved program is + * carried out where the element was written — or, with `as`, bound and left + * alone. What this tier is about is everything that differs from the command — + * the caller's Prompt, the caller's journal, the caller's authority — and + * everything that must not: the ceiling the Agent writes under, the order the + * phases run in, and the exact bytes that come back. * * Every seam is deterministic and in process: the scriptable ACPX runtime, a * scripted review, a recorded draft answer, and an authorship root the case @@ -15,11 +16,11 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { ensure, scoped, until } from "effection"; +import { ensure, scoped, spawn, until } from "effection"; import type { Operation } from "effection"; import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; import { randomUUID } from "node:crypto"; -import { readdir } from "node:fs/promises"; +import { readdir, readFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { @@ -49,6 +50,36 @@ const ROOT = "document.md"; /** A Plan the coding agent replies with, and that admission accepts. */ const PLAN = ["# Say hello", "", "This document greets you.", "", "Hello.", ""].join("\n"); +/** + * The same Plan with one observable, durable effect: a line appended to a log. + * + * A line rather than a file, because the question every case here asks is *how + * many times* — once when the program expanded, never when it was captured or + * refused, and not again when a partial journal resumed inside it. + */ +function programAppending(log: string): string { + return [ + "# Say hello", + "", + "the program ran.", + "", + "```bash exec silent", + `echo ran >> ${log}`, + "```", + "", + ].join("\n"); +} + +/** How many times the program's effect happened, from the log it appends to. */ +function* timesRun(log: string): Operation { + try { + const text = String(yield* until(readFile(log, "utf8"))); + return text.split("\n").filter((line) => line.trim().length > 0).length; + } catch { + return 0; + } +} + /** What one document run produced, and every phase that was reached. */ interface Run { output: string; @@ -79,7 +110,11 @@ function* authorshipRoot(): Operation { function* runDocument(options: { source: string; reply?: string; - reviews?: readonly ("Approve" | "Stop")[]; + /** Turns scripted whole, for a case about how one of them settles. */ + turns?: readonly { reply: string; stopReason?: string }[]; + reviews?: readonly ("Approve" | "Stop" | "Request changes")[]; + /** The properties the enclosing document is run with. */ + props?: Record; root?: string; includes?: readonly string[]; stream?: InMemoryStream; @@ -107,8 +142,13 @@ function* runDocument(options: { if (options.reply !== undefined) { harness.fake.script({ reply: options.reply }); } + for (const turn of options.turns ?? []) { + harness.fake.script(turn); + } for (const decision of options.reviews ?? ["Approve"]) { - harness.script({ decision }); + harness.script( + decision === "Request changes" ? { decision, feedback: "change it" } : { decision }, + ); } let value: Json | undefined; @@ -122,6 +162,7 @@ function* runDocument(options: { ...retainedSource(ROOT, options.source), stream, includes: [...(options.includes ?? [])], + ...(options.props === undefined ? {} : { props: options.props }), }, [ { @@ -284,7 +325,7 @@ describe("Tier PC — in an ordinary document", () => { expect(Reflect.get(Object(plan?.origin), "origin")).toBe(PLAN_ORIGIN); // The description a document author reads is the packaged Component's own // frontmatter, so the asset and the entry describing it are one text. - expect(plan?.description).toContain("Create an XMD program from a Prompt."); + expect(plan?.description).toContain("Create an XMD program from a prompt."); for (const category of catalog.categories) { const names = category.entries.map((entry) => entry.name); @@ -551,6 +592,314 @@ describe("Tier PC — in an ordinary document", () => { }); }); + it("PC19: an approved Plan expands where the element is written", function* () { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + const run = yield* runDocument({ + source: ["before", "", "Write a program.", "", "after", ""].join("\n"), + reply: programAppending(log), + }); + + expect(run.failure).toBe(undefined); + // One program, carried out once, between the two markers the author wrote + // around the element. + expect(yield* timesRun(log)).toBe(1); + const before = run.output.indexOf("before"); + const ran = run.output.indexOf("the program ran."); + const after = run.output.indexOf("after"); + expect(before).toBeGreaterThanOrEqual(0); + expect(ran).toBeGreaterThan(before); + expect(after).toBeGreaterThan(ran); + // What the reader sees is the program's output, not its source. + expect(run.output).not.toContain("```bash exec"); + expect(run.output).not.toContain("echo ran"); + }); + }); + + it("PC20: the same Plan under `as` binds the bytes and carries out none of them", function* () { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + const approved = programAppending(log); + const run = yield* runDocument({ + source: ['Write a program.', "", "got:{program}", ""].join("\n"), + reply: approved, + }); + + expect(run.failure).toBe(undefined); + // Byte for byte, and the effect in those bytes is the negative control. + expect(run.output).toContain(`got:${approved}`); + expect(yield* timesRun(log)).toBe(0); + }); + }); + + it("PC21: the program runs in the calling document's own environment", function* () { + yield* useWorkingDirectory(function* (dir) { + // A component on the caller's own include path. The program reaches it + // because the component selection, the includes and the registry are the + // enclosing execution's rather than a second run's. + yield* writeTextFile(join(dir, "Greeting.md"), "a greeting\n"); + + const approved = [ + "---", + "props:", + " type: object", + " properties:", + " who: { type: string }", + " required: [who]", + " additionalProperties: false", + "release: 1.2.3", + "---", + "", + "# Greet somebody", + "", + "", + "for {props.who} at {meta.release}, greeting {greeting} via ", + "", + "", + "this line is documentation and is not selected.", + "", + ].join("\n"); + + const run = yield* runDocument({ + source: [ + "---", + "props:", + " type: object", + " properties:", + " who: { type: string }", + " required: [who]", + " additionalProperties: false", + "---", + "", + '', + "Write a greeter.", + "", + ].join("\n"), + reply: approved, + includes: [dir], + props: { who: "ada" }, + }); + + expect(run.failure).toBe(undefined); + expect(run.output).toContain("for ada at 1.2.3, greeting hello via a greeting"); + // Its own top-level `` selected what renders, exactly as a root's does. + expect(run.output).not.toContain("this line is documentation"); + }); + }); + + it("PC22: every unsuccessful ending leaves no program effect and no binding", function* () { + const endings: readonly { + name: string; + reviews: readonly ("Approve" | "Stop" | "Request changes")[]; + reply?: string; + turns?: readonly { reply: string; stopReason?: string }[]; + expect: string; + }[] = [ + { name: "stopping", reviews: ["Stop"], expect: "stopped at your request" }, + { + name: "repeated rejection", + reviews: ["Request changes", "Request changes", "Stop"], + expect: "stopped at your request", + }, + { + name: "a failed turn", + reviews: [], + turns: [{ reply: "", stopReason: "refusal" }], + expect: 'stop reason "refusal"', + }, + { + name: "a structural refusal", + reviews: ["Approve"], + reply: "\n", + expect: "does not validate", + }, + { + // The program a `` produces expands under the authority the + // authored site carried, not the declaration's, so the phases only + // those exact bytes may write are not names it can reach. + name: "a private name in the approved source", + reviews: ["Approve"], + reply: '\n', + expect: "does not validate", + }, + ]; + + for (const ending of endings) { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + // A row scripts turns or a reply, never both; the rest approve the + // program whose one effect this case counts. + const scripted = + ending.turns === undefined + ? { reply: ending.reply ?? programAppending(log) } + : { turns: ending.turns }; + + const run = yield* runDocument({ + source: ["Write a program.", "", "after", ""].join("\n"), + ...scripted, + reviews: ending.reviews, + }); + + expect(run.failure).toContain(ending.expect); + // Nothing was carried out and nothing of the program was rendered in its + // place. The ending's name travels into the assertion so a failure says + // which of them produced it. + expect([ending.name, yield* timesRun(log)]).toEqual([ending.name, 0]); + expect(run.output).not.toContain("the program ran."); + }); + } + }); + + it("PC23: cancelling authorship leaves no program effect", function* () { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + const root = yield* authorshipRoot(); + const harness = yield* planDeclarationHarness({ + surface: "component", + authorshipRoot: root, + }); + // The turn never settles, so the run is interrupted while the agent is + // still writing — the one ending no review answer can produce. + harness.fake.script({ reply: programAppending(log), manual: true }); + + yield* scoped(function* () { + yield* installAgentComponents({ defaultAgent: AGENT, permissionMode: "deny-all" }); + const running = yield* spawn(function* () { + return yield* collect( + yield* executeInstalled( + { + ...retainedSource(ROOT, "Write a program.\n"), + stream: new InMemoryStream(), + includes: [], + }, + [ + { + components: agentIdentityComponents(), + declarations: [harness.declaration], + }, + ], + ), + ); + }); + yield* harness.fake.startedTurns(1); + yield* running.halt(); + }); + + expect(harness.reviews).toEqual([]); + expect(yield* timesRun(log)).toBe(0); + expect(yield* until(readdir(root))).toEqual([]); + }); + }); + + it("PC24: a root contract the caller cannot meet refuses before the program runs", function* () { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + const effect = ["```bash exec silent", `echo ran >> ${log}`, "```"].join("\n"); + + // A Plan whose root declares required properties nothing here offers. + // Admission accepts it — a Plan declaring properties is a Plan — and the + // expansion refuses it against the properties this site actually has. + const demanding = yield* runDocument({ + source: ["Write a greeter.", ""].join("\n"), + reply: [ + "---", + "props:", + " type: object", + " properties:", + " who: { type: string }", + " required: [who]", + " additionalProperties: false", + "---", + "", + "# Greet somebody", + "", + effect, + "", + ].join("\n"), + }); + + expect(demanding.failure).toContain("who"); + expect(yield* timesRun(log)).toBe(0); + + // And a Plan declaring a root return, which expanded source has nowhere + // to hand a value back to. + const valued = yield* runDocument({ + source: ["Write a program.", ""].join("\n"), + reply: [ + "---", + "returns:", + " type: string", + "---", + "", + "# Produce a value", + "", + effect, + "", + '', + "", + ].join("\n"), + }); + + expect(valued.failure).toContain("declares `returns`"); + expect(yield* timesRun(log)).toBe(0); + }); + }); + + it("PC25: a partial journal resumes inside the program without repeating it", function* () { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + const first = new InMemoryStream(); + const source = ["before", "", "Write a program.", "", "after", ""].join("\n"); + + const one = yield* runDocument({ + source, + reply: programAppending(log), + stream: first, + }); + expect(one.failure).toBe(undefined); + expect(yield* timesRun(log)).toBe(1); + + // A second run continuing that history, with a provider that would answer + // differently and a review nobody scripted. Authorship, the check, the + // approval and the admission are all restored — and so is the effect the + // program had already performed. + const two = yield* runDocument({ + source, + reply: "# A different Plan\n\nnot this one.\n", + reviews: [], + stream: yield* continuing(first), + }); + + expect(two.failure).toBe(undefined); + expect(two.harness.fake.prompts).toEqual([]); + expect(two.harness.reviews).toEqual([]); + expect(two.harness.checked).toEqual([]); + expect(yield* timesRun(log)).toBe(1); + expect(two.output).toBe(one.output); + }); + }); + + it("PC26: the catalog says the approved plan expands, and validation agrees", function* () { + yield* useWorkingDirectory(function* () { + const catalog = yield* syntaxCatalog([]); + const plan = catalog.categories[1].entries.find((entry) => entry.name === "Plan"); + + expect(plan?.returnDisposition).toEqual({ + kind: "executable-source", + sourceIdentity: "", + }); + expect(plan?.description).toContain("expands the approved plan"); + + // A site without `as` is what the common path writes, so validation has to + // accept it — and it asks the same question expansion does. + const validation = yield* validateDocument({ + ...retainedSource(ROOT, "Write a program.\n"), + declarations: [yield* planComponentDescription()], + }); + expect(validation.outcome).toBe("valid"); + }); + }); + it("PC14: nothing the Component does reaches the caller's filesystem", function* () { yield* useWorkingDirectory(function* (dir) { const run = yield* runDocument({ diff --git a/packages/cli/tests/syntax-cli.test.ts b/packages/cli/tests/syntax-cli.test.ts index 909b7b7e..a0ce31a9 100644 --- a/packages/cli/tests/syntax-cli.test.ts +++ b/packages/cli/tests/syntax-cli.test.ts @@ -266,6 +266,34 @@ describe("Tier SX — the run profile the command describes", () => { ]); }); + it("SX2c: says what returns, in both renderings", function* () { + const catalog = yield* syntaxCatalog([]); + const plan = catalog.categories[1].entries.find((entry) => entry.name === "Plan"); + + // The disposition is part of the contract a document author is reading: + // written without `as`, the approved plan expands where they wrote it. + expect(plan?.returnMode).toBe("value"); + expect(plan?.returnDisposition).toEqual({ + kind: "executable-source", + sourceIdentity: "", + }); + expect(plan?.description).toContain("expands the approved plan"); + expect(plan?.as).toContain("Optional"); + + const markdown = renderSyntaxMarkdown(catalog); + expect(markdown).toContain("**Returns:** executable source"); + expect(markdown).toContain("expands where you wrote it"); + + const json = JSON.parse(renderSyntaxJson(catalog)); + const described = json.categories[1].entries.find( + (entry: { name: string }) => entry.name === "Plan", + ); + expect(described.returnDisposition).toEqual({ + kind: "executable-source", + sourceIdentity: "", + }); + }); + it("SX3: describes without minting an execution claimant", function* () { const catalog = yield* syntaxCatalog([]); const session = catalog.categories[1].entries.find((entry) => entry.name === "Session"); diff --git a/packages/core/mod.ts b/packages/core/mod.ts index d31eb6d1..a9f8e548 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -29,6 +29,7 @@ export type { PartialContent, ComponentRegistry, ComponentSelection, + ExecutableSourceDisposition, Registered, RegistryEntry, SampleContext, diff --git a/packages/core/src/components/declared-markdown.ts b/packages/core/src/components/declared-markdown.ts index 2030b034..9ace7c00 100644 --- a/packages/core/src/components/declared-markdown.ts +++ b/packages/core/src/components/declared-markdown.ts @@ -53,6 +53,7 @@ import { documentationOf } from "./documentation.ts"; import type { ComponentDefinition, ComponentRegistry, + ExecutableSourceDisposition, FunctionComponentDefinition, InvocationForm, PropsSchema, @@ -100,6 +101,14 @@ export interface DeclaredMarkdownComponent { readonly props?: PropsSchema; /** What the host says the source returns. Refused when it disagrees. */ readonly returns?: ReturnsSchema; + /** + * What the host says that return *is*, when a value is not the whole of it. + * + * Host-only, and the one member here that says something the bytes cannot say + * about themselves. Omitting it is ordinary value-component behavior, so + * every declaration that says nothing keeps the contract it always had. + */ + readonly returnDisposition?: ExecutableSourceDisposition; /** Components only elements authored by these exact bytes may resolve. */ readonly privates?: readonly IdentityComponent[]; } @@ -113,6 +122,8 @@ export interface AdmittedDeclaredMarkdown { readonly forms: readonly InvocationForm[]; /** The parse of `source`, produced once and shared by every reader. */ readonly definition: ComponentDefinition; + /** The host's own copy of what it said this return is, or nothing. */ + readonly returnDisposition?: ExecutableSourceDisposition; readonly privates: readonly IdentityComponent[]; } @@ -205,6 +216,8 @@ export function* admitDeclaredMarkdown( ); } + const returnDisposition = admitDisposition(name, declaration.returnDisposition, definition); + // The same admission a registration is held to, so a declared contract is // admissible on exactly the terms every other declared contract is: the // schemas compile here, before a document can write the name. @@ -243,6 +256,7 @@ export function* admitDeclaredMarkdown( digest, forms, definition, + ...(returnDisposition === undefined ? {} : { returnDisposition }), privates, }); } @@ -258,6 +272,47 @@ export function* admitDeclaredMarkdown( return admitted; } +/** + * The host's own copy of what it said this declaration's return is. + * + * Copied rather than referenced, like every other authoritative member: a host + * that hands over a disposition and then mutates the object it still holds has + * replaced nothing. The check is on the source as well as on the statement — a + * disposition describes what the returned *string* is, so bytes that return + * something else, or nothing, cannot carry one. + */ +function admitDisposition( + name: string, + stated: ExecutableSourceDisposition | undefined, + definition: ComponentDefinition, +): ExecutableSourceDisposition | undefined { + if (stated === undefined) { + return undefined; + } + if (stated.kind !== "executable-source") { + throw refuse( + `the declared Markdown component "${name}" states a return disposition this version does ` + + "not know.", + ); + } + if (typeof stated.sourceIdentity !== "string" || stated.sourceIdentity.length === 0) { + throw refuse( + `the declared Markdown component "${name}" states an executable-source return without the ` + + "identity its source is read under.", + ); + } + if (definition.returns?.["type"] !== "string") { + throw refuse( + `the declared Markdown component "${name}" states an executable-source return, which is ` + + "what a returned string is. Its source returns something else.", + ); + } + return Object.freeze({ + kind: stated.kind, + sourceIdentity: stated.sourceIdentity, + }); +} + /** Whether two schemas describe the same thing, whatever order they wrote it in. */ function describesSameSchema(stated: PropsSchema, parsed: PropsSchema): boolean { return canonicalFingerprint(stated) === canonicalFingerprint(parsed); @@ -465,6 +520,28 @@ export class DeclaredImports implements ImportTier { return imported.path === declaration.origin ? this.#closures.get(name) : undefined; } + /** + * What the host said the return of `imported` is, if `imported` is the + * declaration's own definition. + * + * Decided the way a private closure is: from the definition canonical + * resolution retained — the one this expansion is about to invoke, reporting + * the origin core declared — rather than from the name. A repository file, a + * bundled component and a registration all report a different origin, so + * nothing that answers for a declared name by another route carries the + * host's disposition with it. + */ + executableSourceFor( + name: string, + imported: { kind: string; path?: string }, + ): ExecutableSourceDisposition | undefined { + const declaration = this.#catalog.component(name); + if (declaration?.returnDisposition === undefined || imported.kind !== "markdown") { + return undefined; + } + return imported.path === declaration.origin ? declaration.returnDisposition : undefined; + } + /** Whether some declaration keeps this name to itself. */ declaresPrivate(name: string): boolean { return this.#privates.has(name); diff --git a/packages/core/src/components/select.ts b/packages/core/src/components/select.ts index fe132d63..8c58aff0 100644 --- a/packages/core/src/components/select.ts +++ b/packages/core/src/components/select.ts @@ -146,6 +146,9 @@ export function* selectComponent( source: declared.source, forms: declared.forms, definition: declared.definition, + ...(declared.returnDisposition === undefined + ? {} + : { returnDisposition: declared.returnDisposition }), }; } diff --git a/packages/core/src/document-validation.ts b/packages/core/src/document-validation.ts index 5ec432ff..06ac047a 100644 --- a/packages/core/src/document-validation.ts +++ b/packages/core/src/document-validation.ts @@ -336,6 +336,11 @@ interface CompleteContract { readonly captures: readonly string[]; readonly forms: readonly InvocationForm[]; readonly hasReturns: boolean; + /** + * Whether an uncaptured return expands where the component is written, which + * only exact host-declared Markdown can say about itself (§5.3). + */ + readonly expandsSource?: boolean; } /** @@ -873,6 +878,7 @@ class ValidationState { captures: [], forms: selected.forms, hasReturns: selected.definition.returns !== undefined, + expandsSource: selected.returnDisposition !== undefined, }, capture, ); @@ -966,7 +972,12 @@ class ValidationState { // A site whose `as` was already refused is not asked again. const missingCapture = capture.refused ? undefined - : returnCaptureViolation(segment.name, contract.hasReturns, capture.binding); + : returnCaptureViolation( + segment.name, + contract.hasReturns, + capture.binding, + contract.expandsSource === true, + ); if (missingCapture !== undefined) { draft.tokens.push( this.#draft(ordinal, missingCapture.code, { diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index cb18b3e3..7d1f7e06 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -41,6 +41,7 @@ import type { Context } from "effection"; import type { ComponentDefinition, ComponentRegistry, + ExecutableSourceDisposition, FunctionComponent, FunctionComponentDefinition, JsonObject, @@ -221,7 +222,23 @@ type DurableSelection = * host that no longer declares it, refuses rather than continuing somebody * else's policy. */ - | { kind: "declared-markdown"; origin: string; digest: string; content: string } + | { + kind: "declared-markdown"; + origin: string; + digest: string; + content: string; + /** + * What the host said this declaration's return is, when it said anything. + * + * Recorded because it is not in the bytes: the digest above proves the + * source, and only this proves what the run was treating that source's + * return as. A record that carries none was continuing an ordinary value + * component, and a run that now declares an executable-source return + * under the same name is a different contract rather than the same one + * described more fully. + */ + disposition?: ExecutableSourceDisposition; + } /** * A component only the declaration that carries it may write. * @@ -269,15 +286,24 @@ function readDurableSelection(value: unknown): DurableSelection | undefined { const origin = record["origin"]; const digest = record["digest"]; const declaredContent = record["content"]; + const stated = record["disposition"]; if ( - members !== 4 || + (members !== 4 && members !== 5) || + (members === 5 && stated === undefined) || typeof origin !== "string" || typeof digest !== "string" || typeof declaredContent !== "string" ) { return undefined; } - return { kind: "declared-markdown", origin, digest, content: declaredContent }; + if (stated === undefined) { + return { kind: "declared-markdown", origin, digest, content: declaredContent }; + } + const disposition = readRecordedDisposition(stated); + if (disposition === undefined) { + return undefined; + } + return { kind: "declared-markdown", origin, digest, content: declaredContent, disposition }; } const path = record["path"]; @@ -326,6 +352,35 @@ function readDurableSelection(value: unknown): DurableSelection | undefined { return undefined; } +/** + * The return disposition a record carries, read as a closed protocol. + * + * Journal data, so every member is parsed: an unknown kind is malformed rather + * than ignored, because reading it as absent would continue an executable + * source as an ordinary value. + */ +function readRecordedDisposition(value: Json | undefined): ExecutableSourceDisposition | undefined { + if (!isJsonObject(value) || Object.keys(value).length !== 2) { + return undefined; + } + const sourceIdentity = value["sourceIdentity"]; + if (value["kind"] !== "executable-source" || typeof sourceIdentity !== "string") { + return undefined; + } + return { kind: "executable-source", sourceIdentity }; +} + +/** Whether two records describe the same return disposition, absence included. */ +function sameDisposition( + recorded: ExecutableSourceDisposition | undefined, + declared: ExecutableSourceDisposition | undefined, +): boolean { + if (recorded === undefined || declared === undefined) { + return recorded === declared; + } + return recorded.kind === declared.kind && recorded.sourceIdentity === declared.sourceIdentity; +} + /** * What a resumed run says when a recorded component import cannot be read. * @@ -450,6 +505,9 @@ function* durableImportComponent( origin: selected.origin, digest: selected.digest, content: selected.source, + ...(selected.returnDisposition === undefined + ? {} + : { disposition: selected.returnDisposition }), }; case "registered": return { @@ -521,7 +579,12 @@ function* durableImportComponent( declaration === undefined || declaration.origin !== selection.origin || declaration.digest !== selection.digest || - declaration.source !== selection.content + declaration.source !== selection.content || + // What the run was treating the return as is part of the selection, not a + // description of it: continuing a record that carries no disposition as + // executable source — or one that carries a different one — would run + // bytes this run's contract says something else about. + !sameDisposition(selection.disposition, declaration.returnDisposition) ) { throw new Error( `Component ${name} was recorded as the declared Markdown "${selection.origin}", which is ` + @@ -2833,6 +2896,14 @@ function* invoke( ...(declaration.returns === undefined ? {} : { returns: detachedSchema(declaration.returns) }), + ...(declaration.returnDisposition === undefined + ? {} + : { + returnDisposition: Object.freeze({ + kind: declaration.returnDisposition.kind, + sourceIdentity: declaration.returnDisposition.sourceIdentity, + }), + }), ...(declaration.privates === undefined ? {} : { privates: [...declaration.privates].map(retainedIdentityComponent) }), diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index a8359dd2..e7c596a2 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -24,6 +24,7 @@ import type { ComponentFailure, EvalEnv, ExecutableCodeBlock, + ExecutableSourceDisposition, FunctionComponentDefinition, Json, CodeBlockContext, @@ -97,6 +98,7 @@ import type { CheckedFailures } from "./component-failures.ts"; import type { ExpansionAuthority, ImportedDefinition } from "./components/import-authority.ts"; import { DeclaredMarkdownError } from "./components/declared-markdown.ts"; import type { PrivateImport } from "./components/declared-markdown.ts"; +import { parseMarkdownDefinitionPhased } from "./definition.ts"; import CoreTest from "./components/Test.ts"; import { carriesTestActivationDecision } from "./test-activation.ts"; import { declaredRouting, withRouting } from "./foreground.ts"; @@ -2477,6 +2479,13 @@ function* expandComponent( // reporting the origin core declared — rather than from the name alone. const bodyAuthority = authorityForBody(authority, name, definition); + // What the host said this declaration's return is. Read from the definition + // canonical resolution retained rather than from the name, so a repository + // file, a bundle member or a registration answering for a declared name + // carries none of it — and an execution that declares nothing has none to + // read at all. + const executableSource = authority?.declared?.executableSourceFor(name, definition); + const placementError = validateBodyStructure(definition.bodySegments, definition.returns); if (placementError) { return [yield* raise(placementError)]; @@ -2521,7 +2530,12 @@ function* expandComponent( return [yield* raise(schemaValidationErrorSegment(error, name))]; } - const missingCapture = returnCaptureViolation(name, definition.returns !== undefined, asBinding); + const missingCapture = returnCaptureViolation( + name, + definition.returns !== undefined, + asBinding, + executableSource !== undefined, + ); if (missingCapture !== undefined) { return [yield* raise({ type: "error", message: missingCapture.message, source: name })]; } @@ -2695,6 +2709,26 @@ function* expandComponent( return [yield* raise(schemaValidationErrorSegment(error, name))]; } + // Expanded only after the invocation has torn down, so the source runs + // under the authority the site already had rather than under anything the + // component installed for its own body. Under this component's own hide + // set, too: a program a component produced may not re-enter the component + // that produced it. + if (executableSource !== undefined && asBinding === undefined) { + return yield* expandExecutableSource( + name, + executableSource, + value, + callerProps, + newHideSet, + counter, + owner, + path, + checkedFailures, + authority, + ); + } + // Bind only after the invocation has torn down, so the value reaches the // caller's environment and never the component's own. const parentEnv = yield* env; @@ -2764,6 +2798,125 @@ function* expandComponent( return bodyOwner === undefined ? expanded : []; } +/** + * Expand the Executable Markdown a trusted declaration returned, at the site + * that authored it (spec §5.4, §6.10). + * + * This is an **embedded text root**, not another document execution. The source + * is a root — its own frontmatter metadata, its own root props schema and its + * own top-level `` selection apply — and everything else is the + * enclosing expansion's: the component selection and import authority, the + * Workspace, the error mode, the checked-failure ledger, the block counter, the + * output owner, the cancellation scope and the journal. Nothing here starts a + * lifecycle, imports `__root__`, produces a root result, installs a profile, or + * writes to `DocumentOutput`. + * + * The two root contracts are decided before the first effect. A root `returns` + * is refused outright: source expanded where it was written has nowhere to hand + * a value back to. Declared root props are validated against the props ambient + * at the site, which is what a caller can actually offer. + * + * The authority is the *site's*, not the declaration's, so the private closure + * only those declared bytes may write is unreachable from the program they + * produced — a Plan cannot write ``. + * + * Identities extend the authored element's path, so every effect the source + * performs is durable beneath that site and a replay re-enters the same + * expansion rather than deriving a new one. + */ +function* expandExecutableSource( + name: string, + disposition: ExecutableSourceDisposition, + returned: Json, + /** The props ambient at the site, which the embedded root's schema validates. */ + callerProps: Record, + hideSet: Set, + counter: BlockCounter, + owner: Segment[] | undefined, + path: string, + checkedFailures: CheckedFailures | undefined, + authority: ExpansionAuthority | undefined, +): Operation { + if (typeof returned !== "string") { + return [ + yield* raise({ + type: "error", + message: `<${name} /> returns executable source, which is text, and returned something else.`, + source: name, + }), + ]; + } + + const parsed = yield* parseMarkdownDefinitionPhased(name, disposition.sourceIdentity, returned); + if (!parsed.ok) { + return [yield* raise(schemaValidationErrorSegment(parsed.error, name))]; + } + const embedded = parsed.value; + + if (embedded.returns !== undefined) { + return [ + yield* raise({ + type: "error", + message: + `<${name} /> expands its source where it is written, and that source declares ` + + "`returns`. There is nowhere for a root value to go, so nothing was run.", + source: name, + }), + ]; + } + + const structureError = validateBodyStructure(embedded.bodySegments, undefined); + if (structureError) { + return [yield* raise(structureError)]; + } + + let validatedProps: Record; + try { + validatedProps = yield* validateProps(disposition.sourceIdentity, callerProps, embedded.props); + } catch (error) { + return [yield* raise(schemaValidationErrorSegment(error, name))]; + } + + // The caller's bindings, with the root's own validated props over them. The + // layered environment is derived from the caller's, so the source reads what + // the document has bound; its own values object is fresh, so what the source + // binds stays inside it. + const siteEnv = yield* env; + const propsEnv = propsEnvironment(validatedProps); + const sourceEnv = layerEnvironments(siteEnv, propsEnv, false) ?? propsEnv; + + const sourcePath = extendPath(path, { f: "source", at: disposition.sourceIdentity }); + + const expanded = yield* scoped(function* () { + yield* provideEnv(sourceEnv); + // Isolated from the loop the element was written in, for the reason a + // component's own body is: a `` belongs to a `` written in the + // flow that holds it, and this source is its own flow. A `` is + // reserved here for the same reason — the body below is given no return + // frame at all. + yield* ActiveLoop.set(undefined); + return yield* expandBody( + embedded.bodySegments, + [], + embedded.meta, + validatedProps, + hideSet, + counter, + undefined, + passthroughClaim, + owner, + sourcePath, + checkedFailures, + authority, + undefined, + ); + }); + + // The owner already holds what the source selected, exactly as a rendering + // body's does. + return owner === undefined ? expanded : []; +} + // Without `returns`, a function component's rendering is its return value, so // anything else is a contract violation rather than something to stringify. function asText(output: Json): string { diff --git a/packages/core/src/inspect.ts b/packages/core/src/inspect.ts index f2dca639..22532884 100644 --- a/packages/core/src/inspect.ts +++ b/packages/core/src/inspect.ts @@ -4,6 +4,7 @@ import { readTextFile } from "@executablemd/runtime"; import type { ComponentOrigin, ComponentSelection, + ExecutableSourceDisposition, InvocationForm, PropsSchema, ReturnsSchema, @@ -176,6 +177,8 @@ export interface DescribedContract extends ComponentDocumentation { readonly forms: readonly InvocationForm[]; readonly captures: readonly string[]; readonly returnMode: "text" | "value"; + /** What the declaring host says that return is, when it says more (§5.3). */ + readonly returnDisposition?: ExecutableSourceDisposition; } /** @@ -264,11 +267,12 @@ function* completeEntry( } function describedContract(entry: CompleteComponentSyntaxEntry): DescribedContract { - const { forms, captures, returnMode, description, as, context } = entry; + const { forms, captures, returnMode, returnDisposition, description, as, context } = entry; return { forms, captures, returnMode, + ...(returnDisposition === undefined ? {} : { returnDisposition }), ...(description === undefined ? {} : { description }), ...(as === undefined ? {} : { as }), ...(context === undefined ? {} : { context }), @@ -340,6 +344,12 @@ export interface CompleteComponentSyntaxEntry { readonly captures: readonly string[]; readonly returnMode: "text" | "value"; readonly returns: ReturnsSchema; + /** + * What the declaring host says that return is, when a value is not the whole + * of it (§5.3). Only exact declared Markdown carries one, so a reader can see + * which components expand what they return and which bind it. + */ + readonly returnDisposition?: ExecutableSourceDisposition; readonly description?: string; readonly as?: string; readonly context?: string; @@ -536,6 +546,9 @@ function* componentEntry( props: definition.props, captures: [], returns: definition.returns, + ...(selected.returnDisposition === undefined + ? {} + : { returnDisposition: selected.returnDisposition }), documentation: documentationOf(definition.meta), }); } @@ -580,6 +593,8 @@ interface CompleteContract { captures: readonly string[]; /** Absent in text mode, which is what `returnMode` reports. */ returns: ReturnsSchema | undefined; + /** Present only where a declaring host stated one. */ + returnDisposition?: ExecutableSourceDisposition; documentation: ComponentDocumentation; } @@ -600,6 +615,9 @@ function complete( captures: contract.captures, returnMode: contract.returns === undefined ? "text" : "value", returns: contract.returns ?? TEXT_RETURN_SCHEMA, + ...(contract.returnDisposition === undefined + ? {} + : { returnDisposition: contract.returnDisposition }), ...contract.documentation, }; } diff --git a/packages/core/src/invocation-rules.ts b/packages/core/src/invocation-rules.ts index 3564627e..bc65a649 100644 --- a/packages/core/src/invocation-rules.ts +++ b/packages/core/src/invocation-rules.ts @@ -68,13 +68,20 @@ export function capturedBinding(value: Json | undefined): string | undefined { /** * A component that declares `returns` renders nothing, so an invocation that * captures nothing would discard the only thing it produces. + * + * Unless a trusted host declared that return to be executable source (§5.3): + * such a component has somewhere for an uncaptured return to go, because the + * engine expands it where the component was written. Expansion and document + * validation ask this one question, so they cannot disagree about which + * invocations need `as`. */ export function returnCaptureViolation( name: string, declaresReturns: boolean, capture: string | undefined, + expandsSource = false, ): StructuralViolation | undefined { - return declaresReturns && capture === undefined + return declaresReturns && capture === undefined && !expandsSource ? { code: "return-usage-invalid", source: name, diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 4dfe15bf..0fd57384 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -317,6 +317,26 @@ export interface PartialContent { readonly failure?: unknown; } +/** + * What a trusted host says the exact Markdown it declares returns, beyond the + * value its `returns` schema describes (spec §5.3). + * + * `executable-source` says the returned string is Executable Markdown the + * engine expands where the component was written, unless the site captured it + * with `as`. `sourceIdentity` is the identity those bytes are read under — + * source positions and diagnostics report it, the way `` reports supplied + * root text. + * + * Only a declaration carries this. There is no frontmatter field for it, so a + * repository component, a workflow bundle member, a registration and a document + * cannot ask for it, and a declaration that does is refused unless its own + * source returns a string. + */ +export type ExecutableSourceDisposition = { + readonly kind: "executable-source"; + readonly sourceIdentity: string; +}; + /** * Where a selected implementation came from (spec §5.3). * @@ -362,6 +382,8 @@ export type ComponentSelection = source: string; forms: readonly InvocationForm[]; definition: ComponentDefinition; + /** What the host says this declaration's return is, when it says more. */ + returnDisposition?: ExecutableSourceDisposition; } | { kind: "unresolved"; searched: string[]; registered: readonly ComponentOrigin[] }; diff --git a/packages/core/tests/executable-source.test.ts b/packages/core/tests/executable-source.test.ts new file mode 100644 index 00000000..f33f2b0b --- /dev/null +++ b/packages/core/tests/executable-source.test.ts @@ -0,0 +1,637 @@ +/** + * Tier ES — a declared return the engine expands where it was written. + * + * A trusted host may say that the string one of its declared Markdown + * components returns is not an ordinary value but Executable Markdown. Written + * without `as`, canonical core expands those exact bytes at the authored site + * as an **embedded text root**; written with `as`, it binds the same bytes and + * expands none of them. + * + * What this tier measures is that the projection is genuinely embedded rather + * than a second run: the effects it performs are the enclosing document's, they + * receive durable identities beneath the authored element, and a replay resumes + * inside them without repeating one. And that nothing else acquires the + * behavior — an ordinary value component still requires `as`, a repository file + * cannot ask for the disposition in frontmatter, and a journal recorded under a + * different disposition refuses rather than continuing under this one. + * + * Every effect here is in process and deterministic: the program's one durable + * effect is an identity component that records what it was asked, so a case + * about something *not* running is proven by that recorder staying empty rather + * than by absent output. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { ensure, scoped, until } from "effection"; +import { forEach } from "@effectionx/stream-helpers"; +import type { Operation } from "effection"; +import { rm, writeTextFile } from "@effectionx/fs"; +import { mkdtemp, realpath } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { createDurableOperation, InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, Json } from "@executablemd/durable-streams"; +import { collect } from "../src/collect.ts"; +import { executeInstalled, sourceDigest } from "../host.ts"; +import type { + DeclaredMarkdownComponent, + ExecutionInstallation, + IdentityClaimant, + IdentityComponent, +} from "../host.ts"; +import { inspectComponent, inspectSyntax } from "../src/inspect.ts"; +import { validateDocument } from "../src/document-validation.ts"; +import { retainedSource } from "../src/root-source.ts"; +import type { ComponentInvocation } from "../src/invocation-identity.ts"; + +const ROOT_PATH = "documents/root.md"; +const PROGRAM_ORIGIN = "@executablemd/test/Program.md"; +const SOURCE_IDENTITY = ""; + +/** + * The declared component this tier runs against. + * + * It stands where `` stands and does the same one thing an authorship + * workflow does at the end: hand back exact program source. Authorship itself + * is the CLI's tier — here the return is the caller's own text, so every case + * is about what the engine does with approved bytes rather than about how they + * were approved. + */ +const PROGRAM_SOURCE = [ + "---", + "props:", + " type: object", + " properties:", + " source: { type: string }", + " required: [source]", + " additionalProperties: false", + "returns:", + " type: string", + "---", + "", + "", + "", +].join("\n"); + +/** The same component with no disposition: an ordinary string-valued one. */ +function program(overrides: Partial = {}): DeclaredMarkdownComponent { + const source = overrides.source ?? PROGRAM_SOURCE; + return { + name: "Program", + origin: PROGRAM_ORIGIN, + source, + digest: sourceDigest(source), + forms: ["self-closing"], + ...overrides, + }; +} + +/** The same component as the host that expands what it returns declares it. */ +function expanding(overrides: Partial = {}): DeclaredMarkdownComponent { + return program({ + returnDisposition: { kind: "executable-source", sourceIdentity: SOURCE_IDENTITY }, + ...overrides, + }); +} + +/** + * The program's one durable effect, recorded where it happens. + * + * An identity component rather than an ordinary registration, because durable + * work is the point: what it names is journaled, so a replay restores it and + * this recorder stays empty. A case about something not running is proven by + * that, never by absent output. + */ +function effect(performed: string[]): IdentityComponent { + return { + name: "Effect", + origin: `${PROGRAM_ORIGIN}#Effect`, + forms: ["self-closing"], + props: { + type: "object", + properties: { name: { type: "string" } }, + required: ["name"], + additionalProperties: false, + }, + factory: (claim: IdentityClaimant) => + function* Effect( + props: Record, + invocation: ComponentInvocation, + ): Operation { + const id = yield* claim(invocation); + const label = String(props.name); + return yield* (function* (): Operation { + return (yield createDurableOperation( + { type: "call", name: `effect:${id}` }, + // deno-lint-ignore require-yield + function* () { + performed.push(label); + return label; + }, + )) as string; + })(); + }, + }; +} + +/** One document, run against one set of declarations. */ +function* runDocument(options: { + source: string; + declarations: readonly DeclaredMarkdownComponent[]; + performed: string[]; + stream?: InMemoryStream; + includes?: readonly string[]; + props?: Record; +}): Operation<{ output: string; failure: string | undefined; stream: InMemoryStream }> { + const stream = options.stream ?? new InMemoryStream(); + const chunks: string[] = []; + let failure: string | undefined; + yield* scoped(function* () { + try { + const installation: ExecutionInstallation = { + declarations: [...options.declarations], + components: [effect(options.performed)], + }; + const execution = yield* executeInstalled( + { + ...retainedSource(ROOT_PATH, options.source), + stream, + includes: [...(options.includes ?? [])], + ...(options.props === undefined ? {} : { props: options.props }), + }, + [installation], + ); + // Drained the way a consumer drains it, so a run that failed still shows + // what it printed. A refusal proven by absent output would otherwise be + // indistinguishable from output nobody read. + yield* forEach(function* (chunk: string) { + chunks.push(chunk); + }, execution.output); + yield* collect(execution); + } catch (error) { + failure = error instanceof Error ? error.message : String(error); + } + }); + return { output: chunks.join(""), failure, stream }; +} + +/** A document that names its program as a binding and then writes the element. */ +function invoking(source: string, attributes = ""): string { + return [ + "before", + "", + ``, + ``, + "", + "after", + "", + ].join("\n"); +} + +/** A partial continuation of one run: everything it recorded but the terminals. */ +function* continuing(stream: InMemoryStream): Operation { + const partial = new InMemoryStream(); + for (const event of yield* stream.readAll()) { + if (event.type === "close") { + continue; + } + yield* partial.append(event); + } + return partial; +} + +/** Whether one retained event is a component import. */ +function isImport(event: DurableEvent): boolean { + return event.type === "yield" && event.description.type === "import_component"; +} + +/** A temporary directory this scope owns and removes. */ +function* useTempDir(): Operation { + const dir = yield* until(realpath(yield* until(mkdtemp(join(tmpdir(), "xmd-es-"))))); + yield* ensure(() => rm(dir, { recursive: true, force: true })); + return dir; +} + +describe("Tier ES — executable source returns", () => { + it("ES1: the approved source expands once, where the element is written", function* () { + const performed: string[] = []; + const run = yield* runDocument({ + source: invoking( + ["# The program", "", "the program ran.", "", '', ""].join("\n"), + ), + declarations: [expanding()], + performed, + }); + + expect(run.failure).toBe(undefined); + // The effect happened exactly once, and it happened between the markers the + // caller wrote around the element. + expect(performed).toEqual(["one"]); + const before = run.output.indexOf("before"); + const ran = run.output.indexOf("the program ran."); + const after = run.output.indexOf("after"); + expect(before).toBeGreaterThanOrEqual(0); + expect(ran).toBeGreaterThan(before); + expect(after).toBeGreaterThan(ran); + // What the reader sees is the program's output, not its source: the element + // that performed the effect is not printed in its place. + expect(run.output).not.toContain(''); + expect(run.output).not.toContain("returns:"); + }); + + it("ES2: the source runs in the document's own environment", function* () { + const dir = yield* useTempDir(); + // A component the current selection resolves, sitting on the caller's own + // include path. The embedded source reaches it because import authority, + // includes and the registry are the enclosing execution's. + yield* writeTextFile(join(dir, "Greeting.md"), "a greeting\n"); + + const performed: string[] = []; + const source = [ + "---", + "props:", + " type: object", + " properties:", + " who: { type: string }", + " required: [who]", + " additionalProperties: false", + "release: 1.2.3", + "---", + "", + "# The program", + "", + "", + "for {props.who} at {meta.release}, greeting {greeting} via ", + "", + '', + "", + "", + "this line is documentation and is not selected.", + "", + ].join("\n"); + + const run = yield* runDocument({ + source: [ + "---", + "props:", + " type: object", + " properties:", + " who: { type: string }", + " required: [who]", + " additionalProperties: false", + "---", + "", + "before", + "", + '', + ``, + "", + "", + "after", + "", + ].join("\n"), + declarations: [expanding()], + performed, + includes: [dir], + props: { who: "ada" }, + }); + + expect(run.failure).toBe(undefined); + expect(performed).toEqual(["two"]); + // Ambient props reach the root schema, the root's own frontmatter reaches + // `meta`, the caller's binding is still bound, and the imported component + // rendered. + expect(run.output).toContain("for ada at 1.2.3, greeting hello via a greeting"); + // Top-level `` selected, exactly as it does for a root document. + expect(run.output).not.toContain("this line is documentation"); + }); + + it("ES3: the same source under `as` binds byte for byte and runs none of it", function* () { + const performed: string[] = []; + const source = [ + "# The program", + "", + "the program ran.", + "", + '', + "", + ].join("\n"); + const run = yield* runDocument({ + source: [ + ``, + '', + "", + "got:{captured}", + "", + ].join("\n"), + declarations: [expanding()], + performed, + }); + + expect(run.failure).toBe(undefined); + // The negative control: the source was bound, so nothing in it happened. + expect(performed).toEqual([]); + expect(run.output).toContain(`got:${source}`); + }); + + it("ES4: a return that never arrives expands nothing and binds nothing", function* () { + const performed: string[] = []; + // The declaration fails before its ``, which is what every + // unsuccessful authorship ending is from the engine's side. + const refusing = [ + "---", + "props:", + " type: object", + " properties:", + " source: { type: string }", + " required: [source]", + " additionalProperties: false", + "returns:", + " type: string", + "---", + "", + '', + "", + "", + "", + ].join("\n"); + + const source = ["# The program", "", '', ""].join("\n"); + const run = yield* runDocument({ + source: invoking(source), + declarations: [expanding({ source: refusing, digest: sourceDigest(refusing) })], + performed, + }); + + expect(run.failure).toContain("nothing was approved"); + expect(performed).toEqual([]); + expect(run.output).not.toContain("# The program"); + }); + + it("ES5: a root `returns` and a root-props mismatch each refuse before an effect", function* () { + const valued = [ + "---", + "returns:", + " type: string", + "---", + "", + '', + "", + '', + "", + ].join("\n"); + + const valuedPerformed: string[] = []; + const valuedRun = yield* runDocument({ + source: invoking(valued), + declarations: [expanding()], + performed: valuedPerformed, + }); + + expect(valuedRun.failure).toContain("declares `returns`"); + expect(valuedPerformed).toEqual([]); + + const demanding = [ + "---", + "props:", + " type: object", + " properties:", + " who: { type: string }", + " required: [who]", + " additionalProperties: false", + "---", + "", + '', + "", + ].join("\n"); + + const demandingPerformed: string[] = []; + const demandingRun = yield* runDocument({ + source: invoking(demanding), + declarations: [expanding()], + performed: demandingPerformed, + }); + + expect(demandingRun.failure).toContain("who"); + expect(demandingPerformed).toEqual([]); + }); + + it("ES6: a partial journal resumes inside the source without repeating an effect", function* () { + const source = ["# The program", "", "the program ran.", "", '', ""].join( + "\n", + ); + const first: string[] = []; + const one = yield* runDocument({ + source: invoking(source), + declarations: [expanding()], + performed: first, + }); + + expect(one.failure).toBe(undefined); + expect(first).toEqual(["six"]); + + const second: string[] = []; + const two = yield* runDocument({ + source: invoking(source), + declarations: [expanding()], + performed: second, + stream: yield* continuing(one.stream), + }); + + expect(two.failure).toBe(undefined); + // The completed effect was restored rather than performed again, and the + // document produced the same text. + expect(second).toEqual([]); + expect(two.output).toBe(one.output); + }); + + it("ES6b: the source is its own flow, not the flow that wrote the element", function* () { + const performed: string[] = []; + // A stray `` in the program. It belongs to a `` written in + // the program, and there is none — so it is reported there rather than + // silently ending the caller's loop. + const source = ["# The program", "", "", "", '', ""].join("\n"); + const run = yield* runDocument({ + source: [ + "before", + "", + "", + ``, + "", + '', + "", + "", + ].join("\n"), + declarations: [expanding()], + performed, + }); + + // Reported where it was written rather than ending the caller's loop, which + // is what a `` reaching the enclosing frame would have done + // silently — one iteration, no diagnostic, and a document that looked fine. + expect(run.failure).toContain(" must be written inside a "); + expect(performed).toEqual([]); + }); + + it("ES7: the expansion imports no second root", function* () { + const performed: string[] = []; + const run = yield* runDocument({ + source: invoking(["# The program", "", '', ""].join("\n")), + declarations: [expanding()], + performed, + }); + + expect(run.failure).toBe(undefined); + const roots = (yield* run.stream.readAll()) + .filter(isImport) + .filter((event) => event.type === "yield" && event.description.name === "__root__"); + // One root import, the enclosing document's. An embedded projection that + // started another execution would record a second. + expect(roots).toHaveLength(1); + }); + + it("ES8: an ordinary string-valued declaration still requires `as`", function* () { + const performed: string[] = []; + const run = yield* runDocument({ + source: invoking(["# The program", "", '', ""].join("\n")), + declarations: [program()], + performed, + }); + + expect(run.failure).toContain("must be invoked with `as`"); + expect(performed).toEqual([]); + }); + + it("ES9: a repository component cannot ask for the disposition in frontmatter", function* () { + const dir = yield* useTempDir(); + // The frontmatter key a document author might reach for. It is not syntax: + // it lands in ordinary metadata and decides nothing. + yield* writeTextFile( + join(dir, "Opt.md"), + [ + "---", + "returns:", + " type: string", + "returnDisposition:", + " kind: executable-source", + " sourceIdentity: ", + "---", + "", + '"} />', + "", + ].join("\n"), + ); + + const performed: string[] = []; + const run = yield* runDocument({ + source: ["", ""].join("\n"), + declarations: [expanding()], + performed, + includes: [dir], + }); + + expect(run.failure).toContain("must be invoked with `as`"); + expect(performed).toEqual([]); + }); + + it("ES10: a replay whose recorded disposition differs refuses", function* () { + // The same bytes, the same origin and the same digest — and a host that no + // longer says the return is executable source. The record pins what the run + // was treating that return as, so this is a different contract rather than + // the same one described more briefly. + const quiet = ["# The program", "", "the program ran.", ""].join("\n"); + const first: string[] = []; + const one = yield* runDocument({ + source: invoking(quiet), + declarations: [expanding()], + performed: first, + }); + expect(one.failure).toBe(undefined); + + const dropped: string[] = []; + const two = yield* runDocument({ + source: invoking(quiet), + declarations: [program()], + performed: dropped, + stream: yield* continuing(one.stream), + }); + + expect(two.failure).toContain("recorded as the declared Markdown"); + expect(two.output).not.toContain("the program ran."); + expect(dropped).toEqual([]); + + // And the other direction: a run that captured an ordinary value is not + // continued as executable source. The effect in that source is the control + // — a promoted replay would perform it. + const source = ["# The program", "", '', ""].join("\n"); + const captured: string[] = []; + const three = yield* runDocument({ + source: invoking(source, ' as="captured"'), + declarations: [program()], + performed: captured, + }); + expect(three.failure).toBe(undefined); + expect(captured).toEqual([]); + + const promoted: string[] = []; + const four = yield* runDocument({ + source: invoking(source, ' as="captured"'), + declarations: [expanding()], + performed: promoted, + stream: yield* continuing(three.stream), + }); + + expect(four.failure).toContain("recorded as the declared Markdown"); + expect(promoted).toEqual([]); + }); + + it("ES11: a declaration whose source returns no string cannot carry the disposition", function* () { + const text = ["# Not a value component", ""].join("\n"); + const performed: string[] = []; + const run = yield* runDocument({ + source: "\n", + declarations: [expanding({ source: text, digest: sourceDigest(text), forms: ["paired"] })], + performed, + }); + + expect(run.failure).toContain("executable-source return"); + expect(performed).toEqual([]); + }); + + it("ES12: inspection and validation describe the same contract", function* () { + const declarations = [expanding()]; + + const described = yield* inspectComponent({ name: "Program", declarations }); + expect(described.kind).toBe("markdown"); + expect(Reflect.get(Object(described), "returnDisposition")).toEqual({ + kind: "executable-source", + sourceIdentity: SOURCE_IDENTITY, + }); + + const catalog = yield* inspectSyntax({ declarations }); + const entry = catalog.categories[1].entries.find((candidate) => candidate.name === "Program"); + expect(entry?.returnMode).toBe("value"); + expect(entry?.returnDisposition).toEqual({ + kind: "executable-source", + sourceIdentity: SOURCE_IDENTITY, + }); + + // Validation asks the same question expansion does, so a site that needs no + // capture here needs none there. + const valid = yield* validateDocument({ + ...retainedSource(ROOT_PATH, '\n'), + declarations, + }); + expect(valid.outcome).toBe("valid"); + + const ordinary = yield* validateDocument({ + ...retainedSource(ROOT_PATH, '\n'), + declarations: [program()], + }); + expect(ordinary.outcome).toBe("invalid"); + expect(ordinary.diagnostics.map((diagnostic) => diagnostic.code)).toContain( + "return-usage-invalid", + ); + }); +}); diff --git a/scripts/tests/cli-npm-bin.test.ts b/scripts/tests/cli-npm-bin.test.ts index 1e5a14ac..42ca0dc1 100644 --- a/scripts/tests/cli-npm-bin.test.ts +++ b/scripts/tests/cli-npm-bin.test.ts @@ -173,6 +173,15 @@ describe("npm CLI package", { sanitizeOps: false, sanitizeResources: false }, () digest, }); expect(plan.forms).toEqual(["paired"]); + // What that return *is* travels with the declaration rather than with the + // bytes, so the digest above cannot answer for it: a build that shipped the + // Component without its disposition would let a document write `` and + // then refuse it for having captured nothing. + expect(plan.returnDisposition).toEqual({ + kind: "executable-source", + sourceIdentity: "", + }); + expect(plan.description).toContain("expands the approved plan"); // And no private capability is syntax a document may write, in any build. for (const name of ["PlanInputs", "PlanAuthorship", "CheckDraft", "AdmitPlan"]) { expect(entries.map((entry: { name?: string }) => entry?.name)).not.toContain(name); diff --git a/scripts/tests/plan-component-compiled.test.ts b/scripts/tests/plan-component-compiled.test.ts index faa2499f..b6866e65 100644 --- a/scripts/tests/plan-component-compiled.test.ts +++ b/scripts/tests/plan-component-compiled.test.ts @@ -83,6 +83,15 @@ describe("compiled xmd", { sanitizeOps: false, sanitizeResources: false }, () => }); expect(plan.forms).toEqual(["paired"]); expect(plan.returnMode).toBe("value"); + // What that return *is* travels with the declaration rather than with the + // bytes, so the digest above cannot answer for it: a build that shipped the + // Component without its disposition would let a document write `` and + // then refuse it for having captured nothing. + expect(plan.returnDisposition).toEqual({ + kind: "executable-source", + sourceIdentity: "", + }); + expect(plan.description).toContain("expands the approved plan"); // And the private capabilities are not syntax any build lets a document // write. diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index b32d0cfb..f8f71b04 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -2704,11 +2704,11 @@ outside a workflow run learns that a bundle exists. A trusted host may hand one execution exact first-party Markdown: the public name, the reported origin, the source, the SHA-256 of those bytes, the accepted -forms, an optional statement of the props schema and return, and an optional -private component closure. It crosses on an `ExecutionInstallation`, by value, -before any installation, middleware or document code exists — the same terms the -component bundle crosses on — so no caller-facing option selects one, adds one, -or names its source. +forms, an optional statement of the props schema and return, an optional return +disposition, and an optional private component closure. It crosses on an +`ExecutionInstallation`, by value, before any installation, middleware or +document code exists — the same terms the component bundle crosses on — so no +caller-facing option selects one, adds one, or names its source. **It is held to its own bytes.** Canonical core parses the source and refuses the declaration before the root document is imported when the stated digest is @@ -2720,6 +2720,23 @@ declared name therefore fails where it is installed. The prose a reader sees is the source's own frontmatter, exactly as it is for any other Markdown component, so the asset and the entry describing it are one text. +**A declaration may say what its return *is*.** `returnDisposition` is the one +member that states something the bytes cannot state about themselves, and it has +one value: `{ kind: "executable-source", sourceIdentity }`. It says the string +those bytes return is Executable Markdown rather than an ordinary value, and +that a site which captures nothing has somewhere for it to go — canonical core +expands it there, as an embedded text root read under `sourceIdentity` (§5.4). +Written with `as`, the same invocation binds the exact source and expands none +of it (§6.10). + +It is admitted against the source as well as against itself: a declaration whose +Markdown returns anything but a string is refused, so a disposition never +describes a return that is not text. There is no frontmatter field for it. A +repository file, a workflow bundle member, a registration, a function component +and a document-authored component therefore cannot request it, and neither can a +program a `` approves. Omitting it is ordinary value-component behavior, +which is what every declaration that says nothing keeps. + **A declaration may carry private components.** They are ordinary declared components (§5.6) — one domain, one claimant, revoked when the execution is torn down — and nothing registers them. Canonical core offers one for the duration of @@ -2776,9 +2793,9 @@ any other name: an unrelated import in the same execution is the ordinary open import it has always been. **Every declaration is read once, before any installation runs.** The name, -origin, source, digest, forms, prose and each private declaration are captured -by the invocation and held by it, with the factory bound and every array and -schema copied. A schema is copied rather than referenced because it is a whole +origin, source, digest, forms, prose, return disposition and each private +declaration are captured by the invocation and held by it, with the factory +bound and every array, schema and disposition copied. A schema is copied rather than referenced because it is a whole object graph: holding the caller's object would let a hook reach into it after capture and change the contract admission compiles, registration publishes and expansion validates each invocation against. So a host that hands over a @@ -2787,22 +2804,29 @@ holds — has replaced nothing: what is admitted, registered and executed is wha it declared at the moment of capture. **The journal records the asset.** A declared import records exactly -`{ kind: "declared-markdown", origin, digest, content }` and a private one -records exactly `{ kind: "declared-private", origin }`. A continuation reads -both as hostile data and verifies the recorded origin, digest and bytes against -what this run declares, reading no file: a host that no longer declares the name, -or that declares different bytes under it, refuses rather than continuing -somebody else's Markdown, and a recorded private import refuses unless the -element asking is inside the same declaration. +`{ kind: "declared-markdown", origin, digest, content }`, with `disposition` +beside them when the host stated one, and a private one records exactly +`{ kind: "declared-private", origin }`. A continuation reads both as hostile data +and verifies the recorded origin, digest, bytes and disposition against what this +run declares, reading no file: a host that no longer declares the name, that +declares different bytes under it, or that now says something else about what its +return is, refuses rather than continuing somebody else's Markdown, and a +recorded private import refuses unless the element asking is inside the same +declaration. The disposition is compared whole and by absence: a record that +carries none was continuing an ordinary value component, so it is never read as +executable source, and one that carries a different one is a different contract +rather than the same one described more fully. **`` is the one of these the `run` profile has.** The CLI declares `packages/cli/src/documents/Plan.md` to every ordinary run under the origin -`@executablemd/cli/Plan.md`, paired-only, returning a string. Its body is the -Prompt, rendered once with the capabilities the calling document already has; -its `as` receives the exact approved Plan source after the authorship frame has -been dismantled and the bytes have been structurally admitted. Its four private -capabilities — ``, ``, `` and -`` — are the closure those exact bytes carry, and are syntax no +`@executablemd/cli/Plan.md`, paired-only, returning a string, with the +executable-source disposition and the identity ``. Its body is the Prompt, +rendered once with the capabilities the calling document already has. Written +without `as`, the exact approved Plan source expands at the authored site once +the authorship frame has been dismantled and the bytes have been structurally +admitted; written with `as`, that same source is bound and none of it runs. Its +four private capabilities — ``, ``, `` +and `` — are the closure those exact bytes carry, and are syntax no document may write. [The plan command](./plan-command-spec.md) is the contract. Two registrations for one name and kind at the same scope are a configuration @@ -3098,9 +3122,12 @@ own bytes. What inspection does not do is build anything from them. Each declared Markdown name contributes one complete entry under built-in, reporting `declared-markdown` as its source kind and the declared origin and -digest as its origin. Its private closure contributes nothing: those names are -not syntax a document may write, so listing them would describe an environment -that does not exist. +digest as its origin, and carrying the return disposition its host stated. That +last is part of the contract an author is reading rather than a note about it — +it says whether writing the component without `as` expands what it returns — so +the catalog reports it and `xmd syntax` renders it in both formats. Its private +closure contributes nothing: those names are not syntax a document may write, so +listing them would describe an environment that does not exist. **Inspection is observation, never authority.** Building a catalog installs only the declarative registration layer selection needs. It enters no execution, @@ -3215,7 +3242,11 @@ author reading a result is owed all of them at once. **A declared Markdown component is checked as the contract it declares.** An invocation of one is held to the props, forms and return mode parsed from the declared bytes, exactly as a registration's invocation is held to its -declaration, and its origin is recorded on the invocation record. Its body is +declaration, and its origin is recorded on the invocation record. Its return +disposition is part of that contract: a declaration whose host said its return is +executable source validates without `as`, because that is what running it does +(§6.10). The source it would produce does not exist yet and is not validated +here; what runs it admits it structurally first. Its body is not walked: those bytes are the host's, an author cannot change them, and the private names only they may write resolve nowhere else — so reporting on them would report a document's author for the engine's own asset. A private name @@ -3641,6 +3672,60 @@ schema, an invalid value, a body error, and a failure raised after `` all complete `Err`, and body text emitted before the failure remains only on the output stream. +#### Embedded text roots + +A component whose host declared its return to be executable source (§5.3) +produces a third kind of root: source the engine expands **where the component +was written**, inside the execution that wrote it. `` invoked without +`as` is the one the `run` profile has. + +The projection happens after the component's invocation and its teardown have +completed, so the source runs under the authority the site already had rather +than under anything the component installed for its own body. Nothing about it +is a second execution: it calls no `execute()`, imports no `__root__`, produces +no root result, installs no host profile, and writes to no output channel of its +own. + +**It is a root in exactly three respects**, and each is decided before its first +effect: + +- **Its own root contract.** A root `returns` declaration is refused — source + expanded where it was written has nowhere to hand a value back to. A declared + root props schema validates the properties ambient at the site, which is what + a caller can actually offer. +- **Its own frontmatter metadata**, which is what `{meta.key}` reads inside it. +- **Its own top-level `` selection** (§6.9), so what reaches the + surrounding document is what the program selected rather than everything it + rendered. + +**Everything else is the enclosing expansion's.** The component selection and +import authority, the Workspace, the error mode, the checked-failure ledger, the +block counter, the output owner, the cancellation scope and the journal are the +ones already in force. Bindings the document made are readable inside the source +and the validated root props are what it reads under `props`; what the source +binds stays inside it. The authority is the authored site's rather than the +declaration's, so the private closure only the declared bytes may write is +unreachable from the program they produced, and the source expands under the +producing component's own hide set, so it cannot re-enter that component. + +**It is its own flow, too.** A `` written in the source belongs to a +`` written in the source, exactly as a component's own body's does (§6.10), +so a stray one is reported there rather than ending the loop the element was +written in. A `` is reserved for the same reason: an embedded root owns +no return frame at all. + +**Its identities extend the authored element's.** Every effect the source +performs is durable beneath the site the component was written at, so two sites +running the same program are two sets of effects and a replay re-enters the same +expansion rather than deriving a new one. A partial journal therefore restores +whatever produced the source — for ``, the authorship and the admission — +re-enters the source at the same path, restores the effects that had completed, +and continues without repeating one. + +The source is never emitted in place of what it does: an invocation that expands +its return renders that program's selected output, and an invocation that +captures it with `as` renders nothing at all. + #### Document targets A root document addresses its own sections. A **document target** is an @@ -6511,6 +6596,17 @@ value. Absence of the declaration is what selects text mode, so an explicit `returns: { type: string }` is a value component that happens to return a string. +**One exception, and only a trusted host can make it.** Exact declared Markdown +whose host stated an executable-source return disposition (§5.3) may be invoked +without `as`: what it returns is a program rather than a value, and the engine +expands it at the authored site as an embedded text root (§5.4). Written with +`as`, it binds the exact source and expands none of it, so adding `as` is what +prevents every effect of that program. Expansion and document validation ask one +question about this, so a site that needs no capture when it runs needs none when +it is validated. Nothing a document, a repository file, a workflow bundle, a +registration or a function component can write reaches the exception: an +ordinary value component still requires `as`, string-valued ones included. + #### Declaring a return value `returns` is an object: either a draft-07 JSON Schema, marked by `type` or @@ -11917,6 +12013,30 @@ user's own `~/.xmd/repositories`. | ORC20 | Retained workflow regression | Repository and Worktree replay, transactional Git, Push and pull-request history evidence, Issue effects, forks and completed replay keep their records, identities, provider call counts and native-launch refusal unchanged; directory ensure adds only its own `workspace_file` event and resulting retained root | | ORC21 | Compiled binary | A compiled smoke creates a root-level ambient Worktree, runs a command there, proves `.git` is a file and the checkout persists after exit; a second gated process proves lock refusal and release | +### Tier ES — Executable source returns (§5.3, §5.4, §6.10) + +A declared component stands where `` stands and returns the caller's own +text, so every case is about what the engine does with approved bytes rather +than about how they were approved. The program's one effect is a component that +names durable work, so "it did not run" is proven by a recorder that stayed +empty rather than by absent output. + +| # | Test | Verify | +|---|------|--------| +| ES1 | Immediate expansion | An invocation without `as` performs the program's effect exactly once, between the markers surrounding the element, and emits the program's output rather than its source | +| ES2 | The document's own environment | One program uses ambient props through its root schema, its own frontmatter metadata, a caller binding, a component on the caller's include path, and top-level `` | +| ES3 | Capture | The same source under `as` binds byte for byte while the effect in it does not happen | +| ES4 | No return, no expansion | A component that fails before returning expands nothing and binds nothing | +| ES5 | Root preflight | A root `returns` declaration and a root-props mismatch each refuse before a negative-control effect | +| ES6 | Replay | A partial journal resumes inside the source, does not repeat the completed effect, and produces the same output | +| ES6b | Its own flow | A stray `` in the source is reported there rather than ending the loop the element was written in | +| ES7 | One lifecycle | The journal records exactly one root import | +| ES8 | Ordinary value components | A declared string-valued component without the disposition still requires `as` | +| ES9 | No document-authored opt-in | A repository component's similarly named frontmatter grants nothing and still requires `as` | +| ES10 | Disposition is part of the selection | A continuation that drops the disposition, and one that adds it, each refuse | +| ES11 | Admitted against the source | A declaration whose Markdown returns no string cannot carry the disposition | +| ES12 | One described contract | Inspection and the catalog report the disposition; validation accepts an uncaptured site and still refuses one without it | + --- diff --git a/specs/plan-command-spec.md b/specs/plan-command-spec.md index 8f784f9e..f8daf1fe 100644 --- a/specs/plan-command-spec.md +++ b/specs/plan-command-spec.md @@ -266,6 +266,19 @@ document invoking `` expand the same bytes, under the same origin `@executablemd/cli/Plan.md` and the same digest, in every distribution. There is no generated TypeScript copy and no second Markdown implementation. +**What the Component returns is a program, and the engine expands it.** The host +declares the executable-source return disposition on those exact bytes under the +identity `` ([executable MDX](./executable-mdx-spec.md) §5.3), so an +ordinary document writing `…` carries the approved program out +where it wrote the element, as an embedded text root (§5.4). Writing +`` binds the exact approved source instead and performs none +of its effects. Authorship and review are about the program's contents and do +not change between the two. + +`plan-command.md` writes `as`, which is what keeps `xmd plan` source-only: the +command hands the caller bytes, and `--run` is still the one thing that executes +them, through the independent path below. + The Component contains one enclosing `` expansion whose body holds every initial, repair and human-revision ``. A loop inside that one Session creates no second placement; sibling Sessions are not used, because sibling diff --git a/specs/release-process-spec.md b/specs/release-process-spec.md index 48e8a58e..e4d1fb47 100644 --- a/specs/release-process-spec.md +++ b/specs/release-process-spec.md @@ -627,7 +627,12 @@ The checks that hold this together, each proving a different build: asks the built bin — from a directory that is not the package — which Plan `` Component source it would let a document write. The answer carries the origin and the SHA-256 of those bytes, so a build that shipped different ones, or none, - answers differently here rather than at a person's first `xmd plan`. + answers differently here rather than at a person's first `xmd plan`. It carries + the return disposition too: what `` returns is executable source the + engine expands, and that travels with the declaration rather than with the + bytes, so the digest cannot answer for it. A build that shipped the Component + without it would let a document write `` and then refuse it for having + captured nothing. - `scripts/tests/plan-component-compiled.test.ts` asks the same question of the compiled binary, which has no checkout to fall back to. It runs in the `smoke` job, beside the other suites whose subject is `dist/xmd`. From 843f5cc5d09544157e6a213f4272218f117847c5 Mon Sep 17 00:00:00 2001 From: Taras Mankovski <74687+taras@users.noreply.github.com> Date: Tue, 1 Sep 2026 21:18:50 -0400 Subject: [PATCH 2/2] =?UTF-8?q?=E2=9C=85=20Close=20two=20gaps=20in=20the?= =?UTF-8?q?=20unsuccessful-ending=20evidence=20(#711)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PC22's "repeated rejection" row proved only that a Stop ends the run, which the row above it already proved. It is replaced by PC22b: every draft and repair answers with the same candidate, the host finds each unsound, nine rounds request changes and the tenth takes the exhaustion ending. The case asserts the exhaustion sentence exactly — stopping a round earlier produces "stopped at your request" instead — together with ten reviews, forty checks, and a candidate that carried a program effect through all of it and never performed it. ES4 asserted no expansion but never looked at the binding. It now invokes the refusing component with `as` inside ``, so the document survives the refusal and can be asked what it holds, and reads the capture back through `typeof`. A positive control against a declaration that does return proves the probe answers both ways. The authored-refusal half is kept: that failure is the run's own outcome, and it expands nothing either. Claude-Session: https://claude.ai/code/session_01TNJwcFmnt3kYSn9gGsx9u7 --- packages/cli/tests/plan-component.test.ts | 40 ++++++++-- packages/core/tests/executable-source.test.ts | 76 ++++++++++++++++++- specs/executable-mdx-spec.md | 2 +- 3 files changed, 108 insertions(+), 10 deletions(-) diff --git a/packages/cli/tests/plan-component.test.ts b/packages/cli/tests/plan-component.test.ts index 427f6711..83ad2a22 100644 --- a/packages/cli/tests/plan-component.test.ts +++ b/packages/cli/tests/plan-component.test.ts @@ -696,11 +696,6 @@ describe("Tier PC — in an ordinary document", () => { expect: string; }[] = [ { name: "stopping", reviews: ["Stop"], expect: "stopped at your request" }, - { - name: "repeated rejection", - reviews: ["Request changes", "Request changes", "Stop"], - expect: "stopped at your request", - }, { name: "a failed turn", reviews: [], @@ -750,6 +745,41 @@ describe("Tier PC — in an ordinary document", () => { } }); + it("PC22b: ten drafts nobody could approve end in exhaustion, having run nothing", function* () { + yield* useWorkingDirectory(function* (dir) { + const log = join(dir, "log.txt"); + // Every draft and every repair attempt answers with the same candidate, + // and the host finds each of them unsound. That is what drives the + // Component through its three repairs a round and its ten rounds, and + // what makes the tenth round the one with nothing left to revise into. + const candidate = programAppending(log); + const run = yield* runDocument({ + source: ["Write a program.", "", "after", ""].join("\n"), + turns: Array.from({ length: 60 }, () => ({ reply: candidate })), + // deno-lint-ignore require-yield + assess: function* () { + return { valid: false, diagnostics: { problem: "the draft is unsound" } }; + }, + reviews: [...Array.from({ length: 9 }, () => "Request changes" as const), "Stop" as const], + }); + + // The exhaustion ending, not the ordinary one a person choosing Stop + // earlier would have reached. + expect(run.failure).toBe( + "Plan authorship reviewed ten drafts without an approved Plan. No Plan was returned.", + ); + // Ten drafts is what the workflow reviewed, and each was checked once and + // then repaired three times. + expect(run.harness.reviews).toHaveLength(10); + expect(run.harness.checked).toHaveLength(40); + // The candidate carried a program effect through every round of this and + // never performed it. + expect(yield* timesRun(log)).toBe(0); + expect(run.output).not.toContain("the program ran."); + expect(run.leftover).toEqual([]); + }); + }); + it("PC23: cancelling authorship leaves no program effect", function* () { yield* useWorkingDirectory(function* (dir) { const log = join(dir, "log.txt"); diff --git a/packages/core/tests/executable-source.test.ts b/packages/core/tests/executable-source.test.ts index f33f2b0b..6081fa2b 100644 --- a/packages/core/tests/executable-source.test.ts +++ b/packages/core/tests/executable-source.test.ts @@ -336,9 +336,78 @@ describe("Tier ES — executable source returns", () => { }); it("ES4: a return that never arrives expands nothing and binds nothing", function* () { + const source = ["# The program", "", "the program ran.", "", '', ""].join( + "\n", + ); + + // The declaration ends without selecting a value, which is what every + // unsuccessful authorship ending is from the engine's side: the workflow + // finished and there is no approved source to hand back. + const unresolved = [ + "---", + "props:", + " type: object", + " properties:", + " source: { type: string }", + " required: [source]", + " additionalProperties: false", + "returns:", + " type: string", + "---", + "", + "the workflow approved nothing, so it selects no value.", + "", + ].join("\n"); + + // Written with `as`, and inside `` so the refusal is printed + // rather than ending the run: what this case is about is what the document + // holds *afterwards*, and a run that stopped at the element could not be + // asked. + const asking = [ + ``, + "", + '', + "", + "", + '', + "the capture is unset.", + "", + "the capture holds {approved}", + "", + "", + "", + ].join("\n"); + + // The same document against a declaration that does return, so the question + // the `` asks is one this document can answer either way. + const control: string[] = []; + const answered = yield* runDocument({ + source: asking, + declarations: [expanding()], + performed: control, + }); + expect(answered.output).toContain("the capture holds # The program"); + expect(control).toEqual([]); + + const captured: string[] = []; + const capturing = yield* runDocument({ + source: asking, + declarations: [expanding({ source: unresolved, digest: sourceDigest(unresolved) })], + performed: captured, + }); + + // The refusal was reported where the element was written. + expect(capturing.output).toContain("no "); + // And the binding that site named was never written — not to the source, + // and not to anything else. + expect(capturing.output).toContain("the capture is unset."); + expect(capturing.output).not.toContain("the capture holds"); + expect(captured).toEqual([]); + expect(capturing.output).not.toContain("the program ran."); + + // The same refusal at a site that captures nothing: an authored ending this + // time, which the run settles on rather than recovering. const performed: string[] = []; - // The declaration fails before its ``, which is what every - // unsuccessful authorship ending is from the engine's side. const refusing = [ "---", "props:", @@ -357,7 +426,6 @@ describe("Tier ES — executable source returns", () => { "", ].join("\n"); - const source = ["# The program", "", '', ""].join("\n"); const run = yield* runDocument({ source: invoking(source), declarations: [expanding({ source: refusing, digest: sourceDigest(refusing) })], @@ -366,7 +434,7 @@ describe("Tier ES — executable source returns", () => { expect(run.failure).toContain("nothing was approved"); expect(performed).toEqual([]); - expect(run.output).not.toContain("# The program"); + expect(run.output).not.toContain("the program ran."); }); it("ES5: a root `returns` and a root-props mismatch each refuse before an effect", function* () { diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index f8f71b04..b7f4a46e 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -12026,7 +12026,7 @@ empty rather than by absent output. | ES1 | Immediate expansion | An invocation without `as` performs the program's effect exactly once, between the markers surrounding the element, and emits the program's output rather than its source | | ES2 | The document's own environment | One program uses ambient props through its root schema, its own frontmatter metadata, a caller binding, a component on the caller's include path, and top-level `` | | ES3 | Capture | The same source under `as` binds byte for byte while the effect in it does not happen | -| ES4 | No return, no expansion | A component that fails before returning expands nothing and binds nothing | +| ES4 | No return, no expansion | A component that selects no value expands nothing, and the capture its site named is still unset afterwards; an authored refusal is the run's own outcome and expands nothing either | | ES5 | Root preflight | A root `returns` declaration and a root-props mismatch each refuse before a negative-control effect | | ES6 | Replay | A partial journal resumes inside the source, does not repeat the completed effect, and produces the same output | | ES6b | Its own flow | A stray `` in the source is reported there rather than ending the loop the element was written in |