diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7dcb2943..f427ddb0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -222,6 +222,13 @@ jobs: - name: The Component the compiled binary carries run: deno test --allow-all --frozen scripts/tests/plan-component-compiled.test.ts + # The ordinary repository provider is assembled at a runtime-named + # entrypoint and holds managed checkouts with a kernel-released advisory + # lock, so only the binary shows both surviving `deno compile`. The script + # runs two of them at once against a managed root of its own. + - name: Smoke test repository composition with the compiled binary + run: deno run --allow-all --frozen scripts/smoke-run-composition.ts + # `` resolves from core's registry, requests through the contextual # Fetch adapter, and detaches the response before binding it. All three # live in the module graph, so only the binary shows they survived diff --git a/.github/workflows/publish-packages.yml b/.github/workflows/publish-packages.yml index c5297191..9f56a1e9 100644 --- a/.github/workflows/publish-packages.yml +++ b/.github/workflows/publish-packages.yml @@ -90,7 +90,7 @@ jobs: version: ${{ needs.version.outputs.value }} testing: - needs: [version, core, durable-streams] + needs: [version, core, durable-streams, runtime] uses: ./.github/workflows/publish-one.yml with: package: packages/testing diff --git a/architecture.md b/architecture.md index 1609bc6b..b3d26989 100644 --- a/architecture.md +++ b/architecture.md @@ -34,8 +34,8 @@ Existing documents and code get aligned to this section retroactively. | definition base | the Git revision supplied to choose a workflow definition's pinned commit | | Repository base | the optional Git revision from which one named Repository initializes its primary checkout | | Repository selection | plain structural composition data naming the repository one component invocation acts on: an opaque provider-minted selection identifier, the display name, the credential-free repository identity, and the selected checkout path. It carries no credential, provider handle, lock, database, run ID or authority — the installed provider authenticates every selection against private state before it touches Git or a service, so a copied, replaced or rebuilt one can misname a target and be refused but can never reach one | -| ambient Repository | the repository an ordinary document execution was started inside, discovered once before root expansion from the invocation's starting directory. Its identity is the canonical common Git directory and its selected checkout is the canonical checkout root, so starting in a linked worktree names the same repository as starting in the primary checkout while Git operations still act on the worktree. A workflow run has none | -| managed checkout | a Repository or Worktree an ordinary document execution created under the host root `~/.xmd/repositories`, addressed by a digest of its whole identity, described by a closed version 1 sidecar written beside it, and held for one document execution by an exclusive non-blocking advisory lock. It survives every execution: nothing deletes, resets, cleans, fetches or repairs one | +| ambient Repository | the repository an ordinary `xmd run` was started inside, discovered once before root expansion from the invocation's starting directory. Its identity is the canonical common Git directory and its selected checkout is the canonical checkout root, so starting in a linked worktree names the same repository as starting in the primary checkout while Git operations still act on the worktree. A workflow run has none | +| managed checkout | a Repository or Worktree an ordinary `xmd run` created under the host root `~/.xmd/repositories`, addressed by a digest of its whole identity, described by a closed version 1 sidecar written beside it, and held for one document execution by an exclusive non-blocking advisory lock. It survives every execution: nothing deletes, resets, cleans, fetches or repairs one | | ordinary Git identity | the invoking user's effective Git author and committer name and email, captured once from the trusted host's own environment and configuration before a document expands and kept in the provider's closure. It is used for an ordinary `` and nothing else, because that commit lands in the caller's own checkout; a workflow run keeps its one fixed identity, whose whole purpose is that retained Git state does not depend on whose machine made it. It is not a prop, a Context value, a component result or a middleware answer, and nothing else about the environment is borrowed with it — hooks, file-system monitors, signing programs and repository-supplied credential helpers stay disabled by the same fixed command-line configuration. A host where Git can name no identity refuses `` with an actionable sentence rather than substituting the workflow one; every other component is unaffected | | ordinary invocation identity | a fresh opaque random value an ordinary document execution's repository provider mints for itself and keeps in its own closure. It is not a prop, a Context value, a component result, a middleware answer, a lifecycle ID or a retained record, and it is neither addressable nor reusable; live Issue and pull-request idempotency and reconciliation keys are derived from it together with the engine's own expansion identity | | pinned commit | the commit obtained by resolving a base once; it remains the workflow run's starting repository state even as the run creates descendant commits | @@ -1692,19 +1692,28 @@ an admitted `File:read` does not become a write and an admitted `File:write` does not become a read, however many times something is asked. The standard Deno workflow profile's write table is three entries, in the order -it states them: core's paired `File:write`, the workflow package's lexical `Dir` -— built from the same implementation and schema the ordinary registration owns -so the two cannot drift — and core's self-closing -`@executablemd/core#File.Delete`. A host's own extensions come after them. An -admitted mutation is the ordinary component: `` and `` each -cross `API.Files` and therefore the run's transaction-bound provider and its one -effect transaction, so a removal is retained by the same `workspace_file` effect -an authored deletion publishes, carrying `{ kind: "deleted" }`. The evaluator -adds no mutation API, transaction or receipt of its own, and a deletion has no -outcome for one to carry: a write-enabled fragment that only deletes still binds -`{ observations: [], output: "" }`. Local Git, Git-host, issue, process, eval and -exec, native-command, credential and external-write effects are outside the -class. +it states them: core's paired `File:write`, the workflow package's paired `Dir` +under the versioned origin +`@executablemd/workflow/composition/dir-v2#Dir` — built from the same +implementation and schema the ordinary registration owns so the two cannot +drift — and core's self-closing `@executablemd/core#File.Delete`. A host's own +extensions come after them. Selecting `write` intentionally authorizes Dir's +persistent recursive directory creation as well as the mutations File and +File.Delete perform. Each is the ordinary component and crosses `API.Files` +into the run's transaction-bound provider, so an ensured directory is retained +by the same `workspace_file` effect an authored Dir publishes. The evaluator +adds no mutation API, transaction or receipt of its own, and a mutation has no +outcome for one to carry: a write-enabled fragment that only ensures a directory +still binds `{ observations: [], output: "" }`. Local Git, Git-host, issue, +process, eval and exec, native-command, credential and external-write effects +are outside the class. + +The versioned Dir origin is a grant boundary, not a catalog decoration. The +former `@executablemd/workflow/composition#Dir` identity authorized lexical +placement that created nothing; it does not authorize the persistent mutation. +A continuation whose retained write table names that former identity is refused +before generated execution instead of inheriting a broader grant. Read-only +admissions never selected the write table and remain unaffected. That order is retained identity rather than presentation. A continuation is compared against the entries position by position, so adding the deletion @@ -2239,14 +2248,33 @@ answers a document's file operations; what a provider is allowed to do is decided by the provider, from identities the contextual layer cannot supply. The Api's operations are whole semantic acts — read this path, replace this -path, remove this path, list what these patterns select — rather than steps a -caller sequences. +path, ensure this directory exists, remove this path, list what these patterns +select — rather than steps a caller sequences. Every operation is mandatory on +the host and workflow providers; the terminal for an absent provider remains +fail-closed, and separately loaded copies compose through the Api's stable +namespaced name. The one preliminary operation, `checkFilePath`, is deliberately weak: pure path arithmetic, no filesystem access, and it returns nothing usable. `` calls it to decide whether a write's children may expand, and the later write repeats the same admission from the same authored path. Nothing is handed between them, so a check that was skipped or answered elsewhere authorizes nothing. +Directory placement is one semantic act of its own. `` gives +`ensureDirectory` the enclosing working directory and the path the author +selected; a relative path is resolved against that directory and an absolute +path keeps the component's existing meaning and is used as written. The +provider recursively creates a missing target, accepts an existing directory +without replacing or clearing it, and refuses a final or intermediate +non-directory with structural failure data that carries no resolved path or +platform diagnostic. Success carries Unit rather than a path or handle. + +`` performs that ensure before it installs the target as `Env.cwd` and +before its content begins. A refusal therefore runs no content. Once the ensure +succeeds, content expands with the target as cwd and the component's scope +restores the enclosing cwd on success, failure and cancellation; restoration +never deletes the directory. The Repository selection in scope is unchanged — +Dir places content in a directory and selects no Repository. + Removal is one of those acts, and it is mandatory: a provider does not omit it and let a document reach the host instead. Its success is Effection's `Unit`, carrying no value at all, which is what makes "a deletion hands back no receipt" @@ -2267,39 +2295,49 @@ parent, which is otherwise outside it. The two providers make different containment claims, and both are stated rather than implied: -- **`xmd run`** resolves document paths in the caller's own filesystem. It - refuses empty, absolute, and lexically escaping paths without touching the - filesystem, and refuses an observed outward symlink once resolution can see - one. That holds while the host namespace is stable; it is not a sandbox, and - closing the replacement window would require a native dependency this project - does not take. +- **`xmd run`** resolves document paths in the caller's own filesystem. The + ordinary file operations refuse empty, absolute, and lexically escaping paths + without touching the filesystem, and refuse an observed outward symlink once + resolution can see one. Directory ensure is the placement exception above: + it accepts the exact absolute target `` already accepts and resolves a + relative target from the enclosing cwd. It creates directly and enlists in no + transaction, so directories it created persist after the component and the + document end — including when later content fails or is cancelled — and a + recursive attempt is never rolled back or removed by teardown. These claims + hold while the host namespace is stable; this is not a sandbox, and closing + the replacement window would require a native dependency this project does + not take. - **A workflow run** resolves document paths in the run-owned Workspace's logical filesystem, inside the caller-owned transaction. A document path - never becomes a host path, so there is no host namespace to replace. + never becomes a host path, so there is no host namespace to replace. An + absolute Dir target is an absolute logical Workspace path, never a host path. Neither claim covers a native command a document runs. The workflow provider's operations are durable effects. A read, a write, a -deletion and a search each carry a description derived from the current -expansion, the operation and the resolved logical path, so one authored element -is the same effect across replays and a document edited to name another file is -a different one. `checkFilePath` is not among them: it is lexical admission, it performs no -effect, and it appends nothing. +directory ensure, a deletion and a search each carry a description derived from +the current expansion, the operation and the resolved logical path, so one +authored element is the same effect across replays and a document edited to name +another target is a different one. `checkFilePath` is not among them: it is +lexical admission, it performs no effect, and it appends nothing. A mutation, the immutable root that results from it, and the filtered journal -result share the one caller-owned transaction. An ordinary refusal rolls its -mutation savepoint back before that result is published, so the retained -outcome describes a Workspace that is exactly what it was, and the reason that -crosses the boundary is selected from the shared vocabulary rather than derived -from anything the filesystem said. Replay restores those recorded outcomes -without performing the mutation, opening a transaction or consulting the current +result share the one caller-owned transaction. Recursive directory creation is +inside one savepoint and one `workspace_file` effect: successful creation or +existing-directory adoption publishes the resulting root with the recorded +success, while a documented refusal rolls every directory created by the +attempt back and publishes the refusal against the unchanged root. An ordinary +refusal from any mutation follows that rule, so the retained outcome describes +a Workspace that is exactly what it was, and the reason that crosses the +boundary is selected from the shared vocabulary rather than derived from +anything the filesystem said. Replay restores those recorded outcomes without +performing the mutation, opening a transaction or consulting the current frontier, which is why a create/delete/create history replays in order — and why -a completed deletion does not remove a second time even where its path exists -again. A cancellation between a removal and its commit publishes nothing, so the -continuation performs that deletion once, against the retained frontier. A -temporary directory is refused outright: the provider has no host directory to -hand out, and falling through to the caller's would be the uncontained -filesystem the boundary exists to prevent. +a completed ensure does not create again. A cancellation between a mutation and +its commit publishes nothing, so continuation performs it once against the +retained frontier. A temporary directory is refused outright: the provider has +no host directory to hand out, and falling through to the caller's would be the +uncontained filesystem the boundary exists to prevent. Failure data crosses the boundary as a plain frozen object under a stable tag, carrying a reason from a fixed vocabulary and the phase it came from. No @@ -2387,13 +2425,13 @@ on that value. A Repository selection is composition data and is therefore replaceable: a document may bind one, render one, hand one to a child, and construct one that looks exactly like it. Nothing a repository provider does is authorized by the -value it was handed. What stays provider-owned, in the provider's own closure, -is everything a selection is *not*: the advisory locks on managed checkouts, -the canonical Git identity each selection resolves to, an ordinary run's live -Push evidence, its invocation identity, and every reconciliation key derived -from them. A selection that the provider did not mint, or one whose -name, checkout path or identity was edited after it did, is refused before Git -or a service is touched. +value it was handed. What stays provider-owned, in the provider's own closure +for one document execution, is everything a selection is *not*: the advisory +locks on managed checkouts, the canonical Git identity each selection resolves +to, an ordinary run's live Push evidence, its invocation identity, and every +reconciliation key derived from them. A selection that the provider did not +mint, or one whose name, checkout path or identity was edited after it did, is +refused before Git or a service is touched. ### Definition-owned return state @@ -3766,21 +3804,28 @@ Status is measured against main. | `Config` run deadline / exec default / Fetch default / verbosity | three independently owned contextual timeouts, absent unless configured, each read by exactly one consumer, and contextual verbosity — a boolean that is false unless configured, seeded by the command line and overridable for a lexical subtree, bounding nothing and owning no authority | built on this stack | | native session launch (`` / `launchAgentSession()`) | prepares one durable coding-agent session from the rendered body of `` and hands the provider's native UI the terminal for that exact session, then continues the document after it exits. The body renders completely first and only what it rendered crosses as the instruction layer; the launch performs no model turn; the run's one foreground-terminal lease is taken before an agent is resolved, so a host with no terminal refuses without probing for an installed CLI. A session is constructed once, by one of two mechanisms, and its create-once construction route says which. Where the provider returns the identity, the ACPX provider creates the session, installs the layer at creation, releases ACP ownership before the spawn, and marks its handle stale so a later `` reattaches. Where the adapter names its own sessions, it allocates the identity inside ownership before any process exists, the native process creates the session under that name from a private mode-0600 instruction file, and ACP creates nothing — the instruction text reaches neither argv nor environment, and the file is removed on success, failure and cancellation alike while ownership is still held. Neither route converts into the other, and which one governs is chosen by the first operation that consumes the placement rather than by the `` that made it: a fresh `` publishes no route and establishes nothing, so a `` nested inside one constructs the session it placed, while a first subscribed `` publishes ACP-first before it ensures and keeps that account even if the turn that follows is never accepted. An established route is validated eagerly by a later ``, and a launch meeting a published ACP-first route refuses before an identity exists. A `` or `` meeting a bound client-allocated route attaches under the route's exact identity; a legacy unbound route or an unavailable attachment capability refuses before a turn and creates no substitute conversation. Phases are retained as `agent_session_launch` records under one expansion identity — `prepared` before ownership is released, then `detached`, then `exited` — so a completed replay launches nothing, a replay holding only `prepared` proves the handoff never began and may still create under the retained identity, and one holding `detached` resumes and never falls back. The public route carries an opaque one-use launch request and answers nothing; authority to run and retain a phase is delivered to the installed provider directly, so neither a returned completion nor a rebuilt request authors a launch. Every operation that can act on an advertised session takes exclusive ownership under one natural key first, through a coordinator the host built and passed in; contention refuses instead of queueing, and an owner that never proved it stopped leaves a recovery tombstone. A host that cannot say who owns a session refuses every advertised operation, and one that cannot say how a session was constructed additionally refuses an agent that names its own — before any provider effect. Every private setup or child-creation failure is normalized to `process-creation-failed` with fixed provider-owned text, carrying no path, argv, environment or host message. No launch path discards persistent provider state. A client-allocated session is bound to one executable build: the build is observed inside ownership before an identity is allocated, the binding is published with the V2 route and retained beside the prepared record, the native child runs the exact observed path in place of the launcher name, and every later create, resume, attachment and incomplete replay reobserves and compares before a process, an ensure or a turn. A `` or `` meeting a bound client-native route attaches to it: it reobserves the build, requires any retained provider arrangement to assert that same conversation, calls ensure with the route identity as `resumeSessionId`, and requires the provider to report that identity before a turn — refusing on missing capability, build drift, missing history or a differing assertion without creating a substitute conversation. ACP runtimes are partitioned by resolved agent command and binding, each handle is closed by the partition that created it, and a bound partition is torn down when its last handle closes. A legacy V1 client-native route keeps exactly the released native-only behavior and never attaches | built on the #517 stack, extended by the #519 and #561 stacks; Deno and the compiled binary assemble the host — coordinator, route store and executable observer — and Node and Bun keep the same advertised names while assembling none of it, so every advertised operation refuses before provider work; `claude` is advertised for native launch after passing the client-allocated gate at Claude Code 2.1.241 on macOS arm64 (#520) and separately for client-native attachment after passing the native-to-ACP marker gate (#561), and Codex remains unadvertised because nothing has run its provider-returned claims against an installed Codex; `Agent.AddDir` is unbuilt | | `` | performs one XMD-mediated HTTP read through contextual `API.Fetch`, admitting the whole request before transport, and retains the normalized request and the detached response as one `fetch` durable observation; capture decides whether a status is data or a failure, and the trusted host's destination ceiling sits below the component | built on the #456 stack; a generated fragment may name the pinned identity only for a request the trusted host stated exactly, on the #369 stack | -| `API.Files` | routes every document filesystem operation to the installed provider, with no host default and structural failure data | built on the #227 stack | +| `API.Files` | routes every document filesystem operation to the installed provider, with no host default and structural failure data. Its mandatory semantic operations include `ensureDirectory`, which recursively creates or adopts one directory and returns Unit; separately loaded copies compose through the stable Api name | built on the #227 stack; directory ensure added by #643 | | `` | removes one file the document names, inside the contextual working directory. An ordinary overridable core default with a closed schema of one required non-empty `path`, **self-closing only** — a paired spelling never enters its body, because the component declares its one form and canonical invocation-form dispatch enters that body only for the form the scan recorded, before `Env.cwd` is read and before the provider is reached. Neither the composable `Component.hasContent()` chain nor a method on whatever object a caller handed over takes part. It renders the empty string, declares no `returns` and hands back no receipt, so an ordinary `as` captures that empty string; absence is the same success, so deleting a path twice succeeds twice. One regular file or one final symbolic link goes — the link rather than its target, inside or outside — and every directory is refused, an empty one included. Empty, absolute, lexically escaping and parent-link-escaping paths are refused before any removal, and a printed error names only the path the document wrote. One semantic `API.Files.deleteFile` call and no filesystem access of its own; under a workflow run it is one `workspace_file` effect retaining `{ kind: "deleted" }`. The standard Deno workflow profile admits it to generated XMD as the exact self-closing identity `@executablemd/core#File.Delete`, third in the write table, where it performs that same ordinary effect and contributes no evaluator result | built on the #567 stack | -| host Files provider / `useHostFiles()` | resolves document paths in the caller's filesystem, containing them while the host namespace is stable; installed by all four CLI entrypoints | built on the #227 stack | -| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction | built on the #366 stack | +| host Files provider / `useHostFiles()` | resolves document paths in the caller's filesystem, containing them while the host namespace is stable; its directory ensure accepts Dir's existing absolute target or resolves a relative one against the enclosing cwd, recursively creates or adopts it directly, preserves existing contents, refuses a non-directory before content and never rolls back or deletes created directories; installed by all four CLI entrypoints, so Dir remains operational under Node and Bun | built on the #227 stack; directory ensure added by #643 | +| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction; directory ensure is one durable `workspace_file` mutation whose recursive creation, resulting root and recorded success commit together, whose documented refusal rolls its savepoint back, and whose replay restores the record without creating again | built on the #366 stack; directory ensure added by #643 | | `service=` | publishes the attachment's endpoint into the live binding overlay for its invocation | built on main | | `ephemeral eval` | reconstructs live middleware and bindings without a journal entry | built on main | | `useWorkflowServiceDenial()` | provides a non-delegating workflow service denial provider, installed inside every start and resume execution scope | built on the #366 stack | | `xmd workflow start` / `xmd workflow resume` | starts or resumes a workflow run from the CLI, under the Deno entrypoints only | built on the #366 stack; both acquire #367's executor lock before any lifecycle transition | | implicit workflow Workspace | retains provider-neutral filesystem, repository and attachment state by run ID | document filesystem built on the #366 stack and Repository/Worktree composition on the #293 stack; process capabilities unbuilt (#218) | -| repository composition vocabulary | one array of thirteen ordinary, shadowable registrations — `Repository`, `Worktree`, `Dir`, the four `Git.*` operations, `PullRequest` and its three evidence reads, `IssueTracker` and `Issue` — declared once and consumed by the workflow attachment, so one vocabulary is described and resolved wherever it is installed. Registering it installs no provider, discovers no repository, acquires no lock, spawns no Git and reads no credential; what a name does is the installed provider's. A repository-local Markdown or TypeScript component of the same name is chosen ahead of any of them | built on the #643 stack | -| `` / `` / `` composition | names a Git repository and its linked checkouts inside the run-owned Workspace, installs each as contextual working directory, and retains creation identity beside the retained Git bytes | built on the #293 stack, Deno provider only | -| transactional Git effects (`Git.Switch` / `Git.Add` / `Git.Commit`) | publish local Git mutations with their journal result; the enclosing Repository and the contextual working directory select which retained checkout one runs in, and neither observation carries authority — the observed record is compared with the retained row and the directory with the checkouts that row holds, so a failure of authority, of retained state or of an unrecognized native condition fails the run instead of publishing a result | built on the #294 stack, Deno provider only | -| `Git.Push` | publishes the selected checkout's exact current named branch and commit to the same branch on the retained Repository's canonical `origin`, reconciled through the shared Git-host state machine rather than through a Workspace transaction: no props and no component result, no force, no upstream mutation and no implicit staging or committing; the durable request and record carry the Repository's filtered identity without its checkout path, and the transport runs in a provider-owned isolated control repository reading the checkout's objects through an object-source attachment whose alternates chain and object tree are proven contained before the first remote observation, aimed at the exact private retained locator. A destination proven absent is published to once and one already naming this exact commit is adopted; one naming a distinct commit that same authenticated source proves is in this commit's ancestry is a performable pre-state, published over by the same exact non-force refspec and retained as the predecessor with the attested relation, while a divergent commit and one the source cannot read are both conflicts and nothing is fetched to decide either; a completed Push is reconstructed from the Workspace root its own journal event was appended against, read without publishing it or moving the run's frontier, so a branch published more than once resumes | built on the #370 stack, Deno provider only | -| `` | upserts one pull request of the selected checkout's current named branch, reconciled through the shared Git-host state machine: a required `title`, an optional positive-integer `number`, an optional `base` defaulting to the Repository's retained initial branch, an optional `draft`, and the rendered content as the body; it renders nothing and returns stable evidence through `as` — the filtered Repository identity, the provider's own stable pull-request identity, number, URL, open state, and the head and base SHAs of the snapshot it finished at. Without a number it creates one pull request for the head/base pair or adopts the compatible one an interrupted attempt left; with a number it brings that exact pull request's title, body, draft state and base to what the request says, records a no-op when they already match, and refuses a number belonging to another repository, opened from another head, or no longer open. It never pushes, never rewrites a head, and never reopens, merges or comments. The run must already hold its own successful `Git.Push` result for that exact Repository identity, head branch, destination ref and commit — proven by a scan of the whole successful history that requires each relevant record's natural key, inputs and result to describe one publication; a branch is published repeatedly, so the whole history is read in order and the run's last publication of that branch decides — an earlier one behind it is history rather than disagreement, while a last one naming another commit is the branch having moved on; that is conflicting, no relevant record at all is missing, and a relevant record that cannot be read whole is unreadable, each failing locally before the Git host is observed; the first adapter works over `github.com` on REST plus the two GraphQL draft transitions, selected from the private retained locator, credentialed from `GH_TOKEN`, then `GITHUB_TOKEN`, then the machine's own `gh` login, issuing each required mutation at most once per attempt and deciding the outcome by one observation, with the locator, endpoint, credential and payload confined to the per-invocation provider closure | built on the #295 stack, Deno provider only | -| `` | asks one of two questions, decided by its own shape, through a boundary of its own rather than the Git host's. Self-closing with `url` reads that issue and binds `{ url, title, description, tags, assignee }`; paired with `title` upserts and binds exactly `{ url }`, its rendered content being the description. There is no `description` prop. Props are exactly `url`, `title`, optional `tags`, optional `assignee` and — on a read only — optional `provider`; no repository/token/label/milestone/project/comment/close or approval prop. Both forms render nothing. The form is decided before the tracker is read, before any provider is asked and before an `issue_effect` record exists, and that is where a mixed `url`+`title`, a read carrying content or `tags`/`assignee`, an upsert with no content, an upsert naming a `provider`, and an element that is neither are all refused. A read needs no tracker — its URL is the identity; an upsert requires the nearest lexical `` and takes its discriminator only from there. The tracker carries a credential-free `url` and an optional `provider`; the URL is canonicalized — a credential, a query and a fragment are refused rather than stripped — and a nested tracker replaces the whole value for its descendants, never merging members, with the enclosing one restored on leaving. It is composition data, not authority: the provider holds an adapter-private ceiling beside its credentials, admitted before it connects, so a target outside it sends nothing. One stable contextual operation, `executablemd.workflow.issue`, with `read(url, options)` and `upsert(issue, options)`; a provider is ordinary middleware around it, matching its own URLs without a discriminator and only its own name with one, independently per member, with no host-side resolution. Once middleware matches it owns the answer — it never delegates afterwards, and nothing catches its refusal to try somebody else — and a request everyone delegated reaches `NoIssueProvider` unchanged. `issue_effect` records an operation discriminator with the normalized request and result; both forms replay without reaching `IssueApi` and therefore without network access; only an upsert derives an idempotency key, from the operation, the canonical target and the run's own effect identity. Retention excludes credentials, endpoints, payloads, provider identities, origin markers and host paths. Observing, adopting, creating once and recovering an interrupted creation are the provider's, because they are knowledge about what a service can prove; title is never identity, and tags are a code-point-sorted set. The Deno workflow host installs configured GitHub middleware and installs none otherwise, so absence of configuration is fail-closed | built on the #296 stack; GitHub middleware, Deno host | +| `` / `` composition under a workflow run | names a Git repository and its linked checkouts inside the run-owned Workspace, installs each as contextual working directory, and retains creation identity beside the retained Git bytes | built on the #293 stack, Deno provider only | +| `` | asks the mandatory provider-neutral `API.Files.ensureDirectory` operation to recursively create or adopt its target before content begins, then expands content with that target as cwd. Relative paths resolve from the enclosing cwd and absolute paths keep their existing meaning; an existing directory and its contents are preserved, a file or special target refuses before content, and the enclosing cwd is restored on success, failure and cancellation without deleting what was created. Dir changes cwd only and leaves the Repository selection untouched | directory-creation contract added by #643; host and workflow Files providers | +| transactional Git effects (`Git.Switch` / `Git.Add` / `Git.Commit`) under a workflow run | publish local Git mutations with their journal result; the enclosing Repository and the contextual working directory select which retained checkout one runs in, and neither observation carries authority — the observed record is compared with the retained row and the directory with the checkouts that row holds, so a failure of authority, of retained state or of an unrecognized native condition fails the run instead of publishing a result | built on the #294 stack, Deno provider only | +| `Git.Push` under a workflow run | publishes the selected checkout's exact current named branch and commit to the same branch on the retained Repository's canonical `origin`, reconciled through the shared Git-host state machine rather than through a Workspace transaction: no props and no component result, no force, no upstream mutation and no implicit staging or committing; the durable request and record carry the Repository's filtered identity without its checkout path, and the transport runs in a provider-owned isolated control repository reading the checkout's objects through an object-source attachment whose alternates chain and object tree are proven contained before the first remote observation, aimed at the exact private retained locator. A destination proven absent is published to once and one already naming this exact commit is adopted; one naming a distinct commit that same authenticated source proves is in this commit's ancestry is a performable pre-state, published over by the same exact non-force refspec and retained as the predecessor with the attested relation, while a divergent commit and one the source cannot read are both conflicts and nothing is fetched to decide either; a completed Push is reconstructed from the Workspace root its own journal event was appended against, read without publishing it or moving the run's frontier, so a branch published more than once resumes | built on the #370 stack, Deno provider only | +| `` under a workflow run | upserts one pull request of the selected checkout's current named branch, reconciled through the shared Git-host state machine: a required `title`, an optional positive-integer `number`, an optional `base` defaulting to the Repository's retained initial branch, an optional `draft`, and the rendered content as the body; it renders nothing and returns stable evidence through `as` — the filtered Repository identity, the provider's own stable pull-request identity, number, URL, open state, and the head and base SHAs of the snapshot it finished at. Without a number it creates one pull request for the head/base pair or adopts the compatible one an interrupted attempt left; with a number it brings that exact pull request's title, body, draft state and base to what the request says, records a no-op when they already match, and refuses a number belonging to another repository, opened from another head, or no longer open. It never pushes, never rewrites a head, and never reopens, merges or comments. The run must already hold its own successful `Git.Push` result for that exact Repository identity, head branch, destination ref and commit — proven by a scan of the whole successful history that requires each relevant record's natural key, inputs and result to describe one publication; a branch is published repeatedly, so the whole history is read in order and the run's last publication of that branch decides — an earlier one behind it is history rather than disagreement, while a last one naming another commit is the branch having moved on; that is conflicting, no relevant record at all is missing, and a relevant record that cannot be read whole is unreadable, each failing locally before the Git host is observed; the first adapter works over `github.com` on REST plus the two GraphQL draft transitions, selected from the private retained locator, credentialed from `GH_TOKEN`, then `GITHUB_TOKEN`, then the machine's own `gh` login, issuing each required mutation at most once per attempt and deciding the outcome by one observation, with the locator, endpoint, credential and payload confined to the per-invocation provider closure | built on the #295 stack, Deno provider only | +| repository composition vocabulary | one array of thirteen ordinary, shadowable registrations — `Repository`, `Worktree`, `Dir`, the four `Git.*` operations, `PullRequest` and its three evidence reads, `IssueTracker` and `Issue` — consumed by the workflow attachment, by `xmd syntax` and `xmd plan`'s validation and generation, and by an ordinary document execution, so one vocabulary is described and resolved everywhere. Registering it installs no provider, discovers no repository, acquires no lock, spawns no Git and reads no credential; what a name does is the installed provider's. A repository-local Markdown or TypeScript component of the same name is chosen ahead of any of them | built on the #643 stack | +| `` / `` composition under an ordinary run | selects a managed checkout under `~/.xmd/repositories`, addressed by a digest of its whole identity and described by a closed version 1 sidecar written by exclusive temporary sibling plus atomic rename only after the checkout is complete and verified. The slot is entered under an exclusive non-blocking advisory lock held for the whole document execution, so a second process is refused rather than made to wait and a self-closing Worktree captured with `as` stays protected while a later sibling `` and an interactive Session use it. Reuse compares creation identity alone — the immutable request, the recorded creation facts, the canonical checkout and common directory, the object format, the admitted `origin` and the creation commit still being present — and never HEAD, the current branch, the index or the working tree, which are the mutable work the checkout exists to preserve; a conflict refuses and leaves every byte where it was, and nothing resets, switches, cleans, fetches, moves, replaces, repairs or deletes. A metadata-free slot is adopted only after the stricter pre-exposure state is proved — exact owner and locator, the branch and base this request resolves to, the creation commit still being HEAD, the object format, linked-worktree registration where applicable, and nothing in the slot but the checkout — and refuses otherwise. Written outside a lexical ``, a Worktree belongs to the ambient Repository; outside a Git checkout it refuses locally and names how to run inside one | built on the #643 stack, Deno and compiled only | +| local Git operations (`Git.Switch` / `Git.Add` / `Git.Commit`) under an ordinary run | perform the same authored transitions the workflow performers perform — named branches only, explicit Add pathspecs, index-only Commit, no implicit stage or push, and the same fixed provider Git configuration that disables hooks, signing, file-system monitors and repository-supplied helper programs — directly against the authenticated selected checkout. A commit records the invoking user's own effective Git identity, captured once from the trusted host before the document expands and read back off the written object; a host that can name no identity refuses `` and names the commands that fix it rather than writing the workflow identity into somebody's repository, and every other component stays usable. They enlist in no transaction, roll back nothing and replay nothing, and a failure claims neither. Which checkout one runs in is decided by the Repository selection in scope and the contextual working directory, resolved through the provider's own invocation-owned checkout registry rather than through anything the selection says about itself | built on the #643 stack, Deno and compiled only | +| `Git.Push` under an ordinary run | keeps the same observe/adopt/fast-forward/refuse rules and the same isolated transport aimed at the checkout's admitted `origin`: a destination proven absent is published once, one already naming this exact commit is adopted, one holding a proven ancestor is published over by the same exact non-force refspec, and a divergent or unreadable one is a conflict, with an unreachable host never read as absence. It reconciles no Git-host effect and retains nothing. After a verified performed or adopted publication it stores one private evidence entry — the authenticated Repository identity, canonical checkout root, origin, named branch, destination ref and exact commit — in the provider instance's own closure. A checkout with no admitted `origin` refuses before a credential, a session or a transport exists | built on the #643 stack, Deno and compiled only | +| `` and its evidence reads under an ordinary run | share the URL matching, host ceiling, response normalization and low-level GitHub reconciliation, and differ in lifecycle and authority. A read is performed afresh every execution and retained nowhere. An upsert authenticates the Repository selection and the contextual checkout, reads the current named branch and commit, and requires the exact matching entry this provider instance already holds — a Push for another checkout, Repository, origin, destination, branch or commit is irrelevant, a later Push of the same destination supersedes the earlier entry, and missing or conflicting evidence is a local refusal before a credential is opened. Nothing crosses executions: a new run and a new `--journal` run each start with a new invocation identity and empty evidence, and copying a Context value, a component result or a previous trace file grants nothing. Within one invocation the attempt happens at most once; across a process interruption there is no exactly-once claim | built on the #643 stack, Deno and compiled only | +| `` under an ordinary run | reaches the same configured transport under the same host ceiling, with no durable envelope: identity is this execution's own opaque invocation identity together with the engine's expansion identity, so an upsert presents an idempotency key a provider can carry and a second run is a new request rather than a resumption. Absent or out-of-ceiling configuration installs no matching provider and sends no credential and no request | built on the #643 stack, Deno and compiled only | +| ordinary repository provider assembly | the Deno source entrypoint and the compiled binary install the live provider for `xmd run` and for an approved `xmd plan --run`, parameterized by the same credential-helper assembly the workflow host uses and by the two existing host configurations, `XMD_WORKFLOW_GITHUB_ISSUES` and `XMD_WORKFLOW_GITHUB_PULL_REQUESTS`, both read and validated before a document runs. A nested `` child receives a fresh instance — its own invocation identity, leases and Push evidence — so nothing it publishes authorizes its parent or a sibling. The outer `xmd test` command and a workflow execution install none. Node and Bun register the vocabulary and install no operational provider, so every repository operation reports an absent provider before a lock, a credential, a subprocess or a request exists | built on the #643 stack | +| `` under a workflow run | asks one of two questions, decided by its own shape, through a boundary of its own rather than the Git host's. Self-closing with `url` reads that issue and binds `{ url, title, description, tags, assignee }`; paired with `title` upserts and binds exactly `{ url }`, its rendered content being the description. There is no `description` prop. Props are exactly `url`, `title`, optional `tags`, optional `assignee` and — on a read only — optional `provider`; no repository/token/label/milestone/project/comment/close or approval prop. Both forms render nothing. The form is decided before the tracker is read, before any provider is asked and before an `issue_effect` record exists, and that is where a mixed `url`+`title`, a read carrying content or `tags`/`assignee`, an upsert with no content, an upsert naming a `provider`, and an element that is neither are all refused. A read needs no tracker — its URL is the identity; an upsert requires the nearest lexical `` and takes its discriminator only from there. The tracker carries a credential-free `url` and an optional `provider`; the URL is canonicalized — a credential, a query and a fragment are refused rather than stripped — and a nested tracker replaces the whole value for its descendants, never merging members, with the enclosing one restored on leaving. It is composition data, not authority: the provider holds an adapter-private ceiling beside its credentials, admitted before it connects, so a target outside it sends nothing. One stable contextual operation, `executablemd.workflow.issue`, with `read(url, options)` and `upsert(issue, options)`; a provider is ordinary middleware around it, matching its own URLs without a discriminator and only its own name with one, independently per member, with no host-side resolution. Once middleware matches it owns the answer — it never delegates afterwards, and nothing catches its refusal to try somebody else — and a request everyone delegated reaches `NoIssueProvider` unchanged. `issue_effect` records an operation discriminator with the normalized request and result; both forms replay without reaching `IssueApi` and therefore without network access; only an upsert derives an idempotency key, from the operation, the canonical target and the run's own effect identity. Retention excludes credentials, endpoints, payloads, provider identities, origin markers and host paths. Observing, adopting, creating once and recovering an interrupted creation are the provider's, because they are knowledge about what a service can prove; title is never identity, and tags are a code-point-sorted set. The Deno workflow host installs configured GitHub middleware and installs none otherwise, so absence of configuration is fail-closed | built on the #296 stack; GitHub middleware, Deno host | | workflow lifecycle inspection and control | reads status/list/history without advancing a run, recovering a private copy when a crashed source needs rollback; enforces the executor lock, refuses live cancellation, cancels non-live runs under that lock and deletes retained state | direct read-only inspection and control built on the #367 stack; coordinated recovered inspection built on the #513 stack, Deno provider only | | XMD artifact export, inspection and fork source | seals one run's committed retained state, Workspace roots and workflow definition source closure into one immutable `.xmd` evidence file; opens that file read-only for status/history and admits continuation only by creating a new history fork whose lineage names the artifact identity | specified by `specs/xmd-artifact-spec.md`; the version-1 sealed container, its total read-only verifier, `xmd workflow export` and artifact `status`/`history` are built, Deno provider only — the artifact-source fork remains unbuilt. Inspection is two sibling lifecycle operations, `inspectArtifact()` and `historyArtifact()`, taking a path rather than a run id: a run id names live lifecycle authority and a path names immutable evidence, so neither is a mode of the other. They reach no run store, lock, Workspace, definition reader or external provider, and the artifact path never enters the structural answer | | Agent session portability evidence in an XMD artifact | classifies every logical Agent session that contributed a retained Prompt as portable — with ordered provider checkpoint tokens and an opaque Agent session bundle — or as explicitly unavailable, as two content kinds inside the existing version-1 manifest and identity | specified by `specs/xmd-artifact-spec.md` §2.5; the closed union, both content kinds and the complete post-identity profile verifier are built on the #621 stack, Deno provider only. Provider bundle capture, Agent-aware export, intrinsic Agent-aware inspection and artifact-backed fork are unbuilt | @@ -3789,7 +3834,7 @@ Status is measured against main. | workflow Agent session | a workflow document's `` runs under a profile the host attaches only for a live or partial run: an empty host-owned working directory instead of any Workspace, checkout or caller path, no MCP servers, an empty requested native tool set, and `deny-all` with a permission path that denies every native request and fails the turn that asked without reaching the public permission chain. Within a run a session is identified by the Agent/Session expansion identity the engine derived — the authored name is descriptive, so two sibling `` elements are two sessions — routed inside a placement bound to its element and good for one use, so a kept placement cannot be substituted for the next. The conversation is retained as a row in the run's own database with the provider, resolved agent command and policy fingerprint beside it as compatibility attributes. The order is placement, the backend's acceptance of the session's first turn, the provider's canonical tagged assertion, then the mapping commit — and only then is anything that turn produced exposed. A placement is inert: it creates no provider session and writes no row. Occupancy of a provider key is not an assertion, and a record held for a first turn nobody accepted asserts nothing at all; the pre-commit window reconciles only from exactly one canonical assertion, and a missing, conflicting, replaced or ambiguous assertion is one explicit refusal that starts no replacement. Deleting a run removes the row with the run and the provider-session directory beside it, and reports the categories. The profile selects ACP-only capability explicitly — no native-launch advertisement and no client-native attachment advertisement — rather than inheriting the provider package's ordinary-run sets by omission, and it supplies no machine session coordinator, construction-route store or executable observer: a workflow session belongs to a run, and the machine-wide account describes a different thing entirely | built on the #302 stack, with the explicit ACP-only selection from #561; the portable proof that an adapter honours an empty tool set is tracked by #496 and does not widen the ceiling | | generated-XMD admission | admits one Agent-generated fragment through the trusted-host seam: host policy is a `read` table and a `write` table of exact pinned identities, each carrying the authored forms it is admitted for, and an authored `allow` selects a canonical non-empty subset of the closed classes — omitted means `read`. The complete source is preflighted inside one `generated_xmd` durable effect before its first generated effect; only the pinned identity the selected classes hold for that name **and** that form executes; and the admitted source, class selection, selected root, every selected entry with its forms, the identity and form of each element named, and the normalized request policy are retained in that effect's own result — so a continuation restores the decision without reading the current candidate and expands only the retained source. The roots are an as-of-admission retained basis checked by membership — the run's own later root publications and an advanced retained current root pass, while a lost admission root or lost selected root refuses — and every non-root term is checked exactly, refusing a run whose classes, identities, forms or requests have moved. The admission and every nested generated effect are offered inline by the owning expansion in authored order, so a partial continuation restores each completed one without another live execution. Each admitted effect is retained by its own ordinary record, and a read's value is collected while a mutation's is not | built on the #369 stack, continuation basis amended by #589; core owns the mechanics and the workflow policy wrapper is internal | | `` | the workflow host's component an authored document writes where an observation should happen. The host does not register it: it **declares** it to the execution through `ExecutionInstallation.components`, captured before any installation runs, and canonical execution calls its factory once for that attachment with the claimant it minted and registers what comes back. Registration provides availability only — a name a trusted document may write — and carries none of the authority. Its schema is closed on one required `source` string and one optional `allow` array selecting a non-empty duplicate-free subset of the closed effect classes `read` and `write` — omitted means `read` — and paired content is refused. It declares no `returns` and answers with a detached value — `{ observations: [{ name, value }], output }`, each admitted read's own returned value under the name the fragment invoked it by, in invocation order, with whatever the fragment rendered under `output` rather than instead of them, and the pinned identity that produced one left in the retained admission rather than copied here — so an admitted ``, which renders nothing at all, still reaches the document. An admitted mutation contributes no entry and no receipt, so a write-only fragment binds `{ observations: [], output: "" }`; `as` is valid for every selection and binds that same shape. An ordinary `as` captures that value by reference, and an authored `` renders it into the next ``: deciding how a value becomes text is the document's. Every ceiling comes from values the host captured at installation — the run's retained roots and its authoritative current root read from the run's own storage at invocation, as-of-admission provenance a continuation holds by membership so the run's own later publications and an advanced retained current root invalidate nothing, core's pinned self-closing `` read, the write table of core's paired ``, this package's lexical `` built from the definition the ordinary registration owns, and core's self-closing ``, and `` only when the captured request ceiling is non-empty — and no prop, binding, context or middleware return value supplies or widens one. `allow` selects among those tables and adds nothing to them; approval, when a workflow needs one, is authored control flow before the element. Its durable operation is named through that claimant, on the exact invocation the engine handed it and in that invocation's own frame — not from a context a document could rebind, a contextual Api answer, a definition, or a registry answer. Generated source never resolves through the registration: the evaluator consults only its own closed table of pinned identities. It is deliberately not wrapped in `printErrors`, so a refused fragment stops the authored loop rather than becoming text the next turn could read as a read that happened | built on the #302 stack, extended by #369 | -| generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks: the standard Deno profile's write table is core's paired `File:write`, the workflow package's lexical `Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension; admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | +| generated mutation proposals | lets an Agent propose constrained executable changes that a separate admission then performs against the run's own Workspace | built on the #369 and #567 stacks, with directory creation added by #643: the standard Deno profile's write table is core's paired `File:write`, the paired `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes Dir's persistent recursive directory creation; its versioned identity makes every continuation retained under the former non-mutating Dir identity refuse before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, and the evaluator adds no mutation API or receipt. Approval is authored control flow before the write-enabled element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack | | scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt | | `` | retry a region until it completes | defined, unbuilt | diff --git a/issue-643-implementor-handoff.md b/issue-643-implementor-handoff.md new file mode 100644 index 00000000..94765cbd --- /dev/null +++ b/issue-643-implementor-handoff.md @@ -0,0 +1,127 @@ +# Issue #643 — `` production-correction handoff + +## Continuity + +- Stack base commits: + 1. `0b02832d136ab9f847cc7a97ef16ba333c013e3a` + 2. `7be4b873252e1d4c715e04bdacd03ab5b9ce2916` + 3. `e9ff9d60565c9f9986c5371cd3710d4fc122ccaa` +- This correction follows the architecture-artifact commit on top of PR 3. +- Existing PR #689 remains untouched. Pushes and PR editing belong to delivery, + not this feedback step. +- Preserve `scratch/orc19-probe.ts`; do not create, remove, prune or repair a + worktree. + +## Settled correction + +`` is not placement-only. It performs one provider-neutral semantic +directory operation before it installs cwd or expands content: + +- recursively create the specified directory when it is missing; +- use an existing directory without replacing, clearing or otherwise changing + it or its contents; +- refuse a target or intermediate entry that is a file or another + non-directory, before content begins; +- resolve a relative path against the enclosing contextual cwd and preserve the + component's existing absolute-path behavior; +- expand content with the resulting directory as cwd; +- restore the enclosing cwd on success, failure and cancellation; +- leave created directories in place, including after later content failure or + cancellation; and +- change no Repository selection or identity member. + +The contract applies to ordinary and workflow runs. Shared `` calls only +the mandatory stable `API.Files.ensureDirectory` operation. It does not call +`API.Fs`, `@effectionx/fs`, a runtime API or RepositoryComposition. The absent +Files provider remains fail-closed, and independently loaded package copies +compose through the stable Api name. All four host entrypoints install the host +Files provider, so Node and Bun retain operational `` behavior despite +having no ordinary repository provider. + +## Provider contracts + +The host Files provider resolves the target in the contextual filesystem, +recursively creates it directly, accepts an existing directory and maps every +documented refusal to the shared sanitized vocabulary. It is deliberately +non-transactional: successful creation persists, and neither later content +failure nor cancellation triggers rollback or teardown deletion. + +The workflow Files provider publishes exactly one `workspace_file` effect for +the ensure. Recursive mutation runs inside its savepoint. On success, the +mutation, sanitized outcome and resulting Workspace root commit together. A +documented refusal rolls the savepoint back and records the refusal against the +unchanged root. Infrastructure failure or cancellation before commit publishes +neither an outcome nor a root. Completed replay restores the recorded success +and retained root without attempting creation again. No retained field contains +a host path, errno/platform code, raw platform message or cause. + +## Generated-XMD authority + +The product decision is settled: selecting `allow={["write"]}` intentionally +authorizes ``'s persistent recursive directory creation. + +Make that authority change visible in retained identity. The paired write-table +entry is `@executablemd/workflow/composition/dir-v2#Dir`. The former +`@executablemd/workflow/composition#Dir` identity authorized placement that +created nothing and must never authorize creation. A continuation retaining the +former identity refuses before generated execution and mutation. Read-only +admissions remain unaffected because they never selected the write table. + +## Implementation boundaries + +1. Extend the shared `API.Files` contract with mandatory + `ensureDirectory(cwd, path)` returning Effection `Result`. Keep the + existing fail-closed terminal and stable namespaced Api. +2. Implement the operation in the host Files provider through asynchronous + Effection filesystem operations. Preserve the host provider's containment + and sanitization rules, including the established absolute-Dir exception. +3. Implement it in the transaction-bound workflow Files provider as one named + `workspace_file` effect and savepoint, using the logical Workspace path and + root-publication protocol already used by file mutations. +4. Change shared `` ordering to ensure, then install cwd, then expand + content. Scope teardown restores cwd only. +5. Version the pinned paired Dir origin in the generated write table and cover + continuation refusal from the former identity. +6. Keep Repository composition out of the operation. Do not select, replace or + reconstruct a Repository from a directory path. +7. Update the component description to the already accepted text, without + further product interpretation: + + > Create or use a directory. `…` changes the working directory for its content. + +Do not add a second directory API, a test-only provider fallback, host/runtime +detection in shared code, rollback for ordinary host creation or deletion during +Dir teardown. + +## Frozen acceptance evidence + +Evidence must distinguish every row; one broad happy-path test does not replace +the failure, replay or authority cases. + +| ID | Criterion | Discriminating evidence | +| --- | --- | --- | +| ORC6 | Missing directory and ordering | Missing relative parents are created recursively, an absolute target keeps its meaning, and no content begins before the ensure succeeds | +| ORC6a | Existing directory | Existing contents remain byte-identical; the directory is neither replaced nor cleared | +| ORC6b | Non-directory refusal | File and supported special targets, including an intermediate entry, refuse before content with sanitized structural data | +| ORC6c | Cwd restoration | The enclosing cwd is restored after success, printed failure and cancellation | +| ORC6d | Repository invariance | Every Repository identity member and provider selection remains unchanged inside and after Dir | +| ORC6e | Ordinary persistence | Host-created directories remain after success, failed content and cancellation; no teardown delete runs | +| ORC6f | Workflow atomicity | Recursive creation, one `workspace_file` success and its resulting authoritative root commit together before content | +| ORC6g | Workflow refusal rollback | A documented non-directory refusal rolls back its savepoint, creates no partial parent, begins no content and records the unchanged root | +| ORC6h | Workflow replay/cancellation | Completed replay performs no second ensure and restores the retained root; pre-commit cancellation publishes nothing; later content cancellation preserves the committed directory | +| ORC6i | Generated authority | `write` admits only paired `dir-v2#Dir`; the former retained identity refuses before generated execution and creates nothing | +| ORC20 | Retained regression | Repository/Worktree/Git/Issue/PullRequest replay contracts and provider call counts remain unchanged; directory ensure contributes only its own effect/root | + +Also extend the shared host Files contract/compiled probe and the absent-provider +suite. Exercise the ordinary component under Deno, Node and Bun because the +host Files provider is their operational boundary. Exercise workflow creation, +existing-directory adoption, refusal rollback, completed replay, current-root +publication and cancellation at the retained provider boundary. + +## Feedback handoff + +Create one production feedback commit on top of the architecture-artifact +commit after the smallest focused tests that discriminate these rows pass. Hand +back its exact base and SHA, every changed file, every focused command and exact +result, the row-to-test mapping, and any deviation. Do not wait for CI or push +before requesting the architecture verdict. diff --git a/packages/cli/src/bun.ts b/packages/cli/src/bun.ts index 8850382c..1592632e 100644 --- a/packages/cli/src/bun.ts +++ b/packages/cli/src/bun.ts @@ -14,6 +14,7 @@ import { runXmd, XMD_VERSION } from "./cli.ts"; import type { UpgradeAssembly } from "./upgrade.ts"; import { unassembledMachineSessions } from "./session-coordinator.ts"; import { unsupportedWorkflowHost } from "./workflow.ts"; +import { unsupportedRepositories } from "./run-repositories.ts"; import { useBunService } from "./bun-service.ts"; const ENTRYPOINT = fileURLToPath(import.meta.url); @@ -59,10 +60,16 @@ await main(function* (args) { // build either. Advertising the same names is what makes the refusal say so: // every advertised operation stops before provider work, while ordinary ACP // work is unaffected. + // The same thirteen repository components, and no provider that operates + // any of them. This runtime has no kernel-released advisory lock to hold a + // managed checkout with, so a Repository, Worktree, Git, Issue or PullRequest + // operation reports an absent provider before a local or remote change could + // happen. `xmd syntax` still describes one language everywhere. yield* runXmd( args, useBunService, UPGRADE, + unsupportedRepositories, unsupportedWorkflowHost, unassembledMachineSessions(), ); diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 083de179..c0eb2459 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -131,6 +131,8 @@ import type { UpgradeAssembly } from "./upgrade.ts"; import { componentSearchPath, resolveTestTarget } from "./test-target.ts"; import { renderSyntaxJson, renderSyntaxMarkdown, syntaxCatalog } from "./syntax.ts"; import { testingExecutionHost } from "./testing-host.ts"; +import { unsupportedRepositories } from "./run-repositories.ts"; +import type { RepositoryInstaller } from "./run-repositories.ts"; import { EVAL_ALIAS, EVAL_OPTION, evalGrammarError, readEvalFlags } from "./eval-source.ts"; import type { EvalFlags } from "./eval-source.ts"; import { @@ -144,7 +146,7 @@ import type { HostWorkflowInstaller, WorkflowHost, WorkflowStart } from "./workf import { runWorkflowManagement } from "./workflow-management.ts"; import { establishDefinition } from "./workflow-definition.ts"; import type { EstablishedDefinition } from "./workflow-definition.ts"; -import { useWorkflowServiceDenial } from "@executablemd/workflow"; +import { COMPOSITION_REGISTRATIONS, useWorkflowServiceDenial } from "@executablemd/workflow"; import denoJson from "../deno.json" with { type: "json" }; const SECRET_DETECTION_OPTION = "--secret-detection"; @@ -862,6 +864,13 @@ export function* installDocumentComponents(mode: DocumentMode, verbose: boolean) // `` is registered at all. yield* Config.around({ verbose: () => verbose }, { at: "min" }); + // The repository-composition vocabulary, as ordinary shadowable defaults. + // Registering it installs no provider, discovers no repository, acquires no + // lock and reaches no network: what a name *does* is decided by whichever + // provider the command installed, and a runtime that installs none still + // resolves every one of these. + yield* registerComponents(COMPOSITION_REGISTRATIONS); + // Compose testing around the single core execution entrypoint: both // commands register the components (assertions work in regular documents, // explicit boundaries affect the outcome), while `xmd test` @@ -913,6 +922,8 @@ function* runDocument( config: DocumentConfig, mode: DocumentMode, installService: HostServiceInstaller, + installRepositories: RepositoryInstaller, + childRepositories: RepositoryInstaller, ): Operation> { const { root, include, verbose, journal, raw, secretDetection, retainProcessOutput } = config; @@ -1021,6 +1032,15 @@ function* runDocument( // the provider for a service. yield* installService(); + // Repository authority belongs to document execution too, and it is this + // execution's own: the provider it installs holds an invocation identity, the + // leases on the checkouts this document selects, and the evidence of what it + // published. `xmd run` and an approved `xmd plan --run` supply the live one; + // `xmd test` and every runtime without an operational provider supply the one + // that installs nothing, and every repository operation then reports an + // absent provider before touching anything. + yield* installRepositories(); + // What a `` in this document runs a nested execution under. Captured // before document code begins, so a child is offered exactly what this // command assembled — and never a second description of it. @@ -1033,6 +1053,13 @@ function* runDocument( includes: include, secretDetection, installService, + // The *entrypoint's* installer, not this command's. A `host="run"` child is + // an ordinary run whatever command is hosting it, so `xmd test` — which + // installs no repository provider for its own document — still gives one to + // a child that asked to be a run. Passed rather than inherited because a + // child runs in an isolated scope and needs a fresh instance: its own + // invocation identity, its own leases and its own Push evidence. + installRepositories: childRepositories, testAgentWorker: yield* readWorkerCommand(), plan, }); @@ -1134,9 +1161,13 @@ function* runScopedDocument( config: DocumentConfig, mode: DocumentMode, installService: HostServiceInstaller, + installRepositories: RepositoryInstaller, + childRepositories: RepositoryInstaller = installRepositories, ): Operation> { try { - return yield* scoped(() => runDocument(config, mode, installService)); + return yield* scoped(() => + runDocument(config, mode, installService, installRepositories, childRepositories), + ); } catch (error) { return Err(error instanceof Error ? error : new Error(String(error))); } @@ -1166,6 +1197,7 @@ export function planExecutor( stack: AgentStack, sessions: MachineSessionAssembly | undefined, installService: HostServiceInstaller, + installRepositories: RepositoryInstaller, ): (approved: PlanExecution) => Operation> { return (approved) => scoped(function* (): Operation> { @@ -1192,6 +1224,10 @@ export function planExecutor( agent: stack, }, installService, + // An approved plan's second execution is an ordinary run, so it gets + // the ordinary provider — a fresh one, since the authorship profile's + // scope is already gone. + installRepositories, ); }); } @@ -1246,6 +1282,8 @@ function* test( config: TestConfig, args: string[], installService: HostServiceInstaller, + /** What a `` child installs. This command installs none. */ + installRepositories: RepositoryInstaller, ): Operation { const patterns = readPatternFlags(args); if (patterns.missingValue) { @@ -1279,6 +1317,12 @@ function* test( { ...config, root: { path } }, { testing: true }, installService, + // The outer `xmd test` command installs no operational repository + // provider. A test that needs the production behavior exercises an + // explicit `` child, which is an ordinary run and + // is handed the entrypoint's own installer below. + unsupportedRepositories, + installRepositories, ); if (!result.ok) { reportFailure(result.error); @@ -1319,6 +1363,8 @@ function* test( }, { testing: true }, installService, + unsupportedRepositories, + installRepositories, ); if (!result.ok) { reportFailure(result.error, document.relativePath); @@ -2105,6 +2151,7 @@ function* dispatch( helpRequest: { requested: boolean; args: string[] }, installService: HostServiceInstaller, upgrade: UpgradeAssembly, + installRepositories: RepositoryInstaller, workflowHost: WorkflowHost | undefined, sessions: MachineSessionAssembly | undefined, ): Operation { @@ -2206,6 +2253,7 @@ function* dispatch( agent: runStack, }, installService, + installRepositories, ); }); if (!result.ok) { @@ -2263,7 +2311,7 @@ function* dispatch( // A host that answers installs a provider; one that does not installs // none, and nothing downstream reads a profile to find out which. installElicitation: installWebElicitation, - execute: planExecutor(config, planStack, sessions, installService), + execute: planExecutor(config, planStack, sessions, installService, installRepositories), }, ); if (exitCode !== 0) { @@ -2364,6 +2412,7 @@ function* dispatch( { ...command.config, retainProcessOutput: keepsProcessOutput(command.config.journal) }, evalFlags.rest, installService, + installRepositories, ); break; } @@ -2380,7 +2429,7 @@ function* dispatch( yield* exit(1); break; } - process.stdout.write(rendered); + yield* writeStdoutWhole(rendered); break; } case "test-agent": @@ -2469,6 +2518,9 @@ function* dispatch( // service adapter would: installed inside the execution scope, // before the root document is imported. useWorkflowServiceDenial, + // A workflow run's repositories are the retained ones its Workspace + // attachment installs, so this path installs none of its own. + unsupportedRepositories, ), ), ); @@ -2478,6 +2530,27 @@ function* dispatch( } } +/** + * Write to stdout and wait for it to reach the operating system. + * + * `process.stdout` is asynchronous when it is a pipe and synchronous when it is + * a file or a terminal. A large document handed to `write` is therefore still + * sitting in a buffer when the run ends, and the process exits without it: + * `xmd syntax --json > file` is whole, `xmd syntax --json | jq` stops at about + * 64 KiB, in the middle of a token. Waiting for the callback is what makes the + * write finish before anything can exit. + * + * Only the catalog goes through this today, because it is the one output this + * command writes in a single call and the only one already past the buffer. + */ +function* writeStdoutWhole(text: string): Operation { + yield* until( + new Promise((resolve, reject) => { + process.stdout.write(text, (error) => (error ? reject(error) : resolve())); + }), + ); +} + export function* runXmd( args: string[], installService: HostServiceInstaller, @@ -2486,6 +2559,12 @@ export function* runXmd( // command run under any other one refuses with that installation's own remedy // rather than reaching for a release, a lock or a file. upgrade: UpgradeAssembly, + // What an ordinary document execution installs for ``, + // ``, the Git operations, `` and ``. Deno and + // the compiled binary supply the live provider; Node and Bun supply the one + // that installs nothing, so those runtimes describe the same vocabulary and + // operate none of it. + installRepositories: RepositoryInstaller, // Defaults to the host that refuses. A caller driving this without naming a // workflow host has no run store, and inheriting one by omission is the // failure mode the whole boundary exists to prevent — so the default is the @@ -2556,7 +2635,15 @@ export function* runXmd( (selected.name === "run" || selected.name === "plan"); if (!executes) { - return yield* dispatch(evalFlags, helpRequest, installService, upgrade, workflowHost, sessions); + return yield* dispatch( + evalFlags, + helpRequest, + installService, + upgrade, + installRepositories, + workflowHost, + sessions, + ); } const timeouts = resolveRunTimeouts(evalFlags.rest); @@ -2567,6 +2654,14 @@ export function* runXmd( } yield* underRunDeadline(timeouts, () => - dispatch(evalFlags, helpRequest, installService, upgrade, workflowHost, sessions), + dispatch( + evalFlags, + helpRequest, + installService, + upgrade, + installRepositories, + workflowHost, + sessions, + ), ); } diff --git a/packages/cli/src/compiled.ts b/packages/cli/src/compiled.ts index 464f1e77..ae192e3b 100644 --- a/packages/cli/src/compiled.ts +++ b/packages/cli/src/compiled.ts @@ -13,6 +13,7 @@ import { runXmd, XMD_VERSION } from "./cli.ts"; import { compiledUpgradeAssembly } from "./compiled-upgrade.ts"; import { useMachineSessions } from "./session-coordinator.ts"; import { useDenoWorkflowHost } from "./deno-workflow.ts"; +import { denoRunRepositories } from "./deno-repositories.ts"; import { isCredentialHelperMode, runCredentialHelper, @@ -80,10 +81,14 @@ if (isCredentialHelperMode(process.argv.slice(2))) { // two owners of one conversation. // Helper mode receives neither this nor the workflow host: it is not the // public CLI and assembles none of it. + // The ordinary repository provider, on the same terms the Deno entrypoint + // installs it: the binary is Deno, and the helper assembly it hands over is + // the one that names this executable rather than a module path. yield* runXmd( args, useCompiledService, UPGRADE, + denoRunRepositories(HELPER), () => useDenoWorkflowHost(HELPER), useMachineSessions(), ); diff --git a/packages/cli/src/deno-repositories.ts b/packages/cli/src/deno-repositories.ts new file mode 100644 index 00000000..1b834676 --- /dev/null +++ b/packages/cli/src/deno-repositories.ts @@ -0,0 +1,52 @@ +/** + * The live repository provider, assembled where it can be. + * + * Kept apart from `run-repositories.ts` because that module is on the shared + * command path and this one names the Deno adapter, whose module graph reaches + * `node:sqlite`. Bun has no such built-in, so a static import of this from + * shared code would stop `xmd` loading there — not refuse a repository + * operation, but fail to start at all. Only `deno.ts` and `compiled.ts` import + * this file, and both of them are Deno. + * + * Managed checkouts live beneath `~/.xmd/repositories` and survive every + * execution: what is in one is somebody's work, and nothing deletes one. There + * is no environment variable naming a different root, because the only caller + * that needs one is a test, and a test is handed the root directly. + */ + +import type { Operation } from "effection"; +import { cwd } from "@executablemd/runtime"; +import { useRunComposition } from "@executablemd/workflow/deno"; +import type { HelperAssembly } from "@executablemd/workflow/credential-helper"; +import { gitHubIssuesConfiguration } from "./github-issues-config.ts"; +import { gitHubPullRequestsConfiguration } from "./github-pull-requests-config.ts"; +import { DEFAULT_REPOSITORY_ROOT } from "./run-repositories.ts"; +import type { RepositoryInstaller } from "./run-repositories.ts"; + +/** + * The live provider Deno and the compiled binary install. + * + * The two GitHub configurations are read once, when the installer runs, so an + * operator who wrote something this host cannot use learns it before a document + * expands rather than in the middle of one. + */ +export function denoRunRepositories( + helper: HelperAssembly, + root: string = DEFAULT_REPOSITORY_ROOT, +): RepositoryInstaller { + return function* (): Operation { + const gitHubIssues = yield* gitHubIssuesConfiguration(); + const gitHubPullRequests = yield* gitHubPullRequestsConfiguration(); + yield* useRunComposition({ + root, + // The directory this execution starts in, which is where the ambient + // repository is discovered from. Read through the contextual Api rather + // than from the process, so a nested execution that composed its own + // working directory is discovered from that one. + cwd: yield* cwd(), + helper, + ...(gitHubIssues === undefined ? {} : { gitHubIssues }), + ...(gitHubPullRequests === undefined ? {} : { gitHubPullRequests }), + }); + }; +} diff --git a/packages/cli/src/deno.ts b/packages/cli/src/deno.ts index 82917b8b..c12630c9 100644 --- a/packages/cli/src/deno.ts +++ b/packages/cli/src/deno.ts @@ -16,6 +16,7 @@ import { runXmd, XMD_VERSION } from "./cli.ts"; import type { UpgradeAssembly } from "./upgrade.ts"; import { useMachineSessions } from "./session-coordinator.ts"; import { useDenoWorkflowHost } from "./deno-workflow.ts"; +import { denoRunRepositories } from "./deno-repositories.ts"; import { isCredentialHelperMode, runCredentialHelper, @@ -96,10 +97,17 @@ if (isCredentialHelperMode(process.argv.slice(2))) { // two owners of one conversation. // Helper mode receives neither this nor the workflow host: it is not the // public CLI and assembles none of it. + // The ordinary repository provider: managed checkouts under + // `~/.xmd/repositories`, the ambient repository this command was run in, + // and the two GitHub configurations this deployment authorizes. It is + // parameterized by the same credential-helper assembly the workflow host + // uses, because the program that is running is what knows how to re-invoke + // itself as one. yield* runXmd( args, useDenoService, UPGRADE, + denoRunRepositories(HELPER), () => useDenoWorkflowHost(HELPER), useMachineSessions(), ); diff --git a/packages/cli/src/node.ts b/packages/cli/src/node.ts index d07fd86e..927ba236 100755 --- a/packages/cli/src/node.ts +++ b/packages/cli/src/node.ts @@ -20,6 +20,7 @@ import { runXmd, XMD_VERSION } from "./cli.ts"; import type { UpgradeAssembly } from "./upgrade.ts"; import { unassembledMachineSessions } from "./session-coordinator.ts"; import { unsupportedWorkflowHost } from "./workflow.ts"; +import { unsupportedRepositories } from "./run-repositories.ts"; import { useNodeService } from "./node-service.ts"; const ENTRYPOINT = fileURLToPath(import.meta.url); @@ -67,10 +68,16 @@ await main(function* (args) { // build either. Advertising the same names is what makes the refusal say so: // every advertised operation stops before provider work, while ordinary ACP // work is unaffected. + // The same thirteen repository components, and no provider that operates + // any of them. This runtime has no kernel-released advisory lock to hold a + // managed checkout with, so a Repository, Worktree, Git, Issue or PullRequest + // operation reports an absent provider before a local or remote change could + // happen. `xmd syntax` still describes one language everywhere. yield* runXmd( args, useNodeService, UPGRADE, + unsupportedRepositories, unsupportedWorkflowHost, unassembledMachineSessions(), ); diff --git a/packages/cli/src/run-repositories.ts b/packages/cli/src/run-repositories.ts new file mode 100644 index 00000000..859a7b3a --- /dev/null +++ b/packages/cli/src/run-repositories.ts @@ -0,0 +1,60 @@ +/** + * Where an ordinary `xmd run` keeps the repositories it manages, and what it is + * allowed to reach. + * + * This is the only module in the CLI that names the managed root, exactly as + * `deno-workflow.ts` is the only one that names the run store. What a document + * writes decides which repository it wants; this decides where a clone of it + * lands, which issue trackers and pull requests this deployment authorizes, and + * how the host writes its own credential helper. + * + * Managed checkouts live beneath `~/.xmd/repositories` and survive every + * execution: what is in one is somebody's work — a branch, a worktree an agent + * is still editing, an uncommitted change — and nothing deletes one. There is + * no environment variable naming a different root, because the only caller that + * needs one is a test, and a test is handed the root directly. + * + * Node and Bun install none of this. They register the same thirteen + * declarations, so `xmd syntax` describes one language and a document resolves + * the same names everywhere, and every operation then reaches a clear + * provider-absence error before anything local or remote is touched. + * + * Nothing here imports the Deno adapter. This module is on the shared command + * path, and the adapter's module graph reaches `node:sqlite` — a built-in Bun + * does not have — so a static import of it here would stop `xmd` loading there + * at all. The live installer lives beside the entrypoints that can use it, in + * `deno-repositories.ts`. + */ + +import { homedir } from "node:os"; +import { join } from "node:path"; +import type { Operation } from "effection"; + +/** Where managed repositories and worktrees live. */ +export const DEFAULT_REPOSITORY_ROOT: string = join(homedir(), ".xmd", "repositories"); + +/** + * How one document execution obtains repository operations, or does not. + * + * A function rather than a value, because the provider is installed *inside* + * the execution scope and holds that execution's own invocation identity, + * leases and Push evidence. A nested `` calls it again + * and gets a fresh instance, which is what keeps a child's evidence and locks + * out of its parent and its siblings. + */ +export type RepositoryInstaller = () => Operation; + +/** + * The runtimes that register the vocabulary and operate none of it. + * + * Installing nothing is the whole implementation: ``, ``, + * the Git operations, `` and `` each reach their own Api's + * default, which reports an absent provider before a lock, a credential, a + * subprocess or a request exists. `` is unaffected — it needs no provider. + */ +export function unsupportedRepositories(): Operation { + return noRepositories(); +} + +// deno-lint-ignore require-yield +function* noRepositories(): Operation {} diff --git a/packages/cli/src/syntax.ts b/packages/cli/src/syntax.ts index 03b720f2..39915006 100644 --- a/packages/cli/src/syntax.ts +++ b/packages/cli/src/syntax.ts @@ -33,14 +33,16 @@ import type { import { TESTING_REGISTRATIONS } from "@executablemd/testing"; import { WEB_REGISTRATIONS } from "@executablemd/web"; import { VERBOSE_REGISTRATION } from "./verbose-component.ts"; +import { COMPOSITION_REGISTRATIONS } from "@executablemd/workflow"; /** * The catalog for the production `run` profile, in the contextual working * directory. * * The registrations are the ones `installTestingComponents()`, - * `installWebComponents()` and `installAgentComponents()` register, read as - * values so this cannot drift from what a run installs. What those installers + * `installWebComponents()`, `installAgentComponents()` and the + * repository-composition installer register, read as values so this cannot + * drift from what a run installs. What those installers * *also* do — testing activation and its execution middleware, the elicitation * provider, the agent provider, the permission mode, the foreground launcher — * is operational and belongs to a run, so none of it happens here. @@ -82,6 +84,10 @@ export function* useRunProfileRegistry(): Operation { ...AGENT_REGISTRATIONS, ...TESTING_REGISTRATIONS, ...WEB_REGISTRATIONS, + // The repository-composition vocabulary. Registering it is all that happens + // here: catalog construction installs no provider, discovers no ambient + // repository, acquires no lock, spawns no Git and reads no credential. + ...COMPOSITION_REGISTRATIONS, ]); } diff --git a/packages/cli/src/testing-host.ts b/packages/cli/src/testing-host.ts index 3ba748f5..f6d3218c 100644 --- a/packages/cli/src/testing-host.ts +++ b/packages/cli/src/testing-host.ts @@ -26,7 +26,7 @@ import type { DeclaredMarkdownComponent } from "@executablemd/core/host"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; import { forEach } from "@effectionx/stream-helpers"; -import { useHostFiles } from "@executablemd/runtime"; +import { API, useHostFiles } from "@executablemd/runtime"; import { installWebElicitation } from "@executablemd/web"; import type { Operation, Result } from "effection"; import { @@ -49,6 +49,7 @@ import type { } from "@executablemd/testing"; import { installDocumentComponents } from "./cli.ts"; import type { HostServiceInstaller } from "./cli.ts"; +import type { RepositoryInstaller } from "./run-repositories.ts"; /** What the entrypoint already decided, and a child must not decide again. */ export interface TestingHostSettings { @@ -69,6 +70,15 @@ export interface TestingHostSettings { readonly secretDetection: boolean; /** The native service adapter this entrypoint supplies. */ readonly installService: HostServiceInstaller; + /** + * How this entrypoint installs repository operations for one execution. + * + * Called again for every child, so an isolated `` + * constructs a provider instance of its own: its own invocation identity, its + * own leases and its own Push evidence. Nothing it publishes authorizes its + * parent or a sibling, and nothing they published authorizes it. + */ + readonly installRepositories: RepositoryInstaller; /** * How this entrypoint re-invokes itself as the test-agent worker, or why it * cannot. @@ -149,6 +159,23 @@ function* runProfileChild( if (request.host !== "run") { throw new Error(`the ${request.host} host profile is not available on this entrypoint`); } + // First, because everything below resolves against it. This scope inherits no + // `API.Env` handler, so without this the child would stand in the *process* + // directory: a `` around the `` would scope every component + // in it except the child, the root reference would resolve from somewhere the + // document never named, and the repository provider installed below would + // discover its ambient Git from whatever checkout the process was launched + // in. Installed ahead of the root, the provider and the execution, so all + // three agree with the document that asked. + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return invocation.cwd; + }, + }, + { at: "min" }, + ); const root = rootOf(request); // `--journal` is the only thing that asks `xmd run` for a diagnostic record, // and a declaration is the only thing that asks a child for one. Neither @@ -203,6 +230,9 @@ function* runProfileChild( // Native service authority belongs only to document execution, here as in the // command that owns it. yield* settings.installService(); + // And repository authority the same way, from the same installer the command + // used — a fresh instance for this child alone. + yield* settings.installRepositories(); const execution = yield* executeInstalled( { diff --git a/packages/cli/tests/compiled-upgrade.test.ts b/packages/cli/tests/compiled-upgrade.test.ts index 15d73150..6b7664f4 100644 --- a/packages/cli/tests/compiled-upgrade.test.ts +++ b/packages/cli/tests/compiled-upgrade.test.ts @@ -45,6 +45,7 @@ import { join } from "node:path"; import process from "node:process"; import { runXmd } from "../src/cli.ts"; +import { unsupportedRepositories } from "../src/run-repositories.ts"; import { compiledUpgradeAssembly, writeAll } from "../src/compiled-upgrade.ts"; import type { CompiledUpgradeDependencies, @@ -1530,7 +1531,10 @@ function* commandStatus(args: string[], assembly: UpgradeAssembly): Operation { + const home = yield* useTempDirectory("xmd-orc-remote-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-orc-remote-"))); + const seed = join(parent, "seed"); + git(["init", "--initial-branch=main", seed], parent, home); + git(["commit", "--allow-empty", "-m", "first"], seed, home); + const bare = join(parent, "remote.git"); + git(["clone", "--bare", "--", seed, bare], parent, home); + return bare; +} + +/** A repository the command is "run in", and a managed root of this suite's own. */ +function* useAmbient( + locator?: string, +): Operation<{ checkout: string; root: string; home: string }> { + const home = yield* useTempDirectory("xmd-orc-home-"); + // Canonical, so what this fixture names and what Git reports are one string. + const parent = yield* until(realpath(yield* useTempDirectory("xmd-orc-ambient-"))); + const checkout = join(parent, "checkout"); + if (locator === undefined) { + git(["init", "--initial-branch=main", checkout], parent, home); + git(["commit", "--allow-empty", "-m", "first"], checkout, home); + } else { + git(["clone", "--", locator, checkout], parent, home); + } + const managed = yield* until(realpath(yield* useTempDirectory("xmd-orc-managed-"))); + return { checkout, root: join(managed, "repositories"), home }; +} + +/** + * Run one document under the ordinary provider, on a stream a caller chose. + * + * The commit identity is stated rather than read from the host. The provider's + * production default is the caller's own `git config`, so leaving it alone + * makes `` refuse on any machine without `user.name` set — every CI + * runner — and turns these cases into an assertion about who ran them. + */ +function runOrdinary( + source: string, + options: { + root: string; + cwd: string; + journal?: string; + /** Installed after the components, where a provider's own middleware goes. */ + agent?: () => Operation; + }, +): Operation { + return scoped(function* () { + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return options.cwd; + }, + }, + { at: "min" }, + ); + yield* useHostFiles(); + yield* installAgentComponents(); + yield* useCompositionComponents(); + yield* useRunComposition({ + root: options.root, + cwd: options.cwd, + // deno-lint-ignore require-yield + *identity(): Operation { + return "Fixture 0 +0000"; + }, + }); + if (options.agent !== undefined) { + yield* options.agent(); + } + // `--journal` is exactly this: the file-backed stream instead of the + // in-memory one, created by the command before the run begins. + const stream = + options.journal === undefined ? new InMemoryStream() : new FileStream(options.journal); + return yield* collect(yield* execute({ ...inlineSource(source), stream })); + }); +} + +describe("ORC7 — a Session launched in a managed Worktree", () => { + it("hands the launch that worktree's own root and a session key of its own", function* () { + const ambient = yield* useAmbient(); + + /** Every launch this document routed, as the placement it was given. */ + const routed: { cwd: string; session: string | undefined }[] = []; + + // The public launch surface a provider answers. A real `` + // reaches exactly this, through the same installation `xmd run` makes, and + // what it is handed is the placement: the directory the session belongs to. + const capture = function* (): Operation { + // A registered provider, reached the way `` reaches one. + // Only a registered provider is handed the launch authority, so only one + // can settle a launch — middleware can route a request and cannot + // perform it, which is the boundary this uses rather than works around. + yield* registerAgentProvider("probe", function* (options, authority) { + yield* Agent.around( + { + // deno-lint-ignore require-yield + *agent([name]): Operation { + return name ?? options.defaultAgent; + }, + *launch([request]): Operation { + routed.push({ + cwd: request.cwd, + session: typeof request.session === "string" ? request.session : undefined, + }); + // Settled as a refusal rather than performed: this suite is about + // where a launch is placed, and starting a native UI would need a + // terminal nothing here has. + yield* authority.refuse(request, { + phase: "prepared", + agent: "codex", + sessionKey: deriveSessionKey( + "codex", + request.cwd, + typeof request.session === "string" ? request.session : undefined, + ), + provider: "probe", + nativeSessionId: "probe-session", + sessionState: "created", + instructionChannel: "probe", + instructionReconciliation: "installed", + identityProvenance: "provider-returned", + instructionsDigest: "0".repeat(64), + instructions: request.instructions, + cwd: request.cwd, + additionalDirectories: [...request.additionalDirectories], + permissionMode: request.permissionMode, + launcher: "probe", + failure: { + class: "unsupported-capability", + message: "this suite launches nothing", + }, + }); + }, + }, + { at: "min" }, + ); + }); + // The terminal a native launch reserves before it is routed. Reserving is + // what `xmd run` installs a real launcher for; a suite installs one that + // owns nothing, so the launch reaches the surface below rather than + // failing on a host with no terminal. + yield* NativeLauncher.around( + { + // deno-lint-ignore require-yield + *reserve(): Operation {}, + // deno-lint-ignore require-yield + *flush(): Operation {}, + }, + { at: "min" }, + ); + }; + + // One launch inside a managed Worktree, and one in the ambient checkout, in + // the same document — so the two placements are decided by where each + // element was written and by nothing else. + const bound = yield* runOrdinary( + [ + '', + ``, + // The provider settles each launch as a refusal, so the region that + // prints one is what lets the second launch happen at all. What is + // under test is where each was placed, not whether a UI started. + "", + "", + '', + "INSIDE", + "", + "", + '', + "OUTSIDE", + "", + "", + "", + "", + "{w}", + ].join("\n"), + { root: ambient.root, cwd: ambient.checkout, agent: capture }, + ); + const worktree = String(bound).trim().split("\n").at(-1) ?? ""; + expect(yield* exists(worktree)).toBe(true); + + // Both launches were routed, and each received the directory it was + // written in. + expect(routed).toHaveLength(2); + expect(routed[0]?.cwd).toBe(worktree); + expect(routed[1]?.cwd).toBe(ambient.checkout); + expect(routed[0]?.session).toBe("implementer"); + expect(routed[1]?.session).toBe("implementer"); + + // The same agent and the same session name in the two places are two + // sessions, because the placement differs. + const agent = "codex"; + const inWorktree = deriveSessionKey(agent, routed[0]?.cwd ?? "", "implementer"); + const inAmbient = deriveSessionKey(agent, routed[1]?.cwd ?? "", "implementer"); + expect(inWorktree).not.toBe(inAmbient); + + // Supporting evidence for *why* the placement stops at the worktree: `.git` + // there is a file, and the candidate walk is bounded by it. + expect(yield* readTextFile(`${worktree}/.git`)).toContain("gitdir:"); + const candidates = yield* sessionCandidates(agent, worktree); + expect(candidates.map((candidate: { cwd: string }) => candidate.cwd)).toEqual([worktree]); + const ambientCandidates = yield* sessionCandidates(agent, ambient.checkout); + expect(ambientCandidates.map((candidate: { cwd: string }) => candidate.cwd)).toEqual([ + ambient.checkout, + ]); + }); +}); + +describe("ORC18 — the journal is diagnostic", () => { + it("performs the same live work with and without a trace, once each", function* () { + const first = yield* useAmbient(); + const second = yield* useAmbient(); + const trace = join(second.root, "..", "diagnostic.jsonl"); + + const document = [ + ``, + "", + `made`, + ``, + ``, + "", + ].join("\n"); + + yield* runOrdinary(document, { root: first.root, cwd: first.checkout }); + yield* runOrdinary(document, { + root: second.root, + cwd: second.checkout, + journal: trace, + }); + + // One live mutation per invocation, either way: each repository has exactly + // one commit on the branch beyond the one it started with. + for (const ambient of [first, second]) { + expect( + git(["log", "--oneline", "traced"], ambient.checkout, ambient.home).split("\n"), + ).toHaveLength(2); + expect(git(["log", "-1", "--pretty=%s", "traced"], ambient.checkout, ambient.home)).toBe( + "Traced", + ); + } + + // The trace was newly created by that run and holds its events. + expect(yield* exists(trace)).toBe(true); + const written = yield* readTextFile(trace); + expect(written.length).toBeGreaterThan(0); + + // And it is not continuation. A third execution handed that exact trace + // performs its own work against its own repository — the trace neither + // restores the earlier commit nor stands in for one. + const third = yield* useAmbient(); + yield* runOrdinary(document, { + root: third.root, + cwd: third.checkout, + journal: trace, + }); + expect(git(["log", "-1", "--pretty=%s", "traced"], third.checkout, third.home)).toBe("Traced"); + expect( + git(["log", "--oneline", "traced"], third.checkout, third.home).split("\n"), + ).toHaveLength(2); + }); +}); + +describe("ORC15 — a trace is not evidence", () => { + it("refuses a PullRequest handed the trace of an execution that really published", function* () { + const remote = yield* useRemote(); + const first = yield* useAmbient(remote); + const trace = join(first.root, "..", "published.jsonl"); + + // A real publication, written into a real diagnostic trace. + yield* runOrdinary( + [ + ``, + `pushed`, + ``, + ``, + ``, + ].join("\n"), + { root: first.root, cwd: first.checkout, journal: trace }, + ); + const published = git(["rev-parse", "HEAD"], first.checkout, first.home); + expect(git(["rev-parse", "traced-push"], remote, first.home)).toBe(published); + expect(yield* exists(trace)).toBe(true); + const written = yield* readTextFile(trace); + // The trace holds this run's own events, and none of them is the + // publication: an ordinary run journals no repository effect at all, so + // there is not even a record for a later run to misread as evidence. + expect(written.length).toBeGreaterThan(0); + expect(written).toContain("import_component"); + expect(written).not.toContain("git-push"); + expect(written).not.toContain("git_host"); + + // A new execution, on the same checkout, on the same branch, at the same + // commit — handed that exact file as its journal, and containing only a + // pull request. + const failure = yield* raisedValue( + runOrdinary(``, { + root: first.root, + cwd: first.checkout, + journal: trace, + }), + ); + expect(String(failure)).toContain("holds no successful result"); + + // The second run wrote its own events after the first run's, which is what + // a trace is: a file appended to, never a file read back. Nothing in it + // authorized anything, and there is still no publication recorded anywhere + // in it. + const after = yield* readTextFile(trace); + expect(after.startsWith(written)).toBe(true); + expect(after.length).toBeGreaterThan(written.length); + expect(after).not.toContain("git-push"); + expect(after).not.toContain("git_host"); + }); +}); + +/** Whatever this operation raised, as a value. */ +function* raisedValue(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error; + } + throw new Error("the operation did not fail"); +} diff --git a/packages/cli/tests/run-composition-nested.test.ts b/packages/cli/tests/run-composition-nested.test.ts new file mode 100644 index 00000000..a4b701db --- /dev/null +++ b/packages/cli/tests/run-composition-nested.test.ts @@ -0,0 +1,551 @@ +/** + * Tier ORC — a nested `host="run"` child under an ordinary run. + * + * A child is a root execution in a scope that does not descend from the + * document's, so everything it is given has to be given deliberately: its own + * provider instance, and the working directory the `` was written + * in. This file is about both, and about what must *not* cross that boundary. + * + * The root execution's own provider is asked separately, in + * `run-composition-deno.test.ts`. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { type Operation, scoped, until } from "effection"; +import { realpath } from "node:fs/promises"; +import { exists, readdir, readTextFile, writeTextFile } from "@effectionx/fs"; +import { spawnSync } from "node:child_process"; +import { join } from "node:path"; +import process from "node:process"; +import { API, useHostFiles } from "@executablemd/runtime"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { collect, execute, inlineSource, installAgentComponents } from "@executablemd/core"; +import { runCli } from "@executablemd/test-support/launch"; +import { useTempDirectory } from "@executablemd/test-support/temp"; +import { useCompositionComponents } from "@executablemd/workflow"; +import { useRunComposition } from "@executablemd/workflow/deno"; +import { FileStream } from "../src/file-stream.ts"; + +/** Git, with an environment a caller's own configuration cannot reach into. */ +function git(args: readonly string[], cwd: string, home: string): string { + const outcome = spawnSync("git", [...args], { + cwd, + env: { + ...(process.env.PATH === undefined ? {} : { PATH: process.env.PATH }), + HOME: home, + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_CONFIG_SYSTEM: "/dev/null", + GIT_CONFIG_NOSYSTEM: "1", + GIT_TERMINAL_PROMPT: "0", + LC_ALL: "C", + GIT_AUTHOR_NAME: "Fixture", + GIT_AUTHOR_EMAIL: "fixture@example.invalid", + GIT_COMMITTER_NAME: "Fixture", + GIT_COMMITTER_EMAIL: "fixture@example.invalid", + }, + stdio: ["ignore", "pipe", "pipe"], + encoding: "utf8", + }); + if (outcome.status !== 0) { + throw new Error(`git ${args.join(" ")} exited ${outcome.status}: ${outcome.stderr}`); + } + return outcome.stdout.trim(); +} + +/** A bare repository this suite can publish to, made from a real checkout. */ +function* useRemote(): Operation { + const home = yield* useTempDirectory("xmd-orc-remote-home-"); + const parent = yield* until(realpath(yield* useTempDirectory("xmd-orc-remote-"))); + const seed = join(parent, "seed"); + git(["init", "--initial-branch=main", seed], parent, home); + git(["commit", "--allow-empty", "-m", "first"], seed, home); + const bare = join(parent, "remote.git"); + git(["clone", "--bare", "--", seed, bare], parent, home); + return bare; +} + +/** A repository the command is "run in", and a managed root of this suite's own. */ +function* useAmbient( + locator?: string, +): Operation<{ checkout: string; root: string; home: string }> { + const home = yield* useTempDirectory("xmd-orc-home-"); + // Canonical, so what this fixture names and what Git reports are one string. + const parent = yield* until(realpath(yield* useTempDirectory("xmd-orc-ambient-"))); + const checkout = join(parent, "checkout"); + if (locator === undefined) { + git(["init", "--initial-branch=main", checkout], parent, home); + git(["commit", "--allow-empty", "-m", "first"], checkout, home); + } else { + git(["clone", "--", locator, checkout], parent, home); + } + const managed = yield* until(realpath(yield* useTempDirectory("xmd-orc-managed-"))); + return { checkout, root: join(managed, "repositories"), home }; +} + +/** Run one document under the ordinary provider, on a stream a caller chose. */ +function runOrdinary( + source: string, + options: { + root: string; + cwd: string; + journal?: string; + /** Installed after the components, where a provider's own middleware goes. */ + agent?: () => Operation; + }, +): Operation { + return scoped(function* () { + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return options.cwd; + }, + }, + { at: "min" }, + ); + yield* useHostFiles(); + yield* installAgentComponents(); + yield* useCompositionComponents(); + yield* useRunComposition({ root: options.root, cwd: options.cwd }); + if (options.agent !== undefined) { + yield* options.agent(); + } + // `--journal` is exactly this: the file-backed stream instead of the + // in-memory one, created by the command before the run begins. + const stream = + options.journal === undefined ? new InMemoryStream() : new FileStream(options.journal); + return yield* collect(yield* execute({ ...inlineSource(source), stream })); + }); +} + +describe("ORC19 — a nested run profile", () => { + /** + * The claims here are about a real `` child, so they + * are asked of a real `xmd run` — a subprocess, launched the way a person + * launches one. + * + * ## Why a subprocess, and why from nowhere + * + * A child is a root execution in a scope that does not descend from the + * document's, so it inherits no `API.Env` handler and its working directory + * is whatever the host installs for it. When that propagation breaks, the + * child falls back to the *process* directory — and an in-process suite's + * process directory is this repository. A regression would then discover this + * checkout as its ambient repository and operate on it: branches, worktrees + * and commits in the tree the suite is running from. + * + * So the process directory is a temporary directory that is not a Git + * checkout at all. Every repository, every managed root and every document is + * under a fixture-owned temporary directory, and `HOME` is one too — which is + * what moves `~/.xmd/repositories` out of the way, since a run takes its + * managed root from there and no option names another. A break cannot reach a + * shared checkout because, from where these processes stand, there is no + * checkout to reach: the child refuses for want of a repository. + * + * That refusal is the last test below, and it is what keeps the first one + * honest. Without it, "the child worked" would be equally well explained by + * the child having found a repository some other way. + * + * ## Why `` rather than `xmd test` + * + * Two of these claims are about what a *parent* holds while its child runs — + * publication evidence, and a lease. `xmd test` installs no repository + * provider for its own document, by design: only its children get one. So the + * parent here is an ordinary `xmd run`, which has a provider of its own, and + * `` turns on the harness for the region containing the children. + */ + interface Nested { + readonly checkout: string; + readonly home: string; + /** Where a run of this fixture keeps its managed checkouts. */ + readonly managed: string; + /** The process directory every run below is launched from. */ + readonly outside: string; + readonly documents: string; + } + + /** + * A repository to work in, a home to be nobody in, and a directory to stand + * in that is neither. + */ + function* useNested(locator?: string): Operation { + const ambient = yield* useAmbient(locator); + // An ordinary run commits as the invoking user and refuses when the host + // cannot say who that is, so the identity a child would use is configured + // here — in this fixture's `HOME`, never the developer's. + yield* writeTextFile( + join(ambient.home, ".gitconfig"), + ["[user]", "\tname = Nested Fixture", "\temail = nested@example.invalid", ""].join("\n"), + ); + return { + checkout: ambient.checkout, + home: ambient.home, + // Where `denoRunRepositories` puts them when nothing names a root, which + // is every `xmd run`. Fixture-owned because `HOME` is. + managed: join(ambient.home, ".xmd", "repositories"), + outside: yield* until(realpath(yield* useTempDirectory("xmd-orc-outside-"))), + documents: yield* until(realpath(yield* useTempDirectory("xmd-orc-documents-"))), + }; + } + + /** One `xmd run` of `source`, from a directory that is not a repository. */ + function* runNested( + source: string, + fixture: Nested, + expected: "passes" | "fails" = "passes", + ): Operation { + const document = join(fixture.documents, "nested.md"); + yield* writeTextFile(document, source); + const run = yield* runCli(["run", document], { + // The whole point: nothing about where this process stands names a + // repository, so only what the document says can put a child in one. + cwd: fixture.outside, + env: { HOME: fixture.home }, + timeout: 180_000, + }).join(); + const reported = `${run.stdout}\n${run.stderr}`; + const passed = run.code === 0; + if (passed !== (expected === "passes")) { + throw new Error(`xmd run exited ${run.code}, expected to ${expected}:\n${reported}`); + } + return reported; + } + + /** A child document, as one escaped `source` attribute value. */ + function child(source: string): string { + return JSON.stringify(source); + } + + /** The `` child, which reaches its repository ambiently. */ + const AMBIENT_WORKTREE = child('\n\n{w}\n'); + + it("stands the child where the document is, not where the process is", function* () { + const fixture = yield* useNested(); + + // The `` is the only thing that puts anything in a repository. If the + // contextual directory did not reach the child it would stand in + // `fixture.outside` and refuse — which is exactly what the last test here + // shows happens when the `` is absent. + const reported = yield* runNested( + [ + ``, + "", + "", + "", + '', + "", + ``, + '', + "", + // What it bound is a managed checkout under *this* run's root, so the + // child really resolved `` through a provider of its own + // rather than reporting a path it never made. + ``, + "", + "", + "", + "", + "", + "", + "", + "", + ].join("\n"), + fixture, + ); + expect(reported).not.toContain("not inside a Git checkout"); + + // And the worktree is on disk, belonging to the repository the document + // named — observed from outside the run that made it, so this is the state + // the child left rather than a line it printed. + const slots = join(fixture.managed, "worktrees"); + const [repository] = yield* readdir(slots); + const [slot] = yield* readdir(join(slots, repository ?? "")); + const bound = join(slots, repository ?? "", slot ?? "", "checkout"); + expect(yield* readTextFile(join(bound, ".git"))).toContain("gitdir:"); + expect(git(["rev-parse", "--abbrev-ref", "HEAD"], bound, fixture.home)).toBe("child"); + // It is a linked worktree of the ambient checkout the `` named, which + // is the whole claim: the child discovered its repository from the + // directory the *document* was standing in. + expect( + git(["rev-parse", "--path-format=absolute", "--git-common-dir"], bound, fixture.home), + ).toBe(join(fixture.checkout, ".git")); + }); + + it("does not let one child's Push authorize its sibling", function* () { + const remote = yield* useRemote(); + const fixture = yield* useNested(remote); + + // Both sides are children, and both reach the *same* repository — the + // ambient one, which they discover from the `` this `` is + // written in. Sequential sharing is what makes that work: caller-owned + // ambient Git is nobody's managed slot, so it carries no lease, and one + // execution can hand it to the next. + // + // The claim is that each execution gets a provider of its own, so evidence + // one earns does not authorize the next. The parent/child direction of the + // same boundary is asked separately, from a process standing in a + // repository. + const publishes = child( + [ + '', + 'first', + '', + '', + "", + "", + ].join("\n"), + ); + const asks = child('\n'); + + yield* runNested( + [ + ``, + "", + "", + "", + '', + "", + // The first child really publishes, through a provider of its own. + ``, + "", + "", + "", + // The second stands in the same repository, immediately after, and + // holds none of it. If Push evidence outlived one execution this would + // succeed. + ``, + "", + '', + "", + "", + "", + "", + "", + "", + "", + "", + ].join("\n"), + fixture, + ); + + // The first child's publication really happened, so what the sibling + // lacked is evidence rather than a branch: the remote carries the commit, + // observed from outside the run that made it. + expect(git(["rev-parse", "pushed-by-first"], remote, fixture.home)).not.toBe(""); + }); + + it("gives a child its own lease owner, and keeps the parent's when it ends", function* () { + const remote = yield* useRemote(); + const fixture = yield* useNested(remote); + + const selection = ``; + // The same repository and the same worktree name, so parent and child ask + // the operating system for one slot. + const asks = child( + `${selection}\n\n\n`, + ); + + yield* runNested( + [ + selection, + "", + // The parent takes the lease on `shared` and holds it for its whole + // run. + '', + "", + "", + "", + '', + "", + // A provider sharing the parent's held set would answer out of it and + // succeed without asking the operating system anything; one with an + // owner of its own asks, and is refused because the parent is still + // holding it. + ``, + "", + '', + "", + "", + // A second child, after the first has torn down. The parent's lease + // survived that teardown, so this one is refused for the same reason + // rather than finding the slot free. + ``, + "", + '', + "", + "", + "", + "", + "", + "", + "", + "", + ].join("\n"), + fixture, + ); + }); + + /** + * The same fixture, with the process standing *inside* a disposable checkout. + * + * The escape argument is different here and still holds: a regression in + * contextual-directory propagation reaches the process directory, and the + * process directory is a temporary clone this fixture made and owns. There is + * nothing shared to reach. The non-Git case above keeps the other half of the + * argument — that the propagation is real rather than incidentally agreeing + * with where the process happens to stand. + */ + function* useInRepository(): Operation { + const remote = yield* useRemote(); + const fixture = yield* useNested(remote); + // The process stands in the caller's own checkout, which is what an + // ordinary `xmd run` stands in. Nothing here is a managed slot, so nothing + // here is leased, and two executions can use it one after the other. + return { ...fixture, remote, outside: fixture.checkout }; + } + + /** One branch, published once, asked about from both directions. */ + const PUBLISHES = [ + '', + 'published', + '', + '', + "", + ]; + + /** + * `` reaches its Git host through `source.open()`, and the + * evidence gate runs ahead of it. The refusal says so itself — "Nothing was + * observed at the Git host, and no pull request was created" — so that + * sentence is the claim rather than an inference from it. + * + * The second assertion is the corroborating one: a local file origin is not + * a Git host, so an execution that had got past the gate would have failed + * with "no usable origin" instead. Its absence and the sentence's presence + * are two independent readings of the same ordering. + */ + const REFUSED_BEFORE_HOST_ACCESS = "Nothing was observed at the Git host"; + + function refusedBeforeHostAccess(binding: string): string[] { + return [ + ``, + ``, + ``, + ``, + ]; + } + + it("does not let a parent's Push authorize its child", function* () { + const fixture = yield* useInRepository(); + + // The parent publishes in the ambient checkout the process is standing in; + // the child, immediately after, asks for a pull request from that same + // checkout, origin, branch and head. Everything about the repository is + // identical between them. The only thing that differs is which provider + // holds the Push evidence. + yield* runNested( + [ + ...PUBLISHES, + "", + "", + "", + '', + "", + `\n')}} as="opened">`, + ...refusedBeforeHostAccess("opened"), + "", + "", + "", + "", + "", + "", + ].join("\n"), + fixture, + ); + + // The parent's publication was real, so what the child lacked is evidence. + expect(git(["rev-parse", "shared-head"], fixture.remote, fixture.home)).toBe( + git(["rev-parse", "HEAD"], fixture.checkout, fixture.home), + ); + }); + + it("does not let a child's Push authorize its parent", function* () { + const fixture = yield* useInRepository(); + + // The other direction, in the same checkout. The child publishes and tears + // down; the parent — whose provider has been installed the whole time — + // then asks for a pull request from the head its own child just pushed. + // + // The parent's refusal ends the document, which is what a refusal at + // document level is supposed to do, so this run is expected to fail and the + // refusal is read out of what it reported. The child's test block runs + // first and is reported before it. + const reported = yield* runNested( + [ + "", + "", + '', + "", + ``, + "", + "", + "", + "", + "", + "", + "", + '', + "", + ].join("\n"), + fixture, + "fails", + ); + + // The parent is refused for want of evidence, and before its Git host is + // reached. + expect(reported).toContain("holds no successful"); + expect(reported).toContain(REFUSED_BEFORE_HOST_ACCESS); + expect(reported).not.toContain("no usable origin"); + // The child's assertions passed, so the publication it was refused credit + // for really happened. + expect(reported).not.toContain("❌"); + expect(git(["rev-parse", "shared-head"], fixture.remote, fixture.home)).toBe( + git(["rev-parse", "HEAD"], fixture.checkout, fixture.home), + ); + }); + + it("refuses outside a repository when nothing places the child in one", function* () { + const fixture = yield* useNested(); + + // The same child as the first test, with the `` removed and nothing + // else changed. This is what a break in contextual-directory propagation + // looks like from the child's side — and it is a refusal, in a temporary + // directory, rather than work done in whatever checkout the process + // happened to be launched from. + yield* runNested( + [ + "", + "", + '', + "", + ``, + "", + '', + "", + "", + "", + "", + "", + "", + ].join("\n"), + fixture, + ); + + // Nothing was checked out for it. The managed root itself exists — every + // run creates one before a document expands — so what says the child did no + // work is that it never reached a repository to make a slot under. + expect(yield* exists(fixture.managed)).toBe(true); + expect(yield* exists(join(fixture.managed, "worktrees"))).toBe(false); + }); +}); diff --git a/packages/cli/tests/run-composition.test.ts b/packages/cli/tests/run-composition.test.ts new file mode 100644 index 00000000..a7da7254 --- /dev/null +++ b/packages/cli/tests/run-composition.test.ts @@ -0,0 +1,282 @@ +/** + * Tier ORC — how the command line assembles repository operations. + * + * Three claims, and they are about opposite things. Describing the vocabulary + * must reach nothing; one runtime *operates* it; and one only *describes* it. + * All three have to be true at once, so a document written for `xmd run` + * resolves the same thirteen names everywhere, and on a runtime that operates + * none of them every one reports an absent provider before a lock, a + * credential, a subprocess or a request exists. + * + * The declarations are the same array in every case, which is why there is no + * fourth thing to keep in agreement. + * + * Everything here runs under Deno, Node and Bun. The parity claim is not one to + * defer to CI: what it asserts is that a runtime with no operational provider + * still describes and resolves the whole language. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { scoped, suspend, type Operation } from "effection"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { collect, execute, inlineSource, registerComponents } from "@executablemd/core"; +import { API, Service, useHostFiles } from "@executablemd/runtime"; +import type { RuntimeFetchResponse } from "@executablemd/runtime"; +import { exists, readdir, readTextFile, writeTextFile } from "@effectionx/fs"; +import { useTempDirectory } from "@executablemd/test-support/temp"; +import { join } from "node:path"; +import { COMPOSITION_REGISTRATIONS } from "@executablemd/workflow"; +import { syntaxCatalog, useRunProfileRegistry } from "../src/syntax.ts"; +import { DEFAULT_REPOSITORY_ROOT, unsupportedRepositories } from "../src/run-repositories.ts"; + +/** Every element an author can write that needs a repository provider. */ +const OPERATIONS: readonly { readonly name: string; readonly source: string }[] = [ + { + name: "Repository", + source: ``, + }, + { name: "Worktree", source: `` }, + { name: "Git.Switch", source: `` }, + { name: "Git.Add", source: `` }, + { name: "Git.Commit", source: `` }, + { name: "Git.Push", source: `` }, + { name: "PullRequest", source: `` }, + { + name: "PullRequest.Reviews", + source: ``, + }, + { + name: "PullRequest.Comments", + source: ``, + }, + { + name: "PullRequest.Checks", + source: ``, + }, + { name: "Issue", source: `` }, +]; + +/** The thirteen names #643 settled, exactly as a document writes them. */ +const COMPOSITION_NAMES = [ + "Repository", + "Worktree", + "Dir", + "Git.Switch", + "Git.Add", + "Git.Commit", + "Git.Push", + "PullRequest", + "PullRequest.Reviews", + "PullRequest.Comments", + "PullRequest.Checks", + "IssueTracker", + "Issue", +] as const; + +/** + * Run one element with the declarations registered and no provider installed — + * which is exactly what Node and Bun assemble. + */ +function ordinaryWithoutProvider(source: string, cwd: string): Operation { + return scoped(function* () { + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return cwd; + }, + }, + { at: "min" }, + ); + yield* useHostFiles(); + yield* registerComponents(COMPOSITION_REGISTRATIONS); + yield* unsupportedRepositories(); + return yield* collect( + yield* execute({ ...inlineSource(source), stream: new InMemoryStream() }), + ); + }); +} + +describe("ORC1 — describing the vocabulary reaches nothing", () => { + it("builds the catalog without a subprocess, a service, a request or a lock", function* () { + const managed = yield* useTempDirectory("xmd-orc1-managed-"); + const reached: string[] = []; + + const catalog = yield* scoped(function* () { + // Tripwires at every host boundary the provider would use, installed + // beneath everything so nothing can answer ahead of them. Each records + // rather than throwing, so a failure says which boundary was reached. + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *command(): Operation { + reached.push("command"); + return []; + }, + // deno-lint-ignore require-yield + *cwd(): Operation { + return managed; + }, + }, + { at: "min" }, + ); + yield* API.Fetch.around( + { + // deno-lint-ignore require-yield + *fetch(): Operation { + reached.push("fetch"); + throw new Error("the catalog reached the network"); + }, + }, + { at: "min" }, + ); + yield* Service.around( + { + *start(): Operation { + reached.push("service"); + throw new Error("the catalog started a service"); + // deno-lint-ignore no-unreachable + yield* suspend(); + }, + }, + { at: "min" }, + ); + return yield* syntaxCatalog([]); + }); + + // The whole vocabulary is described. + const builtIn = catalog.categories[1].entries.map((entry) => entry.name); + for (const name of COMPOSITION_NAMES) { + expect(builtIn).toContain(name); + } + + // And nothing was reached to describe it: no command was built for a + // subprocess, no request was sent, no service was started. + expect(reached).toEqual([]); + // No managed root, no slot, no lock sidecar — nothing was created at all. + expect(yield* readdir(managed)).toEqual([]); + }); + + it("leaves every repository operation unprovided after inspection", function* () { + // Registering the declarations installs no provider: the Apis still answer + // with their own defaults, which is what a catalog is allowed to leave + // behind. + yield* useRunProfileRegistry(); + const failure = yield* raisedValue( + collect(yield* execute({ ...inlineSource(``), stream: new InMemoryStream() })), + ); + // `` has no lexical Repository, so the first thing it asks for + // is the ambient one — and that is the Api reporting absence. + expect(String(failure)).toContain("no Repository composition provider is installed"); + }); +}); + +describe("ORC2 — one language, described everywhere and operated somewhere", () => { + it("registers the same thirteen declarations the syntax catalog describes", function* () { + // The array itself, rather than a second list: `useRunProfileRegistry()`, + // `installDocumentComponents()` and `useCompositionComponents()` all + // consume this one, so there is nothing for a runtime to disagree about. + expect(COMPOSITION_REGISTRATIONS).toHaveLength(13); + expect([...COMPOSITION_REGISTRATIONS].map((registration) => registration.name).sort()).toEqual( + [...COMPOSITION_NAMES].sort(), + ); + + // And the catalog every runtime builds describes each of them completely. + const catalog = yield* scoped(() => syntaxCatalog([])); + const builtIn = catalog.categories[1].entries; + for (const name of COMPOSITION_NAMES) { + const entry = builtIn.find((candidate) => candidate.name === name); + expect(`${name}: ${entry?.description !== undefined}`).toBe(`${name}: true`); + expect(`${name}: ${(entry?.forms?.length ?? 0) > 0}`).toBe(`${name}: true`); + } + }); + + it("reports an absent provider for every repository operation, and mutates nothing", function* () { + for (const operation of OPERATIONS) { + // The working directory is this repository's own checkout, so a provider + // that *did* discover an ambient repository would find one — and the + // refusal below would then be about something else. + const failure = yield* raisedValue(ordinaryWithoutProvider(operation.source, ".")); + // The element's name travels with the assertion, so a failure says which + // of the twelve reported something else. + const reported = `${operation.name}: ${String(failure)}`; + expect(reported).toMatch( + /provider is not installed|no Repository composition provider|no Git composition provider|no issue provider|no Issue provider|no pull-request provider/, + ); + } + }); + + it("leaves working, because it needs no provider at all", function* () { + const rendered = yield* ordinaryWithoutProvider( + ['', "", "inside", "", ""].join("\n"), + ".", + ); + expect(String(rendered)).toContain("inside"); + }); +}); + +describe("ORC2 — Dir is operational where the repository provider is not", () => { + // The one composition element that needs no repository provider. Every other + // name here reports an absent provider on this runtime; `` reaches the + // host `API.Files` provider instead, which all four entrypoints install — so + // it creates its directory and runs its content on Deno, Node and Bun alike. + // + // This runs in the portable suite deliberately: the operation is the same one + // the tier suites drive under Deno, and the claim being made here is that + // nothing about it is Deno's. + it("creates the directory and runs its content with no repository provider", function* () { + const workspace = yield* useTempDirectory("xmd-orc2-dir-"); + + const rendered = String( + yield* ordinaryWithoutProvider( + '\n\nlanded\n\nINSIDE\n\n\n', + workspace, + ), + ); + + expect(rendered).toContain("INSIDE"); + expect(yield* exists(join(workspace, "made", "deep"))).toBe(true); + expect(yield* readTextFile(join(workspace, "made", "deep", "inside.md"))).toBe("landed"); + }); + + // And the refusal half, on the same runtime: a non-directory is refused + // rather than answered, so `` being operational is not `` being + // permissive. + it("refuses a non-directory target without a repository provider", function* () { + const workspace = yield* useTempDirectory("xmd-orc2-dir-refusal-"); + yield* writeTextFile(join(workspace, "occupied"), "a file"); + + const rendered = String( + yield* ordinaryWithoutProvider( + '\n\n\nINSIDE\n\n\n\n', + workspace, + ), + ); + + expect(rendered).toContain("not a directory"); + expect(rendered).not.toContain("INSIDE"); + expect(yield* readTextFile(join(workspace, "occupied"))).toBe("a file"); + }); +}); + +describe("ORC2 — where the managed root is", () => { + it("names ~/.xmd/repositories and nothing a document can influence", function* () { + expect(DEFAULT_REPOSITORY_ROOT.endsWith("/.xmd/repositories")).toBe(true); + // Describing the vocabulary and refusing an operation both leave it exactly + // as they found it — including not existing. + const before = yield* exists(DEFAULT_REPOSITORY_ROOT); + yield* raisedValue(ordinaryWithoutProvider(``, ".")); + expect(yield* exists(DEFAULT_REPOSITORY_ROOT)).toBe(before); + }); +}); + +/** Whatever this operation raised, as a value. */ +function* raisedValue(operation: Operation): Operation { + try { + yield* operation; + } catch (error) { + return error; + } + throw new Error("the operation did not fail"); +} diff --git a/packages/cli/tests/run-deadline.test.ts b/packages/cli/tests/run-deadline.test.ts index 75c3b3cf..fe6a63f0 100644 --- a/packages/cli/tests/run-deadline.test.ts +++ b/packages/cli/tests/run-deadline.test.ts @@ -21,6 +21,7 @@ import { API, Config, Service, fetch, useHostFiles } from "@executablemd/runtime import { runXmd } from "../src/cli.ts"; import { SOURCE_UPGRADE } from "./support/upgrade-assembly.ts"; +import { unsupportedRepositories } from "../src/run-repositories.ts"; /** * The exit continuation `exit()` reaches for. `main()` installs one under this * name; a suite that drives `runXmd` directly installs its own so a command's @@ -124,6 +125,7 @@ function* drive(args: string[], options: DriveOptions = {}): Operation { } }, SOURCE_UPGRADE, + unsupportedRepositories, ); return { status, stderr, reads, events, serviceInstalled, deadlineReads }; diff --git a/packages/cli/tests/support/run-markdown-tier.ts b/packages/cli/tests/support/run-markdown-tier.ts index e420761f..f63638bc 100644 --- a/packages/cli/tests/support/run-markdown-tier.ts +++ b/packages/cli/tests/support/run-markdown-tier.ts @@ -35,6 +35,7 @@ import { useBunService } from "../../src/bun-service.ts"; import { useDenoService } from "../../src/deno-service.ts"; import { useNodeService } from "../../src/node-service.ts"; +import { unsupportedRepositories } from "../../src/run-repositories.ts"; /** The native service adapter the entrypoint for this runtime installs. */ const SERVICES = { bun: useBunService, @@ -81,6 +82,9 @@ export function runMarkdownTier(document: string): Operation { return renderSyntaxMarkdown(yield* syntaxCatalog(["components", "."])); }, }), + // This harness runs Markdown tiers, not repository work: a child that + // asked for a checkout is told there is no provider. + installRepositories: unsupportedRepositories, }); const execution = yield* executeInstalled({ path: document, stream: new InMemoryStream() }, [ testHarnessInstallation(testingHost), diff --git a/packages/cli/tests/syntax-cli.test.ts b/packages/cli/tests/syntax-cli.test.ts index cd327325..909b7b7e 100644 --- a/packages/cli/tests/syntax-cli.test.ts +++ b/packages/cli/tests/syntax-cli.test.ts @@ -132,6 +132,23 @@ function catalogWith(props: PropsSchema): SyntaxCatalog { }; } +/** The thirteen names #643 settled, exactly as a document writes them. */ +const COMPOSITION_NAMES = [ + "Repository", + "Worktree", + "Dir", + "Git.Switch", + "Git.Add", + "Git.Commit", + "Git.Push", + "PullRequest", + "PullRequest.Reviews", + "PullRequest.Comments", + "PullRequest.Checks", + "IssueTracker", + "Issue", +] as const; + describe("Tier SX — the run profile the command describes", () => { it("SX1: names core, Agent, testing and web defaults, and ", function* () { const catalog = yield* syntaxCatalog([]); @@ -158,6 +175,58 @@ describe("Tier SX — the run profile the command describes", () => { expect(catalog.categories[2].entries).toEqual([]); }); + it("ORC1: names all thirteen repository-composition components, with contracts", function* () { + const catalog = yield* syntaxCatalog([]); + const entries = catalog.categories[1].entries; + const builtIn = names(entries); + + for (const name of COMPOSITION_NAMES) { + expect(builtIn).toContain(name); + } + + // A complete contract, not a bare name: every one of them says what it is + // for, which forms it takes, and what its props are. + for (const name of COMPOSITION_NAMES) { + const entry = entries.find((candidate) => candidate.name === name); + expect(entry?.description ?? "").not.toBe(""); + expect(entry?.forms?.length ?? 0).toBeGreaterThan(0); + // Registered rather than reserved, which is what makes a repository + // component of the same name win. + expect(entry?.origin).toEqual({ + kind: "registered", + origin: "@executablemd/workflow/composition", + reserved: false, + }); + } + + // The ones that produce a value say what `as` binds; the ones that render + // nothing and produce nothing do not pretend to. + expect(entries.find((entry) => entry.name === "Git.Commit")?.as).toContain("object id"); + expect(entries.find((entry) => entry.name === "PullRequest.Reviews")?.as).toContain("Required"); + expect(entries.find((entry) => entry.name === "Git.Push")?.as).toBe(undefined); + }); + + it("ORC1: a repository component of the same name shadows the default", function* () { + yield* useWorkspace( + { + "Worktree.md": [ + "---", + "description: the repository's own Worktree", + "---", + "", + "shadowed", + "", + ].join("\n"), + }, + function* (dir) { + const catalog = yield* syntaxCatalog([dir]); + const provided = catalog.categories[2].entries.find((entry) => entry.name === "Worktree"); + expect(provided).toBeDefined(); + expect(names(catalog.categories[1].entries)).not.toContain("Worktree"); + }, + ); + }); + it("SX2: documents every complete built-in in the profile", function* () { const catalog = yield* syntaxCatalog([]); const undocumented = catalog.categories[1].entries.filter( diff --git a/packages/cli/tests/targets-cli.test.ts b/packages/cli/tests/targets-cli.test.ts index 14541a56..f2fe1d9d 100644 --- a/packages/cli/tests/targets-cli.test.ts +++ b/packages/cli/tests/targets-cli.test.ts @@ -22,6 +22,7 @@ import { runCli } from "@executablemd/test-support/launch"; import { runXmd } from "../src/cli.ts"; import { SOURCE_UPGRADE } from "./support/upgrade-assembly.ts"; +import { unsupportedRepositories } from "../src/run-repositories.ts"; function* useFixture( files: Record, body: (dir: string) => Operation, @@ -587,6 +588,7 @@ function* replacingRun( }); }, SOURCE_UPGRADE, + unsupportedRepositories, ); return { status, stderr, serviceInstalled, serviceStarted, documentReads, reads }; @@ -761,6 +763,7 @@ function* helpRun(args: string[], cwd: string): Operation { serviceInstalled = true; }, SOURCE_UPGRADE, + unsupportedRepositories, ); return { status, stdout, stderr, serviceInstalled }; diff --git a/packages/cli/tests/testing-execution-host.test.ts b/packages/cli/tests/testing-execution-host.test.ts index de3e2fa6..34571d81 100644 --- a/packages/cli/tests/testing-execution-host.test.ts +++ b/packages/cli/tests/testing-execution-host.test.ts @@ -27,6 +27,7 @@ import { runCli } from "@executablemd/test-support/launch"; import { testingExecutionHost } from "../src/testing-host.ts"; import { planComponentDescription } from "../src/plan-component.ts"; +import { unsupportedRepositories } from "../src/run-repositories.ts"; function doc(...lines: string[]): string { return `${lines.join("\n")}\n`; } @@ -501,6 +502,7 @@ describe("deterministic dependencies declared for a nested run", () => { secretDetection: true, // deno-lint-ignore require-yield installService: function* (): Operation {}, + installRepositories: unsupportedRepositories, testAgentWorker: Err(new Error("xmd command not installed")), // The run profile's own Component travels to every child, and this case is // about the relaunch it cannot perform rather than about ``. @@ -518,6 +520,10 @@ describe("deterministic dependencies declared for a nested run", () => { configuration: [{ kind: "test-agent", defaultAgent: "test", scenarios: [] }], }, run: undefined, + // Unobservable here: the refusal is reached before the child stands + // anywhere, and `tmpdir()` is a directory this assertion cannot + // depend on having any particular contents. + cwd: tmpdir(), // deno-lint-ignore require-yield *chunk(): Operation {}, }); diff --git a/packages/cli/tests/workflow-agent.test.ts b/packages/cli/tests/workflow-agent.test.ts index 3fd4e397..fed965e3 100644 --- a/packages/cli/tests/workflow-agent.test.ts +++ b/packages/cli/tests/workflow-agent.test.ts @@ -719,7 +719,7 @@ describe("Tier WAL — the workflow Agent observation loop", () => { expect(recorded).toHaveLength(1); const policy = JSON.stringify(recorded[0]); expect(policy).toContain("File:write"); - expect(policy).toContain("@executablemd/workflow/composition#Dir"); + expect(policy).toContain("@executablemd/workflow/composition/dir-v2#Dir"); expect(policy).toContain('"allow":["write"]'); // And the change is in the run's own Workspace, where an ordinary read // beneath the fragment's own directory finds it. Anchored on the diff --git a/packages/cli/tests/workflow-suspension.test.ts b/packages/cli/tests/workflow-suspension.test.ts index 9a14429b..2b1af9d6 100644 --- a/packages/cli/tests/workflow-suspension.test.ts +++ b/packages/cli/tests/workflow-suspension.test.ts @@ -192,6 +192,47 @@ interface Retained { readonly rootCloses: number; } +/** Every journal record, as stored. */ +function readRecords(path: string): string[] { + const database = new DatabaseSync(path, { readOnly: true }); + try { + return database + .prepare("SELECT record FROM journal_events ORDER BY sequence") + .all() + .map((row) => String(row["record"])); + } finally { + database.close(); + } +} + +/** + * Rewrite every retained record through `edit`. + * + * A continuation admitted under a previous release cannot be produced by + * running this one, so the retained history is edited into the shape that + * release left behind. Nothing else about the run changes — the same source, + * roots and selection — so the identity is the only thing a resume can + * disagree about. + */ +function rewriteRecords(path: string, edit: (record: string) => string): void { + const database = new DatabaseSync(path); + try { + const rows = database + .prepare("SELECT event_id AS id, record FROM journal_events") + .all() + .map((row) => ({ id: String(row["id"]), record: String(row["record"]) })); + const update = database.prepare("UPDATE journal_events SET record = ? WHERE event_id = ?"); + for (const row of rows) { + const edited = edit(row.record); + if (edited !== row.record) { + update.run(edited, row.id); + } + } + } finally { + database.close(); + } +} + function retained(path: string): Retained { const database = new DatabaseSync(path, { readOnly: true }); try { @@ -1467,121 +1508,140 @@ the retained note expect(answers(path)[0]?.state).toBe("consumed"); }); } -}); -/** - * The one question a workflow asks, and the smallest document that asks it. - * - * No effect precedes the wait: what is under test is which provider answers - * ``, and an effect ahead of it would only add ways for the case to - * fail before reaching the boundary it exists to cross. - */ -const ELICITATION = ` -Proceed with the change? - - -decision: {decision.proceed} -`; - -/** A HOME of this case's own, so nothing reaches the developer's configuration. */ -function useIsolatedHome(): Operation { - return resource(function* (provide) { - const home = yield* until(mkdtemp(join(tmpdir(), "xmd-wfs-home-"))); - yield* ensure(function* () { - yield* rm(home, { recursive: true, force: true }); + // WGAC17: `allow={["write"]}` admits the versioned paired ``, and that + // entry now authorizes persistent recursive creation. The former identity + // authorized placement that created nothing, so a run admitted under it must + // not silently receive the wider authority. + // + // Two halves, and neither is evidence alone. The control shows the current + // admission really does execute its generated `` and really does resume + // — without it, the refusal below could be a run that was broken for some + // other reason. The refusal shows the former identity is what stops it. + function* wgac17( + store: string, + ): Operation<{ runId: string; path: string; rendered: string[]; calls: AgentCalls }> { + const fixture = yield* useCheckpointFixture(bundledRoot("", "Propose"), { + Propose: `\\n\\nfrom the fragment\\n\\n"} allow={["write"]} as="observed" />\n`, }); - yield* provide(home); - }); -} + yield* useRepositoryGit(fixture.repository); -/** - * How long one `xmd workflow` invocation may take before it is abandoned. - * - * Bounded rather than left to the default, because the failure this case - * guards against is a *wait*: with a Web provider assembled underneath the - * workflow's own, the start opens a loopback form and blocks for a reader who - * is not coming. `runCli` reports what each channel received before the - * deadline, so an abandoned run still names the form it opened. - */ -const INVOCATION_LIMIT = 40_000; + const calls: AgentCalls = { prompts: [] }; + const rendered: string[] = []; + const started = yield* invoke( + { ...REQUEST, action: "start" }, + yield* startFor(fixture), + productionHost(store, calls), + pinnedBody(rendered), + ); + expect(started.exitCode).toBe(2); + const runId = String(started.written.err.find((line) => line.startsWith("workflow run: "))) + .slice("workflow run: ".length) + .trim(); + const path = workflowRunPath(store, runId); -/** What one `xmd workflow` line published, or nothing when it published none. */ -function published(stderr: string, label: string): string | undefined { - const prefix = `workflow ${label}: `; - const line = stderr.split("\n").find((entry) => entry.startsWith(prefix)); - return line?.slice(prefix.length).trim(); -} + const suspended = retained(path); + expect(suspended.status).toBe("suspended"); + // The admission carries the versioned identity. + expect(readRecords(path).some((record) => record.includes("dir-v2#Dir"))).toBe(true); + + // And the generated `` really executed before the run suspended: the + // retained file effects hold an ensure for `/generated` ahead of the nested + // write beneath it. Read from the committed order rather than from the + // fragment's text, because the text says what was asked for and the order + // says what happened. + const files = orderedEffects(path) + .filter((effect) => effect.type === FILE_EFFECT) + .map((effect) => effect.name); + const ensured = files.findIndex( + (name) => name.startsWith("ensure-directory:") && name.endsWith(":/generated"), + ); + const wrote = files.findIndex( + (name) => name.startsWith("write:") && name.endsWith(":/generated/inside.md"), + ); + expect(ensured).toBeGreaterThanOrEqual(0); + expect(wrote).toBeGreaterThan(ensured); -describe("a workflow elicitation through the shipped CLI assembly", () => { - it("suspends, is answered and resumes without ever opening a browser form", function* () { - // Launched as a process, because the subject is the assembly the entrypoint - // builds around `runScopedDocument`. Calling `runWorkflow()` or - // `executeInstalled()` from here would install the components this suite - // chose and never reach the boundary under test. - const runs = yield* useRunStore(); - const home = yield* useIsolatedHome(); - const fixture = yield* useCheckpointFixture(ELICITATION); - - const xmd = (args: string[]) => - runCli(args, { - cwd: fixture.repository, - env: { HOME: home, XMD_WORKFLOW_RUNS: runs }, - timeout: INVOCATION_LIMIT, - }); + yield* manage( + { + action: "answer", + runId, + suspensionId: suspended.requests[0] ?? "", + value: { proceed: true }, + secretDetection: true, + }, + productionHost(store, calls), + ); + return { runId, path, rendered, calls }; + } - const started = yield* xmd(["workflow", "start", "workflow.md"]).join(); + it("WGAC17: the current admission executes Dir, resumes, and publishes nothing further", function* () { + const store = yield* useRunStore(); + const { runId, path, rendered, calls } = yield* wgac17(store); + const atSuspension = counts(path); + const rootsAtSuspension = workspaceRootState(path); - // Suspension is its own process outcome, and the pair a caller needs in - // order to answer the run reaches standard error beside it. - expect(started.code).toBe(2); - expect(published(started.stderr, "status")).toBe("suspended"); - const runId = published(started.stderr, "run"); - expect(runId).toBeDefined(); - const suspensionId = published(started.stderr, "suspension"); - expect(suspensionId).toBeDefined(); + const resumed = yield* invoke( + { ...REQUEST, action: "resume", target: runId }, + undefined, + productionHost(store, calls), + pinnedBody(rendered), + ); - // Nobody was asked anything: no loopback form was announced and no browser - // launch was attempted, on either channel. - const captured = `${started.stdout}\n${started.stderr}`; - expect(captured).not.toContain("http://127.0.0.1:"); - expect(captured).not.toContain("could not open a browser automatically"); + // It really resumes. + expect(resumed.exitCode).toBe(0); + expect(retained(path).status).toBe("completed"); + expect(rendered).toHaveLength(1); + // And the resume publishes nothing further: no additional effect, and no + // new Workspace root. + // + // That is exactly what these two observables support, and no more. Root + // invariance does not prove that no low-level ensure ran — an ensure + // finding the directory already there would publish no root either. The + // low-level call count is WF26's, which decorates `mkdir` and counts it + // directly; it is not reachable from this harness, because the CLI host + // fixes the private workspace options to `{}` (`run-host.ts`) and threading + // a decorator through would mean changing a production signature for a + // test. + // + // What this case establishes is the pair WF26 cannot: that the generated + // `` effect occurred under the current admission, and that resuming + // that admission publishes no further effect or root. + expect(workspaceRootState(path)).toEqual(rootsAtSuspension); + expect(counts(path)).toEqual(atSuspension); + }); - const path = workflowRunPath(runs, String(runId)); - const suspended = retained(path); - expect(suspended.status).toBe("suspended"); - expect(suspended.requests).toHaveLength(1); - expect(suspended.rootCloses).toBe(0); - // The wait the caller was told to answer is the wait the run retained. - expect(suspensionId).toBe(suspended.requests[0]); - - // The delivery retains a value and moves nothing else. - const delivered = yield* xmd([ - "workflow", - "answer", - String(runId), - String(suspensionId), - '{"proceed":true}', - ]).join(); - expect(delivered.code).toBe(0); - expect(delivered.stdout.trim()).toBe(`workflow answer: ${runId} (${suspensionId})`); - expect(retained(path)).toEqual(suspended); - expect(answers(path)).toEqual([ - { - suspensionId: String(suspensionId), - state: "pending", - answer: JSON.stringify({ proceed: true }), - }, - ]); + it("WGAC17: a continuation retaining the former Dir identity refuses before generated execution and mutation", function* () { + const store = yield* useRunStore(); + const { runId, path, rendered, calls } = yield* wgac17(store); + + // The run becomes one admitted under the former identity. Nothing else + // changes — same source, roots and selection — so the identity is the only + // thing the resume can disagree about. + rewriteRecords(path, (record) => + record.replaceAll( + "@executablemd/workflow/composition/dir-v2#Dir", + "@executablemd/workflow/composition#Dir", + ), + ); + const beforeResume = counts(path); + const rootsBeforeResume = workspaceRootState(path); - // An ordinary resume spends the answer once and reaches the authored value. - const resumed = yield* xmd(["workflow", "resume", String(runId)]).join(); - expect(resumed.code).toBe(0); - expect(published(resumed.stderr, "status")).toBe("completed"); - expect(resumed.stdout).toContain("decision: true"); + const resumed = yield* invoke( + { ...REQUEST, action: "resume", target: runId }, + undefined, + productionHost(store, calls), + pinnedBody(rendered), + ); - const completed = retained(path); - expect(completed.status).toBe("completed"); - expect(completed.requests).toEqual(suspended.requests); - expect(answers(path)[0]?.state).toBe("consumed"); + // Refused, and before anything generated ran: no further effect of any kind + // was committed and no Workspace root was published. + expect(resumed.exitCode).not.toBe(0); + expect(counts(path)).toEqual(beforeResume); + expect(workspaceRootState(path)).toEqual(rootsBeforeResume); + // Nothing after the wait reached the document either. + expect(rendered).toEqual([]); + // And the refusal does not publish which identity moved. + expect(resumed.written.err.join("\n")).not.toContain("dir-v2"); }); }); diff --git a/packages/core/mod.ts b/packages/core/mod.ts index 53b9e61b..d31eb6d1 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -348,6 +348,17 @@ export { createSecretScanner } from "./src/secrets/scanner.ts"; export type { SecretScanner } from "./src/secrets/scanner.ts"; export { SecretDetectedError, SecretScannerError } from "./src/secrets/findings.ts"; export type { SecretFinding } from "./src/secrets/findings.ts"; +/** + * The engine's `API.Files` door, for a component that does not live in core. + * + * `` is the composition package's, and it performs a document filesystem + * act like any other component. It must reach the Api the same way core's own + * components do — through the call that converts an illegal throw into a + * failure the engine can fence — rather than by holding the Api itself, which + * is what "a provider may fail and may not throw" means in practice. + */ +export { ensureDirectory } from "./src/files.ts"; + export { scanFiles } from "./src/secrets/files.ts"; export type { FileSecretFinding } from "./src/secrets/files.ts"; diff --git a/packages/core/src/files.ts b/packages/core/src/files.ts index 7c8f698d..0ba4f39a 100644 --- a/packages/core/src/files.ts +++ b/packages/core/src/files.ts @@ -321,6 +321,26 @@ export function deleteFile(input: FilePathInput): Operation> { return unit(Files.operations.deleteFile(input), { operation: "delete", phase: "access" }); } +/** + * Make this authored path name a directory, with nothing coming back. + * + * Unit for the same reason `deleteFile` is: what the caller asked for is the + * directory's existence, and existence is the whole answer. A receipt saying + * whether this call created it or found it would be a difference the operation + * deliberately does not offer, and a caller that could branch on it would be + * branching on a race. + * + * `access` is the fallback phase, because reaching the directory is the step + * this operation exists for — a refusal the vocabulary does not recognize + * happened while trying to make it, not while reading the path. + */ +export function ensureDirectory(input: FilePathInput): Operation> { + return unit(Files.operations.ensureDirectory(input), { + operation: "ensure-directory", + phase: "access", + }); +} + export function readFileText(input: FilePathInput): Operation> { return outcome(Files.operations.readTextFile(input), { operation: "read", diff --git a/packages/core/tests/files-fatal.test.ts b/packages/core/tests/files-fatal.test.ts index e5e177e3..fe925ac4 100644 --- a/packages/core/tests/files-fatal.test.ts +++ b/packages/core/tests/files-fatal.test.ts @@ -20,6 +20,8 @@ import { expect } from "@executablemd/test-support/expect"; import { ensure, Err, Ok, resource, scoped, spawn, suspend, until, withResolvers } from "effection"; import type { Operation, Result } from "effection"; import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { ensureDirectory } from "../src/files.ts"; +import { registerComponents } from "../src/components/registration.ts"; import { API, FILES_ERROR, @@ -258,9 +260,41 @@ describe("Tier FF — Files infrastructure failure", () => { }); // FF2: the same for every other form. Each stops at its first Files call. - it("FF2: read, delete, Glob and TempDir all stop at their first provider call", function* () { + // + // `` belongs to `@executablemd/workflow`, which depends on core rather + // than the other way round, so the element itself cannot be expanded here. + // What can be — and what FF2 is actually about — is the one call it makes: + // `ensureDirectory` through the engine's door, with no provider installed. + // `` performs exactly that and nothing else, so an absent provider + // reaches this suite by the same path the real component would take. That the + // shipped `` makes this call and no other is asserted where `` + // lives. + it("FF2: read, delete, ensureDirectory, Glob and TempDir all stop at their first provider call", function* () { const dir = yield* useFixture(); + const ensures = function* (): Operation { + yield* registerComponents([ + { + name: "Ensures", + origin: "test://ensures", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + yield* ensureDirectory({ cwd: dir, path: "made" }); + return "REACHED"; + }, + }, + ]); + }; + + const absent = yield* run(dir, "\n\nAFTER", ensures); + expect(absent.ok).toBe(false); + expect(parseFilesFatal(fatalCause(absent.error))?.kind).toBe("provider-unavailable"); + expect(absent.output).not.toContain("AFTER"); + // The operation is what failed, not the element around it: nothing after the + // call inside the component ran either. + expect(absent.output).not.toContain("REACHED"); + expect(yield* exists(join(dir, "made"))).toBe(false); + for (const source of [ '\n\nAFTER', // Deletion has no preliminary check to fail at, so its first and only diff --git a/packages/runtime/files.ts b/packages/runtime/files.ts index 1fddb624..f4207dc4 100644 --- a/packages/runtime/files.ts +++ b/packages/runtime/files.ts @@ -116,12 +116,19 @@ const REASONS: readonly FilesReason[] = [ ]; /** The operations whose failure carries no commit outcome. */ -export type FilesOperation = "check-file-path" | "read" | "delete" | "glob" | "temporary-directory"; +export type FilesOperation = + | "check-file-path" + | "read" + | "delete" + | "ensure-directory" + | "glob" + | "temporary-directory"; const OPERATIONS: readonly FilesOperation[] = [ "check-file-path", "read", "delete", + "ensure-directory", "glob", "temporary-directory", ]; @@ -244,6 +251,20 @@ export interface FilesHandler { * that same success. */ deleteFile(input: FilePathInput): Operation>; + /** + * Make this path name a directory, and answer with nothing. + * + * Recursively creates the target and any missing parent. An existing + * directory is already the answer, so it succeeds without replacing it, + * clearing it or touching what is in it; a file or another non-directory, at + * the target or anywhere on the way to it, is a refusal. + * + * Mandatory like the rest, and Unit for the same reason `deleteFile` is: a + * document that asked for a directory to exist has been answered by its + * existence. Nothing comes back to branch on — no path, no handle, and no + * word on whether this call is what created it. + */ + ensureDirectory(input: FilePathInput): Operation>; /** Sorted, deduplicated, POSIX-separated paths of the regular files that match. */ globFiles(input: GlobInput): Operation>; /** @@ -827,6 +848,10 @@ export const Files: Api = createApi("executablemd.ru throw new FilesProviderUnavailableError(); }, // deno-lint-ignore require-yield + *ensureDirectory(_input: FilePathInput): Operation> { + throw new FilesProviderUnavailableError(); + }, + // deno-lint-ignore require-yield *globFiles(_input: GlobInput): Operation> { throw new FilesProviderUnavailableError(); }, diff --git a/packages/runtime/host-files.ts b/packages/runtime/host-files.ts index 28f8e207..52c2faac 100644 --- a/packages/runtime/host-files.ts +++ b/packages/runtime/host-files.ts @@ -95,7 +95,7 @@ import type { * observable rather than merely stated. */ export interface HostFilesEvent { - readonly operation: "read" | "write" | "delete" | "glob"; + readonly operation: "read" | "write" | "delete" | "ensure-directory" | "glob"; readonly phase: "target" | "access" | "parents" | "temporary" | "commit" | "cleanup" | "read-dir"; } @@ -291,6 +291,43 @@ function* removalDestination(input: FilePathInput): Operation { } } +/** + * The directory a `` names, which is the one target that may be absolute. + * + * Every other operation here refuses an absolute path outright, because a + * document that writes one is naming a place outside the work it was given. + * `` is the established exception: an absolute `path` has always been used + * as written, and this operation exists to serve that component. So an absolute + * target is taken as it stands and is not measured against the working + * directory — there is no base it was ever relative to. + * + * A relative target keeps the ordinary rules: resolved against `cwd`, with both + * sides canonical, so a working directory reached through a symlink is not read + * as an escape and a `..` that genuinely leaves still is. + * + * The final segment is resolved along with the rest. A directory that already + * exists behind a symlink is the directory it points at, and entering it is + * what the document asked for. + */ +function* directoryDestination(input: FilePathInput): Operation { + try { + if (isAbsolute(input.path)) { + return { path: yield* resolveExisting(input.path) }; + } + if (!within(input.cwd, resolve(input.cwd, input.path))) { + return { reason: "lexical-escape" }; + } + const base = (yield* API.Fs.operations.realpath(input.cwd)) ?? input.cwd; + const path = yield* resolveExisting(resolve(input.cwd, input.path)); + if (!within(base, path)) { + return { reason: "resolved-escape" }; + } + return { path }; + } catch (error) { + return { reason: reasonOf(error) }; + } +} + function nonWriteFailure( operation: FilesOperation, phase: FilesPhase, @@ -545,6 +582,59 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { return Ok(undefined); } + /** + * Make the named path a directory, creating what is missing. + * + * Three answers, and the order between them is the contract. An existing + * directory is success without touching it: nothing is replaced, cleared or + * written, because the document asked for the directory to exist and it does. + * An existing entry that is not a directory is a refusal — a file where a + * directory was asked for is a mistake to report, never a thing to remove. + * Anything else is created, recursively, along with every missing parent. + * + * The target is classified before creation is attempted so the refusal for a + * non-directory target is decided here rather than left to whatever the + * platform's `mkdir -p` happens to say. An intermediate non-directory is the + * platform's to report, and `ENOTDIR` already carries it into the shared + * vocabulary — so both refusals arrive as `not-directory` and neither carries + * a host path or a platform message. + * + * Creation is direct and persists. There is no rollback and no teardown + * removal: a later failure of the content that runs inside this directory + * says nothing about whether the directory should exist. + */ + function* ensureDirectory(input: FilePathInput): Operation> { + if (input.path.length === 0) { + return nonWriteFailure("ensure-directory", "lexical", "empty-path"); + } + + const target = yield* directoryDestination(input); + if ("reason" in target) { + return nonWriteFailure("ensure-directory", "resolution", target.reason); + } + + notify(observe, { operation: "ensure-directory", phase: "target" }); + try { + const info = yield* API.Fs.operations.stat(target.path); + if (info.exists && !info.isDirectory) { + return nonWriteFailure("ensure-directory", "target", "not-directory"); + } + if (info.exists) { + return Ok(undefined); + } + } catch (error) { + return nonWriteFailure("ensure-directory", "target", reasonOf(error)); + } + + notify(observe, { operation: "ensure-directory", phase: "access" }); + try { + yield* API.Fs.operations.ensureDir(target.path); + } catch (error) { + return nonWriteFailure("ensure-directory", "access", reasonOf(error)); + } + return Ok(undefined); + } + /** * The regular files under `cwd` that `include` selects and `exclude` does not. * @@ -623,7 +713,15 @@ export function hostFilesHandler(options: HostFilesOptions = {}): FilesHandler { }); } - return { checkFilePath, readTextFile, writeTextFile, deleteFile, globFiles, temporaryDirectory }; + return { + checkFilePath, + readTextFile, + writeTextFile, + deleteFile, + ensureDirectory, + globFiles, + temporaryDirectory, + }; } /** @@ -691,6 +789,9 @@ export function useHostFiles(options: HostFilesOptions = {}): Operation { *deleteFile([input]) { return yield* handler.deleteFile(input); }, + *ensureDirectory([input]) { + return yield* handler.ensureDirectory(input); + }, *globFiles([input]) { return yield* handler.globFiles(input); }, diff --git a/packages/runtime/tests/host-files.test.ts b/packages/runtime/tests/host-files.test.ts index 8aa88d9d..9c5f4717 100644 --- a/packages/runtime/tests/host-files.test.ts +++ b/packages/runtime/tests/host-files.test.ts @@ -857,6 +857,78 @@ describe("Tier HF — host Files provider", () => { expect(observed).toEqual(["notes.md"]); }); + // HF15b: installation forwards every operation, not only the ones a caller + // happens to try first. + // + // `useHostFiles` forwards member by member, and `Files.around` accepts a + // partial handler by design — middleware implementing a subset and delegating + // the rest is the ordinary case. So an operation left out of the forwarding + // list is well-typed, and every caller of it falls through to the + // absent-provider terminal instead. That is not a contract this suite can + // check by construction; it has to be reached. + // + // Asserted through the installed provider rather than the handler, because + // the handler having the method is exactly what the omission looks like. + it("HF15b: every operation is reachable through the installed provider", function* () { + const fixture = yield* useFixture(); + yield* writeTextFile(join(fixture.workspace, "notes.md"), "reachable"); + yield* until(mkdir(join(fixture.workspace, "listed"))); + + yield* scoped(function* () { + yield* useHostFiles(); + // One call per operation the contract declares. A member missing from the + // forwarding list throws `FilesProviderUnavailableError` here rather than + // returning a Result, so an omission fails this case loudly. + expect( + (yield* Files.operations.checkFilePath({ + cwd: fixture.workspace, + path: "notes.md", + })).ok, + ).toBe(true); + expect( + value( + yield* Files.operations.readTextFile({ + cwd: fixture.workspace, + path: "notes.md", + }), + ).length, + ).toBeGreaterThan(0); + expect( + (yield* Files.operations.writeTextFile({ + cwd: fixture.workspace, + path: "written.md", + content: "x", + })).ok, + ).toBe(true); + expect( + (yield* Files.operations.ensureDirectory({ + cwd: fixture.workspace, + path: "made/through/installation", + })).ok, + ).toBe(true); + expect( + (yield* Files.operations.deleteFile({ + cwd: fixture.workspace, + path: "written.md", + })).ok, + ).toBe(true); + expect( + (yield* Files.operations.globFiles({ + cwd: fixture.workspace, + include: ["**/*"], + exclude: [], + })).ok, + ).toBe(true); + yield* scoped(function* () { + expect(value(yield* Files.operations.temporaryDirectory()).length).toBeGreaterThan(0); + }); + }); + + // And the directory really was made, through the installed provider rather + // than a handler the test held itself. + expect(yield* exists(join(fixture.workspace, "made", "through", "installation"))).toBe(true); + }); + // HF16: a junction is the Windows shape of the same limitation, and this is // the row that names it. Elsewhere it is an ordinary directory symlink, so // the case runs on every target rather than only where the reparse point @@ -1101,3 +1173,169 @@ describe("Tier HF — host Files provider", () => { expect(yield* entries(fixture.workspace)).toEqual(["notes.md"]); }); }); + +describe("host Files — making a directory exist", () => { + // HF24: the two things a caller can write, and the guarantee that the + // operation is finished before anything else happens. A relative path is + // resolved against the working directory and every missing parent is made; an + // absolute one names that exact location in the caller's filesystem and is + // used as written, which is the established `` exception and the reason + // this operation resolves its own destination. + it("HF24: creates missing parents for a relative path and uses an absolute one as written", function* () { + const fixture = yield* useFixture(); + const files = handler(); + + const nested = yield* files.ensureDirectory({ + cwd: fixture.workspace, + path: "one/two/three", + }); + expect(nested.ok).toBe(true); + expect(yield* exists(join(fixture.workspace, "one", "two", "three"))).toBe(true); + + // Outside the working directory, and accepted: every other operation here + // refuses an absolute path, so this asserts the exception rather than + // inheriting it. A relative path spelled to reach the same place is refused + // just below, which is what makes the two rules distinguishable. + const elsewhere = join(fixture.outside, "made", "here"); + const absolute = yield* files.ensureDirectory({ cwd: fixture.workspace, path: elsewhere }); + expect(absolute.ok).toBe(true); + expect(yield* exists(elsewhere)).toBe(true); + + const escaping = yield* files.ensureDirectory({ + cwd: fixture.workspace, + path: "../outside/climbed", + }); + expect(parseFilesFailure(failed(escaping))?.reason).toBe("lexical-escape"); + expect(yield* exists(join(fixture.outside, "climbed"))).toBe(false); + }); + + // HF25: an existing directory is the answer already. Nothing is replaced and + // nothing is cleared — asserted on the bytes, because a provider that removed + // and recreated the directory would pass a test that only checked it exists. + it("HF25: an existing directory is used, and its contents are untouched", function* () { + const fixture = yield* useFixture(); + const files = handler(); + const target = join(fixture.workspace, "existing"); + yield* until(mkdir(target)); + yield* writeTextFile(join(target, "kept.txt"), "the bytes that were here"); + yield* until(mkdir(join(target, "sub"))); + + const again = yield* files.ensureDirectory({ cwd: fixture.workspace, path: "existing" }); + expect(again.ok).toBe(true); + expect(yield* readTextFile(join(target, "kept.txt"))).toBe("the bytes that were here"); + expect(yield* entries(target)).toEqual(["kept.txt", "sub"]); + }); + + // HF26: a non-directory is a mistake to report, never a thing to remove. The + // matrix is four cases, not two: the entry may be a regular file or a + // supported special one, and it may stand at the target or on the way to it. + // Those refuse through different code — the target is classified here, an + // intermediate is the platform's `ENOTDIR` carried into the shared + // vocabulary — so a pair that covered only one position would leave the other + // untested. + // + // A symbolic link is this repository's portable special entry, and it points + // at a regular file: a link to a *directory* is a directory to enter, which + // is the parity decision both providers make. + it("HF26: a file or a special entry, at the target or on the way, refuses", function* () { + const fixture = yield* useFixture(); + const files = handler(); + yield* writeTextFile(join(fixture.workspace, "occupied"), "a file, not a directory"); + yield* writeTextFile(join(fixture.workspace, "pointee"), "what the link names"); + yield* until(symlink(join(fixture.workspace, "pointee"), join(fixture.workspace, "linked"))); + + const cases = [ + { what: "a regular file at the target", path: "occupied", phase: "target" }, + { what: "a regular file on the way", path: "occupied/below/here", phase: undefined }, + { what: "a special entry at the target", path: "linked", phase: "target" }, + { what: "a special entry on the way", path: "linked/below/here", phase: undefined }, + ] as const; + + for (const entry of cases) { + const refused = yield* files.ensureDirectory({ cwd: fixture.workspace, path: entry.path }); + const failure = parseFilesFailure(failed(refused)); + // The `what` travels with the assertion, so a failure names which of the + // four reported something else. + expect(`${entry.what}: ${failure?.operation}`).toBe(`${entry.what}: ensure-directory`); + expect(`${entry.what}: ${failure?.reason}`).toBe(`${entry.what}: not-directory`); + if (entry.phase !== undefined) { + expect(`${entry.what}: ${failure?.phase}`).toBe(`${entry.what}: ${entry.phase}`); + } + // Nothing resolved crosses back: no host path, no errno, no platform + // text. Asserted on the whole serialized failure, so a member added later + // that carried one of them would fail here rather than pass unnoticed. + const serialized = JSON.stringify(failed(refused)); + expect(`${entry.what}: ${serialized.includes(fixture.workspace)}`).toBe( + `${entry.what}: false`, + ); + expect(`${entry.what}: ${serialized.includes(fixture.root)}`).toBe(`${entry.what}: false`); + expect(`${entry.what}: ${/ENOTDIR|ENOENT|errno/i.test(serialized)}`).toBe( + `${entry.what}: false`, + ); + } + + // Every refusal changed nothing: the file is still a file, the link is + // still a link, and its target still holds its bytes. + expect(yield* readTextFile(join(fixture.workspace, "occupied"))).toBe( + "a file, not a directory", + ); + expect((yield* until(lstat(join(fixture.workspace, "linked")))).isSymbolicLink()).toBe(true); + expect(yield* readTextFile(join(fixture.workspace, "pointee"))).toBe("what the link names"); + }); + + // HF27: creation is direct and persists. The operation has no rollback and no + // teardown removal, so a directory made inside a scope that then fails or is + // cancelled is still there afterwards — which is the whole difference between + // this provider and the transactional one. + it("HF27: a created directory survives a later failure and a cancellation", function* () { + const fixture = yield* useFixture(); + const files = handler(); + + yield* scoped(function* () { + const made = yield* files.ensureDirectory({ cwd: fixture.workspace, path: "kept-on-fail" }); + expect(made.ok).toBe(true); + try { + throw new Error("the work inside failed"); + } catch { + // Swallowed here: what is under test is the directory, not the failure. + } + }); + expect(yield* exists(join(fixture.workspace, "kept-on-fail"))).toBe(true); + + const created = withResolvers(); + const task = yield* spawn(function* () { + const made = yield* files.ensureDirectory({ + cwd: fixture.workspace, + path: "kept-on-halt", + }); + expect(made.ok).toBe(true); + created.resolve(); + yield* suspend(); + }); + // Halted only once the directory exists. Halting a task that had not run + // yet would assert nothing about teardown — the directory would be absent + // because it was never created, which is a different result wearing the + // same shape. + yield* created.operation; + yield* task.halt(); + expect(yield* exists(join(fixture.workspace, "kept-on-halt"))).toBe(true); + }); + + // HF28: the operation is about a directory and nothing else. It reads no + // repository, and a checkout it is pointed inside of is byte-identical after. + it("HF28: ensuring a directory beneath a checkout leaves the checkout alone", function* () { + const fixture = yield* useFixture(); + const files = handler(); + const checkout = join(fixture.workspace, "checkout"); + yield* until(mkdir(join(checkout, ".git"), { recursive: true })); + yield* writeTextFile(join(checkout, ".git", "HEAD"), "ref: refs/heads/main\n"); + yield* writeTextFile(join(checkout, "tracked.txt"), "committed content"); + + const made = yield* files.ensureDirectory({ cwd: checkout, path: "generated/output" }); + expect(made.ok).toBe(true); + + expect(yield* readTextFile(join(checkout, ".git", "HEAD"))).toBe("ref: refs/heads/main\n"); + expect(yield* readTextFile(join(checkout, "tracked.txt"))).toBe("committed content"); + expect(yield* entries(checkout)).toEqual([".git", "generated", "tracked.txt"]); + }); +}); diff --git a/packages/testing/package.json b/packages/testing/package.json index 01f8f0c6..01beeaa7 100644 --- a/packages/testing/package.json +++ b/packages/testing/package.json @@ -12,6 +12,7 @@ "@effectionx/timebox": "0.4.3", "@executablemd/core": "workspace:*", "@executablemd/durable-streams": "workspace:*", + "@executablemd/runtime": "workspace:*", "effection": "4.1.0" } } diff --git a/packages/testing/src/execution-harness.ts b/packages/testing/src/execution-harness.ts index 2ac135ec..7c915e7f 100644 --- a/packages/testing/src/execution-harness.ts +++ b/packages/testing/src/execution-harness.ts @@ -85,6 +85,7 @@ import type { SourcePosition, } from "@executablemd/core"; import { DeclarationScan } from "@executablemd/core/host"; +import { cwd } from "@executablemd/runtime"; import type { AnswerConfiguration, AnswersPlacement, @@ -939,9 +940,19 @@ function* runChild( // Spent only once the chain has agreed there is a child to run, and never // twice: two nested executions are two authorizations. grant.spend(); + // Read here, on the last line that is still inside this invocation. The + // isolated scope below does not descend from the document's, so this is the + // only place a `` or `` around the `` is still + // observable — one statement later it is the process directory. + const directory = yield* cwd(); return yield* inIsolation(function* (childScope) { return yield* childScope.run(() => - provider.runChild({ request: settled, run: run?.scope, chunk: channel.chunk }), + provider.runChild({ + request: settled, + run: run?.scope, + chunk: channel.chunk, + cwd: directory, + }), ); }); } diff --git a/packages/testing/src/execution-host.ts b/packages/testing/src/execution-host.ts index 234b66d3..087f803c 100644 --- a/packages/testing/src/execution-host.ts +++ b/packages/testing/src/execution-host.ts @@ -234,6 +234,27 @@ export interface WorkflowRunScope { export interface ChildInvocation { /** The profile the terminal recorded. */ readonly request: HostProfileRequest; + /** + * The contextual working directory the `` invocation was standing + * in, captured before the child's isolated scope existed. + * + * A child is a root execution in a scope that does not descend from the + * document's, so it inherits no `API.Env` handler and `cwd()` would fall back + * to the *process* directory. That silently disagrees with the document: a + * `` or `` around an `` would scope every + * component in it except the child, and a run provider discovering its + * ambient repository would find the one the process happens to be standing + * in — a shared checkout nobody addressed. + * + * Deliberately here and not on {@link HostProfileRequest}. The profile is + * what middleware reads, refuses on, and delegates, and a handler that could + * see this could also swap it: the directory a child resolves its root and + * its repository in would become something composed policy chose rather than + * something the document did. So it travels on the private invocation the + * terminal hands the trusted provider, where the same rule already keeps + * `run` and `chunk`. + */ + readonly cwd: string; /** The isolated workflow run this child belongs to, under the workflow profile. */ readonly run: WorkflowRunScope | undefined; /** diff --git a/packages/workflow/src/composition/components/Dir.ts b/packages/workflow/src/composition/components/Dir.ts index 8e04cbfb..4e20e560 100644 --- a/packages/workflow/src/composition/components/Dir.ts +++ b/packages/workflow/src/composition/components/Dir.ts @@ -2,16 +2,26 @@ * `` — lexical working directory * (specs/workflow-workspace-spec.md §6.3). * - * One meaning and one effect on the world: the content expands with `path` as - * the contextual working directory, and the enclosing one is restored when the - * invocation ends — on success, failure and cancellation alike, because the - * installation lives on the invocation's own scope. + * Two acts, in this order. First the directory named by `path` is made to + * exist: it is created recursively when it is missing, used as it stands when + * it is already a directory, and refused when something that is not a directory + * is there or on the way to it. Only then does the content expand, with `path` + * as the contextual working directory. Content never begins on the far side of + * a refusal, because a document that named a directory asked for that directory + * and not for whatever it would otherwise have run in. * - * It creates nothing and retains nothing. A `` inside a `` - * leaves that Repository contextual, which is what lets the self-closing - * Worktree spelling work: the path is bound by `as`, `` moves the working - * directory to it, and the Repository a later Git component would act on is - * still the enclosing one. + * The enclosing working directory is restored when the invocation ends — on + * success, failure and cancellation alike, because the installation lives on + * the invocation's own scope. Restoring it removes nothing: a directory this + * element created stays, and a later failure of the content inside it says + * nothing about whether the directory should exist. There is no rollback here + * and no teardown deletion. + * + * It retains nothing of its own. A `` inside a `` leaves that + * Repository contextual, which is what lets the self-closing Worktree spelling + * work: the path is bound by `as`, `` moves the working directory to it, + * and the Repository a later Git component would act on is still the enclosing + * one. Neither the selection nor any member of its identity is touched. * * Self-closing `` is invalid. A working directory installed for no * content is a directory nothing runs in, and silently rendering nothing would @@ -21,7 +31,8 @@ */ import { API, cwd } from "@executablemd/runtime"; -import { content } from "@executablemd/core"; +import { content, ensureDirectory } from "@executablemd/core"; +import { parseFilesFailure } from "@executablemd/runtime"; import type { FormDeclaration, InvocationForm } from "@executablemd/core"; import type { Operation } from "effection"; import type { Json } from "@executablemd/durable-streams"; @@ -39,6 +50,27 @@ export class DirInvocationError extends Error { override name = "DirInvocationError"; } +/** What a refusal means, in the document's terms rather than the platform's. */ +function reason(value: string): string { + switch (value) { + case "not-directory": + return "something that is not a directory is already there, or is on the way to it"; + case "missing": + return "part of the path is not there and could not be created"; + case "permission-denied": + return "this run is not allowed to create or enter it"; + case "read-only": + return "the filesystem it would be created on is read-only"; + case "lexical-escape": + case "resolved-escape": + return "it is outside the directory this run is working in"; + case "empty-path": + return "the path is empty"; + default: + return "the directory operation did not succeed"; + } +} + function* Dir(props: Record): Operation { const path = props.path; if (typeof path !== "string" || path === "") { @@ -55,6 +87,25 @@ function* Dir(props: Record): Operation { ? path : `${enclosing.endsWith("/") ? enclosing.slice(0, -1) : enclosing}/${path}`; + // Before the working directory is installed and before any content expands. + // A document that names a directory has asked for it to exist, and content + // that ran in a directory the ensure was going to refuse would be content run + // somewhere nobody chose. + // + // One provider-neutral call, and nothing else: which filesystem this reaches + // — the caller's own or a workflow run's logical one — is the installed + // provider's business, and a component that asked would be a component that + // behaves differently under the two profiles. + const refusal = yield* ensureDirectory({ cwd: enclosing, path }); + if (!refusal.ok) { + const failure = parseFilesFailure(refusal.error); + throw new DirInvocationError( + ` could not be used: ${ + failure === undefined ? "the directory operation failed" : reason(failure.reason) + }`, + ); + } + yield* API.Env.around( { // deno-lint-ignore require-yield diff --git a/packages/workflow/src/composition/installation.ts b/packages/workflow/src/composition/installation.ts index c0e55854..ba4a6872 100644 --- a/packages/workflow/src/composition/installation.ts +++ b/packages/workflow/src/composition/installation.ts @@ -7,12 +7,14 @@ * reserved and not structural — so a repository-local component may shadow one * for its own scope. * - * One array rather than a list per caller, because two descriptions of one - * vocabulary would drift. `useCompositionComponents()` registers it inside a - * workflow attachment. Registering it installs no provider, performs no - * repository discovery, acquires no lock and reaches no network: what a name - * *does* is the installed provider's, and describing the environment mints - * none. + * One array, three consumers, because three descriptions of one vocabulary + * would drift. `useCompositionComponents()` registers it inside a workflow + * attachment; `useRunProfileRegistry()` registers it for `xmd syntax` and for + * `xmd plan`'s validation and generation; `installDocumentComponents()` + * registers it for an ordinary run. Registering it installs no provider, + * performs no repository discovery, acquires no lock and reaches no network: + * what a name *does* is the installed provider's, and describing the + * environment mints none. * * A completed root replay attaches no provider and registers nothing, so a * document that already ran through completion re-registers none of these. @@ -59,14 +61,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: repositoryProps, fn: Repository, ...documented({ - description: - "Work in a Git repository by name and url. " + - '`…` clones it once, ' + - "then expands its content with that checkout as the working directory. Written " + - '`` it renders nothing and binds ' + - "the checkout path instead. A second invocation naming the same repository and url " + - "reuses the same checkout, keeping the commits, branches and uncommitted work the " + - "first one left there.", + description: `Clone or use a repository. \`…\` expands its content in the specified checkout.`, as: "Optional. The path of the selected checkout.", context: "The Markdown expanded in that checkout.", }), @@ -77,13 +72,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: worktreeProps, fn: Worktree, ...documented({ - description: - "Work on a branch in a linked checkout of its own. " + - '`` creates the branch when ' + - "it is missing and checks it out beside the repository, so several branches are open " + - "at once without one switch disturbing another. `branch` is required and `name` never " + - "selects one. Written outside a `` it belongs to the repository the " + - "command was run in, where the host has one.", + description: `Create or use a worktree. \`…\` expands its content in the worktree.`, as: "Optional. The path of the linked checkout.", context: "The Markdown expanded in that checkout.", }), @@ -94,12 +83,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: dir.props, fn: dir.fn, ...documented({ - description: - "Run its content in another directory. " + - "`…` expands the Markdown inside with `path` as the " + - "working directory, and restores the enclosing one afterwards. A relative path is " + - "read against the directory already in effect. It selects no repository: Git " + - "elements inside still belong to the enclosing ``.", + description: `Create or use a directory. \`…\` changes the working directory for its content.`, as: null, context: "The Markdown expanded in that directory.", }), @@ -110,11 +94,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: gitSwitchProps, fn: GitSwitch, ...documented({ - description: - "Put the checkout on a named branch. " + - '`` switches to the branch, creating ' + - "it at `base` when it does not exist yet. A branch another checkout already holds, " + - "and local changes the switch would overwrite, are both refused rather than forced.", + description: `Switch or create a branch. \`\` starts a missing branch at the specified base.`, as: null, context: null, }), @@ -125,11 +105,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: gitAddProps, fn: GitAdd, ...documented({ - description: - "Stage exactly the paths you name. " + - '`` stages them as written, from ' + - "the directory the element appears in. `paths` is a Git pathspec and is required; " + - '`"."` is how a document says everything here.', + description: `Stage paths for commit. \`\` resolves Git pathspecs from the working directory.`, as: null, context: null, }), @@ -141,12 +117,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ returns: gitCommitReturns, fn: GitCommit, ...documented({ - description: - "Commit what is staged, and hand back the commit. " + - '`…` commits the index ' + - "alone — nothing is staged for it and nothing is amended. Content expands first and " + - "becomes the message body, so a `` written inside stages before the commit " + - "exists. An index that already matches HEAD is refused rather than committed empty.", + description: `Commit staged changes. \`Prepare 1.4\` uses its expanded content as the commit message.`, as: "Optional. The full object id of the commit.", context: "The message body, expanded before the commit is made.", }), @@ -157,12 +128,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: gitPushProps, fn: GitPush, ...documented({ - description: - "Publish the checkout's current branch to its origin. " + - "`` takes no props: the remote is the repository's `origin`, the branch " + - "is the one the checkout is on, and the commit is the one that branch points at. It " + - "never force-pushes and changes no upstream tracking; a destination naming a commit " + - "this run did not publish from is refused.", + description: `Publish the current branch. \`\` pushes its current commit to the same branch on \`origin\` without forcing.`, as: null, context: null, }), @@ -174,13 +140,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ returns: pullRequestReturns, fn: PullRequest, ...documented({ - description: - "Open a pull request for the branch this run published, or bring one up to date. " + - '`…` asks for one ' + - "pull request from the checkout's branch to `base` to exist; with `number` it updates " + - "that pull request instead. The content is the body. It publishes nothing itself: " + - "write `` first, and this run must hold that push's own successful " + - "result for the same branch and commit.", + description: `Open or update a pull request. \`…\` uses its content as the body after \`\`.`, as: "Optional. The pull request's repository, number, url, state and head and base commits.", context: "The pull request's body.", }), @@ -192,12 +152,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ returns: reviewsReturns, fn: formDispatcher(reviewsForm), ...documented({ - description: - "Read the reviews a pull request holds. " + - '`` binds one array to ' + - "iterate with ``, so an objection reaches an agent's prompt. The url is the " + - "identity — there is no repository or number prop, and no `` to be " + - "inside of. `as` is required.", + description: `Read pull request reviews. \`\` reads reviews from the specified pull request.`, as: "Required. Each review's author, state, body, submission time, commit and url.", context: null, }), @@ -209,11 +164,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ returns: commentsReturns, fn: formDispatcher(commentsForm), ...documented({ - description: - "Read the comments a pull request holds. " + - '`` binds one array of ' + - "both conversation comments and review comments, each saying which kind it is. The " + - "url is the identity. `as` is required.", + description: `Read pull request comments. \`\` reads both conversation and review comments.`, as: "Required. Each comment's kind, author, body, timestamps and url, and a review comment's file, hunk and line.", context: null, }), @@ -225,11 +176,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ returns: checksReturns, fn: formDispatcher(checksForm), ...documented({ - description: - "Read the checks reported against a pull request's head. " + - '`` binds one array of both ' + - "check runs and commit statuses, each saying which kind it is. The url is the " + - "identity. `as` is required.", + description: `Read pull request checks. \`\` reads check runs and commit statuses for its head.`, as: "Required. Each check's kind, name, head commit and outcome.", context: null, }), @@ -240,12 +187,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ props: issueTrackerProps, fn: IssueTracker, ...documented({ - description: - "Say which tracker the issues in its content are filed in. " + - "`…` names the container new issues " + - "are created in — one repository's issues, one project. `provider` " + - "names the only adapter allowed to act on it, for a url nobody recognizes. A nested " + - "tracker replaces the whole target for its own content rather than merging with it.", + description: `Select an issue tracker. \`…\` expands its content with the specified tracker in scope.`, as: null, context: "The Markdown whose issues are filed there.", }), @@ -257,13 +199,7 @@ export const COMPOSITION_REGISTRATIONS: readonly ComponentRegistration[] = [ returns: issueReturns, fn: Issue, ...documented({ - description: - "Read an issue by url, or file one in the tracker in scope. " + - '`` reads the one that url names and needs ' + - 'no tracker. `…` inside an ' + - "`` files an issue whose content is its description, creating it once " + - "and bringing it up to date afterwards. Which of the two it is, is decided by the " + - "spelling: a url reads, a title files.", + description: `Read or file an issue. \`\` reads one. \`…\` files its content in the issue tracker in scope.`, as: "Required for a read, which binds url, title, description, tags and assignee. A file binds the url alone.", context: "The issue's description, for the form that files one.", }), diff --git a/packages/workflow/src/deno/workspace/evaluate.ts b/packages/workflow/src/deno/workspace/evaluate.ts index b569b00b..51d76d8a 100644 --- a/packages/workflow/src/deno/workspace/evaluate.ts +++ b/packages/workflow/src/deno/workspace/evaluate.ts @@ -251,7 +251,14 @@ function createEvaluate( const reads: GeneratedObservation[] = [pinnedFileRead(), ...(options.reads ?? [])]; const writes: GeneratedMutation[] = [ pinnedFileWrite(), - pinnedMutation(dir.name, `${COMPOSITION_ORIGIN}#Dir`, dir, "paired"), + // Versioned, because what this entry authorizes changed. The former + // `@executablemd/workflow/composition#Dir` authorized placement that created + // nothing; `` now recursively creates the directory it names, and that + // is persistent mutation of the run's Workspace. A continuation granted + // under the old identity was granted something narrower than this, so it + // must not silently receive the wider authority — the retained comparison + // sees a different identity and refuses before generated execution. + pinnedMutation(dir.name, `${COMPOSITION_ORIGIN}/dir-v2#Dir`, dir, "paired"), pinnedFileDelete(), ...(options.writes ?? []), ]; diff --git a/packages/workflow/src/deno/workspace/files.ts b/packages/workflow/src/deno/workspace/files.ts index 6f2a72b3..3f24fb0d 100644 --- a/packages/workflow/src/deno/workspace/files.ts +++ b/packages/workflow/src/deno/workspace/files.ts @@ -153,6 +153,7 @@ type FileEffectOutcome = | { readonly kind: "content"; readonly content: string } | { readonly kind: "written" } | { readonly kind: "deleted" } + | { readonly kind: "ensured" } | { readonly kind: "paths"; readonly paths: string[] } | { readonly kind: "refused"; readonly phase: Phase; readonly reason: FilesReason }; @@ -176,6 +177,7 @@ const OUTCOME_MEMBERS: ReadonlyMap = new Map( if (kind === "deleted") { return { kind: "deleted" }; } + if (kind === "ensured") { + return { kind: "ensured" }; + } if (kind === "paths") { const paths = readPaths(record.paths); return paths === undefined ? undefined : { kind: "paths", paths }; @@ -250,6 +255,28 @@ function parseOutcome( return { kind: "refused", phase, reason }; } +/** + * The logical path an authored relative path names, or the refusal to report. + * + * Returns the failure itself rather than a discriminated wrapper, because the + * one caller that needs it has exactly two things to do with the answer. It + * reaches no filesystem and no Api, so it is an ordinary function rather than + * an Operation that would only ever have yielded nothing. + */ +function logicalTarget(input: FilePathInput): string | Result { + const resolved = resolveLogicalPath(input.cwd, input.path); + if (!resolved.ok) { + return Err( + filesFailure({ + operation: "ensure-directory", + phase: "lexical", + reason: lexicalReason(resolved.error), + }), + ); + } + return resolved.value; +} + /** * How one file effect is identified, deterministically. * @@ -375,6 +402,49 @@ function* replace( yield* filesystem.writeFile(path, content); } +/** + * Make one logical directory exist, discarding the whole attempt if any part + * of it refuses. + * + * An existing directory is the answer already: nothing is written, and the + * effect still commits so that replay has a record to restore. An existing + * entry that is not a directory is refused at the target, before any parent is + * created — which is what keeps a refusal from leaving half a path behind. + * + * The recursive creation runs inside one savepoint, so a run that creates two + * parents and then cannot create the third leaves neither. What comes back + * therefore describes a Workspace that is exactly what it was. + */ +function* ensureOutcome( + filesystem: DenoWorkspaceFilesystem, + path: string, +): Operation> { + if (path === WORKSPACE_ROOT) { + return { kind: "ensured" }; + } + // `stat` rather than `lstat`, so a symbolic link is classified by what it + // points at. A link to a directory is a directory to enter, which is what the + // host provider decides too — its resolution follows the whole path — and the + // two profiles must not disagree about the same document. + const info = yield* statPath(filesystem, path); + if (info.ok) { + return info.value.kind === "directory" + ? { kind: "ensured" } + : refused("target", "not-directory"); + } + const reason = refusalReason(info.error); + if (reason !== "missing") { + return refused("target", reason); + } + + try { + yield* savepoint(filesystem.mkdir(path, { recursive: true })); + } catch (error) { + return refused("access", refusalReason(asRefusal(error))); + } + return { kind: "ensured" }; +} + /** * Replace one file, discarding every part of the attempt if any part refuses. * @@ -578,6 +648,7 @@ export interface WorkflowFilesHandler { readTextFile(input: FilePathInput): Operation>; writeTextFile(input: FileWriteInput): Operation>; deleteFile(input: FilePathInput): Operation>; + ensureDirectory(input: FilePathInput): Operation>; globFiles(input: GlobInput): Operation>; temporaryDirectory(): Operation>; } @@ -678,6 +749,39 @@ export function workflowFilesHandler(database: WorkflowRunDatabase): WorkflowFil return Ok(undefined); }, + *ensureDirectory(input: FilePathInput): Operation> { + // A bound Workspace path is already logical and absolute, and `` + // uses one as written. Every other operation refuses an absolute path, + // so the resolution is chosen here rather than by relaxing the shared + // rule that keeps authored paths inside the working directory. + const path = input.path.startsWith("/") ? logicalDirectory(input.path) : logicalTarget(input); + if (typeof path !== "string") { + return path; + } + const outcome = yield* performed( + database, + yield* describeFileEffect("ensure-directory", path, { + path: input.path, + cwd: input.cwd, + }), + parseFilesPhase, + (filesystem) => ensureOutcome(filesystem, path), + ); + if (outcome.kind === "refused") { + return Err( + filesFailure({ + operation: "ensure-directory", + phase: outcome.phase, + reason: outcome.reason, + }), + ); + } + if (outcome.kind !== "ensured") { + throw new FilesInvariantError("protocol"); + } + return Ok(undefined); + }, + *globFiles(input: GlobInput): Operation> { const directory = logicalDirectory(input.cwd); const include = [...input.include]; @@ -737,6 +841,9 @@ export function useWorkflowFiles(database: WorkflowRunDatabase): Operation *deleteFile([input]) { return yield* handler.deleteFile(input); }, + *ensureDirectory([input]) { + return yield* handler.ensureDirectory(input); + }, *globFiles([input]) { return yield* handler.globFiles(input); }, diff --git a/packages/workflow/tests/generated-agent-component.test.ts b/packages/workflow/tests/generated-agent-component.test.ts index e4d206fe..980ff9ba 100644 --- a/packages/workflow/tests/generated-agent-component.test.ts +++ b/packages/workflow/tests/generated-agent-component.test.ts @@ -1027,7 +1027,7 @@ describe("Tier WGAC — the standard write table", () => { { name: "File", identity: "@executablemd/core#File:write", forms: ["paired"] }, { name: "Dir", - identity: "@executablemd/workflow/composition#Dir", + identity: "@executablemd/workflow/composition/dir-v2#Dir", forms: ["paired"], }, { diff --git a/packages/workflow/tests/git-switch.test.ts b/packages/workflow/tests/git-switch.test.ts index e6bee991..61ac4669 100644 --- a/packages/workflow/tests/git-switch.test.ts +++ b/packages/workflow/tests/git-switch.test.ts @@ -14,7 +14,7 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { registerComponents } from "@executablemd/core"; +import { content, registerComponents } from "@executablemd/core"; import type { ComponentRegistration } from "@executablemd/core"; import { collect, execute, inlineSource } from "@executablemd/core"; import { InMemoryStream } from "@executablemd/durable-streams"; @@ -22,7 +22,7 @@ import type { Json } from "@executablemd/durable-streams"; import { scoped, until } from "effection"; import { readTextFile, writeTextFile } from "@effectionx/fs"; import { pathToFileURL } from "node:url"; -import { cwd } from "@executablemd/runtime"; +import { API, cwd } from "@executablemd/runtime"; import type { Operation } from "effection"; import { GitCompositionProviderError, @@ -57,6 +57,7 @@ import { retainedRepositories, retainedWorktrees, runDocument, + runWorkflowDocument, subcommands, survivingRoots, workspaceText, @@ -187,6 +188,47 @@ function document(locator: string, ...lines: string[]): string { return [``, ...lines, ""].join("\n"); } +/** + * A working directory inside the selected checkout that is not a directory. + * + * The subject of the case below is the Git provider's own authority check: a + * contextual working directory the retained checkout does not actually hold is + * refused, before the provider spawns anything. Reaching that state needs a cwd + * that names something which is not there. + * + * `` used to be the way to write it, because it installed a working + * directory and created nothing. It now creates the directory it names (#643), + * so it can no longer produce this state — and pointing it at a file instead + * would prove Dir's own refusal, which is evidence held elsewhere, while + * pointing outside the checkout would repeat the preceding case. + * + * So the fixture does the one thing `` used to: read the real contextual + * directory, install a path beneath it for its content, and touch no + * filesystem at all. It performs no `API.Files` operation and creates nothing, + * which is what leaves the working directory unreal for the Git operation + * inside it. + */ +function nowhere(): ComponentRegistration { + return { + name: "Nowhere", + origin: "test://nowhere", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + const enclosing = yield* cwd(); + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return `${enclosing}/absent`; + }, + }, + { at: "min" }, + ); + return yield* content(); + }, + }; +} + describe("workflow Git.Switch", () => { it("moves the primary checkout to a branch the remote published", function* () { const root = yield* useStorageRoot(); @@ -551,15 +593,15 @@ describe("workflow Git.Switch selection", () => { const database = yield* createRun(); const counting = countingHost(); const failure = yield* raised( - runDocument( + runWorkflowDocument( database, - document( - remote.locator, - ``, - ``, - "", - ), + document(remote.locator, ``, ``, ""), countingOptions(counting), + (execute) => + scoped(function* () { + yield* registerComponents([nowhere()]); + return yield* execute(); + }), ), ); diff --git a/packages/workflow/tests/repository-components.test.ts b/packages/workflow/tests/repository-components.test.ts index 1910582b..9b511c8f 100644 --- a/packages/workflow/tests/repository-components.test.ts +++ b/packages/workflow/tests/repository-components.test.ts @@ -10,6 +10,11 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; import { scoped } from "effection"; +import { exists } from "@effectionx/fs"; +import { join } from "node:path"; +import process from "node:process"; +import { API } from "@executablemd/runtime"; +import { parseFilesFatal } from "@executablemd/runtime"; import type { Operation } from "effection"; import { collect, @@ -19,7 +24,7 @@ import { inlineSource, registerComponents, } from "@executablemd/core"; -import type { ComponentInvocation } from "@executablemd/core"; +import type { ComponentInvocation, ComponentRegistration } from "@executablemd/core"; import type { Json } from "@executablemd/durable-streams"; import type { WorkflowRunDatabase } from "../mod.ts"; import { useCompositionComponents } from "../src/composition/installation.ts"; @@ -720,3 +725,121 @@ describe("workflow Dir under a lying content chain", () => { }); }); }); + +/** The first Files infrastructure failure in a thrown graph. */ +function fatalCause(error: unknown, seen = new Set()): unknown { + if (parseFilesFatal(error) !== undefined) { + return error; + } + if (typeof error !== "object" || error === null || seen.has(error)) { + return undefined; + } + seen.add(error); + const causes = + error instanceof AggregateError + ? error.errors + : error instanceof Error && error.cause !== undefined + ? [error.cause] + : []; + for (const cause of causes) { + const found = fatalCause(cause, seen); + if (found !== undefined) { + return found; + } + } + return undefined; +} + +/** + * The shipped `` against no Files provider at all. + * + * Core's `FF2` proves the fail-closed boundary of `ensureDirectory` itself, + * through a stand-in that makes exactly that one call — it has to, because core + * cannot import the package this component lives in. This is the other half: + * that the component shipped here really does make that call, first, and that a + * document stops at it. + * + * Together they are the whole claim. Neither is sufficient alone: the stand-in + * could pass while `` called something else or called nothing, and this + * could pass while the operation quietly answered instead of refusing. + */ +describe("Dir without a Files provider", () => { + it("stops at ensureDirectory, and no directory, content or sibling follows", function* () { + const calls: string[] = []; + // Sentinel components rather than rendered text. What escapes a failed + // execution is an error, and searching an error's string for a marker + // cannot tell "the content never expanded" from "the content expanded and + // its text is simply not in this message". A component that ran leaves a + // record here whether or not anything was rendered or collected. + const expanded: string[] = []; + const marker = (name: string): ComponentRegistration => ({ + name, + origin: "test://marker", + props: { type: "object", properties: {}, additionalProperties: false }, + // deno-lint-ignore require-yield + *fn(): Operation { + expanded.push(name); + return name; + }, + }); + + const failure = yield* raised( + scoped(function* () { + yield* useCompositionComponents(); + yield* registerComponents([marker("Inside"), marker("After")]); + // Middleware that records and delegates. Delegation is what keeps this + // fail-closed: the absent-provider terminal is still what answers, so + // recording the call cannot be what makes the document stop. + yield* API.Files.around({ + *checkFilePath([input], next) { + calls.push("check-file-path"); + return yield* next(input); + }, + *readTextFile([input], next) { + calls.push("read"); + return yield* next(input); + }, + *writeTextFile([input], next) { + calls.push("write"); + return yield* next(input); + }, + *deleteFile([input], next) { + calls.push("delete"); + return yield* next(input); + }, + *ensureDirectory([input], next) { + calls.push("ensure-directory"); + return yield* next(input); + }, + *globFiles([input], next) { + calls.push("glob"); + return yield* next(input); + }, + *temporaryDirectory([], next) { + calls.push("temporary-directory"); + return yield* next(); + }, + }); + return yield* collect( + yield* execute({ + ...inlineSource('\n\n\n\n\n\n\n'), + stream: new InMemoryStream(), + }), + ); + }), + ); + + // The first Files call the component makes, and the only one it reaches. + expect(calls[0]).toBe("ensure-directory"); + expect(calls).toEqual(["ensure-directory"]); + // Absence is not a refusal the document can print: what escapes is the + // provider-unavailable failure itself. + expect(parseFilesFatal(fatalCause(failure))?.kind).toBe("provider-unavailable"); + // Neither the content inside the region nor the sibling after it ran. This + // is the assertion the failure's message could not make. + expect(expanded).toEqual([]); + // And nothing was made. The path is relative, so a provider that had + // answered would have created it beneath the process directory. + expect(yield* exists(join(process.cwd(), "made"))).toBe(false); + }); +}); diff --git a/packages/workflow/tests/run-composition-ambient.test.ts b/packages/workflow/tests/run-composition-ambient.test.ts index d544bd3c..5fd96385 100644 --- a/packages/workflow/tests/run-composition-ambient.test.ts +++ b/packages/workflow/tests/run-composition-ambient.test.ts @@ -14,9 +14,17 @@ */ import { describe, it } from "@executablemd/test-support/bdd"; +import { join } from "node:path"; +import { symlink } from "node:fs/promises"; +import { API, cwd } from "@executablemd/runtime"; +import { spawn, suspend, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { selectedRepository } from "../src/composition/context.ts"; +import type { RepositorySelection } from "../src/composition/selection.ts"; +import type { ComponentRegistration } from "@executablemd/core"; import { expect } from "@executablemd/test-support/expect"; import { scoped } from "effection"; -import { ensureDir, exists, writeTextFile } from "@effectionx/fs"; +import { ensureDir, exists, readdir, readTextFile, writeTextFile } from "@effectionx/fs"; import { chmod } from "node:fs/promises"; import { until } from "effection"; import { useTempDirectory } from "@executablemd/test-support/temp"; @@ -528,3 +536,301 @@ describe("ORC13 — live local Git", () => { expect(String(failure)).toContain("branch-checked-out-elsewhere"); }); }); + +/** A component that records the whole Repository selection in scope. */ +function selectionProbe(seen: (RepositorySelection | undefined)[]): ComponentRegistration { + return { + name: "Stated", + origin: "test", + props: { type: "object", additionalProperties: false }, + *fn(): Operation { + seen.push(yield* selectedRepository()); + return ""; + }, + }; +} + +describe("ORC6 — Dir makes the directory it names", () => { + // ORC6: the ordering claim and the two path rules, in one document. + it("ORC6: creates missing parents, keeps an absolute path, and finishes before content", function* () { + const root = yield* useManagedRoot(); + const checkout = (yield* useOriginlessCheckout()).root; + const elsewhere = yield* useTempDirectory("xmd-orc6-absolute-"); + + // The ordering probe. A nested `` proves nothing about ordering, + // because a write creates its own parents recursively — it would land + // whether or not the ensure had finished. This runs *first* inside the + // region and records what it finds: whether the directory is already there, + // and what the contextual working directory is at that moment. + const observed: { exists: boolean; cwd: string }[] = []; + const probe: ComponentRegistration = { + name: "Observes", + origin: "test://observes", + props: { type: "object", properties: {}, additionalProperties: false }, + *fn(): Operation { + const here = yield* cwd(); + observed.push({ exists: yield* exists(here), cwd: here }); + return ""; + }, + }; + + yield* runOrdinaryDocument( + [ + '', + "", + "", + "", + 'landed', + "", + "", + "", + ``, + "", + "", + "", + 'also landed', + "", + "", + "", + ].join("\n"), + { root, cwd: checkout, components: [probe] }, + ); + + // Before any other content ran, the directory already existed and was the + // contextual working directory. That is the ordering claim, observed rather + // than inferred. + expect(observed).toHaveLength(2); + expect(observed[0]).toEqual({ exists: true, cwd: join(checkout, "made", "deep") }); + expect(observed[1]).toEqual({ + exists: true, + cwd: join(elsewhere, "written", "here"), + }); + + // Every missing parent, and the file that landed after them. + expect(yield* readTextFile(join(checkout, "made", "deep", "inside.md"))).toBe("landed"); + // The absolute target names exactly that place — not a rebase beneath cwd. + expect(yield* readTextFile(join(elsewhere, "written", "here", "outside.md"))).toBe( + "also landed", + ); + expect(yield* exists(join(checkout, elsewhere.replace(/^\//, "")))).toBe(false); + }); + + // ORC6a: an existing directory is adopted. Asserted on the bytes and on the + // entry list, because a provider that removed and recreated it would pass a + // test that only checked the directory exists. + it("ORC6a: an existing directory keeps its contents", function* () { + const root = yield* useManagedRoot(); + const checkout = (yield* useOriginlessCheckout()).root; + yield* ensureDir(join(checkout, "kept", "sub")); + yield* writeTextFile(join(checkout, "kept", "planted.txt"), "the bytes that were here"); + + yield* runOrdinaryDocument( + '\n\nadded\n\n\n', + { root, cwd: checkout }, + ); + + expect(yield* readTextFile(join(checkout, "kept", "planted.txt"))).toBe( + "the bytes that were here", + ); + expect((yield* readdir(join(checkout, "kept"))).sort()).toEqual([ + "added.md", + "planted.txt", + "sub", + ]); + }); + + // ORC6b: a non-directory refuses before content, at the target and on the way + // to it, and what the document is told carries no host path or platform code. + it("ORC6b: a file or a special entry, at the target or on the way, refuses", function* () { + const root = yield* useManagedRoot(); + const checkout = (yield* useOriginlessCheckout()).root; + yield* writeTextFile(join(checkout, "occupied"), "a file"); + yield* writeTextFile(join(checkout, "pointee"), "what the link names"); + yield* until(symlink(join(checkout, "pointee"), join(checkout, "linked"))); + + // The same four positions the host contract covers, asked of the shipped + // element: a regular file and a supported special entry, each at the target + // and each on the way to it. + for (const path of ["occupied", "occupied/below", "linked", "linked/below"]) { + const printed = String( + yield* runOrdinaryDocument( + `\n\n\nINSIDE\n\n\n\n`, + { root, cwd: checkout }, + ), + ); + expect(`${path}: ${printed.includes("not a directory")}`).toBe(`${path}: true`); + expect(`${path}: ${printed.includes("INSIDE")}`).toBe(`${path}: false`); + expect(`${path}: ${printed.includes(checkout)}`).toBe(`${path}: false`); + expect(`${path}: ${/ENOTDIR|ENOENT|errno/i.test(printed)}`).toBe(`${path}: false`); + } + + // Nothing the refusals touched changed. + expect(yield* readTextFile(join(checkout, "occupied"))).toBe("a file"); + expect(yield* readTextFile(join(checkout, "pointee"))).toBe("what the link names"); + }); + + // ORC6c: the enclosing directory comes back, on each of the three ways out. + // A `` written after the region is what says where the document is + // standing — the path is relative, so it lands wherever cwd points. + it("ORC6c: the enclosing directory is restored after success, failure and cancellation", function* () { + const root = yield* useManagedRoot(); + const checkout = (yield* useOriginlessCheckout()).root; + + yield* runOrdinaryDocument( + [ + '', + "", + 'within', + "", + "", + "", + 'after', + "", + ].join("\n"), + { root, cwd: checkout }, + ); + expect(yield* exists(join(checkout, "inner", "within.md"))).toBe(true); + expect(yield* exists(join(checkout, "after-success.md"))).toBe(true); + + yield* runOrdinaryDocument( + [ + "", + '', + "", + 'no', + "", + "", + "", + "", + 'after', + "", + ].join("\n"), + { root, cwd: checkout }, + ); + // Beside the checkout root, not beside `inner`: the region restored cwd on + // its way out even though the content inside it failed. + expect(yield* exists(join(checkout, "after-failure.md"))).toBe(true); + expect(yield* exists(join(checkout, "inner", "after-failure.md"))).toBe(false); + + // And after cancellation. This one is observed rather than inferred from a + // later run: a fresh execution is handed its cwd explicitly, so where its + // files land says nothing about what the cancelled one restored. + // + // The gate is inside the region, so the halt lands with ``'s cwd + // installed. What the enclosing scope reads afterwards is the restoration. + const reached = withResolvers(); + const seen: string[] = []; + yield* scoped(function* () { + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd(): Operation { + return checkout; + }, + }, + { at: "min" }, + ); + const halted = yield* spawn(() => + runOrdinaryDocument('\n\n\n\n\n', { + root, + cwd: checkout, + components: [ + { + name: "Gate", + origin: "test", + props: { type: "object", additionalProperties: false }, + *fn(): Operation { + // Read from inside the region, so the pair below is + // "installed" then "restored" rather than one reading. + seen.push(yield* cwd()); + reached.resolve(); + yield* suspend(); + return ""; + }, + }, + ], + }), + ); + yield* reached.operation; + yield* halted.halt(); + seen.push(yield* cwd()); + }); + + expect(seen).toHaveLength(2); + expect(seen[0]).toBe(join(checkout, "inner")); + expect(seen[1]).toBe(checkout); + }); + + // ORC6d: the operation is about a directory. The Repository selection in + // scope, and every member of its identity, is the same inside the region and + // after it. + it("ORC6d: Repository selection and identity are unchanged by Dir", function* () { + const root = yield* useManagedRoot(); + const remote = yield* useBareRemote(REMOTE); + const checkout = (yield* useHostCheckout(remote.locator)).root; + const seen: (RepositorySelection | undefined)[] = []; + + yield* runOrdinaryDocument( + [ + '', + '', + "", + '', + "", + "", + "", + '', + "", + ].join("\n"), + { root, cwd: checkout, components: [selectionProbe(seen)] }, + ); + + expect(seen).toHaveLength(3); + // The whole selection, member by member: identifier, name, checkout path + // and every member of the identity. A Dir that had re-selected anything + // would differ in one of them. + expect(seen[0]).toBeDefined(); + expect(seen[1]).toEqual(seen[0]); + expect(seen[2]).toEqual(seen[0]); + }); + + // ORC6e: creation is persistent. A directory made inside a region whose + // content then fails is still there, and so is one made in a run that was + // cancelled — there is no rollback and no teardown removal on this profile. + it("ORC6e: a created directory survives failed content and cancellation", function* () { + const root = yield* useManagedRoot(); + const checkout = (yield* useOriginlessCheckout()).root; + + yield* runOrdinaryDocument( + '\n\n\nno\n\n\n\n', + { root, cwd: checkout }, + ); + expect(yield* exists(join(checkout, "made-then-failed"))).toBe(true); + + // Halted with a component still in flight inside the region. The gate is + // reached only after the ensure, so the halt lands strictly after creation + // — a halt that arrived first would prove nothing about teardown. + const reached = withResolvers(); + const halted = yield* spawn(() => + runOrdinaryDocument('\n\n\n\n\n', { + root, + cwd: checkout, + components: [ + { + name: "Gate", + origin: "test", + props: { type: "object", additionalProperties: false }, + *fn(): Operation { + reached.resolve(); + yield* suspend(); + return ""; + }, + }, + ], + }), + ); + yield* reached.operation; + yield* halted.halt(); + expect(yield* exists(join(checkout, "made-then-halted"))).toBe(true); + }); +}); diff --git a/packages/workflow/tests/workspace-files.test.ts b/packages/workflow/tests/workspace-files.test.ts index ac857585..eecef513 100644 --- a/packages/workflow/tests/workspace-files.test.ts +++ b/packages/workflow/tests/workspace-files.test.ts @@ -15,7 +15,16 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { expect } from "@executablemd/test-support/expect"; -import { createContext, race, scoped, sleep, suspend, type Operation } from "effection"; +import { + createContext, + race, + scoped, + sleep, + spawn, + suspend, + type Operation, + withResolvers, +} from "effection"; import { type Api, createApi } from "@effectionx/context-api"; import { collect, execute, inlineSource, registerComponents } from "@executablemd/core"; import type { Json } from "@executablemd/durable-streams"; @@ -28,6 +37,8 @@ import { withWorkflowWorkspace } from "../src/deno/workspace/host.ts"; import { WORKSPACE_FILE } from "../src/deno/workspace/files.ts"; import { throwWorkspaceFilesystemFailure } from "../src/deno/workspace/errors.ts"; import type { DenoWorkspaceFilesystem } from "../src/deno/workspace/filesystem.ts"; +import { join } from "node:path"; +import { exists } from "@effectionx/fs"; import { transactWorkspaceRoots } from "../src/deno/workspace/private.ts"; import type { PrivateWorkspaceTransaction } from "../src/deno/workspace/private.ts"; import { @@ -232,6 +243,19 @@ function countingRemoves( }); } +/** Every directory this run actually created, in order. */ +function countingMkdirs( + made: string[], +): (filesystem: DenoWorkspaceFilesystem) => DenoWorkspaceFilesystem { + return (filesystem) => ({ + ...filesystem, + *mkdir(path, options) { + made.push(path); + yield* filesystem.mkdir(path, options); + }, + }); +} + /** * A Workspace filesystem that stops one removal after it has happened. * @@ -252,6 +276,78 @@ function suspendingRemove( }); } +/** + * A creation that makes a parent and then refuses. + * + * The refusal a document can be told about arrives *after* part of the path + * exists, which is the only shape that exercises the savepoint: a target that + * refuses before anything is created rolls nothing back, and a test built on + * one would assert an empty Workspace that was never written to. + */ +function partialThenRefuse( + target: string, + parent: string, +): (filesystem: DenoWorkspaceFilesystem) => DenoWorkspaceFilesystem { + return (filesystem) => ({ + ...filesystem, + *mkdir(path, options) { + if (path !== target) { + yield* filesystem.mkdir(path, options); + return; + } + yield* filesystem.mkdir(parent, { recursive: true }); + // Planted the way the Workspace filesystem reports one, so it is a + // journalable condition rather than an infrastructure failure: an + // unrecognized platform error is deliberately fatal here, which is + // exactly what an unadorned `Error` would have produced. + throwWorkspaceFilesystemFailure( + Object.assign(new Error("planted"), { name: "WorkspaceFsError", code: "ENOTDIR" }), + ); + }, + }); +} + +function suspendingWrite( + target: string, + reached: { resolve(): void }, +): (filesystem: DenoWorkspaceFilesystem) => DenoWorkspaceFilesystem { + return (filesystem) => ({ + ...filesystem, + *writeFile(path, content, mode) { + yield* filesystem.writeFile(path, content, mode); + if (path === target) { + reached.resolve(); + yield* suspend(); + } + }, + }); +} + +/** + * A Workspace filesystem that stops one creation after it has happened. + * + * The suspension sits between the mutation and the transaction's commit, which + * is the one window where a Workspace holds a change nothing has published yet. + */ +function suspendingMkdir( + target: string, + reached: { resolve(): void }, +): (filesystem: DenoWorkspaceFilesystem) => DenoWorkspaceFilesystem { + return (filesystem) => ({ + ...filesystem, + *mkdir(path, options) { + yield* filesystem.mkdir(path, options); + if (path === target) { + // Signalled after the mutation and before the commit, so a halt that + // waits for this lands in that window rather than wherever a deadline + // happened to fall. + reached.resolve(); + yield* suspend(); + } + }, + }); +} + /** * A Workspace filesystem whose removal succeeds and then fails the run. * @@ -1346,3 +1442,271 @@ describe("WF workflow document filesystem", () => { }); }); }); + +/** What a logical path is, as a second connection sees it. */ +function* workspaceStat( + database: WorkflowRunDatabase, + path: string, +): Operation { + const read = yield* transactWorkspaceRoots(database, function* (workspace) { + try { + return (yield* workspace.filesystem.stat(path)).kind; + } catch { + return undefined; + } + }); + if (!read.ok) { + throw read.error; + } + return read.value; +} + +describe("Tier WF — the run's own directories", () => { + // ORC6h (cancellation): the two halves the replay case does not reach. + // + // Cancelled before the commit, nothing is published — the directory the + // Workspace briefly held is not visible to a second connection, no effect is + // recorded and the root has not moved. Cancelled after it, in the content + // that runs inside the region, the committed directory stays: the ensure is + // its own transaction and the content's fate is not its business. + it("ORC6h: cancellation before the commit publishes nothing", function* () { + const root = yield* useStorageRoot(); + const reached = withResolvers(); + yield* withStorage( + root, + function* () { + const database = yield* createRun(); + const path = runPath(root, database.record.runId); + const before = committedRoot(path); + + // Halted on the signal rather than on a deadline: a race that fired + // before `mkdir` ran would find the same empty Workspace and pass + // without ever reaching the window under test. + const running = yield* spawn(() => + raised(runDocument(database, '\n\nINSIDE\n\n\n')), + ); + yield* reached.operation; + yield* running.halt(); + + expect(yield* workspaceStat(database, "/never")).toBe(undefined); + expect(committedRoot(path)).toEqual(before); + expect(committedEffects(path, "ensure-directory", "/never")).toEqual(0); + expect(yield* workspaceEvents(database)).toEqual([]); + }, + { decorateFilesystem: suspendingMkdir("/never", reached) }, + ); + }); + + it("ORC6h: cancelling later content leaves the committed directory", function* () { + const root = yield* useStorageRoot(); + const reached = withResolvers(); + yield* withStorage( + root, + function* () { + const database = yield* createRun(); + const path = runPath(root, database.record.runId); + + // The halt lands inside the region, after the ensure committed: the + // write that suspends is the content running in the new directory, and + // the signal is what puts the halt there rather than a deadline. + const running = yield* spawn(() => + raised( + runDocument( + database, + '\n\nx\n\n\n', + ), + ), + ); + yield* reached.operation; + yield* running.halt(); + + // The directory and its effect survived the cancellation of the + // content that was running inside it — the ensure is its own + // transaction and the content's fate is not its business. + expect(yield* workspaceStat(database, "/committed")).toBe("directory"); + expect(committedEffects(path, "ensure-directory", "/committed")).toEqual(1); + // And the interrupted write published nothing. + expect(committedEffects(path, "write", "/committed/inside.md")).toEqual(0); + }, + { decorateFilesystem: suspendingWrite("/committed/inside.md", reached) }, + ); + }); + + // WF23: the mutation, its outcome and the resulting root are one commit, and + // the content runs after it. Ordering is read off the retained effects, in + // the order they committed — the nested `` landing proves nothing on + // its own, because a write creates its own parents recursively and would land + // either way. + it("WF23: recursive creation, its effect and the resulting root commit before content", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + const path = runPath(root, database.record.runId); + const before = committedRoot(path); + + const run = yield* runDocument( + database, + '\n\nlanded\n\n\n', + ); + + expect(yield* workspaceStat(database, "/one")).toBe("directory"); + expect(yield* workspaceStat(database, "/one/two")).toBe("directory"); + expect(yield* workspaceText(database, "/one/two/inside.md")).toBe("landed"); + // One ensure effect, and it committed *before* the write. + // + // The nested `` existing proves nothing on its own: a write creates + // its own parents recursively, so it would land whether or not `` + // had committed first. What settles the ordering is the retained order of + // the effects themselves. + const effects = yield* recordedFileEffects(database); + const ensures = effects.filter((effect) => effect.name.startsWith("ensure-directory:")); + expect(ensures).toHaveLength(1); + const order = effects.map((effect) => effect.name.split(":")[0]); + const ensured = order.indexOf("ensure-directory"); + const wrote = order.indexOf("write"); + expect(ensured).toBeGreaterThanOrEqual(0); + expect(wrote).toBeGreaterThan(ensured); + // The root moved, and the host filesystem was never reached for any of it. + expect(committedRoot(path)).not.toEqual(before); + expect(rootOfLastEvent(path)).toEqual(committedRoot(path)); + expect(run.host.seen).toEqual([]); + }); + }); + + // WF24: a refusal rolls its savepoint back. The target sits below a file, so + // the parent could only be created by an attempt that then had to be undone — + // which is what makes "no partial parent" a real assertion rather than a + // restatement of "nothing happened". + it("WF24: a refusal after a partial creation rolls the savepoint back", function* () { + const root = yield* useStorageRoot(); + yield* withStorage( + root, + function* () { + const database = yield* createRun(); + const path = runPath(root, database.record.runId); + yield* mutateWorkspace(database, function* (workspace) { + yield* workspace.filesystem.writeFile("/kept.txt", "kept"); + }); + const before = committedRoot(path); + + const run = yield* runDocument( + database, + [ + "", + '', + "", + "INSIDE", + "", + "", + "", + "", + 'yes', + ].join("\n"), + ); + + expect(run.output).toContain("not a directory"); + expect(run.output).not.toContain("INSIDE"); + expect(run.host.seen).toEqual([]); + + const recorded = yield* recordedFileEffects(database); + expect(recorded[0]?.result).toEqual({ + status: "ok", + value: { kind: "refused", phase: "access", reason: "not-directory" }, + }); + + // `/deep` was really created inside the savepoint and is gone again. + expect(yield* workspaceStat(database, "/deep")).toBe(undefined); + expect(yield* workspaceStat(database, "/deep/nested")).toBe(undefined); + // What the Workspace already held is what it still holds. + expect(yield* workspaceText(database, "/kept.txt")).toEqual("kept"); + // And this is what says it was the savepoint rather than the whole + // transaction: the next effect still commits. A rollback that took the + // transaction with it would lose this write too, and the refusal alone + // cannot tell the two apart. + expect(recorded[1]?.result).toEqual({ status: "ok", value: { kind: "written" } }); + expect(yield* workspaceText(database, "/after.txt")).toEqual("yes"); + expect(committedRoot(path)).not.toEqual(before); + // Sanitized: no host path, no platform code. + expect(run.output).not.toContain(root); + expect(run.output).not.toMatch(/ENOTDIR|ENOENT|errno/i); + }, + { decorateFilesystem: partialThenRefuse("/deep/nested", "/deep") }, + ); + }); + + // WF25: an existing populated directory is adopted, its contents survive, and + // the effect still commits so replay has something to restore. Also the + // absolute case: a logical path is used as written, is not rebased under the + // contextual directory, and never names anything on the host. + it("WF25: an existing directory is adopted, and an absolute path is logical", function* () { + const root = yield* useStorageRoot(); + yield* withStorage(root, function* () { + const database = yield* createRun(); + const path = runPath(root, database.record.runId); + yield* mutateWorkspace(database, function* (workspace) { + yield* workspace.filesystem.mkdir("/kept", { recursive: true }); + yield* workspace.filesystem.writeFile("/kept/inside.txt", "the bytes that were here"); + }); + + const run = yield* runDocument( + database, + '\n\nA\n\n\n\n\n\nB\n\n\n', + ); + + // Adopted, not replaced. + expect(yield* workspaceText(database, "/kept/inside.txt")).toBe("the bytes that were here"); + // The absolute path named exactly that logical path. Not rebased under the + // contextual directory — `/kept/nested` would be the rebased spelling and + // it does not exist. + expect(yield* workspaceStat(database, "/nested/deep")).toBe("directory"); + expect(yield* workspaceStat(database, "/kept/nested")).toBe(undefined); + expect(yield* workspaceText(database, "/nested/deep/b.md")).toBe("B"); + // And it named nothing on the host: neither the logical path taken as a + // host path, nor one beneath the run's own storage. + expect(yield* exists("/nested/deep")).toBe(false); + expect(yield* exists(join(root, "nested", "deep"))).toBe(false); + expect(run.host.seen).toEqual([]); + // Both ensures committed, and the last event's root is the run's own. + const ensures = (yield* recordedFileEffects(database)).filter((effect) => + effect.name.startsWith("ensure-directory:"), + ); + expect(ensures).toHaveLength(2); + expect(rootOfLastEvent(path)).toEqual(committedRoot(path)); + }); + }); + + // WF26: replay restores rather than repeats. The directory is privately + // removed between the two runs, so a second ensure would be visible as it + // coming back — and the counter says which of the two happened rather than + // leaving it to be inferred. + it("WF26: a completed replay restores the outcome without creating again", function* () { + const root = yield* useStorageRoot(); + const made: string[] = []; + yield* withStorage( + root, + function* () { + const database = yield* createRun(); + const path = runPath(root, database.record.runId); + const source = '\n\nINSIDE\n\n\n'; + + const first = yield* runDocument(database, source); + expect(first.output).toContain("INSIDE"); + expect(made).toEqual(["/once"]); + const after = committedRoot(path); + + // Removed behind the run's back, so a second creation would show. + yield* mutateWorkspace(database, function* (workspace) { + yield* workspace.filesystem.remove("/once", { recursive: true }); + }); + + const replayed = yield* replayDocument(database, source); + expect(replayed.output).toContain("INSIDE"); + // No second ensure: the counter is unchanged. + expect(made).toEqual(["/once"]); + // And the retained root is what a replay restores, not a new one. + expect(rootOfLastEvent(path)).toEqual(after); + }, + { decorateFilesystem: countingMkdirs(made) }, + ); + }); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 214c14bb..81786c4b 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -398,6 +398,9 @@ importers: '@executablemd/durable-streams': specifier: workspace:* version: link:../durable-streams + '@executablemd/runtime': + specifier: workspace:* + version: link:../runtime effection: specifier: 4.1.0 version: 4.1.0 diff --git a/scripts/files-contract-probe.ts b/scripts/files-contract-probe.ts index 7f548162..f6dd5dfa 100644 --- a/scripts/files-contract-probe.ts +++ b/scripts/files-contract-probe.ts @@ -198,6 +198,43 @@ await main(function* () { ); check("nothing beyond that link was removed", yield* exists(join(outside, "secret.txt"))); + // The directory act, which the compiled binary must carry like the rest. A + // relative target creates every missing parent; an absolute one names that + // exact location and is used as written, which is the one place this contract + // admits an absolute path at all. + check( + "a missing relative directory and its parents are created", + (yield* Files.operations.ensureDirectory({ cwd: workspace, path: "made/deep" })).ok && + (yield* until(lstat(join(workspace, "made", "deep")))).isDirectory(), + ); + yield* writeTextFile(join(workspace, "made", "deep", "kept.txt"), "still here"); + check( + "an existing directory is used without clearing it", + (yield* Files.operations.ensureDirectory({ cwd: workspace, path: "made/deep" })).ok && + (yield* readTextFile(join(workspace, "made", "deep", "kept.txt"))) === "still here", + ); + check( + "an absolute directory is created where it names", + (yield* Files.operations.ensureDirectory({ cwd: workspace, path: join(outside, "absolute") })) + .ok && (yield* until(lstat(join(outside, "absolute")))).isDirectory(), + ); + yield* writeTextFile(join(workspace, "occupied"), "a file"); + check( + "a file where a directory was asked for is refused", + reasonOf(yield* Files.operations.ensureDirectory({ cwd: workspace, path: "occupied" })) === + "not-directory", + ); + check( + "a non-directory on the way to the target is refused", + reasonOf( + yield* Files.operations.ensureDirectory({ cwd: workspace, path: "occupied/below" }), + ) === "not-directory", + ); + check( + "the refused file is still a file", + (yield* readTextFile(join(workspace, "occupied"))) === "a file", + ); + let temporary = ""; yield* scoped(function* () { temporary = valueOf(yield* Files.operations.temporaryDirectory()) ?? ""; diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 1f0f4313..22765142 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -563,8 +563,9 @@ const COMPILED_BINARY: RuntimeExclusion[] = [ * * It holds a managed checkout with a kernel-released exclusive advisory lock, * which this repository reaches through the Deno runtime and which Node and Bun - * expose no equivalent of. The provider is not reachable from any entrypoint - * yet — these suites install it themselves through the trusted test installer. + * expose no equivalent of. The declarations it installs, and the fact that a + * runtime without the provider refuses every operation, are covered portably by + * `packages/cli/tests/run-composition.test.ts`, which runs everywhere. */ const DENO_ONLY_REPOSITORY_PROVIDER: RuntimeExclusion[] = [ { @@ -585,6 +586,18 @@ const DENO_ONLY_REPOSITORY_PROVIDER: RuntimeExclusion[] = [ "the same provider, publishing and reconciling against a modeled Git host; the transport and its evidence are Deno-only for the same reason the rest of the provider is", issue: DERIVED_SCOPE, }, + { + path: "packages/cli/tests/run-composition-deno.test.ts", + reason: + "the same provider reached through an ordinary run, asked the questions only a runtime that operates repositories can answer: managed-Worktree session placement, an ordinary journal's diagnostic status, and diagnostic-trace non-authority", + issue: DERIVED_SCOPE, + }, + { + path: "packages/cli/tests/run-composition-nested.test.ts", + reason: + "a nested execution's own provider instance, which needs the same kernel-released advisory lock the root's does, and real subprocesses to prove where a child stands", + issue: DERIVED_SCOPE, + }, ]; const BUN_MISSING_NODE_SQLITE: RuntimeExclusion[] = [ diff --git a/scripts/smoke-run-composition.ts b/scripts/smoke-run-composition.ts new file mode 100644 index 00000000..d078f16b --- /dev/null +++ b/scripts/smoke-run-composition.ts @@ -0,0 +1,223 @@ +/** + * Repository composition through the compiled binary (#643). + * + * The ordinary provider is assembled at a runtime-named entrypoint, holds its + * managed checkouts with a kernel-released advisory lock, and discovers the + * ambient repository from the directory the command was run in. Every one of + * those is a fact about the program that is running, so only the binary shows + * they survived `deno compile`. + * + * Four claims, each observed from outside the process: + * + * 1. a root-level `` belongs to the repository the binary was run in, + * and a command inside it runs there; + * 2. that checkout is a real linked worktree — `.git` is a file — and it is + * still on disk after the process exits; + * 3. a second binary, run while the first still holds the slot, is refused + * without waiting and changes nothing; and + * 4. once the first exits, the slot is taken by the next one, which finds the + * same checkout. + * + * The managed root is a temporary directory named through the same environment + * a person's would be reached through, so nothing here touches + * `~/.xmd/repositories`. + */ + +import { main } from "effection"; +import { sleep, until } from "effection"; +import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { useTempDirectory } from "./lib/temp-directory.ts"; +import * as path from "node:path"; + +const BINARY = path.join(Deno.cwd(), "dist", "xmd"); + +function fail(claim: string): never { + console.error(`run-composition smoke: ${claim}`); + Deno.exit(1); +} + +/** Git, run with an environment a caller's own configuration cannot reach. */ +function git(args: readonly string[], cwd: string, home: string): string { + const outcome = new Deno.Command("git", { + args: [...args], + cwd, + env: { + PATH: Deno.env.get("PATH") ?? "", + HOME: home, + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_CONFIG_SYSTEM: "/dev/null", + GIT_CONFIG_NOSYSTEM: "1", + GIT_TERMINAL_PROMPT: "0", + LC_ALL: "C", + GIT_AUTHOR_NAME: "Smoke", + GIT_AUTHOR_EMAIL: "smoke@example.invalid", + GIT_COMMITTER_NAME: "Smoke", + GIT_COMMITTER_EMAIL: "smoke@example.invalid", + }, + clearEnv: true, + stdout: "piped", + stderr: "piped", + }).outputSync(); + const printed = new TextDecoder().decode(outcome.stdout).trim(); + if (!outcome.success) { + fail(`git ${args.join(" ")} failed: ${new TextDecoder().decode(outcome.stderr)}`); + } + return printed; +} + +/** + * One invocation of the compiled binary on the smoke document. + * + * The long-lived one inherits its streams. A piped stream nobody is draining is + * a buffer that fills, and the run this script gates on is deliberately held + * open — so the one invocation that must not be blocked by its own output is + * the one whose output nothing is reading. + */ +function binary( + cwd: string, + env: Record, + streams: "piped" | "null" = "piped", +): Deno.Command { + return new Deno.Command(BINARY, { + args: ["run", "smoke.md", "--raw"], + cwd, + env, + clearEnv: true, + stdout: streams, + stderr: streams, + }); +} + +function decode(bytes: Uint8Array): string { + return new TextDecoder().decode(bytes); +} + +/** The environment every binary invocation in this smoke runs under. */ +function environment(home: string, managed: string): Record { + return { + PATH: Deno.env.get("PATH") ?? "", + // The managed root follows `HOME`, so this smoke never reaches the real + // `~/.xmd/repositories` and never needs an option that does not exist. + HOME: managed, + GIT_CONFIG_GLOBAL: path.join(home, ".gitconfig"), + GIT_CONFIG_SYSTEM: "/dev/null", + GIT_CONFIG_NOSYSTEM: "1", + GIT_TERMINAL_PROMPT: "0", + LC_ALL: "C", + }; +} + +await main(function* () { + if (!(yield* exists(BINARY))) { + fail(`no compiled binary at ${BINARY} — run \`deno task build\` first`); + } + + const home = yield* useTempDirectory("xmd-smoke-orc-home-"); + const managed = yield* useTempDirectory("xmd-smoke-orc-managed-"); + const workspace = yield* useTempDirectory("xmd-smoke-orc-"); + const checkout = path.join(workspace, "checkout"); + + // A configured identity, because an ordinary run commits as the invoking user + // and refuses when this host cannot say who that is. + yield* writeTextFile( + path.join(home, ".gitconfig"), + ["[user]", "\tname = Smoke Runner", "\temail = smoke@example.invalid", ""].join("\n"), + ); + + git(["init", "--initial-branch=main", checkout], workspace, home); + git(["commit", "--allow-empty", "-m", "first"], checkout, home); + const started = git(["rev-parse", "HEAD"], checkout, home); + + // 1 and 2. A root-level Worktree of the repository the binary was run in, a + // command inside it, and a gate the document holds itself open on. The + // command writes where it is standing to a file rather than to its own + // output, so what this script reads is the checkout Git resolved rather + // than a line it had to parse out of a rendered document. + const release = path.join(workspace, "release"); + const marker = path.join(workspace, "standing-in"); + yield* writeTextFile( + path.join(checkout, "smoke.md"), + [ + "# Ordinary repository composition", + "", + '', + "", + "", + "", + "```bash exec", + `git rev-parse --show-toplevel > ${marker}; while [ ! -f ${release} ]; do sleep 0.05; done`, + "```", + "", + "", + "", + ].join("\n"), + ); + + const holding = binary(checkout, environment(home, managed), "null").spawn(); + + // Observed while the child is still waiting for a file this script has not + // written yet, so the slot is genuinely held when the second binary asks for + // it. + for (let attempt = 0; !(yield* exists(marker)); attempt += 1) { + if (attempt > 600) { + fail("the document never reached its worktree command"); + } + yield* sleep(100); + } + + // 3. A second binary, while the first still holds the slot. + const contended = yield* until(binary(checkout, environment(home, managed)).output()); + const reported = decode(contended.stdout) + decode(contended.stderr); + if (contended.success) { + fail("a second process was allowed into a slot the first was holding"); + } + if (!reported.includes("another process is working in")) { + fail(`a second process refused for the wrong reason: ${reported}`); + } + + yield* writeTextFile(release, "go\n"); + const first = yield* until(holding.status); + if (!first.success) { + fail(`the holding run exited ${first.code}`); + } + + // Where the command inside the Worktree was actually standing. + const slot = (yield* readTextFile(marker)).trim(); + + // The worktree the run made is still there, and it is a real linked one. + if (!(yield* exists(slot))) { + fail(`the managed worktree did not survive the run: ${slot}`); + } + const administration = yield* readTextFile(path.join(slot, ".git")); + if (!administration.startsWith("gitdir:")) { + fail(`the managed worktree's .git is not a file naming its repository: ${administration}`); + } + if (git(["rev-parse", "--show-toplevel"], slot, home) !== slot) { + fail("the managed worktree does not report itself as its own checkout root"); + } + // It belongs to the ambient repository, which is what "ambient" means, and + // the ambient checkout was left where it was. + if (git(["rev-parse", "HEAD"], checkout, home) !== started) { + fail("the ambient checkout moved"); + } + if (git(["rev-parse", "--abbrev-ref", "HEAD"], slot, home) !== "smoke") { + fail("the managed worktree is not on the branch the document asked for"); + } + + // 4. The slot is free again, and the next run finds the same checkout. The + // marker is removed first, so what it holds afterwards is that run's own + // answer rather than the first one's. + yield* rm(marker); + const later = yield* until(binary(checkout, environment(home, managed)).output()); + if (!later.success) { + fail(`the slot was not released for a later run: ${decode(later.stderr)}`); + } + if ((yield* readTextFile(marker)).trim() !== slot) { + fail("a later run did not reuse the checkout the first one made"); + } + if (!(yield* exists(slot))) { + fail("the managed worktree did not survive the later run"); + } + + console.log("run-composition smoke: ok"); +}); diff --git a/scripts/tests/ci-workflow.test.ts b/scripts/tests/ci-workflow.test.ts index 922d5b16..19bf9dc2 100644 --- a/scripts/tests/ci-workflow.test.ts +++ b/scripts/tests/ci-workflow.test.ts @@ -270,6 +270,7 @@ describe("the CI smoke job", () => { "scripts/smoke-foreground.ts", "scripts/smoke-loaded-copy.ts", "scripts/smoke-fetch.ts", + "scripts/smoke-run-composition.ts", ]) { expect(commands).toContain(script); } diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 089ec6e2..b32d0cfb 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -116,17 +116,42 @@ paths relative to cwd, and the engine's own file access is written that way: component search directories (`["./components", "./"]`) are relative, and resolved paths in the journal (`"components/Greeting.md"`) are relative. +#### Lexical working directories + +Three components produce the contextual working directory the operations above +resolve against, and each restores the enclosing one when its invocation ends — +on success, failure and cancellation alike. + +- `` expands its content at the selected checkout. +- `` expands its content at the linked checkout it + selected of the Repository in scope. +- `` recursively creates or adopts `path` before content begins, then + expands its content there. A relative path is read against the directory + already in effect and an absolute path keeps its existing meaning. It selects + no repository. + +Written self-closing with `as`, Repository and Worktree render nothing and bind +the checkout path instead, which is what lets a later sibling `` +render descendants there. + +Every consumer of the contextual working directory observes the selected +checkout: a process a document runs, a `` an agent is launched into, +and every relative document path. Which repository a Git element acts on is +therefore decided by where it is written, and the same element inside a `` +at a linked worktree acts on that worktree. + #### Document data and engine control plane Two kinds of filesystem access are separate boundaries, and the separation is what lets one document mean the same thing in two environments. -**Document data** — the files a document names in its own text — goes through -`API.Files`, a contextual Api of whole semantic operations. `` (§6.13), -`` (§6.13.1), `` (§6.14), and `` (§6.11) speak only -that Api, hold no host path, and never learn which provider answered. `xmd run` installs a host -provider that resolves those paths in the caller's filesystem; a workflow run -installs one whose paths name entries in a logical filesystem the run owns. +**Document data** — the files and directories a document names in its own text +— goes through `API.Files`, a contextual Api of whole semantic operations. +``, `` (§6.13), `` (§6.13.1), `` (§6.14), and +`` (§6.11) speak only that Api, hold no provider handle, and never learn +which provider answered. `xmd run` installs a host provider that resolves those +paths in the caller's filesystem; a workflow run installs one whose paths name +entries in a logical filesystem the run owns. The Api has **no host default**: with no provider installed, every operation fails the execution rather than reaching the host. @@ -135,12 +160,27 @@ replay guards, the eval compiler, the diagnostic journal, and the test target — reads host paths the caller selected, through the low-level `API.Fs`. Those are not document-addressable, and they stay where they are. -A path a document authored is always relative and always resolved by the -provider, against the contextual `Env.cwd` the component supplies with it. -Nothing the provider resolves reaches a printed error or the journal: what -crosses back is a reason from a fixed vocabulary (§6.13), never a resolved -path, a symlink target, a temporary name, an errno code, or a platform -message. +The ordinary file-operation paths a document authors are relative and resolved +by the provider against contextual `Env.cwd`. Directory placement preserves +Dir's existing exception: an absolute `path` is used as written, while a +relative one is resolved against the enclosing cwd. Nothing the provider +resolves reaches a printed error or the journal: what crosses back is a reason +from a fixed vocabulary (§6.13), never a resolved path, a symlink target, a +temporary name, an errno code, or a platform message. + +`API.Files.ensureDirectory` is the mandatory provider-neutral directory act. +It recursively creates a missing target, succeeds for an existing directory +without replacing it or clearing its contents, and refuses a file or another +non-directory. Its success carries no path or handle. The absent-provider +terminal throws like every other Files operation, and the host and workflow +providers must both implement it; the Api's stable name is what lets separately +loaded copies compose. + +Dir asks for that ensure before installing the target as cwd and before +expanding content. If it refuses, content does not begin. After it succeeds, the +directory persists and content failure or cancellation does not remove it; the +component's scope restores only the enclosing cwd. Repository context is not +installed or replaced by Dir. #### The contextual working directory @@ -1910,7 +1950,7 @@ run but are absent from the diagnostic trace. | `packages/cli/src/workflow.ts` | the runtime-neutral `xmd workflow` lifecycle: grammar, run opening, execution records, statuses and exit codes | | `packages/cli/src/workflow-definition.ts` | establishing an immutable Git definition from a working-tree path, and loading a retained one again | | `packages/cli/src/deno-workflow.ts` | the Deno run store and Workspace attachment; Node and Bun install the refusing host instead | -| `packages/workflow/src/deno/workspace/files.ts` | the transaction-bound `API.Files` provider — one durable Workspace effect per document read, write and search | +| `packages/workflow/src/deno/workspace/files.ts` | the transaction-bound `API.Files` provider — one durable Workspace effect per document read, write, directory ensure and search | | `packages/workflow/src/deno/workspace/host.ts` | `withWorkflowWorkspace()` — the run's effect coordinator, logical cwd `/`, and Files provider installed together inside one execution | | `packages/workflow/src/journal.ts` | the `workflow_run` record, canonical-record recognition, and the refusals that name differing fields without their values | | `packages/workflow/src/run.ts` | `workflowInstallation()` / `retainedWorkflowInstallation()` — the `ExecutionInstallation` values a trusted host passes to `executeInstalled()`, each contributing a mandatory run-identity admission and the `prepare` hook that creates or restores the run inside the durable root | @@ -2634,6 +2674,23 @@ absence falls through to a default: a candidate that exists but cannot be read, imported, parsed, or compiled fails where it is loaded, so a broken local component is never quietly replaced. +#### The run profile's repository declarations + +Thirteen names — `Repository`, `Worktree`, `Dir`, `Git.Switch`, `Git.Add`, +`Git.Commit`, `Git.Push`, `PullRequest`, `PullRequest.Reviews`, +`PullRequest.Comments`, `PullRequest.Checks`, `IssueTracker` and `Issue` — are +ordinary registered defaults in tier 5. A repository-local Markdown or +TypeScript component of any of those names is chosen ahead of them, exactly as +it is ahead of any other package's default, and for its own scope alone. + +They are one array with several consumers: an ordinary document execution, a +workflow attachment, `xmd syntax`, and `xmd plan`'s validation and generation. +Registering them installs no provider, discovers no repository, acquires no +lock, spawns no Git and reads no credential — describing an environment mints +nothing. What each name *does* is decided by whichever repository provider the +command installed (§8.1), and a host that installed none still resolves every +one of them. + The bundle tier exists only inside a workflow run, and such a run searches no repository directories at all — so a declared name resolves to the exact source its pinned commit holds and to nothing beside it in a mutable checkout. Core's @@ -8457,6 +8514,52 @@ workflow and returns a `DocumentExecution` handle. Options: - `secretDetection?` — detect credentials before durable events persist (default: enabled) +#### The host's repository provider + +A document execution reaches repository operations only through the provider +the command installed inside the execution scope, before the root document is +imported. There are two, and they differ in lifetime and authority rather than +in what an author writes. + +The **ordinary provider** is what the Deno source entrypoint and the compiled +binary install for `xmd run` and for an approved `xmd plan --run`. Constructing +it mints a fresh opaque invocation identity and empty state, both private to +that one execution: + +- **Ambient discovery** happens once, before root expansion, from the + invocation's starting directory: the canonical checkout root, the canonical + common Git directory, the object format, the current HEAD and branch, the + locally recorded admitted `origin` when there is one, and the recorded default + branch. Being outside a Git checkout is not a startup failure — only an + element that needs a repository refuses, and it names how to run inside one. +- **The Git identity** an ordinary commit records is the invoking user's own, + read once from the trusted host's environment and configuration. It is used + for `` alone and is not otherwise observable; a host that can name + no identity refuses that one component and leaves every other one usable. + Nothing else crosses from the caller's environment: hooks, file-system + monitors, signing programs and repository-supplied credential helpers stay + disabled. +- **The invocation identity** names this execution to a service. It is not a + prop, a Context value, a component result, a middleware answer or a journal + event; it is neither addressable nor reusable, and the engine's own + `Expansion.id` names the authored site inside it. +- **The journal is not authority.** `--journal` writes a diagnostic trace that + starts from a path that did not exist; nothing reads one back. A run with an + in-memory stream and a run with `--journal` perform the same live operations, + the same number of times, and a later run starts with a new identity, empty + evidence and a new request. + +The **retained provider** is what a workflow host installs inside its Workspace +attachment. Its operations are durable effects keyed by the WorkflowRun, so a +completed one restores from the journal without contacting anything. + +Node and Bun install neither. They register the same declarations (§5.3), and +every repository operation there reports an absent provider before a lock, a +credential, a subprocess or a request exists. + +A nested `` child constructs a provider of its own, so its +identity, its locks and its evidence do not reach its parent or a sibling. + #### Secret detection Every execution refuses to persist a durable event that carries a credential. @@ -9087,8 +9190,8 @@ how an eval block compiles, how a service process is hosted, or which runtime it is on**. A runtime-named entrypoint — `deno.ts`, `node.ts`, `bun.ts`, `compiled.ts` — installs its `API.Env` providers with `{ at: "min" }` and passes the matching service -installer and its own upgrade assembly to -`runXmd(args, installService, upgrade)`: +installer, its own upgrade assembly and its own repository installer to +`runXmd(args, installService, upgrade, installRepositories, ...)`: ```typescript yield* API.Env.around( @@ -9103,7 +9206,14 @@ yield* API.Env.around( }, { at: "min" }, ); -yield* runXmd(args, useDenoService, UPGRADE); +yield* runXmd( + args, + useDenoService, + UPGRADE, + denoRunRepositories(HELPER), + () => useDenoWorkflowHost(HELPER), + useMachineSessions(), +); ``` The upgrade assembly is a required parameter rather than an optional one. It @@ -9114,10 +9224,30 @@ document nothing through which a release could be read, downloaded or replaced, and inheriting that authority by omission is impossible. The complete contract is [`xmd upgrade`](./upgrade-command-spec.md). -The installer is invoked only for `xmd run` and `xmd test`, immediately before -`execute()`. Help, inspection and agent-worker paths never install or attach a -service. Document-aware `xmd run … --help` is one of those inspection paths and -never invokes the installer. +The repository installer is required for the same reason and states a different +fact: what this runtime can *operate*, rather than what it is. Deno and the +compiled binary pass the live provider — `denoRunRepositories(HELPER)`, built +from the same credential-helper assembly the workflow host uses — and Node and +Bun pass `unsupportedRepositories`. Every runtime registers the same thirteen +declarations either way, so `xmd syntax` describes one language everywhere and a +runtime that operates none of it reports an absent provider before a lock, a +credential, a subprocess or a request exists (§5.3, §8.1). + +Being *supplied* by the entrypoint and being *used* are different things, and +the commands differ: + +- an ordinary `xmd run`, and an approved `xmd plan --run`, install a fresh + provider instance for that execution; +- `xmd test` installs none for its own root document, and hands the + entrypoint's installer to a `host="run"` child — which is an ordinary run + whatever command hosts it, and receives an instance of its own; +- a workflow execution installs none at all, because its Workspace attachment + owns repositories for the run. + +The service installer is invoked only for `xmd run` and `xmd test`, immediately +before `execute()`. Help, inspection and agent-worker paths never install or +attach a service. Document-aware `xmd run … --help` is one of those inspection +paths and never invokes the installer. A `xmd run` that refuses its document target (§5.4) may or may not have reached the installer: the refusal comes before it when an inspection discovered the @@ -10153,13 +10283,18 @@ platform's. | HF21 | Containment before removal | Empty, absolute, lexically escaping, and parent-link-escaping paths are each refused with their own phase and reason and reach no low-level removal — proven against an admitted deletion that does | | HF22 | Platform failures | A refused removal is a structured `Err` carrying a reason and nothing else; one that reports the path already gone is the same success absence is | | HF23 | Cancellation | A halt before the removal produces no Result, leaves the entry as it was, and leaves nothing behind — deletion acquires nothing | +| HF24 | Recursive directory ensure | A missing relative target and all missing parents are created before `` content begins; the same operation accepts the exact absolute target without rebasing it | +| HF25 | Existing directory | An existing target succeeds without replacement or clearing, and planted contents remain byte-identical after the body | +| HF26 | Non-directory refusal | A regular file and a supported special entry, at the target and as an intermediate entry, each refuse before content with a sanitized reason and no host path, platform code, raw message or cause | +| HF27 | Direct persistence | A directory created by `` remains after successful content, failed content and cancellation; cwd restoration performs no teardown deletion | +| HF28 | Repository independence | Ensuring and entering a directory beneath a selected checkout leaves every Repository identity member and provider selection unchanged | ### Tier FF — Files infrastructure failure | # | Test | Verify | |---|------|--------| | FF1 | Absence before children | A write with no provider fails the execution, expands no children, renders nothing, reaches no `API.Fs` call, and stops the following sibling | -| FF2 | Every other form | Read, ``, ``, and `` each stop at their first provider call, and nothing after them expands | +| FF2 | Every other form | Read, ``, ``, ``, and `` each stop at their first provider call, and nothing after them expands | | FF3 | A refused operation | Denial is fatal, carries its own fixed diagnostic, and renders no content | | FF4 | The check authorizes nothing | A refused check makes exactly one provider call; the children never run and the document carries on | | FF5 | Malformed write data | A phase and target that contradict each other, a reason outside the vocabulary, and an undescribable success are all fatal `protocol` invariants, and the category is not interpolated | @@ -10806,7 +10941,7 @@ Defined in [Workflow runs](./workflow-spec.md) §10. | # | Test | Verify | |---|------|--------| -| WF1 | Public routing | `` and `` reach the run's logical Workspace, and a host `API.Files` spy installed outside the run observes nothing for any read, write, refusal or search | +| WF1 | Public routing | ``, `` and `` reach the run's logical Workspace, and a host `API.Files` spy installed outside the run observes nothing for any read, write, directory ensure, refusal or search | | WF2 | Atomic write | File bytes, the current-root pointer and one filtered Yield are all visible to a second connection together, and the newest journal row names the published root | | WF3 | Recorded read | A read is its own durable effect whose recorded value is the content it read | | WF4 | Historical read | A read restores its recorded content where the current frontier holds something else | @@ -10828,6 +10963,10 @@ Defined in [Workflow runs](./workflow-spec.md) §10. | WF20 | Rollback after the removal | An infrastructure failure between the removal and publication rolls the outer transaction back: the file, the committed root and the committed effect history are all the ones the run started with | | WF21 | Completed replay | A completed deletion restores its outcome without a second low-level removal, even where the path was privately recreated afterwards, and appends nothing | | WF22 | Cancelled deletion | A halt between the removal and the commit publishes no outcome and moves no root; the continuation, which has no record of it, performs and records the deletion exactly once | +| WF23 | Atomic directory ensure | A recursive creation, its one `workspace_file` outcome and the resulting current Workspace root become visible together; content begins only after that commit | +| WF24 | Directory refusal rollback | A target or intermediate non-directory rolls its savepoint back, publishes only the sanitized refusal against the unchanged root and begins no content; neither a partial parent nor host detail survives | +| WF25 | Existing directory and retained root | Ensuring an existing populated directory preserves its contents and commits a successful effect whose resulting root is the run's authoritative current root | +| WF26 | Directory replay and cancellation | Completed replay restores the recorded success and retained root without creating again; cancellation before commit publishes no effect or root, while cancellation of later content leaves the committed directory and restores the enclosing cwd | ### Tier EP — The execution protocol @@ -11057,7 +11196,7 @@ Defined in [Workflow workspaces](./workflow-workspace-spec.md) §8.4. | GXC1 | The selection | Omitting `allow` and stating `read` produce one identical retained policy; a mixed selection is retained in canonical class order, with the read table's entries before the write table's and host order inside each | | GXC2 | Unstateable policy | An empty selection, one class twice, a selected class with no table or an empty one, and a host table holding one name with overlapping forms or two definitions each fail before the candidate is parsed — with a deliberately unparseable candidate, no `generated_xmd` record and nothing of the candidate retained | | GXC3–GXC4 | Name and form | A self-closing and a paired `` in one fragment resolve to the read and the write identity and are retained with their forms; the opposite form under a single-class selection is refused with no read and no write | -| GXC5–GXC6 | Authority | No admitted name — `` in either form, `` or the self-closing `` — is answered by a same-name repository component, the dotted one included; and Git push, pull request, issue, repository, glob and an executable block are outside the tables whatever the selection | +| GXC5–GXC6 | Authority | No admitted name — `` in either form, the versioned paired `` or the self-closing `` — is answered by a same-name repository component, the dotted one included; and Git push, pull request, issue, repository, glob and an executable block are outside the tables whatever the selection | | GXC7 | The result | A write-only fragment observes nothing and renders nothing; a mixed one collects the read's value and not the write's | | GXC8 | Preflight | An unadmitted sibling after an admitted write, an unadmitted child under an admitted parent, and an unadmitted form after an admitted write each perform no write at all | | GXC10 | The form at the invocation | An admitted read still performs exactly its read and an admitted write still performs exactly its write, under a `Component.hasContent` handler outside the generated expansion that lies consistently *and* under one answering `[false, true]` / `[true, false]` — with the retained identity and form unmoved, the read's value collected, and the file bytes proving which effect ran. An admitted read beside each of them reports the chain's answer and consumes exactly one, so every case proves the handler installed and answering before it proves the element ignored it — an inert handler, and an element that consulted the chain and took the next answer, both fail. An invocation the evaluator did not receive is refused with no provider call, and a handler that observes and delegates leaves both forms running through their ordinary providers | @@ -11091,12 +11230,13 @@ Defined in [Workflow workspaces](./workflow-workspace-spec.md) §8.4. | WGAC7 | The import boundary | Through public `importComponent` middleware: calling the implementation with an invocation it built, with the first site's routed at the second, with a live content-bearing parent's routed into the sites inside it, by keeping the declared implementation and running it at another element's invocation, and by keeping one attachment's implementation — with that attachment's whole registration record answered for the name inside a second, simultaneously live attachment — and running it at the second attachment's own `` site, are each refused. Nothing is admitted and no request performed under an identity the author wrote no observation at, and neither run's database receives a record the other's expansion named: a redirected registry answer carries behavior, never the claimant this execution delivered. Honest delegation, forwarding the genuine invocation, admits each site under its own name, nested or not; and an interrupted run resumes into each site's own record — the retained observation restored without re-reading, the interrupted request performed | | WGAC8 | The result shape | The result carries each admitted read's name and returned value in invocation order, with the fragment's rendering under `output` — an admitted `` renders nothing and its response survives anyway. Which pinned identity produced one is not in the value: the retained admission holds that | | WGAC9 | The class selection | An empty, repeated, unknown or non-array `allow` is refused before any admission; `read`, `write` and both in either order are admitted and retained in canonical order; omitting it retains exactly `read`, and the write table is not in that policy at all | -| WGAC10 | The standard write table | An admitted paired `` beneath a generated `` lands in the run's own Workspace at the nested logical path, through the ordinary file effect; the retained policy holds all three standard entries with their forms, in the order the profile states them — core's paired `File:write`, the composition package's paired `Dir`, then core's self-closing `File.Delete` — and a `` and a self-closing `` are each refused under the same selection with no effect | +| WGAC10 | The standard write table | An admitted paired `` beneath a generated `` first recursively creates the directory through one ordinary `workspace_file` effect, then lands the file at the nested logical path through another; the retained policy holds all three standard entries with their forms, in the order the profile states them — core's paired `File:write`, paired `@executablemd/workflow/composition/dir-v2#Dir`, then core's self-closing `File.Delete` — and a `` and a self-closing `` are each refused under the same selection with no effect | | WGAC14 | The admitted deletion | A write-only fragment holding one self-closing `` removes the file from the run's own Workspace, read back through the run's own transaction; it performs exactly one further `workspace_file` effect, whose outcome is `{ kind: "deleted" }`; it is retained under `@executablemd/core#File.Delete` in the self-closing form; and the document binds exactly `{ observations: [], output: "" }` with no receipt of any kind | | WGAC15 | Preflight covers the deletion | A fragment placing an admitted `` first and an executable code block second is refused for the block — the class proving the deletion was admitted, since a refusal names no name — and the file remains with no deletion effect performed or retained | | WGAC11 | What a selection binds | A write-only evaluation binds `observations: []`, and a mixed one binds exactly the read's entry while the write lands in the Workspace | | WGAC12 | Committed mutations | A completed replay of a write-enabled document journals nothing new, performs no second mutation, and leaves the retained content | | WGAC16 | Bundled continuation | `` inside a committed bundled Markdown component, then a parent `` write and the real `` outside it, for a generated read and a generated write alike: the real start suspends holding the exactly ordered `generated_xmd → nested workspace_file → parent workspace_file → suspension_request` subsequence; answer delivery and the completed resume leave the journal counts, that subsequence, the Workspace root-publication count and the authoritative current root unchanged; and the delivered value reaches the document after the wait. `API.Files` component calls are not evidence here — re-expansion legitimately enters that boundary before the durable effect restores | +| WGAC17 | Directory mutation authority | `allow={["write"]}` admits the versioned paired `` and intentionally authorizes its persistent recursive creation; a continuation retaining the former `@executablemd/workflow/composition#Dir` refuses before generated execution and creates nothing, while an unchanged current admission replays without a second ensure | ### Tier WAL — The workflow Agent observation loop @@ -11735,6 +11875,48 @@ drives the `xmd plan` command. | VB9 | Host false is overridden to true for one lexical subtree | | VB10 | Host true is overridden to false, the body is skipped, and sibling state is restored | +### Tier ORC — Repository composition under an ordinary run (§5.3, §8.1) + +Every case distinguishes what a *document* observed from what the *host* +holds, because that is the boundary the two profiles differ across. Real Git +repositories, a real managed root and a real second process carry the +canonical-identity, linked-worktree and kernel-lock claims; injected roots, +transports, credential readers and gates carry the rest. No case points at the +user's own `~/.xmd/repositories`. + +| # | Test | Verify | +|---|------|--------| +| ORC1 | One declaration surface | All thirteen names appear in the catalog with complete contracts; a repository file of one of those names shadows the default; catalog construction performs no ambient discovery, lock, Git, credential or network operation | +| ORC2 | Runtime declaration parity | The same catalog assertion holds under Deno, Node and Bun; on a runtime that installs no operational repository provider, representative Repository, Worktree, Git, Issue and PullRequest forms each report an absent provider with zero mutation, while `` remains operational through that runtime's host `API.Files` provider | +| ORC3 | Ambient primary checkout | From a normal repository, root Switch/Add/Commit select the ambient Repository and the contextual checkout; outside Git, a root Worktree, Git operation or PullRequest refusal names how to run inside one | +| ORC4 | Ambient linked worktree | Invoked from a linked worktree, Repository identity follows the canonical common directory, Git acts on that worktree's root, and the primary checkout is untouched | +| ORC5 | Origin is not local authority | A repository with no `origin` creates a Worktree and performs local Git; Push and PullRequest refuse before a credential, session or transport exists | +| ORC6 | Directory ordering and paths | `` recursively creates a missing relative target from the enclosing cwd, accepts an absolute target without rebasing it, and completes the ensure before installing cwd or beginning content | +| ORC6a | Existing directory | `` uses an existing directory without replacing or clearing it, and its planted contents remain unchanged after content expands | +| ORC6b | Non-directory refusal | A file and a supported special entry at the target or along the route each refuse before content; the error carries only the fixed structural vocabulary and no host path or platform detail | +| ORC6c | Context boundaries | Repository and Worktree bodies observe their checkout; `` content observes the ensured directory; the enclosing cwd is restored after success, printed failure and cancellation | +| ORC6d | Repository remains selected | Entering a directory inside a selected checkout changes only cwd; every Repository identity member and provider selection remains unchanged | +| ORC6e | Host persistence | An ordinary host-created directory persists after successful, failed and cancelled content, and no teardown deletion occurs | +| ORC6f | Workflow atomic ensure | Recursive creation, one durable `workspace_file` success and the resulting Workspace root commit together before content; the root retained by the effect is the authoritative current root | +| ORC6g | Workflow refusal rollback | A target or intermediate non-directory rolls back its mutation savepoint, publishes a sanitized refusal against the unchanged root, creates no partial parent and begins no content | +| ORC6h | Workflow replay and cancellation | Completed replay restores the recorded success and retained root without ensuring again; cancellation before commit publishes neither, while later content cancellation preserves the committed directory and restores cwd | +| ORC6i | Generated write authority | `allow={["write"]}` admits paired `@executablemd/workflow/composition/dir-v2#Dir` and authorizes persistent recursive creation; a retained former `@executablemd/workflow/composition#Dir` entry refuses before generated execution and mutation | +| ORC7 | Session placement | A Session launched in a managed Worktree receives that Git root and a distinct worktree session key; `.git`-file discovery remains the boundary | +| ORC8 | Persistent lifecycle | Managed paths, metadata and working files survive normal completion, authored failure and cancellation; no teardown Git or delete command occurs | +| ORC9 | Compatible reuse | A second invocation with the same immutable request reuses the same path and preserves a moved branch and a later commit while revalidating owner, origin, object format and creation commit | +| ORC10 | Conflict is non-mutating | A changed base, Worktree branch or base, metadata, origin, common directory, object format or owner relationship refuses, and the slot's entries and sidecar are identical before and after | +| ORC11 | Partial creation | A metadata-free slot in exactly the pre-exposure state is adopted and receives its sidecar; an incompatible or non-empty one refuses and remains byte-identical | +| ORC12 | Exclusive ownership | A second process selecting the same slot is refused while the first holds it; another slot succeeds concurrently; normal release permits a later owner and the checkout remains | +| ORC13 | Live local Git | Switch, Add and Commit keep their authored semantics and make real, non-transactional changes; a failure or a cancellation claims neither rollback nor replay; a commit records the invoking user's own identity, an unresolvable one refuses that component alone, and hooks, monitors, signing and repository helpers stay disabled | +| ORC14 | Live Push evidence | A performed or already-equal Push stores exact private evidence; a Push of another branch, checkout, origin, destination or commit does not authorize a PullRequest; the latest publication of a destination decides | +| ORC15 | Evidence cannot cross runs | A PullRequest succeeds only after an exact Push in the same execution; a new run must publish again, and copying a Context value, a result or a previous trace grants nothing | +| ORC16 | Live Issues | Configured reads and upserts use the existing normalized contracts and this execution's own identity; absent or out-of-ceiling configuration sends no credential and no request | +| ORC17 | Live PullRequests | Configured reviews, comments and checks reads, and a Push-authorized upsert, run under ordinary Deno; the read ceiling and the local evidence check both precede credential and network access | +| ORC18 | The journal is diagnostic | The same fixture without and with `--journal` performs the same live operations once per invocation; the trace is newly created and never consumed as continuation or evidence | +| ORC19 | Nested run profile | An isolated `host="run"` child receives the declarations and a fresh provider; its evidence and locks reach neither its parent nor a sibling | +| ORC20 | Retained workflow regression | Repository and Worktree replay, transactional Git, Push and pull-request history evidence, Issue effects, forks and completed replay keep their records, identities, provider call counts and native-launch refusal unchanged; directory ensure adds only its own `workspace_file` event and resulting retained root | +| ORC21 | Compiled binary | A compiled smoke creates a root-level ambient Worktree, runs a command there, proves `.git` is a file and the checkout persists after exit; a second gated process proves lock refusal and release | + --- diff --git a/specs/testing-spec.md b/specs/testing-spec.md index ae4ddff8..016549b2 100644 --- a/specs/testing-spec.md +++ b/specs/testing-spec.md @@ -305,6 +305,14 @@ arbitrary directory is addressable without being a component. A `source` is markdown supplied directly and follows the production `run -e` path: it reports the `` identity and writes no file. `props` are the child root's props. +A child runs in the working directory the `` was written in, not the +one the process was launched from. A `` or `` around an +`` therefore scopes the child as it scopes everything else in it: a +relative `target` resolves there, and a host profile that discovers an ambient +repository discovers it from there. The child's scope does not descend from the +document's, so this is the trusted host installing what the invocation was +standing in rather than the child inheriting it. + `host="workflow"`, and the `` scope it requires, are specified in issue #454 and are not built: a host that provides no workflow profile refuses them, naming that. diff --git a/specs/workflow-workspace-spec.md b/specs/workflow-workspace-spec.md index c56d5e25..349c66f2 100644 --- a/specs/workflow-workspace-spec.md +++ b/specs/workflow-workspace-spec.md @@ -60,6 +60,46 @@ exits. - `--journal` remains a diagnostic trace and is not continuation input. - Agent permissions remain caller-selected. +Repository operations are part of that environment under Deno and inside the +compiled binary. The ordinary provider gives a document the same thirteen +components a workflow run has, over the caller's own filesystem: + +- The **ambient Repository** is the Git checkout the command was run in, + discovered once before root expansion. Its identity is the canonical common + Git directory and its selected checkout is the canonical checkout root, so a + command started in a linked worktree names the same repository as one started + in the primary checkout while Git operations act on the worktree. A document + that never asks for a repository runs unchanged outside one. Each execution + discovers its own: a nested `` builds a provider of its + own and discovers from the working directory that `` was written + in, so a child under a `` is in that directory's repository rather than + in the one the outer command was started in. +- `` selects a **managed checkout** beneath + `~/.xmd/repositories`, and `` selects a linked one of whichever + Repository is in scope. Both survive every execution and are held for one + document execution by an exclusive non-blocking advisory lock. +- Local Git operations happen directly against the selected checkout. There is + no transaction, no rollback and no replay, and none is claimed. A commit is + made under the invoking user's own effective Git identity, captured once from + the trusted host before the document expands; a host where Git can name no + identity refuses `` and names the two commands that fix it, and + every other component stays usable. Nothing else is borrowed from the caller's + environment: hooks, file-system monitors, signing programs and + repository-supplied credential helpers stay disabled exactly as they are for a + workflow run. +- `` keeps the observe/adopt/fast-forward/refuse rules and stores + private evidence of what it published. `` is authorized by that + evidence and by nothing else, so a new run must publish again. +- `` and the pull-request evidence reads reach the same configured + transports under the same host ceilings, retaining nothing. + +Node and Bun register the same thirteen declarations and install no operational +repository provider, so `xmd syntax` describes one language everywhere and +every repository operation there reports an absent provider before a lock, a +credential, a subprocess or a request exists. Their host `API.Files` provider +still supplies ``'s directory operation, so directory composition remains +operational on every runtime. + ### 2.2 `xmd workflow` `xmd workflow` executes supported operations against a retained constrained @@ -813,6 +853,32 @@ run whose history it has just replaced. ## 6. Repository and Worktree +``, `` and `` are one component language with two +providers behind it (§2.1). What an author writes — the forms, the props, what +`as` binds, and the refusal vocabulary — is the same either way; what differs +is what a checkout *is*. + +Under a workflow run a checkout is retained Workspace state: creation identity +is a durable effect, placement is Workspace-relative, and a completed effect +restores from the journal. That is what the rest of this section describes. + +Under an ordinary `xmd run` a checkout is a directory. `` and +`` select a managed checkout beneath `~/.xmd/repositories`, described +by a closed version 1 sidecar and held for one document execution by an +exclusive non-blocking advisory lock; reuse compares creation identity alone +and never resets, cleans, fetches, repairs or deletes; and a Worktree written +outside a lexical `` belongs to the ambient Repository the command +was run in. Nothing there is retained, replayed or forkable, and none of the +durability this section states applies to it. + +What both share is the composition value a component observes: a Repository +selection carries a provider-minted identifier, a display name, the +credential-free repository identity and the selected checkout path, and grants +nothing. Every operation authenticates the selection it was handed against the +installed provider's private state before it touches Git, so a replaced +contextual Repository can misname a checkout and be refused but can never reach +one. + ``, `` and `` are ordinary registered defaults that the workflow host installs for a live or partial execution, alongside the document filesystem. A repository-local component with one of those names is @@ -953,7 +1019,7 @@ checkout path while also rendering them. ### 6.3 Directory context -`` has one meaning: lexical cwd. +`` creates or adopts a directory and gives its content that lexical cwd. ```md @@ -961,18 +1027,40 @@ checkout path while also rendering them. ``` -It restores the previous cwd after its children finish. Self-closing `` -is invalid, and which form it was written as comes from the invocation the -engine issued rather than from the contextual `hasContent()` chain — a handler -installed outside the invocation answers that chain first, and could otherwise -validate a self-closing `` the document wrote no children for, or refuse -a paired one that has them (executable-mdx-spec §5.6). A Prompt sent to an already-established Agent does not acquire a new -cwd merely because its invocation appears inside a later Dir. - -Its paired form is also a pinned identity in the standard generated-XMD write -table (§8.4), built from this same implementation and schema. Being registered -grants that nothing: a generated fragment reaches the pinned identity and never -the registration, and a repository component named `Dir` satisfies neither. +Before installing the cwd or expanding any child, `` asks the mandatory +provider-neutral `API.Files.ensureDirectory` operation to ensure its target. A +relative path resolves from the enclosing contextual cwd; an absolute path +keeps its existing absolute meaning. The operation recursively creates a +missing directory, succeeds without replacing, clearing or otherwise changing +an existing directory and its contents, and refuses when the target or an +intermediate entry is a file or another non-directory. That refusal is fixed +and sanitized, begins no content and exposes no host path or platform error. + +After a successful ensure, `` installs the resulting directory as cwd and +expands its content. It restores the previous cwd on success, ordinary failure +and cancellation. Restoration performs no deletion: a directory created by the +ensure persists even when later content fails or is cancelled. `` changes +only cwd. It neither selects nor replaces the Repository in scope, so a Git +operation inside a directory beneath a selected checkout still addresses that +same Repository. + +Self-closing `` is invalid, and which form it was written as comes from +the invocation the engine issued rather than from the contextual `hasContent()` +chain — a handler installed outside the invocation answers that chain first, +and could otherwise validate a self-closing `` the document wrote no +children for, or refuse a paired one that has them (executable-mdx-spec §5.6). +A Prompt sent to an already-established Agent does not acquire a new cwd merely +because its invocation appears inside a later Dir. + +Its paired form is also the versioned pinned identity +`@executablemd/workflow/composition/dir-v2#Dir` in the standard generated-XMD +write table (§8.4), built from this same implementation and schema. Selecting +`allow={["write"]}` intentionally authorizes its persistent recursive directory +creation. The former `@executablemd/workflow/composition#Dir` identity never +authorizes creation, so a continuation retaining it refuses before generated +execution. Being registered grants the current identity nothing: a generated +fragment reaches the pinned identity and never the registration, and a +repository component named `Dir` satisfies neither. Everything the lexical directory covers reads and writes through it, including `` (§10.1), which removes one file from the Workspace the same @@ -980,6 +1068,24 @@ contextual way `` writes one. ## 7. Git operations +The four Git components are likewise one language with two providers. Which +checkout one acts on is decided the same way in both: by the Repository in +scope and by the contextual working directory, neither of which carries +authority. + +Under a workflow run each is a durable Workspace effect, or — for Push — a +reconciled Git-host effect, and that is what the rest of this section +describes. Under an ordinary `xmd run` the same authored transitions happen +directly against the selected checkout: no transaction encloses them, nothing +rolls back, nothing replays, and no such claim is made. Section 7.3's fixed +Git identity is a workflow run's, for a reason that inverts here — a workflow's +retained state must not depend on whose machine made it, and an ordinary run's +commit lands in that person's own checkout. So an ordinary commit records the +invoking user's own effective identity, and a host that can name none refuses +rather than substituting one. Push keeps the +observe/adopt/fast-forward/refuse rules and, instead of a reconciliation +record, leaves private evidence in the provider instance that verified it. + Git operations require a contextual Repository or Worktree checkout. They use the transactional Workspace Git implementation, not an implicit host command. @@ -1946,11 +2052,11 @@ unadmitted component, live registration or not. Mutation admission is the same boundary asked for the other class. The standard Deno workflow profile's write table holds exactly three identities, in the order -it states them: core's paired `File:write`, this package's lexical `Dir` — built -from the same implementation and schema the ordinary registration owns so the two -cannot drift — and core's self-closing `File.Delete`. A host's own extensions -come after them, and the profile supplies no external-write and no execution -table. +it states them: core's paired `File:write`, the paired versioned +`@executablemd/workflow/composition/dir-v2#Dir` — built from the same +implementation and schema the ordinary registration owns so the two cannot +drift — and core's self-closing `File.Delete`. A host's own extensions come +after them, and the profile supplies no external-write and no execution table. That order is retained identity. A continuation compares the entries position by position (§8.4, *A resumed run is held to its ceilings*), so a run admitted under @@ -1959,15 +2065,22 @@ resumed under the wider one. A read-only admission never selected the write tabl and is unaffected by a change to it. An admitted mutation runs as the ordinary component it is. A paired `` and -a self-closing `` each cross `API.Files`, which under a workflow run -is the transaction-bound provider, so the mutation, the published logical root -and the filtered journal result commit in the run's one effect transaction -(§10.1) — a generated deletion is the same `workspace_file` effect an authored -one publishes, carrying `{ kind: "deleted" }`; `` installs a lexical working -directory for its content and creates nothing of its own. -Generated source receives no special mutation API, and the evaluator adds none: -it owns admission, and the component's own contract owns atomicity, failure and -cancellation. +a paired `` and a self-closing `` each cross `API.Files`, which +under a workflow run is the transaction-bound provider, so each mutation, its +published logical root and its filtered journal result commit in the run's one +effect transaction (§10.1). A generated directory creation is the same +`workspace_file` effect an authored `` publishes, and a generated deletion +is the same effect an authored deletion publishes, carrying +`{ kind: "deleted" }`. Generated source receives no special mutation API, and +the evaluator adds none: it owns admission, and the component's own contract +owns atomicity, failure and cancellation. + +Selecting `allow={["write"]}` therefore intentionally authorizes persistent, +recursive directory creation by the current pinned ``. The former +`@executablemd/workflow/composition#Dir` identity described contextual placement +without creation and is not interchangeable with this authority. A continuation +whose retained table contains that former identity refuses before any generated +component executes; the write ceiling is not silently broadened. Because the whole fragment is decided inside the admission before its first effect, a fragment mixing an admitted write with anything unadmitted — a later @@ -2178,6 +2291,11 @@ retained no checkpoint remains unassociated. ## 9. Replay and continuation +Everything in this section is workflow-only. An ordinary `xmd run` has no +WorkflowRun, no retained history and nothing to replay: its repository +operations are performed once per execution, and a second run is a second +question rather than a continuation of the first. + Replay rehydrates the Effection tree. Ephemeral structure executes again; durable observations and mutations restore. @@ -2185,7 +2303,8 @@ durable observations and mutations restore. | --- | --- | | implicit Workspace | reattach the same run-owned Workspace | | bundled component import | restore the retained `{ kind: "workflow", path, sourceHash, content }` selection and reconstruct the component from that exact source, resolving no name and reading no file | -| lexical Dir/Repository/Worktree | reinstall contextual cwd and live facade | +| lexical Repository/Worktree | reinstall contextual cwd and live facade | +| Dir directory ensure | restore the recorded `workspace_file` success and retained root without creating again, then reinstall the contextual cwd | | Agent provider/session | attach lazily before the first live Agent operation | | Repository base/default resolution | restore pinned result | | Repository/Worktree creation | restore the retained creation record, then reattach and verify the retained Git state without recloning | @@ -2304,6 +2423,29 @@ declarative Git operations use this boundary, and so does a mutation an admitted generated fragment performs (§8.4) — it is the same component crossing the same provider, with no second path of the evaluator's own. +**Document directory ensure** is one of them. `` resolves its path in the +logical Workspace filesystem and publishes one `workspace_file` effect, named +by the expansion, the directory operation and the resolved logical target. A +relative path resolves from the enclosing logical cwd; an absolute path keeps +its established logical-root meaning. The provider recursively creates a +missing target and succeeds unchanged for an existing directory, preserving all +of its contents. A file or any other non-directory at the target or along the +route is a documented refusal before `` installs the cwd or begins content. + +The ensure runs in a mutation savepoint. On success, the mutation and the +resulting Workspace root commit with the sanitized successful outcome; only +then does `` install that directory as cwd for its content. A documented +refusal rolls back the savepoint and publishes its sanitized refusal against the +unchanged root. An infrastructure failure or cancellation before commit rolls +the outer transaction back and publishes neither outcome nor root. No retained +record contains a host path, platform code, raw platform message or cause. + +A completed replay restores the recorded success and retained resulting root +without attempting creation again. Once committed, creation is not owned by the +cwd scope: later content failure or cancellation restores the enclosing cwd but +does not remove the directory. `` neither reads nor changes the Repository +selection in scope. + **Document deletion** is one of them. `` (executable-mdx-spec §6.13.1) removes one file from the run's Workspace, resolving its path inside the logical filesystem so no host path is ever produced. One regular file or one @@ -2335,6 +2477,14 @@ delete` (§12), which no document can reach. ### 10.2 Git-host effects +The reconciliation described here is a workflow run's. Under an ordinary run +there is no history to reconcile against: a pull request is observed once, +created or updated at most once inside that execution, and decided by one exact +observation afterwards, and what authorizes it is the provider instance's own +record of publishing the branch rather than a journal scan. Within one +invocation the attempt happens at most once; across a process interruption +there is no exactly-once claim. + A **Git host** is an external service that owns remote Git repositories and associated collaboration objects such as branches and pull requests. GitHub is one Git-host adapter. A Git host is distinct from the local Git capability of @@ -2487,6 +2637,12 @@ missing result causes reconciliation under the same deterministic identity. ### 10.3 Issue effects +The durable envelope described here is a workflow run's. Under an ordinary run +`` reaches the same configured transport under the same host ceiling +with no envelope at all: identity is that execution's own opaque invocation +identity together with the engine's expansion identity, an upsert presents an +idempotency key derived from them, and nothing is retained. + An **Issue provider** is an external service that owns a collection of issues. GitHub and Atlassian Cloud are Issue-provider adapters. @@ -3007,6 +3163,15 @@ is outside the initial local capability set; Worker Shell follows §10.4. A late Cloudflare-hosted or workerd-backed provider may install the same Workspace and lifecycle contracts; documents do not choose that topology. +An ordinary `xmd run` has a topology of its own beside this one. Managed +checkouts live under `~/.xmd/repositories`, in `repositories//` and +`worktrees///` slots holding a `checkout/` directory and a +`metadata.json` sidecar, with lock sidecars under `locks//.lock` +outside the slot they protect. Every authored string — a name, a locator — is +present only as a digest, so no name a document writes decides a path. It uses +the same host authentication and the same `XMD_WORKFLOW_GITHUB_ISSUES` and +`XMD_WORKFLOW_GITHUB_PULL_REQUESTS` configurations this host already reads. + The local lifecycle adapter owns a non-blocking exclusive advisory lock on one deterministic sidecar per run. The open file belongs to the workflow executor's scope and the operating system releases it when the process exits. The exact @@ -3254,9 +3419,13 @@ fetch operation requires its own language and durability contract. | workflow-run and expansion identity | built by #289 / PR #341 | | retained run record and filtered journal | built by #291 | | caller-owned storage transaction | built by #291; Workspace mutations join it in #365 | -| provider-backed retained Workspace | document filesystem built by #366 and repository composition by #293; document deletion (§10.1) built by #567 for both providers; process capabilities unbuilt (#218) | +| provider-backed retained Workspace | document filesystem built by #366 and repository composition by #293; document deletion (§10.1) built by #567 for both providers; mandatory directory ensure is specified by #643 for both providers; process capabilities unbuilt (#218) | | `xmd workflow start` / `resume` | built by #366, Deno entrypoints only; both acquire #367's executor lock | -| ``, `` and `` composition | built by #293, Deno provider only | +| ``, `` and `` composition under a workflow run | Repository and Worktree built by #293, Deno provider only; ``'s mandatory `API.Files.ensureDirectory` is specified by #643 as one transactional `workspace_file` mutation followed by lexical cwd installation | +| the same thirteen declarations under every runtime | built by #643: one shadowable array consumed by the workflow attachment, `xmd syntax`, `xmd plan` and an ordinary document execution | +| ``, `` and the ambient Repository under an ordinary run | built by #643, Deno and compiled only: managed checkouts under `~/.xmd/repositories` with version 1 sidecars and execution-owned non-blocking locks, and the checkout the command was run in as the default Repository. Node and Bun install no operational provider | +| local Git operations and `Git.Push` evidence under an ordinary run | built by #643, Deno and compiled only: the same authored transitions with no transaction and no replay, commits recorded under the invoking user's own captured Git identity with an actionable refusal when the host can name none, and a private per-execution Push evidence entry that authorizes `` and crosses no run | +| `` and pull-request reads under an ordinary run | built by #643, Deno and compiled only: the same transports and ceilings with no durable envelope, keyed by this execution's own invocation identity | | transactional Git components (`Git.Switch`, `Git.Add`, `Git.Commit`) | built by #294, Deno provider only | | `` read and upsert, and the `issue_effect` boundary (§10.3) | built by #296; GitHub middleware, Deno host | | ``, ``, `` (§7.7) | built by #576; GitHub middleware, Deno host. Named by canonical URL and asked of `PullRequestApi`, which carries the upsert too; ordinary durable reads rather than reconciled effects, inheritable by a fork; complete or unavailable, never truncated. Which URLs may be read is operator configuration | @@ -3271,8 +3440,8 @@ fetch operation requires its own language and durability contract. | workflow Agent isolation | built by #302: no directory attachment, an empty host-owned working directory, no MCP servers, an empty requested tool set and deny-all with a failing permission path; the portable no-tool proof is tracked by #496 | | workflow Agent session retention | built by #302: a row in the run's own database, keyed by the engine-derived Session expansion identity alone — the authored name is descriptive — with provider, agent command and policy fingerprint beside it as compatibility attributes. A `` places one and creates nothing; the first subscribed Prompt constructs it, and the mapping commits after the backend accepted that turn and the provider made its canonical tagged assertion, before anything the turn produced is exposed. Occupancy of a provider key is never identity — including a record held for a first turn nobody accepted — and missing, mismatched, replaced or ambiguous assertions each refuse instead of starting a replacement session | | generated-XMD admission | built by #369, through `@executablemd/core/host`; the workflow policy wrapper is internal. Host policy is a read table and a write table of exact pinned identities, each entry carrying the authored forms it is admitted for, and an authored `allow` selects a canonical subset of the closed classes `read` and `write` — omitted means `read`. The complete fragment is preflighted inside one `generated_xmd` effect before its first generated effect | -| `` and the authored loop | built by #302 and #369: a workflow-host component with a closed schema of one required `source` and one optional `allow`, declared to the execution rather than registered by the attachment — canonical execution calls the host's factory with the claimant it minted and registers what comes back, which provides availability only. Its ceilings come from the run's own storage, core's pinned `` read and write identities and this package's lexical ``; iteration, branching, approval and exhaustion are ordinary Markdown | -| generated-XMD mutation-proposal admission | built by #369 and #567: the standard Deno profile's write table is core's paired `File:write`, this package's lexical `Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension; admitted mutations run as the ordinary components they are through the run's effect transactions, a generated deletion publishing the same `workspace_file` effect an authored one does; the evaluator adds no receipt or result entry, so a write-only fragment still binds `{ observations: [], output: "" }`; and approval is authored control flow before the element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | +| `` and the authored loop | built by #302 and #369: a workflow-host component with a closed schema of one required `source` and one optional `allow`, declared to the execution rather than registered by the attachment — canonical execution calls the host's factory with the claimant it minted and registers what comes back, which provides availability only. Its ceilings come from the run's own storage, core's pinned `` read and write identities and the versioned paired `@executablemd/workflow/composition/dir-v2#Dir`; iteration, branching, approval and exhaustion are ordinary Markdown | +| generated-XMD mutation-proposal admission | built by #369 and #567, amended by #643: the standard Deno profile's write table is core's paired `File:write`, the paired versioned `@executablemd/workflow/composition/dir-v2#Dir` and core's self-closing `File.Delete`, in that retained order and followed by any host extension. `allow={["write"]}` intentionally authorizes the current Dir's persistent recursive creation; the former unversioned identity never authorizes it and a retained table naming that identity refuses before generated execution. Admitted mutations run as the ordinary components they are through the run's effect transactions, directory ensure and deletion each publishing the same `workspace_file` effect their authored forms do; the evaluator adds no receipt or result entry, so a write-only fragment still binds `{ observations: [], output: "" }`; and approval is authored control flow before the element. Local Git, Git-host, issue, process, execution, credential and external-write effects are outside the class | | Deno-local DOFS persistence | POC proven by #349 / PR #350 | | scoped Deno Worker Shell | containment proven by #351 / PR #353 and transactions by #357 / PR #362; production integration unbuilt | | Worker JavaScript | deferred |