Skip to content

Make <Evaluate> evaluate complete XMD programs #713

Description

@taras

Story

As an Executable Markdown author, I want <Evaluate> to evaluate a complete XMD
program at an explicit composition site, so producing a program and running it
remain separate choices.

Common paths

Evaluate a planned program directly while privately capturing the source that
<Plan> produces:

<Evaluate>
  <Plan>
  Inspect the release inputs, recommend a version, and ask me to approve it.
  </Plan>
</Evaluate>

Or preserve the program first and evaluate it later:

<Plan as="plan">
Inspect the release inputs, recommend a version, and ask me to approve it.
</Plan>

The approved program is ready. Carry it out here.

<Evaluate program={plan} />

The paired and program forms evaluate a complete root in the current XMD
execution. They do not start a child document, process, host profile, root
lifecycle, or independent journal.

Three disjoint forms

<Evaluate> has three mutually exclusive inputs:

  1. paired content produces complete-program source in a private buffer;
  2. program={value} supplies complete-program source directly; and
  3. source={fragment} retains the restricted generated-XMD meaning delivered
    by Evaluate Agent-generated XMD through a constrained allowlist #369.

source, program, and paired content cannot be combined. allow is valid
only with source. props={object} is valid only with a complete-program form.
Every ambiguous or misplaced combination refuses before program content or a
program effect runs.

The paired form suppresses its source-producing output from the surrounding
document, removes only indentation contributed by the <Evaluate> wrapper, and
admits the resulting complete root. Bytes emitted by a producer are otherwise
unchanged. With a sole <Plan> producer, paired evaluation and
program={plan} admit the same approved bytes and content digest.

Another source-producing text component may be used in paired content when its
rendered result is a complete admissible root. Paired content is not quotation:
an executable component written literally there executes under ordinary XMD
semantics while producing the text sent to admission.

Complete-program behavior

The complete root's frontmatter, imports, metadata, props schema, returns, and
<Output> selection apply.

  • props={object} is the complete root's props input. It defaults to {};
    ambient root props are never silently adopted.
  • Root props are schema-validated before the first program effect.
  • Ordinary non-props caller bindings are visible read-only. Program-local
    bindings do not escape the evaluation.
  • Relative imports resolve from the source origin of the authored <Evaluate>
    site.
  • A text root executes and renders its selected output without as; with as,
    it still executes, binds that selected text as a string, and emits none of it.
  • A value root requires as before its first effect and binds its
    schema-validated JSON result. Without as, it refuses before program effects.

The ordinary run profile exposes the complete-program forms. A workflow run's
<Evaluate> exposes those forms plus its existing restricted source form.
The author-facing description is:

Evaluate XMD source in the current execution. <Evaluate program={plan} />
evaluates a complete program. Use
<Evaluate source={fragment} allow={["read"]} /> for a restricted generated
fragment.

Structured syntax fields own the form, prop, capture, return, and origin details.

Current-site authority

Complete-program evaluation uses the current execution's lifecycle, journal,
cancellation scope, contextual providers, working directory or Workspace, and
authority at the authored <Evaluate> site. Program source requests behavior;
it grants none.

The producer's temporary authority is not inherited. In particular, <Plan>'s
authorship profile and private components have torn down before evaluation, and
a private component belonging to an enclosing declaration is available only to
elements authored by that declaration's exact bytes. Source returned by a
producer cannot invoke either closure. Ordinary imports and capabilities that
are genuinely present at the evaluation site remain available.

Separate admission and replay

Complete programs use a distinct complete-program admission and durable event;
do not widen or reuse generated_xmd. Before the first program effect, retain
and hostile-parse the exact source and digest, explicit props, evaluation-site
source origin, root mode, resolved component identities and forms, and every
compatibility term needed to prove the same current-site environment.

A partial continuation expands the retained source, restores completed nested
effects, and repeats no planning. Changed source at the same evaluation
occurrence is stale input and refuses before either current or retained source
can run. Neither source silently wins.

The source digest identifies the program artifact. The authored <Evaluate>
site and loop iteration identify one execution occurrence. Deliberately
evaluating the same artifact at two sites or iterations creates independent
nested effect identities; the digest never deduplicates those executions.

Restricted-fragment compatibility

<Evaluate source={fragment} allow={...} as="observation" /> keeps #369's
contract exactly:

  • the closed generated-XMD grammar and selected read/write ceilings;
  • caller-binding isolation;
  • the generated_xmd admission and retained-source replay;
  • pinned component identities and forms; and
  • the exact { observations, output } result.

Complete-program support cannot be selected through source or allow.
Representative Agent, Elicitation, import, binding, executable-block, and
unadmitted-component cases still refuse through source before effects.

Acceptance ownership

This story exclusively owns these Quest cases:

  • Direct composition: paired Plan source evaluates once and is not emitted
    separately.
  • Deferred composition: program={plan} evaluates the same digest without
    repeating planning.
  • Program forms: text and value roots follow their output and return rules;
    ambiguous forms refuse before effects.
  • Root props: explicit props validate before effects and ambient props are
    not adopted.
  • Fragment compatibility: the delivered result and ceilings remain exact;
    Agent and Elicitation remain refused through source.
  • Authority: only evaluation-site authority is usable; producer-private and
    otherwise unavailable capabilities never run.
  • Evaluation replay: retained source resumes nested effects without
    replanning or repeating a completed effect.
  • Changed evaluation source: stale input refuses and neither source runs.
  • Repeated evaluation: two explicit occurrences execute independently and
    are not deduplicated by digest.

Structural preflight cases place a negative-control effect before a later
malformed construct and prove the earlier effect did not run.

Documentation and focused evidence

Update architecture.md, the component/root/return/execution sections of
specs/executable-mdx-spec.md, and the generated-XMD section of
specs/workflow-workspace-spec.md. The documentation names complete-program
admission separately and leaves #369's vocabulary intact. Update xmd syntax
and the component inventory for every host profile that exposes <Evaluate>.

Add focused canonical evidence in
packages/core/tests/program-evaluation.test.ts; public run-profile evidence in
packages/cli/tests/evaluate-program-component.test.ts; and workflow-profile
compatibility evidence in
packages/workflow/tests/generated-agent-component.test.ts. Keep
packages/core/tests/generated-xmd.test.ts as the restricted-fragment negative
control.

Focused feedback evidence:

deno task test \
  packages/core/tests/program-evaluation.test.ts \
  packages/cli/tests/evaluate-program-component.test.ts \
  packages/workflow/tests/generated-agent-component.test.ts \
  packages/core/tests/generated-xmd.test.ts \
  packages/cli/tests/syntax-cli.test.ts

After a feedback commit, run deno task test --changed. Delivery proves npm and
compiled distribution through scripts/tests/cli-npm-bin.test.ts, the ordinary
packaged-document probes, the build, and required CI checks.

Dependencies

There is no dependency on superseded #711 or PR #720.

Out of scope

  • Implicit execution by <Plan> or any other source-producing component.
  • Widening the restricted generated-XMD evaluator.
  • A <Run> component, process isolation, or a child document lifecycle.
  • Granting authority from source or inheriting producer-private closures.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions