Repository navigation
Execution records in use: preview and hub run from records; one presenter for every engine (RIU + RFU + OPR) - #1171
Merged
Merged
Conversation
Contributor
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
A capture ref restored from disk was trusted as fresh until the next edit, so a document changed while the preview was down showed stale output. The preview now recomputes staleness once at startup. The RIU-0 spikes S0-S7 (Python, Julia, knitr, headless Chromium, the real preview and hub, record-only cache entries, Positron export) and the records-in-use epic and plans RIU-0 to RIU-6 they fed are included as notes. Fixtures: 9 files added, none modified, none removed, under crates/quarto-preview/tests/fixtures/records-in-use (6, plus 3 under project/). No .snap files.
… entries (RIU-2) One publish seam encodes a record for the cache and refuses a record with no legacy form when the gate is off. The cache wrapper stores and hits a record-only entry; record sources are walked first-wins with per-source outcomes; re-execute carries a config and an execution guard; record_one walks the sources and policy-off still reads records. Reuse across a restart is pinned. Also here: five review rounds and the RIU-0b/0c spike results that rewrote the epic and plans, and the pointers from the execution-records plans (decisions 4 and 8, EXR-15, EXR-17) at the records-in-use decisions. No fixtures or snapshots added or changed.
export_for_positron filters a record down to what Positron's cache accepts, with one representation per image. Tests run against real Python and Julia records and cover mirror and import round trips. Fixtures: 8 files added under crates/quarto-core/tests/fixtures/records-export, none modified. No .snap files.
…e 0) Research notes only: links, live update, console, stop and clear behavior, the Quarto 1 source reading, and the finding that the pane does not sync. Points at the strand for printing the URL when the server is ready. No code, no fixtures or snapshots.
… (RIU-1) Each surface (preview, provider) has its own writer gate. A document whose engine builds no execution record is stored whole in the legacy capture format and gets the warning Q-18-15 (Q-18-14 went to knitr child documents on main), once per capture run; record copy fails soft. The preview config takes the Preview surface and the provider returns engines_without_record. The provide-hub flip is deferred to RIU-7. Also here: the epic's lanes and ready board, the exit criterion for a native Windows run, and the EXR decision-4 pointer to the runbook gate section. Catalog entry Q-18-15, docs page docs/errors/engine/Q-18-15.qmd and its sidebar line are added. No fixtures or .snap files.
record_from_positron converts a Positron cache into a record; a native-only locator finds the cache (URI-hash name, quarto_source scan, content-hash fallback). The walk gains Positron and directory arms, imported records get a staleness rule via an opt-in language ownership map, and the preview publishes and re-checks them. A minimal --records flag and the provider's publish_record_source are included. Fixtures: 7 files added under crates/quarto-core/tests/fixtures/positron-cache, none modified. No .snap files.
Adds scripts/riu-6-check-export-mirror.sh and records the results of running commands 1 and 3 in a Positron window (directory override, child process, Stop ends q2), plus the plan's resume state. No fixtures or snapshots.
--records takes record sources (a Positron cache directory or the q2 cache) on both commands, with start-up validation, per-command defaults, suggestions for bad entries, and a rule that no engine means never execute. The provider walks the sources through ProviderLookup and keeps the engine-only path unchanged. The share banner and help stop claiming guests can re-run when no engine can run, and the preview shows a per-source diagnostic when no source supplies a record (silent when a source failed). Documented in the guides. No fixtures or snapshots added or changed.
Browser, e2e and two-binary share checks, the guest re-execute run against a scratch ipykernel venv, and provide-hub runs against a scratch hub (gate off, consent refusal, default engine path under --dangerously-accept-requests). The no-record warning now carries its per-source reasons and the fix in the problem line. Plan reconciliation and the board tick are included. Fixtures/e2e: 1 spec added and 1 helper modified under q2-preview-spa/e2e. No .snap files.
Command that writes a record as a Positron cache file, using export_for_positron. Includes the recorded Julia error-document run through it. No fixtures or snapshots added.
Command 2 and the runner, Python and part of Julia done via Playwright, the findings note and a demo script; Phase D and the resume notes are brought up to date. No fixtures or snapshots.
…U-3b) Spliced record output carries a jupyter-output stamp so the viewer can tell a generated output node from the author's text. The AST reconcile remap is reworked for it, record_splice stamps nodes, and the preview renderer treats the new kind as atomic (documented in the provenance contract and the guides). The epic's closing reconciliation (statuses, board, exit criteria, Julia results) lands with it. Tests: a 675-line stamp test module and a preview-renderer integration test are added. No .snap files and no fixture directories.
Cell presentation had separate loops per route (plain render, record replay, splice), each deciding for itself what to emit. A single per-cell step now takes a struct of optional inputs (record, source, options, target) and produces the wrapper and output divs, so a route only decides which inputs it has. Pinning tests for the existing loops, scope, spans and entry paths land first so the unification can be checked against today's behavior. Julia declares its display-only options and acknowledges that core now presents its output. Also adds the planning documents for the one-presenter epic and the follow-up to the execution-records work: the plan DAG, lanes, agent protocol and recorded decisions.
Exporting a record for Positron is a format conversion, so it belongs under `q2 convert` next to the other targets rather than in a one-off `record` subcommand. The old command is removed (it never shipped).
…uarto 1 parity harness The document-level function now returns per-cell outcomes instead of only output, so callers can report what was presented, refused or skipped. The parity harness renders a corpus with both q2 and Quarto 1, normalizes generated ids and empty wrappers, and diffs the two. It separates documents q2 rejects from execution failures, and keeps known differences in an allowlist with explicit sign-off state. The baseline run is recorded.
… instead of --records A cached knitr chunk keeps its cache and is left out of the record, so a record never claims output that was not produced in this run. Warning enforcement now comes from one rule derived from the hash declaration. Record sources are selected by the `execute: record-sources` key (layered across project, directory and document) instead of the `--records` flag, which is removed from preview, provide-hub and convert. The key is excluded from the execution hash (hash definition version 2) since choosing where records come from must not invalidate them.
…unner Render reads `execute: record-sources` per document, with selection, a skip guard and a shared staleness wrapper; a single-file book flags the key as not honored. Preview routes use the `--share` ceiling. The staleness inputs are merged for the render's format, since Julia records otherwise read as stale. QuartoNotebookRunner is pinned to a known revision and re-resolved when the pinned source revision changes, and the three capture facts are read from the cell annotation. A conformance sweep pushes 21 cases through import and splice.
…alence fact Julia capture facts are written as cell metadata under the knitr keys, and `warning: false` hides only Warning-kind stderr when the record carries output kinds. The Positron import gate used to compare engine names. Engines now declare kernel equivalence (trait method, manifest key, oracle), and export no longer skips by engine, so knitr records export too.
An engine offers presentation and core acknowledges it, so deciding who presents a cell no longer involves an engine-name test. Presenting from a record is a separate flag from recording. Also documents `q2 convert` and the Positron export, records the first parity-gate run with two proposed allowlist entries, and notes the results of verifying the Julia reproductions and browser checks.
A knitr chunk served from the cache never re-executes, so the record had no entry for it. A sidecar beside the cache entry now holds what is needed to record the hit, keeping level 1 and 2 behavior distinct. Tests were written first and pin the sidecar checks. Includes the end-to-end notes from running R and Julia through the fork, and the plans for engine-controlled attributes.
An `execute_result` image takes its width and height from the output metadata, so a recorded PNG renders at the size the engine produced. Level-3 cache hits are pinned for `dependson`, `autodep`, `cache.lazy`, `cache.rebuild`, callouts, `child=`, deleted `_files` and the nonce marker. The three non-Julia parity allowlist entries are recorded as signed off.
Engines can now control the attributes on the cell wrapper and on every output div, and the figure file names, instead of core hard-coding Julia's. `quarto_attrs` is validated by an `engine_attrs` module; refused entries are reported as a presentation note. The attributes are display data: they are hashed, survive staleness checks and round trip through splice and render. Julia uses the channel to stamp the plain route's wrapper id, counts, figure names and figure class, and its to-markdown no longer forwards `metadata.quarto` as an attribute. That removes the five Julia entries from the parity allowlist. The engine-host bundle is rebuilt.
ANSI color codes in stream and error output are removed in a single function at presentation, driven by one shared table, so every engine gets the same result. Parity cases cover Jupyter, knitr and Julia.
Widget outputs need their JavaScript and CSS on the page, which Quarto 1 discovers by scanning the output. The scan is now implemented in core and runs on every route (plain, replay, splice), with oracle fixtures generated from Quarto 1. Engines can pass `dependencyData`, which is stored on the record entry together with the presentation acknowledgement and checked by the wire-parity guard. A widget shown in a view that cannot load it raises Q-18-16 once per document. Replay derives dependencies again so older records keep working.
Figure groups (`fig-subcap`, layouts) got their caption and panel ids only on the engine-specific route. The group rule now lives in core, so Jupyter, knitr and Julia all number captions and panels the same way. Later figures keep their numbers, and a panel-id clash stays an author error. Edge-case, widget and fallback-chunk tests cover the rule. Includes the reviews and plans that came out of this work: the split of the whole-document target flip into smaller steps, and the subfloat parity draft.
The parity gate now compares output per target rather than only HTML. A document/pin schema describes what to compare, a normalizer handles Julia's interleaved cells, and an ordered diff reports differences. docx, pptx and epub are unpacked by a zip extractor; a target with no extractor fails loudly instead of passing vacuously. The gate asserts on blockers, on disagreement between runs and on stale allowlist entries, and the report has per-entry columns. Spike results for the target-following work are folded into the plans.
…ell id The cell presenter picks the display format by the render target's priority, the figure directory is named after the output format, and the engine's automatic cell id goes on the wrapper only for html-family targets, matching Quarto 1. A stored Jupyter html-plus-image bundle follows the target. `quarto-unit-id` is reserved on knitr chunks (Q-18-17), so a chunk cannot claim the key the record uses to identify cells.
Julia documents are checked against Quarto 1 for eight targets, including epub with a pinned interleaving entry, echo-false classes and a date-regex fix. A `JULIA_PRESENTED_TARGETS` allowlist decides which targets offer presentation. The gate report proposes five more interleaving entries and a capture-facts pin; all six entries are now signed off.
Plain render of Julia and knitr documents now goes through the recorded route and the one presenter, instead of each engine's own output path. WASM replay tests cover captures stored before the change so they keep rendering.
… (RFU-2 import gate)
…execute keys cfe44f6 put record-sources in the Jupyter hash definition's presentationExecuteKeys; the 13 shadow .record.json goldens were not updated because they only run with a Python kernel (skipped locally, caught by macOS CI). Hand-patched; the patched files equal the CI actual output byte for byte. No .snap files changed.
pptx joins the knitr structured targets (html, revealjs, epub, typst, docx,
gfm), so its chunks go through the shared presenter instead of keeping knitr's
finished markdown.
Evidence:
- The KNR corpus run under pptx (every fixture, KNR_FORMAT override) matched
docx exactly: 256 accepted, 6 allowlisted, 5 aborted by design, 4 unexpected
blockers. The 4 are not pptx: two are the html cache=1/2 gap, two are a
random R temp path in webshot's "screenshot completed" line on fixtures that
only run under html.
- Real render with pandoc 3.13: one document (print, plot with fig-cap, kable
with tbl-cap, fig-subcap pair, cross-references) rendered to pptx down both
routes. All five slides identical as pretty-printed XML, the one media file
byte-identical, same two Q-11-1 warnings. A fallback notice appears only on
the old route; the new route's record holds 4 structured cells.
- Shared limits, not caused by this change: the kable table and the fig-subcap
group reach no slide on either route (pandoc/q2 pptx writer).
Tests (written first, seen failing): target_route covered_targets_are_structured
and gating_tests::the_fallback_table ("pptx target" structured; the uncovered
case now uses context). Corpus manifest: pptx added to the four fixtures that
already run docx. Snapshots: 4 new frozen oracle files, live/rich-output/
{kable,df_print_default,knit_asis,include_graphics}.pptx.md; no existing
snapshot changed (288 matched).
Verified: cargo nextest run -p quarto-core (target_route, the_fallback_table);
cargo clippy -p quarto-core --all-targets -- -D warnings. Not run: the Windows
cross-clippy (aws-lc-sys does not build for the target here; the change is
constants, tests and fixtures only). Not run: the full corpus gate passes for
every pptx row but still reports four html-only failures that predate this
change (s1/14_cache_level1/2 and two stale indent allowlist entries); the
corpus report only runs when KNR_REPORT is set, so CI does not see them.
Corpus tool: KNR_FORMAT=<fmt> overrides each fixture's formats for one report run, to measure a candidate target across the whole corpus without editing the manifest. Report-only; the gate and the default run are unchanged.
A page for the knitr engine, the sibling of the Jupyter kernels page, built from Quarto 1's "Using R" and caching articles and made about Quarto 2: overview and a complete example, installation (R, rmarkdown, knitr 1.44, QUARTO_R), how Quarto picks knitr, cell options (`#|` only, `!expr`, labels, execution options, `output: asis`), `df-print`, the `knitr:` key with `opts_chunk` and `R.options`, inline expressions, caching, and the chunks that keep knitr's markdown (child=, other languages, replaced hooks, formats the record cannot present yet, cache levels 1 and 2). Differences from Quarto 1 are callouts beside each feature. Checked by real renders with the built q2: `knitr:` in the project file and the document, `R.options`, `output: asis`, `df-print: kable`, both inline spellings, and `!expr` in `echo` (works) and in `fig-cap` / `tbl-cap` (shown literally; documented as a difference). The page was rendered with `q2 render --strict` in a copy of the docs site (the `examples/` resource is generated and absent in a fresh worktree): no errors or warnings for the page, every link resolves. Not verified: the list of formats the record presents is as of this branch (pptx and context are being added separately); the Positron statement for cells that keep knitr's markdown comes from the KNR plans and the existing execution-records page, not a Positron run. `--cache` and `--cache-refresh` are documented as having no effect because nothing reads them in the code.
The page listed pptx among the formats that keep knitr's markdown. It joined the presented formats in 9ac905b22 on this branch.
context stays on knitr's finished markdown, like latex, beamer and pdf, and now gets the same diagnostic. Before, it was an "uncovered target", which fell back the same way but under the generic wording and with no stated reason. knitr treats context as a TeX target: pandoc_to in knitr's pandoc.R maps latex, beamer, context and texinfo alike to the pdf figure device, so its output depends on the format the way LaTeX does and the LaTeX epic owns that presentation. A markdown-level corpus run under context cannot see that (it compared like html), so it is not evidence for presenting context from the record. Context is also not a q2 output format today (`q2 render --to context` stops with "Unknown format: context"). The diagnostic now names LaTeX and ConTeXt. Tests (written first, seen failing): target_route the_latex_family_falls_back and the_diagnostic_names_the_target_and_the_reason; gating_tests the_fallback_table gains a "context target" row (it passes before and after, since both fallback reasons write the same record; it pins the behavior, not the reason). The uncovered-target examples now use odt. Verified: cargo nextest run -p quarto-core (target_route, the_fallback_table, gating); cargo clippy -p quarto-core --all-targets -- -D warnings; cargo fmt --check. Not run: the Windows cross-clippy (aws-lc-sys does not build for the target here; constants, tests and a message only).
a_record_only_cache_entry_is_published's second record_one call fell through from the refused cache source to the engine source. With a Jupyter kernel installed (CI's ubuntu job) that ran the document's python cell for real and panicked with 'Cannot start a runtime from within a runtime'; without one the engine source declined and the test passed. Turn the engine off for that call so the outcome does not depend on what is installed.
…4dceefd..493762536 493762536 julia-engine declares dependencyData with the presentation acknowledgement 237876f02 Declare the kernel-equivalence fact in _extension.yml 8044c1a9b Pin QuartoNotebookRunner ffe5dab; re-resolve when the pinned source revision changes bb9d7b09c Exclude `execute: record-sources` from the hash definition (version 2) 9f110c875 Julia declares display-only options and acknowledges core presentation git-subtree-dir: resources/extension-subtrees/julia-engine git-subtree-split: 4937625361d77ac0890cd12da06246e4613b7ea9
gordonwoodhull
force-pushed
the
feature/one-presenter
branch
from
November 12, 2026 05:21
1323b41 to
a76c96a
Compare
gordonwoodhull
marked this pull request as ready for review
November 12, 2026 05:30
gordonwoodhull
temporarily deployed
to
github-pages
November 12, 2026 06:42 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Execution records: preview and hub run from records, and every engine goes through one presenter
Preview and
provide-hubnow work from an ipynb execution record instead of the legacy capture cache. A record can come from an engine run, a file, or a Positron cache. One presenter turns a recorded cell into page output for Jupyter, Julia and knitr alike, and plainq2 renderof Julia and knitr takes the same route. Three epics, developed on one integration branch, land together because each builds on the last. Main has the record writer behind an off-by-default gate; none of the consumer side below exists on main, so the command and metadata names are new, not renames:q2 convertcommand, knitr export, tests, and docs.The branch changes about 390 files, mostly in
quarto-coreandquarto-preview, and no.snapfiles.What changes for users
provide-hub. The record is the only thing the viewer needs; the legacy capture remains only for an engine that builds no record, and a diagnostic says so.execute: record-sourceschooses where a document's record comes from. The source list (engine,cache, a directory, and so on) is one metadata key, set with-Mon the command line.preview,provide-hub,convertandrenderread it through one reader, so a document gets its record from the first source that has one.q2 convert --to positron-ipynbexports a record as a Positron cache file, for Python, Julia and knitr documents.q2 renderof Julia and knitr builds the record in memory and presents it through the Rust presenter, writing nothing under.quarto. A cached knitr chunk becomes a record cell by reading its sidecar (level 3 cache hit), so knitr's cache still works. The LaTeX family,pptx,contextand unknown knitr targets stay on the engine route.quarto_attrs.Jupyter output stays byte-identical unless a cell sets a new option, except for two documented preview and hub fixes (OPR-1a, OPR-1b) and one group-caption fix (OPR-7j). Marimo keeps the legacy capture splice and is outside this work.
How it is built
warningandmessageare enforced exactly where the record's hash definition lists them as display-only. When an option is in the hash, capture already enforced it.q2/node-port. It is restored bycargo xtask pull-extension-subtree julia-engineas the last step before review, so the history ends in a subtree merge commit.What to check
aws-lc-syscannot cross-compile from macOS, so RFU-6 needs a native Windows run of the changed crates and of the tests that exercise paths and file handles.q2 convert,execution-recordsand preview are written, but the embedded-docs refresh (cargo xtask build-agents-docs) is that plan's last step.claude-notes/plans/:2026-11-02-records-in-use-epic.md,2026-11-08-rfu-epic-riu-follow-up.md,2026-11-07-one-presenter-path-epic.md.Testing
cargo xtask lintandcargo clippy -p quarto-core -p pampa --all-targets -- -D warningspass.cargo nextest run -p quarto-core -p pampa: 13772 run, 13772 passed, 36 skipped, with pandoc 3.13 onPATH(the Julia subtree included). Under pandoc 3.12.1, 252 tests fail on the 3.13 version floor that main introduced; that is not a regression from this branch.cargo xtask verify(full, with the hub-client build) was started locally; results are posted below when it finishes.