[cscheid]

Problem

Kernel sessions were keyed by (kernel name, working directory). In a project render the working directory is the project directory, so every Python document of a project shared one kernel:

• Parallel render (QUARTO_JOBS=2): cells of different documents interleaved on that kernel, so a document could read a value another document had just set (doc a sees owner = b). The per-document execute variables from Give code cells the Quarto 1 execute environment (QUARTO_EXECUTE_INFO and friends) #1161 (QUARTO_DOCUMENT_PATH etc.) were also wrong for later cells.
• Serial render (QUARTO_JOBS=1): a document could read the globals of whichever document ran before it, so results depended on render order.
• The daemon-wide sessions lock, taken per cell, also serialized cells on unrelated kernels.

Quarto 1 never shares a kernel between documents. Its one-shot path starts a fresh kernel per document, and its keepalive daemon is keyed by the document's path.

Fix

• SessionKey gains document (ExecutionContext.source_path).
• JupyterDaemon::acquire returns a SessionLease, an owned guard on that session's own lock, which the document holds for its whole run. The map lock is never held across an await, so documents on different kernels now run in parallel. Kernels start without a shared lock. The old start_lock also kept concurrent peek_ports calls apart; a small reserved-port set now does that job, holding each kernel's ports until it is ready.
• KernelScope carries a policy:
  • kernel_scope() (q2 render) shuts a document's kernel down as soon as the document finishes, on both the success and error paths. At most QUARTO_JOBS kernels are alive at once.
  • kernel_scope_retaining() (q2 preview) keeps each document's kernel warm, with its state, for the next render of that document, as Quarto 1's keepalive daemon does. No reset on reuse.
• Docs: guides/authoring/jupyter-kernels.qmd gains "Each document has its own kernel"; execution-context.qmd drops the shared-kernel paragraph.

Design discussion and decisions: claude-notes/plans/2026-11-11-jupyter-kernel-doc-isolation.md.

Tests

The tests in the first two groups failed at 78b4863c6 before the fix:

• Fake kernel (CI-safe), text_execute_isolation.rs: two documents in one directory run on different kernel processes; a document's kernel is gone once it finishes; a retaining scope reuses a document's kernel across renders. The fake kernel gains a KernelPid output so a test can see which process ran a cell.
• Real Python, jupyter_document_isolation_e2e.rs: the interleave under QUARTO_JOBS=2 (four cells, so the interleave happens every time at HEAD; the doc comment explains why) and the globals leak under QUARTO_JOBS=1. render_leaves_no_kernel_behind now expects one kernel per document, and both must be dead after exit.
• Lease tests in fake_kernel.rs: exclusive per document; a held lease doesn't block another document; Release shuts the kernel down; a waiter whose holder releases gets a fresh kernel the daemon tracks.
• The QUARTO_JOBS=1 workaround Give code cells the Quarto 1 execute environment (QUARTO_EXECUTE_INFO and friends) #1161 added to execute_env_cli_e2e.rs is removed.

No snapshot files changed.

Verification

• cargo nextest run --workspace: 19286 passed. Excluded: knitr::capture_tests::capture_classifies_pieces, which fails at main too on this machine because of knitr 1.52 (bd-3p8y4gdt, unrelated).
• cargo xtask verify --skip-hub-build: lints, clippy, build and the strict claude-notes render pass; the test step stops only on the knitr test above. The Jupyter engine is native-only, so the hub/WASM build is unaffected.
• Windows cross-check (cargo clippy --target x86_64-pc-windows-msvc) cannot run from macOS (ring/aws-lc-sys build C). The change adds no platform-specific code or cfg gates.
• End to end with q2 render on the repro fixtures: correct results at QUARTO_JOBS=1 and 2, with no stray kernels afterwards. The interleave fixture renders in 7.2 s at QUARTO_JOBS=2 versus 11.4 s at 1.
• End to end with q2 preview (headless browser, two "Re-execute" clicks): a.qmd re-ran on the same kernel (same PID, runs 1, then runs 2), b.qmd had its own kernel, and after SIGINT both kernels were gone and their connection files removed.

Second commit: graceful shutdown, and kernels dying at startup

The first CI run failed once on Ubuntu: q1_equivalence_fixtures hit failed to create shell socket: Connect timed out after 30s. It never reproduced locally in five full loops of all 301 Jupyter tests, and the rerun on fa24d2750 passed. Investigating it found two things, both fixed in fa24d2750:

• A regression from this PR: kernels were being SIGKILLed. KernelSession::shutdown() was a thin wrapper around shutdown_blocking(), and that function has to skip the graceful shutdown_request when it runs on a tokio runtime thread. On main, kernels were shut down from KernelScope::drop, off the runtime. A document's lease now ends inside engine_runtime().block_on, so every kernel was killed and user atexit handlers never ran. shutdown() now sends the request on the current runtime and waits for the kernel to exit before the kill backstop. A new e2e test, a_finished_documents_kernel_exits_gracefully, has an atexit handler write a marker; it failed before the fix and passes now.
• Diagnostics. A kernel that died while starting surfaced only as that 30 s timeout, and its stderr was piped but never read. The shell connect now races the process exiting, and such a kernel fails at once with ExitedDuringStartup, carrying its exit code and stderr. This is tested with a fake kernel that exits at startup; I wrote that test alongside the change, not before it. If the timeout recurs, the message will now tell a dead kernel from a slow one. The cause of the original timeout is still unknown: a port collision, or a kernel that started slowly under load.

Follow-up found along the way: bd-wjg6sjxj (kernels orphaned when the owning process is killed by a signal; present on main).

🤖 Generated with Claude Code

[posit-snyk-bot]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[cscheid]

@gordonwoodhull FYI, I ran into this pretty bad jupyter kernel bug and it needed a bit of a change on how we manage Jupyter kernels. I hope the changes here don't affect much your concurrent work on #1171 etc, but I thought I'd give you a heads up.

[gordonwoodhull]

This sounds right!

Should be easy to merge. Thanks!