Skip to content

Defer island modules whose recorded load is deterministic (NI-7b/7d/7e/7f) - #1906

Merged
ctate merged 29 commits into
mainfrom
ni/i9-recorded-deferral
Oct 29, 2026
Merged

ctate merged 29 commits into
mainfrom
ni/i9-recorded-deferral

Conversation

@cramforce

@cramforce cramforce commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1905. Last of the nine Node islands PRs. Its tree equals int/islands minus the separately upstreamed #1834–#1837.

What it does

  • The deferral decision is the probe's load recording (node-islands/probe-recorder.ts). For each candidate module it records:

    • inputs: env keys, isTTY, argv, cwd, fs, os, tty;
    • volatile values;
    • effects: stdio, warnings, process listeners, timers, ticks, microtasks, async resources, net, child processes and more;
    • mutations of builtin state;
    • the module's file closure.

    A module defers when its recorded load threw nothing, had no effects and read no inputs other than guarded ones: env keys and isTTY become boot guards, and so do files inside the module's own package (size and mtime). The soundness argument is in the file.

  • Attribution. A call belongs to the module whose load made it. Loader frames, wrappers from earlier modules and builtins acting for their caller are seen through. Getter reads during the snapshot are not part of the load; a member whose getter has inputs or effects starts Node.

  • Per-module evaluation. When every static island import defers, no deferred module mutates builtin state and nothing loads islands dynamically, components of deferred modules evaluate on the first use of their own stubs. Otherwise every deferred module evaluates when Node starts, in import order. A late evaluation sees the env values from import time.

  • A native package's top-level require() of an island defers like a static import. Timers set and then cleared during a load are not an effect. os.homedir() and os.tmpdir() become HOME and TMPDIR guards.

  • Node starts with argv[0] = "node", so Node's warning hint and island process.argv0 read "node" as under Node.

  • The differential harness has an opt-in // @normalize-pid directive.

Evidence

  • Corpus programs, byte-identical with Node:
    • 6843: two packages with disjoint closures, evaluated per module.
    • 6844: an env key read at load and changed before first use.
    • 6845: a native package requires an island at top level, and libnode mapped: false holds before the first call.
    • 6846: a package reads its own files at load. It defers, and touching a file fails the boot guard.
    • 6847: punycode DEP0040 from island code outside node_modules, none inside, and a package that warns at load stays eager.
  • An earlier measurement used a lookalike of a real workload: twelve real npm packages, no private code. All island imports deferred, and Node was not mapped at boot. Boot took 2.9 ms and 3.5 MB deferred, against 124 ms and 112 MB eager.

llvm-differential honors // @normalize-pid too.

Shortcuts

  • Replaying recorded builtin mutations at Node start is not sound, so any mutation means every deferred module evaluates when Node starts.
  • Island runtime code that mutates a builtin before a lazy component evaluates is not detected. This has not been observed. The risk is written down in probe-recorder.ts.
  • ESM islands loaded through import() disable per-module mode.

Gate results

  • Island set in a Vercel Sandbox, with the libnode dev pack rebuilt from this branch's bridge (node-island*.test.ts, npm.test.ts, packages/compiler/src/node-islands, differential 68xx): plain 231/231 + diff 74/74; sanitized 231/231 + 74/74.
  • pnpm lint, pnpm format:check and pnpm test:ts7 --baselines-only pass.
  • The self-hosted compiler (packages/compiler/src/native/cli.ts --emit=llvm) builds.
  • Zero-cost gate: 2834 programs identical (benchmarks, self-hosted compiler, TypeScript corpus) against the base branch. The only DIFFERS is corpus/fs-extra-package (%m0.settleVoid). It differs in every gate of the stack, including Bound V8's stack scan by Node's own stack in Node islands #1898, which does not touch the compiler, so it is nondeterministic on the base. both-fail covers only the island programs, because the gate host has no libnode pack. Compile time stays within tolerance.

CI on the first push found failures outside the focused island set: the ABI in bridge.test.ts, the runtime ABI header declarations, the effect gate on the ts7 allowlist, weak collections in the compiler, the pipeline scout and the Node version line. The fixes are folded into the PRs that introduce each piece, and they are re-checked in a Sandbox on the final tree.

Stack

#1898
#1899
#1900
#1901
#1902
#1903
#1904
#1905
→ #1906
#1907
#1908

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Oct 28, 2026 6:58pm UTC

@vercel-internal-security-reviewer

Copy link
Copy Markdown

Security review details

…e/7f)

Node islands stack, part 9 of 11 (scriptc-perf/results/islands-prs/PLAN.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BSD script takes the command after the transcript file, needs a stdin
that is not a socket, and echoes the EOF it forwards from /dev/null.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A member read in an export getter (a.b or a[b]) can invoke another
accessor that runs code on every read, so the recorder now accepts only a
bare identifier as a plain binding getter. Everything else keeps its
runtime read and is not snapshotted.

The mutation check that makes an earlier island module eager copied only
data properties of a module value, so an object exported through a
binding getter could stay stale when a later load changed it. The shape
now reads binding getters the same way the snapshot does, and compares
them by reading again, so those modules no longer defer.

Corpus fixtures pin both: a getter that reads a member counts every read,
and a later load that mutates a getter-exported object keeps both modules
eager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cramforce

Copy link
Copy Markdown
Contributor Author

Addressed both remaining review threads in d061550.

Binding getters (probe-recorder.ts). Only a bare identifier qualifies now (() => name, function () { return name }, get x() { return name }). A member read inside the getter (() => a.b, () => a[b]) can invoke another accessor that runs on every read, so that member is no longer snapshotted and keeps its runtime read. New fixture tests/fixtures/node-island/deferral/getter-member fails on the previous head and passes here.

Mutation detection (probe-recorder.ts). exportShape copied only data properties, so an object exported through a binding getter was invisible to the check that makes an earlier module eager. The shape now reads binding getters the same way the snapshot does and compares by reading again; a later island load that mutates a getter-exported object now makes both modules eager instead of leaving a stale snapshot. New fixture tests/fixtures/node-island/deferral/mutate-getter fails on the previous head and passes here. Both cases are covered in tests/harness/node-island-deferral.test.ts (6 passed).

The existsSync and isTTY threads were already fixed in 4f07fbd; no action.

Validation on Linux in a sandbox: island harness 46/46, 68xx differential 133/133, npm.test.ts 141/141, lint and format clean.

Zero-cost gate (bin/validate rv-1906 --skip=lanes,bench,t262,san-bootstrap, base = merge base with main f4a4cf8): every benchmark and corpus program has byte-identical IR and LLVM; there are no ir: or llvm: difference lines at all. The reported failures are artifacts of comparing against main while the branch carries the whole islands stack:

  • 915 programs differ only in the linked executable. All of it comes from the runtime pack that the lower PRs (NI-1…NI-8) add — entry-point offsets shift, bytes relocate into a larger scr_error_vts, and scr_json.c's dynamic-tier units show up in corpus/4032-inline-rc-fast-paths.ts. None of these files were touched here (git log f4a4cf865..HEAD -- packages/runtime/src/scr_json.c packages/runtime/src/scr_exception.c names only NI-2…NI-7 merges).
  • 8 build errors are the stack's new node-call programs (6800, 7009, 7661, 7662, …), which do not exist at f4a4cf8 and need the libnode pack (SC8005).

An isolated run against this PR's parent (#1905 head a11b82d) is running so the recorder change is measured on its own; its log is validate/runs/20261028T022102-rv-1906.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cramforce

Copy link
Copy Markdown
Contributor Author

Stack maintenance round; head is now d5dab9243.

CI root cause. test (Linux, plain|san, 10/20) failed in unit-journal.test.ts in a lower branch (the island collections were unclassified). Fixed in 126154edc and carried here by 5f29686b0, which merged the current ni/i8-exit; d5dab9243 carries the lazy-traps divergence classification.

Review threads here remain resolved (the binding-getter recorder fixes in d06155048 are in the head).

Sandbox validation with a libnode pack built from the top branch's bridge (bridgeAbi 10): unit-journal, the eight node-island*.test.ts files (100 tests, including the deferral suites), the 64 island corpus programs, and the npm suite pass.

🤖 Generated with Claude Code

…deferral

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cramforce

Copy link
Copy Markdown
Contributor Author

Restacked after main moved. #1902 was merged at 17:41Z (merge commit 2f3b926de), so this branch now merges origin/ni/i8-exit and the merge was clean (no conflicts, no hand edits; pnpm manifest reproduces the merged surface manifest byte for byte). Head is now afa84f5c5 and a fresh CI run is in progress on it.

🤖 Generated with Claude Code

Base automatically changed from ni/i8-exit to main October 29, 2026 04:43
@ctate
ctate merged commit d972a6b into main Oct 29, 2026
151 of 154 checks passed
@ctate
ctate deleted the ni/i9-recorded-deferral branch October 29, 2026 20:18

This branch was successfully deployed

1 active deployment
Preview — afa84f5c Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants