Skip to content

Defer evaluating Node island modules until the program uses them (NI-7) - #1901

Merged
ctate merged 26 commits into
mainfrom
ni/i4-deferred
Oct 28, 2026
Merged

ctate merged 26 commits into
mainfrom
ni/i4-deferred

Conversation

@cramforce

@cramforce cramforce commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1900. Fourth of nine Node islands PRs. This PR has no bridge change (ABI 6).

What it does

A program that imports an island package but does not use it yet should not pay for starting Node.

  • Lowering. Every static island import loads through nodeIsland.defer(path, snapshot) in the importer's %init prelude.
  • Decision after lowering (node-islands/snapshot.ts). It takes the package's evaluate setting (auto | lazy | eager), then the static effect gate (node-islands/effect-gate.ts, rules in its header), then one build-time probe run for all candidates. A rejected module turns back into an eager nodeIsland.require. IR, LLVM and object outputs, and builds without a probe-able pack, stay eager. SCRIPTC_NODE_ISLANDS_TRACE=defer prints each decision.
  • Probe. packages/runtime/src/tools/scr_node_island_probe.c is a small C host built once with the host compiler and cached by source hash. It dlopens the pack's bridge, loads each candidate, records the effects it can observe, and records an interface snapshot: objects, members, flags, primitive constants and plain classes.
  • Runtime (scr_node_island_boot.c). It checks the boot guards with one stat per file: the pack's files, and every file in the module's closure (size and mtime). It then returns stub refs that answer recorded members, typeof and instanceof. Any other operation starts Node.
    • When Node starts, every deferred module evaluates in import order, and each stub is reconciled so it takes over the real value's bridge id. That keeps === exact.
    • A failed guard loads eagerly at the import position. It never uses a stale snapshot.

The effect gate's TypeScript 5 parser island is on the ts7 lane's allowlist (scripts/test-ts7.mjs).

Evidence

  • 6840-node-island-deferred-unused: imports, typeof, name, length, constants, instanceof and a stub alias, with no calls. It matches Node, and it checks /proc/self/maps itself: libnode mapped: false.
  • 6841-node-island-deferred-call: CJS and ESM deferred packages, with snapshot reads before calls. A function the island returns is === to the stub taken before Node started.
  • 6842-node-island-effectful-eager: a package that prints at load stays eager, and its output order matches Node.
  • Unit tests: effect-gate.test.ts.
  • Startup in the earlier measurement for 6840: 2.9 ms and 2.6 MB RSS deferred, against 37 ms and 86 MB eager.

Shortcuts

  • Once Node starts for any reason, every deferred module evaluates. Per-module evaluation comes in the recorded-deferral PR.
  • The decision uses one probe run plus the static gate, and there are no env or TTY guards. The gate refuses every process reference.
  • Guards compare size and mtime, not content hashes. beside copies therefore run eagerly unless hard links keep the mtimes.
  • The snapshot covers two levels of the module value. Object.keys, in and deeper reads start Node.

Gate results

  • Island set in a Vercel Sandbox, with the libnode dev pack rebuilt from this branch's bridge (node-island*.test.ts, npm.test.ts, packages/compiler/src/node-islands, differential 68xx): plain 223/223 + diff 52/52; sanitized 223/223 + 52/52.
  • pnpm lint, pnpm format:check and pnpm test:ts7 --baselines-only pass.
  • The self-hosted compiler (packages/compiler/src/native/cli.ts --emit=llvm) builds.
  • Zero-cost gate: 2838 programs identical (benchmarks, self-hosted compiler, TypeScript corpus) against the base branch. The only DIFFERS is corpus/fs-extra-package (%m0.settleVoid). It differs in every gate of the stack, including Bound V8's stack scan by Node's own stack in Node islands #1898, which does not touch the compiler, so it is nondeterministic on the base. both-fail covers only the island programs, because the gate host has no libnode pack. Compile time stays within tolerance.

CI on the first push found failures outside the focused island set: the ABI in bridge.test.ts, the runtime ABI header declarations, the effect gate on the ts7 allowlist, weak collections in the compiler, the pipeline scout and the Node version line. The fixes are folded into the PRs that introduce each piece, and they are re-checked in a Sandbox on the final tree.

Stack

#1898
#1899
#1900
→ #1901
#1902
#1903
#1904
#1905
#1906
#1907
#1908

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Oct 28, 2026 7:46am UTC

@vercel-security-reviewer

Copy link
Copy Markdown

Security review details

Comment thread packages/compiler/src/index.ts
Comment thread packages/compiler/src/node-islands/effect-gate.ts
Comment thread packages/compiler/src/node-islands/snapshot.ts
Comment thread packages/compiler/src/node-islands/effect-gate.test.ts
Comment thread packages/compiler/src/node-islands/snapshot.ts Outdated
Node islands stack, part 4 of 11 (scriptc-perf/results/islands-prs/PLAN.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	packages/runtime-pack-common/runtime-pack-matrix.mjs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cramforce and others added 3 commits October 27, 2026 19:14
Reading a class with a static accessor is refused because the accessor
can run import-time effects, but `const B = A` gave the alias the kind
"value", so a later read through `B` was admitted as pure and the
accessor's effects could move with the deferred module. Propagate the
"accessors" kind through alias chains to a fixed point and cover the
alias in the refusal test.

Co-Authored-By: Claude 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude 5.5 <noreply@anthropic.com>
@cramforce

cramforce commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Round complete (head 16f44273e).

  • Class-alias bypass of the effect gate (thread 4355087776): fixed in 1d2ab2e87. The alias scan now propagates the accessors kind through const B = A chains to a fixed point, so a read through an alias is refused like a read of the class itself. effect-gate.test.ts gained the alias refusal case (29 tests pass).
  • --libnode=beside digests (4355087772): fixed earlier in 0f7fd1b; the beside path validates the installation digests and falls back to the eager load.
  • Primitive .prototype classified as a plain class (4355087784): fixed earlier in 0f7fd1b; plainClass now requires an object-valued own prototype.
  • Resolver missing in effect-gate.test.ts (4355087788): fixed earlier in 2f8e8b2; the tests resolve through createRequire(file).resolve and pass standalone.
  • Predictable probe cache (4355087793): fixed earlier in 0f7fd1b; the probe requires ownership/write permission and falls back to a private temp directory.
  • 7cc3a9fe5 is the oxfmt fix for the new loop (lint and format was red on the previous head).

The branch was merged forward with the fixed #1899 heads (7d1a7f388, then c959443a8); CI is green on the new head.

The zero-cost gate against a4a844f44 (measured at 7cc3a9fe5) reports only the two differences the base branch ni/3-hosted-loop already shows on its own gate: @requires-libnode corpus programs that cannot compile in the gate sandbox (no libnode pack there; both sides error identically) and 4032-inline-rc-fast-paths gaining the scr_dyn_*.scrunit.scr_json symbols, which reproduces at ca4a162e3. Bench and self time are within tolerance.

🤖 Generated with Claude Code

Co-Authored-By: Claude 5.5 <noreply@anthropic.com>
Base automatically changed from ni/i3-exports to main October 28, 2026 07:40
@ctate
ctate merged commit e85af20 into main Oct 28, 2026
148 of 150 checks passed
@ctate
ctate deleted the ni/i4-deferred branch October 28, 2026 08:53

This branch was successfully deployed

1 active deployment
Preview — c9c8224a Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants