Repository navigation
Host the runtime's event loop on Node's libuv once Node loads (NI-3) - #1696
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Scope-B milestone NI-3. When a program loads Node in-process, its pending ticks, promise jobs, timers and immediates move onto Node's queues in order (timers keep their deadlines relative to the moment Node began loading; the work of a phase the switch interrupted finishes first) and every later one is scheduled there, so the jobs of both sides interleave as in Node. The runtime's I/O hooks are watched from Node's loop, and its liveness is re-read before each poll phase, after each check phase and after every microtask checkpoint. Node's loop runs on Node's stack, and V8's limit follows whichever stack an entry comes from. Island signal and exit listeners join the runtime's registries at their position, so the runtime keeps owning every signal disposition; Node's own 'exit' emission is left to the runtime's exit sequence, and the exit code island code leaves in process.exitCode becomes the program's. Bridge ABI 3. Ordering programs (tests/corpus-node-islands-order, harness node-island-order.test.ts), corpus programs 6804-6811 and the node-island-process harness pin the behavior against Node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f6f880c to
eaefbdf
Compare
…pcontext warning, fix a race in 6811 - ni_stack_limit classifies the stack by the frame address. Under ASan's use-after-return detection a local lives on a fake stack, so a call into Node from a hosted promise reaction got the wrong V8 stack limit and hit "Maximum call stack size exceeded" (6802, dev, sanitized). - The ordering harness drops ASan's once-per-process makecontext/swapcontext warning from stderr, as the other harnesses do. - 6811's island timers fire at 15 and 40 ms, after the program's immediate and its 5 and 10 ms timers, instead of racing the first loop turn. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the @scriptc/libnode-<platform> packs (linux-x64-gnu, linux-arm64-gnu, darwin-arm64, darwin-x64): the pinned Node.js 24.22.0 built as a stripped shared library with full ICU and its startup snapshot, the scriptc bridge compiled against that release's headers, Node's LICENSE and a manifest with the Node version, NODE_MODULE_VERSION, bridge ABI and file hashes. packages/libnode-common holds the pin, the build (fetch and verify the source tarball, configure --shared, build only the libnode target, strip, stage with headers in a cache keyed by release, flags, platform and toolchain; the bridge rebuilds in seconds against the cached headers), a verifier, and a smoke test: a C program that loads a pack the way the runtime does, starts Node from the snapshot and evaluates expressions synchronously, asynchronously and throwing. The libnode-packs workflow builds the packs on native runners, Linux in the manylinux_2_28 images for a glibc 2.28 floor, caches each libnode build, smoke-tests and uploads every pack. It never runs in the ordinary build or test path. The compiler gains the pack lookup (SCRIPTC_LIBNODE, then the installed package), validation, the nodeIslands.libnode placements and the license notices for the Node island loader to use. The bridge source and its C ABI header are the NI-1 bridge agent's, included here so the packs build; the packs stay private until the in-process island runtime ships. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s in CI Stack the libnode packs on the in-process loader (#1685) and use its pack assembler and manifest format instead of a second one: build-pack.mjs keeps its CLI and fields (schema, nodeVersion, moduleVersion, bridgeAbi, platform, arch, files), adds the libnode and bridge names, strips and (on macOS) re-signs the bridge and links it with -z defs, and exports buildPack(), which the platform packs call with their package, target and build inputs as extra manifest fields. libnodeRecord falls back to the installed @scriptc/libnode-<platform> pack, found from the entry's directory and then next to the compiler, when neither --libnode nor SCRIPTC_LIBNODE names one. The Linux test lanes restore the libnode build that the libnode-packs workflow cached, assemble the linux-x64 pack from it in seconds (build.mjs --from-cache never builds Node) and export SCRIPTC_LIBNODE, so the Node island corpus and harness tests run instead of skipping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An installed @scriptc/libnode-<platform> pack versions in lockstep with the compiler, whose version every executable cache key already holds; keying on it as well made the full compiler's key differ from the bootstrap's routed key wherever a pack directory was present, so routed cache hits stopped. Only directories with a manifest.json count as installed packs, so an unbuilt workspace package is not one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's island lanes fall back to the development pack (dev-pack.json: Node 24.22.1, no package.json) until the libnode-packs cache exists, so the smoke test, which verifies a @scriptc/libnode-<platform> package against the 24.22.0 pin, failed there. It now runs only when SCRIPTC_LIBNODE names a package directory. The pack package versions follow the 0.4.2 release (scripts/sync-versions.mjs), which pin.test.ts checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
g01 (twenty fs.stat round trips against a 2 ms timer), p10 (two 0 ms timers armed on either side of an island call) and s06 (fs callbacks and timers pending when Node loads) each failed one CI run in a different lane: their order depends on how long the island call or the I/O takes, which differs between the sides and grows under ASan. Their native runs now have to produce the lines of one of Node's outputs, like the other race programs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Node islands stack, part 1 of 11 (scriptc-perf/results/islands-prs/PLAN.md). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hosted loop's scheduling entries (a tick, a timer or an immediate goes through the bootstrap's helpers; a checkpoint drains ticks and microtasks) and island process-event listeners called the bridge on whatever stack the runtime was on: the main thread's for the first checkpoint, or an async fiber's. JavaScript then ran there, and a collection it triggered failed V8's "Check failed: isolate_->IsOnCentralStack()" (node-island-handles, about 1 run in 15 sanitized: promise reactions run by the first checkpoint on the main stack allocated a microtask). They now run on Node's stack like every other entry, below the mark when they come from a fiber. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nested trampoline never returns (setcontext back), so the redzones around its locals stayed poisoned in the shadow of Node's stack, and Node's own frames tripped over them later (stack-buffer-underflow in __bzero, sanitized corpus 6822/6823/6831/6832/6894 on macOS). Linux's ASan clears a context's stack in its swapcontext interceptor; macOS's has none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
6809-node-island-stdio armed a 5 ms island timer and a 10 ms native timer next to immediates. On a loaded host the main script can run past 5 ms after arming, so Node runs the island timer before the immediates and the output order depends on load. The timers are now 50 ms and 100 ms, well after the script and the immediates finish; the expected order is unchanged.
# Conflicts: # packages/compiler/src/index.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An island signal registration is now tagged with the raw listener it installed, so a once registration and an on registration of one function dispatch through their own listeners instead of both resolving to the plain function, and the once registration no longer fires indefinitely. The removal handler forgets the entry whose raw listener is no longer on process, so a once wrapper that removes itself leaves its runtime registration behind no longer; duplicate registrations of one function share a raw listener, and the most recent of those is removed. refresh() of a timer armed before Node loads re-arms at now + the delay the timer was created with. Its Node handle was created with the remaining delay at handoff and a due timer had no handle at all, so the handle is replaced rather than refreshed. The signal program registers one function once and on; a new timer program refreshes a timer most of whose delay passed before Node loaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fixes for the three open Vercel Agent findings are pushed as 8887f35 (also merged into #1899 so the islands stack stays consistent). What changed
Tests
Zero cost All C changes are inside CI The earlier red runs were the 🤖 Generated with Claude Code |
|
Validation note — zero-cost gate
IR and LLVM for that program are byte-identical; only the link differs. The cause is the cached base runtime pack, not this branch. Proven by rebuilding the base tree's pack with the helper: the base compiler at Root cause in the tooling: 🤖 Generated with Claude Code |
Host the runtime's event loop on Node's libuv once Node loads (NI-3)
Scope-B milestone NI-3, stacked on #1685 (NI-1: in-process libnode loader, with
@scriptc/nodefrom #1686).Once a program loads Node in-process, there is one event loop, Node's, and one set of queues. The jobs of both sides interleave as they do in Node.
The switch (
scr_loop_host, theSCR_NODE_ISLANDSbuild ofscr_async.c)Right after Node starts, the runtime moves its pending work onto Node's queues, in order:
queueMicrotaskenvelopes become V8 microtasks;process.nextTickcallbacks;Every later scheduling call goes through the bridge:
scr_ready_push, the tick functions,scr_timer_pushand the timer ref, unref, refresh, hasRef and clear calls, and the immediate functions.A switch can happen in the middle of a loop pass. In that case:
UV_RUN_ONCE);Calls into JavaScript no longer drain queues themselves. The runtime's checkpoint (the bridge's
checkpoint) and Node's own dispatch do that. A throw that escapes a runtime job, or an unhandled runtime rejection decided after a V8 microtask checkpoint, ends the process the same way the runtime'smainwould.I/O
The runtime's
scr_loop_set_*slots stay where they are. On Node's side:io_statereports their fds (watched asuv_pollhandles, which are stopped rather than closed when an fd goes away and comes back);io_dispatchruns the dispatch stations with Node's checkpoint after each one. The bridge re-reads the state before each poll phase, after each check phase, and after every microtask checkpoint, so liveness is current wherever Node next decides it. Deferred fs callbacks are delivered on the turn after they are queued, as the runtime's own loop pass delivers them.Process state (the
SCR_NODE_ISLANDSbuild ofscr_events.c,scr_node_island_host.cjs)exitlisteners that island code adds join the runtime's registries at their registration position. The bootstrap removes Node'sstartListeningIfSignal/stopListeningIfSignalhooks, so libuv never owns a signal. Removing an island listener therefore no longer resets the signal toSIG_DFLunderneath the program's own handler.exitemission is left to the runtime's exit sequence, which runs the listeners of both sides in order.process.exitCodebecomes the program's when the loop ends.process.exitalready runs the runtime's exit sequence (NI-1).mainthat calls into Node got "Maximum call stack size exceeded".Bridge ABI
ScrNodeBridgeandScrNodeBridgeHostgain the hosted-loop entries listed inscr_node_bridge.h. NI-1's pump slot (backend_fd,backend_timeout,alive,pump) is gone. The bridge ABI goes from 2 to 3. NI-1's dedicated Node stack is kept: Node's loop runs on it once it is hosted, and V8's limit also follows an async fiber that a hosted job resumes and that calls into Node again (the newstack_limithost callback). The bridge smoke test drives the hosted entry points.Tests (Linux sandbox, community libnode 24.22,
node24.22 as the oracle)Ordering fidelity:
tests/harness/node-island-order.test.tsruns 36 programs intests/corpus-node-islands-order/:At 20 Node runs against 20 native runs, 664/720 native runs produced an order Node produced, and every queue-determined program matched 20/20. The 56 misses are timer-against-I/O races in p05, p08, p11, q04, s01 and s02, where the duration of the I/O differs between the sides (scriptc's fs callbacks complete in one turn, while Node's
readFiletakes several; the first module load inside Node is slower than in a Node process). These programs are marked// @order: race, and for them the harness requires the same lines in any order.Process state, corpus (byte-exact against Node):
process.exitinside a call;process.exitfrom an island timer;process.exitwith island listeners;uncaughtExceptionlistener after the switch.Process state, harness:
tests/harness/node-island-process.test.tscovers uncaught exceptions and unhandled rejections on either side. Exit status and stdout match Node, and stderr carries the error.Cost (sandbox, per callback, Node loaded / Node itself / scriptc without Node):
setImmediatechain: 3.6–4.2 / 2.2 / 0.11 µs;nextTick: 0.31 / 0.14 / 0.01 µs;await: 81 / 72 / 2 ns;setImmediate: 2.1 µs against Node's 1.7 µs.The budget is 12 µs per callback.
Zero cost
Every change to the shared units is behind
SCR_NODE_ISLANDS. In the island-free build,unifdefshows only macro aliases that expand to the old code, plus scoping braces. Zero-cost gate against the NI-1 base,--set=bench,self,corpus --corpus=all, 3546 programs: 3540 identical, 4 allowed (6800-6803, the programs that call into Node), and 2 that differed:2314-process-introspection:scr_active_resourceswas rewritten through a local variable. Fixed (the island-free path is now the original code), and a re-run over the bench, process, timer and event programs found 137 identical.fs-extra-package: the IR records the checkout path of the compiler's fs-extra shim (/vercel/sandbox/prevagainst/vercel/sandbox/cand). The two IR files are identical once that path is normalized, and two builds from the same checkout are identical. This is an artifact of the gate, not of this change.Not in this PR
cwdcache (Node cachesprocess.cwd()in JavaScript, so a nativeprocess.chdiris not visible to island code): a hook inscr_lib.cwould change island-free binaries, so it waits for a variant.beforeExitstays a documented divergence.process.listeners()does not see native signal listeners.🤖 Generated with Claude Code