Skip to content

perf(sdk): start the authored Node probe while the flow loads - #684

Open
miyaontherelay wants to merge 2 commits into
mainfrom
perf/authored-node-probe-prewarm
Open

miyaontherelay wants to merge 2 commits into
mainfrom
perf/authored-node-probe-prewarm

Conversation

@miyaontherelay

@miyaontherelay miyaontherelay commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What

The standalone CLI imported the flow and hashed its Surface package (checkAuthoredTriggers). Only then, at the first authority check before root admission, did it spawn the Node probe and SHA-256 hash the Node executable, about 110 MB. #681 made the second check reuse that answer, but the first one still waited for the whole probe.

runDirectFlow now calls prewarmAuthoredRuntime() before it loads the flow. That starts the same probe and hash without blocking: an async spawn, and the executable streamed through the hash in 1 MB chunks. The prewarm only fills #681's memo, under #681's rules:

  • the same candidate, by path and by realpath
  • an unchanged dev:ino:size:mtime:ctime fingerprint
  • not stored if the file was rewritten while it was being hashed

The first check (assertAuthoredRuntimeAvailable, now async) awaits the prewarm and then validates the memo exactly as before. If the prewarm failed or left nothing usable, that check probes itself and returns the same refusal it always did. The async probe keeps the sync probe's limits: exit 0, at most 4096 output bytes, 10 s.

The prewarm does nothing outside the standalone CLI (the Bun runtime with an embedded Node runner), and on resume and other commands.

The authority code (#681's memo, the probe, and now the prewarm) moves from authored-node-runner.ts into authored-node-authority.ts. The runner was at 500 lines. The moved code is unchanged except for the shared helpers split out for the async path.

What does not change

  • Replay identity and parallel safety. The root metadata, admission spec, executionRuntime values and eligibility code are untouched. The authority object is the same {path, version, executableSha256}, computed from the same bytes.
  • The child check. The Node child still hashes the executable it runs, and the parent still compares that report with the authority, so a binary swapped after the probe is still refused.

Timing

These are standalone bun-darwin-arm64 builds with the same measurement-only marks as #683 (script in #683). Each run is flows run of a one-step flow, with Surface 2.0.54 in the Cloud Bun envelope and relayflowd already started. There were 20 interleaved rounds on a shared macOS arm64 machine, load average 25–60.

Phase (median ms) main (n=20) pr2 (n=20)
Bun CLI start (spawn to runDirectFlow) 156 155
Load flow in Bun 29 67
of which Surface hash #1 18 56
Daemon connect 3 2
Node probe and hash #1 70 0
Parallel eligibility 23 23
Admit, dispatch, authority #2 22 23
Write and re-check payload 12 10
Node child start to modules loaded 154 155
Watchdog and child hashes 80 72
Child loads pinned source 101 98
of which Surface hash #2 41 40
Journal connect to first step 21 21
Spawn to first step 673 634

The probe and hash no longer stall admission (70 → 0 ms). The load phase grows by about 38 ms, because the executable hash shares Bun's main thread with the load: it runs in chunks between the load's own work. Net, spawn to first step improves by 39 ms here. A second batch of 20 rounds gave 703 → 658 ms.

Locally the load is short (about 30 ms), so it can hide only part of the hash. On E2B the load phase was about 356 ms in my earlier profile, so the overlap there should hide the whole probe. That is an estimate and has not been measured.

With #683 (same batch, for context):

Phase (median ms) main (n=20) pr1 (n=20) pr2 (n=20) both (n=20)
Bun CLI start (spawn to runDirectFlow) 156 156 155 155
Load flow in Bun 29 25 67 24
of which Surface hash #1 18 15 56 13
Daemon connect 3 3 2 10
Node probe and hash #1 70 69 0 35
Parallel eligibility 23 23 23 24
Admit, dispatch, authority #2 22 23 23 24
Write and re-check payload 12 13 10 9
Node child start to modules loaded 154 154 155 156
Watchdog and child hashes 80 77 72 73
Child loads pinned source 101 80 98 77
of which Surface hash #2 41 19 40 19
Journal connect to first step 21 21 21 20
Spawn to first step 673 640 634 617

With both PRs the load is too short to cover the hash, so the first check still waits about 35 ms. Moving the hash to a worker thread would remove the main-thread contention; that is not in this PR.

Tests (fail on revert)

New file tests/authored-node-prewarm.test.ts. It uses a stand-in Node that answers the probe and logs each call, with a fresh module per case:

  • probes and hashes once, and the later call reuses the same authority (the sha256 of the file)
  • returns at once and leaves the event loop free while a 0.4 s probe runs
  • probes again when a different candidate is selected by the time it is needed
  • a failed prewarm leaves the probe and refusal to the call
  • does not trust a probe answer over the 4096-byte limit the sync probe enforces

Standalone (tests/authored-node-runtime.test.ts, Bun 1.4.x compiled CLI): starts the Node probe while it imports the flow, and probes once per run. The flow's top-level code records whether the probe wrapper had already run while Bun was importing it. The wrapper must also run exactly once for the whole run.

Mutation checks, each restored with git checkout -- src:

Mutation Result
delete the prewarmAuthoredRuntime() call in direct-run.ts × … starts the Node probe while it imports the flow, and probes once per run → expected 'false' to be 'true'
prewarm calls the sync nodeAuthority() × returns at once and leaves the event loop free while the probe runs → expected 619.67… to be less than 150
async probe without the output limit × does not trust a probe answer over the output limit… → promise resolved "{ …(3) }" instead of rejecting
reuse the memo without checking the selection × probes again when a different candidate is selected… → expected '/Users/…' to be '/Users/…'
restored Tests 5 passed (5)

The #681 tests (tests/authored-node-authority.test.ts, 5 tests) pass unchanged.

Verification (Node 22.23.4, Bun 1.4.2, at 59a59e1)

$ npm run typecheck        # exit 0
$ npm run build            # ok
$ npm run typecheck:tests  # ok
$ npx vitest run tests/authored-parallel-*.test.ts tests/authored-flow-lifecycle-executor.test.ts tests/daemon-lifecycle.test.ts tests/daemon-lifecycle-live.test.ts tests/mcp-lifecycle.test.ts tests/authored-node-runtime.test.ts tests/authored-node-authority.test.ts tests/authored-surface-authority.test.ts tests/cloud-answer.test.ts tests/authored-root.test.ts tests/authored-flow.test.ts tests/authored-node-prewarm.test.ts
 ✓ tests/authored-node-runtime.test.ts (23 tests)
 ✓ tests/authored-parallel-run-live.test.ts (42 tests | 7 skipped)
 ✓ tests/authored-parallel-llm.test.ts (6 tests)
 ✓ tests/authored-parallel-agents.test.ts (8 tests)
 ✓ tests/authored-parallel-eligibility.test.ts (13 tests)
 ✓ tests/authored-parallel-body.test.ts (5 tests)
 ❯ tests/authored-flow-lifecycle-executor.test.ts (51 tests | 3 failed)
 Test Files  1 failed | 15 passed (16)
      Tests  3 failed | 331 passed | 7 skipped (341)

The 3 failures are the completes a signal wait after … round(s) cases. They fail identically on main 2cd9561 on this machine (Tests 3 failed | 48 passed (51)).

This PR is independent of #683, and the two compose without conflicts (the combined build above).

Review round 1 (14a6137)

  • Probe stderr is bounded. spawnSync's maxBuffer limits each stream; I confirmed ENOBUFS on 5000 bytes of stderr on both Node 22 and Bun 1.4.2. The async probe now bounds stdout and stderr separately.
  • An unneeded prewarm is stopped. runDirectFlow stops any prewarm still running on every return path. The probe is SIGKILLed, its pipes closed, and the hash worker terminated, so a refused run no longer waits up to 10 s for a slow Node candidate.
  • The executable is hashed on a worker thread (readFileSync inside an eval worker; same bytes as the sync path). This answers the earlier finding that the main-thread hash slowed the flow load.

New tests, each failing on revert (output is in the review replies):

  • does not trust a probe whose stderr is over the limit…
  • stops a slow probe at once…
  • the worker assertion in probes and hashes once…
  • standalone does not wait for a slow Node probe when the flow is refused before it needs Node (fails with expected 9542 to be less than 5000 when the stop call is removed)

Timing: 20 interleaved rounds, same harness, load average 9–57:

Phase (median ms) main (n=20) #684 at 59a59e1 (n=20) #684 at 14a6137 (n=20) both at 14a6137+e5053b2b (n=20)
Bun CLI start (spawn to runDirectFlow) 154 157 158 157
Load flow in Bun 28 71 36 24
of which Surface hash #1 18 61 22 14
Daemon connect 3 2 3 3
Node probe and hash #1 73 0 45 53
Parallel eligibility 24 24 23 23
Admit, dispatch, authority #2 24 28 24 25
Write and re-check payload 12 10 10 11
Node child start to modules loaded 155 158 158 157
Watchdog and child hashes 80 79 87 80
Child loads pinned source 105 104 101 81
of which Surface hash #2 42 42 41 19
Journal connect to first step 24 21 21 20
Spawn to first step 679 659 665 644

The worker brings the load phase back to near main (71 → 36 ms) because the hash no longer shares the main thread. The authority check now waits about 45 ms, because locally the load is shorter than the probe spawn plus a 110 MB hash. End to end, locally the worker is within noise of the main-thread version. Its benefit depends on the sandbox having a spare core while the load runs; that has not been measured on E2B.

Tests (Node 22.23.4, at 14a6137): Tests 4 failed | 333 passed | 7 skipped (344) over the same suites.

  • 3 known failures: the completes a signal wait after … round(s) cases, which also fail on main.

  • 1 failure in resumes 'llm' branches without replaying completed effects (run_admission_conflict). It also fails with main's built CLI. In 15 interleaved runs each of the same test: main's dist 1 failure, this branch 3. Likely cause, not yet confirmed:

    • In the serial fallback, f.run ids come from call order (nextId('run'), authored-flow-executor.ts:530).
    • Each branch's f.run follows its own f.llm, so the ids follow the order the two llm results settle.
    • If a resume settles them in a different order, run-1 maps to the other command and its admission key is already bound to a different spec.

    Neither change touches step ids or replay. Under the Node CLI, which this test uses, the prewarm is not active. The higher rate on this branch may still come from timing, so this should be fixed separately.

  • CI linux-x64-artifact at 14a6137: pass.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Agent Relay sessions

  • claude session a7546818 · opened via gh pr create · last active 2026-10-26

The standalone CLI imported the flow and hashed its Surface package, and only
then spawned the Node probe and SHA-256 hashed the ~110 MB executable for the
first authority check before root admission.

Start that probe and hash before the flow load, without blocking it. It only
fills #681's memo under the same rules (same candidate by path and realpath,
unchanged stat fingerprint, a file not rewritten while hashed), and the first
authority check awaits it and still validates the memo. A failed or unusable
prewarm leaves the probe, and its refusal, to that check. The async probe
keeps the synchronous probe's contract: exit 0, 4096 output bytes, 10 s.

The authority code moves to authored-node-authority.ts; the runner was at the
500-line limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6ee2ce69-6e45-4333-9310-f699fd886e16
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge; both previous findings are addressed.

Summary

Starts the Node probe while the standalone CLI loads the flow.

  • Direct runs start checking Node while they load the flow.
  • Node authority checks now live in a separate module.

Both previous findings are addressed. No new actionable issues were found.

Diagram

sequenceDiagram
  participant CLI
  participant Flow as Flow loader
  participant Probe as Node probe
  participant Worker as Hash worker
  CLI->>Probe: Start prewarm
  CLI->>Flow: Load flow
  Probe->>Worker: Hash selected executable
  Worker-->>CLI: Remember authority
  Flow-->>CLI: Loaded flow or refusal
  alt Flow accepted
    CLI->>CLI: Await prewarm and recheck authority
  else Flow refused
    CLI->>Probe: Stop probe
    CLI->>Worker: Stop hash
  end
Loading

Reviews (2) · Last reviewed commit: "fix(sdk): bound probe stderr, stop an un..." · Reviewed by Greptile

Comment thread packages/sdk/src/cli/direct-run.ts
Comment thread packages/sdk/src/authored-node-authority.ts Outdated
…a worker

Review (#684):
- The async probe discarded stderr, while spawnSync's maxBuffer bounds each
  stream. A candidate writing over 4096 bytes to stderr passed the prewarm
  and was refused without it. Bound stderr the same way.
- A run refused before it needs Node returned while the prewarm probe was
  still running, so a slow Node candidate kept the CLI alive up to the 10 s
  probe limit. runDirectFlow now stops any prewarm still running when it
  returns: the probe is killed, its pipes closed, the hash worker ended.

Also hash the executable on a worker thread. The chunked main-thread hash
shared Bun's main thread with the flow load and slowed it (#684 timings).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant