Skip to content

feat(apple): serve the Simulator AX bridge from the XCTest runner - #4182

Merged
thymikee merged 25 commits into
mainfrom
feat/runner-hosted-ax-bridge
Nov 5, 2026
Merged

thymikee merged 25 commits into
mainfrom
feat/runner-hosted-ax-bridge

Conversation

@thymikee

@thymikee thymikee commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Summary

Limrun iOS snapshots read the same Simulator AX bridge as a local Simulator. Limrun cannot start the guest bridge process, so the XCTest runner compiles the shared apple/snapshot-bridge reader and serves snapshotBridge off its XCTest main thread. Both channels share one route: generation circuit, surface-absence proof (runner foreground read, then the reader's own host check), and one typed XCTest fallback. snapshot-tree-changed (a list scrolling between two reads) costs one capture, not the generation. A refusal-code fixture and guard keep Objective-C, Swift and TypeScript in step. The Limrun runner must be built with Xcode 26. ADR 0004 is amended.

84 files against main (which now has #4224's runner switch). Follow-ups: #4205, #4241.

Validation

Head da8e78ee7c, base main.

  • pnpm check:affected --run --head da8e78ee7c: passed, exactCommit=true (run-dbxS07).
  • Live at this head, Limrun iOS 26.4 with an Xcode 26.2 runner: bridge captures, both web sheets read by XCTest, appstate before and after the runner starts (evidence).
  • Earlier live runs: Bluesky on Limrun and a local Simulator bridge-served (p50 0.64 s / 0.18 s), no runner restart or circuit retirement (evidence); snapshot-tree-changed → XCTest → bridge on one generation on Limrun (evidence); release-shaped runner, Xcode 26.6 (evidence).

Risks: the abandoned-read path has unit evidence only (#4241).

@vercel

vercel Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
agent-device-website Ready Ready Preview Nov 4, 2026 6:27am UTC

Request Review

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.03 MB 6.06 MB +29.8 kB
Package (unpacked) 6.03 MB 6.06 MB +29.8 kB
Package (download) 1.83 MB 1.83 MB +8.7 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 43.5 ms 35.1 ms -8.3 ms
CLI --help 74.8 ms 66.6 ms -8.2 ms

@thymikee
thymikee force-pushed the feat/runner-hosted-ax-bridge branch 2 times, most recently from c2841b2 to 5445f27 Compare November 3, 2026 08:18
@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

At 5445f27 I found one likely defect in the cancel path and one missing live run. CI is green (17 checks, 0 not passing), and there are no conflicts. Before merge, bridge-read-in-flight needs to be capture-scoped with a cancel-then-capture route test, and a Limrun system-surface run at this head needs to be attached.

I found this by reading the code, not by running it, so please confirm it. If a caller cancels a runner-served describe (a cancelled request, or a wait/batch step that aborts its capture), send() releases the daemon's previousRead turn at once in the finally. The runner does not see the disconnect, so its reader stays in flight until the AX read returns. The next capture of the same generation then gets application_not_responding / bridge-read-in-flight, which maps to kind timeout. bridgeFailureDisposition does not list that code in CAPTURE_SCOPED_REFUSALS, so the route adds the generation to disabledGenerations. On Limrun, one cancelled capture followed by a capture inside that read's lifetime would move every later capture of the app generation to XCTest. Each of those costs 1-1.6 s plus a target round trip and carries the XCTest warning. The refusal only says the reader is busy, so it is not evidence that the generation cannot be served. The local guest path queues the next read instead of retiring the generation. The rule: a refusal that only says another read is still running is capture-scoped on every channel. CAPTURE_SCOPED_REFUSALS enumerates those, and the exhaustive warning switch needs a consequence for the new entry. With that in place, the daemon-side previousRead queue can go, because the runner's admission is the one owner. The ADR sentence that says only an abandoned read can meet this refusal needs correcting. Please add a route test that cancels a describe, sends a second capture that the runner refuses with bridge-read-in-flight, and asserts the third capture is bridge-served. That test should fail without the fix.

On Limrun, systemSurfacePresent is fixed to unknown. So the runner's appState foreground read is the only thing that keeps a capture off an occluded app tree (#2438). The PR body shows bridge-served captures with no warnings, but no run where a registered host (SafariViewService or PassbookUIService) is foreground over the app. The unit tests stub appState, so a wrong answer on the live branch would serve the occluded app tree with no warning. Please run once on Limrun at this head: open an app, present a web sign-in sheet, and take a snapshot. The output should show producer xctest, the surface tree, and no snapshotBridge describe in the diagnostics for that capture. Then dismiss the sheet and show the next capture served by simulator-ax-bridge. Please also attach the command output for the claimed bridge-served and local Simulator runs.

Not blocking, take or leave: the runner budget at runner-snapshot-bridge.ts:152 is computed before send() waits for the previous read's turn, so a read that queues longer than the margin is aborted by the daemon before the runner abandons it (computing it from the deadline left after the turn, or dropping the daemon queue, fixes this); RunnerSnapshotBridgePrimaryApp and RunnerSnapshotBridgePrimaryPid in RunnerSnapshotBridge.m:26 repeat AgentDeviceForegroundElement and the primaryApp lookup in isPrimaryForegroundProcess: (a shared primaryForegroundPid() in SnapshotBridgeRuntime.m would serve both); the comment at runner-runtime.ts:112 says observe-only reads the runner's tree directly, but the provider capture throws UNSUPPORTED_OPERATION for every observe-only input; createAppleRunnerRuntime never calls snapshotRoute.shutdown(), so the transport close and the source's closed state never run; and the foreground reads in system-surface-foreground.ts:69 now go out together for local Simulators too, so a local capture always sends every appState read (fine to keep, but worth naming in the PR body).

What I did not check: whether a timed-out or cancelled read-only runner exchange on the Limrun session triggers runner recovery above runAppleRunnerCommand; whether the AX reader is safe on a background queue while XCTest issues its own AX queries (only the body's live Limrun claim covers that); bridge-source.ts against adapter.ts line by line (I checked the move by structure only); and the check:affected receipt, Swift unit-test results and live runs from the PR body, which I did not re-run. I also could not confirm whether dropping the XCTest keyboard field on the Limrun bridge path differs from local bridge captures. The bridge tree source has no keyboard field, which matches local Simulator behavior.

@thymikee
thymikee force-pushed the stack/limrun-native-runner branch from 1a5ca97 to f165d9c Compare November 3, 2026 12:50
@thymikee
thymikee force-pushed the feat/runner-hosted-ax-bridge branch from 5445f27 to 7e0f29c Compare November 3, 2026 12:50
@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Thanks. Both points are fixed at 7e0f29cf02, and the stack is rebased on current main.

1. bridge-read-in-flight is capture-scoped. You were right. bridge-read-in-flight now has its own consequence (bridge-read-busy) in the failure policy, so the route answers it with the XCTest fallback for that capture only and never retires the generation. The daemon-side previousRead queue is gone; the runner's admission is the one owner. That also removes the budget-before-queue issue you named. New route test: cancel a describe, the second capture meets bridge-read-in-flight and falls back, the third is bridge-served. It fails without the fix. The policy test pins bridge-read-abandoned as generation-scoped. The ADR sentence now says a read a cancelled capture left running meets the refusal too, and that the refusal has capture scope.

2. Live system-surface run on Limrun at this head. No repo app presents a web sign-in sheet, so I used a 60-line UIKit probe app with two buttons: SFSafariViewController and ASWebAuthenticationSession. Same script on Limrun (iPhone, iOS 26.4, Xcode 26.2 runner built from this head) and on a local iOS 26.2 Simulator, snapshot -i --json --debug each time:

capture Limrun producer snapshotBridge cmds Local producer top nodes warning
app simulator-ax-bridge 2 (target + describe, cold) simulator-ax-bridge SheetProbe, Sheet Probe Home —
app simulator-ax-bridge 1 simulator-ax-bridge SheetProbe, Sheet Probe Home —
Safari sheet xctest 0 xctest Safari, Toolbar, Close web sign-in sheet; inapplicable (system-surface-presented)
after Close simulator-ax-bridge 1 simulator-ax-bridge SheetProbe, Sheet Probe Home —
auth sheet xctest 0 xctest Safari, Toolbar, Cancel web sign-in sheet; inapplicable (system-surface-presented)
after Cancel simulator-ax-bridge 1 simulator-ax-bridge SheetProbe, Sheet Probe Home —

Each Limrun sheet capture sent 2 appState + 1 snapshot to the runner and no snapshotBridge. Wall times on Limrun: 0.46–0.53 s bridge, 0.74–1.14 s sheet (first capture 12.4 s includes runner start). One earlier run hit a foreground read past its 1 s connect budget while the sheet was presenting; that capture reported system-surface-probe-unavailable and still went to XCTest with the surface tree, so the fallback stays on the safe side.

Also changed: Xcode 27 is not expected on Limrun. An Xcode 27 simulator build embeds no XCTest frameworks, so it cannot start on Limrun's iOS 26 simulators. The release workflow now selects Xcode 26, the package script refuses a runner without XCTest.framework, and the Limrun products check refuses one before upload with a hint. Docs say so.

Non-blocking items: the observe-only comment is corrected; createAppleRunnerRuntime now has shutdown(), which closes the route and source, and Limrun dispose calls it; the parallel foreground reads are named in the PR body. I left the primaryForegroundPid duplication in RunnerSnapshotBridge.m: sharing it means changing the guest runtime, which I would rather do with #4172.

Gate: FALLOW_AUDIT_BASE=f165d9c46c pnpm check:affected --run --head 7e0f29cf02 passed, exactCommit=true, 42 checks. Swift unit tests: 8 passed.

Still not checked: the native abandoned-read path, and whether a timed-out read-only runner exchange on Limrun triggers runner recovery above runAppleRunnerCommand.

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

The code looks good at 7e0f29c. The earlier review issues are addressed, and the Limrun system-surface run you posted covers the occluded case: supplied signal and deadline aborts now become request cancellation without triggering ready-session recovery, while unrelated transport failures retain recovery. All 17 checks passed. There are no conflicts.

Not blocking: The cancellation assertion can accept an unrelated error; you can take or leave an assertion against cancelled.signal.reason, or a named typed cancellation reason, in the cancellation test: https://github.com/callstack/agent-device/blob/7e0f29c/packages/platform-apple/src/runner-snapshot-bridge.test.ts#L324.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 3, 2026
@thymikee
thymikee force-pushed the stack/limrun-native-runner branch from f165d9c to 00d77a6 Compare November 3, 2026 15:00
@thymikee
thymikee force-pushed the feat/runner-hosted-ax-bridge branch from 7e0f29c to fd56bac Compare November 3, 2026 15:00
@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

The code looks good at 3a4eb6d. The surface proof is now fenced to one runner session, the dead limrun-ios-tree producer is gone, and the refusal codes are guarded against drift. All 16 checks pass and there are no conflicts. The native sources have not changed since fd56bac, so the earlier Limrun and local sheet runs still cover them.

Not blocking: You can take or leave extending the emitter guard to enumerate deadline-helper and native-build forwarding paths and classify their codes in the fixture, while keeping bridgeFailureDisposition as the runtime policy owner; the current guard recognizes only literal TypeScript snapshotSourceError calls. Guard

@thymikee

thymikee commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

The code at c56942e looks good: a tree that changes during a bridge read is now refused for that capture only. This new native path still needs a live run. CI reports 16 passing checks and no failures. There are no known conflicts.

No recorded live run reaches the new refusal on either production channel, so device validation remains incomplete despite green CI (SnapshotBridgeRuntime.m:50). Please rebuild c56942e and record snapshot -i --json --debug on both Limrun and a local Simulator during deep-tree churn. Each run must show snapshot-tree-changed, XCTest fallback, and then simulator-ax-bridge on the same generation; include the commands, device, and SHA. Please attach this evidence before merge.

@thymikee thymikee removed the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 3, 2026
@thymikee
thymikee force-pushed the feat/runner-hosted-ax-bridge branch from c56942e to bacf32e Compare November 4, 2026 06:26
@thymikee
thymikee force-pushed the stack/limrun-native-runner branch from 00d77a6 to 24be243 Compare November 4, 2026 06:26
@thymikee

thymikee commented Nov 4, 2026 •

Copy link
Copy Markdown
Member Author

Live evidence for the snapshot-tree-changed refusal at head bacf32eeb3. This is the stack as rebased onto main and force-pushed at 06:26Z; the PR's 9 commits are patch-equivalent to c56942ed24 per git range-diff. On bacf32eeb3, FALLOW_AUDIT_BASE=24be24328d pnpm check:affected --run --head bacf32eeb3 passed with exactCommit=true (receipt run-CokQ9M); I ran it before this live run. Local Simulator: done. Limrun: live refusal evidence still pending; the Xcode 26 CI runner is now available (update below). The PR stays in draft.

Local Simulator

  • Device: iPhone Air 1B89356B-961C-4993-B0BE-4C0F3917D995, iOS 27.0. Runner built from bacf32eeb3 with Xcode 27.1 into a private derived path. Channel: guest-process bridge.
  • App: Agent Device Tester release build (embedded JS, includes Heavy feed), com.callstack.agentdevicelab.
  • Churn: Heavy feed (30 wrappers; 13–21 continuations per read at 32 accepted levels). A swipe fling followed at once by a capture, so momentum scrolling recycles FlatList cells between a node's first read and its continuation re-read. No product code was changed.

Commands (worktree CLI at bacf32eeb3, private state dir):

AD="env AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH=/private/tmp/ad-4182-local2/derived node bin/agent-device.mjs --state-dir /private/tmp/ad-4182-local2/state --session ax4182b --platform ios --udid 1B89356B-961C-4993-B0BE-4C0F3917D995"
$AD open com.callstack.agentdevicelab --relaunch
$AD open com.callstack.agentdevicelab 'agent-device-test-app://heavy-feed'
# repeated until the refusal appeared (iteration 2):
$AD swipe 210 300 210 880
$AD snapshot -i --json --debug
# then, after the scroll came to rest:
$AD snapshot -i --json --debug

--debug stderr carries only the daemon request lines. The route lines below are from the request diagnostics logs that --debug writes (state/sessions/ax4182b/requests/<requestId>.ndjson). They are raw except that unrelated retry/exec_command/appState lines are left out.

1–2. Refusal, then XCTest serves that capture (request 34837628e06f38d1). After a swipe, the snapshot command takes stabilization captures. The first two are bridge-served. The third meets snapshot-tree-changed, and XCTest serves it, and that XCTest tree is the one returned. Its first warning is: Simulator AX snapshot unavailable (snapshot-tree-changed); used XCTest for this capture, while the screen changed during the bridge read.

{"ts":"2026-11-04T07:04:41.753Z","level":"debug","phase":"ios_snapshot_source_recovery","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","data":{"producer":"simulator-ax-bridge","targetId":"1B89356B-961C-4993-B0BE-4C0F3917D995:com.callstack.agentdevicelab","generation":"15938:UIKitApplication:com.callstack.agentdevicelab[268e][rb-legacy]:Wed Nov  4 08:01:44 2026","requestedLevels":129,"hint":"hinted","hintedLevels":32,"requests":22,"rejected":0,"continuations":21,"acceptedLevels":32,"truncated":false,"learning":"kept"}}
{"ts":"2026-11-04T07:04:41.753Z","level":"info","phase":"ios.snapshot-source.acquire","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":333,"data":{"producer":"simulator-ax-bridge","transport":"guest-process"}}
{"ts":"2026-11-04T07:04:41.756Z","level":"info","phase":"ios.snapshot-source.present","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":3,"data":{"producer":"simulator-ax-bridge"}}
{"ts":"2026-11-04T07:04:42.296Z","level":"debug","phase":"ios_snapshot_source_recovery","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","data":{"producer":"simulator-ax-bridge","targetId":"1B89356B-961C-4993-B0BE-4C0F3917D995:com.callstack.agentdevicelab","generation":"15938:UIKitApplication:com.callstack.agentdevicelab[268e][rb-legacy]:Wed Nov  4 08:01:44 2026","requestedLevels":129,"hint":"hinted","hintedLevels":32,"requests":22,"rejected":0,"continuations":21,"acceptedLevels":32,"truncated":false,"learning":"kept"}}
{"ts":"2026-11-04T07:04:42.296Z","level":"info","phase":"ios.snapshot-source.acquire","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":321,"data":{"producer":"simulator-ax-bridge","transport":"guest-process"}}
{"ts":"2026-11-04T07:04:42.298Z","level":"info","phase":"ios.snapshot-source.present","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":2,"data":{"producer":"simulator-ax-bridge"}}
{"ts":"2026-11-04T07:04:42.824Z","level":"error","phase":"ios.snapshot-source.acquire","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":308,"data":{"producer":"simulator-ax-bridge","transport":"guest-process","error":"iOS Simulator snapshot source malformed-tree: snapshot-tree-changed"}}
{"ts":"2026-11-04T07:04:42.824Z","level":"debug","phase":"ios_snapshot_route_fallback","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","data":{"reason":"snapshot-tree-changed","deviceId":"1B89356B-961C-4993-B0BE-4C0F3917D995","generation":"15938:UIKitApplication:com.callstack.agentdevicelab[268e][rb-legacy]:Wed Nov  4 08:01:44 2026","details":{"guestMessage":"snapshot continuation children unavailable","bridgeFailure":"malformed-tree","bridgeFailureCode":"snapshot-tree-changed"}}}
{"ts":"2026-11-04T07:04:47.971Z","level":"info","phase":"ios_runner_command_send","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":5143,"data":{"command":"snapshot","commandId":"runner-c54196e3-061d-4ec3-893c-f6957caddb34","readOnly":true,"sessionReady":true,"timeoutMs":44999}}
{"ts":"2026-11-04T07:04:47.971Z","level":"info","phase":"snapshot_capture","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","durationMs":5146,"data":{"backend":"xctest"}}
{"ts":"2026-11-04T07:04:47.974Z","level":"warn","phase":"post_gesture_snapshot_stabilization_timeout","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot","data":{"action":"swipe","attempts":3,"durationMs":6569,"lastPairAgreed":false}}
{"ts":"2026-11-04T07:04:47.975Z","level":"info","phase":"request_success","session":"ax4182b","requestId":"34837628e06f38d1","command":"snapshot"}

3. Next capture: simulator-ax-bridge on the same generation (request 30adede85db36841; generation 15938:…[268e]…08:01:44 in both):

{"ts":"2026-11-04T07:04:51.506Z","level":"debug","phase":"ios_snapshot_source_recovery","session":"ax4182b","requestId":"30adede85db36841","command":"snapshot","data":{"producer":"simulator-ax-bridge","targetId":"1B89356B-961C-4993-B0BE-4C0F3917D995:com.callstack.agentdevicelab","generation":"15938:UIKitApplication:com.callstack.agentdevicelab[268e][rb-legacy]:Wed Nov  4 08:01:44 2026","requestedLevels":129,"hint":"hinted","hintedLevels":32,"requests":22,"rejected":0,"continuations":21,"acceptedLevels":32,"truncated":false,"learning":"kept"}}
{"ts":"2026-11-04T07:04:51.506Z","level":"info","phase":"ios.snapshot-source.acquire","session":"ax4182b","requestId":"30adede85db36841","command":"snapshot","durationMs":316,"data":{"producer":"simulator-ax-bridge","transport":"guest-process"}}
{"ts":"2026-11-04T07:04:51.510Z","level":"info","phase":"ios.snapshot-source.present","session":"ax4182b","requestId":"30adede85db36841","command":"snapshot","durationMs":3,"data":{"producer":"simulator-ax-bridge"}}
{"ts":"2026-11-04T07:04:51.511Z","level":"info","phase":"request_success","session":"ax4182b","requestId":"30adede85db36841","command":"snapshot"}

The refusal had capture scope: the generation was not retired (learning: kept, no disabled), and the next capture of the same app process used the bridge again. The same three steps showed up at c56942ed24 before the rebase (requests 9e1372624101587e → bfa156ecb40e8473, generation 44988:…07:31:08). Sessions closed with --shutdown; the daemons are stopped and the iPhone Air is shut down.

Limrun: earlier Xcode 27 attempt

At the time of this attempt, this host had only Xcode 27.1 and iOS 27 runtimes. The local toolchain could not build the release-shaped Limrun runner from this head, and the release runner (v0.21.36) did not contain this PR's native change. The CI build below now clears the toolchain blocker.

As a fallback (at c56942ed24, before the rebase), I built the runner with Xcode 27.1 and embedded the Xcode 27.1 simulator XCTest frameworks, the same set that the v0.21.36 runner carries. I pointed AGENT_DEVICE_LIMRUN_IOS_RUNNER_APP at it:

AGENT_DEVICE_SESSION=ax4182-limrun AGENT_DEVICE_LIMRUN_IOS_RUNNER_APP=<that .app> \
  node bin/agent-device.mjs --state-dir /private/tmp/ad-4182-limrun/state connect limrun --platform ios
… install com.callstack.agentdevicelab ./AgentDeviceTester.app   # Installed
… open com.callstack.agentdevicelab                              # Opened
… snapshot -i --json --debug

The runner never accepted a connection on the Limrun iOS 26 simulator:

{"level":"info","phase":"limrun.runner.launch","data":{"path":"/xctest/run","port":36657}}
{"level":"error","phase":"ios_runner_command_send","data":{"command":"uptime","sessionReady":false,"error":"Runner did not accept connection"}}
{"level":"error","phase":"request_failed","command":"alert","data":{"error":"Runner did not accept connection (launcher exited early)"}}

This matches the Xcode 27 limit that the PR documents. That runner shape is not the shipped one, so this run says nothing about the PR code on Limrun. The Limrun instance I started (lease 8a7c0b9b3d0a8d8922e2b083e07df4f5, ios_euna_01m6jm9gvbfaessq9ttdh4sy3q) is terminated per the Limrun API.

Update: Xcode 26 CI runner available

Built exact source bacf32eeb348b6dd0f2c7cf4436641020226a498 with Xcode 26.6 (17F113) in CI run 41849063216. The runner artifact contains the packaged app, checksum, source SHA and Xcode version. Downloaded it locally, verified its SHA-256, and checked that it contains arm64/x86_64 binaries and embedded XCTest.framework. The artifact expires November 7; the downloaded copy is at /private/tmp/ad-4182-xcode26-runner.

The temporary CI branch was removed after the successful build. No release was created.

This clears the build blocker. It does not provide the requested live Limrun evidence: snapshot-tree-changed, XCTest fallback, then a bridge-served capture on the same generation. This session's device tools expose only local devices, so that run remains pending and the PR stays in draft.

🤖 Generated with Claude Code

@thymikee thymikee closed this Nov 4, 2026
@thymikee thymikee reopened this Nov 4, 2026
@thymikee

thymikee commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

At bacf32e, local live evidence now covers guest-process transport, but the Limrun run could not start the Xcode 27-built runner on its iOS 26 simulator. Could you run this head with an Xcode 26 runner on Limrun and capture snapshot-tree-changed, the XCTest fallback, and a subsequent bridge-served capture on the same generation? The runner bridge refusal remains unverified on the principal changed device route.

CI reports 18 passing checks and no non-passing checks at head. No conflicts are known. Please post the head-specific Limrun evidence before merge.

@thymikee

thymikee commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Update on the remaining Limrun request: the Xcode build blocker is cleared. CI run 41849063216 built exact head bacf32eeb348b6dd0f2c7cf4436641020226a498 with Xcode 26.6 (17F113). The runner artifact is downloaded and checksum-verified, with XCTest embedded and arm64/x86_64 binaries.

I updated the evidence comment to remove the stale toolchain blocker. The requested live Limrun snapshot-tree-changed → XCTest fallback → bridge recovery sequence remains unverified. This session exposes only local device targets. Keeping the PR in draft until that evidence is attached.

@thymikee

thymikee commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Limrun evidence for the snapshot-tree-changed refusal at head 3f4a65ba69. This answers the remaining request.

  • Device: Limrun iOS 26.4 (instance limrun:ios:c461de40f6c2…). The runner was built from 3f4a65ba69 with Xcode 26.2 and embeds XCTest. Channel: runner.
  • App: Agent Device Tester, Release build with the Heavy feed (built with Xcode 27.1, because Xcode 26.2 fails on expo-modules-jsi).
  • Churn: I ran 60 iterations of swipe 210 300 210 860 --count 3 --pause-ms 0, each followed by snapshot -i --json --debug. On Limrun each read takes 9–21 continuations.

The refusal fires inside the post-swipe stabilization of one snapshot request (a8a35de6e259712e). That capture is served by XCTest. The stabilization re-read then uses the bridge again on the same generation:

18:33:50.116Z ios.snapshot-source.acquire  {"producer":"simulator-ax-bridge","transport":"runner","error":"iOS Simulator snapshot source malformed-tree: snapshot-tree-changed"}
18:33:50.117Z ios_snapshot_route_fallback  {"reason":"snapshot-tree-changed","generation":"98789:1793817059.156915","details":{"guestMessage":"snapshot continuation children unavailable","bridgeFailureCode":"snapshot-tree-changed"}}
18:33:52.638Z snapshot_capture             {"backend":"xctest"}
18:33:53.578Z ios_snapshot_source_recovery {"producer":"simulator-ax-bridge","generation":"98789:1793817059.156915","continuations":9,"learning":"learned"}
18:33:53.579Z ios.snapshot-source.acquire  {"producer":"simulator-ax-bridge","transport":"runner"}

The next request (1fe62818c0716d70) is bridge-served on generation 98789:1793817059.156915 with learning: kept, in the same runner session. There is no circuit-disabled and no runner restart.

A second occurrence happened on Bluesky's feed: request 193169c1df0f360a, generation 94658:1793816774.978155. It showed the same refusal, the XCTest capture, and then bridge recovery in the same session.

A full pass on this head ran Bluesky (signed in) on Limrun and on a local iOS 26.2 Simulator: the full flow, 30 scroll-and-capture cycles, busy/cancel, and the web-sheet probe. Every warm app capture was bridge-served, except the expected typed fallbacks:

  • Limrun showed the alert's foreground-owner-unverified, and once system-surface-host-unverified after forced XCTest reads.
  • Local showed the lingering host once.

No app tree was served under a sheet. There was no circuit retirement and no runner restart. The typical capture took 0.65 s on Limrun and 0.19 s locally.

@thymikee
thymikee marked this pull request as ready for review November 4, 2026 18:38

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 84 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread scripts/package-ios-simulator-runner.sh Resolved
Comment thread apple/snapshot-bridge/SnapshotBridge.m Resolved
Comment thread packages/contracts/src/ios-snapshot.ts Resolved
Comment thread packages/kernel/src/snapshot.ts Resolved
@thymikee

thymikee commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

At 3f4a65b, the code changes since the earlier review look right, and all 15 checks pass. No conflicts are known. The remaining gap is live evidence for the runner-served bridge.

The runner-served bridge (https://github.com/callstack/agent-device/blob/3f4a65b/packages/platform-apple/src/runner-snapshot-bridge.ts#L137) is the main device route this PR changes, and no live run on Limrun reaches it yet. The runner also now scans the Simulator process table itself (SnapshotBridgeSurfaceHosts.m inside the XCTest runner), and no run shows what that scan returns inside the runner sandbox. So the snapshot-tree-changed refusal, the XCTest fallback, the bridge recovery and the new runner-side surface host check are all unverified on a real provider Simulator. On Limrun at 3f4a65b with the Xcode 26 runner artifact, please record snapshot -i --json --debug and post the request diagnostics for two cases. First, snapshot-tree-changed, then XCTest for that capture, then simulator-ax-bridge on the same generation. Second, one runner describe that carries surfaceHostExecutables, once while a SafariViewService sheet is presented (it must answer system-surface-host-present and route to the runner surface) and once without it (it must serve from the bridge, or answer system-surface-host-unverified). Please name the device, the commands and the SHA.

Not blocking, and fine to take or leave: in apple/snapshot-bridge/SnapshotBridgeSurfaceHosts.m (https://github.com/callstack/agent-device/blob/3f4a65b/apple/snapshot-bridge/SnapshotBridgeSurfaceHosts.m#L117) an empty candidate set counts as proven absence, so a filtered KERN_PROC_ALL table that returns no error would let the reader serve the occluded app tree, where before this change the runner channel always sent that capture to XCTest; the rule could be that the scan claims absence only from a table it can show is complete, checked in the shared reader for both channels, for example by treating the table as unverified unless it contains the capture target's own pid, which the request already carries as expectedPid.

No review threads are open.

thymikee and others added 18 commits November 5, 2026 19:06
Co-Authored-By: Claude <noreply@anthropic.com>
…drift

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
The bridge reader re-reads a node whose children the first read withheld.
When a scrolling list recycles its cells between the two reads, the
re-read returns fewer children, and the runtime reported every such
reader error as snapshot-tree-malformed. The route then retired the app
generation, so one scroll moved every later capture of the app to XCTest
(seen on Bluesky's feed). The reader errors are now a typed enum, and the
runtime maps each one to its refusal: a changed tree is
snapshot-tree-changed with capture scope, and a foreground change during a
continuation is foreground-owner-changed.

Co-Authored-By: Claude <noreply@anthropic.com>
…e guest channel

#4172 moved the guest channel's presence check into the bridge request: when
no runner can answer the foreground read, the route names the surface host
executables, and the reader checks the process table in place. The runner
channel now takes the same path instead of a constant 'unknown' presence
probe. The runner forwards the executables to the shared reader and names the
Simulator it runs in. A sandbox that refuses the process table answers
system-surface-host-unverified, so the capture uses XCTest and says why.

The surface-host refusal codes join the refusal fixture, and the runner
request fixture pins a describe that carries the executables.

Co-Authored-By: Claude <noreply@anthropic.com>
…lformed

A continuation whose children were not an array shared the tree-changed
code, so its capture-scoped refusal never opened the generation circuit.
Only a child-count shrink is tree-changed now.

Co-Authored-By: Claude <noreply@anthropic.com>
…able rejection

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
… header

The guest bridge and the host decoder count the 4-byte length prefix
against maxResponseBytes; the runner did not, so an envelope could pass
the runner and fail the host. The default bound is the reader's own
frame bound, and the reader request adds only the fields the host sent.

Co-Authored-By: Claude <noreply@anthropic.com>
A describe request cannot omit the identity it is checked against, and
the foreground read always carries the runner session it expects.
Adds the observe-only admission and runner budget regression tests.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…ips it

Co-Authored-By: Claude <noreply@anthropic.com>
The runner compiles the shared reader for the iOS Simulator only, so the
macOS runner unit-test build could not link SnapshotBridgeHandleRequest.

Co-Authored-By: Claude <noreply@anthropic.com>
The host reads still go out together, but a decisive first host now
answers without waiting for the reads after it.

Co-Authored-By: Claude <noreply@anthropic.com>
Off the iOS Simulator the default envelope bound was 0, so the runner's
runner-bridge-unavailable refusal came back as response-too-large when a
request named no bound.

Co-Authored-By: Claude <noreply@anthropic.com>
@thymikee
thymikee changed the base branch from stack/limrun-native-runner to main November 5, 2026 18:08
thymikee and others added 2 commits November 5, 2026 19:10
The wire-vocabulary guard matched SNAPSHOT_BRIDGE_PROTOCOL_VERSION 1 as a
substring, so a native protocol version of 10 still passed. It now matches
each whole #define line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner sent an absent SIMULATOR_UDID as an empty string, and its
bridge handler re-detected that with a second copy of the reader's
64-character UDID bound. The request builder is the only place that
knows the UDID is absent, so it now leaves simulatorUdid out, and the
handler refuses the surface proof with system-surface-host-unverified
when the key is missing. The reader keeps the only UDID validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee
thymikee force-pushed the feat/runner-hosted-ax-bridge branch from aeb439d to da8e78e Compare November 5, 2026 18:10
@thymikee

thymikee commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Live on the new base: da8e78ee7c, rebased onto main, which now carries #4224's switch of Limrun to the XCTest runner. The Limrun runner was built from this head with Xcode 26.2 and run on Limrun iOS 26.4.

  • Captures: bridge → bridge → XCTest (Safari sheet, web sign-in warning, the sheet's own tree) → bridge → XCTest (auth sheet) → bridge.
  • appstate before the runner starts: success, state: null, after the fix(limrun): stop answering appstate for an unreachable iOS instance #4176 instance check.
  • appstate after the captures: success, runningForeground, from the runner.
  • Other commands: install, open, press and screenshot all succeeded. No runner restart.

@thymikee

thymikee commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

At da8e78e, the code looks good. The rebase onto main (with #4224's runner launcher) keeps the bridge on the same runAppleRunnerCommand path with the capture signal, the missing-UDID handling no longer duplicates the reader's bound, and the live Limrun iOS 26.4 run at this head (#4182 (comment)) covers the bridge, XCTest fallback and appstate routes. CI is green (20 checks) and there are no conflicts.

The two inline threads on the protocol.test.ts macro anchors and the literal 64 in RunnerSnapshotBridge.m are fixed and can be resolved. The two on a retired generation after a runner restart and a wedged reader holding the read latch are deferred to #4241; they cost only an extra round trip, since captures still fall back to a correct XCTest tree, so they can be resolved here too.

For the record: the absent-SIMULATOR_UDID branch can only be reached by the unit test, since the runner always runs inside a Simulator that sets it, and cancelling a bridge read through the new launcher was checked in code, not live.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 5, 2026
@thymikee
thymikee merged commit e6ac165 into main Nov 5, 2026
20 checks passed
@thymikee
thymikee deleted the feat/runner-hosted-ax-bridge branch November 5, 2026 18:50
@thymikee

thymikee commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Post-merge record for da8e78ee7c, the merged head.

Base and rebase. I retargeted the PR from stack/limrun-native-runner (#4227) to main and rebased only #4182's own 23 commits onto origin/main. #4227's 4 commits were not carried, because #4224 already brings that work to main.

Review follow-ups (one push):

  • 6fb0dd16cb: the version guard matches each whole #define line. A VERSION 10 mutant now fails it.
  • da8e78ee7c: the Swift request builder leaves simulatorUdid out when SIMULATOR_UDID is absent or empty. RunnerSnapshotBridgeHandle refuses with system-surface-host-unverified only when that key is missing, so the reader keeps the only UDID bound. 6 bridge Swift unit tests passed on an iOS 27 Simulator.

Gate. pnpm check:affected --run --head da8e78ee7c passed with exactCommit=true, receipt run-Kp89ZG. Two earlier runs at the same head failed while the host load average was 290–570, both outside this PR's code:

  • unit: timeouts in daemon-client and platform-android tests. Those files pass alone (32/32).
  • provider-integration: the fake record command in apple-platform-output-guard.test.ts hit its 20 s real-time limit.

The passing run was at a load average of about 60.

Live run at da8e78ee7c. On the iPhone Air (1B89356B-…, iOS 27.0), with the runner built from this head, I repeated the heavy-feed fling churn. On generation 85812:…[d8e4]…19:58:56:

  1. request 3cda1dc2a5e84a55 met snapshot-tree-changed;
  2. XCTest served that capture (snapshot_capture backend: xctest);
  3. request b4fb604b95336d68 was served by simulator-ax-bridge with learning: kept.

Threads. All 4 open threads are answered and resolved: the macro anchors and literal 64 are fixed; the runner restart and wedged reader are deferred to #4241. No open threads remain.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant