[muvaf]

Summary

Replace Limrun iOS AX automation with the native agent-device XCTest runner through Limrun HTTP forwarding. Snapshots, interactions and Apple operations use shared execution paths; commands such as agent-device snapshot -i stay unchanged.

Following #4213, this 140-file replacement has two commits: shared prerequisites, then the provider switch and AX deletion. Keeping the switch and deletion together avoids a temporary mixed backend. Scope also includes the Revyl screenshot cancellation test and physical-iPhone replay adjustment. Local app sync follows in #4225.

Validation

Tested 7e18ad0a6fcdb4b677df5b94602afddaaf8fedcb: pnpm check:affected --run --head HEAD passes with exactCommit: true, including 17,297 unit tests, provider integration and compatibility checks. Public evidence includes the regression comparison against 442a6ce, independent review, and release-runner builds with Xcode 26.4/27.1. The prerequisite commit independently builds and typechecks; earlier full-gate evidence remains attributed to its original SHA.

The packaged 26.4 runner, including verified Testing overlays, passed three connections to one Limrun simulator: 21 snapshot nodes and live foreground state each time. Persistent slot conflicts and sparse captures retain typed recovery errors. Intermittent XCTest capture stalls remain possible; their underlying cause is not claimed fixed. GitHub-owned device/coverage lanes remain CI obligations. No fresh physical-phone run is claimed.

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

[cubic-dev-ai]

All reported issues were addressed across 116 files

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic</sub>

[thymikee]

At 0ae3dc2, one problem needs live proof before merge. All 17 checks pass, but no Limrun lane runs in CI, so CI does not exercise the forwarded-runner route. No conflicts are known.

The PR body says a reused Limrun simulator hit XCTest snapshot timeouts and only a fresh instance passed. Reuse is a supported route: attached instances (LIM_IOS_INSTANCE_URL, closing a lease never terminates the simulator) are reconnected across sessions and daemons. Nothing at this commit handles that state. Launch retries a 409 only when this runner instance canceled a run itself, and a snapshot timeout surfaces as the generic runner timeout. A second session on the same instance can then fail every snapshot with no actionable error, and the root cause is unknown. Could you reproduce this at 0ae3dc2? Connect twice to the same LIM_IOS_INSTANCE_URL, and once more after killing the daemon. Run open and snapshot -i each time, and attach the daemon log phases limrun.runner.prepare and limrun.runner.launch plus the runner stream. Then either fix the cause in the runner launcher (for example reset or reinstall the staged runner, or end the prior run) or fail with a typed details.reason and a hint. A passing snapshot -i on the reused instance, or that typed error, is the proof I need.

This PR is one commit with 116 files and about 10,370 changed lines, against the 2,500-line limit in docs/agents/pull-requests.md. Could the prerequisite exports, helpers and packaging go in a first commit or PR, with the switch and the deletion kept together in the second?

Not blocking, and fine to take or leave: (1) limrun-ios-tree has no producer left since the old snapshot adapter was deleted, so deleting it from the IosSnapshotProducer and IosProviderAcquisitionProducer unions lets the compiler list every table, comment and stub to remove, including the stale entries in interactor-operation-catalog.ts:188, eager-closure-budgets.ts:208 and seams.ts:144; (2) runner deployment and forwarding failures have no details.reason (an occupied XCTest slot is a bare COMMAND_FAILED 409 with no stop-Appium hint, and a runner built for the wrong Xcode or iOS shows as a generic startup timeout), so one helper with a few typed reasons used by every throw in limrun-ios-runner.ts and limrun-ios-runner-products.ts would let tests assert reasons instead of message text; (3) awaitWithSignal in request-cancel.ts repeats awaitLimrunOperation, so keeping one owner in kernel and building the other on it would remove the copy; (4) the Revyl screenshot-cancel test rewrite and the physical-iPhone replay change do not touch Limrun, so they could move to their own PR or be named in the body.

Of the open inline threads, these still apply: download test accepts any failure, download test may skip cleanup, raw timeout from unreachable instance, download errors lack typed reason, app log history lost on reconnect, abort listener never removed and shutdown skipped on throw, plus 11 lower-priority threads. The console launch owner, install-leaves-app-stopped, capabilities.md and shared runner products threads no longer apply.

Before merge, please post the reused-instance run above.

[muvaf]

Addressed the review in 442a6ce44. The history now has two commits: independently buildable shared prerequisites, then the provider switch and AX deletion.

I reproduced the reuse case at 0ae3dc2a and reran the same script against the same attached simulator at the new head. Both final reconnects returned 21 snapshot nodes. After killing the active daemon, the third connection returned limrun_xctest_slot_occupied with recovery instructions. A separate run reproduced the capture stall and verified limrun_runner_snapshot_unavailable, retaining capture-quality details and a recovery hint. The underlying intermittent XCTest stall is not claimed fixed, and we do not terminate another client's run.

Before/after logs, prepare/launch diagnostics, runner streams, scripts and validation receipts are published. pnpm check and the affected gate both pass at 442a6ce44 with exactCommit: true. The prerequisite commit also builds and typechecks independently. GitHub-owned lanes remain CI obligations; no fresh phone run is claimed.

The optional cleanup is included: obsolete producer removal, typed deployment/forwarding failures, one kernel cancellation helper, and explicit scope for the Revyl/iPhone test edits. The inline findings have fixes or documented dispositions. An independent review also caught preparation cancellation races, now covered by regression tests.

[thymikee]

At 442a6ce, most of the earlier review is addressed: the typed runner reasons, the snapshot classifier, the shared await-with-signal, preparation waiter ownership and the download and file-staging cleanups are in, and the reuse logs from your fork cover the route CI cannot run. All 17 checks are green and there are no conflicts. One defect remains.

The 409 retry in limrun-ios-runner.ts:158 never surfaces limrun_xctest_slot_occupied. That branch runs when this daemon's own run ended and it relaunches in the same lease. It is bounded by AbortSignal.timeout(90_000) and the shared launch phase budget, not by retryWithPolicy exhaustion, and the 89 sleeps already total about 87.75 s before round-trip time, so on a remote instance the timer or budget fires first. The loop then throws request_canceled (or the fetch wait rejects with operation_deadline_expired) and the last 409 is dropped. If Limrun keeps the slot, which is the reported reuse failure, the user waits out the startup budget and sees a cancel or a generic deadline error instead of the typed refusal and its recreate hint. Every launch that ends because /xctest/run kept answering 409 should surface limrun_xctest_slot_occupied, whether the retry was skipped, exhausted or cut off by the budget. Could the wait use retryWithPolicy's deadline option (below the launch phase budget) instead of a timer signal, so exhaustion rethrows the last 409, with a fixture test where canceledRunConflicts outlasts the budget and asserts details.reason? This comes from reading the retry and budget code; I did not observe a slot held past the budget.

Not blocking, take or leave:

• a fixture test that drives a sparse runner snapshot reply (capture-failed, no viewport) through createInteractor().snapshot and asserts details.reason, since the classifier is only tested with a hand-built error;
• limrunRunnerError overwrites a caller's hint with hints[reason], so the hint at limrun-ios-runner-products.ts:36-44 never reaches the user (details.hint ?? hints[reason]);
• abortMessage is still threaded through LimrunRequestOperationDrain.wait and awaitLimrunDeploymentOperation though awaitLimrunOperation ignores it;
• website/docs/docs/commands.md:821 still says Limrun iOS has no live app state read, while the capability table now says XCTest foreground state, and limrun.md could name the two new reasons near line 99;
• the preparation waiter refcount is a third copy of the last-waiter-cancels pattern (next to the apple-runner artifact start admission and the recording video waiters), and limrun.runner.prepare is emitted twice on a cold launch.

Open threads: the runner archive overlays question still needs a release-runner build to confirm, and the commands.md appstate sentence still stands. The other threads are fixed or no longer apply at this head and can be resolved.

[muvaf]

Addressed this follow-up in 7e18ad0a6fcdb4b677df5b94602afddaaf8fedcb, keeping the two-commit history.

The shared launcher now receives the start deadline. Limrun gives the 409 retry loop a shorter deadline through retryWithPolicy, preserving limrun_xctest_slot_occupied and the last response when retries end. An admission timeout also preserves the last 409; explicit caller cancellation remains request_canceled. The regression tests independently cover normal and short retry budgets, admission timeout, and caller cancellation, and assert settlement before manually firing admission timeout.

The sparse-response tests go through forwarded HTTP and createInteractor().snapshot(), with both zero nodes and a zero-sized root. They caught a missing shared-path detail: the zero-node rejection now preserves capture quality for the Limrun classifier. Custom hints survive, unused abortMessage arguments are gone, and the docs describe live XCTest app state and both recovery reasons. Preparation diagnostics now have one owner; the live run recorded one preparation event per launch.

For the overlay question, I ran the release workflow's build/package commands with Xcode 26.4 and 27.1. The 26.4 archive contains Testing.framework and all four toolchain overlays, with byte-identical overlay binaries. I deployed that archive to Limrun: three connections each returned 21 interactive snapshot nodes and live runningForeground app state, including a reconnect after stopping the previous lease and restarting our daemon. The test daemon and instance were cleaned up. Xcode 27.1 does not bundle Testing.framework, so its conditional overlay copy does not apply; that archive built and packaged but was not live-tested.

I left generic waiter extraction out after independent review: artifact-start admission, recording waiters, and runner-product preparation differ in retry retention and whether cleanup must drain before replacement. Sharing those policies would be a separate lifecycle change; they already share awaitWithSignal.

pnpm check:affected --run --head HEAD passed with exactCommit: true, including 17,297 unit tests. The same seven focused cases produce five failures/two passes on 442a6ce, then seven passes at the new head. GitHub-owned lanes remain CI obligations.

Regression comparison, exact-commit validation receipt, release build logs/checksums, independent review, and live results. No fresh physical-phone run is claimed, and #4225 is unchanged in this round.

[thymikee]

At 7e18ad0, the code looks good. The retry-deadline fix from the earlier review is in, so a slot held past the retry budget now surfaces limrun_xctest_slot_occupied, and the regression test covers it. All 17 checks pass and there are no conflicts. No CI lane runs a live Limrun instance, so the forwarded /xctest/run route rests on the fixture tests and your live logs; I did not see a slot held past the deadline live.

Not blocking, take or leave: the custom-hint test at limrun-ios-runner-errors.test.ts:59 calls the helper with an invented hint, so the shipped hint in limrun-ios-runner-products.ts is asserted nowhere; driving an invalid AGENT_DEVICE_LIMRUN_IOS_RUNNER_APP through the products path would cover it. The four new canceled-run cases at limrun-ios-runner.test.ts:201 take about 3 s each, over the 2.5 s slow-test budget, likely from a real wait in the cancel-then-restart setup.

The two open inline threads can be resolved: the packaging overlay one is benign (your Xcode 26.4 release build bundled all four overlays, and Xcode 27.1 ships no Testing.framework), and commands.md now describes the live XCTest app state read.