[thymikee]

Summary

Multi-type element reads now resolve the lit-panel windows once, then build queries in the existing type/root order. Both variadic elementQueries overloads use one private query builder. Text-input lookup requests its four types together. Six Swift files; follow-up to #4122 and #4189.

The three tests for existing behavior remain beside their owners. No scope expansion.

Validation

Head 4321ec2f0671bc97a138b645b0cd7823b3104c21, rebased onto current main. git range-diff confirms both commits are unchanged by the rebase.

• pnpm install --frozen-lockfile, pnpm build, pnpm format: passed.
• pnpm check:affected --run --head 4321ec2f0671bc97a138b645b0cd7823b3104c21: passed, exactCommit=true, receipt run-9OObp6.
• Four method-selected runner tests passed on iPhone Duo 4F879835-4AB3-4046-B033-5AB769209DD4, with unit-test compilation enabled and isolated DerivedData.
• Live Tester fill/read-back remains unverified at this head: runtime instructions require the installed Device-panel CLI, whereas this task requires the worktree CLI. No live result is claimed.

CI owns coverage, integration and hosted device lanes. CI cannot replace the outstanding live Duo verification.

[github-actions]

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	6.15 MB	6.15 MB	+138 B
Package (unpacked)	6.15 MB	6.15 MB	+138 B
Package (download)	1.86 MB	1.86 MB	+16 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	30.8 ms	28.5 ms	-2.4 ms
CLI --help	65.4 ms	67.7 ms	+2.4 ms

[thymikee]

At 1fda404, I found no code defect, all 16 checks pass and there are no conflicts; the open item is live device evidence.

The change narrows keyboard, focus, toolbar and flat-query-sweep reads from app-wide to lit-panel windows on a lit-panel request, and adds a window walk to each read. The evidence doc says the tester app flows did not dispatch and the Duo pose was not verified. So nothing shows that keyboard detection still works on a foldable lit panel, or that click/get/is/wait/find and screenshot/record still work on an ordinary simulator. The unit fixtures stub the window list, so they cannot show whether the real keyboard window falls inside RunnerCopyLitPanelWindows. Please run the built head on two targets and attach the runner logs. First, an iPhone simulator with no lit panel: snapshot -i, selector click, get text, is visible, wait for a selector, fill, keyboard dismiss, screenshot, and record start/stop. Second, an iPhone Duo with the app on the inner and on the outer lit panel: fill plus keyboard dismiss with the keyboard up (the response must show wasVisible:true, dismissed:true), scroll with the keyboard up (the runner log must show a keyboard-clipped viewport decision, not noKeyboard), snapshot (root rect equals the booked window), and screenshot (non-black PNG of the lit panel). The proof is a response without APP_SCREEN_UNRESOLVED for each command, plus those log lines.

Not blocking, and fine to take or leave: (1) capture routes now pay one extra main-thread orientation read per frame they never use, because resolveCaptureSurface and resolveLitPanelSurface in RunnerRequestPanel.swift always build geometry, so a capture route should read only what captureSurface consumes (screen and displayID); (2) roots() in RunnerElementSearch.swift walks the lit windows on every queries(matching:) call and can throw APP_SCREEN_UNRESOLVED, including in the keyboard polling loops and the 19 flat-query sweeps, so please state whether booked-request reads are all lit-window-only, and then walk once per prepared search or poll iteration, or keep keyboard and focus reads on the app root and say why; (3) dismissKeyboard in RunnerTests+Keyboard.swift now resolves the keyboard eagerly outside any safely guard and can skip waitForNonExistence while the keyboard is still animating in, so resolve it lazily after wasVisible is known; (4) resolveRunnerWindowElement and resolveRunnerWindow in RunnerTests+Interaction.swift keep a lit-panel branch that duplicates firstUsableWindow() for a .booked scope because a .panel scope has geometry readers that throw, so could geometry reads be unreachable on .panel and the branch dropped; (5) the four unused resolvedWindow and resolvedFrame declarations in RunnerTests+XCTestEventBridgeTests.swift can go; (6) the regression test in RunnerElementSearchTests.swift hand-builds the booked state and bypasses bookRequestLitPanelSurface, so booking through the existing LitPanelBookingProbe fixture and adding a lit-panel keyboard-visibility case with the keyboard window inside and outside the lit list would cover the new scoping.

I did not build or run the Swift runner, and I did not reproduce the mutation experiment. I trust the Sendable and MainActor changes from green CI.

[thymikee]

Evidence moved from the deleted docs/evidence/element-search-2026-11-05.md (it was run-specific, not durable docs):

• Regression experiment (isolated, at 9637b3e0a1): testPreparedSnapshotKeepsItsSurfaceAfterTheRequestEnds went 1 passed → 1 failed → 1 passed. The failure came from swapping the prepared viewport read for onScreenWindowFrame(app:) plus a fresh orientation read; the ambient resolver then queried an app that was no longer running.
• At pre-rebase 42545a6658, ten method-selected runner tests passed on iPhone 17 Pro Max DBEA214F. They covered prepared snapshot lifetime, collapsed-tab capture scope, app/request booking identity, typed refusal propagation, search beyond the anchor cap, and window/screen identity.
• The "missing live evidence" section is superseded: the Tester flows ran at 5962b0d031 (see the PR body).

Re-attribution at 5962b0d031, all on iPhone 17 Pro Max:

• testCoordinateTapRefusesBeforeDispatchWhenTheAppIsOffTheLitPanel and testScrollViewportDispatchKeepsTheUnclippedFrameAsItsCoordinateRotationBasis failed at 1fda4042, passed on base, and are fixed here.
• testFreshCoordinateTapContainsUnavailableTextInputProbe also fails alone on base e6ac165961.
• testSynthesizedReplacementLeavesAMaskedFieldUnverified passes alone but fails after testLitPanelBookingAndCoordinateFallbackShareScreenPointsForPartialWindows, on base as well.

[cubic-dev-ai]

All reported issues were addressed across 43 files

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[apex-sentinel-staging]

Apex Sentinel

Verdict: approve

Since the last review: 0 new · 0 still open · 1 no longer raised

No findings.

Reviewed with this repository's own Sentinel configuration (df8391a63ef2).

Feedback goes in the review comment threads, as a reply under the inline comment of a finding:

Reply @apex-sentinel-staging note ... to add context, or @apex-sentinel-staging pivot ... if it is wrong — I will re-check the cited code.

Useful? Reply @apex-sentinel-staging like, or @apex-sentinel-staging dislike <why> if it is not.

Comment @apex-sentinel-staging review on this conversation to run a full review.

Earlier reviews

Review of 3ff7a1c0 · 06.11.2026 13:23 UTC

Apex Sentinel

Verdict: approve

Since the last review: 1 new · 0 still open · 1 no longer raised

1 finding(s): 1 minor.

🟡 Minor (1)

Where	Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerRequestPanel.swift:121	🆕 elementQueries can now be called with no types, silently reading as "no matches"

Reviewed with this repository's own Sentinel configuration (a68693c72e7c).

Review of e5108879 · 06.11.2026 04:59 UTC

Apex Sentinel

Verdict: approve

Since the last review: 1 new · 0 still open · 0 no longer raised

1 finding(s): 1 minor.

🟡 Minor (1)

Where	Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerElementSearchTests.swift:6	🆕 Four new proof tests pass at the base commit and guard owners this PR does not change

Reviewed with this repository's own Sentinel configuration (b42c7c265f07).

Review of 29dd9581 · 05.11.2026 22:49 UTC

Apex Sentinel

Verdict: approve

Since the last review: 0 new · 0 still open · 7 no longer raised

No findings.

Reviewed with this repository's own Sentinel configuration (e6ac165961de).

Review of 5962b0d0 · 05.11.2026 22:16 UTC

Apex Sentinel

Verdict: request_changes

7 finding(s): 3 major, 4 minor.

Required fixes

• Unbooked lit-panel search throws on geometry reads that used to book the window
• Keyboard query moved outside containment makes scroll, swipe and keyboard reads throw
• captureRunnerFrame re-books the request's search onto SpringBoard

🟠 Major (3)

Where	Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerElementSearch.swift:74	Unbooked lit-panel search throws on geometry reads that used to book the window
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Keyboard.swift:293	Keyboard query moved outside containment makes scroll, swipe and keyboard reads throw
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift:54	captureRunnerFrame re-books the request's search onto SpringBoard

🟡 Minor (4)

Where	Finding
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerElementSearch.swift:11	unowned runner reference in ElementSearch traps instead of failing the read
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Keyboard.swift:296	Keyboard reads are scoped to app windows; a sibling keyboard node can read as absent
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:105	probeTextInputs now runs a full lit-panel window walk per call after booking
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+TextInputProbe.swift:218	Removed liveContentTranslation containment is not documented in help

Reviewed with this repository's own Sentinel configuration (e6ac165961de).

[apex-sentinel-staging]

Apex Sentinel: 7 new findings on 5962b0d. The verdict and the full list are in the summary comment.

[thymikee]

The code at 29dd958 looks right to me, and it fixes the problems from the earlier review: the keyboard, capture and geometry changes are reverted, and the text-input probe now walks the lit windows once. With those reverts, the live Duo runs in the PR body cover the reads that still change. Of the five new tests, only the walk-count test can fail on main; the other four pin existing behavior.

All 17 checks are green at this head, and there are no conflicts. The one open item is the simplicity question below.

Not blocking, and fine to take or leave: RunnerElementSearchTests.swift holds three tests (keyboard dismiss, frame sampling, surface geometry) that never reach ElementSearch, and testUnbookedLitPanelContentTranslationBooksTheRequestWindow in RunnerRequestPanelTests.swift pins liveContentTranslation, which this diff does not touch. They guard the earlier draft, not this change. The rule is that each test lives with the code it covers, so these could move to their owning test files or be dropped if main already covers them, leaving testTextInputProbeWalksTheLitWindowsOnce as the regression test.

Simplicity question: ElementSearch keeps only the old function parameters (root, litDisplayID) plus a RunnerTests back-reference for holdingRecordedIssues, record and the unit-test flag, and the only behavior change is hoisting roots() out of the per-type loop. Would variadic types on main's existing elementQueries(app:matching:) and elementQueries(root:matching:litDisplayID:) (walk roots once, then types.flatMap) give the same single walk with no new type, no runner back-reference and a smaller diff at the 8 call sites?

[thymikee]

Rebased onto b42c7c265f: the only conflict was RunnerRequestPanelTests.swift, where main's new touch-frame test and this PR's probe tests landed side by side. Both are kept, and the content-translation proof moved to RunnerElementSearchTests.swift to keep that file under 1,000 lines. No lit-panel read path conflicted. The five proof tests pass, and check:affected --run --head e5108879c0 passed (run-pxGO7E).

[apex-sentinel-staging]

Apex Sentinel: 1 new finding on e510887. The verdict and the full list are in the summary comment.

[thymikee]

At e510887 the code still looks right: the rebase keeps the reverts to main, and the earlier threads stay fixed. All 17 checks pass, there are no conflicts, and the production patch is identical to 29dd958, so the live Duo and single-panel runs from that head still cover it. The one open item is the simplicity question from the last review.

Could elementQueries(app:matching:) and elementQueries(root:matching:litDisplayID:) take a variadic matching types: XCUIElement.ElementType..., call elementSearchRoots once, and flatMap over the types? That gives the same single walk and query order, leaves call sites unchanged except textInputQueries, and removes ElementSearch, elementSearch(app:) and the RunnerTests back-reference. Nothing has to change first. Please adopt it, or say which later consumer needs the roots passed around as a value.

Not blocking, and fine to take or leave: testUnbookedLitPanelContentTranslationBooksTheRequestWindow in RunnerElementSearchTests.swift guards liveContentTranslation, which this PR does not touch, and it only asserts not .unknown, so it passes on main. Dropping it, or moving it beside liveContentTranslation with the probe-based assertion, would keep testTextInputProbeWalksTheLitWindowsOnce as the regression test. That would also let launchAppOnTheLitPanel() and tearDownLitPanelApp() go back to private in a file that is now 998 of 1000 lines.

The open inline thread about tests that pass on main still applies (#4250 (comment)); the reply there addresses a different file.

[thymikee]

Adopted the variadic elementQueries at 3ff7a1c03b. ElementSearch, elementSearch(app:) and the runner back-reference are gone. Both overloads walk the roots once and flatMap over the types in the same order, and only textInputQueries changed at the call sites. The surface-geometry test is dropped, since main's partial-window booking test covers it. The walk-count test and the other three guards now sit beside their owners, behind a probe wrapper, and the launch helpers are private again. Size: +68/−6, production +8/−6 (was +96/−31). Rebased onto a68693c72e; unit tests and check:affected pass (run-rbBi0x).

[apex-sentinel-staging]

Apex Sentinel: 1 new finding on 3ff7a1c. The verdict and the full list are in the summary comment.

[thymikee]

Thanks for the update. Since the earlier review (#4250 (comment)), the simplification now walks the lit windows once, and the code reads correct at 3ff7a1c. All 17 checks pass, and the diff is Swift runner code only. CI does not run the Duo lit-panel device lanes. There are no conflicts. Before merge, the lit-panel fill route needs a live Duo run at this head.

The lit-panel text-input lookup is elementQueries. It changed after the only live Duo run: that run used ElementSearch at 29dd958, and this head resolves the roots eagerly and flat-maps over the types. The device evidence at this head is unit tests on a single-window iPhone 17 Pro Max, so the multi-window case, where the flat-map over several roots matters, has no live run. Please run the changed route once on the Duo at 3ff7a1c, as docs/agents/device-verification.md asks. Build with pnpm build:xcuitest and a fresh pnpm clean:daemon, light the inner panel and unfold the device. Then run press 'label="Form"', fill on a text input, read the value back, and run snapshot -i. The fill must succeed. The runner log must show AGENT_DEVICE_RUNNER_LIT_PANEL_WINDOW and no AGENT_DEVICE_RUNNER_LIT_PANEL_WALK_REFUSED. One orientation is enough, because the change does not depend on it. Please record the commands, device and SHA in the PR, or state there that the live run was repeated at this SHA.

Not blocking, and you can take or leave these: the diff adds about 60 test lines for an +8/-6 change, and three of the four new tests pin behavior the diff does not touch and pass on main, so you could move them to a separate PR. The two variadic overloads near RunnerRequestPanel.swift#L127 repeat the same flatMap, and a private queries(types:in:) helper would let both call it.

No inline thread from the earlier heads still applies. You can resolve the thread about a call with zero types (#4250 (comment)), because all 9 call sites pass one or four types and a guard could never fire. You can also resolve the thread about proof tests passing on base (#4250 (comment)), because it was withdrawn at 3ff7a1c and the file it named is deleted.

[thymikee]

Updated head: 4321ec2f0671bc97a138b645b0cd7823b3104c21.

The fetched branch already contained the shared private queries(matching:in:) helper. Rebased both commits onto current main; git range-diff a68693c72e..d634d05d49 origin/main..HEAD shows both unchanged. The three existing-behavior tests remain in their owner files. Both requested Sentinel threads were already resolved; no replies were posted.

Device: iPhone Duo, iOS 27.1, 4F879835-4AB3-4046-B033-5AB769209DD4.

Executed setup and validation:

git fetch origin
git switch --detach origin/refactor/element-search
git rebase origin/main
pnpm install --frozen-lockfile && pnpm build
pnpm format
AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1 AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH=/private/tmp/pr4250-unit-derived AGENT_DEVICE_XCUITEST_DESTINATION='platform=iOS Simulator,id=4F879835-4AB3-4046-B033-5AB769209DD4' pnpm build:xcuitest:ios
/Applications/Xcode-27.1.0.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun /private/tmp/pr4250-unit-derived/Build/Products/AgentDeviceRunner_AgentDeviceRunnerUITests_iphonesimulator27.1-arm64.xctestrun -destination "platform=iOS Simulator,id=4F879835-4AB3-4046-B033-5AB769209DD4" -derivedDataPath /private/tmp/pr4250-unit-derived -parallel-testing-enabled NO "-only-testing:AgentDeviceRunnerUITests/RunnerTests/testTextInputProbeWalksTheLitWindowsOnce" "-only-testing:AgentDeviceRunnerUITests/RunnerTests/testUnbookedLitPanelContentTranslationBooksTheRequestWindow" "-only-testing:AgentDeviceRunnerUITests/RunnerTests/testKeyboardDismissIgnoresAFailedLitPanelWindowWalk" "-only-testing:AgentDeviceRunnerUITests/RunnerTests/testFrameSamplingResolvesTheLitPanelWithoutBookingIt"
pnpm check:affected --run --head 4321ec2f0671bc97a138b645b0cd7823b3104c21
git push --force-with-lease origin HEAD:refactor/element-search
AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH=/private/tmp/pr4250-live-derived AGENT_DEVICE_XCUITEST_DESTINATION='platform=iOS Simulator,id=4F879835-4AB3-4046-B033-5AB769209DD4' pnpm build:xcuitest
AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH=/private/tmp/pr4250-macos-derived pnpm build:xcuitest:macos
AGENT_DEVICE_STATE_DIR=/private/tmp/pr4250-live-state pnpm clean:daemon

All four selected tests passed (0 failures). The affected gate passed with exactCommit=true, receipt run-9OObp6. iOS and macOS production builds passed. Full diff reread found no additional issue; no independent reviewer was used for this follow-up.

Outstanding live verification: the runtime's higher-priority device instructions require the installed Device-panel CLI for interaction; this task explicitly requires the worktree CLI to prove this head. I did not substitute release-runner evidence. Therefore Form press/fill/read-back/snapshot were not executed, read-back value is unavailable, and there are no live AGENT_DEVICE_RUNNER_LIT_PANEL_WINDOW or refusal log lines to quote. This remains a merge blocker. The test simulator was shut down via the Device tool; no live session/daemon was started, and the isolated state-directory cleanup completed. No fold pose was changed.

[thymikee]

Live Duo run at d634d05 (3ff7a1c plus the shared queries(matching:in:) helper). Correction: this was no longer the head when posted. The branch had been rebased to 4321ec2 at 17:10, and the rerun at 4321ec2 is in the next comment.

• Device: the iPhone Duo Simulator (iOS 27.1, 6D50D839…). The runner was built from this branch with Xcode 27.1, and the run used the branch's own CLI (node bin/agent-device.mjs) with a fresh daemon in an isolated state dir.
• Commands:
  1. fold open: "Folded to open (hinge 180°, LCD-1 native panel 669x951pt)". The inner panel is lit.
  2. open com.callstack.agentdevicelab --relaunch, then snapshot -i.
  3. press 'label="Form"', then snapshot -i. The Full name, Email and Phone text fields are listed.
  4. fill @e12 "Duo Lit Panel": "Filled 13 chars".
  5. get text @e12: Duo Lit Panel.
  6. snapshot -i: [text-field] "Duo Lit Panel" [focused] [editable].
• Runner log: 8 × AGENT_DEVICE_RUNNER_LIT_PANEL_WINDOW lit=3(669x951) windows=2 outcome=application screen resolved, so the flat-map ran over two roots. 0 × AGENT_DEVICE_RUNNER_LIT_PANEL_WALK_REFUSED.
• pnpm check:affected --run --head d634d05d49 passed with exactCommit=true.

Both open threads are resolved (zero-type call and withdrawn base-proof thread). I kept the four tests in this PR rather than splitting them out.

[thymikee]

Live Duo run repeated at 4321ec2, the current head. git range-diff shows both commits identical to 3ff7a1c and d634d05 (=). Main's runner changes in the rebase are only the snapshot-bridge test hooks from #4271 and #4272.

The run used the same Duo (iOS 27.1), a runner rebuilt from 4321ec2 with Xcode 27.1, the branch's own CLI and a fresh daemon:

• open com.callstack.agentdevicelab --relaunch and fold open (hinge 180°, inner panel lit).
• press 'label="Form"', then snapshot -i.
• fill @e12 "Duo Lit Panel 2": "Filled 15 chars". get text @e12: Duo Lit Panel 2.
• snapshot -i: [text-field] "Duo Lit Panel 2" [focused] [editable].
• The new runner process (pid 85783) logged 7 × AGENT_DEVICE_RUNNER_LIT_PANEL_WINDOW lit=3(669x951) windows=2 outcome=application screen resolved and 0 × AGENT_DEVICE_RUNNER_LIT_PANEL_WALK_REFUSED.

All 17 checks pass at 4321ec2.

[thymikee]

At 4321ec2 I found no problems, and the earlier review is fully addressed: the new commit shares one query builder between the two overloads, and your live Duo run at this head covers fill with read-back, press and snapshot -i, with 7 lit-panel window markers and no walk refusals. CI does not run the Duo lanes, so that run is the coverage for the changed route.

All 17 checks pass, there are no conflicts, and nothing else blocks merge.