Skip to content

perf(ios): decide Simulator AX bridge viability (GO, Node-direct guest reader) - #2237

Merged
thymikee merged 13 commits into
mainfrom
codex/2192-guest-bridge-evidence
Sep 4, 2026
Merged

perf(ios): decide Simulator AX bridge viability (GO, Node-direct guest reader)#2237
thymikee merged 13 commits into
mainfrom
codex/2192-guest-bridge-evidence

Conversation

@thymikee

@thymikee thymikee commented Sep 2, 2026

Copy link
Copy Markdown
Member

Summary

Concludes the Simulator accessibility bridge decision for #2192 with a GO for the narrow Node-direct route.

  • drives the pinned idb v1.5.2 SimulatorFrameworkBridge directly over its private UNIX socket; the supplied executable is SHA-256 verified before launch, and no Python, idb_companion, or gRPC client remains
  • pairs every timed relaunch read with an independent readiness observation for the exact app PID generation and expected screen anchor: 20 samples on each of 6 representative screens
  • splits bridge process/connection/resource ownership from request/response adaptation (290 and 237 lines)
  • reports Simulator preference apply/restore evidence and the private-interface compatibility risk; production routing is unchanged

The readable result stays in-tree at docs/evidence/ios-simulator-ax-bridge-2026-09-02-corrected.md. Raw artifacts are held on evidence/ios-snapshot at immutable tag evidence/ios-snapshot/44995806ea, evidence commit f8b2fab28b8604f20094785c16e16a79fdc651a3, with hashes in that branch's README.

The retained 3,645-line tooling diff is: 1,941 lines of private wire decoding, validation, bounds, adaptation, process ownership, and their tests; 966 lines of fail-closed decision/report generation and tests; 733 lines of live bootstrap/relaunch/recovery orchestration and instructions; 5 lines of repository wiring. A smaller wrapper would not independently validate the unsupported wire shape, target generation, lifecycle recovery, or resource bounds that make the decision auditable.

Validation

  • rebased onto origin/main 941ca0e7e08f50960dc90d3fdafa572355b030a0
  • pnpm check:affected --run — all runnable checks passed
  • focused verifier suite — 18 tests passed; planted generation/hash violations observed red
  • live post-rebase run from clean commit 44995806ea3be09f3c48ceac50ac3cab18462c35 on dedicated iOS 26.2 Simulator ad-2237-axbridge
  • relaunch: 120/120, 6/6 screens, 0 generation mismatches, worst per-screen p95 234.7 ms (<500 ms)
  • bootstrap: 5/5, p95 1136.6 ms (<2000 ms); resources: 129/129 measured, max CPU 220 ms, max RSS 89,407,488 bytes
  • recovery 4/4; hierarchy depth 29; preference apply/restore passed; Simulator shut down with no bridge process remaining

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-04 05:38 UTC

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 2.60 MB 2.60 MB 0 B
JS gzip 875.8 kB 875.8 kB 0 B
npm bundled raw 2.60 MB 2.60 MB 0 B
npm bundled gzip 875.8 kB 875.8 kB 0 B
npm tarball 1.03 MB 1.03 MB +31 B
npm unpacked 3.48 MB 3.48 MB +115 B
npm clean-installed 3.48 MB 3.48 MB +115 B

npm unpacked components

Component Base Current Diff
JS / dist source 2.76 MB 2.76 MB 0 B
Apple runner source/project 544.4 kB 544.4 kB 0 B
Apple snapshot presentation source 33.8 kB 33.8 kB 0 B
macOS helper source 54.8 kB 54.8 kB 0 B
Android helper artifacts 43.3 kB 43.3 kB 0 B
Other package files 46.5 kB 46.6 kB +115 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 30.5 ms 31.9 ms +1.4 ms
CLI --help 79.2 ms 78.6 ms -0.6 ms

Top changed chunks: no changes in the largest emitted chunks.

Top changed packed files

Packed file Base Current Diff
package.json 20.1 kB 20.2 kB +115 B

@thymikee

thymikee commented Sep 2, 2026

Copy link
Copy Markdown
Member Author

BLOCKED — the corrected decision artifact is not yet reviewable.

  1. [P1] Fix generation-readiness accounting before treating relaunch as a hard-gate pass. scripts/ios-ax-bridge-spike/corrected-report.ts collects all non-null generations across a cell, then sets readinessObservedSamples to the full readable sample count whenever that collection is merely non-empty. In the checked-in broad artifact, every warm and relaunch cell has generation evidence for only 1/20 samples, but the generated report says ready=20/20 and PASS. Count readiness per sample, add a mixed null/non-null regression case, and regenerate the report. Until then the corrected relaunch verdict and perf(ios): decide Simulator AX bridge viability with a bounded spike #2192 closure are unsupported.

  2. [P1] Re-capture decisive targeted evidence from a clean, reviewable revision. ios-simulator-ax-bridge-2026-09-02-targeted.json.gz records revision eac2c7… with dirty: true, while the readable report masks that uncertainty. Re-run the narrow bootstrap/recovery probes from a clean commit and regenerate the targeted/corrected artifacts, with an assertion that the evidence revision is clean and tied to the reviewed source. The current NO-GO may be directionally correct, but it is not reproducible from the cited revision.

  3. [P2] Account for or prune the retained NO-GO harness. This PR adds 6,098 lines across 53 files, roughly 5k non-test/non-doc lines of one-off spike machinery, but the body neither itemizes the justified permanent growth nor explains why a smaller retained artifact (raw evidence plus a narrow verifier/reproduction path) was rejected. Keep only the decision evidence and reusable owning tooling, or document the concrete long-lived owner and why the materially smaller design is insufficient.

Production routing remains unchanged and the acquisition-only ADR direction is otherwise aligned. Do not apply ready-for-human; #2192 should remain open pending corrected exact-head evidence.

@thymikee thymikee changed the title perf(ios): decide Simulator AX bridge viability perf(ios): decide Simulator AX bridge viability (GO, Node-direct guest reader) Sep 3, 2026
@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

BLOCKED. The branch conflicts with main and its own Compatibility & Provenance gate fails with Fallow dead-code/clone/high-complexity findings. More importantly, the GO decision lacks the required bounded-resource evidence for the selected Node-direct nested-tree route: targeted samples record cpuMs:null/memoryBytes:null, while older resource samples use the legacy-flat companion+Python route. The checked-in live artifact is also from 999920… with dirty:true, not this exact head. Rebase, resolve Fallow, record CPU/memory bounds for the selected route at a committed exact head, and rerun all gates. iOS Smoke also has the shared truncated absence failure.

@thymikee
thymikee force-pushed the codex/2192-guest-bridge-evidence branch from 62fd102 to 5f25c29 Compare September 3, 2026 18:44
@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Resolved the outstanding review blockers on exact head 5f25c29b5dd6365acabe852d828ea997c5ec242a:

  • rebased onto current origin/main (7bf8d8c4a34c2cee09d407902af1d4591ae02d71); conflicts are gone
  • removed the September 1 Python/idb companion experiment, broad one-off harness, superseded reports, and unrelated benchmark edits
  • retained one narrow Node-direct UNIX-socket verifier and corrected the broad corpus readiness accounting (1/20, not 20/20)
  • added fail-closed CPU/RSS measurement for every successful targeted read; clean live result was 9/9 measured, max 220 ms CPU and 84,787,200 bytes RSS
  • captured clean exact-source live evidence at 268a90275e7a30419e581336b6d85eff680a2eb6; raw artifacts moved to immutable tag evidence/ios-snapshot/268a90275 / evidence commit fdc52f65ed679f1420f91312204f8d558a8c0061
  • final PR diff is 22 files / 3,127 additions with no raw binaries; Fallow reports no issues
  • all exact-head GitHub checks are green, including Compatibility & Provenance, Coverage, Repo Guards, and iOS/Android/macOS/Linux smoke

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

BLOCKED at exact head 5f25c29. The rebase, Fallow/CI, clean tagged evidence, CPU/RSS, bootstrap, recovery, hierarchy, and artifact hashes are now sound, but three blockers remain.

  1. The relaunch hard gate still does not prove per-screen post-readiness latency. corrected-report.ts combines six-screen latency from the legacy broad route with five targeted list-screen bootstrap/readiness samples; latencyGate ignores readinessObservedSamples, and each retained relaunch cell identifies the generation for only 1/20 samples. Pair every relaunch latency sample with the independently observed expected generation through the selected Node-direct route (or rerun that route across the representative corpus), and fail the gate when per-cell evidence is absent.

  2. guest-adapter.ts is 519 lines; AGENTS.md requires extraction before adding behavior past 500. Split process/connection/resource ownership from request-response adaptation. Also itemize the retained ~3,127-line tooling growth and why a materially smaller reproducible verifier is insufficient.

  3. perf(ios): decide Simulator AX bridge viability with a bounded spike #2192 requires the readable decision report to include the Simulator preference control and private-interface compatibility risk. The raw artifact contains the applied/restored AutomationEnabled/IgnoreAXServerEntitlements plist diff, but the checked-in report omits it and does not evaluate reliance on idb’s private guest interface. Restore that decision-risk section.

No ready-for-human label until these are fixed and exact-head CI reruns.

@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

One additional evidence-integrity requirement: the targeted harness executes the supplied --guest-bridge path but never hashes that file. GUEST_MECHANISM_EVIDENCE reports hardcoded expected hashes, so an arbitrary binary could run while the artifact claims the official idb guest hash. Hash the actual supplied binary before execution, fail closed on mismatch, and record the observed hash in the raw artifact.

Replace the idb companion + Python reader in the #2192 spike with a Node client
for idb v1.5.2's in-Simulator SimulatorFrameworkBridge: one private guest per
session spawned through simctl, 4-byte length-prefixed JSON over a UNIX socket,
single-fetch traversal with automation mode asserted per request, nested trees
flattened to parent-linked raw nodes with XCTest type names, and typed
crash/timeout/cancel/stale-generation failures.

The targeted harness now observes app readiness with a throwaway probe instead
of admitting on pid presence, relaunches the app per bootstrap sample, records
host load per sample, and runs recovery probes through the adapter. Hard tiers
follow the corrected #2192 contract (warm 300/500 ms, relaunch 500 ms); the
former 75/150 ms and 250 ms values are reported as stretch findings. Preboot
preference edits are optional and unused by the guest path.

The prototype's targeted artifact is preserved under a -python-prototype name;
its bootstrap and recovery samples measured the packaging, not the mechanism.
@thymikee
thymikee force-pushed the codex/2192-guest-bridge-evidence branch from 5f25c29 to f93beaf Compare September 3, 2026 20:44
@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

Addressed the latest review round on the rebased head f93beafe2d:

  • Relaunch gate: replaced the mixed legacy/targeted verdict with 120 Node-direct samples (20 × 6 screens). Every timed read is paired with an independent probe for the exact relaunched PID generation and expected screen anchor; missing/mismatched samples fail closed. Post-rebase result: 120/120, 0 generation mismatches, worst screen p95 234.7 ms.
  • Adapter size/ownership: split process, socket, frame, teardown, and resource ownership into guest-connection.ts (290 lines); guest-adapter.ts is now request sequencing and response adaptation (237 lines). Fallow passes.
  • Preference/private API risk: the readable report now includes the applied/restored Simulator preferences, fixture compatibility, and the pinned-version/reverification control for the private idb/Apple interface.
  • Binary provenance: the verifier hashes the actual --guest-bridge bytes and aborts on mismatch; the raw/report evidence records both expected and observed SHA-256.
  • Tooling size: the PR description now itemizes all 3,645 retained lines and why each evidence layer is required.

Raw post-rebase artifacts are off-tree at immutable tag evidence/ios-snapshot/44995806ea (evidence commit f8b2fab28b8604f20094785c16e16a79fdc651a3). pnpm check:affected --run passes, and the dedicated Simulator was shut down with no guest bridge process remaining.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 3, 2026
@thymikee

thymikee commented Sep 3, 2026

Copy link
Copy Markdown
Member Author

READY at exact head f93beafe2df22df38bb5667b920738c18a04c399. All prior blockers are resolved. The 120-sample Node-direct relaunch corpus covers 20 launches on each of six screens with exact PID-generation and anchor admission; the raw artifact has 0 generation/anchor mismatches and complete resource metrics. The guest binary is hashed from the supplied file and fails closed against the pinned SHA. The adapter/process split stays below the repository extraction threshold, the readable report now includes Simulator preference control and private-interface risk, and the PR itemizes and justifies its 3,645-line evidence-harness growth. Recomputed evidence hashes match the annotated tag, the clean source revision is an ancestor of this head, all exact-head CI is green, and cleanup leaves the dedicated Simulator shut down with no bridge process. Production routing remains unchanged.

@thymikee
thymikee merged commit 33084c7 into main Sep 4, 2026
21 of 23 checks passed
@thymikee
thymikee deleted the codex/2192-guest-bridge-evidence branch September 4, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant