diff --git a/.gitattributes b/.gitattributes index e6f0ff344..863377d73 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,3 +3,7 @@ graphify-out/**/manifest.json filter=lfs diff=lfs merge=lfs -text graphify-out/**/*.graphml filter=lfs diff=lfs merge=lfs -text graphify-out/**/*.svg filter=lfs diff=lfs merge=lfs -text graphify-out/**/graph.html filter=lfs diff=lfs merge=lfs -text + +# Phase 66 benchmark fixtures are hashed byte for byte by scripts/bench/tasks/core.lock.json; +# never convert their line endings on checkout. +scripts/bench/fixtures/** -text diff --git a/.gitignore b/.gitignore index 979ea5769..2c8a84736 100644 --- a/.gitignore +++ b/.gitignore @@ -47,7 +47,7 @@ lattice/ Logs/ Extenive logs, multiple scenariaos.txt showcase/server/fsb-data.db* -Research/ +/Research/ # Quick task 260630-hct -- self-hosted DB-IP IP-to-City Lite region dataset. # The real ~100MB production artifact is generated by @@ -78,3 +78,6 @@ graphify-out/.graphify_uncached.txt # local secrets directory .secrets/ + +# Phase 66 -- raw benchmark captures (may contain page content; D-21) +scripts/bench/.local/ diff --git a/.planning/MILESTONES.md b/.planning/MILESTONES.md index 595292bfa..71bd869d5 100644 --- a/.planning/MILESTONES.md +++ b/.planning/MILESTONES.md @@ -1,5 +1,28 @@ # Project Milestones: FSB (Full Self-Browsing) +## v0.9.91 MCP Clients as Providers (Shipped: 2026-09-28) + +**Phases completed:** 9 phases (57-65), 61 plans, 157 tasks + +**Delivered:** Installed agent CLIs (Claude Code, OpenCode, Codex) became key-less side-panel providers that drive the live browser back through FSB's own MCP tools. + +**Key accomplishments:** + +- Agent identity capture: MCP `initialize` clientInfo plus a 21-client installed sweep became durable clicked/connected/installed evidence behind one guarded `getMcpClients` view (Phase 57) +- Providers panel: "API Configuration" became "Providers" with explicit `api` vs `agent` kinds, one evidence-based recommendation, and BYOK settings preserved (Phase 58) +- Reverse-request channel: authenticated loopback pairing with a rotating session secret, exact Host/Origin checks, secret redaction, and a permanent CI gate against forbidden agent flags (Phase 59) +- Supervised adapters: one five-method adapter contract and spawn supervisor running Claude Code 2.1.177, OpenCode 1.14.25 and Codex with pinned profiles, recorded fixtures and CI drift checks (Phases 60, 62, 64, 65) +- Delegation UX: consent, live streaming feed, background-tab ownership, Take Control, Stop/reclaim, a 20-second heartbeat and service-worker-eviction recovery (Phase 61) +- Native wake host: an optional native-messaging host that only starts or attaches to `serve` and never spawns agent CLIs (Phase 63) + +**Human UAT:** All 48 pending live scenarios in the Phase 58-65 human-UAT ledgers, and the matching human verification items, were closed at the user's direction on 2026-09-28 as waived, not executed. The archived ledgers keep their original pending entries as the record. + +**Known deferred items at close:** 117 acknowledged (116 quick-task audit entries and 1 diagnosis-only debug session; see STATE.md Deferred Items) + +**Archive:** `.planning/milestones/v0.9.91-ROADMAP.md`, `.planning/milestones/v0.9.91-REQUIREMENTS.md`, `.planning/milestones/v0.9.91-phases/` + +--- + ## v1.1.0 T1 App Execution Expansion (Shipped: 2026-06-30) **Phases completed:** 8 phases (44-51), 29 plans, 17/17 requirements satisfied. @@ -25,11 +48,13 @@ --- -## v1.0.0 Full App Catalog (OpenTabs Parity) (Shipped: 2026-06-29) +## v0.13.0 Full App Catalog (OpenTabs Parity) (Shipped: 2026-06-29) + +*Relabeled from v1.0.0 on 2026-09-28 so that v1.0.0 names the Jev Fast Mode milestone that ships extension 1.0.0. Archive files and references were renamed to match.* **Phases completed:** 10 phases (35-43, including inserted 39.5), 46 plans, 17/17 requirements satisfied. -**Archive:** `.planning/milestones/v1.0.0-ROADMAP.md`, `.planning/milestones/v1.0.0-REQUIREMENTS.md`, `.planning/milestones/v1.0.0-MILESTONE-AUDIT.md`, and `.planning/milestones/v1.0.0-phases/`. +**Archive:** `.planning/milestones/v0.13.0-ROADMAP.md`, `.planning/milestones/v0.13.0-REQUIREMENTS.md`, `.planning/milestones/v0.13.0-MILESTONE-AUDIT.md`, and `.planning/milestones/v0.13.0-phases/`. **Audit status:** `passed` -- automated milestone gate met. Non-blocking debt carried forward: live guarded-write UAT, live discovery first-visit UAT, Pattern-D cross-origin execution, and T1 expansion for the DOM/discovery tail. diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md index 472d7270d..18a22c954 100644 --- a/.planning/PROJECT.md +++ b/.planning/PROJECT.md @@ -10,9 +10,9 @@ FSB is an AI-powered browser automation Chrome extension that executes tasks thr ## Current State -**Active milestone progress:** Phases 57-64 are complete (48/51 requirements). Phase 64 verified 4/4 roadmap success criteria, 66/66 plan truths, 36/36 key links, MULTI-01..03, and all ten registered HIGH/CRITICAL threats; its guarded 25-command matrix and repository test process are green after a clean code-review fix loop. Three genuine OpenCode account/process/browser/accessibility scenarios join the prior 42 in the user-directed milestone-end UAT sweep. Phase 65 (Codex Adapter) is the final implementation phase in v0.9.91; backlog Phase 999.1 remains outside this milestone. +**Active milestone:** v1.0.0 Jev Fast Mode — 25 requirements mapped to Phases 66-71 plus inserted Phase 66.1; Phase 66 context is captured and ready for planning. -**Last completed:** v1.2.0 Showcase i18n Completeness — Phases 52-56 shipped 2026-07-09. Full-page translation audit, 5-id resync + stats-274 retirement + hero/CTA transcreation, stats lint gate flip, permanent `verify-translation-drift` CI gate, and WARNING-02 locale-cookie redirect fix. VISUAL-01 browser UAT remains human_needed (`53-VISUAL-QA.md`). +**Last completed:** v0.9.91 MCP Clients as Providers — Phases 57-65 shipped 2026-09-28. Installed agent CLIs (Claude Code, OpenCode, Codex) became key-less side-panel providers that drive the live browser back through FSB's own MCP tools; 51/51 requirements complete with green automated/source verification. The 48 deferred live UAT scenarios were closed at the user's direction as waived, not executed. Backlog Phase 999.1 remains outside the milestone. > **Post-v1.2 supersession (2026-07-15):** The showcase localization follow-up > localized the dashboard and removed its `lint:i18n` exclusion. The v1.2.0 @@ -20,12 +20,42 @@ FSB is an AI-powered browser automation Chrome extension that executes tasks thr > `showcase/angular/src/locale/I18N-BOUNDARIES.md`. **Recent shipping cadence:** +- v0.9.91 MCP Clients as Providers -- archived 2026-09-28; all deferred human UAT waived by the user - v1.2.0 Showcase i18n Completeness -- archived 2026-07-09 - v1.1.0 T1 App Execution Expansion -- archived 2026-06-30; remaining tail rows carry explicit proof requirements before direct execution -- v1.0.0 Full App Catalog (OpenTabs Parity) -- archived 2026-06-29; T1 expansion debt carried into v1.1.0 +- v0.13.0 Full App Catalog (OpenTabs Parity) -- archived 2026-06-29; T1 expansion debt carried into v1.1.0 -## Current Milestone: v0.9.91 MCP Clients as Providers +## Current Milestone: v1.0.0 Jev Fast Mode + +**Goal:** Make FSB measurably faster in both autopilot and MCP by letting Jev (TypeSafe AI's System One decision model) make the simple step decisions, while the LLM keeps planning, writing text and reading pages, MCP changes stay minimal, and the extension ships as 1.0.0. + +**Target features:** +- **Speed baseline and measurement** -- per-call timing recorded for MCP and autopilot sessions, a working session-detail lookup for journal-backed MCP sessions, and a repeatable benchmark suite comparing wall time per completed task, success rate and cost before and after. +- **FSB-hosted Jev decision service** -- FSB keeps its upstream Jev credential on `showcase/server/` in server-side secret storage. The MV3 service worker sends bounded, redacted decision requests over HTTPS to the FSB gateway; the credential is never sent to the extension or entered by users. Every install may use the service without sign-in or a usage quota. The server monitors aggregate cost, has an operator-wide disable switch, and the extension falls back cleanly when Jev is unavailable. +- **Autopilot fast mode (opt-in)** -- the only fast-mode setting lives in control-panel Advanced Settings and defaults off. The MCP settings tool can also read and write it. Each simple step, Jev picks the operation and target from FSB's element snapshot and checks whether the task is done or stuck; the LLM handles the plan, typed text, extraction and any step below the confidence threshold. +- **MCP speedups with minimal wire change** -- shorter descriptions for all MCP tools (72 at `8a058bdf`, 73 with the settings tool); only `expect` is added as an optional action-tool parameter (outcome checked in the same call). Existing `run_task` follows the Advanced Settings switch and has a bounded fast path without a separate fast-mode argument. +- **MCP settings tool (Phase 66.1, approved 2026-09-29)** -- one new MCP-only tool that shows every control-panel setting compactly and changes one setting by key, including the Jev switch once it exists. Secrets are masked on read and refused on write; every other setting is writable. The autopilot tool list does not change. +- **Safety** -- Jev can only add confirmations, never remove them; page text is treated as untrusted; Jev is never the sole safety gate. +- **Release** -- extension 1.0.0 via a `CHANGELOG.md` v1.0.0 entry and `npm run version:set:extension -- 1.0.0`; MCP gets a minor bump for the additive parameters (0.11.0 to 0.12.0). + +**Key context:** +- Measured baseline (40 recorded MCP sessions, 2026-09-16 to 2026-09-21; the sessions themselves were lost in a 2026-10-09 extension reinstall): median wall time per tool call is 9.5 s with Claude and 4.6 s with Cursor. FSB's own execution is roughly 0.3-1 s per call (a click waits for 300 ms of DOM stability plus 200 ms of network quiet, capped at 3 s), so the calling agent's turns are about 85-90% of MCP time. Autopilot defaults to xAI `grok-4-1-fast`; no recorded autopilot sessions exist to measure. +- MCP tool surface today: 72 tools at `8a058bdf` with 54,024 characters of descriptions (median 730, longest 2,187 for `execute_js`) plus 34,217 characters of parameter descriptions; no MCP `annotations` are defined. +- Jev facts (`.planning/research/JEV-REFERENCE.md`): `POST /v1/systemone` with a `state` plus typed `questions` (Choice up to 255 options, Score 2-10 levels, Noul yes/no probability); about 100 ms server time; $0.042 per million input tokens, output free; 32k tokens for the state plus the longest question; 1,200 requests/min; no SLA. Upstream access and model selection are FSB server responsibilities, not user configuration. +- The user clarified that FSB provides Jev and owns the credential. The reference's BYOK recommendation is superseded. Hosted Jev is planned for every install with no sign-in or user usage quota for now. `showcase/server/src/routes/auth.js` currently allows anyone to create a hash key, so that key is an attribution signal rather than protection for the FSB-owned credential. Phase 67 must bound request shape and concurrent work, monitor aggregate cost, and provide an operator-wide disable switch. Server round-trip time belongs in the whole-task benchmark. +- Evidence to respect: per-decision speed and cost wins are real, but whole-agent tests mostly cut cost without raising task success; a third-party benchmark scored Jev 25/49 on raw DOM controls; confidence ranks answers well, but thresholds must be tuned on FSB's own labelled tasks against a pinned model version. +- Realistic targets: autopilot about 1.5-2x faster on mixed tasks; MCP about 1.3-1.6x on UAT/Jira-style sessions and 2-3x on click-heavy flows; MCP tool-list tokens cut by more than half. +- INV-01 carries forward: MCP wire contracts stay byte-stable; only additive optional parameters, plus the one approved settings tool. +- With Jev disabled, FSB behaves exactly as it did before this milestone (owner direction, 2026-09-29). +- xAI retired `grok-4-1-fast` on 2026-08-15. That name is now served by grok-4.3 at $1.25 / $2.50 per million tokens, while FSB's price table still has $0.20 / $0.50. The Phase 66 benchmark pins an explicit current model. The product default is a separate fix. +- The version bump goes through `scripts/sync-product-version.mjs`, which refuses to run until the `## v1.0.0` CHANGELOG entry exists. +- Tests read archived planning files, and source-pin tripwires on extension files require paired test updates in the same commit. +- Phases continue from 66. The milestone label v1.0.0 matches the extension release; the former v1.0.0 catalog milestone is now v0.13.0. + +## Last Milestone: v0.9.91 MCP Clients as Providers + +**Status:** Archived 2026-09-28. Phases 57-65 shipped; 51/51 requirements complete. The 48 deferred live UAT scenarios were closed at the user's direction as waived, not executed. Archive files live under `.planning/milestones/v0.9.91-*`. **Goal:** Make installed agent CLIs (Claude Code first) first-class side-panel providers -- FSB captures which MCP clients the user installs/connects, presents them as key-less providers in a renamed Providers panel, and delegates side-panel tasks to a spawned agent CLI that drives the browser back through FSB's own MCP tools. @@ -38,7 +68,7 @@ FSB is an AI-powered browser automation Chrome extension that executes tasks thr - **Drift gate + compatibility doctor (Phase 62 complete)** -- one daemon-owned compatibility matrix feeds offline production-parser CI, doctor text/JSON, authenticated durable browser projection, protocol-drift diagnostics, and non-mutating Providers compatibility states. Three genuine installed/rendered/accessibility checks remain pending. - **Native wake path (Phase 63 complete)** -- the optional native-messaging host wakes or attaches only to `serve` through exact one-shot framing, exact-owned cross-platform registration, read-only diagnostics, and a background-authoritative wake UI with doctor fallback; it never spawns agent CLIs, and the Phase 59 channel-gate files are byte-identical to the phase base. Eight genuine install/Chrome/accessibility checks remain pending. - **OpenCode adapter (Phase 64 complete)** -- the unchanged five-method `AgentProviderAdapter` now drives both cold `opencode run` tasks and verified attaches to an FSB-owned `opencode serve`; exact 1.14.25 private policy, schema-derived JSONL fixture, first-commit drift coverage, transient server authentication, role-aware crash recovery, provider-neutral browser persistence, and honest unknown billing are all verified. Three genuine live scenarios remain pending. -- **Codex adapter (Phase 65 next)** -- complete the stable adapter roster with hermetic `codex exec --json`, source-pinned fixture/drift coverage, and honest ChatGPT OAuth versus API-key versus unauthenticated disclosure. Gemini remains explicitly deferred until a live help capture and fixture pin exist. Task-mode only for v0.9.91. +- **Codex adapter (Phase 65 complete)** -- complete the stable adapter roster with hermetic `codex exec --json`, source-pinned fixture/drift coverage, and honest ChatGPT OAuth versus API-key versus unauthenticated disclosure. Gemini remains explicitly deferred until a live help capture and fixture pin exist. Task-mode only for v0.9.91. **Key context:** - The spawn channel is security-critical (RCE-adjacent): extension-origin gating + shared secret + explicit consent tiers required. Bridge already rejects untrusted browser origins (`tests/mcp-bridge-topology.test.js`). @@ -63,7 +93,7 @@ FSB is an AI-powered browser automation Chrome extension that executes tasks thr - Fix WARNING-02 (carried since v0.9.63): picker-set `fsb-locale` cookie no longer short-circuits the bare-`/` Accept-Language redirect for returning fresh-tab/shared-link visitors. - New CI drift-detection gate: fail the build if `messages.xlf` source content changes without a corresponding update to all 5 translated locale files. -**Key context:** Supported locales are fixed (en source + es/de/ja/zh-CN/zh-TW) -- not up for debate, carried over from v0.9.63's `LocaleService` + locale-constants module. Builds on existing tooling: `lint:i18n` eslint check, `verify-locale-sync.mjs`, `ng extract-i18n`. This is the second attempt at closing this exact gap -- v0.9.63 left dashboard + WARNING-02 as accepted debt that then sat untouched through 6+ subsequent milestones (v0.9.69, v0.10.0, v0.11.0, v0.12.0, v1.0.0, v1.1.0). +**Key context:** Supported locales are fixed (en source + es/de/ja/zh-CN/zh-TW) -- not up for debate, carried over from v0.9.63's `LocaleService` + locale-constants module. Builds on existing tooling: `lint:i18n` eslint check, `verify-locale-sync.mjs`, `ng extract-i18n`. This is the second attempt at closing this exact gap -- v0.9.63 left dashboard + WARNING-02 as accepted debt that then sat untouched through 6+ subsequent milestones (v0.9.69, v0.10.0, v0.11.0, v0.12.0, v0.13.0, v1.1.0). **Progress:** Phase 52 (Full-Page Translation Completeness Audit) complete 2026-07-08 -- `audit-translation-completeness.mjs` confirms the true drift/gap scope across all 12 current routes and 5 locales: 5 drifted trans-units (matches the corrected estimate, superseding the original "247" figure), 54 orphaned ids/locale, and the `translations.stats-274.*.json` artifacts traced as 15/21 keys already merged into live XLIFF per locale (6 missing, 0 stale). This is the authoritative scope Phases 53-55 inherit. @@ -71,15 +101,15 @@ FSB is an AI-powered browser automation Chrome extension that executes tasks thr **Status:** Archived on 2026-06-30. Audit passed; automated milestone gates met. Archive files live under `.planning/milestones/v1.1.0-*`. -**Outcome:** FSB converted the v1.0.0 catalog tail from undifferentiated search/discovery support into an explicit readiness and terminal-state model. The milestone added reusable port contracts, same-origin read ports, bridge decision gates, guarded-write evidence gates, terminal-state reporting, and a write/destructive UAT ledger. +**Outcome:** FSB converted the v0.13.0 catalog tail from undifferentiated search/discovery support into an explicit readiness and terminal-state model. The milestone added reusable port contracts, same-origin read ports, bridge decision gates, guarded-write evidence gates, terminal-state reporting, and a write/destructive UAT ledger. **Final counts:** 2,314 descriptors; 1,267 executable T1-ready rows; 556 guarded fail-closed rows; 5 bridge-needed rows; 141 UAT-needed rows; 123 blocked rows; 222 degraded/discovery-pending rows. **Accepted closeout caveats:** v1.1.0 does not claim all apps are T1-ready. Remaining rows are non-invocable until their documented proof requirements are satisfied. Backlog side phase `999.1 MCP tool gaps -- click heuristics` remains outside the milestone. -## Last Milestone: v1.0.0 Full App Catalog (OpenTabs Parity) +## Last Milestone: v0.13.0 Full App Catalog (OpenTabs Parity) -**Status:** Archived on 2026-06-29. Audit passed; automated milestone gate met. Archive files live under `.planning/milestones/v1.0.0-*`. +**Status:** Archived on 2026-06-29. Audit passed; automated milestone gate met. Archive files live under `.planning/milestones/v0.13.0-*`. **Outcome:** FSB imported the full allowed OpenTabs-derived catalog surface into existing capability tiers: 2,314 descriptors across 128 app stems / 129 services, with side-effect classification, backing-status annotation, denylist/sensitive-origin gating, discovery seeding, scale gates, and selected T1 heads. The milestone deliberately did not hand-port every descriptor; that T1 expansion debt is now the explicit v1.1.0 milestone. @@ -314,6 +344,19 @@ Carry-forward backlog candidates: ## Requirements +### Active — v1.0.0 Jev Fast Mode + +The 25 testable active requirements are in `.planning/REQUIREMENTS.md`. They cover: +- timing and a repeatable baseline (MEAS-01..03) +- the hosted Jev gateway and safety policy (JEV-01..07) +- opt-in autopilot fast mode (AUTO-01..05) +- additive MCP speedups (MCP-01..04) +- the MCP settings tool (SET-01..02) +- comparative evaluation (EVAL-01..02) +- versioned release (REL-01..02) + +Every requirement maps to exactly one of Phases 66-71 or inserted Phase 66.1 in `.planning/ROADMAP.md`. + ### Validated - ✓ Chrome Extension MV3 architecture with service worker -- existing @@ -445,7 +488,7 @@ Carry-forward backlog candidates: ### Active -(Milestone v0.9.91 MCP Clients as Providers -- Phases 57-63 are verified complete. NATIVE-01..04 validated in Phase 63: Native-Messaging Host. Remaining active requirements are MULTI-01..06; 45/51 requirements are complete.) +(Milestone v1.0.0 Jev Fast Mode -- requirements are being defined in `.planning/REQUIREMENTS.md`.) ### Validated (v0.9.91) @@ -472,6 +515,9 @@ Carry-forward backlog candidates: - [x] UX-01..06: Consent, delegated routing, live feed, background ownership, Take Control/Stop, and honest usage/offline recovery are implemented without weakening BYOK behavior -- Phase 61. - [x] LIFE-01..04: Session-backed event persistence, exact recovery, active-only heartbeat, hold/resume, and daemon-restart classification survive MV3 worker eviction -- Phase 61. - [x] DRIFT-01..04: Registry-driven offline drift CI, safe doctor text/JSON, sanitized rate-limited runtime diagnostics, and one canonical browser-consumed compatibility matrix are enforced -- Phase 62. +- [x] NATIVE-01..04: An optional native-messaging host registers per platform, wakes or attaches only to `serve`, never spawns agent CLIs, and backs an honest "Agent offline" wake flow -- Phase 63. +- [x] MULTI-01..03: OpenCode 1.14.25 runs through the unchanged five-method adapter contract with cold spawn plus FSB-owned server attach, a pinned profile, fixture and drift coverage -- Phase 64. +- [x] MULTI-04..06: Codex runs through hermetic `codex exec --json` with source-pinned fixture/drift coverage and honest ChatGPT OAuth vs API-key vs unauthenticated disclosure; task-mode only -- Phase 65. ### Validated (v0.9.99) @@ -717,6 +763,14 @@ The following backlog items are formally retired in v0.9.45rc1. Better external | Action tools always return `change_report`; read tools never do | Reduces follow-up `read_page` round-trips on action-heavy flows; keeps reads pristine | Good -- v0.9.60 ~halves tokens-per-task on action sequences; size-capped with `truncated` hint | | Bootstrap-safe recovery tools must work from restricted active tabs | Active `chrome://newtab/` was blocking the same recovery tools the error message advised | Good -- v0.9.60 (Phase 247) `open_tab`/zero-owned `navigate`/`switch_tab`/`list_tabs` recover without content-script attachability while preserving cross-agent rejection | | Background-tab-by-default for MCP-routed surfaces | MCP agents must not steal focus from the user or other agents | Good -- v0.9.60 (Phase 246) `open_tab` defaults background; agent-scoped tab resolution replaces `chrome.tabs.query({active:true})` everywhere except synthesized `legacy:*` agents | +| Security-first reverse channel before any spawn code | Turning a browser click into an agent CLI process is RCE-adjacent | Good -- v0.9.91 Phase 59 landed CHAN-01..07 before Phase 60's spawn code, with a permanent forbidden-flag CI gate | +| Close v0.9.91 with every deferred human UAT waived | The user chose to close the milestone and mark all accumulated live UAT done without running it | Pending -- 48 live scenarios were never executed; the archived ledgers keep their pending entries as the record | +| Baseline before Jev control | Whole-task speed and success must be measured before introducing a decision fast path | Planned in v1.0.0 Phase 66 before provider and loop work | +| Keep Jev optional and bounded | A typed model answer can still be wrong; the LLM and FSB mechanics retain planning, text, execution, and safety authority | Planned in Phases 67-70 with shadow calibration and fallback | +| FSB provides Jev without a user key | The user wants one Advanced Settings switch and no provider setup | Phase 67 holds the upstream credential on the FSB server; Phase 68 adds the switch; Phase 70 follows it for MCP | +| Offer hosted Jev to every install without a usage quota | The owner chose open access for now | Phase 67 measures aggregate usage and cost and includes an operator-wide disable switch | +| Add one MCP settings tool | The owner wants MCP clients to view settings and change any of them efficiently, including the Jev switch | Phase 66.1: secrets are masked and never writable, every other setting can be written, and only MCP clients get the tool | +| Backwards compatible with Jev off | The owner requires FSB to behave as before whenever Jev is disabled | Milestone-wide: every change is additive, and the fast paths activate only with the switch on | ## Evolution @@ -736,4 +790,4 @@ This document evolves at phase transitions and milestone boundaries. 4. Update Context with current state --- -*Last updated: 2026-07-21 -- Phase 64 OpenCode Adapter deterministically complete; 45 live scenarios remain deferred to the milestone-end sweep. Next: Phase 65 Codex Adapter.* +*Last updated: 2026-09-29 -- Phase 66 context gathered; inserted Phase 66.1 MCP Settings Tool (SET-01..02).* diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index a9bf7c9d0..675ff34e4 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -1,203 +1,91 @@ # Requirements: FSB (Full Self-Browsing) -**Defined:** 2026-07-11 -**Core Value:** Reliable single-attempt execution -- the AI decides correctly, the mechanics execute precisely. -**Milestone:** v0.9.91 MCP Clients as Providers +**Defined:** 2026-09-28 +**Core value:** Reliable single-attempt execution — the AI decides correctly; the mechanics execute precisely. +**Milestone:** v1.0.0 Jev Fast Mode +**Source:** `.planning/PROJECT.md` and `.planning/research/JEV-REFERENCE.md` ## v1 Requirements -Requirements for this milestone. Each maps to roadmap phases. +### MEAS — Timing and Baseline -### IDENT -- Agent Identity Capture +- [ ] **MEAS-01:** An operator can inspect per-call wall time, FSB execution time, model time, tokens, and cost for both MCP and autopilot sessions, with unavailable values labelled rather than invented. +- [ ] **MEAS-02:** An operator can retrieve session details for a journal-backed MCP session, including its tool calls and timing records. +- [ ] **MEAS-03:** An operator can run a repeatable, outcome-checked task suite and save a baseline of completion rate, wall time per completed task, and cost before enabling Jev. -- [x] **IDENT-01**: When the user clicks a copy-to-clipboard button on the onboarding MCP-install screen for a specific client (`claude-code`, `cursor`, `vscode`, `windsurf`, `codex`, `opencode`, `openclaw`, `claude-desktop`, `all`), FSB records that client id (with timestamp, deduplicated, all-clients aggregated for multi-select cases) into a durable `fsbAgentProviders.clicked` list in `chrome.storage.local`. -- [x] **IDENT-02**: When an MCP client completes its `initialize` handshake with `fsb-mcp-server` (over any transport: stdio, streamable-HTTP, or the ws://7225 bridge), FSB captures the caller's `clientInfo.name` and `clientInfo.version` and threads them to the extension via an additive field on the existing `agent:register` bridge payload without breaking any existing consumer of that payload (INV-01). -- [x] **IDENT-03**: The extension's agent registry stamps captured `clientInfo` onto each live `AgentRecord` and rolls the identity up into a durable `fsbAgentProviders.connected` entry that survives service-worker eviction and Chrome restart, keyed so re-connections update rather than duplicate. -- [x] **IDENT-04**: `fsb-mcp-server` can enumerate installed MCP-capable clients on the current machine by inspecting the paths already known to `platforms.ts` (per-OS `configPath` for file-mode clients; ` --version` probe for cli-mode `claude-code`) and report each as `installed` / `not-installed` with any parseable version. -- [x] **IDENT-05**: A `getMcpClients` extension runtime message returns a merged view (`clicked` ∪ `installed` ∪ `connected`) with per-client status, so UI surfaces read one consistent structure instead of assembling it themselves. +### JEV — Decision Provider and Safety -### PROV -- Providers Panel +- [ ] **JEV-01:** A user on any FSB install can use FSB-provided Jev without supplying or configuring a Jev or OpenRouter key, signing in, or receiving a per-user or per-install usage quota. (code-complete; live proof deferred to V2) +- [ ] **JEV-02:** FSB's Jev credential stays in server-side secret storage; the extension sends bounded decision requests to the FSB server over HTTPS and never receives the upstream credential. (code-complete; live proof deferred to V2) +- [x] **JEV-03:** The extension and server omit secrets, credential values, password, file, and hidden inputs, and unnecessary page content from Jev requests; page text is treated as untrusted data rather than instructions. (met by automated evidence in Phase 67) +- [ ] **JEV-04:** An operator can inspect the actual returned model version, token usage, FSB-borne cost when available, and typed fallback reason for each Jev decision without exposing request bodies or credentials in logs. (code-complete; live proof deferred to V2) +- [ ] **JEV-05:** Jev can evaluate labelled tasks in shadow mode without acting, so operation, target, completion, stuck, and risk thresholds are calibrated on a pinned model before live use. (code-complete; live proof deferred to V3) +- [x] **JEV-06:** Jev can add a confirmation requirement but cannot remove an existing confirmation, bypass an execution safety breaker, or authorize an action on its own. (met by automated evidence in Phase 67; the Phase 68 verifier re-checked and confirmed that every Jev-accepted action still passes the existing consent, permission and breaker gates) +- [ ] **JEV-07:** The FSB server validates a fixed bounded decision schema, limits request size and concurrent upstream work, records aggregate usage and FSB-borne cost, and supports an operator-wide disable switch; malformed, unavailable, and disabled calls return typed fallbacks. (code-complete; live proof deferred to V2) -- [x] **PROV-01**: The control panel section formerly labeled "API Configuration" is labeled "Providers" (heading, nav label, and any anchor `#api-config` continues to work as a redirect to the new `#providers` anchor for existing bookmarks). -- [x] **PROV-02**: Each provider has an explicit `providerKind` value of either `api` (the existing 7 BYOK LLM providers) or `agent` (a locally installed agent CLI); the kind determines which fields render. -- [x] **PROV-03**: When an `agent`-kind provider is selected, the API-key input, key-URL hint, and per-model key-format hint are hidden; the panel shows instead the provider's install status, auth status (from the CLI's own login state where surfaceable), connection status, and a short "uses your subscription -- no API key needed" caption. -- [x] **PROV-04**: When the user has both an active `agent` provider selection and a valid BYOK key for an `api` provider, `universal-provider.js` continues to see only `api`-kind provider values; the two selections do not collide, and switching between them preserves the other's configuration (INV-03 provider parity for the BYOK side). -- [x] **PROV-05**: The panel visually marks exactly one provider as "Recommended" per session, chosen by a ground-truth cascade: highest-priority = a provider whose CLI is currently connected via MCP `initialize`, next = a provider whose CLI is installed on disk, next = a provider whose copy button the user clicked during onboarding, fallback = the current xAI-default recommendation. The panel never auto-switches the user's selection; the badge is advisory only. -- [x] **PROV-06**: Cost/usage rows for `agent`-kind providers never display fabricated dollar amounts; they display token count, turn count, and duration and label the run as "included in your subscription", with a link to the vendor's current billing page (copy must not promise "free" or "unlimited"). +### AUTO — Autopilot Fast Mode -### CHAN -- Delegation Channel & Security Foundation +- [ ] **AUTO-01:** A user can enable or disable Jev fast mode in the control panel's Advanced Settings, or through the MCP settings tool (SET-01), which changes the same stored setting; it is off by default, explains that bounded page context is sent through FSB to Jev, and existing autopilot behavior remains available when off. (code-complete; live proof deferred to V5 and V6) +- [ ] **AUTO-02:** For a supported simple step, Jev can choose only an offered operation and element from FSB's filtered current snapshot; FSB validates target freshness and executes the action through its existing mechanics. (code-complete; live proof deferred to V3 and V5) +- [x] **AUTO-03:** The LLM retains planning, generated text, page reading, unsupported actions, and decisions below calibrated thresholds. (met by automated evidence in Phase 68) +- [ ] **AUTO-04:** The fast loop uses observable page evidence for completion and stuck checks, enforces task time, action, and Jev-call budgets, and reports the actual terminal state. (code-complete; live proof deferred to V4) +- [ ] **AUTO-05:** Ambiguous targets, changed page state, provider errors, and missing evidence fall back to the existing LLM path or a truthful blocked outcome without repeating an unsafe action. (code-complete; live proof deferred to V5) -- [x] **CHAN-01**: A new bridge message-type family (`ext:request` / `ext:response` / `ext:event`) transports extension→daemon reverse requests over the existing ws://localhost:7225 bridge without changing the byte-shape of any existing `MCPMessageType` value (INV-01 additive proof). -- [x] **CHAN-02**: A relay process signals its ability to fulfill spawn requests by advertising `capabilities: ['agent-spawn']` in its `relay:hello`; the hub routes each `ext:request` locally (if itself the daemon) or to the first relay advertising the required capability. -- [x] **CHAN-03**: The bridge rejects every incoming `ext:*` frame whose WebSocket upgrade did not carry an `Origin` header matching a durable per-install FSB-extension-id allowlist and whose `Host` header is not exactly `127.0.0.1` (or `localhost`) at the loopback port. -- [x] **CHAN-04**: A per-install >=32-byte shared secret is provisioned once between the extension and the daemon, transported only in the `Sec-WebSocket-Protocol` upgrade header (never in URL, never in payloads, never in logs), rotated on daemon restart, and required on every `ext:*` frame. -- [x] **CHAN-05**: `redactForLog` and diagnostic ring-buffer writes strip any string matching the shared-secret token pattern; the drift gate fails the build if a raw secret substring appears in any tracked log fixture. -- [x] **CHAN-06**: The bridge topology test suite covers hub-exit-mid-delegation and relay-mid-`ext:*`-frame scenarios; existing hub-exit-promotion tests still pass byte-for-byte. -- [x] **CHAN-07**: A permanent CI grep gate fails the build if the strings `--dangerously-skip-permissions`, `--yolo`, or `--auto` appear anywhere in `mcp/src/agent-providers/**`, so those flags can never enter the spawn path in any future patch. +### MCP — Faster Existing Tool Surface -### ADAPT -- Adapter Contract & Spawn Supervisor +- [ ] **MCP-01:** MCP clients receive shorter descriptions for all existing tools (the 72 tools at `8a058bdf`, 73 with the Phase 66.1 settings tool) while retaining enough argument and safety guidance to use them correctly. (code-complete; live proof deferred to V4 and V5) +- [ ] **MCP-02:** Action tools accept an optional `expect` value and return a bounded outcome check in the same call, so the caller can distinguish verified effect from uncertainty. (code-complete; live proof deferred to V4 and V5) +- [ ] **MCP-03:** Existing `run_task` honors the extension's Advanced Settings fast-mode toggle, applies bounded Jev decisions when enabled, and reports done, ambiguous, blocked, or needs-confirmation outcomes without claiming success from Jev's answer alone. (code-complete; live proof deferred to V2-V6) +- [x] **MCP-04:** Existing MCP tool names, required arguments, and wire contracts remain byte-stable; this milestone adds only the optional `expect` parameter on action tools and the settings tool approved on 2026-09-29 (SET-01, SET-02). (met by automated evidence in Phase 69, with the recorded D-25 redaction exception; owner items WR-05 and CR-02 in V5) -- [x] **ADAPT-01**: An `AgentProviderAdapter` TypeScript interface in `mcp/src/agent-providers/` defines exactly five methods: `detect() -> {installed, version, authState, binary}`, `buildSpawn(task, ctx) -> SpawnSpec`, `parseEvents(stream) -> AsyncIterable`, `kill(child, {grace}) -> Promise`, and `caps() -> AdapterCapabilities`. -- [x] **ADAPT-02**: A `SpawnSupervisor` module living in the `fsb-mcp-server serve` daemon accepts a validated spawn request, looks up the requested adapter, constructs argv from adapter output plus a daemon-controlled flag allowlist (unknown payload keys rejected), and spawns the child with `{ detached: true, windowsHide: true, stdio: ['pipe','pipe','pipe'] }` and an environment with `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`/`GEMINI_API_KEY` scrubbed. -- [x] **ADAPT-03**: The supervisor never invokes a shell; the user's prompt travels only via `child.stdin` (never argv, never `sh -c`) so shell metacharacters and Windows `.cmd`-shim EINVAL (Node CVE-2024-27980) cannot execute. -- [x] **ADAPT-04**: A `stop`/cancel request triggers SIGTERM at the process-group level (POSIX `process.kill(-child.pid, 'SIGTERM')` after `spawn({detached:true})`, Windows `taskkill /pid /T /F`), escalates to SIGKILL after a grace window, and blocks resolving the delegation until either an exit-signal is observed or the daemon confirms no descendant matches remain. -- [x] **ADAPT-05**: On daemon startup, the supervisor scans for orphaned children matching prior adapter fingerprints and kills them before accepting new spawn requests (recovery from crash). +### SET — Settings Control -### CLAUDE -- Claude Code MVP +- [ ] **SET-01:** An MCP client can use one new tool to view all of FSB's control-panel settings in one compact call and to change a single setting by key in one call. Secret values are masked on read and refused on write. Every other setting is writable. (code-complete; live proof deferred to V5) +- [ ] **SET-02:** Changes made through the settings tool use the control panel's own validation and storage, so the control panel, autopilot, and `run_task` share one value per setting, including the Jev fast-mode switch once Phase 68 adds it. Only MCP clients get the tool, and the autopilot tool list stays unchanged. (code-complete; live proof deferred to V5) -- [x] **CLAUDE-01**: The Claude Code adapter spawns `claude -p --verbose --output-format stream-json --include-partial-messages --strict-mcp-config --mcp-config --agents --agent fsb --permission-mode dontAsk --allowedTools "mcp__fsb" --disallowedTools "Bash,Edit,Write,NotebookEdit,WebFetch,WebSearch" --max-turns 40 --no-session-persistence`, or the version-appropriate equivalent selected by `detect()` output. -- [x] **CLAUDE-02**: The user's task prompt is sent to the spawned CLI via stdin only; the adapter constructs no argv fragment containing user-supplied text. -- [x] **CLAUDE-03**: The adapter's `parseEvents` translates the CLI's stream-json events (`system/init`, `assistant`, `user`, tool-use events, `system/api_retry`, `result`) into a normalized `AgentEvent` schema (`type`, `sessionId`, `payload`), fails loud on unknown event types (surfaced as `agent_protocol_drift` diagnostic), and is covered by a recorded JSONL fixture under `tests/fixtures/agent-streams/claude-code-2.1.177/` so CI runs without a live CLI. -- [x] **CLAUDE-04**: The Claude Code adapter's `detect()` fingerprints the binary via `claude --version`, compares against a minimum supported version, and reports `installed=false` with a doctor-diagnostic message rather than spawning if the version predates the verified stream-json contract. +### EVAL — Comparative Release Gate -### UX -- Delegation UX +- [ ] **EVAL-01:** An operator can compare whole-task completion rate, wall time per completed task, and cost across baseline, fast mode, and a non-Jev control on the same task set. (code-complete; live proof deferred to V1, V3 and V4) +- [ ] **EVAL-02:** Fast-mode release evidence reports pass or fail against predeclared speed, completion, and safety criteria; failed criteria keep the affected fast path experimental or disabled rather than being reported as a win. (code-complete; live proof deferred to V4, with the M-71-00 gate sign-off at V1 and the verdicts applied at V6) -- [x] **UX-01**: A fifth entry `delegated` in `EXECUTION_MODES` (`extension/ai/engine-config.js`) defines: `uiFeedbackChannel: 'popup-sidepanel'`, `animatedHighlights: true`, `safetyLimits: { wallClockMs, eventSilenceMs }` (no iteration cap -- the loop runs in the spawned CLI, not `runAgentLoop`), and is selected when the active provider is `agent`-kind. -- [x] **UX-02**: The side panel renders a live per-run streaming feed with distinct card types for init (client, model, session id, allowed tools), tool-call (name, args summary, tab id), retry (typed error class), and result (usage summary), driven by the normalized `AgentEvent` stream. -- [x] **UX-03**: Before FSB spawns any agent CLI for the first time, the user sees an explicit consent card that names the CLI, what it will be permitted to do (drive the FSB MCP tools on the user's live browser), and what it will not be permitted to do (edit files, run shell, fetch arbitrary URLs). A per-run confirm-to-continue toggle is on by default and can be disabled per provider only via an explicit "trust this agent" setting. -- [x] **UX-04**: A prominent Stop button in the side panel triggers `stopDelegatedTask`, which routes to the supervisor's kill and, on confirmed exit, releases every tab that was owned by the spawned agent (per v0.9.60 ownership) and reports "Agent stopped, N tab(s) released" in the feed. -- [x] **UX-05**: A delegated run opens by default in a new background tab; when the user activates the tab that the agent is driving, a persistent "Take control" affordance appears; clicking it pauses the agent (v0.9.60 ownership release + supervisor grace hold), lets the user interact, and offers "Resume with agent" to give ownership back. -- [x] **UX-06**: A post-run summary card displays tokens (in/out/total), turn count, wall-clock duration, cost bucket (`included in your subscription` for agent kind; real USD for api kind), and a per-tool-call breakdown, expandable to the full tool-call log for the run. +### REL — Versioned Release -### LIFE -- Lifecycle & Persistence +- [ ] **REL-01:** The extension is versioned as 1.0.0 through a `CHANGELOG.md` v1.0.0 entry followed by `npm run version:set:extension -- 1.0.0`. (code-complete; live proof deferred to V6) +- [ ] **REL-02:** The MCP package receives the planned minor version bump for its additive optional parameters and the new settings tool, with compatibility checks against existing callers. (code-complete; live proof deferred to V4 and V6) -- [x] **LIFE-01**: Every progress event received from the supervisor is written to `chrome.storage.session` under a per-delegation key before it fans out to UI subscribers, so a MV3 service worker eviction mid-run reloads exactly the delivered feed on re-open. -- [x] **LIFE-02**: While a delegation is active, the extension pings the bridge every 20 s over the existing WS heartbeat channel to keep the Chrome 116+ SW-lifetime extension applied; if 3 heartbeats are missed the extension shows a `daemon:disconnected` fallback that offers a doctor-relaunch button but does not attempt an in-extension restart. -- [x] **LIFE-03**: If `fsb-mcp-server serve` is not running when a delegated send is attempted, the side panel shows an "Agent offline" state with a deep-link to `fsb-mcp-server doctor` output and does not enqueue or optimistically show the message. -- [x] **LIFE-04**: On daemon restart while a delegation was mid-flight, the supervisor does not re-adopt any surviving spawned CLI; it kills it (LIFE-04 restart-is-clean) and reports `daemon_restart_lost_run` in the side panel so the user knows the run ended. +## Future Requirements -### DRIFT -- CI Drift-Smoke Gate & Doctor Extensions - -- [x] **DRIFT-01**: A CI job runs each shipped adapter against a canned prompt fixture, asserts a known event-type sequence and the presence of required fields on `system/init` and `result`, and fails the build on unknown event types, missing fields, or a `--version` outside the compatibility matrix. -- [x] **DRIFT-02**: `fsb-mcp-server doctor` gains a per-adapter section reporting: binary path, version, auth state (parseable where the CLI exposes it), shared-secret presence, and the current spawn-secret rotation age. -- [x] **DRIFT-03**: The diagnostics ring buffer classifies drift events as `agent_protocol_drift` (with adapter id, expected vs observed) and rate-limits duplicate entries at the existing 1-per-10s bucket. -- [x] **DRIFT-04**: The `doctor` output includes a machine-readable adapter compatibility matrix that both CI and the extension can read to render "supported / degraded / unsupported" states without hardcoding versions in extension code. - -### NATIVE -- Native-Messaging Host - -- [x] **NATIVE-01**: `fsb-mcp-server install --native-host` writes the platform-appropriate native-messaging host manifest (macOS `~/Library/Application Support/Google/Chrome/NativeMessagingHosts/`, Linux `~/.config/google-chrome/NativeMessagingHosts/`, Windows registry `HKCU\Software\Google\Chrome\NativeMessagingHosts\`), allowing the FSB extension id as the sole caller, and installs a host binary that wakes the daemon on demand. -- [x] **NATIVE-02**: The extension's manifest gains a `nativeMessaging` permission entry (additive, no other permission changes), and the extension detects native-host presence at boot; when present, an "Agent offline" state auto-attempts a wake before showing the doctor deep-link. -- [x] **NATIVE-03**: The native host itself does not spawn agents; it only starts `fsb-mcp-server serve` (or attaches to a running one) and exits after handoff. All spawn authority remains inside the `serve` daemon behind the CHAN gates. -- [x] **NATIVE-04**: `fsb-mcp-server uninstall --native-host` removes the manifest, and `doctor` reports native-host install state including manifest path and any Chrome allowlist mismatch. - -### MULTI -- Additional Adapters - -- [x] **MULTI-01**: An OpenCode adapter (`mcp/src/agent-providers/opencode.ts`) implements the `AgentProviderAdapter` contract with `caps.serverMode=true`; the supervisor either spawns `opencode run` cold or attaches to a running `opencode serve` per the adapter's `buildSpawn` output (contract-stresser: the ADAPT contract must accommodate both spawn and attach without hardcoding). -- [x] **MULTI-02**: The OpenCode adapter ships a pinned agent definition (equivalent to Claude Code's `--agents fsb`) using OpenCode's `agent create` / `agents` config surface, keyed to a version pinned during phase spike. -- [x] **MULTI-03**: A recorded OpenCode JSONL fixture under `tests/fixtures/agent-streams/opencode-1.14.25/` (or the latest pinned version) proves the adapter's event schema in CI without a live CLI. -- [x] **MULTI-04**: A Codex adapter (`mcp/src/agent-providers/codex.ts`) implements the `AgentProviderAdapter` contract, invoking `codex exec --json` with the current-verified flag set (v0.142.5 as baseline: use `--ephemeral` + `--ignore-user-config` for hermeticity; do not use the deprecated `--full-auto`). -- [x] **MULTI-05**: The Codex adapter's `detect()` correctly identifies auth via ChatGPT OAuth vs API key vs unauthenticated and surfaces the state in the provider panel so the user knows which billing bucket a run will hit. -- [x] **MULTI-06**: A schema-derived Codex JSONL contract fixture pins the event schema in CI with `liveCapturePending: true` until genuine UAT capture, and the adapter's `caps()` correctly reports `chatMode: false` for v0.9.91 (task-mode only across all adapters). - -## v2 Requirements - -Deferred to future release. Tracked but not in current roadmap. - -### Chat-Mode Continuity - -- **CHAT-FUTURE-01**: Adapters expose `caps.chatMode: true` and the side panel maps a chat thread to `--resume ` (Claude Code) / `codex resume` / `gemini --resume` / `opencode --continue` per adapter. -- **CHAT-FUTURE-02**: The daemon pins per-thread working directory so `claude --resume` finds its history. - -### Gemini CLI Adapter - -- **GEMINI-FUTURE-01**: Gemini CLI adapter after a live `--help` capture and JSONL schema pinning (v0.9.91 lacked a local binary to verify against). - -### Broader Agent Ecosystem - -- **ACP-FUTURE-01**: ACP-based adapter unification (`@zed-industries/agent-client-protocol`) once ≥2 non-Claude adapters have shipped and proven the contract shape. -- **REMOTE-FUTURE-01**: Remote/mobile delegation surfaces (Happy-style approval flows) -- explicitly localhost-only in v0.9.91. +| Candidate | Reason deferred | +| --- | --- | +| A Jev-generated plan, typed prose, or page summary | Jev returns typed decisions; the LLM owns writing and reading. | +| Automatic fast mode for every task | This milestone gathers task-level evidence and keeps activation opt-in. | +| New MCP tools for Jev | Existing action tools and `run_task` cover the planned surface; add a tool only after a separately reviewed need. | +| User-supplied Jev or OpenRouter keys | FSB provides Jev through its hosted server path. | +| Further decision-model gateways or self-hosted models | The v1.0.0 server path uses a pinned Jev upstream. | +| Per-user or per-install Jev usage quotas | The owner chose unrestricted availability for now; usage is monitored centrally. | ## Out of Scope -Explicitly excluded. Documented to prevent scope creep. - -| Feature | Reason | -|---------|--------| -| Embedding `@anthropic-ai/claude-agent-sdk` in FSB | Anthropic policy prohibits third-party products from using consumer subscription auth via the SDK (enforcement Apr 4 2026); shelling to the user's installed `claude` binary is the only compliant "no API key" path. | -| Proxying / spoofing subscription OAuth tokens | Banned outright by Anthropic Apr 4 2026; would be product-killing regardless of technical feasibility. | -| Bundling / silent-installing agent CLIs | Wrong shape for a Chrome extension + npm daemon; would violate Chrome Web Store distribution rules and add attack surface. | -| PTY / TUI screen-scraping of agent CLIs | Structured headless interfaces exist for all four target CLIs; scraping is a maintenance sinkhole. | -| Any `--dangerously-skip-permissions` / `--yolo` / `--auto` flag in the spawn path | 1-click RCE for any prompt-injection incident; CHAN-07 grep gate makes this a permanent invariant. | -| Fabricated dollar costs on subscription-backed runs | Cline established the $0.00 convention; PROV-06 codifies it. | -| Auto-switching the user's provider when a "better" agent appears | Advisory badge only (PROV-05); user consent is the load-bearing property. | -| Forcing users away from BYOK when an agent is available | INV-03 provider parity carries forward; BYOK stays first-class. | -| `Firefox` support | Deferred at project level; MV3 nativeMessaging in NATIVE requirements is Chrome-specific for v0.9.91. | -| Native host that itself spawns agent CLIs | NATIVE-03 explicitly forbids this; all spawn authority lives inside the serve daemon behind CHAN gates. | +| Item | Reason | +| --- | --- | +| Replacing the LLM with Jev | Planning, text generation, extraction, and complex page understanding still require the LLM. | +| Jev inventing selectors or arbitrary tool arguments | It may choose only code-offered options; FSB mechanics validate and execute. | +| Reducing existing purchase, account, publication, or other confirmation gates | Jev can only tighten confirmations. | +| Sending passwords, credential values, or hidden fields to a decision provider | Violates the existing secret-handling boundary. | +| Sending the FSB-owned Jev credential to extension code or storage | A shared credential cannot remain secret in a distributed client. | +| A second fast-mode switch, such as an MCP argument or a separately stored toggle | Advanced Settings holds the only fast-mode setting; the MCP settings tool reads and writes that same value. | +| User sign-in or payment entitlement for Jev | FSB provides Jev to every installation in this milestone. | ## Traceability -Which phases cover which requirements. Populated during roadmap creation. - -| Requirement | Phase | Status | -|-------------|-------|--------| -| IDENT-01 | Phase 57 | Complete | -| IDENT-02 | Phase 57 | Complete | -| IDENT-03 | Phase 57 | Complete | -| IDENT-04 | Phase 57 | Complete | -| IDENT-05 | Phase 57 | Complete | -| PROV-01 | Phase 58 | Complete | -| PROV-02 | Phase 58 | Complete | -| PROV-03 | Phase 58 | Complete | -| PROV-04 | Phase 58 | Complete | -| PROV-05 | Phase 58 | Complete | -| PROV-06 | Phase 58 | Complete | -| CHAN-01 | Phase 59 | Complete | -| CHAN-02 | Phase 59 | Complete | -| CHAN-03 | Phase 59 | Complete | -| CHAN-04 | Phase 59 | Complete | -| CHAN-05 | Phase 59 | Complete | -| CHAN-06 | Phase 59 | Complete | -| CHAN-07 | Phase 59 | Complete | -| ADAPT-01 | Phase 60 | Complete | -| ADAPT-02 | Phase 60 | Complete | -| ADAPT-03 | Phase 60 | Complete | -| ADAPT-04 | Phase 60 | Complete | -| ADAPT-05 | Phase 60 | Complete | -| CLAUDE-01 | Phase 60 | Complete | -| CLAUDE-02 | Phase 60 | Complete | -| CLAUDE-03 | Phase 60 | Complete | -| CLAUDE-04 | Phase 60 | Complete | -| UX-01 | Phase 61 | Complete | -| UX-02 | Phase 61 | Complete | -| UX-03 | Phase 61 | Complete | -| UX-04 | Phase 61 | Complete | -| UX-05 | Phase 61 | Complete | -| UX-06 | Phase 61 | Complete | -| LIFE-01 | Phase 61 | Complete | -| LIFE-02 | Phase 61 | Complete | -| LIFE-03 | Phase 61 | Complete | -| LIFE-04 | Phase 61 | Complete | -| DRIFT-01 | Phase 62 | Complete | -| DRIFT-02 | Phase 62 | Complete | -| DRIFT-03 | Phase 62 | Complete | -| DRIFT-04 | Phase 62 | Complete | -| NATIVE-01 | Phase 63 | Complete | -| NATIVE-02 | Phase 63 | Complete | -| NATIVE-03 | Phase 63 | Complete | -| NATIVE-04 | Phase 63 | Complete | -| MULTI-01 | Phase 64 | Complete | -| MULTI-02 | Phase 64 | Complete | -| MULTI-03 | Phase 64 | Complete | -| MULTI-04 | Phase 65 | Complete | -| MULTI-05 | Phase 65 | Complete | -| MULTI-06 | Phase 65 | Complete | - -**Coverage:** -- v1 requirements: 51 total -- Mapped to phases: 51 / 51 (100%) -- Unmapped: 0 - -**Phase distribution:** -- Phase 57 (IDENT): 5 requirements -- Phase 58 (PROV): 6 requirements -- Phase 59 (CHAN): 7 requirements -- Phase 60 (ADAPT + CLAUDE): 9 requirements -- Phase 61 (UX + LIFE): 10 requirements -- Phase 62 (DRIFT): 4 requirements -- Phase 63 (NATIVE): 4 requirements -- Phase 64 (MULTI-OpenCode): 3 requirements -- Phase 65 (MULTI-Codex): 3 requirements - ---- -*Requirements defined: 2026-07-11* -*Last updated: 2026-07-16 after Phase 62 completion (41/51 requirements complete; all live checks deferred to milestone end)* +| Phase | Requirements | Count | +| --- | --- | ---: | +| 66 — Baseline and Session Timing | MEAS-01, MEAS-02, MEAS-03 | 3 | +| 66.1 — MCP Settings Tool (inserted; code-complete 2026-10-15, live proof deferred to V5) | SET-01, SET-02 | 2 | +| 67 — Hosted Jev Gateway, Policy, and Shadow Mode (code-complete 2026-10-15; JEV-03/JEV-06 met, JEV-01/JEV-02/JEV-04/JEV-07 live proof deferred to V2, JEV-05 calibration deferred to V3) | JEV-01, JEV-02, JEV-03, JEV-04, JEV-05, JEV-06, JEV-07 | 7 | +| 68 — Autopilot Fast Mode (code-complete 2026-10-17; AUTO-03 met, AUTO-01 live proof deferred to V5 and V6, AUTO-02 to V3 and V5, AUTO-04 to V4, AUTO-05 to V5) | AUTO-01, AUTO-02, AUTO-03, AUTO-04, AUTO-05 | 5 | +| 69 — MCP Descriptions and Same-Call Checks (code-complete 2026-10-15; MCP-04 met, MCP-01/MCP-02 live proof deferred to V4 and V5) | MCP-01, MCP-02, MCP-04 | 3 | +| 70 — Bounded MCP Fast Task (code-complete 2026-10-17; MCP-03 live proof deferred to V2-V6, with the WR-01 owner decision at V4 and the D-17 amendment and MCP-enable consent review at V6) | MCP-03 | 1 | +| 71 — Comparative Gate and Release (code-complete 2026-10-17; EVAL-01 live proof deferred to V1, V3 and V4, EVAL-02 to V4 with the M-71-00 gate sign-off at V1 and the verdicts applied at V6, REL-01 to V6, REL-02 to V4 `count_tokens` and the V6 bump and publish) | EVAL-01, EVAL-02, REL-01, REL-02 | 4 | + +**Coverage:** 25/25 requirements mapped exactly once. diff --git a/.planning/RETROSPECTIVE.md b/.planning/RETROSPECTIVE.md index d76be65ad..101fa74b9 100644 --- a/.planning/RETROSPECTIVE.md +++ b/.planning/RETROSPECTIVE.md @@ -214,6 +214,42 @@ --- +## Milestone: v0.9.91 -- MCP Clients as Providers + +**Shipped:** 2026-09-28 +**Phases:** 9 | **Plans:** 61 + +### What Was Built +- Durable MCP client identity (handshake clientInfo, a 21-client installed sweep, copy intent) behind one guarded `getMcpClients` view +- A Providers panel with explicit `api` vs `agent` kinds and one evidence-based recommendation, with BYOK settings preserved +- An authenticated reverse-request channel with a rotating session secret, exact Host/Origin checks, secret redaction and a permanent forbidden-flag CI gate +- One five-method adapter contract and spawn supervisor running Claude Code, OpenCode and Codex, with pinned profiles, recorded fixtures and drift CI +- Delegation UX (consent, live feed, Take Control/Stop, heartbeat, service-worker-eviction recovery) and an optional native wake host that only starts `serve` + +### What Worked +- Security-first ordering: the channel's trust boundary (Phase 59) landed and was tested before any spawn code existed (Phase 60). +- The adapter contract stayed unchanged while OpenCode and Codex were added, which proved the abstraction. +- Source-pin contract tests caught drift early, and every phase kept the full suite green. + +### What Was Inefficient +- Every live check was deferred to one milestone-end sweep that never ran; 48 scenarios accumulated and were waived at close, so the milestone has no live evidence. +- The milestone sat at `human_needed` for two months after implementation finished (2026-07-22 to 2026-09-28). +- Tests read live planning files (phase folders, ROADMAP.md, REQUIREMENTS.md), so archiving the milestone required repointing test paths. + +### Patterns Established +- Closing with waived UAT: record the waiver in STATE.md and MILESTONES.md and leave the UAT ledgers untouched, so no pass is fabricated. +- Tests that inspect planning artifacts read the archived milestone copies once a milestone closes. + +### Key Lessons +1. Deferring all live checks to a single end-of-milestone sweep turns UAT into debt; schedule them per phase, or agree up front that closure is automated-only. +2. Point planning-artifact tests at archive paths (or keep fixtures out of `.planning/`) so milestone archival cannot break the suite. + +### Cost Observations +- Model mix: inherit profile. +- Notable: 157 tasks across 61 plans; implementation ran 2026-07-10 to 2026-07-22. + +--- + ## Cross-Milestone Trends ### Process Evolution @@ -234,6 +270,7 @@ | v0.9.35 | 5 | 15 | MCP reliability before new feature expansion | | v0.9.36 | 3 | 6 | Explicit MCP visual lifecycle and trusted client identity | | v0.9.60 | 11 | 30 | Multi-agent tab concurrency, MCP 0.8.0, run_task lifecycle return, change_report, restricted-tab recovery | +| v0.9.91 | 9 | 61 | Agent CLIs as side-panel providers; security-first reverse channel; all live UAT waived at close | ### Top Lessons (Verified Across Milestones) diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index ec3d07ae3..e82c6162e 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -8,273 +8,324 @@ ## Milestones -- 🚧 **v0.9.91 MCP Clients as Providers** — Phases 57-65, planning 2026-07-11. Extends the Chrome MV3 extension + `fsb-mcp-server` so installed agent CLIs (Claude Code first, then OpenCode + Codex) become first-class side-panel providers: FSB captures which MCP clients the user copies/installs/connects, presents them as key-less providers in a renamed Providers panel, and delegates side-panel tasks to a spawned agent CLI that drives the browser back through FSB's own MCP tools — with a security-first reverse-request channel, an `AgentProviderAdapter` contract, task-mode only (chat-mode deferred), CI drift-smoke gate, native-messaging wake-host, and INV-01 (byte-stable existing MCP wire) preserved end-to-end. +- 🚧 **v1.0.0 Jev Fast Mode** — Phases 66-71, planned 2026-09-28. Establish a whole-task speed and quality baseline, add FSB-hosted Jev with the credential held on the server, expose one opt-in switch in Advanced Settings, add an MCP tool to view and change settings, use bounded autopilot and MCP fast paths, shorten existing MCP tool descriptions, and release the extension as 1.0.0 only with honest comparative evidence. +- ✅ **v0.9.91 MCP Clients as Providers** — Phases 57-65, shipped 2026-09-28. Installed agent CLIs (Claude Code, OpenCode, Codex) became key-less side-panel providers that drive the live browser through FSB's own MCP tools. Archive: `.planning/milestones/v0.9.91-ROADMAP.md`. - ✅ **v1.2.0 Showcase i18n Completeness** — Phases 52-56, shipped 2026-07-09. Archive: `.planning/milestones/v1.2.0-ROADMAP.md`. Closes the translation gap that reopened after v0.9.63 shipped: a full-page audit establishes the true drift/missing scope, resync + stats-page translation + transcreation review close it, a CI drift-detection gate lands on the clean baseline, and the long-deferred WARNING-02 locale-cookie redirect bug is fixed. - ✅ **v1.1.0 T1 App Execution Expansion** — Phases 44-51, shipped 2026-06-30; refreshed 2026-07-01 after post-closeout T1 ports. Expanded proven T1/guarded coverage from 26 baseline descriptors to 1,267 executable T1-ready descriptors plus 556 guarded fail-closed rows, and closed the remaining catalog tail with explicit terminal-state accounting. Archive: `.planning/milestones/v1.1.0-ROADMAP.md`. -- ✅ **v1.0.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v1.0.0-ROADMAP.md`. - -## Current Milestone: v0.9.91 MCP Clients as Providers - -**Milestone Goal:** Make installed agent CLIs (Claude Code first) first-class side-panel providers — FSB captures which MCP clients the user installs/connects, presents them as key-less providers in a renamed Providers panel, and delegates side-panel tasks to a spawned agent CLI that drives the browser back through FSB's own MCP tools. The user's own words: "when I'm sending a message from the side panel, the FSB extension sends a request to the MCP server, which spawns a Claude/Codex/OpenCode agent with the prompt, and that agent uses FSB MCP tools to perform the task — technically driving the local coding agent from the side panel." - -**Key context (non-negotiable):** - -- **Security-first hard rule.** The reverse-request channel is RCE-adjacent by construction: a browser click becomes `execve(claude, -p, )` inside a localhost daemon. The security foundation phase (Phase 59) — reverse-request channel + Origin/Host/secret/consent + CHAN-07 grep gate — MUST land before ANY spawn code exists. It cannot be deferred to a "hardening" phase. All CHAN-01..CHAN-07 requirements land in Phase 59 together. -- **INV-01 discipline.** Every wire addition is additive: new frame types (`ext:request`/`ext:response`/`ext:event`) + optional payload fields only. Existing `MCPMessageType` values and tool schemas stay byte-stable across the entire milestone; a byte-freeze regression test proves this in every phase that touches the wire. -- **Task-mode only.** Chat-mode continuity via `--resume` / `codex resume` / `opencode --continue` is v2 scope (CHAT-FUTURE-01/02); every adapter's `caps()` reports `chatMode: false` for v0.9.91. -- **Adapter breadth: OpenCode + Codex; skip Gemini.** GEMINI-FUTURE-01 defers Gemini until a live `--help` capture and JSONL schema pinning are done. NATIVE and DRIFT are IN scope. -- **No `--dangerously-skip-permissions` / `--yolo` / `--auto`.** Anywhere. Ever. Not even in an "advanced mode." A permanent CI grep gate (CHAN-07) fails the build if those strings appear under `mcp/src/agent-providers/**`. -- **No `@anthropic-ai/claude-agent-sdk` embedded in FSB.** Anthropic banned third-party products from using consumer subscription auth via the SDK (enforcement Apr 4 2026). Shelling to the user's installed `claude` binary is the only compliant "no API key" path and is standard ecosystem practice (Cline, Roo, Zed, Conductor). -- **Extension has no `nativeMessaging` permission in Phases 57-62.** Delegation therefore requires a running `fsb-mcp-server serve` daemon; MVP ships an honest "agent offline → doctor" state. Phase 63 (NATIVE) adds the optional wake-host that closes this UX cliff. -- **Source-pin tripwire discipline.** FSB's test suite pins exact token counts and substrings on extension source (even comments). Every extension-touching commit updates paired tripwires; the full suite runs from commit 1 of every phase. - -**Phase ordering rationale (from research; dependency chain non-negotiable):** identity data (Phase 57) → provider selection UI reads it (Phase 58) → security foundation must exist before any spawn code (Phase 59) → adapter contract needs the channel (Phase 60) → UX/lifecycle needs the adapter (Phase 61) → drift gate and doctor need something to check (Phase 62) → native-host is additive and can only close the offline cliff after the offline state itself exists (Phase 63) → contract must be stable before adapter breadth (Phases 64-65). - -## Phases - -**Phase Numbering:** -- Integer phases (57, 58, 59, 60, 61, 62, 63, 64, 65): Planned milestone work, continuing from v1.2.0's Phase 56 -- Decimal phases (57.1, 57.2): Urgent insertions (marked with INSERTED) - -- [x] **Phase 57: Agent Identity Capture** - Persist copy-clicks, capture MCP `initialize` `clientInfo`, thread it through `agent:register`, add disk-scan detection, expose a unified `getMcpClients` view — additive on both sides of the wire (INV-01 safe), unblocks everything downstream (completed 2026-07-12) -- [x] **Phase 58: Providers Panel** - Rename "API Configuration" → "Providers", introduce `api` vs `agent` provider kinds, hide the key input for agent kind, badge exactly one "Recommended" provider via the connected > installed > copy-clicked cascade, keep `universal-provider.js` unaware of agent values (INV-03 BYOK parity) (completed 2026-07-12; live UAT deferred to milestone-end gate) -- [x] **Phase 59: Reverse-Request Channel & Security Foundation** - **SECURITY-CRITICAL, load-bearing**. Additive `ext:*` frames on ws://localhost:7225, strict Origin allowlist + Host loopback + per-install rotating shared secret in `Sec-WebSocket-Protocol`, log redaction, hub-exit-mid-delegation topology tests, permanent CI grep gate against `--dangerously-skip-permissions` / `--yolo` / `--auto` — completed 2026-07-14; automated/source verification passed and four live checks are deferred to the milestone-end UAT gate -- [x] **Phase 60: Adapter Contract & Claude Code MVP** - `AgentProviderAdapter` interface, `SpawnSupervisor` in the `serve` daemon with argv-only spawn / scrubbed env / SIGTERM-at-process-group / Windows `taskkill /T /F` / orphan scan on startup, Claude Code adapter with verified 2.1.177 flag set + shipped `fsb` agent definition + recorded stream-json JSONL fixture — completed 2026-07-14; automated/source verification passed and seven live checks are deferred to the milestone-end UAT gate -- [x] **Phase 61: Delegation UX & SW-Eviction Persistence** - Fifth `EXECUTION_MODES` entry `delegated`, explicit first-use consent card, live per-tool-call streaming feed, default-background-tab + "Take control" affordance, kill switch that reclaims owned tabs, post-run usage summary, `chrome.storage.session` per-event persistence, 20 s WS heartbeat, "agent offline → `doctor`" deep-link, restart-is-clean semantics (completed 2026-07-15; automated/source verification passed and eight live checks are deferred to the milestone-end UAT gate) -- [x] **Phase 62: CI Drift-Smoke Gate & Doctor Extensions** - Per-adapter CI drift-smoke against canned fixtures (fail-loud on unknown event types / missing fields / version outside compat matrix), `fsb-mcp-server doctor` per-adapter section (binary path, version, auth, secret rotation age), machine-readable adapter compatibility matrix consumed by the extension (completed 2026-07-16; automated/source/full-suite verification passed and three live checks are deferred to the milestone-end UAT gate) -- [ ] **Phase 63: Native-Messaging Host** - `install --native-host` writes the platform-appropriate manifest (mac/Linux/Windows), extension gains additive `nativeMessaging` permission, "Agent offline" state auto-attempts wake before the doctor deep-link; the native host only launches (or attaches to) `serve` — it NEVER spawns agent CLIs directly (all spawn authority stays inside the daemon behind Phase 59 CHAN gates) -- [ ] **Phase 64: OpenCode Adapter** - Second adapter proves the contract accommodates server-mode + attach on top of cold spawn without any Phase 60 rewrite; pinned OpenCode agent definition + recorded JSONL fixture + drift-smoke coverage -- [ ] **Phase 65: Codex Adapter** - Third adapter with `codex exec --json` on the verified 0.142.5 hermetic flag set (`--ephemeral` + `--ignore-user-config`, never the deprecated `--full-auto`), `detect()` surfacing ChatGPT OAuth / API key / unauthenticated so the Providers panel discloses which billing bucket a run will hit; `caps.chatMode: false` across all adapters (automated/source verification passed 2026-07-22; exactly three live UAT scenarios remain pending) - -## Phase Details - -### Phase 57: Agent Identity Capture -**Goal**: FSB knows which MCP-capable agent CLIs the user has installed on disk, has expressed intent to install (via onboarding copy clicks), and has actually connected via MCP `initialize` — surfaced as a single ground-truth view that unblocks every downstream provider-selection decision. -**Depends on**: Nothing (first phase of this milestone; pure additive data layer on both sides of the wire) -**Requirements**: IDENT-01, IDENT-02, IDENT-03, IDENT-04, IDENT-05 -**Success Criteria** (what must be TRUE): - 1. When the user clicks any copy-to-clipboard button on the onboarding MCP-install screen (for `claude-code`, `cursor`, `vscode`, `windsurf`, `codex`, `opencode`, `openclaw`, `claude-desktop`, or `all`), FSB records that client id (with timestamp, deduplicated, all-clients aggregated for multi-select cases) into a durable `fsbAgentProviders.clicked` list in `chrome.storage.local` that survives service-worker eviction and Chrome restart. - 2. When any MCP client completes its `initialize` handshake with `fsb-mcp-server` (over stdio, streamable-HTTP, or the ws://7225 bridge), FSB captures the caller's `clientInfo.name` and `clientInfo.version`, threads them through an additive field on the existing `agent:register` bridge payload, stamps them onto the live `AgentRecord` in the registry, and rolls the identity up into a durable `fsbAgentProviders.connected` entry keyed so re-connections update rather than duplicate. - 3. `fsb-mcp-server` can enumerate installed MCP-capable clients on the current machine by inspecting the paths already known to `platforms.ts` (per-OS `configPath` for file-mode clients; `claude --version` binary probe for cli-mode `claude-code`) and report each as `installed` / `not-installed` with any parseable version. - 4. A single `getMcpClients` extension runtime message returns a merged `clicked ∪ installed ∪ connected` view with per-client status, so UI surfaces read one consistent structure instead of assembling it themselves — and the merged view survives MV3 service-worker eviction. - 5. INV-01 holds: no existing `MCPMessageType` value, no existing tool schema, and no existing consumer of the `agent:register` payload breaks — the additive `clientInfo` field is optional and existing `payload: {}` handlers keep working byte-for-byte. -**Plans**: 3/3 complete -- [x] 57-01: Capture lazy MCP client identity and daemon-side installed-client inventory. -- [x] 57-02: Persist durable clicked, connected, and installed evidence in the extension. -- [x] 57-03: Expose the canonical merged MCP-client evidence view with compatibility locks. - -### Phase 58: Providers Panel -**Goal**: The control panel's "API Configuration" section becomes the "Providers" panel — distinguishing BYOK API providers from installed agent CLIs, badging exactly one provider "Recommended" from ground truth (never auto-switching selection), and shipping honest no-fabrication cost/usage copy for agent-kind providers. -**Depends on**: Phase 57 (needs real `fsbAgentProviders` data to render agent rows and to drive the "Recommended" cascade) -**Requirements**: PROV-01, PROV-02, PROV-03, PROV-04, PROV-05, PROV-06 -**Success Criteria** (what must be TRUE): - 1. The control panel section formerly labeled "API Configuration" is labeled "Providers" (heading, nav label, and the legacy `#api-config` anchor continues to work as a redirect to the new `#providers` anchor for existing bookmarks); the source-pin tripwire suite stays green from commit 1 of this phase. - 2. Every provider row has an explicit `providerKind` of either `api` (the existing 7 BYOK LLM providers) or `agent` (a locally installed agent CLI); the kind determines which fields render — an `agent`-kind selection hides the API-key input, key-URL hint, and per-model key-format hint, and shows install status, auth status (where the CLI exposes it), connection status, and a "uses your subscription — no API key needed" caption instead. - 3. Exactly one provider is badged "Recommended" per session, chosen by the strict ground-truth cascade (highest = a CLI currently connected via MCP `initialize`, next = a CLI installed on disk, next = a CLI whose copy button was clicked during onboarding, fallback = the existing xAI-default recommendation); the badge is advisory only — the user's selection is never auto-switched. - 4. `universal-provider.js` (the existing BYOK request builder) never observes an agent value: switching between an active agent provider and a BYOK api provider preserves the other's configuration, and INV-03 provider parity for the 7 BYOK providers holds unchanged. - 5. Cost/usage rows for `agent`-kind providers display token count, turn count, and duration alongside the label "included in your subscription", with a link to the vendor's current billing page — never a fabricated dollar amount and never the words "free" or "unlimited". -**Plans**: 3/3 complete -**UI hint**: yes - -### Phase 59: Reverse-Request Channel & Security Foundation -**Goal**: Extension → daemon reverse requests transit the existing `ws://localhost:7225` bridge with every documented 2025-2026 CVE-class defense enforced from commit one — Origin allowlist, Host loopback, per-install rotating shared secret, log redaction, additive-only wire evolution, and a permanent grep gate against the three "one-click RCE" flags. This phase is SECURITY-CRITICAL and load-bearing: it ships BEFORE any spawn code exists in Phase 60. -**Depends on**: Nothing (channel design is orthogonal to Phase 57/58 data + UI; can develop in parallel but MUST be code-green before Phase 60 starts) -**Requirements**: CHAN-01, CHAN-02, CHAN-03, CHAN-04, CHAN-05, CHAN-06, CHAN-07 -**Success Criteria** (what must be TRUE): - 1. A new `ext:request` / `ext:response` / `ext:event` bridge message-type family transports extension → daemon reverse requests over the existing ws://localhost:7225 bridge, and a byte-freeze regression test proves every existing `MCPMessageType` value, tool schema, and bridge payload stays byte-identical (INV-01 additive proof). - 2. A relay process advertising `capabilities: ['agent-spawn']` on `relay:hello` becomes the routing target for `ext:*` frames: the hub routes each `ext:request` locally when it is itself the daemon, forwards to the first `agent-spawn`-advertising relay when it is not, and replies `agent_provider_offline` when no supervisor is present — verified by extending `tests/mcp-bridge-topology.test.js` with new hub-exit-mid-delegation and relay-mid-`ext:*`-frame cases without breaking any existing hub-exit-promotion test. - 3. A fixture crafted with `Origin: https://evil.com` on the WebSocket upgrade is rejected before any handler runs; a second fixture with `Host: evil.com:7225` (even when the target actually resolves to 127.0.0.1) is rejected as DNS-rebind defense — the bridge accepts `ext:*` frames only when the WS upgrade carried an `Origin` matching a durable per-install `chrome-extension://` allowlist and a `Host` exactly `127.0.0.1` or `localhost` at the loopback port, and the daemon never binds `0.0.0.0`. - 4. A per-install ≥32-byte shared secret is provisioned once between the extension and the daemon, transported only in the `Sec-WebSocket-Protocol` upgrade header (never in URL, never in payloads, never in logs), rotated on daemon restart, and required on every `ext:*` frame — and `redactForLog` plus every tracked diagnostic ring-buffer sink strips any string matching the shared-secret token pattern, with a build-time drift gate that fails if a raw secret substring appears in any tracked log fixture. - 5. A permanent CI grep gate fails the build if the strings `--dangerously-skip-permissions`, `--yolo`, or `--auto` appear anywhere in `mcp/src/agent-providers/**`, so those "one-click RCE" flags can never enter the spawn path in any future patch. -**Plans**: 4/4 complete - -### Phase 60: Adapter Contract & Claude Code MVP -**Goal**: A shell-free, tree-killable `SpawnSupervisor` in the `serve` daemon spawns the user's installed `claude` CLI via a formal five-method `AgentProviderAdapter` contract — with a shipped `fsb` agent definition (never prompt-stuffing), `--strict-mcp-config` hermeticity, and a recorded stream-json JSONL fixture — so a user with Claude Code installed can send a side-panel message and observe the agent drive their live browser back through FSB's MCP tools with visible per-tool-call feedback. This phase is the integration payoff; the contract is designed for adapter breadth so OpenCode/Codex slot in without rework. -**Depends on**: Phase 58 (provider selection reads the Providers panel), Phase 59 (channel + all CHAN security foundations must be code-green before any spawn code exists) -**Requirements**: ADAPT-01, ADAPT-02, ADAPT-03, ADAPT-04, ADAPT-05, CLAUDE-01, CLAUDE-02, CLAUDE-03, CLAUDE-04 -**Success Criteria** (what must be TRUE): - 1. The `AgentProviderAdapter` TypeScript interface (`mcp/src/agent-providers/adapter.ts`) exposes exactly five methods (`detect()` → `{installed, version, authState, binary}`, `buildSpawn(task, ctx)` → `SpawnSpec`, `parseEvents(stream)` → `AsyncIterable`, `kill(child, {grace})` → `Promise`, `caps()` → `AdapterCapabilities`), and the Claude Code adapter (`mcp/src/agent-providers/claude-code.ts`) implements all five and is registered against the `INSTALL_CLIENTS` / `PLATFORMS` client id `claude-code`. - 2. A user with Claude Code installed can select "Claude Code" as their provider in the side panel, type a task, and observe the spawned CLI drive their live browser back through FSB's MCP tools — with visible per-tool-call feedback — while every existing FSB MCP tool call remains byte-identical on the wire (INV-01 preserved end-to-end through the delegation round-trip; the byte-freeze regression test still passes). - 3. The `SpawnSupervisor` accepts a validated spawn request over Phase 59's `ext:request` channel, looks up the requested adapter, constructs argv from adapter output plus a daemon-controlled flag allowlist (unknown payload keys rejected with a typed error), and spawns the child with `{ detached: true, windowsHide: true, stdio: ['pipe','pipe','pipe'] }` and an environment with `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and `GEMINI_API_KEY` scrubbed; the supervisor never invokes a shell, and the user's prompt reaches the CLI only via `child.stdin` (never argv, never `sh -c`) so shell metacharacters and Windows `.cmd`-shim EINVAL (Node CVE-2024-27980) cannot execute. - 4. A `stop` / cancel request triggers SIGTERM at the process-group level on POSIX (`process.kill(-child.pid, 'SIGTERM')` after `spawn({detached:true})`), `taskkill /pid /T /F` on Windows, escalates to SIGKILL after a grace window, and does not resolve the delegation until either an exit-signal is observed or the daemon confirms no descendant matches remain — grandchildren, MCP sub-servers, and shell tools spawned by the CLI are all terminated. - 5. On daemon startup, the supervisor scans for orphaned children matching prior adapter fingerprints (env-var tag + argv pattern) and kills them before accepting new spawn requests, and reports the scan result in structured diagnostics — so a daemon crash mid-run cannot leave a ghost CLI controlling the browser. - 6. The Claude Code adapter spawns `claude -p --verbose --output-format stream-json --include-partial-messages --strict-mcp-config --mcp-config --agents --agent fsb --permission-mode dontAsk --allowedTools "mcp__fsb" --disallowedTools "Bash,Edit,Write,NotebookEdit,WebFetch,WebSearch" --max-turns 40 --no-session-persistence` (or the version-appropriate equivalent selected by `detect()`); `parseEvents` translates the CLI's stream-json events into a normalized `AgentEvent` schema, fails loud on unknown event types (surfaced as `agent_protocol_drift` diagnostic), and is covered by a recorded JSONL fixture under `tests/fixtures/agent-streams/claude-code-2.1.177/` so CI runs without a live CLI; `detect()` fingerprints via `claude --version`, compares against a minimum supported version, and reports `installed=false` with a doctor-diagnostic message rather than spawning if the version predates the verified stream-json contract. -**Plans**: 4/4 complete -- [x] 60-01: Freeze the exact adapter contract, Claude detection/profile, and shipped FSB agent policy. -- [x] 60-02: Normalize bounded Claude JSONL through the concrete adapter with truthful offline fixture provenance. -- [x] 60-03: Build private runtime state plus verified POSIX/Windows termination and orphan recovery. -- [x] 60-04: Wire the serve-only supervisor, exact-once reverse delegation, full regression gates, and deferred UAT ledger. - -### Phase 61: Delegation UX & SW-Eviction Persistence -**Goal**: The side panel becomes a first-class delegation surface — visible first-use consent gate, live per-tool-call streaming feed, default-background-tab with a "Take control" affordance, kill switch that reclaims owned tabs, honest post-run usage summary, and MV3 SW-eviction-safe persistence so a 45-minute run resumes exactly the feed the user last saw. The extension has no `nativeMessaging` permission at this point — this phase ships the honest "Agent offline → `doctor`" state that Phase 63 later augments with an optional wake path. -**Depends on**: Phase 60 (adapter contract + Claude Code MVP producing the `AgentEvent` stream + tab ownership on spawn); Phase 59 heartbeat plumbing -**Requirements**: UX-01, UX-02, UX-03, UX-04, UX-05, UX-06, LIFE-01, LIFE-02, LIFE-03, LIFE-04 -**Success Criteria** (what must be TRUE): - 1. A fifth `EXECUTION_MODES` entry `delegated` in `extension/ai/engine-config.js` (with `uiFeedbackChannel: 'popup-sidepanel'`, `animatedHighlights: true`, wall-clock + event-silence watchdogs, and no iteration cap — the reasoning loop runs in the spawned CLI, not `runAgentLoop`) is selected automatically whenever the active provider is agent-kind, and the extension's own `runAgentLoop` is never entered for delegated tasks. - 2. Before FSB spawns any agent CLI for the first time, the user sees an explicit consent card that names the CLI, states what it will be permitted to do (drive FSB MCP tools on the user's live browser), and states what it will not (edit files, run shell, fetch arbitrary URLs); a per-run confirm-to-continue toggle is on by default and can be disabled per-provider only via an explicit "trust this agent" setting — the copy never uses "faster mode" language. - 3. The side panel renders a live per-run streaming feed with distinct card types for `init` (client, model, session id, allowed tools), `tool-call` (name, args summary, tab id), `retry` (typed error class), and `result` (usage summary) driven by the normalized `AgentEvent` stream; a delegated run opens by default in a new background tab, and when the user activates the tab the agent is driving, a persistent "Take control" affordance appears that pauses the agent (v0.9.60 ownership release + supervisor grace hold), lets the user interact, and offers "Resume with agent" to give ownership back. - 4. A prominent Stop button in the side panel triggers `stopDelegatedTask`, routes to the supervisor's kill, and — on confirmed exit — releases every tab that was owned by the spawned agent (per v0.9.60 ownership) and reports "Agent stopped, N tab(s) released" in the feed; a post-run summary card displays tokens (in/out/total), turn count, wall-clock duration, cost bucket (`included in your subscription` for agent kind, real USD for api kind), and an expandable per-tool-call breakdown. - 5. Every progress event received from the supervisor is written to `chrome.storage.session` under a per-delegation key BEFORE it fans out to UI subscribers, so a mid-run MV3 service worker eviction reloads exactly the delivered feed on re-open — no ghost state, no fabricated events, no dropped events; the fifth `EXECUTION_MODES` entry works end-to-end across a forced SW eviction in a test fixture. - 6. While a delegation is active, the extension pings the bridge every 20 s over the existing WS heartbeat channel to keep the Chrome 116+ SW-lifetime extension applied; if 3 heartbeats are missed the side panel switches to a `daemon:disconnected` fallback that offers a doctor-relaunch button but does not attempt an in-extension restart; if `fsb-mcp-server serve` is not running when a delegated send is attempted, the side panel shows an "Agent offline" state with a deep-link to `fsb-mcp-server doctor` output and does not enqueue or optimistically show the message; if the daemon restarts while a delegation was mid-flight, the supervisor kills any surviving spawned CLI (never re-adopts) and reports `daemon_restart_lost_run` in the side panel so the user knows the run ended. -**Plans**: 8/8 complete; automated/source verification passed; eight live UAT checks deferred to the milestone-end gate -- [x] 61-01: Freeze exact agent routing, delegated mode, read-only preflight, one-use consent, and provider-local trust. -- [x] 61-02: Build the bounded write-before-fanout event ledger and sole delegated lifecycle controller. -- [x] 61-03: Serialize async bridge observers, add acknowledged active-delegation heartbeat, and pin Chrome 116/no-native authority. -- [x] 61-04: Correlate server delegation ids with extension agents and add sealed complete mapped-tab hold leases. -- [x] 61-05: Add supervisor hold/resume/status plus generation-backed restart-loss evidence. -- [x] 61-06: Integrate start, handoff, Stop, hydration, heartbeat, and recovery in the service worker. -- [x] 61-07: Implement the safe, exact-copy, accessible delegated side-panel feed and lifecycle UI. -- [x] 61-08: Install complete regression/evidence gates and defer genuine live UAT to the milestone-end ledger. -**UI hint**: yes - -### Phase 62: CI Drift-Smoke Gate & Doctor Extensions -**Goal**: An `fsb-mcp-server doctor` operator and CI both catch adapter drift the moment an agent CLI ships a new flag or event-shape — with a machine-readable adapter compatibility matrix the extension consumes to render "supported / degraded / unsupported" states without hardcoding versions anywhere in extension source. -**Depends on**: Phase 60 (needs at least one adapter to run drift-smoke against; Phase 61 UX consumes the compatibility matrix for rendering degraded states) -**Requirements**: DRIFT-01, DRIFT-02, DRIFT-03, DRIFT-04 -**Success Criteria** (what must be TRUE): - 1. A CI job runs each shipped adapter against a canned prompt fixture, asserts the known event-type sequence and the presence of required fields on `system/init` and `result`, and fails the build on unknown event types, missing required fields, or a `--version` outside the compatibility matrix — with no live CLI required (the recorded JSONL fixtures from Phases 60/64/65 suffice), so contributors without every CLI installed can still land safe changes. - 2. `fsb-mcp-server doctor` gains a per-adapter section reporting: binary path, version, auth state (parseable where the CLI exposes it), shared-secret presence, and the current spawn-secret rotation age — so an operator with only `doctor` output can identify which adapter failed and why. - 3. The diagnostics ring buffer classifies drift events as `agent_protocol_drift` with adapter id, expected vs observed fields, and rate-limits duplicate entries at the existing 1-per-10s bucket so a chatty drift does not blow the buffer. - 4. `doctor` emits a machine-readable adapter compatibility matrix that both the CI drift-smoke job and the extension can read; the extension consumes the matrix at boot (and on doctor refresh) to render `supported` / `degraded` / `unsupported` badges in the Providers panel — and never hardcodes CLI versions in extension source. -**Plans**: 6 plans -- [x] 62-01: Establish the canonical adapter compatibility matrix/classifier and generalized offline drift-smoke CI gate. -- [x] 62-02: Extend `doctor` with offline adapter diagnostics, safe spawn-secret metadata, and text/JSON parity. -- [x] 62-03: Carry the authenticated safe compatibility projection into background-owned durable, freshness-aware provider state. -- [x] 62-04: Project sanitized protocol-drift detail and report it exactly once through a true per-adapter pre-throttle. -- [x] 62-05: Render exact, accessible, non-mutating compatibility badges and selected-provider details. -- [x] 62-06: Wire root/source/security gates and preserve all genuine live evidence in the pending milestone-end UAT ledger. - -### Phase 63: Native-Messaging Host -**Goal**: When the user has installed the optional native-messaging host, the "Agent offline" state auto-attempts to wake `fsb-mcp-server serve` before falling back to the Phase 61 doctor deep-link — closing the UX cliff introduced by the extension having no `nativeMessaging` permission through Phases 57-62. All spawn authority stays inside the serve daemon behind Phase 59's CHAN gates; the native host itself never spawns agent CLIs. -**Depends on**: Phase 61 ("Agent offline → doctor" state must already exist for the wake to have a fallback), Phase 62 (doctor extensions already report daemon state) -**Requirements**: NATIVE-01, NATIVE-02, NATIVE-03, NATIVE-04 -**Success Criteria** (what must be TRUE): - 1. `fsb-mcp-server install --native-host` writes the platform-appropriate native-messaging host manifest — macOS `~/Library/Application Support/Google/Chrome/NativeMessagingHosts/`, Linux `~/.config/google-chrome/NativeMessagingHosts/`, Windows registry `HKCU\Software\Google\Chrome\NativeMessagingHosts\` — allowing the FSB extension id as the sole caller, and installs a host binary that can wake the daemon on demand. - 2. The extension's manifest gains a single additive `nativeMessaging` permission entry (no other permission changes); at boot the extension detects native-host presence, and when the host is installed, an "Agent offline" state auto-attempts a wake before showing the doctor deep-link — a user with the host installed sees the delegation come alive on demand instead of the manual `serve` prompt. - 3. The native host itself does NOT spawn agents; it only starts `fsb-mcp-server serve` (or attaches to a running one) and exits after handoff — all spawn authority remains inside the serve daemon behind the Phase 59 CHAN gates (Origin allowlist, shared secret, flag allowlist, argv-only) and no CHAN requirement is relaxed to accommodate the wake path. - 4. `fsb-mcp-server uninstall --native-host` cleanly removes the manifest, and `doctor` reports native-host install state including manifest path, allowlist mismatches, and host-binary reachability so an operator can debug wake failures without shell-level inspection. -**Plans**: 12 plans +- ✅ **v0.13.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v0.13.0-ROADMAP.md`. +## Current Milestone: v1.0.0 Jev Fast Mode + +**Status (2026-10-19):** code-complete; Validation Block V1-V6 pending. Runbook: `.planning/v1.0.0-VALIDATION-BLOCK.md` (the section 1 owner decisions come first, with M-71-00 as the hard gate before V1). + +**Milestone goal:** Make FSB measurably faster on complete autopilot and MCP tasks by delegating bounded simple decisions to Jev while preserving the LLM's planning and writing roles and FSB's existing action and safety boundaries. + +**Scope:** 25 requirements in `.planning/REQUIREMENTS.md`, across Phases 66-71 plus the inserted Phase 66.1; numbering continues after the completed v0.9.91 milestone. The existing Phase 999.1 is a completed historical backlog item outside this milestone. + +**Compatibility:** With Jev disabled, FSB behaves exactly as it did before this milestone (owner direction, 2026-09-29). + +**Provider decision:** FSB supplies Jev to every installation without user sign-in or usage quotas for now. The upstream credential stays on `showcase/server/` (or its server-side secret store); the extension never receives it. A bounded HTTPS gateway carries redacted decision state and monitors aggregate usage and cost. The only fast-mode setting is the control-panel Advanced Settings switch, default off. MCP `run_task` follows it, and the MCP settings tool (Phase 66.1) reads and writes that same value. The server round trip is included in all speed measurements. + +**Validation Block (owner direction 2026-10-15):** every human, live-account, production, or spend validation in this milestone is deferred to one block at the end (V1 baseline at the pinned pre-milestone commit `8a058bdf`, V2 Jev gateway deploy, V3 shadow calibration, V4 comparative gate, V5 human UAT, V6 release). Phases are built and verified with automated checks only. See `.planning/STATE.md` → Decisions. + +**Release evidence:** Compare whole-task completion, wall time per completed task, and cost on the same task set. The project brief's 1.5-2x autopilot and 1.3-1.6x MCP speedups are targets to measure, not results already achieved. A path that fails its predeclared gate remains experimental or disabled. + +### Phase 66: Baseline and Session Timing + +**Goal:** Establish trustworthy timing and task-outcome records before any Jev path changes execution. +**Depends on:** None. +**Requirements:** MEAS-01, MEAS-02, MEAS-03. +**Success Criteria:** + +1. An operator can see per-call wall, FSB execution, and model time, plus tokens and cost where available, for MCP and autopilot sessions; absent values are explicit. +2. Journal-backed MCP sessions resolve to session details with tool-call timing rather than an empty or broken lookup. +3. A reproducible suite runs representative mixed and click-heavy tasks, verifies outcomes from page evidence, and saves completion, wall-time, and cost baselines before fast mode is enabled. +4. The suite records enough task and environment identity to compare later runs without counting Jev's own completion answer as proof. + +**Plans:** 11 plans in 6 waves + +Plans: +**Wave 1** + +- [x] 66-01-PLAN.md — Export the 2026-09-16..21 MCP sessions (H1) and commit sanitized history metrics (wave 1, human) (closed 2026-10-12: source data lost, not executed) +- [x] 66-02-PLAN.md — Install the worktree, record the start-commit test and regression-sweep baselines, add bench statistics, outcome and identity libraries (wave 1) + +**Wave 2** *(blocked on Wave 1 completion)* + +- [x] 66-03-PLAN.md — MCP call timing at ingress through recorder and journal, text-export timing, allowlisted fsb://config identity (wave 2) +- [x] 66-04-PLAN.md — Autopilot model and tool timing, provider-first cost labels, grok-4.3 price rows, history persistence (wave 2) +- [x] 66-05-PLAN.md — Bench task schema, loopback fixture server, Claude Code and Cursor stream parsers (wave 2) + +**Wave 3** *(blocked on Wave 2 completion)* + +- [x] 66-06-PLAN.md — Journal-backed get_session_detail with sanitized timing rows and a filter-proof measurement block (wave 3) +- [x] 66-07-PLAN.md — Frozen 12-task core corpus (6 mixed, 6 click-heavy), live reading set, corpus lock and self-check (wave 3) +- [x] 66-08-PLAN.md — Benchmark runner (preflight, ABBA schedule, per-run saves, CDP pairing), arm drivers, baseline report (wave 3) + +**Wave 4** *(blocked on Wave 3 completion)* + +- [x] 66-09-PLAN.md — Register bench suites in the test chain; full-chain, Node 20, timing and mcp/ audits (wave 4) + +**Wave 5** *(blocked on Wave 4 completion)* + +- [ ] 66-10-PLAN.md — Bench profile setup (H2/H3), CDP pairing, preflight, all-task smoke and live smoke, live operator timing check (wave 5, human) (deferred to Validation Block V1; run at `8a058bdf`) + +**Wave 6** *(blocked on Wave 5 completion)* + +- [ ] 66-11-PLAN.md — Approved baseline campaign (H4), committed sanitized results, restore (H5) (wave 6, human) (deferred to Validation Block V1; run at `8a058bdf`) + +### Phase 66.1: MCP Settings Tool (INSERTED) + +**Goal:** Let an MCP client view FSB's control-panel settings and change one setting at a time through a single new tool, without exposing secrets or changing behavior for existing callers or autopilot. +**Depends on:** Phase 66 (the baseline records the MCP tool list before this phase changes it). +**Requirements:** SET-01, SET-02. +**Status:** Code-complete 2026-10-15; human/live checks deferred to the Validation Block (V5 UAT incl. the WR-02/WR-04 owner decision, V1 digest cross-check, V6 release; see `66.1-HUMAN-UAT.md`). +**Success Criteria:** + +1. One new MCP tool returns every control-panel setting as compact key-value data in a single call, and changes one setting by key in a single call. +2. Secret values (API keys, hash keys, and credential and payment data) are masked on read and refused on write. Every other setting is writable, as the owner chose on 2026-09-29. +3. A write goes through the control panel's own validation and storage, so the control panel, autopilot, and `run_task` all see the new value. An invalid key or value returns a typed error and changes nothing. +4. The Jev fast-mode switch becomes reachable through the same tool once Phase 68 adds it. It stays one stored value shared with Advanced Settings. +5. Only MCP clients get the tool. The autopilot model's tool list and prompt are unchanged, and existing tool names, arguments, and responses stay byte-stable. + +**Open questions for discussion:** + +- Should endpoint settings that receive API keys or page data (custom endpoint, sync server URL, LM Studio URL) count as secrets? +- Should settings changed over MCP be logged for the operator? + +**Open questions auto-resolved (66.1-CONTEXT, 2026-10-15):** (a) yes, for writing: `customEndpoint`, `lmstudioBaseUrl` and `serverUrl` are refused on write like secrets and read as origin only (D-11); (b) yes: every accepted or refused MCP write is logged to `automationLogs` (key, previous/new value or error code, client label, time; never a refused value) independent of session recording, and journaled (D-18). + +**Plans:** 8 plans in 8 waves (serialized: one shared working tree; every plan runs an MCP build or a regression sweep) + +Plans: **Wave 1** -- [x] 63-01: Establish the workspace-safe build guard, release-built Windows bootstrap, and durable bundled runtime contract. + +- [x] 66.1-01-PLAN.md — Pre-edit sweep and tools/list digest baselines; FsbSettingsSchema in config.js with masked reads, origin-only endpoints, effective defaults and strict parsing (wave 1) **Wave 2** *(blocked on Wave 1 completion)* -- [x] 63-02: Fix pre-bind secret rotation and make product-specific serve readiness authoritative. + +- [x] 66.1-02-PLAN.md — Write planner with D-15 provider group rules, shared DEFAULT_MODELS and a value-free Allowed clause in invalid-value errors; inventory, default and bounds parity with the control panel (wave 2) **Wave 3** *(blocked on Wave 2 completion)* -- [x] 63-03: Implement the closed native-messaging framing, validation, one-shot entry, and authority boundary. + +- [x] 66.1-03-PLAN.md — MCP surface: two additive wire types, SETTING_ error passthrough under a Settings label, the settings tool with a value-safe write preflight, runtime wiring and smoke (wave 3) **Wave 4** *(blocked on Wave 3 completion)* -- [x] 63-04: Implement exact readiness probing, tokened wake coalescing, and one shell-free serve handoff. + +- [x] 66.1-04-PLAN.md — Control-panel sync (FSBSettingsSync): touched tracking, no-network in-place refresh of simple fields and the provider group, and a Save-time filter so MCP writes are never reverted (wave 4) **Wave 5** *(blocked on Wave 4 completion)* -- [x] 63-05: Implement exact-owned atomic registration, runtime materialization, install, and uninstall across three OS families. + +- [x] 66.1-05-PLAN.md — Service-worker settings routes, write lock, fail-closed delegated-write refusal, operator log lines, bridge routing and icon class (wave 5) **Wave 6** *(blocked on Wave 5 completion)* -- [x] 63-06: Add explicit native-host CLI install/uninstall routes without widening existing installer behavior. + +- [x] 66.1-06-PLAN.md — Recording hygiene: request/response sanitizers, journal projection, sentinel sweep across every sink, replay classification (wave 6) **Wave 7** *(blocked on Wave 6 completion)* -- [x] 63-07: Add bounded read-only native-host diagnostics with safe local and browser projections. + +- [x] 66.1-07-PLAN.md — Built-runtime contract: tool shape, 8a058bdf tools, prompt and resource digests, MCP-edge rules, end-to-end round trip, SC4 fixture, delegated refusal (wave 7) **Wave 8** *(blocked on Wave 7 completion)* -- [x] 63-08: Add the single manifest permission and background-owned probe/offline-wake controller. -**Wave 9** *(blocked on Wave 8 completion)* -- [x] 63-09: Render the approved intent-fenced checking state in the existing delegation card. +- [x] 66.1-08-PLAN.md — Phase gate: full chain, MCP smoke, Node 20, regression sweep, mcp/ footprint and byte-identity audits, VALIDATION.md results (wave 8) + +### Phase 67: Hosted Jev Gateway, Policy, and Shadow Mode + +**Goal:** Provide version-pinned Jev through an FSB-owned server credential and a bounded, installation-associated HTTPS decision path; validate safe inputs, accounting, fallback, and calibration before live control. +**Depends on:** Phase 66 (baseline and labelled task records). +**Requirements:** JEV-01, JEV-02, JEV-03, JEV-04, JEV-05, JEV-06, JEV-07. +**Status:** Code-complete 2026-10-15; human/live checks deferred to the Validation Block (V2 secrets, branch deploy, switch flip, live smoke, real error mapping and the IN-04/IN-05 owner decision; V3 live shadow calibration; V4 default-off gate; V5 WR-06 and IN-02 owner decisions; V6 release; see `67-HUMAN-UAT.md`). +**Success Criteria:** + +1. The FSB server holds the upstream Jev credential in server-side secret storage, calls a pinned model, and returns typed decisions; no extension bundle, message, response, or storage record contains that credential, and users enter no Jev or OpenRouter key. +2. The HTTPS decision route is available to every install without sign-in or usage quotas. It validates a fixed bounded schema, limits body size and concurrent upstream work, records aggregate usage and FSB-borne cost, and supports an operator-wide disable switch. The existing self-issued hash key may identify a caller but is not treated as a secret protecting the Jev credential. +3. Extension and server validation exclude secrets and sensitive form values, treat page content as data, and avoid raw request-body or credential logging across success and error paths. +4. Timeout, rate-limit, invalid-response, unavailable-model, and key errors produce typed fallback results; actual model version, usage, and FSB-borne cost are recorded where supplied, without fabricated values. +5. Shadow runs on labelled tasks produce operation, target, completion, stuck, and risk accuracy by confidence bucket; live thresholds are selected from that evidence and tied to the pinned model. Tests prove Jev cannot remove confirmation or bypass existing safety breakers. + +**Plans:** 13 plans in 5 waves (all autonomous; live deploy, secret and calibration deferred to Validation Blocks V2/V3) -**Wave 10** *(blocked on Wave 9 completion)* -- [x] 63-10: Wire deterministic focused/root/CI contracts and create the still-pending milestone-end UAT ledger. +Plans: +**Wave 1** + +- [x] 67-01-PLAN.md — Server Jev contract (19-reason fallback enum, key sets, caps), mirrored secret scrubber + injection filter, exact-key request validator, shared fixtures (wave 1) +- [x] 67-02-PLAN.md — Shadow scoring by confidence bucket, predeclared Bonferroni/Wilson threshold selection, committed targets.json (wave 1) + +**Wave 2** *(blocked on Wave 1 completion)* + +- [x] 67-03-PLAN.md — Server-owned versioned instrument with byte budget, calibration profiles and the explicit UNCALIBRATED default (wave 2) +- [x] 67-04-PLAN.md — Boot-time config (route + pin as configuration, operator switch), single-attempt upstream client with D-16 classification, loopback fake upstream (wave 2) +- [x] 67-05-PLAN.md — Body-free SQLite ledger and daily aggregate, housekeeper retention, read-only report CLI (wave 2) +- [x] 67-06-PLAN.md — FsbJevState: mirrored contract with parity test, gateway-response validator, sanitizer and budgeted request builder (wave 2) +- [x] 67-07-PLAN.md — FsbJevPolicy tighten-only lattice with exhaustive tables, real breaker invariance and HookPipeline ordering proof (wave 2) + +**Wave 3** *(blocked on Wave 2 completion)* + +- [x] 67-08-PLAN.md — FsbJevClient.decide(): never-rejecting fixed-endpoint client, typed fallbacks, abort/deadline, in-memory cool-down (wave 3) +- [x] 67-09-PLAN.md — /api/jev/decide router, server.js and global-parser wiring, crash-resistance and log-hygiene proofs (wave 3) +- [x] 67-10-PLAN.md — Labelled shadow step-record format and 25-record seed set over the 12 core fixtures (wave 3) -**Wave 11** *(blocked on Wave 10 completion)* -- [x] 63-11: Run independent blocking code, security, and UI source-review gates. +**Wave 4** *(blocked on Wave 3 completion)* -**Wave 12** *(blocked on Wave 11 completion)* -- [x] 63-12: Run the reviewed focused matrix and workspace-preserving repository-wide regression gate. +- [x] 67-11-PLAN.md — Inert importScripts loads with paired lattice pin bump, extension boundary suite (wave 4) +- [x] 67-12-PLAN.md — Gateway-only shadow harness, offline end-to-end run, profile round-trip, V2 smoke command (wave 4) -### Phase 64: OpenCode Adapter -**Goal**: The `AgentProviderAdapter` contract proves it accommodates a second CLI family — one that supports both cold spawn AND attach-to-running-server (`opencode serve` + `opencode run --attach`) — without any Phase 60 rewrite; OpenCode joins the Providers panel with a pinned agent definition and CI-covered event schema so the drift gate covers it from day one. -**Depends on**: Phase 60 (contract), Phase 62 (drift-smoke gate must accept the new adapter fixture) -**Requirements**: MULTI-01, MULTI-02, MULTI-03 -**Success Criteria** (what must be TRUE): - 1. An OpenCode adapter (`mcp/src/agent-providers/opencode.ts`) implements the `AgentProviderAdapter` contract with `caps.serverMode = true`; the supervisor either spawns `opencode run` cold OR attaches to a running `opencode serve` per the adapter's `buildSpawn` output — and the Phase 60 contract handles both without any hardcoded spawn-vs-attach branch outside the adapter. - 2. The OpenCode adapter ships a pinned agent definition (equivalent to Claude Code's `--agents fsb`) using OpenCode's `agent create` / `agents` config surface, keyed to a version pinned during phase spike, so tool boundaries and system-prompt intent are identical across the two adapters. - 3. A recorded OpenCode JSONL fixture under `tests/fixtures/agent-streams/opencode-/` proves the adapter's event schema in CI without a live CLI; the Phase 62 drift-smoke job includes OpenCode from the first commit of this phase, and unknown event types raise `agent_protocol_drift` with the adapter id `opencode`. Planning-document commits do not count as implementation commits: Plan 64-01 has exactly one implementation task/commit containing the complete parser, fixture, native negative corpus, and generalized drift gate; every later plan depends on it transitively, and Plan 64-05 exposes adapter/registry/matrix/bijection atomically. - 4. A user with OpenCode installed can pick "OpenCode" as their provider in the side panel and observe the delegation UX (streaming feed, kill switch that reclaims tabs, post-run summary, SW-eviction survival) work identically to Claude Code — with no adapter-specific side-panel branches. -**Plans**: 13 plans +**Wave 5** *(blocked on Wave 4 completion)* +- [x] 67-13-PLAN.md — Register the 14 Jev suites in the test chain and refinement contract; full-chain, Node 20 and compatibility audits (wave 5) + +### Phase 68: Autopilot Fast Mode + +**Goal:** Add opt-in, bounded Jev decisions for simple autopilot steps while the LLM handles planning, text, reading, and uncertain work. +**Depends on:** Phase 67 (safe provider and calibrated thresholds). +**Requirements:** AUTO-01, AUTO-02, AUTO-03, AUTO-04, AUTO-05. +**Status:** Code-complete 2026-10-17; human/live checks deferred to the Validation Block (V3 capture campaigns, calibration and the CR-01 eligibility decision; V4 comparative gate; V5 panel card, live fast steps, the WR-05/IN-04/WR-03 paths and the privacy translation review; V6 deployed privacy page, Web Store form, the WR-04/D-16 and IN-05 owner decisions and release; see `68-HUMAN-UAT.md`). +**Success Criteria:** + +1. A user can enable or disable fast mode in control-panel Advanced Settings or through the Phase 66.1 settings tool, which changes the same stored value; the default is off, the setting explains the bounded FSB-to-Jev page-context transfer, and an ordinary task with it off follows the existing autopilot path. +2. On a supported simple step, Jev chooses only among operations and current elements offered by FSB; FSB rechecks target freshness and executes through its existing action machinery. +3. Planning, generated text, extraction, unsupported steps, and below-threshold decisions route to the LLM without the Jev loop fabricating arguments or page understanding. +4. Completion and stuck decisions are checked against observable page evidence; action, request, and wall-clock budgets stop repeated or runaway loops with a truthful outcome. +5. Ambiguous, changed, or failed decisions escalate once to the existing path or stop as blocked, with no duplicated side effect or weakened confirmation. + +**Plans:** 12 plans +- [x] 68-01-PLAN.md — Open Phase 68 safely and land the switch. +- [x] 68-02-PLAN.md — Build the content-side half of the element menu: a separate Jev ref registry, vault-fill marks, and a value-free menu extractor with code-derived operations, a union-based risk classifier, an editable-region-free visible-text excerpt and a state-aware page signature. +- [x] 68-03-PLAN.md — Wire the content menu into the content message router: a `getJevMenu` case, a strict jev branch that runs before every other target-resolution branch in `executeAction`, and an armed, one-shot capture observation at the VIS-01 target-resolution point. +- [x] 68-04-PLAN.md — Create the pure core of `extension/ai/jev-fast-path.js` (global `FsbFastPath`): constants, fallback classification, the menu-to-request mapping, the decision evaluator built only on `FsbJevPolicy.accept`, the static call builder, and the evidence, hand-off text and measurement helpers that plan 68-05's orchestration uses. +- [x] 68-05-PLAN.md — Add the orchestration half of FsbFastPath: `beforeTurn` (eligibility, latches, budgets, menu request, request building, one Jev call, decision, hand-off flush, measurement), `afterStep` (evidence after a fast action), `observeTools` (what the LLM did on its own turns), `latchOff` and `abortInFlight`, all against injected dependencies so they are proven without the agent loop. +- [x] 68-06-PLAN.md — Wire the fast path into the running extension. +- [x] 68-07-PLAN.md — Prove the fast path's safety and truthfulness properties end to end through the real `runAgentIteration`, using the 68-06 harness, the fake gateway and a test-only calibrated result. +- [x] 68-08-PLAN.md — Build the network-free `recorded-llm` capture and its exporter so V3 bench runs produce realistic labelled step records: the service worker records what FSB would have offered Jev before each LLM turn and what the LLM actually did, and a bench script turns those records into the 67-10 record format. +- [x] 68-09-PLAN.md — Make Jev measurement visible and durable for V4 without changing anything for sessions that never used fast mode: persist the session's `jevMeasurement` block through the history store and expose it, plus a per-action decision-source marker, through `get_session_detail`. +- [x] 68-12-PLAN.md — Add the Advanced Settings disclosure card for the fast-mode switch (split out of plan 68-01 for context budget, and placed after the fast path exists so its copy is checked against implemented behaviour). +- [x] 68-10-PLAN.md — Disclose the opt-in page-context transfer on the public privacy page in every showcase locale, keeping the Policy History convention: first archive the current September 29, 2026 text, then add the Jev fast mode section, the qualified TLDR, an External Services bullet, a fast-mode Limited Use statement and the new date. +- [x] 68-11-PLAN.md — Run the Phase 68 gate: the full chain (tolerant mode if the hub holds 7225), validate:extension, Node 20 runs of every new suite, the pin, compatibility and registration audits, and the final 68-VALIDATION.md results with every human, live, production or spend check left as a tagged Manual-Only deferral. + +### Phase 69: MCP Descriptions and Same-Call Checks + +**Goal:** Cut MCP tool-list context and avoid follow-up verification calls through additive optional expectations. +**Depends on:** Phase 66 (tool-call measurement baseline) and Phase 66.1 (the settings tool, `mcp/src/tools/settings.ts` and its 8a058bdf tools/list freeze, which this phase migrates). +**Requirements:** MCP-01, MCP-02, MCP-04. +**Status:** Code-complete 2026-10-15; human/live checks deferred to the Validation Block (V4 exact token counts and expect calibration, V5 UAT incl. the WR-04/WR-05 owner decisions and the CR-02 acknowledgement, V1 digest cross-check, V6 release; see `69-HUMAN-UAT.md`). +**Success Criteria:** + +1. All existing tool descriptions (the 72 tools at `8a058bdf`, 73 with the Phase 66.1 settings tool) are shortened and a repeatable token-count check shows the aggregate tool-list reduction against the Phase 66 baseline while preserving action and safety guidance. +2. Supported action tools accept an optional `expect` and report verified, unmet, or uncertain effects within the same call using a bounded post-action observation. +3. Contract tests prove existing tool names, required arguments, default behavior, and serialized wire shapes remain unchanged when `expect` is omitted. +4. A representative MCP caller can complete an action and read its effect without an extra verification call when the outcome is observable. + +**Plans:** 8 plans in 6 waves + +Plans: **Wave 1** -- [x] 64-01-PLAN.md — Land the complete strict parser, honest 1.14.25 fixture, native negative corpus, and generalized adapter-native Phase 62 CI drift gate as the sole first implementation commit. + +- [x] 69-01-PLAN.md — Record $P69_START and the sweep baseline; freeze the 8a058bdf tools/list fixture with its git-free listing library and --write/--check generator; fixture-integrity, shape-freeze and report-only budget suites (wave 1) **Wave 2** *(blocked on Wave 1 completion)* -- [x] 64-02-PLAN.md — Freeze the exact five-method provider-neutral topology/attestation seam and the public fixed-env versus transient secret-binding contract. + +- [x] 69-02-PLAN.md — MCP-only description module, jsonSchemaToZod text override, short text for the 37 manual action tools, per-tool 2,048 cap on, Phase 66.1 freeze migration (wave 2) +- [x] 69-03-PLAN.md — Extension expect contract: constants, spec normalization, read-only page probe and bounded evaluation loop in action-verification.js; sensitive values redacted in change_report.inputs_changed (D-25, the recorded MCP-04 security exception) (wave 2) **Wave 3** *(blocked on Wave 2 completion)* -- [x] 64-03-PLAN.md — Version the exact delegation/provider-server runtime journal and private-artifact boundary with legacy recovery and structural secret non-retention preserved. + +- [x] 69-04-PLAN.md — Remaining descriptions (read-only, trigger, screenshot, vault, capability, back, observability, autopilot, visual-session, settings), then a gate-only commit: full coverage, must-keep table, D-06 budget floors on (wave 3) +- [x] 69-05-PLAN.md — Dispatcher wiring: ISOLATED probes, commit-gated navigation (back included), back-route hook, bridge-client forwarding, expect stripped from journal and replay payloads (wave 3) **Wave 4** *(blocked on Wave 3 completion)* -- [x] 64-04-PLAN.md — Build exact OpenCode 1.14.25 retained detection, private policy, effective-attestation declarations, and exact server/attach secret-binding descriptors. + +- [x] 69-06-PLAN.md — Optional expect on the 29 tools: zod shape, top-level sidecar, back timeout, version-skew fill, unknown-outcome text; byte-identity goldens without expect (wave 4) **Wave 5** *(blocked on Wave 4 completion)* -- [x] 64-05-PLAN.md — Atomically compose and expose the production adapter, registry row, compatibility row, and exact parser/fixture/native-drift bijection. - -**Wave 6** *(parallel; blocked on Wave 5, with the server lifecycle also directly bound to Wave 3 runtime state)* -- [x] 64-06-PLAN.md — Project the exact roster into bounded inventory/doctor evidence and the browser-safe Claude/OpenCode compatibility plane. -- [x] 64-07-PLAN.md — Implement the generic cold-first, FSB-owned-server lease, health, fallback, idle, and recovery lifecycle. - -**Wave 7** *(parallel; each blocked on its Wave 6 dependency)* -- [x] 64-08-PLAN.md — Interpret closed attestation descriptors through the shared verifier, enforce no replay, and gate results on clean exit/tree settlement. -- [x] 64-09-PLAN.md — Generalize canonical extension preflight, consent, trust, and background-authoritative provider routing. - -**Wave 8** *(parallel; each blocked on its Wave 7 dependency)* -- [x] 64-10-PLAN.md — Generalize durable event/controller hydration and safe per-adapter drift diagnostics. -- [x] 64-11-PLAN.md — Promote the existing OpenCode Providers row from safe evidence with honest auth/billing and no visible layout change. - -**Wave 9** *(blocked on Waves 7-8 completion)* -- [x] 64-12-PLAN.md — Project OpenCode through the existing delegated consent/side-panel/feed/summary UI with no provider-specific branch. - -**Wave 10** *(blocked on Wave 9 completion)* -- [x] 64-13-PLAN.md — Wire exact validation/root/CI/security/full-suite gates and the still-pending three-row human-UAT ledger. - -### Phase 65: Codex Adapter -**Goal**: Task-mode delegation coverage extends to OpenAI Codex with correct auth-state disclosure (ChatGPT OAuth vs API key vs unauthenticated) so users know which billing bucket a run will hit before starting; `caps.chatMode: false` across all v0.9.91 adapters confirms task-mode-only scope for the milestone. -**Depends on**: Phase 60 (contract), Phase 62 (drift-smoke gate) -**Requirements**: MULTI-04, MULTI-05, MULTI-06 -**Success Criteria** (what must be TRUE): - 1. A Codex adapter (`mcp/src/agent-providers/codex.ts`) implements the `AgentProviderAdapter` contract, invoking `codex exec --json` with the verified 0.142.5 flag set (`--ephemeral` + `--ignore-user-config` for hermeticity; the deprecated `--full-auto` is NEVER referenced in adapter source, protected by the CHAN-07 grep gate against `--auto`). - 2. The Codex adapter's `detect()` correctly identifies auth via ChatGPT OAuth, API key, or unauthenticated and surfaces the state in the Providers panel so the user knows which billing bucket a run will hit — with the accepted copy **Included with your ChatGPT plan**, **Billed to the API key stored by Codex; dollar amount not reported.**, or the exact sign-in/status-refresh recovery copy. - 3. A schema-derived Codex JSONL contract fixture pins the event schema in CI under `tests/fixtures/agent-streams/codex-0.142.5/`, honestly retains `liveCapturePending: true`, and awaits genuine capture in UAT65-02; the Phase 62 drift-smoke job includes Codex from the first implementation commit, and the adapter's `caps()` correctly reports `chatMode: false` — matching the milestone-wide task-mode-only posture. - 4. A user with Codex installed can pick "Codex" as their provider and observe the same delegation UX as Claude Code and OpenCode with the correct per-auth-state cost copy, and the Providers panel's `agent`-kind cost row shows tokens / turns / duration + the auth-state-appropriate subscription caption rather than a fabricated dollar amount. -**Plans**: 8/8 complete - -- [x] 65-01-PLAN.md — Establish canonical accepted identity, capability vocabulary, and durable provider-neutral persistence before Codex exposure. -- [x] 65-02-PLAN.md — Bind background-owned preflight, consent, trust, and replay rejection to the exact five-field identity. -- [x] 65-03-PLAN.md — Enforce immediate supervisor re-attestation and exact start echo before any runtime or browser-visible effect. -- [x] 65-04-PLAN.md — Add bounded byte probes, sanitized environments, serve-owned authority descriptors, and empty direct scratch cleanup. -- [x] 65-05-PLAN.md — Atomically expose the complete Codex adapter, profile, parser, fixture, roster, drift, and negative corpus. -- [x] 65-06-PLAN.md — Project safe auth evidence and exact billing copy through the existing three-provider Providers surface. -- [x] 65-07-PLAN.md — Preserve immutable accepted identity through feed/hydration and close shared 44px delegated controls. -- [x] 65-08-PLAN.md — Lock exact validation/UAT contracts and the preservation-safe focused, root, and CI closure gate. - -## Progress - -**Execution Order:** -Phases execute in numeric order: 57 → 58 → 59 → 60 → 61 → 62 → 63 → 64 → 65 -Security-first hard rule: Phase 59 is code-green before Phase 60 spawn code lands, regardless of Phase 57/58 progress. - -| Phase | Plans Complete | Status | Completed | -|-------|-----------------|--------|-----------| -| 57. Agent Identity Capture | 3/3 | Complete | 2026-07-12 | -| 58. Providers Panel | 3/3 | Complete | 2026-07-12 | -| 59. Reverse-Request Channel & Security Foundation | 4/4 | Complete | 2026-07-14 | -| 60. Adapter Contract & Claude Code MVP | 4/4 | Complete | 2026-07-14 | -| 61. Delegation UX & SW-Eviction Persistence | 8/8 | Complete (UAT deferred) | 2026-07-15 | -| 62. CI Drift-Smoke Gate & Doctor Extensions | 6/6 | Complete (UAT deferred) | 2026-07-16 | -| 63. Native-Messaging Host | 12/12 | Complete | 2026-07-20 | -| 64. OpenCode Adapter | 13/13 | Complete | 2026-07-21 | -| 65. Codex Adapter | 8/8 | Complete | 2026-07-22 | + +- [x] 69-07-PLAN.md — expect contract parity test, SC4 in-process caller test (one tools/call), headless Chrome fixture probe test (wave 5) + +**Wave 6** *(blocked on Wave 5 completion)* + +- [x] 69-08-PLAN.md — Tool-count wording fix (72 at 8a058bdf, 73 with settings) and the phase gate: footprint, full chain, Node 20, fixture --check, VALIDATION sign-off (wave 6) + +### Phase 70: Bounded MCP Fast Task + +**Goal:** Let the existing `run_task` use the bounded Jev fast loop when the extension's Advanced Settings switch is enabled. +**Depends on:** Phases 67, 68, and 69 (provider policy, fast-loop mechanics, and additive contract gate). +**Requirements:** MCP-03. +**Status:** Code-complete 2026-10-17; human/live checks deferred to the Validation Block (V2 live `disabled`/`spend_cap_reached` fallbacks; V3 calibrated `run_task` profile; V4 comparative gate and typed-outcome agreement table plus the WR-01 owner decision; V5 real MCP hosts, real-page cases and the PD-70-01 confirmation; V6 release, the D-17 amendment, the MCP-enable consent review and the wording check; see `70-HUMAN-UAT.md`). +**Success Criteria:** + +1. Existing `run_task` reads the extension's Advanced Settings switch and uses a bounded action, Jev-call, and wall-clock budget only when enabled; it has no separate fast-mode argument or toggle. +2. The MCP result distinguishes done, ambiguous, blocked, and needs-confirmation using verified effects and task state, not Jev's answer alone. +3. Provider failures, uncertain targets, and safety gates exit through typed fallback or escalation without duplicate side effects. +4. Existing MCP clients continue to parse responses and invoke tools without any new required argument or new tool. + +**Plans:** 8 plans +- [x] 70-01-PLAN.md — Open Phase 70 safely. +- [x] 70-02-PLAN.md — Let `run_task` join the fast path through the one gate Phase 68 built, with its own named budgets. +- [x] 70-03-PLAN.md — Record the evidence the typed outcome will be computed from. +- [x] 70-04-PLAN.md — Make the fast path usable on real `run_task` goals without sending more than before. +- [x] 70-05-PLAN.md — Compute the typed outcome. +- [x] 70-06-PLAN.md — Put the typed outcome on the existing `run_task` result. +- [x] 70-07-PLAN.md — Prove the result reaches real MCP callers unchanged except for the one additive key, and let V4 measure it. +- [x] 70-08-PLAN.md — Run the Phase 70 gate. + +### Phase 71: Comparative Gate and Release + +**Goal:** Compare full-task outcomes, decide which fast paths merit release, and version the extension and MCP package honestly. +**Depends on:** Phases 66-70. +**Requirements:** EVAL-01, EVAL-02, REL-01, REL-02. +**Status:** Code-complete 2026-10-17; human/live checks deferred to the Validation Block (V1 the M-71-00 owner sign-off on the frozen gate, a hard pre-V1 decision that also takes the WR-03/WR-05 review follow-ups, plus the pinned Claude Code and Chrome, the 30-round `core-baseline-66` campaign, the cherry-pick, the safety sidecar and the identity cross-check; V3 calibrated-profile smoke; V4 the autopilot and Claude gate campaigns, the evaluation and decision file, the conditional `core-rebaseline-71` fallback, `count_tokens` and the IN-05 switch check; V6 step A extension 1.0.0, step B MCP 0.12.0 or the 72-count docs fallback; see `71-HUMAN-UAT.md`). +**Success Criteria:** + +1. The same outcome-checked corpus compares baseline, fast mode, and a non-Jev control on completion rate, wall time per completed task, and cost, with uncertainty and failure cases included. +2. A predeclared gate reports pass or fail for speed, success, and safety; any failing path remains experimental or disabled and no per-decision timing is presented as whole-task speedup. +3. `CHANGELOG.md` has a v1.0.0 entry before `npm run version:set:extension -- 1.0.0` succeeds, and product version checks agree across extension surfaces. +4. The MCP package receives its planned minor bump for optional `expect` and the approved settings tool, with compatibility checks for existing callers and no unreviewed tool additions. + +**Plans:** 12 plans +- [x] 71-01-PLAN.md — Record the sync re-check of every upstream interface Phase 71 depends on, record the pre-edit regression-sweep baseline ($P71_START), and fix F5 in the bench: the fixture run token stops making every bench goal `sensitive_goal`, proven against the real 67-06 scrubber. +- [x] 71-02-PLAN.md — Extend the bench's single row contract (scripts/bench/lib/runs.mjs) for the three-arm comparison: a run-condition dimension in the seeded ABBA schedule, the `condition_not_applied` infra code and the `jev_measurement_missing` flag (set by 71-03 only when a switched-on run's fast_mode summary reports Jev attempts but its jevMeasurement never arrived), and an additive, versioned v2 run row with `condition`, `jev`, `safety`, `endLabel` and `fastMode` blocks, while schema-1 rows (the V1 baseline) validate unchanged. +- [x] 71-03-PLAN.md — Wire the condition dimension into the live runner (apply and verify the fast-mode switch before t0, gate hash in identity, v2 raw records) and make the campaign report group rows by (arm, condition) with honest method notes. +- [x] 71-04-PLAN.md — Add the completion-difference statistic, the shared synthetic-campaign generator, and the power and coverage simulation whose output decides the round counts the gate will freeze. +- [x] 71-05-PLAN.md — Write the gate's exact-key validator, then author and freeze `scripts/bench/gate/release-gate-v1.0.0.json` from the 71-04 simulation and the sync re-check, pin its sha256, and make the runner refuse an invalid gate. +- [x] 71-06-PLAN.md — Build the pure gate evaluator and prove every verdict branch on synthetic campaigns. +- [x] 71-07-PLAN.md — Turn evaluator results into the decision file, the release claims and the gate report, and add the two CLIs V4 will run: the gate evaluator (git and suite checks injected) and the baseline safety sidecar. +- [x] 71-08-PLAN.md — Fix the version setter's MCP README compatibility templates so each MCP release states its own declared minimum extension version, and rehearse both release steps (extension 1.0.0, then MCP 0.12.0) in a temp root with every parity phrase checked. +- [x] 71-09-PLAN.md — Build the three REL-02 checks that can run now: the reviewed tools/list delta for MCP 0.12.0, the release-diff checker that binds shipped code to the V4-measured SHA, and the count_tokens script V4 will run. +- [x] 71-10-PLAN.md — Update the gate-independent documentation: tool counts at 73 in both READMEs with tables that sum to the built tools/list length, the `settings` row and the `expect` reference, the skill decision tree, and count-free onboarding copy, with the parity pins moved in the same commits. +- [x] 71-11-PLAN.md — Move the showcase copy, the llms sources and outputs, and all seven xlf catalogs from 68 to 73 tools with digit-only, line-stable edits, recompute the affected allowlist hashes, and extend the parity test's stale-count ban to 68 and 69. +- [x] 71-12-PLAN.md — Run the Phase 71 gate: the full chain, Node 20 runs of every new suite, the regression sweep against the 71-01 baseline, the footprint audit, and the final 71-VALIDATION.md statuses and sign-off. ## Completed Milestones +
+v0.9.91 MCP Clients as Providers — Phases 57-65, SHIPPED 2026-09-28 + +**Milestone Goal:** Make installed agent CLIs (Claude Code first) first-class side-panel providers that FSB detects, presents key-less in a Providers panel, and delegates tasks to, with the spawned CLI driving the browser back through FSB's own MCP tools. + +**Phase summary:** + +| Phase | Name | Plans | Status | +|-------|------|-------|--------| +| 57 | Agent Identity Capture | 3/3 | Complete | +| 58 | Providers Panel | 3/3 | Complete | +| 59 | Reverse-Request Channel & Security Foundation | 4/4 | Complete | +| 60 | Adapter Contract & Claude Code MVP | 4/4 | Complete | +| 61 | Delegation UX & SW-Eviction Persistence | 8/8 | Complete | +| 62 | CI Drift-Smoke Gate & Doctor Extensions | 6/6 | Complete | +| 63 | Native-Messaging Host | 12/12 | Complete | +| 64 | OpenCode Adapter | 13/13 | Complete | +| 65 | Codex Adapter | 8/8 | Complete | + +Archive files: + +- `.planning/milestones/v0.9.91-ROADMAP.md` +- `.planning/milestones/v0.9.91-REQUIREMENTS.md` +- `.planning/milestones/v0.9.91-phases/` + +Outcome: 51/51 requirements complete with green automated/source verification. The 48 deferred live UAT scenarios were closed at the user's direction on 2026-09-28 as waived, not executed; see `.planning/STATE.md` `## Deferred Items`. + +
+
v1.2.0 Showcase i18n Completeness — Phases 52-56, SHIPPED 2026-07-09 @@ -313,65 +364,81 @@ Outcome: 13/13 v1.2.0 requirements satisfied; audit passed. Phase 52's audit est - Search distinguished "directly invocable T1" from "discovery-pending DOM/T2." ### Phase 44: T1 Readiness Inventory + Status Surface + **Goal:** Create the authoritative T1 readiness matrix for all 2,314 descriptors and make status visible to developers and users so "catalog supported" is never confused with "direct API-ready." **Requirements:** T1R-01, T1R-02, T1R-03. **Plans:** + - [x] 44-01: T1 readiness matrix generator and evidence report. - [x] 44-02: Status-surface and documentation honesty pass. - [x] 44-03: T1 readiness CI guard and phase closeout. ### Phase 45: T1 Porting Scaffold + Handler Contract Hardening + **Goal:** Build the reusable test and implementation scaffold for app ports so each new T1 handler has origin-pin, logged-out guard, shape guard, no-secret logging, consent classification, and byte-stable fallback behavior by default. **Requirements:** T1R-04, T1R-05, T1R-09. **Plans:** + - [x] 45-01: Port contract library and scaffold CLI. - [x] 45-02: Current-catalog contract verifier. - [x] 45-03: Documentation and phase closeout. ### Phase 46: Same-Origin Read Ports — First High-Value Batch + **Goal:** Convert a first batch of high-value read descriptors from T3 to executable T1 where the app's authenticated web runtime uses same-origin APIs and can be proven without weakening Wall 2. **Requirements:** T1R-06. **Plans:** + - [x] 46-01: Candidate selection and Netlify/Bitbucket/CircleCI handler ports. - [x] 46-02: Catalog wiring, search readiness, and same-origin classifier gates. - [x] 46-03: Verification, UAT notes, and phase closeout. ### Phase 47: Pattern-D + GAPI Bridge Architecture + **Goal:** Design and prove the missing architecture for apps whose useful APIs are separate-origin, per-org-subdomain, or page-bridge mediated, without breaking the active-tab credential boundary. **Requirements:** T1R-07, T1R-08. **Plans:** + - [x] 47-01: Pattern-D decision for separate-origin and per-org APIs. - [x] 47-02: GAPI bridge decision for Google Workspace APIs. - [x] 47-03: CI gate, negative controls, and closeout. ### Phase 48: High-Value Read Ports — Second Batch + **Goal:** Use the Phase 47 outcome to port another batch of read descriptors, including Pattern-D/GAPI candidates if the architecture is proven. **Requirements:** T1R-06, T1R-07, T1R-08. **Plans:** + - [x] 48-01: Vercel same-origin read head. - [x] 48-02: CircleCI same-origin read expansion. - [x] 48-03: Gate, search, and readiness-report closeout. ### Phase 49: Guarded Writes Activation Pipeline + **Goal:** Turn fail-closed write/destructive candidates into executable T1 only after live mutation-body capture, consent gate verification, and redacted audit proof. **Requirements:** T1R-10, T1R-11. **Plans:** + - [x] 49-01: Write activation evidence ledger. - [x] 49-02: Evidence verifier and validation gate. - [x] 49-03: Live-UAT template and closeout decision. ### Phase 50: T1 Expansion Gate + Next-Batch Plan + **Goal:** Close the milestone with an honest T1 coverage gate, full regression suite, and a prioritized backlog for the remaining 2,288-descendant tail. **Requirements:** T1R-12. **Plans:** + - [x] 50-01: Regenerate readiness evidence and closeout counts. - [x] 50-02: Full regression gate. - [x] 50-03: Next-batch backlog and milestone closeout. ### Phase 51: Full T1 Tail Migration Across Remaining Catalog + **Goal:** Convert the Phase 51 catalog-tail descriptors to explicit terminal states: executable T1/T1b where safe and technically provable, guarded fail-closed where write/destructive UAT is still required, or blocked where denylist/product/legal policy says no. **Requirements:** T1ALL-01, T1ALL-02, T1ALL-03, T1ALL-04, T1ALL-05. **Plans:** + - [x] 51-01: Full-tail worklist generator, acceptance gate, and batch scheduler. - [x] 51-02: Existing-head and low-risk same-origin read bulk port. - [x] 51-03: Retool CSRF same-origin read head. @@ -387,19 +454,19 @@ Archive files: - `.planning/milestones/v1.1.0-REQUIREMENTS.md` - `.planning/milestones/v1.1.0-MILESTONE-AUDIT.md` -Outcome: v1.1.0 expanded proven direct execution from the v1.0.0 baseline into verified T1-ready read coverage across Netlify, Bitbucket, CircleCI, Vercel, Retool, Asana, and generated same-origin reads, with post-closeout ports reflected in the refreshed artifacts. Phase 51 closes the remaining catalog tail honestly: 1,267 rows are executable T1-ready, 556 rows are guarded fail-closed, and the remaining 491 descriptors are explicitly surfaced as bridge-needed, UAT-needed, blocked-policy, or degraded/discovery-pending rather than overclaimed. +Outcome: v1.1.0 expanded proven direct execution from the v0.13.0 baseline into verified T1-ready read coverage across Netlify, Bitbucket, CircleCI, Vercel, Retool, Asana, and generated same-origin reads, with post-closeout ports reflected in the refreshed artifacts. Phase 51 closes the remaining catalog tail honestly: 1,267 rows are executable T1-ready, 556 rows are guarded fail-closed, and the remaining 491 descriptors are explicitly surfaced as bridge-needed, UAT-needed, blocked-policy, or degraded/discovery-pending rather than overclaimed.
-v1.0.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 +v0.13.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 Archive files: -- `.planning/milestones/v1.0.0-ROADMAP.md` -- `.planning/milestones/v1.0.0-REQUIREMENTS.md` -- `.planning/milestones/v1.0.0-MILESTONE-AUDIT.md` -- `.planning/milestones/v1.0.0-phases/` +- `.planning/milestones/v0.13.0-ROADMAP.md` +- `.planning/milestones/v0.13.0-REQUIREMENTS.md` +- `.planning/milestones/v0.13.0-MILESTONE-AUDIT.md` +- `.planning/milestones/v0.13.0-phases/` Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,314 descriptors across 128 app stems / 129 services. The milestone deliberately fed the existing tier model rather than hand-porting every action: 26 descriptors resolve to T1/T1b today, 5 guarded writes remain fail-closed, and 2,288 descriptors remain T3 DOM/discovery-tail. Full milestone audit passed; non-blocking live UAT and T1 expansion debt are carried forward. @@ -408,7 +475,7 @@ Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,3
v0.9.99 Native Capability Catalog (FSB API Execution) — Phases 26-34, CODE-COMPLETE 2026-06-23 -Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v1.0.0 milestone extends this substrate verbatim. +Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v0.13.0 milestone extends this substrate verbatim. | Phase | Name | Plans | Status | |-------|------|-------|--------| @@ -422,7 +489,7 @@ Gave FSB first-class authenticated-API execution as a fast path alongside DOM au | 33 | PhantomStream Media Mirroring (0.2.1 Uptake) — milestone extension | 1/1 | Complete; live media UAT human_needed | | 34 | Explicit File Upload Tool (upload_file) — milestone extension | 1/1 | Complete; live upload UAT human_needed | -Substrate carried into v1.0.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard. +Substrate carried into v0.13.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard.
@@ -503,10 +570,10 @@ Known deferred closeout evidence: 11 human-gated Chrome MV3/UAT verification ite ## Carry-Forward Candidates -- **v0.9.91 v2 deferred (see REQUIREMENTS.md):** CHAT-FUTURE-01/02 (chat-mode continuity via `--resume` / `codex resume` / `opencode --continue` + per-thread cwd pinning); GEMINI-FUTURE-01 (Gemini CLI adapter after live `--help` capture + JSONL schema pinning); ACP-FUTURE-01 (`@zed-industries/agent-client-protocol` unification once ≥2 non-Claude adapters have shipped); REMOTE-FUTURE-01 (remote/mobile delegation surfaces — v0.9.91 is explicitly localhost-only). -- **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. Does NOT block v0.9.91. +- **v0.9.91 v2 deferred (see `.planning/milestones/v0.9.91-REQUIREMENTS.md`):** CHAT-FUTURE-01/02 (chat-mode continuity via `--resume` / `codex resume` / `opencode --continue` + per-thread cwd pinning); GEMINI-FUTURE-01 (Gemini CLI adapter after live `--help` capture + JSONL schema pinning); ACP-FUTURE-01 (`@zed-industries/agent-client-protocol` unification once ≥2 non-Claude adapters have shipped); REMOTE-FUTURE-01 (remote/mobile delegation surfaces — v0.9.91 is explicitly localhost-only). +- **Consolidated Chrome MV3 UAT debt:** Closed 2026-09-28 — waived by the user at the v0.9.91 close, not executed (see `.planning/STATE.md` `## Deferred Items`). - **v1.2.0 v2 deferred:** QA-01 (native-speaker/bilingual QA pass), I18N-FUTURE-01 (migrate stats page off ad hoc JSON mechanism into main XLIFF pipeline), I18N-FUTURE-02 (full automated per-locale visual regression pipeline), I18N-FUTURE-03 (translation-freshness/"last synced" reporting surface). -- **v2 deferred capability families (acknowledged, out of v1.0.0/v1.1.0):** GAPI-01 (gapi-bridge handler family for Google Workspace); CLOUD-01 (cloud-console Pattern-D ports); UATX-01 (per-app live guarded-write UAT closeout). +- **v2 deferred capability families (acknowledged, out of v0.13.0/v1.1.0):** GAPI-01 (gapi-bridge handler family for Google Workspace); CLOUD-01 (cloud-console Pattern-D ports); UATX-01 (per-app live guarded-write UAT closeout). - **Delegation primitive (Lattice-owned):** Parked from v0.10.0; re-scope as either a Lattice-owned primitive or an FSB-only consumer of Lattice receipt + tripwire surfaces after v0.9.91 stabilizes the adapter contract. ## Backlog diff --git a/.planning/STATE.md b/.planning/STATE.md index e3a126cfb..e598c8f47 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -1,20 +1,20 @@ --- gsd_state_version: 1.0 -milestone: v0.9.91 -milestone_name: MCP Clients as Providers -status: human_needed -stopped_at: Phase 65 automated verification passed; human UAT65-01 through UAT65-03 pending -last_updated: "2026-07-22T18:45:13.807Z" -last_activity: 2026-07-22 +milestone: v1.0.0 +milestone_name: Jev Fast Mode +status: executing +stopped_at: "v1.0.0 code-complete; Validation Block V1-V6 pending (see .planning/v1.0.0-VALIDATION-BLOCK.md)" +last_updated: "2026-10-19T05:45:24.000Z" +last_activity: 2026-10-19 -- v1.0.0 Validation Block runbook and integration check written progress: - total_phases: 18 - completed_phases: 9 - total_plans: 61 - completed_plans: 63 - percent: 50 + total_phases: 7 + completed_phases: 0 + total_plans: 72 + completed_plans: 70 + percent: 0 --- -*Note: the `total_phases`/`completed_phases` counts above are scoped to the active v0.9.91 milestone (Phases 57-65) only. Some GSD tooling (`roadmap.analyze`, `phase.complete`) reports a noisy multi-phase count including collapsed `## Completed Milestones` archive entries and `## Backlog` sections — treat this file's own numbers as authoritative for v0.9.91 progress.* +*Note: the `total_phases`/`completed_phases` counts above are scoped to the active v1.0.0 milestone (Phases 66-71 plus inserted 66.1) only. Some GSD tooling (`roadmap.analyze`, `phase.complete`) reports a noisy multi-phase count including collapsed `## Completed Milestones` archive entries and `## Backlog` sections — treat this file's own numbers as authoritative for v1.0.0 progress.* # Project State @@ -26,25 +26,36 @@ progress: ## Project Reference -See: .planning/PROJECT.md (v0.9.91 MCP Clients as Providers — Current Milestone section, Key context bullets) -See: .planning/ROADMAP.md (v0.9.91 active, Phases 57-65; v1.2.0 / v1.1.0 / v1.0.0 / v0.9.99 / etc. archived and collapsed) -See: .planning/REQUIREMENTS.md (51 v1 requirements across 10 categories: IDENT, PROV, CHAN, ADAPT, CLAUDE, UX, LIFE, DRIFT, NATIVE, MULTI — all mapped to Phases 57-65, 48/51 complete) -See: .planning/research/SUMMARY.md (converged research summary; suggested phase structure; HIGH confidence) -See: .planning/research/PITFALLS.md (16 pitfalls with phase assignments; security section verified against 2025-2026 CVE class incidents) -See: .planning/research/ARCHITECTURE.md (file:line integration seams; brownfield mapping onto existing FSB architecture) +See: .planning/PROJECT.md (updated 2026-09-28 after the v0.9.91 close) +See: .planning/milestones/v0.9.91-ROADMAP.md, .planning/milestones/v0.9.91-REQUIREMENTS.md, .planning/milestones/v0.9.91-phases/ (archived MCP Clients as Providers milestone; 51/51 requirements complete) See: .planning/milestones/v1.2.0-ROADMAP.md, .planning/milestones/v1.2.0-REQUIREMENTS.md, .planning/v1.2.0-MILESTONE-AUDIT.md (archived Showcase i18n Completeness milestone) -**Core value:** Reliable single-attempt execution — the AI decides correctly, the mechanics execute precisely. v0.9.91 does not touch the DOM/automation single-attempt property; it extends the surface so installed agent CLIs (Claude Code first, then OpenCode + Codex) become first-class side-panel providers that drive the same live browser through FSB's own MCP tools. -**Current focus:** Milestone-end human UAT sweep; Phase 65 automated verification is complete +**Core value:** Reliable single-attempt execution — the AI decides correctly, the mechanics execute precisely. +**Current focus:** v1.0.0 milestone-end Validation Block (V1-V6), starting with the owner decisions in `.planning/v1.0.0-VALIDATION-BLOCK.md` section 1; every phase (66, 66.1, 67-71) is code-complete ## Current Position -Phase: 65 (codex-adapter) — EXECUTING -Plan: 8 of 8 -Status: human_needed -Last activity: 2026-07-28 - Completed quick task 260728-k2v: animated toolbar action icon +Milestone: v1.0.0 Jev Fast Mode — CODE-COMPLETE (Phases 66, 66.1, 67, 68, 69, 70, 71); Validation Block V1-V6 pending +Phase: 71 (Comparative Gate and Release) — CODE-COMPLETE, the last phase; verification `human_needed` at 9c8e5d5b (4/4 roadmap criteria, 69/69 plan must-haves, 0 gaps; full chain exit 0, 348 commands, at 29093c4b and again at 9c8e5d5b) +Plan: 70 of 72 plans complete; 66-10 and 66-11 run in Validation Block V1 from a worktree pinned at 8a058bdf, with Phase 71's gate numbers (30 rounds, 720 runs) superseding theirs +Status: v1.0.0 code-complete; Validation Block V1-V6 pending. The runbook `.planning/v1.0.0-VALIDATION-BLOCK.md` orders every deferred item: section 1 lists 35 owner decisions due before V1 (M-71-00 is the hard gate; its D-15 false-fail finding means the frozen gate would almost surely disable Jev, so the recommendation is to amend it before V1), section 2 gives V1-V6 with commands and evidence, and section 3 traces all 25 requirements. The integration check `.planning/v1.0.0-INTEGRATION-CHECK.md` found no blockers (8/8 E2E flows wired; warnings W1-W8 carried into the runbook). No version, CHANGELOG or publish change; nothing pushed; `main` untouched; `scripts/bench/results` and `scripts/bench/gate/decisions` are still empty +Last activity: 2026-10-19 -- v1.0.0 Validation Block runbook and integration check written -## Roadmap At A Glance (v0.9.91, Phases 57-65) +## Roadmap At A Glance (v1.0.0, Phases 66-71) + +| Phase | Name | Requirements | Status | +|-------|------|--------------|--------| +| 66 | Baseline and Session Timing | MEAS-01..03 | Code-complete (9/11; 66-10/66-11 in Validation Block V1) | +| 66.1 | MCP Settings Tool (inserted) | SET-01..02 | Code-complete (8/8; human/live checks in Validation Block V5, V1 cross-check, V6 release) | +| 67 | Hosted Jev Gateway, Policy, and Shadow Mode | JEV-01..07 | Code-complete (13/13; JEV-03/JEV-06 met; human/live checks in Validation Block V2, V3 calibration, V4 gate, V5 owner decisions, V6 release) | +| 68 | Autopilot Fast Mode | AUTO-01..05 | Code-complete (12/12; AUTO-03 met; human/live checks in Validation Block V3 calibration and CR-01 decision, V4 gate, V5 UAT, V6 privacy page, Web Store, WR-04/IN-05 owner decisions and release) | +| 69 | MCP Descriptions and Same-Call Checks | MCP-01, MCP-02, MCP-04 | Code-complete (8/8; human/live checks in Validation Block V4, V5, V1 cross-check, V6 release) | +| 70 | Bounded MCP Fast Task | MCP-03 | Code-complete (8/8; human/live checks in Validation Block V2 fallbacks, V3 calibration, V4 gate and WR-01 owner decision, V5 hosts, real pages and PD-70-01, V6 release, D-17 amendment and consent review) | +| 71 | Comparative Gate and Release | EVAL-01..02, REL-01..02 | Code-complete (12/12; human/live checks in Validation Block V1 gate sign-off (hard pre-V1) and 30-round baseline, V3 calibrated smoke, V4 gate campaigns, decision file and `count_tokens`, V6 release step A extension 1.0.0 and step B MCP 0.12.0) | + +Coverage: 25/25 v1.0.0 requirements mapped; 0/7 phases complete. See `.planning/REQUIREMENTS.md` and `.planning/ROADMAP.md`. FSB supplies the Jev credential server-side to every install without a user usage quota. The only fast-mode setting is the control-panel Advanced Settings switch, which the Phase 66.1 MCP settings tool can also read and write. With Jev disabled, FSB behaves as it did before this milestone. + +## Roadmap At A Glance (v0.9.91, Phases 57-65 — archived 2026-09-28) | Phase | Name | Requirements | Status | |-------|------|--------------|--------| @@ -55,10 +66,10 @@ Last activity: 2026-07-28 - Completed quick task 260728-k2v: animated toolbar ac | 61 | Delegation UX & SW-Eviction Persistence | UX-01..06, LIFE-01..04 | Complete (2026-07-15; UAT deferred to milestone end) | | 62 | CI Drift-Smoke Gate & Doctor Extensions | DRIFT-01, DRIFT-02, DRIFT-03, DRIFT-04 | Complete (2026-07-16; UAT deferred to milestone end) | | 63 | Native-Messaging Host | NATIVE-01, NATIVE-02, NATIVE-03, NATIVE-04 | Complete (2026-07-20; UAT deferred to milestone end) | -| 64 | OpenCode Adapter | MULTI-01, MULTI-02, MULTI-03 | In Progress (9/13) | -| 65 | Codex Adapter | MULTI-04, MULTI-05, MULTI-06 | Not started | +| 64 | OpenCode Adapter | MULTI-01, MULTI-02, MULTI-03 | Complete (2026-07-21) | +| 65 | Codex Adapter | MULTI-04, MULTI-05, MULTI-06 | Complete (2026-07-22) | -Coverage: 51/51 v0.9.91 requirements mapped and automated/source complete; milestone closure awaits the accumulated human UAT ledgers +Coverage: 51/51 v0.9.91 requirements complete. The 48 deferred live UAT scenarios were closed on 2026-09-28 as waived by the user, not executed. ## Hard Invariants (v0.9.91) @@ -77,9 +88,30 @@ Coverage: 51/51 v0.9.91 requirements mapped and automated/source complete; miles ## Accumulated Context +### Roadmap Evolution + +- Phase 66.1 inserted after Phase 66: MCP Settings Tool: one MCP-only tool to view all control-panel settings and set one by key; secrets masked and never writable; approved 2026-09-29 + ### Decisions -Full decision log for prior milestones (v0.9.99 Phase 26-34, v1.0.0, v1.1.0, v1.2.0) lives in PROJECT.md, ROADMAP.md's `## Completed Milestones` section, and the archived `.planning/milestones/*` files. Those decisions concern the T1 capability-catalog surface and the showcase-i18n surface; they have zero shared surface with v0.9.91's provider/delegation work and are not repeated here to keep this file a digest. +Full decision log for prior milestones (v0.9.99 Phase 26-34, v0.13.0, v1.1.0, v1.2.0) lives in PROJECT.md, ROADMAP.md's `## Completed Milestones` section, and the archived `.planning/milestones/*` files. Those decisions concern the T1 capability-catalog surface and the showcase-i18n surface; they have zero shared surface with v0.9.91's provider/delegation work and are not repeated here to keep this file a digest. + +v1.0.0-specific decisions: + +- [Milestone 2026-10-15, owner direction]: **All human validations are deferred to one Validation Block at the end of the milestone.** Phases 66.1 and 67-71 are built and verified with automated checks only (unit, contract, fixture, fake-upstream and headless tests); no phase waits on a human checkpoint. Items that need the operator, live accounts, production, or spend are collected into the Validation Block instead: + - **V1, baseline:** 66-10 (H2/H3, smoke) and 66-11 (H4 campaign, H5 restore), run from a worktree pinned at `8a058bdf`, the last commit before any Phase 66.1+ product code. That keeps the baseline a measurement of FSB before this milestone, as Phase 66 requires. + - **V2, Jev gateway live:** set the server-side Jev credential and deploy the gateway (Fly). Pushing to `main` deploys the server, packages the extension, and publishes MCP to npm (`.github/workflows/deploy.yml`, `chrome-extension.yml`, `npm-publish.yml`), so nothing from this milestone reaches `main` before the Validation Block. + - **V3, shadow calibration (JEV-05):** run shadow mode on the labelled corpus against the pinned live model and set the live thresholds from that evidence. Until then, thresholds ship as explicit uncalibrated defaults, fast mode stays default-off, and anything that would turn it on for users is gated on V3 evidence. + - **V4, comparative gate (EVAL-01/02):** baseline vs. non-Jev control vs. fast mode on the same corpus, with the predeclared gate. + - **V5, human UAT:** every `human_needed` verification item and every UI check from Phases 66.1-71. + - **V6, release:** the version commits can be prepared in code, but publishing and merging to `main` wait for V4's gate and the owner's approval. +- [Phase 66, 2026-10-15]: Phase 66 is code-complete (9/11). 66-10 and 66-11 are deferred to Validation Block V1, and later phases treat the pinned commit `8a058bdf` as "the Phase 66 baseline" (for example, Phase 69's tool-list token count is measured against the tool list at `8a058bdf`). +- [Phase 66.1, 2026-10-15]: Phase 66.1 is code-complete (8/8 plans plus 10 review-fix commits; verification `human_needed`, 62/63, 0 failed). SET-01/SET-02 stay unticked until their live proof runs in V5. One owner decision is pending in V5: accept or revert the review fixes WR-02 (a switch to `custom` keeps a stored model name) and WR-04 (`providerKind=api` needs a ready provider), which depart from the letter of D-15. Open non-blocking follow-up IN-02: add a parity test for the settings literals hand-mirrored in `mcp-tool-dispatcher.js` before Phase 68 extends the schema. +- [Phase 69, 2026-10-15]: Phase 69 is code-complete (8/8 plans plus 7 review-fix commits; verification `human_needed`, 12/12 automated must-haves, 0 failed). MCP-04 is ticked on automated evidence (a 192-call pre/post wire diff against `f978e989` with 0 diffs; the D-25 redaction of sensitive `inputs_changed`/`value` records is its one recorded exception). MCP-01/MCP-02 stay unticked until V4 (exact `count_tokens` ratio; the ceil(chars/4) estimate is a 55.2% cut on description text but 30.4% on whole-list bytes, so the PROJECT.md "more than half" target is unconfirmed) and V5 (description read-through, real-site verdicts, live SC4). The `expect` timing constants ship uncalibrated until V4. Pending in V5: owner decisions WR-04 (an already-true predicate reports `verified`, then "Do not repeat the action" on an unknown outcome) and WR-05 (keep or widen the D-25 sensitive-name list), and the CR-02 acknowledgement (the recorder now sanitizes http(s) URLs under `observed`/`expected`/`before`/`after`, a storage-only change). Non-blocking follow-ups: review IN-01, IN-02, IN-04 (see `69-HUMAN-UAT.md` Gaps). +- [Phase 67, 2026-10-15]: Phase 67 is code-complete (13/13 plans plus 10 review-fix commits; verification `human_needed` at `9cd522c7`, 64/64 automated must-haves, 0 failed). JEV-03 and JEV-06 are ticked on automated evidence. JEV-01, JEV-02, JEV-04 and JEV-07 stay unticked until the V2 gateway deploy. JEV-05 stays unticked until V3 calibration on the V2 pin; until then the shipped profile is UNCALIBRATED and every decision stays below threshold. The extension `jev-*.js` modules are loaded inert, and Phases 68 and 70 are their consumers. Pending owner decisions: V5 WR-06 (a failed spend seed is retried, not failed closed) and IN-02 (client `invalid_request` vs gateway `payload_too_large` for an unshrinkable request); V2 IN-04/IN-05 (refusal and 404 accounting, CORS on `/api/jev`). For Phase 68: `FsbJevPolicy.accept()` is necessary, never sufficient, so every Jev-chosen action must still pass the existing consent, permission and breaker gates. The Phase 68 verifier should check that call site (JEV-06). +- [Phase 68, 2026-10-17]: Phase 68 is code-complete (12/12 plans plus 12 review-fix commits; verification `human_needed` at `38312ca1`, 10/10 automated must-haves, 0 failed). AUTO-03 is ticked on automated evidence. AUTO-01 (live panel V5, deployed privacy page and Web Store V6), AUTO-02 (calibrated live V3/V5), AUTO-04 (whole-task comparison V4) and AUTO-05 (real summaries V5) stay unticked. The JEV-06 re-check is confirmed: a Jev-accepted step passes the same permission hook, breakers and meta-attempt counter, and no path special-cases `source:'jev'`. Fast mode is default-off, and with the switch off the autopilot path is byte-identical (golden committed before the loop edit). Pending owner decisions: V3 CR-01 (structural fast-eligibility allowlist) before freezing the instrument; V6 WR-04 (any fallback latches the session until a calibrated answer, narrowing CONTEXT D-16; update the D-16 record) and IN-05/M-68-12 (MCP can turn the data-sharing switch on). Non-blocking follow-up: review IN-03 (a session's last held capture is not counted as dropped). Phase 70 joins `run_task` through the one gate Phase 68 built. +- [Phase 70, 2026-10-17]: Phase 70 is code-complete (8/8 plans plus 4 review-fix commits; verification `human_needed` at `9ec5093f`, 12/12 automated must-haves, 0 gaps). MCP-03 stays unticked until its live proof runs in V2-V6. `run_task` follows the one Advanced Settings switch under its own frozen `run_task` budget profile (12 / 16 / 20,000 ms / 3 / 3, revisited at V3), with no new MCP argument or tool and no `mcp/` diff. A pure classifier maps task state plus per-call run evidence to done, ambiguous, blocked or needs_confirmation and never reads a Jev answer; the result reaches callers as one additive `fast_mode` key only when the switch was on at start, and switch-off output is byte-identical to the pre-edit golden on all 5 settle paths. Recorded amendments awaiting owner confirmation: PD-70-01 (D-14 observation set narrowed to the 9 page-reading read-only tools plus a delivered hand-off snapshot; V5) and the D-17 amendment (the `fast_mode` key under MCP-04; V6). Pending owner decisions: V4 WR-01 (keep counting a failed read-only call with a null nested result as unresolved uncertainty, or loosen with a one-line `readOnly` guard) and V6 M-70-08 (MCP-enable consent remedy among D-07 options a-d; release blocker for MCP-started page-context transfers). Non-blocking follow-ups: review IN-02 (ignored registration boolean) and IN-04 (stopped-run `jev` counts possibly off by one; read that way at V4); see `70-HUMAN-UAT.md` Gaps. +- [Phase 71, 2026-10-17]: Phase 71 is code-complete (12/12 plans plus 12 review-fix commits; verification `human_needed` at `9c8e5d5b`, 4/4 roadmap criteria and 69/69 automated must-haves, 0 gaps). EVAL-01, EVAL-02, REL-01 and REL-02 stay unticked until their live halves run: EVAL-01 at V1, V3 and V4; EVAL-02 at V4, with the M-71-00 sign-off at V1 and the verdicts applied at V6; REL-01 at V6; REL-02 at V4 (`count_tokens`) and V6. The release gate `scripts/bench/gate/release-gate-v1.0.0.json` is frozen (sha256 `9d97fb90…f8eb8e`, pinned in `tests/bench-gate.test.mjs`, committed in `b7bba953` before any campaign) at R = 30 rounds for every campaign, which is 1,800 paid runs plus 720 for the optional fallback. The evaluator, decision file, release claims and gate report are proven on synthetic campaigns. The version setter refuses 1.0.0 without the CHANGELOG entry, the 0.12.0 tools/list delta is reviewed (`settings` added, `expect` on 29 tools, nothing removed or newly required), and `check-release-diff` binds shipped code to the V4-measured SHA. Docs show 73 tools. Owner escalation, a hard pre-V1 decision at M-71-00: as frozen, D-15 with the `unknownExposure: 'acted'` reading fails Jev safety 83% (p = 0.5%) to 97% (p = 1%) of the time with no true difference, and the non-Jev "any excess" rule fails 39-49%. Without an amendment (new `gateVersion` and sha256 pin before V1), V4 would mechanically disable Jev for every install. M-71-00 also takes items 2-5 (D-27 count-free onboarding, M-70-08 remedy timing, the D-09 item-3 substitution, the D-24 `showcase/server/**` narrowing) and the review follow-ups WR-05 (`showcase/**` and `skills/**` code classed as docs), WR-03 (record the gate sha256 at M-71-01b as an amend-proof anchor) and the WR-01/WR-02/WR-06 rule confirmations. Operator check at V4: review IN-05, so confirm `fsbJevFastModeEnabled` reads false after the autopilot campaign. Hand-off drift: 71-VALIDATION.md does not yet carry the WR-03, WR-05 and IN-05 follow-ups (see `71-HUMAN-UAT.md` Gaps). v0.9.91-specific decisions so far: @@ -248,24 +280,25 @@ v0.9.91-specific decisions so far: ### Pending Todos -None. Phases 57-63 are automated/source complete; Phase 63 automated validation is closed (all 30 per-task rows green) and awaits phase verification plus the milestone-end human UAT sweep. +None. v0.9.91 is archived; all nine phases are automated/source complete. ### Blockers/Concerns No active blocker. -- **Milestone-end UAT gate:** Phase 58's 12 live Providers checks, Phase 59's 4 live pairing/lifecycle/accessibility checks, Phase 60's 7 authenticated CLI/OS/browser checks, Phase 61's 8 consolidated consent/theme/handoff/stream/worker/endurance/POSIX/restart scenarios, Phase 62's 3 doctor/stream/layout/accessibility checks, and Phase 63's eight genuine OS/Chrome/visual/accessibility scenarios remain pending. Per user instruction, all live evidence is accumulated and audited at milestone end; automated/source and independent review evidence is green through Phase 63 Plan 12 (automated validation closed), and no live pass is inferred. +- **Milestone-end UAT gate (closed 2026-09-28):** The 48 accumulated live scenarios (Phase 58: 12, 59: 4, 60: 7, 61: 8, 62: 3, 63: 8, 64: 3, 65: 3) were closed at the user's direction as waived, not executed. The archived human-UAT ledgers keep their original pending entries as the record; no live pass is claimed. - **Phase 59 pairing decision resolved:** Use explicit `fsb-mcp-server pair`, a durable exact extension-Origin binding, a per-daemon 32-byte session credential, `pair --reset` for deliberate rebind, per-frame sessionId revalidation, and a secret-free `bridge.auth-status` acknowledgement. Silent TOFU is rejected. Phase 60 resolved the static inline-agent/profile and Windows shell-free resolution questions in source and deterministic tests. Their genuine installed-CLI corroboration remains in the deferred Phase 60 UAT ledger. Phase 64 Plan 07 resolved the remaining OpenCode server-shape item with cold-first execution plus an FSB-owned, authenticated `opencode serve` lease for later exact-identity attaches. -- Milestone-end UAT gate: Phase 65 adds UAT65-01 through UAT65-03 (genuine auth matrix, live Codex/browser lifecycle, and accessibility/responsive rendering); all remain human_needed, pending, and evidence-empty. +- Phase 65's UAT65-01 through UAT65-03 are included in the 2026-09-28 user waiver above. ### Quick Tasks Completed | # | Description | Date | Commit | Status | Directory | |---|-------------|------|--------|--------|-----------| +| 260930-f7j | Make FSB content click activate its target once (no double activation; toggle-aware click verification) | 2026-09-30 | da7e484f | Verified | [260930-f7j-make-fsb-content-click-activate-its-targ](./quick/260930-f7j-make-fsb-content-click-activate-its-targ/) | | 260728-k2v | Replace the chrome.action connection badge with a state-driven animated toolbar icon | 2026-07-28 | 45aa40e1 | | [260728-k2v-animated-toolbar-icon](./quick/260728-k2v-animated-toolbar-icon/) | | 260720-jb5 | Patch Sheets session redaction and multi-tab visual-session token finalization | 2026-07-20 | 2f8359b2 | Verified | [260720-jb5-patch-sheets-session-redaction-and-multi](./quick/260720-jb5-patch-sheets-session-redaction-and-multi/) | | 260715-hs1 | Replace Google Sheets OAuth with a zero-extra-auth signed-in tab session | 2026-07-15 | 9ab3d40d | Needs Review | [260715-hs1-replace-google-sheets-oauth-with-a-signe](./quick/260715-hs1-replace-google-sheets-oauth-with-a-signe/) | @@ -395,40 +428,47 @@ Phase 60 resolved the static inline-agent/profile and Windows shell-free resolut ## Deferred Items -Items acknowledged and carried forward from previous milestone closes (Chrome MV3/manual UAT evidence gaps, not fabricated passes; procedures archived under `.planning/milestones/*/` and `.planning/phases/*/`). None of this debt blocks v0.9.91. +On 2026-09-28, at the v0.9.91 close, the user directed that every outstanding UAT gap be marked done and every open audit item be closed. UAT rows below are therefore closed as **waived by the user, not executed**; no live pass is claimed, and the original ledgers under `.planning/milestones/*/` keep their pending entries as the record. | Category | Item | Status | Deferred At | |----------|------|--------|-------------| -| uat_gap | Phase 63 Plans 05-06 (real POSIX ownership/modes, Windows HKCU/WOW64/PE/bootstrap behavior, Chrome discovery, production CLI composition, installed launcher/native execution) | deferred to v0.9.91 milestone end; human_needed | Phase 63 Plan 06 | -| uat_gap | Phase 62 / 62-HUMAN-UAT.md (installed doctor/genuine stream, compatibility layout, keyboard/accessibility/live refresh) | deferred to v0.9.91 milestone end; 3 scenarios | Phase 62 Plan 06 | -| uat_gap | Phase 61 / 61-HUMAN-UAT.md (consent/theme/handoff, authenticated stream, worker eviction, 45-minute endurance, POSIX lifecycle, daemon restart classification) | deferred to v0.9.91 milestone end; 8 scenarios | Phase 61 Plan 08 | -| uat_gap | Phase 60 / 60-HUMAN-UAT.md (authenticated isolation, genuine JSONL provenance, live MCP read, POSIX/Windows tree kill, crash recovery, browser ownership/vault/consent) | deferred to v0.9.91 milestone end; 7 scenarios | Phase 60 Plan 04 | -| uat_gap | Phase 59 / 59-HUMAN-UAT.md (live pair, daemon restart invalidation, Chrome session clearing, accessibility/theme) | deferred to v0.9.91 milestone end; 4 scenarios | Phase 59 Plan 04 | -| uat_gap | Phase 58 / 58-HUMAN-UAT.md + 58-VISUAL-QA.md (live Providers theme/responsive/keyboard/motion and extension-state checks) | deferred to v0.9.91 milestone end; 12 scenarios | Phase 58 Plan 03 | -| uat_gap | v1.2.0 Phase 53 / 53-VISUAL-QA.md (VISUAL-01 live DE/CJK browser visual spot-check) | human_needed | v1.2.0 Phase 53 | -| uat_gap | Phase 27 / 27-HUMAN-UAT.md (live FETCH-05 logged-in-shape UAT-27-01 + contrast + origin-pin) | human_needed; 3 scenarios | v0.9.99 Phase 27 | -| uat_gap | Phase 29 / 29-HUMAN-UAT.md ([ASSUMED] internal-endpoint live capture) | human_needed | v0.9.99 Phase 29 | -| uat_gap | Phase 30 / 30-HUMAN-UAT.md (UAT-30-01 live render/Grant/badge smoke) | human_needed | v0.9.99 Phase 30 | -| uat_gap | Phase 31 / live discovery UAT | human_needed | v0.9.99 Phase 31 | -| uat_gap | Phase 32 / 32-HUMAN-UAT.md (UAT-32-01 live self-healing) | human_needed; partial | v0.9.99 Phase 32 | -| uat_gap | Phase 33 / 33-HUMAN-UAT.md (live media playback fidelity) | human_needed | v0.9.99 Phase 33 | -| uat_gap | Phase 34 / 34-HUMAN-UAT.md (live upload fidelity) | partial: UAT-34-02 pass; UAT-34-01 MCP text-file smoke pass, binary/submit checks still human_needed; UAT-34-03/04 still human_needed | v0.9.99 Phase 34 | -| uat_gap | Phases 01/16/20/25 (v0.10/v0.11/v0.12 live-browser) | human_needed/partial | prior closes | +| uat_gap | Phase 65 / 65-HUMAN-UAT.md (genuine Codex auth matrix, live Codex/browser lifecycle, accessibility/responsive rendering) | closed 2026-09-28: waived by user, not executed; 3 scenarios | Phase 65 Plan 08 | +| uat_gap | Phase 64 / 64-HUMAN-UAT.md (genuine OpenCode account/process/browser/accessibility scenarios) | closed 2026-09-28: waived by user, not executed; 3 scenarios | Phase 64 | +| uat_gap | Phase 63 / 63-HUMAN-UAT.md + Plans 05-06 (real POSIX ownership/modes, Windows HKCU/WOW64/PE/bootstrap behavior, Chrome discovery, production CLI composition, installed launcher/native execution) | closed 2026-09-28: waived by user, not executed; 8 scenarios | Phase 63 Plan 06 | +| uat_gap | Phase 62 / 62-HUMAN-UAT.md (installed doctor/genuine stream, compatibility layout, keyboard/accessibility/live refresh) | closed 2026-09-28: waived by user, not executed; 3 scenarios | Phase 62 Plan 06 | +| uat_gap | Phase 61 / 61-HUMAN-UAT.md (consent/theme/handoff, authenticated stream, worker eviction, 45-minute endurance, POSIX lifecycle, daemon restart classification) | closed 2026-09-28: waived by user, not executed; 8 scenarios | Phase 61 Plan 08 | +| uat_gap | Phase 60 / 60-HUMAN-UAT.md (authenticated isolation, genuine JSONL provenance, live MCP read, POSIX/Windows tree kill, crash recovery, browser ownership/vault/consent) | closed 2026-09-28: waived by user, not executed; 7 scenarios | Phase 60 Plan 04 | +| uat_gap | Phase 59 / 59-HUMAN-UAT.md (live pair, daemon restart invalidation, Chrome session clearing, accessibility/theme) | closed 2026-09-28: waived by user, not executed; 4 scenarios | Phase 59 Plan 04 | +| uat_gap | Phase 58 / 58-HUMAN-UAT.md + 58-VISUAL-QA.md (live Providers theme/responsive/keyboard/motion and extension-state checks) | closed 2026-09-28: waived by user, not executed; 12 scenarios | Phase 58 Plan 03 | +| verification_gap | Phases 58-65 / *-VERIFICATION.md `human_needed` items (the human halves of the ledgers above) | closed 2026-09-28: waived by user; automated verification had passed | v0.9.91 phases | +| uat_gap | v1.2.0 Phase 53 / 53-VISUAL-QA.md (VISUAL-01 live DE/CJK browser visual spot-check) | closed 2026-09-28: waived by user, not executed | v1.2.0 Phase 53 | +| uat_gap | Phase 27 / 27-HUMAN-UAT.md (live FETCH-05 logged-in-shape UAT-27-01 + contrast + origin-pin) | closed 2026-09-28: waived by user, not executed; 3 scenarios | v0.9.99 Phase 27 | +| uat_gap | Phase 29 / 29-HUMAN-UAT.md ([ASSUMED] internal-endpoint live capture) | closed 2026-09-28: waived by user, not executed | v0.9.99 Phase 29 | +| uat_gap | Phase 30 / 30-HUMAN-UAT.md (UAT-30-01 live render/Grant/badge smoke) | closed 2026-09-28: waived by user, not executed | v0.9.99 Phase 30 | +| uat_gap | Phase 31 / live discovery UAT | closed 2026-09-28: waived by user, not executed | v0.9.99 Phase 31 | +| uat_gap | Phase 32 / 32-HUMAN-UAT.md (UAT-32-01 live self-healing) | closed 2026-09-28: waived by user, not executed (was partial) | v0.9.99 Phase 32 | +| uat_gap | Phase 33 / 33-HUMAN-UAT.md (live media playback fidelity) | closed 2026-09-28: waived by user, not executed | v0.9.99 Phase 33 | +| uat_gap | Phase 34 / 34-HUMAN-UAT.md (live upload fidelity) | closed 2026-09-28: UAT-34-02 and the UAT-34-01 MCP text-file smoke had passed; the remaining binary/submit checks and UAT-34-03/04 waived by user, not executed | v0.9.99 Phase 34 | +| uat_gap | Phases 01/16/20/25 (v0.10/v0.11/v0.12 live-browser) | closed 2026-09-28: waived by user, not executed | prior closes | +| quick_task | 116 entries flagged by `audit-open` in `.planning/quick/` | closed 2026-09-28 (acknowledged). 104 have summaries and were executed; the audit flags them only because it reads `SUMMARY.md` while GSD's quick workflow writes `-SUMMARY.md` (or records `status: completed`). The other 12 were not completed: 9 plan-only (never executed), `260701-e69` (steam, blocked), `260701-2m4` (sentry, implemented but uncommitted) and `260701-e6d` (teams, implemented) | quick tasks 2026-06-17 through 2026-07-28 | +| debug | fsb-trigger-arm-masked (MCP 0.10.0 trigger arm error masked as page navigation) | closed 2026-09-28 (acknowledged): diagnosis-only; root cause of the masking found, fix not applied | 2026-06-19 | | i18n_debt | WARNING-02 picker-cookie short-circuits bare-`/` Accept-Language redirect | closed in v1.2.0 Phase 56 (ROUTE-01/02) | v0.9.63, carried 6+ milestones | +Phase 66 plan 66-01 (H1 history export) was closed on 2026-10-12, not executed. The 40 MCP sessions from 2026-09-16..21 were lost when the Web Store build was reinstalled on 2026-10-09, and no backup exists. No Phase 66 success criterion or MEAS requirement depends on them; see `66-01-SUMMARY.md`. + Carry-forward publish/tag gates (pre-existing, user-gated): `npm publish fsb-mcp-server@0.9.0`; `npm publish fsb-mcp-server@0.10.0`; branch + tag pushes for v0.9.62 / v0.9.63 / v0.9.69 / v0.10.0 / v0.11.0 / v0.12.0 / v1.2.0; `clawhub publish "skills/FSB Skill"`; public package publication. None of this blocks v0.9.91. v2 deferred (see REQUIREMENTS.md v0.9.91 v2 section): CHAT-FUTURE-01/02 (chat-mode continuity via `--resume` / `codex resume` / `opencode --continue` + per-thread cwd pinning); GEMINI-FUTURE-01 (Gemini CLI adapter after live `--help` capture + JSONL schema pinning); ACP-FUTURE-01 (`@zed-industries/agent-client-protocol` unification once ≥2 non-Claude adapters have shipped); REMOTE-FUTURE-01 (remote/mobile delegation surfaces — v0.9.91 is explicitly localhost-only). ## Session Continuity -Last session: 2026-07-22T18:44:34.205Z -Stopped at: Phase 65 automated verification passed; human UAT65-01 through UAT65-03 pending -Resume file: None +Last session: 2026-10-19T05:45:24.000Z +Stopped at: v1.0.0 code-complete; Validation Block V1-V6 pending (see .planning/v1.0.0-VALIDATION-BLOCK.md) +Resume file: .planning/v1.0.0-VALIDATION-BLOCK.md ## Next Actions -Execute 64-10-PLAN.md next. Generalize durable event/controller hydration and safe per-adapter drift diagnostics while preserving the canonical accepted-run provider context established in Plan 09. +Owner: answer the section 1 decisions in `.planning/v1.0.0-VALIDATION-BLOCK.md` (items 1-10 are signed with M-71-00 after the 66-10 smoke), then run V1 from a worktree pinned at `8a058bdf`. Code-changing decisions (the gate amendment, the M-70-08 remedy, the D-25 widening) must land before the V4 release-candidate SHA. Nothing reaches `main` before V6. ## Performance Metrics @@ -467,3 +507,15 @@ Execute 64-10-PLAN.md next. Generalize durable event/controller hydration and sa | Phase 65 P06 | 40 min | 2 tasks | 16 files | | Phase 65 P07 | 21m | 2 tasks | 8 files | | Phase 65 P08 | 55m | 2 tasks | 22 files | +| Phase 66 P02 | 16 min | 3 tasks | 11 files | +| Phase 66 P03 | 19 min | 3 tasks | 12 files | +| Phase 66 P04 | 24 min | 3 tasks | 9 files | +| Phase 66 P05 | 43 min | 3 tasks | 13 files | +| Phase 66 P06 | 23 min | 3 tasks | 4 files | +| Phase 66 P07 | 46 min | 3 tasks | 34 files | +| Phase 66 P08 | 35 min | 3 tasks | 11 files | +| Phase 66 P09 | 12 min | 2 tasks | 2 files | + +## Operator Next Steps + +- Read `.planning/v1.0.0-VALIDATION-BLOCK.md` section 1 and record the decisions, then start V1: pin worktree at `8a058bdf`, pinned Claude Code and Chrome for Testing, the 66-10 bench setup and smoke, the M-71-00 sign-off, then the 30-round `core-baseline-66` campaign. diff --git a/.planning/milestones/v1.0.0-MILESTONE-AUDIT.md b/.planning/milestones/v0.13.0-MILESTONE-AUDIT.md similarity index 93% rename from .planning/milestones/v1.0.0-MILESTONE-AUDIT.md rename to .planning/milestones/v0.13.0-MILESTONE-AUDIT.md index 1043f68c4..7694c8383 100644 --- a/.planning/milestones/v1.0.0-MILESTONE-AUDIT.md +++ b/.planning/milestones/v0.13.0-MILESTONE-AUDIT.md @@ -1,5 +1,5 @@ --- -milestone: v1.0.0 +milestone: v0.13.0 milestone_name: Full App Catalog (OpenTabs Parity) audited: 2026-06-26 status: passed @@ -28,7 +28,7 @@ tech_debt: - "UATX-01: per-app live guarded-write UAT closeout across the depth tier — v2 (mirrors the v0.9.99 live-UAT posture)." - phase: process items: - - "9/10 v1.0.0 VALIDATION.md files carry a stale `nyquist_compliant: false` frontmatter flag (only Phase 37 was flipped to true). Coverage IS present — every phase has a VALIDATION.md with the per-SC automated-verify map and passed verification green — only the draft flag was never flipped. A doc-flag gap, not a coverage gap." + - "9/10 v0.13.0 VALIDATION.md files carry a stale `nyquist_compliant: false` frontmatter flag (only Phase 37 was flipped to true). Coverage IS present — every phase has a VALIDATION.md with the per-SC automated-verify map and passed verification green — only the draft flag was never flipped. A doc-flag gap, not a coverage gap." nyquist: compliant_phases: ["37"] partial_phases: ["35","36","38","39","39.5","40","41","42","43"] @@ -36,7 +36,7 @@ nyquist: overall: partial # all phases HAVE a VALIDATION.md + passed verification; the nyquist_compliant flag is stale on 9 (coverage present) --- -# v1.0.0 — Full App Catalog (OpenTabs Parity) — Milestone Audit +# v0.13.0 — Full App Catalog (OpenTabs Parity) — Milestone Audit **Status: PASSED.** All 17 requirements satisfied (3-source cross-referenced), all 10 phases verified, cross-phase integration 5/5 wired end-to-end, and the milestone gate (`full npm test EXIT 0`) is met in the real working tree. No critical gaps. Accumulated tech debt is all NON-blocking (human-needed live UAT on fail-closed/inert surfaces, explicitly-deferred v2 families, a post-milestone UI follow-up, and a stale doc-flag). @@ -118,4 +118,4 @@ Every phase has a VALIDATION.md with a per-SC automated-verify map and passed it ## Verdict -**v1.0.0 PASSED.** Ready to complete the milestone (archive + tag). The tech debt is tracked + non-blocking; recommend carrying the live-UAT + v2 families into the backlog. +**v0.13.0 PASSED.** Ready to complete the milestone (archive + tag). The tech debt is tracked + non-blocking; recommend carrying the live-UAT + v2 families into the backlog. diff --git a/.planning/milestones/v1.0.0-REQUIREMENTS.md b/.planning/milestones/v0.13.0-REQUIREMENTS.md similarity index 87% rename from .planning/milestones/v1.0.0-REQUIREMENTS.md rename to .planning/milestones/v0.13.0-REQUIREMENTS.md index 75a629eb4..cb3a7fb3e 100644 --- a/.planning/milestones/v1.0.0-REQUIREMENTS.md +++ b/.planning/milestones/v0.13.0-REQUIREMENTS.md @@ -1,11 +1,11 @@ -# Requirements: FSB (Full Self-Browsing) -- v1.0.0 Full App Catalog (OpenTabs Parity) +# Requirements: FSB (Full Self-Browsing) -- v0.13.0 Full App Catalog (OpenTabs Parity) **Defined:** 2026-06-23 -**Core Value:** Reliable single-attempt execution -- the AI decides correctly; the mechanics execute precisely. v1.0.0 scales the v0.9.99 capability path from a 4-service head to the full ~119-app OpenTabs surface by FEEDING THE EXISTING TIERS (breadth = closed-vocab descriptors as data; depth = hand-ported handlers; tail = seeded discovery + DOM fallback) -- no new MCP tool, no new router branch, INV-01..04 + Walls 1/2 preserved by construction. +**Core Value:** Reliable single-attempt execution -- the AI decides correctly; the mechanics execute precisely. v0.13.0 scales the v0.9.99 capability path from a 4-service head to the full ~119-app OpenTabs surface by FEEDING THE EXISTING TIERS (breadth = closed-vocab descriptors as data; depth = hand-ported handlers; tail = seeded discovery + DOM fallback) -- no new MCP tool, no new router branch, INV-01..04 + Walls 1/2 preserved by construction. ## v1 Requirements -Requirements for the v1.0.0 milestone. Each maps to a roadmap phase (Phases 35+, continuing integer numbering from v0.9.99's Phase 34). Source: github.com/opentabs-dev/opentabs (MIT, 119 plugins / 2,523 ops), pinned by commit SHA; attribution already in README Acknowledgements. +Requirements for the v0.13.0 milestone. Each maps to a roadmap phase (Phases 35+, continuing integer numbering from v0.9.99's Phase 34). Source: github.com/opentabs-dev/opentabs (MIT, 119 plugins / 2,523 ops), pinned by commit SHA; attribution already in README Acknowledgements. ### DENY -- Denylist Expansion & Import-Time Classification Gate (lands FIRST) @@ -44,7 +44,7 @@ Requirements for the v1.0.0 milestone. Each maps to a roadmap phase (Phases 35+, ## v2 Requirements -Acknowledged but deferred beyond v1.0.0; not in this roadmap. +Acknowledged but deferred beyond v0.13.0; not in this roadmap. ### Deferred Capability Families @@ -68,7 +68,7 @@ Explicit exclusions (anti-features from research), documented to prevent scope c Finalized by the roadmapper; each requirement maps to exactly one phase. 100% coverage, 0 orphans. -Note on breadth phases: BRDTH-01/02/03 are OWNED by Phase 37, which establishes the breadth contract (all real apps return from search with intent synonyms + side-effect class + backing-status, each batch gated on denylist coverage). Phases 38 (Comms/Social/Content) and 39 (Commerce/Travel/Misc) are sensitivity-ascending CONTINUATION batches that extend the same three requirements to more categories; they own no additional v1.0.0 REQ-ID. Depth splits cleanly: DEPTH-01 (the hand-port contract + read heads) is owned by Phase 40; DEPTH-02 (guarded writes + DENY-04 re-enforcement + per-app CORS gate) is owned by Phase 41, which continues the same hand-port mechanism for the remaining heads. +Note on breadth phases: BRDTH-01/02/03 are OWNED by Phase 37, which establishes the breadth contract (all real apps return from search with intent synonyms + side-effect class + backing-status, each batch gated on denylist coverage). Phases 38 (Comms/Social/Content) and 39 (Commerce/Travel/Misc) are sensitivity-ascending CONTINUATION batches that extend the same three requirements to more categories; they own no additional v0.13.0 REQ-ID. Depth splits cleanly: DEPTH-01 (the hand-port contract + read heads) is owned by Phase 40; DEPTH-02 (guarded writes + DENY-04 re-enforcement + per-app CORS gate) is owned by Phase 41, which continues the same hand-port mechanism for the remaining heads. | Requirement | Phase | Status | |-------------|-------|--------| @@ -96,5 +96,5 @@ Note on breadth phases: BRDTH-01/02/03 are OWNED by Phase 37, which establishes - Unmapped: 0 --- -*Requirements defined: 2026-06-23 -- v1.0.0 Full App Catalog (OpenTabs Parity)* +*Requirements defined: 2026-06-23 -- v0.13.0 Full App Catalog (OpenTabs Parity)* *Last updated: 2026-06-24 -- traceability finalized by roadmapper (9 phases 35-43; DEPTH-02 mapped to Phase 41; BRDTH continuation batches 38-39 noted)* diff --git a/.planning/milestones/v1.0.0-ROADMAP.md b/.planning/milestones/v0.13.0-ROADMAP.md similarity index 95% rename from .planning/milestones/v1.0.0-ROADMAP.md rename to .planning/milestones/v0.13.0-ROADMAP.md index 815d6a119..c42b54a58 100644 --- a/.planning/milestones/v1.0.0-ROADMAP.md +++ b/.planning/milestones/v0.13.0-ROADMAP.md @@ -1,16 +1,18 @@ # Roadmap: FSB (Full Self-Browsing) +> **Relabel note (2026-09-28):** This milestone shipped as v1.0.0 and was relabeled v0.13.0 so that v1.0.0 names the Jev Fast Mode milestone that ships extension 1.0.0. + ## Milestones - **v0.10.0 Autopilot via Lattice SDK** — Phases 01-13, shipped 2026-06-15. - **v0.11.0 Trigger Tool (Reactive DOM Monitoring)** — Phases 14-20, completed 2026-06-17; release actions and browser UAT remain user-gated. - **v0.12.0 PhantomStream Package Migration** — Phases 21-25, completed 2026-06-17; live Chrome-extension UAT remains user-gated. - **v0.9.99 Native Capability Catalog (FSB API Execution)** — Phases 26-34, code-complete 2026-06-23 (full npm test EXIT 0); live-browser UAT debt carried forward. Phase 999.1 (click heuristics) retained in Backlog. -- 🚧 **v1.0.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, IN PROGRESS. Scales the v0.9.99 capability path from a 4-service head to the full ~119-app / ~2,523-op OpenTabs surface by feeding the existing tiers (breadth = descriptors as data; depth = hand-ported handlers; tail = seeded discovery + DOM fallback). Denylist-first, category-batched. +- 🚧 **v0.13.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, IN PROGRESS. Scales the v0.9.99 capability path from a 4-service head to the full ~119-app / ~2,523-op OpenTabs surface by feeding the existing tiers (breadth = descriptors as data; depth = hand-ported handlers; tail = seeded discovery + DOM fallback). Denylist-first, category-batched. ## Active Milestone -**v1.0.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43. +**v0.13.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43. **Milestone Goal:** Take FSB's capability catalog from the 4-service bundled head to the full ~119-app OpenTabs surface — every app discoverable via `search_capabilities` and invocable through the existing tiers — adding a hand-ported depth tier, an expanded service denylist, and seeded discovery, all WITHOUT violating MV3 Wall 1 (recipes stay closed-vocabulary data, never shipped code). The milestone is **NOT** a rebuild and **NOT** a port of OpenTabs' code: it is "feed the existing tiers, don't add tiers." Breadth = codegen closed-vocabulary **descriptors** (data) so every app returns from search; depth = hand-port ~15-30 top apps as T1a/T1b handlers exactly like the shipped `github.js`; the tail is reached by seeded network-capture discovery (T2) and the universal DOM fallback (T3). No new MCP tool, no new router branch, no new tier — INV-01..04 and Walls 1/2 are preserved by construction. The single new build-time stack is `zod@^4.4.3` + `tsx@^4.22.4` (devDeps only) to run OpenTabs' own `z.toJSONSchema()` extraction; nothing OpenTabs ships into the extension at runtime (Wall 1). @@ -48,7 +50,7 @@ ### Phase 35: Denylist Expansion + Import-Time Classification Gate (LANDS FIRST) **Goal**: Make the service-denylist cover every categorically-prohibited and allowed-but-sensitive origin across the 119-app set, and make that coverage un-bypassable, BEFORE any descriptor that could reach a finance/health/ToS-hostile app is ever emitted — under the shipped opt-out Auto default the denylist is the ONE hard floor, so this is a hard dependency for every later phase, not a parallel track. -**Depends on**: Nothing within v1.0.0 (extends the v0.9.99 substrate; `service-denylist.js` loader unchanged). Strictly first. +**Depends on**: Nothing within v0.13.0 (extends the v0.9.99 substrate; `service-denylist.js` loader unchanged). Strictly first. **Requirements**: DENY-01, DENY-02, DENY-03, DENY-04 **Success Criteria** (what must be TRUE): 1. `service-denylist.json` `deniedOrigins` hard-blocks the categorically-prohibited OpenTabs apps (brokerage/trading: robinhood, fidelity, carta; ToS-hostile media/social: netflix, spotify, twitch, steam, youtube-music, tinder, onlyfans; and the write-paths of instagram/facebook/tiktok/x) — confirmed by a test asserting each prohibited origin classifies `denied`. @@ -94,7 +96,7 @@ ### Phase 38: Breadth B — Comms / Social / Content (sensitivity-screened) **Goal**: Extend the Phase-37 breadth contract to the comms / social / content apps — the first batch where ToS-hostility and write-sensitivity bite — importing each app's descriptors only after Phase 35 covers its origin, and routing ToS-hostile apps to DOM-only/denied rather than API-invocable. -**Depends on**: Phase 37 (reuses the breadth pipeline + the backing-status + the batch-gating contract). Continuation of BRDTH-01/02/03 to additional categories (no new v1.0.0 REQ-ID owned). +**Depends on**: Phase 37 (reuses the breadth pipeline + the backing-status + the batch-gating contract). Continuation of BRDTH-01/02/03 to additional categories (no new v0.13.0 REQ-ID owned). **Success Criteria** (what must be TRUE): 1. Descriptors for the screened comms/social/content apps (discord, bluesky, reddit-extra, chatgpt, claude, …) are imported and returned by `search_capabilities`, each included only after a per-app sensitivity check. 2. A denylist-coverage assertion for this batch passes before merge (every sensitive social/messaging origin in the batch is classified denied-or-sensitive); ToS-hostile apps carry a DOM-only routing marker (T3) and are never fully-API-invocable-by-default. @@ -106,7 +108,7 @@ ### Phase 39: Breadth C — Commerce / Travel / Misc (most-sensitive) **Goal**: Complete descriptor coverage of all real OpenTabs apps by importing the commerce / travel / misc batch (the most-sensitive end of the ascending order), screening out or denying payment-bearing flows so no descriptor arms a money-moving operation under Auto. -**Depends on**: Phase 38 (final, most-sensitive breadth batch on the same gated pipeline). Continuation of BRDTH-01/02/03 (no new v1.0.0 REQ-ID owned). +**Depends on**: Phase 38 (final, most-sensitive breadth batch on the same gated pipeline). Continuation of BRDTH-01/02/03 (no new v0.13.0 REQ-ID owned). **Success Criteria** (what must be TRUE): 1. Descriptors for the commerce/travel/misc apps (booking, airbnb, bestbuy, costco, craigslist, dominos, chipotle, calendly, …) are imported and returned by `search_capabilities`, completing coverage of all real apps (excluding fixtures + the denied set). 2. A denylist-coverage assertion passes for this batch; payment-flow ops fail-closed (T3 DOM) or are denied — no payment-bearing op is API-invocable under Auto. @@ -122,7 +124,7 @@ ### Phase 39.5: Full OpenTabs Source Import (real ~117-plugin / ~2,374-op catalog) [INSERTED] **Goal**: Achieve true OpenTabs parity — replace the hand-authored representative breadth corpus (~53 apps / ~228 descriptors) with the REAL OpenTabs source imported via the proven pipeline. Fetch the pinned-SHA repo tarball, vendor all 117 real plugin metadata slices (no sdk-stub — the real `@opentabs-dev/plugin-sdk` imports cleanly under tsx), run the frozen importer to emit ~2,374 real descriptors, re-screen all ~114 origins, and AUGMENT (preserve the 13 hand-authored-only apps that have no upstream at the SHA). De-risked by FULL-PARITY-SPIKE.md (GO; 117/117 clean dry-run imports). -**Depends on**: Phase 39 (reuses the frozen screen-then-import pipeline + the payment-op guard + the screen-then-import contract at full scale). Continuation of BRDTH-01/02/03 (no new v1.0.0 REQ-ID owned) — closes the milestone's literal "OpenTabs Parity" goal. +**Depends on**: Phase 39 (reuses the frozen screen-then-import pipeline + the payment-op guard + the screen-then-import contract at full scale). Continuation of BRDTH-01/02/03 (no new v0.13.0 REQ-ID owned) — closes the milestone's literal "OpenTabs Parity" goal. **Success Criteria** (what must be TRUE): 1. The 117 real OpenTabs plugins (119 dirs − 2 CI fixtures) are vendored from the pinned SHA (real `src/**` metadata, no stub) and imported as ~2,374 descriptors (excluding grafana/sqlpad empty-origin + carving `target.apply_promo_code`'s `code` forbidden field); the 40 overlapping hand-authored apps are regenerated from real source; the 13 hand-authored-only apps (amazon/shopify/lyft/ubereats/threads/mastodon/etc.) are PRESERVED (augment, not replace). Coverage report shows realAppCount ≈ 117 real + 13 hand-only. 2. STEM_OVERRIDES is extended for all ~40 wrong/colliding-stem apps (6 collision groups incl 4-way `console`) with a CI no-duplicate-stem assertion; 0 colliding `opentabs____*` filenames. @@ -223,7 +225,7 @@ Phases execute in numeric order: 35 → 36 → 37 → 38 → 39 → 40 → 41
v0.9.99 Native Capability Catalog (FSB API Execution) — Phases 26-34, CODE-COMPLETE 2026-06-23 -Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v1.0.0 milestone extends this substrate verbatim. +Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v0.13.0 milestone extends this substrate verbatim. | Phase | Name | Plans | Status | |-------|------|-------|--------| @@ -237,7 +239,7 @@ Gave FSB first-class authenticated-API execution as a fast path alongside DOM au | 33 | PhantomStream Media Mirroring (0.2.1 Uptake) — milestone extension | 1/1 | Complete; live media UAT human_needed | | 34 | Explicit File Upload Tool (upload_file) — milestone extension | 1/1 | Complete; live upload UAT human_needed | -Substrate carried into v1.0.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard. +Substrate carried into v0.13.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard.
@@ -318,8 +320,8 @@ Known deferred closeout evidence: 11 human-gated Chrome MV3/UAT verification ite ## Carry-Forward Candidates -- **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. v1.0.0 does NOT block on it. -- **v2 deferred capability families (acknowledged, out of v1.0.0):** GAPI-01 (gapi-bridge handler family for Google Workspace via the `window.gapi.client.request` trampoline); CLOUD-01 (cloud-console Pattern-D ports — aws-console/azure/google-cloud/terraform-cloud — pending per-app CORS verification); UATX-01 (per-app live guarded-write UAT closeout across the hand-ported depth tier). +- **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. v0.13.0 does NOT block on it. +- **v2 deferred capability families (acknowledged, out of v0.13.0):** GAPI-01 (gapi-bridge handler family for Google Workspace via the `window.gapi.client.request` trampoline); CLOUD-01 (cloud-console Pattern-D ports — aws-console/azure/google-cloud/terraform-cloud — pending per-app CORS verification); UATX-01 (per-app live guarded-write UAT closeout across the hand-ported depth tier). - **Delegation primitive:** Parked from v0.10.0; re-scope as either a Lattice-owned primitive or an FSB-only consumer of Lattice receipt + tripwire surfaces. ## Backlog diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-PLAN.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-PLAN.md similarity index 99% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-PLAN.md index 8f44a7eba..63e36ea0b 100644 --- a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-PLAN.md +++ b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-PLAN.md @@ -113,7 +113,7 @@ Current JSON envelope (extension/config/service-denylist.json, v:1): - extension/utils/service-denylist.js (confirm the exact-vs-suffix matcher so each pattern form is correct) - Append to `deniedOrigins` (DENY-01, categorically prohibited) these EXACT patterns, keeping the existing chase/bankofamerica/wellsfargo/irs.gov entries: `https://*.robinhood.com` (suffix), `https://digital.fidelity.com` (exact — NOT *.fidelity.com), `https://app.carta.com` (exact), `https://*.netflix.com` (suffix), `https://open.spotify.com` (exact), `https://*.twitch.tv` (suffix), `https://store.steampowered.com` (exact), `https://music.youtube.com` (exact — NOT *.youtube.com, which would catch youtube proper), `https://*.tinder.com` (suffix), `https://*.onlyfans.com` (suffix). Append to `sensitiveOrigins` (DENY-02, allowed-but-sensitive) these EXACT patterns, keeping the existing bank/mail/*.gov entries: `https://dashboard.stripe.com` (exact — the dashboard origin, NOT api.stripe.com), `https://*.coinbase.com` (suffix), `https://console.twilio.com` (exact), `https://app.ynab.com` (exact), `https://*.instagram.com` (suffix), `https://*.facebook.com` (suffix), `https://*.tiktok.com` (suffix), `https://*.x.com` (suffix), and the messaging set (locked per 35-RESEARCH Q1/A2 — writes mutating-gated under posture B, reads under Auto): `https://web.whatsapp.com` (exact), `https://web.telegram.org` (exact), `https://*.slack.com` (suffix), `https://discord.com` (exact), `https://teams.live.com`, `https://teams.microsoft.com`, `https://teams.cloud.microsoft` (three exact). instagram/facebook/tiktok/x go in sensitiveOrigins ONLY (NOT deniedOrigins) per CONTEXT — fully-denied is reserved for the brokerage + ToS-hostile-media set. Do NOT change the loader, the `v` version, or `deniedReason`. Optionally extend the leading `_comment` to note the v1.0.0 roster expansion (do not remove the existing comment text). Keep the file ASCII-only, no trailing commas (valid JSON). + Append to `deniedOrigins` (DENY-01, categorically prohibited) these EXACT patterns, keeping the existing chase/bankofamerica/wellsfargo/irs.gov entries: `https://*.robinhood.com` (suffix), `https://digital.fidelity.com` (exact — NOT *.fidelity.com), `https://app.carta.com` (exact), `https://*.netflix.com` (suffix), `https://open.spotify.com` (exact), `https://*.twitch.tv` (suffix), `https://store.steampowered.com` (exact), `https://music.youtube.com` (exact — NOT *.youtube.com, which would catch youtube proper), `https://*.tinder.com` (suffix), `https://*.onlyfans.com` (suffix). Append to `sensitiveOrigins` (DENY-02, allowed-but-sensitive) these EXACT patterns, keeping the existing bank/mail/*.gov entries: `https://dashboard.stripe.com` (exact — the dashboard origin, NOT api.stripe.com), `https://*.coinbase.com` (suffix), `https://console.twilio.com` (exact), `https://app.ynab.com` (exact), `https://*.instagram.com` (suffix), `https://*.facebook.com` (suffix), `https://*.tiktok.com` (suffix), `https://*.x.com` (suffix), and the messaging set (locked per 35-RESEARCH Q1/A2 — writes mutating-gated under posture B, reads under Auto): `https://web.whatsapp.com` (exact), `https://web.telegram.org` (exact), `https://*.slack.com` (suffix), `https://discord.com` (exact), `https://teams.live.com`, `https://teams.microsoft.com`, `https://teams.cloud.microsoft` (three exact). instagram/facebook/tiktok/x go in sensitiveOrigins ONLY (NOT deniedOrigins) per CONTEXT — fully-denied is reserved for the brokerage + ToS-hostile-media set. Do NOT change the loader, the `v` version, or `deniedReason`. Optionally extend the leading `_comment` to note the v0.13.0 roster expansion (do not remove the existing comment text). Keep the file ASCII-only, no trailing commas (valid JSON). node -e "const j=require('./extension/config/service-denylist.json'); const need=['https://digital.fidelity.com','https://music.youtube.com','https://store.steampowered.com','https://open.spotify.com','https://app.carta.com','https://*.robinhood.com','https://*.netflix.com','https://*.twitch.tv','https://*.tinder.com','https://*.onlyfans.com']; const sens=['https://dashboard.stripe.com','https://*.coinbase.com','https://console.twilio.com','https://app.ynab.com','https://*.instagram.com','https://*.facebook.com','https://*.tiktok.com','https://*.x.com']; const md=need.filter(o=>!j.deniedOrigins.includes(o)); const ms=sens.filter(o=>!j.sensitiveOrigins.includes(o)); if(md.length||ms.length){console.error('MISSING denied:',md,'sensitive:',ms);process.exit(1)} if(j.deniedOrigins.includes('https://*.instagram.com')){console.error('instagram must NOT be denied');process.exit(1)} console.log('denylist roster OK')" diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-SUMMARY.md similarity index 98% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-SUMMARY.md index 15c1761e9..99dbfa065 100644 --- a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-SUMMARY.md +++ b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-01-SUMMARY.md @@ -75,7 +75,7 @@ Each task was committed atomically (TDD RED→GREEN; Task 3 is the GREEN verific **Plan metadata:** see final docs commit. ## Files Created/Modified -- `extension/config/service-denylist.json` - Appended 10 denied roster patterns + 15 sensitive roster patterns (8 core + 7 messaging) to the existing seed arrays; extended `_comment` to note the v1.0.0 roster expansion and the exact-host rationale. `v:1` and `deniedReason` unchanged. +- `extension/config/service-denylist.json` - Appended 10 denied roster patterns + 15 sensitive roster patterns (8 core + 7 messaging) to the existing seed arrays; extended `_comment` to note the v0.13.0 roster expansion and the exact-host rationale. `v:1` and `deniedReason` unchanged. - `tests/service-denylist.test.js` - Added a Phase-35 assertion block (inside the existing async IIFE, after the GOV-08 checks): per-origin `classify().denied===true` over the denied roster, per-origin `{sensitive:true, denied:false}` over the sensitive roster, three exact-host discrimination assertions, negative controls (www.fidelity.com non-denied, github.com clean), and a chase-seed regression guard. The pre-existing GOV-08 assertions were extended, not replaced. ## Decisions Made diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-PLAN.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-PLAN.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-PLAN.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-CONTEXT.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-RESEARCH.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-RESEARCH.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-RESEARCH.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-RESEARCH.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-REVIEW.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-REVIEW.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VALIDATION.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/35-denylist-expansion-import-time-classification-gate-lands-fir/35-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-PLAN.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-PLAN.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-PLAN.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-PLAN.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-CONTEXT.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-RESEARCH.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-RESEARCH.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-RESEARCH.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-RESEARCH.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-REVIEW.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-REVIEW.md similarity index 98% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-REVIEW.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-REVIEW.md index b2a7bbed8..ef6d4fee4 100644 --- a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-REVIEW.md +++ b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-REVIEW.md @@ -279,7 +279,7 @@ The override only works because the slug-suffix branch (lines 179-188) saves it. **File:** `package.json:3` (`"version": "0.9.90"`), `:71` (`"license": "MIT"`), `:133` (badge `license-MIT`), `:24` (`package` script hardcodes `fsb-v0.9.90.zip`) -**Issue:** The package declares `version: 0.9.90` while the milestone/requirements target v1.0.0, and the `package` zip name is hardcoded to the version string (drifts silently on bump). The SPDX field, README badge, and vendored OpenTabs descriptors now consistently advertise MIT. The remaining version drift is not a Phase-36 regression or a code defect. +**Issue:** The package declares `version: 0.9.90` while the milestone/requirements target v0.13.0, and the `package` zip name is hardcoded to the version string (drifts silently on bump). The SPDX field, README badge, and vendored OpenTabs descriptors now consistently advertise MIT. The remaining version drift is not a Phase-36 regression or a code defect. **Fix:** Out of Phase-36 scope; recommend a version-metadata reconciliation pass that parameterizes the zip name. diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VALIDATION.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/36-codegen-pipeline-no-dead-entry-resolution/36-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-PLAN.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-PLAN.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-PLAN.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-PLAN.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-CONTEXT.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-REVIEW.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-REVIEW.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VALIDATION.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/37-breadth-a-dev-productivity-least-sensitive/37-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-PLAN.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-PLAN.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-PLAN.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-CONTEXT.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-CONTEXT.md similarity index 98% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-CONTEXT.md index b0ae28349..7dabe6b07 100644 --- a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-CONTEXT.md +++ b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-CONTEXT.md @@ -11,7 +11,7 @@ Extend the Phase-37 breadth contract to the comms / social / content apps — th where ToS-hostility + write-sensitivity bite. Import each app's descriptors ONLY after Phase 35 covers its origin (the merge-time classifyGate is the fail-closed enforcer), and route ToS-hostile apps to DOM-only/denied rather than API-invocable. Continues BRDTH-01/02/03 (no new -v1.0.0 REQ-ID). Reuses the FROZEN Phase-37 machinery verbatim + ADDS the per-app sensitivity +v0.13.0 REQ-ID). Reuses the FROZEN Phase-37 machinery verbatim + ADDS the per-app sensitivity screening (a denylist expansion for this batch's origins). **In scope:** the comms/social/content import batch; the per-app ToS/sensitivity classification diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-REVIEW.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-REVIEW.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VALIDATION.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VERIFICATION.md similarity index 99% rename from .planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VERIFICATION.md index 7f1fe0731..2ac52ab39 100644 --- a/.planning/milestones/v1.0.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VERIFICATION.md +++ b/.planning/milestones/v0.13.0-phases/38-breadth-b-comms-social-content-sensitivity-screened/38-VERIFICATION.md @@ -91,7 +91,7 @@ re_verification: | BRDTH-02 | 38-01, 38-03 | Apps imported in sensitivity-ascending batches; each gated on origins being denylist-classified before merge | ✓ SATISFIED | classifyGate fail-closed proven (aborts on unclassified social origin); all batch origins screened before 02/03 import. | | BRDTH-03 | 38-01, 38-02, 38-03 | Each descriptor carries invocability/backing-status signal | ✓ SATISFIED | All 24 descriptors backing='dom' → invocable=false / discovery-pending; backing-status annotation distinguishes invocable vs DOM-only. | -BRDTH-01/02/03 are Phase-37-owned; Phase 38 extends them to the comms/social/content category with no new v1.0.0 REQ-ID (consistent with REQUIREMENTS.md line 71). No orphaned requirements. +BRDTH-01/02/03 are Phase-37-owned; Phase 38 extends them to the comms/social/content category with no new v0.13.0 REQ-ID (consistent with REQUIREMENTS.md line 71). No orphaned requirements. ### Anti-Patterns Found diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-05-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-PLAN.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-PLAN.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-07-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-CONTEXT.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VALIDATION.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/FULL-PARITY-SPIKE.md b/.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/FULL-PARITY-SPIKE.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/FULL-PARITY-SPIKE.md rename to .planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/FULL-PARITY-SPIKE.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-01-PLAN.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-02-PLAN.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-03-PLAN.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-04-PLAN.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-05-PLAN.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-05-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-05-PLAN.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-05-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-05-SUMMARY.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-05-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-05-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-05-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-CONTEXT.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-REVIEW.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-REVIEW.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-VALIDATION.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-VERIFICATION.md similarity index 99% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-VERIFICATION.md index 63b710cde..f8a23807c 100644 --- a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/39.5-VERIFICATION.md +++ b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/39.5-VERIFICATION.md @@ -105,7 +105,7 @@ No `scripts/*/tests/probe-*.sh` probes are declared for this phase; verification ### Requirements Coverage -This phase is a continuation of BRDTH-01/02/03 (no new v1.0.0 REQ-ID owned — it closes the milestone's literal "OpenTabs Parity" goal). PLAN frontmatters declare: +This phase is a continuation of BRDTH-01/02/03 (no new v0.13.0 REQ-ID owned — it closes the milestone's literal "OpenTabs Parity" goal). PLAN frontmatters declare: | Requirement | Source Plans | Description | Status | Evidence | |-------------|--------------|-------------|--------|----------| diff --git a/.planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/deferred-items.md b/.planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/deferred-items.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/39.5-full-opentabs-source-import/deferred-items.md rename to .planning/milestones/v0.13.0-phases/39.5-full-opentabs-source-import/deferred-items.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-01-PLAN.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-02-PLAN.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-03-PLAN.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-04-PLAN.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-05-PLAN.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-05-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-05-PLAN.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-05-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-05-SUMMARY.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-05-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-05-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-05-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-CONTEXT.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-REVIEW.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-REVIEW.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-VALIDATION.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/40-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/40-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/deferred-items.md b/.planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/deferred-items.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/40-depth-1-top-read-hand-ports/deferred-items.md rename to .planning/milestones/v0.13.0-phases/40-depth-1-top-read-hand-ports/deferred-items.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-PLAN.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-PLAN.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-PLAN.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-PLAN.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-PLAN.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-PLAN.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-SUMMARY.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-05-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-CONTEXT.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-DEFERRAL.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-DEFERRAL.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-DEFERRAL.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-DEFERRAL.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-HUMAN-UAT.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-HUMAN-UAT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-HUMAN-UAT.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-HUMAN-UAT.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-REVIEW.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-REVIEW.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VALIDATION.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/41-depth-2-remaining-hand-ports-guarded-writes/41-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-01-PLAN.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-02-PLAN.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-03-PLAN.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-04-PLAN.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-04-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-04-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-04-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-04-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-05-PLAN.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-05-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-05-PLAN.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-05-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-05-SUMMARY.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-05-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-05-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-05-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-CONTEXT.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-CONTEXT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-CONTEXT.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-HUMAN-UAT.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-HUMAN-UAT.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-HUMAN-UAT.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-HUMAN-UAT.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-REVIEW.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-REVIEW.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-REVIEW.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-REVIEW.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-VALIDATION.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-VALIDATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-VALIDATION.md diff --git a/.planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-VERIFICATION.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/42-discovery-seeding-tail-learn/42-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/42-discovery-seeding-tail-learn/42-VERIFICATION.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-01-PLAN.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-01-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-01-PLAN.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-01-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-01-SUMMARY.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-01-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-01-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-01-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-02-PLAN.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-02-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-02-PLAN.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-02-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-02-SUMMARY.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-02-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-02-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-02-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-03-PLAN.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-03-PLAN.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-03-PLAN.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-03-PLAN.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-03-SUMMARY.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-03-SUMMARY.md similarity index 100% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-03-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-03-SUMMARY.md diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-04-PLAN.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-04-PLAN.md similarity index 97% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-04-PLAN.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-04-PLAN.md index 7f7460e24..1a3530058 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-04-PLAN.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-04-PLAN.md @@ -35,7 +35,7 @@ must_haves: --- -THE MILESTONE GATE (the final close of v1.0.0). This plan is mostly a verification/sign-off +THE MILESTONE GATE (the final close of v0.13.0). This plan is mostly a verification/sign-off battery: it asserts that the WHOLE milestone is green -- the SCALE-01 scale + precision HARD bars, the SCALE-02 self-heal tests, INV-03 7-provider byte-equality, MIT provenance, INV-01..04 + Walls 1/2 guards -- and that full `npm test` EXITS 0. That npm-test-EXIT-0 over the complete @@ -49,7 +49,7 @@ is recorded as a DOCUMENT (debt with optional live-confirmation steps), not a bl consistent with 43-VALIDATION.md ("no NEW human_needed item; carried-forward UAT is non-blocking debt") and the autonomous-run posture. -Purpose: prove the v1.0.0 milestone is met -- everything from breadth + depth + discovery + scale + +Purpose: prove the v0.13.0 milestone is met -- everything from breadth + depth + discovery + scale + self-heal is green and the full suite passes, with the live-UAT debt honestly surfaced. Output: the 43-MILESTONE-GATE.md sign-off record (every gate command + measured result + the carried-forward NON-blocking UAT debt + its optional live-confirmation steps). @@ -155,7 +155,7 @@ Sub-gates (record each number/result): THE MILESTONE GATE -- the whole suite: npm test Record the EXIT code. It MUST be 0. The whole-suite pass over breadth + depth + discovery + scale + -self-heal + provider parity + provenance + every guard IS the v1.0.0 milestone gate. +self-heal + provider parity + provenance + every guard IS the v0.13.0 milestone gate. If `npm test` exits non-zero: capture the FIRST failing test + its output, STOP, and report it as a milestone-blocking gap (the fix belongs in the responsible plan, not in this sign-off plan). Do NOT @@ -174,7 +174,7 @@ edit tests to make them pass. Create .planning/phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md recording the milestone sign-off from Task 1's measured results. Use the Write tool. Include: - - A header: "Phase 43 / v1.0.0 Milestone Gate -- Sign-Off" + the date. + - A header: "Phase 43 / v0.13.0 Milestone Gate -- Sign-Off" + the date. - A GATE RESULTS table: one row per gate (INV-03 provider parity, MIT provenance, SCALE-01 scale, SCALE-01 precision, SCALE-02 coalescing, SCALE-02 recurrence, SCALE-02 degraded, INV-01 MCP surface, INV-02 autopilot parity, validate:extension structural guards, full npm test) with @@ -182,7 +182,7 @@ milestone sign-off from Task 1's measured results. Use the Write tool. Include: (e.g. scale: "1.40MB / 11ms / 621B / 2318 descriptors"; precision: "wrong-invoke= (HARD bar), recall@5="; provider: "distinct=[], length=1"; provenance: " apps MIT + SHA-pinned"; npm test: "EXIT 0"). - - THE MILESTONE GATE line: full `npm test` EXIT 0 -- the authoritative v1.0.0 milestone signal. + - THE MILESTONE GATE line: full `npm test` EXIT 0 -- the authoritative v0.13.0 milestone signal. - A CARRIED-FORWARD NON-BLOCKING UAT DEBT section listing the two human_needed items verbatim from (41-HUMAN-UAT.md guarded-write mutation bodies; 42-HUMAN-UAT.md discovery first-visit), each marked NON-blocking with its fail-closed/inert rationale + a pointer to the @@ -228,14 +228,14 @@ ASCII-only, no emojis. This file is the milestone audit's input. - Every milestone sub-gate is green with its measured result captured. -- Full `npm test` EXITS 0 -- the v1.0.0 milestone gate. +- Full `npm test` EXITS 0 -- the v0.13.0 milestone gate. - INV-03 7-provider byte-equality + MIT provenance + INV-01..04 + Walls 1/2 all green. - 43-MILESTONE-GATE.md is the sign-off record with the gate table + the carried-forward NON-blocking UAT debt for the milestone audit. - This wave delivers SC3 (and confirms SC1 + SC2) BECAUSE it ASSERTS the whole milestone is green -- the typed fallback reason is byte-equal across all 7 providers, MIT provenance is complete, full npm test EXITS 0, and INV-01..04 + Walls 1/2 guards all pass -- with the live-UAT debt honestly recorded as - NON-blocking carry-forward. The v1.0.0 milestone is met. + NON-blocking carry-forward. The v0.13.0 milestone is met. diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-04-SUMMARY.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-04-SUMMARY.md similarity index 95% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-04-SUMMARY.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-04-SUMMARY.md index 9483d2950..5e0c64709 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-04-SUMMARY.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-04-SUMMARY.md @@ -2,13 +2,13 @@ phase: 43-catalog-scale-milestone-gate plan: 04 subsystem: milestone-gate-signoff -tags: [SCALE-01, SCALE-02, milestone-gate, INV-03, provenance, npm-test-exit-0, v1.0.0] +tags: [SCALE-01, SCALE-02, milestone-gate, INV-03, provenance, npm-test-exit-0, v0.13.0] requires: - all of 43-01/43-02/43-03 (the deliverables this gate confirms) - tests/provider-parity.test.js (INV-03) - tests/provenance-scaffold.test.js (MIT provenance) provides: - - the v1.0.0 milestone sign-off (full npm test EXIT 0 over the whole catalog + all guards) + - the v0.13.0 milestone sign-off (full npm test EXIT 0 over the whole catalog + all guards) - 43-MILESTONE-GATE.md (the gate-results record + carried-forward NON-blocking UAT debt) affects: - the post-milestone lifecycle (audit -> complete -> cleanup) @@ -30,7 +30,7 @@ metrics: # Phase 43 Plan 04: THE MILESTONE GATE -- Sign-Off Summary -THE FINAL CLOSE of v1.0.0. A verification/sign-off battery asserting the WHOLE milestone is green +THE FINAL CLOSE of v0.13.0. A verification/sign-off battery asserting the WHOLE milestone is green -- the SCALE-01 scale + precision HARD bars, the SCALE-02 self-heal tests, INV-03 7-provider byte-equality, MIT provenance, INV-01..04 + Walls 1/2 guards -- and that full `npm test` EXITS 0. That npm-test-EXIT-0 over the complete catalog IS the milestone gate. @@ -40,7 +40,7 @@ That npm-test-EXIT-0 over the complete catalog IS the milestone gate. **Confirmed EXIT 0.** The whole suite over breadth + depth + discovery + scale + self-heal + provider parity + provenance + every INV/Wall guard passes (0 suites failed; 24 suites reported `failed: 0`; the &&-chain completed through the last test `no-orphan-descriptor: 10 passed, 0 -failed`). The v1.0.0 milestone is met. +failed`). The v0.13.0 milestone is met. ## Measured Sub-Gate Results diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-CONTEXT.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-CONTEXT.md similarity index 97% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-CONTEXT.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-CONTEXT.md index a8dc61435..ac955521c 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-CONTEXT.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-CONTEXT.md @@ -2,12 +2,12 @@ **Gathered:** 2026-06-26 **Status:** Ready for planning -**Mode:** Orchestrator-gathered (the search/eval/self-heal substrate is shipped; the scale budget is already met; the precision re-tune + the self-heal hardening are the two substantive deliverables). The FINAL v1.0.0 phase — `npm test` EXIT 0 IS the milestone gate. +**Mode:** Orchestrator-gathered (the search/eval/self-heal substrate is shipped; the scale budget is already met; the precision re-tune + the self-heal hardening are the two substantive deliverables). The FINAL v0.13.0 phase — `npm test` EXIT 0 IS the milestone gate. ## Phase Boundary -CLOSE the v1.0.0 milestone: prove full-corpus performance, drive full-corpus search PRECISION to the +CLOSE the v0.13.0 milestone: prove full-corpus performance, drive full-corpus search PRECISION to the hard bar (DEF-39.5-04-A), harden recipe-rot self-heal for the now-119-app surface, and gate on the full test suite — the v0.9.99 Phase-32 milestone-gate posture. OWNS **SCALE-01** + **SCALE-02**. @@ -140,7 +140,7 @@ lifecycle (audit → complete → cleanup). recurrence distinguishes a one-off blip from a site-wide change; degraded surfacing makes a stale app visible instead of silently failing. - THE close is the gate: full npm test EXIT 0 over breadth + depth + discovery + scale, INV-01..04 + - Walls 1/2 green, MIT provenance complete — the v1.0.0 milestone is met. + Walls 1/2 green, MIT provenance complete — the v0.13.0 milestone is met. diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md similarity index 95% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md index 7db3d343a..b3ffc1eda 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-MILESTONE-GATE.md @@ -1,12 +1,12 @@ -# Phase 43 / v1.0.0 Milestone Gate -- Sign-Off +# Phase 43 / v0.13.0 Milestone Gate -- Sign-Off **Date:** 2026-06-26 -**Milestone:** v1.0.0 (Full App Catalog -- OpenTabs Parity) +**Milestone:** v0.13.0 (Full App Catalog -- OpenTabs Parity) **Phase:** 43 (Catalog-Scale + Milestone Gate, SCALE-01/02) -- the FINAL phase **Result:** PASS -- full `npm test` EXIT 0 over the whole catalog + all guards. The whole-suite pass over breadth + depth + discovery + scale + self-heal + provider parity + -provenance + every INV/Wall guard IS the v1.0.0 milestone gate (the v0.9.99 Phase-32 +provenance + every INV/Wall guard IS the v0.13.0 milestone gate (the v0.9.99 Phase-32 milestone-gate posture). Every sub-gate below was run DIRECTLY first (fast, attributable) and then confirmed inside the whole-suite run. @@ -29,9 +29,9 @@ then confirmed inside the whole-suite run. ## THE MILESTONE GATE -**Full `npm test` EXIT 0** -- the authoritative v1.0.0 milestone signal. The whole suite over +**Full `npm test` EXIT 0** -- the authoritative v0.13.0 milestone signal. The whole suite over breadth + depth + discovery + scale + self-heal + provider parity + provenance + every INV/Wall -guard passes. The v1.0.0 milestone is met. +guard passes. The v0.13.0 milestone is met. ## Carried-Forward NON-Blocking UAT Debt (recorded for the milestone audit) diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-REVIEW.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-REVIEW.md similarity index 99% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-REVIEW.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-REVIEW.md index 04b8db742..f479c953e 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-REVIEW.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-REVIEW.md @@ -30,7 +30,7 @@ resolution_commits: LO-02: e0bcf4fe --- -# Phase 43: Code Review Report (Catalog-Scale + Milestone Gate, SCALE-01/02 — the v1.0.0 close) +# Phase 43: Code Review Report (Catalog-Scale + Milestone Gate, SCALE-01/02 — the v0.13.0 close) **Reviewed:** 2026-06-26 **Depth:** deep (cross-file: importer data-map -> emitted corpus -> eval; scheduler/store -> router wire-up; full gate run) @@ -122,7 +122,7 @@ route-through is a confirmed-by-milestone-gate follow-up, not in this plan's sco was wired"). That is honest disclosure, but it means the SCALE-02 CONTEXT promises — "one coalesced re-learn per origin (no thundering-herd)" and "degraded surfacing makes a stale app visible instead of silently failing" — are **NOT delivered by this milestone**. The gate is green only because the tests -exercise the isolated modules, not the integration. For the FINAL v1.0.0 phase this is a material gap +exercise the isolated modules, not the integration. For the FINAL v0.13.0 phase this is a material gap between the claimed close and the shipped behavior, not a cosmetic nit — hence HIGH, not MEDIUM. **Fix:** Either (a) wire it before declaring SCALE-02 met — route `_quarantineAndRelearn`'s re-learn diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-VALIDATION.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-VALIDATION.md similarity index 98% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-VALIDATION.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-VALIDATION.md index 559b34c7b..567bc6390 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-VALIDATION.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-VALIDATION.md @@ -9,7 +9,7 @@ created: 2026-06-26 # Phase 43 — Validation Strategy -> Close v1.0.0: full-corpus scale + the wrong-invoke precision re-tune (DEF-39.5-04-A) + recipe-rot +> Close v0.13.0: full-corpus scale + the wrong-invoke precision re-tune (DEF-39.5-04-A) + recipe-rot > self-heal hardening (per-origin coalescing/back-off, recurrence, degraded surfacing) + INV-03/7-provider > + MIT provenance + **full `npm test` EXIT 0 = THE MILESTONE GATE**. Source of truth: 43-CONTEXT.md. diff --git a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-VERIFICATION.md b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-VERIFICATION.md similarity index 95% rename from .planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-VERIFICATION.md rename to .planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-VERIFICATION.md index 524305047..8d39670df 100644 --- a/.planning/milestones/v1.0.0-phases/43-catalog-scale-milestone-gate/43-VERIFICATION.md +++ b/.planning/milestones/v0.13.0-phases/43-catalog-scale-milestone-gate/43-VERIFICATION.md @@ -34,7 +34,7 @@ human_verification: # Phase 43: Catalog-Scale + Milestone Gate (SCALE-01/02) Verification Report -**Phase Goal:** Close the v1.0.0 milestone by proving full-corpus performance, driving full-corpus search PRECISION to the hard bar (DEF-39.5-04-A), hardening recipe-rot self-heal for the 119-app surface, and gating on the full test suite. `npm test` EXIT 0 IS the milestone gate. +**Phase Goal:** Close the v0.13.0 milestone by proving full-corpus performance, driving full-corpus search PRECISION to the hard bar (DEF-39.5-04-A), hardening recipe-rot self-heal for the 119-app surface, and gating on the full test suite. `npm test` EXIT 0 IS the milestone gate. **Verified:** 2026-06-26 **Status:** passed **Re-verification:** No -- initial verification (the phase went through 43-REVIEW + a full fix pass; this is the first goal-backward verification). @@ -42,7 +42,7 @@ human_verification: ## Verification Posture -This is the FINAL v1.0.0 phase. The starting hypothesis was adversarial: the 43-REVIEW found 2 material HIGH findings (HI-01 the scheduler was DEAD/unwired; HI-02 wrong-invoke=0 was partially overfit + overclaimed). Both were claimed RESOLVED. SUMMARY/MILESTONE-GATE claims were NOT trusted -- every assertion was re-run against the actual code in the real tree, the HI-01 wiring was traced line-by-line in `capability-router.js`, and the wiring test's non-vacuousness was PROVEN by a sandbox sabotage. The authoritative milestone signal -- `npm test` EXIT 0 -- was run HERE in the real tree (not a throwaway worktree). +This is the FINAL v0.13.0 phase. The starting hypothesis was adversarial: the 43-REVIEW found 2 material HIGH findings (HI-01 the scheduler was DEAD/unwired; HI-02 wrong-invoke=0 was partially overfit + overclaimed). Both were claimed RESOLVED. SUMMARY/MILESTONE-GATE claims were NOT trusted -- every assertion was re-run against the actual code in the real tree, the HI-01 wiring was traced line-by-line in `capability-router.js`, and the wiring test's non-vacuousness was PROVEN by a sandbox sabotage. The authoritative milestone signal -- `npm test` EXIT 0 -- was run HERE in the real tree (not a throwaway worktree). ## Goal Achievement -- Per-Success-Criterion @@ -68,7 +68,7 @@ This is the FINAL v1.0.0 phase. The starting hypothesis was adversarial: the 43- | Chain completion | Ran to the last test `no-orphan-descriptor: 10 passed, 0 failed` (the &&-chain did not short-circuit) | | Showcase-infra dep concern | DID NOT materialize -- the showcase-server suites ran WITH deps present. The single stack trace (`PayloadTooLargeError`, server-telemetry-body-cap INGEST-07a) is an INTENTIONAL 413-rejection test that PASSED (3 passed, 0 failed); the express log line is the server logging the deliberately-rejected 33 KB body, not a test failure. | -The whole-suite pass over breadth + depth + discovery + scale + self-heal + provider parity + provenance + every INV/Wall guard IS the v1.0.0 milestone gate. **The v1.0.0 milestone is genuinely met.** +The whole-suite pass over breadth + depth + discovery + scale + self-heal + provider parity + provenance + every INV/Wall guard IS the v0.13.0 milestone gate. **The v0.13.0 milestone is genuinely met.** ### Every Sub-Gate (each re-run directly, read-only) @@ -177,7 +177,7 @@ Scanned the phase-modified files (relearn-scheduler.js, capability-router.js, le - **Observation:** `FsbLearnedRecipeStore.getOriginHealth(origin)` is defined, exported (learned-recipe-store.js:667), substantive (LO-01 dead branch removed), and -- after the HI-01 wiring made `recordRot` live -- is now backed by accurate production data. It is consumed by `tests/app-degraded-surfacing.test.js` and `tests/relearn-router-wiring.test.js`, but NO production code path (search hit, status query, sidepanel/UI) calls it. In `extension/` it appears outside its own module only in a COMMENT in capability-router.js:212. - **Why this is a WARNING, not a BLOCKER:** The ROADMAP SC2 names "an app-level degraded/needs-re-port surfacing"; the 43-CONTEXT (lines 80-81) explicitly defines the deliverable as exposing degraded state "(via the catalog/search result **OR a status accessor**)"; and the 43-03 PLAN must_have contracts specifically "an app-level degraded/needs-re-port **status accessor** (`getOriginHealth`)." The contracted deliverable is the ACCESSOR -- which exists, is correct, is bounded, and is now fed live data via the HI-01 router wiring (the chain recordRot -> dispositionFor 'systemic' -> getOriginHealth degraded:true is proven END-TO-END through the live router in relearn-router-wiring.test.js 9/0). The CONTEXT's "or a status accessor" wording makes the accessor a valid delivery form, and the HI-01 fix's own claim is precise ("the getOriginHealth degraded surfacing is no longer INERT" -- i.e. the state is live -- not "a UI displays it"). - **What is NOT yet delivered:** a production READER that turns the accessor into a user/agent-VISIBLE "this app needs re-learning" cue. No 43-03/43-04 PLAN task scoped a search-hit/UI consumer, so this is a follow-up to complete the user-visible loop, not an unmet phase task. -- **Recommendation (human decision):** Accept as the contracted-accessor form for v1.0.0 (the milestone gate is green and the state is live + proven), and track "wire a degraded cue into a search hit / status surface" as a post-milestone follow-up alongside the carried-forward UAT debt. This does NOT block the milestone. +- **Recommendation (human decision):** Accept as the contracted-accessor form for v0.13.0 (the milestone gate is green and the state is live + proven), and track "wire a degraded cue into a search hit / status surface" as a post-milestone follow-up alongside the carried-forward UAT debt. This does NOT block the milestone. ## Human Verification Required (carried-forward, NON-blocking milestone debt) diff --git a/.planning/milestones/v0.9.91-REQUIREMENTS.md b/.planning/milestones/v0.9.91-REQUIREMENTS.md new file mode 100644 index 000000000..0b8ef13e3 --- /dev/null +++ b/.planning/milestones/v0.9.91-REQUIREMENTS.md @@ -0,0 +1,212 @@ +# Requirements Archive: v0.9.91 MCP Clients as Providers + +**Archived:** 2026-09-28 +**Status:** SHIPPED + +For current requirements, see `.planning/REQUIREMENTS.md`. + +--- + +# Requirements: FSB (Full Self-Browsing) + +**Defined:** 2026-07-11 +**Core Value:** Reliable single-attempt execution -- the AI decides correctly, the mechanics execute precisely. +**Milestone:** v0.9.91 MCP Clients as Providers + +## v1 Requirements + +Requirements for this milestone. Each maps to roadmap phases. + +### IDENT -- Agent Identity Capture + +- [x] **IDENT-01**: When the user clicks a copy-to-clipboard button on the onboarding MCP-install screen for a specific client (`claude-code`, `cursor`, `vscode`, `windsurf`, `codex`, `opencode`, `openclaw`, `claude-desktop`, `all`), FSB records that client id (with timestamp, deduplicated, all-clients aggregated for multi-select cases) into a durable `fsbAgentProviders.clicked` list in `chrome.storage.local`. +- [x] **IDENT-02**: When an MCP client completes its `initialize` handshake with `fsb-mcp-server` (over any transport: stdio, streamable-HTTP, or the ws://7225 bridge), FSB captures the caller's `clientInfo.name` and `clientInfo.version` and threads them to the extension via an additive field on the existing `agent:register` bridge payload without breaking any existing consumer of that payload (INV-01). +- [x] **IDENT-03**: The extension's agent registry stamps captured `clientInfo` onto each live `AgentRecord` and rolls the identity up into a durable `fsbAgentProviders.connected` entry that survives service-worker eviction and Chrome restart, keyed so re-connections update rather than duplicate. +- [x] **IDENT-04**: `fsb-mcp-server` can enumerate installed MCP-capable clients on the current machine by inspecting the paths already known to `platforms.ts` (per-OS `configPath` for file-mode clients; ` --version` probe for cli-mode `claude-code`) and report each as `installed` / `not-installed` with any parseable version. +- [x] **IDENT-05**: A `getMcpClients` extension runtime message returns a merged view (`clicked` ∪ `installed` ∪ `connected`) with per-client status, so UI surfaces read one consistent structure instead of assembling it themselves. + +### PROV -- Providers Panel + +- [x] **PROV-01**: The control panel section formerly labeled "API Configuration" is labeled "Providers" (heading, nav label, and any anchor `#api-config` continues to work as a redirect to the new `#providers` anchor for existing bookmarks). +- [x] **PROV-02**: Each provider has an explicit `providerKind` value of either `api` (the existing 7 BYOK LLM providers) or `agent` (a locally installed agent CLI); the kind determines which fields render. +- [x] **PROV-03**: When an `agent`-kind provider is selected, the API-key input, key-URL hint, and per-model key-format hint are hidden; the panel shows instead the provider's install status, auth status (from the CLI's own login state where surfaceable), connection status, and a short "uses your subscription -- no API key needed" caption. +- [x] **PROV-04**: When the user has both an active `agent` provider selection and a valid BYOK key for an `api` provider, `universal-provider.js` continues to see only `api`-kind provider values; the two selections do not collide, and switching between them preserves the other's configuration (INV-03 provider parity for the BYOK side). +- [x] **PROV-05**: The panel visually marks exactly one provider as "Recommended" per session, chosen by a ground-truth cascade: highest-priority = a provider whose CLI is currently connected via MCP `initialize`, next = a provider whose CLI is installed on disk, next = a provider whose copy button the user clicked during onboarding, fallback = the current xAI-default recommendation. The panel never auto-switches the user's selection; the badge is advisory only. +- [x] **PROV-06**: Cost/usage rows for `agent`-kind providers never display fabricated dollar amounts; they display token count, turn count, and duration and label the run as "included in your subscription", with a link to the vendor's current billing page (copy must not promise "free" or "unlimited"). + +### CHAN -- Delegation Channel & Security Foundation + +- [x] **CHAN-01**: A new bridge message-type family (`ext:request` / `ext:response` / `ext:event`) transports extension→daemon reverse requests over the existing ws://localhost:7225 bridge without changing the byte-shape of any existing `MCPMessageType` value (INV-01 additive proof). +- [x] **CHAN-02**: A relay process signals its ability to fulfill spawn requests by advertising `capabilities: ['agent-spawn']` in its `relay:hello`; the hub routes each `ext:request` locally (if itself the daemon) or to the first relay advertising the required capability. +- [x] **CHAN-03**: The bridge rejects every incoming `ext:*` frame whose WebSocket upgrade did not carry an `Origin` header matching a durable per-install FSB-extension-id allowlist and whose `Host` header is not exactly `127.0.0.1` (or `localhost`) at the loopback port. +- [x] **CHAN-04**: A per-install >=32-byte shared secret is provisioned once between the extension and the daemon, transported only in the `Sec-WebSocket-Protocol` upgrade header (never in URL, never in payloads, never in logs), rotated on daemon restart, and required on every `ext:*` frame. +- [x] **CHAN-05**: `redactForLog` and diagnostic ring-buffer writes strip any string matching the shared-secret token pattern; the drift gate fails the build if a raw secret substring appears in any tracked log fixture. +- [x] **CHAN-06**: The bridge topology test suite covers hub-exit-mid-delegation and relay-mid-`ext:*`-frame scenarios; existing hub-exit-promotion tests still pass byte-for-byte. +- [x] **CHAN-07**: A permanent CI grep gate fails the build if the strings `--dangerously-skip-permissions`, `--yolo`, or `--auto` appear anywhere in `mcp/src/agent-providers/**`, so those flags can never enter the spawn path in any future patch. + +### ADAPT -- Adapter Contract & Spawn Supervisor + +- [x] **ADAPT-01**: An `AgentProviderAdapter` TypeScript interface in `mcp/src/agent-providers/` defines exactly five methods: `detect() -> {installed, version, authState, binary}`, `buildSpawn(task, ctx) -> SpawnSpec`, `parseEvents(stream) -> AsyncIterable`, `kill(child, {grace}) -> Promise`, and `caps() -> AdapterCapabilities`. +- [x] **ADAPT-02**: A `SpawnSupervisor` module living in the `fsb-mcp-server serve` daemon accepts a validated spawn request, looks up the requested adapter, constructs argv from adapter output plus a daemon-controlled flag allowlist (unknown payload keys rejected), and spawns the child with `{ detached: true, windowsHide: true, stdio: ['pipe','pipe','pipe'] }` and an environment with `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`/`GEMINI_API_KEY` scrubbed. +- [x] **ADAPT-03**: The supervisor never invokes a shell; the user's prompt travels only via `child.stdin` (never argv, never `sh -c`) so shell metacharacters and Windows `.cmd`-shim EINVAL (Node CVE-2024-27980) cannot execute. +- [x] **ADAPT-04**: A `stop`/cancel request triggers SIGTERM at the process-group level (POSIX `process.kill(-child.pid, 'SIGTERM')` after `spawn({detached:true})`, Windows `taskkill /pid /T /F`), escalates to SIGKILL after a grace window, and blocks resolving the delegation until either an exit-signal is observed or the daemon confirms no descendant matches remain. +- [x] **ADAPT-05**: On daemon startup, the supervisor scans for orphaned children matching prior adapter fingerprints and kills them before accepting new spawn requests (recovery from crash). + +### CLAUDE -- Claude Code MVP + +- [x] **CLAUDE-01**: The Claude Code adapter spawns `claude -p --verbose --output-format stream-json --include-partial-messages --strict-mcp-config --mcp-config --agents --agent fsb --permission-mode dontAsk --allowedTools "mcp__fsb" --disallowedTools "Bash,Edit,Write,NotebookEdit,WebFetch,WebSearch" --max-turns 40 --no-session-persistence`, or the version-appropriate equivalent selected by `detect()` output. +- [x] **CLAUDE-02**: The user's task prompt is sent to the spawned CLI via stdin only; the adapter constructs no argv fragment containing user-supplied text. +- [x] **CLAUDE-03**: The adapter's `parseEvents` translates the CLI's stream-json events (`system/init`, `assistant`, `user`, tool-use events, `system/api_retry`, `result`) into a normalized `AgentEvent` schema (`type`, `sessionId`, `payload`), fails loud on unknown event types (surfaced as `agent_protocol_drift` diagnostic), and is covered by a recorded JSONL fixture under `tests/fixtures/agent-streams/claude-code-2.1.177/` so CI runs without a live CLI. +- [x] **CLAUDE-04**: The Claude Code adapter's `detect()` fingerprints the binary via `claude --version`, compares against a minimum supported version, and reports `installed=false` with a doctor-diagnostic message rather than spawning if the version predates the verified stream-json contract. + +### UX -- Delegation UX + +- [x] **UX-01**: A fifth entry `delegated` in `EXECUTION_MODES` (`extension/ai/engine-config.js`) defines: `uiFeedbackChannel: 'popup-sidepanel'`, `animatedHighlights: true`, `safetyLimits: { wallClockMs, eventSilenceMs }` (no iteration cap -- the loop runs in the spawned CLI, not `runAgentLoop`), and is selected when the active provider is `agent`-kind. +- [x] **UX-02**: The side panel renders a live per-run streaming feed with distinct card types for init (client, model, session id, allowed tools), tool-call (name, args summary, tab id), retry (typed error class), and result (usage summary), driven by the normalized `AgentEvent` stream. +- [x] **UX-03**: Before FSB spawns any agent CLI for the first time, the user sees an explicit consent card that names the CLI, what it will be permitted to do (drive the FSB MCP tools on the user's live browser), and what it will not be permitted to do (edit files, run shell, fetch arbitrary URLs). A per-run confirm-to-continue toggle is on by default and can be disabled per provider only via an explicit "trust this agent" setting. +- [x] **UX-04**: A prominent Stop button in the side panel triggers `stopDelegatedTask`, which routes to the supervisor's kill and, on confirmed exit, releases every tab that was owned by the spawned agent (per v0.9.60 ownership) and reports "Agent stopped, N tab(s) released" in the feed. +- [x] **UX-05**: A delegated run opens by default in a new background tab; when the user activates the tab that the agent is driving, a persistent "Take control" affordance appears; clicking it pauses the agent (v0.9.60 ownership release + supervisor grace hold), lets the user interact, and offers "Resume with agent" to give ownership back. +- [x] **UX-06**: A post-run summary card displays tokens (in/out/total), turn count, wall-clock duration, cost bucket (`included in your subscription` for agent kind; real USD for api kind), and a per-tool-call breakdown, expandable to the full tool-call log for the run. + +### LIFE -- Lifecycle & Persistence + +- [x] **LIFE-01**: Every progress event received from the supervisor is written to `chrome.storage.session` under a per-delegation key before it fans out to UI subscribers, so a MV3 service worker eviction mid-run reloads exactly the delivered feed on re-open. +- [x] **LIFE-02**: While a delegation is active, the extension pings the bridge every 20 s over the existing WS heartbeat channel to keep the Chrome 116+ SW-lifetime extension applied; if 3 heartbeats are missed the extension shows a `daemon:disconnected` fallback that offers a doctor-relaunch button but does not attempt an in-extension restart. +- [x] **LIFE-03**: If `fsb-mcp-server serve` is not running when a delegated send is attempted, the side panel shows an "Agent offline" state with a deep-link to `fsb-mcp-server doctor` output and does not enqueue or optimistically show the message. +- [x] **LIFE-04**: On daemon restart while a delegation was mid-flight, the supervisor does not re-adopt any surviving spawned CLI; it kills it (LIFE-04 restart-is-clean) and reports `daemon_restart_lost_run` in the side panel so the user knows the run ended. + +### DRIFT -- CI Drift-Smoke Gate & Doctor Extensions + +- [x] **DRIFT-01**: A CI job runs each shipped adapter against a canned prompt fixture, asserts a known event-type sequence and the presence of required fields on `system/init` and `result`, and fails the build on unknown event types, missing fields, or a `--version` outside the compatibility matrix. +- [x] **DRIFT-02**: `fsb-mcp-server doctor` gains a per-adapter section reporting: binary path, version, auth state (parseable where the CLI exposes it), shared-secret presence, and the current spawn-secret rotation age. +- [x] **DRIFT-03**: The diagnostics ring buffer classifies drift events as `agent_protocol_drift` (with adapter id, expected vs observed) and rate-limits duplicate entries at the existing 1-per-10s bucket. +- [x] **DRIFT-04**: The `doctor` output includes a machine-readable adapter compatibility matrix that both CI and the extension can read to render "supported / degraded / unsupported" states without hardcoding versions in extension code. + +### NATIVE -- Native-Messaging Host + +- [x] **NATIVE-01**: `fsb-mcp-server install --native-host` writes the platform-appropriate native-messaging host manifest (macOS `~/Library/Application Support/Google/Chrome/NativeMessagingHosts/`, Linux `~/.config/google-chrome/NativeMessagingHosts/`, Windows registry `HKCU\Software\Google\Chrome\NativeMessagingHosts\`), allowing the FSB extension id as the sole caller, and installs a host binary that wakes the daemon on demand. +- [x] **NATIVE-02**: The extension's manifest gains a `nativeMessaging` permission entry (additive, no other permission changes), and the extension detects native-host presence at boot; when present, an "Agent offline" state auto-attempts a wake before showing the doctor deep-link. +- [x] **NATIVE-03**: The native host itself does not spawn agents; it only starts `fsb-mcp-server serve` (or attaches to a running one) and exits after handoff. All spawn authority remains inside the `serve` daemon behind the CHAN gates. +- [x] **NATIVE-04**: `fsb-mcp-server uninstall --native-host` removes the manifest, and `doctor` reports native-host install state including manifest path and any Chrome allowlist mismatch. + +### MULTI -- Additional Adapters + +- [x] **MULTI-01**: An OpenCode adapter (`mcp/src/agent-providers/opencode.ts`) implements the `AgentProviderAdapter` contract with `caps.serverMode=true`; the supervisor either spawns `opencode run` cold or attaches to a running `opencode serve` per the adapter's `buildSpawn` output (contract-stresser: the ADAPT contract must accommodate both spawn and attach without hardcoding). +- [x] **MULTI-02**: The OpenCode adapter ships a pinned agent definition (equivalent to Claude Code's `--agents fsb`) using OpenCode's `agent create` / `agents` config surface, keyed to a version pinned during phase spike. +- [x] **MULTI-03**: A recorded OpenCode JSONL fixture under `tests/fixtures/agent-streams/opencode-1.14.25/` (or the latest pinned version) proves the adapter's event schema in CI without a live CLI. +- [x] **MULTI-04**: A Codex adapter (`mcp/src/agent-providers/codex.ts`) implements the `AgentProviderAdapter` contract, invoking `codex exec --json` with the current-verified flag set (v0.142.5 as baseline: use `--ephemeral` + `--ignore-user-config` for hermeticity; do not use the deprecated `--full-auto`). +- [x] **MULTI-05**: The Codex adapter's `detect()` correctly identifies auth via ChatGPT OAuth vs API key vs unauthenticated and surfaces the state in the provider panel so the user knows which billing bucket a run will hit. +- [x] **MULTI-06**: A schema-derived Codex JSONL contract fixture pins the event schema in CI with `liveCapturePending: true` until genuine UAT capture, and the adapter's `caps()` correctly reports `chatMode: false` for v0.9.91 (task-mode only across all adapters). + +## v2 Requirements + +Deferred to future release. Tracked but not in current roadmap. + +### Chat-Mode Continuity + +- **CHAT-FUTURE-01**: Adapters expose `caps.chatMode: true` and the side panel maps a chat thread to `--resume ` (Claude Code) / `codex resume` / `gemini --resume` / `opencode --continue` per adapter. +- **CHAT-FUTURE-02**: The daemon pins per-thread working directory so `claude --resume` finds its history. + +### Gemini CLI Adapter + +- **GEMINI-FUTURE-01**: Gemini CLI adapter after a live `--help` capture and JSONL schema pinning (v0.9.91 lacked a local binary to verify against). + +### Broader Agent Ecosystem + +- **ACP-FUTURE-01**: ACP-based adapter unification (`@zed-industries/agent-client-protocol`) once ≥2 non-Claude adapters have shipped and proven the contract shape. +- **REMOTE-FUTURE-01**: Remote/mobile delegation surfaces (Happy-style approval flows) -- explicitly localhost-only in v0.9.91. + +## Out of Scope + +Explicitly excluded. Documented to prevent scope creep. + +| Feature | Reason | +|---------|--------| +| Embedding `@anthropic-ai/claude-agent-sdk` in FSB | Anthropic policy prohibits third-party products from using consumer subscription auth via the SDK (enforcement Apr 4 2026); shelling to the user's installed `claude` binary is the only compliant "no API key" path. | +| Proxying / spoofing subscription OAuth tokens | Banned outright by Anthropic Apr 4 2026; would be product-killing regardless of technical feasibility. | +| Bundling / silent-installing agent CLIs | Wrong shape for a Chrome extension + npm daemon; would violate Chrome Web Store distribution rules and add attack surface. | +| PTY / TUI screen-scraping of agent CLIs | Structured headless interfaces exist for all four target CLIs; scraping is a maintenance sinkhole. | +| Any `--dangerously-skip-permissions` / `--yolo` / `--auto` flag in the spawn path | 1-click RCE for any prompt-injection incident; CHAN-07 grep gate makes this a permanent invariant. | +| Fabricated dollar costs on subscription-backed runs | Cline established the $0.00 convention; PROV-06 codifies it. | +| Auto-switching the user's provider when a "better" agent appears | Advisory badge only (PROV-05); user consent is the load-bearing property. | +| Forcing users away from BYOK when an agent is available | INV-03 provider parity carries forward; BYOK stays first-class. | +| `Firefox` support | Deferred at project level; MV3 nativeMessaging in NATIVE requirements is Chrome-specific for v0.9.91. | +| Native host that itself spawns agent CLIs | NATIVE-03 explicitly forbids this; all spawn authority lives inside the serve daemon behind CHAN gates. | + +## Traceability + +Which phases cover which requirements. Populated during roadmap creation. + +| Requirement | Phase | Status | +|-------------|-------|--------| +| IDENT-01 | Phase 57 | Complete | +| IDENT-02 | Phase 57 | Complete | +| IDENT-03 | Phase 57 | Complete | +| IDENT-04 | Phase 57 | Complete | +| IDENT-05 | Phase 57 | Complete | +| PROV-01 | Phase 58 | Complete | +| PROV-02 | Phase 58 | Complete | +| PROV-03 | Phase 58 | Complete | +| PROV-04 | Phase 58 | Complete | +| PROV-05 | Phase 58 | Complete | +| PROV-06 | Phase 58 | Complete | +| CHAN-01 | Phase 59 | Complete | +| CHAN-02 | Phase 59 | Complete | +| CHAN-03 | Phase 59 | Complete | +| CHAN-04 | Phase 59 | Complete | +| CHAN-05 | Phase 59 | Complete | +| CHAN-06 | Phase 59 | Complete | +| CHAN-07 | Phase 59 | Complete | +| ADAPT-01 | Phase 60 | Complete | +| ADAPT-02 | Phase 60 | Complete | +| ADAPT-03 | Phase 60 | Complete | +| ADAPT-04 | Phase 60 | Complete | +| ADAPT-05 | Phase 60 | Complete | +| CLAUDE-01 | Phase 60 | Complete | +| CLAUDE-02 | Phase 60 | Complete | +| CLAUDE-03 | Phase 60 | Complete | +| CLAUDE-04 | Phase 60 | Complete | +| UX-01 | Phase 61 | Complete | +| UX-02 | Phase 61 | Complete | +| UX-03 | Phase 61 | Complete | +| UX-04 | Phase 61 | Complete | +| UX-05 | Phase 61 | Complete | +| UX-06 | Phase 61 | Complete | +| LIFE-01 | Phase 61 | Complete | +| LIFE-02 | Phase 61 | Complete | +| LIFE-03 | Phase 61 | Complete | +| LIFE-04 | Phase 61 | Complete | +| DRIFT-01 | Phase 62 | Complete | +| DRIFT-02 | Phase 62 | Complete | +| DRIFT-03 | Phase 62 | Complete | +| DRIFT-04 | Phase 62 | Complete | +| NATIVE-01 | Phase 63 | Complete | +| NATIVE-02 | Phase 63 | Complete | +| NATIVE-03 | Phase 63 | Complete | +| NATIVE-04 | Phase 63 | Complete | +| MULTI-01 | Phase 64 | Complete | +| MULTI-02 | Phase 64 | Complete | +| MULTI-03 | Phase 64 | Complete | +| MULTI-04 | Phase 65 | Complete | +| MULTI-05 | Phase 65 | Complete | +| MULTI-06 | Phase 65 | Complete | + +**Coverage:** +- v1 requirements: 51 total +- Mapped to phases: 51 / 51 (100%) +- Unmapped: 0 + +**Phase distribution:** +- Phase 57 (IDENT): 5 requirements +- Phase 58 (PROV): 6 requirements +- Phase 59 (CHAN): 7 requirements +- Phase 60 (ADAPT + CLAUDE): 9 requirements +- Phase 61 (UX + LIFE): 10 requirements +- Phase 62 (DRIFT): 4 requirements +- Phase 63 (NATIVE): 4 requirements +- Phase 64 (MULTI-OpenCode): 3 requirements +- Phase 65 (MULTI-Codex): 3 requirements + +--- +*Requirements defined: 2026-07-11* +*Last updated: 2026-07-16 after Phase 62 completion (41/51 requirements complete; all live checks deferred to milestone end)* diff --git a/.planning/milestones/v0.9.91-ROADMAP.md b/.planning/milestones/v0.9.91-ROADMAP.md new file mode 100644 index 000000000..1df7234a4 --- /dev/null +++ b/.planning/milestones/v0.9.91-ROADMAP.md @@ -0,0 +1,521 @@ +# Roadmap: FSB (Full Self-Browsing) + +> **Historical-scope note (2026-07-15):** The Phase 54 dashboard-exclusion +> entries below preserve the completed v1.2.0 roadmap. A later showcase +> localization follow-up superseded that policy by localizing the dashboard and +> removing its lint exclusion. Current policy lives in +> `showcase/angular/src/locale/I18N-BOUNDARIES.md`. + +## Milestones + +- 🚧 **v0.9.91 MCP Clients as Providers** — Phases 57-65, planning 2026-07-11. Extends the Chrome MV3 extension + `fsb-mcp-server` so installed agent CLIs (Claude Code first, then OpenCode + Codex) become first-class side-panel providers: FSB captures which MCP clients the user copies/installs/connects, presents them as key-less providers in a renamed Providers panel, and delegates side-panel tasks to a spawned agent CLI that drives the browser back through FSB's own MCP tools — with a security-first reverse-request channel, an `AgentProviderAdapter` contract, task-mode only (chat-mode deferred), CI drift-smoke gate, native-messaging wake-host, and INV-01 (byte-stable existing MCP wire) preserved end-to-end. +- ✅ **v1.2.0 Showcase i18n Completeness** — Phases 52-56, shipped 2026-07-09. Archive: `.planning/milestones/v1.2.0-ROADMAP.md`. Closes the translation gap that reopened after v0.9.63 shipped: a full-page audit establishes the true drift/missing scope, resync + stats-page translation + transcreation review close it, a CI drift-detection gate lands on the clean baseline, and the long-deferred WARNING-02 locale-cookie redirect bug is fixed. +- ✅ **v1.1.0 T1 App Execution Expansion** — Phases 44-51, shipped 2026-06-30; refreshed 2026-07-01 after post-closeout T1 ports. Expanded proven T1/guarded coverage from 26 baseline descriptors to 1,267 executable T1-ready descriptors plus 556 guarded fail-closed rows, and closed the remaining catalog tail with explicit terminal-state accounting. Archive: `.planning/milestones/v1.1.0-ROADMAP.md`. +- ✅ **v0.13.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v0.13.0-ROADMAP.md`. + +## Current Milestone: v0.9.91 MCP Clients as Providers + +**Milestone Goal:** Make installed agent CLIs (Claude Code first) first-class side-panel providers — FSB captures which MCP clients the user installs/connects, presents them as key-less providers in a renamed Providers panel, and delegates side-panel tasks to a spawned agent CLI that drives the browser back through FSB's own MCP tools. The user's own words: "when I'm sending a message from the side panel, the FSB extension sends a request to the MCP server, which spawns a Claude/Codex/OpenCode agent with the prompt, and that agent uses FSB MCP tools to perform the task — technically driving the local coding agent from the side panel." + +**Key context (non-negotiable):** + +- **Security-first hard rule.** The reverse-request channel is RCE-adjacent by construction: a browser click becomes `execve(claude, -p, )` inside a localhost daemon. The security foundation phase (Phase 59) — reverse-request channel + Origin/Host/secret/consent + CHAN-07 grep gate — MUST land before ANY spawn code exists. It cannot be deferred to a "hardening" phase. All CHAN-01..CHAN-07 requirements land in Phase 59 together. +- **INV-01 discipline.** Every wire addition is additive: new frame types (`ext:request`/`ext:response`/`ext:event`) + optional payload fields only. Existing `MCPMessageType` values and tool schemas stay byte-stable across the entire milestone; a byte-freeze regression test proves this in every phase that touches the wire. +- **Task-mode only.** Chat-mode continuity via `--resume` / `codex resume` / `opencode --continue` is v2 scope (CHAT-FUTURE-01/02); every adapter's `caps()` reports `chatMode: false` for v0.9.91. +- **Adapter breadth: OpenCode + Codex; skip Gemini.** GEMINI-FUTURE-01 defers Gemini until a live `--help` capture and JSONL schema pinning are done. NATIVE and DRIFT are IN scope. +- **No `--dangerously-skip-permissions` / `--yolo` / `--auto`.** Anywhere. Ever. Not even in an "advanced mode." A permanent CI grep gate (CHAN-07) fails the build if those strings appear under `mcp/src/agent-providers/**`. +- **No `@anthropic-ai/claude-agent-sdk` embedded in FSB.** Anthropic banned third-party products from using consumer subscription auth via the SDK (enforcement Apr 4 2026). Shelling to the user's installed `claude` binary is the only compliant "no API key" path and is standard ecosystem practice (Cline, Roo, Zed, Conductor). +- **Extension has no `nativeMessaging` permission in Phases 57-62.** Delegation therefore requires a running `fsb-mcp-server serve` daemon; MVP ships an honest "agent offline → doctor" state. Phase 63 (NATIVE) adds the optional wake-host that closes this UX cliff. +- **Source-pin tripwire discipline.** FSB's test suite pins exact token counts and substrings on extension source (even comments). Every extension-touching commit updates paired tripwires; the full suite runs from commit 1 of every phase. + +**Phase ordering rationale (from research; dependency chain non-negotiable):** identity data (Phase 57) → provider selection UI reads it (Phase 58) → security foundation must exist before any spawn code (Phase 59) → adapter contract needs the channel (Phase 60) → UX/lifecycle needs the adapter (Phase 61) → drift gate and doctor need something to check (Phase 62) → native-host is additive and can only close the offline cliff after the offline state itself exists (Phase 63) → contract must be stable before adapter breadth (Phases 64-65). + +## Phases + +**Phase Numbering:** +- Integer phases (57, 58, 59, 60, 61, 62, 63, 64, 65): Planned milestone work, continuing from v1.2.0's Phase 56 +- Decimal phases (57.1, 57.2): Urgent insertions (marked with INSERTED) + +- [x] **Phase 57: Agent Identity Capture** - Persist copy-clicks, capture MCP `initialize` `clientInfo`, thread it through `agent:register`, add disk-scan detection, expose a unified `getMcpClients` view — additive on both sides of the wire (INV-01 safe), unblocks everything downstream (completed 2026-07-12) +- [x] **Phase 58: Providers Panel** - Rename "API Configuration" → "Providers", introduce `api` vs `agent` provider kinds, hide the key input for agent kind, badge exactly one "Recommended" provider via the connected > installed > copy-clicked cascade, keep `universal-provider.js` unaware of agent values (INV-03 BYOK parity) (completed 2026-07-12; live UAT deferred to milestone-end gate) +- [x] **Phase 59: Reverse-Request Channel & Security Foundation** - **SECURITY-CRITICAL, load-bearing**. Additive `ext:*` frames on ws://localhost:7225, strict Origin allowlist + Host loopback + per-install rotating shared secret in `Sec-WebSocket-Protocol`, log redaction, hub-exit-mid-delegation topology tests, permanent CI grep gate against `--dangerously-skip-permissions` / `--yolo` / `--auto` — completed 2026-07-14; automated/source verification passed and four live checks are deferred to the milestone-end UAT gate +- [x] **Phase 60: Adapter Contract & Claude Code MVP** - `AgentProviderAdapter` interface, `SpawnSupervisor` in the `serve` daemon with argv-only spawn / scrubbed env / SIGTERM-at-process-group / Windows `taskkill /T /F` / orphan scan on startup, Claude Code adapter with verified 2.1.177 flag set + shipped `fsb` agent definition + recorded stream-json JSONL fixture — completed 2026-07-14; automated/source verification passed and seven live checks are deferred to the milestone-end UAT gate +- [x] **Phase 61: Delegation UX & SW-Eviction Persistence** - Fifth `EXECUTION_MODES` entry `delegated`, explicit first-use consent card, live per-tool-call streaming feed, default-background-tab + "Take control" affordance, kill switch that reclaims owned tabs, post-run usage summary, `chrome.storage.session` per-event persistence, 20 s WS heartbeat, "agent offline → `doctor`" deep-link, restart-is-clean semantics (completed 2026-07-15; automated/source verification passed and eight live checks are deferred to the milestone-end UAT gate) +- [x] **Phase 62: CI Drift-Smoke Gate & Doctor Extensions** - Per-adapter CI drift-smoke against canned fixtures (fail-loud on unknown event types / missing fields / version outside compat matrix), `fsb-mcp-server doctor` per-adapter section (binary path, version, auth, secret rotation age), machine-readable adapter compatibility matrix consumed by the extension (completed 2026-07-16; automated/source/full-suite verification passed and three live checks are deferred to the milestone-end UAT gate) +- [ ] **Phase 63: Native-Messaging Host** - `install --native-host` writes the platform-appropriate manifest (mac/Linux/Windows), extension gains additive `nativeMessaging` permission, "Agent offline" state auto-attempts wake before the doctor deep-link; the native host only launches (or attaches to) `serve` — it NEVER spawns agent CLIs directly (all spawn authority stays inside the daemon behind Phase 59 CHAN gates) +- [ ] **Phase 64: OpenCode Adapter** - Second adapter proves the contract accommodates server-mode + attach on top of cold spawn without any Phase 60 rewrite; pinned OpenCode agent definition + recorded JSONL fixture + drift-smoke coverage +- [ ] **Phase 65: Codex Adapter** - Third adapter with `codex exec --json` on the verified 0.142.5 hermetic flag set (`--ephemeral` + `--ignore-user-config`, never the deprecated `--full-auto`), `detect()` surfacing ChatGPT OAuth / API key / unauthenticated so the Providers panel discloses which billing bucket a run will hit; `caps.chatMode: false` across all adapters (automated/source verification passed 2026-07-22; exactly three live UAT scenarios remain pending) + +## Phase Details + +### Phase 57: Agent Identity Capture +**Goal**: FSB knows which MCP-capable agent CLIs the user has installed on disk, has expressed intent to install (via onboarding copy clicks), and has actually connected via MCP `initialize` — surfaced as a single ground-truth view that unblocks every downstream provider-selection decision. +**Depends on**: Nothing (first phase of this milestone; pure additive data layer on both sides of the wire) +**Requirements**: IDENT-01, IDENT-02, IDENT-03, IDENT-04, IDENT-05 +**Success Criteria** (what must be TRUE): + 1. When the user clicks any copy-to-clipboard button on the onboarding MCP-install screen (for `claude-code`, `cursor`, `vscode`, `windsurf`, `codex`, `opencode`, `openclaw`, `claude-desktop`, or `all`), FSB records that client id (with timestamp, deduplicated, all-clients aggregated for multi-select cases) into a durable `fsbAgentProviders.clicked` list in `chrome.storage.local` that survives service-worker eviction and Chrome restart. + 2. When any MCP client completes its `initialize` handshake with `fsb-mcp-server` (over stdio, streamable-HTTP, or the ws://7225 bridge), FSB captures the caller's `clientInfo.name` and `clientInfo.version`, threads them through an additive field on the existing `agent:register` bridge payload, stamps them onto the live `AgentRecord` in the registry, and rolls the identity up into a durable `fsbAgentProviders.connected` entry keyed so re-connections update rather than duplicate. + 3. `fsb-mcp-server` can enumerate installed MCP-capable clients on the current machine by inspecting the paths already known to `platforms.ts` (per-OS `configPath` for file-mode clients; `claude --version` binary probe for cli-mode `claude-code`) and report each as `installed` / `not-installed` with any parseable version. + 4. A single `getMcpClients` extension runtime message returns a merged `clicked ∪ installed ∪ connected` view with per-client status, so UI surfaces read one consistent structure instead of assembling it themselves — and the merged view survives MV3 service-worker eviction. + 5. INV-01 holds: no existing `MCPMessageType` value, no existing tool schema, and no existing consumer of the `agent:register` payload breaks — the additive `clientInfo` field is optional and existing `payload: {}` handlers keep working byte-for-byte. +**Plans**: 3/3 complete +- [x] 57-01: Capture lazy MCP client identity and daemon-side installed-client inventory. +- [x] 57-02: Persist durable clicked, connected, and installed evidence in the extension. +- [x] 57-03: Expose the canonical merged MCP-client evidence view with compatibility locks. + +### Phase 58: Providers Panel +**Goal**: The control panel's "API Configuration" section becomes the "Providers" panel — distinguishing BYOK API providers from installed agent CLIs, badging exactly one provider "Recommended" from ground truth (never auto-switching selection), and shipping honest no-fabrication cost/usage copy for agent-kind providers. +**Depends on**: Phase 57 (needs real `fsbAgentProviders` data to render agent rows and to drive the "Recommended" cascade) +**Requirements**: PROV-01, PROV-02, PROV-03, PROV-04, PROV-05, PROV-06 +**Success Criteria** (what must be TRUE): + 1. The control panel section formerly labeled "API Configuration" is labeled "Providers" (heading, nav label, and the legacy `#api-config` anchor continues to work as a redirect to the new `#providers` anchor for existing bookmarks); the source-pin tripwire suite stays green from commit 1 of this phase. + 2. Every provider row has an explicit `providerKind` of either `api` (the existing 7 BYOK LLM providers) or `agent` (a locally installed agent CLI); the kind determines which fields render — an `agent`-kind selection hides the API-key input, key-URL hint, and per-model key-format hint, and shows install status, auth status (where the CLI exposes it), connection status, and a "uses your subscription — no API key needed" caption instead. + 3. Exactly one provider is badged "Recommended" per session, chosen by the strict ground-truth cascade (highest = a CLI currently connected via MCP `initialize`, next = a CLI installed on disk, next = a CLI whose copy button was clicked during onboarding, fallback = the existing xAI-default recommendation); the badge is advisory only — the user's selection is never auto-switched. + 4. `universal-provider.js` (the existing BYOK request builder) never observes an agent value: switching between an active agent provider and a BYOK api provider preserves the other's configuration, and INV-03 provider parity for the 7 BYOK providers holds unchanged. + 5. Cost/usage rows for `agent`-kind providers display token count, turn count, and duration alongside the label "included in your subscription", with a link to the vendor's current billing page — never a fabricated dollar amount and never the words "free" or "unlimited". +**Plans**: 3/3 complete +**UI hint**: yes + +### Phase 59: Reverse-Request Channel & Security Foundation +**Goal**: Extension → daemon reverse requests transit the existing `ws://localhost:7225` bridge with every documented 2025-2026 CVE-class defense enforced from commit one — Origin allowlist, Host loopback, per-install rotating shared secret, log redaction, additive-only wire evolution, and a permanent grep gate against the three "one-click RCE" flags. This phase is SECURITY-CRITICAL and load-bearing: it ships BEFORE any spawn code exists in Phase 60. +**Depends on**: Nothing (channel design is orthogonal to Phase 57/58 data + UI; can develop in parallel but MUST be code-green before Phase 60 starts) +**Requirements**: CHAN-01, CHAN-02, CHAN-03, CHAN-04, CHAN-05, CHAN-06, CHAN-07 +**Success Criteria** (what must be TRUE): + 1. A new `ext:request` / `ext:response` / `ext:event` bridge message-type family transports extension → daemon reverse requests over the existing ws://localhost:7225 bridge, and a byte-freeze regression test proves every existing `MCPMessageType` value, tool schema, and bridge payload stays byte-identical (INV-01 additive proof). + 2. A relay process advertising `capabilities: ['agent-spawn']` on `relay:hello` becomes the routing target for `ext:*` frames: the hub routes each `ext:request` locally when it is itself the daemon, forwards to the first `agent-spawn`-advertising relay when it is not, and replies `agent_provider_offline` when no supervisor is present — verified by extending `tests/mcp-bridge-topology.test.js` with new hub-exit-mid-delegation and relay-mid-`ext:*`-frame cases without breaking any existing hub-exit-promotion test. + 3. A fixture crafted with `Origin: https://evil.com` on the WebSocket upgrade is rejected before any handler runs; a second fixture with `Host: evil.com:7225` (even when the target actually resolves to 127.0.0.1) is rejected as DNS-rebind defense — the bridge accepts `ext:*` frames only when the WS upgrade carried an `Origin` matching a durable per-install `chrome-extension://` allowlist and a `Host` exactly `127.0.0.1` or `localhost` at the loopback port, and the daemon never binds `0.0.0.0`. + 4. A per-install ≥32-byte shared secret is provisioned once between the extension and the daemon, transported only in the `Sec-WebSocket-Protocol` upgrade header (never in URL, never in payloads, never in logs), rotated on daemon restart, and required on every `ext:*` frame — and `redactForLog` plus every tracked diagnostic ring-buffer sink strips any string matching the shared-secret token pattern, with a build-time drift gate that fails if a raw secret substring appears in any tracked log fixture. + 5. A permanent CI grep gate fails the build if the strings `--dangerously-skip-permissions`, `--yolo`, or `--auto` appear anywhere in `mcp/src/agent-providers/**`, so those "one-click RCE" flags can never enter the spawn path in any future patch. +**Plans**: 4/4 complete + +### Phase 60: Adapter Contract & Claude Code MVP +**Goal**: A shell-free, tree-killable `SpawnSupervisor` in the `serve` daemon spawns the user's installed `claude` CLI via a formal five-method `AgentProviderAdapter` contract — with a shipped `fsb` agent definition (never prompt-stuffing), `--strict-mcp-config` hermeticity, and a recorded stream-json JSONL fixture — so a user with Claude Code installed can send a side-panel message and observe the agent drive their live browser back through FSB's MCP tools with visible per-tool-call feedback. This phase is the integration payoff; the contract is designed for adapter breadth so OpenCode/Codex slot in without rework. +**Depends on**: Phase 58 (provider selection reads the Providers panel), Phase 59 (channel + all CHAN security foundations must be code-green before any spawn code exists) +**Requirements**: ADAPT-01, ADAPT-02, ADAPT-03, ADAPT-04, ADAPT-05, CLAUDE-01, CLAUDE-02, CLAUDE-03, CLAUDE-04 +**Success Criteria** (what must be TRUE): + 1. The `AgentProviderAdapter` TypeScript interface (`mcp/src/agent-providers/adapter.ts`) exposes exactly five methods (`detect()` → `{installed, version, authState, binary}`, `buildSpawn(task, ctx)` → `SpawnSpec`, `parseEvents(stream)` → `AsyncIterable`, `kill(child, {grace})` → `Promise`, `caps()` → `AdapterCapabilities`), and the Claude Code adapter (`mcp/src/agent-providers/claude-code.ts`) implements all five and is registered against the `INSTALL_CLIENTS` / `PLATFORMS` client id `claude-code`. + 2. A user with Claude Code installed can select "Claude Code" as their provider in the side panel, type a task, and observe the spawned CLI drive their live browser back through FSB's MCP tools — with visible per-tool-call feedback — while every existing FSB MCP tool call remains byte-identical on the wire (INV-01 preserved end-to-end through the delegation round-trip; the byte-freeze regression test still passes). + 3. The `SpawnSupervisor` accepts a validated spawn request over Phase 59's `ext:request` channel, looks up the requested adapter, constructs argv from adapter output plus a daemon-controlled flag allowlist (unknown payload keys rejected with a typed error), and spawns the child with `{ detached: true, windowsHide: true, stdio: ['pipe','pipe','pipe'] }` and an environment with `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and `GEMINI_API_KEY` scrubbed; the supervisor never invokes a shell, and the user's prompt reaches the CLI only via `child.stdin` (never argv, never `sh -c`) so shell metacharacters and Windows `.cmd`-shim EINVAL (Node CVE-2024-27980) cannot execute. + 4. A `stop` / cancel request triggers SIGTERM at the process-group level on POSIX (`process.kill(-child.pid, 'SIGTERM')` after `spawn({detached:true})`), `taskkill /pid /T /F` on Windows, escalates to SIGKILL after a grace window, and does not resolve the delegation until either an exit-signal is observed or the daemon confirms no descendant matches remain — grandchildren, MCP sub-servers, and shell tools spawned by the CLI are all terminated. + 5. On daemon startup, the supervisor scans for orphaned children matching prior adapter fingerprints (env-var tag + argv pattern) and kills them before accepting new spawn requests, and reports the scan result in structured diagnostics — so a daemon crash mid-run cannot leave a ghost CLI controlling the browser. + 6. The Claude Code adapter spawns `claude -p --verbose --output-format stream-json --include-partial-messages --strict-mcp-config --mcp-config --agents --agent fsb --permission-mode dontAsk --allowedTools "mcp__fsb" --disallowedTools "Bash,Edit,Write,NotebookEdit,WebFetch,WebSearch" --max-turns 40 --no-session-persistence` (or the version-appropriate equivalent selected by `detect()`); `parseEvents` translates the CLI's stream-json events into a normalized `AgentEvent` schema, fails loud on unknown event types (surfaced as `agent_protocol_drift` diagnostic), and is covered by a recorded JSONL fixture under `tests/fixtures/agent-streams/claude-code-2.1.177/` so CI runs without a live CLI; `detect()` fingerprints via `claude --version`, compares against a minimum supported version, and reports `installed=false` with a doctor-diagnostic message rather than spawning if the version predates the verified stream-json contract. +**Plans**: 4/4 complete +- [x] 60-01: Freeze the exact adapter contract, Claude detection/profile, and shipped FSB agent policy. +- [x] 60-02: Normalize bounded Claude JSONL through the concrete adapter with truthful offline fixture provenance. +- [x] 60-03: Build private runtime state plus verified POSIX/Windows termination and orphan recovery. +- [x] 60-04: Wire the serve-only supervisor, exact-once reverse delegation, full regression gates, and deferred UAT ledger. + +### Phase 61: Delegation UX & SW-Eviction Persistence +**Goal**: The side panel becomes a first-class delegation surface — visible first-use consent gate, live per-tool-call streaming feed, default-background-tab with a "Take control" affordance, kill switch that reclaims owned tabs, honest post-run usage summary, and MV3 SW-eviction-safe persistence so a 45-minute run resumes exactly the feed the user last saw. The extension has no `nativeMessaging` permission at this point — this phase ships the honest "Agent offline → `doctor`" state that Phase 63 later augments with an optional wake path. +**Depends on**: Phase 60 (adapter contract + Claude Code MVP producing the `AgentEvent` stream + tab ownership on spawn); Phase 59 heartbeat plumbing +**Requirements**: UX-01, UX-02, UX-03, UX-04, UX-05, UX-06, LIFE-01, LIFE-02, LIFE-03, LIFE-04 +**Success Criteria** (what must be TRUE): + 1. A fifth `EXECUTION_MODES` entry `delegated` in `extension/ai/engine-config.js` (with `uiFeedbackChannel: 'popup-sidepanel'`, `animatedHighlights: true`, wall-clock + event-silence watchdogs, and no iteration cap — the reasoning loop runs in the spawned CLI, not `runAgentLoop`) is selected automatically whenever the active provider is agent-kind, and the extension's own `runAgentLoop` is never entered for delegated tasks. + 2. Before FSB spawns any agent CLI for the first time, the user sees an explicit consent card that names the CLI, states what it will be permitted to do (drive FSB MCP tools on the user's live browser), and states what it will not (edit files, run shell, fetch arbitrary URLs); a per-run confirm-to-continue toggle is on by default and can be disabled per-provider only via an explicit "trust this agent" setting — the copy never uses "faster mode" language. + 3. The side panel renders a live per-run streaming feed with distinct card types for `init` (client, model, session id, allowed tools), `tool-call` (name, args summary, tab id), `retry` (typed error class), and `result` (usage summary) driven by the normalized `AgentEvent` stream; a delegated run opens by default in a new background tab, and when the user activates the tab the agent is driving, a persistent "Take control" affordance appears that pauses the agent (v0.9.60 ownership release + supervisor grace hold), lets the user interact, and offers "Resume with agent" to give ownership back. + 4. A prominent Stop button in the side panel triggers `stopDelegatedTask`, routes to the supervisor's kill, and — on confirmed exit — releases every tab that was owned by the spawned agent (per v0.9.60 ownership) and reports "Agent stopped, N tab(s) released" in the feed; a post-run summary card displays tokens (in/out/total), turn count, wall-clock duration, cost bucket (`included in your subscription` for agent kind, real USD for api kind), and an expandable per-tool-call breakdown. + 5. Every progress event received from the supervisor is written to `chrome.storage.session` under a per-delegation key BEFORE it fans out to UI subscribers, so a mid-run MV3 service worker eviction reloads exactly the delivered feed on re-open — no ghost state, no fabricated events, no dropped events; the fifth `EXECUTION_MODES` entry works end-to-end across a forced SW eviction in a test fixture. + 6. While a delegation is active, the extension pings the bridge every 20 s over the existing WS heartbeat channel to keep the Chrome 116+ SW-lifetime extension applied; if 3 heartbeats are missed the side panel switches to a `daemon:disconnected` fallback that offers a doctor-relaunch button but does not attempt an in-extension restart; if `fsb-mcp-server serve` is not running when a delegated send is attempted, the side panel shows an "Agent offline" state with a deep-link to `fsb-mcp-server doctor` output and does not enqueue or optimistically show the message; if the daemon restarts while a delegation was mid-flight, the supervisor kills any surviving spawned CLI (never re-adopts) and reports `daemon_restart_lost_run` in the side panel so the user knows the run ended. +**Plans**: 8/8 complete; automated/source verification passed; eight live UAT checks deferred to the milestone-end gate +- [x] 61-01: Freeze exact agent routing, delegated mode, read-only preflight, one-use consent, and provider-local trust. +- [x] 61-02: Build the bounded write-before-fanout event ledger and sole delegated lifecycle controller. +- [x] 61-03: Serialize async bridge observers, add acknowledged active-delegation heartbeat, and pin Chrome 116/no-native authority. +- [x] 61-04: Correlate server delegation ids with extension agents and add sealed complete mapped-tab hold leases. +- [x] 61-05: Add supervisor hold/resume/status plus generation-backed restart-loss evidence. +- [x] 61-06: Integrate start, handoff, Stop, hydration, heartbeat, and recovery in the service worker. +- [x] 61-07: Implement the safe, exact-copy, accessible delegated side-panel feed and lifecycle UI. +- [x] 61-08: Install complete regression/evidence gates and defer genuine live UAT to the milestone-end ledger. +**UI hint**: yes + +### Phase 62: CI Drift-Smoke Gate & Doctor Extensions +**Goal**: An `fsb-mcp-server doctor` operator and CI both catch adapter drift the moment an agent CLI ships a new flag or event-shape — with a machine-readable adapter compatibility matrix the extension consumes to render "supported / degraded / unsupported" states without hardcoding versions anywhere in extension source. +**Depends on**: Phase 60 (needs at least one adapter to run drift-smoke against; Phase 61 UX consumes the compatibility matrix for rendering degraded states) +**Requirements**: DRIFT-01, DRIFT-02, DRIFT-03, DRIFT-04 +**Success Criteria** (what must be TRUE): + 1. A CI job runs each shipped adapter against a canned prompt fixture, asserts the known event-type sequence and the presence of required fields on `system/init` and `result`, and fails the build on unknown event types, missing required fields, or a `--version` outside the compatibility matrix — with no live CLI required (the recorded JSONL fixtures from Phases 60/64/65 suffice), so contributors without every CLI installed can still land safe changes. + 2. `fsb-mcp-server doctor` gains a per-adapter section reporting: binary path, version, auth state (parseable where the CLI exposes it), shared-secret presence, and the current spawn-secret rotation age — so an operator with only `doctor` output can identify which adapter failed and why. + 3. The diagnostics ring buffer classifies drift events as `agent_protocol_drift` with adapter id, expected vs observed fields, and rate-limits duplicate entries at the existing 1-per-10s bucket so a chatty drift does not blow the buffer. + 4. `doctor` emits a machine-readable adapter compatibility matrix that both the CI drift-smoke job and the extension can read; the extension consumes the matrix at boot (and on doctor refresh) to render `supported` / `degraded` / `unsupported` badges in the Providers panel — and never hardcodes CLI versions in extension source. +**Plans**: 6 plans +- [x] 62-01: Establish the canonical adapter compatibility matrix/classifier and generalized offline drift-smoke CI gate. +- [x] 62-02: Extend `doctor` with offline adapter diagnostics, safe spawn-secret metadata, and text/JSON parity. +- [x] 62-03: Carry the authenticated safe compatibility projection into background-owned durable, freshness-aware provider state. +- [x] 62-04: Project sanitized protocol-drift detail and report it exactly once through a true per-adapter pre-throttle. +- [x] 62-05: Render exact, accessible, non-mutating compatibility badges and selected-provider details. +- [x] 62-06: Wire root/source/security gates and preserve all genuine live evidence in the pending milestone-end UAT ledger. + +### Phase 63: Native-Messaging Host +**Goal**: When the user has installed the optional native-messaging host, the "Agent offline" state auto-attempts to wake `fsb-mcp-server serve` before falling back to the Phase 61 doctor deep-link — closing the UX cliff introduced by the extension having no `nativeMessaging` permission through Phases 57-62. All spawn authority stays inside the serve daemon behind Phase 59's CHAN gates; the native host itself never spawns agent CLIs. +**Depends on**: Phase 61 ("Agent offline → doctor" state must already exist for the wake to have a fallback), Phase 62 (doctor extensions already report daemon state) +**Requirements**: NATIVE-01, NATIVE-02, NATIVE-03, NATIVE-04 +**Success Criteria** (what must be TRUE): + 1. `fsb-mcp-server install --native-host` writes the platform-appropriate native-messaging host manifest — macOS `~/Library/Application Support/Google/Chrome/NativeMessagingHosts/`, Linux `~/.config/google-chrome/NativeMessagingHosts/`, Windows registry `HKCU\Software\Google\Chrome\NativeMessagingHosts\` — allowing the FSB extension id as the sole caller, and installs a host binary that can wake the daemon on demand. + 2. The extension's manifest gains a single additive `nativeMessaging` permission entry (no other permission changes); at boot the extension detects native-host presence, and when the host is installed, an "Agent offline" state auto-attempts a wake before showing the doctor deep-link — a user with the host installed sees the delegation come alive on demand instead of the manual `serve` prompt. + 3. The native host itself does NOT spawn agents; it only starts `fsb-mcp-server serve` (or attaches to a running one) and exits after handoff — all spawn authority remains inside the serve daemon behind the Phase 59 CHAN gates (Origin allowlist, shared secret, flag allowlist, argv-only) and no CHAN requirement is relaxed to accommodate the wake path. + 4. `fsb-mcp-server uninstall --native-host` cleanly removes the manifest, and `doctor` reports native-host install state including manifest path, allowlist mismatches, and host-binary reachability so an operator can debug wake failures without shell-level inspection. +**Plans**: 12 plans + +**Wave 1** +- [x] 63-01: Establish the workspace-safe build guard, release-built Windows bootstrap, and durable bundled runtime contract. + +**Wave 2** *(blocked on Wave 1 completion)* +- [x] 63-02: Fix pre-bind secret rotation and make product-specific serve readiness authoritative. + +**Wave 3** *(blocked on Wave 2 completion)* +- [x] 63-03: Implement the closed native-messaging framing, validation, one-shot entry, and authority boundary. + +**Wave 4** *(blocked on Wave 3 completion)* +- [x] 63-04: Implement exact readiness probing, tokened wake coalescing, and one shell-free serve handoff. + +**Wave 5** *(blocked on Wave 4 completion)* +- [x] 63-05: Implement exact-owned atomic registration, runtime materialization, install, and uninstall across three OS families. + +**Wave 6** *(blocked on Wave 5 completion)* +- [x] 63-06: Add explicit native-host CLI install/uninstall routes without widening existing installer behavior. + +**Wave 7** *(blocked on Wave 6 completion)* +- [x] 63-07: Add bounded read-only native-host diagnostics with safe local and browser projections. + +**Wave 8** *(blocked on Wave 7 completion)* +- [x] 63-08: Add the single manifest permission and background-owned probe/offline-wake controller. + +**Wave 9** *(blocked on Wave 8 completion)* +- [x] 63-09: Render the approved intent-fenced checking state in the existing delegation card. + +**Wave 10** *(blocked on Wave 9 completion)* +- [x] 63-10: Wire deterministic focused/root/CI contracts and create the still-pending milestone-end UAT ledger. + +**Wave 11** *(blocked on Wave 10 completion)* +- [x] 63-11: Run independent blocking code, security, and UI source-review gates. + +**Wave 12** *(blocked on Wave 11 completion)* +- [x] 63-12: Run the reviewed focused matrix and workspace-preserving repository-wide regression gate. + +### Phase 64: OpenCode Adapter +**Goal**: The `AgentProviderAdapter` contract proves it accommodates a second CLI family — one that supports both cold spawn AND attach-to-running-server (`opencode serve` + `opencode run --attach`) — without any Phase 60 rewrite; OpenCode joins the Providers panel with a pinned agent definition and CI-covered event schema so the drift gate covers it from day one. +**Depends on**: Phase 60 (contract), Phase 62 (drift-smoke gate must accept the new adapter fixture) +**Requirements**: MULTI-01, MULTI-02, MULTI-03 +**Success Criteria** (what must be TRUE): + 1. An OpenCode adapter (`mcp/src/agent-providers/opencode.ts`) implements the `AgentProviderAdapter` contract with `caps.serverMode = true`; the supervisor either spawns `opencode run` cold OR attaches to a running `opencode serve` per the adapter's `buildSpawn` output — and the Phase 60 contract handles both without any hardcoded spawn-vs-attach branch outside the adapter. + 2. The OpenCode adapter ships a pinned agent definition (equivalent to Claude Code's `--agents fsb`) using OpenCode's `agent create` / `agents` config surface, keyed to a version pinned during phase spike, so tool boundaries and system-prompt intent are identical across the two adapters. + 3. A recorded OpenCode JSONL fixture under `tests/fixtures/agent-streams/opencode-/` proves the adapter's event schema in CI without a live CLI; the Phase 62 drift-smoke job includes OpenCode from the first commit of this phase, and unknown event types raise `agent_protocol_drift` with the adapter id `opencode`. Planning-document commits do not count as implementation commits: Plan 64-01 has exactly one implementation task/commit containing the complete parser, fixture, native negative corpus, and generalized drift gate; every later plan depends on it transitively, and Plan 64-05 exposes adapter/registry/matrix/bijection atomically. + 4. A user with OpenCode installed can pick "OpenCode" as their provider in the side panel and observe the delegation UX (streaming feed, kill switch that reclaims tabs, post-run summary, SW-eviction survival) work identically to Claude Code — with no adapter-specific side-panel branches. +**Plans**: 13 plans + +**Wave 1** +- [x] 64-01-PLAN.md — Land the complete strict parser, honest 1.14.25 fixture, native negative corpus, and generalized adapter-native Phase 62 CI drift gate as the sole first implementation commit. + +**Wave 2** *(blocked on Wave 1 completion)* +- [x] 64-02-PLAN.md — Freeze the exact five-method provider-neutral topology/attestation seam and the public fixed-env versus transient secret-binding contract. + +**Wave 3** *(blocked on Wave 2 completion)* +- [x] 64-03-PLAN.md — Version the exact delegation/provider-server runtime journal and private-artifact boundary with legacy recovery and structural secret non-retention preserved. + +**Wave 4** *(blocked on Wave 3 completion)* +- [x] 64-04-PLAN.md — Build exact OpenCode 1.14.25 retained detection, private policy, effective-attestation declarations, and exact server/attach secret-binding descriptors. + +**Wave 5** *(blocked on Wave 4 completion)* +- [x] 64-05-PLAN.md — Atomically compose and expose the production adapter, registry row, compatibility row, and exact parser/fixture/native-drift bijection. + +**Wave 6** *(parallel; blocked on Wave 5, with the server lifecycle also directly bound to Wave 3 runtime state)* +- [x] 64-06-PLAN.md — Project the exact roster into bounded inventory/doctor evidence and the browser-safe Claude/OpenCode compatibility plane. +- [x] 64-07-PLAN.md — Implement the generic cold-first, FSB-owned-server lease, health, fallback, idle, and recovery lifecycle. + +**Wave 7** *(parallel; each blocked on its Wave 6 dependency)* +- [x] 64-08-PLAN.md — Interpret closed attestation descriptors through the shared verifier, enforce no replay, and gate results on clean exit/tree settlement. +- [x] 64-09-PLAN.md — Generalize canonical extension preflight, consent, trust, and background-authoritative provider routing. + +**Wave 8** *(parallel; each blocked on its Wave 7 dependency)* +- [x] 64-10-PLAN.md — Generalize durable event/controller hydration and safe per-adapter drift diagnostics. +- [x] 64-11-PLAN.md — Promote the existing OpenCode Providers row from safe evidence with honest auth/billing and no visible layout change. + +**Wave 9** *(blocked on Waves 7-8 completion)* +- [x] 64-12-PLAN.md — Project OpenCode through the existing delegated consent/side-panel/feed/summary UI with no provider-specific branch. + +**Wave 10** *(blocked on Wave 9 completion)* +- [x] 64-13-PLAN.md — Wire exact validation/root/CI/security/full-suite gates and the still-pending three-row human-UAT ledger. + +### Phase 65: Codex Adapter +**Goal**: Task-mode delegation coverage extends to OpenAI Codex with correct auth-state disclosure (ChatGPT OAuth vs API key vs unauthenticated) so users know which billing bucket a run will hit before starting; `caps.chatMode: false` across all v0.9.91 adapters confirms task-mode-only scope for the milestone. +**Depends on**: Phase 60 (contract), Phase 62 (drift-smoke gate) +**Requirements**: MULTI-04, MULTI-05, MULTI-06 +**Success Criteria** (what must be TRUE): + 1. A Codex adapter (`mcp/src/agent-providers/codex.ts`) implements the `AgentProviderAdapter` contract, invoking `codex exec --json` with the verified 0.142.5 flag set (`--ephemeral` + `--ignore-user-config` for hermeticity; the deprecated `--full-auto` is NEVER referenced in adapter source, protected by the CHAN-07 grep gate against `--auto`). + 2. The Codex adapter's `detect()` correctly identifies auth via ChatGPT OAuth, API key, or unauthenticated and surfaces the state in the Providers panel so the user knows which billing bucket a run will hit — with the accepted copy **Included with your ChatGPT plan**, **Billed to the API key stored by Codex; dollar amount not reported.**, or the exact sign-in/status-refresh recovery copy. + 3. A schema-derived Codex JSONL contract fixture pins the event schema in CI under `tests/fixtures/agent-streams/codex-0.142.5/`, honestly retains `liveCapturePending: true`, and awaits genuine capture in UAT65-02; the Phase 62 drift-smoke job includes Codex from the first implementation commit, and the adapter's `caps()` correctly reports `chatMode: false` — matching the milestone-wide task-mode-only posture. + 4. A user with Codex installed can pick "Codex" as their provider and observe the same delegation UX as Claude Code and OpenCode with the correct per-auth-state cost copy, and the Providers panel's `agent`-kind cost row shows tokens / turns / duration + the auth-state-appropriate subscription caption rather than a fabricated dollar amount. +**Plans**: 8/8 complete + +- [x] 65-01-PLAN.md — Establish canonical accepted identity, capability vocabulary, and durable provider-neutral persistence before Codex exposure. +- [x] 65-02-PLAN.md — Bind background-owned preflight, consent, trust, and replay rejection to the exact five-field identity. +- [x] 65-03-PLAN.md — Enforce immediate supervisor re-attestation and exact start echo before any runtime or browser-visible effect. +- [x] 65-04-PLAN.md — Add bounded byte probes, sanitized environments, serve-owned authority descriptors, and empty direct scratch cleanup. +- [x] 65-05-PLAN.md — Atomically expose the complete Codex adapter, profile, parser, fixture, roster, drift, and negative corpus. +- [x] 65-06-PLAN.md — Project safe auth evidence and exact billing copy through the existing three-provider Providers surface. +- [x] 65-07-PLAN.md — Preserve immutable accepted identity through feed/hydration and close shared 44px delegated controls. +- [x] 65-08-PLAN.md — Lock exact validation/UAT contracts and the preservation-safe focused, root, and CI closure gate. + +## Progress + +**Execution Order:** +Phases execute in numeric order: 57 → 58 → 59 → 60 → 61 → 62 → 63 → 64 → 65 +Security-first hard rule: Phase 59 is code-green before Phase 60 spawn code lands, regardless of Phase 57/58 progress. + +| Phase | Plans Complete | Status | Completed | +|-------|-----------------|--------|-----------| +| 57. Agent Identity Capture | 3/3 | Complete | 2026-07-12 | +| 58. Providers Panel | 3/3 | Complete | 2026-07-12 | +| 59. Reverse-Request Channel & Security Foundation | 4/4 | Complete | 2026-07-14 | +| 60. Adapter Contract & Claude Code MVP | 4/4 | Complete | 2026-07-14 | +| 61. Delegation UX & SW-Eviction Persistence | 8/8 | Complete (UAT deferred) | 2026-07-15 | +| 62. CI Drift-Smoke Gate & Doctor Extensions | 6/6 | Complete (UAT deferred) | 2026-07-16 | +| 63. Native-Messaging Host | 12/12 | Complete | 2026-07-20 | +| 64. OpenCode Adapter | 13/13 | Complete | 2026-07-21 | +| 65. Codex Adapter | 8/8 | Complete | 2026-07-22 | + +## Completed Milestones + +
+v1.2.0 Showcase i18n Completeness — Phases 52-56, SHIPPED 2026-07-09 + +**Milestone Goal:** Close the translation gap that reopened after v0.9.63 shipped -- full, drift-free coverage across all six supported locales (en, es, de, ja, zh-CN, zh-TW) for every showcase marketing page plus the stats page, the long-deferred locale-cookie redirect bug, and a CI gate that catches future drift automatically. + +**Phase summary:** + +| Phase | Name | Plans | Status | +|-------|------|-------|--------| +| 52 | Full-Page Translation Completeness Audit | 1/1 | Complete | +| 53 | Trans-Unit Resync, Stats Translation & Transcreation Review | 3/3 | Complete; VISUAL-01 human_needed | +| 54 | Stats Lint Gate Flip & Dashboard Boundary Documentation | 1/1 | Complete | +| 55 | CI Drift-Detection Gate | 1/1 | Complete | +| 56 | Locale-Cookie Redirect Fix (WARNING-02) | 1/1 | Complete | + +Archive files: + +- `.planning/milestones/v1.2.0-ROADMAP.md` +- `.planning/milestones/v1.2.0-REQUIREMENTS.md` +- `.planning/v1.2.0-MILESTONE-AUDIT.md` + +Outcome: 13/13 v1.2.0 requirements satisfied; audit passed. Phase 52's audit established the true 5-drifted/54-orphaned baseline, superseding the milestone brief's original "247 trans-units" estimate. Phase 53 resynced the 5 drifted units + retired the stats-274 JSON artifacts + transcreated the 19 hero/CTA strings across 5 non-en locales. Phase 54 flipped `lint:i18n` to cover stats and documented the dashboard exclusion as permanent. Phase 55 landed `verify-translation-drift.mjs` as a hard-fail CI gate on a clean drift-free baseline. Phase 56 fixed WARNING-02's picker-cookie short-circuit on the bare-`/` Accept-Language redirect. VISUAL-01 browser UAT remains `human_needed` (`53-VISUAL-QA.md`). + +
+ +
+v1.1.0 T1 App Execution Expansion — Phases 44-51, SHIPPED 2026-06-30 + +**Hard invariants (v1.1.0):** + +- Kept the two-tool MCP surface: `search_capabilities` and `invoke_capability`; no one tool per app. +- Preserved MV3 Wall 1: descriptors and recipes are closed-vocabulary data; no OpenTabs runtime/plugin code ships. +- Preserved Wall 2 unless explicitly extended by a verified Pattern-D design: credentialed execution stays origin-pinned and same-session. +- Writes and destructive actions stayed fail-closed until live request shape, consent behavior, and no-secret logging were proven. +- Denylisted origins remained blocked; sensitive origins stayed flagged/audited under the existing invoke/discovery consent semantics. +- Search distinguished "directly invocable T1" from "discovery-pending DOM/T2." + +### Phase 44: T1 Readiness Inventory + Status Surface +**Goal:** Create the authoritative T1 readiness matrix for all 2,314 descriptors and make status visible to developers and users so "catalog supported" is never confused with "direct API-ready." +**Requirements:** T1R-01, T1R-02, T1R-03. +**Plans:** +- [x] 44-01: T1 readiness matrix generator and evidence report. +- [x] 44-02: Status-surface and documentation honesty pass. +- [x] 44-03: T1 readiness CI guard and phase closeout. + +### Phase 45: T1 Porting Scaffold + Handler Contract Hardening +**Goal:** Build the reusable test and implementation scaffold for app ports so each new T1 handler has origin-pin, logged-out guard, shape guard, no-secret logging, consent classification, and byte-stable fallback behavior by default. +**Requirements:** T1R-04, T1R-05, T1R-09. +**Plans:** +- [x] 45-01: Port contract library and scaffold CLI. +- [x] 45-02: Current-catalog contract verifier. +- [x] 45-03: Documentation and phase closeout. + +### Phase 46: Same-Origin Read Ports — First High-Value Batch +**Goal:** Convert a first batch of high-value read descriptors from T3 to executable T1 where the app's authenticated web runtime uses same-origin APIs and can be proven without weakening Wall 2. +**Requirements:** T1R-06. +**Plans:** +- [x] 46-01: Candidate selection and Netlify/Bitbucket/CircleCI handler ports. +- [x] 46-02: Catalog wiring, search readiness, and same-origin classifier gates. +- [x] 46-03: Verification, UAT notes, and phase closeout. + +### Phase 47: Pattern-D + GAPI Bridge Architecture +**Goal:** Design and prove the missing architecture for apps whose useful APIs are separate-origin, per-org-subdomain, or page-bridge mediated, without breaking the active-tab credential boundary. +**Requirements:** T1R-07, T1R-08. +**Plans:** +- [x] 47-01: Pattern-D decision for separate-origin and per-org APIs. +- [x] 47-02: GAPI bridge decision for Google Workspace APIs. +- [x] 47-03: CI gate, negative controls, and closeout. + +### Phase 48: High-Value Read Ports — Second Batch +**Goal:** Use the Phase 47 outcome to port another batch of read descriptors, including Pattern-D/GAPI candidates if the architecture is proven. +**Requirements:** T1R-06, T1R-07, T1R-08. +**Plans:** +- [x] 48-01: Vercel same-origin read head. +- [x] 48-02: CircleCI same-origin read expansion. +- [x] 48-03: Gate, search, and readiness-report closeout. + +### Phase 49: Guarded Writes Activation Pipeline +**Goal:** Turn fail-closed write/destructive candidates into executable T1 only after live mutation-body capture, consent gate verification, and redacted audit proof. +**Requirements:** T1R-10, T1R-11. +**Plans:** +- [x] 49-01: Write activation evidence ledger. +- [x] 49-02: Evidence verifier and validation gate. +- [x] 49-03: Live-UAT template and closeout decision. + +### Phase 50: T1 Expansion Gate + Next-Batch Plan +**Goal:** Close the milestone with an honest T1 coverage gate, full regression suite, and a prioritized backlog for the remaining 2,288-descendant tail. +**Requirements:** T1R-12. +**Plans:** +- [x] 50-01: Regenerate readiness evidence and closeout counts. +- [x] 50-02: Full regression gate. +- [x] 50-03: Next-batch backlog and milestone closeout. + +### Phase 51: Full T1 Tail Migration Across Remaining Catalog +**Goal:** Convert the Phase 51 catalog-tail descriptors to explicit terminal states: executable T1/T1b where safe and technically provable, guarded fail-closed where write/destructive UAT is still required, or blocked where denylist/product/legal policy says no. +**Requirements:** T1ALL-01, T1ALL-02, T1ALL-03, T1ALL-04, T1ALL-05. +**Plans:** +- [x] 51-01: Full-tail worklist generator, acceptance gate, and batch scheduler. +- [x] 51-02: Existing-head and low-risk same-origin read bulk port. +- [x] 51-03: Retool CSRF same-origin read head. +- [x] 51-04: Asana same-origin Pattern-D carveout. +- [x] 51-05: Pattern-D and GAPI bridge implementation waves. +- [x] 51-06: Sensitive consumer/social and blocked-policy triage. +- [x] 51-07: Write/destructive live-UAT activation waves. +- [x] 51-08: Final all-tail regression, UAT ledger, and closeout. + +Archive files: + +- `.planning/milestones/v1.1.0-ROADMAP.md` +- `.planning/milestones/v1.1.0-REQUIREMENTS.md` +- `.planning/milestones/v1.1.0-MILESTONE-AUDIT.md` + +Outcome: v1.1.0 expanded proven direct execution from the v0.13.0 baseline into verified T1-ready read coverage across Netlify, Bitbucket, CircleCI, Vercel, Retool, Asana, and generated same-origin reads, with post-closeout ports reflected in the refreshed artifacts. Phase 51 closes the remaining catalog tail honestly: 1,267 rows are executable T1-ready, 556 rows are guarded fail-closed, and the remaining 491 descriptors are explicitly surfaced as bridge-needed, UAT-needed, blocked-policy, or degraded/discovery-pending rather than overclaimed. + +
+ +
+v0.13.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 + +Archive files: + +- `.planning/milestones/v0.13.0-ROADMAP.md` +- `.planning/milestones/v0.13.0-REQUIREMENTS.md` +- `.planning/milestones/v0.13.0-MILESTONE-AUDIT.md` +- `.planning/milestones/v0.13.0-phases/` + +Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,314 descriptors across 128 app stems / 129 services. The milestone deliberately fed the existing tier model rather than hand-porting every action: 26 descriptors resolve to T1/T1b today, 5 guarded writes remain fail-closed, and 2,288 descriptors remain T3 DOM/discovery-tail. Full milestone audit passed; non-blocking live UAT and T1 expansion debt are carried forward. + +
+ +
+v0.9.99 Native Capability Catalog (FSB API Execution) — Phases 26-34, CODE-COMPLETE 2026-06-23 + +Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v0.13.0 milestone extends this substrate verbatim. + +| Phase | Name | Plans | Status | +|-------|------|-------|--------| +| 26 | Recipe Schema + Bundled Interpreter + MV3 CI Guard | 3/3 | Complete | +| 27 | Authenticated Fetch Primitive (MAIN-world) + Origin-Pin + Resume-Sidecar | 3/3 | Complete; live FETCH-05 UAT human_needed | +| 28 | Lean MCP Surface + Capability Search + Eval Harness | 4/4 | Complete | +| 29 | Catalog + Tiered Router + Bundled Head + Declarative Tail + Autopilot Parity | 5/5 | Complete; live-capture UAT human_needed | +| 30 | Consent Governance + Recipe Signature Verification + Audit + Legal Posture | 4/4 | Complete; live smoke human_needed | +| 31 | Network-Capture Discovery + Recipe Synthesis + Learned Recipes | 6/6 | Complete; live UAT human_needed | +| 32 | Self-Healing Fallback + Recipe-Rot + Re-Learn + Provider/Schema-Lock Tests + UAT | 5/5 | Complete; live self-heal UAT human_needed | +| 33 | PhantomStream Media Mirroring (0.2.1 Uptake) — milestone extension | 1/1 | Complete; live media UAT human_needed | +| 34 | Explicit File Upload Tool (upload_file) — milestone extension | 1/1 | Complete; live upload UAT human_needed | + +Substrate carried into v0.13.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard. + +
+ +
+v0.12.0 PhantomStream Package Migration — Phases 21-25, COMPLETED 2026-06-17 + +Archive files: + +- `.planning/milestones/v0.12.0-ROADMAP.md` +- `.planning/milestones/v0.12.0-REQUIREMENTS.md` +- `.planning/milestones/v0.12.0-MILESTONE-AUDIT.md` +- `.planning/milestones/v0.12.0-phases/` + +Phase summary: + +| Phase | Name | Plans | Status | +|-------|------|-------|--------| +| 21 | Package Intake & Contract Mapping | 3/3 | Complete | +| 22 | Capture Adapter Migration | 4/4 | Complete | +| 23 | Dashboard Renderer Migration | 4/4 | Complete | +| 24 | Transport, Relay & Remote Control Integration | 4/4 | Complete | +| 25 | Parity Removal, Docs & Browser UAT | 4/4 | Complete; human UAT debt recorded | + +Known deferred closeout evidence: live Chrome-extension dashboard preview and remote-control UAT remains `human_needed`; see `.planning/milestones/v0.12.0-phases/25-parity-removal-docs-browser-uat/25-HUMAN-UAT.md`. + +
+ +
+v0.11.0 Trigger Tool (Reactive DOM Monitoring) — Phases 14-20, COMPLETED 2026-06-17 + +Phase summary: + +| Phase | Name | Plans | Status | +|-------|------|-------|--------| +| 14 | Trigger Survivability Foundation | 3/3 | Complete | +| 15 | Fire-Condition Engine & Value Extraction | 3/3 | Complete | +| 16 | Live-Observe Watch & Analyzing Pulse | 4/4 | Complete | +| 17 | Refresh-Poll Watch (Tab-Owning Background Reload) | 4/4 | Complete | +| 18 | Shared Tool Registry & Dispatcher Wiring | 4/4 | Complete | +| 19 | MCP Tools & Blocking/Detached Reporting | 3/3 | Complete | +| 20 | Integration, Cap UI, Docs & Edge Cases | 5/5 | Complete; human UAT debt recorded | + +Known deferred closeout evidence: live-browser/composed trigger UAT remains `human_needed`; publish/tag/release actions remain user-gated. + +
+ +
+v0.10.0 Autopilot via Lattice SDK (Phases 01-13) — SHIPPED 2026-06-15 + +Archive files: + +- `.planning/milestones/v0.10.0-ROADMAP.md` +- `.planning/milestones/v0.10.0-REQUIREMENTS.md` +- `.planning/milestones/v0.10.0-MILESTONE-AUDIT.md` +- `.planning/milestones/v0.10.0-phases/` + +Phase summary: + +| Phase | Name | Plans | Status | +|-------|------|-------|--------| +| 01 | Lattice SDK gap survey + integration scaffolding | 2/2 | Complete | +| 02 | Lattice tripwire + receipt primitives extension | 5/5 | Complete | +| 03 | Observability + step-markers extension | 3/3 | Complete | +| 04 | Provider adapter alignment | 5/5 | Complete | +| 05 | MV3-survivability adapter contract + bundler infra + hybrid offscreen Lattice host | 6/6 | Complete | +| 06 | FSB engine consumes Lattice provider abstraction | 7/7 | Complete | +| 07 | Archive FSB custom provider stack | 4/4 | Complete | +| 08 | FSB agent brain on Lattice runtime | 3/3 | Complete | +| 09 | FSB SurvivabilityAdapter activated for MV3 SW eviction resumption | 3/3 | Complete | +| 10 | MCP-philosophy parity for autopilot driver | 3/3 | Complete | +| 11 | Tab-aware side panel surface | 5/5 | Complete | +| 12 | Side panel follows automation | 5/5 | Complete | +| 13 | Public Lattice package integration | 1/1 | Complete | + +Known deferred closeout evidence: 11 human-gated Chrome MV3/UAT verification items were acknowledged at close. See `.planning/STATE.md` `## Deferred Items`. + +
+ +## Carry-Forward Candidates + +- **v0.9.91 v2 deferred (see REQUIREMENTS.md):** CHAT-FUTURE-01/02 (chat-mode continuity via `--resume` / `codex resume` / `opencode --continue` + per-thread cwd pinning); GEMINI-FUTURE-01 (Gemini CLI adapter after live `--help` capture + JSONL schema pinning); ACP-FUTURE-01 (`@zed-industries/agent-client-protocol` unification once ≥2 non-Claude adapters have shipped); REMOTE-FUTURE-01 (remote/mobile delegation surfaces — v0.9.91 is explicitly localhost-only). +- **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. Does NOT block v0.9.91. +- **v1.2.0 v2 deferred:** QA-01 (native-speaker/bilingual QA pass), I18N-FUTURE-01 (migrate stats page off ad hoc JSON mechanism into main XLIFF pipeline), I18N-FUTURE-02 (full automated per-locale visual regression pipeline), I18N-FUTURE-03 (translation-freshness/"last synced" reporting surface). +- **v2 deferred capability families (acknowledged, out of v0.13.0/v1.1.0):** GAPI-01 (gapi-bridge handler family for Google Workspace); CLOUD-01 (cloud-console Pattern-D ports); UATX-01 (per-app live guarded-write UAT closeout). +- **Delegation primitive (Lattice-owned):** Parked from v0.10.0; re-scope as either a Lattice-owned primitive or an FSB-only consumer of Lattice receipt + tripwire surfaces after v0.9.91 stabilizes the adapter contract. + +## Backlog + +### Phase 999.1: MCP tool gaps — click heuristics + +**Status:** Completed historical backlog work retained outside milestone archival. + +- `999.1-01`: Route-aware MCP bridge dispatch + `execute_js` background handler. +- `999.1-02`: Text-based click targeting with TreeWalker visible-text matching. + +Artifacts remain in `.planning/phases/999.1-mcp-tool-gaps-click-heuristics/`. diff --git a/.planning/phases/57-agent-identity-capture/57-01-PLAN.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-01-PLAN.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-01-PLAN.md diff --git a/.planning/phases/57-agent-identity-capture/57-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-01-SUMMARY.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-01-SUMMARY.md diff --git a/.planning/phases/57-agent-identity-capture/57-02-PLAN.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-02-PLAN.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-02-PLAN.md diff --git a/.planning/phases/57-agent-identity-capture/57-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-02-SUMMARY.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-02-SUMMARY.md diff --git a/.planning/phases/57-agent-identity-capture/57-03-PLAN.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-03-PLAN.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-03-PLAN.md diff --git a/.planning/phases/57-agent-identity-capture/57-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-03-SUMMARY.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-03-SUMMARY.md diff --git a/.planning/phases/57-agent-identity-capture/57-CONTEXT.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-CONTEXT.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-CONTEXT.md diff --git a/.planning/phases/57-agent-identity-capture/57-DISCUSSION-LOG.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-DISCUSSION-LOG.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-DISCUSSION-LOG.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-DISCUSSION-LOG.md diff --git a/.planning/phases/57-agent-identity-capture/57-RESEARCH.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-RESEARCH.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-RESEARCH.md diff --git a/.planning/phases/57-agent-identity-capture/57-REVIEW-FIX.iter2.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW-FIX.iter2.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-REVIEW-FIX.iter2.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW-FIX.iter2.md diff --git a/.planning/phases/57-agent-identity-capture/57-REVIEW-FIX.iter3.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW-FIX.iter3.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-REVIEW-FIX.iter3.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW-FIX.iter3.md diff --git a/.planning/phases/57-agent-identity-capture/57-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW-FIX.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW-FIX.md diff --git a/.planning/phases/57-agent-identity-capture/57-REVIEW.iter2.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW.iter2.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-REVIEW.iter2.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW.iter2.md diff --git a/.planning/phases/57-agent-identity-capture/57-REVIEW.iter3.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW.iter3.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-REVIEW.iter3.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW.iter3.md diff --git a/.planning/phases/57-agent-identity-capture/57-REVIEW.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-REVIEW.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-REVIEW.md diff --git a/.planning/phases/57-agent-identity-capture/57-UI-REVIEW.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-UI-REVIEW.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-UI-REVIEW.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-UI-REVIEW.md diff --git a/.planning/phases/57-agent-identity-capture/57-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-UI-SPEC.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-UI-SPEC.md diff --git a/.planning/phases/57-agent-identity-capture/57-VALIDATION.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-VALIDATION.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-VALIDATION.md diff --git a/.planning/phases/57-agent-identity-capture/57-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-VERIFICATION.md similarity index 100% rename from .planning/phases/57-agent-identity-capture/57-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/57-agent-identity-capture/57-VERIFICATION.md diff --git a/.planning/phases/58-providers-panel/58-01-PLAN.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-01-PLAN.md similarity index 100% rename from .planning/phases/58-providers-panel/58-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-01-PLAN.md diff --git a/.planning/phases/58-providers-panel/58-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-01-SUMMARY.md similarity index 97% rename from .planning/phases/58-providers-panel/58-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-01-SUMMARY.md index 8751e6d55..4600e7636 100644 --- a/.planning/phases/58-providers-panel/58-01-SUMMARY.md +++ b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-01-SUMMARY.md @@ -87,7 +87,7 @@ None - plan executed exactly as written. ## Issues Encountered -- The pre-existing deletion of `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` caused the first raw root-suite run to stop in `coverage-report.test.js`. Each required root-suite gate used a temporary symlink to the identical archived v1.0.0 milestone copy; the link and temporary directory were removed after each run, preserving the user's deletion exactly. +- The pre-existing deletion of `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` caused the first raw root-suite run to stop in `coverage-report.test.js`. Each required root-suite gate used a temporary symlink to the identical archived v0.13.0 milestone copy; the link and temporary directory were removed after each run, preserving the user's deletion exactly. ## Verification diff --git a/.planning/phases/58-providers-panel/58-02-PLAN.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-02-PLAN.md similarity index 100% rename from .planning/phases/58-providers-panel/58-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-02-PLAN.md diff --git a/.planning/phases/58-providers-panel/58-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-02-SUMMARY.md similarity index 98% rename from .planning/phases/58-providers-panel/58-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-02-SUMMARY.md index 6b71a270c..dace7b5a8 100644 --- a/.planning/phases/58-providers-panel/58-02-SUMMARY.md +++ b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-02-SUMMARY.md @@ -105,7 +105,7 @@ Each task was committed atomically: ## Issues Encountered -- The pre-existing deletion of `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` caused the first raw Task 1 root-suite run to stop at `coverage-report.test.js`. Required full-suite gates temporarily used the identical archived v1.0.0 milestone copy via a symlink; the link and temporary directory were removed after each run, preserving the deletion. +- The pre-existing deletion of `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` caused the first raw Task 1 root-suite run to stop at `coverage-report.test.js`. Required full-suite gates temporarily used the identical archived v0.13.0 milestone copy via a symlink; the link and temporary directory were removed after each run, preserving the deletion. ## Verification diff --git a/.planning/phases/58-providers-panel/58-03-PLAN.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-03-PLAN.md similarity index 100% rename from .planning/phases/58-providers-panel/58-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-03-PLAN.md diff --git a/.planning/phases/58-providers-panel/58-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-03-SUMMARY.md similarity index 98% rename from .planning/phases/58-providers-panel/58-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-03-SUMMARY.md index a24238aec..a7ad44dad 100644 --- a/.planning/phases/58-providers-panel/58-03-SUMMARY.md +++ b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-03-SUMMARY.md @@ -99,7 +99,7 @@ None - plan behavior and test scope were implemented as specified. ## Issues Encountered -- The shared Conductor worktree intentionally lacks the old Phase 39 coverage manifest. Full-suite gates temporarily linked the identical archived v1.0.0 milestone copy and removed the link afterward, preserving the user's deletions. +- The shared Conductor worktree intentionally lacks the old Phase 39 coverage manifest. Full-suite gates temporarily linked the identical archived v0.13.0 milestone copy and removed the link afterward, preserving the user's deletions. - The first clean-worktree dependency provisioning used symlink paths with one extra parent segment, so the environment stopped at `tsc: command not found` with exit `127`. Corrected ignored dependency links restored `tsc`/Angular executables; a clean pre-test status was reconfirmed and the complete suite then exited `0`. - The in-app Browser runtime initialized, but browser discovery returned `[]` after its troubleshooting flow. No live visual item or screenshot was claimed; `58-VISUAL-QA.md` records exact human reproduction for each item. diff --git a/.planning/phases/58-providers-panel/58-CONTEXT.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-CONTEXT.md similarity index 100% rename from .planning/phases/58-providers-panel/58-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-CONTEXT.md diff --git a/.planning/phases/58-providers-panel/58-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-HUMAN-UAT.md similarity index 100% rename from .planning/phases/58-providers-panel/58-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-HUMAN-UAT.md diff --git a/.planning/phases/58-providers-panel/58-RESEARCH.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-RESEARCH.md similarity index 100% rename from .planning/phases/58-providers-panel/58-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-RESEARCH.md diff --git a/.planning/phases/58-providers-panel/58-REVIEW-FIX.iter2.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW-FIX.iter2.md similarity index 100% rename from .planning/phases/58-providers-panel/58-REVIEW-FIX.iter2.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW-FIX.iter2.md diff --git a/.planning/phases/58-providers-panel/58-REVIEW-FIX.iter3.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW-FIX.iter3.md similarity index 100% rename from .planning/phases/58-providers-panel/58-REVIEW-FIX.iter3.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW-FIX.iter3.md diff --git a/.planning/phases/58-providers-panel/58-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW-FIX.md similarity index 100% rename from .planning/phases/58-providers-panel/58-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW-FIX.md diff --git a/.planning/phases/58-providers-panel/58-REVIEW.iter2.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW.iter2.md similarity index 100% rename from .planning/phases/58-providers-panel/58-REVIEW.iter2.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW.iter2.md diff --git a/.planning/phases/58-providers-panel/58-REVIEW.iter3.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW.iter3.md similarity index 100% rename from .planning/phases/58-providers-panel/58-REVIEW.iter3.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW.iter3.md diff --git a/.planning/phases/58-providers-panel/58-REVIEW.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW.md similarity index 100% rename from .planning/phases/58-providers-panel/58-REVIEW.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-REVIEW.md diff --git a/.planning/phases/58-providers-panel/58-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-UI-SPEC.md similarity index 100% rename from .planning/phases/58-providers-panel/58-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-UI-SPEC.md diff --git a/.planning/phases/58-providers-panel/58-VALIDATION.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-VALIDATION.md similarity index 100% rename from .planning/phases/58-providers-panel/58-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-VALIDATION.md diff --git a/.planning/phases/58-providers-panel/58-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-VERIFICATION.md similarity index 100% rename from .planning/phases/58-providers-panel/58-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-VERIFICATION.md diff --git a/.planning/phases/58-providers-panel/58-VISUAL-QA.md b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-VISUAL-QA.md similarity index 99% rename from .planning/phases/58-providers-panel/58-VISUAL-QA.md rename to .planning/milestones/v0.9.91-phases/58-providers-panel/58-VISUAL-QA.md index c24670b17..27c78a35c 100644 --- a/.planning/phases/58-providers-panel/58-VISUAL-QA.md +++ b/.planning/milestones/v0.9.91-phases/58-providers-panel/58-VISUAL-QA.md @@ -57,7 +57,7 @@ Result: exit `0`. The final output ended with `no-orphan-descriptor: 10 passed, An initial environment-only provisioning attempt used dependency links with one extra parent segment and exited `127` at `tsc: command not found`. No product assertion failed. The links were corrected, both `mcp/node_modules/.bin/tsc` and `showcase/angular/node_modules/.bin/ng` were confirmed executable, the worktree was confirmed clean again, and the complete suite then exited `0`. As expected, the suite's build steps regenerated `mcp/build/index.js` and showcase discovery files after the clean precondition had been recorded. The temporary verification worktree was removed afterward. -The shared Conductor worktree also completed `npm test` with exit `0`; its only environment accommodation was a temporary link restoring the user-removed Phase 39 coverage manifest from `.planning/milestones/v1.0.0-phases/`. That link was removed after the run. +The shared Conductor worktree also completed `npm test` with exit `0`; its only environment accommodation was a temporary link restoring the user-removed Phase 39 coverage manifest from `.planning/milestones/v0.13.0-phases/`. That link was removed after the run. ## Browser availability diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-01-PLAN.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-01-PLAN.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-01-PLAN.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-01-SUMMARY.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-01-SUMMARY.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-02-PLAN.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-02-PLAN.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-02-PLAN.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-02-SUMMARY.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-02-SUMMARY.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-03-PLAN.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-03-PLAN.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-03-PLAN.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-03-SUMMARY.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-03-SUMMARY.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-04-PLAN.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-04-PLAN.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-04-PLAN.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-04-SUMMARY.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-04-SUMMARY.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-CONTEXT.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-CONTEXT.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-CONTEXT.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-HUMAN-UAT.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-HUMAN-UAT.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-PATTERNS.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-PATTERNS.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-PATTERNS.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-PATTERNS.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-RESEARCH.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-RESEARCH.md similarity index 99% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-RESEARCH.md index 0e228fe7c..8b6649a4b 100644 --- a/.planning/phases/59-reverse-request-channel-security-foundation/59-RESEARCH.md +++ b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-RESEARCH.md @@ -191,7 +191,7 @@ All Critical/High threats are blocking. No plan may defer one to Phase 60, becau | Full suite | `npm test` | | Expected feedback | focused tests under ~20 seconds; full suite several minutes | -The current workspace has user-owned deletions of old phase artifacts, while one legacy test expects `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`. For full-suite verification only, create a temporary symlink to `.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`, remove it in a shell trap, and never stage it. This is workspace hygiene, not Phase 59 product work. +The current workspace has user-owned deletions of old phase artifacts, while one legacy test expects `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`. For full-suite verification only, create a temporary symlink to `.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`, remove it in a shell trap, and never stage it. This is workspace hygiene, not Phase 59 product work. ### Per-requirement verification map diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-REVIEW-FIX.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-REVIEW-FIX.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-REVIEW.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-REVIEW.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-REVIEW.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-REVIEW.md diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-VALIDATION.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-VALIDATION.md similarity index 98% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-VALIDATION.md index bbecba730..816b772ff 100644 --- a/.planning/phases/59-reverse-request-channel-security-foundation/59-VALIDATION.md +++ b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-VALIDATION.md @@ -23,7 +23,7 @@ created: 2026-07-12 | **Full suite command** | `npm test` | | **Estimated runtime** | focused slices < 20 seconds; full suite several minutes | -The current workspace has user-owned deletions of historical phase artifacts. For the full suite only, create a temporary untracked symlink from `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` to `.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`, remove it with a shell trap, and never stage it. +The current workspace has user-owned deletions of historical phase artifacts. For the full suite only, create a temporary untracked symlink from `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` to `.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`, remove it with a shell trap, and never stage it. --- diff --git a/.planning/phases/59-reverse-request-channel-security-foundation/59-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-VERIFICATION.md similarity index 100% rename from .planning/phases/59-reverse-request-channel-security-foundation/59-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/59-reverse-request-channel-security-foundation/59-VERIFICATION.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-01-PLAN.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-01-PLAN.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-01-PLAN.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-01-SUMMARY.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-01-SUMMARY.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-02-PLAN.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-02-PLAN.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-02-PLAN.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-02-SUMMARY.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-02-SUMMARY.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-03-PLAN.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-03-PLAN.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-03-PLAN.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-03-SUMMARY.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-03-SUMMARY.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-04-PLAN.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-04-PLAN.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-04-PLAN.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-04-SUMMARY.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-04-SUMMARY.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-AI-SPEC.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-AI-SPEC.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-AI-SPEC.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-AI-SPEC.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-CONTEXT.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-CONTEXT.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-CONTEXT.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-DISCUSSION-LOG.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-DISCUSSION-LOG.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-DISCUSSION-LOG.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-DISCUSSION-LOG.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-HUMAN-UAT.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-HUMAN-UAT.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-RESEARCH.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-RESEARCH.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-RESEARCH.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter2.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter2.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter2.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter2.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter3.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter3.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter3.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.iter3.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW-FIX.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter2.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter2.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter2.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter2.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter3.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter3.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter3.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW.iter3.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-REVIEW.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-REVIEW.md diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-VALIDATION.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-VALIDATION.md similarity index 98% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-VALIDATION.md index e564b253c..fd660cd1d 100644 --- a/.planning/phases/60-adapter-contract-claude-code-mvp/60-VALIDATION.md +++ b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-VALIDATION.md @@ -23,7 +23,7 @@ created: 2026-07-14 | **Full suite command** | `node scripts/run-phase60-full-tests.mjs` (preflight + temporary Phase 39 link + `npm test` + finally cleanup + dirty/staged preservation checks) | | **Estimated runtime** | focused slices < 30 seconds; full root suite several minutes | -The workspace contains user-owned deletions of historical Phase 39 artifacts. For full-suite runs only, create an untracked symlink at `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` pointing to `.planning/milestones/v1.0.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`, install a shell trap that removes the link and empty directory, and never stage or commit it. +The workspace contains user-owned deletions of historical Phase 39 artifacts. For full-suite runs only, create an untracked symlink at `.planning/phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md` pointing to `.planning/milestones/v0.13.0-phases/39-breadth-c-commerce-travel-misc-most-sensitive/39-06-REMAINING-APPS.md`, install a shell trap that removes the link and empty directory, and never stage or commit it. --- diff --git a/.planning/phases/60-adapter-contract-claude-code-mvp/60-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-VERIFICATION.md similarity index 100% rename from .planning/phases/60-adapter-contract-claude-code-mvp/60-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/60-adapter-contract-claude-code-mvp/60-VERIFICATION.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-01-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-01-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-01-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-01-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-01-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-02-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-02-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-02-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-02-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-02-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-03-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-03-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-03-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-03-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-03-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-04-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-04-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-04-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-04-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-04-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-05-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-05-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-05-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-05-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-05-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-05-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-05-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-05-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-06-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-06-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-06-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-06-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-06-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-06-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-06-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-06-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-07-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-07-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-07-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-07-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-07-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-07-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-07-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-07-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-08-PLAN.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-08-PLAN.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-08-PLAN.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-08-PLAN.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-08-SUMMARY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-08-SUMMARY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-08-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-08-SUMMARY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-CONTEXT.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-CONTEXT.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-CONTEXT.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-DISCUSSION-LOG.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-DISCUSSION-LOG.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-DISCUSSION-LOG.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-DISCUSSION-LOG.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-HUMAN-UAT.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-HUMAN-UAT.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-PATTERNS.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-PATTERNS.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-PATTERNS.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-PATTERNS.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-RESEARCH.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-RESEARCH.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-RESEARCH.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW-FIX.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW-FIX.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-REVIEW.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-SECURITY.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-SECURITY.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-SECURITY.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-SECURITY.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-UI-REVIEW.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-UI-REVIEW.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-UI-REVIEW.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-UI-REVIEW.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-UI-SPEC.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-UI-SPEC.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-VALIDATION.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-VALIDATION.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-VALIDATION.md diff --git a/.planning/phases/61-delegation-ux-sw-eviction-persistence/61-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-VERIFICATION.md similarity index 100% rename from .planning/phases/61-delegation-ux-sw-eviction-persistence/61-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/61-delegation-ux-sw-eviction-persistence/61-VERIFICATION.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-PLAN.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-PLAN.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-PLAN.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-SUMMARY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-01-SUMMARY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-PLAN.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-PLAN.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-PLAN.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-SUMMARY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-02-SUMMARY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-PLAN.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-PLAN.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-PLAN.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-SUMMARY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-03-SUMMARY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-PLAN.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-PLAN.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-PLAN.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-SUMMARY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-04-SUMMARY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-PLAN.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-PLAN.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-PLAN.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-PLAN.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-SUMMARY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-SUMMARY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-05-SUMMARY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-PLAN.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-PLAN.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-PLAN.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-PLAN.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-SUMMARY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-SUMMARY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-06-SUMMARY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-CONTEXT.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-CONTEXT.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-CONTEXT.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-HUMAN-UAT.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-HUMAN-UAT.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-RESEARCH.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-RESEARCH.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-RESEARCH.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW-FIX.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW-FIX.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-REVIEW.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-SECURITY.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-SECURITY.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-SECURITY.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-SECURITY.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-REVIEW.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-REVIEW.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-REVIEW.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-REVIEW.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-SPEC.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-UI-SPEC.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-VALIDATION.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-VALIDATION.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-VALIDATION.md diff --git a/.planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-VERIFICATION.md similarity index 100% rename from .planning/phases/62-ci-drift-smoke-gate-doctor-extensions/62-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/62-ci-drift-smoke-gate-doctor-extensions/62-VERIFICATION.md diff --git a/.planning/phases/63-native-messaging-host/63-01-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-01-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-01-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-01-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-01-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-02-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-02-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-02-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-02-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-02-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-03-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-03-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-03-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-03-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-03-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-04-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-04-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-04-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-04-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-04-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-05-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-05-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-05-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-05-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-05-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-05-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-05-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-05-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-06-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-06-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-06-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-06-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-06-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-06-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-06-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-06-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-07-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-07-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-07-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-07-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-07-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-07-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-07-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-07-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-08-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-08-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-08-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-08-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-08-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-08-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-08-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-08-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-09-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-09-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-09-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-09-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-09-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-09-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-09-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-09-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-10-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-10-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-10-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-10-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-10-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-10-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-10-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-10-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-11-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-11-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-11-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-11-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-11-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-11-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-11-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-11-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-12-PLAN.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-12-PLAN.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-12-PLAN.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-12-PLAN.md diff --git a/.planning/phases/63-native-messaging-host/63-12-SUMMARY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-12-SUMMARY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-12-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-12-SUMMARY.md diff --git a/.planning/phases/63-native-messaging-host/63-CONTEXT.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-CONTEXT.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-CONTEXT.md diff --git a/.planning/phases/63-native-messaging-host/63-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-HUMAN-UAT.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-HUMAN-UAT.md diff --git a/.planning/phases/63-native-messaging-host/63-PATTERNS.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-PATTERNS.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-PATTERNS.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-PATTERNS.md diff --git a/.planning/phases/63-native-messaging-host/63-RESEARCH.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-RESEARCH.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-RESEARCH.md diff --git a/.planning/phases/63-native-messaging-host/63-REVIEW.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-REVIEW.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-REVIEW.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-REVIEW.md diff --git a/.planning/phases/63-native-messaging-host/63-SECURITY.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-SECURITY.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-SECURITY.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-SECURITY.md diff --git a/.planning/phases/63-native-messaging-host/63-UI-REVIEW.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-UI-REVIEW.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-UI-REVIEW.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-UI-REVIEW.md diff --git a/.planning/phases/63-native-messaging-host/63-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-UI-SPEC.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-UI-SPEC.md diff --git a/.planning/phases/63-native-messaging-host/63-VALIDATION.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-VALIDATION.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-VALIDATION.md diff --git a/.planning/phases/63-native-messaging-host/63-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/63-native-messaging-host/63-VERIFICATION.md similarity index 100% rename from .planning/phases/63-native-messaging-host/63-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/63-native-messaging-host/63-VERIFICATION.md diff --git a/.planning/phases/64-opencode-adapter/64-01-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-01-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-01-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-01-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-01-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-02-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-02-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-02-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-02-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-02-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-03-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-03-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-03-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-03-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-03-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-04-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-04-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-04-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-04-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-04-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-05-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-05-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-05-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-05-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-05-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-05-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-05-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-05-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-06-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-06-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-06-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-06-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-06-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-06-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-06-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-06-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-07-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-07-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-07-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-07-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-07-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-07-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-07-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-07-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-08-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-08-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-08-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-08-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-08-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-08-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-08-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-08-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-09-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-09-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-09-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-09-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-09-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-09-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-09-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-09-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-10-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-10-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-10-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-10-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-10-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-10-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-10-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-10-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-11-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-11-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-11-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-11-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-11-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-11-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-11-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-11-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-12-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-12-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-12-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-12-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-12-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-12-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-12-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-12-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-13-PLAN.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-13-PLAN.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-13-PLAN.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-13-PLAN.md diff --git a/.planning/phases/64-opencode-adapter/64-13-SUMMARY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-13-SUMMARY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-13-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-13-SUMMARY.md diff --git a/.planning/phases/64-opencode-adapter/64-CONTEXT.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-CONTEXT.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-CONTEXT.md diff --git a/.planning/phases/64-opencode-adapter/64-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-HUMAN-UAT.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-HUMAN-UAT.md diff --git a/.planning/phases/64-opencode-adapter/64-PATTERNS.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-PATTERNS.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-PATTERNS.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-PATTERNS.md diff --git a/.planning/phases/64-opencode-adapter/64-RESEARCH.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-RESEARCH.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-RESEARCH.md diff --git a/.planning/phases/64-opencode-adapter/64-REVIEW-FIX.iter2.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW-FIX.iter2.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-REVIEW-FIX.iter2.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW-FIX.iter2.md diff --git a/.planning/phases/64-opencode-adapter/64-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW-FIX.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW-FIX.md diff --git a/.planning/phases/64-opencode-adapter/64-REVIEW.iter2.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW.iter2.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-REVIEW.iter2.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW.iter2.md diff --git a/.planning/phases/64-opencode-adapter/64-REVIEW.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-REVIEW.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-REVIEW.md diff --git a/.planning/phases/64-opencode-adapter/64-SECURITY.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-SECURITY.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-SECURITY.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-SECURITY.md diff --git a/.planning/phases/64-opencode-adapter/64-UI-REVIEW.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-UI-REVIEW.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-UI-REVIEW.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-UI-REVIEW.md diff --git a/.planning/phases/64-opencode-adapter/64-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-UI-SPEC.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-UI-SPEC.md diff --git a/.planning/phases/64-opencode-adapter/64-VALIDATION.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-VALIDATION.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-VALIDATION.md diff --git a/.planning/phases/64-opencode-adapter/64-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/64-opencode-adapter/64-VERIFICATION.md similarity index 100% rename from .planning/phases/64-opencode-adapter/64-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/64-opencode-adapter/64-VERIFICATION.md diff --git a/.planning/phases/65-codex-adapter/65-01-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-01-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-01-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-01-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-01-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-01-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-01-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-01-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-02-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-02-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-02-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-02-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-02-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-02-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-02-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-02-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-03-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-03-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-03-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-03-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-03-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-03-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-03-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-03-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-04-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-04-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-04-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-04-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-04-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-04-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-04-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-04-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-05-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-05-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-05-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-05-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-05-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-05-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-05-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-05-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-06-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-06-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-06-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-06-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-06-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-06-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-06-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-06-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-07-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-07-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-07-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-07-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-07-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-07-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-07-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-07-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-08-PLAN.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-08-PLAN.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-08-PLAN.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-08-PLAN.md diff --git a/.planning/phases/65-codex-adapter/65-08-SUMMARY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-08-SUMMARY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-08-SUMMARY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-08-SUMMARY.md diff --git a/.planning/phases/65-codex-adapter/65-CONTEXT.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-CONTEXT.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-CONTEXT.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-CONTEXT.md diff --git a/.planning/phases/65-codex-adapter/65-HUMAN-UAT.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-HUMAN-UAT.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-HUMAN-UAT.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-HUMAN-UAT.md diff --git a/.planning/phases/65-codex-adapter/65-RESEARCH.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-RESEARCH.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-RESEARCH.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-RESEARCH.md diff --git a/.planning/phases/65-codex-adapter/65-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-REVIEW-FIX.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-REVIEW-FIX.md diff --git a/.planning/phases/65-codex-adapter/65-REVIEW.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-REVIEW.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-REVIEW.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-REVIEW.md diff --git a/.planning/phases/65-codex-adapter/65-SECURITY.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-SECURITY.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-SECURITY.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-SECURITY.md diff --git a/.planning/phases/65-codex-adapter/65-UI-REVIEW-FIX.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-UI-REVIEW-FIX.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-UI-REVIEW-FIX.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-UI-REVIEW-FIX.md diff --git a/.planning/phases/65-codex-adapter/65-UI-REVIEW.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-UI-REVIEW.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-UI-REVIEW.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-UI-REVIEW.md diff --git a/.planning/phases/65-codex-adapter/65-UI-SPEC.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-UI-SPEC.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-UI-SPEC.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-UI-SPEC.md diff --git a/.planning/phases/65-codex-adapter/65-VALIDATION.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-VALIDATION.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-VALIDATION.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-VALIDATION.md diff --git a/.planning/phases/65-codex-adapter/65-VERIFICATION.md b/.planning/milestones/v0.9.91-phases/65-codex-adapter/65-VERIFICATION.md similarity index 100% rename from .planning/phases/65-codex-adapter/65-VERIFICATION.md rename to .planning/milestones/v0.9.91-phases/65-codex-adapter/65-VERIFICATION.md diff --git a/.planning/milestones/v1.1.0-REQUIREMENTS.md b/.planning/milestones/v1.1.0-REQUIREMENTS.md index 05fcb77a8..0a9f16740 100644 --- a/.planning/milestones/v1.1.0-REQUIREMENTS.md +++ b/.planning/milestones/v1.1.0-REQUIREMENTS.md @@ -11,7 +11,7 @@ For current requirements, see `.planning/REQUIREMENTS.md`. **Defined:** 2026-06-29 **Extended:** 2026-06-29 with Phase 51 full-tail migration. -**Core Value:** Reliable single-attempt execution -- the AI decides correctly; the mechanics execute precisely. v1.1.0 focuses on converting the v1.0.0 catalog tail from search/discovery support into verified direct T1 execution where it is technically and safely provable. +**Core Value:** Reliable single-attempt execution -- the AI decides correctly; the mechanics execute precisely. v1.1.0 focuses on converting the v0.13.0 catalog tail from search/discovery support into verified direct T1 execution where it is technically and safely provable. ## v1 Requirements @@ -27,7 +27,7 @@ Requirements for the v1.1.0 milestone. Each maps to roadmap phases beginning at - [x] **T1R-06**: Port a first batch of high-value read descriptors to executable T1/T1b, selected by readiness feasibility, user value, and low side-effect risk. - [x] **T1R-07**: Implement or explicitly reject a Pattern-D execution architecture for separate-origin/per-org-subdomain APIs, with negative-control tests proving unsafe ports fail closed. - [x] **T1R-08**: Spike and decide the Google Workspace GAPI bridge handler family for Docs, Drive, and Calendar through a page-owned `window.gapi.client.request` trampoline. -- [x] **T1R-09**: Preserve all v1.0.0 security walls: no OpenTabs runtime/plugin code, no new MCP tool-per-app surface, no secret persistence, and denylist/sensitive-origin behavior unchanged unless deliberately re-scoped. +- [x] **T1R-09**: Preserve all v0.13.0 security walls: no OpenTabs runtime/plugin code, no new MCP tool-per-app surface, no secret persistence, and denylist/sensitive-origin behavior unchanged unless deliberately re-scoped. - [x] **T1R-10**: Generalize the live UAT template for guarded writes/destructive actions, recording method/path/body shape and CSRF/token location redacted to shape only. - [x] **T1R-11**: Activate write/destructive T1 handlers only when live mutation-body UAT and consent/audit checks pass; otherwise keep them fail-closed with explicit reasons. - [x] **T1R-12**: Close with a before/after T1 coverage report, full regression suite, validate:extension, and prioritized next-batch plan for the remaining tail. @@ -89,4 +89,4 @@ Acknowledged but deferred beyond v1.1.0 unless pulled into an approved phase. - Unmapped: 0 --- -*Requirements defined: 2026-06-29 -- v1.1.0 T1 App Execution Expansion milestone started after closing v1.0.0 and was extended with Phase 51 full-tail migration on 2026-06-29.* +*Requirements defined: 2026-06-29 -- v1.1.0 T1 App Execution Expansion milestone started after closing v0.13.0 and was extended with Phase 51 full-tail migration on 2026-06-29.* diff --git a/.planning/milestones/v1.1.0-ROADMAP.md b/.planning/milestones/v1.1.0-ROADMAP.md index 8f10c98ea..2be71d80f 100644 --- a/.planning/milestones/v1.1.0-ROADMAP.md +++ b/.planning/milestones/v1.1.0-ROADMAP.md @@ -2,7 +2,7 @@ ## Milestones -- ✅ **v1.0.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v1.0.0-ROADMAP.md`. +- ✅ **v0.13.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v0.13.0-ROADMAP.md`. - ✅ **v1.1.0 T1 App Execution Expansion** — Phases 44-51, shipped 2026-06-30; refreshed 2026-07-01 after post-closeout T1 ports. Expanded proven T1/guarded coverage from 26 baseline descriptors to 1,267 executable T1-ready descriptors plus 556 guarded fail-closed rows, then closed the remaining catalog tail with explicit terminal-state accounting. ## Milestone Closed @@ -44,7 +44,7 @@ **Goal:** Create the authoritative T1 readiness matrix for all 2,314 descriptors and make status visible to developers and users so “catalog supported” is never confused with “direct API-ready.” -**Depends on:** v1.0.0 archive. +**Depends on:** v0.13.0 archive. **Requirements:** T1R-01, T1R-02, T1R-03. @@ -221,14 +221,14 @@ ## Completed Milestones
-v1.0.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 +v0.13.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 Archive files: -- `.planning/milestones/v1.0.0-ROADMAP.md` -- `.planning/milestones/v1.0.0-REQUIREMENTS.md` -- `.planning/milestones/v1.0.0-MILESTONE-AUDIT.md` -- `.planning/milestones/v1.0.0-phases/` +- `.planning/milestones/v0.13.0-ROADMAP.md` +- `.planning/milestones/v0.13.0-REQUIREMENTS.md` +- `.planning/milestones/v0.13.0-MILESTONE-AUDIT.md` +- `.planning/milestones/v0.13.0-phases/` Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,314 descriptors across 128 app stems / 129 services. The milestone deliberately fed the existing tier model rather than hand-porting every action: 26 descriptors resolve to T1/T1b today, 5 guarded writes remain fail-closed, and 2,288 descriptors remain T3 DOM/discovery-tail. Full milestone audit passed; non-blocking live UAT and T1 expansion debt are carried forward. @@ -239,7 +239,7 @@ Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,3
v0.9.99 Native Capability Catalog (FSB API Execution) — Phases 26-34, CODE-COMPLETE 2026-06-23 -Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v1.0.0 milestone extends this substrate verbatim. +Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v0.13.0 milestone extends this substrate verbatim. | Phase | Name | Plans | Status | |-------|------|-------|--------| @@ -253,7 +253,7 @@ Gave FSB first-class authenticated-API execution as a fast path alongside DOM au | 33 | PhantomStream Media Mirroring (0.2.1 Uptake) — milestone extension | 1/1 | Complete; live media UAT human_needed | | 34 | Explicit File Upload Tool (upload_file) — milestone extension | 1/1 | Complete; live upload UAT human_needed | -Substrate carried into v1.0.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard. +Substrate carried into v0.13.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard.
@@ -334,8 +334,8 @@ Known deferred closeout evidence: 11 human-gated Chrome MV3/UAT verification ite ## Carry-Forward Candidates -- **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. v1.0.0 does NOT block on it. -- **v2 deferred capability families (acknowledged, out of v1.0.0):** GAPI-01 (gapi-bridge handler family for Google Workspace via the `window.gapi.client.request` trampoline); CLOUD-01 (cloud-console Pattern-D ports — aws-console/azure/google-cloud/terraform-cloud — pending per-app CORS verification); UATX-01 (per-app live guarded-write UAT closeout across the hand-ported depth tier). +- **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. v0.13.0 does NOT block on it. +- **v2 deferred capability families (acknowledged, out of v0.13.0):** GAPI-01 (gapi-bridge handler family for Google Workspace via the `window.gapi.client.request` trampoline); CLOUD-01 (cloud-console Pattern-D ports — aws-console/azure/google-cloud/terraform-cloud — pending per-app CORS verification); UATX-01 (per-app live guarded-write UAT closeout across the hand-ported depth tier). - **Delegation primitive:** Parked from v0.10.0; re-scope as either a Lattice-owned primitive or an FSB-only consumer of Lattice receipt + tripwire surfaces. ## Backlog diff --git a/.planning/milestones/v1.2.0-ROADMAP.md b/.planning/milestones/v1.2.0-ROADMAP.md index c049b04f5..fdcafd0f2 100644 --- a/.planning/milestones/v1.2.0-ROADMAP.md +++ b/.planning/milestones/v1.2.0-ROADMAP.md @@ -4,7 +4,7 @@ - 🚧 **v1.2.0 Showcase i18n Completeness** — Phases 52-56, in progress. Closes the translation gap that reopened after v0.9.63 shipped: a full-page audit establishes the true drift/missing scope, resync + stats-page translation + transcreation review close it, a CI drift-detection gate lands on the clean baseline, and the long-deferred WARNING-02 locale-cookie redirect bug is fixed. - ✅ **v1.1.0 T1 App Execution Expansion** — Phases 44-51, shipped 2026-06-30; refreshed 2026-07-01 after post-closeout T1 ports. Expanded proven T1/guarded coverage from 26 baseline descriptors to 1,267 executable T1-ready descriptors plus 556 guarded fail-closed rows, and closed the remaining catalog tail with explicit terminal-state accounting. Archive: `.planning/milestones/v1.1.0-ROADMAP.md`. -- ✅ **v1.0.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v1.0.0-ROADMAP.md`. +- ✅ **v0.13.0 Full App Catalog (OpenTabs Parity)** — Phases 35-43, shipped 2026-06-29. Full OpenTabs-derived catalog/search/discovery surface: 2,314 descriptors across 128 app stems / 129 services; 26 T1/T1b descriptors today; 2,288 descriptors intentionally remain DOM/discovery-tail. Archive: `.planning/milestones/v0.13.0-ROADMAP.md`. ## Current Milestone: v1.2.0 Showcase i18n Completeness @@ -181,19 +181,19 @@ Archive files: - `.planning/milestones/v1.1.0-REQUIREMENTS.md` - `.planning/milestones/v1.1.0-MILESTONE-AUDIT.md` -Outcome: v1.1.0 expanded proven direct execution from the v1.0.0 baseline into verified T1-ready read coverage across Netlify, Bitbucket, CircleCI, Vercel, Retool, Asana, and generated same-origin reads, with post-closeout ports reflected in the refreshed artifacts. Phase 51 closes the remaining catalog tail honestly: 1,267 rows are executable T1-ready, 556 rows are guarded fail-closed, and the remaining 491 descriptors are explicitly surfaced as bridge-needed, UAT-needed, blocked-policy, or degraded/discovery-pending rather than overclaimed. +Outcome: v1.1.0 expanded proven direct execution from the v0.13.0 baseline into verified T1-ready read coverage across Netlify, Bitbucket, CircleCI, Vercel, Retool, Asana, and generated same-origin reads, with post-closeout ports reflected in the refreshed artifacts. Phase 51 closes the remaining catalog tail honestly: 1,267 rows are executable T1-ready, 556 rows are guarded fail-closed, and the remaining 491 descriptors are explicitly surfaced as bridge-needed, UAT-needed, blocked-policy, or degraded/discovery-pending rather than overclaimed.
-v1.0.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 +v0.13.0 Full App Catalog (OpenTabs Parity) — Phases 35-43, SHIPPED 2026-06-29 Archive files: -- `.planning/milestones/v1.0.0-ROADMAP.md` -- `.planning/milestones/v1.0.0-REQUIREMENTS.md` -- `.planning/milestones/v1.0.0-MILESTONE-AUDIT.md` -- `.planning/milestones/v1.0.0-phases/` +- `.planning/milestones/v0.13.0-ROADMAP.md` +- `.planning/milestones/v0.13.0-REQUIREMENTS.md` +- `.planning/milestones/v0.13.0-MILESTONE-AUDIT.md` +- `.planning/milestones/v0.13.0-phases/` Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,314 descriptors across 128 app stems / 129 services. The milestone deliberately fed the existing tier model rather than hand-porting every action: 26 descriptors resolve to T1/T1b today, 5 guarded writes remain fail-closed, and 2,288 descriptors remain T3 DOM/discovery-tail. Full milestone audit passed; non-blocking live UAT and T1 expansion debt are carried forward. @@ -202,7 +202,7 @@ Outcome: full OpenTabs-derived catalog/search/discovery surface shipped with 2,3
v0.9.99 Native Capability Catalog (FSB API Execution) — Phases 26-34, CODE-COMPLETE 2026-06-23 -Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v1.0.0 milestone extends this substrate verbatim. +Gave FSB first-class authenticated-API execution as a fast path alongside DOM automation, between Wall 1 (closed-vocabulary recipe DATA bound by a fixed interpreter) and Wall 2 (MAIN-world authenticated fetch). Full `npm test` EXIT 0; live-browser UAT debt (UAT-27/29/30/31/32-01) carried forward. The v0.13.0 milestone extends this substrate verbatim. | Phase | Name | Plans | Status | |-------|------|-------|--------| @@ -216,7 +216,7 @@ Gave FSB first-class authenticated-API execution as a fast path alongside DOM au | 33 | PhantomStream Media Mirroring (0.2.1 Uptake) — milestone extension | 1/1 | Complete; live media UAT human_needed | | 34 | Explicit File Upload Tool (upload_file) — milestone extension | 1/1 | Complete; live upload UAT human_needed | -Substrate carried into v1.0.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard. +Substrate carried into v0.13.0 (FIXED — do not redesign): tiers T0/T1a/T1b/T2-learned/T3-DOM; the closed-vocab interpreter; the consent gate (opt-out Auto default, denylist = the ONE hard floor); the 2 out-of-`TOOL_REGISTRY` MCP tools; `capability-catalog.js resolve()` / `capability-router.js invoke()` / `capability-search.js buildIndex()`; `scripts/package-extension.mjs readJsonDir` + the generated `recipe-index.generated.js` IIFE; the `github.js` T1a hand-port contract; `service-denylist.js` loader; `network-capture.js` discovery path; `verify-recipe-path-guard.mjs` Wall-1 guard.
@@ -298,7 +298,7 @@ Known deferred closeout evidence: 11 human-gated Chrome MV3/UAT verification ite ## Carry-Forward Candidates - **Consolidated Chrome MV3 UAT debt:** Run and capture archived v0.10/v0.11/v0.12 + v0.9.99 (UAT-27/29/30/31/32-01) browser evidence if release policy requires post-close proof. Does NOT block v1.2.0. -- **v2 deferred capability families (acknowledged, out of v1.0.0/v1.1.0):** GAPI-01 (gapi-bridge handler family for Google Workspace); CLOUD-01 (cloud-console Pattern-D ports); UATX-01 (per-app live guarded-write UAT closeout). +- **v2 deferred capability families (acknowledged, out of v0.13.0/v1.1.0):** GAPI-01 (gapi-bridge handler family for Google Workspace); CLOUD-01 (cloud-console Pattern-D ports); UATX-01 (per-app live guarded-write UAT closeout). - **v1.2.0 v2 deferred (see REQUIREMENTS.md):** QA-01 (native-speaker/bilingual QA pass), I18N-FUTURE-01 (migrate stats page off ad hoc JSON mechanism into main XLIFF pipeline), I18N-FUTURE-02 (full automated per-locale visual regression pipeline), I18N-FUTURE-03 (translation-freshness/"last synced" reporting surface). - **Delegation primitive:** Parked from v0.10.0; re-scope as either a Lattice-owned primitive or an FSB-only consumer of Lattice receipt + tripwire surfaces. diff --git a/.planning/phases/66-baseline-and-session-timing/.gitkeep b/.planning/phases/66-baseline-and-session-timing/.gitkeep new file mode 100644 index 000000000..8b1378917 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/.gitkeep @@ -0,0 +1 @@ + diff --git a/.planning/phases/66-baseline-and-session-timing/66-01-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-01-PLAN.md new file mode 100644 index 000000000..2117d8c47 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-01-PLAN.md @@ -0,0 +1,245 @@ +--- +phase: 66-baseline-and-session-timing +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - .gitignore + - scripts/bench/export-history.mjs + - scripts/bench/history-export-snippet.js + - tests/bench-export-history.test.mjs + - tests/fixtures/bench/history-export-sample.json + - scripts/bench/results/history-2026-09-16_21/metrics.json +autonomous: false +requirements: [MEAS-03] +user_setup: [] + +must_haves: + truths: + - "The 40 MCP sessions recorded 2026-09-16..21 in the Web Store build are exported before any benchmark recording, so pruning (~2026-10-16..21) cannot lose them (D-20)." + - "Raw session exports live only under the gitignored scripts/bench/.local/ tree; only sanitized per-session metrics are committed (D-21)." + - "Committed history metrics contain no session ids, task text, URLs, or request/result values, and label their method as journal-timestamp deltas that include caller gap and IndexedDB write-queue delay (D-04 caveat, D-21)." + - "The history metrics reproduce per-client (Claude, Cursor) per-call medians so the Phase 66 baseline can be compared with the PROJECT.md figures (9.5 s / 4.6 s)." + artifacts: + - path: ".gitignore" + provides: "Ignore rule for raw benchmark captures" + contains: "scripts/bench/.local/" + - path: "scripts/bench/export-history.mjs" + provides: "Pure deriveHistoryMetrics + assertNoLeak + CLI with direct-run guard" + exports: ["deriveHistoryMetrics", "assertNoLeak", "HISTORY_METRICS_METHOD"] + - path: "scripts/bench/history-export-snippet.js" + provides: "DevTools console snippet for the Web Store control panel (H1)" + contains: "getSessionDetail" + - path: "tests/bench-export-history.test.mjs" + provides: "Node 20-safe node:test coverage of derivation and leak check" + - path: "scripts/bench/results/history-2026-09-16_21/metrics.json" + provides: "Committed sanitized history metrics" + key_links: + - from: "scripts/bench/history-export-snippet.js" + to: "extension/background.js getSessionDetail runtime case" + via: "chrome.runtime.sendMessage({action:'getSessionDetail', sessionId, afterSequence, limit:500})" + pattern: "action: 'getSessionDetail'" + - from: "scripts/bench/export-history.mjs" + to: "scripts/bench/results/history-2026-09-16_21/metrics.json" + via: "CLI --input --out after assertNoLeak passes" + pattern: "assertNoLeak" +--- + + +Export the 40 MCP sessions recorded 2026-09-16..21 from the Web Store FSB build before they are pruned. Then commit only sanitized per-session metrics derived from that export. + +Purpose: D-20 requires this export before any new recording. These sessions are the only real-world MCP timing history, and the 50-session and 30-day caps will prune them around 2026-10-16..21. D-21 requires raw exports to stay out of git. +Output: a gitignore rule for `scripts/bench/.local/`, a derivation CLI plus its test, a DevTools snippet for the human step (H1), and `scripts/bench/results/history-2026-09-16_21/metrics.json`. + +This plan is URGENT and runs in wave 1. Its human checkpoint (H1) should be completed as soon as it is reached, before any further MCP use in Chrome Profile 2. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/ROADMAP.md +@.planning/REQUIREMENTS.md +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +Runtime message the snippet uses (extension/background.js, `case 'getSessionDetail'`, ~12513-12536). Request: `{action:'getSessionDetail', sessionId, afterSequence, limit}`. Journal response: `{success:true, session:, events:[...], nextSequence, hasMore}`. Legacy response: `{success:true, session:, events:[], nextSequence:null, hasMore:false, legacy:true}`. Failure: `{success:false, error}`. + +Journal event shape (extension/utils/mcp-lattice-journal.js `makeEvent` / `recordDispatchNow` metadata ~1210-1229): `{sequence, kind:'tool.call'|..., timestamp:, metadata:{tool, route, client, success, tabId, logicalTab, request:, result:, ...}}`. Descriptors carry `storage: 'inline'|'artifact'|'omitted'` and, when inline, the raw value under `inline` (page content; never commit). + +Session index entries (`fsbSessionIndex` in chrome.storage.local): `{id, task, startTime, endTime, status, actionCount, eventCount, mode, mcpClient, storageBackend, outcome, outcomeDetails, ...}`. MCP sessions have `mode === 'mcp-agent'`. + +Module style to copy: scripts/report-t1-readiness.mjs (header, pure exports, `writeReport`, direct-run guard `if (import.meta.url === pathToFileURL(process.argv[1] || '').href)`). ESM test header style: tests/dashboard-localization-fallback.test.mjs:1-6. + + + + + + + Task 1: Gitignore raw captures, write the export snippet, and build the history-metrics derivation with a leak check + .gitignore, scripts/bench/history-export-snippet.js, scripts/bench/export-history.mjs, tests/bench-export-history.test.mjs, tests/fixtures/bench/history-export-sample.json + + - .gitignore (current sections; global `build/`, `dist/`, `*.log` ignores already exist) + - extension/background.js lines 12494-12540 (`exportSessionHumanReadable` and `getSessionDetail` runtime cases the snippet calls) + - extension/ui/options.js lines 3430-3490 (`journalDescriptorPreview`, `hydrateJournalSessionDetail`, `loadSessionDetail` paging with limit 500) + - extension/utils/mcp-lattice-journal.js lines 836-872 (`buildIndexEntry` fields) and 1615-1648 (`eventForDetail`, `getSessionDetail`) + - scripts/report-t1-readiness.mjs lines 14-24 and 1241-1280 (module header, writeReport, direct-run guard) + - tests/dashboard-localization-fallback.test.mjs lines 1-6 (ESM test header) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md sections "Pattern 10" and "Code Examples / Exporting the 40 journal sessions" + + + - deriveHistoryMetrics(sampleExport) returns sessionCount 3 and callCount equal to the number of tool.call events in the sample. + - byClient.Claude.deltaMs.median equals the median of consecutive tool.call timestamp deltas across the Claude sessions in the sample (hand-computed in the test). + - Sessions are identified only by ordinal (1..N in startTime order); no output value equals any sample session id. + - assertNoLeak(sample, metrics) passes for the derived metrics, and throws when a metrics copy has a sample task string, a sample URL, or a sample inline request/result string injected into it. + - Allowlisted enum values are exempt from the leak check (W13). assertNoLeak must not throw when a raw inline payload string is exactly a value the derivation legitimately emits: an outcome enum value (for example `completed`), a validated tool name (for example `mcp:read-page`), a sanitized client label, or the method string. The test puts `completed` and `mcp:read-page` as inline result strings in the raw sample and asserts no leak, while a real task string or URL in the same sample still throws. + - Output is deterministic: two derivations of the same input serialize byte-identically. + - A client label that fails /^[A-Za-z0-9 ._-]{1,40}$/ is reported as "other". + + +1. Append to `.gitignore` a commented block: the comment line `# Phase 66 -- raw benchmark captures (may contain page content; D-21)`, followed by the rule `scripts/bench/.local/`. Do not name any committed output directory `build`, `dist`, or give it a `.log` extension. + +2. Create `scripts/bench/history-export-snippet.js`. It is a plain browser-console script: one async IIFE with no imports, run from the Web Store control panel's DevTools console at `chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk/ui/control_panel.html`. Its behavior: + - Read `fsbSessionIndex` via `chrome.storage.local.get(['fsbSessionIndex'])`. + - Select entries with `mode === 'mcp-agent'` and `startTime` in [local 2026-09-16T00:00:00, local 2026-09-22T00:00:00). + - For each selected id, page `chrome.runtime.sendMessage({action:'getSessionDetail', sessionId, afterSequence, limit: 500})`, starting at `afterSequence = -1` and stopping when `hasMore` is false or after 200 pages. Keep `session`, the concatenated `events`, and `legacy` (true when the response had `legacy: true`). + - Build `{schema:'fsb-history-export', schemaVersion:1, exportedAt, extensionId: chrome.runtime.id, version: chrome.runtime.getManifest().version, window:{from:'2026-09-16', to:'2026-09-21'}, index:, sessions:[{indexEntry, legacy, session, events}]}`. + - Download it as `fsb-history-2026-09-16_21.json` through a Blob object URL and a temporary anchor click. + - `console.log` a line `exported sessions ( events)`. + + Put a header comment in the file saying it is the H1 export for Phase 66, runs only in the Web Store control panel console, and its output must be moved into `scripts/bench/.local/`. + +3. Create `scripts/bench/export-history.mjs` (ESM, Node 20-safe, no dependencies), with these exports: + - `HISTORY_METRICS_METHOD`: the exact string `journal event timestamp deltas between consecutive tool.call events; include caller gap and IndexedDB write-queue delay; not FSB execution time`. + - `deriveHistoryMetrics(exportJson)`. It returns `{v:1, method, source:{exportedAt, extensionVersion, extensionIsWebStore: extensionId === 'badgafnfchcihdfnjneklogedcdkmjfk', window}, sessionCount, callCount, byClient:{ + + node --test tests/bench-export-history.test.mjs && git check-ignore -q scripts/bench/.local/probe.json && node --check scripts/bench/history-export-snippet.js + + + - `git check-ignore -q scripts/bench/.local/probe.json` exits 0, and `git check-ignore -q scripts/bench/results/x.json` exits 1. + - `node --test tests/bench-export-history.test.mjs` exits 0 with at least 7 passing tests, including the W13 enum-exemption case. + - `node -e "import('./scripts/bench/export-history.mjs').then(m=>{if(typeof m.deriveHistoryMetrics!=='function'||typeof m.assertNoLeak!=='function'||typeof m.HISTORY_METRICS_METHOD!=='string')process.exit(1)})"` exits 0. + - Running the CLI on the sample fixture writes JSON that contains neither `SAMPLE_PAGE_TEXT_7c1d` nor `SAMPLE_TASK_TEXT_a91f` nor `://`. Check with `node scripts/bench/export-history.mjs --input tests/fixtures/bench/history-export-sample.json --out /tmp/fsb66-history-probe.json && ! grep -q -e SAMPLE_PAGE_TEXT_7c1d -e SAMPLE_TASK_TEXT_a91f -e '://' /tmp/fsb66-history-probe.json`. + - `grep -c "action: 'getSessionDetail'" scripts/bench/history-export-snippet.js` prints at least 1, and `grep -c "limit: 500" scripts/bench/history-export-snippet.js` prints at least 1. + - `package.json` is unchanged (`git diff --quiet -- package.json`). + + Raw captures are ignored by git. The snippet exists. The derivation and leak check are implemented and pass their tests on synthetic data with fake page text and URLs. + + + + Task 2 (H1): Export the 2026-09-16..21 MCP sessions from the Web Store build + none (human action in Chrome Profile 2; the export downloads to ~/Downloads) + + - scripts/bench/history-export-snippet.js (the snippet the human pastes) + + Run the committed export snippet in the Web Store FSB control panel's DevTools console (Chrome Profile 2) to download the 40 recorded MCP sessions. This needs the user's live Chrome profile and the extension's privileged page. No CLI or API can reach that IndexedDB, and the MCP route is the bug this phase fixes. + + Claude already created `scripts/bench/history-export-snippet.js` and a gitignored destination (`scripts/bench/.local/`), and copied the snippet to your clipboard at the end of Task 1. If the clipboard has changed since, open `scripts/bench/history-export-snippet.js` in the repo and copy its contents. + 1. Do this before any other MCP use in Chrome Profile 2. Each new MCP session there can push an old one out of the 50-session history cap. + 2. In Profile 2 (the Web Store FSB build, id `badgafnfchcihdfnjneklogedcdkmjfk`), open `chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk/ui/control_panel.html`. + 3. Open DevTools (Cmd+Opt+I), then the Console tab. If Chrome asks, type `allow pasting` first. + 4. Paste the snippet and press Enter. Wait for the line `exported N sessions (M events)`. The browser downloads `fsb-history-2026-09-16_21.json`, normally into ~/Downloads. + 5. Reply with the N and M values from the console line, and the full path of the downloaded file if it is not in ~/Downloads. + + Claude then confirms the file exists, parses as JSON with `schema === 'fsb-history-export'`, and has `sessions.length === N`. + + - The downloaded file exists at the reported path and parses as JSON with `schema` equal to `fsb-history-export`. + - `sessions.length` equals the N the human reported. + + + node -e "const f=process.env.FSB66_HISTORY_EXPORT||require('os').homedir()+'/Downloads/fsb-history-2026-09-16_21.json'; const j=JSON.parse(require('fs').readFileSync(f,'utf8')); if(j.schema!=='fsb-history-export'||!Array.isArray(j.sessions))process.exit(1); console.log('sessions', j.sessions.length)" + + The human has downloaded the export. Claude has confirmed it parses, and that its session count equals the reported N. + Reply "exported N sessions (M events)" plus the file path if not ~/Downloads, or describe any console error. + + + + Task 3: Move the raw export into .local, derive sanitized metrics, and commit only the metrics + scripts/bench/results/history-2026-09-16_21/metrics.json + + - scripts/bench/export-history.mjs (CLI flags and leak check from Task 1) + - .gitignore (the scripts/bench/.local/ rule) + + +1. Move the downloaded export from Task 2 (default `~/Downloads/fsb-history-2026-09-16_21.json`) to `scripts/bench/.local/history-2026-09-16_21/raw.json`. Create the directory. Use `mv`, not a copy, so no raw copy stays in ~/Downloads. +2. Confirm that `git check-ignore -q scripts/bench/.local/history-2026-09-16_21/raw.json` exits 0 and that `git status --porcelain` does not list the raw file. +3. Run `node scripts/bench/export-history.mjs --input scripts/bench/.local/history-2026-09-16_21/raw.json --out scripts/bench/results/history-2026-09-16_21/metrics.json`. The CLI's built-in `assertNoLeak` must pass. If it throws, fix the derivation in export-history.mjs and extend the Task 1 test with a case reproducing the leak (update both files). Never hand-edit metrics.json to hide a leak. +4. Record in the plan SUMMARY: the session count (it must equal the N from Task 2; report the actual number, even if it is not 40), the call count, per-client median/p90 per-call deltas, and a comparison with PROJECT.md's 9.5 s (Claude) / 4.6 s (Cursor). State that the history metric is a journal-timestamp delta (it includes caller gap and IndexedDB write-queue delay), not FSB execution time. +5. Commit only `scripts/bench/results/history-2026-09-16_21/metrics.json`, plus any Task 1 file fixes, with a message like `docs(66-01): commit sanitized 2026-09-16..21 MCP history metrics`. Never add a Co-Authored-By line or any AI/Claude attribution. Never stage anything under scripts/bench/.local/. + + + node -e "const m=require('fs').readFileSync('scripts/bench/results/history-2026-09-16_21/metrics.json','utf8'); const j=JSON.parse(m); const r=JSON.parse(require('fs').readFileSync('scripts/bench/.local/history-2026-09-16_21/raw.json','utf8')); if(j.sessionCount!==r.sessions.length||m.includes('://'))process.exit(1)" && git check-ignore -q scripts/bench/.local/history-2026-09-16_21/raw.json + + + - `scripts/bench/results/history-2026-09-16_21/metrics.json` exists, and its `sessionCount` equals `sessions.length` in the raw export. + - metrics.json contains no `://` substring, and no value equal to any raw session id (checked by assertNoLeak during the CLI run, and by the verify command for URLs). + - `git ls-files scripts/bench/.local` prints nothing. + - `git log -1 --format=%B` contains no `Co-Authored-By` line. + + The 2026-09-16..21 sessions are preserved locally (gitignored). Sanitized per-session and per-client metrics are committed. The SUMMARY records the actual session count and per-client medians. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Web Store extension storage → local disk | The raw export holds page content, typed values and URLs from real sites. | +| Local raw export → git | Only derived metrics may cross into version control. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-02 | Information disclosure | scripts/bench/results/history-2026-09-16_21/metrics.json | mitigate | Raw file lives only in gitignored `scripts/bench/.local/`. `deriveHistoryMetrics` emits only an allowlisted set (ordinals, client labels, tool names that pass a regex, counts, deltas, date-only start). `assertNoLeak` fails the CLI when any session id, task string, inline payload string (8+ chars) or URL appears. The unit test injects each leak type. | +| T-66-01 | Information disclosure | history-export-snippet.js | accept | The snippet reads only the session index and session detail through the extension's own runtime message. It never reads API-key or bridge-auth storage keys, and its output goes to local disk only. | +| T-66-01b | Tampering | raw export handling | mitigate | The export is moved with `mv` (no stray copy in ~/Downloads), and the verify step confirms `git check-ignore` before any commit. | + + + +- `node --test tests/bench-export-history.test.mjs` passes. +- `git check-ignore` confirms scripts/bench/.local/ is ignored, and `git ls-files scripts/bench/.local` is empty. +- metrics.json session count matches the raw export, and it contains no URLs or ids. + + + +- D-20: the historical sessions are exported before any benchmark recording. +- D-21: raw data stays local; only sanitized metrics are committed. +- The SUMMARY records the actual per-client medians with an explicit method label. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages. This is the user's global rule. Stage only this plan's files, and never anything under `scripts/bench/.local/`. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-01-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-01-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-01-SUMMARY.md new file mode 100644 index 000000000..02b29c3d7 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-01-SUMMARY.md @@ -0,0 +1,102 @@ +--- +phase: 66-baseline-and-session-timing +plan: 01 +subsystem: benchmark +tags: [benchmark, history-export, d-20, d-21, closed-not-executed] + +# Dependency graph +requires: [] +provides: + - "scripts/bench/export-history.mjs, scripts/bench/history-export-snippet.js and tests/bench-export-history.test.mjs (Task 1 only; still in the root chain, 14 cases)" + - "The gitignore rule for scripts/bench/.local/ (D-21), which 66-10 and 66-11 rely on for raw captures" + - "No history metrics: scripts/bench/results/history-2026-09-16_21/metrics.json was never produced" +affects: [66-10, 66-11] + +# Tech tracking +tech-stack: + added: [] + patterns: [] + +key-files: + created: + - scripts/bench/export-history.mjs + - scripts/bench/history-export-snippet.js + - tests/bench-export-history.test.mjs + - tests/fixtures/bench/history-export-sample.json + modified: + - .gitignore + +key-decisions: + - "Closed 2026-10-12 without running H1 or Task 3. The 40 MCP sessions recorded 2026-09-16..21 were lost when the Web Store FSB was reinstalled on 2026-10-09, and no backup exists" + - "The Task 1 code stays. It is tested, harmless, and works for any future export of the same shape" + - "66-10 no longer depends on 66-01, and 66-11 no longer compares live call intervals with the history" + +patterns-established: [] + +requirements-completed: [] + +# Metrics +duration: 0min +completed: 2026-10-12 +--- + +# Phase 66 Plan 01: History Export (Closed, Not Executed) Summary + +**Closed 2026-10-12: the source data was lost in the 2026-10-09 reinstall, so the plan was not executed. Task 1 (the export snippet, the sanitizing derivation and its test) shipped on 2026-09-30. The human export H1 and the metrics commit (Task 3) never ran. The 40 MCP sessions they were meant to save no longer exist in any Chrome profile on this machine.** + +## What Was Done + +- **Task 1 (shipped):** + - `f406736b` test(66-01) added the failing test (RED). + - `b4fe9d41` feat(66-01) added the code (GREEN): + - the `scripts/bench/.local/` gitignore rule; + - `export-history.mjs` (`deriveHistoryMetrics`, `assertNoLeak`, `HISTORY_METRICS_METHOD`); + - the DevTools snippet; + - the test and its fixture. + - Plan 66-09 registered `tests/bench-export-history.test.mjs` in the root chain, where it passes 14 cases. +- **Task 2 (H1) and Task 3: not executed.** + +## Why the Export Is Impossible + +All checks were read-only, on 2026-10-09: + +- **Fresh install in both profiles.** Chrome Default and Profile 2 both show the Web Store build (`badgafnfchcihdfnjneklogedcdkmjfk`, now 0.9.92) with `first_install_time` of 2026-10-09 around 15:58 UTC. An update keeps `chrome.storage.local`; a reinstall wipes it. +- **Empty stores.** + - In both profiles, `fsbSessionIndex` is `[]` and `fsbSessionLogs` is `{}`. + - Each extension-settings LevelDB holds only `MANIFEST-000001` and `000003.log`, so the store was created fresh. + - The IndexedDB directories are about 20 KB each. +- **No copy anywhere.** + - No extension storage in any profile, including the unpacked build in Profile 5, contains the session mode string `mcp-agent`. + - `~/Downloads` holds no `fsb-history*` file. + - Time Machine has no destination configured. +- The only remaining trace of those sessions is the pair of medians quoted in `PROJECT.md` (9.5 s per call with Claude, 4.6 s with Cursor). They can no longer be re-derived. + +## Effect on the Phase + +- **Phase 66 still meets its success criteria and requirements.** `66-01-PLAN.md` lists `metrics.json` as a must-have, but none of the four ROADMAP success criteria for Phase 66 and none of MEAS-01..03 mention the history. The ROADMAP success criteria take precedence over a plan's must-haves. The MEAS-03 baseline comes from the 66-11 campaign. +- **66-10** dropped 66-01 from `depends_on`. +- **66-11** dropped the history-versus-live call-interval comparison (W6). It still reports the live `callIntervalMs` median and p90. +- **The bench report's call-interval label** no longer names the 66-01 history, so the committed baseline will not point at an artifact that does not exist. +- **D-20 in 66-CONTEXT.md** has a dated amendment: the export part is superseded, and the save-each-run rule still applies. + +## Deviations from Plan + +The plan was closed instead of executed. Its human checkpoint became impossible when the source data was deleted outside the project. + +## Issues Encountered + +- The plan sent the operator to Chrome Profile 2. By 2026-10-07, though, Profile 2's Web Store slot was an empty stub, and the published build ran in Default. So the instruction had already gone stale before the reinstall. +- The `list_sessions` safety-net snapshot suggested in 66-RESEARCH.md was never taken. + +## User Setup Required + +None. + +## Next Phase Readiness + +- 66-10 is next. It has no dependency left on this plan. +- **Blockers:** none. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-10-12 (closed, not executed)* diff --git a/.planning/phases/66-baseline-and-session-timing/66-02-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-02-PLAN.md new file mode 100644 index 000000000..2326bfb6c --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-02-PLAN.md @@ -0,0 +1,291 @@ +--- +phase: 66-baseline-and-session-timing +plan: 02 +type: execute +wave: 1 +depends_on: [] +files_modified: + - scripts/bench/sweep-tests.mjs + - .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json + - scripts/bench/lib/stats.mjs + - scripts/bench/lib/outcome.mjs + - scripts/bench/lib/identity.mjs + - tests/bench-stats.test.mjs + - tests/bench-outcome.test.mjs + - tests/bench-identity.test.mjs +autonomous: true +requirements: [MEAS-03] +user_setup: [] + +must_haves: + truths: + - "The worktree has every dependency CI installs, and the start-commit npm test state is recorded with port 7225 status before any extension file is edited. Pre-existing failures are recorded, not fixed (orchestrator decision 13)." + - "A regression sweep (scripts/bench/sweep-tests.mjs) runs every test that references a given source file with its package.json invocation, bounded per test without the `timeout` command. Its start-commit baseline over the nine extension files Phase 66 edits is committed, so plans 66-03, 66-04 and 66-06 report only new failures (W11)." + - "Statistics follow D-16: geometric mean of per-task median ratios with a hierarchical bootstrap CI (tasks, then runs within a task), seeded and deterministic. A CI crossing 1.0 is 'inconclusive'. Tasks with no completed run in an arm are dropped and listed." + - "An A/A noise floor (odd vs even rounds of one arm) is computable from the same bootstrap." + - "Task success is decided only from evidence independent of FSB and the agent: request log, final URL, DOM markers, and live-page text (D-17). Agent claims are never read." + - "Run identity is canonical and hashable (D-18). The preflight refuses a Web Store extension id, a missing Phase 66 measurement marker, a non-api provider kind, a non-xai provider, any autopilot model other than grok-4.3 (D-19), a missing --claude-model for Claude runs, a hub on 7225 that the runner's own server does not own, and a dirty tree for campaigns." + artifacts: + - path: "scripts/bench/sweep-tests.mjs" + provides: "Referencing-test regression sweep CLI (--files, --baseline, --out, --per-test-timeout-ms)" + - path: ".planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json" + provides: "Start-commit sweep results for the nine Phase 66 extension files" + - path: "scripts/bench/lib/stats.mjs" + provides: "median, quantile, mulberry32, wilson, summarize, geoMeanMedianRatio, aaNoiseFloor" + exports: ["median", "quantile", "mulberry32", "wilson", "summarize", "geoMeanMedianRatio", "aaNoiseFloor"] + - path: "scripts/bench/lib/outcome.mjs" + provides: "evaluateOutcome, normalizeFinalUrl, stripRunPrefix, CHECK_KINDS" + exports: ["evaluateOutcome", "normalizeFinalUrl", "stripRunPrefix", "CHECK_KINDS"] + - path: "scripts/bench/lib/identity.mjs" + provides: "canonicalJson, sha256Hex, toolsListDigest, gitIdentity, evaluatePreflight, servedModelMatches, constants" + exports: ["canonicalJson", "sha256Hex", "toolsListDigest", "gitIdentity", "evaluatePreflight", "servedModelMatches", "WEB_STORE_EXTENSION_ID", "PINNED_AUTOPILOT_MODEL"] + key_links: + - from: "scripts/bench/lib/outcome.mjs" + to: "fixture-server evidence (plan 66-05) and CDP evidence (plan 66-08)" + via: "Evidence contract {requests[], finalUrl, markers{}, text}" + pattern: "evaluateOutcome" + - from: "scripts/bench/lib/identity.mjs" + to: "fsb://config payload from _handleGetConfig (plan 66-03)" + via: "config.extension.measurement.callTiming === 1 and sessionDetailJournal === 1" + pattern: "measurement_marker_missing" +--- + + +Prepare the fresh worktree, record the start-commit test baseline, and build the three pure benchmark libraries that everything else depends on: statistics, outcome evaluation, and run identity plus preflight. + +Purpose: VALIDATION requires the environment and a recorded baseline before the first extension edit. That is why this is the first autonomous plan and runs in wave 1, parallel to the human-gated history export. The libraries encode D-16 (statistics), D-17 (outcome proof) and D-18/D-19 (identity and preflight) as tested, pure code. +Output: installed deps (untracked), a recorded baseline in the SUMMARY, the committed regression sweep `scripts/bench/sweep-tests.mjs` with its start-commit baseline (W11), and `scripts/bench/lib/{stats,outcome,identity}.mjs` with Node 20-safe tests. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/ROADMAP.md +@.planning/REQUIREMENTS.md +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-VALIDATION.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +Shared contracts other plans implement against. They are stated verbatim here, and this plan owns their implementation. + +Evidence (produced by scripts/bench/fixture-server.mjs in plan 66-05 and by CDP reads in plan 66-08): +`{ requests: [{ t:, method:'GET'|'POST'|..., path: prefix already removed>, query:{:}, bodyType:'json'|'form'|'text'|'none', body: }], finalUrl:, markers:{:}, text: }` + +Task success checks (task JSON `success.all[]`, validated by scripts/bench/lib/tasks.mjs in plan 66-05): +- `{kind:'request', method, path, count:= 0>, jsonEquals?:}`. It passes when the number of evidence requests with an equal method (case-insensitive) and path (exact), and, if jsonEquals is present, a canonically equal body, is exactly `count`. `count: 0` asserts absence. +- `{kind:'finalUrl', pathEquals?:, queryIncludes?:{:}, urlIncludes?:}`. Core tasks use pathEquals/queryIncludes on the normalized final URL. Live tasks use urlIncludes on the absolute final URL. +- `{kind:'marker', name, equals:}`. Passes when `evidence.markers[name] === equals`. +- `{kind:'text', includes:}` (live only). Passes when `evidence.text` includes the string (case-sensitive). + +fsb://config payload (implemented by plan 66-03 in extension/ws/mcp-bridge-client.js `_handleGetConfig`). The MCP resource wraps it as `{success:true, config:{providerKind, modelProvider, modelName, maxIterations?, domOptimization?, maxDOMElements?}, extension:{id, version, measurement:{callTiming:1, sessionDetailJournal:1}}}`. + +Code to reuse as patterns: `stableJson` in extension/utils/mcp-lattice-journal.js lines 110-130 (canonical JSON); `captureGitBuffer` in scripts/run-phase60-full-tests.mjs lines 51-62 (git via spawnSync, GIT_OPTIONAL_LOCKS=0, shell:false); module style from scripts/lib/t1-port-contract.mjs lines 1-60; statistics code from 66-RESEARCH.md "Pattern 9" (use verbatim). + + + + + + + Task 1: Install the worktree's dependencies, record the start-commit npm test baseline, and add the regression sweep with its baseline + scripts/bench/sweep-tests.mjs, .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json (node_modules trees, mcp/build and extension/dist are gitignored build outputs) + + - package.json (scripts.pretest, scripts.test, scripts.showcase:install, scripts.build) + - .github/workflows/ci.yml (the install set CI uses: root, mcp ci, showcase/server, showcase/angular) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Common Pitfalls" 13-14 and "Project Constraints" (7225 chain halt, Node 20) + + +1. Record the start commit: `git rev-parse HEAD`, `node --version` and `npm --version`. +2. Install and build, in order, from the repo root. Stop and report on the first non-zero exit. + 1. `npm install --no-audit --no-fund` + 2. `npm --prefix mcp ci --no-audit --no-fund` + 3. `npm --prefix mcp run build` + 4. `npm --prefix showcase/server install --no-audit --no-fund` + 5. `npm run showcase:install` (showcase/angular; CI installs it, and pretest's dashboard-localization-fallback plus the fsb-telemetry-service and stats-view-state suites need it) + 6. `npm run build` (esbuild writes the gitignored extension/dist/offscreen/lattice-host.js) +3. Check the hub ports with `lsof -nP -iTCP:7225 -sTCP:LISTEN` and `lsof -nP -iTCP:7226 -sTCP:LISTEN`, and record the holders (PID and command via `ps -p -o pid,command`). Do NOT kill user processes (for example Claude Desktop's FSB server or the installed native host). +4. Run the baseline: + - If 7225 is free: run `npm test` in the background with stdout and stderr redirected to `/tmp/fsb-66-02-baseline.log`, and write the exit code to `/tmp/fsb-66-02-baseline.exit`. Poll until the exit file exists. The full chain takes several minutes, beyond one Bash call's timeout. Judge the run by the exit code and the last suite name in the log, not by counting FAIL lines. + - If 7225 is held: run a tolerant pass instead. Take `scripts.pretest` then `scripts.test`, split each on the exact separator ` && `, and run every command through `/bin/sh -c` from the repo root with stdin from `/dev/null`, continuing past failures. Write one JSON line per command (`{command, exitCode, durationMs}`) to `/tmp/fsb-66-02-baseline.jsonl`. Expect only port-bound suites (for example `tests/mcp-install-platforms.test.js`, `tests/mcp-diagnostics-status.test.js`) to fail from the held port. +5. Revert tracked files that installs, builds or the chain rewrote, and never commit them. Run `git status --porcelain --untracked-files=no`. For any modified tracked path under `showcase/angular/public/` (llms.txt, llms-full.txt, sitemap.xml are known to be rewritten), under `mcp/build/`, `mcp/ai/tool-definitions.cjs`, or any `package-lock.json`, restore it with `git checkout -- `. Leave `.gitignore` alone: plan 66-01 edits it in parallel. +6. Create `scripts/bench/sweep-tests.mjs`, the regression sweep that plans 66-03, 66-04 and 66-06 run after each extension edit (W11). It is ESM with Node built-ins only, has a direct-run guard, and never modifies files. + - CLI: `node scripts/bench/sweep-tests.mjs --files [ ...] [--baseline ] [--out ] [--per-test-timeout-ms 180000]`, plus `--help`. + - Selection: every `tests/*.test.js` and `tests/*.test.mjs` file whose text contains the basename without extension of any `--files` path (for example `mcp-bridge-client`). + - Invocation: find the test path in package.json scripts (all scripts, split on ` && `). Use that command's flags for a single file (for example `node --test ` or `node --import tsx `). When the file is in no script, use `node `. + - Execution: `child_process.spawn` with `shell:false`, stdin ignored, and output to a per-test log under `os.tmpdir()`. The per-test bound is implemented in the script: SIGTERM at the limit, then SIGKILL after 5 s. Never use the `timeout` command; it is not installed on this macOS machine. + - Output: JSON `{startedAt, head, port7225Held, files, results:[{test, command, exitCode, signal, timedOut, durationMs}], failures:[...]}` to `--out` when given, and a one-line PASS/FAIL list on stdout. + - Exit code: with `--baseline `, print the NEW failures (in `failures` but not in the baseline's) and exit 1 only if there are any. Without a baseline, exit 1 on any failure. +7. Run the sweep baseline on the start commit over the nine extension files Phase 66 edits: `extension/ws/mcp-bridge-client.js extension/ws/mcp-tool-dispatcher.js extension/utils/mcp-session-recorder.js extension/utils/mcp-lattice-journal.js extension/utils/spreadsheet-record-redaction.js extension/ai/cost-tracker.js extension/ai/agent-loop.js extension/ai/action-history.js extension/utils/automation-logger.js`. Use `--out .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. Revert tracked files the tests rewrote (as in step 5). +8. Record in this plan's SUMMARY: + - the start commit SHA, Node and npm versions, and the install results; + - the 7225/7226 holders and which baseline mode ran; + - the chain exit code, the last suite reached, and the exact names of any failing suites; + - the sweep baseline's failure list and whether 7225 was held during it. + The failing-suite list becomes the "pre-existing failures" set that later plans compare against. Also note the known out-of-chain failures from operator memory (mcp-lifecycle-smoke 0/5 and six unscripted tests), marked "not in the chain". + +Commit `scripts/bench/sweep-tests.mjs` and `66-02-SWEEP-BASELINE.json` together: `chore(66-02): add referencing-test regression sweep and start-commit baseline`. No Co-Authored-By line or any AI/Claude attribution. + + + test -d node_modules && test -d mcp/node_modules/@modelcontextprotocol/sdk && test -d showcase/server/node_modules && test -d showcase/angular/node_modules && test -f mcp/build/index.js && test -f extension/dist/offscreen/lattice-host.js && (test -f /tmp/fsb-66-02-baseline.exit || test -f /tmp/fsb-66-02-baseline.jsonl) && node scripts/bench/sweep-tests.mjs --help && node -e "const b=require('./.planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json');if(!Array.isArray(b.results)||b.results.length<10||!Array.isArray(b.failures))process.exit(1)" && test -z "$(git status --porcelain --untracked-files=no -- mcp showcase extension package.json package-lock.json)" + + + - `node -e "console.log(require('./mcp/node_modules/@modelcontextprotocol/sdk/package.json').version)"` prints `1.29.0`. + - A baseline artifact exists: `/tmp/fsb-66-02-baseline.exit` (full chain) or `/tmp/fsb-66-02-baseline.jsonl` (tolerant). + - `git status --porcelain --untracked-files=no -- mcp showcase extension package.json package-lock.json` prints nothing. + - The SUMMARY lists the start SHA, the baseline mode, the exit code, the last suite, and the failing suite names (or "none"). + - The sweep baseline JSON exists with at least 10 results and a `failures` array, and covers the nine files listed in step 7. + - `grep -c "spawn" scripts/bench/sweep-tests.mjs` prints at least 1, and the script never invokes a `timeout` binary: `node -e "const s=require('fs').readFileSync('scripts/bench/sweep-tests.mjs','utf8');if(/['\"]timeout['\"]\s*,|spawn\(\s*['\"]timeout/.test(s))process.exit(1)"` exits 0. + + The environment is installed. The start-commit test state and the sweep baseline are recorded with their port context. Only the sweep tool and its baseline are committed; no other tracked file is modified. + + + + Task 2: Statistics library with hierarchical bootstrap and A/A noise floor + scripts/bench/lib/stats.mjs, tests/bench-stats.test.mjs + + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 9: Hierarchical bootstrap" (code to use verbatim) + - scripts/lib/t1-port-contract.mjs lines 1-60 (module header and frozen-export style) + - tests/dashboard-localization-fallback.test.mjs lines 1-6 and tests/mcp-lattice-journal.test.js lines 1-7 (ESM and node:test style) + + + - median([3,1,2]) === 2; median([1,2,3,4]) === 2.5; Number.isNaN(median([])). + - quantile([1,2,3,4,5,6,7,8,9,10], 0.9) is 9.1 within 1e-9; quantile([], 0.5) is NaN. + - summarize([5,1,3]) returns {n:3, total:9, median:3, p90, min:1, max:5}; summarize([]) returns {n:0, total:null, median:null, p90:null, min:null, max:null} (null, not 0). + - mulberry32(66) yields the same first 5 values on two calls; all values are in [0,1). + - wilson(8,10) ≈ [0.4902, 0.9433] (±1e-4); wilson(0,0) returns [NaN, NaN]. + - geoMeanMedianRatio on 12 synthetic tasks x 10 runs where arm b is 0.7x arm a (±3% seeded noise): estimate within 0.05 of 0.7, verdict 'faster', ci[1] < 1. + - geoMeanMedianRatio on identical-distribution arms (A/A): ci[0] < 1 < ci[1] and verdict 'inconclusive'. + - A task whose b array is empty is excluded from the estimate and listed in droppedTasks; usedTasks lists the rest. + - The same seed gives deep-equal results; the default iterations is 10000 and the default seed is 66. + - aaNoiseFloor([{id, runs:[{round, wallMs}]}...]) splits odd and even rounds, returns the geoMeanMedianRatio result plus label 'A/A odd-vs-even rounds', and drops tasks that lack either half. + + +Create `scripts/bench/lib/stats.mjs` (ESM, pure, Node 20-safe, no imports beyond none needed). Use the 66-RESEARCH.md Pattern 9 code verbatim for `mulberry32`, `median`, `quantile` (linear interpolation), the internal `resample`, `geoMeanMedianRatio(tasks, {iterations=10000, seed=66, level=0.95})` and `wilson(successes, n, z=1.959963984540054)`. Keep the return shape `{estimate, ci:[lo,hi], verdict:'faster'|'slower'|'inconclusive', usedTasks, droppedTasks, iterations, seed}`, where verdict is 'slower' if lo > 1, 'faster' if hi < 1, and 'inconclusive' otherwise (a CI crossing 1.0 is inconclusive, per D-16). + +Add: +- `summarize(xs)`: the n/total/median/p90/min/max object. Use null for every statistic when xs is empty. +- `aaNoiseFloor(tasks, opts)`: tasks are `[{id, runs:[{round:, wallMs:}]}]`. It builds `a` from odd rounds and `b` from even rounds for each task, calls geoMeanMedianRatio with the same opts, and returns its result with `label: 'A/A odd-vs-even rounds'`. + +Put a header comment stating: Phase 66 benchmark statistics (D-16); pure and seeded; per-task CIs are descriptive only; five runs per task is a smoke test, not a baseline. + +Create `tests/bench-stats.test.mjs` (node:test, node:assert/strict, relative import of `../scripts/bench/lib/stats.mjs`). Cover every behavior listed above. Generate the synthetic arrays with mulberry32 so the test is deterministic. No Node 22+ globals. Do NOT register the test in package.json (plan 66-09 does that). + +Commit: `feat(66-02): add seeded benchmark statistics with hierarchical bootstrap`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-stats.test.mjs + + + - `node --test tests/bench-stats.test.mjs` exits 0 with at least 9 passing tests. + - `node -e "import('./scripts/bench/lib/stats.mjs').then(m=>{for(const k of ['median','quantile','mulberry32','wilson','summarize','geoMeanMedianRatio','aaNoiseFloor'])if(typeof m[k]!=='function')process.exit(1)})"` exits 0. + - The stats module imports nothing outside Node built-ins: `grep -c "^import" scripts/bench/lib/stats.mjs` prints 0. + + Deterministic statistics exist and are tested against known answers, a known 0.7x effect, and an A/A null. + + + + Task 3: Outcome evaluator (claims never read) and run identity plus preflight + scripts/bench/lib/outcome.mjs, scripts/bench/lib/identity.mjs, tests/bench-outcome.test.mjs, tests/bench-identity.test.mjs + + - extension/utils/mcp-lattice-journal.js lines 110-130 (stableJson: sorted keys, drops undefined/functions, throws on cycles) + - scripts/run-phase60-full-tests.mjs lines 51-62 (captureGitBuffer pattern) + - extension/ws/mcp-bridge-client.js lines 2702-2708 (current `_handleGetConfig`, which plan 66-03 replaces with the payload in ) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 6" and "Pattern 7" (identity, evidence, task schema) and "Plan Decomposition Hints" (preflight refusal list) + + + - outcome: every check kind passes on matching evidence and fails on non-matching evidence. For request checks, method matching is case-insensitive, path matching is exact, and count is exact (count:0 passes only when no request matches). jsonEquals compares canonical JSON, so key order does not matter and array order does. + - outcome: evidence carrying `claimed:{status:'completed', complete_task:true}` with failing checks yields pass:false. The evaluator never reads `claimed`: the test passes a Proxy that throws on access to `claimed`. + - outcome: a text check's detail is `{matched:boolean}` only. The test asserts JSON.stringify(result) does not contain the page text. + - outcome: an unknown check kind makes that check fail with detail `{error:'unknown check kind'}`; a task with zero checks fails. + - outcome: normalizeFinalUrl('http://127.0.0.1:5000/r/tok12345/fixtures/a/index.html?view=done', 'tok12345') returns {path:'/fixtures/a/index.html', query:{view:'done'}}, and stripRunPrefix('/r/tok12345/api/x', 'tok12345') returns '/api/x'. + - identity: canonicalJson({b:1,a:{d:2,c:3}}) === canonicalJson({a:{c:3,d:2},b:1}); it throws on a cycle; it drops undefined values. + - identity: sha256Hex('abc') === 'ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad'. + - identity: toolsListDigest is unchanged by reordering tools or object keys, and changes when any description changes. count equals tools.length. + - identity: gitIdentity(repoRoot) returns a 40-hex sha and a boolean dirty flag. + - identity: evaluatePreflight with an all-good input returns {ok:true, refusals:[]}. Each refusal code fires alone when only its condition is broken: config_unavailable, measurement_marker_missing, webstore_extension_loaded, extension_id_mismatch, provider_kind_not_api, provider_not_xai, model_not_pinned, claude_model_required, hub_not_owned, dirty_tree, tool_list_empty. + - identity: servedModelMatches('grok-4.3') and servedModelMatches('grok-4.3-latest') are true; 'grok-4-1-fast-reasoning' is false. + + +Create `scripts/bench/lib/outcome.mjs` (ESM, pure, Node 20-safe): +- `CHECK_KINDS`: frozen `['request','finalUrl','marker','text']`. +- `stripRunPrefix(pathname, token)`: removes a leading `/r/`. +- `normalizeFinalUrl(url, token)`: returns `{path, query}` for core runs (parse with `new URL`, strip the prefix, and turn searchParams into a plain object). Return null for a null or unparseable URL. +- `evaluateOutcome(task, evidence)`: returns `{pass, checks:[{kind, pass, detail}]}`. It implements the check semantics in exactly and uses a local canonical-JSON comparison for jsonEquals. It reads only `task.success.all`, `task.set`, `evidence.requests`, `evidence.finalUrl`, `evidence.markers`, `evidence.text`, and the run token when passed as a third argument `{token}`. + - It never reads `evidence.claimed` or any other agent-claim field (D-17). + - Details hold counts and booleans, never page text or request bodies. + +Create `scripts/bench/lib/identity.mjs` (ESM): +- Constants: `WEB_STORE_EXTENSION_ID = 'badgafnfchcihdfnjneklogedcdkmjfk'`, `PINNED_AUTOPILOT_MODEL = 'grok-4.3'` (D-19), and a regex `SERVED_MODEL_PATTERN = /^grok-4\.3/`. +- `canonicalJson(value)`: follows the stableJson semantics. +- `sha256Hex(input)`: uses node:crypto createHash('sha256'). +- `toolsListDigest(tools)`: sorts by name, projects `{name, description, inputSchema}`, then hashes the canonical JSON. Returns `{count, sha256}`. +- `gitIdentity(repoRoot)`: runs spawnSync git with `GIT_OPTIONAL_LOCKS=0`, `shell:false`, cwd repoRoot. Uses `rev-parse HEAD` and `status --porcelain --untracked-files=no`, and returns `{sha, dirty, dirtyPaths}`. +- `servedModelMatches(model)`. +- `evaluatePreflight({mode, arms, configPayload, hub:{listenerPid, serverPid}, claudeModel, expectExtensionId, git:{dirty}, allowDirty, toolsList:{count}})`: returns `{ok, refusals:[{code, detail}]}` with the eleven codes listed in . + - Provider and model checks apply only when `arms` includes `autopilot`. + - claude_model_required applies only when `arms` includes `claude` and mode is `smoke` or `campaign`. + - dirty_tree applies only to mode `campaign` without allowDirty. + - hub_not_owned fires when either pid is missing or they differ. + - Refusal details may name model and provider strings, but never echo any other config value. + +Create `tests/bench-outcome.test.mjs` and `tests/bench-identity.test.mjs` (node:test, node:assert/strict, relative imports). Cover every behavior above. Build the gitIdentity case on the real repo root: `path.resolve(fileURLToPath(new URL('..', import.meta.url)))`. No Node 22+ globals. Do NOT register the tests in package.json (plan 66-09 does that). + +Commit: `feat(66-02): add claim-blind outcome evaluator and run identity preflight`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-outcome.test.mjs tests/bench-identity.test.mjs + + + - `node --test tests/bench-outcome.test.mjs tests/bench-identity.test.mjs` exits 0. + - The claim-blindness test exists: `grep -c "new Proxy" tests/bench-outcome.test.mjs` prints at least 1, and that case passes. It fails if `evaluateOutcome` touches `claimed`. + - `node -e "import('./scripts/bench/lib/identity.mjs').then(m=>{if(m.WEB_STORE_EXTENSION_ID!=='badgafnfchcihdfnjneklogedcdkmjfk'||m.PINNED_AUTOPILOT_MODEL!=='grok-4.3')process.exit(1)})"` exits 0. + - Each of the eleven refusal codes appears in tests/bench-identity.test.mjs: `for c in config_unavailable measurement_marker_missing webstore_extension_loaded extension_id_mismatch provider_kind_not_api provider_not_xai model_not_pinned claude_model_required hub_not_owned dirty_tree tool_list_empty; do grep -q "$c" tests/bench-identity.test.mjs || exit 1; done` exits 0. + + Outcome evaluation is claim-blind and tested. Identity hashing and the preflight refusal matrix are tested. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Agent under test → outcome decision | Agent claims (run_task success, complete_task, Claude result) must not decide success. | +| fsb://config → benchmark identity | The extension config payload feeds committed identity records. | +| npm registry → worktree | Installs use the committed lockfiles only. No new packages. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-04 | Spoofing / Tampering | scripts/bench/lib/outcome.mjs | mitigate | `evaluateOutcome` reads only request-log, finalUrl, marker and text evidence. A Proxy test proves `claimed` is never accessed, so an agent's self-report cannot pass a run. | +| T-66-01 | Information disclosure | scripts/bench/lib/identity.mjs | mitigate | Identity records only ids, versions, model/provider names and SHA-256 digests. Preflight details never echo config values other than model and provider names. The config payload is already allowlisted by plan 66-03. | +| T-66-01b | Elevation of privilege | preflight | mitigate | Preflight refuses the Web Store extension id, a missing Phase 66 marker, and a 7225 hub the runner does not own. Benchmark runs therefore never target the user's everyday extension or a stale squatter hub. | +| T-66-SC | Tampering | npm installs in Task 1 | accept | No new packages are added. `npm --prefix mcp ci` installs exactly the committed lock (SDK 1.29.0), and the other installs match CI's commands. Lockfile rewrites are reverted and never committed. | + + + +- `node --test tests/bench-stats.test.mjs tests/bench-outcome.test.mjs tests/bench-identity.test.mjs` passes. +- The SUMMARY records the baseline chain result on the start commit, and `66-02-SWEEP-BASELINE.json` is committed. +- No tracked file outside this plan's files_modified is modified. + + + +- The environment is ready, and the baseline test state is recorded before any extension edit (VALIDATION Wave 0). +- The D-16 statistics, D-17 claim-blind outcome, and D-18/D-19 identity plus preflight are implemented and tested. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Stage only this plan's files. Never commit lockfile, `showcase/angular/public/*`, or `mcp/build/*` rewrites produced by installs or test runs. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md new file mode 100644 index 000000000..33ec41e99 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md @@ -0,0 +1,290 @@ +--- +phase: 66-baseline-and-session-timing +plan: 02 +subsystem: testing +tags: [benchmark, bootstrap, statistics, outcome-evaluation, preflight, regression-sweep, node-test] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 01) + provides: scripts/bench/ directory and the .gitignore entry for scripts/bench/.local/ (already committed before this plan ran) +provides: + - "Installed worktree (root, mcp ci, mcp build, showcase/server, showcase/angular, esbuild) with lockfile rewrites reverted" + - "Start-commit test baseline (tolerant mode, 7225 held): 311/311 chain commands exit 0" + - "scripts/bench/sweep-tests.mjs referencing-test regression sweep (W11) with --baseline new-failure gating" + - "66-02-SWEEP-BASELINE.json: 73/73 referencing tests pass over the nine Phase 66 extension files at b4fe9d41" + - "scripts/bench/lib/stats.mjs: median, quantile, mulberry32, wilson, summarize, geoMeanMedianRatio, aaNoiseFloor" + - "scripts/bench/lib/outcome.mjs: claim-blind evaluateOutcome, normalizeFinalUrl, stripRunPrefix, CHECK_KINDS" + - "scripts/bench/lib/identity.mjs: canonicalJson, sha256Hex, toolsListDigest, gitIdentity, servedModelMatches, evaluatePreflight (11 refusal codes)" +affects: [66-03, 66-04, 66-05, 66-06, 66-07, 66-08, 66-09, 66-10, 66-11, benchmark runner, fixture server, live baseline campaign] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Regression sweep: run every test that mentions an edited file's basename with its package.json invocation; compare failures to a committed baseline" + - "Per-process bound in-process (SIGTERM to the process group, SIGKILL 5 s later); never the timeout binary" + - "Claim-blind outcome evaluation: read four evidence fields by name, never enumerate the evidence object" + - "Preflight refusal matrix: one code per broken condition; config-derived checks are skipped when config is unavailable" + +key-files: + created: + - scripts/bench/sweep-tests.mjs + - .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json + - scripts/bench/lib/stats.mjs + - scripts/bench/lib/outcome.mjs + - scripts/bench/lib/identity.mjs + - tests/bench-stats.test.mjs + - tests/bench-outcome.test.mjs + - tests/bench-identity.test.mjs + modified: [] + +key-decisions: + - "Baseline ran in tolerant mode because 7225 is held by the user's hub (pid 42278, npm exec fsb-mcp-server@latest); the port was never freed" + - "Sweep failures are keyed by test path; with --baseline only failures absent from the baseline fail the run" + - "The sweep records 7225 state with a loopback connect probe (no bind), so it cannot race a hub that is starting" + - "evaluateOutcome fails closed when the request log is absent (not an array), so count:0 absence checks cannot pass vacuously" + - "evaluatePreflight skips config-derived checks when fsb://config is unavailable, and treats unknown git state as dirty for campaigns" + - "TDD tasks were committed as RED test + GREEN feat pairs (matching plan 66-01); the GREEN commits carry the plan's exact messages" + +patterns-established: + - "Later plans (66-03, 66-04, 66-06) run: node scripts/bench/sweep-tests.mjs --files --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json" + - "Bench libraries are pure ESM with node:test suites that avoid Node 22+ APIs; they are not registered in package.json until plan 66-09" + +requirements-completed: [MEAS-03] + +# Metrics +duration: 16min +completed: 2026-09-30 +--- + +# Phase 66 Plan 02: Environment, Baseline and Bench Libraries Summary + +**Worktree installed with a green start-commit baseline (311/311 chain commands, 7225 held), a referencing-test regression sweep with a committed 73/73 baseline, and tested D-16 statistics (seeded hierarchical bootstrap plus an A/A floor), a D-17 claim-blind outcome evaluator, and D-18/D-19 identity and preflight.** + +This plan delivers MEAS-03's foundation slice only. The runner, tasks, fixtures, and live baseline campaign land in later Phase 66 plans. + +## Performance + +- **Duration:** 16 min +- **Started:** 2026-09-30T06:23:22Z +- **Completed:** 2026-09-30T06:39:22Z +- **Tasks:** 3/3 +- **Files created:** 8 (no existing tracked file modified) + +## Start-Commit Baseline (the record later plans compare against) + +### Environment + +| Item | Value | +|------|-------| +| Start commit | `b4fe9d415b62cd1c6f7c0a22a7838eab068b2efb` (branch `Jev`) | +| Node / npm | v24.14.1 / 11.19.0 | +| MCP SDK | `@modelcontextprotocol/sdk` 1.29.0 (from `npm --prefix mcp ci`) | + +### Install and build results (in plan order, all exit 0) + +| Step | Result | +|------|--------| +| `npm install --no-audit --no-fund` | exit 0, added 44 packages (1 s) | +| `npm --prefix mcp ci --no-audit --no-fund` | exit 0, added 105 packages (1 s) | +| `npm --prefix mcp run build` | exit 0 (2 s); `verify-agent-provider-flags` and `verify-native-host-boundary` (source and compiled) PASS | +| `npm --prefix showcase/server install --no-audit --no-fund` | exit 0, added 110 packages (15 s); better-sqlite3 loads under Node 24 | +| `npm run showcase:install` | exit 0, added 684 packages (4 s); npm audit reports 38 advisories in showcase/angular's committed lock (pre-existing, not acted on) | +| `npm run build` (esbuild) | exit 0; `extension/dist/offscreen/lattice-host.js` written (gitignored) | + +- npm 11.19 printed advisory `allowScripts` warnings for install scripts (esbuild, fsevents, better-sqlite3, lmdb, and others). The build and better-sqlite3 both work regardless. +- **Tracked files rewritten and reverted (never committed):** + - The root `npm install` rewrote `package-lock.json` (root `engines.chrome` `>=88.0.0` → `>=116.0.0`). + - `showcase:install` rewrote `showcase/angular/package-lock.json` (it dropped the optional `@types/node` 26.4.0 and `undici-types` entries under `@angular/build`). + - The test pass rewrote `showcase/angular/public/{llms.txt,llms-full.txt,sitemap.xml}`. + - The three `npm --prefix mcp run build` runs (one explicit, two inside the chain) left `mcp/build/*` and `mcp/ai/tool-definitions.cjs` byte-identical. + - `.gitignore` was not touched. + +### Hub ports and baseline mode + +| Port | Holder | +|------|--------| +| 7225 | **held**: pid 42278 `node /Users/lakshman/.npm/_npx/d92c25bff7522293/node_modules/.bin/fsb-mcp-server` (parent 42179 `npm exec fsb-mcp-server@latest`, started 2026-09-29 15:24:36). This is the user's hub; it was not killed or restarted. | +| 7226 | free | + +**Mode: tolerant pass** (7225 held). `scripts.pretest`, then `scripts.test`, were split on ` && `. Each of the 311 commands ran through `/bin/sh -c` from the repo root, with stdin from `/dev/null`, continuing past failures. +- Runner: a throwaway `/tmp/fsb-66-02-tolerant-runner.mjs`, not committed. +- Artifacts: `/tmp/fsb-66-02-baseline.jsonl` (one `{command, exitCode, durationMs}` row per command, plus id, phase, signal, timedOut and startedAt), `/tmp/fsb-66-02-baseline.log` and `/tmp/fsb-66-02-baseline.done`. + +### Chain result + +- **Exit code:** in tolerant mode there is no single chain exit code. All 311 commands exited 0 (13/13 pretest, 298/298 test), so the equivalent `npm test` result is 0. Total time was 148.0 s, and no command reached the runner's 20-minute per-command bound. +- **Last suite reached:** T298 `node --import tsx tests/no-orphan-descriptor.test.js` (the end of the chain). +- **Failing suites:** **none.** +- **Port-bound suites passed with 7225 held**, contrary to the plan's expectation: + - T44 `tests/mcp-diagnostics-status.test.js`: 254 passed, 0 failed. + - T45 `tests/mcp-install-platforms.test.js`: 108 passed, 0 failed. +- The slowest commands were T103 `showcase-build-smoke` (22.1 s), T45 `mcp-install-platforms` (20.5 s) and T8 `phase60-full-tests-harness` (17.1 s). +- No stray `mcp/build/index.js` relay from this workspace was left running. 7225 was still held by the same pid 42278 afterwards. + +### Pre-existing failures set (for later plans) + +- **In the `npm test` chain:** none. +- **In the sweep baseline:** none (see below). +- **Known out-of-chain failures from operator memory, marked "not in the chain":** these were not re-run here. + - `tests/mcp-lifecycle-smoke.test.js` (`npm run test:mcp-smoke`), 0/5 on origin/main. + - Six unscripted tests: `agent-manager-start-mode`, `back-tool`, `dashboard-analytics-refresh`, `foreground-audit`, `site-search` and `trigger-cap-settings-ui` (`.test.js`). + - One of those six, `tests/foreground-audit.test.js`, is in this plan's sweep set and **passed** at `b4fe9d41` on `Jev`. + +### Sweep baseline (`66-02-SWEEP-BASELINE.json`) + +- **Command:** `node scripts/bench/sweep-tests.mjs --files --out .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` +- **Files covered:** + - `extension/ws/mcp-bridge-client.js` and `extension/ws/mcp-tool-dispatcher.js` + - `extension/utils/mcp-session-recorder.js`, `mcp-lattice-journal.js`, `spreadsheet-record-redaction.js` and `automation-logger.js` + - `extension/ai/cost-tracker.js`, `agent-loop.js` and `action-history.js` +- **Result:** + - 73 referencing tests ran, and all 73 passed. **`failures: []`**. + - **7225 was held during the sweep** (`port7225Held: true`), with `head` = `b4fe9d41`. + - The per-test limit was 180000 ms. The whole sweep took about 9.0 s. +- **Invocations:** 69 tests ran as `node ` and 4 as `node --test `. Of the 73, 12 are in no package.json script and ran as plain `node `. +- **After the sweep:** no tracked file was rewritten. + +## Accomplishments + +- The environment matches CI's install set, and the start commit's test state is recorded before any extension edit. The whole chain is green even with the hub on 7225. +- **`scripts/bench/sweep-tests.mjs`:** + - selects every `tests/*.test.{js,mjs}` file that mentions an edited file's basename, and resolves that file's own package.json invocation (it handles `--test`, `--import tsx` and trailing script arguments); + - bounds each test in-process and writes per-test logs under the OS temp dir; + - with `--baseline`, fails only on new failures. + + Verified cases: help, usage errors (exit 2), a 30 ms bound killing a test by SIGTERM, a new failure against a clean baseline (exit 1), a known failure (exit 0), and "no longer failing" reporting. +- **D-16 statistics:** + - Seeded (66) hierarchical bootstrap with 10000 iterations by default. + - A known 0.7x effect gives estimate 0.7047 with CI [0.6995, 0.7098], verdict "faster". + - An A/A comparison gives CI [0.954, 1.064], verdict "inconclusive". It was inconclusive for 40 of 40 synthetic data seeds, so the null test is not seed-lucky. +- **D-17 claim-blind outcome evaluator:** a Proxy that throws on any read, `in` probe, descriptor lookup or enumeration of `claimed` proves the evaluator never touches it. Mutation runs on temporary copies confirmed the test catches a direct read, an `in` probe, an object spread, and a claim-trusting shortcut. +- **D-18/D-19 identity and preflight:** + - Canonical JSON, the SHA-256 "abc" vector, an order-independent tools/list digest, git identity, and grok-4.3 served-model matching. + - Each of the eleven refusal codes is shown to fire alone. + - Refusal details never echo config values other than provider and model names. + +## Task Commits + +1. **Task 1: Install, record baseline, add regression sweep and its baseline:** `ee447d4d` (chore) +2. **Task 2: Statistics library:** + - `0fac17e6` (test, RED) + - `cb98a67c` (feat, GREEN) +3. **Task 3: Outcome evaluator and run identity plus preflight:** + - `3d3c598c` (test, RED) + - `b19e65e8` (feat, GREEN) + +**Plan metadata:** see the `docs(66-02)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `scripts/bench/sweep-tests.mjs`: referencing-test regression sweep CLI (`--files`, `--baseline`, `--out`, `--per-test-timeout-ms`, `--help`). It uses Node built-ins only, has a direct-run guard, and never modifies files. +- `.planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`: start-commit sweep results for the nine Phase 66 extension files. +- `scripts/bench/lib/stats.mjs`: `mulberry32`, `median`, `quantile`, `geoMeanMedianRatio` and `wilson` from Pattern 9 verbatim, plus `summarize`, `aaNoiseFloor` and `AA_NOISE_FLOOR_LABEL`. It has no imports. +- `scripts/bench/lib/outcome.mjs`: `CHECK_KINDS`, `stripRunPrefix`, `normalizeFinalUrl` and `evaluateOutcome`. +- `scripts/bench/lib/identity.mjs`: + - constants `WEB_STORE_EXTENSION_ID`, `PINNED_AUTOPILOT_MODEL`, `SERVED_MODEL_PATTERN` and `PREFLIGHT_REFUSAL_CODES`; + - functions `canonicalJson`, `sha256Hex`, `toolsListDigest`, `gitIdentity`, `servedModelMatches` and `evaluatePreflight`. +- `tests/bench-stats.test.mjs`: 11 node:test cases. +- `tests/bench-outcome.test.mjs`: 13 cases. +- `tests/bench-identity.test.mjs`: 26 cases. + +All three test files are unregistered; plan 66-09 adds them to the chain. + +## Decisions Made + +- **Tolerant baseline instead of `npm test`:** 7225 is held by the user's hub, and the rules forbid freeing it. Because every command passed, the tolerant result is equivalent to a green chain. +- **Sweep identity and exit codes:** + - A failure is keyed by test path. + - A test fails on a non-zero exit, a signal, or hitting the time bound. + - Exit codes are 0 (no new failures), 1 (failures, or new failures with `--baseline`) and 2 (usage or setup error, including a `--files` path that does not exist). + - `--files`, `--out` and `--baseline` resolve against the current directory. +- **Sweep port probe:** a loopback TCP connect to 127.0.0.1:7225 that sends no bytes, rather than a bind that could race a hub that is starting. +- **Outcome evaluator:** + - For `set: 'live'` tasks, final-URL checks use the URL as-is. For other sets, the `/r/` prefix is stripped when `{token}` is passed. + - Check objects that don't match the contract fail with `{error:'invalid check'}`. + - Query values are compared as strings. +- **Preflight ordering and scope:** + - Refusals appear in the order of the plan's list. + - `extension_id_mismatch` fires only when `expectExtensionId` is given. + - The `hub_not_owned` detail names the two PIDs, which are not config values. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 2 - Missing critical functionality] Outcome evaluator fails closed on absent request evidence** +- **Found during:** Task 3 +- **Issue:** Treating a missing `evidence.requests` as an empty log would let a `count: 0` absence check pass when the fixture server's evidence was never collected. +- **Fix:** A non-array `requests` makes every request check fail with `{error:'no request evidence'}`. An empty array `[]` is still valid evidence. +- **Files modified:** `scripts/bench/lib/outcome.mjs`, `tests/bench-outcome.test.mjs` +- **Verification:** the case "missing request evidence fails closed, even for count: 0" passes. +- **Committed in:** `3d3c598c` (test) / `b19e65e8` (feat) + +**2. [Rule 2 - Missing critical functionality] Preflight fails closed on unknown git state for campaigns** +- **Found during:** Task 3 +- **Issue:** Firing `dirty_tree` only when `git.dirty === true` would let a campaign run when git state could not be read. +- **Fix:** A campaign without `allowDirty` refuses unless `git.dirty === false`. +- **Files modified:** `scripts/bench/lib/identity.mjs`, `tests/bench-identity.test.mjs` +- **Verification:** the case "dirty_tree applies only to campaigns without allowDirty", including its unknown-state assertion, passes. +- **Committed in:** `3d3c598c` / `b19e65e8` + +**3. [Rule 2 - Missing critical functionality] Timed-out tests cannot leave straggler processes** +- **Found during:** Task 1 +- **Issue:** Clearing the SIGKILL timer as soon as the direct child exits would let descendants that ignored SIGTERM keep running into later tests. +- **Fix:** After a timed-out test closes, its process group gets one more SIGKILL. +- **Files modified:** `scripts/bench/sweep-tests.mjs` +- **Verification:** a 30 ms bound run killed the test by SIGTERM, and no process was left afterwards. +- **Committed in:** `ee447d4d` + +### Execution notes (not rule deviations) + +- **TDD commit split:** Tasks 2 and 3 are `tdd="true"`, so each has a RED `test(66-02)` commit before its GREEN `feat(66-02)` commit. The plan named one commit per task; the GREEN commits use the plan's messages verbatim. This matches plan 66-01's history. +- **Branch safety check:** the executor's `worktree-agent-*` branch allow-list was skipped because the orchestrator directed commits on `Jev` in this workspace. The cwd-drift sentinel and the protected-branch deny-list ran before every commit. +- **Tolerant runner additions:** the runner prepended `node_modules/.bin` to `PATH` to mirror `npm run`, added a 20-minute per-command safety bound (never hit), and wrote extra JSONL fields. None of this changes what counts as a pass. +- **Extra output fields:** the sweep JSON adds `finishedAt`, `nodeVersion` and `perTestTimeoutMs` to the specified shape. `gitIdentity` uses `status --porcelain -z` so paths with spaces and renames parse correctly, and it accepts 64-hex SHA-256 object IDs. `toolsListDigest` breaks ties between equal names on canonical JSON, so duplicate names cannot make the digest order-dependent. `PREFLIGHT_REFUSAL_CODES` and `AA_NOISE_FLOOR_LABEL` are additive exports. +- **Drafting order:** the sweep script was drafted in /tmp while the tolerant pass ran, so no file was written into the repo during the baseline. + +--- + +**Total deviations:** 3 auto-fixed (all Rule 2, fail-closed correctness). +**Impact on plan:** they tighten evidence and preflight correctness. There is no scope creep, and the footprint is exactly the plan's `files_modified`. + +## Issues Encountered + +- The plan expected port-bound suites to fail while 7225 was held. They passed (T44 254/0, T45 108/0), so the pre-existing failure set in the chain is empty. +- No local Node 20 binary exists, and CI runs Node 20. Instead, a static scan found no Node 22+ API in the eight new files (`Object.groupBy`, `Promise.withResolvers`, `crypto.hash`, `fs.globSync`, `process.getBuiltinModule`, global `WebSocket`, Set methods, iterator helpers, `toSorted` and similar). Running the tests under the official Node 20 tarball remains part of the phase gate. + +## Known Stubs + +None. The modules are complete, pure libraries with no data wiring left open. + +## Threat Flags + +| Flag | File | Description | +|------|------|-------------| +| threat_flag: loopback-probe | scripts/bench/sweep-tests.mjs | Opens and immediately closes one TCP connection to 127.0.0.1:7225 per sweep to record whether the hub port is held. It sends no bytes. This is not in the plan's threat register, and it is read-only toward the hub. | + +## User Setup Required + +None. No external service configuration is required. + +## Next Phase Readiness + +- **Plans 66-03, 66-04 and 66-06:** after each extension edit, run `node scripts/bench/sweep-tests.mjs --files --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It exits 1 only on new failures. The baseline was taken with 7225 held, and the sweep prints a note when the port state differs. +- **Plans 66-05, 66-06 and 66-08:** they implement against `evaluateOutcome` (evidence contract), `evaluatePreflight` (fsb://config payload from plan 66-03) and `geoMeanMedianRatio`/`aaNoiseFloor`/`wilson`/`summarize`. +- **Plan 66-09:** registers `tests/bench-{stats,outcome,identity}.test.mjs` in the `test` chain and in `REFINEMENT_ROOT_TEST_COMMANDS`. +- **Blockers:** none. + +## Self-Check: PASSED + +- All 8 created files were found on disk. +- All 5 task commits were found: `ee447d4d`, `0fac17e6`, `cb98a67c`, `3d3c598c`, `b19e65e8`. +- No commit in `b4fe9d41..HEAD` has a Co-Authored-By line. +- `node --test tests/bench-stats.test.mjs tests/bench-outcome.test.mjs tests/bench-identity.test.mjs` passes 50/50. +- `git status --porcelain --untracked-files=no -- mcp showcase extension package.json package-lock.json` is empty. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json b/.planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json new file mode 100644 index 000000000..0bdd23f38 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json @@ -0,0 +1,606 @@ +{ + "startedAt": "2026-09-30T06:30:04.914Z", + "finishedAt": "2026-09-30T06:30:13.929Z", + "head": "b4fe9d415b62cd1c6f7c0a22a7838eab068b2efb", + "nodeVersion": "v24.14.1", + "port7225Held": true, + "perTestTimeoutMs": 180000, + "files": [ + "extension/ws/mcp-bridge-client.js", + "extension/ws/mcp-tool-dispatcher.js", + "extension/utils/mcp-session-recorder.js", + "extension/utils/mcp-lattice-journal.js", + "extension/utils/spreadsheet-record-redaction.js", + "extension/ai/cost-tracker.js", + "extension/ai/agent-loop.js", + "extension/ai/action-history.js", + "extension/utils/automation-logger.js" + ], + "results": [ + { + "test": "tests/action-history.test.js", + "command": "node tests/action-history.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 47 + }, + { + "test": "tests/action-icon-behavior.test.js", + "command": "node tests/action-icon-behavior.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 166 + }, + { + "test": "tests/action-tool-agent-scoped.test.js", + "command": "node tests/action-tool-agent-scoped.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 68 + }, + { + "test": "tests/agent-bridge-routes.test.js", + "command": "node tests/agent-bridge-routes.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/agent-grace.test.js", + "command": "node tests/agent-grace.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 960 + }, + { + "test": "tests/agent-loop-empty-contents.test.js", + "command": "node tests/agent-loop-empty-contents.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 92 + }, + { + "test": "tests/agent-loop-iterator-guard.test.js", + "command": "node tests/agent-loop-iterator-guard.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + }, + { + "test": "tests/agent-pooling.test.js", + "command": "node tests/agent-pooling.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 110 + }, + { + "test": "tests/agent-registry.test.js", + "command": "node tests/agent-registry.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 89 + }, + { + "test": "tests/automation-logger-mcp-retention.test.js", + "command": "node tests/automation-logger-mcp-retention.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 331 + }, + { + "test": "tests/autopilot-agent-scope.test.js", + "command": "node tests/autopilot-agent-scope.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 83 + }, + { + "test": "tests/back-tool-ownership.test.js", + "command": "node tests/back-tool-ownership.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 66 + }, + { + "test": "tests/capability-autopilot-parity.test.js", + "command": "node tests/capability-autopilot-parity.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 69 + }, + { + "test": "tests/capability-search-agent-scope.test.js", + "command": "node tests/capability-search-agent-scope.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + }, + { + "test": "tests/change-report-dispatcher.test.js", + "command": "node tests/change-report-dispatcher.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 563 + }, + { + "test": "tests/change-report-read-tools-excluded.test.js", + "command": "node tests/change-report-read-tools-excluded.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 81 + }, + { + "test": "tests/change-report-toggle.test.js", + "command": "node tests/change-report-toggle.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 568 + }, + { + "test": "tests/cost-tracker-ordering.test.js", + "command": "node tests/cost-tracker-ordering.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 72 + }, + { + "test": "tests/cost-tracker.test.js", + "command": "node tests/cost-tracker.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 64 + }, + { + "test": "tests/delegation-phase-contract.test.js", + "command": "node tests/delegation-phase-contract.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 167 + }, + { + "test": "tests/delegation-sidepanel-tab-follow.test.js", + "command": "node tests/delegation-sidepanel-tab-follow.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 73 + }, + { + "test": "tests/extension-content-script-files-completeness.test.js", + "command": "node tests/extension-content-script-files-completeness.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 56 + }, + { + "test": "tests/foreground-audit.test.js", + "command": "node tests/foreground-audit.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 53 + }, + { + "test": "tests/goal-progress-tracker.test.js", + "command": "node tests/goal-progress-tracker.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/install-identity.test.js", + "command": "node tests/install-identity.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + }, + { + "test": "tests/lattice-provider-bridge-smoke.test.js", + "command": "node tests/lattice-provider-bridge-smoke.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 84 + }, + { + "test": "tests/lattice-replay-background-contract.test.js", + "command": "node tests/lattice-replay-background-contract.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 66 + }, + { + "test": "tests/lattice-replay-playback.test.js", + "command": "node --test tests/lattice-replay-playback.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 206 + }, + { + "test": "tests/lattice-step-emitter-smoke.test.js", + "command": "node tests/lattice-step-emitter-smoke.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 60 + }, + { + "test": "tests/lattice-survivability-smoke.test.js", + "command": "node tests/lattice-survivability-smoke.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 82 + }, + { + "test": "tests/list-tabs-agent-scope.test.js", + "command": "node tests/list-tabs-agent-scope.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 57 + }, + { + "test": "tests/lmstudio-agent-request.test.js", + "command": "node tests/lmstudio-agent-request.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + }, + { + "test": "tests/mcp-agent-providers-storage.test.js", + "command": "node tests/mcp-agent-providers-storage.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 96 + }, + { + "test": "tests/mcp-bridge-background-dispatch.test.js", + "command": "node tests/mcp-bridge-background-dispatch.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 169 + }, + { + "test": "tests/mcp-bridge-client-lifecycle.test.js", + "command": "node tests/mcp-bridge-client-lifecycle.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 76 + }, + { + "test": "tests/mcp-client-identity-integration.test.js", + "command": "node tests/mcp-client-identity-integration.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 151 + }, + { + "test": "tests/mcp-dispatcher-client-label.test.js", + "command": "node tests/mcp-dispatcher-client-label.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/mcp-in-flight-session-lookup.test.js", + "command": "node tests/mcp-in-flight-session-lookup.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/mcp-lattice-journal.test.js", + "command": "node --test tests/mcp-lattice-journal.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 572 + }, + { + "test": "tests/mcp-metrics-recorder.test.js", + "command": "node tests/mcp-metrics-recorder.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 65 + }, + { + "test": "tests/mcp-philosophy-parity-smoke.test.js", + "command": "node tests/mcp-philosophy-parity-smoke.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 56 + }, + { + "test": "tests/mcp-restricted-tab.test.js", + "command": "node tests/mcp-restricted-tab.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/mcp-screenshot-route.test.js", + "command": "node --test tests/mcp-screenshot-route.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 156 + }, + { + "test": "tests/mcp-session-recorder-journal.test.js", + "command": "node tests/mcp-session-recorder-journal.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 76 + }, + { + "test": "tests/mcp-session-recorder.test.js", + "command": "node tests/mcp-session-recorder.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 77 + }, + { + "test": "tests/mcp-tool-routing-contract.test.js", + "command": "node tests/mcp-tool-routing-contract.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + }, + { + "test": "tests/mcp-version-parity.test.js", + "command": "node tests/mcp-version-parity.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 502 + }, + { + "test": "tests/mcp-visual-session-contract.test.js", + "command": "node tests/mcp-visual-session-contract.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 83 + }, + { + "test": "tests/mcp-visual-tick-lifecycle.test.js", + "command": "node tests/mcp-visual-tick-lifecycle.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 73 + }, + { + "test": "tests/meta-cognitive-tracker.test.js", + "command": "node tests/meta-cognitive-tracker.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 58 + }, + { + "test": "tests/multi-agent-regression.test.js", + "command": "node tests/multi-agent-regression.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 71 + }, + { + "test": "tests/open-tab-background-default.test.js", + "command": "node tests/open-tab-background-default.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + }, + { + "test": "tests/openai-agent-request.test.js", + "command": "node tests/openai-agent-request.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 61 + }, + { + "test": "tests/ownership-error-codes.test.js", + "command": "node tests/ownership-error-codes.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/ownership-gate.test.js", + "command": "node tests/ownership-gate.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 64 + }, + { + "test": "tests/ownership-leakage-regression.test.js", + "command": "node tests/ownership-leakage-regression.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 67 + }, + { + "test": "tests/provider-parity.test.js", + "command": "node tests/provider-parity.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 68 + }, + { + "test": "tests/read-tool-tab-resolution.test.js", + "command": "node tests/read-tool-tab-resolution.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 61 + }, + { + "test": "tests/recovery-tools-bootstrap.test.js", + "command": "node tests/recovery-tools-bootstrap.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 65 + }, + { + "test": "tests/run-task-cleanup-paths.test.js", + "command": "node tests/run-task-cleanup-paths.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 56 + }, + { + "test": "tests/run-task-heartbeat.test.js", + "command": "node tests/run-task-heartbeat.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 65 + }, + { + "test": "tests/run-task-resolve-discipline.test.js", + "command": "node tests/run-task-resolve-discipline.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 97 + }, + { + "test": "tests/sidepanel-message-log-smoke.test.js", + "command": "node tests/sidepanel-message-log-smoke.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 65 + }, + { + "test": "tests/sidepanel-tab-aware-smoke.test.js", + "command": "node tests/sidepanel-tab-aware-smoke.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 61 + }, + { + "test": "tests/spreadsheet-record-redaction.test.js", + "command": "node --test tests/spreadsheet-record-redaction.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 152 + }, + { + "test": "tests/stuck-action-repetition.test.js", + "command": "node tests/stuck-action-repetition.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 112 + }, + { + "test": "tests/tool-definitions-parity.test.js", + "command": "node tests/tool-definitions-parity.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 76 + }, + { + "test": "tests/tool-executor-readonly.test.js", + "command": "node tests/tool-executor-readonly.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 111 + }, + { + "test": "tests/trigger-blocking-reporting.test.js", + "command": "node tests/trigger-blocking-reporting.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 107 + }, + { + "test": "tests/trigger-tool-dispatcher.test.js", + "command": "node tests/trigger-tool-dispatcher.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 185 + }, + { + "test": "tests/universal-provider-lmstudio.test.js", + "command": "node tests/universal-provider-lmstudio.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 62 + }, + { + "test": "tests/upload-file-route.test.js", + "command": "node tests/upload-file-route.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 58 + }, + { + "test": "tests/visual-session-agent-scoped.test.js", + "command": "node tests/visual-session-agent-scoped.test.js", + "exitCode": 0, + "signal": null, + "timedOut": false, + "durationMs": 59 + } + ], + "failures": [] +} diff --git a/.planning/phases/66-baseline-and-session-timing/66-03-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-03-PLAN.md new file mode 100644 index 000000000..071f92a09 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-03-PLAN.md @@ -0,0 +1,298 @@ +--- +phase: 66-baseline-and-session-timing +plan: 03 +type: execute +wave: 2 +depends_on: ["66-02"] +files_modified: + - extension/ws/mcp-bridge-client.js + - extension/utils/spreadsheet-record-redaction.js + - extension/utils/mcp-session-recorder.js + - extension/utils/mcp-lattice-journal.js + - tests/mcp-bridge-background-dispatch.test.js + - tests/spreadsheet-record-redaction.test.js + - tests/mcp-session-recorder-journal.test.js + - tests/mcp-lattice-journal.test.js +autonomous: true +requirements: [MEAS-01, MEAS-03] +user_setup: [] + +must_haves: + truths: + - "Every MCP call entering the extension is stamped with receivedAt on the non-enumerable Symbol.for('fsb.mcp.call-timing') at _handleMessage. The stamp never appears in any tool response or JSON (D-04)." + - "Recorder entries carry timing {v:1, receivedAt, completedAt, fsbMs}, with completedAt stamped synchronously at the recording call site. Journal tool.call events store it as metadata.timing only when valid, never 0 and never derived from event timestamps (D-04)." + - "Timing is never added to the beginCall identity. It is never written to chrome.storage.local per call. Google Sheets entries keep a validated timing through the redaction allowlist." + - "Legacy-only MCP sessions carry timing and durationMs on their actionHistory rows, so both recording paths hold timing (D-04)." + - "Both journal text exports show per-call FSB milliseconds and a caller gap labelled as an upper bound on host thinking time. MCP queue wait is stated as not measured (D-03, D-09). The two producers stay byte-identical." + - "fsb://config returns only providerKind, modelProvider, modelName (plus optional maxIterations/domOptimization/maxDOMElements), the loaded extension id, the manifest version and a Phase 66 measurement marker. No API key or secret-like key is ever read or returned (D-18, D-01)." + - "No file under mcp/ changes. Routing, tool names, schemas and response shapes are unchanged (D-01, D-02)." + artifacts: + - path: "extension/ws/mcp-bridge-client.js" + provides: "MCP_BRIDGE_CALL_TIMING stamp, _buildMcpCallTiming, timing on the action tap, allowlisted _handleGetConfig" + contains: "Symbol.for('fsb.mcp.call-timing')" + - path: "extension/utils/mcp-session-recorder.js" + provides: "_sanitizeCallTiming and timing forwarding on recordAction, _snapshotJournalEntry, _snapshotEntry, legacy actionHistory, recordTaskOutcome" + contains: "_sanitizeCallTiming" + - path: "extension/utils/mcp-lattice-journal.js" + provides: "metadata.timing and the shared per-call timing formatter for both text exports" + contains: "formatCallTimingSuffix" + - path: "extension/utils/spreadsheet-record-redaction.js" + provides: "validated timing preserved in baseFields" + contains: "timing" + key_links: + - from: "extension/ws/mcp-bridge-client.js _handleMessage" + to: "extension/ws/mcp-bridge-client.js _recordMcpSessionAction" + via: "same payload object; payload[MCP_BRIDGE_CALL_TIMING] read by _buildMcpCallTiming" + pattern: "_buildMcpCallTiming\\(payload\\)" + - from: "extension/utils/mcp-session-recorder.js _snapshotJournalEntry" + to: "extension/utils/mcp-lattice-journal.js recordDispatchNow" + via: "entry.timing -> metadata.timing" + pattern: "metadata\\.timing" + - from: "extension/utils/mcp-lattice-journal.js exportHumanReadable" + to: "extension/utils/mcp-lattice-journal.js streamSessionExport" + via: "shared formatCallTimingSuffix with previousCompletedAt carried across pages" + pattern: "formatCallTimingSuffix\\(" +--- + + +Capture per-call MCP timing where calls enter the extension, and carry it through the session recorder into the Lattice journal (and into legacy sessions). Show it in both journal text exports. Replace `_handleGetConfig` with an allowlisted run-identity projection. + +Purpose: MEAS-01 needs per-call FSB execution time and caller gap for MCP sessions. MEAS-03 (D-18) needs `fsb://config` to prove which extension build, provider and model are loaded. Everything is extension-only (D-01) and additive (D-02). +Output: timing stamp and tap in the bridge client, validated timing in recorder and journal entries, timing in the text exports, and a secret-free config projection, each with paired tests. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/REQUIREMENTS.md +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +SHARED CONTRACT (verbatim; plans 66-04 and 66-06 implement against the same names): +- Symbol: `Symbol.for('fsb.mcp.call-timing')`. The bridge client's top-level identifier is `MCP_BRIDGE_CALL_TIMING`. The dispatcher (plan 66-06) uses a DIFFERENT identifier, `MCP_DISPATCHER_CALL_TIMING`, because both files share one classic-script global lexical scope, and a duplicate top-level `const` throws on the second importScripts. +- Ingress stamp value: `{ receivedAt: }`, defined with `Object.defineProperty(payload, MCP_BRIDGE_CALL_TIMING, { value, enumerable: false, configurable: true })`. +- Recorder entry field: `timing: { v: 1, receivedAt, completedAt, fsbMs }`. completedAt is `Date.now()` taken synchronously at the recording call site, before any fire-and-forget enqueue; `fsbMs = completedAt - receivedAt`. When there is no stamp, the field is null or omitted, never 0. +- Validation used by the recorder, journal and Sheets redaction: v === 1; receivedAt and completedAt are safe non-negative integers; completedAt >= receivedAt; fsbMs is recomputed as completedAt - receivedAt. Anything else is dropped. +- Journal: `metadata.timing` has the same shape and is present only when valid. +- Legacy MCP actionHistory row: adds `timing` (same shape) and `durationMs = timing.fsbMs`, only when timing is valid. Plan 66-04 makes automation-logger persist both. +- fsb://config payload (the `_handleGetConfig` return): `{ config: { providerKind, modelProvider, modelName, maxIterations?, domOptimization?, maxDOMElements? }, extension: { id, version, measurement: { callTiming: 1, sessionDetailJournal: 1 } } }`. +- Use `Date.now()` only. `performance` is undefined in the vm harnesses. + +Existing code (verified 2026-09-29): +- mcp-bridge-client.js: `_handleMessage` ~1144-1199 (insert after `if (!id || !type) return;`); `_recordMcpSessionAction` ~1689-1743 (`let sessionRecordEntry = {` ~1712-1726); `_handleExecuteAction` starts ~1745 (`resolveAgentTabOrError` is at offset 3845 of its 4500-char budget); `_handleGetConfig` ~2702-2708. The defaults to copy come from extension/background.js ~1726-1739 `fsbReadAuthoritativeProviderConfig`. +- spreadsheet-record-redaction.js `baseFields(entry)` ~246-255. +- mcp-session-recorder.js (IIFE): `_snapshotEntry` ~854-880; `_snapshotJournalEntry` ~918-966; `recordTaskOutcome` ~1751-1800 (journal entry literal ~1777-1786); legacy `session.actionHistory.push` ~2146-2152 (keep ~2157-2170 byte-intact); `recordDispatch` ~2222-2280; `recordAction` ~2336-2352. +- mcp-lattice-journal.js (IIFE): `recordDispatchNow` metadata literal ~1210-1229 (add timing before `makeEvent`; the compact budget marker ~1243-1246 reuses the same object); `exportHumanReadable` ~2019-2045; `streamSessionExport` text branch ~2121-2153 (`callIndex` persists across 500-event pages). + + + + + + + Task 1: Ingress stamp, action-tap timing, Sheets timing allowlist, and allowlisted fsb://config identity + extension/ws/mcp-bridge-client.js, extension/utils/spreadsheet-record-redaction.js, tests/mcp-bridge-background-dispatch.test.js, tests/spreadsheet-record-redaction.test.js + + - extension/ws/mcp-bridge-client.js lines 1-60, 1140-1200, 1685-1760, 2698-2712 + - extension/ws/mcp-tool-dispatcher.js lines 157-160 and 3184-3197 (the MCP_REPLAY_RECORD_CONTEXT hidden-symbol precedent) + - extension/background.js lines 1726-1739 (fsbReadAuthoritativeProviderConfig defaults) + - extension/config/config.js lines 14-60 (secret keys that must never be read: apiKey, geminiApiKey, openaiApiKey, anthropicApiKey, openrouterApiKey, customApiKey, captchaApiKey, customEndpoint) + - extension/utils/spreadsheet-record-redaction.js lines 225-290 + - tests/mcp-bridge-background-dispatch.test.js lines 195-330 (harness, chrome mock), 395-490 (lease lifecycle cases), 3600-3642 (run()) + - tests/spreadsheet-record-redaction.test.js lines 1-60 and 770-790 + - tests/mcp-session-recorder.test.js lines 860-955 (TRIPWIRES: exactly 1 "never let session recording break the action" comment; exactly 3 `this._recordMcpSessionAction(` calls; the `let sessionRecordEntry = {` span contains `resolveMcpClientLabel(payload)`; exactly 1 `recordSafely(... 'recordAction' ... sessionRecordEntry ...)`; `resolveAgentTabOrError` within 4500 chars after `async _handleExecuteAction`) + + + - _handleMessage on `{id, type:'mcp:execute-action', payload:{tool:'click', params:{}}}` passes _routeMessage a payload whose `Symbol.for('fsb.mcp.call-timing')` value has a finite receivedAt. `JSON.stringify(payload)` and `Object.keys(payload)` do not show it, and the JSON handed to _sendResult contains none of `receivedAt`, `fsbMs`, `call-timing`. + - The lease lifecycle order stays ['begin','route','result'|'error','end'], and begin/end identities still deep-equal (existing cases unchanged). + - _recordMcpSessionAction with a stamped payload (receivedAt = now - 40) hands recordAction an entry whose timing is {v:1, receivedAt, completedAt, fsbMs}, with fsbMs >= 40 and completedAt >= receivedAt. With an unstamped payload, timing is null. + - Spoofing: a payload whose JSON carries a string key `timing: {v:1, receivedAt:1, completedAt:2, fsbMs:1}` still gets entry.timing.receivedAt from the Symbol stamp (>= the time the test started), not 1. + - _handleGetConfig with storage {providerKind:'api', modelProvider:'xai', modelName:' grok-4.3 ', maxIterations:50, apiKey:'xai-SECRET-1', geminiApiKey:'g-SECRET-2', customEndpoint:'https://secret.example.test'} and getManifest returning {version:'0.9.91'} returns exactly {config:{providerKind:'api', modelProvider:'xai', modelName:'grok-4.3', maxIterations:50}, extension:{id:'dispatch-test-extension', version:'0.9.91', measurement:{callTiming:1, sessionDetailJournal:1}}}. The JSON contains neither `SECRET` nor `secret.example`. + - _handleGetConfig with empty storage and no getManifest returns config {providerKind:'api', modelProvider:'xai', modelName:''} and extension.version null, without throwing. + - Sheets: sanitizeEntry on a Google Sheets entry with a valid timing keeps a deep-equal timing. With an invalid timing (negative, string, completedAt < receivedAt, v:2), no timing key is present. A non-Sheets entry is returned unchanged (same object). + + +In extension/ws/mcp-bridge-client.js: +1. Add a top-level `const MCP_BRIDGE_CALL_TIMING = Symbol.for('fsb.mcp.call-timing');` beside the other top-level consts near the file head. Add a two-line comment: in-process-only call timing stamp (D-04); JSON and structuredClone drop Symbol keys, so it never reaches responses or storage. +2. In `_handleMessage`, directly after `if (!id || !type) return;` and before the `recordingIdentity` block: when `payload && typeof payload === 'object'`, define the stamp `{ receivedAt: Date.now() }` with `Object.defineProperty(..., { value, enumerable: false, configurable: true })` inside try/catch. Use the comment `/* timing capture never gates routing */`. Do NOT touch `recordingIdentity`, `beginCall` or `endCall`. +3. Add a class method `_buildMcpCallTiming(payload)` immediately before `_recordMcpSessionAction`. It reads `payload && payload[MCP_BRIDGE_CALL_TIMING]`, requires `Number.isSafeInteger(receivedAt) && receivedAt >= 0`, takes `completedAt = Date.now()`, and returns `{ v: 1, receivedAt, completedAt, fsbMs: Math.max(0, completedAt - receivedAt) }`. Otherwise it returns null. It never reads string keys named timing. +4. In `_recordMcpSessionAction`, add the property `timing: this._buildMcpCallTiming(payload)` as the last property of the existing `let sessionRecordEntry = {` literal. Do not add a new call site. Do not alter the `recordSafely`/`recordAction` branch, the `isMcpSpreadsheetRecord` logic, or the existing `/* never let session recording break the action */` comment. All edits sit before `async _handleExecuteAction`, so its 4500-char budget is unaffected. Do not edit anything inside `_handleExecuteAction`. +5. Replace the body of `_handleGetConfig()`: + - Read only `chrome.storage.local.get(['providerKind','modelProvider','modelName','maxIterations','domOptimization','maxDOMElements'])`. + - Build `config` with the fsbReadAuthoritativeProviderConfig defaults: providerKind is the string value or 'api'; modelProvider is the string value or 'xai'; modelName is the trimmed string or ''. Include maxIterations and maxDOMElements only when Number.isFinite, and domOptimization only when boolean. + - Read the manifest version via `typeof chrome.runtime.getManifest === 'function'` inside try/catch (null on failure). + - Return `{ config, extension: { id: (chrome.runtime && chrome.runtime.id) || null, version, measurement: { callTiming: 1, sessionDetailJournal: 1 } } }`. + - Never request, read or return any other storage key (D-18; the resource description promises "API keys are redacted"). + +In extension/utils/spreadsheet-record-redaction.js: add a private helper inside the IIFE, `safeTiming(value)`, implementing the contract validation (it returns a fresh `{v:1, receivedAt, completedAt, fsbMs}` or null). In `baseFields(entry)`, set `out.timing` only when `safeTiming(entry.timing)` is non-null. Change nothing else in the module. + +Tests: +- Append to tests/mcp-bridge-background-dispatch.test.js three async cases: `runCallTimingIngressCase`, `runCallTimingActionTapCase`, `runGetConfigIdentityCase`. Register them in `run()` after `runRecordingLeaseDoesNotGateRoutingCase()`. + - Build them with the existing `buildClientHarness`, `assertDeepEqual`, `assert`, `toPlainObject` helpers. + - Seed the chrome mock's `storage.local` through its own API, and attach `chrome.runtime.getManifest` only in the manifest sub-case. + - Access the symbol in tests as `Symbol.for('fsb.mcp.call-timing')` (the registry is shared across vm realms). + - Keep every existing case unchanged. +- Add a `node:test` case to tests/spreadsheet-record-redaction.test.js for valid, invalid and non-Sheets timing, using the module's `sanitizeEntry`/`recordSafely`. Leave the source-pin test at ~773-782 untouched. + +Run the touched suites, then the cross-file pin suites: `node tests/mcp-session-recorder.test.js`, `node tests/mcp-dispatcher-client-label.test.js`, `node tests/lattice-replay-background-contract.test.js`. If a pin fails on a file this plan does not own, re-run once after confirming no parallel plan is mid-edit. Never edit files outside files_modified. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-bridge-client.js extension/utils/spreadsheet-record-redaction.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit (source and paired tests together, per tripwire discipline): `feat(66-03): stamp MCP call timing at ingress and expose allowlisted run identity`. No Co-Authored-By or AI attribution. + + + node tests/mcp-bridge-background-dispatch.test.js && node --test tests/spreadsheet-record-redaction.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js + + + - `grep -c "Symbol.for('fsb.mcp.call-timing')" extension/ws/mcp-bridge-client.js` prints 1, and `grep -c "MCP_BRIDGE_CALL_TIMING" extension/ws/mcp-bridge-client.js` prints at least 3 (declaration, stamp, read). + - `grep -c "this._recordMcpSessionAction(" extension/ws/mcp-bridge-client.js` still prints 3. + - `node -e "const s=require('fs').readFileSync('extension/ws/mcp-bridge-client.js','utf8');const a=s.indexOf('async _handleExecuteAction');const o=s.indexOf('resolveAgentTabOrError',a)-a;if(o<0||o>4500)process.exit(1);console.log(o)"` exits 0 and prints 3845 (unchanged). + - `node -e "const s=require('fs').readFileSync('extension/ws/mcp-bridge-client.js','utf8');const b=s.slice(s.indexOf('async _handleGetConfig()'), s.indexOf('async _handleGetSiteGuides'));for(const k of ['apiKey','ApiKey','selectedModel','customEndpoint','captcha'])if(b.includes(k))process.exit(1)"` exits 0. + - `node tests/mcp-bridge-background-dispatch.test.js`, `node --test tests/spreadsheet-record-redaction.test.js`, `node tests/mcp-session-recorder.test.js` and `node tests/mcp-dispatcher-client-label.test.js` all exit 0. + - `git status --porcelain -- mcp/` prints nothing and no commit from this task touches mcp/ (D-01). + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-bridge-client.js extension/utils/spreadsheet-record-redaction.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + Calls are stamped at ingress. The action tap records validated timing. Sheets entries keep timing. fsb://config returns only the allowlisted identity plus the Phase 66 marker. All pinned suites stay green. + + + + Task 2: Carry validated timing through the session recorder on journal, legacy and task-outcome paths + extension/utils/mcp-session-recorder.js, tests/mcp-session-recorder-journal.test.js + + - extension/utils/mcp-session-recorder.js lines 850-970, 1745-1805, 2040-2180, 2218-2285, 2326-2356 + - tests/mcp-session-recorder-journal.test.js lines 1-200 (makeJournalStub, reset, journalEntry, and the beginCall identity deep-equal at ~92, which must stay unchanged) plus the file's existing recordTaskOutcome and legacy-client cases + - tests/mcp-session-recorder.test.js lines 1350-1360 (TRIPWIRE: zero `setTimeout(` in the recorder) + - tests/lattice-replay-background-contract.test.js lines 385-450 (TRIPWIRES: substrings `requestPayload: _sanitizeRequestPayload(payload, entry.response)`, `redactedInputs: entry.redactedInputs === true`, `reason: 'idle_timeout'` must remain) + + + - recordDispatch(journalEntry + valid timing) makes journalStub.calls.dispatches[0].timing deep-equal {v:1, receivedAt, completedAt, fsbMs}. + - recordDispatch with each invalid timing (negative receivedAt, string, completedAt < receivedAt, v:2, fsbMs inconsistent) produces a dispatch without an own `timing` property. An inconsistent fsbMs is recomputed from completedAt - receivedAt, so it is kept but corrected. + - recordAction({..., timing}) forwards timing to the journal dispatch. + - recordTaskOutcome({tool:'complete_task', params, payload, response:{status:'completed'}, timing}) through the journal path makes calls.outcomes[0].value.timing deep-equal the validated timing. + - A legacy-only client (no valid recordingRunId sidecar) with timing gets `timing` on its session.actionHistory row and `durationMs === timing.fsbMs`. Without timing, the row has neither key. + - The existing beginCall identity deep-equal case (7 keys) passes unchanged. + + +All edits go inside the recorder IIFE: +1. Add `_sanitizeCallTiming(value)` next to `_snapshotEntry`. It implements the contract validation and returns a fresh `{v:1, receivedAt, completedAt, fsbMs}` (fsbMs recomputed) or null. Use `Number.isSafeInteger`. +2. `recordAction(input)`: add `timing: input.timing` to the object passed to `recordDispatch`. +3. `_snapshotJournalEntry(entry)`: keep every existing field byte-identical. After the object is built, attach `timing` only when `_sanitizeCallTiming(entry && entry.timing)` is non-null, then return the object. Restructure `return {...}` into a local variable plus return. Do not change any existing property line. +4. `_snapshotEntry(entry)` (legacy): same conditional timing attachment. +5. In the legacy `_recordDispatchNow` actionHistory push (~2146-2152): build the row object with the same five fields. When the snapshotted entry has timing, set `row.timing = entry.timing` and `row.durationMs = entry.timing.fsbMs`. Keep the replayEntries push (~2157-2170) byte-identical. +6. `recordTaskOutcome(input)`: add `timing: input.timing` to the `_snapshotJournalEntry({...})` literal (~1777-1786). The legacy `_recordTaskOutcomeNow` path is unchanged. +7. Add no timers and no `setTimeout` token anywhere. Use `Date.now()` only if a timestamp is needed; none is expected, because completedAt arrives from the call site. + +Tests: add `node:test` cases to tests/mcp-session-recorder-journal.test.js covering every behavior above. Use the existing `reset`, `makeJournalStub` and `journalEntry` helpers and `recorder._drainForTests()` to flush the queue. For the legacy row, follow the file's existing legacy-client case. If no accessor exposes open legacy sessions, capture `sessionData` by overriding `globalThis.automationLogger.saveSession` in that test. + +Run `node tests/mcp-session-recorder-journal.test.js`, `node tests/mcp-session-recorder.test.js` and `node tests/lattice-replay-background-contract.test.js`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/utils/mcp-session-recorder.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-03): carry validated MCP call timing through the session recorder`. No Co-Authored-By or AI attribution. + + + node tests/mcp-session-recorder-journal.test.js && node tests/mcp-session-recorder.test.js && node tests/lattice-replay-background-contract.test.js + + + - `grep -c "_sanitizeCallTiming(" extension/utils/mcp-session-recorder.js` prints at least 3 (definition plus the journal and legacy snapshots). + - `node -e "const s=require('fs').readFileSync('extension/utils/mcp-session-recorder.js','utf8');if(s.includes('setTimeout('))process.exit(1)"` exits 0. + - `grep -c "requestPayload: _sanitizeRequestPayload(payload, entry.response)" extension/utils/mcp-session-recorder.js` prints 1, and `grep -c "redactedInputs: entry.redactedInputs === true" extension/utils/mcp-session-recorder.js` prints at least 1. + - The three suites in exit 0. + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/utils/mcp-session-recorder.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + Validated timing flows from recorder entries into journal dispatches, task outcomes and legacy rows. Invalid timing is dropped. All recorder pins hold. + + + + Task 3: Store metadata.timing in the journal and add per-call timing to both text exports through one shared formatter + extension/utils/mcp-lattice-journal.js, tests/mcp-lattice-journal.test.js + + - extension/utils/mcp-lattice-journal.js lines 1-130 (limits, INTERNAL_PAYLOAD_KEYS which must NOT change, clone, stableJson), 1165-1260, 1495-1530, 1748-1770 (replayEntryFromEvent explicit copy), 2005-2050, 2100-2160, 2195-2250 (api object) + - tests/mcp-lattice-journal.test.js lines 1-100 (setup, entry, allDetail), 405-445 (TRIPWIRE: streamSessionExport text must byte-equal exportHumanReadable), 600-645 (TRIPWIRE: budget-exhausted compact marker byte costs), 1375-1400 (TRIPWIRE: `/Calls retained: 505/`, `/Executable replay: latest 100 of 505/`, `/Call 505: read_step_504/`) + + + - recordDispatch(entry with valid timing) creates a tool.call event whose metadata.timing deep-equals the validated timing. Without timing, or with invalid timing, metadata has no own `timing` key. + - The existing budget-exhausted and byte-cost cases pass unchanged, because entries without timing add no bytes. + - Replay projection is unchanged: replay entries built from events never include `timing`. + - A session with calls A (receivedAt 1000, completedAt 1120), B (receivedAt 1620, completedAt 1700), C (no timing) and D (receivedAt 1650, completedAt 1800) exports lines ending `| fsb 120 ms`, `| fsb 80 ms | caller gap 500 ms (upper bound)`, `| timing not recorded`, and `| fsb 150 ms`. D has no gap because C was untimed. + - An overlapping call (receivedAt before the previous completedAt) ends with `| overlapped previous call` after its fsb part. + - Both reports contain the legend line exactly once: `Timing: fsb = extension receive to completion; caller gap = receive(n) - completion(n-1), an upper bound on host thinking time; MCP queue wait not measured`. + - streamSessionExport(id,'text') output byte-equals exportHumanReadable(id) for a timed session of 505 tool.call events, so the caller gap is continuous across the 500-event page boundary. + + +All edits go inside the journal IIFE: +1. Add `sanitizeCallTiming(value)` with the contract validation (fresh object or null). +2. In `recordDispatchNow`, after the `var metadata = {...}` literal and before the first `makeEvent`: `var callTiming = sanitizeCallTiming(entry.timing); if (callTiming) metadata.timing = callTiming;`. Leave INTERNAL_PAYLOAD_KEYS, stripInternalPayload and replayEntryFromEvent unchanged. +3. Add a constant `CALL_TIMING_LEGEND` holding the legend string from . Add `formatCallTimingSuffix(metadata, previousCompletedAt)`: + - It returns ` | timing not recorded` when metadata.timing is invalid. + - Otherwise it returns ` | fsb ms`, followed by ` | caller gap ms (upper bound)` when previousCompletedAt is a finite number and g = receivedAt - previousCompletedAt >= 0, or ` | overlapped previous call` when g < 0. +4. In `exportHumanReadable`, push `CALL_TIMING_LEGEND` right after the `Calls retained` line and the optional `Executable replay` line, before the blank line. Append the formatter suffix to each call line, tracking `previousCompletedAt`: set it to the call's completedAt when its timing is valid, otherwise to null. +5. In the `streamSessionExport` text branch, add the same legend in the same header position. Append the same suffix per call, with `previousCompletedAt` declared beside `callIndex` so it survives page boundaries. The two producers must stay byte-identical (existing parity test). + +Tests: add `node:test` cases to tests/mcp-lattice-journal.test.js for every behavior above. Use `setup()` and `entry(...)`, extended with a `timing` override passed through as `timing` on the entry. Build the 505-call parity case the way the existing 505-call test does (~1380-1398). Keep every existing assertion unchanged. + +Run `node --test tests/mcp-lattice-journal.test.js tests/mcp-session-export-port.test.js`, then `npm run test:mcp-journal`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/utils/mcp-lattice-journal.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-03): store MCP call timing in the journal and show per-call ms in text exports`. No Co-Authored-By or AI attribution. + + + node --test tests/mcp-lattice-journal.test.js tests/mcp-session-export-port.test.js && npm run test:mcp-journal + + + - `grep -c "formatCallTimingSuffix(" extension/utils/mcp-lattice-journal.js` prints at least 3 (definition plus the two producers). + - `grep -c "metadata.timing = " extension/utils/mcp-lattice-journal.js` prints 1. + - The INTERNAL_PAYLOAD_KEYS block is unchanged: `git diff HEAD -- extension/utils/mcp-lattice-journal.js | grep -c "INTERNAL_PAYLOAD_KEYS"` prints 0. + - The suites in exit 0, including the existing byte-equality and 505-call regex assertions. + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/utils/mcp-lattice-journal.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + The journal stores validated per-call timing. Both text exports show per-call FSB ms and labelled caller gaps, and remain byte-identical to each other. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| MCP caller → extension ingress (_handleMessage) | The payload JSON is attacker-influenced. String keys must never be read as timing. | +| Extension → MCP resource fsb://config | The response crosses to any connected MCP client. | +| Extension → IndexedDB journal | Persisted metadata must not grow unbounded or carry forged values. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-05 | Tampering | _buildMcpCallTiming / recorder / journal | mitigate | Timing is read only from `Symbol.for('fsb.mcp.call-timing')`, which JSON cannot carry. The recorder, journal and Sheets redaction re-validate (safe non-negative integers, completedAt >= receivedAt, fsbMs recomputed). A spoofed string key `timing` is ignored (unit test). | +| T-66-01 | Information disclosure | _handleGetConfig | mitigate | Explicit six-key storage read with type-checked defaults. The unit test seeds seven secret-bearing keys and asserts none appear. A source check forbids `apiKey`/`ApiKey`/`customEndpoint`/`captcha` inside the method body. | +| T-66-03 | Information disclosure | journal text exports | accept | The suffix adds only integers and fixed labels. No request/result content is added to text exports (they list tool names only, as before). | +| T-66-05b | Denial of service | timing capture | mitigate | Every stamp and read sits in try/catch with `timing capture never gates routing`. Recording stays fire-and-forget. No per-call chrome.storage.local writes (D-04). | + + + +- `node tests/mcp-bridge-background-dispatch.test.js && node --test tests/spreadsheet-record-redaction.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node tests/mcp-session-recorder-journal.test.js && node tests/lattice-replay-background-contract.test.js && node --test tests/mcp-lattice-journal.test.js tests/mcp-session-export-port.test.js` +- No recent commit touches mcp/ (D-01): `git log --name-only --format= -n 10 | grep -c '^mcp/'` prints 0. + + + +- MEAS-01 (MCP side): per-call FSB ms and caller gap are captured and persisted, and visible in text exports. +- D-18: fsb://config proves the loaded build, provider and model without secrets. +- Every pinned suite listed above stays green in every commit. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Each commit pairs the source change with its tripwire and test updates. Never stage files under mcp/. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-03-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-03-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-03-SUMMARY.md new file mode 100644 index 000000000..75359efb1 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-03-SUMMARY.md @@ -0,0 +1,259 @@ +--- +phase: 66-baseline-and-session-timing +plan: 03 +subsystem: observability +tags: [mcp, per-call-timing, session-recorder, lattice-journal, fsb-config, run-identity, redaction, node-test] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 02) + provides: start-commit baseline (311/311 chain commands) and the W11 regression sweep with 66-02-SWEEP-BASELINE.json +provides: + - "Ingress stamp { receivedAt } on the non-enumerable Symbol.for('fsb.mcp.call-timing') (MCP_BRIDGE_CALL_TIMING) in _handleMessage" + - "_buildMcpCallTiming: { v:1, receivedAt, completedAt, fsbMs } or null on the bridge action-tap entry" + - "Sheets redaction keeps a validated timing (safeTiming in baseFields)" + - "fsb://config run identity: config { providerKind, modelProvider, modelName, maxIterations?, domOptimization?, maxDOMElements? } plus extension { id, version, measurement { callTiming:1, sessionDetailJournal:1 } }" + - "Recorder _sanitizeCallTiming and timing forwarding: recordAction, _snapshotJournalEntry, _snapshotEntry, recordTaskOutcome, legacy actionHistory rows (timing + durationMs), SW-eviction restore" + - "Journal metadata.timing on tool.call events, plus CALL_TIMING_LEGEND and formatCallTimingSuffix shared by both text exports" +affects: [66-04, 66-05, 66-06, 66-08, benchmark runner preflight, get_session_detail journal branch] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Per-call timing travels on a registry Symbol that JSON, structuredClone and spreads drop; each classic-script file gets its own top-level identifier for it" + - "One v1 timing contract, re-validated at every hop (bridge producer, Sheets redactor, recorder, journal): v === 1, safe non-negative integer times, completedAt >= receivedAt, fsbMs recomputed" + - "Optional measurement fields are attached only when valid; an unmeasured call carries no key, never a 0" + - "One shared line formatter, plus state that survives pages, keeps two text producers byte-identical" + +key-files: + created: [] + modified: + - extension/ws/mcp-bridge-client.js + - extension/utils/spreadsheet-record-redaction.js + - extension/utils/mcp-session-recorder.js + - extension/utils/mcp-lattice-journal.js + - tests/mcp-bridge-background-dispatch.test.js + - tests/spreadsheet-record-redaction.test.js + - tests/mcp-session-recorder-journal.test.js + - tests/mcp-lattice-journal.test.js + +key-decisions: + - "_buildMcpCallTiming returns null when the wall clock stepped backwards (completedAt < receivedAt), rather than clamping fsbMs to 0" + - "Legacy actionHistory rows keep a validated timing and a re-derived durationMs through _sanitizeActionHistory, so a service-worker eviction restore does not strip them" + - "Each tdd task was committed once (tests and source together): the plan requires every pinned suite to be green in every commit. RED was run and recorded before each implementation" + - "fsb://config returns the manifest version only when it is a string; mistyped optional settings are omitted, never coerced" + - "The journal's timing helpers sit next to the text exporters, far from the INTERNAL_PAYLOAD_KEYS block, which is untouched" + +patterns-established: + - "Plan 66-06 reads the same Symbol.for('fsb.mcp.call-timing') as MCP_DISPATCHER_CALL_TIMING and passes timing into recordDispatch / recordTaskOutcome. The recorder and journal need no further change" + - "Text-export call lines end in one of: '| fsb N ms', '| fsb N ms | caller gap N ms (upper bound)', '| fsb N ms | overlapped previous call', or '| timing not recorded'" + +requirements-completed: [MEAS-01, MEAS-03] + +# Metrics +duration: 19min +completed: 2026-09-30 +--- + +# Phase 66 Plan 03: MCP Call Timing and Run Identity Summary + +**MCP calls are stamped with `receivedAt` on a hidden `Symbol.for('fsb.mcp.call-timing')` where they enter the extension. The action tap turns the stamp into validated `{v:1, receivedAt, completedAt, fsbMs}` timing, which reaches the recorder, the journal's `metadata.timing`, legacy `actionHistory` rows and Sheets records. Both journal text exports show per-call FSB milliseconds and a caller gap labelled as an upper bound. `fsb://config` now returns a secret-free run identity (provider, model, extension id, manifest version, Phase 66 marker).** + +This plan delivers the MCP-side slice of MEAS-01 and the run-identity slice of MEAS-03 (D-18). `requirements-completed` above lists the plan's requirement IDs for traceability only. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched, because the orchestrator owns them. + +## Performance + +- **Duration:** 19 min +- **Started:** 2026-09-30T06:43:01Z +- **Completed:** 2026-09-30T07:02Z +- **Tasks:** 3/3 +- **Files modified:** 8 (4 extension sources, 4 paired test files); none created + +## Accomplishments + +- **Ingress stamp (D-04):** + - `_handleMessage` defines `{ receivedAt: Date.now() }` on `payload[MCP_BRIDGE_CALL_TIMING]` with `enumerable: false`, inside a try/catch (`timing capture never gates routing`). + - Tests prove that the stamp is absent from `Object.keys`, from `JSON.stringify` and from the wire result frame, even when a handler echoes the payload back. + - The `beginCall` identity keeps exactly its seven keys. +- **Action-tap timing:** + - `sessionRecordEntry.timing = this._buildMcpCallTiming(payload)` is taken synchronously at the tap. It is null when there is no stamp or the stamp is malformed, and never 0. + - A caller-supplied string key `timing` is ignored: the spoof test shows `receivedAt` always comes from the Symbol. +- **Sheets (orchestrator decision 9):** `baseFields` keeps a fresh validated `timing`. A real-bridge `fillsheet` test shows that the ingress timing survives redaction. +- **`fsb://config` (D-18, T-66-01):** + - It reads exactly six storage keys (asserted on the storage mock) and applies the `fsbReadAuthoritativeProviderConfig` defaults. + - Seven secret-bearing keys seeded in the test never appear in the result or on the wire. + - It returns the extension id, the manifest version, and `measurement: {callTiming:1, sessionDetailJournal:1}`. + - The payload shape matches `evaluatePreflight` in `scripts/bench/lib/identity.mjs` from plan 66-02. +- **Recorder:** + - `_sanitizeCallTiming` enforces the contract. + - Timing flows through `recordAction`, `_snapshotJournalEntry`, `_snapshotEntry` and `recordTaskOutcome` (the journal `task-status` entry). + - Legacy-only sessions get `timing` plus `durationMs = timing.fsbMs` on their `actionHistory` rows. These fields survive an SW-eviction restore and are handed to `automationLogger.saveSession` on close; plan 66-04 persists them. +- **Journal:** + - `recordDispatchNow` sets `metadata.timing` only when the timing is valid, so untimed events keep their byte cost, and the budget-exhausted compact marker keeps it. + - Both text exports print the legend once, after `Calls retained` / `Executable replay`, and share `formatCallTimingSuffix`. + - `previousCompletedAt` survives the 500-event stream pages. A timed 505-call session exports byte-identically from both producers, with a continuous caller gap at call 500 (`fsb 102 ms | caller gap 899 ms (upper bound)`). + +Sample export (rendered from the committed code): + +``` +Calls retained: 3 +Timing: fsb = extension receive to completion; caller gap = receive(n) - completion(n-1), an upper bound on host thinking time; MCP queue wait not measured + +[OK] Call 1: navigate | fsb 812 ms +[OK] Call 2: click | fsb 412 ms | caller gap 9101 ms (upper bound) +[OK] Call 3: read_page | timing not recorded +``` + +## Task Commits + +1. **Task 1: Ingress stamp, action-tap timing, Sheets timing allowlist, allowlisted fsb://config identity:** `a70ac9b6` (feat) +2. **Task 2: Carry validated timing through the session recorder:** `68cdb830` (feat) +3. **Task 3: Journal metadata.timing and per-call timing in both text exports:** `a90c8281` (feat) + +**Plan metadata:** see the `docs(66-03)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `extension/ws/mcp-bridge-client.js`: + - `MCP_BRIDGE_CALL_TIMING` constant, the ingress stamp, and `_buildMcpCallTiming`. + - The `timing` property on the existing tap literal (still exactly 3 `this._recordMcpSessionAction(` calls, and the `_handleExecuteAction` resolver offset is still 3845). + - The allowlisted `_handleGetConfig`. +- `extension/utils/spreadsheet-record-redaction.js`: private `safeTiming`, and `baseFields` keeps a valid timing. Nothing else changed. +- `extension/utils/mcp-session-recorder.js`: + - `_sanitizeCallTiming`. + - Timing forwarding in `recordAction`, `_snapshotJournalEntry`, `_snapshotEntry` and `recordTaskOutcome`. + - Legacy row `timing`/`durationMs`, and `_sanitizeActionHistory` keeps them on restore. + - The replayEntries push is byte-identical, and there is still no `setTimeout(`. +- `extension/utils/mcp-lattice-journal.js`: + - `sanitizeCallTiming`, `CALL_TIMING_LEGEND`, `formatCallTimingSuffix` and `callTimingCompletedAt`. + - `metadata.timing` in `recordDispatchNow`. + - Both text producers updated. + - `INTERNAL_PAYLOAD_KEYS`, `stripInternalPayload` and `replayEntryFromEvent` unchanged. +- `tests/mcp-bridge-background-dispatch.test.js`: `runCallTimingIngressCase`, `runCallTimingActionTapCase` and `runGetConfigIdentityCase`, registered after `runRecordingLeaseDoesNotGateRoutingCase()`. 455 → 456 assertions pass, including a clock-rollback case. +- `tests/spreadsheet-record-redaction.test.js`: two cases, one for valid, invalid and non-Sheets timing and one real-bridge `fillsheet` case. The source-pin test is untouched. +- `tests/mcp-session-recorder-journal.test.js`: four cases, covering journal dispatches (valid, 10 invalid shapes, corrected fsbMs, untimed), `recordAction` forwarding, task outcomes, and legacy rows (live, restored, handed to `saveSession`). +- `tests/mcp-lattice-journal.test.js`: + - The `entry()` helper passes `timing` through only when the override is given, so existing entries are byte-identical. + - Four cases: metadata validation plus replay isolation, the compact marker, gap/overlap/untimed/FAILED lines with the legend, and a timed 505-call parity export. + +## Verification + +| Check | Result | +|-------|--------| +| Task 1 verify (`mcp-bridge-background-dispatch`, `spreadsheet-record-redaction`, `mcp-session-recorder`, `mcp-dispatcher-client-label`) | exit 0: 456/0, 20/0, 353/0, 51/0 | +| Task 2 verify (`mcp-session-recorder-journal`, `mcp-session-recorder`, `lattice-replay-background-contract`) | exit 0: 12/0, 353/0, 59 passed | +| Task 3 verify (`node --test mcp-lattice-journal + mcp-session-export-port`, `npm run test:mcp-journal`) | exit 0: 42/0; 12/0 + 39/0 | +| Plan `` chain (plus `mcp-in-flight-session-lookup`) | exit 0 (15/0 for the in-flight suite) | +| W11 sweeps vs `66-02-SWEEP-BASELINE.json` | Task 1: 24/24 (run twice); Task 2: 5/5; Task 3: 1/1; all four files combined: 27/27. No new failures (7225 held, the same state as the baseline) | +| Full `npm test` chain, tolerant mode (7225 held by pid 42278) | **311/311 commands exit 0** (13 pretest + 298 test), 207 s. Artifacts: `/tmp/fsb-66-03-chain.{jsonl,log,done}` | +| Acceptance greps | `Symbol.for('fsb.mcp.call-timing')` = 1; `MCP_BRIDGE_CALL_TIMING` = 3; tap calls = 3; resolver offset = 3845; `_handleGetConfig` secret-token scan passes; `_sanitizeCallTiming(` = 5; no `setTimeout(`; `requestPayload: _sanitizeRequestPayload(payload, entry.response)` = 1; `redactedInputs: entry.redactedInputs === true` = 3; `formatCallTimingSuffix(` = 3; `metadata.timing = ` = 1; `INTERNAL_PAYLOAD_KEYS` lines in the journal diff = 0 | +| D-01 | `git status --porcelain -- mcp/` is empty, and `git log --name-only --format= -n 10 \| grep -c '^mcp/'` = 0 | + +- The chain rewrote `showcase/angular/public/{llms.txt,llms-full.txt,sitemap.xml}`. They were reverted with `git checkout --` and never committed. +- No lockfile and no `mcp/` file changed. + +## Decisions Made + +- **Clock-rollback guard at the source.** See Deviation 1. +- **Restore keeps legacy timing.** See Deviation 2. +- **Commit granularity:** one commit per task (see TDD Gate Compliance). +- **`fsb://config` hardening beyond the plan text:** + - The manifest version is kept only when it is a string. + - `(await chrome.storage.local.get(...)) || {}` tolerates a storage mock that returns undefined. + - `chrome.runtime` is read once, with a guard. +- **Timing validators are fail-closed and exception-safe.** `_buildMcpCallTiming`, `_sanitizeCallTiming` and the journal's `sanitizeCallTiming` each catch their own errors and return null, so a malformed timing can never drop the whole record (T-66-05b). +- **Where timing is written:** + - The legacy open-session buffer is `chrome.storage.session` (the existing per-mutation SW-eviction buffer), not `chrome.storage.local`. Timing integers on legacy rows therefore travel only with that existing write. + - `chrome.storage.local` receives them only at session close, through `saveSession`. That satisfies D-04's "never per call to `chrome.storage.local`". + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] `_buildMcpCallTiming` returns null on wall-clock rollback instead of an invented 0** +- **Found during:** Task 1 +- **Issue:** The plan's literal builder returns `fsbMs: Math.max(0, completedAt - receivedAt)`. If the clock steps backwards, that object has `completedAt < receivedAt` with `fsbMs: 0`, which is exactly the invented zero the contract forbids. Every downstream validator would drop it, but the in-memory entry would carry it. +- **Fix:** When `completedAt < receivedAt` (or `completedAt` is not a safe integer), the builder returns null. Otherwise `fsbMs = completedAt - receivedAt`. +- **Files modified:** `extension/ws/mcp-bridge-client.js`, `tests/mcp-bridge-background-dispatch.test.js` +- **Verification:** the case "a stamp with receivedAt after completion (clock stepped back) records timing null" passes. +- **Committed in:** `a70ac9b6` + +**2. [Rule 2 - Missing critical functionality] Legacy row timing was lost on service-worker eviction restore** +- **Found during:** Task 2 +- **Issue:** `_restoreFromBuffer` (and the one-time redaction scrub) rebuild `actionHistory` rows through `_sanitizeActionHistory`, which copies an explicit five-field list. `timing`/`durationMs` would silently vanish from every legacy session that outlived an MV3 eviction. That breaks the must-have "Legacy-only MCP sessions carry timing and durationMs on their actionHistory rows". +- **Fix:** `_sanitizeActionHistory` keeps a `_sanitizeCallTiming`-validated `timing` and re-derives `durationMs` from it. Untimed rows keep their exact five keys in the same order. +- **Files modified:** `extension/utils/mcp-session-recorder.js`, `tests/mcp-session-recorder-journal.test.js` +- **Verification:** the restore segment of "legacy-only sessions carry timing and durationMs on actionHistory rows" passes, and the 353-assertion `mcp-session-recorder` suite (restore cases included) is unchanged. +- **Committed in:** `68cdb830` + +--- + +**Total deviations:** 2 auto-fixed (1 Rule 1, 1 Rule 2). +**Impact on plan:** both close correctness gaps inside the plan's own contract. There is no scope creep, and the footprint is exactly the plan's `files_modified`. + +## TDD Gate Compliance + +All three tasks are `tdd="true"`, but the plan type is `execute`, and the plan requires every pinned suite to "stay green in every commit". A standalone RED commit would have left in-chain suites (for example, `tests/mcp-bridge-background-dispatch.test.js`) failing at that commit. So RED was run before each implementation but not committed separately. Each task has one `feat(66-03)` commit that pairs the tests with the source. + +| Task | RED (tests written first, before source) | GREEN | +|------|-------------------------------------------|-------| +| 1 | bridge 432 pass / 23 fail; Sheets 18 / 2 (all failures in new cases) | 456 / 0; 20 / 0 | +| 2 | recorder-journal 8 / 4 (the 4 new cases) | 12 / 0 | +| 3 | lattice-journal 35 / 4 (the 4 new cases; the extended `entry()` kept all 35 existing cases green) | 39 / 0 (+3 export-port) | + +## Issues Encountered + +- graphify's graph is stale for these files, as 66-PATTERNS.md notes: a query returned unrelated nodes. The verified line numbers in the plan and in PATTERNS were used instead. Every one matched the working tree. +- The pre-commit `worktree-agent-*` branch allow-list was skipped because the orchestrator directed commits on `Jev`, as in 66-02. The cwd-drift sentinel and the protected-branch deny-list ran before each commit. + +## Known Stubs + +These are intentional and resolved by plan 66-06: +- **Dispatcher-routed calls are not timed yet.** Only `mcp:execute-action` calls, which go through the bridge action tap, carry timing today. Message-route calls (read-only tools recorded by the dispatcher's `recordDispatch` site) and `complete_task`/`partial_task`/`fail_task` outcomes show `| timing not recorded` until 66-06 adds `MCP_DISPATCHER_CALL_TIMING` and passes `timing` into `recordDispatch` / `recordTaskOutcome`. The recorder already accepts `input.timing` on `recordTaskOutcome`; no production caller passes it yet. +- **The `measurement.sessionDetailJournal: 1` marker comes before the feature it names.** `fsb://config` advertises it as the fixed Phase 66 build marker, but the journal-first `get_session_detail` branch lands in 66-06. A build between 66-03 and 66-06 advertises it early. The live campaign runs only after the whole phase is in, so preflight is unaffected in practice. + +## Threat Model Coverage + +- **T-66-05 (tampering):** + - Timing is read only from the Symbol. + - The string-key spoof test passes. + - The recorder, journal and Sheets redactor re-validate and recompute `fsbMs`. + - Ten invalid shapes are dropped in the recorder tests, and seven in the journal tests. +- **T-66-01 (information disclosure):** + - A six-key read, asserted on the storage mock. + - The seeded-secret test covers both the return value and the wire frame. + - The source scan forbids `apiKey`/`ApiKey`/`selectedModel`/`customEndpoint`/`captcha` inside `_handleGetConfig`. +- **T-66-03 (accepted):** text exports add only integers and fixed labels. +- **T-66-05b (denial of service):** + - Every stamp and read is wrapped in try/catch. + - Recording stays fire-and-forget. + - No timer was added, and there is no per-call `chrome.storage.local` write. + +No new security surface beyond the plan's threat register. + +## User Setup Required + +None. + +## Next Phase Readiness + +- **66-04 (automation-logger):** closing a legacy MCP session hands `actionHistory` rows with `timing` and `durationMs` to `saveSession` (proved by test). The logger's two `.map` projections must keep both fields. +- **66-06 (dispatcher):** + - Declare `const MCP_DISPATCHER_CALL_TIMING = Symbol.for('fsb.mcp.call-timing');`. + - Add `timing` (built the same way as `_buildMcpCallTiming`, with `completedAt` taken synchronously) to the message-route `var sessionRecordEntry = {`. + - Read the stamp from the original payload at the top of `dispatchMcpToolRoute`, before canonicalization, and pass `timing` into `recordTaskOutcome({ tool, params, payload, response, timing })`. + - The `format:'text'` branch can call `journal.exportHumanReadable`, which already shows per-call ms. +- **66-05 / 66-08 (bench runner):** `fsb://config` now satisfies `evaluatePreflight`'s marker, provider and model checks, with no `mcp/` change. +- **Blockers:** none. + +## Self-Check: PASSED + +- All 8 modified files exist on disk. No file was created. +- All three task commits exist: `a70ac9b6`, `68cdb830`, `a90c8281`. +- No commit in `783b8650..HEAD` has a Co-Authored-By or AI attribution line. +- `git status --porcelain -- mcp/` is empty. The working tree was clean before this SUMMARY was written. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-04-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-04-PLAN.md new file mode 100644 index 000000000..82f3653d3 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-04-PLAN.md @@ -0,0 +1,336 @@ +--- +phase: 66-baseline-and-session-timing +plan: 04 +type: execute +wave: 2 +depends_on: ["66-02"] +files_modified: + - extension/ai/cost-tracker.js + - extension/ai/agent-loop.js + - extension/ai/action-history.js + - extension/utils/automation-logger.js + - tests/cost-tracker.test.js + - tests/cost-tracker-ordering.test.js + - tests/action-history.test.js + - tests/automation-logger-mcp-retention.test.js + - tests/fixtures/bench/xai-chat-completion.json +autonomous: true +requirements: [MEAS-01] +user_setup: [] + +must_haves: + truths: + - "Each autopilot iteration records model time from apiCallStartTime (a retry is marked modelRetried) and the summed time of the tools it executed, measured between the permission check and the successful action-history push. Unmeasured values are null or omitted, never 0 (D-05)." + - "Executed autopilot actions carry actionHistory[].durationMs. Denied and malformed-argument calls carry none. createActionEvent defaults durationMs to null, not 0 (D-05, orchestrator decision 5)." + - "Provider-reported usage and cost come first: xAI usage.cost_in_usd_ticks / 1e10 is the cost when present. The price table is a labelled estimate. Reasoning tokens are billed at the output rate where they are reported separately (xai, gemini). Every iteration records usageSource and costSource (D-10)." + - "Unknown models are never priced as Grok in measurement records: estimateCostDetailed returns source 'unavailable'. The legacy numeric estimateCost fallback is decoupled from the Grok rows (D-10, orchestrator decision 9)." + - "Grok rows follow the verified xAI retirement: grok-4.3 and every slug it now serves at $1.25/$2.50 per 1M tokens, and grok-code-fast-1/grok-build-0.1 at $1.00/$2.00. Exact grok-4.3 rows precede the legacy grok-4 substring row (D-11). Other price tables (analytics.js, background.js ~10867) are unchanged (orchestrator decision 6)." + - "iterationTiming (last 100 flat rows), measurementSummary and durationMs survive automationLogger.saveSession in both branches, and the autopilot text export shows per-iteration and per-call milliseconds (D-05, D-09). Each save also stores the legacy total's basis label, `totalCostBasis`, on the session record and on its fsbSessionIndex entry, so list_sessions and get_session_detail report the same label (I-2)." + - "Legacy numeric fields (usage.input || 0, session.totalCost, CostTracker totals) keep their meaning. With measurement off or unavailable, autopilot decisions and execution are unchanged (D-02). No stored measurement key contains the substring 'token', and none is named stepName." + artifacts: + - path: "extension/ai/cost-tracker.js" + provides: "D-11 rows, resolvePricingKey, readProviderUsage, estimateCostDetailed, buildIterationTimingRow, appendIterationTiming, addIterationToolTime" + exports: ["MODEL_PRICING", "estimateCost", "CostTracker", "resolvePricingKey", "readProviderUsage", "estimateCostDetailed", "buildIterationTimingRow", "appendIterationTiming", "addIterationToolTime", "XAI_TICKS_PER_USD", "LEGACY_UNKNOWN_MODEL_PRICING"] + - path: "extension/ai/agent-loop.js" + provides: "Inline model/tool stamps wired to the cost-tracker helpers" + contains: "_al_appendIterationTiming(" + - path: "extension/utils/automation-logger.js" + provides: "Persistence of iterationTiming, measurementSummary, durationMs and legacy MCP timing, the save-time totalCostBasis label on the record and index entry (I-2), plus timing lines in the text export" + contains: "cloneIterationTimingForPersistence" + - path: "tests/fixtures/bench/xai-chat-completion.json" + provides: "Synthetic xAI chat-completion body with cost ticks, reasoning and cached tokens" + key_links: + - from: "extension/ai/agent-loop.js runAgentIteration" + to: "extension/ai/cost-tracker.js appendIterationTiming" + via: "_al_ resolver one-liners (globals in the SW, require() in Node) with null fallbacks" + pattern: "_al_appendIterationTiming\\(session" + - from: "extension/ai/agent-loop.js saveToLogger" + to: "extension/utils/automation-logger.js _saveSessionUnlocked" + via: "session.iterationTiming / session.measurementSummary / actionHistory[].durationMs copied in both branches" + pattern: "cloneIterationTimingForPersistence\\(" +--- + + +Give autopilot sessions honest per-iteration model time, per-tool time, and provider-first usage and cost. Label estimates, record unmeasured values as null, keep the legacy numbers' meaning, and persist everything through the history store. + +Purpose: MEAS-01 (autopilot side), implementing D-05, D-09, D-10 and D-11 exactly. The benchmark's autopilot arm (D-19, grok-4.3) reads these values through get_session_detail (plan 66-06). +Output: pure pricing and measurement helpers with tests, inline agent-loop stamps guarded by the source pins, and logger persistence plus text export. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +SHARED CONTRACT (verbatim; plan 66-06 projects these fields in get_session_detail): +- `session.iterationTiming[]`: flat rows, last 100 kept. `{ v:1, iteration, startedAt, modelMs, modelRetried, toolMs, toolCount, usage:{ input, output, reasoning, cachedInput }, usageSource:'provider'|'unavailable', costUsd, costSource:'provider'|'estimate'|'unavailable', estimateUsd, estimateDeltaPct, pricingKey, requestedModel, servedModel, provider }`. Numbers are finite and non-negative or null. toolMs is null until the first measured tool. +- `session.measurementSummary`: `{ v:1, iterations, modelMsTotal, toolMsTotal, toolCount, usage:{ input, output, reasoning, cachedInput }, usageIterations, usageSource:'provider'|'mixed'|'unavailable', costUsd, costIterations, costSource:'provider'|'estimate'|'mixed'|'unavailable', requestedModel, servedModels:[<=10 strings] }`. Totals stay null until the first finite value arrives. +- `actionHistory[].durationMs`: a finite ms value on executed tools; absent on denied or malformed calls. The persisted value is null when absent. +- Legacy MCP rows (written by plan 66-03's recorder) carry `timing:{v:1, receivedAt, completedAt, fsbMs}` and `durationMs`. This plan's logger must persist both. +- No stored key contains "token" (the dispatcher's secret filter drops /token/i keys), and no key is named `stepName` (tests/lattice-step-emitter-smoke.test.js pin). + +Existing code (verified 2026-09-29): +- cost-tracker.js: MODEL_PRICING 23-54; estimateCost 67-92 (Grok fallback 85-88); CostTracker.record 128-135; module.exports 171-173. +- tool-use-adapter.js extractUsage 312-337: anthropic usage.input_tokens/output_tokens; gemini usageMetadata.promptTokenCount/candidatesTokenCount; the default OpenAI-compatible branch uses usage.prompt_tokens/completion_tokens. +- agent-loop.js: + - resolvers at 155-165 (`var _al_estimateCost = ...`; `var _al_CostTracker = ...`), with `_al_costTracker = require('./cost-tracker.js')` at ~69; + - `var apiCallStartTime = Date.now();` ~1936, then `var response;`; + - inner provider try with the one-time image-retry catch ~1953-1979; + - `markDelivered` block ~1980-1985 and debug log ~1986-1992; + - usage block `// h. Extract and accumulate usage` ~2005-2029 ending `session.iterationCount = iterNum;`; + - malformed-args branch ~2227-2250 (continue); + - permission hook and denial branch ~2258-2293 (continue); + - `session._currentStepName = 'BEFORE_TOOL_EXECUTION';` ~2298; + - terminal complete/partial/fail branches return early ~2437-2610; + - metrics recorder block with `client: 'FSB Autopilot'` ~2769-2795, followed by the SUCCESS `// ADOPT-04` push ~2797-2812 and `// Emit afterToolExecution hook` ~2814; + - module.exports ~3131-3133. +- action-history.js createActionEvent 38-53 (`durationMs: f.durationMs || 0` at 51). +- automation-logger.js (helpers inside the `else` block of the load guard; cloneReplayRecordForPersistence ~225): `_saveSessionUnlocked` existing branch ~863-920 (action map ~904-916), new branch ~921-976 (action map ~963-973), index ~982-1012 (leave lean); `exportHumanReadable` ~689-714 (must stay after `async getReplayData(`). + +Verified external facts (docs.x.ai, fetched 2026-09-29): +- The retirement took effect May 15, 2026 12:00 PT. grok-4-1-fast-reasoning, grok-4-1-fast-non-reasoning, grok-4-fast-reasoning, grok-4-fast-non-reasoning, grok-4-0709 and grok-3 now serve grok-4.3. grok-code-fast-1 now serves grok-build-0.1. +- Prices (<200k-token tier): grok-4.3 costs $1.25 in, $0.20 cached in, $2.50 out; grok-build-0.1 costs $1.00 in, $0.20 cached in, $2.00 out. The 200k+ tier doubles these and is not modelled. +- Cost is carried in `usage.cost_in_usd_ticks`, with 1e10 ticks = $1. FSB's autopilot path is non-streaming, so the field is on the response body. + + + + + + + Task 1: Price rows, provider-first usage/cost readers, and iteration-row helpers in cost-tracker.js + extension/ai/cost-tracker.js, tests/cost-tracker.test.js, tests/fixtures/bench/xai-chat-completion.json + + - extension/ai/cost-tracker.js (whole file, 173 lines) + - tests/cost-tracker.test.js (whole file). TRIPWIRE: Grok prices pinned at lines 52-54, 73-76, 86, 92, 134-147, 155, 176-183, 190-197, 202-221, 228-233; the unknown-model fallback currently reads the grok-4-1-fast-reasoning row. + - extension/ai/tool-use-adapter.js lines 305-337 (provider usage shapes) + - mcp/src/tools/pricing.ts lines 68-74, 152-158, 246-275 (labelled nullable result and finite-token gate; read-only reference, never edit) + - extension/utils/delegation-event-store.js lines 274-282 and 551-581 (null-not-0 idiom) + - tests/fixtures/model-discovery-responses.js lines 1-28 (synthetic-fixture header convention) + + + - MODEL_PRICING['grok-4.3'] and ['grok-4.3-latest'] are {input:1.25, output:2.50}. 'grok-build-0.1' and 'grok-code-fast-1' are {1.00, 2.00}. 'grok-4-1-fast', 'grok-4-1-fast-reasoning', 'grok-4-1-fast-non-reasoning', 'grok-4-fast-reasoning', 'grok-4-fast-non-reasoning', 'grok-4-0709' and 'grok-3' are {1.25, 2.50}. The first three keys of MODEL_PRICING are 'grok-4.3', 'grok-4.3-latest', 'grok-build-0.1'. + - estimateCost('grok-4.3', 1e6, 1e6) === 3.75 (today it wrongly returns 18.00 via the 'grok-4' substring). estimateCost('grok-4-1-fast-reasoning', 1e6, 1e6) === 3.75. The existing prefix-match case (grok-4-1-fast-reasoning followed by a version suffix, as in tests/cost-tracker.test.js line 86) gives 1.25 for 1e6 input and 0 output. + - estimateCost('totally-unknown-model', 1e6, 1e6) === 0.70, from the frozen LEGACY_UNKNOWN_MODEL_PRICING that no longer reads any Grok row. estimateCost(null, 0, 0) === 0. The lmstudio provider gives 0. + - resolvePricingKey('grok-4.3') === 'grok-4.3'; resolvePricingKey('x-ai/grok-4.3') === 'grok-4.3'; resolvePricingKey('totally-unknown-model') === null. + - readProviderUsage(xaiFixture, 'xai') returns {input:12000, output:350, reasoning:420, cachedInput:8000, providerCostUsd:0.008525, usageSource:'provider', servedModel:'grok-4.3'}. + - readProviderUsage({}, 'xai') returns all numbers null, usageSource 'unavailable', servedModel null. Non-finite or negative fields become null. + - The anthropic shape {usage:{input_tokens:10, output_tokens:5, cache_read_input_tokens:3}} maps to input 10, output 5, cachedInput 3, reasoning null. The gemini shape maps promptTokenCount, candidatesTokenCount, thoughtsTokenCount and cachedContentTokenCount. The openrouter shape uses a finite `usage.cost` as providerCostUsd. + - estimateCostDetailed('totally-unknown-model', usage, 'xai') returns {usd:null, source:'unavailable', pricingKey:null}. + - estimateCostDetailed('grok-4.3', {input:1e6, output:1e6, reasoning:1e6}, 'xai').usd === 6.25 (reasoning billed at the output rate). For provider 'openai', reasoning is not added. + - estimateCostDetailed with input null returns source 'unavailable'. The lmstudio provider gives {usd:0, source:'estimate', pricingKey:'lmstudio-local'}. + - buildIterationTimingRow with a provider cost gives costSource 'provider' and estimateDeltaPct = round4((estimate - provider)/provider). Without a provider cost but with an estimate, it gives 'estimate'. With neither, it gives costUsd null and 'unavailable'. A non-finite modelMs becomes null. + - appendIterationTiming keeps the last `cap` rows (default 100), keeps summary totals across all appended rows, and sets usageSource/costSource to 'provider', 'estimate', 'mixed' or 'unavailable' by the rules in . It returns the stored row object. + - addIterationToolTime(session, row, 412) turns row.toolMs from null to 412 and toolCount to 1, and updates summary.toolMsTotal and toolCount. Bad inputs (null row, NaN, negative) change nothing and do not throw. + - A recursive key scan of a row and of the summary finds no key matching /token/i and no key named stepName. + - The CostTracker record/total/checkBudget assertions are recomputed with the new Grok rows (1M/1M = 3.75; 500k/200k = 1.125; 1M/500k = 2.50), keeping the same or a higher assertion count. + + +1. MODEL_PRICING (D-11): insert `'grok-4.3'`, `'grok-4.3-latest'` (1.25/2.50) and `'grok-build-0.1'` (1.00/2.00) as the first three keys. Substring matching iterates in insertion order, so they must precede `'grok-4'`. Set these to 1.25/2.50: `'grok-4-0709'`, `'grok-4-1-fast-reasoning'`, `'grok-4-1-fast-non-reasoning'`, `'grok-4-fast-reasoning'`, `'grok-4-fast-non-reasoning'`, `'grok-3'`, `'grok-4-1-fast'`. Set `'grok-code-fast-1'` to 1.00/2.00. Leave every other row unchanged. + Add a comment block citing the May 15, 2026 12:00 PT retirement, the two docs.x.ai pages, the <200k tier, and "provider-reported cost is authoritative (D-10)". Do NOT touch extension/utils/analytics.js or the table in extension/background.js (orchestrator decision 6). +2. Add `var LEGACY_UNKNOWN_MODEL_PRICING = Object.freeze({ input: 0.20, output: 0.50 });`, commented as the pre-Phase-66 display fallback for unknown models, decoupled from the Grok rows. Measurement records use estimateCostDetailed, which reports unknown models as unavailable (D-10). + Add `resolvePricingKey(model)`: exact key first, then the existing startsWith/includes loop over keys in insertion order; returns null when unmatched. Refactor `estimateCost` to use it and fall back to LEGACY_UNKNOWN_MODEL_PRICING. Keep the lmstudio early return. CostTracker is unchanged. +3. Add `var XAI_TICKS_PER_USD = 1e10;`. Then add, in the file's ES5 style (var, function): + - `readProviderUsage(response, providerKey)` + - `estimateCostDetailed(model, usage, provider)`: reasoning is billed only when provider is 'xai' or 'gemini' and reasoning is finite; the result carries a boolean `reasoningBilled`. + - `buildIterationTimingRow(fields)`, where fields are `{iteration, startedAt, modelMs, modelRetried, usage, estimate, requestedModel, provider}`. + - `appendIterationTiming(session, row, cap)` + - `addIterationToolTime(session, row, durationMs)` + + All of them follow the list and the contract. Use a local helper for finite non-negative numbers (null otherwise). Never return an invented 0 except the lmstudio local estimate. +4. Extend the `module.exports` object with the new functions and the two constants. These names become service-worker globals through `importScripts('ai/cost-tracker.js')`; confirm with `grep -rnw extension --include='*.js'` that each is unused elsewhere. +5. Create `tests/fixtures/bench/xai-chat-completion.json`: a synthetic non-streaming xAI chat.completion body with model 'grok-4.3', one tool_call choice, and usage `{prompt_tokens:12000, completion_tokens:350, total_tokens:12770, prompt_tokens_details:{cached_tokens:8000}, completion_tokens_details:{reasoning_tokens:420}, cost_in_usd_ticks:85250000}`. Add a top-level `_fixture` string saying the body is synthetic, shaped per docs.x.ai cost-tracking, and that no network call was made. +6. Update tests/cost-tracker.test.js. Recompute every Grok-priced expectation from the new rows. Keep the unknown-model assertion at 0.70 and relabel it `unknown model uses the frozen legacy display fallback (decoupled from Grok rows)`. Add cases for every behavior above, loading the fixture with `require('./fixtures/bench/xai-chat-completion.json')`. Keep the file's plain-node assert style and exit code. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/ai/cost-tracker.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-04): price grok-4.3 correctly and add provider-first usage and cost labels`. No Co-Authored-By or AI attribution. + + + node tests/cost-tracker.test.js + + + - `node tests/cost-tracker.test.js` exits 0. + - `node -e "const c=require('./extension/ai/cost-tracker.js');if(Object.keys(c.MODEL_PRICING).slice(0,3).join()!=='grok-4.3,grok-4.3-latest,grok-build-0.1')process.exit(1);if(Math.abs(c.estimateCost('grok-4.3',1e6,1e6)-3.75)>1e-9)process.exit(2);if(c.estimateCostDetailed('totally-unknown-model',{input:1,output:1,reasoning:null,cachedInput:null},'xai').source!=='unavailable')process.exit(3)"` exits 0. + - `git diff --quiet HEAD -- extension/utils/analytics.js extension/background.js` (other price tables untouched). + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/ai/cost-tracker.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + The pin target is priced correctly. Provider cost is read first. Unknown models are "unavailable" in measurement. Legacy numbers keep working, and the tests are updated in the same commit. + + + + Task 2: Inline model and tool timing in agent-loop.js, null-default durationMs, and source-contract assertions + extension/ai/agent-loop.js, extension/ai/action-history.js, tests/action-history.test.js, tests/cost-tracker-ordering.test.js + + - extension/ai/agent-loop.js lines 60-170, 1930-2070, 2195-2310, 2425-2475, 2760-2830, 3125-3135 + - extension/ai/action-history.js lines 20-60 and 215-226 + - tests/action-history.test.js lines 55-70 (TRIPWIRE at 67: default durationMs currently asserted 0) + - tests/cost-tracker-ordering.test.js (whole file; you append a Phase 66 section) + - TRIPWIRE suites and what they pin in agent-loop.js: + - tests/lattice-survivability-smoke.test.js ~371-373 and ~503-511: exactly 8 `setTimeout` tokens, comments included; 4 `session._nextIterationTimer = setTimeout` sites; no `_currentStepName` inside a timer lambda. + - tests/mcp-philosophy-parity-smoke.test.js ~373-398: the same 8-token count; no recordDispatch inside a timer lambda. + - tests/agent-loop-iterator-guard.test.js ~49-69: 3 schedule lines byte-exact, 4 call sites. + - tests/lattice-step-emitter-smoke.test.js ~157-167 and ~246-250: exactly one `stepName: 'LLM_TURN'`, one `stepName: 'TOOL_DISPATCH'`, and no other `stepName: '...'` literal. + - tests/cost-tracker-ordering.test.js: its anchors. + - tests/meta-cognitive-tracker.test.js ~40-48: markers. + - tests/ownership-leakage-regression.test.js ~334-335: substrings. + - tests/run-task-cleanup-paths.test.js: `fsbBroadcastAutomationLifecycle(`. + - tests/turn-result.test.js:50: lastTurnResult.durationMs is session-elapsed and must not be repurposed. + + +In extension/ai/action-history.js, change createActionEvent's field to `durationMs: Number.isFinite(f.durationMs) ? f.durationMs : null`, so a measured 0 stays 0 and an absent value becomes null (D-05, orchestrator decision 5). In the same commit, update tests/action-history.test.js line 67 to expect null (label `default durationMs is null (unmeasured, never 0)`). Add assertions that `createActionEvent({durationMs:125}).durationMs === 125` and `createActionEvent({durationMs:0}).durationMs === 0`. + +In extension/ai/agent-loop.js (inline stamps only, per research Pattern 4; no new module, so the importScripts pins 333/329 stay untouched): +1. After the existing `var _al_CostTracker = ...` resolver line, add five resolvers in the same idiom, each with a `null` fallback, never a 0-returning function: `_al_readProviderUsage`, `_al_estimateCostDetailed`, `_al_buildIterationTimingRow`, `_al_appendIterationTiming`, `_al_addIterationToolTime`. Each resolves the same-named global or `_al_costTracker?.`. +2. In runAgentIteration, directly after `var apiCallStartTime = Date.now();`, declare `var _timingModelRetried = false;` and `var _timingModelMs = null;`. +3. In the image-rejection catch, directly after `if (!imageFallbackCode) throw imageRequestError;`, set `_timingModelRetried = true;`. +4. Directly after the inner provider try/catch closes, and before the `if (pendingScreenshotAttachments.length > 0` markDelivered block, set `_timingModelMs = Date.now() - apiCallStartTime;`. Leave the existing debug and error logs unchanged. +5. Directly after `session.iterationCount = iterNum;` (end of step h), add: + - `var _timingRow = null;` + - a block guarded by all four `_al_` helpers being truthy, wrapped in try/catch with the comment `/* measurement is diagnostic-only */`, that computes `var _timingUsage = _al_readProviderUsage(response, providerKey);` and then `_timingRow = _al_appendIterationTiming(session, _al_buildIterationTimingRow({ iteration: iterNum, startedAt: apiCallStartTime, modelMs: _timingModelMs, modelRetried: _timingModelRetried, usage: _timingUsage, estimate: _al_estimateCostDetailed(model, _timingUsage, providerKey), requestedModel: model, provider: providerKey }), 100);`. + + Appending here means the row exists on every path: end_turn, tool loops, and the early-returning complete_task/partial_task/fail_task branches. +6. Directly before `session._currentStepName = 'BEFORE_TOOL_EXECUTION';`, which runs after the permission hook returned non-denied, add `var _timingToolStartedAt = Date.now();`. +7. At the SUCCESS push: the `// ADOPT-04` block that immediately follows the metrics block containing `client: 'FSB Autopilot'`. + - Before it, add `var _timingToolMs = Date.now() - _timingToolStartedAt;`. + - Add `durationMs: _timingToolMs` to BOTH object literals in that block: the `session._actionHistory.push({...})` literal and the `_al_createActionEvent({...})` literal. + - After the block and before `// Emit afterToolExecution hook`, call `_al_addIterationToolTime(session, _timingRow, _timingToolMs)`, guarded by `_al_addIterationToolTime && _timingRow` and wrapped in try/catch with the same diagnostic-only comment. + - Do NOT add durationMs to the malformed-args or permission-denied pushes. +8. Do not change `var inputTokens = usage.input || 0;` and the rest of the legacy usage block, the AFTER_API_RESPONSE payload, lastTurnResult, the metrics recordDispatch block, or module.exports. Add no timer, no word "setTimeout" in any comment, no `stepName:` literal, and no recordDispatch call. + +Append a `--- Phase 66 measurement wiring ---` section to tests/cost-tracker-ordering.test.js. It asserts on agentLoopSource: +- `_al_appendIterationTiming(session` occurs exactly once, after `var usage = _extractUsage(response, providerKey);` and before `// i. Check if AI is done`; +- `var _timingToolStartedAt = Date.now();` occurs once and precedes `session._currentStepName = 'BEFORE_TOOL_EXECUTION';`; +- `durationMs: _timingToolMs` occurs exactly 2 times; +- the slice between `if (call.argsParseError) {` and the first following `continue; // Skip to next tool` contains no `durationMs`, and the same holds for the slice between `if (denialResult && denialResult.denial) {` and its following `continue; // Skip to next tool`; +- `var inputTokens = usage.input || 0;` is still present (D-02). + +Run every agent-loop suite: `node tests/action-history.test.js && node tests/cost-tracker.test.js && node tests/cost-tracker-ordering.test.js && node tests/turn-result.test.js && node tests/lattice-survivability-smoke.test.js && node tests/mcp-philosophy-parity-smoke.test.js && node tests/agent-loop-iterator-guard.test.js && node tests/lattice-step-emitter-smoke.test.js && node tests/meta-cognitive-tracker.test.js && node tests/ownership-leakage-regression.test.js && node tests/run-task-cleanup-paths.test.js && node tests/lattice-provider-bridge-smoke.test.js && node tests/openai-agent-request.test.js && node tests/agent-loop-empty-contents.test.js && node tests/lmstudio-agent-request.test.js && node tests/stuck-action-repetition.test.js && node tests/goal-progress-tracker.test.js && node tests/provider-parity.test.js && node tests/capability-autopilot-parity.test.js && node tests/tool-executor-readonly.test.js && node tests/tool-definitions-parity.test.js`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/ai/agent-loop.js extension/ai/action-history.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit (source and paired pins together): `feat(66-04): record autopilot model and tool time per iteration`. No Co-Authored-By or AI attribution. + + + node tests/action-history.test.js && node tests/cost-tracker-ordering.test.js && node tests/lattice-survivability-smoke.test.js && node tests/mcp-philosophy-parity-smoke.test.js && node tests/agent-loop-iterator-guard.test.js && node tests/lattice-step-emitter-smoke.test.js && node tests/meta-cognitive-tracker.test.js && node tests/ownership-leakage-regression.test.js && node tests/run-task-cleanup-paths.test.js && node tests/turn-result.test.js && node tests/lattice-provider-bridge-smoke.test.js && node tests/openai-agent-request.test.js && node tests/agent-loop-empty-contents.test.js + + + - `node -e "const s=require('fs').readFileSync('extension/ai/agent-loop.js','utf8');if((s.match(/setTimeout/g)||[]).length!==8)process.exit(1);if((s.match(/stepName: '/g)||[]).length!==2)process.exit(2);if((s.match(/durationMs: _timingToolMs/g)||[]).length!==2)process.exit(3)"` exits 0. + - `node -e "const s=require('fs').readFileSync('extension/ai/action-history.js','utf8');if(!s.includes('durationMs: Number.isFinite(f.durationMs) ? f.durationMs : null'))process.exit(1)"` exits 0. + - All suites in exit 0, plus `node tests/lmstudio-agent-request.test.js`, `node tests/stuck-action-repetition.test.js`, `node tests/goal-progress-tracker.test.js`, `node tests/provider-parity.test.js`, `node tests/capability-autopilot-parity.test.js`, `node tests/tool-executor-readonly.test.js` and `node tests/tool-definitions-parity.test.js`. + - `git diff --quiet HEAD -- extension/background.js` (importScripts pins untouched; no new module). + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/ai/agent-loop.js extension/ai/action-history.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + Autopilot iterations record model time and summed tool time on every path. Executed actions carry durationMs. All agent-loop tripwires hold in the same commit. + + + + Task 3: Persist measurement fields through saveSession and add timing lines to the autopilot text export + extension/utils/automation-logger.js, tests/automation-logger-mcp-retention.test.js + + - extension/utils/automation-logger.js lines 1-260 (load guard; helpers such as cloneReplayRecordForPersistence ~225), 620-720 (getReplayData then exportHumanReadable), 836-1020 (_saveSessionUnlocked both branches and index) + - tests/automation-logger-mcp-retention.test.js lines 1-145 (harness loadLogger, storage mock, first case pattern including how session logs are seeded so saveSession proceeds) + - .planning/phases/66-baseline-and-session-timing/66-06-PLAN.md "BASIS LABELS" (the legacy cost-basis rule that `computeLegacyCostBasis` implements verbatim; I-2) + - tests/lattice-replay-background-contract.test.js lines 105-120 (TRIPWIRE: `async getReplayData(` must precede `async exportHumanReadable(`) + - tests/delegation-sidepanel-tab-follow.test.js line ~470 (TRIPWIRE: `var AUTOMATION_LOGS_PERSISTED_TAIL = (\d+);` >= 200, untouched region) + + + - New-session branch: saving {actionHistory:[{tool:'click', result:{success:true}, timestamp, durationMs:412}, {tool:'type_text', result:{success:true}, timestamp}], iterationTiming:[r1,r2], measurementSummary:S} persists durationMs 412 and null respectively. iterationTiming deep-equals [r1,r2], and measurementSummary deep-equals S. + - Existing-session branch: a second save with a new iterationTiming (3 rows) and summary replaces the stored ones. A second save without these fields keeps the stored ones. + - Cap: a 150-row iterationTiming persists the last 100 rows. + - A legacy MCP row with valid timing {v:1, receivedAt:1000, completedAt:1400, fsbMs:400} and durationMs 400 persists both. A row with invalid timing (completedAt < receivedAt) persists no timing key and durationMs null. + - The index entry gains exactly one field, `totalCostBasis` (I-2). It still holds no iterationTiming or measurementSummary, and every other index field is unchanged. + - Save-time cost basis (I-2): the saved session record and its fsbSessionIndex entry carry the same `totalCostBasis`: + - totalCost 0.70 with measurementSummary.costSource 'unavailable' gives 'unavailable'; + - a summary with costSource 'estimate' where any iterationTiming row has `pricingKey: null` gives 'unavailable'; + - a summary with costSource 'provider' whose rows all carry a pricingKey gives 'estimate', including when totalCost is 0; + - no summary with totalCost 0.12 gives 'estimate', and no summary with totalCost 0 gives 'unavailable'. + A second save with a new summary recomputes the value in both places. A second save without summary fields keeps the stored summary, and so the same value. + - exportHumanReadable for a session with a measurementSummary includes a line starting `Timing: model ` and a line starting `Iteration 1: model `. For a session with a durationMs row, it includes a line matching /^Call \d+: \S+ \| \d+ ms$/. A session with neither includes `Timing: not recorded for this session`. + - The existing retention and concurrency cases pass unchanged. + + +Inside the load-guard `else` block, next to cloneReplayRecordForPersistence, add these helpers. Confirm each name is unused in extension/ first. +- `cloneIterationTimingForPersistence(rows)`: returns undefined when rows is not an array. Otherwise it JSON-clones the last 100 plain-object rows. +- `cloneMeasurementSummaryForPersistence(summary)`: returns a JSON clone for a plain object, else undefined. +- `projectPersistedActionTiming(a)`: returns `{ durationMs: Number.isFinite(a.durationMs) ? a.durationMs : null }`, plus `timing: {v:1, receivedAt, completedAt, fsbMs}` only when a.timing passes the contract validation (safe non-negative integers, completedAt >= receivedAt, fsbMs recomputed). +- `computeLegacyCostBasis(session)` (I-2): implements the cost half of the BASIS LABELS contract in 66-06-PLAN.md verbatim. The legacy totalCost is a price-table sum, so the result is never 'provider'. With a plain-object measurementSummary, return 'unavailable' when `measurementSummary.costSource === 'unavailable'` or any stored iterationTiming row has `pricingKey === null`, else 'estimate' (even when totalCost is 0). Without a summary, return 'estimate' when totalCost is finite and > 0, else 'unavailable'. +- `formatSessionTimingLines(session)`: returns an array of strings. + - With measurementSummary, it emits `Timing: model ms over iterations | tools ms over calls | cost <$ with 6 decimals + ' (' + costSource + ')' | 'unavailable'>`. Then, per iterationTiming row, it emits `Iteration : model ms<', retried' when modelRetried> | tools ms ( calls) | cost | served `. + - Then, for the last 100 actionHistory rows with a finite durationMs, it emits `Call : | ms`. + - If neither kind of data exists, it emits the single line `Timing: not recorded for this session`. + +In `_saveSessionUnlocked`: +- In BOTH action-row `.map(a => ({...}))` projections, add the spread of `projectPersistedActionTiming(a)`. +- In the existing-session branch, set `existing.iterationTiming = cloneIterationTimingForPersistence(sessionData.iterationTiming) || existing.iterationTiming;` and do the same for measurementSummary, following the `existing.replay = ... || existing.replay || null` precedent. +- In the new-session literal, add `iterationTiming` and `measurementSummary` from the helpers. +- Leave totalCost, totalInputTokens and totalOutputTokens and their `|| 0` behavior unchanged (D-02: legacy numbers keep their meaning; labelled values live in measurementSummary per D-10). +- Right after `const savedSession = sessionStorage[sessionId];` (after both branches have merged), set `savedSession.totalCostBasis = computeLegacyCostBasis(savedSession);`, which persists with fsbSessionLogs. Add `totalCostBasis: savedSession.totalCostBasis` to the index entry literal (I-2). Do not add iterationTiming or measurementSummary to the index entry. Index entries are not shape-pinned by any test (checked: tests read specific fields only), so this lean field breaks no pinned projection. + +In `exportHumanReadable`, after the existing step lines, append `''` followed by `formatSessionTimingLines(session)`. Keep the method order (getReplayData before exportHumanReadable). The file also loads in content scripts and the control panel, so use no service-worker-only API. + +Tests: add cases to tests/automation-logger-mcp-retention.test.js covering every behavior above, using `loadLogger` and `makeStorage`, and seeding session logs the way the first case does so saveSession proceeds. Then run `node tests/automation-logger-mcp-retention.test.js && node tests/lattice-replay-background-contract.test.js && node tests/delegation-sidepanel-tab-follow.test.js && node tests/mcp-session-recorder.test.js`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/utils/automation-logger.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-04): persist autopilot and MCP call timing through session history`. No Co-Authored-By or AI attribution. + + + node tests/automation-logger-mcp-retention.test.js && node tests/lattice-replay-background-contract.test.js && node tests/delegation-sidepanel-tab-follow.test.js && node tests/mcp-session-recorder.test.js + + + - `grep -c "projectPersistedActionTiming(a)" extension/utils/automation-logger.js` prints 2 (both action projections). + - `grep -c "cloneIterationTimingForPersistence(" extension/utils/automation-logger.js` prints at least 3 (definition plus both branches). + - `grep -c "computeLegacyCostBasis(" extension/utils/automation-logger.js` prints at least 2 (definition plus the save), and `grep -c "totalCostBasis" extension/utils/automation-logger.js` prints at least 2 (session record and index entry) (I-2). + - `node -e "const s=require('fs').readFileSync('extension/utils/automation-logger.js','utf8');if(s.indexOf('async getReplayData(')>s.indexOf('async exportHumanReadable('))process.exit(1)"` exits 0. + - All suites in exit 0. + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/utils/automation-logger.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + Measurement fields and per-call durations survive history persistence in both branches. The session record and index entry carry the same save-time totalCostBasis. The autopilot text export shows per-iteration and per-call ms. Legacy totals are unchanged. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| LLM provider response → autopilot loop | The usage and cost fields come from a remote API and may be missing, malformed or hostile. | +| Session object → chrome.storage.local history | Persisted fields are later projected over MCP (plan 66-06). | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-05 | Tampering | readProviderUsage / buildIterationTimingRow | mitigate | Only finite non-negative numbers are accepted; everything else becomes null. servedModel is capped at 128 chars. The helpers run inside try/catch in the loop, so a hostile body cannot break autopilot (D-02). | +| T-66-01 | Information disclosure | iterationTiming / measurementSummary | mitigate | Rows are flat numbers plus model and provider names. No request, response or page text is stored. Key names avoid "token" so the MCP secret filter never has to drop them, and plan 66-06's explicit projection controls exposure. | +| T-66-02b | Repudiation (mislabelled cost) | cost labels | mitigate | Each iteration records costSource/usageSource. Unknown models are 'unavailable', never Grok-priced. estimateDeltaPct records estimate-versus-provider divergence, so a wrong estimate formula is detectable. | +| T-66-DoS | Denial of service | history size | mitigate | iterationTiming is capped at 100 rows per session. Summary totals are O(1). No per-iteration chrome.storage writes are added; persistence happens at the existing saveSession points. | + + + +- `node tests/cost-tracker.test.js && node tests/action-history.test.js && node tests/cost-tracker-ordering.test.js && node tests/automation-logger-mcp-retention.test.js`, plus the full agent-loop pin list from Task 2. +- `git diff --quiet HEAD -- extension/background.js extension/utils/analytics.js mcp/`. + + + +- MEAS-01 (autopilot side): model time, tool time, tokens and cost per iteration are recorded with honest labels and persisted. +- D-05, D-09, D-10 and D-11 are implemented exactly. D-02 legacy behavior is preserved, and every tripwire is updated in the same commit as its source change. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Each extension commit includes its paired tripwire and test updates. Never stage files under mcp/. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-04-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-04-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-04-SUMMARY.md new file mode 100644 index 000000000..7ef161f35 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-04-SUMMARY.md @@ -0,0 +1,266 @@ +--- +phase: 66-baseline-and-session-timing +plan: 04 +subsystem: observability +tags: [autopilot, per-iteration-timing, provider-cost, xai, cost-tracker, automation-logger, source-pins, d-02] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 02) + provides: start-commit baseline and the W11 regression sweep with 66-02-SWEEP-BASELINE.json + - phase: 66-baseline-and-session-timing (plan 03) + provides: legacy MCP actionHistory rows carrying a v1 timing and durationMs, handed to automationLogger.saveSession +provides: + - "D-11 Grok rows: grok-4.3 / grok-4.3-latest / grok-build-0.1 first in MODEL_PRICING; retired slugs at $1.25/$2.50, grok-code-fast-1 at $1.00/$2.00" + - "LEGACY_UNKNOWN_MODEL_PRICING (frozen 0.20/0.50) decoupled from the Grok rows; resolvePricingKey (own keys only)" + - "readProviderUsage: provider-first usage and cost (xAI cost_in_usd_ticks / 1e10, OpenRouter usage.cost), null when unmeasured" + - "estimateCostDetailed: labelled estimate; unknown model or missing usage -> 'unavailable'; reasoning billed at the output rate for xai and gemini" + - "buildIterationTimingRow / appendIterationTiming / addIterationToolTime: the v1 session.iterationTiming rows (last 100) and session.measurementSummary" + - "agent-loop.js: model time per iteration (modelRetried on the image retry), tool time from permission check to success push, actionHistory[].durationMs on executed calls only" + - "createActionEvent durationMs defaults to null" + - "automation-logger.js: iterationTiming, measurementSummary, durationMs and legacy MCP timing persisted in both save branches; save-time totalCostBasis on the record and its index entry (I-2); timing lines in the autopilot text export" + - "tests/fixtures/bench/xai-chat-completion.json: synthetic xAI body with cost ticks, reasoning and cached input" +affects: [66-06, 66-08, benchmark autopilot arm, get_session_detail measurement projection, list_sessions totalCostBasis] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Provider-reported numbers first, price table second, and every value carries a source label (provider / estimate / mixed / unavailable)" + - "Inline agent-loop stamps call pure cost-tracker helpers through _al_ resolvers with null fallbacks, so a missing helper skips measurement instead of inventing a 0" + - "Measurement blocks sit in try/catch (/* measurement is diagnostic-only */) and never gate what the loop decides or executes" + - "Save-time basis labels: computed once when the record is written and copied to the lean index entry" + +key-files: + created: + - tests/fixtures/bench/xai-chat-completion.json + modified: + - extension/ai/cost-tracker.js + - extension/ai/agent-loop.js + - extension/ai/action-history.js + - extension/utils/automation-logger.js + - tests/cost-tracker.test.js + - tests/cost-tracker-ordering.test.js + - tests/action-history.test.js + - tests/automation-logger-mcp-retention.test.js + +key-decisions: + - "The legacy estimateCost keeps its 0.20/0.50 unknown-model rate through a frozen constant, so legacy totals for unknown models do not move; only estimateCostDetailed (the measurement path) reports them as unavailable" + - "resolvePricingKey matches own keys only: prototype names ('constructor', '__proto__') and non-string models now fall back instead of yielding NaN or throwing" + - "Executed-tool time that comes out negative (wall clock stepped back) is recorded as null, never a negative number or a clamped 0" + - "The logger persists durationMs only when finite and non-negative, and keeps an MCP timing only when it passes the v1 contract, with fsbMs recomputed" + - "Text-export timing lines print a unit only beside a measured number ('model 2001 ms'), and a word for a missing value ('model n/a', 'tools none')" + - "Each task committed once, tests with source: every touched suite is in the npm test chain and must stay green in every commit; RED was run and recorded before each implementation" + +patterns-established: + - "Plan 66-06 reads session.iterationTiming / session.measurementSummary / actionHistory[].durationMs exactly as the v1 contract in 66-04-PLAN.md describes, and uses the stored totalCostBasis when it is 'estimate' or 'unavailable'" + - "New cost-tracker helpers become service-worker globals through importScripts; agent-loop resolves them with the typeof-global-else-require idiom" + +requirements-completed: [MEAS-01] + +# Metrics +duration: 24min +completed: 2026-09-30 +--- + +# Phase 66 Plan 04: Autopilot Iteration Timing and Provider-First Cost Summary + +**Autopilot iterations now record model time (with a retry mark), summed executed-tool time and provider-first usage and cost with source labels. xAI's `cost_in_usd_ticks` is the cost when present; the price table is a labelled estimate that prices grok-4.3 at $1.25/$2.50 instead of the old $3/$15 substring match, and never prices an unknown model as Grok. Executed calls carry `durationMs` (none on denied or malformed calls). Everything persists through `automationLogger.saveSession`, including a save-time `totalCostBasis` label on the record and its index entry, and the autopilot text export shows per-iteration and per-call milliseconds. Legacy totals and autopilot behaviour are unchanged (D-02).** + +This plan delivers the autopilot slice of MEAS-01 (D-05, D-09, D-10, D-11). `requirements-completed` lists the plan's requirement ID for traceability only. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched, because the orchestrator owns them. + +## Performance + +- **Duration:** 24 min +- **Started:** 2026-09-30T07:06:35Z +- **Completed:** 2026-09-30T07:30Z +- **Tasks:** 3/3 +- **Files:** 9 (4 extension sources and 4 paired test files modified, 1 test fixture created); exactly the plan's `files_modified` + +## Accomplishments + +- **Pricing (D-11, D-10):** + - `grok-4.3`, `grok-4.3-latest` and `grok-build-0.1` are the first three `MODEL_PRICING` keys, so the substring loop can no longer send `grok-4.3` to the legacy `grok-4` row. `estimateCost('grok-4.3', 1e6, 1e6)` is 3.75 (it was 18.00). + - The seven slugs xAI now serves with grok-4.3 bill at $1.25/$2.50. `grok-code-fast-1` bills at $1.00/$2.00. Every other row is unchanged, and so are the tables in `analytics.js` and `background.js` (orchestrator decision 6). + - Unknown models keep the frozen 0.20/0.50 display rate in the legacy numbers (still 0.70 per 1M/1M). The labelled measurement path reports them as `unavailable`. +- **Provider-first readers:** + - `readProviderUsage` maps the OpenAI-compatible, Anthropic and Gemini usage shapes to `{input, output, reasoning, cachedInput}`. It takes xAI cost from `cost_in_usd_ticks / 1e10` and OpenRouter cost from `usage.cost`, and caps `servedModel` at 128 characters. Missing, non-finite, negative and string values become null. + - `estimateCostDetailed` bills reasoning at the output rate only for xai and gemini, where it is reported outside the output count. LM Studio is a labelled $0 estimate. +- **Iteration rows:** + - `buildIterationTimingRow` produces the exact v1 row. Provider cost wins over the estimate, and `estimateDeltaPct = round4((estimate - provider) / provider)`. + - The fixture's delta is 0.9853, which is the cached-input discount the estimate does not model. + - `appendIterationTiming` keeps the last 100 rows and O(1) summary totals that stay null until the first finite value. A source is the rows' shared value, or `mixed` once rows differ. +- **Agent loop (inline, no new module, `background.js` untouched):** + - Five `_al_` resolvers with null fallbacks. + - Model time is stamped once the provider call, and any text-only image retry, resolves. The retry sets `modelRetried`. + - The iteration row is appended right after the legacy usage block, before the end_turn check, so the terminal `complete_task`/`partial_task`/`fail_task` iterations record it too. + - Tool time runs from the permission check to the success push, lands on both push literals, and is folded into the iteration row. + - `setTimeout` still counts 8, the two `stepName` literals are untouched, and the legacy usage lines, hooks, metrics and exports are unchanged. +- **Persistence (D-05, D-09, I-2):** + - Both `saveSession` action projections spread `projectPersistedActionTiming(a)`. + - `iterationTiming` and `measurementSummary` persist in the new-session branch. In the existing-session branch they are replaced, or kept when a save omits them. + - `computeLegacyCostBasis` implements the cost half of the 66-06 BASIS LABELS contract verbatim. The same `totalCostBasis` goes on the record and its index entry, which gains only that field. + - `exportHumanReadable` appends the timing lines after the step lines. + +Sample export tail, from an end-to-end run of the real agent loop and logger (throwaway harness, below): + +``` +Timing: model 54 ms over 2 iterations | tools 0 ms over 1 calls | cost $0.017050 (provider) +Iteration 1: model 28 ms | tools 0 ms (1 calls) | cost $0.008525 (provider) | served grok-4.3 +Iteration 2: model 26 ms | tools none (0 calls) | cost $0.008525 (provider) | served grok-4.3 +Call 1: get_site_guide | 0 ms +``` + +## Task Commits + +1. **Task 1: Price rows, provider-first usage/cost readers, and iteration-row helpers in cost-tracker.js:** `b073fbd5` (feat) +2. **Task 2: Inline model and tool timing in agent-loop.js, null-default durationMs, and source-contract assertions:** `62ac836a` (feat) +3. **Task 3: Persist measurement fields through saveSession and add timing lines to the autopilot text export:** `1bc21ddd` (feat) + +**Plan metadata:** see the `docs(66-04)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `extension/ai/cost-tracker.js`: + - D-11 rows and the retirement comment block. + - Constants `LEGACY_UNKNOWN_MODEL_PRICING` and `XAI_TICKS_PER_USD`. + - `resolvePricingKey` and the refactored `estimateCost`. + - `readProviderUsage`, `estimateCostDetailed`, `buildIterationTimingRow`, `appendIterationTiming` and `addIterationToolTime`. + - Private `_ct*` helpers; every new top-level name was grep-checked as unused in `extension/`. + - `CostTracker` is unchanged. `module.exports` gains the six new functions and two constants. +- `extension/ai/agent-loop.js`: the five resolvers, `_timingModelRetried`/`_timingModelMs`, the h1a measurement block, `_timingToolStartedAt`, `_timingToolMs` with the clock-rollback guard, `durationMs: _timingToolMs` on both success-push literals, and the guarded `_al_addIterationToolTime` call (+52/-2 lines). +- `extension/ai/action-history.js`: `durationMs: Number.isFinite(f.durationMs) ? f.durationMs : null`. +- `extension/utils/automation-logger.js`: + - New helpers `cloneIterationTimingForPersistence`, `cloneMeasurementSummaryForPersistence`, `projectPersistedActionTiming`, `computeLegacyCostBasis` and `formatSessionTimingLines`. + - Both save branches, the save-time basis, the index field, and the export lines. +- `tests/cost-tracker.test.js`: Grok expectations recomputed. The unknown-model case is relabelled and stays 0.70. Cases cover every behaviour in the plan, including recursive `/token/i` and `stepName` key scans. It went from 67 to 299 assertions. +- `tests/cost-tracker-ordering.test.js`: a `--- Phase 66 measurement wiring ---` section with 20 source assertions (placement, the resolver lines, `durationMs` counts, the malformed and denied slices, D-02 anchors). It went from 17 to 37. +- `tests/action-history.test.js`: the default `durationMs` is now null, with measured, zero and invalid cases added. It went from 65 to 71. +- `tests/automation-logger-mcp-retention.test.js`: seven Phase 66 sections covering both branches, the cap, the row filter, MCP timing in both branches, index leanness, the seven basis cases plus recompute and keep, and the export lines. The 82 existing checks are unchanged; the file went from 82 to 138. +- `tests/fixtures/bench/xai-chat-completion.json`: a synthetic body whose ticks equal the true cached-input bill: 4000 × $1.25/M + 8000 × $0.20/M + 770 × $2.50/M = $0.008525. + +## Verification + +| Check | Result | +|-------|--------| +| Task 1 verify `node tests/cost-tracker.test.js` + acceptance `node -e` (first keys, 3.75, unavailable) + `git diff --quiet` on analytics.js/background.js | exit 0 (299/0); all acceptance exits 0 | +| Task 2: the 13 verify suites, the 7 extra acceptance suites and cost-tracker (21 in all) | all exit 0 (action-history 71/0, cost-tracker-ordering 37/0, turn-result 69/0, meta-cognitive 19/0, provider-parity 118/0, tool-definitions-parity 268/0, ...) | +| Task 2 acceptance `node -e` (setTimeout 8, `stepName: '` 2, `durationMs: _timingToolMs` 2; `Number.isFinite(f.durationMs) ? f.durationMs : null` present) and `git diff --quiet HEAD -- extension/background.js` | exit 0 | +| Task 3 verify (retention, lattice-replay-background-contract, delegation-sidepanel-tab-follow, mcp-session-recorder) | exit 0 (138/0, 59 passed, pass, 353/0) | +| Task 3 acceptance greps | `projectPersistedActionTiming(a)` = 2; `cloneIterationTimingForPersistence(` = 3; `computeLegacyCostBasis(` = 2; `totalCostBasis` = 2; getReplayData precedes exportHumanReadable | +| W11 sweeps vs `66-02-SWEEP-BASELINE.json` (7225 held, same as the baseline) | Task 1: 4/4; Task 2: 24/24; Task 3: 6/6. No new failures | +| Plan `` | the four suites exit 0; `git diff --quiet eaa1ed03 HEAD -- extension/background.js extension/utils/analytics.js mcp/` exits 0 | +| Full `npm test` chain, tolerant mode (7225 held by pid 42278) | **311/311 commands exit 0** (13 pretest + 298 test), 204 s. Artifacts: `/tmp/fsb-66-04-chain.{jsonl,log,done}`. The chain rewrote `showcase/angular/public/{llms.txt,llms-full.txt,sitemap.xml}`; they were reverted with `git checkout --` and never committed. No lockfile or `mcp/` file changed, and 7225 is still held by the same pid | + +**End-to-end harness (throwaway, `/tmp/fsb-66-04-agentloop-harness.js`, never committed).** No suite runs `runAgentIteration`, so a harness drove the real `runAgentLoop` in Node with a stubbed xAI bridge. Iteration 1 returned an executed local tool (`get_site_guide`) and a malformed-argument call; iteration 2 returned `complete_task`. +- **Result:** two v1 rows, including the terminal iteration's row. The executed call carries `durationMs` and the malformed call carries none. The summary totals are right. With the real logger loaded, the saved record, the index basis and the export lines are all correct. +- **D-02:** the same run with the Phase 66 helpers removed from `cost-tracker.js`, and the same run against the pre-change `agent-loop.js`/`action-history.js`, produced the same status, two provider calls, `totalCost` 0.03175, `totalInputTokens` 24000 and the same action rows. The only difference was the additive `durationMs` key. + +## Decisions Made + +- **Legacy fallback kept numerically, decoupled structurally.** Orchestrator decision 9 asks for decoupling. A frozen constant keeps the legacy unknown-model rate at its old value, so D-02 legacy totals do not move. The measurement path reports unknown models as `unavailable`. +- **Model time covers the one retry.** When the text-only image retry fires, model time spans both attempts plus the rejection bookkeeping, and `modelRetried: true` says so. This follows research Pattern 4. +- **The requested model is the latest one.** `measurementSummary.requestedModel` tracks the most recent non-null requested model. `servedModels` keeps up to 10 distinct served strings in first-seen order. +- **Export wording.** See key-decisions: a unit is printed only beside a measured number. Call numbers are the row's position in the persisted `actionHistory`, so `Call N` lines up with `Step N`. +- **Commit granularity.** One commit per task, tests with source (see TDD Gate Compliance). + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] A backwards wall-clock step could record a negative tool duration** +- **Found during:** Task 2 +- **Issue:** The plan's `var _timingToolMs = Date.now() - _timingToolStartedAt;` goes straight onto the action row. A clock step during a tool call would persist a negative `durationMs`, which the v1 contract ("finite and non-negative or null") forbids. 66-03 hit the same problem on the MCP side (its Deviation 1). +- **Fix:** One guard line, `if (!(_timingToolMs >= 0)) _timingToolMs = null;`, so the call is recorded as unmeasured, never negative and never a clamped 0. The plan's literal line is kept, and `durationMs: _timingToolMs` still appears exactly twice. `buildIterationTimingRow` already nulls a negative `modelMs`, and `addIterationToolTime` rejects a negative duration. +- **Files modified:** `extension/ai/agent-loop.js` +- **Verification:** the Task 2 acceptance `node -e` exits 0 and the pin suites pass. A throwaway run of the loop shows non-negative durations. +- **Committed in:** `62ac836a` + +**2. [Rule 2 - Missing critical functionality] The persisted durationMs must also be non-negative** +- **Found during:** Task 3 +- **Issue:** The plan's projection was finite-only (`Number.isFinite(a.durationMs) ? a.durationMs : null`), so a negative value from any producer would reach history and `get_session_detail`. +- **Fix:** `projectPersistedActionTiming` keeps `durationMs` only when it is finite and `>= 0`, else null. The text export applies the same rule. +- **Files modified:** `extension/utils/automation-logger.js`, `tests/automation-logger-mcp-retention.test.js` (a -5 case) +- **Verification:** "a non-v1 timing and a negative durationMs are dropped" passes in both save branches. +- **Committed in:** `1bc21ddd` + +**3. [Rule 1 - Bug] Prototype names and non-string models broke the price lookup** +- **Found during:** Task 1 +- **Issue:** The old exact match `MODEL_PRICING[model]` returned `Object.prototype.constructor` for the model `'constructor'`, which gave a NaN cost. A non-string truthy model threw a TypeError at `.toLowerCase()`. +- **Fix:** `resolvePricingKey` requires a non-empty string and matches own keys only. Both cases now resolve to null. The legacy path then uses the display fallback, and the measurement path reports `unavailable`. +- **Files modified:** `extension/ai/cost-tracker.js`, `tests/cost-tracker.test.js` +- **Verification:** the `'constructor'`, `'__proto__'` and `42` cases pass. `estimateCost('constructor', 1e6, 1e6)` is 0.70, not NaN. +- **Committed in:** `b073fbd5` + +**4. [Rule 3 - Blocking] The resolver edit trimmed a space from an adjacent line** +- **Found during:** Task 2 diff review +- **Issue:** The multi-line edit that inserted the resolvers dropped the space in `var _al_TranscriptStore = (`, giving `=(`. +- **Fix:** Restored the line byte-for-byte before committing. The committed diff for that region is purely additive. +- **Files modified:** `extension/ai/agent-loop.js` +- **Committed in:** `62ac836a` (never committed in the broken form) + +--- + +**Total deviations:** 4 auto-fixed (2 Rule 1, 1 Rule 2, 1 Rule 3). +**Impact on plan:** each one tightens the plan's own contract (never negative, never NaN, never invented) or keeps the diff clean. There is no scope creep, and the footprint is exactly `files_modified`. + +## TDD Gate Compliance + +Tasks 1 and 3 are `tdd="true"`, but the plan type is `execute` and every touched suite is in the `npm test` chain. A standalone RED commit would leave the chain red at that commit, so RED was run and recorded before each implementation, and each task has one `feat(66-04)` commit with the tests and source together. This matches 66-03. + +| Task | RED (tests first, unchanged source) | GREEN | +|------|-------------------------------------|-------| +| 1 | cost-tracker 66 pass / 58 fail (every failure in a repriced or new case) | 299 / 0 | +| 2 (tests first, not tdd) | action-history 66 / 5; cost-tracker-ordering 22 / 15 | 71 / 0; 37 / 0 | +| 3 | retention: the 82 existing checks and 9 new ones pass, 11 new checks fail, then a new case aborts on the missing rows. That case was hardened to report FAIL instead of throwing | 138 / 0 | + +## Issues Encountered + +- **LM Studio legacy basis (observation for 66-06; pre-existing, not changed).** The loop's legacy `session._costTracker.record(model, inputTokens, outputTokens)` does not pass the provider. That line is unchanged, per D-02 and the plan's "do not change the legacy usage block". An LM Studio session's legacy `totalCost` is therefore priced at the unknown-model display rate unless its model name matches a table row, exactly as before Phase 66. + - The measurement path records LM Studio correctly: a $0 estimate with pricingKey `lmstudio-local`. + - Implemented verbatim, the BASIS LABELS rule then labels that legacy total `'estimate'`, as the contract explicitly intends ("even when the total is 0, for example a local lmstudio run"). + - If 66-06 or the deferred default-model fix wants the label to reflect the non-zero legacy total, the fix belongs in that contract or in the legacy `record()` call, not here. +- **Sub-millisecond tools record 0.** Local tools such as `get_site_guide` finish within one `Date.now()` tick, so `durationMs` 0 is a measured value, not an invented one. +- graphify's graph is stale for these files, as 66-PATTERNS.md notes. It was used for orientation only; every line anchor was re-verified in the working tree. +- The pre-commit `worktree-agent-*` branch allow-list was skipped because the orchestrator directed commits on `Jev`, as in 66-02 and 66-03. The cwd-drift sentinel and the protected-branch deny-list ran before each commit. + +## Known Stubs + +None. Every new field is written by the live autopilot loop, persisted by the logger, and exercised end to end by the throwaway harness. Plan 66-06 adds the `get_session_detail` projection that reads it over MCP; that is the planned next step, not a stub here. + +## Threat Model Coverage + +- **T-66-05 (tampering):** + - Only finite, non-negative provider numbers are accepted, and `servedModel`/`requestedModel` are capped at 128 characters. + - Both agent-loop measurement blocks are in try/catch, and a hostile body leaves the row all-null (hostile-shape tests in cost-tracker). + - Legacy MCP timing is re-validated with `fsbMs` recomputed. +- **T-66-01 (information disclosure):** rows hold only numbers, model and provider names, and labels, with no request, response or page text. Recursive key scans find no `/token/i` key and no `stepName` key. The index carries only the basis label. +- **T-66-02b (mislabelled cost):** every row carries `usageSource`/`costSource`. Unknown models are `unavailable`, never Grok-priced. `estimateDeltaPct` records estimate-versus-provider drift. +- **T-66-DoS:** 100-row cap, O(1) summary updates, and no new `chrome.storage` writes. Persistence rides the existing `saveSession` calls. + +No new security surface beyond the plan's threat register. + +## User Setup Required + +None. + +## Next Phase Readiness + +- **66-06:** + - `session.iterationTiming`, `session.measurementSummary` and `actionHistory[].durationMs` follow the v1 contract in 66-04-PLAN.md ``. + - Every session saved from now on carries `totalCostBasis` on its record and its `fsbSessionIndex` entry, always `'estimate'` or `'unavailable'`. + - Legacy MCP rows carry `timing` and `durationMs` through history. +- **66-08 (autopilot arm):** grok-4.3 is priced correctly, and xAI ticks are recorded as the provider cost with the estimate delta beside them. +- **Blockers:** none. + +## Self-Check: PASSED + +- All 9 plan files exist on disk (8 modified, 1 created), as does this SUMMARY. +- All three task commits exist: `b073fbd5`, `62ac836a`, `1bc21ddd`. +- No commit in `eaa1ed03..HEAD` has a Co-Authored-By or AI attribution line. +- `git diff --name-only eaa1ed03 HEAD -- .planning/STATE.md .planning/ROADMAP.md .planning/REQUIREMENTS.md mcp/` is empty. +- The working tree was clean apart from this SUMMARY before it was committed. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-05-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-05-PLAN.md new file mode 100644 index 000000000..e34a19e39 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-05-PLAN.md @@ -0,0 +1,328 @@ +--- +phase: 66-baseline-and-session-timing +plan: 05 +type: execute +wave: 2 +depends_on: ["66-02"] +files_modified: + - scripts/bench/lib/tasks.mjs + - scripts/bench/fixture-server.mjs + - scripts/bench/lib/host-stream.mjs + - tests/bench-tasks.test.mjs + - tests/bench-fixture-server.test.mjs + - tests/bench-claude-stream.test.mjs + - tests/fixtures/bench/tasks-sample/tasks/core/core-sample-save.json + - tests/fixtures/bench/tasks-sample/tasks/live/live-sample-read.json + - tests/fixtures/bench/tasks-sample/fixtures/sample/index.html + - tests/fixtures/bench/claude-stream-result.jsonl + - tests/fixtures/bench/cursor-stream-result.jsonl + - tests/fixtures/bench/claude-stream-usage-cap.jsonl + - tests/fixtures/bench/claude-error-text-cases.json +autonomous: true +requirements: [MEAS-03] +user_setup: [] + +must_haves: + truths: + - "Every task JSON is validated against an exact-key schema before use. Core tasks run only from local fixture files; live tasks run only against absolute https URLs (D-15, T-66-06)." + - "A task's identity is id + version + fixtureSha256, computed over the canonical task JSON and every fixture file's bytes (D-18)." + - "The fixture server binds 127.0.0.1 only, serves fixtures under a per-run /r// prefix, logs every request (method, path, query, parsed body) per run, and blocks path traversal. Its evidence endpoint requires a runner-held secret, so an agent cannot read or rewrite evidence (D-13, D-17, T-66-04)." + - "The Claude Code argv mirrors the pinned v0.9.91 task-mode flags from claude-profile.ts, plus an explicit --model, with no permission-bypass flags (D-14, orchestrator decision 3)." + - "Host metrics come only from the driver's stream: Claude's init/result give tool-loading mode, durations, turns, usage and cost, with cost labelled 'list-price estimate, not billed' when apiKeySource is none or costBasis is list. Cursor reports duration only, and tokens and cost are 'unavailable (host does not report)' (D-12, D-18)." + - "The Claude summary exposes usage-cap, rate-limit and auth signals (`rateLimit`, `errorKind` with values usage_limit, rate_limit, auth, api_error, other) without storing any result text. The runner (66-08) can then classify a Claude usage cap as `agent_cap` and a login or key failure as `agent_auth`, and pause the campaign instead of counting failed runs (W4, W-A). Numeric codes classify only with an `HTTP` or `status` prefix, so text like '500 ms' or the bare word 'provider' never triggers a pause." + - "No file under mcp/ changes (D-01). The bench fixtures live under tests/fixtures/bench/, not tests/fixtures/agent-streams/." + artifacts: + - path: "scripts/bench/lib/tasks.mjs" + provides: "validateTask, loadTasks, fixtureSha256, corpusLock, renderInstruction, roots and constants" + exports: ["validateTask", "loadTasks", "fixtureSha256", "corpusLock", "renderInstruction", "TASK_SCHEMA_VERSION", "BENCH_TASKS_ROOT", "BENCH_FIXTURES_ROOT"] + - path: "scripts/bench/fixture-server.mjs" + provides: "startFixtureServer (in-process evidence accessor + secret-gated HTTP evidence endpoint) and a direct-run CLI" + exports: ["startFixtureServer"] + - path: "scripts/bench/lib/host-stream.mjs" + provides: "NDJSON parsing, Claude/Cursor summarizers, Claude argv and agent serialization" + exports: ["splitNdjson", "createNdjsonAccumulator", "readFsbAgentPolicy", "serializeFsbAgents", "buildClaudeArgv", "buildCursorArgv", "summarizeClaudeStream", "summarizeCursorStream", "FORBIDDEN_HOST_FLAGS", "COST_LABEL_LIST_PRICE", "HOST_UNAVAILABLE_LABEL"] + key_links: + - from: "scripts/bench/fixture-server.mjs" + to: "scripts/bench/lib/outcome.mjs" + via: "request log entries in the Evidence contract; stripRunPrefix imported from outcome.mjs" + pattern: "stripRunPrefix" + - from: "scripts/bench/lib/tasks.mjs" + to: "scripts/bench/lib/identity.mjs" + via: "canonicalJson + sha256Hex for fixtureSha256" + pattern: "fixtureSha256" + - from: "tests/bench-claude-stream.test.mjs" + to: "mcp/src/agent-providers/claude-profile.ts" + via: "source-text parity of every task-mode flag (read-only)" + pattern: "claude-profile.ts" +--- + + +Build the rest of the benchmark's pure core: the task schema and loader, the local fixture server that collects outcome evidence, and the host-stream parsers that turn Claude Code and Cursor NDJSON into labelled host metrics. + +Purpose: MEAS-03 needs repeatable, outcome-checked tasks (D-15, D-17) with enough identity to compare later runs (D-18). The server and parsers are pure or local, and are tested on Node 20 without a browser. +Output: `scripts/bench/lib/tasks.mjs`, `scripts/bench/fixture-server.mjs`, `scripts/bench/lib/host-stream.mjs`, a small sample task set, stream fixtures, and three node:test suites (registered in the chain later by plan 66-09). + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +From plan 66-02 (already implemented): +- `scripts/bench/lib/outcome.mjs`: `CHECK_KINDS`, `stripRunPrefix(pathname, token)`, `normalizeFinalUrl(url, token)`, `evaluateOutcome(task, evidence, {token})`. +- `scripts/bench/lib/identity.mjs`: `canonicalJson(value)`, `sha256Hex(input)`. +- Evidence request entries are `{ t, method, path, query, bodyType:'json'|'form'|'text'|'none'|'too_large', body }`. + +TASK SCHEMA (verbatim contract; plan 66-07 writes the corpus against it): +- Core task: exactly these keys: `schemaVersion` (1), `id`, `version`, `set` ('core'), `category` ('mixed'|'click-heavy'), `startPath`, `instruction`, `timeBudgetMs`, `success`, `fixtureFiles`. +- Live task: exactly these keys: `schemaVersion`, `id`, `version`, `set` ('live'), `category` ('reading'), `startUrl`, `instruction`, `timeBudgetMs`, `success`. +- `id`: /^(core|live)-[a-z0-9]+(-[a-z0-9]+)*$/, and its prefix equals `set`. The file is named `.json` under `//`. +- `version`: a positive integer. `timeBudgetMs`: an integer in 30000..900000. +- `instruction`: 20..2000 chars, containing the template token `{startUrl}` exactly once. +- `startPath` (core): starts with `/fixtures/`, ends with `.html`, has no `..`, `//`, `?` or `#`. `fixtureFiles` (core): a non-empty array of relative paths under the fixtures root (no leading `/`, no `..`), and it must include the startPath's file (startPath minus the leading `/fixtures/`). +- `startUrl` (live): an absolute `https://` URL. +- `success`: exactly `{ all: [check, ...] }`, with 1..20 checks. + - `request` (core only): keys among {kind, method, path, count, jsonEquals}. method is one of GET|POST|PUT|PATCH|DELETE; path starts with `/` and has no `..`; count is an integer in 0..100. + - `finalUrl`: core uses keys among {kind, pathEquals, queryIncludes} and requires pathEquals; live uses exactly {kind, urlIncludes}. + - `marker` (core only): exactly {kind, name, equals}; name matches /^[a-z0-9-]{1,40}$/; equals is a string of at most 200 chars. + - `text` (live only): exactly {kind, includes}, where includes is 1..200 chars. + +FIXTURE SERVER CONTRACT (verbatim; plans 66-07 and 66-08 rely on it): +- `startFixtureServer({ root = BENCH_FIXTURES_ROOT, port = 0, secret = , maxRequestsPerRun = 2000, maxBodyBytes = 65536 })` resolves to `{ port, baseUrl, secret, newRunToken(), runUrl(token, startPath), evidence(token), clear(token), close() }`. The host is always 127.0.0.1 and not configurable. +- Pages: `GET /r//fixtures/` serves `/`. The token matches /^[A-Za-z0-9_-]{8,64}$/; otherwise 404 and not logged. +- App API: any method on `/r//api/<...>` returns 200 `{"ok":true}` (application/json). The body is parsed by content-type: JSON gives bodyType 'json'; urlencoded gives 'form' (an object); anything else gives 'text'. A body over maxBodyBytes gives 413 and bodyType 'too_large'. +- Every request under a valid `/r//` is logged as `{t, method, path: stripRunPrefix(pathname, token), query, bodyType, body}`. Static GETs use bodyType 'none'. +- `GET /__bench/evidence?run=` and `DELETE /__bench/evidence?run=` require header `x-bench-secret`, compared in constant time, else 403. The in-process `evidence(token)` returns a deep copy `{requests, dropped}`. +- Every response sends `cache-control: no-store`. + +Claude Code task-mode argv to mirror (mcp/src/agent-providers/claude-profile.ts lines ~192-211, read-only): `-p --verbose --output-format stream-json --include-partial-messages --setting-sources "" --disable-slash-commands --no-chrome --strict-mcp-config --mcp-config --agents --agent fsb --permission-mode dontAsk --tools "" --allowedTools mcp__fsb --disallowedTools Bash,Edit,Write,NotebookEdit,WebFetch,WebSearch --max-turns 40 --no-session-persistence`. The agent is serialized from mcp/ai/agents/fsb.json in key order description, prompt, tools, disallowedTools, permissionMode, maxTurns, under key `fsb` (`serializeStaticAgentPolicy` ~93-105; validation `loadStaticAgentPolicy` ~53-89). + + + + + + + Task 1: Task schema validator, loader, fixture hashing and sample task set + scripts/bench/lib/tasks.mjs, tests/bench-tasks.test.mjs, tests/fixtures/bench/tasks-sample/tasks/core/core-sample-save.json, tests/fixtures/bench/tasks-sample/tasks/live/live-sample-read.json, tests/fixtures/bench/tasks-sample/fixtures/sample/index.html + + - scripts/bench/lib/outcome.mjs and scripts/bench/lib/identity.mjs (from 66-02; import from them rather than duplicating) + - mcp/src/agent-providers/claude-profile.ts lines 44-89 (exact-sorted-key validation style to mirror) + - tests/fixtures/agent-streams/claude-code-2.1.177/manifest.json (versioned manifest style; do NOT add files under agent-streams/) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 7" (task schema example) + + + - The sample core task validates (ok true, errors []). Removing `version`, adding an unknown key, using category 'reading' for core, a startPath containing `..`, an instruction lacking `{startUrl}`, timeBudgetMs 10, a request check with count -1, a marker name 'Bad Name', or a text check in a core task each makes it invalid with a specific error message. + - The sample live task validates. A live task with an `http://` startUrl, a `marker` check, or a core-style finalUrl is invalid. + - loadTasks({tasksRoot, fixturesRoot, set:'core'}) returns tasks sorted by id. A file whose name differs from `.json` throws. A missing fixture file listed in fixtureFiles throws. + - fixtureSha256(task, root) is stable across two calls and across a key-reordered copy of the task. It changes when one byte of a fixture file changes (the test copies the sample into a temp dir to mutate it). + - corpusLock(tasks, root) returns [{id, version, fixtureSha256}] sorted by id. + - renderInstruction(task, 'http://127.0.0.1:1/r/tok/fixtures/sample/index.html') replaces the single `{startUrl}` token. + + +Create `scripts/bench/lib/tasks.mjs` (ESM, Node 20-safe) implementing the TASK SCHEMA contract exactly: +- `TASK_SCHEMA_VERSION = 1`, `CORE_CATEGORIES = ['mixed','click-heavy']`, `LIVE_CATEGORIES = ['reading']`. +- `BENCH_TASKS_ROOT` and `BENCH_FIXTURES_ROOT`, resolved relative to the module as `scripts/bench/tasks` and `scripts/bench/fixtures`. +- `validateTask(task, {fixturesRoot}?)` returns `{ok, errors}` and collects every error rather than stopping at the first. It checks file existence only when fixturesRoot is given, and resolves every fixture path with a containment check (the resolved path must stay inside fixturesRoot). +- `loadTasks({tasksRoot = BENCH_TASKS_ROOT, fixturesRoot = BENCH_FIXTURES_ROOT, set})` reads `//*.json` and throws one Error listing every invalid file. +- `fixtureSha256(task, fixturesRoot)`: sha256Hex of canonicalJson `{task, files:[{path, sha256}]}` with files sorted by path. Live tasks hash the canonical task alone. +- `corpusLock(tasks, fixturesRoot)` and `renderInstruction(task, startUrl)`. + +Create the sample set used by this plan's tests and by plan 66-08's dry run: +- `tests/fixtures/bench/tasks-sample/tasks/core/core-sample-save.json`: category click-heavy, startPath `/fixtures/sample/index.html`, fixtureFiles `["sample/index.html"]`. Its success checks are a POST `/api/sample/save` request with count 1 and jsonEquals `{"value":"alpha"}`, a finalUrl pathEquals `/fixtures/sample/index.html` with queryIncludes `{"view":"saved"}`, and a marker `saved-value` equal to `alpha`. +- `tests/fixtures/bench/tasks-sample/tasks/live/live-sample-read.json`: startUrl `https://issues.apache.org/jira/browse/HADOOP-1`, with finalUrl urlIncludes `/jira/browse/HADOOP-1` and text includes `HADOOP-1`. +- `tests/fixtures/bench/tasks-sample/fixtures/sample/index.html`: a self-contained page, relative URLs only, no external resources. It has a text input and a Save button. On save it runs `fetch('../../api/sample/save', {method:'POST', headers:{'content-type':'application/json'}, body: JSON.stringify({value})})`, then does `history.replaceState(null, '', '?view=saved')` and sets `data-bench-marker="saved-value"` text to the value. The relative API path resolves to `/r//api/sample/save` from `/r//fixtures/sample/index.html`. Verify that resolution in the test with `new URL`. + +Create `tests/bench-tasks.test.mjs` (node:test) covering every behavior above. Create the temp dir with `fs.mkdtempSync(os.tmpdir())` and remove it in a finally block. No Node 22+ globals. + +Commit: `feat(66-05): add benchmark task schema, loader and fixture hashing`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-tasks.test.mjs + + + - `node --test tests/bench-tasks.test.mjs` exits 0. + - `node -e "import('./scripts/bench/lib/tasks.mjs').then(async m=>{const t=m.loadTasks({tasksRoot:'tests/fixtures/bench/tasks-sample/tasks',fixturesRoot:'tests/fixtures/bench/tasks-sample/fixtures',set:'core'});if(t.length!==1||t[0].id!=='core-sample-save')process.exit(1)})"` exits 0. + - `ls tests/fixtures/agent-streams` lists the same directories as before this plan (`git status --porcelain tests/fixtures/agent-streams` is empty). + + Tasks are schema-validated, loaded deterministically and hashed with their fixtures. A sample set exists for the tests and the runner's dry run. + + + + Task 2: Loopback fixture server with per-run request log and a secret-gated evidence endpoint + scripts/bench/fixture-server.mjs, tests/bench-fixture-server.test.mjs + + - tests/screenshot-uat.test.js lines 63-85 (startFixtureServer precedent and traversal guard), 214-216 and 379-384 (lifecycle and finally close) + - scripts/report-t1-readiness.mjs lines 1273-1280 (direct-run guard) + - scripts/bench/lib/outcome.mjs (stripRunPrefix) + + + - The server listens on 127.0.0.1 with port > 0 and port !== 7225, and baseUrl equals `http://127.0.0.1:`. + - GET runUrl(token, '/fixtures/sample/index.html') returns 200 with content-type text/html, and evidence(token).requests has one entry {method:'GET', path:'/fixtures/sample/index.html', bodyType:'none'}. + - Traversal attempts (`/r//fixtures/../../../package.json`, a percent-encoded `%2e%2e` variant, and an absolute-path variant) never return 200 and never return file bytes from outside root. + - A malformed token (`/r/bad/fixtures/sample/index.html`) returns 404 and is logged nowhere. + - POST JSON {value:'alpha'} to `/r//api/sample/save` returns 200 `{"ok":true}`, and the evidence holds {method:'POST', path:'/api/sample/save', bodyType:'json', body:{value:'alpha'}}. A urlencoded POST gives bodyType 'form' with an object body. A body over maxBodyBytes returns 413 and is logged with bodyType 'too_large'. + - Two tokens have isolated logs, and clear(token) empties one log only. + - GET /__bench/evidence?run= returns 403 with no header, 403 with a wrong secret, and 200 with the right secret (same requests as evidence(token)). + - maxRequestsPerRun:3 keeps 3 entries and reports dropped:1 after 4 requests. + - close() resolves, and the port stops accepting connections. + + +Create `scripts/bench/fixture-server.mjs` (ESM, node:http/node:fs/node:path/node:crypto only) implementing the FIXTURE SERVER CONTRACT exactly: +- Always bind '127.0.0.1'. Use the precedent's containment check: decode the pathname, `path.resolve(root, '.' + rel)`, and require `resolved === root || resolved.startsWith(root + path.sep)`. +- Serve content types for .html, .js, .css, .json, .svg and .png, and application/octet-stream for anything else. +- Compare secrets with `crypto.timingSafeEqual` on equal-length Buffers. A length mismatch is treated as a failure. +- Never log or print the secret. Generate it with `crypto.randomBytes(32).toString('hex')` when not provided. +- Add a CLI behind the direct-run guard: `node scripts/bench/fixture-server.mjs [--port N]` prints `fixture server on http://127.0.0.1: (evidence secret held in memory)` and closes on SIGINT/SIGTERM. +- Add a header comment: Phase 66 benchmark fixture server (D-13, D-17). It runs on request only and is never used by npm test except this unit test. Evidence is the app's own requests; in-page probes are forbidden because they perturb FSB's network-quiet settle wait (Pitfall 9). + +Create `tests/bench-fixture-server.test.mjs` (node:test) covering every behavior above. Use root `tests/fixtures/bench/tasks-sample/fixtures` and the global `fetch` (available on Node 20). Close the server in `finally`. Bind port 0 only, never 7225. + +Commit: `feat(66-05): add loopback fixture server with per-run evidence log`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-fixture-server.test.mjs + + + - `node --test tests/bench-fixture-server.test.mjs` exits 0. + - `grep -c "0.0.0.0" scripts/bench/fixture-server.mjs` prints 0, and `grep -c "'127.0.0.1'" scripts/bench/fixture-server.mjs` prints at least 1. + - `grep -c "timingSafeEqual" scripts/bench/fixture-server.mjs` prints at least 1. + - No `console.log` in the file prints the secret: `node -e "const s=require('fs').readFileSync('scripts/bench/fixture-server.mjs','utf8');if(/console\.(log|error)\([^)]*secret\b/.test(s))process.exit(1)"` exits 0. + + A loopback-only fixture server serves per-run pages, records request evidence, blocks traversal, and gates HTTP evidence behind a secret. + + + + Task 3: Claude Code and Cursor stream parsers, argv builders and agent serialization + scripts/bench/lib/host-stream.mjs, tests/bench-claude-stream.test.mjs, tests/fixtures/bench/claude-stream-result.jsonl, tests/fixtures/bench/cursor-stream-result.jsonl, tests/fixtures/bench/claude-stream-usage-cap.jsonl, tests/fixtures/bench/claude-error-text-cases.json + + - mcp/src/agent-providers/claude-profile.ts lines 20-110 and 176-240 (DENIED_TOOLS, loadStaticAgentPolicy, serializeStaticAgentPolicy, task-mode argv; read-only) + - mcp/src/agent-providers/claude-stream.ts lines 100-185 and 350-400 (ResultSchema passthrough; attestInit, which rejects non-mcp__fsb tools, so do NOT reuse it; NDJSON loop with CR stripping and trailing flush) + - mcp/ai/agents/fsb.json (agent policy) + - scripts/verify-agent-provider-flags.mjs lines 15-60 (forbidden flag list; line-scanning precedent) + - tests/fixtures/agent-streams/claude-code-2.1.177/contract-stream.jsonl (event shapes; its result line lacks duration and cost) + - tests/mcp-agent-drift-smoke.test.js lines 190-200 and 355-366 (TRIPWIRE: it enumerates tests/fixtures/agent-streams/*/manifest.json, so bench fixtures must stay out of agent-streams/) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 8" + + + - buildClaudeArgv({mcpConfigPath:'/tmp/x/mcp.json', agentsJson, model:'claude-model-fixture'}) contains, in the same relative order as the task-mode array in claude-profile.ts, every flag literal from that array (parsed from the TS source text). Each value follows its flag (stream-json, fsb, dontAsk, mcp__fsb, the denied-tools list, 40, and empty strings for --setting-sources and --tools). The array ends with `--model`, `claude-model-fixture`. + - buildClaudeArgv throws for a missing or invalid model (/^[A-Za-z0-9._:\[\]-]{1,80}$/ fails) and for a relative mcpConfigPath. No output of buildClaudeArgv or buildCursorArgv contains any FORBIDDEN_HOST_FLAGS entry (`--dangerously-skip-permissions`, `--yolo`, `--auto`, `--force`). + - serializeFsbAgents(readFsbAgentPolicy(repoRoot)) parses to an object whose only key is `fsb`, and whose value has keys in order [description, prompt, tools, disallowedTools, permissionMode, maxTurns]. readFsbAgentPolicy throws on a policy with an extra key. + - summarizeClaudeStream(fixture events) gives init.toolLoading 'tool-search' (tools include ToolSearch), init.apiKeySource 'none', result.durationMs 91234, durationApiMs 70211, numTurns 14, totalCostUsd 0.4312, and costLabel 'list-price estimate, not billed'. toolUses counts FSB tools from complete assistant messages only; partial stream_event blocks are not counted twice. claimedComplete is true, and JSON.stringify(summary) does not contain 'CLAUDE_RESULT_TEXT_3b7e'. + - A stream whose init.tools includes WaitForMcpServers gives toolLoading 'eager'. A stream with no result event gives result.present false. A result with a costBasis 'list' field and apiKeySource 'user' still gets the list-price label; without either signal the label is 'client-side estimate'. + - Usage caps (W4): summarizeClaudeStream on the usage-cap fixture gives `rateLimit.limited === true`, `rateLimit.events >= 1` and `errorKind === 'usage_limit'`, and JSON.stringify(summary) does not contain the fixture's limit message sentinel `CLAUDE_LIMIT_TEXT_5e0a`. On the normal fixture it gives `rateLimit.limited === false` and `errorKind === null`. The classification table in tests/fixtures/bench/claude-error-text-cases.json holds one `test()` per case. Each case wraps its text in a minimal init plus result stream with is_error as given, and asserts the expected errorKind and that JSON.stringify(summary) does not contain the text: + - `HTTP 429 Too Many Requests` and `rate limit exceeded` give 'rate_limit'; + - `Claude AI usage limit reached` gives 'usage_limit'; + - `API Error: 401 authentication_error: invalid x-api-key` and `Invalid API key · Please run /login` give 'auth'; + - `HTTP 503 Service Unavailable` and `API Error: 500 Internal server error` give 'api_error'; + - the negatives `Step failed after 500 ms waiting for selector` and `The provider page did not load` (is_error true) give 'other'; + - `Done in 500 ms` with is_error false gives null. + - createNdjsonAccumulator given the fixture split into 7-byte chunks with CRLF endings emits the same events as splitNdjson on the whole text, and end() flushes a trailing line that lacks a newline. Invalid JSON lines are counted in parseErrors, not thrown. + - summarizeCursorStream(cursor fixture) gives result.durationMs 45678, tokens null, costUsd null, label 'unavailable (host does not report)', and does not contain 'CURSOR_RESULT_TEXT_91aa'. + + +Create `scripts/bench/lib/host-stream.mjs` (ESM, Node 20-safe): +- `splitNdjson(text)` and `createNdjsonAccumulator(onEvent)` with `{push(chunk), end()}`. Strip CR, skip blank lines, count invalid lines. +- `CLAUDE_DENIED_TOOLS` (the six names). +- `readFsbAgentPolicy(repoRoot)`: reads `mcp/ai/agents/fsb.json` and validates the same sorted key set and values as loadStaticAgentPolicy. +- `serializeFsbAgents(policy)`. +- `buildClaudeArgv({mcpConfigPath, agentsJson, model})`: builds exactly the argv in , followed by `--model ` (orchestrator decision 3: the model is an explicit runner parameter with no built-in default). +- `buildCursorArgv({model})`: returns `['-p','--output-format','stream-json']`, plus `['--model', model]` when a model is given. The prompt is appended by the driver as the final positional argument. +- `summarizeClaudeStream(events, {toolSearchEnv})` and `summarizeCursorStream(events)`, as in . Never copy `result.result` text, assistant text, or tool inputs or outputs into a summary. +- Add two fields to the Claude summary (W4): + - `rateLimit: {events, limited, statuses}`. It counts `rate_limit_event` lines (shape: `{type:'rate_limit_event', session_id, rate_limit_info:{...passthrough}}`, as in mcp/src/agent-providers/claude-stream.ts ~110-113). `limited` is true when any event's `rate_limit_info.status` is a string other than `allowed` or `allowed_warning`. `statuses` holds up to 5 distinct status strings, each at most 40 chars. It never holds reset times or messages. + - `errorKind: 'usage_limit'|'rate_limit'|'auth'|'api_error'|'other'|null`. It is null unless the result has `is_error: true` or the stream shows `rateLimit.limited`. Classify the result text in memory only, never storing it, trying these rules in order (W-A): + - /usage limit|limit reached|out of (?:extra )?usage/i gives 'usage_limit'; + - /\b(?:HTTP|status)[ :]*429\b|rate.?limit|too many requests|overloaded/i gives 'rate_limit'; + - /\b(?:HTTP|status)[ :]*40[13]\b|unauthori[sz]ed|forbidden|authentication|invalid (?:x-)?api[ -]?key|not logged in|log ?in required|please run \/login/i gives 'auth'; + - an `api_error_status` field, or /\b(?:HTTP|status)[ :]*5\d\d\b|api error/i, gives 'api_error'; + - anything else gives 'other'. + A limited rateLimit with no error result gives 'usage_limit'. A number never classifies without its `HTTP` or `status` prefix, and the word "provider" alone classifies nothing. + - These rate-limit shapes are passthrough, and the exact status strings are verified during the 66-10 smoke; the fixture is labelled synthetic. +- Constants `FORBIDDEN_HOST_FLAGS`, `COST_LABEL_LIST_PRICE = 'list-price estimate, not billed'`, `COST_LABEL_CLIENT_ESTIMATE = 'client-side estimate'`, `HOST_UNAVAILABLE_LABEL = 'unavailable (host does not report)'`. + +Create `tests/fixtures/bench/claude-stream-result.jsonl`, one JSON object per line: +- first a `{"type":"_fixture","note":"synthetic; shaped per code.claude.com headless and costs docs; no network"}` line (summarizers ignore unknown types); +- a `system`/`init` event with model `claude-model-fixture`, tools including `ToolSearch`, `mcp__fsb__open_tab`, `mcp__fsb__click` and `mcp__fsb__complete_task`, mcp_servers `[{"name":"fsb","status":"connected"}]`, and apiKeySource `none`; +- two `stream_event` partial lines; +- assistant messages whose tool_use blocks call mcp__fsb__open_tab, mcp__fsb__click (x3) and mcp__fsb__complete_task, with user tool_result lines between them; +- a final `result` event with subtype success, is_error false, duration_ms 91234, duration_api_ms 70211, num_turns 14, result `CLAUDE_RESULT_TEXT_3b7e`, total_cost_usd 0.4312, usage `{input_tokens:1200, output_tokens:900, cache_read_input_tokens:88000, cache_creation_input_tokens:4000}`, modelUsage for claude-model-fixture, and permission_denials []. + +Create `tests/fixtures/bench/cursor-stream-result.jsonl` with the same `_fixture` line, an init event, one assistant event, and a result event with duration_ms 45678, duration_api_ms 40111 and result `CURSOR_RESULT_TEXT_91aa`. + +Create `tests/fixtures/bench/claude-stream-usage-cap.jsonl`, a synthetic capped run: +- the `_fixture` line; +- an init event; +- one `rate_limit_event` with `rate_limit_info: {"status":"rejected","rateLimitType":"five_hour"}`; +- a final `result` with subtype `error_during_execution`, is_error true, num_turns 1, and result text `CLAUDE_LIMIT_TEXT_5e0a: Claude usage limit reached`; +- no FSB tool_use blocks. + +Create `tests/fixtures/bench/claude-error-text-cases.json`: an array of `{name, isError, text, expected}` covering every classification case in , plus a top-level `_fixture` note that the texts are synthetic. + +Create `tests/bench-claude-stream.test.mjs` (node:test) covering every behavior above. +- Parse the task-mode array from claude-profile.ts source text: take the slice after the `] : [` that follows `const argv = ctx.purpose === 'connection_test'`, up to the next `];`, and collect its single-quoted string literals and its identifier arguments (for example `ctx.privateMcpConfigPath`, `SERIALIZED_FSB_AGENTS`). +- Use `fileURLToPath(new URL('..', import.meta.url))` for the repo root. +- No Node 22+ globals. + +Commit: `feat(66-05): add Claude Code and Cursor host-stream parsers with argv parity`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-claude-stream.test.mjs && node tests/mcp-agent-drift-smoke.test.js + + + - `node --test tests/bench-claude-stream.test.mjs` exits 0. + - `node tests/mcp-agent-drift-smoke.test.js` exits 0 (the agent-streams manifest set is unchanged). If it needs `npm --prefix mcp run build` first, run that; mcp/build is gitignored. + - `git status --porcelain -- mcp/` prints nothing (D-01). + - `node -e "const s=require('fs').readFileSync('scripts/bench/lib/host-stream.mjs','utf8');if(!s.includes(\"'--model'\"))process.exit(1)"` exits 0. + + The Claude and Cursor stream summaries and the argv builders are implemented, tested against the pinned v0.9.91 profile, and label unavailable host metrics honestly. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Agent-controlled browser → fixture server | Pages and agents can issue arbitrary requests to 127.0.0.1. | +| Task JSON files → runner | Task definitions drive navigation and file serving. | +| Host CLI stream → committed metrics | Claude and Cursor stdout can contain page content and model text. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-04 | Tampering / Information disclosure | fixture-server evidence endpoint | mitigate | `/__bench/evidence` requires `x-bench-secret`, compared with timingSafeEqual. The secret is held in runner memory only and never printed. The in-process accessor is used by the runner. An agent's `fetch` from a page cannot read or clear evidence (unit test: no header and wrong header both return 403). | +| T-66-06 | Tampering / Information disclosure | validateTask, fixture-server path resolution | mitigate | Exact-key schema validation. Fixture paths and startPath reject `..`, leading `/` and `//`, and are containment-checked against the root. The server rejects traversal (including percent-encoded) and malformed tokens (unit tests). | +| T-66-06b | Elevation of privilege | fixture-server bind address | mitigate | The server binds 127.0.0.1 only (host not configurable), and a source check forbids `0.0.0.0`. | +| T-66-02 | Information disclosure | summarizeClaudeStream / summarizeCursorStream | mitigate | Summaries hold counts, durations, token numbers and labels only. Result text and tool inputs and outputs are never copied (unit test asserts sentinel strings are absent). | +| T-66-ARGV | Elevation of privilege | buildClaudeArgv | mitigate | The argv mirrors the pinned dontAsk / allowedTools mcp__fsb / disallowed-tools profile. FORBIDDEN_HOST_FLAGS are asserted absent from all built argv. | + + + +- `node --test tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs tests/bench-claude-stream.test.mjs` +- `node tests/mcp-agent-drift-smoke.test.js` still passes. +- `git status --porcelain -- mcp/ tests/fixtures/agent-streams/` is empty. + + + +- The task schema, fixture evidence server and host-stream parsers exist with tests, ready for the corpus (66-07) and the runner (66-08). +- D-12, D-13, D-14, D-15, D-17 and D-18 building blocks are covered without touching mcp/ (D-01). + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Do not edit package.json or tests/delegation-phase-contract.test.js; plan 66-09 registers the new tests in one commit. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-05-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-05-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-05-SUMMARY.md new file mode 100644 index 000000000..9f9d9aa2e --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-05-SUMMARY.md @@ -0,0 +1,294 @@ +--- +phase: 66-baseline-and-session-timing +plan: 05 +subsystem: testing +tags: [benchmark, task-schema, fixture-server, evidence, ndjson, claude-code, cursor-agent, argv-parity, node-test] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 02) + provides: "scripts/bench/lib/outcome.mjs (stripRunPrefix, evaluateOutcome) and identity.mjs (canonicalJson, sha256Hex); bench test conventions" +provides: + - "scripts/bench/lib/tasks.mjs: exact-key task schema (validateTask), loadTasks, fixtureSha256, corpusLock, renderInstruction, BENCH_TASKS_ROOT/BENCH_FIXTURES_ROOT" + - "scripts/bench/fixture-server.mjs: startFixtureServer (127.0.0.1 only, per-run request log, secret-gated evidence endpoint) plus a direct-run CLI" + - "scripts/bench/lib/host-stream.mjs: NDJSON accumulator, Claude/Cursor stream summaries with honest labels, buildClaudeArgv (claude-profile.ts parity plus --model), buildCursorArgv, FSB agent policy reader/serializer" + - "A sample task set (1 core, 1 live) and a self-contained sample page under tests/fixtures/bench/tasks-sample/, for these tests and the 66-08 dry run" + - "Synthetic Claude result, usage-cap and Cursor stream fixtures, and an error-text classification table, under tests/fixtures/bench/" +affects: [66-07, 66-08, 66-09, 66-10, 66-11, benchmark corpus, benchmark runner, live baseline campaign] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Exact-key validation that collects every error, with one specific message per broken rule and no cascades" + - "Two-step containment for any fixture path: lexical startsWith(root + sep), then realpath against the real root" + - "Byte-level NDJSON splitting on the LF byte, so a UTF-8 character split across chunks survives" + - "Source-text parity tests that tokenize claude-profile.ts instead of importing mcp/build" + - "Safety refusals that can bind a port are tested through a pure options resolver, never through listen" + +key-files: + created: + - scripts/bench/lib/tasks.mjs + - scripts/bench/fixture-server.mjs + - scripts/bench/lib/host-stream.mjs + - tests/bench-tasks.test.mjs + - tests/bench-fixture-server.test.mjs + - tests/bench-claude-stream.test.mjs + - tests/fixtures/bench/tasks-sample/tasks/core/core-sample-save.json + - tests/fixtures/bench/tasks-sample/tasks/live/live-sample-read.json + - tests/fixtures/bench/tasks-sample/fixtures/sample/index.html + - tests/fixtures/bench/claude-stream-result.jsonl + - tests/fixtures/bench/cursor-stream-result.jsonl + - tests/fixtures/bench/claude-stream-usage-cap.jsonl + - tests/fixtures/bench/claude-error-text-cases.json + modified: [] + +key-decisions: + - "Fixture paths (startPath and fixtureFiles) must be plain segments of letters, digits, '.', '_' or '-', which also rules out '%2e%2e', backslashes and whitespace" + - "The fixture server refuses any Host header other than 127.0.0.1: or localhost: (DNS rebinding) and any unknown option, including a host" + - "The evidence secret is compared as equal-length SHA-256 digests with timingSafeEqual, and is non-enumerable on the server handle" + - "A structured api_error_status is classified as 'status ', so 429 is rate_limit and 401/403 is auth; a refused rate limit turns an 'other' error into usage_limit" + - "A model value may not start with '-', and agentsJson must be exactly the serializeFsbAgents output" + - "Loaded tasks are deep-frozen so a task cannot drift from its recorded fixtureSha256" + - "TDD tasks were committed as RED test plus GREEN feat pairs (as in 66-01 and 66-02); the GREEN commits carry the plan's messages" + +patterns-established: + - "Plan 66-07 writes scripts/bench/tasks/{core,live}/.json and scripts/bench/fixtures/** against validateTask; loadTasks reports every bad file in one Error" + - "Plan 66-08 mints one token per run with newRunToken(), reads evidence in process with evidence(token), then clear(token)" + - "Plan 66-08 records identity from corpusLock(tasks, root) and summary.init.toolLoading/toolSearchEnv/model, and pauses on errorKind usage_limit/rate_limit (agent_cap) or auth (agent_auth)" + +requirements-completed: [MEAS-03] + +# Metrics +duration: 43min +completed: 2026-09-30 +--- + +# Phase 66 Plan 05: Benchmark Task Schema, Fixture Server and Host-Stream Parsers Summary + +**Exact-key task schema with fixture hashing, a loopback-only fixture server whose evidence endpoint answers only to a runner-held secret, and Claude Code and Cursor stream summaries with honest cost labels and an argv proven identical to the pinned v0.9.91 task-mode profile.** + +This plan delivers only part of MEAS-03: the pure core the corpus (66-07) and the runner (66-08) build on. The corpus, runner, live smoke and baseline land in later plans. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched; the orchestrator owns them. + +## Performance + +- **Duration:** 43 min +- **Started:** 2026-09-30T07:32:44Z +- **Completed:** 2026-09-30T08:15:34Z +- **Tasks:** 3/3 +- **Files created:** 13 (exactly the plan's `files_modified`; no existing tracked file changed) + +## Accomplishments + +- **Task schema (`tasks.mjs`):** + - `validateTask` checks the exact core and live key sets and every rule in the contract, and collects every error. Each broken rule has one specific message, and dependent checks are skipped so one mistake never cascades into several. + - With a `fixturesRoot`, each fixture file must resolve inside the root both lexically and after symlinks, and must be a regular file. + - `loadTasks` reads `//*.json`, also checks the `.json` file name and the directory's set, and throws one Error listing every bad file. It returns tasks sorted by id and deep-frozen. + - `fixtureSha256` is SHA-256 over `canonicalJson({task, files:[{path, sha256}]})`, with files sorted by path. It depends only on bytes and relative paths, so an identical copy elsewhere hashes the same. A live task hashes its canonical JSON alone. + - `corpusLock` gives `[{id, version, fixtureSha256}]` sorted by id. `renderInstruction` uses split and join, so `$&`-style patterns in a URL stay literal. +- **Sample set:** `core-sample-save` (click-heavy: type `alpha`, press Save) and `live-sample-read` (public Jira HADOOP-1). + - The sample page is self-contained and posts to `../../api/sample/save`, which resolves to `/r//api/sample/save`. + - Evidence shaped like that page passes the sample task through the real `evaluateOutcome`. +- **Fixture server (`fixture-server.mjs`):** implements the contract exactly: + - It binds `127.0.0.1` only, logs requests per `/r//` run, parses JSON, form and text bodies, answers an oversized body with 413, keeps the first `maxRequestsPerRun` entries and counts the rest as `dropped`, and sends `cache-control: no-store` on every response. + - `evidence(token)` returns a deep copy. + - The HTTP evidence endpoint returns 403 without the secret, with a wrong secret of the same length, and with a shorter or longer one. + - The CLI prints `fixture server on http://127.0.0.1: (evidence secret held in memory)` and exits 0 on SIGTERM. The test checks that no 64-hex string is ever printed. +- **Host streams (`host-stream.mjs`):** + - `buildClaudeArgv` equals the claude-profile.ts task-mode array element for element, followed by `--model `. The parity test tokenizes the TS source and maps its four identifier elements (the binary prefix spread, the MCP config path, the serialized agents and the denied-tools join). An unmapped element fails the test. + - The denied tools, the agent key set, the serialization key order and the 24/200/40 value bounds are also read from the TS source. + - The Claude summary reports the tool-loading mode, durations, turns, usage, per-model usage, the cost with its label, FSB tool counts from complete assistant messages only (deduplicated by tool_use id), `rateLimit {events, limited, statuses}` and `errorKind`. + - The Cursor summary reports durations only, with `tokens: null`, `costUsd: null` and `label: 'unavailable (host does not report)'`. + - Six sentinels (result text, assistant text, tool input, tool output, a partial delta, and the non-ASCII `µ`) never appear in a summary. + +## Interface notes for plans 66-07 and 66-08 + +- **Summary field locations:** + - Claude cost figures live under `result`: `result.durationMs`, `result.durationApiMs`, `result.numTurns`, `result.usage {input, output, cacheRead, cacheCreation}`, `result.modelUsage[]`, `result.totalCostUsd`, `result.costLabel` and `result.costBasis`. + - Top-level Claude fields are `claimedComplete` (result `success` and not an error), `errorKind` and `rateLimit`. + - Cursor puts `tokens`, `costUsd` and `label` at the top level and durations under `result`. +- **Unmeasured values:** they are `null`, never 0. When no cost number exists, `costLabel` is `unavailable (host does not report)`. +- **NDJSON output:** `splitNdjson(text)` returns `{events, parseErrors, oversizeLines}`. The accumulator's `end()` returns `{events, parseErrors, oversizeLines}` as counts. +- **Additive exports** (beyond the plan's lists): + - `resolveFixtureServerOptions` (pure; used to test the 7225 refusal without binding) + - `runTokens()` on the server handle + - `TASK_SETS`, `CLAUDE_DENIED_TOOLS`, `COST_LABEL_CLIENT_ESTIMATE`, `CLAUDE_ERROR_KINDS` and `NDJSON_MAX_LINE_BYTES` +- **Server behaviors beyond the written contract:** + - A POST to a fixture page is logged and serves the page, so a native form can post to a result page. Other non-GET methods on pages get 405. + - A repeated form key becomes an array. A repeated query key keeps its first value, matching `normalizeFinalUrl`. + - A log entry is added once its body has been read, and `t` is the arrival time. +- **CLI root:** `--root DIR` exists because the default `scripts/bench/fixtures` does not exist until plan 66-07. + +## Task Commits + +1. **Task 1: Task schema validator, loader, fixture hashing and sample task set** + - `6bc03933` (test, RED) + - `b427e7af` (feat, GREEN): `feat(66-05): add benchmark task schema, loader and fixture hashing` +2. **Task 2: Loopback fixture server with per-run request log and a secret-gated evidence endpoint** + - `359f8c09` (test, RED) + - `bd4dd117` (feat, GREEN): `feat(66-05): add loopback fixture server with per-run evidence log` +3. **Task 3: Claude Code and Cursor stream parsers, argv builders and agent serialization** + - `55bd2238` (test, RED) + - `ee086201` (feat, GREEN): `feat(66-05): add Claude Code and Cursor host-stream parsers with argv parity` + +**Plan metadata:** the `docs(66-05)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `scripts/bench/lib/tasks.mjs`: task schema, loader, fixture hashing, corpus lock and instruction rendering. +- `scripts/bench/fixture-server.mjs`: loopback fixture and evidence server, plus its CLI. +- `scripts/bench/lib/host-stream.mjs`: NDJSON parsing, Claude and Cursor summaries, argv builders and the FSB agent policy. +- `tests/bench-tasks.test.mjs`: 76 node:test cases. +- `tests/bench-fixture-server.test.mjs`: 29 cases. +- `tests/bench-claude-stream.test.mjs`: 46 cases. +- `tests/fixtures/bench/tasks-sample/**`: the sample core task, live task and page. +- `tests/fixtures/bench/{claude-stream-result,cursor-stream-result,claude-stream-usage-cap}.jsonl` and `claude-error-text-cases.json`: synthetic fixtures. Each opens with a `_fixture` line (or key) that says so. + +All three test files are unregistered; plan 66-09 adds them to the chain. + +## Decisions Made + +- **Paths:** `startPath` and fixture paths use plain segments only. This makes the containment rule easy to state and test, and it cannot reject any name a fixture author would reasonably use. +- **Evidence secret:** it is compared as SHA-256 digests with `timingSafeEqual`. The buffers always have equal length, so timing does not reveal the guess's length, and a length mismatch still fails. +- **Port 7225 test:** the refusal is tested only through the pure resolver. The hub listens on `127.0.0.1:7225`. A refusal bug in a test that calls `listen` could otherwise try to bind the user's hub port. +- **Classification input:** error text is read from `result.result`, `result.error` and `result.errors[]`, bounded to 16 KB, and dropped after classification. +- **Tool names:** names are counted only if they match `^[A-Za-z0-9_.:-]{1,64}$`; anything else is counted as `[unrecognized]`. At most 64 distinct names are kept. +- **Rate-limit statuses:** at most 5 distinct statuses are kept, 40 characters each. Reset times, messages and utilization are never copied. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 2 - Security] Symlink containment in addition to the lexical check** +- **Found during:** Tasks 1 and 2 +- **Issue:** A lexical `startsWith(root + sep)` check passes a symlink inside the root that points outside it. +- **Fix:** + - `validateTask` and `fixtureSha256` realpath each fixture and re-check it against the real root. + - The server does the same before reading a file. +- **Files modified:** `scripts/bench/lib/tasks.mjs`, `scripts/bench/fixture-server.mjs`, and both tests +- **Verification:** a symlink-escape case is refused in both suites. A mutation that removes the realpath check fails both suites. +- **Committed in:** `6bc03933`/`b427e7af` and `359f8c09`/`bd4dd117` + +**2. [Rule 2 - Security] Host header allowlist and strict options on the fixture server** +- **Found during:** Task 2 +- **Issue:** A loopback server that accepts any Host header can be reached through DNS rebinding. An unknown `host` option would be silently ignored. +- **Fix:** + - Any Host other than `127.0.0.1:` or `localhost:` gets 403 and is not logged. + - Unknown option keys are refused, and port 7225 is refused. + - The secret is non-enumerable on the handle, so it never appears in `JSON.stringify` or `Object.keys`. +- **Files modified:** `scripts/bench/fixture-server.mjs`, `tests/bench-fixture-server.test.mjs` +- **Verification:** a foreign-Host case returns 403 with an empty log. A mutation that removes the Host check fails the suite. +- **Committed in:** `359f8c09`/`bd4dd117` + +**3. [Rule 2 - Security] Tighter path and URL rules in the task schema** +- **Found during:** Task 1 +- **Issue:** The written rules (`..`, `//`, `?`, `#`) still let `%2e%2e`, backslashes, whitespace and `.` segments through. They also allowed a repeated fixture path, and a live `startUrl` carrying credentials. +- **Fix:** each case now has its own specific error message: + - a plain-segment rule for `startPath` and `fixtureFiles` + - a repeated-path error + - an explicit `https://` prefix check + - a credentials check on `startUrl` +- **Files modified:** `scripts/bench/lib/tasks.mjs`, `tests/bench-tasks.test.mjs` +- **Verification:** 41 core and 10 live error cases pin exact error arrays. +- **Committed in:** `6bc03933`/`b427e7af` + +**4. [Rule 1 - Bug] A structured `api_error_status` 429 or 401 would have been an `api_error`** +- **Found during:** Task 3 +- **Issue:** Read literally, rule 4 sends every `api_error_status` to `api_error`. A structured 429 or 401 would then be counted as a failed run instead of pausing the campaign (agent_cap or agent_auth), which is what W4 exists to prevent. +- **Fix:** + - The code is classified as the in-memory text `status `, so it goes through the same prefixed rules: 429 gives rate_limit, 401/403 give auth, and 5xx or any other code gives api_error. + - When an error result is otherwise `other` and the stream shows a refused rate limit, errorKind becomes `usage_limit`. + - The text rules and their order are unchanged. +- **Files modified:** `scripts/bench/lib/host-stream.mjs`, `tests/bench-claude-stream.test.mjs` +- **Verification:** + - a structured-status case covers 429, '401', 403, 529, 400, null and undefined, and usage-limit text beats a 429; + - all 18 error-text table cases pass. +- **Committed in:** `55bd2238`/`ee086201` + +**5. [Rule 2 - Security] Argv hardening beyond the plan's model regex** +- **Found during:** Task 3 +- **Issue:** `/^[A-Za-z0-9._:\[\]-]{1,80}$/` accepts `--yolo` or `-p` as a model value. `agentsJson` was not checked at all. +- **Fix:** + - A model may not start with `-`. + - `agentsJson` must equal the `serializeFsbAgents` output byte for byte. That rules out widened tools or permission mode, extra agents and pretty-printing. + - A post-condition refuses any argv element equal to a forbidden flag, or starting with that flag followed by `=`. +- **Files modified:** `scripts/bench/lib/host-stream.mjs`, `tests/bench-claude-stream.test.mjs` +- **Verification:** + - the model, agentsJson and forbidden-flag cases pass; + - `FORBIDDEN_HOST_FLAGS` is checked to cover every flag in `scripts/verify-agent-provider-flags.mjs`. +- **Committed in:** `55bd2238`/`ee086201` + +**6. [Rule 2 - Missing critical functionality] Bounded NDJSON lines** +- **Found during:** Task 3 +- **Issue:** An unbounded partial line from a runaway host process could exhaust memory. +- **Fix:** + - `maxLineBytes` (default 4 MiB) drops an over-long line and counts it in `parseErrors` and `oversizeLines`. + - Non-object JSON lines count as parse errors, and a leading BOM is stripped. +- **Files modified:** `scripts/bench/lib/host-stream.mjs`, `tests/bench-claude-stream.test.mjs` +- **Verification:** the over-long-line and invalid-line cases pass. +- **Committed in:** `55bd2238`/`ee086201` + +### Execution notes (not rule deviations) + +- **TDD commit split:** every task is `tdd="true"`, so each has a RED `test(66-05)` commit before its GREEN `feat(66-05)` commit. The GREEN commits use the plan's messages verbatim. +- **Temp dirs:** the plan says `fs.mkdtempSync(os.tmpdir())`. Taken literally, that creates a sibling of the temp directory. The tests use `fs.mkdtempSync(path.join(os.tmpdir(), 'fsb-bench-...-'))` and remove the directory in `finally`. +- **Raw requests:** the traversal cases use `node:http` with raw request targets. The global `fetch` normalizes `..` and `%2e%2e` before sending. Every other case uses the global `fetch`, as the plan says. +- **Extra table cases:** the error-text table has 18 cases, 8 more than the plan lists: `status: 429`, `overloaded`, `out of extra usage`, `HTTP 403`, `not logged in`, bare `429` and bare `401` negatives, and a non-error mentioning a usage limit. +- **Branch checks:** the executor's `worktree-agent-*` branch allow-list was skipped, because the orchestrator directed commits on `Jev`. The cwd-drift sentinel and the protected-branch deny-list ran before every commit. + +--- + +**Total deviations:** 6 auto-fixed (1 Rule 1, 5 Rule 2). +**Impact on plan:** each change tightens a security or evidence boundary the plan's threat register names (T-66-02, T-66-04, T-66-06, T-66-ARGV), or keeps W4's pause behavior correct. The footprint is exactly the plan's 13 files, and none of the changes adds scope. + +## Issues Encountered + +- **A typo in a verification loop:** a stray `$=0;` made zsh run itself as a command. That child shell waited on stdin until the 600 s tool timeout, and none of the intended suites ran. Both processes were mine (pids 98374 and 98376) and were stopped. No user process was touched, and the hub (pid 42278 on `127.0.0.1:7225`) was never disturbed. The suites were then re-run correctly (below). +- **A mutation probe that survived:** routing `stream_event` into `countToolUses` survived, because partial blocks sit at `event.event.content_block`, not `message.content`, so that mutation changes nothing. A real mutation that counts partial `content_block_start` tool_use blocks is caught. +- **No Node 20 run:** no Node 20 binary exists on this machine, and none was downloaded. A static scan of all six new files found no Node 22+ API (`Object.groupBy`, `Promise.withResolvers`, `crypto.hash`, `fs.globSync`, `process.getBuiltinModule`, global `WebSocket`, `import.meta.dirname`, `toSorted`, Set methods, `fs.cpSync`, and others). Running the bench tests under Node 20 remains part of the phase gate. + +## Verification Evidence + +- **New suites:** `node --test tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs tests/bench-claude-stream.test.mjs` passes 151/151 (76 + 29 + 46). +- **66-02 suites:** `node --test tests/bench-stats.test.mjs tests/bench-outcome.test.mjs tests/bench-identity.test.mjs` still passes 50/50. +- **Drift smoke:** `node tests/mcp-agent-drift-smoke.test.js` PASS. The fixture roster is still `claude-code, grok-build`, and `tests/fixtures/agent-streams` is unchanged. +- **Acceptance checks, all as specified:** + - `loadTasks` on the sample set returns exactly `core-sample-save`. + - `grep -c "0.0.0.0"` prints 0. + - `grep -c "'127.0.0.1'"` prints 1. + - `grep -c timingSafeEqual` prints 3. + - The console-secret regex check exits 0. + - `host-stream.mjs` contains `'--model'`. + - `git status --porcelain -- mcp/` is empty. +- **Mutation probes:** each mutation was run on the committed modules and then restored byte for byte, and each made its suite fail: + - tasks: realpath containment removed, `String.replace` instead of split/join, the startPath include check cascading; + - server: both containment checks removed, realpath containment removed, Host check removed, secret check removed; + - host-stream: the partial-messages flag dropped, result text copied into the summary, tool_use id deduplication removed, an unprefixed 401 rule, partial tool_use blocks counted. +- **Chain suites that look at git state:** `node tests/phase60-full-tests-harness.test.js`, `phase64-...`, `phase65-...` and `node tests/mcp-native-host-packaging.test.js --section workflow-and-pack` all exit 0. The tree stayed clean afterwards. +- **Leftovers:** no fixture-server process remained after the CLI tests, and no tracked file was rewritten by any run. + +## Known Stubs + +None. The three modules are complete, and the sample set is intentionally one core and one live task (the 66-08 dry run relies on exactly that). The real corpus is plan 66-07's. + +## User Setup Required + +None. No external service configuration is required. + +## Next Phase Readiness + +- **Plan 66-07:** writes the corpus under `scripts/bench/tasks/{core,live}/` and `scripts/bench/fixtures/`, and runs `loadTasks({set:'core'})` and `loadTasks({set:'live'})` to validate it. Fixture pages must use relative URLs only and create server-visible effects: fetch or form posts under `../../api/...`, or a POST to a result page. +- **Plan 66-08:** + - drives the arms with `buildClaudeArgv` (explicit `--claude-model`) and `buildCursorArgv`, sending the task on stdin to Claude and as the final positional argument to Cursor; + - parses stdout with `createNdjsonAccumulator`, summarizes with `summarizeClaudeStream(events, {toolSearchEnv})` or `summarizeCursorStream`, and collects evidence in process from `startFixtureServer`; + - pauses on `errorKind` `usage_limit` or `rate_limit` (agent_cap) and on `auth` (agent_auth). +- **Plan 66-09:** registers `tests/bench-{tasks,fixture-server,claude-stream}.test.mjs` in the `test` chain and in `REFINEMENT_ROOT_TEST_COMMANDS`. +- **Plan 66-10 smoke:** confirms the real `rate_limit_event` status strings and whether a result carries `api_error_status` or `costBasis`. The shapes used here are passthrough and labelled synthetic. +- **Blockers:** none. + +## Self-Check: PASSED + +- All 13 created files and this SUMMARY were found on disk. +- All 6 task commits were found: `6bc03933`, `b427e7af`, `359f8c09`, `bd4dd117`, `55bd2238`, `ee086201`. +- No commit in `0ec2abe7..HEAD` has a Co-Authored-By line. +- `git status --porcelain -- mcp/ tests/fixtures/agent-streams/ extension/ package.json .planning/STATE.md .planning/ROADMAP.md` is empty. diff --git a/.planning/phases/66-baseline-and-session-timing/66-06-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-06-PLAN.md new file mode 100644 index 000000000..2f839d5e9 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-06-PLAN.md @@ -0,0 +1,304 @@ +--- +phase: 66-baseline-and-session-timing +plan: 06 +type: execute +wave: 3 +depends_on: ["66-03", "66-04"] +files_modified: + - extension/ws/mcp-tool-dispatcher.js + - tests/mcp-in-flight-session-lookup.test.js +autonomous: true +requirements: [MEAS-01, MEAS-02] +user_setup: [] + +must_haves: + truths: + - "get_session_detail resolves journal-backed MCP sessions. The lookup order is: journal hasSession then getSessionDetail, then the legacy loadSession, then in-flight sessions. The journal branch is skipped when the journal module is missing (D-07). This is the one intended behavior change (D-02)." + - "The response keeps the {success:true, session:{...}} envelope. Journal tool.call events become a sanitized session.actionHistory built through the existing sanitizeActionHistoryEntry allowlist, with timing added. Raw journal events and inline request/result payloads are never returned (D-08, T-66-03)." + - "get_session_detail gains no new parameters. Paging over journal events happens internally, bounded to 20 pages of 500, and truncation is visible (D-08, MCP-04)." + - "format:'text' for a journal session returns the journal's exportHumanReadable, which carries the per-call timing lines from plan 66-03 (D-07, D-09)." + - "Every session detail carries an explicit measurement block attached after the generic sanitizer, so token counts and timing survive the /token/i key filter. MCP sessions report per-call fsb ms, caller gap labelled as an upper bound, activeSpanMs (never endTime - startTime), endReason with idle_timeout flagged and the idle tail excluded, queueWaitStatus 'not_measured', and host model time, tokens and cost 'unavailable' (D-03, D-06, D-12)." + - "Autopilot sessions report iterations, model and tool ms, provider-first usage and cost with sources, requested and served models, and per-iteration rows. Pre-Phase-66 sessions report recorded:false plus labelled legacy totals, never invented values (D-05, D-10)." + - "No legacy total appears over MCP without a basis label. get_session_detail and list_sessions carry `totalCostBasis` beside the legacy `totalCost`, and `measurement.legacyTotals` carries `costBasis` and `usageBasis`. The legacy numbers keep their values (D-02) while the labelled measurement fields are authoritative (D-10), per the D-02/D-10 reconciliation in orchestrator decision 9 (W1). Both routes return the same `totalCostBasis` for the same session, because both prefer the value plan 66-04 stores at save time (I-2)." + - "MCP measurement reports `callIntervalMs {median, p90, count}` (receive-to-receive between consecutive timed calls), which is like-for-like with the 66-01 history deltas. Autopilot perIteration rows carry `startedAt` (W6)." + - "The dispatcher's two recording paths carry call timing: the message-route recorder entry, and recordTaskOutcome, whose timing is read from the original payload before visual-session canonicalization drops the Symbol (D-04)." + artifacts: + - path: "extension/ws/mcp-tool-dispatcher.js" + provides: "MCP_DISPATCHER_CALL_TIMING, readDispatcherCallStamp, buildDispatcherCallTiming, journal-first handleGetSessionMessageRoute, measurement projection, finite-only durationMs/timing in sanitizeActionHistoryEntry" + contains: "MCP_DISPATCHER_CALL_TIMING" + - path: "tests/mcp-in-flight-session-lookup.test.js" + provides: "vm-harness cases using the real journal module (memory backend) plus stub-journal truncation and error cases" + key_links: + - from: "extension/ws/mcp-tool-dispatcher.js handleGetSessionMessageRoute" + to: "globalThis.FsbMcpLatticeJournal.getSessionDetail / exportHumanReadable" + via: "typeof-guarded journal-first branch before the automationLogger check" + pattern: "FsbMcpLatticeJournal" + - from: "journal tool.call metadata.request/result descriptors" + to: "sanitizeActionHistoryEntry" + via: "pseudo-action {tool, timestamp, iteration:null, params, result} built only from inline descriptors" + pattern: "sanitizeActionHistoryEntry\\(" + - from: "extension/ws/mcp-tool-dispatcher.js dispatchMcpToolRoute" + to: "fsbMcpSessionRecorder.recordTaskOutcome" + via: "callTimingStamp read before canonicalizeVisualSessionTaskParams, passed as timing" + pattern: "timing: buildDispatcherCallTiming\\(callTimingStamp\\)" +--- + + +Make `get_session_detail` return journal-backed MCP sessions with sanitized per-call timing. Add an explicit, filter-proof `measurement` block for MCP and autopilot sessions. Carry call timing on the dispatcher's own recording paths. + +Purpose: MEAS-02 (journal sessions resolve instead of "not found") and MEAS-01 (operators can inspect per-call wall and FSB time, model time, tokens and cost, with unavailable values labelled). The existing `{success, session}` envelope and the tool's parameters stay unchanged (D-08, MCP-04). +Output: the dispatcher changes and paired vm-harness tests. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/REQUIREMENTS.md +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-03-SUMMARY.md +@.planning/phases/66-baseline-and-session-timing/66-04-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +Shared contracts from plans 66-03 and 66-04 (already implemented): +- Symbol `Symbol.for('fsb.mcp.call-timing')`. The stamp is `{receivedAt}`. This file's identifier must be `MCP_DISPATCHER_CALL_TIMING`; the bridge client already declares `MCP_BRIDGE_CALL_TIMING`, and both share one global lexical scope. +- Recorder entry `timing: {v:1, receivedAt, completedAt, fsbMs}`. The recorder's `recordTaskOutcome(input)` forwards `input.timing`. +- Journal event: `{sequence, kind, timestamp:, metadata:{tool, route, client, success, tabId, logicalTab, request:, result:, timing?}}`. A descriptor has `storage:'inline'|'artifact'|'omitted'`; `inline` holds the raw value only when storage is 'inline'. The inline request value is the recorder-sanitized payload object; its `params` holds selector/url. +- Journal API: `hasSession(id)`; `getSessionDetail({sessionId, afterSequence, limit})` returns `{session:, events, nextSequence, hasMore}`, with limit clamped to 500; `exportHumanReadable(id)`. The index entry has `id, task, status, startTime, endTime, actionCount, eventCount, mode:'mcp-agent', mcpClient, storageBackend, recordingState, outcome, outcomeDetails:{reason:'idle_timeout',...}, ...`. +- Autopilot stored fields: `session.iterationTiming[]` flat rows `{v, iteration, startedAt, modelMs, modelRetried, toolMs, toolCount, usage:{input, output, reasoning, cachedInput}, usageSource, costUsd, costSource, estimateUsd, estimateDeltaPct, pricingKey, requestedModel, servedModel, provider}`, and `session.measurementSummary {v, iterations, modelMsTotal, toolMsTotal, toolCount, usage:{...}, usageIterations, usageSource, costUsd, costIterations, costSource, requestedModel, servedModels}`. `actionHistory[].durationMs` is finite or null. Legacy MCP rows may carry `timing`. + +MEASUREMENT BLOCK (verbatim contract; plan 66-08's runner reads it): +- MCP: `{ v:1, kind:'mcp', source:'journal'|'legacy', calls, callsWithTiming, fsbMs:{total, median, p90}, callerGapMs:{total, median, p90, count, overlapped, label:'upper bound on host thinking time'}, callIntervalMs:{median, p90, count}, activeSpanMs, endReason, idleTimeout:, idleTailExcluded:true, queueWaitMs:null, queueWaitStatus:'not_measured', host:{modelMs:null, inputTokens:null, outputTokens:null, costUsd:null, status:'unavailable', reason:'host does not report to FSB'}, truncated, pagesRead }`. fsbMs and callerGapMs statistics are null when there are no values. activeSpanMs is lastCompletedAt - firstReceivedAt over timed calls, or null. For caller gaps, an untimed call breaks the chain and a negative gap increments `overlapped` without entering the statistics. callIntervalMs uses receivedAt(n) - receivedAt(n-1) over consecutive timed calls in sequence order; an untimed call breaks the chain, and negative intervals are excluded. This is the live counterpart of the 66-01 history deltas, which measured journal timestamps between consecutive calls (W6). +- Autopilot: `{ v:1, kind:'autopilot', recorded, iterations, modelMs:{total, median, p90}, toolMs:{total, median, p90, calls}, inputTokens, outputTokens, reasoningTokens, cachedInputTokens, usageSource, costUsd, costSource, costIterations, requestedModel, servedModels, perIteration:[last 100 {iteration, startedAt, modelMs, modelRetried, toolMs, toolCount, inputTokens, outputTokens, reasoningTokens, cachedInputTokens, usageSource, costUsd, costSource, servedModel, estimateDeltaPct}], legacyTotals:{costUsd, costBasis, inputTokens, outputTokens, usageBasis, label:'legacy price-table total; not provider-reported'} }`. When measurementSummary is absent: recorded:false, every measured value null, sources 'unavailable', perIteration []. legacyTotals values are null unless > 0. + +BASIS LABELS (verbatim contract; W1, orchestrator decision 9 D-02/D-10 reconciliation): +- The legacy `totalCost` is always a price-table sum, so `'provider'` never applies to it. Its basis depends on whether the session has a `measurementSummary` (I-B): + - With a summary: `'unavailable'` when `measurementSummary.costSource === 'unavailable'` or any stored iterationTiming row has `pricingKey === null`, because the legacy total then includes the unknown-model display fallback rather than a real price-table estimate. Otherwise `'estimate'`, even when the total is 0 (for example a local lmstudio run). + - Without a summary (pre-Phase-66 sessions): `'estimate'` when totalCost is finite and > 0, else `'unavailable'`. +- The legacy token totals sum provider-reported usage. Their basis is `'provider'` when either total is > 0, and `'unavailable'` otherwise. +- `measurement.legacyTotals.costBasis` and `measurement.legacyTotals.usageBasis` carry these two bases. +- Every get_session_detail session object that contains a `totalCost` key gains `totalCostBasis` beside it. Every list_sessions entry that contains a `totalCost` key gains `totalCostBasis` too. +- Stored value (I-2): sessions saved since plan 66-04 carry `totalCostBasis`, computed by this same cost rule at save time (`computeLegacyCostBasis` in automation-logger.js), on both the session record and its fsbSessionIndex entry. Index entries never carry measurementSummary or iterationTiming, so the rule cannot be recomputed from a list entry. Both routes therefore use a stored 'estimate' or 'unavailable' value as is, and compute the rule only when none is stored (pre-Phase-66 sessions, which have no summary). A stored value outside those two is ignored and recomputed. list_sessions and get_session_detail therefore agree for every session. +- The token totals never appear bare over MCP, because the /token/i filter removes them from the generic output; they appear only inside `legacyTotals`, where `usageBasis` labels them. +- The fields are additive and optional (INV-01 / MCP-04 allow optional response fields). No existing field changes value. + +Existing dispatcher code (verified 2026-09-29): MCP_REPLAY_RECORD_CONTEXT 157-160; payload spread in canonicalizeVisualSessionOwnershipToken 576-579; `dispatchMcpToolRoute` 639-731 (canonicalize 649-652; recordTaskOutcome call ~722); `dispatchMcpMessageRoute` 733-874 (`var sessionRecordEntry = {` 829-851); isSensitiveKey/sanitizeValue 2027-2050; sanitizeSessionMetadata/sanitizeActionHistoryEntry/sanitizeSessionDetail 2064-2116; `handleGetSessionMessageRoute` 2904-2957; `_mcp_dispatcher_exports` 3926-3960. Mirror (do not copy) background.js 12496-12536 and options.js 3443-3487. Do NOT reuse `journalDescriptorPreview`, which returns inline values verbatim. + + + + + + + Task 1: Carry call timing on the dispatcher's message-route recorder and task-outcome paths + extension/ws/mcp-tool-dispatcher.js, tests/mcp-in-flight-session-lookup.test.js + + - extension/ws/mcp-tool-dispatcher.js lines 150-165, 560-600, 636-735, 815-875, 3920-3990 + - tests/mcp-in-flight-session-lookup.test.js (whole file, 207 lines: harness, run(), exit code) + - TRIPWIRES in tests/mcp-session-recorder.test.js lines 860-925: + - exactly 1 comment `never let session recording break dispatch`; + - exactly 1 `recordSafely(... 'recordDispatch' ... sessionRecordEntry ...)`; + - the `var sessionRecordEntry = {` span contains `resolveMcpClientLabel(payload)`; + - the first `fsbMcpSessionRecorder.recordDispatch` comes after `async function dispatchMcpMessageRoute`; + - exactly 1 `if (!_mcpMetricsSuppressInner) {`; + - `/globalThis\.fsbMcpMetricsRecorder\.recordDispatch\(\{[\s\S]*?\}\);/g` matches exactly 2. + - TRIPWIRES elsewhere: tests/mcp-dispatcher-client-label.test.js lines 215-240 (same metrics regex, count 2); tests/spreadsheet-record-redaction.test.js lines 773-782 (dispatcher regexes `spreadsheetInvoke[\s\S]*recordSafely\([\s\S]*'recordDispatch'` and `if \(!spreadsheetRecord && !unresolvedRecordTarget\)[\s\S]*recordDispatch\(sessionRecordEntry\)`). + + + - dispatchMcpMessageRoute({type:'mcp:list-sessions', payload}) where payload carries the Symbol stamp (receivedAt = now - 30) hands the stubbed fsbMcpSessionRecorder.recordDispatch an entry with timing {v:1, receivedAt, completedAt >= receivedAt, fsbMs >= 30}. Without a stamp, timing is null. + - readDispatcherCallStamp returns null for a payload that only has a string key `timing`, for a non-integer or negative receivedAt, and for non-objects. It returns {receivedAt} for a valid stamp. + - buildDispatcherCallTiming(null) is null. For a valid stamp it returns an object whose fsbMs equals completedAt - receivedAt. + - Source contract: inside dispatchMcpToolRoute, `const callTimingStamp = readDispatcherCallStamp(payload);` appears before `canonicalizeVisualSessionTaskParams(tool, params, payload)`, and the recordTaskOutcome call passes `timing: buildDispatcherCallTiming(callTimingStamp)`. + - The existing four in-flight and session_not_found cases pass unchanged. + + +1. Beside `MCP_REPLAY_RECORD_CONTEXT`, add `const MCP_DISPATCHER_CALL_TIMING = Symbol.for('fsb.mcp.call-timing');` with a one-line comment. It is the same registry symbol the bridge client stamps (D-04); the identifier differs because both files share the service-worker global scope. +2. Add two top-level functions: + - `readDispatcherCallStamp(payload)`: accepts only the Symbol property's object with a safe non-negative integer receivedAt, and returns `{receivedAt}` or null. + - `buildDispatcherCallTiming(stamp)`: returns `{v:1, receivedAt, completedAt: Date.now(), fsbMs}` or null. + Use `Date.now()` only; `performance` does not exist in the vm harness. +3. In `dispatchMcpMessageRoute`, add `timing: buildDispatcherCallTiming(readDispatcherCallStamp(payload))` as the last property of the existing `var sessionRecordEntry = {` literal. Change nothing else in that finally block. +4. In `dispatchMcpToolRoute`, directly before `const canonicalized = canonicalizeVisualSessionTaskParams(tool, params, payload);`, add `const callTimingStamp = readDispatcherCallStamp(payload);`. The canonicalized payload is a spread copy that has lost the Symbol (Pitfall 5). Change the existing call to `globalThis.fsbMcpSessionRecorder.recordTaskOutcome({ tool, params, payload, response, timing: buildDispatcherCallTiming(callTimingStamp) });`. Do not add any other recorder call to dispatchMcpToolRoute, and do not touch either metrics `recordDispatch` block. +5. Add both helpers to `_mcp_dispatcher_exports` under a `// Phase 66 call timing (test-only)` comment. + +Tests: append two cases to tests/mcp-in-flight-session-lookup.test.js, `runDispatcherMessageRouteTimingCase` and `runDispatcherTimingHelpersCase`. Set `context.fsbMcpSessionRecorder` in the harness context after build, and leave FsbSpreadsheetRecordRedaction undefined so the plain recordDispatch branch runs. Use `Symbol.for('fsb.mcp.call-timing')` from the test realm. Add a source-contract case reading the dispatcher file. Register the new cases in `run()`. Keep the plain-node `assert`/`passed`/`failed` style. + +Run `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node --test tests/spreadsheet-record-redaction.test.js && node tests/mcp-tool-routing-contract.test.js`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-tool-dispatcher.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-06): record MCP call timing on dispatcher recording paths`. No Co-Authored-By or AI attribution. + + + node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node --test tests/spreadsheet-record-redaction.test.js + + + - `grep -c "Symbol.for('fsb.mcp.call-timing')" extension/ws/mcp-tool-dispatcher.js` prints 1, and `grep -c "const MCP_BRIDGE_CALL_TIMING" extension/ws/mcp-tool-dispatcher.js` prints 0. + - `node -e "const s=require('fs').readFileSync('extension/ws/mcp-tool-dispatcher.js','utf8');const f=s.slice(s.indexOf('async function dispatchMcpToolRoute'),s.indexOf('async function dispatchMcpMessageRoute'));if(!(f.indexOf('readDispatcherCallStamp(payload)')>-1&&f.indexOf('readDispatcherCallStamp(payload)') exit 0, and `node tests/mcp-tool-routing-contract.test.js` exits 0. + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-tool-dispatcher.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + The dispatcher's message-route and task-outcome recordings carry validated call timing. Every dispatcher recording pin holds. + + + + Task 2: Journal-first get_session_detail with a sanitized, paged actionHistory projection and a journal text path + extension/ws/mcp-tool-dispatcher.js, tests/mcp-in-flight-session-lookup.test.js + + - extension/ws/mcp-tool-dispatcher.js lines 2020-2120 and 2866-2960 + - extension/background.js lines 12494-12540 (journal-first control-panel lookup to mirror; do NOT copy the `...detail` spread, which returns raw events) + - extension/ui/options.js lines 3430-3490 (field mapping to mirror; do NOT reuse journalDescriptorPreview) + - extension/utils/mcp-lattice-journal.js lines 836-872, 1615-1648, 2005-2050, 2195-2250 + - tests/mcp-lattice-journal.test.js lines 1-96 (how to reset the real journal with memory, local-storage, time and alarm shims) + - mcp/src/tools/observability.ts lines 43-66 (read-only: the tool forwards only {sessionId, format} with a 10 s timeout, so no new parameters are possible) + + + - With the real journal module (memory backend) injected as `context.FsbMcpLatticeJournal`, and 4 recorded tool.call events (3 with metadata.timing, 1 without; request params with selector '#save' and url 'https://intranet.example.test/secret-path?token=abc'; one result carrying 'PAGE_TEXT_SENTINEL_5d2c'), dispatchMcpMessageRoute({type:'mcp:get-session', payload:{sessionId}}) returns success:true with the {session} envelope: + - session.id === sessionId, and session.storageBackend equals the index entry's value; + - session.actionHistory has 4 rows in sequence order, each with tool, a numeric timestamp, a sequence, domain 'intranet.example.test' (never the path or query), selector '#save', and result.success; + - the 3 timed rows carry timing and durationMs === timing.fsbMs; the untimed row has neither key; + - JSON.stringify(response) contains none of 'PAGE_TEXT_SENTINEL_5d2c', '/secret-path', 'token=abc'. + - The same session with format:'text' returns {success:true, format:'text', text}. The text contains `FSB MCP SESSION REPORT`, the plan 66-03 timing legend line, and `| fsb `. + - 1203 recorded calls give actionHistory length 100 (the last 100, ending with the last call), measurement.calls 1203, truncated false, and pagesRead 3. + - A stub journal whose getSessionDetail always returns hasMore:true stops after 20 pages with measurement.truncated true. + - A stub journal whose hasSession throws falls through to the legacy and in-flight lookup. A stub whose getSessionDetail throws returns a route error containing 'Journal session detail unavailable', not session_not_found. + - Round trip: after wiring the harness automationLogger.listSessions to return the journal's `fsbSessionIndex` from its local-storage shim, the id from `mcp:list-sessions` resolves through `mcp:get-session`. + - With no journal module (the existing four cases), behavior is byte-identical to before. + + +In extension/ws/mcp-tool-dispatcher.js: +1. Add a top-level constant `JOURNAL_DETAIL_MAX_PAGES = 20` and these helpers (confirm each name is unused in extension/ first): + - `projectJournalDescriptorObject(descriptor)`: returns `descriptor.inline` only when `descriptor.storage === 'inline'` and inline is a plain non-array object; otherwise `{}`. + - `validProjectionTiming(value)`: the contract validation (v === 1, safe non-negative integers, completedAt >= receivedAt, fsbMs recomputed); returns a fresh object or null. + - `projectJournalToolCall(event)`: builds the pseudo-action `{tool: metadata.tool || '', timestamp: Date.parse(event.timestamp), iteration: null, params: .params when it is a plain object else {}, result: }`, passes it through the EXISTING `sanitizeActionHistoryEntry`, then attaches `sequence`, `route` (string, 40 chars max), `logicalTab` (string, 40 chars max), and, when timing is valid, `timing` and `durationMs`. + - `async collectJournalToolCalls(journal, sessionId)`: pages `journal.getSessionDetail({sessionId, afterSequence, limit: 500})` from afterSequence -1 until hasMore is false or JOURNAL_DETAIL_MAX_PAGES pages are read. It keeps the last 100 projected tool.call rows and running arrays of fsbMs values, caller gaps (computed as in the contract, including the overlapped count) and receive-to-receive call intervals, plus the first receivedAt, the last completedAt, the total call count, pagesRead and truncated. It returns `{session, rows, stats}`. + - `projectJournalSessionForMcp(collected)`: returns the `sanitizeSessionMetadata(collected.session)` result plus `logs: []`, `actionHistory: rows`, `callsTotal`, `actionHistoryTruncated: callsTotal > rows.length`, and `measurement` (built in Task 3; in this task, attach whatever Task 3's builder returns, or build the MCP block inline if doing Task 3 in the same commit). +2. In `handleGetSessionMessageRoute`, after the sessionId check and BEFORE the automationLogger availability check, add the journal-first branch: + - Resolve `globalThis.FsbMcpLatticeJournal` via `typeof globalThis !== 'undefined'` and require `hasSession` and `getSessionDetail` to be functions; otherwise skip the branch. + - Compute `isJournal` with try/catch; a throw means false. + - If `isJournal` and `payload.format === 'text'`: return `{ success: true, format: 'text', text: await journal.exportHumanReadable(payload.sessionId) }`, or the existing `createMcpRouteError(... 'Human-readable session exporter unavailable')` shape when the exporter is missing. + - If `isJournal`, otherwise: return `{ success: true, session: projectJournalSessionForMcp(await collectJournalToolCalls(journal, payload.sessionId)) }`. Wrap that in try/catch returning `createMcpRouteError('get_session_detail', 'observability', MCP_ROUTE_RECOVERY_HINT, { error: 'Journal session detail unavailable' })`. + - Leave the legacy, in-flight and session_not_found code paths below it byte-identical. +3. Use `Array.isArray` (never `instanceof Array`), because the real journal returns objects from the Node realm while the dispatcher runs in the vm realm. Add no top-level `require(...)`: the harness throws on unknown requires. + +Tests: extend `buildHarness` with an optional `journal` argument that sets `context.FsbMcpLatticeJournal`, and an optional `listSessions` override. Do not change the defaults used by the existing four cases. Add cases for every behavior above, resetting the real journal exactly as tests/mcp-lattice-journal.test.js `setup()` does. Register them in `run()`. + +Run `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/mcp-lattice-journal.test.js`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-tool-dispatcher.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-06): resolve journal-backed MCP sessions in get_session_detail`. No Co-Authored-By or AI attribution. + + + node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/mcp-lattice-journal.test.js + + + - `node -e "const s=require('fs').readFileSync('extension/ws/mcp-tool-dispatcher.js','utf8');const f=s.slice(s.indexOf('async function handleGetSessionMessageRoute'));if(!(f.indexOf('FsbMcpLatticeJournal')>-1&&f.indexOf('FsbMcpLatticeJournal') + Journal sessions resolve through get_session_detail in JSON and text with sanitized, timing-bearing rows. Paging is bounded, and every legacy path is unchanged. + + + + Task 3: Explicit measurement projection for MCP and autopilot sessions, plus finite-only durationMs and timing in the action allowlist + extension/ws/mcp-tool-dispatcher.js, tests/mcp-in-flight-session-lookup.test.js + + - extension/ws/mcp-tool-dispatcher.js lines 2020-2120 (sanitizers; `sanitizeSessionDetail` is the pattern: delete the raw field, then attach an explicit projection, as `replaySummary` does) and lines 2866-2880 (`handleListSessionsMessageRoute`) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Open Questions (RESOLVED)", decision 9 (the D-02/D-10 reconciliation) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 3" and "Pitfall 1" (the /token/i key filter and depth-4 flattening) + - extension/ai/cost-tracker.js and extension/utils/automation-logger.js (the field shapes produced by plan 66-04, including the save-time `totalCostBasis` on the session record and index entry; I-2) + + + - A legacy autopilot session (through the harness automationLogger.loadSession) with measurementSummary {iterations:2, modelMsTotal:5000, toolMsTotal:900, toolCount:3, usage:{input:18000, output:700, reasoning:300, cachedInput:9000}, usageSource:'provider', costUsd:0.0321, costSource:'provider', requestedModel:'grok-4.3', servedModels:['grok-4.3']} and two iterationTiming rows yields session.measurement.kind 'autopilot', recorded true, inputTokens 18000, outputTokens 700, reasoningTokens 300, cachedInputTokens 9000, and costUsd 0.0321 with costSource 'provider'. Token counts survive the /token/i filter. perIteration has 2 rows with modelMs values, and session.iterationTiming and session.measurementSummary are absent from the generic output. + - An autopilot session without measurementSummary but with totalCost 0.12 yields recorded:false, costUsd null, costSource 'unavailable', and legacyTotals.costUsd 0.12 with the label. With totalCost 0, legacyTotals.costUsd is null. + - actionHistory rows with durationMs 412 keep durationMs 412. Rows without durationMs, or with a null or NaN one, have no durationMs key (never 0). + - A legacy MCP session (mode 'mcp-agent') whose actionHistory rows carry timing yields measurement.kind 'mcp' and source 'legacy', with fsbMs and callerGapMs computed from row timing and host.status 'unavailable'. + - A journal session whose index entry has outcomeDetails.reason 'idle_timeout' yields measurement.endReason 'idle_timeout', idleTimeout true, idleTailExcluded true, and activeSpanMs = lastCompletedAt - firstReceivedAt (not endTime - startTime), with queueWaitMs null and queueWaitStatus 'not_measured'. + - Caller gaps: three timed calls A (1000..1120), B (1620..1700) and C (1650..1800, overlapping B) yield callerGapMs.count 1, total 500, and overlapped 1. + - The in-flight snapshot's currentAction keeps its shape. It gains durationMs only when the live action has a finite one. + - Call intervals: timed calls received at 1000, 1600 and 2600 (and completed after) yield callIntervalMs {median 800, p90 per the linear-interpolation rule, count 2}. An untimed call between two timed calls breaks the chain. + - Autopilot perIteration rows carry `startedAt` equal to the stored iteration row's startedAt. + - Basis labels (W1): an autopilot session with totalCost 0.12 and no measurementSummary returns `session.totalCostBasis === 'estimate'`, `measurement.legacyTotals.costBasis === 'estimate'` and `measurement.legacyTotals.usageBasis === 'provider'` (with totalInputTokens > 0). A session with totalCost 0 and no summary returns 'unavailable' for both bases. + - Basis labels (I-B): a session with totalCost 0.70 whose measurementSummary has costSource 'unavailable' returns `totalCostBasis === 'unavailable'` and `legacyTotals.costBasis === 'unavailable'`, not 'estimate'. The same holds when any iterationTiming row has `pricingKey: null`, even if costSource is 'estimate'. A session whose summary has costSource 'provider' and whose rows all carry a pricingKey returns 'estimate' for the legacy total. list_sessions entries with a `totalCost` key gain `totalCostBasis`, and entries without one gain nothing. No response contains a `totalCost` key without a sibling `totalCostBasis` (checked by a recursive scan in the test). + - List and detail agree (I-2): a session with totalCost 0.70 whose record and index entry both store `totalCostBasis: 'unavailable'` (as 66-04 saves it; the index entry has no measurementSummary) returns 'unavailable' from mcp:list-sessions and from mcp:get-session, and `measurement.legacyTotals.costBasis` is 'unavailable' too. Without the stored value, that list entry would compute 'estimate', which is the disagreement this prevents. A stored 'provider' is ignored and recomputed. + + +1. Extend `sanitizeActionHistoryEntry` with two conditional spreads, using the idiom already in that function: `...(Number.isFinite(action?.durationMs) ? { durationMs: action.durationMs } : {})` and `...(validProjectionTiming(action?.timing) ? { timing: validProjectionTiming(action.timing) } : {})`. +2. Add these builders following the MEASUREMENT BLOCK contract exactly (numbers null when absent; medians and p90 by linear interpolation over finite values): + - `buildMcpMeasurementFromStats(stats, sessionMeta, source)` + - `buildMcpMeasurementFromLegacy(session)`: computes the same stats from `actionHistory[].timing`. + - `buildAutopilotMeasurement(session)` + Map stored `usage.input/output/reasoning/cachedInput` to `inputTokens/outputTokens/reasoningTokens/cachedInputTokens` ONLY inside this post-sanitization block. +3. In `sanitizeSessionDetail(session)`, after the existing logic: `delete sanitized.iterationTiming; delete sanitized.measurementSummary;` then set `sanitized.measurement` to `buildMcpMeasurementFromLegacy(session)` when `session.mode === 'mcp-agent'`, else `buildAutopilotMeasurement(session)`. Then apply the BASIS LABELS contract through a helper `withLegacyCostBasis(obj, source)`: when obj has a `totalCost` key, set `totalCostBasis` to `source.totalCostBasis` when that is 'estimate' or 'unavailable' (the value stored at save time; I-2), else compute it from `source` by the contract. Here `source` is the unsanitized session, because the summary has already been deleted from `sanitized`. `buildAutopilotMeasurement` resolves `legacyTotals.costBasis` the same way, so the two labels never disagree. Include `costBasis`/`usageBasis` in legacyTotals, `callIntervalMs` in both MCP builders, and `startedAt` in perIteration rows. +3b. In `handleListSessionsMessageRoute`, map each sanitized entry through `withLegacyCostBasis(entry, entry)`. The entry carries the stored `totalCostBasis` for sessions saved since 66-04 (it passes the key filter), and pre-Phase-66 entries fall back to the no-summary rule, so the list returns the same label as get_session_detail (I-2). This adds only the optional `totalCostBasis` field; the limit, order and every other field are unchanged. +4. In the journal projection from Task 2, set `measurement: buildMcpMeasurementFromStats(stats, collected.session, 'journal')`. +5. Keep every existing sanitizer behavior otherwise unchanged. The measurement block must be attached after `sanitizeSessionMetadata`, never passed through `sanitizeValue`. + +Tests: add cases for every behavior above to tests/mcp-in-flight-session-lookup.test.js, using `historicalSession` for the legacy and autopilot sessions and the real or stub journal for the MCP ones. Register them in `run()`. + +Run `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/spreadsheet-record-redaction.test.js`. + +Regression sweep (W11): run `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-tool-dispatcher.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json`. It must exit 0, meaning no failure beyond the 66-02 start-commit baseline. If 7225 is held now but was free during the baseline, rerun the reported port-bound tests after the holder is gone before calling a failure new. Never use the `timeout` command. + +Commit: `feat(66-06): project filter-proof timing, token and cost measurement in session detail`. No Co-Authored-By or AI attribution. + + + node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/spreadsheet-record-redaction.test.js + + + - `grep -c "queueWaitStatus: 'not_measured'" extension/ws/mcp-tool-dispatcher.js` prints at least 1, and `grep -c "upper bound on host thinking time" extension/ws/mcp-tool-dispatcher.js` prints at least 1. + - `grep -c "host does not report to FSB" extension/ws/mcp-tool-dispatcher.js` prints at least 1. + - Session end and start are never subtracted: `node -e "const s=require('fs').readFileSync('extension/ws/mcp-tool-dispatcher.js','utf8');if(/endTime\s*-\s*[a-zA-Z_.]*startTime/.test(s))process.exit(1)"` exits 0. + - `grep -c "totalCostBasis" extension/ws/mcp-tool-dispatcher.js` prints at least 1, `grep -c "callIntervalMs" extension/ws/mcp-tool-dispatcher.js` prints at least 1, and `grep -c "withLegacyCostBasis(" extension/ws/mcp-tool-dispatcher.js` prints at least 3 (definition, session detail, list_sessions). + - All suites in exit 0. + - The W11 sweep `node scripts/bench/sweep-tests.mjs --files extension/ws/mcp-tool-dispatcher.js --baseline .planning/phases/66-baseline-and-session-timing/66-02-SWEEP-BASELINE.json` exits 0. + + Operators get per-call timing plus model, token and cost measurements for MCP and autopilot sessions over MCP. Unavailable values are labelled and never 0. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Extension journal (IndexedDB) → MCP client | Journal events hold full request and result payloads from real pages. | +| MCP caller payload → dispatcher recording | String keys are attacker-influenced. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-03 | Information disclosure | handleGetSessionMessageRoute journal branch | mitigate | Raw events are never returned. Each tool.call becomes a pseudo-action fed through the existing `sanitizeActionHistoryEntry` allowlist (selector at most 250 chars, domain only, error at most 500 chars). Artifact and omitted descriptors contribute nothing. The unit test asserts that page text, URL path and query sentinels are absent from the whole response. | +| T-66-05 | Tampering | readDispatcherCallStamp | mitigate | Timing is read only from the registry Symbol. String keys are ignored (unit test). Values are re-validated. | +| T-66-DoS | Denial of service | collectJournalToolCalls | mitigate | Paging is bounded to 20 pages of 500 events. The ring keeps 100 rows. Truncation is reported. The MCP tool's 10 s timeout is respected by the bound. | +| T-66-03b | Information disclosure | measurement block | accept | The block contains only numbers, model and provider names, and fixed labels. It is built from stored flat numeric fields, with no page content. | + + + +- `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/spreadsheet-record-redaction.test.js tests/mcp-lattice-journal.test.js` +- `git status --porcelain -- mcp/` is empty. + + + +- MEAS-02: journal-backed MCP sessions resolve with tool calls and timing records. +- MEAS-01: per-call and per-iteration timing, tokens and cost are inspectable over MCP with explicit unavailable labels. +- D-03, D-06, D-07, D-08, D-09 and D-12 hold; parameters and envelope are unchanged. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Each commit pairs the dispatcher change with its test updates. Never stage files under mcp/. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-06-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-06-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-06-SUMMARY.md new file mode 100644 index 000000000..804d1bef4 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-06-SUMMARY.md @@ -0,0 +1,300 @@ +--- +phase: 66-baseline-and-session-timing +plan: 06 +subsystem: observability +tags: [mcp, get-session-detail, lattice-journal, per-call-timing, measurement, cost-basis, sanitizer, list-sessions, node-test] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 02) + provides: start-commit baseline and the W11 regression sweep with 66-02-SWEEP-BASELINE.json + - phase: 66-baseline-and-session-timing (plan 03) + provides: ingress stamp on Symbol.for('fsb.mcp.call-timing'), recorder timing forwarding (recordDispatch and recordTaskOutcome), journal metadata.timing and the timed text exports + - phase: 66-baseline-and-session-timing (plan 04) + provides: session.iterationTiming, session.measurementSummary, actionHistory[].durationMs, and the save-time totalCostBasis on the record and its index entry +provides: + - "MCP_DISPATCHER_CALL_TIMING, readDispatcherCallStamp and buildDispatcherCallTiming: timing on the message-route recorder entry and on recordTaskOutcome (stamp read before visual-session canonicalization)" + - "Journal-first get_session_detail: hasSession, then bounded internal paging (20 pages of 500), then sanitized actionHistory rows (last 100) with sequence, route, logicalTab, timing and durationMs; callsTotal and actionHistoryTruncated; format text = the journal's exportHumanReadable" + - "measurement block v1 for MCP sessions (journal and legacy) and autopilot sessions, attached after the generic sanitizer" + - "totalCostBasis beside every totalCost in get_session_detail and list_sessions (stored value first, W1/I-B rule otherwise)" + - "sanitizeActionHistoryEntry keeps a measured durationMs and a v1-valid timing" + - "Save-time computeLegacyCostBasis aligned: a positive legacy total beside LM Studio rows is 'unavailable'" +affects: [66-07, 66-08, benchmark runner, list_sessions consumers] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Journal events leave only as pseudo-actions through the existing sanitizeActionHistoryEntry allowlist; inline descriptors contribute, artifact and omitted ones contribute nothing" + - "One per-call statistics accumulator shared by the journal and legacy MCP paths, so both report identical measures" + - "Quantiles use the bench report's rule (scripts/bench/lib/stats.mjs), and a test pins the parity" + - "Measurement blocks are attached after sanitizeSessionMetadata, never passed through the /token/i key filter" + - "Basis labels: the stored save-time value wins when valid; otherwise the dispatcher computes the same rule the logger uses" + +key-files: + created: [] + modified: + - extension/ws/mcp-tool-dispatcher.js + - tests/mcp-in-flight-session-lookup.test.js + - extension/utils/automation-logger.js + - tests/automation-logger-mcp-retention.test.js + +key-decisions: + - "LM Studio (orchestrator carry-over): a positive legacy totalCost beside rows with pricingKey 'lmstudio-local' is 'unavailable', because the legacy record() call prices it at the unknown-model fallback. A legacy total of 0 stays 'estimate', as the contract's own lmstudio example says. The same rule is applied at save time in automation-logger.js and in the dispatcher, so list_sessions and get_session_detail agree" + - "Journal rows take result.success from the inline result when it is boolean, else from the journal's metadata.success, so an artifact-stored result (over 64 KB) does not read as a failure" + - "The MCP measurement builder landed with Task 2 (the journal branch needs measurement.calls, truncated and pagesRead); Task 3 added the legacy MCP and autopilot builders, the allowlist and the basis labels" + - "activeSpanMs is the latest completedAt minus the earliest receivedAt over timed calls. This equals first and last in sequence order for sequential calls, and stays correct when concurrent calls complete out of order" + - "Autopilot modelMs and toolMs median and p90 cover the stored per-iteration rows (the last 100). toolMs.calls is the session's executed-tool count" + - "Legacy MCP measurement covers the stored actionHistory rows with truncated false and pagesRead 0, because no paging applies" + - "Each tdd task was committed once, tests with source: every touched suite is in the npm test chain and must stay green in every commit. RED was run and recorded before each implementation" + +patterns-established: + - "Plan 66-08 reads session.measurement from get_session_detail: kind 'mcp' (journal or legacy) or 'autopilot', v 1; the MCP arm's sessions are journal sessions" + - "Any new legacy total shown over MCP gets a sibling basis label through withLegacyCostBasis" + +requirements-completed: [MEAS-01, MEAS-02] + +# Metrics +duration: 23min +completed: 2026-09-30 +--- + +# Phase 66 Plan 06: Journal-Backed Session Detail and Measurement Projection Summary + +**`get_session_detail` now resolves journal-backed MCP sessions. It checks the journal first and pages it internally within a bound. Each tool call comes back as a sanitized `actionHistory` row carrying its timing, and `format: 'text'` returns the journal's timed report. Every history session gains a filter-proof `measurement` block:** +- **MCP sessions:** per-call FSB milliseconds, a caller gap labelled as an upper bound, receive-to-receive call intervals, an active span that leaves out the idle tail, a flagged `idle_timeout`, queue wait labelled "not measured", and host model time, tokens and cost labelled "unavailable". +- **Autopilot sessions:** iterations, model and tool time, provider-first tokens and cost with their sources, requested and served models, per-iteration rows, and labelled legacy totals. + +**Every legacy `totalCost` shown over MCP now carries a `totalCostBasis` label. The dispatcher's own recording paths now carry call timing.** + +This plan delivers MEAS-02 (journal sessions resolve instead of "not found") and the MCP-inspection slice of MEAS-01. `requirements-completed` lists the plan's requirement IDs for traceability only. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched, because the orchestrator owns them. + +## Performance + +- **Duration:** 23 min +- **Started:** 2026-09-30T08:22:47Z +- **Completed:** 2026-09-30T08:45Z +- **Tasks:** 3/3, plus one orchestrator-directed alignment commit +- **Files modified:** 4 (2 extension sources, 2 paired test files); none created + +## Accomplishments + +- **Dispatcher call timing (D-04), Task 1:** + - `MCP_DISPATCHER_CALL_TIMING` is the bridge client's registry symbol under a name local to the dispatcher. `Symbol.for('fsb.mcp.call-timing')` appears once in the file, and `MCP_BRIDGE_CALL_TIMING` appears 0 times. + - `readDispatcherCallStamp` reads only the symbol, so a caller-supplied string key named `timing` is ignored. `buildDispatcherCallTiming` returns null on a clock rollback, never 0. + - The message-route `sessionRecordEntry` gains `timing` as its last property. + - `dispatchMcpToolRoute` reads the stamp before `canonicalizeVisualSessionTaskParams`, whose payload copy drops the symbol, and passes `timing` to `recordTaskOutcome`. + - A behavioural test drives the visual-session path. The recorder receives the canonicalized copy, which has lost the symbol, and still gets valid timing. +- **Journal-first `get_session_detail` (D-07, D-08), Task 2:** + - Lookup order: `hasSession` (a throw means not a journal session and falls through), then `getSessionDetail` paged 500 at a time for at most 20 pages, then the legacy `loadSession`, then in-flight sessions. + - Rows are pseudo-actions sent through the existing `sanitizeActionHistoryEntry`. They keep the selector (at most 250 characters), the domain only, and a bounded error, and add `sequence`, `route`, `logicalTab`, `timing` and `durationMs`. + - Raw events, inline payloads and descriptors never leave. The test checks that page-text, URL-path and query sentinels are absent from the whole response. + - `callsTotal`, `actionHistoryTruncated`, `measurement.truncated` and `pagesRead` make both the row cap and the page bound visible. + - `format: 'text'` returns `exportHumanReadable` verbatim, including the 66-03 legend and the `| fsb N ms` lines. + - A failure to read the detail or the export returns the route error `Journal session detail unavailable`, never `session_not_found`, and never echoes the internal error. + - With no journal module, or for sessions the journal does not hold, responses are byte-identical to before (asserted with `JSON.stringify` equality). +- **Measurement and labels (D-03, D-05, D-06, D-10, D-12, W1, W6, I-B, I-2), Task 3:** + - `sanitizeSessionDetail` deletes the raw `iterationTiming` and `measurementSummary`, then attaches the block after the generic sanitizer. Token counts survive the `/token/i` filter, and no `'[object]'` appears in the block. + - `sanitizeActionHistoryEntry` keeps a finite, non-negative `durationMs` and a v1-valid `timing` (with fsbMs recomputed). The in-flight `currentAction` keeps its pre-Phase-66 key order unless the live action measured a duration. + - `list_sessions` maps each entry through `withLegacyCostBasis(entry, entry)`. Only `totalCostBasis` is added; the limit, order and all other fields are unchanged. + +Sample output from the committed code, rendered by a throwaway harness (`/tmp/fsb-66-06-sample.js`, not committed). The first block is a journal session that closed on the idle timeout; the second is an autopilot session. + +``` +{"v":1,"kind":"mcp","source":"journal","calls":3,"callsWithTiming":3,"fsbMs":{"total":1604,"median":412,"p90":732}, + "callerGapMs":{"total":17876,"median":8938,"p90":9068.4,"count":2,"overlapped":0,"label":"upper bound on host thinking time"}, + "callIntervalMs":{"median":9550,"p90":9840.4,"count":2},"activeSpanMs":19480,"endReason":"idle_timeout","idleTimeout":true, + "idleTailExcluded":true,"queueWaitMs":null,"queueWaitStatus":"not_measured", + "host":{"modelMs":null,"inputTokens":null,"outputTokens":null,"costUsd":null,"status":"unavailable","reason":"host does not report to FSB"}, + "truncated":false,"pagesRead":1} + +{"v":1,"kind":"autopilot","recorded":true,"iterations":2,"modelMs":{"total":5000,"median":2750,"p90":2950}, + "toolMs":{"total":900,"median":475,"p90":495,"calls":3},"inputTokens":18000,"outputTokens":700,"reasoningTokens":300, + "cachedInputTokens":9000,"usageSource":"provider","costUsd":0.0321,"costSource":"provider","costIterations":2, + "requestedModel":"grok-4.3","servedModels":["grok-4.3"],"perIteration":[{"iteration":1,"startedAt":1790726401000,...}], + "legacyTotals":{"costUsd":0.03,"costBasis":"estimate","inputTokens":18000,"outputTokens":700,"usageBasis":"provider", + "label":"legacy price-table total; not provider-reported"}} +``` + +## Task Commits + +1. **Task 1: Carry call timing on the dispatcher's message-route recorder and task-outcome paths:** `8beef0db` (feat) +2. **Task 2: Journal-first get_session_detail with a sanitized, paged actionHistory projection and a journal text path:** `7b4e8de9` (feat) +3. **Task 3: Explicit measurement projection for MCP and autopilot sessions, plus finite-only durationMs and timing in the action allowlist:** `ea0cbfe0` (feat) +4. **Orchestrator carry-over: align the save-time LM Studio basis rule:** `c8a22e4d` (fix) + +**Plan metadata:** see the `docs(66-06)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `extension/ws/mcp-tool-dispatcher.js`: + - The call-timing constant and helpers, the recorder-entry `timing` and the `recordTaskOutcome` timing. + - The journal branch in `handleGetSessionMessageRoute`. + - `projectJournalDescriptorObject`, `validProjectionTiming`, `projectJournalToolCall`, `createMcpCallStats`/`addMcpCallToStats`, `measurementQuantile`/`summarizeMeasurementValues`, `readMcpEndReason` and `buildMcpMeasurementFromStats`. + - `collectJournalToolCalls` and `projectJournalSessionForMcp`. + - `buildMcpMeasurementFromLegacy`, `buildAutopilotMeasurement` and `projectAutopilotIterationRow`. + - `legacyCostBasisFor`, `resolveLegacyCostBasis`, `legacyUsageBasisFor` and `withLegacyCostBasis`. + - The allowlist spreads, the `sanitizeSessionDetail` block and the `list_sessions` mapping. + - Every new top-level name was grep-checked as unused in `extension/`. No top-level `require`. +- `tests/mcp-in-flight-session-lookup.test.js`: + - `buildHarness` gains optional `journal` and `listSessions` arguments; the defaults are unchanged. + - 19 new cases (Task 1: 4, Task 2: 9, Task 3: 6). The suite went from 15 to 214 assertions, and the original 4 cases are untouched. + - Real-journal cases reset the module exactly as `tests/mcp-lattice-journal.test.js` `setup()` does. +- `extension/utils/automation-logger.js`: `computeLegacyCostBasis` adds the LM Studio clause. `computeLegacyCostBasis(` still appears 2 times and `totalCostBasis` 2 times (66-04's pins). +- `tests/automation-logger-mcp-retention.test.js`: LM Studio cases for a positive total (unavailable) and a 0 total (estimate), and a check that the legacy value is kept. It went from 138 to 141 assertions. + +## Verification + +| Check | Result | +|-------|--------| +| Task 1 verify, plus `mcp-tool-routing-contract` | exit 0: in-flight 62/0, recorder 353/0, client-label 51/0, redaction 20/0, routing 239/0 | +| Task 1 acceptance | symbol count 1; `const MCP_BRIDGE_CALL_TIMING` 0; the ordering and metrics-count `node -e` exits 0 | +| Task 2 verify (in-flight, routing contract, `node --test` journal) | exit 0: 140/0, 239/0, 39/0 | +| Task 2 acceptance | journal checked before `automationLogger?.loadSession`; `journalDescriptorPreview` 0; `git status --porcelain -- mcp/` empty | +| Task 3 verify (in-flight, recorder, client-label, routing, redaction) | exit 0: 214/0, 353/0, 51/0, 239/0, 20/0 | +| Task 3 acceptance | `queueWaitStatus: 'not_measured'` 1; `upper bound on host thinking time` 1; `host does not report to FSB` 1; no `endTime - startTime`; `totalCostBasis` 3; `callIntervalMs` 1; `withLegacyCostBasis(` 3 | +| Logger alignment | retention 141/0; `lattice-replay-background-contract` 59 passed; `delegation-sidepanel-tab-follow` 62/0 | +| W11 sweeps vs `66-02-SWEEP-BASELINE.json` (7225 held, the same state as the baseline) | Task 1: 35/35; Task 2: 35/35; Task 3: 35/35; `automation-logger.js`: 6/6. No new failures | +| Plan `` command, run verbatim | exit 0: 214/0, 353/0, 51/0, 239/0, `node --test` 59/0 | +| Full `npm test` chain, tolerant mode (7225 held by pid 42278) | **311/311 commands exit 0** (13 pretest + 298 test), 204 s. Artifacts: `/tmp/fsb-66-06-chain.{jsonl,log,done}` | +| D-01 | `git status --porcelain -- mcp/` is empty; no `mcp/` file appears in `d73d74ce..HEAD` | + +- The chain rewrote `showcase/angular/public/{llms.txt,llms-full.txt,sitemap.xml}`. They were reverted with `git checkout --` and never committed. +- No lockfile and no `mcp/` file changed. No stray relay was left running, and 7225 is still held by the same pid. + +## Decisions Made + +- **LM Studio basis label.** This settles the orchestrator's carry-over from 66-04. + - **Background:** the legacy `CostTracker.record()` call does not pass the provider, so an LM Studio run's legacy `totalCost` is priced at the unknown-model display fallback (or at a table row whose name happens to match). Its iteration rows carry the local $0 estimate with pricingKey `'lmstudio-local'`. The 66-04 save-time rule labelled that total `'estimate'`. + - **Rule:** within I-B, those iterations have no real price row. So a positive legacy total beside `'lmstudio-local'` rows is `'unavailable'`. A legacy total of exactly 0 stays `'estimate'`, which is the contract's own lmstudio example. + - **Where it applies:** `computeLegacyCostBasis` (save time, `c8a22e4d`) and `legacyCostBasisFor` (dispatcher recompute, `ea0cbfe0`) now match. Both routes prefer the stored label, so they agree for every session. + - **What is unchanged:** no legacy number changed (D-02). + - **Not covered:** an LM Studio session saved by a build between 66-04 and this plan keeps its stored `'estimate'`. That only affects unreleased development builds. +- **`endReason`** is `outcomeDetails.reason`, cut to 64 characters, or null. A running session has no end reason yet. +- **Autopilot `recorded`** requires `measurementSummary.v === 1`. A malformed or future-version summary reads as not recorded rather than being partly trusted. +- **Source and label allowlists:** + - Summary `usageSource`: provider, mixed or unavailable. + - Summary `costSource`: provider, estimate, mixed or unavailable. + - Row `usageSource`: provider or unavailable. + - Row `costSource`: provider, estimate or unavailable. + - Anything else reads `'unavailable'`. + - Model names are cut to 128 characters, and at most 10 served models are kept. +- **Commit granularity:** see TDD Gate Compliance. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Journal rows reported artifact-stored results as failures** +- **Found during:** Task 2 +- **Issue:** The plan's pseudo-action uses the inline result object as `result`. A result over 64 KB (for example a large `read_page`) is stored as an artifact with no inline value, so `Boolean(result.success)` read `false` for a successful call. The same happens for any inline result without a boolean `success`. +- **Fix:** The pseudo-action's result is `{ success: , error: }`, which is still sent through `sanitizeActionHistoryEntry`. Artifact content contributes nothing. +- **Files modified:** `extension/ws/mcp-tool-dispatcher.js`, `tests/mcp-in-flight-session-lookup.test.js` (`runJournalArtifactResultCase`) +- **Verification:** "an artifact result keeps the journal success flag and the request domain" passes, and the artifact sentinel is absent. +- **Committed in:** `7b4e8de9` + +**2. [Rule 2 - Missing critical functionality] Journal text export and a vanished session could escape as raw errors** +- **Found during:** Task 2 +- **Issue:** The plan wraps only the JSON projection in try/catch. A throwing `exportHumanReadable` would reach the route wrapper `maybeBuildRestrictedResponse`. That wrapper rethrows the raw message to the MCP client, or, when the active tab happens to be restricted, returns a misleading restricted-page response. A session deleted between `hasSession` and `getSessionDetail` would have produced an empty session object. +- **Fix:** The text export gets the same `Journal session detail unavailable` route error. `collectJournalToolCalls` throws when no page returned a session, and that throw becomes the route error. +- **Files modified:** `extension/ws/mcp-tool-dispatcher.js`, `tests/mcp-in-flight-session-lookup.test.js` (`runJournalLookupErrorCases`) +- **Verification:** the throwing-exporter and vanished-session cases pass, and the internal message (`cursor 7`) is never echoed. +- **Committed in:** `7b4e8de9` + +**3. [Rule 2 - Missing critical functionality] The allowlisted durationMs must also be non-negative** +- **Found during:** Task 3 +- **Issue:** The plan's spread is finite-only, so a negative `durationMs` from any producer would reach MCP. 66-04's persisted contract is finite and non-negative (its Deviation 2). +- **Fix:** `...(Number.isFinite(action?.durationMs) && action.durationMs >= 0 ? { durationMs } : {})` +- **Files modified:** `extension/ws/mcp-tool-dispatcher.js`, `tests/mcp-in-flight-session-lookup.test.js` +- **Verification:** the -5 row carries no `durationMs`, and a measured 0 is kept. +- **Committed in:** `ea0cbfe0` + +**4. [Rule 1 - Bug, orchestrator-directed] LM Studio legacy totals were labelled 'estimate' at save time** +- **Found during:** the orchestrator's carry-over note from 66-04 +- **Issue:** see Decisions Made. The fix has to sit in `automation-logger.js`, which is outside this plan's `files_modified`: `list_sessions` can only use the stored label, because index entries carry no rows. +- **Fix:** add the LM Studio clause to `computeLegacyCostBasis`, and use the same clause in the dispatcher's `legacyCostBasisFor`. +- **Files modified:** `extension/utils/automation-logger.js`, `tests/automation-logger-mcp-retention.test.js` (`c8a22e4d`); `extension/ws/mcp-tool-dispatcher.js`, `tests/mcp-in-flight-session-lookup.test.js` (`ea0cbfe0`) +- **Verification:** the two LM Studio cases pass in both suites. The logger sweep ran 6/6, and the full chain ran 311/311. +- **Committed in:** `ea0cbfe0`, `c8a22e4d` + +### Execution notes (not rule deviations) + +- **Where the MCP measurement builder landed:** `buildMcpMeasurementFromStats` shipped with Task 2, because the Task 2 behaviours assert `measurement.calls`, `truncated` and `pagesRead`. The plan allows this ("build the MCP block inline if doing Task 3 in the same commit"). The journal-based idle-timeout, overlap and call-interval tests shipped with it. +- **Test additions beyond the plan:** + - A behavioural visual-session `recordTaskOutcome` test (Pitfall 5). + - A quantile parity check against `scripts/bench/lib/stats.mjs`, loaded with a dynamic `import()` through `pathToFileURL`. + - An artifact-result case. + - A byte-identical fallthrough case. +- **Small implementation choices that do not change the contract:** + - Journal tool names are cut to 100 characters. + - The row ring keeps the last 100 raw `tool.call` events and projects them once at the end. + - A stalled cursor (a `nextSequence` that does not advance) ends paging with `truncated: true`. +- **Branch checks:** the executor's `worktree-agent-*` branch allow-list was skipped because the orchestrator directed commits on `Jev`, as in 66-02 to 66-04. The cwd-drift sentinel and the protected-branch deny-list ran before every commit. + +--- + +**Total deviations:** 4 auto-fixed (2 Rule 1, of which 1 orchestrator-directed; 2 Rule 2). +**Impact on plan:** each one closes a correctness gap inside the plan's own contract. +- The only footprint beyond `files_modified` is the orchestrator-directed `automation-logger.js` alignment and its paired test. +- There is no new tool parameter and no `mcp/` change. + +## TDD Gate Compliance + +All three tasks are `tdd="true"`, but the plan type is `execute`, and every touched suite is in the `npm test` chain. A standalone RED commit would leave the chain red at that commit. So RED was run and recorded before each implementation, and each task has one `feat(66-06)` commit with the tests and source together. This matches 66-03 and 66-04. +- For Tasks 1 and 2, RED ran against the previous commit's dispatcher. That file was copied into place and then restored; the restore was checked by SHA-1. +- For Task 3, RED ran against the committed Task 2 dispatcher. + +| Task | RED (tests first, previous source) | GREEN | +|------|------------------------------------|-------| +| 1 | in-flight 26 pass / 13 fail (all failures in new cases) | 62 / 0 | +| 2 | 85 / 55 (all failures in new journal cases) | 140 / 0 | +| 3 | the first run aborted on an unguarded read. The two assertions were hardened, and the rerun gave 158 / 56 | 214 / 0 | +| Logger alignment | retention 140 / 1 (the LM Studio fallback case) | 141 / 0 | + +## Issues Encountered + +- graphify's graph is stale for the dispatcher (for example, it places `handleGetSessionMessageRoute` at line 2278; the working tree had it at 2904), as the earlier plans noted. It was used for orientation only, and every anchor was re-read in the working tree. +- 7225 was held by the user's hub (pid 42278) throughout, the same state as the baseline. No port-bound test failed. + +## Known Stubs + +None. Journal sessions return `logs: []` by design, as the plan specifies; the journal keeps no session logs. In-flight snapshots do not carry a `measurement` block, which the plan does not ask for. The runner re-queries after completion. + +## Threat Model Coverage + +- **T-66-03 (information disclosure):** + - Raw events never leave. Only the allowlist fields are returned: selector, domain and error. Artifact and omitted descriptors contribute nothing. + - The page-text, URL-path, query and artifact sentinels are absent from the whole response. + - The response contains no `"inline"`, `"events"` or `"metadata"` key. +- **T-66-05 (tampering):** timing is read only from the symbol, and the string-key spoof test passes. Every timing is re-validated with fsbMs recomputed (`validProjectionTiming`). +- **T-66-DoS:** + - At most 20 pages of 500 events are read, and the ring keeps 100 rows. The 1203-call case reads 3 pages. + - A journal whose pages never end stops at 20 pages with `truncated: true`, and a stalled cursor stops early. +- **T-66-03b (accepted):** the measurement block holds only numbers, model names and fixed labels. + +No new security surface: the journal index fields that `get_session_detail` returns (id, task, status, agent and recording ids, tab ids, outcome) already appear in `list_sessions`. + +## User Setup Required + +None. + +## Next Phase Readiness + +- **66-08 (runner):** + - Read `session.measurement` from `get_session_detail`. MCP-arm sessions are journal sessions (`source: 'journal'`). Use `fsbMs`, `callerGapMs` (an upper bound), `callIntervalMs` (like-for-like with the 66-01 history deltas) and `activeSpanMs`, and check `truncated`. + - Autopilot sessions give provider-first cost and tokens with sources, and `perIteration[].startedAt`. + - Retry `get_session_detail` briefly after `run_task`, because `saveSession` is asynchronous. +- **66-07:** the whole chain is green at `c8a22e4d` (311/311 with 7225 held). A Node 20 run remains part of the phase gate. The new tests use only APIs available in Node 20: dynamic `import()`, `structuredClone` and `pathToFileURL`. +- **Blockers:** none. + +## Self-Check: PASSED + +- All 4 modified files exist on disk. No file was created apart from this SUMMARY. +- All four commits exist: `8beef0db`, `7b4e8de9`, `ea0cbfe0`, `c8a22e4d`. +- No commit in `d73d74ce..HEAD` has a Co-Authored-By or AI attribution line. +- `git diff --name-only d73d74ce HEAD -- .planning/STATE.md .planning/ROADMAP.md .planning/REQUIREMENTS.md mcp/` is empty. +- The working tree was clean apart from this SUMMARY before it was committed. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-07-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-07-PLAN.md new file mode 100644 index 000000000..e992ce210 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-07-PLAN.md @@ -0,0 +1,361 @@ +--- +phase: 66-baseline-and-session-timing +plan: 07 +type: execute +wave: 3 +depends_on: ["66-02", "66-05"] +files_modified: + - scripts/bench/fixtures/_shared/bench.js + - scripts/bench/fixtures/invoice-lookup/index.html + - scripts/bench/fixtures/ticket-triage/index.html + - scripts/bench/fixtures/search-detail/index.html + - scripts/bench/fixtures/search-detail/results.html + - scripts/bench/fixtures/search-detail/detail.html + - scripts/bench/fixtures/search-detail/catalog.js + - scripts/bench/fixtures/wizard-submit/index.html + - scripts/bench/fixtures/settings-change/index.html + - scripts/bench/fixtures/paginate-find/index.html + - scripts/bench/fixtures/paginate-find/items.js + - scripts/bench/fixtures/checklist/index.html + - scripts/bench/fixtures/custom-dropdowns/index.html + - scripts/bench/fixtures/accordion-drilldown/index.html + - scripts/bench/fixtures/confirm-modals/index.html + - scripts/bench/fixtures/tabbed-toggles/index.html + - scripts/bench/fixtures/kanban-moves/index.html + - scripts/bench/tasks/core/core-mixed-invoice-lookup.json + - scripts/bench/tasks/core/core-mixed-ticket-triage.json + - scripts/bench/tasks/core/core-mixed-search-detail.json + - scripts/bench/tasks/core/core-mixed-wizard-submit.json + - scripts/bench/tasks/core/core-mixed-settings-change.json + - scripts/bench/tasks/core/core-mixed-paginate-find.json + - scripts/bench/tasks/core/core-click-checklist.json + - scripts/bench/tasks/core/core-click-custom-dropdowns.json + - scripts/bench/tasks/core/core-click-accordion-drilldown.json + - scripts/bench/tasks/core/core-click-confirm-modals.json + - scripts/bench/tasks/core/core-click-tabbed-toggles.json + - scripts/bench/tasks/core/core-click-kanban-moves.json + - scripts/bench/tasks/live/live-jira-apache-hadoop-1.json + - scripts/bench/tasks/live/live-jira-atlassian-jraserver-1369.json + - scripts/bench/tasks/live/live-uat-showcase-privacy.json + - scripts/bench/tasks/live/live-uat-showcase-support.json + - scripts/bench/tasks/core.lock.json + - tests/bench-corpus.test.mjs +autonomous: true +requirements: [MEAS-03] +user_setup: [] + +must_haves: + truths: + - "The core set is exactly 12 local fixture tasks, 6 mixed and 6 click-heavy. Each needs a server-visible effect (the app's own POST) that the fixture server logs, plus a final-URL state and DOM markers read after the timed window (D-15, D-16, D-17)." + - "The core set is frozen through Phase 71: core.lock.json pins every task's id, version and fixtureSha256, and a test fails on any drift (D-15)." + - "A small live-site set covers both styles D-15 names, with no login needed: two public Jira issue reads, plus two UAT-style reads that confirm FSB's own production showcase (https://full-selfbrowsing.com) shows the expected content. It is defined separately, checked by final URL and page text only, reported separately, and never counted in the release gate. An unreachable site is recorded as skipped (D-15, orchestrator decision 8)." + - "This plan intentionally stays one plan (checker W8). Its size risk is mitigated by the 66-10 one-round smoke over all 12 core tasks (W3), which surfaces any fixture defect per task before the H4 campaign approval." + - "Fixture pages use relative URLs only, load no external resources, keep no cross-run browser state, and contain no in-page probe that posts on DOM mutation, so they do not perturb FSB's settle waits (Pitfall 9)." + - "Every core task's success spec is internally consistent. An offline self-check drives its request evidence through the real fixture server and the claim-blind evaluator: correct bodies pass, and a wrong body fails." + artifacts: + - path: "scripts/bench/tasks/core.lock.json" + provides: "Frozen core corpus lock (id, version, fixtureSha256)" + contains: "fixtureSha256" + - path: "tests/bench-corpus.test.mjs" + provides: "Corpus composition, lock, static-safety and offline evidence self-check" + - path: "scripts/bench/fixtures/_shared/bench.js" + provides: "Shared page helper: benchPost, setMarker, setView" + key_links: + - from: "scripts/bench/tasks/core/*.json" + to: "scripts/bench/lib/tasks.mjs validateTask" + via: "exact-key schema from plan 66-05" + pattern: "schemaVersion" + - from: "scripts/bench/fixtures/*/index.html" + to: "scripts/bench/fixture-server.mjs /r//api/*" + via: "relative fetch('../../api//') with JSON bodies" + pattern: "api/" +--- + + +Author the frozen benchmark corpus: 12 local fixture apps with task definitions (6 mixed, 6 click-heavy), four public live-site reading tasks (2 Jira-style, 2 UAT-style on FSB's production showcase), a lock file that freezes the core set through Phase 71, and a corpus test proving every task is well-formed and verifiable by independent evidence. + +Purpose: D-15 (a core set reused unchanged through Phase 71, with live tasks reported separately), D-16 (12 tasks for about 240 baseline runs) and D-17 (success decided by code from the server log, final URL and DOM markers). +Output: fixture pages, task JSON, `core.lock.json` and `tests/bench-corpus.test.mjs`. + +Plan size note (W8): this plan stays whole on purpose. Its risk, that a fixture has an in-page defect a unit test cannot see, is covered by the 66-10 one-round smoke over all 12 core tasks and both arms (W3). That smoke gives per-task results in the H4 packet before any campaign spend. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-05-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +From plan 66-05 (already implemented): `scripts/bench/lib/tasks.mjs` (`validateTask`, `loadTasks`, `fixtureSha256`, `corpusLock`, `renderInstruction`, `BENCH_TASKS_ROOT`, `BENCH_FIXTURES_ROOT`) and `scripts/bench/fixture-server.mjs` (`startFixtureServer` → `{port, baseUrl, secret, newRunToken, runUrl(token, startPath), evidence(token), clear, close}`), plus the TASK SCHEMA and FIXTURE SERVER CONTRACT stated in 66-05-PLAN.md. From plan 66-02: `scripts/bench/lib/outcome.mjs` (`evaluateOutcome(task, evidence, {token})`). + +FIXTURE PAGE CONVENTIONS (verbatim; every page below follows them): +- Served at `/r//fixtures//`. Every link, script and fetch is RELATIVE. The app API path from a page in `/` is `../../api//`, which resolves to `/r//api//`. +- Shared helper `scripts/bench/fixtures/_shared/bench.js` (loaded as `../_shared/bench.js`, a classic script) defines: + - `benchPost(action, body)`: POSTs JSON to `../../api//`. The slug comes from `document.documentElement.dataset.benchApp`, so every page's `` element carries the literal attribute `data-bench-app=""`. It resolves with the parsed `{ok}`. Pages call it with a literal action string, for example `benchPost('save', ...)`, which lets the corpus test tie each request check to its page source (W2). + - `setMarker(name, value)`: finds or creates a visible element with `data-bench-marker=""` and sets its textContent. + - `setView(params)`: `history.replaceState(null, '', '?' + new URLSearchParams(params))`. + - At load it sets `document.documentElement.dataset.fixtureReady = 'true'`. + Every task's `fixtureFiles` lists `_shared/bench.js`. +- Deterministic in-memory state only: no localStorage, sessionStorage, IndexedDB, cookies, Math.random or Date-dependent data. All fixture runs share one origin (127.0.0.1:), so stored state would leak between runs. +- No external resources: no http(s) URLs, CDNs, remote fonts or images. No MutationObserver, setInterval or background fetches. The only network traffic is page and script GETs plus the explicit user-action POST(s). +- JSON request bodies have a fixed key order and sorted arrays, so `jsonEquals` is exact. +- Accessible markup: labels tied to inputs, buttons with visible text, custom widgets with role and aria attributes. Custom (div-based) controls are deliberate on click-heavy tasks. +- Each page shows a small title and instructions-free UI (the task text lives only in the task JSON). + + + + + + + Task 1: Shared helper and the six mixed fixture apps with their task definitions + scripts/bench/fixtures/_shared/bench.js, scripts/bench/fixtures/invoice-lookup/index.html, scripts/bench/fixtures/ticket-triage/index.html, scripts/bench/fixtures/search-detail/index.html, scripts/bench/fixtures/search-detail/results.html, scripts/bench/fixtures/search-detail/detail.html, scripts/bench/fixtures/search-detail/catalog.js, scripts/bench/fixtures/wizard-submit/index.html, scripts/bench/fixtures/settings-change/index.html, scripts/bench/fixtures/paginate-find/index.html, scripts/bench/fixtures/paginate-find/items.js, scripts/bench/tasks/core/core-mixed-invoice-lookup.json, scripts/bench/tasks/core/core-mixed-ticket-triage.json, scripts/bench/tasks/core/core-mixed-search-detail.json, scripts/bench/tasks/core/core-mixed-wizard-submit.json, scripts/bench/tasks/core/core-mixed-settings-change.json, scripts/bench/tasks/core/core-mixed-paginate-find.json + + - scripts/bench/lib/tasks.mjs (schema rules; validate every JSON with it) + - scripts/bench/fixture-server.mjs (URL layout and API behavior) + - tests/fixtures/bench/tasks-sample/fixtures/sample/index.html and tests/fixtures/bench/tasks-sample/tasks/core/core-sample-save.json (the working example to follow) + - tests/fixtures/screenshot-uat.html line ~119 (fixtureReady marker precedent) + + +Create `_shared/bench.js` per the FIXTURE PAGE CONVENTIONS. Then create each app and its task JSON. Every task has schemaVersion 1, version 1, set 'core', category 'mixed', timeBudgetMs 300000 (paginate-find: 360000), and startPath `/fixtures//index.html`. fixtureFiles lists `_shared/bench.js` plus every file of that app. Each instruction starts with `Open {startUrl}. `. Design the data so each answer is unique and has at least one plausible distractor. + +1. invoice-lookup: a table of 15 invoices (id `INV-10xx`, customer, amount, status Paid/Unpaid) across 4 customers, and a review form (text input labelled "Invoice ID" and a Submit button). The answer is the unpaid Northwind Traders invoice with the highest amount. Distractors: a higher paid Northwind invoice, and a higher unpaid invoice from another customer. Submit calls `benchPost('select', {invoiceId})`, then `setView({view:'selected'})` and `setMarker('selected-invoice', invoiceId)`. + Instruction: "Open {startUrl}. Find the unpaid invoice with the highest amount for customer Northwind Traders, enter its invoice ID in the review form, and submit it." + Success: + - request POST `/api/invoice-lookup/select` count 1, jsonEquals `{"invoiceId":""}` + - request POST `/api/invoice-lookup/select` count 1 (no other submissions) + - finalUrl pathEquals `/fixtures/invoice-lookup/index.html` queryIncludes `{"view":"selected"}` + - marker `selected-invoice` equals the answer +2. ticket-triage: a Jira-like list of 10 tickets (key OPS-2xx, title, labels, reporter, status). Clicking a row opens a detail panel on the same page with a div-based status dropdown (role listbox: To Do, In Progress, Blocked, Done) and a Save button. The answer is the Open ticket labelled `billing` and reported by Dana; distractors are a billing ticket reported by someone else and a Dana ticket without the billing label. Save calls `benchPost('update', {ticketId, status})`, `setView({view:'saved', ticket: ticketId})` and `setMarker('ticket-status', ticketId + ':' + status)`. + Instruction: "Open {startUrl}. Find the open ticket labelled billing that was reported by Dana, change its status to In Progress, and save it." + Success: + - request with jsonEquals `{"ticketId":"","status":"In Progress"}` count 1 + - total update requests count 1 + - finalUrl queryIncludes `{"view":"saved","ticket":""}` + - marker `ticket-status` equals `:In Progress` +3. search-detail: index.html has a search form that navigates (GET) to `results.html?q=`. results.html filters 30 products from `catalog.js` (fields sku, name, category, waterproof, price) client-side and links each result to `detail.html?sku=`. detail.html shows the attributes and an "Add to list" button that calls `benchPost('add', {sku})`, `setView({sku, view:'added'})` and `setMarker('list-added', sku)`. Exactly one trail running shoe is waterproof and under $120; distractors are a waterproof one at $135 and a non-waterproof one at $95. + Instruction: "Open {startUrl}. Search for trail running shoe, open the result that is waterproof and costs under $120, and add it to the list." + Success: + - request jsonEquals `{"sku":""}` count 1 + - total add requests count 1 + - finalUrl pathEquals `/fixtures/search-detail/detail.html` queryIncludes `{"sku":"","view":"added"}` + - marker `list-added` equals the answer +4. wizard-submit: a 3-step wizard on one page. Step 1 is an "Account name" text input; step 2 is plan radios (Starter, Team, Enterprise) and a "Seats" number input; step 3 is a review summary with Back and Submit. Submit calls `benchPost('submit', {accountName, plan, seats})`, with plan in lowercase and seats as a number, then `setView({view:'done'})` and `setMarker('wizard-result', accountName + '|' + plan + '|' + seats)`. + Instruction: "Open {startUrl}. Complete the setup wizard for the account Acme Robotics on the Team plan with 12 seats and submit it." + Success: + - request jsonEquals `{"accountName":"Acme Robotics","plan":"team","seats":12}` count 1 + - total submit count 1 + - finalUrl queryIncludes `{"view":"done"}` + - marker `wizard-result` equals `Acme Robotics|team|12` +5. settings-change: a read-only profile card shows "Time zone: America/New_York (UTC-05:00)". A settings form has a "Digest time zone" select (at least 6 IANA zones, default UTC), a "Weekly digest" toggle (role switch, default off) and Save. Save calls `benchPost('save', {digestTimezone, weeklyDigest})`, `setView({view:'saved'})` and `setMarker('settings-saved', digestTimezone + '|' + (weeklyDigest ? 'on' : 'off'))`. + Instruction: "Open {startUrl}. Read the time zone shown on the profile card, set the digest time zone to the same zone, turn on the weekly digest, and save." + Success: + - request jsonEquals `{"digestTimezone":"America/New_York","weeklyDigest":true}` count 1 + - total save count 1 + - finalUrl queryIncludes `{"view":"saved"}` + - marker `settings-saved` equals `America/New_York|on` +6. paginate-find: index.html renders page N (query `page`, default 1) of 50 items from `items.js`, 10 per page, with Previous and Next buttons that set `?page=N` by navigation. Each item has an "Add to cart" button. The answer is item `ITM-0437` "Blue ceramic mug, 12 oz" on page 4; distractors are a 16 oz blue mug on page 2 and a 12 oz green mug on page 5. Add calls `benchPost('add', {itemId})`, `setView({page:String(N), view:'added'})` and `setMarker('cart-count', '1')`. + Instruction: "Open {startUrl}. Page through the catalog to find the 12 oz blue ceramic mug and add it to the cart." + Success: + - request jsonEquals `{"itemId":"ITM-0437"}` count 1 + - total add count 1 + - finalUrl queryIncludes `{"page":"4","view":"added"}` + - marker `cart-count` equals `1` + +After writing, validate every task: `loadTasks({set:'core'})` from scripts/bench/lib/tasks.mjs must not throw for these six. Commit: `feat(66-07): add mixed benchmark fixtures and tasks`. No Co-Authored-By or AI attribution. + + + node -e "import('./scripts/bench/lib/tasks.mjs').then(m=>{const t=m.loadTasks({set:'core'});const mixed=t.filter(x=>x.category==='mixed');if(mixed.length!==6)process.exit(1);console.log(mixed.map(x=>x.id).join(','))})" + + + - The verify command prints six ids, all starting `core-mixed-`, and exits 0. + - `grep -rlE "https?://" scripts/bench/fixtures` prints nothing. + - `grep -rlE "localStorage|sessionStorage|indexedDB|document\.cookie|MutationObserver|setInterval|Math\.random" scripts/bench/fixtures` prints nothing. + + Six mixed fixture apps with unique, distractor-bearing answers exist, and their tasks validate against the schema. + + + + Task 2: The six click-heavy fixture apps, their task definitions, and the live reading set + scripts/bench/fixtures/checklist/index.html, scripts/bench/fixtures/custom-dropdowns/index.html, scripts/bench/fixtures/accordion-drilldown/index.html, scripts/bench/fixtures/confirm-modals/index.html, scripts/bench/fixtures/tabbed-toggles/index.html, scripts/bench/fixtures/kanban-moves/index.html, scripts/bench/tasks/core/core-click-checklist.json, scripts/bench/tasks/core/core-click-custom-dropdowns.json, scripts/bench/tasks/core/core-click-accordion-drilldown.json, scripts/bench/tasks/core/core-click-confirm-modals.json, scripts/bench/tasks/core/core-click-tabbed-toggles.json, scripts/bench/tasks/core/core-click-kanban-moves.json, scripts/bench/tasks/live/live-jira-apache-hadoop-1.json, scripts/bench/tasks/live/live-jira-atlassian-jraserver-1369.json, scripts/bench/tasks/live/live-uat-showcase-privacy.json, scripts/bench/tasks/live/live-uat-showcase-support.json + + - scripts/bench/fixtures/_shared/bench.js and one mixed app from Task 1 (to match conventions) + - scripts/bench/lib/tasks.mjs (live task rules: https startUrl, finalUrl urlIncludes, text includes, category reading) + + +Each core task here uses category 'click-heavy', timeBudgetMs 300000, the same schema and conventions as Task 1, and no typing. + +1. checklist: 10 labelled checkboxes (Alpha, Bravo, Charlie, Delta, Echo, Foxtrot, Golf, Hotel, India, Juliet), none checked initially, and a Save button. Save calls `benchPost('save', {checked:})`, `setView({view:'saved'})` and `setMarker('saved-count', String(n))`. + Instruction: "Open {startUrl}. Tick Alpha, Delta, Echo and Juliet, leave every other box unticked, then press Save." + Success: + - request jsonEquals `{"checked":["alpha","delta","echo","juliet"]}` count 1 + - total save count 1 + - finalUrl queryIncludes `{"view":"saved"}` + - marker `saved-count` equals `4` +2. custom-dropdowns: four div-based dropdowns (role combobox and listbox, opened by click) with the option sets below, and a Submit button. Values are slugs; visible labels differ. + - Region: US East `us-east`, US West `us-west`, Europe (West) `eu-west`, Asia Pacific `ap-south` + - Size: Small, Medium, Large, X-Large + - Color: Silver, Graphite, Navy, Sand + - Shipping: Standard, Express, Overnight + Submit calls `benchPost('submit', {region, size, color, shipping})`, `setView({view:'submitted'})` and `setMarker('selection', region + '|' + size + '|' + color + '|' + shipping)`. + Instruction: "Open {startUrl}. Choose Europe (West) for Region, Large for Size, Graphite for Color and Express for Shipping, then submit." + Success: + - request jsonEquals `{"region":"eu-west","size":"large","color":"graphite","shipping":"express"}` count 1 + - total submit count 1 + - finalUrl queryIncludes `{"view":"submitted"}` + - marker `selection` equals `eu-west|large|graphite|express` +3. accordion-drilldown: a 3-level accordion (collapsed initially). Organization expands to Security, Billing and Members. Security expands to Sessions, Passwords and Audit. Sessions contains toggles "Require re-authentication for exports" and "Require re-authentication for imports". Audit contains a distractor "Require re-authentication for exports (audit copy)". Clicking a toggle calls `benchPost('toggle', {path, enabled})`, where path is slash-joined lowercase ids (for example `organization/security/sessions/reauth-exports`), then `setView({view:'updated'})` and `setMarker(, enabled ? 'on' : 'off')`. + Instruction: "Open {startUrl}. In Organization, then Security, then Sessions, turn on Require re-authentication for exports. Change nothing else." + Success: + - request jsonEquals `{"path":"organization/security/sessions/reauth-exports","enabled":true}` count 1 + - total toggle count 1 + - finalUrl queryIncludes `{"view":"updated"}` + - marker `reauth-exports` equals `on` +4. confirm-modals: 6 records (REC-1..REC-6 with names; REC-3 is "Q3 draft forecast" and REC-5 is "Q3 final forecast"), each with a Delete button. Delete opens modal 1 ("Delete ?" with Continue and Cancel). Continue opens modal 2 ("This cannot be undone." with Delete permanently and Cancel). Delete permanently calls `benchPost('delete', {recordId})`, removes the row, calls `setView({view:'deleted'})` and sets `setMarker('deleted-records', )`. + Instruction: "Open {startUrl}. Delete the record named Q3 draft forecast, confirming both prompts. Do not delete anything else." + Success: + - request jsonEquals `{"recordId":"REC-3"}` count 1 + - total delete count 1 + - finalUrl queryIncludes `{"view":"deleted"}` + - marker `deleted-records` equals `REC-3` +5. tabbed-toggles: 3 tabs (role tablist), each with 3 switches. + - General: Compact mode (off), Show avatars (on), Reduce motion (off) + - Notifications: Email mentions (on), Push mentions (on), Weekly summary (off) + - Privacy: Hide online status (off), Share read receipts (on), Allow indexing (off) + A Save button calls `benchPost('save', )`, `setView({view:'saved'})` and `setMarker('changed-count', String())`. + Instruction: "Open {startUrl}. Turn on Compact mode on the General tab, turn off Email mentions on the Notifications tab, turn on Hide online status on the Privacy tab, then press Save." + Success: + - request jsonEquals with the full nine-key object reflecting exactly those three changes, count 1 + - total save count 1 + - finalUrl queryIncludes `{"view":"saved"}` + - marker `changed-count` equals `3` +6. kanban-moves: columns To do, Doing and Done, with 6 cards. Initially: To do holds Write release notes, Fix login redirect, Update dependencies and Draft roadmap; Doing holds Review PR 42; Done holds Set up CI. Each card has a "Move to" button opening a menu of the other columns. A move calls `benchPost('move', {card, column})`, with column slugs `todo`/`doing`/`done`, updates the board, and calls `setMarker('column-doing', )` and `setMarker('column-done', ...)`. The URL is unchanged. + Instruction: "Open {startUrl}. Move Write release notes to Done, Fix login redirect to Doing, and Update dependencies to Done." + Success: + - three request checks, each count 1 with jsonEquals `{"card":"Write release notes","column":"done"}`, `{"card":"Fix login redirect","column":"doing"}` and `{"card":"Update dependencies","column":"done"}` + - total move count 3 + - finalUrl pathEquals `/fixtures/kanban-moves/index.html` + - marker `column-done` equals `Set up CI|Update dependencies|Write release notes` + - marker `column-doing` equals `Fix login redirect|Review PR 42` + +Live set (set 'live', category 'reading', version 1, timeBudgetMs 180000). These pages were verified public and login-free on 2026-09-29. +7. live-jira-apache-hadoop-1: startUrl `https://issues.apache.org/jira/browse/HADOOP-1`. + Instruction: "Open {startUrl} and report the issue's status and resolution." + Success: + - finalUrl urlIncludes `/jira/browse/HADOOP-1` + - text includes `HADOOP-1` + - text includes `Fixed` +8. live-jira-atlassian-jraserver-1369: startUrl `https://jira.atlassian.com/browse/JRASERVER-1369`. + Instruction: "Open {startUrl} and report the issue's resolution." + Success: + - finalUrl urlIncludes `/browse/JRASERVER-1369` + - text includes `JRASERVER-1369` + +UAT-style live reads (orchestrator decision 8). These confirm, as a UAT pass would, that FSB's own production showcase shows the expected content. The production pages are prerendered English pages, verified reachable on 2026-09-30, and each has a stable heading plus a stable text fact from the showcase source (`showcase/angular/src/app/pages/privacy/privacy-page.component.html`, `showcase/angular/src/app/pages/support/support-page.component.html`). `/pricing` does not exist (404), so it is not used. +9. live-uat-showcase-privacy: startUrl `https://full-selfbrowsing.com/privacy`. + Instruction: "Open {startUrl} and check, as a UAT reviewer would, that the privacy policy has a section stating FSB does no third-party tracking. Report that section's heading." + Success: + - finalUrl urlIncludes `/privacy` (robust to a locale prefix; live evidence is already limited to the startUrl host) + - text includes `Privacy Policy` + - text includes `No Third-Party Tracking` +10. live-uat-showcase-support: startUrl `https://full-selfbrowsing.com/support`. + Instruction: "Open {startUrl} and check, as a UAT reviewer would, that the support page has a Frequently Asked Questions section and a Common Issues section. Report both headings." + Success: + - finalUrl urlIncludes `/support` + - text includes `Frequently Asked Questions` + - text includes `Common Issues` + +Live tasks are reported separately and never part of the core lock or the release gate. A task whose site is unreachable is recorded as skipped and does not block Phase 66 (orchestrator decision 8). The UAT checks expect the English pages, and the bench Chrome uses English as its first language (66-10 H2). + +Validate with `loadTasks({set:'core'})` (12 tasks, 6 per category) and `loadTasks({set:'live'})` (4 tasks: 2 Jira-style, 2 UAT-style). Commit: `feat(66-07): add click-heavy benchmark fixtures and the live reading set`. No Co-Authored-By or AI attribution. + + + node -e "import('./scripts/bench/lib/tasks.mjs').then(m=>{const c=m.loadTasks({set:'core'});const l=m.loadTasks({set:'live'});const k=c.filter(x=>x.category==='click-heavy').length;if(c.length!==12||k!==6||l.length<2||l.length>4)process.exit(1);console.log('core',c.length,'click-heavy',k,'live',l.length)})" + + + - The verify command prints `core 12 click-heavy 6 live 4` and exits 0. The live count stays within 2..4. + - `grep -rlE "https?://" scripts/bench/fixtures` prints nothing, and `grep -rlE "localStorage|sessionStorage|indexedDB|document\.cookie|MutationObserver|setInterval|Math\.random" scripts/bench/fixtures` prints nothing. + - All four live task files contain `"category": "reading"` or its compact JSON form `"category":"reading"`, and exactly two startUrls begin with `https://full-selfbrowsing.com/`. + + The 12-task core corpus (6+6) and a 4-task live reading set (2 Jira-style, 2 UAT-style) exist and validate. + + + + Task 3: Freeze the core corpus with a lock file, and add the corpus test with an offline evidence self-check + scripts/bench/tasks/core.lock.json, tests/bench-corpus.test.mjs + + - scripts/bench/lib/tasks.mjs (corpusLock, fixtureSha256), scripts/bench/fixture-server.mjs, scripts/bench/lib/outcome.mjs + - tests/bench-tasks.test.mjs and tests/bench-fixture-server.test.mjs (test style from plan 66-05) + + + - loadTasks({set:'core'}) returns exactly 12 tasks with unique ids, 6 'mixed' and 6 'click-heavy'. loadTasks({set:'live'}) returns 4 'reading' tasks with https startUrls (within the allowed 2..4): two Jira-style and two whose startUrl begins with `https://full-selfbrowsing.com/`. + - `core.lock.json`'s tasks array deep-equals corpusLock(loadTasks({set:'core'}), BENCH_FIXTURES_ROOT). The lock's schemaVersion is 1, and its note states the D-15 freeze. + - For every core task, derive `` and `` from each request check path `/api//`. The task's fixture files must contain the literal attribute `data-bench-app=""` and the literal call prefix `benchPost(''` (W2). benchPost builds the URL at runtime, so the literal path is not asserted. Each marker name must appear literally in the fixture files, and fixtureFiles must include `_shared/bench.js`. + - Static safety: no fixture file contains an http(s) URL, storage APIs, document.cookie, MutationObserver, setInterval or Math.random. + - Offline self-check against the real fixture server (port 0), for every core task with a fresh token: + - POST each request check's jsonEquals body `count` times to `/r//api/...`; + - build finalUrl from the finalUrl check and markers from the marker checks; + - evaluateOutcome(task, {requests: evidence(token).requests, finalUrl, markers}, {token}) passes; + - after sending one extra POST whose body differs from any jsonEquals, the total-count checks fail and evaluateOutcome returns pass:false. + - renderInstruction(task, server.runUrl(token, task.startPath)) contains `/r//fixtures/` and no `{startUrl}` token. + + +1. Generate `scripts/bench/tasks/core.lock.json` with a one-off `node -e` that imports `corpusLock` and `loadTasks` from scripts/bench/lib/tasks.mjs. Write `{ "schemaVersion": 1, "note": "Core benchmark set frozen through Phase 71 (D-15). Changing any core task or fixture requires a version bump and a deliberately regenerated lock.", "tasks": [ {id, version, fixtureSha256}, ... ] }` as pretty JSON with a trailing newline. Add no timestamp (the lock must be deterministic). +2. Create `tests/bench-corpus.test.mjs` (node:test, relative imports, Node 20-safe) covering every behavior above. Close the server in `finally`. Build the core-task finalUrl as `server.runUrl(token, check.pathEquals) + (queryIncludes ? '?' + new URLSearchParams(queryIncludes) : '')`, or the task's startPath URL when the check has no pathEquals. +3. Run `node --test tests/bench-corpus.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs`. Do NOT register the test in package.json (plan 66-09 does that). + +Commit: `test(66-07): freeze the core benchmark corpus and self-check every task's evidence`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-corpus.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs + + + - `node --test tests/bench-corpus.test.mjs` exits 0. + - `node -e "const l=require('./scripts/bench/tasks/core.lock.json');if(l.schemaVersion!==1||l.tasks.length!==12||l.tasks.some(t=>!/^[0-9a-f]{64}$/.test(t.fixtureSha256)))process.exit(1)"` exits 0. + - `git diff --quiet HEAD -- package.json tests/delegation-phase-contract.test.js` (the chain is untouched in this plan). + + The core corpus is frozen by a verified lock. Every task's success spec is proven evaluable through the real server and the claim-blind evaluator. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Agent under test → fixture pages | Agents interact with pages and could try to satisfy checks by side channels. | +| Live public sites → bench | Live pages are third-party content. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-04 | Spoofing | task success specs | mitigate | Success needs the server-logged POST with an exact body AND an exact total count (so extra or wrong actions fail) AND URL and DOM state read after the window. The evidence endpoint is secret-gated (66-05). Agent claims are never inputs (66-02). | +| T-66-06 | Tampering | fixture files | mitigate | Relative URLs only, no external resources, no storage, no timers or observers (static test). Fixtures are frozen by core.lock.json (hash drift fails the test). | +| T-66-02 | Information disclosure | live tasks | mitigate | Live checks store pass/fail only (text never persisted; 66-02 evaluator details are booleans). The targets are public, login-free issue pages, so no private data is involved. | +| T-66-DoS | Denial of service | live sites | accept | Two public pages at low frequency, reported separately and optional (orchestrator decision 8). | + + + +- `node --test tests/bench-corpus.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs` +- The static-safety greps in Tasks 1 and 2 print nothing. + + + +- D-15, D-16 and D-17: 12 frozen, verifiable core tasks (6 mixed, 6 click-heavy) and a separate live set exist, and every task's evidence path is proven offline. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Do not edit package.json or tests/delegation-phase-contract.test.js in this plan. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-07-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-07-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-07-SUMMARY.md new file mode 100644 index 000000000..3f8ee4f03 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-07-SUMMARY.md @@ -0,0 +1,347 @@ +--- +phase: 66-baseline-and-session-timing +plan: 07 +subsystem: testing +tags: [benchmark, corpus, fixtures, core-lock, live-set, evidence, chrome-headless-shell, node-test] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 02) + provides: "scripts/bench/lib/outcome.mjs evaluateOutcome (claim-blind)" + - phase: 66-baseline-and-session-timing (plan 05) + provides: "scripts/bench/lib/tasks.mjs (validateTask, loadTasks, fixtureSha256, corpusLock, renderInstruction) and scripts/bench/fixture-server.mjs" +provides: + - "12 frozen core fixture apps and task definitions (6 mixed, 6 click-heavy) under scripts/bench/fixtures/ and scripts/bench/tasks/core/" + - "scripts/bench/fixtures/_shared/bench.js: benchPost, setMarker, setView, onActivate, the fixtureReady marker, and a one-click-one-activation guard" + - "4 live reading tasks under scripts/bench/tasks/live/: 2 public Jira issues and 2 UAT-style reads of https://full-selfbrowsing.com" + - "scripts/bench/tasks/core.lock.json: id, version and fixtureSha256 for every core task, frozen through Phase 71" + - "tests/bench-corpus.test.mjs: 17 node:test cases, unregistered until 66-09" +affects: [66-08, 66-09, 66-10, 66-11, 71, benchmark runner, baseline campaign] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "A fixture app is a static page with relative URLs and in-memory state. Each user action sends one explicit POST, and that request is the evidence" + - "One click, one activation: a document capture listener cancels a repeat click on the same element before the microtask queue drains" + - "Every exact-body request check is paired with one exact total for its endpoint" + - "Byte-hashed fixtures are marked -text in .gitattributes, so no checkout can rewrite them" + +key-files: + created: + - scripts/bench/fixtures/_shared/bench.js + - scripts/bench/fixtures/invoice-lookup/index.html + - scripts/bench/fixtures/ticket-triage/index.html + - scripts/bench/fixtures/search-detail/index.html + - scripts/bench/fixtures/search-detail/results.html + - scripts/bench/fixtures/search-detail/detail.html + - scripts/bench/fixtures/search-detail/catalog.js + - scripts/bench/fixtures/wizard-submit/index.html + - scripts/bench/fixtures/settings-change/index.html + - scripts/bench/fixtures/paginate-find/index.html + - scripts/bench/fixtures/paginate-find/items.js + - scripts/bench/fixtures/checklist/index.html + - scripts/bench/fixtures/custom-dropdowns/index.html + - scripts/bench/fixtures/accordion-drilldown/index.html + - scripts/bench/fixtures/confirm-modals/index.html + - scripts/bench/fixtures/tabbed-toggles/index.html + - scripts/bench/fixtures/kanban-moves/index.html + - scripts/bench/tasks/core/*.json (12 files) + - scripts/bench/tasks/live/*.json (4 files) + - scripts/bench/tasks/core.lock.json + - tests/bench-corpus.test.mjs + - .planning/phases/66-baseline-and-session-timing/deferred-items.md + modified: + - .gitattributes + +key-decisions: + - "bench.js cancels a repeat click on the same element within one synchronous turn. FSB's content click fires two click events per click, so without the guard every toggle flips back and every save posts twice" + - "Custom options carry tabindex 0, because FSB's snapshot lists [tabindex] but not role=option. Repeated buttons carry visually hidden item context, and listboxes and menus expand inline so nothing overlays the next control" + - "Pages fit a short viewport (FSB snapshots the viewport only): invoice table and form side by side, ticket list without wrapping cells" + - "The lock is deterministic pretty JSON with no timestamp; .gitattributes marks scripts/bench/fixtures/** -text" + - "TDD for Task 3 was a RED test commit then a GREEN commit; the GREEN commit keeps the plan's own test(66-07) message" + +patterns-established: + - "A core task changes only by a version bump plus a deliberately regenerated core.lock.json; tests/bench-corpus.test.mjs fails on any byte of drift" + - "Runner-side marker reads: [data-bench-marker] textContent after the timed window; the kanban markers exist from page load, the others appear on the first successful action" + +requirements-completed: [MEAS-03] + +# Metrics +duration: 46min +completed: 2026-09-30 +--- + +# Phase 66 Plan 07: Frozen Benchmark Corpus Summary + +**A frozen corpus of 12 local fixture apps (6 mixed, 6 click-heavy) whose success is decided by the app's own server-logged POST, the final URL and DOM markers, plus a separate 4-task live reading set. core.lock.json pins every byte through Phase 71. Every task was driven end to end in a headless browser with FSB's exact click sequence, which revealed that FSB's content click fires twice.** + +This plan delivers part of MEAS-03: the corpus the runner (66-08) and the baseline campaign measure. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched; the orchestrator owns them. + +## Performance + +- **Duration:** 46 min +- **Started:** 2026-09-30T08:48:39Z +- **Completed:** 2026-09-30T09:34:37Z +- **Tasks:** 3/3 +- **Files:** 35 created (exactly the plan's `files_modified`), plus `.gitattributes` modified and `deferred-items.md` created + +## The core set + +Every task posts JSON through `benchPost` to `/r//api//`. Each passes only if all of these hold: +- the exact body is logged the expected number of times; +- the endpoint's total count is exact; +- the final URL matches; +- the DOM marker(s) match. + +| Task | The agent must | Answer | Distractors | +|------|----------------|--------|-------------| +| core-mixed-invoice-lookup | find the largest unpaid Northwind Traders invoice among 15, type its ID, submit | INV-1029 ($2,145.75) | paid Northwind $4,310 and $2,990; other customers' unpaid $3,975, $3,120.40, $2,780; the close second INV-1038 ($1,980) | +| core-mixed-ticket-triage | open the open billing ticket reported by Dana, set In Progress in a div combobox, save | OPS-226 | billing by Priya or Sam; Dana without billing; billing by Dana but Done (OPS-233) | +| core-mixed-search-detail | search, open details, add the waterproof trail running shoe under $120 | TRS-4427 ($112) | waterproof at $135 (TRS-4431), dry at $95 (TRS-4410), dry at $104, laces at $12. Waterproofing shows only on the detail page | +| core-mixed-wizard-submit | three steps: Acme Robotics, Team, 12 seats, submit | `{accountName, plan:"team", seats:12}` | the Starter and Enterprise radios; review, Back and validation paths | +| core-mixed-settings-change | copy the profile time zone into a native select, turn a div switch on, save | America/New_York, on | America/Toronto has the same UTC-05:00 offset; the default is UTC | +| core-mixed-paginate-find | page with Next to page 4, add the 12 oz blue ceramic mug | ITM-0437 | 16 oz blue mug (page 2), blue stainless travel mug 12 oz (page 3), green mug 12 oz (page 5), blue ceramic bowl 12 oz (page 1) | +| core-click-checklist | tick Alpha, Delta, Echo and Juliet out of 10 native checkboxes, save | `["alpha","delta","echo","juliet"]` | the six other boxes | +| core-click-custom-dropdowns | pick in four div comboboxes whose values differ from their labels, submit | eu-west, large, graphite, express | X-Large contains "Large"; slug values such as `ap-south` for Asia Pacific | +| core-click-accordion-drilldown | Organization, then Security, then Sessions; turn one auto-saving switch on | path `organization/security/sessions/reauth-exports` | the "(audit copy)" switch under Audit, and the imports switch | +| core-click-confirm-modals | Delete, Continue, then "Delete permanently" for one record | REC-3 "Q3 draft forecast" | REC-5 "Q3 final forecast", REC-6 "Travel policy draft" | +| core-click-tabbed-toggles | three switches across three tabs, save all nine keys | 3 changed from defaults | six more switches, some on by default | +| core-click-kanban-moves | three moves through "Move to" menus | done: Set up CI, Update dependencies, Write release notes; doing: Fix login redirect, Review PR 42 | Draft roadmap, and moving through Doing first | + +## The live set (reported separately, never in the lock or the release gate) + +| Task | Start URL | Checks | +|------|-----------|--------| +| live-jira-apache-hadoop-1 | `https://issues.apache.org/jira/browse/HADOOP-1` | URL has `/jira/browse/HADOOP-1`; text has `HADOOP-1` and `Fixed` | +| live-jira-atlassian-jraserver-1369 | `https://jira.atlassian.com/browse/JRASERVER-1369` | URL has `/browse/JRASERVER-1369`; text has `JRASERVER-1369` | +| live-uat-showcase-privacy | `https://full-selfbrowsing.com/privacy` | URL has `/privacy`; text has `Privacy Policy` and `No Third-Party Tracking` | +| live-uat-showcase-support | `https://full-selfbrowsing.com/support` | URL has `/support`; text has `Frequently Asked Questions` and `Common Issues` | + +- All four facts were re-checked with plain HTTPS on 2026-09-30: + - privacy: `Privacy Policy` ×1, `No Third-Party Tracking` ×4 + - support: both headings ×1 + - HADOOP-1: `Fixed` ×1 + - JRASERVER-1369: the key ×23 +- None of the checked showcase headings is styled `text-transform: uppercase`. That rule applies only to `.section-label`, `.trouble-fix-label` and footer `h4`, so `innerText` keeps the checked case. + +## Key finding: FSB's content click fires twice + +`extension/content/actions.js` `click` (lines ~2436-2441) and `clickAtCoordinates` (~148-155) dispatch synthetic `mousedown`/`mouseup`/`click` and then also call `element.click()`. Both benchmark arms use this path. + +I replayed that exact sequence from an isolated world in chrome-headless-shell 146, both after an `await` (as FSB does) and synchronously: + +| Page | Toggle button | Native checkbox (direct / via label) | `role="switch"` div | Form submit | +|------|---------------|--------------------------------------|---------------------|-------------| +| without the guard | flips back (2 activations) | ends unchecked | ends unchanged | 2 submits | +| with `bench.js` guard | on (1) | checked | on | 1 submit | +| trusted CDP click, either page | on (1) | checked | on | 1 submit | + +Without the guard, every click-heavy task would be unmeasurable. Every switch would read "no effect", and the accordion's auto-saving switch would send on and off (count 2). `bench.js` therefore cancels a repeat click on the same element before the microtask queue drains. Real input never produces such a repeat. + +The product bug is logged in `deferred-items.md`; it is out of scope here (no `extension/**` edits). + +## Verification Evidence + +- **Functional check of every fixture in a real browser** (throwaway harness in `/tmp`, never committed): + - It used the standalone puppeteer `chrome-headless-shell` 146 with a temporary profile. The user's Chrome was not touched. + - **101/101 scenarios** behaved as expected. + - Each task's intended path passes under FSB-style clicks and under trusted CDP clicks. + - Variants that pass include keyboard use, label clicks, the `check_box` path, Enter via the `press_enter` fallback, detours, cancels and outside clicks. + - Every distractor, double save and extra action fails. + - Every scenario also required: no page exception, no HTTP status of 400 or more, and no GET outside the task's `fixtureFiles`. +- **Layout:** at 1280×800 the content of all 12 start pages ends by 669px, measured with a task's popup open where it has one. Invoice was 841px and ticket triage 756px before the fix. The search results and detail pages are shorter than the start page. +- **Tests:** + - `node --test tests/bench-corpus.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs` passes **122/122** on Node 24.14.1. + - The same run passes 122/122 on **Node 20.20.2**, a binary already in the local npx cache (nothing was downloaded). + - The other bench suites (stats, outcome, identity, claude-stream) pass 96/96. +- **Mutation probes (10), all caught, and each file was restored byte for byte:** + + | Mutation | Guard that failed | + |----------|-------------------| + | one fixture byte | lock | + | `setInterval` in a page | static safety | + | total count raised | pairing and self-checks | + | answer moved to INV-1038 | invoice answer | + | `data-bench-app` renamed | W2 | + | action renamed | W2 | + | absolute helper path | static safety, W2 and served references | + | timestamp in the lock | lock byte determinism | + | marker renamed | W2 | + | unlisted file in an app directory | listed-and-hashed | + +- **Acceptance checks, all as specified:** + - The Task 1 command printed six `core-mixed-` ids. + - The Task 2 command printed `core 12 click-heavy 6 live 4`. + - Both greps over `scripts/bench/fixtures` printed nothing. + - 4 live files are `"category": "reading"`, and exactly 2 start with `https://full-selfbrowsing.com/`. + - The lock check exited 0. + - `git diff --quiet HEAD -- package.json tests/delegation-phase-contract.test.js` exited 0. + +## Task Commits + +1. **Task 1: Shared helper and the six mixed apps:** `d07021ad` (feat) +2. **Layout fix to two Task 1 pages** (found in Task 2's viewport check): `75e69893` (fix) +3. **Task 2: Six click-heavy apps and the live reading set:** `e830f2dd` (feat) +4. **Task 3: Lock and corpus test (TDD):** + - `5d8204f1` (test, RED): 16/17 passed; the lock case failed with ENOENT. + - `f7b9e6ed` (test, GREEN): the plan's message `test(66-07): freeze the core benchmark corpus and self-check every task's evidence`. + +**Plan metadata:** the `docs(66-07)` commit that adds this SUMMARY and `deferred-items.md`. + +## Files Created/Modified + +- `scripts/bench/fixtures/_shared/bench.js`: the shared page helper and the click guard. +- `scripts/bench/fixtures/<12 apps>/…` (16 files): the fixture pages, plus `catalog.js` and `items.js`. +- `scripts/bench/tasks/core/*.json` (12) and `scripts/bench/tasks/live/*.json` (4): task definitions, all `version: 1`. +- `scripts/bench/tasks/core.lock.json`: the frozen lock. +- `tests/bench-corpus.test.mjs`: 17 cases. + - Composition of both sets. + - Lock equality and byte determinism. + - Every app file listed and hashed. + - W2 ties: request path to `data-bench-app` and `benchPost(''`, and markers to page source. + - Exact bodies paired with exact totals. + - Static safety. + - Answers re-derived from the page data for 11 of the 12 tasks. + - The offline self-check: correct bodies pass, while one extra body or a near-miss body fails. + - Served bytes and relative references. + - `renderInstruction`. + + It is not registered; 66-09 adds it to the chain. +- `.gitattributes`: `scripts/bench/fixtures/** -text`. +- `.planning/phases/66-baseline-and-session-timing/deferred-items.md`: the FSB double-click finding. + +## Decisions Made + +- **Answers are validated before a request is sent only where a real app would:** + - Invoice IDs are trimmed, uppercased and must exist in the table. + - The wizard checks each step. + - The dropdowns refuse a submit until all four are chosen. + + A wrong but valid choice (a distractor) is still posted and fails. +- **Busy guards only stop overlapping requests.** A second deliberate save after the first finishes is posted and fails the exact total. "Add" buttons become "Added" and stay disabled, as a real shop would. +- **Navigation targets are idempotent:** wizard buttons go to a named step, kanban ignores a move to the card's own column, and the confirm dialogs track their step. +- **The search index adds a `waterproof` tag.** `waterproof trail running shoe` narrows the results to three. The literal query from the instruction returns 7 results and does not show waterproofing. +- **Markers:** + - `cart-count` reports this page view's adds, so it is `1` on success. + - The kanban markers are set at load, so they are always truthful. + - Accordion markers are created in a "Saved changes" list on the first toggle. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] FSB's content click activated every fixture control twice** +- **Found during:** Task 1 (read from `extension/content/actions.js`, then measured) +- **Issue:** FSB's click fires two click events per click. Toggles flip back, native checkboxes stay unchecked, and submit or auto-save handlers post twice. The click-heavy corpus could not have been completed through FSB's `click` tool. +- **Fix:** + - `bench.js` cancels a repeat click on the same element within one synchronous turn. + - `bench.js` also exports `onActivate` (click plus Enter or Space for div controls), in addition to the plan's three helpers. +- **Files modified:** `scripts/bench/fixtures/_shared/bench.js` +- **Verification:** the guard off/on experiment above, and all 101 scenarios in both click modes. +- **Committed in:** `d07021ad` + +**2. [Rule 2 - Missing critical functionality] Custom controls FSB could not see or tell apart** +- **Found during:** Task 1 +- **Issue:** + - FSB's snapshot candidate selector (`dom-analysis.js` ~1848) has no `[role="option"]`. + - Its element description prefers `aria-label`, then text under 50 characters, so "Move to", "Add to cart" and "Delete" repeated on every row would be indistinguishable. +- **Fix:** + - Options have `tabindex="0"`. + - Comboboxes and switches carry `aria-label`. + - Repeated buttons carry visually hidden item names. + - Listboxes and menus expand inline rather than overlay. +- **Files modified:** the ticket-triage, custom-dropdowns, paginate-find, confirm-modals and kanban-moves pages +- **Committed in:** `d07021ad`, `e830f2dd` + +**3. [Rule 1 - Bug] Two pages taller than a real browser viewport** +- **Found during:** Task 2's layout check +- **Issue:** FSB's default snapshot is viewport-only. Invoice (841px) and ticket triage with its panel open (756px) would have cost scroll steps that are not part of the task. +- **Fix:** invoice table and form side by side, and ticket cells that do not wrap (669px and 566px). +- **Files modified:** `invoice-lookup/index.html`, `ticket-triage/index.html` +- **Committed in:** `75e69893` + +**4. [Rule 2 - Missing critical functionality] Line-ending conversion would break the byte-hashed lock** +- **Found during:** Task 3 +- **Issue:** + - A Windows checkout with `core.autocrlf=true` rewrites fixture bytes, so the lock would report drift that is not a content change. + - CI's `npm test` job is Linux-only, but the corpus is frozen for any machine. +- **Fix:** `.gitattributes` gets `scripts/bench/fixtures/** -text`. The files were already LF; `git status` showed no renormalization. +- **Committed in:** `f7b9e6ed` + +### Execution notes (not rule deviations) + +- **Wording differences from the plan:** + - paginate-find's Previous and Next are links styled as buttons; they navigate to `?page=N` as specified. + - Every core `finalUrl` check has `pathEquals`, because the 66-05 schema requires it. +- **Extra distractors:** + - a Done billing ticket by Dana, which makes "open" matter; + - America/Toronto, with the same offset as New York; + - the close second invoice INV-1038; + - the travel mug and the bowl in the catalog. +- **Corpus test additions beyond the plan's behaviors:** + - every app file listed and hashed; + - bodies paired with totals; + - answers re-derived from page data; + - near-miss bodies; + - served bytes and relative references; + - a wider static list: timers, clocks, randomness, background channels, embeds and CSS resources, `fetch(` only inside `bench.js`, relative `src`/`href`/`action`, and a `data:,` favicon on every page. +- **Branch checks:** the executor's `worktree-agent-*` allow-list was skipped, because the orchestrator directs commits on `Jev`. The cwd-drift sentinel and the protected-branch deny-list ran before every commit. + +--- + +**Total deviations:** 4 auto-fixed (2 Rule 1, 2 Rule 2). +**Impact on plan:** +- Each fix keeps the corpus measurable by both arms and keeps the frozen identity honest. +- The footprint is the plan's 35 files plus one `.gitattributes` line and the deferred-items log. +- No `extension/**`, `mcp/**`, `package.json`, STATE.md or ROADMAP.md change. + +## TDD Gate Compliance + +- Task 3 has a RED `test(66-07)` commit (`5d8204f1`, failing on the missing lock), followed by its GREEN commit (`f7b9e6ed`). +- The GREEN commit keeps the plan's own `test(66-07)` message, as earlier plans kept the plan messages on GREEN. So this task has no `feat` commit. +- The plan is `type: execute` with task-level TDD, so the plan-level gate does not apply. + +## Issues Encountered + +- **zsh parsing:** zsh treats a leading `=` as command expansion, so `echo ======` failed. BSD `cat` has no `-A`. Node 24's default test reporter prints `ℹ` summary lines, not TAP `#` lines. Each was re-run correctly. No process was left running. +- **Viewport measurement:** `scrollHeight` is floored at the viewport height, so the layout check used the bottom of `
` instead. +- **Known limitation of the live checks:** `urlIncludes` and `includes` are substring checks. `/jira/browse/HADOOP-1` and the text `HADOOP-1` would also match HADOOP-10..19. The live set is separate and never gates, so the planned checks were kept. +- **Text-click traps an agent may hit:** FSB's text click picks the last visible match in document order. + - `Large` also matches `X-Large`. + - With both panels open, the accordion's exports text also matches the audit copy. + + These are realistic, and are recorded for the 66-10 smoke. + +## Known Stubs + +None. Placeholder text such as "Choose a region" is the intended empty state of a custom combobox. + +## User Setup Required + +None. No external service configuration is required. + +## Next Phase Readiness + +- **66-08:** + - Drive tasks with `loadTasks` and `renderInstruction` against `startFixtureServer()`, which uses the default root `scripts/bench/fixtures`. + - Read `[data-bench-marker]` text after the window. The kanban page has markers from load, so "the target with the most markers" still resolves to the task tab. + - `data-fixture-ready="true"` is set at DOMContentLoaded. + - Record `corpusLock(tasks)` for run identity. It equals `core.lock.json` today. +- **66-09:** register `tests/bench-corpus.test.mjs`. It is Node 20-safe and takes about 0.2 s. +- **66-10 smoke:** + - Expect one POST per completed action. + - If a task fails while the page shows the right state, look at FSB's double click first; `bench.js` should already neutralize it. + - The live UAT checks need the bench Chrome in English. +- **Blockers:** none. + +## Self-Check: PASSED + +- All 35 planned files, `deferred-items.md` and this SUMMARY were found on disk. +- All 5 task commits were found: `d07021ad`, `75e69893`, `e830f2dd`, `5d8204f1`, `f7b9e6ed`. +- No commit in `333ed5bc..HEAD` has a Co-Authored-By line. +- `git status --porcelain -- extension/ mcp/ package.json .planning/STATE.md .planning/ROADMAP.md` was empty. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-08-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-08-PLAN.md new file mode 100644 index 000000000..5e5edb3ca --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-08-PLAN.md @@ -0,0 +1,438 @@ +--- +phase: 66-baseline-and-session-timing +plan: 08 +type: execute +wave: 3 +depends_on: ["66-02", "66-05"] +files_modified: + - scripts/bench/lib/mcp-client.mjs + - scripts/bench/lib/cdp.mjs + - scripts/bench/lib/claude-arm.mjs + - scripts/bench/lib/cursor-arm.mjs + - scripts/bench/lib/runs.mjs + - scripts/bench/run-bench.mjs + - scripts/bench/report.mjs + - tests/bench-report.test.mjs + - tests/bench-drivers.test.mjs + - tests/fixtures/bench/run-task-provider-error.json + - tests/fixtures/bench/infra-classification-cases.json +autonomous: true +requirements: [MEAS-01, MEAS-03] +user_setup: [] + +must_haves: + truths: + - "The headline metric is wall time per completed task, measured on the runner's own clock (t0 before run_task or before spawning claude; t1 at run_task resolution or at Claude's result event). Session endTime is never used (D-06)." + - "The benchmark runs only on request (`node scripts/bench/run-bench.mjs`), adds no dependencies, and takes the MCP SDK (1.29.0) and ws from mcp/node_modules. No mcp/ source changes (D-01, D-13)." + - "Arms: the autopilot arm is run_task from the runner's own SDK stdio client (after open_tab outside the timed window) against an API-kind xAI grok-4.3 configuration. The MCP arm is headless Claude Code with the pinned flags and an explicit --claude-model. The Cursor driver exists but is excluded from the default two-arm baseline (D-14, D-16, D-19, orchestrator decisions 3, 4, 7)." + - "Success comes only from fixture-server evidence and post-window CDP reads (final URL, DOM markers, live text). Agent claims are stored as 'claimed' and never decide pass (D-17)." + - "Each run's detail is written to disk the moment it ends: the raw record under gitignored scripts/bench/.local/, and one sanitized, allowlisted row appended to the campaign's runs.jsonl (D-20, D-21)." + - "Each campaign records identity (D-18): git SHA and dirty state; loaded extension id, version and measurement marker; MCP server and SDK version; tools/list count and hash; provider kind, requested and served model; host name, version, model and tool-loading mode (init.tools plus ENABLE_TOOL_SEARCH); task id, version and fixture hash." + - "The schedule is 12 tasks x 10 ABBA-alternating rounds x 2 arms (240 runs), seeded and resumable. The report gives completion with Wilson CIs, wall time per completed task, labelled cost per completed task, and the A/A noise floor. A campaign under 10 rounds is labelled a smoke test, and live tasks are reported separately (D-15, D-16)." + - "MCP host model time, tokens and cost come only from the driver's stream, with Claude's cost labelled 'list-price estimate, not billed' when appropriate and Cursor's labelled unavailable (D-12)." + - "Host CLIs never inherit the parent environment (B3). Claude and Cursor get an allowlisted env: HOME, PATH, USER, LOGNAME, SHELL, TMPDIR, LANG, LC_*, and HTTP_PROXY/HTTPS_PROXY/NO_PROXY in either case. CLAUDECODE, CLAUDE_CODE_*, CLAUDE_* and every DELEGATION_PROVIDER_KEY_NAMES entry (imported read-only from mcp/build under D-01 / orchestrator decision 10) are dropped. ENABLE_TOOL_SEARCH is set only when requested. identity.json records the sorted env key names and their sha256, never values." + - "Claude usage caps (`agent_cap`), Claude login or key failures (`agent_auth`) and autopilot provider failures (`provider_error`) are classified from stream and response signals as infra errors, never as task failures. The first one pauses the campaign with exit 75, and `--resume` re-runs every runId whose latest attempt ended in an infra error (W4, W-A)." + - "A sustained xAI rate limit or quota stop still pauses the campaign even though agent-loop.js (~3096-3101) retries HTTP 429 every 5 s with no cap and so never surfaces a status code (W-A). An autopilot run that timed out whose session detail IS present with `measurement.recorded !== true`, meaning no provider response was ever measured, is `provider_error`. The loop's terminal texts `API call failed:`, `API rejected request` and `API key invalid` are `provider_error` too. A timeout whose session detail cannot be found is an ordinary task failure (infraError null, flag `session_detail_missing`) and never pauses (B-TIMEOUT). On timeout the runner calls `get_task_status` before `stop_task` to learn the session id, so the detail is normally found. Numeric codes classify only with an `HTTP` or `status` prefix, so '500 ms' or the bare word 'provider' never pause. agent-loop.js behavior is not changed." + - "No benchmark run can pollute FSB's production telemetry or its public /stats dashboard (W-B, T-66-07). The preflight reads `fsbTelemetryOptOut` from the bench extension's service worker over CDP, refuses with `telemetry_not_opted_out` unless it is strictly true, and records the value in identity.json." + - "Argument validation rejects an `--expect-extension-id` that is not 32 letters a-p, or that equals the Web Store id, before any pairing or spawn (W9). Claude and Cursor run in their own process group and are killed as a group, so relay children cannot outlive a run (W10)." + - "The bridge pairing code is never printed. The runner reads it from `pair --json` in-process only, and a source check covers both cdp.mjs and run-bench.mjs (B2)." + - "Run rows carry `fsb.callIntervalMs {median, p90, count}` for like-for-like comparison with the 66-01 history deltas (W6). Identity records the bench Chrome's launch flags (W12)." + artifacts: + - path: "scripts/bench/run-bench.mjs" + provides: "CLI: --dry-run, --preflight, --smoke, --campaign, --resume; per-run immediate save" + - path: "scripts/bench/report.mjs" + provides: "loadCampaign, buildSummary, renderMarkdown, writeReport, CLI" + exports: ["loadCampaign", "buildSummary", "renderMarkdown", "writeReport"] + - path: "scripts/bench/lib/runs.mjs" + provides: "buildSchedule (seeded ABBA), toCommittedRow, validateCommittedRow, classifyInfra, sessionDetailFlags, evaluateBenchSafety, RUN_ROW_KEYS, INFRA_CODES, PAUSING_INFRA_CODES" + exports: ["buildSchedule", "toCommittedRow", "validateCommittedRow", "classifyInfra", "sessionDetailFlags", "evaluateBenchSafety", "RUN_ROW_KEYS", "INFRA_CODES", "PAUSING_INFRA_CODES"] + - path: "scripts/bench/lib/mcp-client.mjs" + provides: "openRunnerClient, collectServerIdentity, readFsbConfig, callToolJson, openTab, runTaskTimed, getSessionDetailWithRetry, listSessions" + - path: "scripts/bench/lib/cdp.mjs" + provides: "readDevToolsPort, browserVersion, listPageTargets, collectCoreEvidence, collectLiveEvidence, closeTargets, pairExtensionViaCdp, readExtensionStorageFlag" + - path: "scripts/bench/lib/claude-arm.mjs" + provides: "runClaudeTask, claudeVersion, buildBenchChildEnv, loadProviderKeyNames" + - path: "tests/bench-drivers.test.mjs" + provides: "Fake-based driver tests: env allowlist, CDP pairing and evidence via a stub ws CDP endpoint, kill-signal ladder and process-group kill via a fake claude, stop_task on timeout via a fake MCP client" + - path: "scripts/bench/lib/cursor-arm.mjs" + provides: "runCursorTask, cursorVersion (experimental; not in the baseline)" + key_links: + - from: "scripts/bench/run-bench.mjs" + to: "scripts/bench/lib/identity.mjs evaluatePreflight" + via: "refuses before any run when the hub, marker, extension id, provider, model, Claude model or tree fails" + pattern: "evaluatePreflight\\(" + - from: "scripts/bench/run-bench.mjs" + to: "scripts/bench/lib/outcome.mjs evaluateOutcome" + via: "evidence = fixture-server request log + CDP finalUrl/markers (+ live text); claims are never passed" + pattern: "evaluateOutcome\\(" + - from: "scripts/bench/run-bench.mjs" + to: "get_session_detail measurement block (plan 66-06)" + via: "runner SDK client fetches session detail after each run and copies measurement numbers into the row" + pattern: "get_session_detail" +--- + + +Build the benchmark runner, the arm drivers, and the report. The runner verifies run identity, schedules seeded ABBA runs, drives the autopilot and Claude Code MCP arms against local fixtures or live pages, collects independent evidence, saves every run immediately, and aggregates a baseline report with the statistics Phase 71 needs. + +Purpose: MEAS-03 (a repeatable, outcome-checked suite with saved completion, wall-time and cost baselines) and the MCP-side host metrics of MEAS-01 (D-12). +Output: `scripts/bench/run-bench.mjs`, `scripts/bench/report.mjs`, `scripts/bench/lib/{mcp-client,cdp,claude-arm,cursor-arm,runs}.mjs`, `tests/bench-report.test.mjs`, `tests/bench-drivers.test.mjs`, and the provider-error fixture. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md +@.planning/phases/66-baseline-and-session-timing/66-05-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +Libraries already implemented (plans 66-02 and 66-05): +- stats.mjs: `median, quantile, mulberry32, wilson, summarize, geoMeanMedianRatio, aaNoiseFloor`. +- outcome.mjs: `evaluateOutcome(task, evidence, {token})`, `normalizeFinalUrl`, `stripRunPrefix`. +- identity.mjs: `canonicalJson, sha256Hex, toolsListDigest, gitIdentity, evaluatePreflight({mode, arms, configPayload, hub:{listenerPid, serverPid}, claudeModel, expectExtensionId, git:{dirty}, allowDirty, toolsList:{count}}), servedModelMatches, WEB_STORE_EXTENSION_ID, PINNED_AUTOPILOT_MODEL`. +- tasks.mjs: `loadTasks({tasksRoot, fixturesRoot, set}), fixtureSha256, renderInstruction, BENCH_TASKS_ROOT, BENCH_FIXTURES_ROOT`. There is a sample set at tests/fixtures/bench/tasks-sample/{tasks,fixtures}. +- fixture-server.mjs: `startFixtureServer({root, port, secret})` resolves to `{port, baseUrl, secret, newRunToken(), runUrl(token, startPath), evidence(token), clear(token), close()}`. +- host-stream.mjs: `createNdjsonAccumulator, readFsbAgentPolicy, serializeFsbAgents, buildClaudeArgv({mcpConfigPath, agentsJson, model}), buildCursorArgv({model}), summarizeClaudeStream(events, {toolSearchEnv}), summarizeCursorStream(events), FORBIDDEN_HOST_FLAGS, COST_LABEL_LIST_PRICE, HOST_UNAVAILABLE_LABEL`. The Claude summary also carries `rateLimit:{events, limited, statuses}` and `errorKind:'usage_limit'|'rate_limit'|'api_error'|'other'|null` (66-05, W4). The fixture is tests/fixtures/bench/claude-stream-usage-cap.jsonl. +- mcp/build/agent-providers/spawn-environment.js (read-only import, D-01 / orchestrator decision 10): `DELEGATION_PROVIDER_KEY_NAMES`, the frozen list of provider credential variable names from mcp/src/agent-providers/spawn-environment.ts ~175-325. + +Session detail measurement (plan 66-06; reached through the runner's MCP client): +- `get_session_detail({sessionId})` returns `{success, session:{..., actionHistory, measurement}}`. +- MCP measurement: `{kind:'mcp', calls, callsWithTiming, fsbMs:{total, median}, callerGapMs:{total, median, label}, callIntervalMs:{median, p90, count}, endReason, idleTimeout, ...}`. +- Autopilot measurement: `{kind:'autopilot', recorded, iterations, modelMs:{total}, toolMs:{total}, inputTokens, outputTokens, costUsd, costSource, requestedModel, servedModels}`. +- `list_sessions({limit})` returns index entries with `id, startTime, mode, mcpClient, ...`. +- `run_task({task})` resolves `{success, sessionId, status?, partial_outcome?, ...}`. +- fsb://config: `{success, config:{providerKind, modelProvider, modelName}, extension:{id, version, measurement:{callTiming, sessionDetailJournal}}}`. + +SDK facts (mcp/node_modules/@modelcontextprotocol/sdk 1.29.0): `Client` from `client/index.js`; `StdioClientTransport` from `client/stdio.js` (options command, args, stderr:'pipe'; exposes `pid`); `client.getServerVersion()`, `listTools()` (paginate on `nextCursor`), `readResource({uri})`, `callTool(params, resultSchema, {onprogress, resetTimeoutOnProgress, timeout, maxTotalTimeout})`. The default request timeout is 60 s, so run_task must pass these options (Pitfall 7). Resolve with `createRequire(path.join(repoRoot,'mcp','package.json'))`, as tests/write-activation-evidence.test.js:7-10 and tests/mcp-bridge-topology.test.js:10-11 do. + +Signal and child patterns: scripts/run-phase65-full-tests.mjs lines 169-287 (bounded forwarding, signal forwarding, exit 130/143). Claude timeout: SIGINT first, because SIGTERM exits without a result event; then SIGTERM after a 10 s grace, then SIGKILL after 5 s. + +RUN ROW CONTRACT (verbatim; report.mjs and plan 66-11 rely on it). `RUN_ROW_KEYS` in this exact set: +- `schemaVersion` (1), `campaignId`, `runId`, `attempts` (integer >= 1; attempts made for this runId), `set`, `taskId`, `taskVersion`, `category`, `fixtureSha256`, `arm`, `round`, `order`, `startedAt` (ISO), `wallMs`, `wallFromInitMs`, `pass`, `timedOut`. +- `checks:[{kind, pass}]` +- `infraError`: null or one of `INFRA_CODES`: + - `agent_cap`: the host agent (Claude Code) hit a usage or rate cap, from `rateLimit.limited` or `errorKind` in {'usage_limit', 'rate_limit'}; + - `agent_auth`: the host agent cannot authenticate (summary `errorKind === 'auth'`: 401/403 with an `HTTP` or `status` prefix, or login/key wording); + - `provider_error`: the autopilot model provider failed. Either the error text (run_task error or the session's terminal error, held in memory only) matches /\b(?:HTTP|status)[ :]*(?:429|401|403|5\d\d)\b|API call failed:|API rejected request|API key invalid|rate.?limit|too many requests|quota|overloaded/i, or the run timed out AND its session detail is present with `measurement.recorded !== true` (no provider response ever measured; the uncapped 429 retry at agent-loop.js ~3096-3101). A timeout with no retrievable session detail is NOT an infra error: it stays an ordinary task failure with the flag `session_detail_missing` and never pauses (B-TIMEOUT); + - `fsb_agent_cap`: FSB's AGENT_CAP_REACHED; + - `host_api_error`: the host agent hit a non-auth API error (summary `errorKind === 'api_error'`); + - `claude_no_init`, `host_no_result`, `extension_not_connected`, `hub_lost`, `spawn_failed`, `mcp_session_not_found_no_calls`. + - `PAUSING_INFRA_CODES` = `agent_cap`, `agent_auth`, `provider_error`. The first of these pauses the campaign (exit 75). Any other code pauses only on two consecutive occurrences. +- `claimed:{status, partialOutcome, isError, subtype, completeTask}` +- `fsb:{kind, calls, callsWithTiming, fsbMsTotal, fsbMsMedian, callerGapMsTotal, callerGapMsMedian, callIntervalMs:{median, p90, count}, iterations, modelMsTotal, toolMsTotal, endReason, idleTimeout, sessionMatch}` +- `host:{name, model, toolLoading, durationMs, durationApiMs, numTurns, inputTokens, outputTokens, cacheReadTokens, costUsd, costLabel}` +- `cost:{usd, source, label}` +- `provider:{requestedModel, servedModels, servedModelMatch}` +- `flags:[]` +Every string is 200 chars at most and never contains `://`. No request bodies, page text, claimed summary text or model text appear anywhere in a row. + +IDENTITY ADDITIONS (verbatim): identity.json also records: +- `hosts.claude.envKeyNames` (the sorted allowlisted env key names given to the child, names only) and `hosts.claude.envKeyNamesSha256` (B3); +- `chrome.launchFlags`: the sorted `flags` array from the launch record (default `scripts/bench/.local/bench-chrome.json`, written by 66-10). That array holds flag names only, for example `--user-data-dir` and `--host-resolver-rules` (W12, I-3). The record's `flagValues` stays in the gitignored file and is not copied into identity. +- `extension.telemetryOptOut`: the strict boolean read from the bench extension's `chrome.storage.local.fsbTelemetryOptOut` (W-B). A campaign only runs when it is `true`. + + + + + + + Task 1: Arm drivers and clients: runner MCP client, CDP evidence client, Claude Code and Cursor drivers, with fake-based tests + scripts/bench/lib/mcp-client.mjs, scripts/bench/lib/cdp.mjs, scripts/bench/lib/claude-arm.mjs, scripts/bench/lib/cursor-arm.mjs, tests/bench-drivers.test.mjs + + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Code Examples / Runner", "Pattern 7", "Pattern 8", "Pitfall 7", "Pitfall 10" + - tests/screenshot-uat.test.js lines 50-140 (DevToolsActivePort, CDP client) and 225-240 (target discovery) + - tests/mcp-bridge-topology.test.js lines 1-20 (ws from mcp/node_modules) + - tests/write-activation-evidence.test.js lines 1-15 (createRequire resolution) + - scripts/run-phase65-full-tests.mjs lines 160-290 (child and signal handling) + - scripts/bench/lib/host-stream.mjs (argv builders and summarizers) + - mcp/src/tools/autopilot.ts lines 25-60 and 210-250, and mcp/src/tools/observability.ts lines 40-66 (read-only: tool names and argument shapes; stop_task returns no session id) + - mcp/src/queue.ts lines 38-70 and extension/ws/mcp-tool-dispatcher.js lines 2738-2756 (read-only: get_task_status is on the read-only bypass list, and handleGetStatusRoute returns the agent's `currentSessionId`; B-TIMEOUT) + - mcp/src/agent-providers/spawn-environment.ts lines 170-330 (read-only: DELEGATION_PROVIDER_KEY_NAMES; import the compiled list from mcp/build, never edit mcp/) + - extension/ws/mcp-bridge-client.js lines 395-412 and ~3049 (pairing record validation and the `mcpBridgeClient` global the CDP pairing evaluates; read-only) + + + - Env allowlist (B3): given a parent env holding HOME, PATH, USER, LOGNAME, SHELL, TMPDIR, LANG, LC_ALL, LC_CTYPE, HTTP_PROXY, https_proxy, NO_PROXY, CLAUDECODE, CLAUDE_CODE_ENTRYPOINT, CLAUDE_CONFIG_DIR, ANTHROPIC_API_KEY, XAI_API_KEY, OPENAI_API_KEY, NODE_OPTIONS and RANDOM_VAR, `buildBenchChildEnv(parent, {toolSearch:'unset', providerKeyNames})` returns only the allowlisted names. The test also sets every name in `DELEGATION_PROVIDER_KEY_NAMES` in the parent and asserts none survives. With `toolSearch:'true'` the output gains `ENABLE_TOOL_SEARCH='true'`; with 'unset' it has no ENABLE_TOOL_SEARCH even if the parent has one. The result carries `keyNames` (sorted) and `keyNamesSha256`, and never any value in its metadata. + - Kill ladder and process group (W7, W10): the test writes a fake claude script to a temp dir. It ignores SIGINT, logs every received signal name to a file, spawns a long-lived grandchild and records its PID. It is run as `runClaudeTask({command: process.execPath, commandArgsPrefix: [fakePath], budgetMs: 200, sigintGraceMs: 200, sigtermGraceMs: 200, ...})`. The fake receives SIGINT and then SIGTERM, in that order; the result has `timedOut: true`; and after resolution the grandchild PID is gone (`process.kill(pid, 0)` throws ESRCH) because the group was killed. + - Normal fake run: a fake that emits init, two FSB tool_use blocks and a result on stdout gives `initAt` and `t1` set, `summary.result.present` true, the raw stream file written, and the temp workdir removed. + - Timeout path (W7, B-TIMEOUT), using a fake MCP client whose `callTool` for `run_task` rejects with a request-timeout error: + - `runTaskTimed` records the call order ['run_task', 'get_task_status', 'stop_task']: `get_task_status` comes BEFORE `stop_task`, and `stop_task` is called exactly once; + - it returns `timedOut: true` and `sessionId` equal to the fake status's `currentSessionId`; + - when `get_task_status` rejects or returns isError, `stop_task` is still called exactly once; + - when the status has no `currentSessionId`, it falls back to `list_sessions` after `stop_task`. With `{listAttempts: 2, delayMs: 0}` it calls list_sessions at most twice and returns the id of the latest entry whose `mode` is not 'mcp-agent' and whose startTime is in [t0, t1], or null when none matches. + - CDP pairing and evidence (W7): a stub CDP endpoint in the test serves `/json/list`, `/json/version`, `/json/close/` and `PUT /json/new`, with a ws server from mcp/node_modules/ws answering Runtime.evaluate. + - `pairExtensionViaCdp(port, id, code)` evaluates one expression containing `fsbMcpBridgePairing` and `reloadPairingAndReconnect`, returns `{ok:true}`, and prints nothing: the test captures console output and asserts the code never appears. + - With no service-worker target, it opens the control panel through `PUT /json/new` and retries. + - `collectCoreEvidence(port, token)` returns the finalUrl and markers of the target whose URL contains `/r//`. + - `closeTargets` hits `/json/close/` for each id. + - `readExtensionStorageFlag(port, id, 'fsbTelemetryOptOut')` returns true when the stub's evaluate result is true, false when it is false or undefined, and null when no service-worker target exists after the wake-and-retry window. The evaluated expression names only that one key (W-B). The stub asserts that the Runtime.evaluate message for this read carries `awaitPromise: true` and `returnByValue: true` (I-1). + + +1. `scripts/bench/lib/mcp-client.mjs`: + - `openRunnerClient({repoRoot, stderrLogPath})`: spawns `process.execPath /mcp/build/index.js` through StdioClientTransport with `stderr:'pipe'`, draining stderr into stderrLogPath under .local. Returns `{client, transport, sdkVersion, close()}`, with the client name `fsb-bench`. + - `collectServerIdentity(client, transport, sdkVersion)`: returns `{serverVersion, sdkVersion, serverPid: transport.pid, toolsList: toolsListDigest()}`. + - `readFsbConfig(client)`: returns `{ok:true, payload}`, or `{ok:false, reason}` when the resource text is not JSON (for example the not-connected text). + - `callToolJson(client, name, args, opts)`: returns `{isError, json, text}`, parsing `content[0].text` when it is JSON. + - `openTab(client, url)`: calls `open_tab` with `{url, active:true}`, so the autopilot start tab is foregrounded and not subject to background-tab timer throttling. Identity records `startTabActive: true`. + - `runTaskTimed(client, instruction, budgetMs, {listAttempts = 5, delayMs = 1000} = {})`: calls `run_task` with `{onprogress: () => {}, resetTimeoutOnProgress: true, timeout: 60000, maxTotalTimeout: budgetMs}` and returns `{t0, t1, timedOut, json, errorText, sessionId}`. `errorText` is kept in memory only, for classification in Task 2. `sessionId` is `json.sessionId` on success. + - Timeout path (B-TIMEOUT): the client limit (300-360 s task budgets) fires before the server's 600 s safety net (mcp/src/tools/autopilot.ts ~130), so run_task returns nothing. `stop_task` returns only `{success, message}`. + - Therefore, on a timeout error, call `get_task_status` FIRST and keep its `currentSessionId`. It is read-only and bypasses the server's task queue (mcp/src/queue.ts lists it as read-only), so it answers while the abandoned run_task still holds the queue. A get_task_status failure is caught and never skips the next step. Then call `stop_task` exactly once and set timedOut. + - If the status gave no id, call `list_sessions` up to `listAttempts` times, `delayMs` apart, and take the latest entry whose `mode` is not 'mcp-agent' and whose startTime is in [t0, t1]. The retry covers the save that stop_task triggers. If nothing matches, the id is null. + - `getSessionDetailWithRetry(client, sessionId, {attempts:10, delayMs:1000, until})`: returns the first detail for which `until` passes. If `until` never passes, it returns the last successfully fetched detail (so a present detail with `recorded: false` reaches the classifier). It returns null only when sessionId is null or every fetch failed. Also `listSessions(client, limit)`. +2. `scripts/bench/lib/cdp.mjs` (ws via createRequire from mcp/package.json; never the global WebSocket, which is absent on Node 20): + - `readDevToolsPort(profileDir, timeoutMs)`, `browserVersion(port)` (GET /json/version), `listPageTargets(port)` (GET /json/list, type 'page'). + - `evaluate(target, expression)`: Runtime.evaluate with `returnByValue: true` and `awaitPromise: true` (I-1), closing the socket each call. The pairing write, the storage-flag read and 66-10's host-block fetch check all evaluate expressions that return a Promise; without awaitPromise the caller would get the unresolved Promise object instead of its value. When the response has `exceptionDetails` (a thrown error or a rejected promise), evaluate rejects with an Error whose message is the fixed string `cdp_evaluate_exception`. It never includes the exception text or the expression, because the pairing expression carries the code. pairExtensionViaCdp maps that rejection to `{ok:false, reason:'cdp_evaluate_exception'}`. + - `collectCoreEvidence(port, token)`: over targets whose URL contains `/r//`, reads `location.href` and a name-to-text map of `[data-bench-marker]` elements. It prefers the target with the most markers, then the last in the list, and returns `{finalUrl, markers, targetIds}`. + - `collectLiveEvidence(port, task)`: matches targets whose URL host equals the task's startUrl host, reads `location.href` and `document.body.innerText` (first 200000 chars, kept in memory only), and returns `{finalUrl, text, targetIds}`. + - `closeTargets(port, ids)`: GET `/json/close/`, best effort. + - `readExtensionStorageFlag(port, extensionId, key)` (W-B): finds the extension's `service_worker` target the same way pairExtensionViaCdp does (waking it through the control panel if asleep). It evaluates `chrome.storage.local.get([]).then(d => d[] === true)` for exactly one key and returns `true` only for a strict `true`. It returns `false` when the value is false or absent, and `null` when the worker cannot be reached or the evaluation rejects (preflight detail 'unverifiable'). No other storage key is read. + - `pairExtensionViaCdp(port, extensionId, pairingCode)`: bench-profile pairing delivery. + - Find the `service_worker` target whose URL starts with `chrome-extension:///`. If the worker is asleep, open `chrome-extension:///ui/control_panel.html` through CDP `PUT /json/new?` to wake it, then retry for up to 10 s. + - Evaluate `chrome.storage.session.set({ fsbMcpBridgePairing: { pairingCode: , storedAt: Date.now() } })` in the worker, then `mcpBridgeClient.reloadPairingAndReconnect()`. `mcpBridgeClient` is a top-level `const` in the service worker (mcp-bridge-client.js ~3049), so evaluate it by name. The record has exactly the two keys the bridge client validates (`_isValidPairingRecord`, ~402-411). + - Return `{ok, reason}`. The pairing code is a function argument only: never logged, never written to disk, never included in errors. + - Why this path exists: the installed native host rejects any caller whose origin differs from its owner marker (`validateNativeInvocation` in mcp/src/native-host/protocol.ts; the marker origin is the Web Store id). So a bench-profile manifest copy cannot deliver the code to the unpacked build. This harness path uses the extension's own storage contract without touching mcp/ (D-01) or the global native host. +3. `scripts/bench/lib/claude-arm.mjs`: + - `claudeVersion()`: spawnSync `claude --version`, shell false, with the allowlisted env. + - `loadProviderKeyNames(repoRoot)`: dynamic `import()` of `/mcp/build/agent-providers/spawn-environment.js` and return `DELEGATION_PROVIDER_KEY_NAMES`. This is a read-only import from mcp/build (D-01, orchestrator decision 10). If the build is missing, throw an error telling the operator to run `npm --prefix mcp run build`. + - `buildBenchChildEnv(parentEnv, {toolSearch, providerKeyNames})` (B3): + - Copy ONLY these names: HOME, PATH, USER, LOGNAME, SHELL, TMPDIR, LANG, any name starting `LC_`, and HTTP_PROXY, HTTPS_PROXY, NO_PROXY, http_proxy, https_proxy, no_proxy. + - Then assert that no copied name is CLAUDECODE, starts with `CLAUDE_`, or is in providerKeyNames. It is defense in depth; the allowlist already excludes them. + - Set `ENABLE_TOOL_SEARCH` only when toolSearch is one of 'auto', 'true', 'false'. + - Return `{env, keyNames: , keyNamesSha256: sha256Hex(keyNames.join('\n'))}`. + - Never log or return values outside `env`. + - `runClaudeTask({repoRoot, instruction, model, budgetMs, rawStreamPath, toolSearch, command = 'claude', commandArgsPrefix = [], sigintGraceMs = 10000, sigtermGraceMs = 5000})`. The injectable command, prefix and grace times exist for the fake-based tests; the runner uses the defaults. + - Make a temp workdir with `fs.mkdtempSync(os.tmpdir())`. It stays empty apart from `mcp.json`, so no project skills or settings load. Write `mcp.json` as `{"mcpServers":{"fsb":{"type":"stdio","command":,"args":[/mcp/build/index.js]}}}`. + - Build argv with buildClaudeArgv and serializeFsbAgents(readFsbAgentPolicy(repoRoot)). Assert that no FORBIDDEN_HOST_FLAGS entry is present. + - Build the child env with `buildBenchChildEnv(process.env, {toolSearch, providerKeyNames: await loadProviderKeyNames(repoRoot)})`. Never copy `process.env`. + - Record t0 immediately before `spawn(command, [...commandArgsPrefix, ...argv], {cwd, env, stdio:['pipe','pipe','pipe'], shell:false, detached:true})`. `detached: true` puts the CLI and its MCP relay children in their own process group (W10). Write the instruction to stdin and end it (the task goes on stdin, as in the product profile). + - Tee stdout to rawStreamPath and parse it with createNdjsonAccumulator. Record initAt at the init event and t1 at the result event. + - On budget expiry, signal the whole group: `process.kill(-child.pid, 'SIGINT')`, then SIGTERM to the group after sigintGraceMs, then SIGKILL to the group after sigtermGraceMs. After normal exit, also send SIGTERM to the group (ignoring ESRCH), so no relay child survives the run (W10). + - Resolve on close with `{t0, initAt, t1, exitCode, signal, timedOut, summary, envKeyNames, envKeyNamesSha256}`, where the summary comes from summarizeClaudeStream. Keep the stderr tail only in the raw file. Remove the workdir in finally. +4. `scripts/bench/lib/cursor-arm.mjs`: `cursorVersion()` and `runCursorTask({repoRoot, instruction, model, budgetMs, rawStreamPath, command = 'cursor-agent', commandArgsPrefix = [], sigintGraceMs, sigtermGraceMs})`. Use the same lifecycle as the Claude driver: the same `buildBenchChildEnv` allowlist imported from claude-arm.mjs (B3), `detached: true` with group kill (W10), a temp workdir containing `.cursor/mcp.json` that points at the same server, argv from buildCursorArgv plus the instruction as the final positional argument, and the summary from summarizeCursorStream. The header comment marks it experimental and excluded from the Phase 66 baseline (orchestrator decision 7). It passes no permission-bypass flag; if Cursor needs interactive MCP approval, the run ends as an infra error. + +All four modules are ESM with no dependencies beyond Node built-ins and mcp/node_modules (SDK, ws). Resolve the SDK and ws lazily, inside the functions that use them rather than at module top level. That way `run-bench.mjs --dry-run` and the unit tests import these modules without needing mcp/node_modules. The modules never log secrets or environment values. + +5. `tests/bench-drivers.test.mjs` (node:test, Node 20-safe, no network, no real claude/cursor/Chrome) covers every behavior above. + - Resolve ws via `createRequire(/mcp/package.json)`. + - Write the fake CLI scripts into `fs.mkdtempSync(os.tmpdir())` at test time; no fixture file is needed. + - Clean up every child, temp dir and server in `finally`. + - Do not register it in package.json; plan 66-09 registers all bench suites in one commit. + +Commit: `feat(66-08): add benchmark MCP, CDP, Claude Code and Cursor drivers`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-drivers.test.mjs && node -e "Promise.all(['mcp-client','cdp','claude-arm','cursor-arm'].map(n=>import('./scripts/bench/lib/'+n+'.mjs'))).then(([m,c,a,u])=>{const need=[[m,['openRunnerClient','collectServerIdentity','readFsbConfig','callToolJson','openTab','runTaskTimed','getSessionDetailWithRetry','listSessions']],[c,['readDevToolsPort','browserVersion','listPageTargets','collectCoreEvidence','collectLiveEvidence','closeTargets','pairExtensionViaCdp','readExtensionStorageFlag']],[a,['runClaudeTask','claudeVersion','buildBenchChildEnv','loadProviderKeyNames']],[u,['runCursorTask','cursorVersion']]];for(const [mod,ks] of need)for(const k of ks)if(typeof mod[k]!=='function'){console.error('missing '+k);process.exit(1)}})" + + + - The verify command exits 0 (all driver exports present and importable on Node with mcp/node_modules installed). + - `grep -c "createRequire" scripts/bench/lib/cdp.mjs` prints at least 1, and `grep -cE "globalThis\.WebSocket|typeof WebSocket" scripts/bench/lib/cdp.mjs` prints 0 (ws comes from mcp/node_modules; no reliance on the Node 22+ global). + - No driver references a permission-bypass flag literally: `grep -rlE "dangerously-skip-permissions|--yolo" scripts/bench/lib/claude-arm.mjs scripts/bench/lib/cursor-arm.mjs` prints nothing. + - `grep -c "resetTimeoutOnProgress: true" scripts/bench/lib/mcp-client.mjs` prints at least 1. + - The pairing code is never printed: `node -e "for (const f of ['scripts/bench/lib/cdp.mjs']) { const s=require('fs').readFileSync(f,'utf8'); if(/console\.(log|error|warn)\([^)]*pairingCode/.test(s)) process.exit(1) }"` exits 0. Task 2 extends this check to run-bench.mjs. + - `node --test tests/bench-drivers.test.mjs` exits 0, including the env allowlist, kill ladder with group kill, the timeout path (get_task_status before stop_task, sessionId capture, list_sessions fallback; B-TIMEOUT), CDP pairing and evidence cases, and the `awaitPromise: true` assertion on the flag read (I-1). + - `grep -c "get_task_status" scripts/bench/lib/mcp-client.mjs` prints at least 1, and `grep -c "awaitPromise: true" scripts/bench/lib/cdp.mjs` prints at least 1. + - Neither driver copies the parent environment: `node -e "for (const f of ['scripts/bench/lib/claude-arm.mjs','scripts/bench/lib/cursor-arm.mjs']) { const s=require('fs').readFileSync(f,'utf8'); if(/\{\s*\.\.\.process\.env/.test(s)||/env:\s*process\.env/.test(s)) process.exit(1) }"` exits 0, and `grep -c "detached: true" scripts/bench/lib/claude-arm.mjs` prints at least 1. + + The drivers and clients exist, import cleanly, and follow the pinned host profile and Node 20 constraints. + + + + Task 2: Seeded ABBA schedule, sanitized run rows, and the run-bench CLI with preflight, smoke, campaign and resume + scripts/bench/lib/runs.mjs, scripts/bench/run-bench.mjs, tests/bench-report.test.mjs, tests/fixtures/bench/run-task-provider-error.json, tests/fixtures/bench/infra-classification-cases.json + + - scripts/bench/lib/identity.mjs (evaluatePreflight refusal codes), scripts/bench/lib/outcome.mjs, scripts/bench/lib/tasks.mjs, scripts/bench/fixture-server.mjs + - scripts/bench/lib/mcp-client.mjs, cdp.mjs, claude-arm.mjs, cursor-arm.mjs (Task 1) + - tests/fixtures/bench/claude-stream-usage-cap.jsonl and scripts/bench/lib/host-stream.mjs `rateLimit`/`errorKind` (66-05, for the agent_cap and agent_auth classifiers) + - extension/ai/agent-loop.js lines ~3070-3125 (read-only: the 401/403 terminal text "API key invalid or expired", the 400 text "API rejected request (400): …", the uncapped 429 retry every 5 s, and the terminal "API call failed: …"; do NOT change this behavior) + - extension/utils/install-identity.js ~129-145 and extension/utils/telemetry-collector.js ~630-660 (read-only: `fsbTelemetryOptOut` is true only when the user opted out, and every flush re-reads it) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 9" (ABBA), "Runtime and Operational State Touched by the Benchmark", "Pitfalls 8, 10-12" + - scripts/report-t1-readiness.mjs lines 1273-1280 (direct-run guard) + + + - buildSchedule({taskIds: 12 ids, rounds: 10, arms:['autopilot','claude'], seed: 66}) returns 240 entries. Each (round, task, arm) appears exactly once. Within odd rounds each task runs autopilot then claude; within even rounds it runs claude then autopilot. The task order within a round is a seeded permutation, identical across calls with the same seed and different for seed 67. runId is `-r--`, with campaignId passed in. + - toCommittedRow(raw) returns an object whose keys equal RUN_ROW_KEYS exactly. It never copies raw evidence bodies, page text, claimed summary or result text, or URLs. + - validateCommittedRow rejects an extra key, a missing key, any string containing `://`, any string over 200 chars, and an infraError outside the vocabulary. + - The CLI's `--dry-run` with the sample task set prints a JSON schedule summary (runs: 4 for 1 task x 2 rounds x 2 arms) and exits 0 without spawning a server or browser. + - Argument validation collects every failing code and prints them all at once. `--smoke` with the claude arm and no `--claude-model` exits 2 with `claude_model_required` in its output before any process is spawned; `extension_id_invalid` may be listed alongside when the id is also missing. An unknown flag exits 2 with usage text. + - Extension id validation (W9): `--pair-via-cdp`, `--smoke` or `--campaign` with `--expect-extension-id` missing, not matching /^[a-p]{32}$/, or equal to `badgafnfchcihdfnjneklogedcdkmjfk` exits 2 with `extension_id_invalid` before any pairing, server or process spawn. A valid 32-letter id passes this step. The test checks the exit code and that no child process was started, by passing `--dry-run` alongside and asserting the validation still fires first. + - classifyInfra (W4, W-A). Each case in tests/fixtures/bench/infra-classification-cases.json is its own `test()`: + - A Claude summary parsed from tests/fixtures/bench/claude-stream-usage-cap.jsonl (via host-stream) gives `agent_cap`. + - A Claude summary with `errorKind: 'auth'` gives `agent_auth`, and one with `errorKind: 'api_error'` gives `host_api_error`. + - An autopilot run_task response from tests/fixtures/bench/run-task-provider-error.json gives `provider_error`. + - Autopilot error texts `API call failed: Failed to fetch`, `API rejected request (400): bad tool schema`, `API key invalid or expired. Please check your API key in settings.` and `HTTP 503 Service Unavailable` each give `provider_error`. + - An autopilot run with `timedOut: true` and a PRESENT session detail whose `measurement.recorded` is false gives `provider_error`. The same timeout with `measurement.recorded: true` gives null: an ordinary timeout counts as a task failure. + - An autopilot run with `timedOut: true` and a MISSING session detail (null) gives null, never `provider_error`. `sessionDetailFlags({arm:'autopilot', sessionDetail:null})` returns `['session_detail_missing']`, and the runner merges it into that row's flags. The run stays in the completion denominator as a task failure without pausing (B-TIMEOUT). The case table holds this as `autopilot-timeout-missing-detail`. A present detail gives `sessionDetailFlags(...)` of `[]`. + - Negative cases give null and never pause: `Element not found after 500 ms`, `The provider page did not load`, and a status-less `500`. + - A run_task error containing `AGENT_CAP_REACHED` gives `fsb_agent_cap`. + - A Claude summary with no init gives `claude_no_init`. + - A normal failed task (checks failed, no infra signal) gives null, so it counts as a task failure. + - Telemetry refusal (W-B): `evaluateBenchSafety({telemetryOptOut})` returns `{ok:true, refusals:[]}` only for `true`. `false`, `undefined`, `null` (unverifiable) and the string `'true'` each return `ok:false` with refusal code `telemetry_not_opted_out`. The runner's preflight path is tested with an injected flag reader: a stubbed reader returning false makes the preflight step exit 2 with `telemetry_not_opted_out` before any run is scheduled, and identity.json is not written. + - Pause and resume (W4), with a simulated schedule in the test driving the runner's loop helper against injected fake arms: + - the first `agent_cap`, `agent_auth` or `provider_error` stops the loop with exit code 75 and a printed `--resume ` hint; + - two consecutive other infra errors also stop it with 75; + - `--resume` re-runs every runId whose latest raw record has a non-null infraError and skips runIds whose latest record has none; + - after resume, runs.jsonl has exactly one row per runId, with `attempts` counting every attempt. + - Rows carry `fsb.callIntervalMs` copied from the MCP measurement (null fields for autopilot), and `attempts >= 1`. + + +1. `scripts/bench/lib/runs.mjs` (pure) implements the contract exactly: + - `RUN_ROW_KEYS` (the RUN ROW CONTRACT top-level keys, frozen), `INFRA_CODES` and `PAUSING_INFRA_CODES` (frozen); + - `evaluateBenchSafety({telemetryOptOut})`: returns `{ok, refusals:[{code:'telemetry_not_opted_out', detail}]}` unless `telemetryOptOut === true` (W-B). The detail says 'false', 'absent' or 'unverifiable', and nothing else; + - `buildSchedule({campaignId, taskIds, rounds, arms, seed})`: a seeded shuffle per round via mulberry32(seed + round), with ABBA arm order; + - `toCommittedRow(raw)` and `validateCommittedRow(row)`; + - `classifyInfra({arm, hostSummary, taskJson, errorText, sessionErrorText, timedOut, sessionDetail})`. It returns an INFRA_CODES value or null. Error texts (run_task's error or message, and the session's terminal `error`) are read only in memory and never stored. Rules, in the contract's terms (W-A): + - host arms: `rateLimit.limited` or `errorKind` in {'usage_limit', 'rate_limit'} gives `agent_cap`; `errorKind === 'auth'` gives `agent_auth`; `errorKind === 'api_error'` gives `host_api_error`; no init gives `claude_no_init`; no result gives `host_no_result`. + - autopilot: either text matching the provider pattern in the contract gives `provider_error`, as does `timedOut === true` with a present `sessionDetail` (non-null, with a `session` object) whose `session.measurement?.recorded !== true`. A timeout whose `sessionDetail` is null or has no `session` object returns null; the runner flags `session_detail_missing` through `sessionDetailFlags` (B-TIMEOUT). `AGENT_CAP_REACHED` gives `fsb_agent_cap`. + - Never match a bare number or the bare word "provider". Every numeric alternative carries the `(?:HTTP|status)[ :]*` prefix. + - Do not change extension/ai/agent-loop.js: its uncapped 429 retry is handled here, by classification. + - `sessionDetailFlags({arm, sessionDetail})` (B-TIMEOUT): returns `['session_detail_missing']` for the autopilot arm when `sessionDetail?.session` is not an object, else `[]`. run-bench merges the result into the row's flags. + - `tests/fixtures/bench/run-task-provider-error.json`: a synthetic run_task response `{success:false, error:'Provider request failed: HTTP 429 rate limit exceeded (PROVIDER_ERROR_TEXT_77c1)'}` with a `_fixture` note. The test asserts the sentinel never appears in any row. + - `tests/fixtures/bench/infra-classification-cases.json`: an array of `{name, input:{arm, hostSummary?, errorText?, sessionErrorText?, timedOut?, sessionDetail?}, expected}` covering every classifyInfra case in , including the three negatives and `autopilot-timeout-missing-detail` (expected null; B-TIMEOUT). It has a `_fixture` note that all texts are synthetic. + +2. `scripts/bench/run-bench.mjs`: a CLI with a direct-run guard and exit codes 0 ok, 1 error, 2 refusal/usage, 75 paused (infra), 130 SIGINT, 143 SIGTERM. + - Flags: + - modes: `--dry-run`, `--preflight`, `--smoke`, `--campaign` + - task selection: `--set core|live` (default core), `--tasks `, `--rounds N` (smoke 2, campaign 10), `--arms ` (default `autopilot,claude`; `cursor` only when listed, and it is flagged experimental_arm) + - schedule: `--seed N` (default 66) + - hosts: `--claude-model ` (required for smoke/campaign with claude; no built-in default, per orchestrator decision 3), `--claude-tool-search unset|auto|true|false` (default unset), `--cursor-model ` + - browser and identity: `--chrome-profile ` (default `~/.fsb-bench/chrome-profile`), `--expect-extension-id ` (validated per W9 below), `--chrome-launch-record ` (default `scripts/bench/.local/bench-chrome.json`; its flag names go into identity `chrome.launchFlags`, W12) + - campaign bookkeeping: `--campaign-id ` (default `--`), `--resume `, `--allow-dirty` + - paths and pacing: `--tasks-root `, `--fixtures-root `, `--cooldown-ms N` (default 3000) + - pairing: `--pair-via-cdp` (requires `--expect-extension-id` and `--chrome-profile`). After the runner's own server owns 7225 and before the preflight: + - read the current pairing code with `node /mcp/build/index.js pair --json` (execFile, shell false; parse `pairingCode` from stdout; never print it). This command does not rotate the secret; only `--reset` or a serve daemon does. + - call `pairExtensionViaCdp`, then poll `readFsbConfig` for up to 30 s until the extension is connected. + - drop the code from memory after use. + - Validate arguments before spawning or pairing anything. For `--pair-via-cdp`, `--smoke` and `--campaign`, `--expect-extension-id` is required, must match /^[a-p]{32}$/, and must differ from `badgafnfchcihdfnjneklogedcdkmjfk`; otherwise exit 2 with `extension_id_invalid` (W9). This check runs before `--dry-run` handling, so it is testable without spawning. + - `--smoke` defaults to EVERY task in the selected set with `--rounds 1` when `--tasks`/`--rounds` are absent. It then doubles as the per-task reference pass that plan 66-10 feeds into the H4 packet (W3): 12 core tasks x 1 round x 2 arms = 24 runs. + - `--dry-run`: load tasks, compute fixture hashes, build the schedule, print `{campaignId, runs, arms, rounds, tasks:[{id, version, fixtureSha256}]}` and exit 0. + - Otherwise: + - open the runner client (the runner's own server must become the hub on 7225); + - read the 7225 listener PID with `lsof -nP -iTCP:7225 -sTCP:LISTEN -t` (a missing lsof counts as not owned); + - collect server identity, fsb://config and gitIdentity; + - run `evaluatePreflight`. Then read the bench extension's telemetry opt-out with `readExtensionStorageFlag(cdpPort, expectExtensionId, 'fsbTelemetryOptOut')` and run `evaluateBenchSafety` (W-B). The flag reader is injectable for tests. If either check refuses, print every refusal code (for example `telemetry_not_opted_out`) and exit 2 with no runs and no identity.json. Record `extension.telemetryOptOut` in the preflight summary and in identity.json. + - `--preflight` prints a secret-free identity summary and exits 0. + - Smoke and campaign: + - Start the fixture server (core set), read the CDP port from the chrome profile, and record browserVersion and claudeVersion (plus cursorVersion if used). + - Write identity.json, including the IDENTITY ADDITIONS (Claude env key names and sha256 from the driver, Chrome launch flag names from the launch record). Campaign output goes to `scripts/bench/results//`; smoke output goes to `scripts/bench/.local//`. + - Build the schedule. With `--resume`, skip only runIds whose latest raw record has `infraError === null`; re-run runIds whose latest attempt ended in an infra error (W4). Archive a superseded attempt as `runs/.attempt.json` before writing the new one. + - Per run: + - core: newRunToken, runUrl, renderInstruction. live: task.startUrl, no token. + - autopilot: openTab outside the window, then runTaskTimed, then getSessionDetailWithRetry on runTaskTimed's `sessionId` until `session.measurement.recorded === true`. On timeout that id comes from `get_task_status` (called before `stop_task`) or the list_sessions fallback. When no id or no detail is found, pass `sessionDetail: null` to classifyInfra and add `sessionDetailFlags(...)` (here `session_detail_missing`) to the row's flags (B-TIMEOUT). + - claude: runClaudeTask, then listSessions, selecting mcp-agent sessions with startTime in [t0, t1], and fetch each detail. Set sessionMatch to unique, multiple or none, and flag `multiple_mcp_sessions` when needed. + - evidence: core runs combine server.evidence(token).requests with collectCoreEvidence; live runs use collectLiveEvidence. + - `evaluateOutcome(task, evidence, {token})`. + - classify infraError with `classifyInfra`, and set timedOut; set servedModelMatch via servedModelMatches over measurement.servedModels (flag `served_model_mismatch`); flag `idle_timeout_session` when measurement.idleTimeout; copy `measurement.callIntervalMs` into `fsb.callIntervalMs` (W6). + - write the raw record to `scripts/bench/.local//runs/.json`; + - regenerate `//runs.jsonl` from the latest raw record of every runId: `toCommittedRow` plus `validateCommittedRow`, written to a temp file and renamed atomically. This happens right after each run, so each run is saved immediately (D-20) and runs.jsonl never holds duplicate runIds; + - update `.local//progress.json`; + - closeTargets, server.clear(token), then the cooldown. + - Pause rules (W4, W-A): the first code in `PAUSING_INFRA_CODES` (`agent_cap`, `agent_auth`, `provider_error`) exits 75 immediately. Any other two consecutive infra errors also exit 75. Every pause prints the reason code and the exact `--resume ` command, never error text. + - SIGINT or SIGTERM finish the current run's cleanup and exit 130 or 143. + - Never print or persist the fixture-server secret, API keys, bridge credentials or environment values. Identity records only the fields listed in must_haves. +3. Add `node:test` cases for runs.mjs and the CLI behaviors to `tests/bench-report.test.mjs`. Task 3 adds report cases to the same file. Spawn the CLI with `process.execPath` against the sample task set, and use no network. + +Commit: `feat(66-08): add seeded ABBA benchmark runner with preflight and per-run saves`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-report.test.mjs && node scripts/bench/run-bench.mjs --dry-run --tasks-root tests/fixtures/bench/tasks-sample/tasks --fixtures-root tests/fixtures/bench/tasks-sample/fixtures --rounds 2 --arms autopilot,claude --claude-model claude-model-fixture + + + - `node --test tests/bench-report.test.mjs` exits 0. + - The dry-run command exits 0 and prints JSON with `"runs":4`. + - `node scripts/bench/run-bench.mjs --smoke --arms claude --tasks-root tests/fixtures/bench/tasks-sample/tasks --fixtures-root tests/fixtures/bench/tasks-sample/fixtures; test $? -eq 2` succeeds, and its output contains `claude_model_required`. + - `grep -c "evaluatePreflight(" scripts/bench/run-bench.mjs` and `grep -c "evaluateOutcome(" scripts/bench/run-bench.mjs` each print at least 1. + - `git diff --quiet HEAD -- scripts/bench/lib/outcome.mjs scripts/bench/lib/identity.mjs` exits 0 (this plan does not modify the claim-blind evaluator or the preflight). + - The pairing code is never printed by either file (B2): `node -e "for (const f of ['scripts/bench/lib/cdp.mjs','scripts/bench/run-bench.mjs']) { const s=require('fs').readFileSync(f,'utf8'); if(/console\.(log|error|warn|info)\([^)]*(pairingCode|fsb-auth\.)/.test(s)||/process\.(stdout|stderr)\.write\([^)]*pairingCode/.test(s)) process.exit(1) }"` exits 0. + - `node scripts/bench/run-bench.mjs --dry-run --smoke --expect-extension-id badgafnfchcihdfnjneklogedcdkmjfk --claude-model m --tasks-root tests/fixtures/bench/tasks-sample/tasks --fixtures-root tests/fixtures/bench/tasks-sample/fixtures; test $? -eq 2` succeeds, and the output contains `extension_id_invalid` (W9). + - Classification and safety are mechanically checked (W-A, W-B): `node -e "import('./scripts/bench/lib/runs.mjs').then(m=>{const c=m.classifyInfra;if(c({arm:'autopilot',errorText:'Element not found after 500 ms',timedOut:false})!==null)process.exit(1);if(c({arm:'autopilot',errorText:'The provider page did not load',timedOut:false})!==null)process.exit(2);if(c({arm:'autopilot',errorText:'API call failed: Failed to fetch'})!=='provider_error')process.exit(3);if(c({arm:'autopilot',timedOut:true,sessionDetail:{session:{measurement:{recorded:false}}}})!=='provider_error')process.exit(4);if(c({arm:'autopilot',timedOut:true,sessionDetail:null})!==null)process.exit(9);if(m.sessionDetailFlags({arm:'autopilot',sessionDetail:null})[0]!=='session_detail_missing'||m.sessionDetailFlags({arm:'autopilot',sessionDetail:{session:{}}}).length!==0)process.exit(10);if(c({arm:'claude',hostSummary:{init:{},result:{present:true},rateLimit:{limited:false},errorKind:'auth'}})!=='agent_auth')process.exit(5);if(!m.PAUSING_INFRA_CODES.includes('agent_auth'))process.exit(6);const r=m.evaluateBenchSafety({telemetryOptOut:false});if(r.ok||r.refusals[0].code!=='telemetry_not_opted_out')process.exit(7);if(!m.evaluateBenchSafety({telemetryOptOut:true}).ok)process.exit(8)})"` exits 0. + - agent-loop.js is untouched by this plan: `git diff --quiet HEAD -- extension/ai/agent-loop.js` exits 0 (W-A is handled by classification only). + + The runner schedules, verifies, drives, evaluates and saves each run immediately, with sanitized committed rows and resumable campaigns. + + + + Task 3: Baseline report with completion CIs, wall time per completed task, labelled cost, host metrics and the A/A noise floor + scripts/bench/report.mjs, tests/bench-report.test.mjs + + - scripts/bench/lib/stats.mjs and scripts/bench/lib/runs.mjs + - scripts/report-t1-readiness.mjs lines 14-24 and 1241-1280 (writeReport and direct-run guard) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Pattern 9" (A/A, dropped tasks, Wilson) + + + - Given synthetic identity plus 120 rows (3 tasks x 10 rounds x 2 arms; some failed, 2 infra errors), buildSummary reports per set and per arm: runs, infraErrors (excluded from denominators and listed), completed, completionRate, and completionCi95 from wilson. + - Also per arm: perTask medians and p90 of wallMs over completed runs; wallPerCompletedTask.geoMeanOfTaskMediansMs; costPerCompletedTask with a sources count and label; and aaNoiseFloor from stats. + - Rounds under 10 set `kind` to 'smoke (not a baseline)'; 10 or more set 'baseline'. Live rows appear only under `live` and never alter `core` aggregates. + - One-round report (I-C): with 12 tasks x 1 round x 2 arms of synthetic rows (the 66-10 smoke shape), summary.json reports `aaNoiseFloor: {status:'n/a', reason:'needs at least 2 rounds'}` per arm instead of a bootstrap, and report.md prints `n/a` in the A/A section. Neither file contains `NaN`, `null%` or `Infinity`, and the per-task table still renders. + - fsb per-call medians come from row fsb fields, with caller gap labelled 'upper bound on host thinking time'. The summary also reports `fsb.callIntervalMsMedian` over MCP-arm rows, labelled as the live counterpart of the 66-01 history deltas (W6). Claude host metrics report a durationApiMs/durationMs median share and a numTurns median, with the cost label passed through. + - Infra errors are broken down by code (for example `agent_cap: 2, provider_error: 1`) and total attempts are reported, so the report shows capped or failed-provider attempts separately from task failures (W4). + - renderMarkdown contains the headings `## Identity`, `## Core set`, `## A/A noise floor`, `## Costs`, `## Host metrics`, `## Live set (reported separately)` and `## Method`. The Method section states that wall time is the runner's clock, claims are not counted, caller gap is an upper bound, queue wait is not measured, Claude cost is a list-price estimate not billed, and arms are not compared with each other. The markdown contains no `://`. + - The report CLI on a temp campaign dir writes summary.json and report.md, and exits 1 when any row fails validateCommittedRow. + + +Create `scripts/bench/report.mjs` (ESM, direct-run guard): `loadCampaign(dir)` (reads identity.json and runs.jsonl, validating every row), `buildSummary({identity, runs}, {iterations = 10000, seed = 66})`, `renderMarkdown(summary)` and `writeReport(summary, dir)`. The CLI is `node scripts/bench/report.mjs `. Implement every behavior above using stats.mjs only (no new statistics code). Never compute a cross-arm speed ratio; state that Phase 71 compares each arm with its own fast-mode run. + +Add the report cases to `tests/bench-report.test.mjs` (node:test; temp dirs removed in finally; no Node 22+ globals). + +Commit: `feat(66-08): add baseline benchmark report with A/A noise floor`. No Co-Authored-By or AI attribution. + + + node --test tests/bench-report.test.mjs + + + - `node --test tests/bench-report.test.mjs` exits 0. + - `node -e "import('./scripts/bench/report.mjs').then(m=>{for(const k of ['loadCampaign','buildSummary','renderMarkdown','writeReport'])if(typeof m[k]!=='function')process.exit(1)})"` exits 0. + - `grep -c "smoke (not a baseline)" scripts/bench/report.mjs` prints at least 1, and `grep -c "list-price estimate" scripts/bench/report.mjs` prints at least 1 (or the label is imported from host-stream.mjs, verified by `grep -c "COST_LABEL_LIST_PRICE" scripts/bench/report.mjs`). + + The report turns saved runs into the baseline statistics and labels Phase 71 needs, with smoke and live results kept distinct. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Runner → user's machine state | The runner spawns an MCP hub on 7225, Claude/Cursor processes, and reads a Chrome profile over CDP. | +| Agent and pages → evidence | Pages and agents act within the timed window; evidence must be independent. | +| Raw run records → git | Raw records hold request bodies and live page text. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-01 | Information disclosure | identity.json / runs.jsonl | mitigate | Identity records ids, versions, digests, model and provider names only. No env dump, keys, bridge credentials or fixture secret. `validateCommittedRow` enforces an exact key set, bounded strings and no `://`. | +| T-66-02 | Information disclosure | committed results | mitigate | Raw records (bodies, live text, streams) go only to gitignored `scripts/bench/.local/`. Committed rows carry check pass/fail and numbers only. The report refuses invalid rows. | +| T-66-04 | Tampering | outcome evaluation | mitigate | Evidence comes from the fixture server's in-process log (secret never exposed) and post-window CDP reads. Claims are recorded separately and never passed to evaluateOutcome. | +| T-66-ARGV | Elevation of privilege | Claude/Cursor spawn | mitigate | The argv comes from buildClaudeArgv (pinned dontAsk, allowedTools mcp__fsb, disallowed Bash/Edit/Write/...), with an assertion that no FORBIDDEN_HOST_FLAGS entry is present. The empty temp cwd means no project skills or settings. The task goes to Claude via stdin; no shell. | +| T-66-01b | Elevation of privilege | hub ownership | mitigate | The preflight refuses unless the 7225 listener is the runner's own server PID, so runs never go through a squatter hub or the user's everyday extension (Web Store id refused). | +| T-66-01c | Information disclosure | --pair-via-cdp pairing code | mitigate | The code comes from the product's own `pair --json`, lives only in a local variable, and is written only into the bench-profile extension's `chrome.storage.session` over the local CDP socket. It is never logged, persisted, placed in identity or rows, or included in error messages. A source check forbids printing it. | +| T-66-01d | Information disclosure / Elevation of privilege | host CLI environment | mitigate | `buildBenchChildEnv` passes only allowlisted names (HOME, PATH, USER, LOGNAME, SHELL, TMPDIR, LANG, LC_*, proxy variables) and asserts that CLAUDECODE, CLAUDE_CODE_*, CLAUDE_* and every DELEGATION_PROVIDER_KEY_NAMES entry are absent. Provider keys and nested-session markers therefore never reach Claude or Cursor, which keeps the MCP arm on the user's own login. identity.json records key names and a hash, never values (B3; unit-tested in tests/bench-drivers.test.mjs). | +| T-66-SPEND-PAUSE | Denial of service (financial) | capped or failing providers | mitigate | The first `agent_cap`, `agent_auth` or `provider_error` pauses the campaign (exit 75) instead of burning runs, including the silent uncapped-429 case detected by a timeout whose present session detail has no recorded provider response. A timeout with missing detail never pauses (B-TIMEOUT), so legitimate timeouts cannot trigger a false pause-and-resume loop. `--resume` re-runs only the infra-failed runIds (W4, W-A). Status-less numbers never pause. | +| T-66-07 | Tampering (integrity of production data) | benchmark telemetry pollutes production stats | mitigate | The preflight refuses with `telemetry_not_opted_out` unless the bench extension's `fsbTelemetryOptOut` is strictly true, as read over CDP (W-B). No run can start with telemetry on, identity.json records the value, and 66-10 closes the install-announcement window before the flag can be set. | + + + +- `node --test tests/bench-report.test.mjs` +- The dry run on the sample set prints 4 runs, and the smoke run without `--claude-model` exits 2 before spawning. +- `git status --porcelain -- mcp/` is empty. + + + +- MEAS-03 runner and report exist, and live-ready runs are gated by preflight (the live execution is plans 66-10 and 66-11). +- D-06, D-12, D-13, D-14, D-16, D-17, D-18, D-19, D-20 and D-21 are implemented in code with tests where pure. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Do not edit package.json or tests/delegation-phase-contract.test.js in this plan; plan 66-09 registers new tests. Never stage `scripts/bench/.local/`. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-08-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-08-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-08-SUMMARY.md new file mode 100644 index 000000000..3d793a90a --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-08-SUMMARY.md @@ -0,0 +1,354 @@ +--- +phase: 66-baseline-and-session-timing +plan: 08 +subsystem: testing +tags: [benchmark, runner, mcp-sdk, cdp, claude-code, cursor-agent, abba-schedule, infra-classification, report, wilson, bootstrap, node-test] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 02) + provides: "stats.mjs (wilson, median, quantile, summarize, aaNoiseFloor, mulberry32), outcome.mjs (claim-blind evaluateOutcome), identity.mjs (evaluatePreflight, toolsListDigest, gitIdentity, servedModelMatches)" + - phase: 66-baseline-and-session-timing (plan 03) + provides: "fsb://config run identity with the Phase 66 measurement marker" + - phase: 66-baseline-and-session-timing (plan 05) + provides: "tasks.mjs, fixture-server.mjs, host-stream.mjs (argv builders, stream summaries, errorKind, rateLimit)" + - phase: 66-baseline-and-session-timing (plan 06) + provides: "get_session_detail measurement block (kind mcp or autopilot, recorded, callIntervalMs, servedModels)" + - phase: 66-baseline-and-session-timing (plan 07) + provides: "frozen core corpus, live set and [data-bench-marker] fixture contract" +provides: + - "scripts/bench/lib/mcp-client.mjs: runner SDK client, server identity, fsb://config reader, run_task with progress-reset timeouts plus an exact budget timer, B-TIMEOUT session recovery, session detail retry" + - "scripts/bench/lib/cdp.mjs: loopback CDP client on mcp/node_modules/ws: evidence reads, target cleanup, worker wake, telemetry flag read, pairing delivery" + - "scripts/bench/lib/claude-arm.mjs: allowlisted host env (B3), shared host-CLI lifecycle with process-group kill ladder (W7, W10), headless Claude Code driver" + - "scripts/bench/lib/cursor-arm.mjs: experimental Cursor driver on the same lifecycle (not in the baseline)" + - "scripts/bench/lib/runs.mjs: seeded ABBA schedule, RUN_ROW_KEYS rows with validation, classifyInfra, pause rule, telemetry safety gate" + - "scripts/bench/run-bench.mjs: CLI with --dry-run, --preflight, --smoke, --campaign, --resume, --pair-via-cdp; per-run saves; exit 75 pauses" + - "scripts/bench/report.mjs: loadCampaign, buildSummary, renderMarkdown, writeReport and its CLI" + - "tests/bench-drivers.test.mjs (27 cases) and tests/bench-report.test.mjs (79 cases), unregistered until 66-09" + - "tests/fixtures/bench/run-task-provider-error.json and infra-classification-cases.json (39 synthetic cases)" +affects: [66-09, 66-10, 66-11, 71, benchmark smoke, baseline campaign, bench report] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Every CLI dependency is injectable (main(argv, overrides)), so preflight, pairing, pause and resume are tested with fakes and nothing live" + - "Argument validation collects every refusal code before any spawn, pairing or file write" + - "Host CLIs run detached in their own process group; budget expiry sends SIGINT, SIGTERM, SIGKILL to the group, and every exit reaps the group" + - "Raw records keep evidence and measurement blocks under gitignored .local; committed rows are rebuilt from the latest raw record of every runId and validated field by field" + - "Error texts are classified in memory only and never stored, printed or committed" + +key-files: + created: + - scripts/bench/lib/mcp-client.mjs + - scripts/bench/lib/cdp.mjs + - scripts/bench/lib/claude-arm.mjs + - scripts/bench/lib/cursor-arm.mjs + - scripts/bench/lib/runs.mjs + - scripts/bench/run-bench.mjs + - scripts/bench/report.mjs + - tests/bench-drivers.test.mjs + - tests/bench-report.test.mjs + - tests/fixtures/bench/run-task-provider-error.json + - tests/fixtures/bench/infra-classification-cases.json + modified: [] + +key-decisions: + - "run_task also gets an exact budget timer (AbortSignal): SDK 1.29 checks maxTotalTimeout only when a progress event arrives, and the heartbeat ticks every 30 s" + - "The SDK version is read from the real package root; the exports map sends '/package.json' to a versionless dist/cjs/package.json" + - "A host run that used its whole budget is a task failure, not host_no_result, so timeouts stay in the completion denominator" + - "A host run whose FSB calls never produced an MCP session, or any run whose post-window CDP read failed, is extension_not_connected (infra), not a task failure" + - "--smoke defaults to one round (the W3 revision); the plan's flag list still said 2" + - "order is the 1-based position in the whole campaign schedule, so drift over a campaign can be checked" + - "Cost per completed task is the cost of every counted run divided by completed runs; infra-error runs are excluded" + - "A resumed campaign must keep its stable identity (git sha, extension id and version, tools hash, models, task lock, schedule) or it is refused" + - "TDD: each task has a RED test commit before its GREEN commit; the GREEN commits carry the plan's messages" + +patterns-established: + - "66-10 reads the MCP session id of a Claude run from .local//runs/.json sessions[].id" + - "Plan 66-09 appends 'tests/bench-report.test.mjs tests/bench-drivers.test.mjs' to the single bench node --test command" + +requirements-completed: [MEAS-01, MEAS-03] + +# Metrics +duration: 35min +completed: 2026-09-30 +--- + +# Phase 66 Plan 08: Benchmark Runner, Arm Drivers and Baseline Report Summary + +**A benchmark runner that owns the 7225 hub through its own MCP SDK client, refuses to start unless identity, hub ownership and the bench extension's telemetry opt-out all check out, runs a seeded ABBA schedule of autopilot and headless Claude Code runs on its own clock, judges each run only on fixture-server and post-window CDP evidence, saves every run the moment it ends, and pauses (exit 75) on the first usage cap, auth failure or provider failure. A report turns the saved rows into completion with Wilson CIs, wall time per completed task, labelled cost, host metrics and the A/A noise floor.** + +This plan delivers the runner and report of MEAS-03 and the driver-side host metrics of MEAS-01 (D-12). No live system was touched: every test uses fakes, and the live smoke and campaign are plans 66-10 and 66-11. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched; the orchestrator owns them. + +## Performance + +- **Duration:** 35 min +- **Started:** 2026-09-30T09:52:25Z +- **Completed:** 2026-09-30T10:27:38Z +- **Tasks:** 3/3 +- **Files:** 11 created, exactly the plan's `files_modified`; no existing file changed + +## Accomplishments + +- **Runner MCP client (`mcp-client.mjs`):** + - `openRunnerClient` spawns `node mcp/build/index.js` over stdio with the SDK from `mcp/node_modules` (resolved lazily) and drains the server's stderr into a `.local` log. + - `collectServerIdentity` reads `tools/list` across every page before digesting it. + - `runTaskTimed` sends the plan's options (`resetTimeoutOnProgress: true`, `timeout: 60000`, `maxTotalTimeout`). + - On a timeout it calls `get_task_status` first, then `stop_task` exactly once, then the `list_sessions` fallback (latest non-`mcp-agent` session inside [t0, t1]). +- **CDP client (`cdp.mjs`):** + - Every `Runtime.evaluate` passes `returnByValue: true, awaitPromise: true`. An exception rejects with the fixed message `cdp_evaluate_exception`. + - Evidence reads: core runs take the target with the most `[data-bench-marker]` elements (a tie goes to the later target); live runs match the start URL's host and keep up to 200000 characters of text in memory. + - A sleeping worker is woken through `PUT /json/new` on the control panel, and the wake tab is closed afterwards. + - Only loopback debugger URLs are accepted. +- **Host drivers (`claude-arm.mjs`, `cursor-arm.mjs`):** + - `buildBenchChildEnv` copies the allowlist only, then refuses anything named `CLAUDECODE`, `CLAUDE_*` or in the 149-name `DELEGATION_PROVIDER_KEY_NAMES` list (imported from `mcp/build`). + - Claude runs in an empty temp workdir holding only `mcp.json`, and takes the task on stdin. + - The shared `runHostCli` lifecycle tees stdout to a 0600 raw file and keeps only a `{type}` stub for partial-message events. + - The ladder ends in a group SIGKILL, and the group is reaped after every exit. +- **Run contract (`runs.mjs`):** + - Seeded ABBA schedule: 240 entries for 12 tasks x 10 rounds x 2 arms. + - `toCommittedRow` builds each row by picking named fields. `validateCommittedRow` checks exact keys at every level, types, the flag vocabulary, and that strings are at most 200 characters with no `://`. + - `classifyInfra` only counts a number as a status after `HTTP` or `status`. + - `pauseDecision` and `evaluateBenchSafety` (strict `true` only). +- **CLI (`run-bench.mjs`):** + - Every argument error is listed at once and exits 2 before any side effect. That includes W9's extension-id check, which also fires with `--dry-run`. + - `--pair-via-cdp` hands the `pair --json` code straight to `pairExtensionViaCdp`; it is never printed. + - Preflight = `evaluatePreflight` plus the `fsbTelemetryOptOut` gate. + - Identity records env key names and their SHA-256, Chrome launch flag names, `startTabActive: true`, the agent-policy hash, and the tool-loading modes observed in `init.tools`. + - Per-run raw record plus an atomic `runs.jsonl` rewrite; exit 75 pauses with the exact `--resume` command; `--resume` re-runs only runIds whose latest attempt hit an infra error. +- **Report (`report.mjs`):** + - Per set and per arm: infra errors leave the denominator and are listed by code; completion with a Wilson CI; per-task median and p90; geometric mean of the task medians; cost per completed task with sources and label. + - FSB per-call medians, with the caller-gap and call-interval labels (W6); Claude API share, turns and the list-price label. + - A/A floor from `stats.aaNoiseFloor`, reported as `n/a` below two rounds. Under 10 rounds the campaign is labelled `smoke (not a baseline)`; live runs are reported separately. + +Sample one-round report excerpt (rendered from synthetic rows): + +``` +- Completion: 1/2 = 50.0% (Wilson 95% CI 9.5% to 90.5%); timed out 0 +- Wall time per completed task: 42,000 ms, the geometric mean of per-task median wall times over completed runs on the runner's clock (1 task; no completed run: core-b-two) +## A/A noise floor +- core / autopilot: n/a (needs at least 2 rounds) +- core / claude FSB: 7 calls per run (median); FSB time per call 310 ms; caller gap per call 8,200 ms (upper bound on host thinking time); call interval 8,600 ms (receive-to-receive call interval; live counterpart of the 66-01 history deltas); queue wait not measured +``` + +## Interface notes for plans 66-09, 66-10 and 66-11 + +- **66-09:** add `tests/bench-report.test.mjs tests/bench-drivers.test.mjs` to the bench `node --test` command. + - Both are Node 20-safe; together they take about 3 s (the kill-ladder cases use real 150-250 ms graces). + - `bench-drivers` needs the compiled `mcp/build/agent-providers/spawn-environment.js` and `mcp/node_modules/ws`. Both exist in the chain once `npm --prefix mcp run build` has run earlier in it. +- **66-10:** + - `--help` lists every flag, including `--pair-via-cdp`. + - `--preflight` prints JSON `{ok, mode, hub:{listenerPid, serverPid, owned}, identity}`, where `identity.extension.telemetryOptOut` is the flag read over CDP. It writes no `identity.json`. + - A smoke with no `--tasks`/`--rounds` is every task x 1 round. + - Outputs under `scripts/bench/.local//`: + - `runs/.json` (raw record, with `sessions[].id` and `measurement`); + - `runs/.attempt.json` (superseded attempts); + - `streams/.attempt.jsonl` (host streams); + - `progress.json` and `mcp-server.stderr.log`. + - For a smoke, `identity.json` and `runs.jsonl` live here too. + - Pairing refusals print `pairing_failed: `, for example `service_worker_unreachable`, `cdp_unavailable`, `pair_json_failed` or `extension_not_connected_after_pairing`. + - Reusing a campaign id that already has saved runs is refused as `campaign_exists` unless `--resume` is given. +- **66-11:** + - A campaign writes `identity.json` and `runs.jsonl` to `scripts/bench/results//`. + - `node scripts/bench/report.mjs ` adds `summary.json` and `report.md`, and exits 1 on any invalid row. None of the four files carries `://`. + - A resume refuses `resume_identity_mismatch` if the build, tools hash, models, task lock or schedule changed, and records each resume in `identity.resumes`. + +## Task Commits + +1. **Task 1: Arm drivers and clients** + - `3f5f37f2` (test, RED): all imports missing + - `6378fafd` (feat, GREEN): `feat(66-08): add benchmark MCP, CDP, Claude Code and Cursor drivers` +2. **Task 2: Seeded ABBA schedule, sanitized rows, run-bench CLI** + - `f9f060e7` (test, RED): `runs.mjs` missing + - `4bd408d5` (feat, GREEN): `feat(66-08): add seeded ABBA benchmark runner with preflight and per-run saves` +3. **Task 3: Baseline report** + - `56c0b5f8` (test, RED): the 69 Task 2 cases pass, the 10 report cases fail + - `a9a491b3` (feat, GREEN): `feat(66-08): add baseline benchmark report with A/A noise floor` + +**Plan metadata:** the `docs(66-08)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `scripts/bench/lib/mcp-client.mjs`: the runner's SDK client and tool helpers. +- `scripts/bench/lib/cdp.mjs`: CDP evidence, cleanup, worker wake, flag read and pairing. +- `scripts/bench/lib/claude-arm.mjs`: env allowlist, the shared host lifecycle, version probes and the Claude driver. +- `scripts/bench/lib/cursor-arm.mjs`: the experimental Cursor driver. +- `scripts/bench/lib/runs.mjs`: schedule, rows, validation, classification, pause rule and safety gate. +- `scripts/bench/run-bench.mjs`: the CLI, arms, executor, loop, identity and preflight. +- `scripts/bench/report.mjs`: the report library and CLI. +- `tests/bench-drivers.test.mjs`: 27 cases with fake CLIs, a fake MCP client and a ws CDP stub. +- `tests/bench-report.test.mjs`: 79 cases. 39 are classification cases, and the in-process runner cases use injected fakes. +- `tests/fixtures/bench/run-task-provider-error.json` and `infra-classification-cases.json`: synthetic fixtures, each with a `_fixture` note. + +## Decisions Made + +- **Raw records** keep: + - the session ids and 66-06 measurement blocks; + - the host summary; + - the safe run_task fields (status, partial outcome, sw_evicted); + - the evidence (request bodies, final URL, markers, live text) under `.local` only. + + They never keep the full session detail, the run_task error, or the session's terminal error text. +- **Host session selection:** the Claude arm takes `mcp-agent` sessions whose start falls in [t0, t1 or exit]. When there are several, it uses the one with the most calls and flags `multiple_mcp_sessions`. Journal sessions are indexed when they are created (`isNew` in `mcp-lattice-journal.js`), so they can be listed right after the host exits. +- **Autopilot start failure:** if `open_tab` fails, the run never starts. It is classified from its text (`extension_not_connected`, `fsb_agent_cap`, ...), and otherwise as `spawn_failed`. +- **Arm crash:** an exception thrown inside an arm is `spawn_failed`. The error name and a bounded message go into the raw record for debugging. +- **Telemetry refusal detail:** `false` gives 'false', `undefined` gives 'absent', and anything else (null, the string 'true') gives 'unverifiable'. +- **Stderr tail:** the host's stderr tail is appended to the raw stream file as one `_bench_stderr_tail` NDJSON line, so the file stays one valid NDJSON artifact per run. +- **Identity tool loading:** `hosts.claude.toolLoadingObserved` collects the tool-loading modes that runs actually report in `init.tools` (D-18). `toolSearchEnv` records the launched `ENABLE_TOOL_SEARCH`. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] The SDK version would always read as undefined** +- **Found during:** Task 1 +- **Issue:** RESEARCH's `require('@modelcontextprotocol/sdk/package.json').version` goes through the SDK's `./*` exports map to `dist/cjs/package.json`, which has no version (verified: it printed `undefined`). +- **Fix:** `resolveSdk` resolves the client entry, then walks up to the package root whose `name` is `@modelcontextprotocol/sdk`. +- **Files modified:** `scripts/bench/lib/mcp-client.mjs` +- **Committed in:** `6378fafd` + +**2. [Rule 1 - Bug] run_task could overrun its budget by up to 30 s** +- **Found during:** Task 1 (read from SDK 1.29 `protocol.js`) +- **Issue:** `maxTotalTimeout` is checked only inside `_resetTimeout`, which runs when a progress notification arrives (every 30 s). A 300 s budget could therefore end anywhere up to 330 s. +- **Fix:** an AbortSignal timer at exactly `budgetMs` is combined with the runner's stop signal and passed as `signal`. The plan's four options are passed unchanged. +- **Verification:** the case "runTaskTimed enforces the budget itself when no progress event arrives" passes. +- **Committed in:** `6378fafd` + +**3. [Rule 2 - Missing critical functionality] Some infra failures would have counted as task failures** +- **Found during:** Task 2 review of live paths +- **Issue, extension disconnected with Chrome still up:** Claude's FSB calls never reach the extension, so no journal session exists, and the run would count as a failure. Runs alternate between arms (ABBA), so each Claude failure would reset the "two consecutive infra errors" count and the campaign would never pause. +- **Issue, Chrome gone:** the post-window CDP read fails, and every run would become a task failure. +- **Fix:** + - A host run whose FSB calls produced no MCP session in the window is `extension_not_connected`. `classifyInfra` gains an additive `sessionMatch` input, so a session that was found but whose detail could not be read is not misclassified. + - A failed evidence read is `extension_not_connected` too. +- **Verification:** three new classification cases, and an executor case for a thrown and a flagged evidence read. +- **Committed in:** `4bd408d5` + +**4. [Rule 2 - Missing critical functionality] Campaign-directory and resume safety** +- **Found during:** Task 2 +- **Issue:** starting a new campaign under an existing id would mix two campaigns' runs, and a resume on a different build would mix measurements. +- **Fix:** + - `campaign_exists` refuses a non-resume start when runs are already saved. + - `resume_not_found` refuses a resume with nothing to continue. + - `resume_identity_mismatch` compares the stable identity fields; resumes are appended to `identity.resumes`. +- **Committed in:** `4bd408d5` + +**5. [Rule 2 - Security] CDP input hardening** +- **Found during:** Task 1 +- **Fix:** + - `evaluate` accepts only loopback `ws://` debugger URLs. + - Storage keys must match `^[A-Za-z0-9_]{1,64}$` before one is written into an expression. + - `/json/close` ids are pattern-checked. + - A run token must match the fixture server's pattern. +- **Committed in:** `6378fafd` + +**6. [Rule 1 - Bug] Test-data errors in the RED commits, fixed in GREEN** +- **Task 1:** the sentinel pairing code was 42 characters after `fsb-auth.`, one short of the real 43, so the validator rightly refused it. The env-metadata check compared values against key names, and the short value `'1'` appeared in the SHA-256 hex. +- **Task 3:** the plan says "120 rows (3 tasks x 10 rounds x 2 arms)", but 3 x 10 x 2 is 60. The test keeps 120 rows and 10 rounds by using 6 tasks. +- **Committed in:** `6378fafd`, `a9a491b3` + +### Execution notes (not rule deviations) + +- **Smoke rounds:** the plan's flag list says `--rounds N (smoke 2, campaign 10)`, while its W3 revision says a smoke defaults to `--rounds 1`. 66-10 depends on 12 x 1 x 2 = 24 runs, so the default is 1. +- **Host timeout without a result:** such a run is a task failure (null), not `host_no_result`. Otherwise timed-out Claude runs would leave the denominator and inflate completion. A cap, auth or API error signal still wins, and so does a missing init. +- **Additive exports:** + - `claude-arm.mjs`: `runHostCli`, `hostVersion`, `BENCH_ENV_ALLOWLIST`, `TOOL_SEARCH_MODES` + - `cdp.mjs`: `evaluate`, `listTargets`, `findExtensionWorker`, `LIVE_TEXT_MAX_CHARS` + - `mcp-client.mjs`: `resolveSdk`, `isRequestTimeoutError`, constants + - `runs.mjs`: `pauseDecision`, `servedModelMatch`, `RUN_FLAGS`, `ARMS`, `HOST_ARMS`, `CAMPAIGN_ID_PATTERN` + - `run-bench.mjs`: `main`, `parseArgs`, `resolveOptions`, `createRunExecutor`, `runCampaignLoop`, the two arm runners, `readPairingCode`, `readHubListenerPid`, `buildResumeCommand`, `loadLatestRawRecords`, `defaultDeps` + - `report.mjs`: labels and constants +- **`classifyInfra` inputs:** besides the plan's list, it accepts `startFailed` and `sessionMatch`. +- **Branch checks:** the executor's `worktree-agent-*` allow-list was skipped, because the orchestrator directs commits on `Jev`. The cwd-drift sentinel and the protected-branch deny-list ran before every commit. + +--- + +**Total deviations:** 6 auto-fixed (3 Rule 1, 3 Rule 2). +**Impact on plan:** +- Each fix keeps a measurement honest: an exact budget, the true SDK version, no infra failure counted as a task failure, and no mixed campaigns. The CDP fix closes an input path. +- The footprint is exactly the plan's 11 files. + +## TDD Gate Compliance + +Every task is `tdd="true"`, and each has a `test(66-08)` RED commit before its `feat(66-08)` GREEN commit: `3f5f37f2` then `6378fafd`, `f9f060e7` then `4bd408d5`, and `56c0b5f8` then `a9a491b3`. The new suites are not in the `npm test` chain until 66-09, so a RED commit cannot turn the chain red. The plan type is `execute`, so the plan-level gate does not apply. + +## Verification Evidence + +- **Plan ``:** + - `node --test tests/bench-report.test.mjs` passes 79/79. + - The sample dry run prints `"runs":4`. + - `--smoke --arms claude` with no model exits 2 with `claude_model_required` and `extension_id_invalid`. + - `git status --porcelain -- mcp/` is empty. +- **Task verify commands:** all exit 0, including the driver-export `node -e` check and the W-A/W-B classifier `node -e` check. +- **Acceptance greps:** + + | Check | Result | + |-------|--------| + | `createRequire` in cdp.mjs | 3 | + | global WebSocket references | 0 | + | bypass-flag grep | empty | + | `resetTimeoutOnProgress: true` | 1 | + | `get_task_status` in mcp-client.mjs | 4 | + | `awaitPromise: true` in cdp.mjs | 1 | + | `detached: true` in claude-arm.mjs | 1 | + | no-env-copy check | exit 0 | + | no-print check on cdp.mjs and run-bench.mjs | exit 0 | + | `evaluatePreflight(` / `evaluateOutcome(` in run-bench.mjs | 1 / 1 | + | `smoke (not a baseline)` in report.mjs | 1 | + | `COST_LABEL_LIST_PRICE` in report.mjs | 2 | + | outcome.mjs, identity.mjs, agent-loop.js unchanged | yes | + +- **Node 20.20.2** (the binary already in the local npx cache; nothing was downloaded): the two new suites pass 106/106. **Node 24.14.1:** 106/106. The driver suite was run three times in a row with no flake. +- **Regression:** + - The seven earlier bench suites pass 218/218. + - `tests/mcp-in-flight-session-lookup.test.js` (the one existing test that imports `scripts/bench`) passes 214/0. +- **Mutation probes:** each was applied to the committed modules and then restored byte for byte, and each was caught. + - runs.mjs and run-bench.mjs (6): status prefix dropped, timeout without detail treated as a provider failure, attempt archiving removed, error text stored in the raw record, pausing codes no longer immediate, a truthy telemetry flag accepted. + - report.mjs (5): infra rows kept in the denominator, the one-round A/A guard removed, the baseline threshold lowered, live rows mixed into core, row validation skipped. +- **No leftovers:** + - no fake CLI or grandchild process survived; + - `scripts/bench/.local/` is still empty, and `scripts/bench/results/` does not exist; + - no tracked file was rewritten; + - nothing ran against 7225, Chrome or a real host CLI. + +## Issues Encountered + +- **graphify is stale:** its graph predates `scripts/bench/` (built 2026-07), so the query returned unrelated nodes. Every anchor used here was read from the working tree. +- **Stale anchors in the plan:** it cites `extension/ws/mcp-tool-dispatcher.js` lines 2738-2756 for `handleGetStatusRoute`. The route is now at ~3180. It still returns `currentSessionId`, and `stop_task` still resolves the in-flight session from the same map. +- **Not yet proven live:** these behaviors are covered by fakes only, and 66-10 is the first live run: + - `Runtime.evaluate` on a real MV3 service-worker target; + - `PUT /json/new` with a `chrome-extension://` URL; + - the real `claude` stream through this driver; + - lsof-based hub ownership. + +## Known Stubs + +None. Every module is complete and wired: `run-bench.mjs` composes the real drivers, fixture server, CDP client and report inputs. The Cursor arm is intentionally experimental and runs only when `--arms` lists it. + +## Threat Flags + +None beyond the plan's threat register: +- `mcp-server.stderr.log` and the raw host streams are raw captures under the gitignored `.local` tree (T-66-02). +- The `lsof`, `git` and `pair --json` child processes are the read-only probes the plan names (T-66-01b, T-66-01c). + +## User Setup Required + +None for this plan. The bench profile, xAI key, Claude model choice and telemetry opt-out are human steps in 66-10. + +## Next Phase Readiness + +- **66-09:** registers the two new suites (see the interface notes). +- **66-10:** runs `--preflight --pair-via-cdp` and then the 24-run smoke. Pairing and CDP are exercised live there for the first time. If the worker wake fails, its human checkpoint offers "open the control panel once, then retry". +- **Blockers:** none. + +## Self-Check: PASSED + +- All 11 created files exist on disk. +- All 6 task commits exist: `3f5f37f2`, `6378fafd`, `f9f060e7`, `4bd408d5`, `56c0b5f8`, `a9a491b3`. +- No commit in `698016a1..HEAD` has a Co-Authored-By or AI attribution line. +- `git diff --name-only 698016a1..HEAD -- .planning/STATE.md .planning/ROADMAP.md .planning/REQUIREMENTS.md extension mcp package.json .gitignore` and the 66-01 files is empty. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-09-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-09-PLAN.md new file mode 100644 index 000000000..a9d9d58c9 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-09-PLAN.md @@ -0,0 +1,170 @@ +--- +phase: 66-baseline-and-session-timing +plan: 09 +type: execute +wave: 4 +depends_on: ["66-01", "66-02", "66-03", "66-04", "66-05", "66-06", "66-07", "66-08"] +files_modified: + - package.json + - tests/delegation-phase-contract.test.js +autonomous: true +requirements: [MEAS-01, MEAS-02, MEAS-03] +user_setup: [] + +must_haves: + truths: + - "All ten new benchmark test suites (nine planned originally plus tests/bench-drivers.test.mjs from the 66-08 revision) run in the root npm test chain through one appended command, and that command is listed in REFINEMENT_ROOT_TEST_COMMANDS in the same commit. The historical pre-phase hashes and existing chain order are untouched." + - "The full npm test chain (with pretest) shows no failure beyond the start-commit baseline recorded in 66-02-SUMMARY.md, judged by exit code and last suite reached with port 7225 free. npm run validate:extension passes." + - "The benchmark unit suites pass under the official Node 20 binary, matching CI." + - "No timing appears in any tool response. The timing Symbol is declared only in the bridge client and dispatcher, and fsbMs appears only in the recording, journal, projection and history modules (D-04)." + - "No source file under mcp/ changed during Phase 66 (D-01). The benchmark never runs inside npm test or CI; only its pure unit tests do (D-13)." + artifacts: + - path: "package.json" + provides: "The root test chain with one appended `node --test tests/bench-*.test.mjs` command" + contains: "tests/bench-corpus.test.mjs" + - path: "tests/delegation-phase-contract.test.js" + provides: "REFINEMENT_ROOT_TEST_COMMANDS entry for the bench command" + contains: "tests/bench-report.test.mjs" + key_links: + - from: "package.json scripts.test" + to: "tests/delegation-phase-contract.test.js REFINEMENT_ROOT_TEST_COMMANDS" + via: "identical command string appears exactly once in both" + pattern: "node --test tests/bench-export-history.test.mjs" +--- + + +Register every new benchmark unit test in the root chain in one commit, then run the phase-level integration verification: the full chain against the recorded baseline, extension validation, the Node 20 bench run, the "no timing in responses" audit, and the "no mcp/ change" audit. + +Purpose: This is the phase-level integration step the orchestrator requires (research plan 66-07). It proves MEAS-01..03 code is wired and green before any live run. +Output: package.json and tests/delegation-phase-contract.test.js updates, plus a verification record in the SUMMARY. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-VALIDATION.md +@.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md +@.planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +The exact command string to register (one entry; the Google Sheets precedent shows a multi-file `node --test` entry is one command): +`node --test tests/bench-export-history.test.mjs tests/bench-stats.test.mjs tests/bench-outcome.test.mjs tests/bench-identity.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs tests/bench-claude-stream.test.mjs tests/bench-corpus.test.mjs tests/bench-report.test.mjs tests/bench-drivers.test.mjs` + +Chain rules (tests/delegation-phase-contract.test.js ~129-155, ~2373-2391, ~3118-3134; tests/mcp-client-identity-integration.test.js ~469-486): +1. Append the command at the TAIL of `scripts.test`, joined with ` && `. Today the tail is `node --import tsx tests/no-orphan-descriptor.test.js`. +2. Add the identical string as the last element of `REFINEMENT_ROOT_TEST_COMMANDS`. +3. Never edit `PRE_PHASE61_ROOT_TEST_HASH`, `PRE_PHASE62_ROOT_TEST_HASH` or any other historical constant. +4. Never reorder existing entries. + + + + + + + Task 1: Register the ten benchmark suites in the root chain and the refinement contract in one commit + package.json, tests/delegation-phase-contract.test.js + + - package.json (scripts.test tail, scripts.pretest) + - tests/delegation-phase-contract.test.js lines 1-30, 120-160, 2365-2395, 3115-3140 + - tests/mcp-client-identity-integration.test.js lines 460-490 + - The ten test files, which must all exist: tests/bench-export-history.test.mjs, tests/bench-stats.test.mjs, tests/bench-outcome.test.mjs, tests/bench-identity.test.mjs, tests/bench-tasks.test.mjs, tests/bench-fixture-server.test.mjs, tests/bench-claude-stream.test.mjs, tests/bench-corpus.test.mjs, tests/bench-report.test.mjs, tests/bench-drivers.test.mjs + + +1. Confirm all ten files exist and pass together: `node --test `. If any file is missing or failing, stop and report which upstream plan owns it. Do not edit files owned by other plans. +2. Edit package.json `scripts.test` by appending ` && ` plus the exact command string from at the end. Change nothing else in package.json. Keep JSON formatting and key order: do a textual edit of the one string value, not a re-serialization that reorders keys. +3. Append the identical string as a new final element of `REFINEMENT_ROOT_TEST_COMMANDS` in tests/delegation-phase-contract.test.js, with a trailing comma consistent with the array's style. Touch nothing else in that file. +4. Run `node tests/delegation-phase-contract.test.js && node tests/mcp-client-identity-integration.test.js`, and confirm the chain still has exactly one `npm --prefix mcp run build` entry. + +Commit both files together: `test(66-09): run the benchmark unit suites in the root test chain`. No Co-Authored-By or AI attribution. + + + node tests/delegation-phase-contract.test.js && node tests/mcp-client-identity-integration.test.js && node -e "const c=require('./package.json').scripts.test.split(' && ');const cmd='node --test tests/bench-export-history.test.mjs tests/bench-stats.test.mjs tests/bench-outcome.test.mjs tests/bench-identity.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs tests/bench-claude-stream.test.mjs tests/bench-corpus.test.mjs tests/bench-report.test.mjs tests/bench-drivers.test.mjs';if(c.filter(x=>x===cmd).length!==1||c[c.length-1]!==cmd)process.exit(1)" + + + - The registered command appears exactly once in `scripts.test` and is the last entry (verify command). + - `node tests/delegation-phase-contract.test.js` and `node tests/mcp-client-identity-integration.test.js` exit 0. + - `git diff HEAD~1 -- tests/delegation-phase-contract.test.js | grep -cE "^[-+].*PRE_PHASE6[0-9]_ROOT_TEST_HASH"` prints 0 (no historical hash edited). + - `git show --stat HEAD` lists exactly package.json and tests/delegation-phase-contract.test.js. + + The benchmark unit suites are part of the CI chain through one registered command, with the refinement contract updated in the same commit. + + + + Task 2: Phase integration verification: full chain against the baseline, validate:extension, Node 20 bench run, and the timing and mcp/ audits + none (verification only; results go in the SUMMARY) + + - .planning/phases/66-baseline-and-session-timing/66-02-SUMMARY.md (start SHA, baseline mode, pre-existing failures) + - .planning/phases/66-baseline-and-session-timing/66-VALIDATION.md (sampling rules and sign-off list) + + +1. Port hygiene: run `lsof -nP -iTCP:7225 -sTCP:LISTEN` and `lsof -nP -iTCP:7226 -sTCP:LISTEN`. Do not kill user processes. + - If 7225 is free, run `npm test` (pretest included) in the background, logging to `/tmp/fsb-66-09-full.log` and writing the exit code to `/tmp/fsb-66-09-full.exit`, and poll until done. + - If 7225 is held, run the tolerant per-command pass exactly as in 66-02 Task 1 (stdin from /dev/null, continuing past failures, one JSON line per command in `/tmp/fsb-66-09-full.jsonl`). + - Compare against the 66-02 baseline. Every failing suite must already be in the recorded pre-existing set, or be port-bound because 7225 is held. Any new failure blocks this plan: report the suite and the owning plan (by files_modified) instead of editing another plan's files. + - Judge by exit code and last suite reached, not by FAIL counts. +2. Revert run side effects: `git checkout --` any modified tracked file under `showcase/angular/public/` (llms.txt, llms-full.txt, sitemap.xml), `mcp/build/`, `mcp/ai/tool-definitions.cjs`, and any package-lock.json. Confirm `git status --porcelain --untracked-files=no` is empty. +3. Run `npm run validate:extension`, which must exit 0. +4. Node 20 run: + - Download the official Node 20 binary for this machine (`uname -m`: arm64 means darwin-arm64, else darwin-x64). Resolve the exact file name from `https://nodejs.org/dist/latest-v20.x/SHASUMS256.txt`, download `node-v20.*-darwin-.tar.gz` to /tmp, verify its SHA-256 against SHASUMS256.txt, and extract it. Do NOT use `npx node@20`. + - Run `/tmp/node-v20.../bin/node --test` on the ten bench files; it must exit 0. Record the exact Node 20 version. + - If the download is impossible (no network), record that explicitly in the SUMMARY as an unverified item, and still run the suites under the local Node to confirm there are no regressions. Never claim a Node 20 pass that did not happen. +5. Timing audits (D-04): + - `grep -rln "Symbol.for('fsb.mcp.call-timing')" extension` must list exactly extension/ws/mcp-bridge-client.js and extension/ws/mcp-tool-dispatcher.js. + - `grep -rln "fsbMs" extension --include='*.js'` must be a subset of: mcp-bridge-client.js, mcp-tool-dispatcher.js, mcp-session-recorder.js, mcp-lattice-journal.js, spreadsheet-record-redaction.js, automation-logger.js. + - `node tests/mcp-bridge-background-dispatch.test.js` must pass; its ingress case asserts that `_sendResult` JSON carries no timing. +6. D-01 audit: `git diff --stat ..HEAD -- mcp/` prints nothing; fall back to `50334671` if the SUMMARY lacks the SHA. Also confirm `git diff --stat ..HEAD -- extension/utils/analytics.js` is empty (orchestrator decision 6). +7. Record every result in the SUMMARY: chain mode, exit code, last suite, failure comparison, validate:extension, the Node 20 version and result, each audit's output, and the D-01 diff. + + + npm run validate:extension && test "$(grep -rln "Symbol.for('fsb.mcp.call-timing')" extension | sort | tr '\n' ' ')" = "extension/ws/mcp-bridge-client.js extension/ws/mcp-tool-dispatcher.js " && node tests/mcp-bridge-background-dispatch.test.js && test -z "$(git status --porcelain --untracked-files=no)" + + + - `/tmp/fsb-66-09-full.exit` contains 0, or every failing command in `/tmp/fsb-66-09-full.jsonl` / the log is in the 66-02 pre-existing set or is port-bound with 7225 held. The SUMMARY lists the comparison explicitly. + - `npm run validate:extension` exits 0. + - A Node 20 run of the ten bench suites exits 0 and its version is recorded, or its impossibility is recorded as an explicit unverified item. + - The timing audits produce exactly the file sets above, and the D-01 diff over mcp/ is empty. + - `git status --porcelain --untracked-files=no` is empty after the run. + + The phase's code is integrated and verified against the recorded baseline, on CI's Node version, with the timing-exposure and mcp/ boundaries audited. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Repo test chain → CI | Chain edits change what CI proves. | +| Downloaded Node 20 binary → local execution | An external binary is executed for verification. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-09a | Tampering | package.json / delegation-phase-contract | mitigate | One appended command plus the same string in REFINEMENT_ROOT_TEST_COMMANDS. Historical hashes stay untouched, and the contract test enforces that the pre-phase chain still hashes to its constants. | +| T-66-09b | Tampering | Node 20 download | mitigate | Official nodejs.org dist only, SHA-256 verified against SHASUMS256.txt before extraction, run from /tmp, never installed globally. | +| T-66-05 | Information disclosure | timing in responses | mitigate | The grep audits bound where timing identifiers live, and the bridge ingress test asserts `_sendResult` JSON has no timing. | +| T-66-01 | Information disclosure | test logs in /tmp | accept | The logs contain test output only (no user data); they are local and not committed. | + + + +- Task 1 and Task 2 verify commands pass, and the SUMMARY records the full-chain comparison, the Node 20 result and the audits. + + + +- The phase-level integration verification is complete, with honest results. The code for MEAS-01, MEAS-02 and MEAS-03 is proven green and wired before the live baseline (plans 66-10 and 66-11). + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Commit only package.json and tests/delegation-phase-contract.test.js. Never commit files rewritten by test runs. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-09-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-09-SUMMARY.md b/.planning/phases/66-baseline-and-session-timing/66-09-SUMMARY.md new file mode 100644 index 000000000..81d77a3ae --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-09-SUMMARY.md @@ -0,0 +1,307 @@ +--- +phase: 66-baseline-and-session-timing +plan: 09 +subsystem: testing +tags: [benchmark, test-chain, ci, node20, integration-verification, timing-audit, d-01, d-04, d-13] + +# Dependency graph +requires: + - phase: 66-baseline-and-session-timing (plan 01) + provides: "tests/bench-export-history.test.mjs (Task 1 only; 66-01 still paused at H1)" + - phase: 66-baseline-and-session-timing (plan 02) + provides: "start-commit baseline (311/311, tolerant mode, 7225 held) and the stats, outcome and identity suites" + - phase: 66-baseline-and-session-timing (plans 03, 04, 06) + provides: "the extension timing code that the D-04 audits bound" + - phase: 66-baseline-and-session-timing (plans 05, 07, 08) + provides: "the tasks, fixture-server, claude-stream, corpus, report and drivers suites" +provides: + - "One appended root-chain command (T299) runs all ten tests/bench-*.test.mjs suites; the same string is the last REFINEMENT_ROOT_TEST_COMMANDS entry" + - "Phase integration record: 312/312 chain commands exit 0 against the 311/311 baseline, validate:extension green, official Node 20.20.2 bench run 338/338, D-04 timing audits, D-01 and D-13 audits" +affects: [66-10, 66-11, 71, CI, gsd-verify-work] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Bench unit suites live in one multi-file `node --test` chain entry; adding a suite changes that string in package.json and REFINEMENT_ROOT_TEST_COMMANDS in the same commit" + - "Node 20 evidence comes from the official nodejs.org tarball, checked against SHASUMS256.txt and run from /tmp with its bin dir first on PATH" + +key-files: + created: [] + modified: + - package.json + - tests/delegation-phase-contract.test.js + +key-decisions: + - "The chain was verified in the tolerant per-command mode because 7225 is held by the user's hub (pid 42278). The hub was not touched, so `npm test` itself (fail-fast, 7225 free) was not run locally" + - "Node 20 evidence uses the official v20.20.2 darwin-arm64 tarball (SHA-256 verified), not npx node@20 and not the npx-cache binary earlier plans used" + - "D-01 was audited from both the 66-02 start commit b4fe9d41 (per the plan) and the pre-phase commit 50334671, so 66-01's own commits are covered too" + - "Task 2 is verification-only (its files list is none), so it has no commit; its evidence is this SUMMARY" + +patterns-established: + - "Phase gates compare a tolerant JSONL pass with the 66-02 baseline by command identity and exit code, and treat per-suite count drops as a warning sign" + +requirements-completed: [MEAS-01, MEAS-02, MEAS-03] + +# Metrics +duration: 12min +completed: 2026-09-30 +--- + +# Phase 66 Plan 09: Bench Suites in the Root Chain and Phase Integration Verification Summary + +**The ten benchmark unit suites (338 cases) now run in the root `npm test` chain through one appended `node --test` command, registered in the refinement contract in the same commit. The full chain passes 312/312 commands in tolerant mode against the 311/311 baseline, with no new failure. `validate:extension` is green. All ten suites pass under the official Node 20.20.2 binary. The audits confirm the timing symbol lives only in the two ingress files, `fsbMs` only in the six allowed modules, no tool result carries timing, and nothing under `mcp/` changed during Phase 66.** + +`requirements-completed` lists the plan's IDs for traceability only. This plan proves the MEAS-01..03 code is wired and green before any live run. The live smoke and baseline (66-10, 66-11) and 66-01's H1 export are still open. REQUIREMENTS.md, STATE.md and ROADMAP.md were not touched; the orchestrator owns them. + +**66-01 status when this check ran:** still paused at its human checkpoint H1, after Task 1 (`f406736b`, `b4fe9d41`). +- Its suite `tests/bench-export-history.test.mjs` is registered here and passes (14 cases). +- Its Task 3 (move the raw export into `scripts/bench/.local/`, commit metrics under `scripts/bench/results/history-2026-09-16_21/`) has not run: `scripts/bench/.local/` is empty and `scripts/bench/results/` does not exist. +- This plan touched none of 66-01's files. + +## Performance + +- **Duration:** 12 min +- **Started:** 2026-09-30T10:32:03Z +- **Completed:** 2026-09-30T10:45Z +- **Tasks:** 2/2 +- **Files modified:** 2 (`package.json`, `tests/delegation-phase-contract.test.js`) + +## Task 1: Chain Registration + +The registered command, identical in both files: + +``` +node --test tests/bench-export-history.test.mjs tests/bench-stats.test.mjs tests/bench-outcome.test.mjs tests/bench-identity.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs tests/bench-claude-stream.test.mjs tests/bench-corpus.test.mjs tests/bench-report.test.mjs tests/bench-drivers.test.mjs +``` + +- **Pre-check:** all ten files exist, and together they pass 338/338 under Node 24.14.1 (3.1 s). +- **package.json:** a textual edit appended ` && ` after `node --import tsx tests/no-orphan-descriptor.test.js`. + - The chain went from 298 to 299 entries, and the first 298 are unchanged. + - Top-level key order, script key order and every other value are deep-equal to HEAD~1. +- **Refinement contract:** the command was appended as the last `REFINEMENT_ROOT_TEST_COMMANDS` element, with a trailing comma. That is the file's only change. + +| Check | Result | +|-------|--------| +| Task 1 `` verify | exit 0 | +| `node tests/delegation-phase-contract.test.js` | 1205 passed, 0 failed. The new "appears exactly once in the refinements root chain" check passes, and both historical-hash checks pass ("removing the six new Phase 61 gates…" and "removing the three Phase 62 gates… reproduces the exact prior serial chain") | +| `node tests/mcp-client-identity-integration.test.js` | PASS | +| Command is the last entry and appears exactly once | yes (verify `node -e`) | +| `npm --prefix mcp run build` entries in the chain | 1 | +| `git diff HEAD~1 -- tests/delegation-phase-contract.test.js \| grep -cE "^[-+].*PRE_PHASE6[0-9]_ROOT_TEST_HASH"` | 0 | +| `git show --stat HEAD` | exactly `package.json` (+1/-1) and `tests/delegation-phase-contract.test.js` (+1) | +| Other suites that read `scripts.test` (capability-*, classification-gate, lattice-*, qr-pairing, trigger-tool-dispatcher) | none pins the tail entry or the chain length; all passed in the chain run below | + +## Task 2: Integration Verification Record + +### Ports and chain mode + +| Port | Holder | +|------|--------| +| 7225 | **held** by pid 42278, `node …/_npx/d92c25bff7522293/node_modules/.bin/fsb-mcp-server` (parent 42179, started 2026-09-29 15:24:36). This is the same user hub as in the 66-02 baseline. It was not killed or restarted. | +| 7226 | free | + +**Mode: tolerant pass, as in 66-02 Task 1.** +- The runner was the 66-02 runner with only its output paths changed. +- `scripts.pretest`, then `scripts.test`, were split on ` && `. Each command ran through `/bin/sh -c` from the repo root, with stdin from `/dev/null`, `node_modules/.bin` first on PATH, a 20-minute per-command bound, and continuing past failures. +- The only addition is `/tmp/fsb-66-09-full.exit`. It holds the exit code `npm test` would have returned: the first non-zero command's code, or 0. It is derived from the JSONL, not a real chain exit. +- Artifacts: `/tmp/fsb-66-09-full.jsonl` (one row per command), `/tmp/fsb-66-09-full.log`, `/tmp/fsb-66-09-full.done` and `/tmp/fsb-66-09-full.exit`. None is committed. + +### Chain result and baseline comparison + +| Item | 66-02 baseline (`b4fe9d41`) | 66-09 (`b381c0ba`) | +|------|-----------------------------|--------------------| +| Commands | 311 (13 pretest + 298 test) | 312 (13 pretest + 299 test) | +| Exit 0 | 311/311 | **312/312** (13/13 pretest, 299/299 test) | +| Equivalent `npm test` exit | 0 | **0** (`/tmp/fsb-66-09-full.exit`) | +| Last suite reached | T298 `no-orphan-descriptor` (end of chain) | **T299, the bench command (end of chain):** 338 pass, 0 fail, 3.2 s | +| Failing commands | none | **none** | +| Timed out (20-minute bound) | none | none | +| Total time | 148.0 s | 207.7 s (same commands; the machine was more loaded) | + +- **Command identity:** the first 311 commands match the baseline's 311 exactly, by id (P1..P13, T1..T298) and by text. T299 is the only addition. No other commit in Phase 66 touched `package.json`. +- **Failure comparison:** the baseline's pre-existing failure set in the chain is empty, and this run has no failure. So no command is new, pre-existing or port-bound, and nothing blocks the plan. +- **Port-bound suites, with 7225 held as in the baseline:** + - T44 `mcp-diagnostics-status`: 254 passed, 0 failed (baseline 254/0). + - T45 `mcp-install-platforms`: 108 passed, 0 failed (baseline 108/0). +- **Slowest commands:** + - T103 `showcase-build-smoke`: 30.5 s + - T45 `mcp-install-platforms`: 28.9 s + - T8 `phase60-full-tests-harness`: 24.1 s + - T18 `mcp-native-host-packaging --section workflow-and-pack`: 19.4 s +- **Per-suite count check** (a sanity check, not the pass rule, which is exit codes): 146 suites print a parseable pass count. + - 136 are unchanged, and none decreased. + - The 10 increases are all Phase 66 test extensions: + + | Suite | Count | Source | + |-------|-------|--------| + | P6 `mcp-session-recorder-journal` | 8 → 12 | 66-03 | + | P7 `mcp-lattice-journal` + `export-port` | 38 → 42 | 66-03 | + | T2 `cost-tracker-ordering` | 17 → 37 | 66-04 | + | T17 `delegation-phase-contract` | 1204 → 1205 | this plan | + | T28 `mcp-bridge-background-dispatch` | 412 → 456 | 66-03 | + | T34 `mcp-in-flight-session-lookup` | 15 → 214 | 66-06 | + | T56 `action-history` | 65 → 71 | 66-04 | + | T63 `cost-tracker` | 67 → 299 | 66-04 | + | T70 `automation-logger-mcp-retention` | 82 → 141 | 66-04, 66-06 | + | T198 Google Sheets group | 60 → 62 | 66-03 Sheets redaction cases | +- **After the run:** 7225 was still held by the same pid 42278. No `mcp/build/index.js` relay from this workspace was running. + +### Run side effects (step 2) + +- **Rewritten and reverted with `git checkout --`:** `showcase/angular/public/llms.txt`, `llms-full.txt` and `sitemap.xml`. The only changes were generated date stamps (2026-09-01 → 2026-09-30): one header line in each llms file and 77 `` lines in the sitemap. +- **Not changed:** `mcp/build/`, `mcp/ai/tool-definitions.cjs`, the tracked `extension/dist/offscreen/lattice-host.js` and every `package-lock.json`. +- **Untracked files:** none were created. +- `git status --porcelain --untracked-files=no` is empty. + +### validate:extension (step 3) + +`npm run validate:extension` exits 0: all 15 validators and tests in the script, ending with `write-activation-evidence: 16 passed, 0 failed`. It ran twice, once on its own and once inside the Task 2 verify, and the tree stayed clean. + +### Node 20 run (step 4) + +- **Source:** `https://nodejs.org/dist/latest-v20.x/SHASUMS256.txt` names `node-v20.20.2-darwin-arm64.tar.gz` (`uname -m` is arm64). +- **Download:** the tarball went to `/tmp/fsb-66-09-node20/` (41,511,345 bytes). +- **SHA-256:** `466e05f3477c20dfb723054dfebffe55bc74660ee77f612166fca121dacb65b6`. `shasum -a 256 -c` against the SHASUMS256.txt line reported OK, and was checked again right before extraction. +- **Binary:** `/tmp/fsb-66-09-node20/node-v20.20.2-darwin-arm64/bin/node` reports **v20.20.2**, darwin-arm64, release `node`, LTS `Iron` (Mach-O arm64). It was not installed globally. npx node@20 and the npx-cache binary were not used. +- **Run:** the binary's bin dir went first on PATH, so any child `node` is also 20.20.2. The suites spawn children through `process.execPath` anyway. + - **All ten together:** `node --test ` gave **338/338 pass, 0 fail, exit 0** (2.3 s). + - **Each on its own:** all exit 0. The counts sum to 338. + + | Suite | Pass | + |-------|------| + | export-history | 14 | + | stats | 11 | + | outcome | 13 | + | identity | 26 | + | tasks | 76 | + | fixture-server | 29 | + | claude-stream | 46 | + | corpus | 17 | + | report | 79 | + | drivers | 27 | +- **Scope:** only the ten bench suites ran under Node 20, as the plan scopes it. The rest of the chain ran on Node 24.14.1 locally, as in the 66-02 baseline. CI runs the whole chain on Node 20. + +### Timing audits (step 5, D-04) + +| Audit | Output | Verdict | +|-------|--------|---------| +| `grep -rln "Symbol.for('fsb.mcp.call-timing')" extension` | `extension/ws/mcp-bridge-client.js`, `extension/ws/mcp-tool-dispatcher.js` (1 occurrence each). No other spelling of `fsb.mcp.call-timing` anywhere under `extension/` | exact set: PASS | +| `grep -rln "fsbMs" extension --include='*.js'` | `utils/automation-logger.js` (2), `utils/mcp-lattice-journal.js` (4), `utils/mcp-session-recorder.js` (5), `utils/spreadsheet-record-redaction.js` (2), `ws/mcp-bridge-client.js` (1), `ws/mcp-tool-dispatcher.js` (6). No non-`.js` file under `extension/` mentions it | subset of the six allowed files: PASS | +| `node tests/mcp-bridge-background-dispatch.test.js` | 456 passed, 0 failed | PASS | + +- The ingress case replaces only `_send`, so the real `_sendResult` builds the frame. +- Even with a handler that echoes the payload, it asserts the wire frame never contains `receivedAt`, `completedAt`, `fsbMs` or `call-timing`. +- Timing reaches operators only through the deliberate `get_session_detail` measurement projection (D-09), never on a tool call's own result. + +### D-01, orchestrator decision 6 and D-13 audits (step 6) + +| Audit | Output | +|-------|--------| +| `git diff --stat b4fe9d41..HEAD -- mcp/` (the 66-02 start SHA) | empty (0 bytes) | +| `git diff --stat 50334671..HEAD -- mcp/` (pre-phase; covers 66-01's commits too) | empty (0 bytes). 0 of the 50 phase commits touch `mcp/` | +| `git diff --stat b4fe9d41..HEAD -- extension/utils/analytics.js` (decision 6) | empty. Also empty from `50334671` | +| D-13: the benchmark is not in `npm test` or CI | `scripts.test` has exactly one bench entry, the unit-test command. No npm script invokes `run-bench.mjs`, `report.mjs`, `fixture-server.mjs` or `sweep-tests.mjs`. `.github/workflows/ci.yml` and `chrome-extension.yml` run only installs, `npm run validate:extension` and `npm test` | + +### CI parity (beyond the plan's list) + +These checks ask whether the newly registered command will also pass on Linux CI. + +- **Dependencies:** + - `ci.yml` uses Node `'20'` and runs `npm --prefix mcp ci`; `ws` 8.19.0 is an `mcp` production dependency. + - The chain builds `mcp/` at T13, long before T299, so `bench-drivers` has `mcp/build/agent-providers/spawn-environment.js` and `ws` when it runs. +- **Path case:** this Mac's APFS volume is case-insensitive, so a wrong-case path would pass here and fail on Linux. A static check resolved 157 path references with exact case. It covered imports, `path.join` targets, `new URL` targets, task fixture files and start paths, and fixture HTML `src`/`href`. It found **0 mismatches**. +- **Untracked files:** the only referenced file that git does not track is `mcp/build/agent-providers/spawn-environment.js`, which CI's chain builds. + +## Accomplishments + +- **CI coverage:** every Phase 66 bench unit suite now runs in CI through one registered chain command. The refinement contract pins it, and the pre-Phase-61 and pre-Phase-62 chain hashes still reproduce. +- **Phase gate:** its automated evidence is complete. + - The chain is green against the recorded baseline, with command identity checked and no suite count dropping. + - `validate:extension` is green. + - The Node 20 bench run used a verified official binary. + - The D-04 timing audits, the D-01 `mcp/` audit and the D-13 no-benchmark-in-CI audit all pass. + +## Task Commits + +1. **Task 1: Register the ten benchmark suites in the root chain and the refinement contract:** `b381c0ba` (test). Message: `test(66-09): run the benchmark unit suites in the root test chain`. +2. **Task 2: Phase integration verification:** no commit. The task is verification-only (`none`), and its results are this SUMMARY. + +**Plan metadata:** the `docs(66-09)` commit that adds this SUMMARY. + +## Files Created/Modified + +- `package.json`: one appended `scripts.test` command. +- `tests/delegation-phase-contract.test.js`: one appended `REFINEMENT_ROOT_TEST_COMMANDS` element. + +## Decisions Made + +- **Tolerant mode:** 7225 is held by the user's hub, and the rules forbid freeing it. The plan's own branch for a held port applies, and the result is judged by exit codes and the last suite reached. +- **Official Node 20 tarball:** the plan requires it, and it gives an auditable binary identity (URL, SHA-256, version). The same version (20.20.2) was already in the npx cache, but it was not used. +- **Two D-01 ranges:** `b4fe9d41` is 66-01's GREEN commit, so the plan's range starts after 66-01's first two commits. Adding `50334671` covers the whole phase. + +## Deviations from Plan + +None. The plan was executed as written. + +### Execution notes (not rule deviations) + +- **Extra evidence beyond the plan's list:** + - `/tmp/fsb-66-09-full.exit`, derived from the JSONL + - the per-suite count comparison + - per-file Node 20 runs + - the D-01 check from the pre-phase commit + - the D-13 audit + - the CI-parity checks + + None of these changed a file. +- **Branch checks:** the executor's `worktree-agent-*` allow-list was skipped, because the orchestrator directs commits on `Jev`, as in 66-02..66-08. The cwd-drift sentinel and the protected-branch deny-list ran before the commit. +- **graphify:** the graph predates `scripts/bench/` and returned unrelated nodes, as earlier plans found. Every edit anchor was read from the working tree. + +## Issues Encountered + +- **"With port 7225 free" is not met literally.** The must-have asks for a chain result with 7225 free, but the user's hub holds it and must not be disturbed. The tolerant pass is the plan's stated substitute. Both port-bound suites passed with the port held, exactly as at the baseline. The first fail-fast `npm test` with 7225 free will be CI's. +- **This chain run was slower than the baseline** (207.7 s vs 148.0 s). No command came near the bound, and every result matched. + +## Known Stubs + +None. This plan only registers existing suites and verifies them. + +## Threat Model Coverage + +- **T-66-09a (tampering, chain edits):** + - One appended command, plus the identical string in `REFINEMENT_ROOT_TEST_COMMANDS`, in one commit. + - The historical hash constants are untouched (0 changed lines), and the contract's hash-reproduction checks pass. +- **T-66-09b (tampering, Node 20 download):** + - The binary comes from the official nodejs.org dist only, and its SHA-256 was verified against SHASUMS256.txt before extraction. + - It runs from /tmp and was never installed globally. +- **T-66-05 (information disclosure, timing in responses):** the Symbol and `fsbMs` greps bound where timing lives, and the ingress test proves the wire frame carries no timing. +- **T-66-01 (accepted, /tmp logs):** + - The /tmp logs hold test output only and are not committed. + - The downloaded tarball and extracted binary remain under `/tmp/fsb-66-09-node20/`. + +No new security surface. + +## User Setup Required + +None. + +## Next Phase Readiness + +- **66-10 (live smoke):** the code under test is wired and green. + - The live paths the fakes cannot prove are unchanged from 66-08's list: `Runtime.evaluate` on an MV3 worker, `PUT /json/new`, the real `claude` stream, and lsof hub ownership. + - If a later plan wants a Node 20 binary, `/tmp/fsb-66-09-node20/node-v20.20.2-darwin-arm64/bin/node` is verified and still present. +- **66-01:** its H1 export and Task 3 remain. Its suite is already in the chain, so Task 3 needs no chain edit. +- **Blockers:** none. + +## Self-Check: PASSED + +- `package.json`, `tests/delegation-phase-contract.test.js` and this SUMMARY exist on disk. +- The task commit `b381c0ba` exists and has no Co-Authored-By or AI attribution line. +- `git diff --name-only 1098472e HEAD -- .planning/STATE.md .planning/ROADMAP.md .planning/REQUIREMENTS.md mcp/` is empty. +- The tracked working tree was clean before this SUMMARY was committed; the SUMMARY was the only untracked file. +- Re-checked counts: `validate:extension` has 15 commands, and `50334671..b381c0ba` holds 50 commits. + +--- +*Phase: 66-baseline-and-session-timing* +*Completed: 2026-09-30* diff --git a/.planning/phases/66-baseline-and-session-timing/66-10-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-10-PLAN.md new file mode 100644 index 000000000..ee2cb1899 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-10-PLAN.md @@ -0,0 +1,268 @@ +--- +phase: 66-baseline-and-session-timing +plan: 10 +type: execute +wave: 5 +depends_on: ["66-09"] +files_modified: [] +autonomous: false +requirements: [MEAS-01, MEAS-02, MEAS-03] +user_setup: + - service: xai + why: "Autopilot arm of the baseline runs on xAI grok-4.3 (D-19)" + env_vars: [] + dashboard_config: + - task: "Enter the xAI API key in the bench Chrome profile's FSB control panel (Providers), select provider xAI and model grok-4.3" + location: "Bench Chrome profile -> FSB control panel -> Providers" + - service: claude-code + why: "MCP arm of the baseline is headless Claude Code on the user's own login (D-14)" + env_vars: [] + dashboard_config: + - task: "Choose the Claude model id to benchmark (passed as --claude-model)" + location: "Reply at checkpoint H2" + +must_haves: + truths: + - "A dedicated bench Chrome profile runs the worktree's unpacked Phase 66 extension, paired to a runner-owned hub on 7225. The everyday profile's Web Store build, Claude Desktop's FSB server, the fsb MCP server of every other Claude Code/Codex/Conductor session, and any stray fsb-mcp-server or mcp/build/index.js relay are kept off the hub for the campaign window, and H5 restores them (research runtime state; Pitfall 8; W10)." + - "The bench Chrome runs without background throttling (--disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling), and its launch flags are recorded for identity (W12)." + - "No benchmark traffic reaches FSB's production telemetry or public /stats dashboard (W-B, T-66-07). The setup launch makes full-selfbrowsing.com unresolvable, so the `install_announce` that fires 30 s after Load unpacked fails and stays queued. H2 turns 'Send anonymous usage data' off (fsbTelemetryOptOut: true) before any run, the first flush after that clears the queue unsent, and the runner's preflight refuses unless the opt-out is strictly true. The bench Chrome is then relaunched without the block, because the live UAT reads need the site." + - "The unpacked extension is paired without any global native-host change, and that promise holds on every path (B1). `pair --reset --json >/dev/null` unpins the origin without printing the new code (B2), and the runner delivers the product's own `pair --json` code into the bench extension's session storage over local CDP. The code is never logged or persisted. If CDP pairing fails, the plan stops at a blocking human-action checkpoint; there is no reinstall fallback." + - "The runner's preflight passes: the Phase 66 measurement marker is present, the extension id is the unpacked one (not the Web Store id), providerKind is api, provider is xai, model is grok-4.3, a Claude model is given, and the runner owns 7225 (D-18, D-19)." + - "The smoke run covers all 12 core tasks x 1 round x 2 arms = 24 runs, so it doubles as the per-task reference pass (W3). A small live smoke (4 live tasks x 1 round x 2 arms = 8 runs, unless the human declines) checks live reachability and cost (W5). Both complete with independent evidence and saved per-run records, are labelled smoke rather than baseline, and are kept out of git (D-16, D-20, D-21)." + - "An operator can see per-call timing in a live session: get_session_detail JSON for an MCP smoke run carries per-call timing and the measurement block, and its text export shows per-call fsb ms and labelled caller gaps (MEAS-01, MEAS-02, D-09)." + artifacts: [] + key_links: + - from: "scripts/bench/run-bench.mjs --pair-via-cdp" + to: "unpacked extension chrome.storage.session fsbMcpBridgePairing" + via: "pair --json code written through CDP, then mcpBridgeClient.reloadPairingAndReconnect()" + pattern: "--pair-via-cdp" + - from: "scripts/bench/run-bench.mjs --preflight" + to: "fsb://config extension.measurement marker" + via: "evaluatePreflight refusal matrix" + pattern: "--preflight" +--- + + +**Deferred 2026-10-15 (owner direction):** this plan runs in the milestone-end Validation Block (V1), from a git worktree checked out at the baseline pin `8a058bdf`, so its `git rev-parse HEAD` is the pin and the baseline measures FSB before any Phase 66.1+ code. + +**Superseded by 71 (2026-10-16):** use the Chrome for Testing binary from `.planning/phases/71-comparative-gate-and-release/71-VALIDATION.md` M-71-01c wherever this plan launches `/Applications/Google Chrome.app` (Task 1 steps 3-4 and the Task 3 relaunch), and the Claude Code binary pinned by M-71-01 on PATH. The H4 projections use 24 × N core runs, N = `campaigns.baseline.minRounds` in `scripts/bench/gate/release-gate-v1.0.0.json` on Jev. This smoke is not a gate campaign; its raw records feed the owner sign-off M-71-00 (step 0 prints per-arm safety event rates), which is recorded before 66-11's H4 approval. See 71-VALIDATION.md "Upstream V1/V4/V6 rows reconciled". + +Stand up the live benchmark topology, prove it with the runner's preflight, and run the smoke test. The smoke doubles as the live check that operators can see per-call timing in session detail. + +Purpose: MEAS-03's live baseline needs a verified, isolated browser setup before any spend. It also covers MEAS-01/MEAS-02's live operator check (VALIDATION manual-only rows). This plan contains research human checkpoints H2 (bench profile setup) and H3 (reload the unpacked extension after the last extension commit). +Output: a working bench profile (outside the repo), a local smoke campaign and report under the gitignored `scripts/bench/.local/`, and a SUMMARY with the review packet for H4 (plan 66-11). + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md +@.planning/phases/66-baseline-and-session-timing/66-08-SUMMARY.md +@.planning/phases/66-baseline-and-session-timing/66-09-SUMMARY.md + + +Machine facts (verified 2026-09-29): +- The everyday Chrome (Profile 2) runs the Web Store FSB build `badgafnfchcihdfnjneklogedcdkmjfk`. +- `~/.fsb/bridge-auth.json` pins `allowedExtensionOrigin` to that origin. `pair --reset` clears the pin AND rotates the session secret (`resetBridgePairing`, mcp/src/bridge-auth.ts ~180-190). `pair --json` prints `{protocol, pairingCode, rotatedAt}` without rotating. Only a `serve` daemon start (prepareBridgeAuth) or `--reset` rotates. +- CORRECTION to research assumption A4: native messaging cannot pair the bench build, for two reasons. + - The installed native host validates Chrome's caller origin against its owner-marker origin (`validateNativeInvocation`, mcp/src/native-host/protocol.ts ~220-235; marker origin = the Web Store id). + - A Chrome started with `--user-data-dir` looks for user-level hosts in `/NativeMessagingHosts`, not in `~/Library/Application Support/Google/Chrome/NativeMessagingHosts` where the product installs its manifest (mcp/src/native-host-install/platform.ts ~96-104). A global reinstall would therefore change the everyday setup without helping the bench (B1). + The bench uses `run-bench.mjs --pair-via-cdp` only (plan 66-08). Do not create a native-messaging manifest in the bench profile. Never run `install --native-host` or `uninstall --native-host`, and never edit the global manifest. +- `pair --reset` without `--json` prints the new pairing code to stdout (mcp/src/index.ts ~443-451). Always run `node mcp/build/index.js pair --reset --json >/dev/null 2>"scripts/bench/.local/pair-reset.stderr"`: stdout is discarded, and the warning goes to stderr (B2). +- Selecting `grok-4.3` in the bench control panel depends on live xAI model discovery with a valid key, because grok-4.3 is not in `FALLBACK_MODELS` (extension/ai/model-discovery.js ~35-42). Adding it to the product list is a product change outside Phase 66 and must not be done (I3). +- The extension hard-codes `ws://localhost:7225`, and the hub keeps one extension socket. Claude Desktop may run an FSB MCP server that grabs 7225; the native-host serve daemon uses 7226 and rotates the secret when it starts. +- Branded Chrome ignores `--load-extension`, so the unpacked extension is loaded by hand (H2). `--remote-debugging-port=0` writes `/DevToolsActivePort`. +- Runner CLI (plan 66-08): `node scripts/bench/run-bench.mjs --preflight|--smoke|--campaign [--set core|live] [--arms autopilot,claude] --claude-model --expect-extension-id --chrome-profile [--chrome-launch-record ] [--pair-via-cdp] [--campaign-id ] [--resume ]`. `--smoke` defaults to every task in the set x 1 round. Exit codes: 0 ok, 2 refusal, 75 paused (the reason code is printed). Report: `node scripts/bench/report.mjs `. +- Rough per-run cost and time (research A8): autopilot on grok-4.3 costs about $0.05-0.40 per run; Claude runs consume subscription usage; runs take about 1.5-3 minutes each. + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + + + + + + Task 1: Build the Phase 66 extension and MCP server, create the bench profile, and launch the bench Chrome + none tracked (extension/dist, mcp/build and ~/.fsb-bench are outside version control) + + - .planning/phases/66-baseline-and-session-timing/66-09-SUMMARY.md (integration results; do not proceed if it records a blocking failure) + - .planning/phases/66-baseline-and-session-timing/66-RESEARCH.md "Runtime and Operational State Touched by the Benchmark" and "Plan Decomposition Hints / Human checkpoints" + + +1. Confirm `git status --porcelain --untracked-files=no` is empty and record `git rev-parse HEAD` as the bench build SHA. +2. Run `npm run build` (extension/dist/offscreen/lattice-host.js) and `npm --prefix mcp run build`. Restore any tracked file the build rewrote (`git checkout -- `). +3. `mkdir -p "$HOME/.fsb-bench/chrome-profile"`. Record `"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --version`. +4. Launch the bench Chrome detached for SETUP, with output going to `scripts/bench/.local/bench-chrome.log`: `"" --user-data-dir="$HOME/.fsb-bench/chrome-profile" --remote-debugging-port=0 --no-first-run --no-default-browser-check --disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling --host-resolver-rules="MAP full-selfbrowsing.com ~NOTFOUND, MAP *.full-selfbrowsing.com ~NOTFOUND" chrome://extensions`. + - The three disable flags stop Chrome from throttling hidden or background tabs, which would distort FSB's settle waits (W12). + - The host-resolver rule applies only to this setup launch (W-B). The extension's `install_announce` fires 30 s after every onInstalled (extension/background.js ~20970-20985; install, update and reload alike) and POSTs to `https://full-selfbrowsing.com/api/telemetry/events`, which is faster than a human can reach the opt-out toggle. With the host unresolvable, that POST fails and the event stays queued. The collector re-reads `fsbTelemetryOptOut` on every flush, and once it is true it clears the queue without sending (extension/utils/telemetry-collector.js ~636-652). No extension source is edited. + - Record the PID in `scripts/bench/.local/bench-chrome.pid`, and write `scripts/bench/.local/bench-chrome.json` as `{pid, version, flags:[...], flagValues:{...}, setupHostBlock:true}` (I-3): + - `flags` lists every flag above by NAME only, meaning the part before any `=`, for example `--host-resolver-rules` and `--user-data-dir`. The `chrome://extensions` URL is not a flag and is not listed. + - `flagValues` maps each valued flag that is not path-valued to its value, for example `{"--remote-debugging-port": "0", "--host-resolver-rules": "MAP full-selfbrowsing.com ~NOTFOUND, MAP *.full-selfbrowsing.com ~NOTFOUND"}`. Path-valued flags (`--user-data-dir`) have no flagValues entry. + - Task 3 rewrites the file in the same shape after the campaign relaunch. The runner reads `flags` (names) from the final file into identity `chrome.launchFlags`. +5. Wait up to 15 s for `$HOME/.fsb-bench/chrome-profile/DevToolsActivePort`, and record the port. Verify the block with a one-off CDP `Runtime.evaluate` in a fresh `about:blank` page (open it with `PUT /json/new?about:blank`, close it afterwards with `/json/close/`). Pass `awaitPromise: true` and `returnByValue: true`, the same options as the runner's `evaluate()` helper (I-1); without awaitPromise the result is an unresolved Promise object and proves nothing. Evaluate `fetch('https://full-selfbrowsing.com/', {mode:'no-cors'}).then(() => 'resolved', e => 'rejected:' + e.name)`, which must return `'rejected:TypeError'` (a network error). This is a page view, never the telemetry endpoint. If it returns `'resolved'`, the block is not active: stop with a blocking `checkpoint:human-action` offering (a) turn off the Mac's network before Load unpacked and back on only after the H2 opt-out, or (b) accept and record one install announcement. +6. Inspect the hub ports and relays (W10) and record the output in the packet: + - `lsof -nP -iTCP:7225,7226 -sTCP:LISTEN` + - `ps -axo pid,ppid,command | grep -E "fsb-mcp-server|mcp/build/index.js|fsb-native-host|native-host/runtime" | grep -v grep` +7. Print the checkpoint packet: + - the absolute path of `/extension` to load unpacked, and the bench profile path; + - the listener and relay list from step 6; + - a reminder that Claude will run `pair --reset --json >/dev/null` after H2 and again at H5, so the Web Store build re-binds; + - the smoke scope and rough cost from the H2 instructions. + + + test -f extension/dist/offscreen/lattice-host.js && test -f mcp/build/index.js && test -s "$HOME/.fsb-bench/chrome-profile/DevToolsActivePort" && kill -0 "$(cat scripts/bench/.local/bench-chrome.pid)" && node -e "const r=require('./scripts/bench/.local/bench-chrome.json');for(const f of ['--disable-backgrounding-occluded-windows','--disable-renderer-backgrounding','--disable-background-timer-throttling','--host-resolver-rules','--user-data-dir'])if(!r.flags.includes(f))process.exit(1);if(r.setupHostBlock!==true)process.exit(2);if(r.flags.some(f=>f.includes('=')))process.exit(3);if(!/~NOTFOUND/.test((r.flagValues||{})['--host-resolver-rules']||''))process.exit(4);if('--user-data-dir' in (r.flagValues||{}))process.exit(5)" + + + - The verify command exits 0 (build outputs exist, and the bench Chrome is running with remote debugging). `bench-chrome.json` lists flags by exact name with no `=` in any entry, keeps the host rule's value only in `flagValues`, and has no value for the path-valued `--user-data-dir` (I-3). + - `git status --porcelain --untracked-files=no` is empty. + - `git check-ignore -q scripts/bench/.local/bench-chrome.pid` exits 0. + + The Phase 66 build is ready, and a dedicated bench Chrome is running for the human setup step. + + + + Task 2 (H2 + H3): Load the unpacked extension in the bench profile, free the hub, configure xAI grok-4.3, and choose the Claude model + none (human action in the bench Chrome profile and the everyday environment) + + - The Task 1 checkpoint packet (extension path, profile path, port holders) + + Complete the browser and account steps that have no CLI or API: load the unpacked extension in branded Chrome, enter the xAI API key, and pick the benchmark models. Take the everyday FSB build, Claude Desktop's FSB server, the fsb MCP server of other Claude Code/Codex/Conductor sessions, and stray relays off the hub ports for the benchmark window (W10). + + Claude has built the extension and MCP server and launched a dedicated bench Chrome (profile `~/.fsb-bench/chrome-profile`) on chrome://extensions. + 1. In the bench Chrome, turn on Developer mode, click "Load unpacked", and select the `extension/` folder of this worktree (path printed above). H3: if this profile already had the unpacked extension from an earlier session, click Reload on it instead, so the loaded code is the latest commit. + 2. FIRST, before anything else (W-B): open the FSB control panel (the extension's options page). Go to "Advanced Settings" in the left sidebar, find the "Privacy & Telemetry" card, and switch OFF "Send anonymous usage data". The switch saves `fsbTelemetryOptOut: true` immediately, with no Save button. A fresh profile has telemetry on by default, and unpacked builds are not exempt, so bench runs would otherwise post model and token totals to FSB's production /stats. About the window: the install announcement fires 30 s after Load unpacked, but this setup Chrome was started with full-selfbrowsing.com unresolvable, so it fails and stays queued. Once the switch is off, the next flush discards it unsent. If Claude reported at Task 1 that the block was not active and you chose option (b), one install announcement (an install id, 'unknown' model, 0 tokens) may already have reached production; it cannot be recalled. + 3. Copy the new extension's ID (32 letters a-p). It must NOT be `badgafnfchcihdfnjneklogedcdkmjfk`. + 4. Take everything else off the hub for the benchmark window (W10). Everything returns at H5. + - In your everyday Chrome profile, disable the FSB extension (or quit everyday Chrome). + - Quit Claude Desktop (or disable its fsb MCP server). + - In every other Claude Code, Codex or Conductor session that has the fsb MCP server configured, disable that server or end the session for the campaign window. That includes the session running this phase: Conductor sessions spawn `fsb-mcp-server@latest` relays (I-E). For Claude Code, `/mcp` lets you disable `fsb`; otherwise stop that session's relay PID from the list below. The phase executor does not need FSB MCP tools, because the runner uses its own client. + - Inspect what still holds the ports or runs a relay: + `lsof -nP -iTCP:7225,7226 -sTCP:LISTEN` + `ps -axo pid,ppid,command | grep -E "fsb-mcp-server|mcp/build/index.js|fsb-native-host|native-host/runtime" | grep -v grep` + - Stop each listed PID that is not the bench Chrome with `kill ` (use `kill -9 ` only if it survives 5 s). Alternatively, reply "kill " to authorize Claude to stop exactly those PIDs. This includes the native-host serve daemon on 7226, which rotates the pairing secret if it restarts. + 5. In the bench Chrome, open the FSB control panel (extension options). Under Providers choose API, provider xAI, paste your xAI API key, and select the model `grok-4.3`. The benchmark pins grok-4.3 (D-19); preflight refuses any other model. grok-4.3 appears only through live model discovery with a valid key, because it is not in the extension's built-in fallback list. If it is missing, re-check the key and reopen the model list; do not type another model (I3). + 6. Choose the Claude model id for the MCP arm (for example the model you normally use with Claude Code). The runner has no default (orchestrator decision 3). + 7. The live set needs no login. It has 2 public Jira reads (issues.apache.org, jira.atlassian.com) and 2 UAT-style reads of FSB's production showcase (full-selfbrowsing.com/privacy and /support), which check that the deployed pages show the expected content. The UAT checks expect English pages, so keep English as the bench Chrome's first language. Reply "no live" to skip the live set; an unreachable site is recorded as skipped either way (orchestrator decision 8). + 8. What starts immediately after your reply (W5): + - a relaunch of the bench Chrome without the full-selfbrowsing.com block (the live UAT reads need the site; telemetry is already off, so nothing is sent); + - `pair --reset --json >/dev/null` (the new code is never printed; it unpins the everyday build until H5); + - CDP pairing of the unpacked extension, with no global native-host change on any path (B1); + - the preflight, which refuses unless `fsbTelemetryOptOut` is true (W-B); + - the core smoke: all 12 core tasks x 1 round x 2 arms = 24 runs. It takes roughly 40-80 minutes and costs roughly $0.60-4.80 of xAI spend for the 12 autopilot runs, plus Claude usage for 12 Claude runs; + - unless you replied "no live", the live smoke: 4 live tasks x 1 round x 2 arms = 8 runs, roughly 10-25 minutes and $0.20-1.60 of xAI spend plus 4 Claude runs. + Nothing larger runs before your H4 approval (plan 66-11). + + Claude checks that the reported id matches /^[a-p]{32}$/ and is not the Web Store id. Task 3's preflight confirms the loaded id, marker, provider and model. + + FSB66_BENCH_EXTENSION_ID="" node -e "const id=process.env.FSB66_BENCH_EXTENSION_ID||'';if(!/^[a-p]{32}$/.test(id)||id==='badgafnfchcihdfnjneklogedcdkmjfk')process.exit(1)" + + + - The human reply contains an extension id that matches /^[a-p]{32}$/ and is not `badgafnfchcihdfnjneklogedcdkmjfk`, plus a Claude model id, and confirms 'Send anonymous usage data' is off (W-B). + - The human confirms the everyday FSB extension is disabled or quit, and that Claude Desktop's FSB server, the fsb MCP server of every other Claude Code/Codex/Conductor session, stray relays and the 7226 serve daemon are stopped (or authorizes Claude to stop specific PIDs). + + The unpacked Phase 66 extension is loaded in the bench profile, the xAI key and grok-4.3 are set, the hub holders are stopped, and the Claude model is chosen. + Reply: "loaded ; telemetry off; claude model ; everyday FSB off; other fsb MCP servers off (including this session); relays stopped" (add "kill " to authorize stopping listed PIDs, and "no live" to skip the live set). + + + + Task 3: Pair the unpacked extension, run the preflight and the smoke campaign, and verify the operator timing view live + none tracked (smoke output under gitignored scripts/bench/.local/) + + - scripts/bench/run-bench.mjs --help output (including --pair-via-cdp), scripts/bench/lib/cdp.mjs pairExtensionViaCdp, scripts/bench/report.mjs + + +1. Set `FSB66_BENCH_EXTENSION_ID` and `FSB66_CLAUDE_MODEL` from the H2 reply and use them in every command below. Record them in `scripts/bench/.local/bench-session.json` as `{"extensionId": , "claudeModel": , "live": }`, which plan 66-11 reads. +2. If the H2 reply authorized "kill ", stop exactly those PIDs (SIGTERM, then SIGKILL after 5 s) and nothing else. Then confirm `lsof -nP -iTCP:7225,7226 -sTCP:LISTEN` is empty and the relay `ps` check from Task 1 lists nothing. If anything remains, stop with a blocking `checkpoint:human-action` that lists the holders, and never kill unlisted user processes. Record in `scripts/bench/.local/restore-notes.txt` which sessions and services the human stopped, so H5 can restore them. +2b. Relaunch the bench Chrome for the campaign (W-B): + - Stop the PID in `scripts/bench/.local/bench-chrome.pid`, and relaunch with exactly the Task 1 flags minus `--host-resolver-rules` and minus the `chrome://extensions` URL. + - Rewrite `bench-chrome.json` in the Task 1 shape (I-3) with the new pid, `flags` by name without `--host-resolver-rules`, `flagValues` without a `--host-resolver-rules` key, and `setupHostBlock: false`. + - Wait for DevToolsActivePort. + - Confirm through the runner's `readExtensionStorageFlag` (via the preflight in step 4) that `fsbTelemetryOptOut` is still true. + The relaunch runs onStartup, not onInstalled, so no new install announcement is scheduled. Pairing happens after this relaunch, because session storage is cleared on browser restart. +3. Reset pairing without printing the code (B2): `node mcp/build/index.js pair --reset --json >/dev/null 2>scripts/bench/.local/pair-reset.stderr; echo "exit=$?"`. Confirm the reset without exposing the code: `node mcp/build/index.js pair --json | node -e "let d='';process.stdin.on('data',c=>d+=c).on('end',()=>{const j=JSON.parse(d);console.log('rotatedAt',j.rotatedAt)})"` prints only rotatedAt. Record in restore-notes.txt that H5 must run the same reset command so the Web Store build re-binds. +4. Preflight with pairing: `node scripts/bench/run-bench.mjs --preflight --pair-via-cdp --arms autopilot,claude --claude-model "$FSB66_CLAUDE_MODEL" --expect-extension-id "$FSB66_BENCH_EXTENSION_ID" --chrome-profile "$HOME/.fsb-bench/chrome-profile"`. It must exit 0. Retry once after 30 s if pairing times out. + - If CDP pairing still fails, STOP with a blocking `checkpoint:human-action` (B1). Report the runner's pairing reason code and offer only these options: + (a) reload the unpacked extension in the bench Chrome, open its control panel once to wake the service worker, then reply "retry"; + (b) confirm the bench Chrome was launched from Task 1 (remote debugging on, the DevToolsActivePort file present) and that the extension id in the reply is correct; + (c) stop the phase here and record the failure as a gap. + Never offer or run a native-host install or uninstall, and never touch the global manifest. + - For any other refusal code (marker, id, provider, model, hub), stop with a blocking `checkpoint:human-action` that reports the code. +5. Core smoke, which is also the per-task reference pass (W3): `node scripts/bench/run-bench.mjs --smoke --pair-via-cdp --arms autopilot,claude --claude-model "$FSB66_CLAUDE_MODEL" --expect-extension-id "$FSB66_BENCH_EXTENSION_ID" --chrome-profile "$HOME/.fsb-bench/chrome-profile" --campaign-id smoke-`. That is all 12 core tasks x 1 round x 2 arms = 24 runs. Then run `node scripts/bench/report.mjs scripts/bench/.local/smoke-`. + - Unless `bench-session.json` has `live: false`, also run the live smoke (W5): add `--set live --campaign-id smoke-live-`, giving 4 live tasks x 1 round x 2 arms = 8 runs, and then its report. Record unreachable live tasks as skipped. + - On exit 75 (an `agent_cap`, `agent_auth` or `provider_error` pause), stop with a blocking `checkpoint:human-action` that gives the reason code and the exact `--resume` command. + - A preflight refusal `telemetry_not_opted_out` stops with a blocking `checkpoint:human-action`: ask the human to switch "Send anonymous usage data" off in the bench control panel, then rerun the preflight (W-B). +6. Live operator check (MEAS-01/MEAS-02, D-09): + - From the smoke's raw records, take one Claude-arm run's MCP session id. With a one-off Node script that uses scripts/bench/lib/mcp-client.mjs (open the runner client, pair via `--pair-via-cdp` semantics if needed), call `get_session_detail` (JSON) and `get_session_detail` with `format:'text'`. + - Confirm the JSON has `session.measurement.kind === 'mcp'` with `callsWithTiming >= 1`, `queueWaitStatus === 'not_measured'` and `host.status === 'unavailable'`, and actionHistory rows with `timing`. Confirm the text contains `| fsb ` and the timing legend. + - Confirm one autopilot run's detail has `measurement.kind === 'autopilot'`, `recorded === true`, and servedModels matching /^grok-4\.3/. + - Save both responses to `scripts/bench/.local/smoke-/operator-view/`. +7. Write the H4 review packet into the SUMMARY: + - preflight identity (extension id and version, marker, provider and model, server and SDK version, tools/list count and sha256, Claude version, model and tool-loading mode, Claude env key-name sha256, Chrome version and launch flags); + - a per-task table for all 12 core tasks x both arms (W3): pass or fail, wall ms, the failed check kinds, and infra codes. A task that fails in BOTH arms is flagged "possible fixture defect: inspect raw evidence before approving"; + - per-arm smoke completion, wall times, cost so far with labels, infra errors and flags; + - live smoke results per task (pass, fail or skipped-unreachable); + - the operator-view check results; + - projections, each labelled as an extrapolation: full core campaign (per-run medians x 240) and live campaign (x 80) duration and cost (W5). + Do not commit smoke outputs. + + + node scripts/bench/run-bench.mjs --preflight --pair-via-cdp --arms autopilot,claude --claude-model "$FSB66_CLAUDE_MODEL" --expect-extension-id "$FSB66_BENCH_EXTENSION_ID" --chrome-profile "$HOME/.fsb-bench/chrome-profile" && node -e "const fs=require('fs'),p=require('path');const b='scripts/bench/.local';const d=fs.readdirSync(b).filter(x=>/^smoke-2/.test(x)).sort().pop();const ids=new Set(fs.readFileSync(p.join(b,d,'runs.jsonl'),'utf8').trim().split('\n').filter(Boolean).map(l=>JSON.parse(l).runId));if(ids.size<24||!fs.existsSync(p.join(b,d,'summary.json')))process.exit(1);console.log('distinct runIds',ids.size)" && ! grep -q "fsb-auth\." scripts/bench/.local/pair-reset.stderr + + + - The preflight exits 0 with the unpacked extension id, the measurement marker, providerKind api, provider xai, model grok-4.3 and runner-owned 7225. + - The core smoke's runs.jsonl holds at least 24 DISTINCT runIds, all 12 tasks x 2 arms (I-D). Count runIds, not files under runs/, because archived `*.attemptN.json` files would inflate a file count. A smoke summary.json exists labelled 'smoke (not a baseline)', the live smoke exists unless declined, and `git ls-files scripts/bench/.local` is empty. + - The preflight summary shows `telemetryOptOut: true`, and `bench-chrome.json` shows the campaign relaunch with `--host-resolver-rules` absent from both `flags` and `flagValues` (`setupHostBlock: false`) (W-B, I-3). + - No pairing code was captured (B2): the reset ran with stdout redirected to /dev/null, `grep -c "fsb-auth\." scripts/bench/.local/pair-reset.stderr` prints 0, and the SUMMARY records the exact reset command. + - The operator-view files show MCP measurement with timed calls and the text-export timing lines, plus an autopilot measurement with recorded:true and a grok-4.3 served model. Any mismatch is reported as a gap (a gap-closure item), not waived. + - `git diff --quiet HEAD -- mcp/` exits 0. The global native-host manifest is unchanged on every path: its allowed_origins is still exactly `chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk/` (B1). + - The SUMMARY contains the H4 review packet. + + The live topology is verified by preflight, the all-task core smoke and the live smoke have run, per-call timing is proven visible to an operator in a real session, and the H4 packet (with per-task results and projections) is ready. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Bench profile → xAI API | A real API key is entered by the human and used by autopilot. | +| Global pairing state | pair --reset changes machine-wide FSB pairing; there is no native-host change on any path. | +| Runner → bench extension over CDP | The runner writes the pairing record into the bench extension's session storage. | +| Live public sites | Third-party pages are read during live tasks. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-01 | Information disclosure | xAI key / bridge pairing code | mitigate | The key is entered only in the bench profile UI by the human, and is never passed on a command line, written to bench artifacts or logged. fsb://config exposes no keys (66-03). The pairing code is read through `pair --json` inside the runner process and written only into the bench extension's `chrome.storage.session` over local CDP; it is never printed or persisted (66-08 source check). | +| T-66-01b | Elevation of privilege | pair --reset | mitigate | The only machine-wide change is the origin pin plus secret rotation from `pair --reset --json >/dev/null`, recorded in restore-notes.txt and reversed at H5. There is no native-host install or uninstall on any path (B1); a CDP pairing failure stops at a human-action checkpoint. | +| T-66-01e | Information disclosure | pair --reset output | mitigate | The reset always runs with stdout discarded and stderr captured to a file checked for the absence of `fsb-auth.`. The follow-up confirmation prints only `rotatedAt` (B2). | +| T-66-01f | Denial of service | stray relays and other sessions | mitigate | Listeners and relays are inspected with lsof and ps, including this phase's own session relay (I-E). The human stops them or authorizes exact PIDs, and they are restored at H5 (W10). | +| T-66-07 | Tampering (integrity of production data) | benchmark telemetry pollutes production stats | mitigate | The setup launch makes full-selfbrowsing.com unresolvable (verified by a page fetch), so the 30 s install announcement fails and stays queued. H2 switches telemetry off first, the collector's live opt-out check then discards the queue unsent, and the runner's preflight refuses unless `fsbTelemetryOptOut` is true (W-B). Residual risk, stated honestly: the block applies to the extension's service-worker fetches through Chrome's shared network stack, which Task 1 verifies only for a page fetch. If the block were bypassed, or the human chose option (b), at most one install announcement (install id, 'unknown' model, 0 tokens) could reach production; no bench run's model or token totals can. | +| T-66-02 | Information disclosure | smoke outputs | mitigate | Everything goes to gitignored `scripts/bench/.local/`, and nothing from smoke is committed. | +| T-66-01c | Spoofing | hub on 7225 | mitigate | The preflight requires the runner's own server PID to own 7225. Holders are stopped by the human rather than killed by automation, and the Web Store id is refused. | + + + +- The preflight exits 0. The core smoke produced 24+ saved runs and a smoke-labelled summary, plus the live smoke unless declined. The operator-view check is recorded. No pairing code was printed. +- `git status --porcelain --untracked-files=no` is empty, and `git ls-files scripts/bench/.local` is empty. + + + +- The live benchmark topology is proven and the smoke test is complete (D-16 smoke before campaign). +- The MEAS-01/MEAS-02 live operator view is verified in a real session. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). This plan commits nothing except its SUMMARY. Never stage scripts/bench/.local/ or anything outside the repo. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-10-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-11-PLAN.md b/.planning/phases/66-baseline-and-session-timing/66-11-PLAN.md new file mode 100644 index 000000000..2b9c8367a --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-11-PLAN.md @@ -0,0 +1,222 @@ +--- +phase: 66-baseline-and-session-timing +plan: 11 +type: execute +wave: 6 +depends_on: ["66-10"] +files_modified: + - scripts/bench/results/core-baseline-66/identity.json + - scripts/bench/results/core-baseline-66/runs.jsonl + - scripts/bench/results/core-baseline-66/summary.json + - scripts/bench/results/core-baseline-66/report.md + - scripts/bench/results/live-baseline-66/identity.json + - scripts/bench/results/live-baseline-66/runs.jsonl + - scripts/bench/results/live-baseline-66/summary.json + - scripts/bench/results/live-baseline-66/report.md +autonomous: false +requirements: [MEAS-03] +user_setup: [] + +must_haves: + truths: + - "The full baseline campaign (12 core tasks x 10 ABBA rounds x 2 arms, about 240 runs) runs only after the human approves the smoke results and the projected spend, and before any Jev path exists (D-16, roadmap success criterion 3)." + - "Completion rate, wall time per completed task (runner clock) and labelled cost per completed task are saved as the committed baseline, with enough identity to compare later runs. Agent claims never count as success (D-06, D-17, D-18)." + - "The live reading set is reported separately and never folded into the core baseline. It is optional when the sites are unreachable or the human declines (D-15, orchestrator decision 8)." + - "Only sanitized per-run rows, identity and aggregates are committed. Raw records, streams and page text stay under the gitignored .local tree (D-21)." + - "After the campaign, the everyday setup is restored: the bench Chrome is stopped, and pairing is reset with `pair --reset --json >/dev/null` (the code is never printed) so the Web Store build re-binds. The human re-enables everyday FSB, Claude Desktop, and the fsb MCP servers of the other Claude Code/Codex/Conductor sessions stopped at H2 (H5, W10). No native-host change was ever made, so none needs reversing (B1)." + artifacts: + - path: "scripts/bench/results/core-baseline-66/summary.json" + provides: "Core baseline aggregates (per arm completion + Wilson CI, wall per completed task, cost per completed task, A/A noise floor)" + - path: "scripts/bench/results/core-baseline-66/report.md" + provides: "Human-readable baseline report with method notes and labels" + - path: "scripts/bench/results/core-baseline-66/identity.json" + provides: "Run identity for comparison in Phases 69 and 71" + - path: "scripts/bench/results/core-baseline-66/runs.jsonl" + provides: "Sanitized per-run rows (RUN ROW CONTRACT)" + key_links: + - from: "scripts/bench/results/core-baseline-66/runs.jsonl" + to: "scripts/bench/report.mjs" + via: "validateCommittedRow on load; the report refuses invalid rows" + pattern: "runs.jsonl" +--- + + +**Deferred 2026-10-15 (owner direction):** this plan runs in the milestone-end Validation Block (V1), from a git worktree checked out at the baseline pin `8a058bdf`, so its `git rev-parse HEAD` is the pin and the baseline measures FSB before any Phase 66.1+ code. + +**Superseded by 71 (2026-10-16):** Phase 71's predeclared release gate replaces this plan's round count, run count, browser, host binary and approval order. Before H4 the V1 operator reads `scripts/bench/gate/release-gate-v1.0.0.json` on the Jev branch (`campaigns.baseline.minRounds` = N) and these rows of `.planning/phases/71-comparative-gate-and-release/71-VALIDATION.md`: M-71-00 (owner sign-off on the frozen gate, recorded after the 66-10 smoke and before the H4 approval), M-71-01 (pinned Claude Code on PATH), M-71-01b (run the core campaign with `--rounds N`, so runs.jsonl holds 24 × N rows, not 240; a resume cannot change the round count, and a 10-round baseline leaves the non-Jev verdict `inconclusive`), M-71-01c (Chrome for Testing instead of `/Applications/Google Chrome.app`, as in 66-10), M-71-02 and M-71-02b (cherry-pick the results onto Jev and derive `safety.jsonl`), M-71-03 (identity cross-check, including `schedule.rounds` = N), and its section "Upstream V1/V4/V6 rows reconciled". Where this plan says 10 rounds, about 240 runs or no gate sign-off, those rows win. The live set is not a gate input and keeps `--rounds 10`. + +With the human's approval, run the full baseline campaign, produce and commit the sanitized baseline, run the live set separately, and restore the everyday environment. + +Purpose: MEAS-03's saved baseline of completion rate, wall time per completed task and cost is captured before fast mode exists (roadmap success criterion 3). It is the reference Phases 69 and 71 compare against. This plan contains research human checkpoints H4 (campaign approval) and H5 (restore). +Output: `scripts/bench/results/core-baseline-66/` (and `live-baseline-66/` when run) with identity.json, runs.jsonl, summary.json and report.md. The everyday setup is restored. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md +@.planning/phases/66-baseline-and-session-timing/66-10-SUMMARY.md +@.planning/phases/66-baseline-and-session-timing/66-01-SUMMARY.md + +Citations of the form "orchestrator decision N" refer to 66-RESEARCH.md → "Open Questions (RESOLVED)" → "Orchestrator decisions recorded for the plans (2026-09-30)". + + +- `scripts/bench/.local/bench-session.json` (written by 66-10) holds `{extensionId, claudeModel, live}`. `scripts/bench/.local/restore-notes.txt` lists what to restore at H5: the pairing reset, plus the sessions, servers and relays the human stopped at H2. It never lists a native-host change, because none is ever made (B1). +- Runner: `node scripts/bench/run-bench.mjs --campaign --pair-via-cdp --arms autopilot,claude --claude-model --expect-extension-id --chrome-profile "$HOME/.fsb-bench/chrome-profile" --campaign-id core-baseline-66 [--resume core-baseline-66]`. Exit 75 means paused (for example a Claude usage window or two consecutive infra errors) and prints the resume command. Live: add `--set live --campaign-id live-baseline-66`. +- Report: `node scripts/bench/report.mjs scripts/bench/results/` writes summary.json and report.md, and refuses invalid rows. + + + + + + + Task 1 (H4): Review the smoke results and approve the full baseline campaign and its spend + none (human approval) + + - .planning/phases/66-baseline-and-session-timing/66-10-SUMMARY.md (the H4 review packet) + + Approve, adjust or stop the roughly 240-run campaign. It spends the user's xAI budget and Claude usage over several hours, so it needs explicit human authorization; this checkpoint type is used because auto-advance must not approve spend. + + Claude prints the H4 packet from the 66-10 SUMMARY: + - the per-task table for all 12 core tasks x both arms, from the one-round core smoke (W3). Tasks that failed in both arms are marked as possible fixture defects; + - smoke completion per arm, with wall times; + - live smoke results per task (pass, fail or skipped-unreachable), unless declined; + - identity (extension id and version, marker, grok-4.3 served model, Claude version, model and tool-loading mode, env key-name hash, Chrome launch flags, tools/list count and hash, and `telemetryOptOut: true`, W-B); + - the operator-view check; + - projected duration and cost, each labelled as an extrapolation: the core campaign (240 runs) and the live campaign (80 runs) (W5). + 1. Review the packet. If anything looks wrong (for example a task marked as a possible fixture defect, or a task the smoke failed for a fixture reason rather than an agent reason), reply "fix: ". Claude will then stop and open a gap-closure item instead of running the campaign. + 2. H3: if any extension code changed since the smoke, reload the unpacked extension in the bench profile first. + 3. Keep the machine awake and plugged in. The campaign can span Claude usage windows. When the runner pauses (exit 75), Claude asks you when to resume. + 4. Confirm the Claude model (same as smoke unless you change it) and whether to run the live set afterwards. + + Claude reruns the preflight with the confirmed model and id before starting (the same command as 66-10 Task 3 step 4). It must exit 0, which includes the `telemetry_not_opted_out` check (W-B). + + node -e "const s=require('./scripts/bench/.local/bench-session.json'); if(!/^[a-p]{32}$/.test(s.extensionId||'')||!s.claudeModel)process.exit(1)" + + + - The human replies "approve campaign" with the Claude model and a live yes or no, or replies "fix: ..." (which halts this plan with a recorded gap). + - The preflight exits 0 after approval. + + The human has approved the campaign, its model and its spend. + Reply "approve campaign; claude model ; live yes|no" or "fix: ". + + + + Task 2: Run the core campaign (resumable) and the live set, build the reports, commit the sanitized baseline, and restore automatable state + scripts/bench/results/core-baseline-66/identity.json, scripts/bench/results/core-baseline-66/runs.jsonl, scripts/bench/results/core-baseline-66/summary.json, scripts/bench/results/core-baseline-66/report.md, scripts/bench/results/live-baseline-66/identity.json, scripts/bench/results/live-baseline-66/runs.jsonl, scripts/bench/results/live-baseline-66/summary.json, scripts/bench/results/live-baseline-66/report.md + + - scripts/bench/.local/bench-session.json and scripts/bench/.local/restore-notes.txt + - scripts/bench/report.mjs (validation and output) + - .planning/phases/66-baseline-and-session-timing/66-01-SUMMARY.md (records that 66-01 was closed without a history export, so there is no history comparison) + + +1. Start the core campaign in the background under `caffeinate -i` (macOS keep-awake), logging to `scripts/bench/.local/core-baseline-66/runner.log`: `caffeinate -i node scripts/bench/run-bench.mjs --campaign --pair-via-cdp --arms autopilot,claude --claude-model --expect-extension-id --chrome-profile "$HOME/.fsb-bench/chrome-profile" --campaign-id core-baseline-66`. +2. Poll `scripts/bench/.local/core-baseline-66/progress.json` every few minutes, and report progress lines (runs done out of 240, infra errors). + - On exit 75, stop with a blocking `checkpoint:human-action` that gives the printed reason code and the exact resume command. Example codes: `agent_cap` during a Claude usage window, `agent_auth` when Claude needs a re-login, or `provider_error` for an xAI rate limit, quota or key failure, including the silent uncapped-429 case detected as a timeout whose session detail shows no recorded provider response (W-A). A timeout whose session detail cannot be found is a task failure flagged `session_detail_missing` and never pauses (B-TIMEOUT). After the human says to continue, rerun with `--resume core-baseline-66`; resume re-runs every runId whose latest attempt ended in an infra error (W4). + - On exit 2 with `telemetry_not_opted_out`, stop with a blocking `checkpoint:human-action` asking the human to switch "Send anonymous usage data" off in the bench control panel (Advanced Settings → Privacy & Telemetry). Never start or resume a campaign with telemetry on (W-B, T-66-07). + - On any other non-zero exit, stop and report; do not fabricate results. +3. When 240 rows exist (or the human ends the campaign early, recorded as such), run `node scripts/bench/report.mjs scripts/bench/results/core-baseline-66`. +4. Unless the human said "live no": run `--set live --campaign-id live-baseline-66 --rounds 10` the same way, then its report. If the live sites are unreachable (every run an infra error or a blocked page), record that in the SUMMARY and skip committing live results (optional per orchestrator decision 8). +5. Sanitization gate before committing: + - `node scripts/bench/report.mjs` must have succeeded, since it validates every row; + - `grep -rl "://" scripts/bench/results/core-baseline-66 scripts/bench/results/live-baseline-66` prints nothing; + - `grep -rliE "apikey|sessionSecret|pairingCode|authorization" scripts/bench/results/*-baseline-66` prints nothing; + - `git check-ignore -q scripts/bench/.local/core-baseline-66/runs` exits 0. +6. Commit only the `scripts/bench/results/*-baseline-66/` files: `docs(66-11): commit the Phase 66 benchmark baseline`. No Co-Authored-By or AI attribution. Never stage .local. +7. Automatable restore (the H5 prerequisites): + - kill the bench Chrome PID in scripts/bench/.local/bench-chrome.pid; + - reset pairing without printing the code (B2): `node mcp/build/index.js pair --reset --json >/dev/null 2>scripts/bench/.local/pair-reset-restore.stderr`, so the next authenticated extension (the Web Store build) re-binds. Confirm with the rotatedAt-only pipe used in 66-10 (`pair --json` piped into a node one-liner that prints only rotatedAt); + - confirm, read-only, that the global native-host manifest was never changed: its allowed_origins is still exactly `chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk/` (B1). Run no install or uninstall command; + - print the restore list from restore-notes.txt (sessions, servers and relays stopped at H2) for the H5 checkpoint. +8. SUMMARY: + - per-arm completion rate with its 95% CI; + - wall time per completed task (geometric mean of per-task medians) and per-task medians; + - cost per completed task with source labels; + - A/A noise floor width; + - Claude host metrics (duration_api share, turns, list-price label); + - FSB per-call timing medians, caller gap (upper bound) and call interval (`callIntervalMs`, W6); + - identity digest (git SHA, extension id and version, tools/list count and hash, served model); + - the live set, reported separately; + - the live MCP-arm `callIntervalMs` median and p90 (W6). There is no 66-01 history to compare them with: those sessions were lost before H1 (see 66-01-SUMMARY.md); + - the early-stop reason, if any. + + + node scripts/bench/report.mjs scripts/bench/results/core-baseline-66 && node -e "const fs=require('fs');const rows=fs.readFileSync('scripts/bench/results/core-baseline-66/runs.jsonl','utf8').trim().split('\n');const s=require('./scripts/bench/results/core-baseline-66/summary.json');if(rows.length<1||!s)process.exit(1);console.log('rows',rows.length)" && test -z "$(grep -rl '://' scripts/bench/results/core-baseline-66)" && git check-ignore -q scripts/bench/.local/core-baseline-66/runs + + + - scripts/bench/results/core-baseline-66/ holds identity.json, runs.jsonl, summary.json and report.md. The report exits 0, which means every row validated. + - runs.jsonl has 240 rows, or the SUMMARY records the human-approved early stop and the actual count. summary.json labels the campaign 'baseline' when rounds reached 10. + - No committed result file contains `://` or a secret-like key. `git ls-files scripts/bench/.local` is empty. + - The bench Chrome is stopped. The pairing reset ran with stdout discarded, and `grep -c "fsb-auth\." scripts/bench/.local/pair-reset-restore.stderr` prints 0 (B2). The global native-host manifest still allows exactly the Web Store origin and no install or uninstall ran (B1). + - `git log -1 --format=%B` has no Co-Authored-By line. + + The Phase 66 baseline is saved and committed in sanitized form, and machine state is restored as far as automation can go. + + + + Task 3 (H5): Restore the everyday FSB setup + none (human action in the everyday Chrome profile and Claude Desktop) + + - scripts/bench/.local/restore-notes.txt (the pairing reset Claude already ran, and the sessions, servers and relays stopped at H2) + + Re-enable the everyday FSB extension, Claude Desktop, and the fsb MCP servers of the other sessions stopped at H2, so the Web Store build re-pairs with the reset bridge (W10). Re-enabling an extension and restarting desktop apps have no CLI. + + Claude has stopped the bench Chrome and reset the bridge pairing without printing the code. No native-host change was made at any point (B1). + 1. In your everyday Chrome profile, re-enable the FSB extension (or reopen Chrome). Open its side panel once so it re-bootstraps its pairing through the native host. + 2. Restart Claude Desktop (or re-enable its fsb MCP server). + 3. Re-enable the fsb MCP server in each Claude Code, Codex or Conductor session you disabled at H2 (W10), and restart anything from the restore list Claude printed (for example the native-host serve daemon starts again on the next native wake). + 4. Optional: delete `~/.fsb-bench/chrome-profile` if you do not plan to rerun the benchmark. Phase 71 reuses the same setup, so keeping it saves time. + + Claude runs `fsb-mcp-server status --json` and confirms the extension is connected with the Web Store origin (or reports exactly what is still disconnected). + + fsb-mcp-server status --json --timeout 5000 + + + - `fsb-mcp-server status --json` reports the extension connected, and the doctor or status output shows the Web Store extension origin. Any remaining issue is recorded verbatim in the SUMMARY. + + The everyday FSB setup works again with the Web Store build. + Reply "restored", or describe what is not working. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Campaign → paid APIs | The campaign spends xAI budget and Claude usage. | +| Raw run records → git | Committed baseline files must carry no page content or secrets. | +| Machine-wide FSB state | Pairing and the sessions stopped at H2 must be restored; no native-host state is ever changed. | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-66-02 | Information disclosure | scripts/bench/results/*-baseline-66 | mitigate | The report validates every row against the exact RUN ROW CONTRACT, and grep gates reject `://` and secret-like keys. Only the results directories are staged; raw data stays in gitignored .local. | +| T-66-01 | Information disclosure | identity.json | mitigate | Identity holds ids, versions, digests, model and provider names only (66-08). The secret-key grep gate runs before commit. | +| T-66-SPEND | Denial of service (financial) | campaign | mitigate | The human-action approval (H4) is required, with a projected cost. The runner pauses (exit 75) on consecutive infra errors, and resume needs the human's go-ahead. | +| T-66-07 | Tampering (integrity of production data) | campaign telemetry | mitigate | Every campaign start and resume passes the runner preflight, which refuses unless the bench extension's `fsbTelemetryOptOut` is true. identity.json records the value (W-B). | +| T-66-01b | Elevation of privilege | restore | mitigate | `pair --reset --json >/dev/null` runs automatically (the code is never printed, B2), the untouched global manifest is verified read-only (B1), and H5 restores the stopped sessions and confirms the everyday connection through `status --json` (W10). | + + + +- The report builds from the committed rows, and the sanitization gates pass. +- The everyday setup is restored and verified by `fsb-mcp-server status --json`. + + + +- Roadmap Phase 66 success criteria 3 and 4 are met: a reproducible, outcome-checked suite saved completion, wall-time and cost baselines before fast mode, with enough identity to compare later runs. +- D-15, D-16, D-17, D-18 and D-21 hold in the committed artifacts. + + + +Never add a `Co-Authored-By` line or any AI/Claude attribution to commit messages (the user's global rule). Commit only `scripts/bench/results/*-baseline-66/`. Never stage scripts/bench/.local/. + + + +Create `.planning/phases/66-baseline-and-session-timing/66-11-SUMMARY.md` when done. + diff --git a/.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md b/.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md new file mode 100644 index 000000000..52bb14341 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-CONTEXT.md @@ -0,0 +1,261 @@ +# Phase 66: Baseline and Session Timing - Context + +**Gathered:** 2026-09-29 (assumptions mode) +**Status:** Ready for planning + + +## Phase Boundary + +Establish trustworthy timing and task-outcome records before any Jev path changes execution (MEAS-01, MEAS-02, MEAS-03). + +In scope: +- per-call timing for MCP and autopilot sessions, with honest "unavailable" labels +- a working `get_session_detail` for journal-backed MCP sessions +- a repeatable, outcome-checked benchmark suite with a saved baseline and enough run identity to compare later runs + +Out of scope: +- any change to how FSB decides or executes (no Jev, no description edits, no `expect`) +- control-panel UI changes +- the MCP settings tool (now Phase 66.1) + + + +## Implementation Decisions + +### Change footprint and compatibility +- **D-01:** Keep the MCP diff minimal: Phase 66 changes no source file under `mcp/`. The extension does the work (timing capture, the session-detail fix, run identity, cost labels), and the benchmark lives in `scripts/bench/`. `fsb://config` already passes the extension's payload through unchanged (`mcp/src/resources/index.ts:138-162`), so identity fields are an extension-only edit. +- **D-02:** Every Phase 66 change is additive and does not change behavior: + - Autopilot decisions, action execution, tool names, schemas, descriptions and response shapes stay as they are, apart from added optional fields. + - The one intended behavior change: `get_session_detail` returns journal sessions instead of "not found". + - Milestone-wide rule from the user: with Jev disabled, FSB behaves exactly as it does today. +- **D-03:** No MCP-server-side timestamps in this phase. Time spent in the MCP `TaskQueue` is labelled "not measured" and is never quietly folded into another figure. Caller gap is the extension's receive time for call n minus its completion time for call n−1; both are measured on the same machine. + +### Timing capture and storage +- **D-04:** MCP per-call timing is captured in the extension, where calls arrive (`_handleMessage`, `extension/ws/mcp-bridge-client.js` ~1144-1199). + - It travels to both recording paths on a hidden symbol property, the same pattern as `MCP_REPLAY_RECORD_CONTEXT` (`extension/ws/mcp-tool-dispatcher.js` ~160). + - It is stored, with a completion time, as an added optional `metadata.timing` field on the journal `tool.call` event. + - It is never added to tool responses and never written to `chrome.storage.local` on each call. + - Do not compute call time from journal event timestamps: they include IndexedDB write-queue delay. + - Do not put timing on the `beginCall` identity: `tests/mcp-session-recorder-journal.test.js:92` deep-equals it. +- **D-05:** Autopilot records, for each iteration, model time (from `apiCallStartTime`, `extension/ai/agent-loop.js` ~1936) and each tool's time (the span between the `BEFORE_TOOL_EXECUTION` and `AFTER_TOOL_EXECUTION` hooks). Both must survive `automationLogger.saveSession` and the `sanitizeActionHistoryEntry` allowlist. A value that was not measured is null or omitted, never 0. Today `createActionEvent.durationMs` defaults to 0 and is dropped on save. +- **D-06:** The headline metric is wall time per completed task, measured by the benchmark runner's own clock. Session `endTime` is never the metric: + - `closeIdleRun` (60 s idle alarm, `mcp-lattice-journal.js` ~1530) sets `endTime` late. + - `saveSession` resets `endTime` on every save. + - Runs closed with `idle_timeout` are flagged, and their idle tail is excluded. + +### Session detail and operator surface +- **D-07:** `handleGetSessionMessageRoute` gets a journal-first branch, mirroring the control-panel lookup at `extension/background.js` ~12496-12536. Lookup order: + 1. `globalThis.FsbMcpLatticeJournal`: `hasSession`, then `getSessionDetail` + 2. the legacy `loadSession` + 3. in-flight sessions + + The branch is skipped when the journal module is missing, which `tests/mcp-in-flight-session-lookup.test.js` requires. `format:'text'` goes to the journal's `exportHumanReadable`. +- **D-08:** The response keeps the existing `{success:true, session:{...}}` envelope. + - Journal `tool.call` events become a sanitized `session.actionHistory` with timing fields added. The control panel already does this conversion (`extension/ui/options.js` ~3444-3476). + - Raw journal events, which carry the full request and result payloads, are never returned. + - `get_session_detail` gains no new parameters: MCP-04 allows only `expect`. This overrides the research suggestion of `afterSequence`/`limit`. +- **D-09:** Operators see timing in: + - `get_session_detail` JSON + - its text export, which gains per-call milliseconds + - the benchmark report + + The control-panel session viewer does not change. (User: 5a.) + +### Tokens and cost +- **D-10:** Autopilot cost comes from the provider when it reports one. For xAI that is `usage.cost_in_usd_ticks` (10^10 ticks = $1; when streaming, only the final chunk has it). + - The price-table figure becomes a labelled fallback estimate. Each iteration records `usageSource`/`costSource`. + - Session and baseline records mark values "unavailable" or "estimate" instead of inventing them at these three places: + - `usage.input || 0` (`agent-loop.js` ~2007) + - unknown models priced as Grok (`cost-tracker.js` ~85-88) + - `totalCost || 0` (`automation-logger.js` ~884) + - Reasoning tokens are included in cost estimates. They are reported separately from `completion_tokens` (`completion_tokens_details.reasoning_tokens`) and billed at the output rate. +- **D-11:** Update the Grok rows of the `extension/ai/cost-tracker.js` price table. xAI retired `grok-4-1-fast` on 2026-08-15, and that name is now served by grok-4.3 at $1.25 input / $2.50 output per million tokens. The table still says $0.20 / $0.50. (User: 4a.) +- **D-12:** For MCP sessions, FSB marks the calling host's model time, tokens and cost as unavailable. Caller gap is labelled an upper bound on the host's thinking time. Host metrics come only from the benchmark driver: + - **Claude Code:** the `result` event supplies `duration_ms`, `duration_api_ms`, `num_turns`, `usage` and `modelUsage`. `total_cost_usd` is labelled "list-price estimate, not billed" when `apiKeySource` is `none` or `costBasis` is `list`. + - **Cursor:** reports duration only; tokens and cost are "unavailable (host does not report)". + - v0.9.91 made the same call for agent dollar cost (`usd = null`, `extension/utils/delegation-event-store.js` ~569). + +### Benchmark suite +- **D-13:** The benchmark is plain Node scripts in a new `scripts/bench/`: a runner, a report, statistics helpers, and a fixture server on 127.0.0.1 with local fixture pages. + - It runs only on request, never in `npm test` or CI. + - It adds no dependencies. The MCP client comes from `mcp/node_modules`, with the SDK kept at 1.29.0. + - Precedent for a fixture server plus Chrome as an opt-in test: `tests/screenshot-uat.test.js`. +- **D-14:** How each arm is driven: + - **MCP arm:** scripted headless Claude Code, launched from `scripts/bench/` with the pinned v0.9.91 flags (`mcp/src/agent-providers/claude-profile.ts` ~179-207). Nothing under `mcp/` changes (D-01). + - **Optional Cursor arm:** scripted with `cursor-agent -p --output-format stream-json`. + - **Autopilot arm:** `run_task` from an SDK stdio client, with an API-kind provider selected. An agent-kind provider returns a consent response before the autopilot loop runs (`background.js` ~13639-13647). +- **D-15:** Tasks: a core set on local fixture pages stays unchanged through Phase 71, and only this set is used for the release comparison. A small live-site set (UAT- and Jira-style reading) is reported separately and does not count toward the release gate. (User: 2, recommended.) +- **D-16:** Size: about 12 core tasks (6 mixed, 6 click-heavy) × 10 alternating runs per arm × 2 baseline arms (autopilot, Claude Code MCP), roughly 240 runs. The baseline must collect enough for Phase 71's method: + - Headline: the geometric mean of per-task median ratios, with a hierarchical bootstrap CI (resample tasks, then runs within each task). + - Per-task CIs are descriptive only. + - A CI that crosses 1.0 is reported as "inconclusive". + - Five runs per task is a smoke test only. + + (User: 3, recommended.) +- **D-17:** Success is decided by code, from evidence independent of both FSB and the agent: + - Fixture tasks: the fixture server's log of submitted requests, the final URL, and DOM markers. + - Live tasks: URL and text checks after the timed window. + - Agent claims are stored as "claimed" only and never count as proof: autopilot `end_turn`/DONE, `complete_task`, and `run_task` success, including `partial_outcome:'timeout'`. +- **D-18:** Each run records its identity: + - git SHA + - extension manifest version and the ID of the extension actually loaded + - MCP server version and SDK version + - provider kind, requested model, and the model string the provider returned + - host name, version and model + - host tool-loading mode. For Claude Code: whether `init.tools` contains `ToolSearch` or `WaitForMcpServers`, plus the `ENABLE_TOOL_SEARCH` value it was launched with. + - task ID and version, and a hash of the fixture content + - the MCP tool count and a hash of the `tools/list` payload, read by the runner through its own SDK client. The tool list changes in Phases 66.1 and 69, so later comparisons must account for it. + + In the extension, fix `_handleGetConfig` (`mcp-bridge-client.js` ~2702). It currently reads keys that nothing writes. It should return the keys that exist (`providerKind`/`modelProvider`/`modelName`, `extension/config/config.js` ~18-21) plus the manifest version. The change is read-only, and `fsb://config` is a resource, not a tool. +- **D-19:** The autopilot baseline pins an explicit current xAI model ID instead of the retired `grok-4-1-fast` name, and records the model string xAI returns. Changing FSB's product default model is not part of this phase. (User: 4a.) +- **D-20:** The runner saves each run's detail to disk as soon as the run ends. Before any new recording, it exports the 40 recorded MCP sessions from 2026-09-16..21. History keeps 50 sessions per mode and the journal keeps 30 days, so those sessions would be pruned around 2026-10-16..21, sooner as benchmark runs push them out. + - *Amended 2026-10-12:* the export half of D-20 is superseded. The Web Store build was reinstalled on 2026-10-09, which wiped the 40 sessions before the export ran, and no backup exists, so 66-01 was closed without executing it (see 66-01-SUMMARY.md). The save-each-run rule still applies. +- **D-21:** Raw session exports stay local and out of git, because they can contain page content from real sites. Only sanitized per-run metrics, identity and aggregates are committed as the baseline. + +### Claude's Discretion +- Autopilot timing capture: a new hook module (e.g. `extension/ai/hooks/timing-hook.js`) or inline stamps. Weigh the tripwires: + - A new module raises the pinned `importScripts` count (333, `tests/lattice-provider-bridge-smoke.test.js:634-635`). + - Editing `agent-loop.js` touches pinned counts such as the 8 `setTimeout` tokens and fixed anchor text. +- The exact xAI model ID to pin (a current grok-4.3 variant), confirmed during research. +- Where the committed baseline artifacts live (`scripts/bench/results/` or the phase directory), within D-21. +- The exact fixture tasks, the internal layout of the benchmark scripts, and the report format. +- Whether the text export shows per-call timing as columns or inline. + +### Folded Todos +None. No pending todos matched Phase 66. + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Scope and requirements +- `.planning/ROADMAP.md` — Phase 66 goal and success criteria, and the inserted Phase 66.1. This file has the authoritative phase numbering. +- `.planning/REQUIREMENTS.md` — MEAS-01..03, and MCP-04 (the only new parameter allowed is `expect`). +- `.planning/PROJECT.md` — v1.0.0 milestone section, the measured 40-session baseline, and facts about the MCP tool surface. +- `.planning/STATE.md` — current position and the invariants carried forward (INV-01, INV-04, tripwire discipline). + +### Research (v1.0.0; uses an older 8-phase numbering, so match sections by content, not phase number) +- `.planning/research/SUMMARY.md` — "Phase 66 — Measurement floor" section; Pitfalls 1, 2, 11, 15. +- `.planning/research/ARCHITECTURE.md` — per-call timing at extension boundaries, the journal-branch fix, Anti-Pattern 7 (no timing in responses), and the tripwire register. +- `.planning/research/PITFALLS.md` — Pitfalls 1-3 and 17; the "Looks Done But Isn't" checklist. +- `.planning/research/STACK.md` — the "Per-Call Timing" and "Benchmark and Measurement Tooling" tiers. +- `.planning/research/FEATURES.md` — "Benchmark and measurement"; the automated caller-side driver. +- `.planning/research/JEV-REFERENCE.md` — background only; Phase 66 makes no Jev calls. + +### Codebase maps +- `.planning/codebase/TESTING.md` — test runner, the `npm test` chain, and suites that bind ports. +- `.planning/codebase/CONVENTIONS.md` — classic-script modules and the `module.exports` guard. + +### External (verified 2026-09-29) +- xAI model retirement: https://docs.x.ai/developers/migration/may-15-retirement +- xAI cost tracking (`cost_in_usd_ticks`): https://docs.x.ai/developers/cost-tracking +- xAI pricing: https://docs.x.ai/developers/pricing +- Claude Code costs and the headless `result` event: https://code.claude.com/docs/en/costs +- Claude Code MCP tool search: https://code.claude.com/docs/en/mcp +- Cursor CLI output format: https://cursor.com/docs/cli/reference/output-format + + + +## Existing Code Insights + +### Reusable Assets +- Journal API in `extension/utils/mcp-lattice-journal.js`: + - `hasSession` + - `getSessionDetail` (up to 500 events; ~1631-1648) + - `exportHumanReadable` (~2008-2045) + - `streamSessionExport` (~2103) + - module exports (~2199-2240) +- Control-panel lookup that checks the journal first and converts results to the legacy shape: `extension/background.js` ~12496-12536 and `extension/ui/options.js` ~3444-3486. +- Sanitizers `sanitizeSessionMetadata`, `sanitizeActionHistoryEntry` and `sanitizeSessionDetail`: `extension/ws/mcp-tool-dispatcher.js` ~2064-2116. The hidden-symbol context pattern is at ~160. +- Autopilot hook points in `extension/ai/agent-loop.js`: + - `apiCallStartTime` (~1936) + - `AFTER_API_RESPONSE` (~2055) + - `BEFORE_TOOL_EXECUTION` (~2260) and `AFTER_TOOL_EXECUTION` (~2816) + - `saveToLogger` (~1457-1465) + + Also `createActionEvent.durationMs` in `extension/ai/action-history.js` ~38-53. +- Existing examples of metrics that are labelled and may be null: `McpPricingResult` (`mcp/src/tools/pricing.ts` ~68-74) and `_projectMetrics` (`extension/utils/delegation-event-store.js` ~551-580). +- Timing data already recorded: + - `change_report.settle_ms` (dispatcher ~3891) + - the `run_task` heartbeat's `elapsed_ms` and `FsbMcpTaskStore.started_at` (`mcp-bridge-client.js` ~2384-2565) + - `automationLogger.logTiming` (`automation-logger.js` ~557); most callers pass `sessionId: null` +- Building blocks for the benchmark driver: + - the Claude Code launch profile (`mcp/src/agent-providers/claude-profile.ts` ~179-207) + - the stream parser `claude-stream.ts`; its `ResultSchema` uses `.passthrough()` + - the fixture harness precedent in `tests/screenshot-uat.test.js` + - the in-process server `createRuntime` in `mcp/src/runtime.ts` + + The fixture `tests/fixtures/agent-streams/claude-code-2.1.177` is synthetic, and its `result` line has no duration or cost fields. + +### Established Patterns +- Wire changes are additive only. Recording sidecars are stripped by `INTERNAL_PAYLOAD_KEYS` (`mcp-lattice-journal.js` ~44-48) and checked by `RECORDING_SIDECAR_FIELDS` (`tests/mcp-tool-smoke.test.js:58`). +- Recording never blocks routing: it is fire-and-forget inside try/catch. +- Service-worker modules are classic `importScripts` scripts with a `module.exports` guard. +- Missing values are null, never 0. History keeps 50 sessions per mode; the journal keeps 30 days. +- Source-pin tripwires: every commit that touches the extension updates the tests that pin it in the same commit. The full suite stays green from the phase's first commit. + +### Integration Points +- Bridge client, `mcp-bridge-client.js`: + - `_handleMessage`, where calls arrive (~1144-1199) + - `_recordMcpSessionAction` (~1689-1743) + - `_handleGetConfig` (~2702-2707) +- Dispatcher: the message-route recorder (~829-851) and `handleGetSessionMessageRoute` (~2904-2957). +- Recorder, `extension/utils/mcp-session-recorder.js`: + - `_snapshotJournalEntry` (~918-966) + - the legacy `actionHistory.push` (~2146-2152) + - the `run_task` skip (~2226); `run_task` is never journaled +- Journal: + - `recordDispatchNow` metadata (~1210-1229) + - `buildIndexEntry` (~836-872) + - `closeIdleRun` (~1530-1564) +- Logger: the `saveSession` projections (`automation-logger.js` ~904-916 and ~963-974; index ~982-1012). +- Usage and cost: + - agent loop, `agent-loop.js` ~2005-2029 + - usage extraction, `extension/ai/tool-use-adapter.js` ~312-337 + - reasoning tokens, `universal-provider.js` ~676 +- Tests pinned to these files: + - `tests/mcp-in-flight-session-lookup.test.js` + - `tests/mcp-session-recorder-journal.test.js` + - `tests/mcp-tool-smoke.test.js` + - `tests/action-history.test.js:67` + - `tests/turn-result.test.js:50` + - `tests/lattice-provider-bridge-smoke.test.js:634-635` + - `tests/lattice-survivability-smoke.test.js:371-372` + - `tests/mcp-philosophy-parity-smoke.test.js` + - `tests/mcp-lattice-journal.test.js` + - `tests/cost-tracker.test.js` + + + +## Specific Ideas + +- User direction on the MCP arm: "minimal diff (as minimal changes to MCP as possible) (extension does most of the heavy lifting)". Recorded as D-01. A scripted host still drives the MCP arm from `scripts/bench/`, which touches nothing under `mcp/`. +- User direction for the milestone: with Jev disabled, all of FSB must stay backwards compatible (D-02). +- Measured facts the baseline should reproduce: + - Median MCP wall time is 9.5 s per call with Claude and 4.6 s with Cursor. (The 40 source sessions were lost on 2026-10-09, so these two figures can no longer be re-derived; see the D-20 amendment.) + - FSB's own execution takes about 0.3-1 s per call. + - Caller turns are 85-90% of MCP time. + - No autopilot sessions have been recorded yet. + + + +## Deferred Ideas + +- **MCP settings tool.** Shows all control-panel settings compactly and sets any one of them, including the Jev fast-mode toggle once Phase 68 adds it. + - Reads hide secrets. + - Writes may change any setting except secrets and credentials; the user chose this over an allowlist. + - The user approved this new tool on 2026-09-29, and it is inserted as **Phase 66.1**. MCP-04 and the out-of-scope entry against a second user-facing fast-mode switch are amended to match. +- **FSB's default autopilot model has been retired.** Requests for `grok-4-1-fast` are now served by grok-4.3, at about 5-6× the cost FSB assumes. Updating the product default is a separate fix; Phase 66 only pins the model the benchmark uses (D-19). +- **MCP-server-side timestamps** (`serverReceivedAt`, measured `TaskQueue` wait). Add these later only if runs with concurrent agents need queue wait separated from caller gap (D-03). +- **Per-call timing columns in the control-panel session viewer.** Not planned (D-09). + +### Reviewed Todos (not folded) +None. + + +--- + +*Phase: 66-baseline-and-session-timing* +*Context gathered: 2026-09-29* diff --git a/.planning/phases/66-baseline-and-session-timing/66-DISCUSSION-LOG.md b/.planning/phases/66-baseline-and-session-timing/66-DISCUSSION-LOG.md new file mode 100644 index 000000000..1659a3cb5 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-DISCUSSION-LOG.md @@ -0,0 +1,80 @@ +# Phase 66: Baseline and Session Timing - Discussion Log (Assumptions Mode) + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are recorded in CONTEXT.md; this log preserves the analysis behind them. + +**Date:** 2026-09-29 +**Phase:** 66-baseline-and-session-timing +**Mode:** assumptions +**Areas analyzed:** Timing capture and storage; Session detail and operator surface; Tokens and cost; Benchmark suite, outcome proof, and run identity + +## Assumptions Presented + +### Timing capture and storage +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| MCP timing is captured where calls enter the extension and stored as `tool.call` metadata, never in responses | Likely | `mcp-bridge-client.js` 1144-1199; `mcp-lattice-journal.js` 1169, 1385; `tests/mcp-session-recorder-journal.test.js:92` | +| No server-side timestamps in Phase 66; queue wait is labelled "not measured" | Unclear | `mcp/src/queue.ts` 64-77; `mcp/src/agent-bridge.ts` 71-104; `tests/mcp-tool-smoke.test.js:58` | +| Autopilot records model and tool time per iteration; missing values are null, not 0 | Likely | `agent-loop.js` 1936, 2260, 2816; `action-history.js:51`; `automation-logger.js` 909-915 | +| Wall time per completed task comes from the runner's clock; `idle_timeout` runs are flagged | Confident | `mcp-lattice-journal.js` 1530-1537; `automation-logger.js` 874, 931 | + +### Session detail and operator surface +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| `get_session_detail` checks the journal first, guarded when the journal is absent | Confident | dispatcher 2904-2957; `background.js` 12496-12536; `tests/mcp-in-flight-session-lookup.test.js` | +| Same response envelope, with a sanitized `actionHistory` projection and no new parameters | Likely | REQUIREMENTS MCP-04; `options.js` 3444-3476; journal 1620-1647 | +| Operators see timing in MCP session detail, the text export and the bench report; no UI change | Unclear | journal text export 2019-2045; `options.js` 4225-4255 | + +### Tokens and cost +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| Autopilot tokens and cost are labelled by source; no more invented zeros or Grok prices | Likely | `agent-loop.js` 2007-2008; `cost-tracker.js` 85-88; `automation-logger.js` 884-886; `pricing.ts` 68-74 | +| FSB marks the host's model time, tokens and cost as unavailable; host metrics come only from the bench driver | Likely | `mcp-metrics-recorder.js` 70-84; `claude-stream.ts` 115-120; `delegation-event-store.js` 569-570 | + +### Benchmark suite, outcome proof, and run identity +| Assumption | Confidence | Evidence | +|------------|-----------|----------| +| Plain Node scripts in `scripts/bench/`, run on request only, with no dependencies | Confident | `tests/screenshot-uat.test.js`; `mcp/package.json:57` | +| The autopilot arm runs `run_task` over an SDK stdio client; the MCP arm scripts headless Claude Code | Unclear | `background.js` 13639-13647; `mcp-session-recorder.js:2226`; `claude-profile.ts` 179-207 | +| Success is checked in code; agent claims are stored as "claimed" | Likely | `agent-loop.js` 2091-2095; `agent-bridge.ts` 123-134; `mcp-bridge-client.js` 2534-2540 | +| Each run records its identity; `fsb://config` returns the real config keys plus the manifest version | Likely | `mcp-bridge-client.js` 2702-2707; `config.js` 18-21; `ws-client.js:1547` | +| Each run is saved immediately; the 40 recorded sessions are exported first | Likely | `automation-logger.js` 20-28; journal 28-29 | + +## Corrections Made + +### MCP arm and change footprint (question 1) +- **Original assumption:** Scripted headless Claude Code drives the MCP arm, with an optional scripted Cursor arm. +- **User correction:** "minimal diff (as minimal changes to MCP as possible) (extension does most of the heavy lifting)". +- **Recorded as:** D-01: no `mcp/` source changes in Phase 66; the extension does the work. The scripted Claude Code driver stays, because it lives in `scripts/bench/` and touches nothing under `mcp/` (D-14). + +### Task source (question 2) +- **User choice:** Recommended. A core set of local fixture tasks stays unchanged through Phase 71; a small live-site set is reported separately. + +### Suite size (question 3) +- **User choice:** Recommended. About 12 tasks × 10 alternating runs × 2 baseline arms. + +### Autopilot baseline model (question 4) +- **User choice:** Recommended. Pin an explicit current xAI model ID, record the model string xAI returns, and fix the Grok rows of the price table. + +### Operator surface (question 5) +- **User choice:** 5a. Timing appears in `get_session_detail`, its text export and the bench report; the control panel is unchanged. + +### Added by the user +- **Settings MCP tool:** a new tool to view all FSB control-panel settings and set a single one efficiently, including the Jev toggle. + - **Placement:** a new phase right after 66 (recommended option chosen). + - **Write scope:** "everything except secrets". The user chose this over the recommended allowlist that also excluded safety settings. +- **Backwards compatibility:** with Jev disabled, all of FSB must stay backwards compatible (D-02). + +## External Research + +- **Claude Code 2.1.283 `result` event.** It carries `duration_ms`, `duration_api_ms`, `total_cost_usd`, `usage`, `modelUsage`, `num_turns` and `ttft_ms`. + - On a subscription login `costBasis` is `list`, so the cost is a list-price estimate, not a bill. + - Whether tool search is on shows in `init.tools`: it lists `ToolSearch` when on and `WaitForMcpServers` when off. + - Sources: a local run; https://code.claude.com/docs/en/costs; https://code.claude.com/docs/en/mcp +- **xAI pricing and model.** `grok-4-1-fast` was retired on 2026-08-15 and is now served by grok-4.3. + - grok-4.3 costs $1.25 / $2.50 per million tokens; FSB's price table still says $0.20 / $0.50. + - Responses carry `usage.cost_in_usd_ticks`. + - Reasoning tokens are reported separately from `completion_tokens`. + - Sources: https://docs.x.ai/developers/migration/may-15-retirement; https://docs.x.ai/developers/cost-tracking; https://docs.x.ai/developers/pricing +- **Cursor.** `cursor-agent` 2026.09.08 runs headless (`-p --output-format stream-json`), but its result reports only duration: no tokens, cost or model. Source: https://cursor.com/docs/cli/reference/output-format +- **Runs per arm.** A simulated bootstrap power check shows 5 runs per task is only a smoke test. The practical minimum is 10 alternating runs, combined across tasks with a hierarchical bootstrap. A confidence interval that crosses 1.0 counts as "inconclusive". diff --git a/.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md b/.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md new file mode 100644 index 000000000..ae8ed520d --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-PATTERNS.md @@ -0,0 +1,887 @@ +# Phase 66: Baseline and Session Timing - Pattern Map + +**Mapped:** 2026-09-29 +**Files analyzed:** 41 (8 extension sources modified, 8 tests modified, 2 config files modified, 15 new bench scripts or data trees, 6 new bench tests, 2 new test fixtures) +**Analogs found:** 36 / 41 (5 have only partial or no analog; see "No Analog Found") +**Line numbers:** verified against the working tree on 2026-09-29 (commit `55789b51`). graphify's graph is stale for these files (for example, it places `handleGetSessionMessageRoute` at L2278; it is at L2904). Trust the numbers below. + +Nothing under `mcp/` is modified (D-01). The `mcp/` files cited below are read-only references for the bench driver. + +--- + +## File Classification + +### Extension sources (modified; every one has source-pin tripwires, see "Tripwire Register") + +| File | Role | Data Flow | Closest Analog | Match | +|------|------|-----------|----------------|-------| +| `extension/ws/mcp-bridge-client.js` | middleware (MCP ingress, action tap, config resource) | request-response | `extension/ws/mcp-tool-dispatcher.js:157-160, 3184-3197` (`MCP_REPLAY_RECORD_CONTEXT`); `extension/background.js:1726-1739` (`fsbReadAuthoritativeProviderConfig`) | exact | +| `extension/ws/mcp-tool-dispatcher.js` | controller (message routes and sanitizers) | request-response | `extension/background.js:12496-12536` (journal-first lookup); `extension/ui/options.js:3444-3476` (journal-to-legacy conversion); its own sanitizers `:2027-2116` | exact | +| `extension/utils/mcp-session-recorder.js` | service (recorder, fire-and-forget queue) | event-driven | its own `_snapshotJournalEntry` `:918-966` / `recordAction` `:2336-2352` | exact | +| `extension/utils/mcp-lattice-journal.js` | store (IndexedDB journal) | file-I/O | its own `recordDispatchNow` metadata `:1210-1229`; text exports `:2019-2045`, `:2121-2153` | exact | +| `extension/ai/agent-loop.js` | service (autopilot iterator) | event-driven | its own `apiCallStartTime` `:1936` and action-push `:2797-2812` | exact | +| `extension/ai/cost-tracker.js` | utility (pure pricing) | transform | `mcp/src/tools/pricing.ts:68-74, 152-158, 246-275` (`McpPricingResult`); `extension/utils/delegation-event-store.js:551-581` (`_projectMetrics`) | role-match | +| `extension/utils/automation-logger.js` | store (session history) | CRUD | its own `saveSession` projections `:904-916`, `:963-973` | exact | +| `extension/ai/action-history.js` (optional) | model (event factory) | transform | itself `:38-53` | exact | + +### Tests and config (modified) + +| File | Role | Data Flow | Closest Analog | Match | +|------|------|-----------|----------------|-------| +| `tests/mcp-bridge-background-dispatch.test.js` | test (vm harness) | request-response | its own `runRecordingLeaseLifecycleCase` `:399-434` | exact | +| `tests/mcp-in-flight-session-lookup.test.js` | test (vm harness) | request-response | its own `buildHarness` `:34-103` + `runGetSessionDetailFallbackCase` `:105-135` | exact | +| `tests/mcp-session-recorder-journal.test.js` | test (`node:test` + journal stub) | event-driven | its own `makeJournalStub` `:27-72`, `journalEntry` `:131-151` | exact | +| `tests/mcp-lattice-journal.test.js` | test (`node:test`, memory backend) | file-I/O | its own `setup`/`entry`/`allDetail` `:30-96`; text parity `:417-440` | exact | +| `tests/cost-tracker.test.js` | test (plain node) | transform | itself (price pins must change with D-11) | exact | +| `tests/action-history.test.js` (only if the default changes) | test | transform | itself `:67` | exact | +| `tests/automation-logger-mcp-retention.test.js` | test (vm harness) | CRUD | its own `loadLogger` `:60-73` + first case `:108-141` | exact | +| `tests/delegation-phase-contract.test.js` | test (chain contract) | config | its own `REFINEMENT_ROOT_TEST_COMMANDS` `:129-155` | exact | +| `package.json` (`scripts.test`) | config | config | tail of the 298-command chain; ordering rule in `tests/mcp-client-identity-integration.test.js:469-486` | exact | +| `.gitignore` | config | config | existing commented sections (e.g. `extension/dist/` block) | exact | + +### Benchmark (new, `scripts/bench/`) + +| File | Role | Data Flow | Closest Analog | Match | +|------|------|-----------|----------------|-------| +| `scripts/bench/run-bench.mjs` | CLI runner | batch | `scripts/run-phase65-full-tests.mjs:169-287` (bounded spawn, signals, exit codes) | role-match | +| `scripts/bench/report.mjs` | CLI report | transform | `scripts/report-t1-readiness.mjs:14-24, 1241-1280` (pure exports + `writeReport` + direct-run guard) | exact | +| `scripts/bench/export-history.mjs` | CLI transform | file-I/O | `scripts/report-t1-readiness.mjs` (same shape); input format from `extension/ui/options.js:3478-3487` paging | role-match | +| `scripts/bench/fixture-server.mjs` | server | request-response | `tests/screenshot-uat.test.js:63-85` (`startFixtureServer`, path-traversal guard) | exact | +| `scripts/bench/lib/stats.mjs` | utility (pure) | transform | module style only: `scripts/lib/t1-port-contract.mjs:1-60`; math from RESEARCH Pattern 9 | partial | +| `scripts/bench/lib/outcome.mjs` | utility (pure) | transform | `scripts/lib/t1-port-contract.mjs` (validator returning check lists) | role-match | +| `scripts/bench/lib/tasks.mjs` | utility (load + validate) | file-I/O | `scripts/lib/t1-port-contract.mjs` validators; manifest style `tests/fixtures/agent-streams/claude-code-2.1.177/manifest.json` | role-match | +| `scripts/bench/lib/identity.mjs` | utility (hashing) | transform | `extension/utils/mcp-lattice-journal.js:110-130` (`stableJson`); `scripts/run-phase60-full-tests.mjs:51-62` (`captureGitBuffer`) | role-match | +| `scripts/bench/lib/claude-arm.mjs` | driver | streaming | `mcp/src/agent-providers/claude-profile.ts:93-105, 192-211`; `mcp/src/agent-providers/claude-stream.ts:353-400` | exact | +| `scripts/bench/lib/cursor-arm.mjs` (optional) | driver | streaming | same NDJSON loop as `claude-stream.ts:353-400` | partial | +| `scripts/bench/lib/mcp-client.mjs` | client | request-response | `tests/write-activation-evidence.test.js:7-10` (createRequire from a package.json); RESEARCH "Runner" code example | partial | +| `scripts/bench/lib/cdp.mjs` | client | request-response | `tests/screenshot-uat.test.js:50-61, 87-135, 229-234`; `ws` resolution `tests/mcp-bridge-topology.test.js:10-11` | exact | +| `scripts/bench/tasks/{core,live}/*.json` | config (versioned task data) | config | `tests/fixtures/agent-streams/claude-code-2.1.177/manifest.json` (versioned manifest + provenance) | partial | +| `scripts/bench/fixtures//**` | static fixture pages | request-response | `tests/fixtures/screenshot-uat.html` (`dataset.fixtureReady` marker at `:119`) | role-match | +| `scripts/bench/results/**` | committed data | batch output | none | none | + +### Benchmark tests and fixtures (new) + +| File | Role | Data Flow | Closest Analog | Match | +|------|------|-----------|----------------|-------| +| `tests/bench-stats.test.mjs` | test (ESM, `node:test`) | transform | `tests/dashboard-localization-fallback.test.mjs:1-6` (ESM header); `tests/mcp-lattice-journal.test.js:1-7` (`node:test` + `assert/strict`) | role-match | +| `tests/bench-outcome.test.mjs` | test | transform | same | role-match | +| `tests/bench-tasks.test.mjs` | test | file-I/O | same | role-match | +| `tests/bench-identity.test.mjs` | test | transform | same | role-match | +| `tests/bench-claude-stream.test.mjs` | test | streaming | same + source-text parity idea from `scripts/verify-agent-provider-flags.mjs:36-52` | role-match | +| `tests/bench-fixture-server.test.mjs` | test (local http) | request-response | `tests/screenshot-uat.test.js:63-85, 214-216, 379-384` | exact | +| `tests/fixtures/bench/claude-stream-result.jsonl` (new dir; NOT under `agent-streams/`) | test fixture | streaming | `tests/fixtures/agent-streams/claude-code-2.1.177/contract-stream.jsonl` | exact | +| `tests/fixtures/bench/xai-chat-completion.json` (or `.js`) | test fixture | transform | `tests/fixtures/model-discovery-responses.js:1-28` (synthetic, documented provider shapes) | role-match | + +--- + +## Pattern Assignments + +### Track A: MCP call timing capture and run identity + +#### `extension/ws/mcp-bridge-client.js` (middleware, request-response) + +**Analog 1: the hidden-symbol recorder context.** Declaration, `extension/ws/mcp-tool-dispatcher.js:157-160`: +```js +// Non-enumerable, in-process-only recorder context. A Symbol prevents an MCP +// caller from spoofing the resolved tab/origin metadata and keeps it out of +// the bridge response JSON. +const MCP_REPLAY_RECORD_CONTEXT = Symbol('fsb-mcp-replay-record-context'); +``` +Attachment, `extension/ws/mcp-tool-dispatcher.js:3184-3197`: +```js +Object.defineProperty(payload, MCP_REPLAY_RECORD_CONTEXT, { + value: { tabId, targetUrl: ..., targetOrigin: origin, slug: ..., /* ... */ routeFamily: 'capabilities' }, + enumerable: false, + configurable: true +}); +``` +Differences for Phase 66: use `Symbol.for('fsb.mcp.call-timing')`, not `Symbol(...)`, so the two SW files and the vm harnesses share one symbol. Give the top-level `const` a **different identifier in each file** (for example `MCP_BRIDGE_CALL_TIMING` here and `MCP_DISPATCHER_CALL_TIMING` in the dispatcher). Both files are classic scripts in one global lexical scope (`background.js:81` loads the dispatcher, `:122` loads the bridge client), so a duplicate `const` throws on the second load. All proposed names are unused today (grep checked). + +**Where the stamp goes: `_handleMessage`, `:1144-1199`.** Insert right after `:1164-1165`: +```js +const { id, type, payload } = msg; +if (!id || !type) return; +``` +It must come before the `recordingIdentity` block (`:1167-1187`). Do **not** put timing on `recordingIdentity`: `tests/mcp-session-recorder-journal.test.js:92` and `tests/mcp-bridge-background-dispatch.test.js:430` deep-equal it. Follow the ingress's existing "never gate routing" guard, `:1182-1187`: +```js +if (recordingIdentity && recorder && typeof recorder.beginCall === 'function') { + try { + Promise.resolve(recorder.beginCall(recordingIdentity, payload.recordingLeaseMs)) + .catch(function () { /* recording never gates routing */ }); + } catch (_error) { /* recording never gates routing */ } +} +``` + +**Payload identity is preserved to both recording paths (verified).** +- `_routeMessage` passes the same `payload` object to `dispatchMcpMessageRoute({ type, payload, ... })` (`:1255`), to `dispatchMcpToolRoute({ ..., payload })` for `mcp:task-status` (`:1266-1281`), and to `_handleExecuteAction(payload)` (`:1283-1284`). +- `_handleExecuteAction` calls `this._recordMcpSessionAction(payload, ...)` at `:1802`, `:1831` and `:1933`, with no spread in between. + +**Where timing is read: `_recordMcpSessionAction`, `:1689-1743`.** Add `timing: (payload)` to the object literal at `:1712-1726`: +```js +let sessionRecordEntry = { + client: (typeof globalThis.resolveMcpClientLabel === 'function') + ? globalThis.resolveMcpClientLabel(payload) + : null, + tool: payload && payload.tool, + // ... + replayContext +}; +``` +Stamp `completedAt` here, synchronously, before the fire-and-forget enqueue. Omit the key (or pass `null`) when no stamp exists. Never write `0`. + +**Sheets calls (new finding).** With the redactor loaded (production), `FsbSpreadsheetRecordRedaction.recordSafely` returns non-Sheets entries unchanged. It **rebuilds** Google Sheets entries through a `baseFields` allowlist (`extension/utils/spreadsheet-record-redaction.js:246-255`: `client, tool, success, tabId, dispatcher_route`), which drops `timing`. Choose one: +- extend `baseFields` with a validated optional `timing`, and add a case to `tests/spreadsheet-record-redaction.test.js`, which deep-equals only params and response; +- or accept "timing not recorded" for Sheets calls, and state that in the plan. + +**`_handleGetConfig` fix, `:2702-2708`.** Current code: +```js +async _handleGetConfig() { + const config = await chrome.storage.local.get([ + 'selectedModel', 'selectedProvider', 'defaultModel', + 'maxIterations', 'domOptimization', 'maxDOMElements', + ]); + return { config }; +} +``` +Analog for defaults, `extension/background.js:1726-1739`. Copy the same defaults so `fsb://config` and autopilot start agree: +```js +async function fsbReadAuthoritativeProviderConfig() { + const stored = await chrome.storage.local.get([ + 'providerKind', 'agentProviderId', 'modelProvider', 'modelName', 'lmstudioBaseUrl' + ]); + return { + providerKind: typeof stored.providerKind === 'string' ? stored.providerKind : 'api', + agentProviderId: typeof stored.agentProviderId === 'string' ? stored.agentProviderId : '', + modelProvider: typeof stored.modelProvider === 'string' ? stored.modelProvider : 'xai', + modelName: typeof stored.modelName === 'string' ? stored.modelName.trim() : '', + // ... + }; +} +``` +The replacement shape is in RESEARCH Pattern 6 (`{ config, extension: { id, version, measurement: { callTiming: 1, sessionDetailJournal: 1 } } }`). +- The vm harness chrome mock has `runtime.id` but **no `getManifest`** (`tests/mcp-bridge-background-dispatch.test.js:208-223`), so guard it with `typeof`. +- `mcp/src/diagnostics.ts:1067` calls `normalizeConfig(payload?.config ?? payload)`, so a top-level `extension` sibling is harmless. +- `fsb://config` passes the payload through unchanged (`mcp/src/resources/index.ts:148-161`). + +#### `extension/utils/mcp-session-recorder.js` (service, event-driven) + +The whole file is an IIFE, so helper names do not leak into the SW scope. + +**`recordAction`, `:2336-2352`.** It builds a new entry, so add a line forwarding `timing: input.timing`: +```js +function recordAction(input) { + try { + if (!input || typeof input !== 'object') return; + recordDispatch({ + client: input.client, + tool: input.tool, + requestPayload: (input.payload && typeof input.payload === 'object') + ? input.payload + : { tool: input.tool, params: input.params || {}, agentId: input.agentId }, + response: input.response, + success: input.success !== false, + dispatcher_route: 'bridge-action', + tabId: input.tabId, + replayContext: input.replayContext + }); + } catch (_e) { /* fire-and-forget */ } +} +``` + +**`_snapshotJournalEntry`, `:918-966`.** It returns an explicit field list ending at `:963-965`: +```js + targetRedacted: entry.targetRedacted === true || + (rawTargetUrl !== null && safeContext.targetUrl !== rawTargetUrl) + }; +``` +Add `timing: _sanitizeCallTiming(entry.timing)`, keeping only finite non-negative numbers. For the finite-number idiom, copy `extension/utils/delegation-event-store.js:274-282`: +```js +function _nonnegativeIntegerOrNull(value) { + return typeof value === 'number' + && Number.isFinite(value) + && Number.isInteger(value) + && value >= 0 + && value <= MAX_SAFE_INTEGER + ? value + : null; +} +``` + +**`recordTaskOutcome`, `:1751-1798`.** The journal entry is built at `:1777-1786`; add `timing: input.timing` to that literal: +```js +var journalEntry = _snapshotJournalEntry({ + client: payload.visualSession && payload.visualSession.client, + tool: input.tool, + requestPayload: payload, + response: input.response, + success: !(input.response && input.response.success === false), + dispatcher_route: 'task-status', + tabId: snapshot.tabId, + replayContext: { routeFamily: 'lifecycle' } +}); +``` + +**Legacy path.** +- `_snapshotEntry` (`:854-880`, same explicit-field style) gains `timing`. +- `session.actionHistory.push({...})` (`:2146-2152`) gains `durationMs` and `timing`, omitted when absent: +```js +session.actionHistory.push({ + tool: storedTool, + params: replayParams, + result: replayResult, + timestamp: now, + logicalTab: logicalTab +}); +``` +Keep `:2157-2170` byte-intact, because `tests/lattice-replay-background-contract.test.js:391-394` pins `requestPayload: _sanitizeRequestPayload(payload, entry.response)` (`:2162`) and `redactedInputs: entry.redactedInputs === true` (`:2168`). + +#### `extension/utils/mcp-lattice-journal.js` (store, file-I/O) + +This file is also an IIFE: `(function (globalScope) {` at `:9` through `:2248`. + +**Metadata, `:1210-1229`.** Add `if (entry.timing) metadata.timing = ;` after the literal and before `makeEvent` at `:1233`: +```js +var metadata = { + schemaVersion: SCHEMA_VERSION, + agentId: run.agentId, + // ... + targetRedacted: entry.targetRedacted === true, + lateAfterClose: run.status !== 'running' +}; +``` +The budget-exhausted compact marker reuses the same `metadata` object (`:1243-1246`), so timing survives there too. Replay is unaffected: `replayEntryFromEvent` (`:1752-1766`) copies an explicit field list. `INTERNAL_PAYLOAD_KEYS` (`:44-48`) does not change. + +**Text exports: one shared formatter for both.** Today the two producers emit identical call lines: +- `exportHumanReadable`, `:2039-2043`: +```js +rows.forEach(function (event, index) { + var metadata = event.metadata || {}; + lines.push((metadata.success === false ? '[FAILED] ' : '[OK] ') + + 'Call ' + String(index + 1) + ': ' + (metadata.tool || 'unknown')); +}); +``` +- `streamSessionExport` text branch, `:2137-2151`. It pages 500 at a time; `callIndex` survives across pages, so add `prevCompletedAt` beside it: +```js +var textAfter = -1; +var callIndex = 0; +while (true) { + var textRows = await backend().getEvents(sessionId, textAfter, DETAIL_MAX_LIMIT); + // ... + await emitChunked('\n' + (textMetadata.success === false ? '[FAILED] ' : '[OK] ') + + 'Call ' + String(callIndex) + ': ' + (textMetadata.tool || 'unknown')); +``` +Append the suffix after the tool name; the formatter is in RESEARCH "Journal text export line". The pins are regex `assert.match(report, /Call 505: read_step_504/)` (`tests/mcp-lattice-journal.test.js:1399`, unanchored, so a suffix is fine) and **byte equality** of the two producers (`:432-439`). + +**Journal read API used by Track B** (unchanged): `hasSession` `:2008-2011`, `getSessionDetail` `:1631-1648` (limit clamped to `DETAIL_MAX_LIMIT = 500`, `:25`; returns `{session: buildIndexEntry(run), events, nextSequence, hasMore}`), `exportHumanReadable` `:2019`, api object `:2199-2244`. + +--- + +### Track B: `get_session_detail` journal branch and measurement projection + +#### `extension/ws/mcp-tool-dispatcher.js` (controller, request-response) + +**Analog 1: the control-panel journal-first lookup**, `extension/background.js:12500-12505` (text) and `:12513-12531` (detail): +```js +const isJournalSession = globalThis.FsbMcpLatticeJournal && + typeof globalThis.FsbMcpLatticeJournal.hasSession === 'function' && + await globalThis.FsbMcpLatticeJournal.hasSession(request.sessionId); +const text = isJournalSession && typeof globalThis.FsbMcpLatticeJournal.exportHumanReadable === 'function' + ? await globalThis.FsbMcpLatticeJournal.exportHumanReadable(request.sessionId) + : await automationLogger.exportHumanReadable(request.sessionId); +``` +```js +const journal = globalThis.FsbMcpLatticeJournal; +const detail = journal && typeof journal.getSessionDetail === 'function' + ? await journal.getSessionDetail({ sessionId: request.sessionId, afterSequence: request.afterSequence, limit: request.limit }) + : null; +if (detail) { sendResponse({ success: true, ...detail }); return; } +const session = await automationLogger.loadSession(request.sessionId); +``` +**Do not copy the `...detail` spread.** It returns raw events (D-08). + +**Analog 2: journal-to-legacy conversion**, `extension/ui/options.js:3466-3472`: +```js +session.actionHistory = events.filter(event => event.kind === 'tool.call').map((event) => ({ + tool: event.metadata?.tool || '', + params: journalDescriptorPreview(event.metadata?.request), + result: journalDescriptorPreview(event.metadata?.result), + timestamp: Date.parse(event.timestamp), + logicalTab: event.metadata?.logicalTab || 'primary' +})); +``` +Mirror its field mapping, but **do not reuse `journalDescriptorPreview`** (`:3433-3442`), because it returns `descriptor.inline` values verbatim. Feed each pseudo-action through the existing `sanitizeActionHistoryEntry` instead. The paging loop to mirror is `loadSessionDetail` (`options.js:3478-3487`), using `afterSequence`/`limit: 500`. + +**Target route: `handleGetSessionMessageRoute`, `:2904-2957`.** Insert the journal branch between `:2907` and `:2908`, so it runs after the sessionId check and before the `automationLogger` availability check: +```js +async function handleGetSessionMessageRoute({ payload }) { + if (!payload.sessionId) { + return createMcpInvalidParamsError('get_session_detail', 'get_session_detail requires sessionId', { routeFamily: 'observability' }); + } + if (typeof automationLogger === 'undefined' || typeof automationLogger?.loadSession !== 'function') { + return createMcpRouteError('get_session_detail', 'observability', MCP_ROUTE_RECOVERY_HINT, { error: 'Automation logger session loader unavailable' }); + } + const session = await automationLogger.loadSession(payload.sessionId); + if (session) { + if (payload.format === 'text') { /* ... automationLogger.exportHumanReadable ... */ } + return { success: true, session: sanitizeSessionDetail(session) }; + } + // in-flight fallback (:2928-2947), then session_not_found (:2949-2956) -- unchanged +``` +Keep the envelopes `{ success: true, session }` and `{ success: true, format: 'text', text }` byte-compatible. The error shape for a missing exporter is `:2916-2918`. + +**Sanitizers to reuse, `:2027-2116`.** Mind the `/token/i` key filter: +```js +function isSensitiveKey(key) { + return /password|passcode|token|secret|apikey|api_key|authorization|credential|cardnumber|card_number|cvv|cvc|vault|payment|privatekey|private_key/i.test(String(key || '')); +} +``` +`sanitizeValue` flattens objects at depth >= `maxDepth` to `'[object]'` (`:2041`). `sanitizeActionHistoryEntry` (`:2072-2086`) is the allowlist to extend with a finite-only `durationMs` (and `timing`): +```js +function sanitizeActionHistoryEntry(action) { + const params = (action?.params && typeof action.params === 'object') ? action.params : {}; + const result = (action?.result && typeof action.result === 'object') ? action.result : {}; + return { + tool: action?.tool || null, + timestamp: action?.timestamp || null, + iteration: action?.iteration || null, + ...(params.selector ? { selector: boundedString(params.selector, 250) } : {}), + ...(params.url ? { domain: getDomainFromUrl(params.url) } : {}), + result: { + success: Boolean(result.success), + ...(result.error ? { error: boundedString(result.error, 500) } : {}) + } + }; +} +``` +The conditional-spread idiom (`...(cond ? {k} : {})`) already in this function is the one to copy for "omit when unmeasured". `sanitizeSessionDetail` (`:2088-2116`) is the pattern for post-sanitizer additions: it `delete`s a raw field and then attaches an explicit projection (`replaySummary`, `:2097-2105`). Build the `measurement` block the same way, after the generic sanitizer. + +**Timing on the dispatcher's two recording paths.** +- **Message route:** add `timing` to `var sessionRecordEntry = {` (`:829-851`), read from `payload[MCP_DISPATCHER_CALL_TIMING]`. +- **Task outcomes:** in `dispatchMcpToolRoute` (`:639-731`), read the symbol from the **original** payload before `canonicalizeVisualSessionTaskParams` (`:649-652`). `canonicalizeVisualSessionOwnershipToken` spreads the payload (`:576-579`, `return { ...payload, ownershipToken: ... }`), which drops Symbols. Pass the timing explicitly at `:722`: +```js +globalThis.fsbMcpSessionRecorder.recordTaskOutcome({ tool, params, payload, response }); +``` + +**Exports.** If the planner wants the new pure projection helpers unit-tested, add them to `_mcp_dispatcher_exports` (`:3926-3960`), whose comment style is "(test-only)". Nothing pins that key list. + +--- + +### Track C: Autopilot timing, provider cost labels, price rows + +#### `extension/ai/agent-loop.js` (service, event-driven): **highest tripwire load** + +**Model-time stamp.** Stamp model time from `apiCallStartTime` at `:1936`, next to the existing completion log at `:1986-1992`, which already computes the same delta: +```js +if (typeof automationLogger !== 'undefined') { + automationLogger.debug('Agent iteration API call completed', { + sessionId: sessionId, iteration: iterNum, + durationMs: Date.now() - apiCallStartTime, + hasToolCalls: _isToolCallResponse(response, providerKey) + }); +} +``` +The single text-only retry is at `:1957-1979`; mark `modelRetried: true` when that path runs. + +**Usage and cost block, `:2005-2029`.** Keep the legacy numeric flow. Add labelled fields beside it; do not replace these lines: +```js +var usage = _extractUsage(response, providerKey); +var inputTokens = usage.input || 0; +var outputTokens = usage.output || 0; +var iterationCost = 0; +if (session._costTracker) { + iterationCost = session._costTracker.record(model, inputTokens, outputTokens); +``` +The existing precedent for reading the raw xAI body directly off `response` is `:2775-2779`: +```js +var _phase10ReasoningTokens = undefined; +if (_phase10ProviderKey === 'xai' && typeof response !== 'undefined' && + response && response.usage && response.usage.completion_tokens_details) { + _phase10ReasoningTokens = response.usage.completion_tokens_details.reasoning_tokens; +} +``` + +**Tool-time bracket.** Start after the permission hook returns non-denied, at the step marker `:2295-2298`: +```js +// Phase 9 FINT-14 -- mark in-flight tool dispatch boundary. ... +session._currentStepName = 'BEFORE_TOOL_EXECUTION'; +``` +End at the success push, `:2797-2812`, and add `durationMs` inside both object literals: +```js +if (session._actionHistory) { + session._actionHistory.push({ + tool: call.name, params: call.args, + result: { success: result.success, hadEffect: result.hadEffect, error: result.error || null }, + timestamp: Date.now(), iteration: iterNum + }); + session.actionHistory = session._actionHistory.events; // backward compat sync +} else { + if (!session.actionHistory) session.actionHistory = []; + session.actionHistory.push(_al_createActionEvent({ /* same fields */ })); +} +``` +The malformed-argument branch (`:2228-2252`) and the permission-denied branch (`:2258-2293`) never ran a tool. Leave `durationMs` absent there. + +**Getting new cost-tracker helpers into the loop.** Use the resolver idiom at `:155-157`, which works in both the SW (globals) and Node (`require`, `:65-83`): +```js +var _al_estimateCost = (typeof estimateCost !== 'undefined') ? estimateCost : (_al_costTracker?.estimateCost || function() { return 0; }); +var _al_CostTracker = (typeof CostTracker !== 'undefined') ? CostTracker : (_al_costTracker?.CostTracker || null); +``` +Add the same one-liner for each new helper, with a null-returning fallback, never a zero-returning one. `_al_costTracker` is already `require('./cost-tracker.js')` at `:69`. + +**Persistence hand-off.** `saveToLogger` (`:1457-1465`) calls `automationLogger.saveSession(sid, sess)` with the live session, so fields such as `session.iterationTiming` and `session.measurementSummary` reach the logger without an edit here. + +**Exports.** `module.exports` (`:3131-3133`) can export a pure row builder additively; nothing pins its key list. + +#### `extension/ai/cost-tracker.js` (utility, transform) + +**Price table, `:23-55`.** Substring matching iterates insertion order (`estimateCost`, `:74-84`). New `grok-4.3`/`grok-4.3-latest` rows must come **before** the `'grok-4'` key (`:36`); today `'grok-4.3'` matches `grok-4` at $3/$15. The unknown-model fallback, `:85-88`, is D-10's second "invented value" site: +```js +// Default to grok-4-1-fast pricing if model not found +if (!pricing) { + pricing = MODEL_PRICING['grok-4-1-fast-reasoning'] || { input: 0.20, output: 0.50 }; +} +``` +The row values already exist in the MCP pricing data (read-only reference): `mcp/data/mcp-pricing-data.json` has `grok-4.3` at `1.25 / 2.5`, confidence `HIGH`, `source_url` `https://docs.x.ai/developers/models`. + +**Labelled nullable result: copy the envelope from `mcp/src/tools/pricing.ts`.** Type, `:68-74`: +```ts +export type McpPricingResult = { + cost: number | null; + source: 'lookup' | 'fallback' | 'unknown'; + model_used: string | null; + pricing_confidence: 'HIGH' | 'MEDIUM' | 'LOW' | 'fallback' | null; + pricing_source_date: '2026-05-14'; +}; +``` +Unknown envelope, `:152-158`, and the finite, non-negative token gate, `:254-267`: +```ts +const tin = typeof tokensIn === 'number' && Number.isFinite(tokensIn) && tokensIn >= 0 ? tokensIn : null; +const tout = typeof tokensOut === 'number' && Number.isFinite(tokensOut) && tokensOut >= 0 ? tokensOut : null; +let cost: number | null = null; +if (tin !== null && tout !== null) { /* compute */ } +``` +The second analog is "dollar cost is deliberately null", `extension/utils/delegation-event-store.js:567-570`: +```js +var billingKind = context.acceptedIdentity.billingKind; +// Neither shipped agent provider exposes authoritative dollar billing. +// Payload/context dollar claims are deliberately non-representable. +var usd = null; +``` +Write the new functions in this file's ES5 style (`var`, `function`). Extend the exports line at `:171-173`: +```js +if (typeof module !== 'undefined' && module.exports) { + module.exports = { MODEL_PRICING: MODEL_PRICING, estimateCost: estimateCost, CostTracker: CostTracker }; +} +``` +The provider-usage shapes to mirror are in `extension/ai/tool-use-adapter.js:312-337`: `extractUsage` handles anthropic `usage.input_tokens`, gemini `usageMetadata.promptTokenCount`, and OpenAI-compatible `usage.prompt_tokens`. + +#### `extension/utils/automation-logger.js` (store, CRUD) + +Two parallel explicit projections must both change. +- **Existing-session branch.** Totals at `:884-886` (`existing.totalCost = sessionData.totalCost || existing.totalCost || 0;`, the "invented 0" in D-10). Action rows at `:904-916`: +```js +if (sessionData.actionHistory) { + existing.actionHistory = (sessionData.actionHistory || []) + .filter(a => a.result?.success) + .slice(-100) + .map(a => ({ + tool: a.tool, + params: a.params, + result: a.result, + timestamp: a.timestamp, + logicalTab: a.logicalTab || 'primary' + })); +} +``` +- **New-session branch.** Totals at `:951-953`; action rows at `:963-973`, with the same `.map`. +- **Index entry.** `:982-1012` (`totalCost: savedSession.totalCost || 0` at `:993`). Leave the index lean; the index feeds `list_sessions`. + +Add `durationMs: Number.isFinite(a.durationMs) ? a.durationMs : null` to **both** `.map`s. Copy `iterationTiming` (capped) and `measurementSummary` in **both** branches, following the `existing.replay = cloneReplayRecordForPersistence(...) || existing.replay || null` precedent at `:917-919`. + +`exportHumanReadable` (`:689-714`) is the autopilot text export, if per-call ms is also shown for autopilot sessions. Keep the method order: `tests/lattice-replay-background-contract.test.js:111-114` slices from `async getReplayData(` to `async exportHumanReadable(`. + +#### `extension/ai/action-history.js` (model, transform; optional) + +`createActionEvent` (`:38-53`) has `durationMs: f.durationMs || 0` at `:51`; it runs only on the fallback path. `ActionHistory.prototype.push` (`:91-100`) stores objects that already have `tool` and `timestamp` as-is, so the agent-loop literal is what persists. If `:51` becomes null-when-absent, update `tests/action-history.test.js:67` (`assertEqual(evt.durationMs, 0, 'default durationMs is 0')`) in the same commit. Leave `tests/turn-result.test.js:50` alone: `lastTurnResult.durationMs` is session-elapsed time (`agent-loop.js:2111`). + +--- + +### Track D: Bench core (helpers, tasks, fixtures, server, tests, chain registration) + +#### `scripts/bench/fixture-server.mjs` (server, request-response) + +Analog: `tests/screenshot-uat.test.js:63-85`. Copy the traversal guard and the `listen(0, '127.0.0.1')` idiom: +```js +function startFixtureServer() { + const server = http.createServer(async (request, response) => { + try { + const pathname = decodeURIComponent(new URL(request.url, 'http://127.0.0.1').pathname); + const absolute = path.resolve(WORKSPACE, `.${pathname}`); + if (absolute !== WORKSPACE && !absolute.startsWith(`${WORKSPACE}${path.sep}`)) { + response.writeHead(403).end(); + return; + } + const bytes = await fsp.readFile(absolute); + const type = absolute.endsWith('.html') ? 'text/html; charset=utf-8' + : (absolute.endsWith('.ttf') ? 'font/ttf' : 'application/octet-stream'); + response.writeHead(200, { 'content-type': type, 'cache-control': 'no-store' }); + response.end(bytes); + } catch (_error) { + response.writeHead(404).end(); + } + }); + return new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', () => resolve(server)); + }); +} +``` +Differences: +- Root is `scripts/bench/fixtures`, not the workspace. Strip the `/r//` prefix before resolving. +- Add JSON/JS content types, POST body capture into a per-token request log, and a secret-gated `GET /__bench/evidence`. +- Export a `startFixtureServer(options)` function so `tests/bench-fixture-server.test.mjs` can import it without the CLI running (direct-run guard below). + +#### `scripts/bench/fixtures//**` (static pages) + +Analog: `tests/fixtures/screenshot-uat.html`, which sets a readiness marker at `:119` (`document.documentElement.dataset.fixtureReady = 'true';`). Use `data-bench-marker` attributes the same way. **Differences:** that fixture uses an absolute asset URL (`/extension/assets/...`, `:10`). Bench pages must use **relative** links only, so the `/r//` prefix is preserved (RESEARCH Pattern 7). Do not add in-page probes that post on DOM mutation (Pitfall 9). + +#### `scripts/bench/lib/*.mjs`: pure ESM library style + +Analog: `scripts/lib/t1-port-contract.mjs:1-60`, with a doc header, `export const` frozen enums and small private helpers: +```js +/** + * Phase 45 / Plan 01 -- reusable T1 port contract helpers. + * This module is tooling-only. ... + */ +'use strict'; + +export const PORT_TYPES = Object.freeze({ SAME_ORIGIN_READ: 'same-origin-read', /* ... */ }); + +function isObject(value) { + return !!value && typeof value === 'object' && !Array.isArray(value); +} +function hasOwn(obj, key) { + return Object.prototype.hasOwnProperty.call(obj, key); +} +``` +- `lib/stats.mjs`: no statistics code exists in the repo. Use the RESEARCH Pattern 9 code verbatim; it is pure and seeded with `mulberry32`. +- `lib/outcome.mjs`: return `{ pass, checks: [{kind, pass, detail}] }`. The t1-port-contract validators return failure lists the same way. The evaluator must never read `claimed`. +- `lib/tasks.mjs`: schema-check task JSON the way `claude-profile.ts:53-79` validates `fsb.json`, with an exact sorted key set and explicit value checks: +```ts +const keys = Object.keys(policy).sort(); +if (JSON.stringify(keys) !== JSON.stringify(STATIC_AGENT_KEYS)) { + throw new Error('Invalid shipped FSB agent policy keys'); +} +``` +- `lib/identity.mjs`: the canonical-JSON analog is `extension/utils/mcp-lattice-journal.js:110-130` (`stableJson`: sorted keys, drops `undefined`/functions, throws on cycles). Hash with `createHash('sha256')`, as in `scripts/build-native-host-windows.mjs:102`. For the git SHA and dirty state, copy `captureGitBuffer` from `scripts/run-phase60-full-tests.mjs:51-62`: +```js +function captureGitBuffer(args, label) { + const result = spawnSync('git', args, { + cwd: repositoryRoot, + env: { ...process.env, GIT_OPTIONAL_LOCKS: '0' }, + maxBuffer: maximumGitOutput, + shell: false, + }); + if (result.error || result.signal || result.status !== 0) { + throw new Error(`${label} failed`); + } + return Buffer.from(result.stdout); +} +``` + +#### `scripts/bench/tasks/{core,live}/*.json` (versioned data) + +Partial analog: `tests/fixtures/agent-streams/claude-code-2.1.177/manifest.json`, a versioned JSON with `schemaVersion`, provenance and sanitization flags. The task schema itself is in RESEARCH Pattern 7 (`id`, `version`, `set`, `category`, `startPath`, `instruction`, `timeBudgetMs`, `success.all[]`, `fixtureFiles[]`). + +#### `tests/bench-*.test.mjs` (tests, ESM, Node 20-safe) + +The ESM header comes from `tests/dashboard-localization-fallback.test.mjs:1-6`: +```js +import assert from 'node:assert/strict'; +import { createRequire } from 'node:module'; +import path from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +``` +The `node:test` structure comes from `tests/mcp-lattice-journal.test.js:3-7` (`test('...', async () => { ... })` with `assert.equal`/`assert.deepEqual`/`assert.match`). +- **Difference:** the only existing `.mjs` test runs under `node --import tsx` (pretest). Bench tests are plain ESM with no TS imports: run them as `node tests/bench-x.test.mjs` or `node --test tests/bench-*.test.mjs ...`, and import `../scripts/bench/lib/*.mjs` by relative path. +- **CI is Node 20** (`.github/workflows/ci.yml:22`). No global `WebSocket`, `Object.groupBy`, `Promise.withResolvers`, `crypto.hash`, `fs.globSync` or `process.getBuiltinModule`. +- **`bench-fixture-server.test.mjs`:** copy the lifecycle from `tests/screenshot-uat.test.js:214-216` (start server, read `server.address().port`) and `:379-384` (`finally { await new Promise((resolve) => server.close(resolve)); }`). Bind port 0, never 7225. +- **`bench-claude-stream.test.mjs` argv parity:** read `mcp/src/agent-providers/claude-profile.ts` as text and assert that each task-mode flag (`:192-211`) appears in the bench argv. `scripts/verify-agent-provider-flags.mjs:36-52` is the precedent for line-scanning TS source from a script. + +#### `tests/fixtures/bench/claude-stream-result.jsonl` (new fixture) + +The analog is `tests/fixtures/agent-streams/claude-code-2.1.177/contract-stream.jsonl`. Its `result` line (line 6) has no duration or cost fields: +```json +{"type":"result","subtype":"success","session_id":"synthetic-session-01","is_error":false,"num_turns":2,"usage":{"input_tokens":10,"output_tokens":20}} +``` +The new fixture adds `duration_ms`, `duration_api_ms`, `total_cost_usd`, `modelUsage`, an init with `tools` containing `ToolSearch`, and `apiKeySource`. + +**Put it outside `tests/fixtures/agent-streams/`.** `tests/mcp-agent-drift-smoke.test.js:193-200, 358-366` enumerates every `agent-streams/*/manifest.json` and deep-equals the list against the adapter roster. + +#### `tests/fixtures/bench/xai-chat-completion.json` (new fixture) + +The analog is `tests/fixtures/model-discovery-responses.js:1-28`: synthetic data with a doc header naming the documented shape and stating that no network calls were made. The existing minimal xAI body shape is `tests/lattice-provider-bridge-smoke.test.js:334` (`usage: { prompt_tokens: 1, completion_tokens: 1 }`). Add `usage.cost_in_usd_ticks`, `usage.completion_tokens_details.reasoning_tokens`, `usage.prompt_tokens_details.cached_tokens` and `model`. + +#### `package.json` + `tests/delegation-phase-contract.test.js` (chain registration, one plan only) + +Registration list, `tests/delegation-phase-contract.test.js:129-155`: +```js +const REFINEMENT_ROOT_TEST_COMMANDS = Object.freeze([ + 'node tests/mcp-agent-connection-test.test.js', + // ... + 'node --test tests/google-sheets-session.test.js tests/google-sheets-content-actions.test.js tests/gsheets-handler.test.js tests/google-sheets-wiring.test.js tests/spreadsheet-record-redaction.test.js', + 'node tests/sidepanel-automation-runner.test.js', +]); +``` +The enforcing checks are at `:2373-2391` and `:3126-3134`: +```js +const rootCommands = packageJson.scripts.test.split(' && '); +for (const command of REFINEMENT_ROOT_TEST_COMMANDS) { + check(rootCommands.filter((candidate) => candidate === command).length === 1, ...); +} +// every command NOT in a known list must hash to PRE_PHASE61_ROOT_TEST_HASH / PRE_PHASE62_ROOT_TEST_HASH +``` +Rules: +1. Append each new command string to the **tail** of `scripts.test`, joined with `' && '`. Today the last entry is `node --import tsx tests/no-orphan-descriptor.test.js`, and no test pins it. +2. Add the identical string to `REFINEMENT_ROOT_TEST_COMMANDS`. +3. Never edit the `PRE_PHASE6x_ROOT_TEST_HASH` constants (`:12`, `:16`). +4. Do not reorder existing entries (`tests/mcp-client-identity-integration.test.js:469-486`). + +A multi-file `node --test a b c` entry is one command string, as the Sheets precedent shows. + +#### `.gitignore` + +Add a commented block such as `# Phase 66 -- raw benchmark captures (may contain page content; D-21)` followed by `scripts/bench/.local/`. The global `build/`, `dist/` and `*.log` ignores already exist, so do not name committed result directories `build`/`dist`, and do not give committed artifacts a `.log` extension. No test pins `.gitignore`. + +--- + +### Track E: Runner, arms, report + +#### `scripts/bench/run-bench.mjs` (CLI runner, batch) + +Analog: `scripts/run-phase65-full-tests.mjs`. Copy the bounded child-output forwarding and the signal forwarding at `:169-227`: +```js +let activeChild = null; +let receivedSignal = null; +function onSignal(signal) { + if (receivedSignal) return; + receivedSignal = signal; + if (activeChild && activeChild.exitCode === null && activeChild.signalCode === null) { + try { activeChild.kill(signal); } catch { /* ... */ } + } +} +process.on('SIGINT', () => onSignal('SIGINT')); +process.on('SIGTERM', () => onSignal('SIGTERM')); + +function runChild(command, label) { + return new Promise((resolveResult) => { + let settled = false; + const settle = (result) => { if (settled) return; settled = true; activeChild = null; resolveResult(result); }; + try { + const child = spawn(command[0], command.slice(1), { cwd: repositoryRoot, env: process.env, stdio: ['ignore', 'pipe', 'pipe'], shell: false }); + activeChild = child; + // forwardBounded(child.stdout, ...), child.once('error'...), child.once('close', (status, signal) => ...) + } catch (error) { settle(Object.freeze({ label, error })); } + }); +} +``` +Also copy the exit-code convention at `:274-276`: 130 for SIGINT, 143 for SIGTERM. + +**Differences:** +- The Claude arm needs stdin (the task goes on stdin), so use `stdio: ['pipe', 'pipe', 'pipe']`. +- On a timeout, send SIGINT first, wait a grace period, then SIGTERM. SIGTERM produces no `result` event (RESEARCH Pattern 8). +- Write each run's JSON immediately (D-20). + +#### `scripts/bench/report.mjs` and `scripts/bench/export-history.mjs` (CLI transform) + +Analog: `scripts/report-t1-readiness.mjs`. Copy the header at `:14-24`, the pure exported `renderMarkdown`/`writeReport` at `:1241-1247`, and the direct-run guard at `:1273-1280`. The guard is what lets `tests/bench-*.test.mjs` import the module without side effects: +```js +export function writeReport(report, paths) { + const jsonPath = (paths && paths.jsonPath) || JSON_OUT; + const mdPath = (paths && paths.mdPath) || MD_OUT; + mkdirSync(dirname(jsonPath), { recursive: true }); + writeFileSync(jsonPath, JSON.stringify(report, null, 2) + '\n'); + writeFileSync(mdPath, renderMarkdown(report)); +} +// ... +if (import.meta.url === pathToFileURL(process.argv[1] || '').href) { + try { runCli(); } catch (err) { console.error('...: ERROR ' + (err && err.message ? err.message : String(err))); process.exit(1); } +} +``` +**Difference:** that report writes into `.planning/phases/44-*`. Bench output goes to `scripts/bench/results//`, per RESEARCH "Recommended Project Structure" and Pitfall 22. The `export-history.mjs` input is the JSON produced by the RESEARCH console snippet. Its paging loop mirrors `extension/ui/options.js:3478-3487`. + +#### `scripts/bench/lib/claude-arm.mjs` (driver, streaming) + +- **argv:** mirror the task-mode argv at `mcp/src/agent-providers/claude-profile.ts:192-211`: +```ts +'-p', '--verbose', '--output-format', 'stream-json', '--include-partial-messages', +'--setting-sources', '', '--disable-slash-commands', '--no-chrome', '--strict-mcp-config', +'--mcp-config', ctx.privateMcpConfigPath, '--agents', SERIALIZED_FSB_AGENTS, '--agent', 'fsb', +'--permission-mode', 'dontAsk', '--tools', '', '--allowedTools', 'mcp__fsb', +'--disallowedTools', DENIED_TOOLS.join(','), '--max-turns', '40', '--no-session-persistence', +``` +- **`--agents` value:** build it the way `serializeStaticAgentPolicy` does (`:93-105`), with key order `description, prompt, tools, disallowedTools, permissionMode, maxTurns` under the key `fsb`. Read `mcp/ai/agents/fsb.json` (`loadStaticAgentPolicy`, `:53-89`). `DENIED_TOOLS` is at `:24-31`. +- **Forbidden flags:** never use `--dangerously-skip-permissions`, `--yolo` or `--auto` (`scripts/verify-agent-provider-flags.mjs:19-23`). +- **NDJSON reader:** copy the line-splitting loop at `mcp/src/agent-providers/claude-stream.ts:369-397`, including CR stripping and flushing the trailing partial line. + +**Do NOT reuse `attestInit` (`:163-182`).** It rejects any tool that is not `mcp__fsb*`, so it would throw on `ToolSearch`/`WaitForMcpServers`, which D-18 must *record*. The bench summarizer is lenient and only records. The `ResultSchema` is `.passthrough()` (`:115-120`), so `duration_ms`, `duration_api_ms`, `total_cost_usd`, `modelUsage`, `apiKeySource` and `permission_denials` arrive as extra keys. + +#### `scripts/bench/lib/cdp.mjs` (client, request-response) + +The analog is `tests/screenshot-uat.test.js`: +- `waitForDevToolsPort` (`:50-61`) reads `/DevToolsActivePort`. +- `createCdpClient` (`:87-135`) is an id-correlated `pending` Map plus one-shot `listeners`. +- Target discovery is at `:231-232`: `fetch('http://127.0.0.1:' + port + '/json/list')`, then `targets.find(t => t.type === 'page')`. + +**Difference:** that test uses the global `WebSocket` (Node 22+). Resolve `ws` from `mcp/node_modules` as `tests/mcp-bridge-topology.test.js:10-11` does, or with `createRequire` in ESM: +```js +const repoRoot = path.resolve(__dirname, '..'); +const WebSocket = require(path.join(repoRoot, 'mcp', 'node_modules', 'ws')); +``` + +#### `scripts/bench/lib/mcp-client.mjs` (client, request-response) + +No in-repo code uses the SDK `Client` (grep-verified). The module-resolution analog is `tests/write-activation-evidence.test.js:7-10`: +```js +const { createRequire } = require('node:module'); +const REPO_ROOT = path.resolve(__dirname, '..'); +const requireFromRoot = createRequire(path.join(REPO_ROOT, 'package.json')); +``` +Use `createRequire(path.join(repo, 'mcp', 'package.json'))`, then follow the RESEARCH "Runner: own SDK client" code example. Pass `run_task` with `{ onprogress, resetTimeoutOnProgress: true, timeout: 60000, maxTotalTimeout }`, because the SDK default is 60 s. The server passes `fsb://config` through (`mcp/src/resources/index.ts:155-160`, 10 s timeout). `get_session_detail` forwards only `{ sessionId, format }` (`mcp/src/tools/observability.ts:46-57`, 10 s timeout), so no new parameters are possible without an `mcp/` change. + +#### `scripts/bench/lib/cursor-arm.mjs` (optional) + +There is no Cursor code in the repo. Reuse the NDJSON loop from `claude-arm.mjs`. The `result` event has durations only, so tokens and cost are labelled `unavailable (host does not report)`. + +--- + +## Shared Patterns + +### S1. Hidden side-channel on the payload (never serialized, never spoofable) +**Source:** `extension/ws/mcp-tool-dispatcher.js:157-160, 838-850, 3184-3197` +**Apply to:** `mcp-bridge-client.js` (stamp at ingress, read in the action tap) and `mcp-tool-dispatcher.js` (read in the message-route recorder and in `dispatchMcpToolRoute`) +```js +payload && payload[MCP_REPLAY_RECORD_CONTEXT] + ? payload[MCP_REPLAY_RECORD_CONTEXT].tabId + : (readRouteInspection.recordContext ? readRouteInspection.recordContext.tabId : null) +``` +Any `{ ...payload }` spread or `structuredClone`/JSON round-trip drops the symbol. Read it before `canonicalizeVisualSessionTaskParams` (`:649`). + +### S2. Recording never gates routing (fire-and-forget in its own try/catch) +**Source:** `extension/ws/mcp-bridge-client.js:1182-1197`, `extension/ws/mcp-tool-dispatcher.js:823-871`, `extension/utils/mcp-session-recorder.js:2222-2280` +**Apply to:** every new timing read or write in the bridge client, dispatcher, recorder and agent-loop. +- `Date.now()` only. `performance` is undefined in the vm harnesses. +- Do **not** reuse the exact comment strings `never let session recording break dispatch` / `never let session recording break the action`: `tests/mcp-session-recorder.test.js:867-875` counts each at exactly 1. + +### S3. Unmeasured means null or omitted, never 0 +**Sources:** `extension/utils/delegation-event-store.js:274-282` (`_nonnegativeIntegerOrNull`), `:567-570` (`usd = null`); `mcp/src/tools/pricing.ts:152-158` (unknown envelope), `:254-267` (finite gate) +**Apply to:** `_sanitizeCallTiming` (recorder), journal `metadata.timing`, `sanitizeActionHistoryEntry.durationMs`, agent-loop iteration rows, cost-tracker labelled helpers, automation-logger projections. +**Conditional-spread idiom** (`mcp-tool-dispatcher.js:2079-2080`): +```js +...(params.selector ? { selector: boundedString(params.selector, 250) } : {}), +``` + +### S4. Labelled source/status fields beside the number +**Sources:** `McpPricingResult.source` / `pricing_confidence` (`pricing.ts:68-74`); `_projectMetrics.billingKind` (`delegation-event-store.js:567, 577`) +**Apply to:** `usageSource`, `costSource` (`'provider' | 'estimate' | 'mixed' | 'unavailable'`), `queueWaitStatus: 'not_measured'`, `host.status: 'unavailable'`, Claude `total_cost_usd` → `"list-price estimate, not billed"`. + +### S5. Classic service-worker script conventions +**Sources:** `extension/ai/cost-tracker.js:171-173` (`module.exports` guard); `extension/ai/agent-loop.js:65-83, 94, 155-165` (`var` + `typeof X !== 'undefined' ? X : require-fallback`); `extension/utils/mcp-lattice-journal.js:9, 2246-2248` (IIFE with `globalScope`) +**Apply to:** all extension edits. +- The dispatcher, bridge client, cost-tracker and agent-loop are **not** IIFEs, so their top-level names are shared SW globals. All proposed new names are currently unused. +- Do not add top-level `require(...)` calls to the dispatcher: `tests/mcp-in-flight-session-lookup.test.js:65-74` throws on unknown requires. + +### S6. Feature-detect optional SW modules through `globalThis` +**Source:** `extension/background.js:12500-12505` +**Apply to:** the journal branch in `handleGetSessionMessageRoute` +```js +const journal = (typeof globalThis !== 'undefined') ? globalThis.FsbMcpLatticeJournal : null; +if (journal && typeof journal.hasSession === 'function' && typeof journal.getSessionDetail === 'function' + && await journal.hasSession(payload.sessionId)) { /* ... */ } +``` +In the vm harness `context.globalThis = context` (`tests/mcp-in-flight-session-lookup.test.js:83`), so new cases set `context.FsbMcpLatticeJournal = require('../extension/utils/mcp-lattice-journal.js')`, after `journal._resetForTests()` and the time/local shims (`tests/mcp-lattice-journal.test.js:30-49`). The existing four cases must still pass with the module absent. + +### S7. Test harness styles (copy, do not share; repo convention per `tests/mcp-bridge-background-dispatch.test.js:23-25`) +- **Plain node plus counters:** `assert`/`passed`/`failed`, then `process.exit(failed > 0 ? 1 : 0)` (`tests/mcp-in-flight-session-lookup.test.js:21-32, 192-207`). New cases are appended to `run()` (`:192-200`); in the bridge test, append to `run()` at `:3607-3633`. +- **vm harness:** read the source, build `context`, set `context.globalThis = context`, then `vm.runInNewContext(source, context, { filename })`, as in `tests/mcp-in-flight-session-lookup.test.js:34-103` and `tests/mcp-bridge-background-dispatch.test.js:278-326` (the footer exposes internals through `this.__dispatchTest`). +- **Cross-realm deep-equal:** normalize with `JSON.parse(JSON.stringify(x))` before `util.isDeepStrictEqual` (`tests/mcp-bridge-background-dispatch.test.js:100-108`). +- **`node:test`:** `test('...', async () => {...})` with `node:assert/strict`, as in `tests/mcp-lattice-journal.test.js` and `tests/mcp-session-recorder-journal.test.js`. + +### S8. Sensitive-key filter trap (`/token/i`) +**Source:** `extension/ws/mcp-tool-dispatcher.js:2027-2046` +**Apply to:** every stored or returned measurement field. Name usage fields `usage: { input, output, reasoning, cachedInput }` (no "token" substring), keep stored rows flat, and emit the `measurement` block *after* `sanitizeSessionMetadata` (RESEARCH Pitfall 1). + +--- + +## Tripwire Register (update the paired test in the SAME commit) + +| Modified file | Test pin | What it pins | Current value (measured 2026-09-29) | +|---|---|---|---| +| `extension/ai/agent-loop.js` | `tests/lattice-survivability-smoke.test.js:371-373`, `tests/mcp-philosophy-parity-smoke.test.js:373-378`, `tests/lattice-step-emitter-smoke.test.js:302-307` | `setTimeout` token count, comments included | **8** (`:5, 1416, 1602, 2177, 3028, 3031, 3100, 3110`). Add no timers; do not write the word in comments | +| `extension/ai/agent-loop.js` | `tests/agent-loop-iterator-guard.test.js:49-69` | 3 schedule lines byte-exact; 4 schedule call sites | 4 | +| `extension/ai/agent-loop.js` | `tests/lattice-step-emitter-smoke.test.js:157-167, 246-250` | exactly one `stepName: 'LLM_TURN'`, one `stepName: 'TOOL_DISPATCH'`; **no other `stepName: '...'` literal anywhere** | 1 / 1. Do not name a timing-row key `stepName` | +| `extension/ai/agent-loop.js` | `tests/mcp-philosophy-parity-smoke.test.js:384-398` | no `recordDispatch`/`recordVisualSessionTick` inside a `setTimeout(function` lambda body (80-line scan) | 0 | +| `extension/ai/agent-loop.js` | `tests/cost-tracker-ordering.test.js:37-60` | anchors `var sessionConfig = await _al_loadSessionConfig(...)`, `hydrateCostTracker(session);`, `session._costTracker.totalCost = session.totalCost \|\| 0;` (and input/output), text `Cost circuit breaker REMOVED in Phase 231` | present | +| `extension/ai/agent-loop.js` | `tests/meta-cognitive-tracker.test.js:40-48` | markers `var _META_INTERACT_TOOLS = {` and `\nfunction _trackMetaAttempt(session, call, iterNum) {` | present | +| `extension/ai/agent-loop.js` | `tests/ownership-leakage-regression.test.js:334-335` | substrings `spawnProvenance.begin({`, `openTabRegistry.bindTab(openTabAgentId, newTabId)` | present | +| `extension/ai/agent-loop.js` | `tests/run-task-cleanup-paths.test.js` (not in any npm script; run manually) | `fsbBroadcastAutomationLifecycle(` usage | present | +| `extension/ai/cost-tracker.js` | `tests/cost-tracker.test.js:52-54, 74-76, 86, 92, 134-147, 176-183, 190-197, 212-221, 228-233` | `grok-4-1-fast-reasoning` = 0.20/0.50 and **the unknown-model fallback reads that row** (`cost-tracker.js:87`) | Changing that row to 1.25/2.50 (D-11) changes every one of these expectations, including `unknown model defaults to ...` (0.70 becomes 3.75) unless the fallback is decoupled from the row. Decide explicitly; update the test in the same commit | +| `extension/ai/action-history.js` | `tests/action-history.test.js:67` | `createActionEvent({}).durationMs === 0` | only if the default changes | +| `extension/ws/mcp-bridge-client.js` | `tests/mcp-session-recorder.test.js:928-940` | exactly 1 `recordSafely(... 'recordAction' ... sessionRecordEntry ...)`; `let sessionRecordEntry = {` span contains `resolveMcpClientLabel(payload)`; exactly **3** `this._recordMcpSessionAction(` calls | 1 / yes / 3. Add timing *inside* the tap, not a 4th call site | +| `extension/ws/mcp-bridge-client.js` | `tests/mcp-session-recorder.test.js:947-951` | `resolveAgentTabOrError` within 4500 chars after `async _handleExecuteAction` | offset **3845** (655 chars of headroom; edits in `_handleMessage`/`_recordMcpSessionAction` sit before it and do not count) | +| `extension/ws/mcp-bridge-client.js` | `tests/mcp-session-recorder.test.js:873-875` | exactly 1 comment `never let session recording break the action` | 1 | +| `extension/ws/mcp-bridge-client.js` | `tests/spreadsheet-record-redaction.test.js:779, 781` | regex `if \(!spreadsheetTool && !unresolvedRecordTarget\)[\s\S]*recordAction\(sessionRecordEntry\)` | matches | +| `extension/ws/mcp-bridge-client.js` | `tests/mcp-bridge-background-dispatch.test.js:399-487` | lifecycle order `begin, route, result/error, end`; begin/end identity deep-equal | unchanged by design | +| `extension/ws/mcp-tool-dispatcher.js` | `tests/mcp-session-recorder.test.js:887-911` | exactly 1 `recordSafely(... 'recordDispatch' ... sessionRecordEntry ...)`; `var sessionRecordEntry = {` span contains `resolveMcpClientLabel(payload)`; first `fsbMcpSessionRecorder.recordDispatch` occurs after `async function dispatchMcpMessageRoute`; exactly 1 `if (!_mcpMetricsSuppressInner) {` | 1 / yes / yes / 1. Do not add a session-recorder call in `dispatchMcpToolRoute` (`recordTaskOutcome` is fine) | +| `extension/ws/mcp-tool-dispatcher.js` | `tests/mcp-session-recorder.test.js:915-923`, `tests/mcp-dispatcher-client-label.test.js:219-236` | regex `/globalThis\.fsbMcpMetricsRecorder\.recordDispatch\(\{[\s\S]*?\}\);/g` matches exactly **2**, each containing `resolveMcpClientLabel(payload)` and not `fsbMcpSessionRecorder` | 2. Do not put timing in metrics payloads | +| `extension/ws/mcp-tool-dispatcher.js` | `tests/mcp-session-recorder.test.js:867-870` | exactly 1 comment `never let session recording break dispatch` | 1 | +| `extension/ws/mcp-tool-dispatcher.js` | `tests/spreadsheet-record-redaction.test.js:778, 780` | regexes `spreadsheetInvoke[\s\S]*recordSafely\([\s\S]*'recordDispatch'` and `if \(!spreadsheetRecord && !unresolvedRecordTarget\)[\s\S]*recordDispatch\(sessionRecordEntry\)` | match | +| `extension/ws/mcp-tool-dispatcher.js` | `tests/mcp-in-flight-session-lookup.test.js` | in-flight fallback, `session_not_found` (`errorCode`, `recoveryHint` mentions `list_sessions` and `get_task_status`), harness has no journal module and throws on unknown `require` | extend, do not rewrite | +| `extension/utils/mcp-session-recorder.js` | `tests/mcp-session-recorder-journal.test.js:92` | `beginCall` identity deep-equal (7 keys) | unchanged by design | +| `extension/utils/mcp-session-recorder.js` | `tests/mcp-session-recorder.test.js:1356-1357` | no `setTimeout(` anywhere in the recorder | 0 | +| `extension/utils/mcp-session-recorder.js` | `tests/lattice-replay-background-contract.test.js:391-394, 446` | substrings `requestPayload: _sanitizeRequestPayload(payload, entry.response)`, `redactedInputs: entry.redactedInputs === true`, `reason: 'idle_timeout'` | present (`:2162`, `:2168`, `:1625`) | +| `extension/utils/mcp-lattice-journal.js` | `tests/mcp-lattice-journal.test.js:432-439` | `streamSessionExport(id,'text')` output **byte-equals** `exportHumanReadable(id)` | shared formatter required | +| `extension/utils/mcp-lattice-journal.js` | `tests/mcp-lattice-journal.test.js:1396-1399` | `/Calls retained: 505/`, `/Executable replay: latest 100 of 505/`, `/Call 505: read_step_504/` (unanchored regex) | suffix-safe | +| `extension/utils/mcp-lattice-journal.js` | `tests/mcp-lattice-journal.test.js:605-640` | budget-exhausted compact marker path | add `metadata.timing` only when present | +| `extension/utils/automation-logger.js` | `tests/lattice-replay-background-contract.test.js:111-114` | `async getReplayData(` precedes `async exportHumanReadable(` | keep method order | +| `extension/utils/automation-logger.js` | `tests/delegation-sidepanel-tab-follow.test.js:470` | `var AUTOMATION_LOGS_PERSISTED_TAIL = (\d+);` >= 200 | untouched region | +| `extension/background.js` (only if a SW module is added; **not recommended**) | `tests/lattice-provider-bridge-smoke.test.js:634-635, 683-684` | `importScripts` tokens **333**, call sites **329** | inline stamps avoid this | +| `package.json` | `tests/delegation-phase-contract.test.js:2373-2391, 3126-3134` | refinement commands appear exactly once; the chain minus known lists hashes to the pre-phase constants | append to tail + `REFINEMENT_ROOT_TEST_COMMANDS` | +| `package.json` | `tests/mcp-client-identity-integration.test.js:469-486` | Phase 57 cluster order and count | do not reorder | +| new fixture dirs | `tests/mcp-agent-drift-smoke.test.js:193-200, 358-366` | exact set of `tests/fixtures/agent-streams/*/manifest.json` | keep bench fixtures out of `agent-streams/` | + +**Suites to run on every Track C commit** (from RESEARCH Validation Architecture): `node tests/mcp-philosophy-parity-smoke.test.js && node tests/lattice-survivability-smoke.test.js && node tests/agent-loop-iterator-guard.test.js && node tests/cost-tracker-ordering.test.js && node tests/meta-cognitive-tracker.test.js && node tests/ownership-leakage-regression.test.js && node tests/run-task-cleanup-paths.test.js && node tests/lattice-step-emitter-smoke.test.js && node tests/cost-tracker.test.js && node tests/action-history.test.js && node tests/turn-result.test.js`. + +**Suites to run on every Track A/B commit:** `node tests/mcp-bridge-background-dispatch.test.js && node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node tests/mcp-session-recorder-journal.test.js && node --test tests/mcp-lattice-journal.test.js tests/mcp-session-export-port.test.js tests/spreadsheet-record-redaction.test.js && node tests/lattice-replay-background-contract.test.js`. + +--- + +## No Analog Found + +Files with no close match. The planner should use the RESEARCH.md patterns named here. + +| File | Role | Data Flow | Reason | Use instead | +|------|------|-----------|--------|-------------| +| `scripts/bench/lib/stats.mjs` | utility | transform | no statistics or bootstrap code in the repo | RESEARCH Pattern 9 (verbatim, pure, seeded) + module style from `scripts/lib/t1-port-contract.mjs` | +| `scripts/bench/lib/mcp-client.mjs` | client | request-response | no in-repo use of the SDK `Client`/`StdioClientTransport` (grep-verified) | RESEARCH "Runner: own SDK client" example; resolution via `createRequire(/mcp/package.json)` | +| `scripts/bench/lib/cursor-arm.mjs` | driver | streaming | no Cursor integration exists | the `claude-arm.mjs` NDJSON loop; RESEARCH Pattern 8 (Cursor result has durations only) | +| `scripts/bench/results/**` | committed data | batch output | first committed benchmark artifacts | RESEARCH "Recommended Project Structure" (`runs.jsonl`, `summary.json`, `report.md`, `identity.json`); D-21 split raw vs sanitized | +| `scripts/bench/tasks/**/*.json` | config | config | no task-definition corpus exists (manifest style only is analogous) | RESEARCH Pattern 7 schema + suggested 6 mixed / 6 click-heavy set | + +--- + +## Metadata + +**Analog search scope:** `extension/ws/`, `extension/utils/`, `extension/ai/`, `extension/ui/options.js`, `extension/background.js` (targeted ranges), `extension/config/config.js`, `mcp/src/tools/`, `mcp/src/agent-providers/`, `mcp/src/resources/`, `mcp/src/diagnostics.ts`, `mcp/data/`, `scripts/*.mjs`, `scripts/lib/`, `tests/*.test.js`, `tests/*.test.mjs`, `tests/fixtures/`, `package.json`, `.gitignore`, `.github/workflows/ci.yml` +**Files scanned:** ~60 (full reads of the small files; grep-located, non-overlapping reads of the large ones) +**New findings not in RESEARCH.md:** +1. `FsbSpreadsheetRecordRedaction.sanitizeEntry` rebuilds Sheets entries via `baseFields` (`spreadsheet-record-redaction.js:246-255`), dropping any new `timing` field. +2. `tests/mcp-agent-drift-smoke.test.js` enumerates `tests/fixtures/agent-streams/*/manifest.json`, so the new Claude stream fixture must live elsewhere. +3. `tests/lattice-step-emitter-smoke.test.js:246-250` forbids any `stepName: '...'` literal other than the two existing ones in `agent-loop.js`. +4. `tests/mcp-session-recorder.test.js:867-875` counts the two "never let session recording break ..." comments at exactly 1 each. +5. `tests/cost-tracker.test.js` couples the unknown-model fallback value to the `grok-4-1-fast-reasoning` row, so D-11's price change moves ~15 assertions. +6. `resolveAgentTabOrError` currently sits 3845 chars into `_handleExecuteAction` (limit 4500). + +**Pattern extraction date:** 2026-09-29 diff --git a/.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md b/.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md new file mode 100644 index 000000000..7566d6d15 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-RESEARCH.md @@ -0,0 +1,989 @@ +# Phase 66: Baseline and Session Timing - Research + +**Researched:** 2026-09-29 +**Domain:** MV3 service-worker instrumentation (MCP ingress, session recorder, Lattice journal, autopilot loop), provider usage/cost accounting, and an on-request Node benchmark harness (fixture server, headless Claude Code driver, MCP SDK client, bootstrap statistics) +**Confidence:** HIGH for code integration points and tripwires (read and probed in this session); MEDIUM for the live benchmark topology (Chrome/native-host/pairing behaviour partly documented, partly inferred); MEDIUM for xAI facts (official docs, but the response `model` string after redirect is undocumented) + +## User Constraints (from CONTEXT.md) + + + +### Locked Decisions + +#### Change footprint and compatibility +- **D-01:** Keep the MCP diff minimal: Phase 66 changes no source file under `mcp/`. The extension does the work (timing capture, the session-detail fix, run identity, cost labels), and the benchmark lives in `scripts/bench/`. `fsb://config` already passes the extension's payload through unchanged (`mcp/src/resources/index.ts:138-162`), so identity fields are an extension-only edit. +- **D-02:** Every Phase 66 change is additive and does not change behavior: + - Autopilot decisions, action execution, tool names, schemas, descriptions and response shapes stay as they are, apart from added optional fields. + - The one intended behavior change: `get_session_detail` returns journal sessions instead of "not found". + - Milestone-wide rule from the user: with Jev disabled, FSB behaves exactly as it does today. +- **D-03:** No MCP-server-side timestamps in this phase. Time spent in the MCP `TaskQueue` is labelled "not measured" and is never quietly folded into another figure. Caller gap is the extension's receive time for call n minus its completion time for call n−1; both are measured on the same machine. + +#### Timing capture and storage +- **D-04:** MCP per-call timing is captured in the extension, where calls arrive (`_handleMessage`, `extension/ws/mcp-bridge-client.js` ~1144-1199). + - It travels to both recording paths on a hidden symbol property, the same pattern as `MCP_REPLAY_RECORD_CONTEXT` (`extension/ws/mcp-tool-dispatcher.js` ~160). + - It is stored, with a completion time, as an added optional `metadata.timing` field on the journal `tool.call` event. + - It is never added to tool responses and never written to `chrome.storage.local` on each call. + - Do not compute call time from journal event timestamps: they include IndexedDB write-queue delay. + - Do not put timing on the `beginCall` identity: `tests/mcp-session-recorder-journal.test.js:92` deep-equals it. +- **D-05:** Autopilot records, for each iteration, model time (from `apiCallStartTime`, `extension/ai/agent-loop.js` ~1936) and each tool's time (the span between the `BEFORE_TOOL_EXECUTION` and `AFTER_TOOL_EXECUTION` hooks). Both must survive `automationLogger.saveSession` and the `sanitizeActionHistoryEntry` allowlist. A value that was not measured is null or omitted, never 0. Today `createActionEvent.durationMs` defaults to 0 and is dropped on save. +- **D-06:** The headline metric is wall time per completed task, measured by the benchmark runner's own clock. Session `endTime` is never the metric: + - `closeIdleRun` (60 s idle alarm, `mcp-lattice-journal.js` ~1530) sets `endTime` late. + - `saveSession` resets `endTime` on every save. + - Runs closed with `idle_timeout` are flagged, and their idle tail is excluded. + +#### Session detail and operator surface +- **D-07:** `handleGetSessionMessageRoute` gets a journal-first branch, mirroring the control-panel lookup at `extension/background.js` ~12496-12536. Lookup order: + 1. `globalThis.FsbMcpLatticeJournal`: `hasSession`, then `getSessionDetail` + 2. the legacy `loadSession` + 3. in-flight sessions + + The branch is skipped when the journal module is missing, which `tests/mcp-in-flight-session-lookup.test.js` requires. `format:'text'` goes to the journal's `exportHumanReadable`. +- **D-08:** The response keeps the existing `{success:true, session:{...}}` envelope. + - Journal `tool.call` events become a sanitized `session.actionHistory` with timing fields added. The control panel already does this conversion (`extension/ui/options.js` ~3444-3476). + - Raw journal events, which carry the full request and result payloads, are never returned. + - `get_session_detail` gains no new parameters: MCP-04 allows only `expect`. This overrides the research suggestion of `afterSequence`/`limit`. +- **D-09:** Operators see timing in: + - `get_session_detail` JSON + - its text export, which gains per-call milliseconds + - the benchmark report + + The control-panel session viewer does not change. (User: 5a.) + +#### Tokens and cost +- **D-10:** Autopilot cost comes from the provider when it reports one. For xAI that is `usage.cost_in_usd_ticks` (10^10 ticks = $1; when streaming, only the final chunk has it). + - The price-table figure becomes a labelled fallback estimate. Each iteration records `usageSource`/`costSource`. + - Session and baseline records mark values "unavailable" or "estimate" instead of inventing them at these three places: + - `usage.input || 0` (`agent-loop.js` ~2007) + - unknown models priced as Grok (`cost-tracker.js` ~85-88) + - `totalCost || 0` (`automation-logger.js` ~884) + - Reasoning tokens are included in cost estimates. They are reported separately from `completion_tokens` (`completion_tokens_details.reasoning_tokens`) and billed at the output rate. +- **D-11:** Update the Grok rows of the `extension/ai/cost-tracker.js` price table. xAI retired `grok-4-1-fast` on 2026-08-15, and that name is now served by grok-4.3 at $1.25 input / $2.50 output per million tokens. The table still says $0.20 / $0.50. (User: 4a.) +- **D-12:** For MCP sessions, FSB marks the calling host's model time, tokens and cost as unavailable. Caller gap is labelled an upper bound on the host's thinking time. Host metrics come only from the benchmark driver: + - **Claude Code:** the `result` event supplies `duration_ms`, `duration_api_ms`, `num_turns`, `usage` and `modelUsage`. `total_cost_usd` is labelled "list-price estimate, not billed" when `apiKeySource` is `none` or `costBasis` is `list`. + - **Cursor:** reports duration only; tokens and cost are "unavailable (host does not report)". + - v0.9.91 made the same call for agent dollar cost (`usd = null`, `extension/utils/delegation-event-store.js` ~569). + +#### Benchmark suite +- **D-13:** The benchmark is plain Node scripts in a new `scripts/bench/`: a runner, a report, statistics helpers, and a fixture server on 127.0.0.1 with local fixture pages. + - It runs only on request, never in `npm test` or CI. + - It adds no dependencies. The MCP client comes from `mcp/node_modules`, with the SDK kept at 1.29.0. + - Precedent for a fixture server plus Chrome as an opt-in test: `tests/screenshot-uat.test.js`. +- **D-14:** How each arm is driven: + - **MCP arm:** scripted headless Claude Code, launched from `scripts/bench/` with the pinned v0.9.91 flags (`mcp/src/agent-providers/claude-profile.ts` ~179-207). Nothing under `mcp/` changes (D-01). + - **Optional Cursor arm:** scripted with `cursor-agent -p --output-format stream-json`. + - **Autopilot arm:** `run_task` from an SDK stdio client, with an API-kind provider selected. An agent-kind provider returns a consent response before the autopilot loop runs (`background.js` ~13639-13647). +- **D-15:** Tasks: a core set on local fixture pages stays unchanged through Phase 71, and only this set is used for the release comparison. A small live-site set (UAT- and Jira-style reading) is reported separately and does not count toward the release gate. (User: 2, recommended.) +- **D-16:** Size: about 12 core tasks (6 mixed, 6 click-heavy) × 10 alternating runs per arm × 2 baseline arms (autopilot, Claude Code MCP), roughly 240 runs. The baseline must collect enough for Phase 71's method: + - Headline: the geometric mean of per-task median ratios, with a hierarchical bootstrap CI (resample tasks, then runs within each task). + - Per-task CIs are descriptive only. + - A CI that crosses 1.0 is reported as "inconclusive". + - Five runs per task is a smoke test only. + + (User: 3, recommended.) +- **D-17:** Success is decided by code, from evidence independent of both FSB and the agent: + - Fixture tasks: the fixture server's log of submitted requests, the final URL, and DOM markers. + - Live tasks: URL and text checks after the timed window. + - Agent claims are stored as "claimed" only and never count as proof: autopilot `end_turn`/DONE, `complete_task`, and `run_task` success, including `partial_outcome:'timeout'`. +- **D-18:** Each run records its identity: + - git SHA + - extension manifest version and the ID of the extension actually loaded + - MCP server version and SDK version + - provider kind, requested model, and the model string the provider returned + - host name, version and model + - host tool-loading mode. For Claude Code: whether `init.tools` contains `ToolSearch` or `WaitForMcpServers`, plus the `ENABLE_TOOL_SEARCH` value it was launched with. + - task ID and version, and a hash of the fixture content + - the MCP tool count and a hash of the `tools/list` payload, read by the runner through its own SDK client. The tool list changes in Phases 66.1 and 69, so later comparisons must account for it. + + In the extension, fix `_handleGetConfig` (`mcp-bridge-client.js` ~2702). It currently reads keys that nothing writes. It should return the keys that exist (`providerKind`/`modelProvider`/`modelName`, `extension/config/config.js` ~18-21) plus the manifest version. The change is read-only, and `fsb://config` is a resource, not a tool. +- **D-19:** The autopilot baseline pins an explicit current xAI model ID instead of the retired `grok-4-1-fast` name, and records the model string xAI returns. Changing FSB's product default model is not part of this phase. (User: 4a.) +- **D-20:** The runner saves each run's detail to disk as soon as the run ends. Before any new recording, it exports the 40 recorded MCP sessions from 2026-09-16..21. History keeps 50 sessions per mode and the journal keeps 30 days, so those sessions would be pruned around 2026-10-16..21, sooner as benchmark runs push them out. +- **D-21:** Raw session exports stay local and out of git, because they can contain page content from real sites. Only sanitized per-run metrics, identity and aggregates are committed as the baseline. + +### Claude's Discretion +- Autopilot timing capture: a new hook module (e.g. `extension/ai/hooks/timing-hook.js`) or inline stamps. Weigh the tripwires: + - A new module raises the pinned `importScripts` count (333, `tests/lattice-provider-bridge-smoke.test.js:634-635`). + - Editing `agent-loop.js` touches pinned counts such as the 8 `setTimeout` tokens and fixed anchor text. +- The exact xAI model ID to pin (a current grok-4.3 variant), confirmed during research. +- Where the committed baseline artifacts live (`scripts/bench/results/` or the phase directory), within D-21. +- The exact fixture tasks, the internal layout of the benchmark scripts, and the report format. +- Whether the text export shows per-call timing as columns or inline. + +### Deferred Ideas (OUT OF SCOPE) +- **MCP settings tool.** Shows all control-panel settings compactly and sets any one of them, including the Jev fast-mode toggle once Phase 68 adds it. + - Reads hide secrets. + - Writes may change any setting except secrets and credentials; the user chose this over an allowlist. + - The user approved this new tool on 2026-09-29, and it is inserted as **Phase 66.1**. MCP-04 and the out-of-scope entry against a second user-facing fast-mode switch are amended to match. +- **FSB's default autopilot model has been retired.** Requests for `grok-4-1-fast` are now served by grok-4.3, at about 5-6× the cost FSB assumes. Updating the product default is a separate fix; Phase 66 only pins the model the benchmark uses (D-19). +- **MCP-server-side timestamps** (`serverReceivedAt`, measured `TaskQueue` wait). Add these later only if runs with concurrent agents need queue wait separated from caller gap (D-03). +- **Per-call timing columns in the control-panel session viewer.** Not planned (D-09). + + + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|------------------| +| MEAS-01 | An operator can inspect per-call wall time, FSB execution time, model time, tokens, and cost for both MCP and autopilot sessions, with unavailable values labelled rather than invented. | MCP ingress stamp + journal `metadata.timing` (Pattern 1); autopilot `iterationTiming`/`durationMs` inline stamps (Pattern 4); provider-first usage/cost with labels (Pattern 5); explicit `measurement` projection that survives the `/token/i` key filter (Pitfall 1); host metrics from the driver only (Pattern 8). | +| MEAS-02 | An operator can retrieve session details for a journal-backed MCP session, including its tool calls and timing records. | Journal-first branch in `handleGetSessionMessageRoute` with paging, `actionHistory` projection built through the existing `sanitizeActionHistoryEntry`, text export through `exportHumanReadable` (Patterns 2-3); vm-harness guard (Pitfall 4). | +| MEAS-03 | An operator can run a repeatable, outcome-checked task suite and save a baseline of completion rate, wall time per completed task, and cost before enabling Jev. | `scripts/bench/` runner, fixture server with request-log evidence and post-window CDP checks (Pattern 7), run identity via the fixed `fsb://config` + runner's own SDK client (Pattern 6), hierarchical bootstrap helper (Pattern 9), history export of the 40 sessions (Pattern 10), bench topology preflight and human checkpoints (Plan Decomposition). | + + +## Summary + +All three requirements are reachable with extension-only changes plus a new `scripts/bench/` tree, exactly as D-01 prescribes. Every integration point named in CONTEXT.md was re-read at its current line numbers (drift was small; see the Integration Point Index). Four findings from this session change how the work must be done and are not in CONTEXT.md or the prior research: + +1. **`get_session_detail` strips every key containing "token".** `sanitizeValue` drops keys matching `/password|...|token|.../i` (`mcp-tool-dispatcher.js:2027-2046`). A probe of the current route (run in this session) returned a saved autopilot session with `totalInputTokens`, `totalOutputTokens`, `inputTokens`, `outputTokens` and `reasoningTokens` all removed, and `actionHistory[].durationMs` removed by the allowlist. Tokens are therefore invisible over MCP today, and any new field named `*Tokens` would silently vanish. MEAS-01 needs an explicit, post-sanitization `measurement` projection, and stored field names should avoid the substring "token". +2. **FSB's price table misprices the pin target.** `estimateCost('grok-4.3', 1e6, 1e6)` returns **$18.00** today because the substring fallback matches the legacy `grok-4` row ($3/$15); the true list price is $3.75. Unknown models such as `claude-sonnet-4-6` (which FSB's own model list offers) are priced as Grok ($0.70). D-11 must add exact `grok-4.3` rows ahead of the legacy keys, and the benchmark must prefer `usage.cost_in_usd_ticks`. +3. **The autopilot request path is non-streaming and returns the raw xAI body unchanged** (`offscreen/lattice-host.js:878-906` → `lattice-provider-bridge.js:235` → `agent-loop.js:1954`). `usage.cost_in_usd_ticks`, `usage.completion_tokens_details.reasoning_tokens` and `response.model` are already on the `response` object in the loop; no request change is needed. +4. **Cross-agent verification through FSB is impossible by design**, so D-17 evidence must come from outside FSB: `list_tabs` returns only the caller's own tabs (`mcp-tool-dispatcher.js` `handleListTabsRoute`), and the runner's SDK client is a different agent from Claude's. Fixture evidence therefore comes from the fixture server's request log, and DOM markers / final URL / tab cleanup come from a post-window CDP check against a dedicated bench Chrome profile. + +The live benchmark is the riskiest part, and the risk is operational rather than code: the worktree extension must be side-loaded (the everyday Chrome runs the Web Store build `badgafnf…`), bridge pairing is pinned to one extension origin in `~/.fsb/bridge-auth.json`, the FSB native host manifest allows only the Web Store origin, the extension hard-codes hub port 7225, and branded Chrome 137+ ignores `--load-extension`. The recommended topology is a dedicated `--user-data-dir` bench profile with its own `NativeMessagingHosts/` manifest copy, a runner-owned hub on 7225, and explicit human checkpoints for loading, pairing and restoring. + +**Primary recommendation:** Build in four parallel code tracks (MCP timing capture; session-detail + measurement projection; autopilot timing + provider cost; bench helpers + fixtures), keep all timing out of tool responses and out of per-call `chrome.storage.local` writes, add no service-worker module (inline stamps, zero `importScripts` churn), pin `grok-4.3`, and gate the live campaign behind a runner preflight that refuses to run unless `fsb://config` proves the Phase 66 build, model and topology. + +## Project Constraints (from CLAUDE.md, STATE.md and operator memory) + +There is no `./CLAUDE.md` (it is gitignored under "Development files"). Actionable constraints that carry the same weight: + +| Source | Directive | +|--------|-----------| +| `~/.claude/CLAUDE.md` (user global) | Never add "Co-Authored-By" or AI attribution lines to commits, PR descriptions or pushes. | +| STATE.md INV-01 | MCP wire is byte-stable; additions are optional fields only. | +| STATE.md INV-03 | `universal-provider.js` sees only the 7 API-kind providers. Do not route benchmark or Jev logic through it. | +| STATE.md INV-04 | `setTimeout`-chained iterator preserved; `agent-loop.js` must keep exactly 8 `setTimeout` tokens (comments count) and the 4 schedule lines byte-identical. | +| STATE.md tripwire discipline | Every extension-touching commit updates its paired source-pin tests in the same commit; full suite green from the phase's first commit. | +| STATE.md same-context dispatch | Inside the SW use `globalThis.fsbDispatchInternalMessage` (typeof-guarded), never `chrome.runtime.sendMessage`, to reach background handlers. | +| Operator memory: test chain | `npm test` is one `&&` chain; it halts at the first port-bound suite if 7225 is held. Free the hub and stray `mcp/build/index.js` relays before a full run. | +| Operator memory: CI Node 20 | CI runs Node 20 (`.github/workflows/ci.yml:22`); root `engines` says ≥24. CI-run tests must avoid Node 22+ globals (`WebSocket`, `Object.groupBy`, `Promise.withResolvers`, `crypto.hash`, `fs.globSync`, `process.getBuiltinModule`). | +| Operator memory: minimal MCP diff | Touch `mcp/src` only when a tool schema must change (none in this phase). | +| Operator memory: pre-existing failures | On clean main, `tests/mcp-lifecycle-smoke.test.js` (in `test:mcp-smoke`) fails 0/5, and 6 unscripted tests fail. Not Phase 66's fault; do not "fix" them in this phase. | +| Operator memory: MCP driving | Visual-session `client` labels must come from the badge allowlist ("Claude", not "claude-code"); never pass device-emulation args to `capture_screenshot` in the suite (wedges the per-tab CDP lease). | +| `tests/delegation-phase-contract.test.js:129-155, 2373-2391` | New root test commands must be appended to the `package.json` `test` chain **and** listed in `REFINEMENT_ROOT_TEST_COMMANDS`; never edit the `PRE_PHASE6x_ROOT_TEST_HASH` values. | +| `tests/mcp-client-identity-integration.test.js:469-486` | Append new chain entries at the tail; do not reorder existing ones. | +| `.gitignore` | Global `build/`, `dist/`, `*.log` ignores exist; do not name bench output dirs `build` or `dist`. | + +## Architectural Responsibility Map + +| Capability | Primary Tier | Secondary Tier | Rationale | +|------------|-------------|----------------|-----------| +| Per-call MCP timing capture (receive, completion) | Extension SW (`mcp-bridge-client.js` ingress, dispatcher recorder hook) | — | D-01/D-03: single machine clock at the extension's MCP ingress; no server stamps. | +| Timing persistence | Extension storage: Lattice journal (IndexedDB) `tool.call.metadata.timing` | Legacy recorder → `fsbSessionLogs` via `saveSession` (legacy-only clients) | Append-only IDB; never per-call `chrome.storage.local` writes (D-04). | +| Caller gap / host-think upper bound | Projection time (dispatcher projection, journal text export, bench report) | — | Derived, never stored; depends on neighbouring calls. | +| Autopilot model/tool time, usage, cost labels | Extension SW (`agent-loop.js` inline stamps, `cost-tracker.js` pure helpers) | `automation-logger.js` persistence | The loop already owns `apiCallStartTime` and the tool bracket; the logger owns durable history. | +| Session detail over MCP (MEAS-02) | Extension SW dispatcher route | MCP server passes through unchanged | D-01: no `mcp/` change; the tool already forwards `{sessionId, format}`. | +| Run identity (`fsb://config`) | Extension SW `_handleGetConfig` | MCP resource passes through (`mcp/src/resources/index.ts:138-162`) | Read-only allowlisted projection. | +| Wall clock per task, outcome evidence, statistics | Benchmark runner (Node, `scripts/bench/`) | Fixture server (Node http), CDP post-window check | D-06, D-17: independent of FSB and the agent. | +| Host (Claude Code/Cursor) model time, tokens, cost | Benchmark driver (stream-json parser) | — | D-12: FSB cannot observe host internals. | + +## Standard Stack + +### Core (all already present; nothing to install) + +| Library / Runtime | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| Node.js built-ins (`node:http`, `node:crypto`, `node:child_process`, `node:test`, `node:assert`, `node:fs`) | Node 24.14.1 local, Node 20 in CI [VERIFIED: `node --version`, `.github/workflows/ci.yml:22`] | Fixture server, hashing, spawning `claude`, unit tests | D-13: no dependencies; matches `tests/screenshot-uat.test.js` precedent. | +| `@modelcontextprotocol/sdk` (from `mcp/node_modules`) | 1.29.0 [VERIFIED: `mcp/package.json` dependency pin; installed copy at `/opt/homebrew/lib/node_modules/fsb-mcp-server/node_modules/@modelcontextprotocol/sdk/package.json` = 1.29.0] | Runner's own client: `Client`, `StdioClientTransport` (`client/stdio.js`), `listTools`, `readResource`, `callTool`, `getServerVersion` | D-13; already the product's SDK. Resolve with `createRequire(path.join(repo,'mcp','package.json'))`, following `tests/mcp-bridge-topology.test.js:11`. | +| `ws` (from `mcp/node_modules`) | as locked in `mcp/package-lock.json` | CDP client in the runner (post-window checks, tab cleanup) | Node 20-safe alternative to the global `WebSocket`; precedent `tests/mcp-smoke-harness.js:10`. | +| Claude Code CLI | 2.1.283 installed [VERIFIED: `claude --version`] | MCP arm host | D-14. | +| cursor-agent CLI | 2026.09.08-6caf4ff installed [VERIFIED: `cursor-agent --version`] | Optional Cursor arm | D-14 (optional). | +| esbuild (root devDependency, via `npm run build`) | as locked | Builds `extension/dist/offscreen/lattice-host.js`, which the offscreen host loads (`extension/offscreen/lattice-host.html:35`) | Required before side-loading the worktree extension; `extension/dist/` is gitignored. | + +### Supporting + +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| Google Chrome (branded) | installed; version recorded by the runner from CDP `/json/version` | Bench browser with a dedicated `--user-data-dir` and `--remote-debugging-port=0` | Always for the live campaign. | +| Chrome for Testing / Chromium | not installed | Would allow `--load-extension` automation | Not recommended in this phase (adds a download step; D-13 spirit). | + +### Alternatives Considered + +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| Inline timing stamps in `agent-loop.js` | New `extension/ai/hooks/timing-hook.js` | A hook module adds +1 to both `importScripts` pins (333/329) and a registration edit, but `agent-loop.js` must still change (the `AFTER_API_RESPONSE` context lacks model time and the provider usage fields), so the module saves nothing. Recommend inline. | +| Fixture-server request log + post-window CDP | FSB read tools from the runner's own agent | FSB read tools are agent-scoped (`list_tabs` returns only owned tabs) and are not independent of FSB (violates D-17). | +| An in-page telemetry probe posting DOM state | Server-visible effects of the fixture app itself | A probe that sends requests on DOM mutations would lengthen FSB's network-quiet settle wait (a click waits for 200 ms of network quiet) and distort FSB execution time. | +| Dedicated bench Chrome profile | Side-load into the everyday Profile 2 | Profile 2 carries real browsing state (other tabs/extensions = timing noise) and its IndexedDB holds the 40 sessions; a separate profile also keeps bench recordings from evicting them. Profile 2 is only attractive for logged-in live-site tasks. | + +**Installation:** none. Environment setup for a fresh worktree (this worktree has no `node_modules` at all [VERIFIED: `ls`]): + +```bash +npm install --no-audit --no-fund # root (esbuild, tsx, test deps) +npm --prefix mcp ci --no-audit --no-fund # MCP SDK 1.29.0, ws, zod 3 +npm --prefix mcp run build # full mcp/build (only 6 files are tracked) +npm --prefix showcase/server install # server-telemetry suites in the npm test chain +npm run build # extension/dist/offscreen/lattice-host.js +``` + +## Package Legitimacy Audit + +This phase installs **no external packages**. The benchmark imports `@modelcontextprotocol/sdk` 1.29.0 and `ws` from the existing `mcp/package-lock.json`, which CI already installs with `npm --prefix mcp ci`. slopcheck was not run because there is nothing new to vet. + +| Package | Registry | Age | Downloads | Source Repo | slopcheck | Disposition | +|---------|----------|-----|-----------|-------------|-----------|-------------| +| (none new) | — | — | — | — | — | — | + +**Packages removed due to slopcheck [SLOP] verdict:** none +**Packages flagged as suspicious [SUS]:** none + +## Architecture Patterns + +### System Architecture Diagram + +``` + ┌────────────────────── benchmark runner (Node, scripts/bench) ──────────────────────┐ + │ preflight ─► identity (git SHA, tree hash, fsb://config, tools/list hash) │ + task JSON ──► scheduler │ ABBA-alternating runs ─► t0 ─► arm driver ─► t1 ─► evidence ─► outcome check │ + (core/live) │ │ ▲ │ │ + │ │ │ ▼ │ + │ │ fixture server per-run JSON (raw, │ + │ │ request log gitignored) + metrics │ + │ │ + CDP post-window (committed, sanitized)│ + └──────────────────────────────────┼─────────────────────────────────────────────────┘ + │ + ┌───────────────────────────────┬───────────────────┴───────────────────────┐ + ▼ MCP arm ▼ autopilot arm ▼ observability (runner client) + claude -p (pinned v0.9.91 flags) runner SDK client: open_tab(startUrl) list_sessions / get_session_detail / + stream-json ─► init/result events then run_task(task) [timed] readResource(fsb://config) / listTools + │ stdio │ stdio │ + ▼ ▼ ▼ + FSB MCP server (relay) ──────────► FSB MCP server = HUB on 127.0.0.1:7225 (runner-owned, workspace build) + │ ws (bridge-auth pinned to the unpacked extension origin) + ▼ + Chrome bench profile ─► unpacked worktree extension SW + _handleMessage: [stamp receivedAt on hidden Symbol] ─► route ─► execute (+ change_report settle) + ─► recorder entry.timing {receivedAt, completedAt, fsbMs} ─► journal tool.call.metadata.timing (IndexedDB) + autopilot loop: apiCallStart ─► provider (offscreen fetch, non-streaming xAI) ─► usage/cost labels + ─► tool bracket (after permission check) ─► actionHistory.durationMs + iterationTiming ─► saveSession + get_session_detail: journal-first ─► paged tool.call projection + explicit measurement block ─► JSON / text + │ + ▼ + fixture pages on http://127.0.0.1:/r//... + (form POSTs / save fetches logged server-side = primary evidence) +``` + +### Recommended Project Structure + +``` +scripts/bench/ +├── run-bench.mjs # CLI: preflight, schedule, drive arms, save per-run detail immediately +├── report.mjs # CLI: aggregate runs → summary.json + report.md (+ A/A noise floor) +├── export-history.mjs # CLI: turn the raw 40-session export into sanitized per-call metrics +├── fixture-server.mjs # node:http on 127.0.0.1:0; /r// prefix; request log; evidence endpoint +├── lib/ +│ ├── stats.mjs # median, quantile, wilson, mulberry32, hierarchical bootstrap (pure) +│ ├── outcome.mjs # evaluateOutcome(task, evidence) (pure) +│ ├── tasks.mjs # loadTasks + validateTask (pure schema check) + fixture hashing +│ ├── identity.mjs # canonicalJson, sha256, toolsListDigest, treeHash (pure + fs) +│ ├── claude-arm.mjs # argv builder mirroring claude-profile.ts; NDJSON stream summarizer (pure parts) +│ ├── cursor-arm.mjs # optional +│ ├── mcp-client.mjs # SDK client from mcp/node_modules; run_task with progress-reset timeout +│ └── cdp.mjs # minimal CDP client over mcp/node_modules/ws (post-window checks, cleanup) +├── tasks/core/*.json # 12 frozen tasks (6 mixed, 6 click-heavy), versioned +├── tasks/live/*.json # small live set, reported separately +├── fixtures//... # static HTML/JS fixture apps +├── results// # COMMITTED: runs.jsonl (sanitized), summary.json, report.md, identity.json +└── .local/ # GITIGNORED: raw per-run JSON, stream-json logs, session exports +tests/ +├── bench-stats.test.mjs, bench-outcome.test.mjs, bench-tasks.test.mjs, +├── bench-identity.test.mjs, bench-claude-stream.test.mjs, bench-fixture-server.test.mjs # Node 20-safe, in the npm test chain +``` + +Keep baseline artifacts in `scripts/bench/results/`, not the phase directory: Pitfall 22 of the prior research notes that tests reading live `.planning/phases/66-*` files break when the milestone is archived, and Phase 71 needs a stable path. + +### Pattern 1: Hidden-symbol call timing at MCP ingress (D-04) + +**What:** Stamp `receivedAt` on the parsed payload under a registry Symbol at `_handleMessage`, read it where recorder entries are built, and pass it to the recorder as an explicit `timing` field. The Symbol never serializes (JSON, `structuredClone` and non-enumerable spread all drop it — verified in this session). + +**When to use:** Every recorded MCP call (bridge action tap and dispatcher message-route recorder, plus terminal task-status outcomes). + +**Rules:** +- Use `Symbol.for('fsb.mcp.call-timing')` so `mcp-bridge-client.js` and `mcp-tool-dispatcher.js` share the symbol without load-order coupling, and so `vm` harnesses see the same symbol (`Symbol.for` is shared across realms — verified). Give each file's top-level `const` a **different identifier**: both files declare top-level `const`s in the shared classic-script global lexical scope, so a duplicate name throws on the second `importScripts`. +- Use `Date.now()` only. `performance` is **undefined** in every `vm` harness that loads these files (verified); guard it with `typeof` or avoid it. +- Define the property with `Object.defineProperty(..., { enumerable: false, configurable: true })` exactly like `mcp-tool-dispatcher.js:3184-3197`. +- `canonicalizeVisualSessionOwnershipToken` returns `{...payload, ownershipToken}` (`mcp-tool-dispatcher.js:576-579`), which drops a non-enumerable Symbol. Read the timing from the **original** payload at the top of `dispatchMcpToolRoute`, before canonicalization, and pass it explicitly to `recordTaskOutcome`. +- Stamp `completedAt` at the recorder call site, synchronously, before the fire-and-forget enqueue. The recorder queue and the journal write queue add delay; never derive call time from `event.timestamp` (D-04). + +```js +// mcp-bridge-client.js (top level; unique identifier) +const MCP_BRIDGE_CALL_TIMING = Symbol.for('fsb.mcp.call-timing'); + +// in _handleMessage, right after `if (!id || !type) return;` +if (payload && typeof payload === 'object') { + try { + Object.defineProperty(payload, MCP_BRIDGE_CALL_TIMING, { + value: { receivedAt: Date.now() }, enumerable: false, configurable: true + }); + } catch (_e) { /* timing never gates routing */ } +} + +// shared shape builder (one per file, or a small pure helper in the dispatcher) +function buildMcpCallTiming(source) { + const stamp = source && source[MCP_BRIDGE_CALL_TIMING]; + if (!stamp || !Number.isFinite(stamp.receivedAt)) return null; // absent -> omitted, never 0 + const completedAt = Date.now(); + return { v: 1, receivedAt: stamp.receivedAt, completedAt, + fsbMs: Math.max(0, completedAt - stamp.receivedAt) }; +} + +// _recordMcpSessionAction: add to sessionRecordEntry +// timing: buildMcpCallTiming(payload), +``` + +Recorder and journal changes that carry it through (all additive; omit the key when timing is null so existing byte-cost and deep-equal tests are unaffected): +- `recordAction(input)` builds a **new** entry (`mcp-session-recorder.js:2336-2352`); forward `timing: input.timing`. +- `_snapshotJournalEntry` returns an explicit field list (`:918-966`); add `timing: _sanitizeCallTiming(entry.timing)` (finite non-negative numbers only). +- `recordTaskOutcome` (`:1751-1800`) builds its journal entry through `_snapshotJournalEntry`; pass timing into it. +- Legacy path: `_snapshotEntry` (`:854-880`) and `session.actionHistory.push` (`:2146-2152`) gain `timing`/`durationMs` so D-04's "both recording paths" holds for legacy-only clients. +- Journal `recordDispatchNow` metadata (`mcp-lattice-journal.js:1210-1229`): `if (timing) metadata.timing = timing;`. Replay is unaffected because `replayEntryFromEvent` (`:1752-1766`) copies an explicit field list that does not include metadata extras [VERIFIED: codebase]. +- `INTERNAL_PAYLOAD_KEYS` (`:44-48`) and `RECORDING_SIDECAR_FIELDS` (`tests/mcp-tool-smoke.test.js:58`) **do not change**: the timing never becomes a string key on any payload, `stripInternalPayload` clones via `structuredClone`/JSON (`:76-86`, `:821-834`) which drops Symbol keys, and the server-side sidecars are untouched (D-01). + +### Pattern 2: Journal-first session lookup with paged projection (D-07, D-08) + +```js +// mcp-tool-dispatcher.js handleGetSessionMessageRoute, after the sessionId check, +// BEFORE the automationLogger availability check (so journal sessions resolve even without it) +const journal = (typeof globalThis !== 'undefined') ? globalThis.FsbMcpLatticeJournal : null; +if (journal && typeof journal.hasSession === 'function' && typeof journal.getSessionDetail === 'function' + && await journal.hasSession(payload.sessionId)) { + if (payload.format === 'text') { + if (typeof journal.exportHumanReadable !== 'function') { + return createMcpRouteError('get_session_detail', 'observability', MCP_ROUTE_RECOVERY_HINT, + { error: 'Human-readable session exporter unavailable' }); + } + return { success: true, format: 'text', text: await journal.exportHumanReadable(payload.sessionId) }; + } + const collected = await collectJournalToolCalls(journal, payload.sessionId); // pages 500 at a time, bounded + return { success: true, session: projectJournalSessionForMcp(collected) }; +} +// then the existing automationLogger.loadSession branch, then the in-flight fallback, unchanged +``` + +- `getSessionDetail` caps a page at 500 events (`mcp-lattice-journal.js:24-25, 1631-1648`); `sanitizeSessionDetail` keeps the last 100 actions (`mcp-tool-dispatcher.js:2108`). Page with `afterSequence = nextSequence` until `hasMore` is false, keep only compact projected `tool.call` rows in a ring of 100, and keep totals over all calls. Bound the loop (for example 20 pages) and set `actionHistoryTruncated`/`callsTotal` so truncation is visible. The MCP tool's bridge timeout is 10 s (`mcp/src/tools/observability.ts:56-58`). +- Build each row by feeding a pseudo-action `{tool, timestamp: Date.parse(event.timestamp), iteration: null, params: , result: }` through the **existing** `sanitizeActionHistoryEntry`, then attach `sequence`, `route`, `logicalTab` and `timing`. This reuses the allowlist (selector ≤250 chars, domain only, error ≤500 chars) and guarantees raw payloads never leave. Descriptors stored as `artifact` or `omitted` contribute nothing. +- Session metadata: `sanitizeSessionMetadata(detail.session)` on the `buildIndexEntry` shape (`id`, `task`, `status`, `startTime`, `endTime`, `actionCount`, `eventCount`, `mcpClient`, `storageBackend: 'journal-v2'`, `recordingState`, `outcome`, `outcomeDetails`, ...). Keep the `{success:true, session:{...}}` envelope (D-08). +- Mirror of the control panel: `hydrateJournalSessionDetail` (`extension/ui/options.js:3443-3476`) does the same field mapping; do not reuse its `journalDescriptorPreview`, which returns inline payload values. + +### Pattern 3: Explicit measurement projection (survives the key filter) + +Build a `measurement` block **after** the generic sanitizer and attach it to the sanitized session. Delete `iterationTiming`/`measurementSummary` from the generic output first (the generic sanitizer would already have stripped their token fields and flattened anything at depth 4 to `'[object]'`, `mcp-tool-dispatcher.js:2041`). + +```js +// MCP (journal) sessions +measurement: { + v: 1, kind: 'mcp', + calls: 37, callsWithTiming: 37, + fsbMs: { total: 21340, median: 512, p90: 1210 }, + callerGapMs: { total: 301220, median: 7110, label: 'upper bound on host thinking time' }, + activeSpanMs: 322560, // lastCompletedAt - firstReceivedAt; never endTime - startTime (D-06) + endReason: 'idle_timeout', // from outcomeDetails.reason; idle tail excluded + queueWaitMs: null, queueWaitStatus: 'not_measured', // D-03 + host: { modelMs: null, inputTokens: null, outputTokens: null, costUsd: null, + status: 'unavailable', reason: 'host does not report to FSB' } // D-12 +} +// autopilot sessions +measurement: { + v: 1, kind: 'autopilot', + iterations: 9, modelMs: { total: 23110, median: 2380 }, toolMs: { total: 6120, median: 410 }, + inputTokens: 184233, outputTokens: 2310, reasoningTokens: 5120, cachedInputTokens: 90112, + usageSource: 'provider', + costUsd: 0.2481, costSource: 'provider', // 'provider' | 'estimate' | 'mixed' | 'unavailable' + requestedModel: 'grok-4.3', servedModels: ['grok-4.3'], + perIteration: [ /* last 100 flat rows */ ] +} +``` + +Add `durationMs` to `sanitizeActionHistoryEntry` **only when finite** (`...(Number.isFinite(action?.durationMs) ? { durationMs: action.durationMs } : {})`) so unmeasured values are omitted, not 0 (D-05). No test deep-equals sanitized action entries [VERIFIED: grep]. + +### Pattern 4: Inline autopilot stamps (recommended over a hook module) + +- **Model time:** `apiCallStartTime` exists at `agent-loop.js:1936`. Take `var apiCallElapsedMs = Date.now() - apiCallStartTime;` immediately after the provider call(s) resolve, next to the existing debug log at `:1986-1992`. It includes the one text-only image-rejection retry when that fires (`:1957-1979`); record `modelRetried: true` in that case rather than pretending it is one call. +- **Tool time:** the `BEFORE_TOOL_EXECUTION` emit (`:2260-2293`) runs the permission check. Today only the synchronous `permHook` is registered on that event (`background.js:4577`; `hooks/permission-hook.js:38-51` returns immediately), but Phase 68 may add confirmation work there, so start the stamp **after** it returns non-denied (just before `session._currentStepName = 'BEFORE_TOOL_EXECUTION'` at `:2298`) and end it at the successful action-history push (`:2797-2812`), which happens just before `AFTER_TOOL_EXECUTION` (`:2816`). Denied (`:2268-2291`) and malformed-argument (`:2229-2251`) branches never executed a tool: leave `durationMs` absent there. +- `ActionHistory.push` stores the object **as-is** when it has `tool` and `timestamp` (`action-history.js:91-100`), so adding `durationMs: Date.now() - toolStartedAt` to the object literal at `:2799-2803` is sufficient on the normal path. `createActionEvent`'s `durationMs: f.durationMs || 0` (`:51`) only runs on the fallback path; changing that default to `null` is honest but trips `tests/action-history.test.js:67` (paired update in the same commit). +- **Per-iteration record:** push one flat row into `session.iterationTiming` (keep the last 100) after the tool loop and on the `end_turn` path, and keep running totals in `session.measurementSummary`. Store usage as `usage: { input, output, reasoning, cachedInput }` (no key contains "token"). +- **Testability:** no test runs `runAgentIteration` end to end; existing agent-loop tests call only the exported `callProviderWithTools` with a stubbed `executeViaBridge` (`tests/openai-agent-request.test.js:43-56`, `tests/agent-loop-empty-contents.test.js:36-41`). Put the logic in pure helpers so it can be unit-tested: usage/cost readers in `cost-tracker.js`, and a row builder/appender (for example `appendIterationTiming(session, row, cap)`) either in `cost-tracker.js` or exported additively from `agent-loop.js` (its `module.exports` at `:3131-3133` has no test pinning its key list [VERIFIED: grep]). The agent-loop edit itself then stays a few call sites, which the pin suites cover. +- **Persistence:** `automation-logger.js` projects explicit fields in both branches: the existing-session branch (`:863-920`, action rows `:904-916`) and the new-session branch (`:921-976`, action rows `:963-973`). Add `iterationTiming` (capped copy), `measurementSummary`, and `durationMs: Number.isFinite(a.durationMs) ? a.durationMs : null` to the action-row map in **both** branches. `persistSession` (`background.js:4958-4992`) persists only stop-button essentials, so no survivability change is needed (sessions are not resumable after SW restart). +- `saveToLogger` runs before completion is broadcast (`agent-loop.js:1637-1662`) but `saveSession` is async; the runner must retry `get_session_detail` briefly after `run_task` resolves. +- Tripwires in `agent-loop.js`: keep 8 `setTimeout` tokens (no new timers, no comment mentioning the word), keep the 4 schedule lines byte-identical, keep one `stepName: 'LLM_TURN'` and one `TOOL_DISPATCH`, and keep the anchor substrings used by `cost-tracker-ordering`, `meta-cognitive-tracker`, `ownership-leakage-regression` and `run-task-cleanup-paths`. `lastTurnResult.durationMs` is session-elapsed time, not model time (`:2111`); do not repurpose it (and leave `tests/turn-result.test.js:50` alone). + +### Pattern 5: Provider-first usage and cost with labels (D-10, D-11) + +The xAI path is non-streaming (`agent-loop.js:1109-1115` builds `{model, messages, tools, temperature:0, max_tokens:4096}`; the offscreen host `fetch`es and returns `await fetchResp.json()` unchanged, `offscreen/lattice-host.js:878-906`; the bridge returns `rawResponse`, `lattice-provider-bridge.js:235`). Everything D-10 needs is already on `response`. + +```js +// cost-tracker.js additions (pure; module.exports guard already exists) +var XAI_TICKS_PER_USD = 1e10; +function readProviderUsage(response, providerKey) { // returns nulls, never invented zeros + var u = response && response.usage; + if (!u) return { input: null, output: null, reasoning: null, cachedInput: null, + providerCostUsd: null, usageSource: 'unavailable', servedModel: (response && response.model) || null }; + var num = function (v) { return Number.isFinite(v) ? v : null; }; + // OpenAI-compatible (xai/openai/openrouter/custom/lmstudio); anthropic/gemini branches mirror extractUsage + return { + input: num(u.prompt_tokens), output: num(u.completion_tokens), + reasoning: num(u.completion_tokens_details && u.completion_tokens_details.reasoning_tokens), + cachedInput: num(u.prompt_tokens_details && u.prompt_tokens_details.cached_tokens), + providerCostUsd: providerKey === 'xai' && Number.isFinite(u.cost_in_usd_ticks) ? u.cost_in_usd_ticks / XAI_TICKS_PER_USD : null, + usageSource: (num(u.prompt_tokens) !== null || num(u.completion_tokens) !== null) ? 'provider' : 'unavailable', + servedModel: response.model || null + }; +} +function estimateCostDetailed(model, usage, provider) { // no Grok fallback: unknown -> unavailable + if ((provider || '').toLowerCase() === 'lmstudio') return { usd: 0, source: 'estimate', pricingKey: 'lmstudio-local' }; + var key = resolvePricingKey(model); // exact first, then prefix/substring over ordered keys + if (!key || usage.input === null || usage.output === null) return { usd: null, source: 'unavailable', pricingKey: key }; + var p = MODEL_PRICING[key]; + var billedOutput = usage.output + (usage.reasoning || 0); // D-10: reasoning billed at output rate + return { usd: (usage.input / 1e6) * p.input + (billedOutput / 1e6) * p.output, source: 'estimate', pricingKey: key }; +} +``` + +- Keep the legacy numeric `estimateCost`/`CostTracker.record` behaviour (including the unknown-model fallback that `tests/cost-tracker.test.js` pins) so `session.totalCost`, the sidepanel and `BackgroundAnalytics` keep working unchanged (D-02). The labelled fields are additive and are what MEAS-01 and the benchmark read. +- D-11 rows: add exact `grok-4.3` and `grok-4.3-latest` at 1.25/2.50 **first in the object** (substring matching iterates insertion order and today `grok-4.3` hits `grok-4`); set every slug xAI now redirects to grok-4.3 (`grok-4-1-fast`, `grok-4-1-fast-reasoning`, `grok-4-1-fast-non-reasoning`, `grok-4-fast-reasoning`, `grok-4-fast-non-reasoning`, `grok-4-0709`, `grok-3`) to 1.25/2.50; set `grok-code-fast-1` and add `grok-build-0.1` at 1.00/2.00 [CITED: docs.x.ai/developers/migration/may-15-retirement; docs.x.ai/developers/pricing]. These are the <200k-token tier; label estimates accordingly. +- Validate the estimate formula live: on each xAI iteration where both are available, record `estimateDeltaPct = (estimate - provider) / provider`. This settles whether `completion_tokens` already includes reasoning tokens and the effect of the cached-input discount (not modelled in the estimate). +- Two other Grok price tables exist and are **not** in D-11's scope: `extension/utils/analytics.js:28-36` (dashboard) and `extension/background.js:10867`. Leave them (the control-panel dashboard is out of scope, D-09); see Open Questions. + +### Pattern 6: Run identity from `fsb://config` (D-18) + +`_handleGetConfig` (`mcp-bridge-client.js:2702-2708`) reads `selectedModel`/`selectedProvider`/`defaultModel`, which nothing writes. Replace it with a read-only allowlist: + +```js +async _handleGetConfig() { + const stored = await chrome.storage.local.get([ + 'providerKind', 'modelProvider', 'modelName', 'maxIterations', 'domOptimization', 'maxDOMElements' + ]); + const config = { + providerKind: typeof stored.providerKind === 'string' ? stored.providerKind : 'api', // same defaults as + modelProvider: typeof stored.modelProvider === 'string' ? stored.modelProvider : 'xai', // fsbReadAuthoritativeProviderConfig + modelName: typeof stored.modelName === 'string' ? stored.modelName.trim() : '', // (background.js:1726-1739) + ...(Number.isFinite(stored.maxIterations) ? { maxIterations: stored.maxIterations } : {}), + ...(typeof stored.domOptimization === 'boolean' ? { domOptimization: stored.domOptimization } : {}), + ...(Number.isFinite(stored.maxDOMElements) ? { maxDOMElements: stored.maxDOMElements } : {}) + }; + let manifest = null; + try { manifest = chrome.runtime && typeof chrome.runtime.getManifest === 'function' ? chrome.runtime.getManifest() : null; } catch (_e) {} + return { config, extension: { + id: (chrome.runtime && chrome.runtime.id) || null, + version: manifest && manifest.version || null, + measurement: { callTiming: 1, sessionDetailJournal: 1 } // proves the Phase 66 build is loaded + } }; +} +``` + +- Never return API keys (the `mcp/src/types.ts:27` comment promises "keys redacted"). Test that no key named like a secret appears. +- The doctor already prefers `modelProvider`/`modelName` (`mcp/src/diagnostics.ts:729-743`), so the fix also makes `doctor --include-config` report a configured model. `normalizeConfig` reads `payload?.config ?? payload`, so a top-level `extension` sibling is harmless. +- The Web Store build and the worktree both report manifest `0.9.91`; only the `measurement` marker (and the extension ID) distinguishes them. The runner must refuse to run when the marker is absent or the ID equals `badgafnfchcihdfnjneklogedcdkmjfk`. +- `getManifest` may be missing in `vm` harness chrome mocks; guard it. + +### Pattern 7: Fixture evidence and post-window checks (D-15, D-17) + +- **Isolation by URL prefix:** each run gets a random token; all fixture pages are served under `http://127.0.0.1:/r//...`. Relative links keep the prefix, so the server attributes every request to its run without cookies. +- **Primary evidence = the app's own requests.** Design every core task so success requires a server-visible effect that is part of normal page operation: a form POST, a "Save" `fetch` with a JSON body, or a navigation to a result page. The server logs `{t, method, path, query, body}` per run. Do not add an in-page probe that sends requests on DOM mutation during the timed window (Pitfall 9). +- **Evidence endpoint:** `GET /__bench/evidence?run=` requires a runner-held secret header, so an agent calling `fetch` from `execute_js` cannot read or rewrite evidence. +- **Final URL and DOM markers:** after t1, the runner uses CDP on the bench Chrome (`--remote-debugging-port=0`, port read from `/DevToolsActivePort`, same as `tests/screenshot-uat.test.js:52-62`): find page targets whose URL contains `/r//`, read `location.href` and `[data-bench-marker]` values with `Runtime.evaluate`, then close those targets (`Target.closeTarget`) so tabs do not accumulate over 240 runs. Claude's tabs stay open after `claude` exits (the registry releases ownership after the 10 s reconnect grace, `extension/utils/agent-registry.js:75`, but does not close tabs). +- **Pure outcome evaluation:** `evaluateOutcome(task, evidence)` returns `{pass, checks:[{kind, pass, detail}]}`; `claimed` (run_task status, `complete_task`, Claude `result.is_error === false`) is stored separately and never read by the evaluator. +- **Live tasks:** URL and text checks through the same post-window CDP read (independent of FSB). They need logins in the bench profile and are reported in a separate section. + +Task schema (versioned; `fixtureSha256` computed over the task JSON plus its fixture files, canonicalized): + +```json +{ + "id": "core-click-checklist", "version": 1, "set": "core", "category": "click-heavy", + "startPath": "/fixtures/checklist/index.html", + "instruction": "Open {startUrl}. Tick the items Alpha, Delta, Echo and Kilo, then press Save.", + "timeBudgetMs": 240000, + "success": { "all": [ + { "kind": "request", "method": "POST", "path": "/api/checklist/save", "count": 1, + "jsonEquals": { "checked": ["alpha", "delta", "echo", "kilo"] } }, + { "kind": "finalUrl", "pathEquals": "/fixtures/checklist/saved.html" }, + { "kind": "marker", "name": "saved-count", "equals": "4" } + ] }, + "fixtureFiles": ["fixtures/checklist/index.html", "fixtures/checklist/saved.html", "fixtures/checklist/app.js"] +} +``` + +Suggested core set (6 mixed, 6 click-heavy; all server-verifiable): mixed — invoice lookup (read table, submit the max invoice), Jira-like ticket triage (find by label, change status in a custom dropdown, save), search → results → open a specific detail page, three-step wizard submit, settings read-then-change, paginate five pages to find and add an item; click-heavy — ten-checkbox checklist, four custom (div) dropdowns, nested accordion drill-down, chained confirm modals, tabbed editor toggles, kanban moves via "Move to" menus. + +### Pattern 8: Host metrics from the driver only (D-12, D-14) + +- **Claude Code arm argv** mirrors `claude-profile.ts:187-207` (task-mode branch): `-p --verbose --output-format stream-json --include-partial-messages --setting-sources "" --disable-slash-commands --no-chrome --strict-mcp-config --mcp-config --agents --agent fsb --permission-mode dontAsk --tools "" --allowedTools mcp__fsb --disallowedTools Bash,Edit,Write,NotebookEdit,WebFetch,WebSearch --max-turns 40 --no-session-persistence`, task on **stdin** [VERIFIED: codebase]. Build `--agents` from `mcp/ai/agents/fsb.json` with the same key order as `serializeStaticAgentPolicy`, and record the agent-prompt hash in identity. The bench MCP config points at `node /mcp/build/index.js`. +- Run `claude` in an **empty temp cwd** so the repo's `.claude/skills` and gitignored `CLAUDE.md` cannot load; record whether `~/.claude/CLAUDE.md` could have loaded (unverified, see Assumptions). +- Parse NDJSON: from `system/init` record `model`, `tools` (contains `ToolSearch` → tool search on; `WaitForMcpServers` → off), `mcp_servers[].status`, `apiKeySource`, `mcp_server_errors`; count `tool_use` blocks by name (FSB tools vs `ToolSearch`); from `result` record `duration_ms`, `duration_api_ms`, `num_turns`, `usage`, `modelUsage`, `total_cost_usd`, `is_error`, `subtype`, `permission_denials` [CITED: code.claude.com/docs/en/headless; code.claude.com/docs/en/costs]. Label cost "list-price estimate, not billed" when `apiKeySource` is `none` or a `costBasis: 'list'` field is present (D-12; the docs say all such figures are client-side estimates). +- Record `ENABLE_TOOL_SEARCH` as launched (`unset` = default `auto`) [CITED: code.claude.com/docs/en/mcp]. +- Timeout: send **SIGINT** first (SIGTERM exits 143 with no `result` event [CITED: code.claude.com/docs/en/headless]), wait a grace, then SIGTERM. +- Cursor (optional): `result` has `duration_ms`, `duration_api_ms` and no tokens or cost [CITED: cursor.com/docs/cli/reference/output-format] → label "unavailable (host does not report)". + +### Pattern 9: Hierarchical bootstrap of the geometric mean of per-task median ratios (D-16) + +```js +// scripts/bench/lib/stats.mjs (pure, deterministic, Node 20-safe) +export function mulberry32(seed) { + let a = seed >>> 0; + return () => { a = (a + 0x6D2B79F5) >>> 0; let t = a; + t = Math.imul(t ^ (t >>> 15), t | 1); t ^= t + Math.imul(t ^ (t >>> 7), t | 61); + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; +} +export function median(xs) { const s = [...xs].sort((a, b) => a - b); const n = s.length; + if (!n) return NaN; return n % 2 ? s[(n - 1) / 2] : (s[n / 2 - 1] + s[n / 2]) / 2; } +export function quantile(xs, q) { const s = [...xs].sort((a, b) => a - b); if (!s.length) return NaN; + const h = (s.length - 1) * q; const lo = Math.floor(h); return s[lo] + (h - lo) * ((s[Math.ceil(h)] ?? s[lo]) - s[lo]); } +function resample(xs, rnd) { const out = new Array(xs.length); + for (let i = 0; i < xs.length; i++) out[i] = xs[Math.floor(rnd() * xs.length)]; return out; } + +// tasks: [{ id, a: [wallMs of completed runs], b: [wallMs of completed runs] }]; ratio = median(b)/median(a) +export function geoMeanMedianRatio(tasks, { iterations = 10000, seed = 66, level = 0.95 } = {}) { + const usable = tasks.filter(t => t.a.length > 0 && t.b.length > 0); + const dropped = tasks.filter(t => !(t.a.length > 0 && t.b.length > 0)).map(t => t.id); + const logRatio = (t) => Math.log(median(t.b) / median(t.a)); + const estimate = Math.exp(usable.reduce((s, t) => s + logRatio(t), 0) / usable.length); + const rnd = mulberry32(seed); const draws = new Array(iterations); + for (let i = 0; i < iterations; i++) { + let sum = 0; + for (let k = 0; k < usable.length; k++) { // level 1: resample tasks + const t = usable[Math.floor(rnd() * usable.length)]; + sum += Math.log(median(resample(t.b, rnd)) / median(resample(t.a, rnd))); // level 2: runs within task + } + draws[i] = Math.exp(sum / usable.length); + } + const lo = quantile(draws, (1 - level) / 2), hi = quantile(draws, 1 - (1 - level) / 2); + const verdict = lo > 1 ? 'slower' : hi < 1 ? 'faster' : 'inconclusive'; // CI crossing 1.0 => inconclusive + return { estimate, ci: [lo, hi], verdict, usedTasks: usable.map(t => t.id), droppedTasks: dropped, iterations, seed }; +} +export function wilson(successes, n, z = 1.959963984540054) { + if (!n) return [NaN, NaN]; const p = successes / n; const d = 1 + z * z / n; + const c = p + z * z / (2 * n); const m = z * Math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)); + return [(c - m) / d, (c + m) / d]; +} +``` + +- Tasks where either arm has no completed run are dropped from the wall-time headline and listed; completion rate (with Wilson CI) and cost per completed task are reported beside it. +- Add an **A/A noise-floor check** to the baseline report: split each arm's runs into odd/even rounds and run the same bootstrap. Its CI should straddle 1.0; its width is the smallest effect Phase 71 can claim. +- Schedule rounds ABBA (round 1: autopilot then MCP per task; round 2: MCP then autopilot; ...) with a seeded task order, and record `round`, `order`, `startedAt` per run. + +### Pattern 10: Exporting the 40 recorded sessions (D-20, D-21) + +The 40 sessions live in the **Web Store build's** IndexedDB (extension `badgafnf…`, Chrome Profile 2). Worktree code and the bench profile cannot read them. Available paths today: + +| Path | Works on the Web Store build? | Effort | Output | +|------|------|--------|--------| +| MCP `get_session_detail` | No — the bug this phase fixes (journal sessions return `session_not_found`) | — | — | +| MCP `list_sessions` | Yes (reads `fsbSessionIndex`) | 1 call | Run-level metadata only: `startTime`, `endTime`, `actionCount`, `eventCount`, `mcpClient`, `outcome` | +| Control panel → Session History → per-session JSON download | Yes (`saveJournalSessionExport` → port `fsb-session-export-v2` → `streamSessionExport`, `options.js:3508-3548`) | 40 Save-As dialogs | Full JSON incl. `event.timestamp`, metadata and resolved request/result values | +| DevTools console on the Web Store control-panel page | Yes (`chrome.runtime.sendMessage({action:'getSessionDetail', ...})`, `background.js:12513-12536`) | One snippet | One JSON file with every tool.call event (timestamps, tool, success, route, client); request/result stay as descriptors | + +Recommended: first snapshot `list_sessions` (safety net), then run the console snippet (Code Examples) from `chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk/ui/control_panel.html` DevTools. Save raw output to `scripts/bench/.local/history-2026-09-16_21/` (gitignored). `scripts/bench/export-history.mjs` derives sanitized per-call metrics (inter-event deltas labelled "journal-timestamp deltas; include caller gap and IndexedDB write-queue delay", grouped by `mcpClient`) into `scripts/bench/results/history-2026-09-16_21/metrics.json`, which is committed. Because the bench uses a separate profile and extension ID, benchmark recordings do not evict these sessions; ordinary MCP use in Profile 2 still does, so this is the first task of the phase. + +### Anti-Patterns to Avoid + +- **Timing in tool responses** (ARCHITECTURE Anti-Pattern 7): adds tokens to every call; timing appears only in `get_session_detail`, its text export and the bench report. +- **Returning `journal.getSessionDetail` events as-is:** they carry inline request/result payloads (`eventForDetail`, `mcp-lattice-journal.js:1620-1629`). +- **Adding `afterSequence`/`limit` to `get_session_detail`:** forbidden by MCP-04 (D-08). +- **Using `event.timestamp` differences as FSB time:** includes the IndexedDB write queue (D-04). +- **Using session `endTime`** for anything (D-06). +- **Adding a timer or the word "setTimeout" to `agent-loop.js`** (8-token pin). +- **Changing legacy numeric cost/token fields' meaning** (sidepanel and dashboard consumers; D-02). Add labelled fields instead. +- **Letting the SDK default timeout govern `run_task`:** SDK 1.29 `DEFAULT_REQUEST_TIMEOUT_MSEC = 60000` [VERIFIED: sdk/dist/esm/shared/protocol.js:8]; a timed-out client leaves autopilot running in the browser and corrupts the next run. + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| MCP client for the runner | Raw JSON-RPC over stdio | `@modelcontextprotocol/sdk` 1.29.0 `Client` + `StdioClientTransport` from `mcp/node_modules` | Handles initialize, progress tokens, `resetTimeoutOnProgress`, `maxTotalTimeout`, and exposes `transport.pid` and `getServerVersion()`. | +| Sanitizing projected actions | A new allowlist | Existing `sanitizeActionHistoryEntry` / `sanitizeSessionMetadata` | Same privacy boundary as today's legacy path. | +| Journal paging, text export | New IndexedDB reads | `getSessionDetail`, `exportHumanReadable`, `streamSessionExport` | Tested, budget-aware, artifact-verified. | +| Session export UI | New control-panel code | Existing streaming export / `getSessionDetail` runtime message | D-09 forbids control-panel changes. | +| Provider cost math for xAI | Token × price only | `usage.cost_in_usd_ticks` | Covers cached-input discounts, tiers and tool costs in one number. | +| Tab ownership bypass for verification | A privileged FSB route | CDP on the dedicated bench profile | Independent of FSB (D-17); no new product surface. | +| Chrome with an extension flag | `--load-extension` on branded Chrome | One-time manual "Load unpacked" in the bench profile | Branded Chrome 137+ ignores `--load-extension` [CITED: prior STACK.md research]. | + +**Key insight:** the measurement must not change what it measures. Every new write is either off the hot path (fire-and-forget recorder, end-of-session save) or a synchronous `Date.now()`; every evidence source sits outside FSB and the agent. + +## Runtime and Operational State Touched by the Benchmark + +Not a rename phase, but the live campaign changes machine state that git does not track. The planner needs explicit set-up and restore steps. + +| Category | Items Found | Action Required | +|----------|-------------|------------------| +| Stored data | 40 journal sessions (2026-09-16..21) in the Web Store extension's IndexedDB, Profile 2; 30-day retention and 50-session cap [VERIFIED: `mcp-lattice-journal.js:28-29`] | Export first (human checkpoint H1). | +| Live service config | `~/.fsb/bridge-auth.json` `allowedExtensionOrigin = chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk` [VERIFIED: file read, secrets redacted]; binding is sticky (`bindAllowedExtensionOrigin`, `mcp/src/bridge-auth.ts:161-178`) | `fsb-mcp-server pair --reset` before the campaign so the unpacked extension binds; `pair --reset` again afterwards so the Web Store build re-binds. | +| OS-registered state | Native messaging host `io.github.fullselfbrowsing.fsb_native_host` in `~/Library/Application Support/Google/Chrome/NativeMessagingHosts/`, `allowed_origins` = Web Store origin only [VERIFIED: file read]. The extension gets its bridge pairing code only via this host (`background.js:2204-2233` → `chrome.storage.session`) | Put a copy of the manifest, with the unpacked origin, in `/NativeMessagingHosts/` (user-level hosts resolve inside the active user data dir [CITED: developer.chrome.com native messaging]). Leave the global manifest untouched. | +| Secrets/env vars | xAI API key must be entered in the bench profile's control panel; `ENABLE_TOOL_SEARCH` env recorded per run | Human checkpoint; never write the key to bench artifacts. | +| Build artifacts / installed packages | `extension/dist/offscreen/lattice-host.js` (esbuild, gitignored); `mcp/build` (only 6 files tracked); global `fsb-mcp-server` 0.11.0 at `/opt/homebrew/bin` and the packed native-host runtime 0.11.0 at `~/.fsb/native-host/runtime/package` [VERIFIED] | `npm run build` and `npm --prefix mcp run build` before loading; the runner spawns `node /mcp/build/index.js` and verifies it owns 7225 (listener PID = `transport.pid`). | +| Hub port | Extension hard-codes `ws://localhost:7225` (`mcp-bridge-client.js:12`); the hub keeps one extension socket | Disable FSB in everyday Chrome (or quit it) for the campaign window; free 7225 before starting. | + +## Common Pitfalls + +### Pitfall 1: `get_session_detail` silently drops token fields and nested timing +**What goes wrong:** `sanitizeValue` removes every key matching `/token/i` and turns objects at depth ≥ 4 into `'[object]'`; `sanitizeActionHistoryEntry` drops `durationMs`. +**Why it happens:** the secret filter predates token accounting (`mcp-tool-dispatcher.js:2027-2050`, `:2064-2086`). +**How to avoid:** store usage under names without "token" (`usage.input`), keep stored rows flat, and emit an explicit post-sanitization `measurement` block. Test that a session with token counts comes back with them. +**Warning signs:** `measurement` or `iterationTiming` rows missing `inputTokens` or showing `'[object]'` [VERIFIED: probe run in this session]. + +### Pitfall 2: Pricing the pin target as Grok 4 +**What goes wrong:** `estimateCost('grok-4.3', …)` substring-matches `grok-4` ($3/$15): 4.8× too high. Unknown models are priced as Grok ($0.20/$0.50 today). +**How to avoid:** exact `grok-4.3`/`grok-4.3-latest` rows first; `estimateCostDetailed` returns `unavailable` for unknown models; prefer provider ticks [VERIFIED: node probe]. + +### Pitfall 3: Duplicate top-level identifiers across SW classic scripts +**What goes wrong:** `mcp-tool-dispatcher.js` (loaded at `background.js:81`) and `mcp-bridge-client.js` (`:122`) share one global lexical scope; a second `const X` throws and the bridge client fails to load. +**How to avoid:** distinct identifiers per file, both assigned `Symbol.for('fsb.mcp.call-timing')`. + +### Pitfall 4: `vm` harness differences +**What goes wrong:** harnesses load a single file with a minimal context: no `performance`, no `structuredClone`, no `FsbMcpLatticeJournal`, often no `chrome.runtime.getManifest` [VERIFIED: `tests/mcp-in-flight-session-lookup.test.js:34-94`, `tests/mcp-bridge-background-dispatch.test.js:306-313`, node vm probe]. +**How to avoid:** `Date.now()` only; `typeof` guards on `globalThis.FsbMcpLatticeJournal`, `performance`, `getManifest`; never reference the other file's top-level identifiers at evaluation time. + +### Pitfall 5: The `recordTaskOutcome` path loses the Symbol +**What goes wrong:** visual-session canonicalization spreads the payload (`mcp-tool-dispatcher.js:576-579`), so `complete_task`/`fail_task` calls lose timing. +**How to avoid:** read timing from the original payload at the start of `dispatchMcpToolRoute` and pass it explicitly. + +### Pitfall 6: Journal byte-cost tests and text-export parity +**What goes wrong:** `tests/mcp-lattice-journal.test.js:416-431` asserts `streamSessionExport(id,'text')` output **equals** `exportHumanReadable(id)`, and other cases assert exact counts/regexes (`/Call 505: read_step_504/`). +**How to avoid:** change both text producers through one shared formatter; append timing after the tool name (e.g. `[OK] Call 3: click — fsb 412 ms, gap 8113 ms (upper bound)`, or `— timing not recorded` for older events); add `metadata.timing` only when present so existing budget fixtures keep their byte costs. + +### Pitfall 7: `run_task` needs an owned tab and a long timeout +**What goes wrong:** for a registered agent, `handleStartAutomationRoute` requires `resolveAgentTabOrError` to succeed (`mcp-tool-dispatcher.js:2636-2677`), so a fresh runner agent gets `NO_OWNED_TAB`; and the SDK times out at 60 s. +**How to avoid:** before t0, the runner's agent opens the start URL (`open_tab`) outside the timed window; call `run_task` with `{ onprogress, resetTimeoutOnProgress: true, timeout: 60000, maxTotalTimeout: task.timeBudgetMs }`; on timeout call `stop_task` and mark the run failed. Record the asymmetry (the MCP arm opens its own tab inside the window); it does not affect within-arm Phase 71 comparisons. + +### Pitfall 8: Hub, pairing and origin mismatches look like "extension not connected" +**What goes wrong:** the everyday Web Store FSB grabs the single 7225 slot, the bridge refuses the unpacked origin (sticky pin), or native messaging refuses the unpacked origin so no pairing code is ever delivered. Doctor may still look healthy (operator memory). +**How to avoid:** the preflight checklist in Plan Decomposition, and a runner preflight that requires `fsb://config` to return the `measurement` marker and the expected unpacked extension ID before any run. + +### Pitfall 9: Evidence collection that perturbs FSB's settle waits +**What goes wrong:** a page probe posting DOM state on every mutation keeps the network busy; FSB's click settle waits for network quiet (PROJECT.md: 300 ms DOM stability plus 200 ms network quiet, capped at 3 s), so FSB time inflates. +**How to avoid:** evidence = the app's own submissions during the window; DOM markers via CDP after the window. + +### Pitfall 10: Tab and state accumulation across 240 runs +**What goes wrong:** Claude's tabs survive process exit; hundreds of open tabs slow the profile over the campaign and bias later runs. +**How to avoid:** CDP cleanup after each run; restart the bench Chrome between campaign sessions; record run order so drift can be checked. + +### Pitfall 11: Model drift under an alias +**What goes wrong:** `grok-4.3` is the model name and `grok-4.3-latest` an alias; xAI publishes no dated grok-4.3 snapshot [CITED: docs.x.ai/developers/models/grok-4.3], and the `model` string returned after redirects is undocumented. +**How to avoid:** record `servedModels` and `system_fingerprint` (if present) per iteration; the runner flags runs whose served model does not match `^grok-4\.3`. + +### Pitfall 12: History caps evict bench detail before it is saved +**What goes wrong:** autopilot history keeps 50 sessions per mode (`automation-logger.js:20`); 120 autopilot runs overflow it. +**How to avoid:** fetch `get_session_detail` right after each run (with a short retry for the async save) and write the per-run file immediately (D-20). + +### Pitfall 13: Test-chain registration and Node 20 +**What goes wrong:** a new test file not listed in `REFINEMENT_ROOT_TEST_COMMANDS` breaks `tests/delegation-phase-contract.test.js`'s pre-phase hash checks; a Node 22+ global passes locally and fails CI. +**How to avoid:** prefer extending existing, already-chained test files for extension changes; register new bench tests once, in one plan; write them as plain `node:test` ESM without Node 22+ APIs. + +### Pitfall 14: `npm test` false reds from port 7225 +**What goes wrong:** a running bench hub or stray relay holds 7225 and the chain halts at the first port-bound suite (operator memory). +**How to avoid:** never run the campaign and the full chain at the same time; run per-file checks during development and the full chain with 7225 free. + +## Code Examples + +### Runner: own SDK client, identity, and `run_task` with progress-reset timeout +```js +// scripts/bench/lib/mcp-client.mjs (runs locally only; Node 24 fine, but nothing here needs >20) +import { createRequire } from 'node:module'; +import { createHash } from 'node:crypto'; +import path from 'node:path'; +const repo = path.resolve(new URL('../../..', import.meta.url).pathname); +const mcpRequire = createRequire(path.join(repo, 'mcp', 'package.json')); +const { Client } = mcpRequire('@modelcontextprotocol/sdk/client/index.js'); +const { StdioClientTransport } = mcpRequire('@modelcontextprotocol/sdk/client/stdio.js'); +const sdkVersion = mcpRequire('@modelcontextprotocol/sdk/package.json').version; + +export async function openRunnerClient() { + const transport = new StdioClientTransport({ + command: process.execPath, args: [path.join(repo, 'mcp', 'build', 'index.js')], stderr: 'pipe' + }); + const client = new Client({ name: 'fsb-bench', version: '1' }); + await client.connect(transport); + const tools = (await client.listTools()).tools; + const canonical = JSON.stringify(tools.map(t => ({ name: t.name, description: t.description, inputSchema: t.inputSchema })) + .sort((a, b) => a.name.localeCompare(b.name))); + const config = await client.readResource({ uri: 'fsb://config' }); + return { client, transport, identity: { + serverVersion: client.getServerVersion(), sdkVersion, serverPid: transport.pid, + toolsList: { count: tools.length, sha256: createHash('sha256').update(canonical).digest('hex') }, + config: JSON.parse(config.contents[0].text) // must contain extension.measurement.callTiming === 1 + } }; +} + +export async function runTaskTimed(client, task, budgetMs) { + const t0 = Date.now(); + const res = await client.callTool({ name: 'run_task', arguments: { task } }, undefined, { + onprogress: () => {}, resetTimeoutOnProgress: true, timeout: 60_000, maxTotalTimeout: budgetMs + }); + return { t0, t1: Date.now(), claimed: res }; // "claimed" only (D-17) +} +``` + +### Exporting the 40 journal sessions from the Web Store build (paste in the control-panel DevTools console) +```js +// chrome-extension://badgafnfchcihdfnjneklogedcdkmjfk/ui/control_panel.html -> DevTools -> Console +(async () => { + const { fsbSessionIndex = [] } = await chrome.storage.local.get(['fsbSessionIndex']); + const from = Date.parse('2026-09-16T00:00:00'), to = Date.parse('2026-09-22T00:00:00'); + const ids = fsbSessionIndex.filter(s => s.storageBackend === 'journal-v2' && s.startTime >= from && s.startTime < to).map(s => s.id); + const out = { exportedAt: new Date().toISOString(), extensionId: chrome.runtime.id, + version: chrome.runtime.getManifest().version, sessions: [] }; + for (const sessionId of ids) { + let after = -1, events = [], session = null; + for (let page = 0; page < 50; page++) { + const r = await chrome.runtime.sendMessage({ action: 'getSessionDetail', sessionId, afterSequence: after, limit: 500 }); + if (!r || !r.success) break; + session = r.session; events = events.concat(r.events || []); + if (!r.hasMore || r.nextSequence == null) break; after = r.nextSequence; + } + out.sessions.push({ session, events }); + } + const a = document.createElement('a'); + a.href = URL.createObjectURL(new Blob([JSON.stringify(out)], { type: 'application/json' })); + a.download = 'fsb-history-2026-09-16_21.json'; a.click(); + console.log('exported', out.sessions.length, 'sessions'); +})(); +``` + +### Journal text export line (shared by both text producers) +```js +function formatCallTimingSuffix(metadata, previousCompletedAt) { + const t = metadata && metadata.timing; + if (!t || !Number.isFinite(t.fsbMs)) return ' | timing not recorded'; + let gap = ''; + if (Number.isFinite(previousCompletedAt) && Number.isFinite(t.receivedAt)) { + const g = t.receivedAt - previousCompletedAt; + gap = g >= 0 ? ' | caller gap ' + g + ' ms (upper bound)' : ' | overlapped previous call'; + } + return ' | fsb ' + t.fsbMs + ' ms' + gap; +} +``` + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| `grok-4-1-fast*`, `grok-4-fast*`, `grok-4-0709`, `grok-3` served as themselves | Retired; requests redirect to `grok-4.3` (low or none reasoning effort) and bill at $1.25/$2.50 per M | May 15, 2026 12:00 PT [CITED: docs.x.ai/developers/migration/may-15-retirement] | FSB's default model now costs ~5-6× the table value; pin `grok-4.3` for the baseline. | +| Token × price estimates | `usage.cost_in_usd_ticks` on every response (1e10 ticks = $1; streaming: final chunk only, needs `stream_options.include_usage`) | [CITED: docs.x.ai/developers/cost-tracking] | Provider cost is authoritative; FSB's autopilot path is non-streaming, so the field is on the body. | +| Claude Code loads all MCP tool definitions at start | Tool search on by default (v2.1.232+ with feature flags); `ENABLE_TOOL_SEARCH=auto|true|false`; `WaitForMcpServers` when off; descriptions truncated at 2,048 chars | [CITED: code.claude.com/docs/en/mcp] | Record the mode per run; Phase 69's description diet must be judged by wall time. | +| `--load-extension` on branded Chrome | Ignored on branded Chrome 137+ | [CITED: prior STACK.md research, MEDIUM] | Load the unpacked extension manually once in the bench profile. | + +**Deprecated/outdated in the codebase:** `.planning/codebase/TESTING.md` (dated 2026-02-03) says no tests exist; ignore it. `config.js:82` still recommends `grok-4-1-fast` (product default; out of scope per Deferred Ideas). + +## Assumptions Log + +| # | Claim | Section | Risk if Wrong | +|---|-------|---------|---------------| +| A1 | The bare slug `grok-4-1-fast` (FSB's default) is also redirected to grok-4.3; the retirement doc lists only the `-reasoning`/`-non-reasoning` variants | Summary, Pattern 5 | Low for Phase 66: the benchmark pins `grok-4.3` and records the served model; affects only the deferred default fix. | +| A2 | xAI's response `model` field after redirect and for `grok-4.3` echoes a grok-4.3 string | Pitfall 11 | Runner mismatch check could false-flag; make the pattern configurable and inspect the first smoke run. | +| A3 | xAI reports reasoning tokens separately from `completion_tokens` for grok-4.3 (D-10 states it; FSB's legacy parser assumes it) | Pattern 5 | Estimates double-count reasoning; the recorded `estimateDeltaPct` against provider ticks detects it on the first run. | +| A4 | A user-level native messaging manifest placed in `/NativeMessagingHosts/` is honoured for a custom `--user-data-dir` on macOS | Runtime State | If wrong, the unpacked extension never receives a pairing code; fallback is `fsb-mcp-server install --native-host --extension-id ` (changes the global manifest; restore afterwards). | +| A5 | Branded Chrome keeps a manually loaded unpacked extension across restarts of the bench profile, and `--remote-debugging-port` works with a non-default `--user-data-dir` | Pattern 7 | Extra manual reload steps; CDP checks would need a different browser build. | +| A6 | `--setting-sources ""` plus an empty cwd keeps project skills out; whether `~/.claude/CLAUDE.md` still loads is unverified | Pattern 8 | Small prompt differences across machines; record it and keep the machine fixed. | +| A7 | The direct `mcp:start-automation` message may create a small journal-v2 run for the runner's agent (the recorder skips only the literal tool name `run_task`, `mcp-session-recorder.js:2226`); the runner's `open_tab` pre-step is an action tool with visual-session fields (`tool-definitions.js:628-650`) and is journaled under the runner's agent | Pattern 7 / runner | Extra rows in the bench profile's session history; the runner must select the autopilot session by the `sessionId` that `run_task` returns, and MCP-arm sessions by client label, time window and run token in `startUrl`, excluding the runner's own agent. | +| A8 | Campaign cost and duration: autopilot on grok-4.3 roughly $0.05-0.40 per run (≈$6-50 for 120 runs); the Claude arm consumes subscription usage and may hit 5-hour windows; 240 runs take roughly 5-8 hours of wall time | Plan Decomposition | Budget or scheduling surprises; run the smoke first and extrapolate. | +| A9 | The 9.5 s / 4.6 s per-call medians in PROJECT.md were computed from journal timestamps or session spans; the method is not recorded in the repo | Pattern 10 | The history analyzer may not reproduce them exactly; report its method explicitly. | + +## Open Questions (RESOLVED) + +1. **Retirement date discrepancy.** CONTEXT D-11 says xAI retired `grok-4-1-fast` on 2026-08-15; the official page says May 15, 2026 12:00 PT. The decision is unaffected; fix the wording when the deferred default-model fix is written. + **RESOLVED:** No Phase 66 change. The prices D-11 uses stand, and the date is corrected in the deferred default-model fix. +2. **Other Grok price tables.** `extension/utils/analytics.js:28-36` and `extension/background.js:10867` keep $0.20/$0.50. D-11 names only `cost-tracker.js`, and updating the dashboard tables changes displayed numbers. Recommendation: leave them in Phase 66 and add them to the deferred default-model fix; ask the user only if they want dashboard parity now. + **RESOLVED:** Both tables stay unchanged in Phase 66 and move to the deferred default-model fix (orchestrator decision 6). +3. **Which Claude model for the MCP arm.** The v0.9.91 profile passes no `--model`. For reproducibility add `--model ` in the bench driver and record `init.model`/`modelUsage`. The user should pick the model (the 40-session baseline used whatever each session used). + **RESOLVED:** Explicit `--claude-model`, with no default. It is required for `--smoke`/`--campaign` and recorded in run identity. The user picks it at H2/H4 (decision 3). +4. **Should `createActionEvent`'s default change to `null`?** Recommended (D-05 wording) with the paired `tests/action-history.test.js:67` update; it only affects the fallback path. Low risk either way. + **RESOLVED:** Yes. The default becomes `null`, with the paired test update in the same task (decision 5). +5. **Live-site set membership and logins.** D-15 wants UAT- and Jira-style reading tasks; they need accounts in the bench profile. Recommend 2-4 tasks, reported separately, optional for completing the phase if logins are unavailable. + **RESOLVED:** The live set keeps both categories and needs no logins (decision 8): + - 2 public Jira-style reads + - 2 UAT-style reads against FSB's own public production showcase, `https://full-selfbrowsing.com`. These check that deployed pages show the expected content, as a UAT pass would. + + It stays 2-4 tasks, reported separately and excluded from the release gate. A task whose site is unreachable is recorded as skipped and does not block the phase. +6. **Cursor arm.** Optional per D-14; recommend implementing the parser/driver but not running it in the Phase 66 baseline (D-16 sizes two arms). + **RESOLVED:** Build and unit-test the driver and parser, but keep them out of the Phase 66 baseline campaign (decision 7). + +### Orchestrator decisions recorded for the plans (2026-09-30) + +Plans cite these as "orchestrator decision N". They are Claude's-discretion choices made within the locked D-01..D-21. + +1. **Plan split:** follows the research decomposition, adjusted by the planner (11 plans, 6 waves). No two plans in the same wave share a file. +2. **H1 export first:** 66-01 is non-autonomous; H2-H5 live in the live plans. Every spend-bearing or everyday-setup-changing step is a blocking `checkpoint:human-action`, because `auto_advance` is on. +3. **Claude model for the MCP arm:** an explicit `--claude-model` with no default, recorded in identity. +4. **Autopilot model:** pinned to `grok-4.3`. Preflight refuses any other model, and the model string xAI returns is recorded. +5. **`createActionEvent`:** `durationMs` defaults to `null`, with the paired test updated. +6. **Other Grok price tables:** `analytics.js` and `background.js` stay unchanged. +7. **Cursor arm:** built and tested, but not part of the baseline campaign. +8. **Live set:** 2 public Jira reads plus 2 UAT-style reads on `https://full-selfbrowsing.com`. It is reported separately and optional if a site is unreachable. +9. **Pattern-map findings are honored:** + - Sheets entries keep `timing`. + - The new Claude fixture lives under `tests/fixtures/bench/`. + - Timing rows carry no `stepName` key. + - Comments use fresh wording that avoids pinned text. + - The cost-tracker unknown-model path is decoupled from the Grok row. + - `_handleExecuteAction` stays within its size budget. + + **D-02/D-10 reconciliation:** the legacy numeric totals (`totalCost`, `totalInputTokens`, …) keep their existing values for backward compatibility (D-02). The new labelled fields are the authoritative measurement (D-10), and the MCP `measurement` projection adds a basis label next to each legacy total so a bare number is never shown unlabelled. +10. **No `mcp/` source changes (D-01):** bench scripts may import read-only from `mcp/node_modules` and `mcp/build` and may spawn `mcp/build/index.js`. +11. **Commit messages:** never add `Co-Authored-By` or any AI attribution line. +12. **Tripwires:** every extension edit runs its paired pinned tests. New chain entries are appended to `REFINEMENT_ROOT_TEST_COMMANDS`, and the historical hashes are never edited. +13. **Environment:** the first autonomous plan installs dependencies and records the start-commit `npm test` baseline with port 7225 free. Pre-existing failures are recorded, not fixed. + +## Environment Availability + +| Dependency | Required By | Available | Version | Fallback | +|------------|------------|-----------|---------|----------| +| Node.js | everything | ✓ | 24.14.1 (CI: 20) | — | +| npm | installs | ✓ | 11.19.0 | — | +| root `node_modules` | tests, esbuild | ✗ (fresh worktree) | — | `npm install` (Wave 0) | +| `mcp/node_modules` (SDK 1.29.0, ws, zod) | runner, MCP tests | ✗ (fresh worktree) | — | `npm --prefix mcp ci` (Wave 0) | +| `mcp/build` (full) | runner, Claude arm, tests | partial (6 tracked files) | — | `npm --prefix mcp run build` | +| `extension/dist/offscreen/lattice-host.js` | autopilot in the unpacked extension | ✓ (built 2026-09-28) | — | `npm run build` | +| Claude Code | MCP arm | ✓ | 2.1.283 | — | +| cursor-agent | optional arm | ✓ | 2026.09.08-6caf4ff | skip arm | +| Google Chrome (branded) | live campaign | ✓ (in use, Profile 2) | recorded at run time | Chromium/Chrome for Testing (not installed) | +| Global `fsb-mcp-server` / native-host runtime | pairing commands, native host launcher | ✓ | 0.11.0 / 0.11.0 | workspace `node mcp/build/index.js pair --reset` | +| xAI API key in the bench profile | autopilot arm | unknown | — | human checkpoint | + +**Missing dependencies with no fallback:** none (all installable). +**Missing dependencies with fallback:** installs above; the xAI key and bench-profile setup are human steps. + +## Validation Architecture + +### Test Framework + +| Property | Value | +|----------|-------| +| Framework | Plain Node scripts (`node tests/x.test.js`, custom `assert`/`check` helpers) plus `node:test` for some suites (`node --test tests/mcp-lattice-journal.test.js`) | +| Config file | none; the root `package.json` `test` script is a 298-command `&&` chain; `pretest` runs `mcp-session-recorder-journal`, `mcp-lattice-journal` and `mcp-session-export-port` | +| Quick run command | `node tests/.test.js` (or `node --test tests/.test.js`) for each touched suite | +| Full suite command | `npm test` with port 7225 free (runs `pretest` first) | +| CI | Node 20, `npm run validate:extension && npm test` (`.github/workflows/ci.yml`) | + +### Phase Requirements → Test Map + +| Req ID | Behavior | Test Type | Automated Command | File Exists? | +|--------|----------|-----------|-------------------|-------------| +| MEAS-01 | `_handleMessage` stamps a non-enumerable timing symbol; lifecycle order `begin, route, result, end` unchanged; timing never appears in `_sendResult` JSON | unit (vm) | `node tests/mcp-bridge-background-dispatch.test.js` | ✅ extend | +| MEAS-01 | `_recordMcpSessionAction` and the dispatcher message-route recorder attach `timing {receivedAt, completedAt, fsbMs}`; terminal task-status outcomes keep timing through canonicalization | unit (vm) | `node tests/mcp-bridge-background-dispatch.test.js`; new case in `node tests/mcp-session-recorder.test.js` or a new `tests/mcp-call-timing.test.js` | ✅ extend / ❌ new | +| MEAS-01 | Recorder forwards timing into `_snapshotJournalEntry` / legacy snapshot; `beginCall` identity deep-equal unchanged | unit | `node tests/mcp-session-recorder-journal.test.js` | ✅ extend | +| MEAS-01 | Journal stores `metadata.timing` only when supplied; replay projection unchanged; both text exports show per-call ms and stay byte-equal | unit | `node --test tests/mcp-lattice-journal.test.js` | ✅ extend | +| MEAS-01 | `cost_in_usd_ticks` → USD; reasoning billed at output rate; unknown model → `unavailable`; `grok-4.3` exact row; legacy `estimateCost` pins updated to new Grok prices | unit | `node tests/cost-tracker.test.js` | ✅ update | +| MEAS-01 | Autopilot iteration rows and `durationMs` survive `saveSession` in both branches; unmeasured → null/omitted | unit | `node tests/automation-logger-mcp-retention.test.js` (new case) or new `tests/autopilot-measurement.test.js` | ✅ extend / ❌ new | +| MEAS-01 | `agent-loop.js` edits keep INV-04 and anchors | source pins | `node tests/mcp-philosophy-parity-smoke.test.js && node tests/lattice-survivability-smoke.test.js && node tests/agent-loop-iterator-guard.test.js && node tests/cost-tracker-ordering.test.js && node tests/meta-cognitive-tracker.test.js && node tests/ownership-leakage-regression.test.js && node tests/run-task-cleanup-paths.test.js && node tests/lattice-step-emitter-smoke.test.js` | ✅ run | +| MEAS-01 | `measurement` projection returns token counts and timing despite the `/token/i` filter; `durationMs` omitted when unmeasured | unit (vm) | `node tests/mcp-in-flight-session-lookup.test.js` (new cases) | ✅ extend | +| MEAS-01 / D-18 | `_handleGetConfig` returns `providerKind`/`modelProvider`/`modelName`, manifest version, extension id, `measurement` marker, and no secret-like keys | unit (vm) | `node tests/mcp-bridge-background-dispatch.test.js` (new case) | ✅ extend | +| MEAS-02 | Journal session resolves in JSON (envelope unchanged, `storageBackend: 'journal-v2'`, sanitized `actionHistory` with timing, no raw payload strings) and in text; branch skipped when the journal module is missing; legacy and in-flight fallbacks and `session_not_found` unchanged; paging beyond 500 events | unit (vm, real journal module with memory backend) | `node tests/mcp-in-flight-session-lookup.test.js` | ✅ extend | +| MEAS-02 | Round trip: an id from `list_sessions` (index projection) resolves via `get_session_detail` | integration (vm) | same file or new `tests/mcp-session-detail-journal.test.js` | ❌ new (optional) | +| MEAS-03 | Stats: median/quantile known answers; bootstrap deterministic by seed; A/A → inconclusive; known 0.7× effect → "faster" with CI excluding 1; zero-success task dropped and listed; Wilson bounds | unit | `node tests/bench-stats.test.mjs` | ❌ Wave 0 | +| MEAS-03 | Outcome evaluator: request/finalUrl/marker checks, count and exact-body rules, claims ignored | unit | `node tests/bench-outcome.test.mjs` | ❌ Wave 0 | +| MEAS-03 | Task schema validation, fixture hashing stable across runs, 6+6 core composition | unit | `node tests/bench-tasks.test.mjs` | ❌ Wave 0 | +| MEAS-03 | Identity: canonical JSON + tools/list digest stable under key reordering; runner refuses Web Store id / missing marker / wrong model | unit | `node tests/bench-identity.test.mjs` | ❌ Wave 0 | +| MEAS-03 | Claude stream summarizer: init/result extraction, ToolSearch detection, cost labelling; argv parity with `claude-profile.ts` source text | unit | `node tests/bench-claude-stream.test.mjs` | ❌ Wave 0 | +| MEAS-03 | Fixture server: port 0 on 127.0.0.1, path-traversal guard, per-run token attribution, evidence endpoint requires the secret | unit (local http) | `node tests/bench-fixture-server.test.mjs` | ❌ Wave 0 | +| MEAS-03 | Preflight, smoke, and the ~240-run baseline in a live browser | manual / live | `node scripts/bench/run-bench.mjs --preflight`, `--smoke`, `--campaign`; `node scripts/bench/report.mjs ` | ❌ live-only | + +### Sampling Rate +- **Per task commit:** the touched suites' quick commands above, plus the pin suites whenever `agent-loop.js`, `background.js` or the dispatcher changes. +- **Per wave merge:** `npm run test:mcp-journal && npm test` with 7225 free. +- **Phase gate:** full `npm test` green on the final commit; `npm run validate:extension` green; the six `tests/bench-*.test.mjs` run under Node 20 once (official Node 20 tarball, not `npx node@20`); live smoke and baseline campaign reviewed. + +### Wave 0 Gaps +- [ ] `tests/bench-stats.test.mjs`, `tests/bench-outcome.test.mjs`, `tests/bench-tasks.test.mjs`, `tests/bench-identity.test.mjs`, `tests/bench-claude-stream.test.mjs`, `tests/bench-fixture-server.test.mjs` — append to the `test` chain and to `REFINEMENT_ROOT_TEST_COMMANDS` in one commit. +- [ ] A synthetic Claude stream-json fixture that includes a realistic `result` event (the existing `tests/fixtures/agent-streams/claude-code-2.1.177` fixture has no duration or cost fields). +- [ ] An xAI chat-completions response fixture with `usage.cost_in_usd_ticks`, `completion_tokens_details.reasoning_tokens` and `prompt_tokens_details.cached_tokens`. +- [ ] Environment: `npm install`, `npm --prefix mcp ci`, `npm --prefix mcp run build`, `npm --prefix showcase/server install`; confirm the chain is green on the start commit before the first extension edit. + +## Security Domain + +### Applicable ASVS Categories + +| ASVS Category | Applies | Standard Control | +|---------------|---------|-----------------| +| V2 Authentication | no (no new auth); bench changes bridge pairing | Use the product's `pair --reset`; never copy `sessionSecret` into bench artifacts | +| V3 Session Management | no | — | +| V4 Access Control | yes | Fixture server binds 127.0.0.1 only; evidence endpoint requires a runner-held secret; no new privileged FSB route | +| V5 Input Validation | yes | Task JSON validated before use; fixture server path-traversal guard (same as `tests/screenshot-uat.test.js:65-84`); timing numbers validated as finite non-negative before persistence | +| V6 Cryptography | yes (hashing only) | `node:crypto` `createHash('sha256')`; never hand-roll | +| V7/V8 Data protection and logging | yes | `_handleGetConfig` explicit allowlist (no keys); raw exports in gitignored `scripts/bench/.local/`; committed results carry metrics and identity only; `get_session_detail` projection reuses the existing sanitizers | + +### Known Threat Patterns + +| Pattern | STRIDE | Standard Mitigation | +|---------|--------|---------------------| +| API key or bridge secret leaking through `fsb://config` or bench identity | Information disclosure | Allowlist projection; unit test asserts no secret-like keys; identity records only IDs, versions and hashes | +| Page content from real sites committed in baseline files | Information disclosure | D-21 split: `.local/` raw vs `results/` sanitized; `.gitignore` entry for `scripts/bench/.local/` | +| Raw journal payloads returned over MCP | Information disclosure | Projection through `sanitizeActionHistoryEntry`; test asserts no inline request/result values in the response | +| Agent tampering with or reading outcome evidence | Tampering / spoofing | Evidence endpoint secret; evaluator reads only server-side logs and post-window CDP; claims stored separately | +| Spoofed timing via payload keys | Tampering | Timing lives on a Symbol, which JSON cannot carry; string payload keys are never read as timing | + +## Plan Decomposition Hints (for the planner) + +**Dependencies:** extension tracks are independent of each other except for shared contracts (field names below) and shared files; the live campaign depends on every code track plus the human preflight. + +Shared contracts to state verbatim in each plan: +- Symbol: `Symbol.for('fsb.mcp.call-timing')`; stamp `{ receivedAt }`; recorder entry field `timing: { v:1, receivedAt, completedAt, fsbMs }`; journal `metadata.timing` (same shape, omitted when null). +- Autopilot: `session.iterationTiming[]` (flat rows, last 100, `usage: {input, output, reasoning, cachedInput}`, `usageSource`, `costUsd`, `costSource`, `servedModel`, `modelMs`, `toolMs`, `toolCount`), `session.measurementSummary`, `actionHistory[].durationMs`. +- Config: `{ config: {providerKind, modelProvider, modelName, ...}, extension: {id, version, measurement:{callTiming:1, sessionDetailJournal:1}} }`. + +| Wave | Plan (suggested) | Files owned | Autonomous? | Notes | +|------|------|-------------|-------------|-------| +| 0 | **66-01 History export and hygiene** | `scripts/bench/export-history.mjs`, `.gitignore` (`scripts/bench/.local/`), `scripts/bench/results/history-2026-09-16_21/` | **No — human checkpoint H1 first** | Time-critical (prune ~2026-10-16..21). Snapshot `list_sessions`, run the console snippet in the Web Store control panel, then derive sanitized metrics. Also confirms `npm test` green on the start commit. | +| 1 | **66-02 MCP call timing capture + identity** | `extension/ws/mcp-bridge-client.js`, `extension/utils/mcp-session-recorder.js`, `extension/utils/mcp-lattice-journal.js`, tests `mcp-bridge-background-dispatch`, `mcp-session-recorder-journal`, `mcp-lattice-journal` | yes | Includes `_handleGetConfig` (same file) and both journal text exports. | +| 1 | **66-03 Session detail journal branch + measurement projection** | `extension/ws/mcp-tool-dispatcher.js` (message-route recorder timing, `dispatchMcpToolRoute` timing pass-through, `handleGetSessionMessageRoute`, sanitizer additions), test `mcp-in-flight-session-lookup` | yes | Parallel with 66-02 using the shared contract; its journal test cases use the real journal module with a memory backend. | +| 1 | **66-04 Autopilot timing, provider cost labels, price rows** | `extension/ai/agent-loop.js`, `extension/ai/cost-tracker.js`, `extension/utils/automation-logger.js`, optionally `extension/ai/action-history.js`; tests `cost-tracker`, `action-history`, `automation-logger-mcp-retention` (+ pin suites) | yes | Heaviest tripwire load; run the eight agent-loop pin suites on every commit. | +| 1 | **66-05 Bench core: helpers, tasks, fixtures, server** | `scripts/bench/lib/*`, `scripts/bench/tasks/**`, `scripts/bench/fixtures/**`, `scripts/bench/fixture-server.mjs`, `tests/bench-*.test.mjs`, `package.json` (test chain), `tests/delegation-phase-contract.test.js` (`REFINEMENT_ROOT_TEST_COMMANDS`) | yes | The only plan that edits the test chain. Node 20-safe tests. | +| 2 | **66-06 Runner, arms, report** | `scripts/bench/run-bench.mjs`, `scripts/bench/report.mjs`, `scripts/bench/lib/{mcp-client,claude-arm,cursor-arm,cdp}.mjs` | yes (code) | Depends on 66-05; runtime depends on 66-02..04. Preflight refuses on: 7225 not owned by the runner's server, missing `measurement` marker, Web Store extension id, `providerKind !== 'api'`, `modelProvider !== 'xai'`, `modelName !== 'grok-4.3'`. | +| 2 | **66-07 Integration verification** | none or test-only | yes | Full `npm test` with 7225 free; Node 20 run of bench tests; manual review that no timing appears in tool responses. | +| 3 | **66-08 Live baseline capture** | `scripts/bench/results//**` | **No — human checkpoints H2-H5** | Smoke (2-3 tasks × 2 runs × 2 arms) → review evidence and identity → full campaign (12 × 10 × 2, ABBA) → report with A/A noise floor → commit sanitized results → restore everyday setup. | + +**Human checkpoints (cannot be automated):** +- **H1 (first, urgent):** export the 40 sessions from the Web Store build in Profile 2 before further MCP use there. +- **H2 (bench profile setup, once):** create `~/.fsb-bench/chrome-profile`; launch branded Chrome with `--user-data-dir= --remote-debugging-port=0 --no-first-run`; enable Developer mode and "Load unpacked" the worktree `extension/` (after `npm run build`); note the extension ID; place the native-host manifest copy with that origin in `/NativeMessagingHosts/`; disable FSB in the everyday profile (or quit everyday Chrome); run `fsb-mcp-server pair --reset`; open the bench Chrome so the unpacked extension pairs; in its control panel set provider xAI, enter the API key, select `grok-4.3`; sign into live-site accounts if the live set is used. +- **H3 (after the last extension commit):** reload the unpacked extension in the bench profile; the runner verifies the marker, extension ID and model through `fsb://config`. +- **H4 (campaign):** start the smoke, review, then the full campaign (may span Claude usage windows; keep the machine awake). +- **H5 (restore):** close the bench Chrome, `fsb-mcp-server pair --reset`, re-enable FSB in the everyday profile so the Web Store build re-binds; optionally delete the bench profile. + +## Integration Point Index (verified line numbers, 2026-09-29) + +| File | Lines | What | +|------|-------|------| +| `extension/ws/mcp-bridge-client.js` | 12-97 top-level consts; 1144-1199 `_handleMessage`; 1689-1743 `_recordMcpSessionAction`; 1745-1944 `_handleExecuteAction` (record at 1933); 2349-2625 `_handleStartAutomation`; 2702-2708 `_handleGetConfig` | ingress, action tap, run_task settle, config | +| `extension/ws/mcp-tool-dispatcher.js` | 160 `MCP_REPLAY_RECORD_CONTEXT`; 576-579 payload spread; 639-731 `dispatchMcpToolRoute` (recordTaskOutcome 714-724); 733-874 `dispatchMcpMessageRoute` (recorder entry 823-871); 2027-2050 `isSensitiveKey`/`sanitizeValue`; 2064-2116 sanitizers; 2636-2677 `handleStartAutomationRoute`; 2870-2957 list/get session routes; 3184-3197 defineProperty precedent; 3926 exports | | +| `extension/utils/mcp-session-recorder.js` | 100 caps; 854-880 `_snapshotEntry`; 918-966 `_snapshotJournalEntry`; 1751-1800 `recordTaskOutcome`; 2048-2220 legacy `_recordDispatchNow` (push 2146-2152); 2222-2280 `recordDispatch` (run_task skip 2226); 2282-2325 call identity/lease; 2336-2352 `recordAction` | | +| `extension/utils/mcp-lattice-journal.js` | 24-30 limits/retention/idle; 44-48 `INTERNAL_PAYLOAD_KEYS`; 76-86 `clone`; 821-872 strip/index entry; 1168-1330 `recordDispatchNow` (metadata 1210-1229); 1530-1564 `closeIdleRun`; 1620-1648 detail; 1752-1766 replay entry; 2008-2045 `hasSession`/`exportHumanReadable`; 2103-2180 `streamSessionExport`; 2199-2247 api | | +| `extension/ai/agent-loop.js` | 1457-1465 `saveToLogger`; 1637-1669 `finalizeSession`; 1936 `apiCallStartTime`; 1954-1992 provider call; 2005-2029 usage/cost; 2053-2063 `AFTER_API_RESPONSE`; 2066-2146 end_turn; 2229-2293 malformed/permission/denial; 2298 step marker; 2769-2795 metrics row (reasoning tokens); 2797-2821 action push + `AFTER_TOOL_EXECUTION`; 2899-2906 `AFTER_ITERATION` | | +| `extension/ai/cost-tracker.js` | 23-54 table; 67-92 `estimateCost` (fallback 85-88); 128-135 `record` | | +| `extension/ai/action-history.js` | 38-53 `createActionEvent`; 91-100 `push`; 219-225 serializer | | +| `extension/ai/tool-use-adapter.js` | 312-337 `extractUsage` | | +| `extension/utils/automation-logger.js` | 20 cap; 557 `logTiming`; 689 `exportHumanReadable`; 840-1040 `saveSession` (884 totalCost; 904-916 and 963-973 action projections; 982-1012 index) | | +| `extension/background.js` | 81/94/97/104/122 SW loads; 803-816 agent-engine loads; 1726-1739 authoritative provider; 2204-2233 pairing via native host; 4958 `persistSession`; 12495-12536 control-panel session handlers; 13624-13647 start automation / agent consent | | +| `extension/offscreen/lattice-host.js` | 183-213 `computeUrl`; 878-906 non-streaming fetch | | +| `extension/ui/options.js` | 3443-3487 journal hydrate/load; 3508-3548 streaming export | | +| `mcp/src/agent-providers/claude-profile.ts` | 24-31 denied tools; 187-207 task-mode argv | | +| `mcp/src/tools/observability.ts` | 43-66 `get_session_detail` (10 s timeout) | | +| `mcp/src/queue.ts` | 16-22, 64-77 read-only bypass vs serialized mutations | | +| Tests (pins) | `lattice-provider-bridge-smoke.test.js:634-685`; `lattice-survivability-smoke.test.js:371-372`; `mcp-philosophy-parity-smoke.test.js:373-378`; `agent-loop-iterator-guard.test.js:49-69`; `cost-tracker.test.js` (Grok prices, unknown fallback); `action-history.test.js:67`; `turn-result.test.js:50`; `mcp-session-recorder-journal.test.js:92`; `mcp-lattice-journal.test.js:416-431, 1380-1398`; `mcp-tool-smoke.test.js:58`; `delegation-phase-contract.test.js:129-155, 2373-2391` | | + +## Sources + +### Primary (HIGH confidence) +- Codebase reads and probes in this session (files and line ranges in the Integration Point Index), including a `vm` probe of the current `get_session_detail` sanitizer, a node probe of `estimateCost` matches, and node checks of Symbol/`vm` semantics. +- `/opt/homebrew/lib/node_modules/fsb-mcp-server/node_modules/@modelcontextprotocol/sdk` 1.29.0 — `client/stdio.d.ts` (`pid`, `stderr`), `client/index.d.ts` (`getServerVersion`, `listTools`, `readResource`, `callTool`), `shared/protocol.js:8` (60 s default timeout), `protocol.d.ts` (`resetTimeoutOnProgress`, `maxTotalTimeout`). +- Local files: `~/.fsb/bridge-auth.json` (origin pin; secrets not read into output), native messaging host manifests, `~/.fsb/native-host/install-receipt.json`. + +### Secondary (MEDIUM confidence — official docs fetched 2026-09-29) +- https://docs.x.ai/developers/migration/may-15-retirement — retired slugs and grok-4.3 redirect, pricing. +- https://docs.x.ai/developers/cost-tracking — `usage.cost_in_usd_ticks`, 1e10 ticks, streaming final chunk. +- https://docs.x.ai/developers/pricing and https://docs.x.ai/developers/models/grok-4.3 — `grok-4.3` ($1.25/$2.50, alias `grok-4.3-latest`, default reasoning `low`, function calling yes). +- https://code.claude.com/docs/en/headless — `system/init`, `result`, SIGTERM/SIGINT behaviour, `--mcp-config` startup wait. +- https://code.claude.com/docs/en/costs — cost figures are client-side list-price estimates. +- https://code.claude.com/docs/en/mcp — `ENABLE_TOOL_SEARCH`, `ToolSearch`/`WaitForMcpServers`, 2,048-char truncation. +- https://cursor.com/docs/cli/reference/output-format — result has durations only. +- https://developer.chrome.com/docs/extensions/develop/concepts/native-messaging — user-level hosts live under the profile's `NativeMessagingHosts`. + +### Tertiary (LOW confidence) +- Prior milestone research (`.planning/research/STACK.md`) for branded Chrome 137+ ignoring `--load-extension`; not re-verified this session. + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH — no new dependencies; SDK and CLIs verified locally. +- Architecture (extension changes): HIGH — every integration point read at current lines; sanitizer and pricing behaviour probed. +- Benchmark topology (Chrome profile, native host, pairing): MEDIUM — mechanisms verified in code and local files; Chrome behaviour for custom profiles partly documented. +- Pitfalls: HIGH for code pitfalls (verified); MEDIUM for campaign cost and duration. + +**Research date:** 2026-09-29 +**Valid until:** 2026-10-13 for external facts (xAI models and Claude Code change weekly); code facts until the next edit of the listed files. diff --git a/.planning/phases/66-baseline-and-session-timing/66-VALIDATION.md b/.planning/phases/66-baseline-and-session-timing/66-VALIDATION.md new file mode 100644 index 000000000..e88fe4b75 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/66-VALIDATION.md @@ -0,0 +1,156 @@ +--- +phase: 66 +slug: baseline-and-session-timing +status: draft +nyquist_compliant: false +wave_0_complete: false +created: 2026-09-29 +--- + +# Phase 66 — Validation Strategy + +> Per-phase validation contract for feedback sampling during execution. +> Source: `66-RESEARCH.md` → `## Validation Architecture`. Task IDs match the final plans, 66-01..66-11. + +--- + +## Test Infrastructure + +| Property | Value | +|----------|-------| +| **Framework** | Plain Node test scripts (`node tests/.test.js`, custom assert/check helpers), plus `node:test` for some suites (e.g. `node --test tests/mcp-lattice-journal.test.js`). New bench suites use `.test.mjs` with `node --test`. | +| **Config file** | none. The root `package.json` `test` script is a ~298-command `&&` chain, and `pretest` runs `mcp-session-recorder-journal`, `mcp-lattice-journal` and `mcp-session-export-port`. | +| **Quick run command** | the `` command of the task (per-suite `node …` runs) | +| **Full suite command** | `npm test` with loopback port 7225 free (`lsof -nP -iTCP:7225 -sTCP:LISTEN` returns nothing) | +| **CI** | Node 20: `npm run validate:extension && npm test` (`.github/workflows/ci.yml`) | +| **Estimated runtime** | quick: < 30 s per suite. Full chain: several minutes. | + +--- + +## Sampling Rate + +- **After every task commit:** run the task's `` command. + - Whenever `extension/ai/agent-loop.js`, `extension/background.js` or `extension/ws/mcp-tool-dispatcher.js` changes, also run the pin suites listed in 66-04 Task 2 (`mcp-philosophy-parity-smoke`, `lattice-survivability-smoke`, `agent-loop-iterator-guard`, `lattice-step-emitter-smoke`, `lattice-provider-bridge-smoke`, and the others). +- **After every plan wave:** run `npm run test:mcp-journal && npm test` with port 7225 free. + - A red `EADDRINUSE` means the chain stopped early and the later suites never ran. It is not a pass. +- **Before `/gsd-verify-work` (66-09 Task 2):** + - the full `npm test` is green against the start-commit baseline from 66-02 Task 1 + - `npm run validate:extension` is green + - the ten `tests/bench-*.test.mjs` suites have passed once under Node 20, using the official nodejs.org tarball rather than `npx node@20` +- **Regression sweep (66-03, 66-04, 66-06):** after each task, `scripts/bench/sweep-tests.mjs` runs every test that references an edited file and compares the result with `66-02-SWEEP-BASELINE.json`, so a failure that already existed is not reported as new. The `timeout` command is not installed on this machine, so no check uses it. +- **Max feedback latency:** 60 seconds per task check. + +--- + +## Per-Task Verification Map + +| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status | +|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------| +| 66-01-01 | 01 | 1 | MEAS-03 (D-20, D-21) | T-66-02 | Raw captures are gitignored under `scripts/bench/.local/`, and the derivation's leak check rejects page content | unit | `node --test tests/bench-export-history.test.mjs && git check-ignore -q scripts/bench/.local/probe.json && node --check scripts/bench/history-export-snippet.js` | ❌ created in task | ⬜ pending | +| 66-01-02 | 01 | 1 | MEAS-03 (D-20) | — | H1: the export file parses as `fsb-history-export` | manual + check | export-file shape check in the plan | — | ⬜ pending | +| 66-01-03 | 01 | 1 | MEAS-03 (D-21) | T-66-02 | Committed `metrics.json` has no URLs, and the raw file stays ignored | unit | metrics/raw count check `&& git check-ignore -q scripts/bench/.local/history-2026-09-16_21/raw.json` | ❌ created in task | ⬜ pending | +| 66-02-01 | 02 | 1 | MEAS-01..03 | — | Installs complete; the start-commit `npm test` baseline is recorded; `scripts/bench/sweep-tests.mjs` and its `66-02-SWEEP-BASELINE.json` exist; no tracked files modified | env | installs + baseline file + `node scripts/bench/sweep-tests.mjs --help` + sweep-baseline check + clean tracked tree (see plan) | ❌ created in task | ⬜ pending | +| 66-02-02 | 02 | 1 | MEAS-03 (D-16) | — | Median/quantile known answers; bootstrap deterministic by seed; A/A reads as inconclusive; a known effect gives a CI that excludes 1 | unit | `node --test tests/bench-stats.test.mjs` | ❌ created in task | ⬜ pending | +| 66-02-03 | 02 | 1 | MEAS-03 (D-17, D-18) | T-66-01, T-66-04 | The evaluator never reads agent claims. Identity is canonical and hides secrets, and preflight refuses the wrong extension or model. | unit | `node --test tests/bench-outcome.test.mjs tests/bench-identity.test.mjs` | ❌ created in task | ⬜ pending | +| 66-03-01 | 03 | 2 | MEAS-01 (D-04, D-18) | T-66-01, T-66-05 | Timing is stamped on `Symbol.for('fsb.mcp.call-timing')` and never serialized. Sheets entries keep timing. `fsb://config` is an allowlist with no secret-like keys. | unit (vm) | `node tests/mcp-bridge-background-dispatch.test.js && node --test tests/spreadsheet-record-redaction.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js` | ✅ extend | ⬜ pending | +| 66-03-02 | 03 | 2 | MEAS-01 (D-04) | T-66-05 | The recorder carries validated timing on journal, legacy and task-outcome paths, and the `beginCall` identity is unchanged | unit | `node tests/mcp-session-recorder-journal.test.js && node tests/mcp-session-recorder.test.js && node tests/lattice-replay-background-contract.test.js` | ✅ extend | ⬜ pending | +| 66-03-03 | 03 | 2 | MEAS-01 (D-04, D-09) | — | Journal `metadata.timing` is written only when supplied. Both text exports gain per-call ms through one shared formatter. | unit | `node --test tests/mcp-lattice-journal.test.js tests/mcp-session-export-port.test.js && npm run test:mcp-journal` | ✅ extend | ⬜ pending | +| 66-04-01 | 04 | 2 | MEAS-01 (D-10, D-11) | — | Provider-reported cost is used first. `grok-4.3` has exact price rows. Unknown models are labelled unavailable in the labelled path and are never priced as Grok. | unit | `node tests/cost-tracker.test.js` | ✅ update | ⬜ pending | +| 66-04-02 | 04 | 2 | MEAS-01 (D-02, D-05) | — | Model and tool timing are stamped inline, `durationMs` defaults to null, and INV-04 plus all agent-loop pins still hold | unit + source pins | pin-suite chain in 66-04 Task 2 `` | ✅ run | ⬜ pending | +| 66-04-03 | 04 | 2 | MEAS-01 (D-05, D-06) | — | `saveSession` keeps the measurement fields, and the autopilot text export gains timing lines | unit | `node tests/automation-logger-mcp-retention.test.js && node tests/lattice-replay-background-contract.test.js && node tests/delegation-sidepanel-tab-follow.test.js && node tests/mcp-session-recorder.test.js` | ✅ extend | ⬜ pending | +| 66-05-01 | 05 | 2 | MEAS-03 (D-15) | T-66-06 | Task JSON is validated before use, and fixture hashing is stable | unit | `node --test tests/bench-tasks.test.mjs` | ❌ created in task | ⬜ pending | +| 66-05-02 | 05 | 2 | MEAS-03 (D-13, D-17) | T-66-04, T-66-06 | The fixture server binds 127.0.0.1 only and guards against path traversal; the evidence endpoint requires the runner secret | unit (local http) | `node --test tests/bench-fixture-server.test.mjs` | ❌ created in task | ⬜ pending | +| 66-05-03 | 05 | 2 | MEAS-03 (D-12, D-14) | — | The Claude and Cursor stream parsers keep argv parity with `claude-profile.ts`, and the agent-streams manifest set is unchanged | unit | `node --test tests/bench-claude-stream.test.mjs && node tests/mcp-agent-drift-smoke.test.js` | ❌ created in task | ⬜ pending | +| 66-06-01 | 06 | 3 | MEAS-01 (D-04) | T-66-05 | Dispatcher message-route and task-outcome paths carry timing | unit (vm) | `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node --test tests/spreadsheet-record-redaction.test.js` | ✅ extend | ⬜ pending | +| 66-06-02 | 06 | 3 | MEAS-02 (D-07, D-08) | T-66-03 | Lookup checks the journal first and returns sanitized, paged `actionHistory` with no raw payloads. The journal-absent guard still holds and `session_not_found` is unchanged. | unit (vm) | `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/mcp-lattice-journal.test.js` | ✅ extend | ⬜ pending | +| 66-06-03 | 06 | 3 | MEAS-01 | T-66-03 | The `measurement` block survives the `/token/i` filter, and `durationMs` appears only when finite | unit (vm) | `node tests/mcp-in-flight-session-lookup.test.js && node tests/mcp-session-recorder.test.js && node tests/mcp-dispatcher-client-label.test.js && node tests/mcp-tool-routing-contract.test.js && node --test tests/spreadsheet-record-redaction.test.js` | ✅ extend | ⬜ pending | +| 66-07-01 | 07 | 3 | MEAS-03 (D-15) | — | The six mixed fixture apps and their tasks load | unit | mixed-count loader check (see plan) | ❌ created in task | ⬜ pending | +| 66-07-02 | 07 | 3 | MEAS-03 (D-15) | — | Core has 12 tasks (6 click-heavy); the live set has 2-4 | unit | core/live count loader check (see plan) | ❌ created in task | ⬜ pending | +| 66-07-03 | 07 | 3 | MEAS-03 (D-15, D-17) | T-66-06 | The core corpus is frozen by `core.lock.json`, and the offline evidence self-check passes | unit | `node --test tests/bench-corpus.test.mjs tests/bench-tasks.test.mjs tests/bench-fixture-server.test.mjs` | ❌ created in task | ⬜ pending | +| 66-08-01 | 08 | 3 | MEAS-03 (D-14, D-18) | T-66-01, T-66-01d | The drivers build the child env from an allowlist, dropping `CLAUDECODE`, `CLAUDE_*` and every provider key. They spawn in their own process group, stop tasks on timeout, and never log or store the pairing code during CDP pairing. | unit (fakes) | `node --test tests/bench-drivers.test.mjs` + module-export check (see plan) | ❌ created in task | ⬜ pending | +| 66-08-02 | 08 | 3 | MEAS-03 (D-16, D-20) | T-66-02, T-66-SPEND-PAUSE, T-66-07 | Seeded ABBA schedule produces sanitized run rows; a dry run completes. `agent_cap`, `agent_auth` and `provider_error` (including silent 429 timeouts) pause the run with exit 75, and "500 ms" is not a match. Resume re-runs infra-error runs. `--expect-extension-id` is validated before pairing. The preflight refuses with `telemetry_not_opted_out` unless the opt-out is on. | unit + CLI | `node --test tests/bench-report.test.mjs && node scripts/bench/run-bench.mjs --dry-run …` (see plan) | ❌ created in task | ⬜ pending | +| 66-08-03 | 08 | 3 | MEAS-03 (D-12, D-16) | — | The report shows completion CIs, wall time per completed task, labelled cost, and the A/A noise floor | unit | `node --test tests/bench-report.test.mjs` | ❌ created in task | ⬜ pending | +| 66-09-01 | 09 | 4 | MEAS-03 | — | The ten bench suites are registered once, at the end of the chain and in `REFINEMENT_ROOT_TEST_COMMANDS`, with the historical hashes untouched | contract | `node tests/delegation-phase-contract.test.js && node tests/mcp-client-identity-integration.test.js` + chain check | ✅ run | ⬜ pending | +| 66-09-02 | 09 | 4 | MEAS-01..03 (D-01, D-02) | T-66-05 | The full chain matches the baseline, the timing symbol appears only in the two ingress files, and nothing under `mcp/` changed | integration | `npm run validate:extension` + audits (see plan) | ✅ run | ⬜ pending | +| 66-10-01 | 10 | 5 | MEAS-03 | — | The extension and MCP server build, and the bench Chrome starts with DevTools on its own profile | env | build + DevToolsActivePort + pid check | — | ⬜ pending | +| 66-10-02 | 10 | 5 | MEAS-03 | T-66-01 | H2+H3: the unpacked extension id is valid and is not the Web Store id | manual + check | extension-id check (see plan) | — | ⬜ pending | +| 66-10-03 | 10 | 5 | MEAS-01, MEAS-03 | T-66-01, T-66-01e | Preflight passes, including the telemetry opt-out check. The saved core smoke has 24 distinct runIds (12 tasks × 1 round × 2 arms), plus up to 8 live runs unless declined. Operators can see per-call timing live. | live | `node scripts/bench/run-bench.mjs --preflight --pair-via-cdp …` + smoke checks that count distinct runIds (see plan) | — | ⬜ pending | +| 66-11-01 | 11 | 6 | MEAS-03 | — | H4: the human approves the campaign and its spend | manual | bench-session check (see plan) | — | ⬜ pending | +| 66-11-02 | 11 | 6 | MEAS-03 (D-16, D-21) | T-66-02 | The campaign runs; committed results contain no URLs and raw runs stay ignored | live + check | `node scripts/bench/report.mjs scripts/bench/results/core-baseline-66` + leak and ignore checks | — | ⬜ pending | +| 66-11-03 | 11 | 6 | — | — | H5: the everyday setup is restored | manual | `fsb-mcp-server status --json --timeout 5000` | — | ⬜ pending | + +*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky* + +Threat refs (from `66-RESEARCH.md` → Security Domain; each plan's `` restates the ones it touches): + +| Ref | Threat | +|-----|--------| +| T-66-01 | An API key or bridge secret leaks through `fsb://config`, benchmark identity, or CDP pairing | +| T-66-02 | Page content from real sites is committed in baseline files | +| T-66-03 | Raw journal payloads are returned over MCP | +| T-66-04 | An agent tampers with or reads outcome evidence | +| T-66-05 | Timing is spoofed through payload keys | +| T-66-06 | Malformed task JSON is accepted, or a fixture path-traversal read succeeds | +| T-66-01d | The host environment (Claude Code session variables, provider keys) leaks into benchmarked child processes | +| T-66-01e | `pair --reset` output exposes the pairing code | +| T-66-01f | Stray FSB relays take over port 7225 during the campaign | +| T-66-SPEND-PAUSE | Paid runs continue after a usage cap, auth failure or provider error | +| T-66-07 | Benchmark telemetry pollutes the production `/stats` dashboard. Mitigations: a setup-time host block, the H2 opt-out, and a preflight refusal unless `fsbTelemetryOptOut` is true. | + +--- + +## Wave 0 Requirements + +There is no separate Wave 0 plan. Each new test suite is created by the TDD task that needs it, and 66-09 registers all ten in the test chain: + +| Suite | Created by | +|-------|------------| +| `tests/bench-export-history.test.mjs` | 66-01 | +| `tests/bench-stats.test.mjs` | 66-02 | +| `tests/bench-outcome.test.mjs` | 66-02 | +| `tests/bench-identity.test.mjs` | 66-02 | +| `tests/bench-tasks.test.mjs` | 66-05 | +| `tests/bench-fixture-server.test.mjs` | 66-05 | +| `tests/bench-claude-stream.test.mjs` | 66-05 | +| `tests/bench-corpus.test.mjs` | 66-07 | +| `tests/bench-report.test.mjs` | 66-08 | +| `tests/bench-drivers.test.mjs` | 66-08 | + +- [ ] 66-09 registers all ten in the root `test` chain and in `REFINEMENT_ROOT_TEST_COMMANDS`, in one commit. +- [ ] 66-02 Task 1 creates the regression sweep `scripts/bench/sweep-tests.mjs` and its start-commit baseline `66-02-SWEEP-BASELINE.json`. 66-03, 66-04 and 66-06 run the sweep on every file they edit. +- [ ] New fixtures, all under `tests/fixtures/bench/`: + + | Fixture | Plan | + |---------|------| + | `claude-stream-usage-cap.jsonl` | 66-05 | + | `claude-error-text-cases.json` | 66-05 | + | `run-task-provider-error.json` | 66-08 | + | `infra-classification-cases.json` | 66-08 | +- [ ] A synthetic Claude stream-json fixture with a realistic `result` event goes under `tests/fixtures/bench/`, not `tests/fixtures/agent-streams/`. +- [ ] An xAI chat-completions response fixture with `usage.cost_in_usd_ticks` and `completion_tokens_details.reasoning_tokens` (66-04). +- [ ] 66-02 Task 1 installs dependencies and records the start-commit `npm test` baseline before any extension edit. + +--- + +## Manual-Only Verifications + +| Behavior | Requirement | Why Manual | Test Instructions | +|----------|-------------|------------|-------------------| +| Export the 40 recorded MCP sessions (2026-09-16..21) before they are pruned | MEAS-03 (D-20) | They exist only in the Web Store build's IndexedDB in Chrome Profile 2 | 66-01 Task 2 (H1): run `scripts/bench/history-export-snippet.js` in the Web Store control panel's DevTools console and save the output file | +| Bench profile: load the unpacked extension, free the hub, set xAI `grok-4.3`, choose the Claude model | MEAS-03 | Loading an unpacked extension and entering a key can't be automated | 66-10 Task 2 (H2 + H3) | +| Approve the paid campaign | MEAS-03 | It spends money and uses Claude usage windows | 66-11 Task 1 (H4) | +| Restore the everyday setup | — | Pairing is global and sticky | 66-11 Task 3 (H5) | +| Per-call timing visible to an operator in a live session | MEAS-01 | End-to-end check against the real extension | 66-10 Task 3: call `get_session_detail` (JSON and `format:'text'`) on a bench session | + +--- + +## Validation Sign-Off + +- [ ] All tasks have `` verify or Wave 0 dependencies +- [ ] Sampling continuity: no 3 consecutive tasks without automated verify +- [ ] Wave 0 covers all MISSING references +- [ ] No watch-mode flags +- [ ] Feedback latency < 60s +- [ ] `nyquist_compliant: true` set in frontmatter + +**Approval:** pending diff --git a/.planning/phases/66-baseline-and-session-timing/deferred-items.md b/.planning/phases/66-baseline-and-session-timing/deferred-items.md new file mode 100644 index 000000000..8a866a114 --- /dev/null +++ b/.planning/phases/66-baseline-and-session-timing/deferred-items.md @@ -0,0 +1,24 @@ +# Phase 66 deferred items + +Out-of-scope discoveries logged by plan executors. Nothing here was fixed in the plan that found it. + +## From 66-07 (2026-09-30) + +### FSB's content click activates the target twice + +**RESOLVED in 260930-f7j (commit da7e484f):** the click tool and `clickAtCoordinates` now dispatch one composed synthetic click. Click verification counts toggle state (ARIA/data-state/open/checked, a label's control, a toggle-like ancestor, the target's text), so the CDP fallback no longer undoes a correct toggle. Regression test: `tests/content-click-single-activation.test.js`. The bench guard in `scripts/bench/fixtures/_shared/bench.js` was left untouched and is now a no-op. + +- **Where:** `extension/content/actions.js`. + - The `click` tool (lines ~2436-2441) dispatches synthetic `mousedown`, `mouseup` and `click` events, then also calls `element.click()`, unconditionally. + - `clickAtCoordinates` (lines ~148-155) does the same. +- **Effect, measured:** chrome-headless-shell 146 ran the same sequence from an isolated world, as a content script would. One FSB-style click: + - left a toggle button, a native checkbox (clicked directly or through its `