test(browser): stage controlled Agent Task without workflow mutation - #288
test(browser): stage controlled Agent Task without workflow mutation#288seonghobae wants to merge 37 commits into
Conversation
Preserve the reviewed pinned-Chrome Agent Task runner, sandbox contract, post-condition evidence, cleanup checks, and scoped documentation on current protected main without carrying the leaf .github workflow mutation. Workflow activation remains owned by #212. Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head review on e051a3d06a613233781272ffdc0e564023ba52b0: this successor is directly based on protected main@87c4daa1830bac5a5228b6036752ad5633232085, 1 ahead / 0 behind with merge base exactly main, and contains the nine reviewed non-workflow #70 blobs only. No .github/** delta is present. CI 33931732869 and MV3 33931733072 completed skipped because this PR is intentionally Draft under #286; that is expected admission behavior, not GREEN. Security 33931732996, Semgrep 33931732911, and CodeQL 33931732899 remain queued. Product acceptance is still a hosted sandboxed pinned-Chromium execution proving semantic observation → native action → page-observed post-condition → cleanup; ACK-only success and --no-sandbox remain rejected. #212 owns the missing workflow activation/setup.
|
Exact-head local review on |
Remove the workflow-owned sandbox-helper assertion from the staged runner slice and document that hosted sandbox activation remains separately governed. Commit-Message-Assisted-by: Claude (via Claude Code) Signed-off-by: Seongho Bae <me@seonghobae.me>
|
정확한 head |
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head security finding: the new tests/test_mv3_browser_sandbox_contract.py is intentionally RED on this generation. _run_agent_task_browser_pass is sandbox-preserving, but ordinary _run_browser_pass still contains "--no-sandbox". This reintroduces a security/evidence behavior already causally repaired in #43 by a45c83e4d8988fe89920ecb6a9eac469815f5b9b, whose minimal production change was removal of that one launch override. Do not import #43's broader runner/download delta and do not mutate .github/** from this scheduled product lane. Repair only the ordinary launch argument, preserve the new two-path regression, then require complete current-head Python GREEN plus #212-owned sandbox-helper workflow setup and fresh pinned-Chromium execution. Draft/skipped hosted jobs are not GREEN.
seonghobae
left a comment
There was a problem hiding this comment.
Current-head review: doctoring now records the Chromium primary-source security boundary and exact owner path, but this generation is intentionally still RED. tests/test_mv3_browser_sandbox_contract.py requires both real-browser launch paths to omit --no-sandbox; ordinary _run_browser_pass has not yet adopted the reviewed #43 one-line causal repair. Do not call this head GREEN. The safe next product mutation is the narrow runner-line removal only; #212 remains responsible for .github/** sandbox-helper setup and hosted sandboxed Chrome evidence.
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head repair verification: the test-first sandbox contract is now source-level GREEN. Current ordinary MV3 Chrome options and Agent Task Chrome options both omit --no-sandbox. The two repair commits after doctoring have one net semantic line of delta versus 99fea898...: deletion of the ordinary launch override; the intermediate accidental diagnostic wording change was immediately restored by 0f434bc.... This adopts #43's causal invariant without importing its unrelated downloads/diagnostic tree. Keep Draft: protected #286 skips native CI/MV3 for Drafts, and #212 still owns the hosted chrome_sandbox setup needed for real pinned-Chromium GREEN. Skipped/absent hosted evidence is not acceptance.
|
Fresh RCA for the exact-head CodeQL failures: jobs Canonical owner is |
|
Writer lease ACTIVE — exact |
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head security RED on a71af97982b5e25cefc9f828501e610be636a875: the controlled Agent Task lane already treats page content as untrusted, but the ordinary MV3 evidence path still serializes page-derived values into CI exceptions. _exercise_real_click raises real click post-condition failed: {text!r}, and _wait_for_extension_evidence raises with observed={latest!r}. Those values originate from WebDriver/page observations and therefore may decide pass/fail but must not become diagnostic payload. Commit a71af979... adds behavioral regressions that inject a hostile marker and require stable closed diagnostics (real click post-condition failed, MV3 fixture did not converge) with no marker retention. This is intentionally test-first; current production is expected to fail both contracts. Minimal causal fix is to keep the observed values only in the local comparison state, emit no raw observed/page text on mismatch/timeout, and preserve the existing real-browser post-condition, 3-trial denominators, sandbox-enabled launch and cleanup semantics. This finding is separate from #148's ChromeDriver process/protocol diagnostic lane and does not authorize copying its active source or touching .github/**.
|
Writer lease RELEASED — exact |
|
PR-state lease ACTIVE — exact |
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head follow-up on c4ed05d94ad29621265c38a2458eee8e1cdcc59d: the extension-isolation evidence finding is repaired without manufacturing a browser observation. Test-first 380aa4899846730415f729d4796330e5ca693c42 requires the Agent Task launch request to be named separately from verified surfaces; e2230f6ee90e4b40b14439c186cdf035321bdd9a keeps --disable-extensions but records it as extensions_disabled_requested metadata and removes it from _agent_task_surfaces_complete; 922e8ba38c507fe8b48049b21ec3a5b414fd5ed9 makes the non-transitive semantics explicit; c4ed05d94ad29621265c38a2458eee8e1cdcc59d aligns CHANGELOG. The actual observed acceptance surfaces remain semantic role/name, native action outcome, exact synthetic echo, URL stability, and profile cleanup. Draft-policy CI/MV3 skips are not executable GREEN, so keep Draft until #212 supplies the authorized sandbox-helper workflow and a fresh pinned-Chromium run.
seonghobae
left a comment
There was a problem hiding this comment.
Current-head readback on 46f90e941c03dd12b6f0a25a3503d404426f013a: the requested-vs-observed extension-isolation repair remains intact, and the two pre-existing test doubles now use extensions_disabled_requested rather than the retired verified-surface spelling. This child changes tests only; it does not add browser observation, workflow authority, or executable GREEN. Verified Agent Task completeness still depends only on browser-computed semantics, native action/post-condition, exact synthetic echo, URL stability, and profile cleanup. Keep Draft until the authorized #212 workflow path supplies sandbox-helper activation and a fresh exact-head pinned-Chromium 3-trial run.
|
Writer lease RELEASED — exact |
|
Writer lease ACTIVE — exact |
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head finding on b9707975a605347b573b992cfe178150feda6a95: the controlled Agent Task lane proves a page-observed submitted state and exact synthetic echo after the native click, but it never proves those success surfaces were false before the action. A pre-fired fixture or regression can therefore satisfy the post-action assertions without establishing that the WebDriver action caused the state transition. Test-first tests/test_agent_task_action_transition_evidence_contract.py now requires a closed, non-echoing pre-action validator, baseline observation before "/click", and a pre_action_baseline_verified surface that is mandatory for _agent_task_surfaces_complete. Minimal repair: observe #task-result before typing/clicking, fail closed if data-state is already submitted or result text already equals the synthetic task input, then keep the existing semantic checks, native clear/type/click, URL-stability check, submitted-state check, exact echo, cleanup and 3-trial denominator unchanged. Do not treat command ACK as success, do not serialize page-controlled baseline values into diagnostics, and do not move this into #148 protocol diagnostics or #212 workflow/sandbox authority. This is a source-semantic/test-first RED; Draft policy skipped CI/MV3 on this exact head, so no hosted RED or browser GREEN is claimed.
seonghobae
left a comment
There was a problem hiding this comment.
Current-head review on 2e1ca45266bfc64e17e6d40e0a4b5bb2aa587a2e: the causal post-condition ambiguity identified in review 5147216471 is minimally repaired in source commit 8b7aa28ecf7acb1e3f3b2dcadd4cb3cbf59ea01f. The runner now observes the controlled #task-result baseline (data-state=idle, rendered text idle) before clear/type/click, rejects any unexpected/pre-fired state with one closed diagnostic that does not echo page data, preserves the native action and existing URL/submitted/exact-echo checks, and propagates pre_action_baseline_verified: true into per-trial evidence. Compare 4d61c2f...8b7aa28 changes only scripts/ci/run_mv3_compatibility.py by 25 added lines; no workflow, trial denominator, sandbox flag, protocol-diagnostic owner, or post-condition criterion changed. docs/traceability/agent-task-action-transition-evidence.md pins the repair and the W3C WebDriver 2 Working Draft dated 2026-07-02. This is not approval or browser GREEN: current CI/MV3 are Draft-policy skipped and current security/CodeQL lanes are still pending at this read. Fresh sandbox-enabled pinned-Chromium 3-trial execution remains required through #212.
seonghobae
left a comment
There was a problem hiding this comment.
Current-head follow-up on ec131617a2d90657911c6b072982b079c4a79fc2: the original causal-evidence finding is now closed at the acceptance predicate, not merely at evidence emission. Test-first 42a9a129... requires _agent_task_surfaces_complete to reject a successful-looking post-condition record that lacks the pre-action witness; ed6af6eb... updates the existing successful-trial doubles with pre_action_baseline_verified: true; production commit e1dd5099... adds exactly one acceptance line requiring that witness in every successful trial. Compare ed6af6eb...e1dd5099 changes only scripts/ci/run_mv3_compatibility.py by +1 line. The browser pass still proves the controlled idle/idle baseline before native clear/type/click and preserves URL, submitted-state, exact-echo, cleanup, trial denominator, sandbox, workflow, and #148 diagnostic boundaries. Traceability ec131617... records the stronger gate and W3C WebDriver 2 2026-07-02 provenance. No approval/browser GREEN is claimed: CI/MV3 are Draft-policy skipped and current security lanes are pending at this read.
|
Writer lease ACTIVE — exact |
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head test-first RED on 07540f0cdb3178998d305382d9176cccaeabce57: the current runner observes the idle result before clear/type/click, but never re-observes that result after typing and immediately before the submit click. A fixture regression can therefore transition #task-result during input and leave the later click with an already-satisfied submitted/echo post-condition. tests/test_agent_task_action_transition_evidence_contract.py now requires a second closed baseline after "/value" and before "/click", plus a mandatory pre_click_baseline_verified witness in _agent_task_surfaces_complete. Minimal repair is to reuse the existing non-echoing idle validator on a fresh browser observation immediately before click, thread that boolean through trial evidence, and keep the existing semantic target, native click, submitted/exact-echo post-condition, URL stability, cleanup, 3-trial denominator, Draft state, and #212/#148 authority boundaries unchanged. This is source-semantic/test-first RED unless hosted checks execute it; Draft-policy skips are not RED execution evidence.
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head follow-up on 81b7869dbfaa9927a7cea0c02698c8544d0ea056: the immediate pre-click causal-evidence RED from 07540f0cdb3178998d305382d9176cccaeabce57 is minimally repaired. Production 942e4c1a44119384d01ee4c7ec4168e6c5ab38b5 re-observes the existing #task-result after WebDriver typing and submit-target semantic verification, immediately before click, reuses the closed/non-echoing idle validator, carries pre_click_baseline_verified, and makes it mandatory in _agent_task_surfaces_complete. 3d3166ec... and efca7d69... align successful evidence doubles instead of weakening the new predicate. Traceability f82eb097... and TEST_STRATEGY 81b7869d... make the stronger sequence explicit: idle before input → idle immediately before click → native click → observed submitted/exact echo. No browser version, trial denominator, URL/post-condition, cleanup, extension-isolation semantics, sandbox/workflow, or #148 protocol-diagnostic authority changed. This is not approval or executable browser GREEN: exact CI/MV3 are Draft-policy skipped, and sandbox-enabled pinned-Chromium 3-trial evidence remains #212-dependent.
|
Writer lease RELEASED — exact |
Why this split exists
PR #70 contains valid controlled Agent Task browser work and a leaf
.github/workflows/mv3-compatibility.ymlmutation. Protected-mainAGENTS.mdreserves workflow changes for the authorized workflow-owner path, so #288 retains browser/test/documentation work while #212 owns workflow/sandbox activation. Raw WebDriver/ChromeDriver protocol diagnostics remain #148 authority.Exact lineage
main@87c4daa1830bac5a5228b6036752ad5633232085;88abfe308def92ef88fde54f4200b56149e36854;46f90e941c03dd12b6f0a25a3503d404426f013a;e1dd50999cd3a52977179047e8d5d77a2e85aef9;07540f0cdb3178998d305382d9176cccaeabce57;942e4c1a44119384d01ee4c7ec4168e6c5ab38b5;3d3166ec1e3a7c5aaee1f2dae92f09a7acc294f4andefca7d69c3bbc459bc218b9142a6dd4e58828076;f82eb097035b6dc4819345f2f827466283f906cf;81b7869dbfaa9927a7cea0c02698c8544d0ea056.The branch remains Draft and mergeable. This lineage does not modify
.github/**, workflow/ruleset/secrets, browser version, trial denominator, sandbox configuration, provider/model authority, base, release state, or #148 protocol diagnostics.Buyer-visible browser evidence boundary
The controlled Agent Task fixture uses pinned Chrome for Testing / ChromeDriver, browser-computed textbox/button role and accessible name, native WebDriver clear/type/click, browser-observed page state, exact synthetic input echo, URL stability, session cleanup, and profile deletion. WebDriver command acknowledgement is not task success. Page-derived values remain untrusted observation and are not policy or instruction authority.
--disable-extensionsis launch intent only and remainsextensions_disabled_requested; it is not counted as verified extension isolation without independent browser observation.Click-causal transition repair
The first causal repair observed
#task-resultas exactdata-state="idle"plus renderedidlebefore clear/type/click and requiredpre_action_baseline_verified. That closes a fixture already pre-fired at navigation time, but it did not prove the submit click caused the final transition: a regression could change the result while the WebDriver value command was typing and leavesubmittedplus the expected echo already satisfied before click.07540f0c...adds the test-first contract for a second closed baseline after"/value"and before"/click".942e4c1a...minimally repairs production by re-reading the existing result element after typing and submit-target semantic verification, immediately before click, and reusing_validate_agent_task_pre_action_state. Successful evidence now recordspre_click_baseline_verified: true;_agent_task_surfaces_completerequires both baseline witnesses.3d3166ec...andefca7d69...update successful test doubles rather than weakening that predicate.The accepted sequence is therefore idle before input → native clear/type → idle immediately before click → native click → page-observed submitted + exact synthetic echo → URL stability → cleanup. Unexpected page-controlled baseline values remain locally compared and are not serialized into CI diagnostics.
Review
5147463445records the test-first finding on07540f0c...; current-head COMMENT review5147534162records the minimal repair and documentation alignment. Neither is approval or executable browser GREEN.docs/traceability/agent-task-action-transition-evidence.mdanddocs/TEST_STRATEGY.mdare code-current for the two-baseline causal rule and retain the W3C WebDriver 2 Working Draft (2 July 2026) provenance. The stronger causality requirement is an OriginWeave evidence invariant above WebDriver command completion.Exact-head verification state
For exact
81b7869dbfaa9927a7cea0c02698c8544d0ea056at the latest read:34284661457: terminal skipped because this PR is Draft;34284661449: terminal skipped for the same Draft policy;34284661463: pending;34284661408: queued;34284661384: queued;The new regression is test-first/source-semantic evidence, not claimed hosted RED. Draft-policy skips and predecessor runs are not exact-head GREEN. Full Python repository contracts, rustfmt/workspace tests, strict Clippy, rustdoc, exact owned-production 100% coverage, current-head security/governance verdicts, and sandbox-enabled pinned-Chromium execution remain required.
Remaining owner path
#212 owns the current-generation least-privilege
chrome_sandboxworkflow activation. The exact81b7869d...consumer handoff is recorded in #212 comment5592582108. Once authorized, replay the exact successor lineage on Chrome/ChromeDriver150.0.7871.129for three independent trials and require the full two-baseline action transition plus URL stability and session/profile cleanup.Command ACK, launch-argument intent, mock output, cleanup alone, skipped jobs, or another head's browser result are insufficient. Keep Draft until exact consumer evidence and the active ruleset are satisfied. Do not restore
--no-sandbox, disable Ubuntu security restrictions runner-wide, reduce trials, copy #148 source, force-push, destructively rebase, self-approve, bypass, weaken workflows/rulesets/secrets, mark Ready, merge, tag, release, or close predecessor work before every valid delta is fully adopted by an authorized owner.