[andrewleesteele]

Stacked on #10 (retail-cart), which is stacked on #9.

Summary

Runs the retail-cart guest add-to-cart tasks on Kernel, Browserbase, Browser Use, Steel, Hyperbrowser and Anchor browsers. Within an agent, every arm uses the same tools and the same task container, so the only thing that differs between arms is the browser.

• Environments, chosen per arm with environment:
  • remote-browser (RemoteBrowserEnvironment): runs the task container on the runner's Docker. Once the container is up, it creates a browser through harness/providers.py, writes its CDP URL to /run/browser/cdp-url in the container, and releases the browser when the trial ends.
  • kernel-vm-remote-browser (KernelVmRemoteBrowserEnvironment): the same, but the task container runs on a Kernel VM used only as compute, so no local Docker is needed. Kernel arms still get a separate browser session, reached over the network like every other provider's.
  • In both, a browser that can't be created fails the trial with BrowserCreationError. The default environment is still kernel.
• Providers: Kernel through the SDK (create params the SDK doesn't know yet, such as the closed-beta fingerprint, go in the request body); Browserbase, Browser Use, Steel, Hyperbrowser and Anchor through their REST session APIs. An arm's [browser.config.params] is the provider's own session create body, so the arm file shows exactly which stealth, proxy and CAPTCHA settings each provider got.
• playwright-mcp toolset: Playwright MCP (pinned) attached over CDP, working in the browser's existing context and page.
• retail-cart:
  • open-start-url and score use cdp-execute when the CDP URL file exists. Kernel-VM arms are unchanged.
  • 28 sites in three tiers: 2 easy controls, 13 standard and 13 hard. Run one tier with --suite-arg tiers=hard.
  • The verifier no longer counts an empty cart as verified when its "you may also like" recommendations happen to match the item.
  • The pinned configurator runs sites it has no measurements for on the arm's own config.
• Arms:

Provider	Arms	Settings
Kernel	codex-pw-kernel, claude-pw-kernel	stealth: true (managed ISP proxy and CAPTCHA solver)
Kernel, macOS	claude-pw-kernel-macos	as above, plus fingerprint: {os: macos} (closed beta)
Browserbase	codex-pw-browserbase, claude-pw-browserbase	solveCaptchas, proxies: true, keepAlive. No Advanced Stealth: Browserbase rejects it outside the Enterprise plan
Browser Use	codex-pw-browser-use, claude-pw-browser-use	US proxy
Steel	codex-pw-steel, claude-pw-steel	useProxy, solveCaptcha; 15-minute sessions, the Launch plan's maximum
Hyperbrowser	codex-pw-hyperbrowser, claude-pw-hyperbrowser	useStealth, useProxy, solveCaptchas
Anchor	codex-pw-anchor	US Anchor proxy, extra_stealth, captcha_solver

codex-pw-* arms run on remote-browser and claude-pw-* arms on kernel-vm-remote-browser. Claude Code is claude-sonnet-5-5.

Sessions have to survive a CDP disconnect, because the start page, the agent and the verifier each connect separately:

• Browserbase gets keepAlive.

• Hyperbrowser gets keepAlive=true on its CDP URL. Without it Hyperbrowser stops the session at the first disconnect, so every trial lost its browser before the agent started.

• Anchor gets a 30-minute idle timeout.

• Workflow: retail-cart-providers.yml, 10 attempts by default, concurrency 4 per arm. _run.yml passes BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID, BROWSER_USE_API_KEY, STEEL_API_KEY, STEEL_PROJECT_ID (needed for an organization-scoped Steel key), HYPERBROWSER_API_KEY and ANCHOR_API_KEY. A run checks for the keys of the providers its arms use before it starts.

First results (claude-pw-*, one try per site)

28 sites × 6 arms, run locally on 2026-11-06. A pass means the agent reported success and the verifier found the item in the cart.

	Kernel	Kernel macOS	Browserbase	Browser Use	Hyperbrowser	Steel
easy (2)	2	2	2	1	2	1
standard (13)	10	8	7	11	5	2
hard (13)	10	5	6	12	4	1
all 28	22	15	15	24	11	4

• Almost every miss was a bot block. There were no harness errors in 168 trials.
• The macOS fingerprint won Target, Radwell and Newark, where default Kernel failed (Target's HUMAN press-and-hold check stopped appearing). But it lost several Akamai sites that default Kernel passed: Costco, Home Depot, Lowe's, Walmart, Adidas, Kohl's, Chewy, Footlocker and Nike. With either Kernel setup counted, Kernel passed 25/28.
• Steel's eBay and Wayfair runs reported success, but the verifier couldn't reconnect to the session to check them, possibly because of the 15-minute limit.
• One try per site is noisy: borderline sites (Adidas, Zoro, Radwell) changed between runs. The workflow's 10 attempts is the number to quote.

Before a full run

• The repo needs the provider keys above as secrets. I couldn't check the repo's secrets from here.
• Anchor is still untested live (no key). The other five providers' arms each created and released a live browser with this code, and Hyperbrowser and Kernel macOS each ran a full trial on it.
• This compares add-to-cart on 28 US sites. It does not measure purchase completion.

Testing

• ruff check, ruff format --check and pytest (44 passed). New tests cover Browser Use, Hyperbrowser's keep-alive URL, Steel's project, Kernel's extra create params, the Kernel-VM remote-browser environment and retail-cart tiers.
• bench prepare retail-cart-providers --tasks all for all 12 arms (28 tasks each).
• Live:
  • Each Kernel, Browserbase, Browser Use, Hyperbrowser and Steel arm created and released a browser with this branch's providers.
  • Full trials on this branch: claude-pw-hyperbrowser passed costco-microfiber-towels (reward 1, cart_verified 1). claude-pw-kernel-macos ran end to end but Costco showed it no Add to Cart button, which matches the Akamai pattern above.
  • The results table above came from the same design run from a local checkout before it was ported onto this branch.

🤖 Generated with Claude Code