[earakely-scale]

What and why

A live test now decides each remote sandbox provider's WEBSOCKETS declaration, and boat and CoreWeave, which pass it, declare it. Modal and Modal VM, already on from documentation, pass it too. M8.3 (#506) added the declaration and turned it on only where there was evidence, which left e2b, tensorlake, vercel, coreweave, daytona and boat off until tested live. This is part of FD-4973 (M8.25). Its outcome and four decisions are recorded there.

The test: tst/integration/providers/sandbox_providers/websockets_live_test.py, one shared file parametrized over the twelve remote providers (decisions 2 and 3). Each case is skipped without that provider's credentials and is int_test_slow.

• It starts a stdlib-only WebSocket server in a sandbox of the provider (a VM where the provider makes them, else a container of a stock Python image) and, through the provider's tunnel, checks that:
  • a CDP Browser.getVersion is answered;
  • a 10 MB frame comes back intact;
  • after 5 idle minutes, a connection pinged every 20 s, as agentenv's driver and gateway ping, still answers.
• A second connection, never pinged, records the tunnel's own idle limit and doesn't decide (decision 4).
• It fails when the result doesn't match the provider's declaration, so it catches a declaration that's wrong either way. Each case prints a websockets-check JSON line with its findings.

Declarations: WEBSOCKETS = True on BoatSandboxProvider and CoreWeaveSandboxProvider, with their rows in the capabilities table. The providers' AGENTS.md names the live test as the evidence a declaration needs.

Not in this PR:

• the other eight providers, e2b, tensorlake, vercel, daytona, boxd, sail_vm, tenki and agent37, which there are no credentials to run. Their declarations are unchanged;
• the real browser half of decision 1, which deploys the chromium browser computer from the dev stores through each tunnel.

How it was tested

• ruff check: clean.

• Unit tests: the sandbox provider and computer env tests, 1803 passed, 7 skipped. CI runs the full unit tier.

• Live: boat and CoreWeave from an agent-environments-admin VM (boat from this machine too; this machine's network path to CoreWeave's API fails TLS), Modal and Modal VM from this machine:

  Provider	CDP answer	10 MB frame	Pinged, after 5 min idle	Never pinged
  boat	yes	intact	answers	open after 5 min
  CoreWeave	yes	intact	answers	open after 5 min
  Modal	yes	intact	answers	open after 5 min
  Modal VM	yes	intact	answers	open after 5 min

  Before the declarations changed, boat and CoreWeave failed with the declaration mismatch. After, both pass: 2 passed, the other 10 skipped without credentials. Modal and Modal VM ran from this machine in the genai-agent-env Modal environment (MODAL_ENVIRONMENT), because the workspace default, main, is over its spend limit. modal makes only containers, so its case ran the server in a stock Python image.

• Long hold: CoreWeave closes an HTTPS request to an exposed port after 900 s at most, which could cut a long CDP session. One pinged connection per provider, probed every minute, still answered after 25 minutes on both CoreWeave and boat.

• Local: the server and the checks against each other on this machine, all three checks pass. Two more local runs cover Greptile's findings:

  • through a proxy that closes a connection after 10 s idle, the unpinged one is recorded at 10 s, and the pinged one is still checked after the full idle window;
  • when the unpinged connection fails to open, its error is recorded and the pinged one still decides.

  boat, Modal and Modal VM ran again after those fixes.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge, with all previous findings addressed.

Summary

Adds a shared live WebSocket test and enables WEBSOCKETS for boat and CoreWeave.

• A live test checks whether remote provider tunnels carry browser WebSocket traffic.
• Boat and CoreWeave now qualify for browser-based computer environments.

_{Reviews (2) · Last reviewed commit: "fix(providers): the pinged connection wa..." · Reviewed by Greptile}