Repository navigation
Description
Priority
extreme: this reaches the standard runner, machine placement and credential delivery. Native Priority: High; P2 is planned feature work.
Components
- app/nullius (backend)
- app/web (routes)
- templates and static files
- docs and the site
- catalogue (benchmarks, agents, capabilities)
- the standard (ports, kits, relay, runs)
Today
The Where step mixes the machine choice with the execution backend. VM benchmarks can be offered a container that cannot start their desktop VMs. The agent-only Boat backend cannot run a whole desktop task, and setup/check execution can fall back to the launching machine. The interface does not express the account's VM capacity and benchmark size requirements together.
Wanted
Use the approved layout: a Hosts/Sandboxes switch, host cards or a fixed Boat card, compact VM-size cards, then Isolation. Hosts and Boat must never mix in a recipe. Read Boat sizes and concurrent capacity from the API, disable sizes that cannot fit the benchmark's declared guest memory, and show “Too small.”
Ordinary Boat runs keep the agent alone in its VM and the driver, answers and model key on the originating machine. VM benchmarks run each task whole in a Boat VM, with Docker providing the nested desktop and a separate relay container handling model calls. Forward only the chosen provider credentials and declared benchmark credentials over SSH stdin.
Screenshots
| Before | After |
|---|---|
![]() |
![]() |
Acceptance
- Hosts and Sandboxes are mutually exclusive; host selections still run one shard per host.
- Sandboxes show a separate fixed Boat card, API-derived concurrent capacity and machine sizes, with no invented fallback count in the UI.
- Size eligibility uses declared guest memory; the short “Too small” label and compact cards have matching dimensions across HumanEval and OSWorld2 at the same viewport.
- Ordinary benchmarks offer Local processes or Container on Hosts and fixed agent-only VM isolation on Boat; VM benchmarks show fixed desktop VM isolation and their provider.
- Each whole-task Boat VM runs one positional shard, returns its results, is deleted, and contributes to the merged job; unsupported agent-only placements are refused rather than silently run locally.
- Whole-task model calls use a separate relay container inside the Boat VM; task processes receive a placeholder key. Model settings, declared credentials and limits are forwarded, with total spending caps divided between tasks.
- Unit and route coverage checks placement, sizing, credentials and limits; focused browser checks cover light/dark themes and desktop/phone layouts.
Dependencies and related
Builds on merged #150 (agent-only Boat execution), #134 (OSWorld2), #188 (computer port) and #213 (Where icons). Duplicate searches found no open implementation of this redesign.
Implementation: #228. After integrating main, 664 route tests passed; the 16 browser captures cover the approved layout. The backend suite ran 3,073 tests with one remaining OSWorld2 log-reaper failure, also reproduced in an isolated rerun. Its cause is not established. Full validation and real-Boat coverage limits are recorded in the PR.
Requirement layout refinement
Compact named requirements with Check and Install scoped to each requirement; manual commands stay collapsed with Copy. Removed the prerequisite subtitle and shard count. All 664 route tests and 11 focused checks passed; light/dark desktop/mobile disclosure and Copy checks passed. Screenshots: light | dark. Implemented in #228.


Activity