Legend: implemented (tested, usable), experimental (works, but has limited validation or unstable interfaces), planned.
| Area | Status | Evidence / notes |
|---|---|---|
| Repo bootstrap, licensing, DCO, CI | implemented | make check; license matrix verified 2026-10-03 |
| Typed config + model profiles | implemented | strict YAML, validation tests |
| SQLite state + redacted audit log | implemented | internal/store, internal/telemetry |
doctor, init |
implemented | |
bcode chat, setup, one-command install |
experimental | The chat maps messages to new, clarified and follow-up tasks, with queueing and interrupt (model tests). setup steps (backend tests, plus a pty first run with declined downloads). task apply and follow-ups from a branch (backend tests with real git). The installer was verified against the published v0.1.0-alpha.2 release binary (checksum-verified) and on its build-from-source path |
Terminal interface (tui) |
experimental | internal/tui. It covers every CLI command: structured views read the ledger and audit log, and actions run the CLI commands in-process, so policy and audit are unchanged. Model-level tests drive keys and forms against a fake backend at 80×24 to 200×60. Backend tests drive the real CLI (init, workspace, create, cancel, cleanup). The real binary was smoke-tested in a pty |
| llama.cpp supervision (start/reuse/stop, health, OOM classification, external server, idle sleep) | implemented | internal/inference/llamacpp; idle sleep measured 2026-10-04 (RSS 19.3 → 0.8 GiB, VRAM 7.1 → 0.2 GiB, wake 1.7 s) |
| Infrastructure benchmark (feasibility search, prompt/decode, cache reuse, thermals) | implemented | bench infra; reports in benchmarks/reports/ |
| OpenHands adapter (JSON-RPC/stdio, LLM tunnel, resume, condense, protocol v1 handshake) | implemented | real SDK through the adapter, scripted LLM, host and container tests; malformed/partial/crash/deadline/fuzz tests on the protocol |
| Agent runs against the local model end to end | implemented | milestone 1 (kill/resume, condensation, local-only); 11-task suite on 3 models |
| Workspaces (add/remove/disable, per-workspace config) + codebase-memory-mcp (persistent MCP session, version gate, CLI fallback, per-worktree impact) | implemented | Phase 3 gap report; 5-repo lifecycle test; detect_changes limitation and workaround recorded 2026-10-04 |
| Cross-service contract analysis (HTTP, OpenAPI, gRPC, protobuf, SQL, topics, env, Terraform) | implemented | docs/design/cross-service-analysis.md: 7/7 payment-platform links and 24/24 multi-language fixture links with no extras (TestContractsGroundTruth), idiom tests per language, fuzzing, directional contract check in the run loop (TestCounterpartsDirection); on Online Boutique (Go, C#, Node.js, Python, Java) all 21 gRPC client→server edges and no others; not yet exercised in a task run |
| Cross-repository compatibility gate (gRPC/protobuf with Go sides, OpenAPI with spec-reading tests) | experimental | internal/compat, design. For each affected link, the dependent repository's own checks run in the sandbox against the candidate commits (Go workspace composition with coverage; OpenAPI knock-out), with base-commit controls. Results are per link and per check run, keyed by commits (resume, staleness). Integrated into the run loop (broken fails the attempt; untested withholds TASK_VERIFIED), task status, verify --full and the TUI. Tests on the contract-break fixture: compatible, breaking change missed by single-repository verification, untested (5 kinds), unrelated change, resume, stale, bypass, OpenAPI, run-loop retries. The real CLI with the Docker sandbox was run on the fixture. Not run on a real task; other languages are reported untested |
| Serena v1.7.0 symbol navigation (optional, MIT-pinned) | implemented | ADR-0008: per-worktree MCP instances, read-only, graph-for-breadth routing with per-symbol fallback; fake and real-Serena tests (Go, TS, worktree freshness, malicious repo config, multi-repo, crash, cancellation, resume); bench intel overlap study; Stage 2 edit experiment; CI pin guard. Optional, off by default: A/B 9/9 vs 9/9, efficiency trend only (benchmarks/reports/*-ab-serena-summary.md) |
| Task ledger + context planner (resume packs) | implemented | run lease and crash recovery (TestSIGKILLResume, real SIGKILL), changed symbols, step progress, confined and redacted packs, lexical (ripgrep) stage |
Verification engine (task_verified vs tests_green via behavioural evidence, Go build-tag variants, presets for Go, JS/TS, Python, Rust, Java (Maven/Gradle), C/C++ (CMake/Meson/Autotools/Make), Ruby, PHP, Terraform-fmt and Helm plus a make test fallback, impact-selected Go packages, diff scope, protected paths, gitleaks) |
implemented | config and presets read from the base commit (TestAgentCannotWeakenVerification); JS/TS stages use the checkout's installed node_modules (mounted read-only) and fail when a declared script has no installed dependencies; Terraform/Helm stages are optional and skipped when the tool is absent from the sandbox image. Other languages use the checkout's .venv/vendor and the host's Cargo/Maven/Gradle caches, read-only (TestPolyglotVerification: one project per language verified offline in the real image). Evidence compares the base with and without the changed tests (Go per test function, in the packages whose test code or testdata changed; pytest, unittest, Cargo, Surefire, Gradle, minitest, RSpec, PHPUnit, CTest and Meson per named test; other stages per stage); tests that do not compile or load on the base, and timeouts, are not evidence (TestBehaviourEvidenceGoCases, unit tests only, not yet re-validated) |
| Runaway control (reasoning budget, per-response output cap, progress-aware strategy budget) | implemented | governor_test.go; calibrated by replaying archived runs (docs/benchmarks/targeted-engineering-pass-2026-10.md); no live stop observed yet |
Task contract and ambiguity handling (task.ambiguity: ask blocks for task run --clarify; proceed records SPEC_AMBIGUOUS) |
experimental | model-derived; in the second validation 2/6 derivations were empty and 1/6 flagged a false material ambiguity. Since then: a material ambiguity must survive a grounding check before it blocks (a second local-model call, made only when there is one, quotes the request; it is dropped if a quote that settles it occurs in the request, or if fewer than two readings have a supporting quote there; demotions are recorded as decisions; a failed check keeps it material); implementation no longer counts as material; an empty contract is retried once, issue-template HTML comments are stripped, and task.contract_failed carries the redacted raw reply. Unit tests with scripted replies only; untested on real tasks |
| Ranked retrieval seeds (diagnostics first, prose names, config keys, samples last; common names demoted) | implemented | seeds_test.go; replay on the development issues |
| Sandbox + policy (container, masks, read-only git, host git hardening) | implemented | adversarial tests incl. admin-dir tampering, planted symlinks, prompt-injected agent, policy bypasses; residual risks in docs/design/sandbox.md |
| Frontier escalation Z1–Z4 (Codex subscription, manual) | implemented | live Codex run (Phase 7) fixed a task the local model failed; the contained provider is verified live (TestCodexContainedLive) |
Direct-mode suite (bench direct: 13 everyday Go tasks, 2 multi-turn sessions, and a change and a question each in Python, TypeScript, Rust, Java and a Go + TypeScript monorepo, hidden checks or expected answers, per-task model calls, tool calls, prompt-cache hits, tokens, set-aside results, checks caught, time) |
experimental | internal/benchmark/direct.go, benchmarks/direct/; harness test with a fake model (TestDirectSuiteScoresAnswersAndChanges), suite definitions test; --compare for two reports. Five runs with Qwen3.6-35B-A3B on 2026-10-26 (record); two runs of one build size the noise at one task of pass rate. bench lease and bench trail score request-derived leases and history-based file suggestions against real commits, with no model (lease study, trail study); githistory: task sources replay a repository's own commits . Per-call time to first token, prefill and decode, reasoning tokens, time by phase, context by section, peak memory and failure classes; environment pinned; bench compare (paired per-task verdicts, A/A-checked). Baseline of 3 runs per mode on 2026-10-27 (record) |
| Engineering benchmark harness (14 tasks, hidden checks) | implemented | every task fails on its base (TestHiddenChecksFailOnBase, 14/14 on 2026-10-04); the model evaluation used the original 11 |
| Model evaluation (Qwen3.6, Laguna XS 2.1, Qwen3-Coder-Next) | implemented | docs/design/model-evaluation.md. Default: Qwen3.6 (ADR-0007, accepted). |
Success metrics from real task history (stats, frontier list) |
implemented | local-only rate, escalation rate, token share, verified tasks/hour; escalations record model and whether the code changed after the advice |
| Cloud model providers (OpenAI, Anthropic, Gemini, OpenAI-compatible) and API-key storage (ADR-0010) | experimental | provider command; keys in the OS credential store or an owner-only file, used only by the host gateway; request/response translation with verbatim replay of Anthropic thinking blocks and Gemini thought signatures; per-provider usage and cost in stats. Tested against recorded API shapes and fake servers (providers_test.go, provider_test.go, TestCloudFailures); no live API calls and no real tasks yet |
| Hardware detection and model choice (`model recommend | fetch | use |
| Set-up wizard and model/provider settings in the interface | experimental | internal/tui/setupwizard.go; driven end to end against a fake backend (TestWizardLocal, TestWizardCloud, TestWizardValidation, TestWizardFitsSmallTerminals, TestRuntimeModelActions); not yet used by a real user |
| macOS support | experimental | compiles and vets for darwin/arm64 and darwin/amd64; Go installers with pinned macOS assets (real archive unpacked on Linux). Unit tests on the CI macOS runner (a8fce66): 26 of 31 test packages pass; cli, frontier, gitops, sandbox and verify fail (the CI step reports only). The full flow has not been run on a Mac |
| Native Windows support | experimental | compiles and vets for windows/amd64 and arm64; container path translation, relative worktree pointers, Windows packet sanitizing, ACLs. Unit tests on the CI Windows runner (a8fce66): 17 of 31 test packages pass; cli, config, frontier, gitops, install, model, policy, repointel/serena, sandbox, secrets, store, verify, xservice and scripts/serenaguard fail (the CI step reports only). install.ps1 parses in CI but has not been run. The full flow has not been run on Windows; Serena unsupported there |
| Automatic routing between chat, code and verify (ADR-0012): rules on intent, risk keywords and paths, model fallback when unsure, chat→code switch at the first edit, code→verify hand-off before a risky edit, downgrade with a reason when a bounded task cannot run | experimental | internal/direct/router.go; rule table, hand-off and fallback tests in direct, cli and tui. Not measured on real requests |
Tool parity with Pi, Qwen Code and Claude Code (matrix): multi_edit, read_many, notebooks, background shells, web fetch/search, todo list, ask_user, plan mode, sub-agents, skills, memory, Serena symbols, MCP servers, images and @file, checkpoints/undo, /export, permission rules, hooks, command templates, ! shell |
experimental | internal/direct/tools_more_test.go and friends: one test per capability, with a fake MCP server (python3) and a fake web server; TUI tests for /tools, /undo, /export, /todos, templates, !, the question dialog. Audit 2026-10-25 (internal/direct/audit_test.go): every tool call gets a result when a turn ends early, plan mode runs safe commands, agents' tools: honoured, grep output modes, template arguments. Web search parses DuckDuckGo's HTML (fragile) unless direct.web.search_url is set. Not used on real work |
Direct mode: native agent loop for everyday development (bcode chat default, bcode chat -p, bcode -p) |
experimental | ADR-0011, design. internal/direct: read/write/edit/bash/grep/glob/ls confined to the project, approval modes (ask, edits, auto) with the deterministic command policy and secret-path mask in every mode, JSONL sessions with --continue/--resume, context compaction, project instruction files, metering under source=direct and audit events. Unit tests for every tool (incl. symlink escapes, policy before approval), TUI tests (direct default, queue/interrupt, /task, approvals, sessions) and a headless CLI test against a fake OpenAI-compatible server. Smoke runs with a real local model (Qwen3-4B on CPU) on the demo fixture: a question answered through tools; the bug fix made correctly in 2 of 3 runs, and in 1 run claimed without an edit (the changed-files footer was added for this). Fast-and-lean pass 2026-10-26 (internal/direct/workingset.go, check.go, outline.go): stable system prompt (environment and mode framing travel with the user message, so the server's prompt cache holds across turns), a working-set budget with set-aside and recall of older tool results before any summary, provider-sized tool-result caps, outlines for long files, grep collapse, a syntax/build check after every change (with a note when a Go edit returns a pointer to a range loop's copy, and, in a git repository, a note naming the files its history usually changes with the edited file: 70–81% right on held-out commits, trail study, not yet measured on agent runs), the request's names looked up before the first call (seed pack), a project map of every project in the top four directory levels in any language with its build and test commands and missing tools (internal/ecosystem; checked on 12 public repositories, commands not run, record), and streaming of the answer (llama.cpp, OpenAI, OpenAI-compatible, Anthropic, Gemini; the cloud streams tested against recorded event sequences, not live); unit tests for each. Not used on real work. Measured by the direct-mode suite (series 2026-10-26): 12–13/13 tasks, 4.4–5.1 model calls and 5.3–5.6 tool calls per task, 85–89% prompt-cache hits, about 30 s per task with Qwen3.6-35B-A3B; two multi-turn session tasks pass, one of them six messages on golang.org/x/tools that stays correct with a 16K working set (five results set aside). The suite's tasks are in Go projects: direct mode has not been measured on other languages |
| Direct-mode security policy: sensitive-operation guard, provenance, fail-closed verification fallback, MCP trust and pinning | experimental | Security policy. internal/policy/sensitive.go (classifier, language-independent) and internal/direct/guard.go, provenance.go, mcptrust.go. Adversarial unit and end-to-end tests with a scripted, injected model: README, attached issue, written script and package.json, MCP result and definition change, instruction-file and memory injection, summary and sub-agent laundering, verification downgrade, wrappers, quoting, globs, symlinks, substitutions and one-liners (internal/policy/sensitive_test.go, internal/direct/security_test.go), plus 60 ordinary commands across ecosystems that must not be flagged. Measured on the direct-mode suite with Qwen3.6-35B-A3B, two runs against the three-run baseline (benchmarks/reports/20261028-direct-security-r{1,2}, bench compare): |
- 21/23 and 22/23 passed; no task changed from always passing to always failing.
- Time, model and tool calls, and processed tokens are within noise.
- Prompt tokens are 13% higher (median per-task ratio, outside the noise band). This is the trust section of the system prompt, about 250 tokens on every call; it is served from the prompt cache.
- One guard refusal in 46 tasks: a
web_searchthe model tried during a code question, refused because no one could be asked. The model answered from the code and the task passed.
Not used on real work. Adversarial tests use a scripted model, not a real injected one. The race detector was not run locally (no C toolchain); CI runs it. |
| Direct-mode command sandbox (internal/execbox): bubblewrap (strong), Landlock (partial), none; allowlisting network proxy; per-session fallback authorization | experimental | Command sandbox. Adversarial integration tests on Linux, Ubuntu 26.04, kernel 7.0, bwrap 0.11, Landlock ABI 8 (internal/execbox/execbox_linux_test.go, internal/direct/sandbox_linux_test.go):
- unauthorized reads of home credentials, project secrets, other processes' environment and session sockets;
- writes outside the project, and symlink escapes;
- git hook and config tampering;
- network egress: direct, proxy bypass, UDP,
/dev/tcp, DNS, loopback through the proxy; - environment inheritance;
- background,
setsidandnohupprocesses surviving a command, a timeout, cancellation or the session; - a repository script stealing a key and phoning home.
Real downloads through the proxy from the Go module proxy, PyPI and npm (BOUNDEDCODE_NET_TESTS=1).
Overhead:
- per command about 8 ms (bwrap) and 2 ms (Landlock);
- direct-mode suite, two runs against the baseline (
benchmarks/reports/20261028-direct-sandbox-r{1,2}): task time and tokens within noise, tool time +0.02 s per task, no refusals. fix-balance-query, a pitfall task, failed both runs because of a wrong model fix. It also failed in the Phase 00 fix-check run, before any prompt or sandbox change: 5 of 10 recorded runs pass.
macOS and Windows have no OS sandbox (reported as "none"). The Windows job-object code is compiled, not run. Not used on real work. |
| Languages: capability architecture (internal/lang registry of 50 languages and formats, internal/ecosystem table of 26 build systems), generic text navigator, registry-driven post-edit checks, bcode capabilities, verification causes | experimental | Languages. The polyglot fixture (15 projects) is detected exactly. The detected commands build and test Go, JavaScript, Python, Rust, Java (Maven), Kotlin (Gradle), C (CMake), C++ (Make), PHP (Composer) and Ruby (Bundler) with native tools in the sandbox image (TestNativeToolchainsInSandboxImage; Maven and Gradle with the network for plugins).
C#, Swift, Dart and Terraform were detected but never built or run: no toolchain was available, and they are reported as missing tools. TypeScript type checking was not run (it needs node_modules).
Direct-mode suite, two runs against the baseline (benchmarks/reports/20261028-direct-languages-r{1,2}):
- 23/23 and 22/23 tasks passed;
- time, calls and processed tokens are within noise;
- prompt tokens are 1.12× (outside the noise band: the Phase 01 trust rules and the two symbol tools now always offered, all in the cached prefix);
find_symbolwas used by the model on the large-repository task;- the one failure (
session-ledger, an answer-wording check) also occurred in the baseline's repeat run.
Not used on real work in languages other than Go. |
| Verification 2.0: eight layers with per-repository evidence, test integrity, bounded mutation assessment (Go AST, Python tokenizer, C-family lexer), optional counterexamples, bcode bench verify | experimental | Verification, benchmark.
On a 36-case corpus in 7 language groups, with hidden tests as ground truth, in the Docker sandbox:
- false acceptance fell from 14/21 to 7/21, with false rejection 0/15;
mutation: strictreaches 0/21 at 6/15 rejection (correct changes whose tests check one input);- mutation adds about 10 s a case.
The corpus is synthetic and small: it shows what each layer catches, not a rate on real work. Counterexamples are tested with scripted proposals only; they were not measured with a model. The lexical mutation provider is unit-tested for C++ and PHP, and not run for Kotlin, C, C#, Swift, Dart, Scala or Groovy projects. |
| Daemon, GUI | planned | |
| Release packaging, SBOM | implemented | make dist (binaries and SBOMs for linux, darwin and windows on amd64 and arm64, reproducible licenses archive, checksums), scripts/sbom; published as v0.1.0-alpha.1 (source), v0.1.0-alpha.2 (linux/amd64 binary) v0.1.0-alpha.3 (six binaries with SBOMs), v0.1.0-alpha.4 (the same, adding the cross-repository gate), v0.1.0-alpha.5 (built and attested by the release workflow from the tag; SBOMs reproducible) and v0.1.0-alpha.6 (licenses archive reproducible regardless of umask). The workflow only drafts a release; publishing needs maintainer approval. |
| Local escalation tier (switch to a stronger local model before frontier) | planned | motivated by the 4/4 vs 1/4 result in ADR-0007 |
An implementation audit against both specifications (the original project specification and the Serena integration) was done on 2026-10-04; see audit-2026-10-04.md.
A small real-world validation (8 public SWE-bench-style tasks) was run on
2026-10-04: 0/8 on the frozen build, 1/8 after fixing five defects it found;
see the report.
A failure-driven engineering pass on 2026-10-05 used those failures as a
development corpus (development evidence, not a validation): 1/4 development tasks pass on the
final build (prometheus, previously failing); see
the engineering report.
A targeted pass (runaway control, ambiguity, retrieval seeds) followed, then a
second validation on 6 tasks not used during development and never shown to
the agent, screened for issue-derivable acceptance tests: 5 of 6
TASK_VERIFIED and passing the datasets' hidden acceptance tests (hidden from
the agent), all 5 local-only: no frontier calls (escalation enabled, not
triggered), and 0 false verification passes among those 5 on the screened
set (development runs with the same gate design did have such passes); see
the targeted pass and
the validation report.