[remigiusz-antczak]

Summary

The review cost gap against Anthropic's pr-review-toolkit has been reported as 1.84 times (V2s arm) and higher. That ratio compares two different harnesses: the toolkit arm ran with --system-prompt, which replaces the Claude Code system prompt, while the Perun arms ran on the default harness. On the one matched pair in the old bench (an unshipped inline Perun prompt against the toolkit, both as --system-prompt), the ratio was 1.05 times.

This PR adds a pre-registered, harness-matched bench (docs/bench/review-cost-matched-1023/) that measures the shipped products on one harness:

• PERUN: the shipped /review command plus the skill.
• TK: the toolkit's code-reviewer agent loaded as a plugin, fetched at run time from a pinned upstream commit, with a per-file SHA-256 check. No rival text is committed.

It also corrects the docs that quoted the unmatched ratio (README, perun-vs-rivals, capability-matrix, review-cost-per-recall, roadmap). Each now says the ratio is not harness-matched, and that 1.05 times belongs to the unshipped V2 arm, not to the shipped /review.

Status

Built and dry-run only. No paid call was made; spend is $0. The paid run (44 reviews, estimate $2.4, hard cap $4, stops at plan utilization 0.8) waits on owner approval. The keep rule, metrics and stops are frozen in PREREGISTERED.md before any paid call.

Safety

• scripts/test_review_cost_matched.py patches subprocess.run to raise for the whole module, so no test can start claude.
• The cap and quota checks run before any subprocess call, and a test asserts that order.
• The tests kill mutants in the prompt text, the shared flags, the tool lists, the ratio direction and the fetch-before-probe order.

Review

• The first independent review (lane commit 339b6e087) found 🔴0 🟡3 🔵4: a false "never committed" claim, the 1.05 times ratio placed next to the shipped skill, and surviving mutants.
• The delta review, pinned to 0d3cb910996aa344ed465cc36ee21e4661f7696b, confirmed all three fixes and found 🔴0 🟡0 🔵3. The nits are queued for the next nits PR:
  • the run_start ledger line is never asserted;
  • the retry and cached-skip paths in review() are untested;
  • the toolkit prompt tail is not pinned.

This branch is that commit cherry-picked onto main 31419323d5860b34faa9b39dafe15e632ee34c94 with no conflicts.

Tests

• test_review_cost_matched 26/26.
• Gates pass locally on the landed tree: size, size-ratchet, rules, mustload, check_section_pointers, skill_index --check and bench_check --changed.
• CI on the exact SHA is the blocking gate.

🤖 Generated with Claude Code