Repository navigation
Add the verification loop and a /verify recipe - #51
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Before each commit, the agent runs /simplify, /code-review, scripts/test --all and /verify, as in code-mode and shall. The /verify recipe (.claude/skills/verify/) has drive.py. It makes a sandbox data repo with seed pages and the fake transcripts of tests/fixtures.py. Then it runs this checkout's kb, or headless Claude Code with this checkout as the plugin. Every run gets the sandbox KB_CONFIG and KB_ROOT, and a folder without the sandbox marker is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Add the code-mode / shall verification loop to retroagent. Before each commit, the agent runs
/simplify, then/code-review, thenscripts/test --all, then/verify./verifyuses a project recipe,.claude/skills/verify/(SKILL.mdanddrive.py). It runs this checkout'skband the plugin hooks (headlessclaude -p --plugin-dir .) against a temporary sandbox data repo, never the real one.Drift
KB_CONFIG, it reached the real data repo. Sodrive.pycame back. It setsKB_CONFIGandKB_ROOT(andHOMEforkbruns) for each run, and it refuses a folder that has no sandbox marker.claude plugin validate .andclaude plugin test ., and the hook run shows the mod load inpluginLog./code-reviewfixes (about 20 lines) changed code. I did not run/simplifyand/code-reviewagain for them. I checked them inline and with the drive.Decisions
HOME, because the Claude login needs it. Sokb-hintin a hook run can use your local embedding model (method: semantic). Akbrun gets the sandboxHOME.drive.pyreusestests/fixtures.py(init_remote,clone,make_claude_tree,make_codex_tree,git) andkb.adapters.common.iter_records.src/kb/code, hook or skill changes.Evidence
Before:
mainhas no.claude/skills/verify/(git ls-tree -r --name-only main .claudeprints nothing)./verifyhas no recipe and nothing to drive.After: real
drive.pyruns on 2026-10-13. Each run used one sandbox under$TMPDIR/ra-verify-*.new→ first sync in the sandbox:kb <dir> -- find flaky:hook <dir> "Run exactly this Bash command: sh -c 'echo \"ModuleNotFoundError: No module named yaml\" >&2; exit 1'"($0.0028):kb-hook):Project page of demo-app:kb page demo-app(1 open thread, 2 errors → fixes). ...·- [s-000001] Rules · Wait for CI with a watch, not a sleep loop·1 retro suggestion waits for the user to accept or reject ...kb-hint):retroagent: seen before →make checkfails: ModuleNotFoundError no module named yaml → install the dev extras withpip install -e .[dev](kb summary 1a2b3c4d)pluginLog:hooks module retroagent@inline loaded ...,$.process.run (retroagent): bin/kb exited 0 in 150ms(the decide mod'skb decide --json),PreToolUse:Bash ["bin/kb-guard"] ... status 0 (90ms),PostToolUseFailure:Bash ["bin/kb-hint"] ... status 0 (204ms)data.loganddata.remoteLogdid not change (auto_sync: false).kb-guarddeny, from a run of the first version ofdrive.py(15:12, before the review changes; the session code is the same apart from the log file names):SessionStart sync with
new --config '{"auto_sync": true, "debounce_minutes": 0}', thenkb <dir> -- status:last sync: 2026-10-13T15:14:29Z(the first sync ofnewran at 15:14:06).Guards:
kb <dir>on the temp folder itself →not a sandbox: /private/var/folders/.../T.kb <dir> --timeout 0.01 -- status→"exit": "timeout"with the evidence JSON. The drive found this bug: before the fix,--timeoutwent to kb as an argument.scripts/test --all:1726 passed in 81.06s.Reading map
Config:
.claude/skills/verify/seed/pages/suggestions.json,.claude/skills/verify/seed/pages/decisions.json(sandbox seed data, made up)Error handling:
.claude/skills/verify/drive.py:71(raise SystemExitfor a folder that is not a sandbox),:84-90(kb timeout),:150and:182(session kill timer),:159(BrokenPipeError),:168(JSONDecodeError)Stack
Step 1/1. Previous: none. Next: none.
Story
Add the verification loop (
/simplify,/code-review, checks,/verify) and a/verifyrecipe for retroagent.CODING_STANDARDS.mdanddocs/development.mdx./verifyrecipe:.claude/skills/verify/SKILL.md,drive.py(new,kb,hook,clean) and the seed pages.drive.pyoutput for thekband hook probes.scripts/test --allpasses.🤖 Generated with Claude Code