Repository navigation
docs: design for the ponsbot/panicbot scanner cutover (design only) - #666
Conversation
… ponsbot/panicbot Design only; nothing is cut over. Inventories every direct invocation of pons-asinorum and panic-attack across standards, gitbot-fleet and hypatia, specifies a ponsbot adapter, the panicbot parity work, a phased cutover (shadow first), risks and rollback, and six open owner decisions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZpE5cb1MCPbmPbDjHg
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
Codex Review Summary: ❌ FailedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0a1334e0a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | Remove the panic-attack and pons permanent rows from `verdict-grid.sh` and its jq library and tests; delete the direct install/run steps and `panic-ref`/`pons-ref` inputs from `hypatia-scan-reusable.yml`; keep the `PA_GATES` block-on-critical behaviour, now reading panicbot's row | ||
| | standards | ||
| | Grid shows panicbot and ponsbot rows only; callers that passed `panic-ref`/`pons-ref` are updated in the same wave (§7) |
There was a problem hiding this comment.
Add the deprecation cycle before deleting reusable inputs
Phase 2 deletes panic-ref and pons-ref while expecting every caller to be updated in the same wave, but §7 explicitly states that estate-wide callers passing these inputs will break and requires accepting them with a warning for one cycle first. Because no phase implements that compatibility cycle, a consumer that advances its reusable-workflow pin without simultaneous input cleanup will fail workflow validation. Add the deprecation step to an earlier phase and postpone deletion to a later phase.
Useful? React with 👍 / 👎.
| * **Evidence class is carried, not dropped.** Each finding keeps pons' tier | ||
| and evidence class. SPECULATIVE findings are omitted, matching the hard | ||
| gate's `--no-speculative`, or carried with an explicit class that the | ||
| grid never grades as blocking. Whichever is chosen, a heuristic must never | ||
| read as a proof (pons ADR-0004). |
There was a problem hiding this comment.
Preserve speculative findings in the shared adapter output
If D1 selects the offered omission option, ponsbot will serve both the advisory grid and the stricter deep-recon gate while discarding speculative findings for both. That regresses the advisory contract documented in §2.1 and prevents phase 1 from meeting its “same findings” exit criterion; only the deep gate currently requests --no-speculative. Emit all findings with their evidence class from the adapter and let the deep-recon consumer filter speculative evidence.
Useful? React with 👍 / 👎.
| | 1 (shadow) | ||
| | Enrol panicbot and ponsbot in standards **alongside** the permanent rows, for a fixed number of PRs; compare counts row by row | ||
| | standards | ||
| | Same findings, same statuses, for the agreed sample; differences explained |
There was a problem hiding this comment.
Roll out enrollment changes before flipping the grid
Phase 1 enrolls the new rows only in the standards repository, and no later phase updates estate repositories' manifests before phase 2 removes their permanent scanner rows. This is especially problematic because scripts/fix-fleet-enrolment.sh leaves existing manifests untouched and its generator has no ponsbot entry, while its current lint recommends disabling panicbot when the direct lane exists. Consequently, consumers will have neither a ponsbot entry nor, in many cases, an enabled panicbot entry, so the grid flip removes their scanner coverage entirely. Add an estate-wide manifest/template migration and make its completion an exit criterion before phase 2.
Useful? React with 👍 / 👎.
| * Driver integration: one `case` arm in `scripts/fleet-report-back.sh`, one stub | ||
| in `tests/fleet-report-back-test.sh`, one build in | ||
| `scripts/fleet-report-back-build.sh`, as `docs/REPORT-BACK.adoc` prescribes for | ||
| a new adapter. |
There was a problem hiding this comment.
Register ponsbot in every driver lookup
A single invocation case arm is insufficient to integrate this adapter: run_member first calls crate_dir_of and binary_of, neither of which recognizes ponsbot, so it writes an unknown-member or missing-adapter error before the proposed arm can run. The priority registry and display-title lookup also need the new member. Expand the phase-0 checklist and tests to cover all driver registries rather than prescribing only one case arm.
Useful? React with 👍 / 👎.
| | gitbot-fleet `fleet-coordinator.sh`, `hooks/pre-push-gate.sh`, `scripts/maintenance-hard-pass.sh` | ||
| | Fall back to the bare `panic-attack` binary when panicbot is absent, or select it by name | ||
| | Local and operator tooling. The pre-push gate selects `panic-attack hypatia` by default. |
There was a problem hiding this comment.
Include every direct invocation in the operator cutover
The inventory and phase 4 omit direct invocations found by a repo-wide search: the root Justfile runs panic-attack assail, scripts/fix-missing-assail-recipe.sh installs that same direct recipe into other repositories, and robot-repo-automaton/templates/skeleton/Justfile seeds it into new repositories. Updating only the three listed operator scripts therefore leaves current local scans bypassing panicbot and continues creating new direct lanes after the cutover. Add these callers and their tests/templates to the phase-4 migration.
Useful? React with 👍 / 👎.
| D3. Does `hooks/pre-push-gate.sh`'s default (`panic-attack hypatia`) become | ||
| `panicbot ponsbot hypatia`, or does pons stay out of the local pre-push | ||
| gate because of its build cost? |
There was a problem hiding this comment.
Preserve existing pre-push scanner configuration
D3 considers only the default scanner list, but hooks/pre-push-gate.sh also reads the persistent fleet.prePushScanners Git configuration and currently accepts panic-attack as an explicit value. If phase 4 replaces that case with panicbot while dropping the bare-scanner path, users who previously configured panic-attack will hit “unknown configured scanner” and every push will be blocked. Specify a compatibility alias or a configuration migration before removing the old name.
Useful? React with 👍 / 👎.
…ty fixes, owner rulings) (#670) ## What this is Phase 0 of `docs/migrations/scanner-cutover-ponsbot-panicbot.adoc` (#666). Nothing is enrolled, no grid row or direct lane is removed, no cutover workflow changes. ### Owner rulings recorded (2026-11-12) D0 (scanners stay external engines; only direct lanes retire), D1 (SPECULATIVE pons findings carried, never graded blocking), G1 (pons gates), G2 (panic-attack does not gate; panicbot advisory). The phase-2 "keep PA_GATES block-on-critical" carry-over is removed. Scope sentence added: the bots are estate tooling, external users scan for themselves for now. Labels G1/G2 are the doc's own. ### ponsbot (`bots/ponsbot`) - `ponsbot --format json scan <dir>` runs pons as a subprocess (cwd = scanned dir, `scan --format json .`, no `--no-speculative`), writes nothing into the tree. - Exit mapping: pons 0/1 with a valid report -> ran; 2 -> error with pons' message; missing binary -> not_run; timeout, unreadable report, wrong or unstamped `source_commit` -> error. Nothing maps to a pass. Non-ran results print `{"status","reason"}` (exit 2/3), never an empty array. - Findings keep `tier`, `evidence_class`, `pons_severity`. pons ERROR -> `error`, WARN -> `warning`; SPECULATIVE (and unknown classes) are carried at `info`. Heuristics are labelled in the message. - Pin: `bots/ponsbot/canon/pin.toml` = `bca1ea289a4f8a86bac6212731e6f8ec15c67ad4`, the `pons-ref` default in standards `hypatia-scan-reusable.yml` (main f0b3cd4). Not invented. - Driver: case arm, built-in title, tool_version, `--pons SHA` install in `fleet-report-back-build.sh` (cached by commit, never fatal). Added to the `rust.yml` matrix. - `BotId::Ponsbot` added (Display, from_str, tier, BotInfo) but **left out of `all()`**: the adapter has report-back mode only, and `all()` feeds `register_all_bots`, so it would make verifier-tier completion wait on a bot the coordinator cannot run. Test asserts this. ### Gating (for phase 1 enrolments) The renderer reads gating from the enrolment's per-reporter `gating` boolean (`verdict-grid.jq`), not the envelope, so the envelope carries no gating field. **Enrolments must set `"gating": true` for ponsbot and leave panicbot without it.** Tested against the real renderer when `STANDARDS_DIR` is set. ### panicbot parity (design section 5): three real holes, fixed at source Tests written first failed on the old code: 1. the timeout was never enforced (a hung panic-attack hung the lane); 2. a non-zero exit with parseable stdout was accepted as a run; 3. a report with no positive `statistics.total_lines` (nothing analysed) was an empty `ran`. `assail` is now strict (non-zero exit, no/invalid report, unknown severity, zero lines -> error). Other subcommands keep their tolerance. `bots/panicbot/canon/pin.toml` gains `[build]` = `410a8f82...`, the standards Hypatia lane / fork-governance value. ### Pins disagree (new open decision D7, please rule) panic-attack: Hypatia lane `410a8f82`; this repo's `fleet-report-back-reusable.yml` `b689a74d` (its comment claims equality; it is not equal); deep-recon `b5530834`. pons: Hypatia lane `bca1ea28`, deep-recon `51b8920f` (older ancestor). Workflows do not read the pin files yet. Also D6: does deep-recon's must-run check on panic-attack survive now that panicbot is advisory? ## Tests (actual) - `cargo test` ponsbot: 16 unit + 13 integration pass. Real pons built at the pinned commit: `PONSBOT_REAL_PONS=... cargo test real_pons` passes (finds `constant-condition` HEURISTIC; empty tree -> error). Without that env var the test prints SKIP. - panicbot: 49 lib + 5 parity + others pass. shared-context: all pass (incl. 2 new). - `STANDARDS_DIR=... bash tests/fleet-report-back-test.sh`: 206 passed, 0 failed (192 without STANDARDS_DIR, renderer parts SKIP); build test 65 passed; priority-drift, fix-fleet-enrolment, rule-ids tests pass. - clippy `-D warnings` and `cargo fmt --check` clean for ponsbot, panicbot, shared-context. ## Not done / unverified - Reusable workflow does not install pons yet: an enrolled ponsbot reads `not_run: pons is not installed` until phase 1 (nothing is enrolled). - The `--pons` install is tested with a stubbed cargo, not a live `cargo install`. - SARIF (phase 3), fleet dispatch / `all()` (phase 4) deferred. Reviewers: this touches panicbot's scan strictness; a panic-attack version that exits non-zero on findings would now read `error` (the direct lane does the same; panic-attack 2.5.5 `assail` exits 0 with findings). --- _Generated by [Claude Code](https://claude.ai/code/session_0136P7ZkERqsQWMogHRpM2z8)_ --------- Co-authored-by: CLAUDE <355856132+HoTT-stuff@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>



Summary
This PR is a design for retiring the estate's direct pons-asinorum and panic-attack lanes in favour of the fleet adapters
panicbot(exists) andponsbot(new). It adds one file,docs/migrations/scanner-cutover-ponsbot-panicbot.adoc. Design only: nothing is cut over, and no workflow, enrolment, grid row or pin changes.The doc covers:
errorhard gate, panic-attack block-on-critical, must-run checks).Notable finding
panicbot-sweep.ymlinstalls fromhyperpolymath/panic-attacker.git, unpinned, while every other site usespanic-attack. It's recorded as D5 and was not changed.Test plan
hypatia-scan-reusable.yml, panicbot's build infleet-report-back-build.sh, and thatpropagate-workflow-pins.shexists.Generated by Claude Code