Skip to content

Harden the codebase: types, module layers and checks catch wrong code - #11

Merged
gabe4coding merged 1 commit into
mainfrom
claude/codebase-hardening-audit-c98be0
Oct 13, 2026
Merged

gabe4coding merged 1 commit into
mainfrom
claude/codebase-hardening-audit-c98be0

Conversation

@gabe4coding

Copy link
Copy Markdown
Owner

Summary

This change implements the 13 findings of a hardening audit. Each fix moves a class of mistake from code review to a check that fails in CI.

  • Types: statuses, levels, verdicts and actions are Literal types. pyright (standard mode, exhaustive match) now runs in make check.
  • Exit codes: one ExitCode enum. A usage error exits with 1.
  • Findings: Finding is frozen, and blocking is computed from the corpus and the verification. Only validate_findings and read_findings make a Finding (new findings.py).
  • Processes: git.py and claude.py are the only modules that start processes.
  • Module layers: import-linter checks that deterministic modules never import a model module, and that only shall.jev imports the TypeSafe SDK.
  • Config: unknown keys and bad hooks.actions rows in the local shall.yaml stop shall at load. In the extended org config, shall drops a row that it cannot read and writes a note to stderr, so the hooks do not fail open.
  • Models: the agents and the review model are pinned to claude-sonnet-5, and a test checks that they agree.
  • Test fakes: tests/fakes.py has a FakeJev that pyright checks against JevLike.
  • RFCs: each JTOOL RFC lists the test or check for each statement.
  • CI: a new workflow runs make test and make check, and it pins the Claude Code CLI.

The plugin version is now 0.4.1, because the agents changed.

Code-review fixes

  • shall list and shall show printed "None" for an invalid status. They now print "invalid" (Rfc.status_label).
  • shall eval-verify ignored review.verify_model. Now Config.verify_model is the one rule, and verify and triage use it too.
  • load_json crashed on a cache file that is not a JSON object.
  • One unreadable org hooks.actions row made the whole hook fail open.
  • CI installed an unpinned Claude Code CLI.
  • pyright downloaded Node at run time. It now uses pyright[nodejs].

The self-review also found that run_claude retried claude -p at once (JRFC-0006.2). It now waits with exponential backoff and jitter.

Verification

  • make test: 200 passed.
  • make check: corpus lint, pyright 0 errors, import-linter 2 contracts kept, index current.
  • make self-check: OK.
  • make test-plugin: 7 passed.
  • make self-review: 0 findings.
  • /verify, through headless Claude Code with the plugin:
    • a force push was denied by JRFC-0012.2, with and without hookd;
    • a write of an AWS example key was denied by the scanner (JRFC-0004.1);
    • a file write, then Stop, gave no action and a fail-open Stop without a Jev key;
    • CLI probes showed that a bad config row stops shall or is dropped with a note, and that list and show show "invalid".

Not in this change

The self-review gave two advisory findings, JTOOL-0002.6 and JTOOL-0004.2. They are about code that existed before this change and only moved to findings.py. A separate task is fixing them.

Known friction: shall writes the note about dropped org rows each time it loads the config. A hook loads the config on each tool call, so the note repeats.

🤖 Generated with Claude Code

Implement the 13 findings of the hardening audit. Each one moves a
mistake from code review to a check that fails in CI:

- Statuses, levels, verdicts and actions are Literal types. pyright
  (standard mode, exhaustive match) runs in `make check`.
- Exit codes are one ExitCode enum. A usage error exits with 1.
- Finding is frozen. Its `blocking` value is computed from the corpus
  and the verification. Only validate_findings and read_findings make
  a Finding.
- git.py and claude.py are the only modules that start processes.
  import-linter checks that deterministic modules never import a model
  module and that only shall.jev imports the TypeSafe SDK.
- The config rejects unknown keys and bad hooks.actions rows in the
  local shall.yaml. In the extended org config, shall drops a row that
  it cannot read and writes a note to stderr, so the hooks do not fail
  open.
- Agents and the review model are pinned to claude-sonnet-5. A test
  checks that they agree.
- tests/fakes.py gives a FakeJev that pyright checks against JevLike.
- The JTOOL RFCs list the test or check for each statement.
- CI runs `make check` and pins the Claude Code CLI.

Code-review findings that were fixed (the code that showed each one):
- cli.py cmd_list printed "None" for an invalid status: format of
  rfc.status. Now Rfc.status_label.
- cli.py cmd_eval_verify ignored review.verify_model. Now
  Config.verify_model, also used by verify.py and triage.py.
- config.py load_json crashed on a cache file that is not an object.
- hooks.py action_for raised on an org row that it could not read, so
  the whole hook failed open. Now config.py drops the row at load.
- ci.yml installed an unpinned Claude Code CLI.
- pyproject.toml: pyright needed a network download of Node. Now
  pyright[nodejs].

Self-review: claude.py run_claude retried the `claude -p` call at once,
with no delay (JRFC-0006.2). Now it waits with backoff and jitter.

Verification: make test (200 passed), make check, make self-check,
make test-plugin (7 passed), make self-review (0 findings), /verify
(hook drives: force push and secret key denied, write and Stop
fail-open; CLI probes for config and status).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gabe4coding
gabe4coding merged commit d293ba6 into main Oct 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants