[gabe-bots]

Intent

Start the shadow week of the validation loop on this repo, the same setup as gabe4coding/puredori#10 and gabe4coding/retroagent#58. Two new checks run on every PR. Both are advisory: they are not required, they never fail a PR, and nothing here merges, approves or changes repo settings.

• review (.github/workflows/review.yml): two passes in one job.
  1. shall: the diff against the engineering standards, adapted from shall's ci/github/shall-review.yml.
  2. General pass (.github/scripts/general_review.py): one tool-less, isolated claude -p call that checks for bugs, missing or weaker tests, silent behavior changes, PR body vs diff drift and irreversible changes. The PR text is passed as untrusted data. It also records the risk-judgment JSON from the design, which does not change the tier yet.
  • Findings go in PR comments and the job summary. This week the job always ends green.
  • If secrets are missing, each pass skips with a neutral ::notice that names the secret.
• risk (.github/workflows/risk.yml, .github/scripts/risk_tier.py, .github/risk.yml): the deterministic tiers HOLD / T2-hard / T2-soft / T1 / T0.
  • The rules and the script are read from the base branch, so a PR cannot loosen its own tier. This PR is the bootstrap and uses its own copy.
  • The tier is posted as a risk tier check run (success for T0/T1, neutral for T2/HOLD, never failure) and as a risk:* label.
  • Kill switch: if the repo variable AUTOMERGE is off, every PR becomes T2-hard. It is read only.
  • There is no LLM layer yet. The vendor names that HOLD a PR are stored as sha256 hashes, so this public repo never spells them out.
• Sensitive files (T2-soft): runtime/runner.mjs (the sandbox) and tests/runner.integration.mjs, the test that proves the sandbox layers. I added the test because weakening it weakens the sandbox guarantee.
• tests/*.check.mjs and tests/check.mjs are CI gates, so they count as code, not tests.
• The new YAML files are formatted with scripts/format.sh, so the format job passes. No plugin files change, so no version bump.
• Tests: .github/scripts/test_risk_tier.py and test_general_review.py, run by the risk (classifier tests) job.

Secrets to add (Settings > Secrets and variables > Actions):

• TYPESAFE_API_KEY: needed for shall's Jev rule selection.
• ANTHROPIC_API_KEY or CLAUDE_CODE_OAUTH_TOKEN: needed for claude -p in both passes.
• Data note: diffs and PR text go to TypeSafe (shall) and Anthropic (both passes).

This PR is T2-hard by its own rules, because it adds .github/risk.yml and the review and risk workflows. Gabe reviews and merges it.

Evidence

• uv run --no-project --python 3.12 --with pyyaml==6.0.2 --with pytest -- python -m pytest -q .github/scripts passes. actionlint is clean on both workflows.
• Repo checks: sh scripts/format.sh --check ., node tests/code.check.mjs, node tests/docs.check.mjs and node tests/version.check.mjs origin/main all pass ("no plugin files changed").
• The classifier on this branch gives T2-hard (hard paths: .github/risk.yml, the scripts and both workflows).
• Backtest on the last 20 merged PRs (Count direct MCP calls in the bench #13–run_code: name the tools that need an allow rule; auto mode docs for 2.1.301 #32), using these rules (PR base and head from GitHub, diff from the merge base):

PR	Title	Size	Tier	Why
#32	run_code: name the tools that need an allow rule; auto mode docs for 2	+213/-43, 9 files	T1	source-change: 4 code files, 84 changed lines
#31	Sandbox: start Node with no environment variables	+76/-22, 6 files	T2-soft	sensitive-path: touches runtime/runner.mjs; sensitive-path: touches tests/runner.integration.mjs
#30	README refresh: sharper hook, install first, demo section	+32/-32, 1 files	T0	docs-only: 1 files
#29	Run stale decision-key deletes in parallel	+2/-2, 2 files	T1	source-change: 2 code files, 4 changed lines
#28	Format the code with Prettier before each commit	+2289/-625, 33 files	T2-hard	hard-path: touches .githooks/pre-commit; workflow-weakened: .github/workflows/test.yml removes - cron: "0 6 * * 1"
#27	Harden the codebase: one shape, one helper and one test per mistake cl	+954/-498, 19 files	T2-soft	workflow-additive: .github/workflows/test.yml: additive edit or pinned version bump; sensitive-path: touches tests/runner.integration.mjs
#26	Let the model change hints and saved functions, with approval (0.12.0)	+940/-213, 14 files	T2-soft	sensitive-path: touches runtime/runner.mjs; sensitive-path: touches tests/runner.integration.mjs
#25	Add /code-review to the verification loop	+12/-8, 3 files	T1	source-change: 2 code files, 12 changed lines
#24	Keep only the newest 500 decisions in the store (0.11.2)	+54/-9, 4 files	T1	source-change: 3 code files, 34 changed lines
#23	Build the verification loop: /simplify, checks, /verify recipe, and a	+392/-25, 13 files	T1	source-change: 9 code files, 362 changed lines
#22	Add an end-to-end smoke test, and fix the two gaps it found (0.10.2)	+290/-8, 9 files	T1	source-change: 3 code files, 17 changed lines
#21	Give proposals a block to paste where the band does not show (0.11.0)	+178/-10, 10 files	T1	source-change: 5 code files, 92 changed lines
#20	Show calls that did not finish in the run footer (0.10.1)	+108/-16, 7 files	T2-soft	sensitive-path: touches tests/runner.integration.mjs
#19	Add the voxel builder from the explainer video to the banner	+39/-1, 1 files	T0	docs-only: 1 files
#18	Add the explainer video to the README	+2/-0, 1 files	T0	docs-only: 1 files
#17	Show proposals one at a time with labeled fields, and ask for them soo	+363/-198, 11 files	T1	source-change: 5 code files, 366 changed lines
#16	Add saved functions, MCP resources and a network block on Linux (0.9.0	+851/-61, 20 files	T2-soft	workflow-additive: .github/workflows/test.yml: additive edit or pinned version bump; sensitive-path: touches runtime/runner.mjs
#15	Add saveFile/readFile, learned argument names and a check before each	+768/-32, 12 files	T2-soft	sensitive-path: touches runtime/runner.mjs; sensitive-path: touches tests/runner.integration.mjs
#14	Add sleep(), a limit on parallel calls and learned result shapes (0.7.	+317/-17, 11 files	T2-soft	sensitive-path: touches runtime/runner.mjs; sensitive-path: touches tests/runner.integration.mjs
#13	Count direct MCP calls in the bench	+45/-11, 2 files	T1	source-change: 1 code files, 52 changed lines

Totals: T0 3, T1 9, T2-soft 7, T2-hard 1.

Things to compare by hand this week: whether each T2-soft really needed you, and whether any T1 should have been T2.

[gabe-bots]

@greptileai review

[greptile-apps]

Confidence Score: 3/5

[Critical impact] Adds the risk classification system and CI gates.

Fix the stale review tiers and the classification of weakened CI before merging.

Findings

1.  Review results leave stale tiers ▶
2.  Weakened checks get lower tiers ▶

Summary

Adds an advisory risk classifier, PR labels, check summaries, and Python tests. The general review script remains, but its workflow was intentionally removed.

• Pull requests get an advisory risk tier, check, and label.
• A Claude review command reports findings and can post them on a pull request.

Acknowledgment: Harness intentionally removed the review workflow in commit 7d77358 because Greptile is the hosted reviewer.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[PR opened or updated] --> B[Read current Greptile check]
  B --> C{Review passed?}
  C -->|Yes| D[Keep file-based tier]
  C -->|No| E[Raise T0 or T1 to T2-soft]
  D --> F[Post label and check]
  E --> F
  G[Greptile finishes later] --> H[No tier refresh]

Loading

_{Reviews (1) · Last reviewed commit: "Read the Greptile check and close the ri..." · Reviewed by Greptile}

[greptile-apps]

Review results leave stale tiers

If risk reads a missing or pending Greptile review after a push, it posts T2-soft. A later successful review does not trigger another run, so the label and check stay wrong until someone edits or updates the PR. This makes the shadow-week results misleading. Recompute the tier when the review finishes, while keeping the classifier on trusted base-branch code.

[greptile-apps]

Weakened checks get lower tiers

workflow_reason treats every addition-only edit as T2-soft. Adding continue-on-error: true to the existing sandbox test, or adding if: false to skip it, removes no lines and reaches this return. Those edits weaken CI but receive the tier intended for additive jobs instead of T2-hard. Check where added lines land and whether they disable existing checks.