Skip to content

feat(cortex): the one check-in door — pull, score, render, push, summarise by project #349

Closed

Description

@Jammy2211

Overview

Phase 2 of the cortex-checkin epic. Phase 1 shed the review-slot and gate apparatus and left the check-in primitives decoupled (run_pull, score_phase, collect_report, apply_ops, the payload builders, dashboard --apply). This phase composes them into one door — pyauto-brain cortex checkin — so that "check in on my science" is a single command: pull every active project, score every live phase, move what finished, regenerate the board, optionally push the ledger, and print a summary keyed by project that a fresh chat can paste from. It also installs /cortex, which skills/COMMANDS.md has advertised since the conductor was born but which was never symlinked, and rewrites the skill body so it is the check-in sequence rather than a verb menu.

Stacked on the unmerged phase-1 PRs (PyAutoBrain#348, PyAutoCortex#10) — both repos branch from feature/cortex-checkin-p1-shed-review-slot and the PRs target it; they retarget to main when phase 1 merges.

Plan

  • Add a checkin verb to the Cortex conductor that composes phase 1's primitives: sync → score → render → (optionally) push → summarise.
  • --dry-run lists every status: active project with the exact <local_path>/<sync_cli> pull it would run and every live phase it would score; it touches nothing and never reaches the cluster.
  • --apply runs each project's own sync CLI with its output streamed, isolates per-project failures (a non-zero pull is recorded and the sweep continues), writes/merges <pull root>/.cortex/pull.json, scores every submitted|running phase, applies the state moves, and regenerates dashboard.md/.html.
  • --push (guarded: gh auth status succeeds and the Cortex checkout is clean on main) cuts claude/checkin-<date> from a fresh origin/main, refuses a code-classified diff, commits ledger paths explicitly, pushes, and states that ledger_merge.yml will land it. Never main, never --force.
  • The last thing printed is the by-project summary: state counts, each live/awaiting/ready phase with its health verdict and the copy-ready prompt that already exists for that state, the project's paths and the run directories the scorer touched.
  • Rewrite skills/cortex/SKILL.md + cortex.md so the body is the check-in sequence (verbs demoted to an appendix), refresh the stale /cortex row in COMMANDS.md and the conductor's AGENTS.md, install the slash command via bin/install.sh (Claude commands + committed .claude/.codex discovery), and add a short "check in" pointer to PyAutoCortex's README/AGENTS plus a ## Check-in section in REFERENCE.md documenting the .cortex/pull.json keys.
  • Brain tests cover the dry-run listing, per-project pull-failure isolation, manifest write/merge, the no-push path and the push refusal on a code-classified diff.
Detailed implementation plan

Affected Repositories

  • PyAutoBrain (primary) — the conductor, the skill, the tests, the installed command
  • PyAutoCortex — docs only (README/AGENTS pointer, REFERENCE.md "Check-in")

Branch Survey

Repository Current Branch Dirty?
./PyAutoBrain main clean
./PyAutoCortex main clean

Suggested branch: feature/cortex-checkin-p2-the-door, cut from origin/feature/cortex-checkin-p1-shed-review-slot in both repos (stacked; PRs target that branch).

Implementation Steps

  1. agents/conductors/cortex/_cortex.py — new section # ---- the check-in ---- holding:
    • checkin_projects(mod, root, projects, only) → the ordered project keys: every row with status: active, plus any project owning a phase in submitted|running, narrowed by --project.
    • pull_cmd(row) → (cmd, cwd) or a reason string when the row has no pull verb.
    • run_pull_streamed(projects, keys) → runs each CLI with output inherited (not captured — a pull takes minutes), returns {key: (rc, note)}; a non-zero exit or an OSError is recorded and the sweep continues.
    • pull_root(row) → mirror if set else local_path; write_pull_manifest(root_dir, key, cmd, rc, phases_live) → merges {project, pulled_at, cmd, rc, phases_live} into any existing .cortex/pull.json (autolens_profiling's own sync CLI writes a richer one — keys are merged, never clobbered).
    • project_digest(mod, key, row, rows, scored) → one project's summary block; checkin_summary(...) → the whole by-project summary (printed last).
    • push_preflight(root) → (ok, reason) from gh auth status + branch main + clean tree, evaluated before any write; push_ledger(root, paths, date) → fetch, checkout -b claude/checkin-<date> origin/main, ledger_merge.py classify <paths> (refuse on code), add -- <paths>, commit, push, print the auto-merge expectation.
    • cmd_checkin(root, mod, a) → the door.
  2. build_parser() — checkin sub-parser: --dry-run, --apply, --push/--no-push, --project KEY (repeatable), --skip-pull, --refreshed ISO, --cortex DIR. Default is dry-run; push defaults to the preflight verdict and the resolution is printed.
  3. main() — dispatch checkin beside collect, with the same CortexError/OSError handling.
  4. Module docstring + agents/conductors/cortex/AGENTS.md — checkin documented as the front door, collect as the primitive it composes.
  5. skills/cortex/SKILL.md + skills/cortex/cortex.md — body becomes the check-in sequence; the verb table moves to an appendix; the stale plan / gates --grade rows (retired in phase 1) go.
  6. skills/COMMANDS.md — the /cortex row names the check-in, not the retired verbs.
  7. tests/test_cortex_conductor.py — a # --- the check-in --- block over the existing board fixture: dry-run listing, per-project pull failure isolation (a fake sync_cli exiting 1 for one project only), manifest write + merge of an existing manifest, the no-push path, and the push refusal on a code-classified diff (a throwaway git repo, no remote).
  8. bash bin/install.sh — installs ~/.claude/commands/cortex.md + the Claude/Codex skills; --write-project-discovery adds the committed .claude/.codex links for the cortex skill.
  9. PyAutoCortex: README.md + AGENTS.md short "check in" pointer; REFERENCE.md new ## Check-in section documenting the .cortex/pull.json keys and the merge rule.

Key Files

  • PyAutoBrain/agents/conductors/cortex/_cortex.py — the conductor (the door lands here)
  • PyAutoBrain/agents/conductors/cortex/AGENTS.md — the conductor's own doc
  • PyAutoBrain/skills/cortex/{SKILL.md,cortex.md} — the /cortex body
  • PyAutoBrain/skills/COMMANDS.md — the command veneer index
  • PyAutoBrain/tests/test_cortex_conductor.py, PyAutoBrain/tests/_cortex_board.py — the fixture and tests
  • PyAutoCortex/{README.md,AGENTS.md,REFERENCE.md} — the pointer and the manifest contract

Testing

python3 -m pytest tests/test_cortex_conductor.py in the Brain worktree (with PYAUTO_CORTEX pointed at the Cortex worktree), plus the full Brain suite. No real hpc/sync pull is run at any point — every pull under test is a fake sync_cli script; the only live exercise is checkin --dry-run, which SSHes nowhere. The known worktree-only failure test_cortex_conductor::test_a_fixture_tree_finds_the_schema_its_checkout_ships is expected.

Original Prompt

Click to expand starting prompt
# `/cortex` — the one check-in door: pull every active project, score, render, push, hand back prompts

Type: feature
Target: pyautobrain
Repos:
- PyAutoBrain
- PyAutoCortex
Themes:
- mind-workflow
Difficulty: medium
Autonomy: supervised
Priority: high
Status: draft
Consequence: judge
Witness: `pyauto-brain cortex checkin --dry-run` lists every `status: active` project in `projects.yaml` with the exact `<local_path>/<sync_cli> pull` it would run and every live phase it would score; `pyauto-brain cortex checkin --apply --no-push` on the live tree pulls each (a failing pull is reported per project and does not abort the sweep), writes `<local_path>/.cortex/pull.json` for each project pulled, scores every submitted/running phase, applies `running → pulled → awaiting-ruling`, regenerates `dashboard.md`/`.html` (`--check` current), and prints a summary keyed BY PROJECT (state counts, live phases with health, the existing rule / launch / jobs prompts) that a fresh chat can paste from; with `--push` it commits the ledger diff on a `claude/checkin-<YYYY-MM-DD>` branch with explicit paths, pushes, and reports that `ledger_merge.yml` will land it; `~/.claude/commands/cortex.md` exists and `/cortex` in Claude reads `skills/cortex/SKILL.md`, whose body is the check-in sequence, not a verb menu; Brain + Cortex tests cover dry-run, per-project pull failure isolation, manifest write and the no-push path
Review-minutes: 20
Unattended: ready
Epic: cortex-checkin
Phase: 2
Filed: 2026-09-03

Phase 2 of `cortex-checkin`. Branch from phase 1's branches (stacked) if they
are unmerged. Two PRs (Brain, Cortex — the Cortex side is docs + a
`checkin` pointer in README/AGENTS + any `check` rule the manifest needs).

## The door — `pyauto-brain cortex checkin`

Composes the primitives phase 1 kept. Flags: `--dry-run`, `--apply`,
`--push` / `--no-push` (default: `--no-push` in a cloud session, ask-free
`--push` on a laptop where `gh auth status` succeeds — state the rule),
`--project <key>` to narrow, `--skip-pull` for an offline re-score.

1. **Sync.** Iterate `projects.yaml` rows with `status: active` (plus any row
   that owns a phase in `submitted|running`). For each, run
   `<local_path>/<sync_cli> pull` (the verb every CLI implements — verified
   2026-09-03 across all seven), streaming its output under a project
   heading. A non-zero exit is recorded per project and the sweep continues.
   After a successful pull write `<local_path>/.cortex/pull.json`
   `{project, pulled_at, cmd, rc, phases_live: [...]}` so the scorer's
   checkpoint leg stops reading UNOBSERVABLE everywhere except
   autolens_profiling (which already writes its own — do not clobber a
   richer manifest; merge keys).
2. **Score.** Run the decoupled `collect` scorer over every submitted/running
   phase; `--apply` performs the state moves. No batch record.
3. **Render + push.** `dashboard --apply`; then, with `--push`, in the Cortex
   checkout: `git checkout -b claude/checkin-<date>` from a fresh `origin/main`,
   commit the phase files + dashboard pages with explicit paths, push, and
   print the auto-merge expectation (`ledger_merge.yml`). If the diff
   classifies as code (`scripts/ledger_merge.py classify <paths>`), stop
   before pushing and say why. Never push `main` directly, never force.
4. **Summarise, by project.** For each active project: state counts, each
   live/awaiting/ready phase with its health verdict and the copy-ready
   prompt that already exists for that state (`_ruling_payload`,
   `launch_payload`, `_live_payload`), the project's `local_path`, `mirror`,
   `ral_root` and the run directories the scorer touched. Phase 3 enriches
   this; here it must at least exist and be keyed by project. Print it as
   the LAST thing so a chat sees it above the fold.

## Install `/cortex`

The slash command is advertised in `skills/COMMANDS.md` but
`~/.claude/commands/cortex.md` does not exist. Rewrite
`skills/cortex/SKILL.md` + `cortex.md` so the body IS the check-in
sequence (run `checkin`, read the summary, offer the prompts; the individual
verbs become an appendix) and install the symlink the same way the other 29
commands are installed (find the installer — PyAutoBrain owns discovery per
PyAutoMind/AGENTS.md — and add `cortex`; if installation is a documented
manual step, do it and document it). Codex discovery likewise if the others
have it.

## Guardrails

Never arm a timer, subscription, cron or loop; the door runs once and ends.
Never touch `rulings/` (append-only, human's turn). `hpc/sync pull` may take
minutes per project — stream output, do not buffer. RAL is reached only via
the projects' own CLIs; the door adds no SSH of its own.

Activity

  1. Jammy2211 commented on Sep 4, 2026

    @Jammy2211
    ContributorAuthor

    Shipped to PR-open (merge stays human).

    Both retarget to main once phase 1 merges. Merge order: PyAutoBrain#348 → PyAutoCortex#10 → PyAutoBrain#350 → PyAutoCortex#11.

    The CLI as shipped

    pyauto-brain cortex checkin [--dry-run|--apply] [--push|--no-push]
                                [--project KEY]... [--skip-pull] [--refreshed ISO]
                                [--cortex DIR]
    

    --dry-run is the default and reaches nothing. --push resolves from the rule — gh auth status succeeds and the Cortex is clean on main — read before any write, and stated in the output either way.

    Live check, no RAL traffic: checkin --dry-run on the real tree lists 3 active projects (euclid, inference_programme, subhalo_validation), the exact cd … && hpc/sync pull for each, and 2 live phases (both subhalo_validation). --apply was exercised only against fixtures — a real pull is minutes of cluster traffic and the human's call.

    Tests: Brain 874 passed / 3 failed → one was this branch's (test_gh_surface, fixed); the two test_branch_sweep failures reproduce unchanged on the phase-1 base. Targeted re-run: 68 passed. Cortex: cortex.py check OK, 115 passed, dashboard current.

    /cortex installed: bash PyAutoBrain/bin/install.sh created ~/.claude/commands/cortex.md, ~/.claude/skills/cortex and ~/.codex/skills/cortex (and pruned two stale links, register-and-iterate / run-queue, whose skills no longer exist). The installer needed no change — nobody had run it since the skill dir was added.

    Heart: RED at ship time, unacknowledged — release validation FAILED (stage integrate) (red) and PyAutoArray: open PR 11d old (yellow). Both are release-chain facts about other repos; neither of these two is in the release chain. PRs opened, nothing merged.

  2. Jammy2211 commented on Sep 4, 2026

    @Jammy2211
    ContributorAuthor

    Shipped

    Phase 2 of the cortex-checkin epic — the /cortex check-in door.

    PR Merge sha
    #350 9a039d92a5ac2c9c1ebce52636bc3b42acedd4e8
    PyAutoLabs/PyAutoCortex#11 284097de3787aac22156dd1337e2c6749a9d5088

    Both merged to main on 2026-09-04, Brain before Cortex, after phase 1
    (PyAutoBrain#348 / PyAutoCortex#10) had landed. GitHub retargeted both PRs from
    the phase-1 branch to main automatically once phase 1 merged.

    PyAutoBrain#350: Brain Tests green on 3.12 and 3.13, docs / docs-build
    green. PyAutoCortex#11's effective diff against main is three documentation
    files (AGENTS.md, README.md, REFERENCE.md), which match no path filter in
    either Cortex workflow, so no run fired — by design, not a missing check.

    Heart was RED at merge time with two reasons — "release validation FAILED
    (stage integrate)" and "PyAutoArray: open PR 12d old" — both release-chain facts
    about other repos, neither of which is in the release chain. Merged under an
    explicit human acknowledgement.

    Closed by /prm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions