From 455dd47bf2207f2960cd7c39c424b3dc8e8f4160 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Thu, 3 Sep 2026 20:19:27 -0400 Subject: [PATCH] docs: the check-in door and the `.cortex/pull.json` contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Brain's cortex conductor gained one door — `pyauto-brain cortex checkin` — that pulls every active project through its own `sync_cli`, scores every live phase, moves what came back, re-renders the board, pushes the ledger where the rule allows and summarises by project. This is the Cortex's half of it: the pointer a reader arrives at, and the contract the door writes against. - `REFERENCE.md` gains a `## Check-in` section: the flags, the sweep rule (`status: active` plus any project owning a live phase), the per-project failure isolation, a table of every `.cortex/pull.json` key and who writes it, why the file is merged and never clobbered (autolens_profiling's own `hpc/sync` writes the `checkpoints`/`runs` tables that make the checkpoint leg scorable at all), and the push rule — `gh auth status` plus a clean `main`, `claude/checkin-` from a fresh `origin/main`, explicit paths, a code-classified diff refused before the branch is cut. - `README.md` and `AGENTS.md` name `checkin` as the one command rather than `collect --pull --apply`, which is now the scorer it composes. The manifests live outside this repository, in the science trees, so `cortex.py check` says nothing about them — deliberately: this repo holds rulings, not mirrors. Stacked on #10 — retargets to main when it merges. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_013HsZA1ufn3msgPiDFxEXa6 --- AGENTS.md | 8 +++++-- README.md | 8 ++++--- REFERENCE.md | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 77 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b5dc90c..0211267 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -115,6 +115,7 @@ state table. | Verb | What it does | |------|--------------| +| `checkin [--dry-run \| --apply] [--push \| --no-push] [--project KEY] [--skip-pull]` | **the check-in** — the one door: pull every active project through its own `sync_cli`, score every live phase, move what came back, re-render the board, push the ledger where the rule allows, and summarise **by project**. `--dry-run` is the default and reaches nothing | | `census [--json]` | what the Cortex is holding, by state — the one-screen answer | | `dashboard --check` \| `--apply` | render `dashboard.md` + `dashboard.html`; `--check` exits **1 on drift**, **2 on bad args**, anything else = the renderer could not run | | `gates` | every gated phase, its refs and their URLs — read-only, offline | @@ -126,8 +127,11 @@ spells it `--write`. **Nothing here submits a job.** A human runs the project's own `sync_cli submit` line and the `cortex.py move submitted --run ` follow-up. -`collect --pull` is the one leg that shells out, and only to the project's own -CLI. +`checkin` and `collect --pull` are the only legs that shell out, and only to +the project's own CLI — the conductor adds no SSH of its own. Each project +`checkin` pulls gets a `/.cortex/pull.json` manifest +(REFERENCE.md "Check-in"), merged into whatever that project's CLI already +wrote there. ### What runs by itself diff --git a/README.md b/README.md index b17f2f2..2a3497c 100644 --- a/README.md +++ b/README.md @@ -35,9 +35,11 @@ In full those four steps are ten states — `planned`, `gated`, `ready`, or `dropped` — of which only the last three are reachable, and only through `scripts/cortex.py rule` (`legacy` and `legacy_wrong` are states of a *run*, never of a phase). Checking in is one command from the Brain's cortex -conductor — `pyauto-brain cortex collect --pull --apply` pulls every submitted -or running phase's results, scores them and moves what came back to -`awaiting-ruling`. `batches/` is closed history: the three 2026-08/09 batch +conductor — **`pyauto-brain cortex checkin --apply`** (see `--dry-run` first) +pulls every active project through its own sync CLI, scores every submitted or +running phase against its pre-registered witness, moves what came back to +`awaiting-ruling`, re-renders the board, pushes the ledger where it is allowed +to, and prints a summary **by project** with the next prompt for each phase. `batches/` is closed history: the three 2026-08/09 batch records and the human's verbatim reviews stay readable because 13 rulings cite them, and nothing writes new ones. diff --git a/REFERENCE.md b/REFERENCE.md index 0cf46d0..534a18e 100644 --- a/REFERENCE.md +++ b/REFERENCE.md @@ -485,20 +485,84 @@ Stdlib only; `main(argv)`; no import-time side effects; every verb takes --- +## Check-in + +**One command.** `pyauto-brain cortex checkin` is the door the human types when +they want to know where their science is: + +```bash +pyauto-brain cortex checkin --dry-run # what it would pull and score +pyauto-brain cortex checkin --apply # pull, score, move, render, push +pyauto-brain cortex checkin --apply --no-push # ... without the push +pyauto-brain cortex checkin --apply --project subhalo_validation +pyauto-brain cortex checkin --apply --skip-pull # re-score what is already here +``` + +It sweeps every `status: active` row of `projects.yaml`, plus any project that +owns a phase in `submitted | running`, and runs **that project's own** +`/ pull` — the verb all seven implement. Nothing else +reaches a cluster; the conductor adds no SSH. A pull that exits non-zero is +recorded against its project and the sweep continues. Then it scores every live +phase (the six legs below), moves `running → pulled → awaiting-ruling`, +re-renders `dashboard.md` + `dashboard.html`, and prints a summary keyed **by +project** — the last thing on screen, so a chat sees it first. + +### `.cortex/pull.json` — the pull manifest + +After a successful pull the door writes the manifest at the top of that +project's pull root (its `mirror` if it has one, else its `local_path`) — +the same file `collect`'s checkpoint leg reads: + +| Key | Written by | Meaning | +|---|---|---| +| `project` | the check-in | the `projects.yaml` key this tree belongs to | +| `pulled_at` | the check-in | UTC `YYYY-MM-DDTHH:MMZ` of the pull that just finished | +| `cmd` | the check-in | the pull as a human would type it (`cd && pull`) | +| `rc` | the check-in | that pull's exit code — `0`, or the manifest is not written | +| `phases_live` | the check-in | the phase paths that were `submitted \| running` when it ran | +| `schema` | a project's own CLI | `1` where the CLI writes the richer shape; absent = the older `runs`-only one | +| `checkpoints` | a project's own CLI | `{"": {"bytes": N, "mtime": ISO}}` | +| `runs` | a project's own CLI | `{"": {"checkpoint_bytes": N, "checkpoint_mtime": ISO}}` | + +**Merged, never clobbered.** `autolens_profiling`'s own `hpc/sync` already +writes `checkpoints` / `runs` here — the only window onto the RAL-only +`search_internal/checkpoint.hdf5`, which no pull mirrors. The check-in adds its +five keys and leaves every other key exactly as it found it, so the richer +manifest keeps making the checkpoint leg scorable. Without a manifest that leg +is `UNOBSERVABLE`, not `FAIL`. + +The manifests live **outside** this repository, in the science trees, so +`cortex.py check` says nothing about them; the check-in's own note line +(`: pulled → `) is the record that one was written. + +### The push + +`--push` is allowed only when **`gh auth status` succeeds** *and* **this +checkout is clean on `main`** — checked before anything is written, since +"clean" stops being true the moment the phases move. That is the cloud/laptop +split and it is also the default: a laptop pushes without asking, a cloud +session cannot and says so. The push cuts `claude/checkin-` from a +fresh `origin/main`, commits the changed paths explicitly and pushes; +`ledger_merge.yml` then merges it into `main` and deletes the branch (below). +`scripts/ledger_merge.py classify` is asked **first** and a code-classified +diff — `projects.yaml`, `scripts/`, the prose pages — is refused before the +branch is cut. Never `main` directly, never `--force`. + ## Driving the Cortex — the conductor and the workflows The Cortex is state plus `cortex.py`. The reasoning over it — the board and the check-in — is the Brain's **cortex conductor**: ```bash +pyauto-brain cortex checkin [--dry-run|--apply] [--push|--no-push] [--project KEY] [--skip-pull] [--refreshed ISO] pyauto-brain cortex census [--json] # what is held, by state pyauto-brain cortex dashboard --check | --apply # render the two pages pyauto-brain cortex gates # the gated phases and their refs pyauto-brain cortex collect [--phase REL] [--pull] [--refreshed ISO] [--apply] [--out F] ``` -`collect` with no `--phase` scopes to **every** phase in `submitted | running` -— that is the check-in: pull what came back, score it, move it on. +`checkin` composes the others (above). `collect` with no `--phase` scopes to +**every** phase in `submitted | running` — the scorer on its own. With no Brain install, or from a workflow: