Skip to content

Commit da6743f

Browse files
authored
Merge pull request #546 from PyAutoLabs/feature/cortex-birth
feat(cortex): birth — register a scaffolded Science project in one step
2 parents 68916b6 + cfc137f commit da6743f

4 files changed

Lines changed: 436 additions & 0 deletions

File tree

‎agents/conductors/cortex/AGENTS.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ note`).
3131
| `checkin [--dry-run \| --apply] [--push \| --no-push] [--project KEY]` | **Where is my science?** | the door — the sequence below, ending in a summary keyed by project; reaches no cluster; exit **1** when the tree does not check |
3232
| `census [--json]` | What is the Cortex holding? | per project: status, summary, runs by state, log length, last update; totals (active projects, runs open, runs running); the check-in stamp; the problems `cortex.py check` would report |
3333
| `drift [--science-root PATH] [--json]` | **Does the register match the Science tree?** | the laptop Science tree (`<root>/<category>/<project>`, root derived from the rows' `local_path`s) against `projects.yaml`: errors — a folder in `active/` or `development/` with no row, a non-retired row whose folder is gone, a folder whose category disagrees with its status, a derived root that does not exist while other rows' folders do (the tree moved); warnings — an unregistered folder in `complete/` or `archive/`, a `ral_root` in a different category. Writes nothing. Exit **1** on an error · 0 otherwise, including off the laptop (a one-line skip). `census` and `checkin` print its one-line summary; `census()` itself stays drift-free so the CI board is machine-independent |
34+
| `birth <key> --category active\|development --assistant A --summary S [--apply] [--no-push]` | **Register a scaffolded project** | derives the `projects.yaml` row from `<Science root>/<category>/<key>/` (remote from its git origin, `ral_root` = the rows' RAL base + category + key, `sync_verbs` from its `hpc/sync` dispatch `case`, the ledger file) and prints it. `--apply` (needs the Cortex clean on `main`): cut `claude/birth-<key>` from `origin/main`, write through `scripts/cortex.py birth`, commit, push only when `ledger_merge.py classify --base origin/main` calls it a birth, return to `main`. Scaffolding is the assistant's `start-new-project`, never this verb; it creates no remote and reaches no cluster. Exit **2** refused · **1** git or write failed |
3435
| `dashboard --check` | Are the committed pages current? | exit 0 current · **1 stale** · 2 no checkout · 3 unreadable tree |
3536
| `dashboard --apply` | — | writes `dashboard.md` + `dashboard.html` + `state.json` — the organ-cockpit feed (`board/_state.py` v1): red on `cortex.py check` problems, yellow while a run is running or the check-in is stale (the feed's 180-min rule, judged at render), items carry `checkin_payload` / `resume_payload`; `--check` ignores its `updated` |
3637
| `issue [--project KEY] [--apply]` | What sits at the top of each project's issue? | the fenced ledger block (`cortex.py`'s `issue_block`) per project with an `Issue:`; `--apply` writes it into the issue body through `gh` (replacing the block between the markers, or prepending it); never creates an issue; exit **1** without `gh` |

‎agents/conductors/cortex/_cortex.py‎

Lines changed: 238 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@
3535
[--dry-run|--apply] [--push|--no-push] [--project KEY]` (the door, reaches no
3636
cluster) · `census [--json]` · `drift [--science-root PATH] [--json]`
3737
(the laptop Science tree against `projects.yaml`; reads, never writes) ·
38+
`birth <key> --category active|development --assistant A --summary S
39+
[--apply] [--no-push]` (a scaffolded project's row and ledger, written by
40+
the Cortex's own `cortex.py birth` on `claude/birth-<key>`) ·
3841
`dashboard --check|--apply` · `issue [--project KEY] [--apply]`.
3942
4043
Exit codes: 0 ok · 1 dashboard drift (the `dashboard_refresh.yml` contract),
@@ -1239,6 +1242,221 @@ def cmd_drift(root: Path, mod, a) -> int:
12391242
return RC_DRIFT if report["errors"] else RC_OK
12401243

12411244

1245+
# ------------------------------------------------------------ the birth ---
1246+
# Scaffolding belongs to the domain assistant (`start-new-project`); the
1247+
# birth is the Cortex half — a row and a ledger for a project folder that
1248+
# already exists. The conductor derives the row and hands it to the Cortex's
1249+
# own `scripts/cortex.py birth`, the verb that writes; it writes nothing of
1250+
# its own here but a git branch.
1251+
BIRTH_BRANCH_PREFIX = "claude/birth-"
1252+
BIRTH_CATEGORIES = ("active", "development")
1253+
_DISPATCH_RE = re.compile(r'^case "\$\{?cmd\}?" in$(.*?)^esac$', re.M | re.S)
1254+
_LABEL_RE = re.compile(r"^\s+([a-z][a-z0-9-]*)(?:\|[^)\n]*)?\)", re.M)
1255+
_GITHUB_RE = re.compile(r"github\.com[:/]([\w.-]+/[\w.-]+?)(?:\.git)?/?$")
1256+
LEDGER_CANDIDATES = ("wiki/project/state.md", "README.md", "AGENTS.md")
1257+
1258+
1259+
def sync_verbs_of(cli: Path) -> list[str]:
1260+
"""The verbs a project's sync CLI dispatches: the labels of its
1261+
`case "$cmd" in … esac` block, in order — `help` aside, which is the
1262+
CLI's usage, not a verb a conductor runs. Empty when there is none."""
1263+
if not cli.is_file():
1264+
return []
1265+
m = _DISPATCH_RE.search(cli.read_text(encoding="utf-8", errors="replace"))
1266+
verbs: list[str] = []
1267+
for verb in _LABEL_RE.findall(m.group(1)) if m else []:
1268+
if verb != "help" and verb not in verbs:
1269+
verbs.append(verb)
1270+
return verbs
1271+
1272+
1273+
def remote_of(local: Path) -> str:
1274+
"""`owner/repo` of the project's GitHub `origin`, else `none`."""
1275+
r = _git(local, "remote", "get-url", "origin")
1276+
m = _GITHUB_RE.search(r.stdout.strip()) if r.returncode == 0 else None
1277+
return m.group(1) if m else "none"
1278+
1279+
1280+
def ral_base_from(projects: dict) -> str | None:
1281+
"""The RAL base the rows agree on: the most common `<base>` of the
1282+
non-retired `ral_root`s shaped `<base>/<category>/<project>`."""
1283+
bases = [str(Path(ral).parent.parent) for row in projects.values()
1284+
if row.get("status") != "retired"
1285+
for ral in [_norm(row.get("ral_root", ""))]
1286+
if ral and Path(ral).parent.name in SCIENCE_CATEGORIES]
1287+
return max(sorted(set(bases)), key=bases.count) if bases else None
1288+
1289+
1290+
def birth_row(projects: dict, key: str, category: str, assistant: str, *,
1291+
science_root: str | None = None, partition: str = "ral",
1292+
sync_verbs: list | None = None) -> tuple[dict, list[str]]:
1293+
"""`(row, problems)` — the `projects.yaml` row a birth would write for
1294+
`<science root>/<category>/<key>`, derived from the folder itself and from
1295+
the other rows. Any problem refuses the birth."""
1296+
problems: list[str] = []
1297+
root = science_root or science_root_from(projects)
1298+
if category not in BIRTH_CATEGORIES:
1299+
problems.append(f"a project is born into {' or '.join(BIRTH_CATEGORIES)}/, "
1300+
f"not {category}/")
1301+
if not root:
1302+
return {}, problems + ["no row places a project in a Science category, so "
1303+
"the Science root is unknown — pass --science-root"]
1304+
local = Path(_norm(root)) / category / key
1305+
if key in projects:
1306+
problems.append(f"{key} is already a projects.yaml key")
1307+
if any(_real(local_root(r)) == _real(str(local)) for r in projects.values()):
1308+
problems.append(f"{local} is already some row's local_path")
1309+
if not local.is_dir():
1310+
problems.append(f"{local} does not exist — scaffold it first (the "
1311+
"assistant's start-new-project)")
1312+
verbs = list(sync_verbs or sync_verbs_of(local / "hpc" / "sync"))
1313+
if local.is_dir() and not verbs:
1314+
problems.append(f"no verbs found in {local}/hpc/sync (its `case \"$cmd\" "
1315+
"in` block) — scaffold hpc/ or pass --sync-verbs")
1316+
base = ral_base_from(projects)
1317+
if not base:
1318+
problems.append("no row's ral_root is shaped <base>/<category>/<project>, "
1319+
"so the RAL base is unknown")
1320+
ledger = next((c for c in LEDGER_CANDIDATES if (local / c).is_file()),
1321+
LEDGER_CANDIDATES[0])
1322+
row = {
1323+
"remote": remote_of(local) if local.is_dir() else "none",
1324+
"local_path": str(local),
1325+
"ral_root": f"{base}/{category}/{key}" if base else "",
1326+
"mirror": "none",
1327+
"sync_cli": "hpc/sync",
1328+
"sync_verbs": verbs,
1329+
"ledger": ledger,
1330+
"assistant": assistant,
1331+
"partition": partition,
1332+
"status": "active",
1333+
}
1334+
return row, problems
1335+
1336+
1337+
def _birth_argv(root: Path, key: str, row: dict, a) -> list[str]:
1338+
argv = [sys.executable, str(root / "scripts" / "cortex.py"), "birth", key,
1339+
"--root", str(root), "--summary", a.summary, "--issue", a.issue,
1340+
"--sync-verbs", ",".join(row["sync_verbs"])]
1341+
for field in ("local_path", "ral_root", "remote", "mirror", "sync_cli",
1342+
"ledger", "assistant", "partition", "status"):
1343+
argv += [f"--{field.replace('_', '-')}", row[field]]
1344+
if a.note:
1345+
argv += ["--note", a.note]
1346+
return argv
1347+
1348+
1349+
def _abandon_birth(root: Path, branch: str) -> None:
1350+
"""Leave the Cortex as the birth found it: nothing staged or written,
1351+
back on `main`, the birth branch gone."""
1352+
_git(root, "reset", "-q")
1353+
_git(root, "checkout", "--", ".")
1354+
_git(root, "clean", "-fdq", "--", "projects")
1355+
_git(root, "checkout", "main")
1356+
_git(root, "branch", "-D", branch)
1357+
1358+
1359+
def cmd_birth(root: Path, mod, a) -> int:
1360+
"""Register a scaffolded Science project: a row and a ledger, on
1361+
`claude/birth-<key>`. Dry run by default; never creates a remote and
1362+
never reaches RAL."""
1363+
projects, problems = mod.load_projects(root)
1364+
if problems:
1365+
print("cortex birth: projects.yaml does not check — fix it first", file=sys.stderr)
1366+
return RC_DRIFT
1367+
row, refused = birth_row(projects, a.key, a.category, a.assistant,
1368+
science_root=a.science_root or None,
1369+
partition=a.partition,
1370+
sync_verbs=[v for v in a.sync_verbs.split(",") if v] or None)
1371+
print(f"== Cortex birth — {a.key} ({'apply' if a.apply else 'dry run'}) ==")
1372+
for field, value in row.items():
1373+
shown = f"[{', '.join(value)}]" if isinstance(value, list) else value
1374+
print(f" {field + ':':<14} {shown}")
1375+
words = len(a.summary.split())
1376+
if words > mod.SUMMARY_MAX_WORDS:
1377+
refused.append(f"--summary is {words} words; the ledger title takes at most "
1378+
f"{mod.SUMMARY_MAX_WORDS}")
1379+
if refused:
1380+
for p in refused:
1381+
print(f" cannot: {p}")
1382+
return RC_USAGE
1383+
if not a.apply:
1384+
print(f"\nwould cut {BIRTH_BRANCH_PREFIX}{a.key} from origin/main, write this row "
1385+
f"and projects/{a.key}.md through scripts/cortex.py birth, and "
1386+
f"{'push it' if a.push else 'leave it committed locally'}. "
1387+
"Run with --apply to do it.")
1388+
return RC_OK
1389+
1390+
ok, why = push_preflight(root)
1391+
if not ok:
1392+
print(f"cortex birth: REFUSED — {why}; a birth is cut from a clean `main`",
1393+
file=sys.stderr)
1394+
return RC_DRIFT
1395+
branch = f"{BIRTH_BRANCH_PREFIX}{a.key}"
1396+
mine: list[dict] = []
1397+
for step in (("fetch", "origin"), ("checkout", "-b", branch, "origin/main")):
1398+
r = _git(root, *step)
1399+
if r.returncode != 0:
1400+
print(f"cortex birth: REFUSED — `git {' '.join(step)}`: "
1401+
f"{r.stderr.strip()[:200]}", file=sys.stderr)
1402+
_git(root, "checkout", "main")
1403+
return RC_DRIFT
1404+
rc = RC_OK
1405+
try:
1406+
r = subprocess.run(_birth_argv(root, a.key, row, a), capture_output=True, text=True)
1407+
if r.returncode != 0:
1408+
print((r.stderr or r.stdout).strip(), file=sys.stderr)
1409+
_abandon_birth(root, branch)
1410+
return RC_DRIFT
1411+
print(r.stdout.strip())
1412+
for step in (("add", "--", "projects.yaml", f"projects/{a.key}.md"),
1413+
("commit", "-m", f"cortex: birth of {a.key}\n\n{a.summary}\n")):
1414+
g = _git(root, *step)
1415+
if g.returncode != 0:
1416+
print(f"cortex birth: `git {step[0]}` failed — nothing committed, "
1417+
f"the branch is dropped: {(g.stderr or g.stdout).strip()[:200]}",
1418+
file=sys.stderr)
1419+
_abandon_birth(root, branch)
1420+
return RC_DRIFT
1421+
# The path-only classifier calls projects.yaml code; against a base it
1422+
# recognises an add-only row-plus-ledger diff as a birth.
1423+
birth = subprocess.run(
1424+
[sys.executable, str(root / "scripts" / "ledger_merge.py"), "classify",
1425+
"--base", "origin/main"], cwd=str(root), capture_output=True, text=True
1426+
).returncode == 0
1427+
mine = [f for f in drift_report(mod.load_projects(root)[0],
1428+
a.science_root or None)["findings"]
1429+
if f["key"] == a.key or f["path"] == row["local_path"]]
1430+
if not birth:
1431+
rc = RC_DRIFT
1432+
print(f"push: REFUSED — the classifier does not call this a birth; "
1433+
f"`{branch}` is committed locally for a human")
1434+
elif a.push:
1435+
pushed = _git(root, "push", "-u", "origin", branch)
1436+
if pushed.returncode != 0:
1437+
rc = RC_DRIFT
1438+
print(f"push: `{branch}` pushed — `ledger_merge.yml` lands a birth by itself"
1439+
if pushed.returncode == 0 else
1440+
f"push: FAILED — {pushed.stderr.strip()[:200]}; the commit is on `{branch}`")
1441+
else:
1442+
print(f"push: off — `{branch}` is committed locally")
1443+
finally:
1444+
_git(root, "checkout", "main")
1445+
1446+
print(drift_line({"science_root": row["local_path"], "present": True,
1447+
"findings": mine, "errors": sum(f["severity"] == "error" for f in mine),
1448+
"warnings": sum(f["severity"] == "warning" for f in mine)}))
1449+
print("\nNext, by hand:")
1450+
if row["remote"] == "none":
1451+
owners = [r["remote"].split("/")[0] for r in projects.values()
1452+
if "/" in (r.get("remote") or "")]
1453+
owner = max(sorted(set(owners)), key=owners.count) if owners else "<owner>"
1454+
print(f" gh repo create {owner}/{a.key} --private --source {row['local_path']}")
1455+
print(f" cd {row['local_path']} && hpc/sync push # the RAL tree, {row['ral_root']}")
1456+
print(f" git -C {root} pull # once the birth has landed on main")
1457+
return rc
1458+
1459+
12421460
def run_jobs(row: dict) -> str | None:
12431461
"""The project's own `<sync_cli> jobs`, verbatim — stdout and stderr as
12441462
they came, no parsing. None when the row has no `jobs` verb."""
@@ -1725,6 +1943,24 @@ def common(p):
17251943
"local_path)")
17261944
r.add_argument("--json", dest="as_json", action="store_true")
17271945

1946+
b = common(sub.add_parser(
1947+
"birth", help="register a scaffolded Science project: its projects.yaml "
1948+
"row and ledger on claude/birth-<key> (dry run by default)"))
1949+
b.add_argument("key", help="the project key = its folder name")
1950+
b.add_argument("--category", required=True, choices=BIRTH_CATEGORIES)
1951+
b.add_argument("--assistant", required=True,
1952+
help="the domain assistant it was scaffolded from, or none")
1953+
b.add_argument("--summary", required=True, help="the ledger title, one line")
1954+
b.add_argument("--issue", default="none", help="Repo#N or an issue URL")
1955+
b.add_argument("--note", default="", help="a free-text note for the row")
1956+
b.add_argument("--partition", default="ral", choices=("ral", "gpu", "both"))
1957+
b.add_argument("--sync-verbs", dest="sync_verbs", default="",
1958+
help="comma-separated; default: read from the project's hpc/sync")
1959+
b.add_argument("--science-root", dest="science_root", default="")
1960+
b.add_argument("--apply", action="store_true", help="cut the branch and write")
1961+
b.add_argument("--no-push", dest="push", action="store_false",
1962+
help="commit on the branch but do not push")
1963+
17281964
i = common(sub.add_parser(
17291965
"issue", help="the ledger block that sits at the top of each "
17301966
"project's issue; --apply writes it there with gh"))
@@ -1757,6 +1993,8 @@ def main(argv=None) -> int:
17571993
return cmd_issue(root, mod, a)
17581994
if verb == "drift":
17591995
return cmd_drift(root, mod, a)
1996+
if verb == "birth":
1997+
return cmd_birth(root, mod, a)
17601998
c = census(root)
17611999
except mod.CortexError as e:
17622000
print(f"cortex: {root}: {e}", file=sys.stderr)

‎skills/cortex/cortex.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,29 @@ nothing until they say.** Work on the project itself is delegated the way
108108
[`../WORKFLOW.md`](../WORKFLOW.md) "Cortex project work" says; the resume is
109109
the entry protocol, not the work.
110110

111+
### 6. Start a new project
112+
113+
A new Science project is born in two halves, in this order:
114+
115+
1. **Scaffold** — through the domain assistant the project will work with
116+
(`autolens_assistant` or `autofit_assistant`): run its
117+
`skills/start-new-project.md` (Create phase) with the target folder
118+
`<Science root>/<active|development>/<key>/` (`development/` for a project
119+
whose purpose is developing or validating the software). Take data from the
120+
Science tree: copy a sample from `dataset/<sample>/`, or run a standard lens
121+
from `simulators/<type>/<lens>.py --dataset-path dataset` inside the new
122+
folder. The assistant owns the scaffold — never hand-copy an older project.
123+
2. **Register** — `pyauto-brain cortex birth <key> --category <active|development>
124+
--assistant <name> --summary "<one line>"`. The dry run prints the row it
125+
derived from the folder (remote, `ral_root` in the same category, the
126+
`hpc/sync` verbs, the ledger file); show it to the human, then `--apply`:
127+
it cuts `claude/birth-<key>`, writes the row and ledger through
128+
`scripts/cortex.py birth`, and pushes when the Cortex's classifier calls it
129+
a birth (which lands by itself). It creates no GitHub remote and reaches no
130+
cluster — it prints those lines for the human.
131+
132+
`pyauto-brain cortex drift` should then be clean for the new project.
133+
111134
## The push rule
112135

113136
`--push` is allowed when **the Cortex checkout is clean on `main` with a
@@ -166,4 +189,6 @@ checkout.
166189
| `dashboard --check` | Are the committed pages current? Exit **1** = stale (the refresh workflow's contract) |
167190
| `dashboard --apply` | Regenerate `dashboard.md` + `dashboard.html` — never hand-edit those two |
168191
| `issue [--project KEY] [--apply]` | The ledger block at the top of each project's issue (Now, Runs, the last five). `--apply` writes it there through `gh`; never creates an issue; exit **1** without `gh` |
192+
| `drift [--science-root PATH] [--json]` | Does the register match the laptop Science tree? Errors: unregistered `active/`/`development/` folders, vanished rows, category vs status. Reads, never writes; exit **1** on an error |
193+
| `birth <key> --category C --assistant A --summary S [--apply] [--no-push]` | Register a scaffolded project: its row and ledger on `claude/birth-<key>`, written by `cortex.py birth`. Dry run by default; exit **2** = refused (no folder, key or path taken, no sync verbs) |
169194
| `retire <project> --why "…"` (`scripts/cortex.py`) | Is this project over? Flips the row to `status: retired`, stamps the note, logs it. Exit **1** = an unknown key, an already-retired row, or a run still listed |

0 commit comments

Comments
 (0)