Skip to content

feat(cortex): drift check between the Science tree and projects.yaml #543

Closed

Description

@Jammy2211

Overview

The Cortex register (PyAutoCortex/projects.yaml) and the laptop Science tree silently diverged after Science moved under Workspace\: pull --dry-run reached 3 of 12 active projects and nobody noticed for weeks. This adds a report-only drift check so the Cortex notices when its body map and the Science folders disagree — phase 1 of making the Cortex the main tool for managing Science projects (phase 2, cortex birth, follows as its own issue).

Plan

  • New read-only pyauto-brain cortex drift verb comparing the Science folders with projects.yaml; writes nothing.
  • Flags: a project folder with no row; a row whose folder doesn't exist; a folder whose category (active/, development/, complete/, archive/) disagrees with its status; and (warning only) a row whose RAL category differs from its local one.
  • census and checkin show a one-line drift summary on the laptop and a short "skipped" notice anywhere else.
  • Science root derived from the local_paths already in the register (--science-root overrides); repo-backed rows under ~/Code/PyAutoLabs exempt.
  • Lands after projects.yaml: Workspace/Science paths; retire subhalo, cowls, cowls_ansofi PyAutoCortex#77 (path fixes), so it starts from a clean register.

Tier: judge — merge mode: human /prm

Detailed implementation plan

Affected Repositories

  • PyAutoBrain (primary — the cortex conductor)
  • PyAutoCortex (docs only: REFERENCE.md, docs/schema_decisions.md)

Branch Survey

Repository Current Branch Dirty?
./organs/PyAutoBrain main clean
./organs/PyAutoCortex main clean

Suggested branch: feature/cortex-science-drift

Implementation Steps

  1. agents/conductors/cortex/_cortex.py: drift_findings(projects, science_root) -> list[Finding] (kind, key, path, message) and science_root_from(projects) deriving the root from <root>/<category>/<project> local_paths. Kinds: unregistered, missing, category_status, ral_category (warning). Status rules: active/development ↔ active|dormant|planned; complete/archive ↔ retired|dormant.
  2. drift verb: --json, --science-root; exit 0 clean / 1 drift / 2 bad args; root absent → skip notice, exit 0.
  3. One-line summary in census and the checkin read-out (laptop only).
  4. Document in the conductor AGENTS.md, PyAutoCortex REFERENCE.md (Check-in section) and a dated docs/schema_decisions.md entry.
  5. Tests in tests/test_cortex_conductor.py with a tmp Science tree: each finding kind, exempt Code rows, skip notice, --json.
  6. Validate on the real tree: expect only the four development projects' ral_category warnings (RAL trees still under active/).

Key Files

  • PyAutoBrain/agents/conductors/cortex/_cortex.py — conductor (census, checkin, new verb)
  • PyAutoBrain/tests/test_cortex_conductor.py — tests
  • PyAutoCortex/REFERENCE.md, PyAutoCortex/docs/schema_decisions.md — docs

Original Prompt

Click to expand starting prompt

Cortex as the Science project manager: drift check and project birth

Type: feature
Target: cortex
Repos:

  • PyAutoBrain
  • PyAutoCortex
    Themes:
  • science
  • cortex
    Difficulty: medium
    Autonomy: human-required
    Priority: high
    Status: formalised
    Consequence: judge
    Filed: 2026-10-29

Original request (verbatim)

"5) Bring this all together with PyAutoCortex which needs to become the main tool to manage Science
projects, its meant to be but hasnt been used as such for a while."

Follow-up (2026-10-29), after the Science clean-up session proposed next steps
(1 push the path-fix PR, 2 a drift check comparing the Science folders to the register,
3 a new-project command that scaffolds a project in active/ or development/, seeds it from
dataset/ or simulators/ and registers it in one step): "yeah do it".

Context

The Science folder (/mnt/c/Users/Jammy/Workspace/Science) was reorganised on 2026-10-29:
development/ holds software-development projects, simulators/ holds the standard simulated
lenses (shared --dataset-path / --dataset-types CLI, default output dataset/simulated/),
profiling/ holds RAL pull mirrors with a generated INDEX.md, and Science/AGENTS.md names
the Cortex as the project register. Before that, the move from C:\Users\Jammy\Science to
Workspace\Science had left most projects.yaml local_paths pointing at nothing, unnoticed for
weeks: pull --dry-run reached 3 of 12 active projects. PyAutoCortex#77 fixes those paths and
retires subhalo, cowls, cowls_ansofi; it lands before this work.

The failure mode to prevent: the register and the Science tree silently diverge. And starting a
project today is manual copying from an older project, so new projects never get a row.

Scope

Phase 1 — drift check (conductor verb, report-only)

pyauto-brain cortex drift [--science-root PATH] [--json], also summarised in census and
checkin when the Science root is on this machine (laptop-only, like pull; elsewhere a one-line
skip notice, never a failure). It writes nothing. Findings:

  1. a project folder under active/, development/, complete/, archive/ with no row
    (matched by local_path);
  2. a non-retired row whose local_path does not exist;
  3. category vs status: rows under active/ or development/ should be active|dormant|planned;
    under complete/ or archive/ should be retired|dormant;
  4. (warning) a non-retired row whose ral_root category differs from its local_path category
    (decision 61: one layout on both machines).

The Science root is derived from the local_paths (<root>/<category>/<project>), overridable
with --science-root; no new absolute path enters the organism outside projects.yaml.
Rows under ~/Code/PyAutoLabs (repo-backed campaign roots) are exempt from 1/3/4.

Phase 2 — project birth (conductor verb)

pyauto-brain cortex birth <key> --category active|development --summary "<one line>" [--dataset <sample>[/<lens> ...]] [--simulator <type>/<lens> [--dataset-types ...]] [--apply]:

  • scaffold Science/<category>/<key>/ from a project template kept beside the conductor
    (AGENTS.md/CLAUDE.md skeleton, wiki/project/state.md, .gitignore, config/, scripts/,
    hpc/ with sync, sync.conf from the example, batch_cpu/, batch_gpu/), template derived
    from the current best project (active/euclid_detailed) with project specifics stripped;
  • seed dataset/ by copying from Science/dataset/<sample> or running
    Science/simulators/<type>/<lens>.py --dataset-path dataset;
  • add the projects.yaml row (ral_root mirroring the category) and cortex.py new the ledger,
    on a claude/birth-<key> branch — an add-only birth diff, which lands by itself under the
    existing ledger-merge rule;
  • dry-run by default; --apply writes. Never creates a GitHub remote or touches RAL (prints the
    gh repo create / hpc/sync push-data-init lines for the human).

Out of scope

Moving trees on RAL; the human-readable dashboard work (draft/feature/cortex/output_summary.md).

Activity

  1. Jammy2211 commented on Oct 29, 2026

    @Jammy2211
    ContributorAuthor

    Library PRs created

    Workspace impact: none (organ tooling only), so no workspace phase. Tier judge: the human merges with /prm.

    Heart RED override (development only)

    Authorized by the live human on 2026-10-29: "I authorize" (for PyAutoBrain#543, after the exact RED reasons were shown). Development ship only: commit, push, PR-open. Merge needs a separate human /prm with every required check green. No release.

    Exact RED reasons (pyauto-heart readiness, 2026-10-29):

    • PyAutoArray: 4 uncommitted source change(s)
    • PyAutoGalaxy: 9 commit(s) behind origin
    • PyAutoLens: 7 commit(s) behind origin
    • release validation incomplete: no rehearsal for current source
    • release push credential check stale (pyautolabs-bot, 49h old)

    None of these involves PyAutoBrain or PyAutoCortex. This branch does not fix them.

    Passed branch gates: Brain tests/test_cortex_conductor.py 71 passed (12 new); Cortex cortex check: OK + 65 tests; no new ruff findings; no smoke leg applies (no workspace or library code); independent review FINDINGS, all addressed (see below).

    🤖 Generated with Claude Code

  2. Jammy2211 commented on Oct 29, 2026

    @Jammy2211
    ContributorAuthor

    Shipped

    Phase 1 merged on 2026-10-29: #544 (cortex drift verb + tests) and PyAutoLabs/PyAutoCortex#78 (docs, decision 62), after PyAutoLabs/PyAutoCortex#77 (Workspace/Science paths; subhalo, cowls and cowls_ansofi retired). Against the real tree on merged main: 0 errors, 11 warnings.

    Phase 2 (cortex birth) is re-filed as its own prompt and issue.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions