Skip to content

Cortex: route project work through the domain assistant #362

Closed

Description

@Jammy2211

Overview

PyAutoCortex manages the science projects but has no notion of the domain assistants (autolens_assistant and siblings), and the assistants have no notion of the Cortex. Only one active project is assistant-born, and even there the assistant is unused (unfilled profile.md, stale state.md, drift-only journal mentions). This task makes the assistant the entry protocol of the execution subagents the single Cortex chat spawns per project: a one-way pointer from project to assistant. The check-in door and rulings stay assistant-free, and no assistant repo is edited, so assistants remain usable by people who never touch the Cortex.

Plan

  • Add a required assistant: field to every Cortex project row (autolens_assistant for the two active lensing projects, none for every dormant or retired row), validated by check.
  • Teach the Brain cortex conductor to append an assistant entry-protocol brief to the three work-shaped prompts (next task, rerun, planned) when the row's assistant is not none. Check-in, ruling, live, gate and retire prompts do not change.
  • Write a "Work on a project" subagent contract into skills/cortex/cortex.md and a sibling subsection in skills/WORKFLOW.md.
  • Backfill the two live science checkouts outside the workspace with local commits (refer-back block, assistant export and wiki/project/ scaffold for euclid_dr1_prelim; filled profile.md for subhalo_validation).
  • Merge order: PyAutoBrain first (its reader defaults a missing field to none), then PyAutoCortex.
Detailed implementation plan

Affected Repositories

  • PyAutoBrain (primary)
  • PyAutoCortex
  • Outside the workspace, local commits only: /mnt/c/Users/Jammy/Science/euclid_dr1_prelim, /mnt/c/Users/Jammy/Science/subhalo_validation

Branch Survey

Repository Current Branch Dirty?
./PyAutoBrain main clean
./PyAutoCortex main clean

Suggested branch: feature/cortex-assistant-entry
Work Classification: Library (organ code + docs)
Worktree root: ~/Code/PyAutoLabs-wt/cortex-assistant-entry/

Implementation Steps

PyAutoCortex

  1. scripts/cortex.py: add assistant to PROJECT_FIELDS (line ~106). In _finish_row (~349-369) add a value rule: none, or a bare workspace-relative directory name matching ^[A-Za-z0-9_.-]+$ (no /, not absolute). No filesystem stat, matching local_path/ral_root (~366-369) — the Cortex CI render must not need an assistant checkout. retire_project (~1029-1101) needs no change: it matches only status: / note: lines.
  2. projects.yaml: header comment (lines 1-43, field list 16-30, "every field but note is required" 33-34) documents the new field; every row gains assistant: — subhalo_validation and euclid_dr1_prelim → autolens_assistant; every dormant/retired row (incl. retired inference_programme) → none.
  3. REFERENCE.md ## projects.yaml (373-421): a bullet for assistant beside the required-field bullets (391-398).
  4. tests/fixtures/skeleton/projects.yaml: add assistant: none. tests/test_cortex.py: clone test_note_is_optional_but_never_empty_and_the_field_set_still_closes (~266) into a test that assistant is required, accepts none and a bare name, rejects an empty value, a path with / and an absolute path; extend test_the_live_body_map_parses_clean (~191) to assert rows["subhalo_validation"]["assistant"] == "autolens_assistant" and rows["inference_programme"]["assistant"] == "none".
  5. Regenerate dashboard.md / dashboard.html if the renderer output changes (pyauto-brain cortex dashboard --apply from the worktree with PYAUTO_CORTEX pointed at the worktree Cortex; --check must pass).

PyAutoBrain
6. agents/conductors/cortex/_cortex.py: new helper _work_brief(r, projects) near the payload banner (~521). Reads projects.get(r["project"], {}).get("assistant", "none") (tolerates a Cortex main without the field). Returns "" for none; otherwise a fenced block that (a) cd <local_path> && source activate.sh, (b) names the assistant by repo name and says to resolve it as $AUTOLENS_ASSISTANT or <workspace root>/<assistant> and to read its AGENTS.md first and follow its constitution, (c) then read wiki/project/state.md, the newest wiki/project/YYYY-MM-DD-*.md, and the skills the task names, (d) then the task file <rel> whose ## Witness is the contract, (e) finish by writing today's journal entry, rewriting wiki/project/state.md, and returning the outcome plus a list of assistant drift (a skill or wiki page that was wrong, missing or stale). Never resolves or stats the path (the conductor names no path — see its AGENTS.md).
7. Append the brief to _next_task_payload (~634-660), _rerun_payload (~663-679) and _planned_payload (~682-684; give it projects via task_chips). Leave _ruling_payload, _live_payload, _gate_payload, _checkin_payload, _retire_payload, launch_payload untouched. task_chips (~697-724) remains the single funnel so both board twins and census --by-project agree.
8. agents/conductors/cortex/AGENTS.md "What it reads, and what it refuses to read" (~47): one bullet — assistant is a row field rendered by name into work briefs, never resolved on disk.
9. skills/cortex/cortex.md: new ### 4. Work on a project after step 3 (~55-74) and before ## The push rule (~75): the architect pastes the brief into an execution-tier subagent (Agent(model=<one rung down>)), never works the project in the Cortex chat; the subagent's return has two parts (outcome; assistant drift); the architect files drift through /intake targeting the assistant repo; if the brief is empty the project declared assistant: none and the work is plain workspace work. Add to ## The rules that do not bend (~94-123): "the door names the assistant, never reads it — no assistant page is loaded in the Cortex chat".
10. skills/WORKFLOW.md: a subsection "Cortex project work" after the subagent prompt contract (~154-169), before "Progress heartbeat" (~171): the brief is the prompt, the finish rule (journal + state.md rewrite), the two-part return, drift goes to /intake, and the heartbeat contract applies to long runs.
11. skills/COMMANDS.md:40 (/cortex row): append "…and hands each work-shaped task a brief that enters the project through its declared assistant".
12. tests/test_cortex_conductor.py (census fixture helper ~764): a project with assistant: autolens_assistant carries the brief on the next-task, rerun and planned chips on both twins and in census --by-project; a none row carries no brief; the ruling and check-in payloads contain no assistant substring; test_the_chips_a_state_carries_are_the_same_everywhere (~831) still passes; a census whose project row lacks the key renders as none.

Backfill (outside the workspace; stage by filename, local commits, no push)
13. Science/euclid_dr1_prelim/AGENTS.md: after the intro paragraph (lines 1-19), before ## Repository Structure, insert the ## The assistant (skills + wiki) and ## Code gate blocks from autolens_assistant/skills/start-new-project.md (~156-172), with the machine note that the clone lives at /home/jammy/Code/PyAutoLabs/autolens_assistant and activate.sh exports AUTOLENS_ASSISTANT (mirror Science/subhalo_validation/AGENTS.md 24-43). Add export AUTOLENS_ASSISTANT=/home/jammy/Code/PyAutoLabs/autolens_assistant to its activate.sh. Scaffold wiki/project/ from the assistant templates (_state_template.md → state.md filled from the Cortex task tasks/euclid_dr1_prelim/*.md and .cortex/pull.json; profile.md copied from the filled subhalo profile; a README.md line saying the rest of the wiki is the assistant's). Do not add project.yaml (not assistant-born). Its origin is the pipeline repo: commit locally, never push.
14. Science/subhalo_validation/wiki/project/profile.md: fill every section from what the workspace records — lead developer of PyAutoLens (lensing + PyAutoLens background), interaction mode assistant with autonomous multi-session runs, HPC: RAL via alias euclid_jump through bastion jump_finan, no MFA/VPN needed from this machine (the sync CLI runs unattended), agent-driven remote execution permitted (ruling 2026-09-05), automation level assistant-runs. No _unrecorded_ left; last_touched updated. Commit locally.

Key Files

  • PyAutoCortex/scripts/cortex.py — PROJECT_FIELDS, _finish_row, check_problems
  • PyAutoCortex/projects.yaml, PyAutoCortex/REFERENCE.md — schema of record
  • PyAutoCortex/tests/test_cortex.py, tests/fixtures/skeleton/projects.yaml
  • PyAutoBrain/agents/conductors/cortex/_cortex.py — payload functions, task_chips
  • PyAutoBrain/agents/conductors/cortex/AGENTS.md, PyAutoBrain/skills/cortex/cortex.md, PyAutoBrain/skills/WORKFLOW.md, PyAutoBrain/skills/COMMANDS.md
  • PyAutoBrain/tests/test_cortex_conductor.py
  • autolens_assistant/skills/start-new-project.md (read only — the refer-back block source)

Testing

  • pytest PyAutoCortex/tests and pytest PyAutoBrain/tests/test_cortex_conductor.py from the task worktree with PYAUTO_CORTEX=<worktree>/PyAutoCortex.
  • python3 scripts/cortex.py check clean on the edited projects.yaml; pyauto-brain cortex checkin --dry-run shows the brief on a planned task of subhalo_validation and none for a none row.
  • grep -ril cortex autolens_assistant still empty.

Original Prompt

Click to expand starting prompt

Cortex projects route through the domain assistant at execution time

Type: feature
Target: pyautobrain
Repos:

  • PyAutoBrain
  • PyAutoCortex
    Themes:
  • mind-workflow
  • assistants
    Difficulty: medium
    Autonomy: human-required
    Priority: high
    Status: formalised
    Consequence: judge
    Witness: projects.yaml rows carry an assistant: field (workspace-relative assistant repo path or none) that cortex.py check validates and that the retired inference_programme row leaves as none; every work-shaped /cortex payload (ruling-follow-up, rerun, next-task, planned, gate) prints a subagent brief that starts cd <local_path> && source activate.sh, then reads the assistant's AGENTS.md, the project's wiki/project/state.md, its newest dated journal entry and the named skills, and ends by writing a journal entry, rewriting state.md and returning assistant-drift findings; the check-in door and the ruling prompt are unchanged and name no assistant; skills/cortex/cortex.md and skills/WORKFLOW.md carry a "work on a project" subagent contract; grep -ril cortex autolens_assistant still returns nothing; Science/euclid_dr1_prelim/AGENTS.md carries the assistant refer-back block and Science/subhalo_validation/wiki/project/profile.md has no _unrecorded_ field left; Brain + Cortex tests cover the field, the check rule and the brief text
    Review-minutes: 25
    Unattended: never
    Filed: 2026-09-07

Human direction (2026-09-07, verbatim):

"""
PyAutoCortex has become my science project manager, but I have noted that it
means I do not necessary interface with or use autolens_assistant (or another
assistant). When I checked, it wasnt clear the existing project suse
autolens_assistant. However, I want to be sure that Cortex projects always
route through the asssistant, as this should help AI work and also ensure the
assistant is functioning as best and possible and is being used. Do an
assessment of how Cortex, and assistants, should interface. Currently, I find
myself using Cortex as a single chat to manage all projects, rather than
individual chats per project. I therefor think given this is the use I find
myself oging towards we should continue to build arond that work flow, but that
probably means the project-to-assistant interface needs to be a bit more
dynamic, maybe only using the assistant when sub agents spawn for work on a
particular project maybe? I guess its also worth assesing for sure that going
via assistants makes sense with Cortex.
"""

After the assessment:

"""
Sounds good, assistants should not "know" about Cortex ideally as users may use
them without ever using Cortex, other than tht I agree with the plan but note
that inference has been retired so dont worry bout setting this up for that.
"""

Assessment of record (2026-09-07)

  • The Cortex schema has no assistant field; the Brain cortex conductor and the
    /cortex skill contain no "assistant" hit; all five assistants contain no
    "cortex" hit. The only link is implicit: ledger: wiki/project/state.md is
    the file the assistant's start-new-project scaffolds.
  • Only subhalo_validation is assistant-born (project.yaml assistant_ref,
    refer-back AGENTS.md); euclid_dr1_prelim is a clone of the Euclid pipeline
    repo with no refer-back. Even the assistant-born project shows non-use: its
    profile.md is the unfilled template, state.md was last rewritten
    2026-09-02 while runs launched 2026-09-07, its journals name the assistant
    only as drift, and the assistant benchmark harness has never recorded a run.
  • The check-in door is mechanical (pull, score, move, render, push, print
    prompts) and reasons about no science. Its payloads are pastes for a fresh
    chat with no stated working directory.

Ruling: the assistant is the entry protocol of the execution subagents the
single Cortex chat spawns per project, never part of the check-in door or the
ruling. The pointer is one-way, project → assistant; assistants stay
Cortex-unaware so users who never use the Cortex are unaffected.

Scope

  1. Cortex schema. assistant: row field in projects.yaml (workspace-
    relative path such as autolens_assistant, or none), parsed like note:
    is today, validated by check (known key, none or an existing directory
    holding an AGENTS.md), documented in REFERENCE.md and the file's header
    comment. Rows: subhalo_validation and euclid_dr1_prelim →
    autolens_assistant; every dormant/retired row → none.
  2. Brain briefs. Work-shaped payloads in agents/conductors/cortex/_cortex.py
    become subagent briefs carrying the entry protocol when the row's
    assistant is not none; the paste form survives for the board. Mechanical
    payloads (check-in, ruling, submit, retire) do not change.
  3. Contract. A "Work on a project" section in skills/cortex/cortex.md and
    a sibling subsection under "Model delegation" in skills/WORKFLOW.md: the
    architect delegates project work one rung down with the brief; the subagent
    ends by writing the journal entry and rewriting state.md, and returns the
    outcome plus assistant-drift findings, which the architect files via
    /intake as assistant work.
  4. Backfill (outside the workspace, local commits). Refer-back block in
    Science/euclid_dr1_prelim/AGENTS.md (its origin is the pipeline repo:
    commit locally, never push there); fill
    Science/subhalo_validation/wiki/project/profile.md from what this
    workspace already records about the human (push to the project's remote).

Out of scope: inference_programme (retired 2026-09-07); any edit to an
assistant repo; the check-in door; ruling prompts.

Activity

  1. Jammy2211 commented on Sep 7, 2026

    @Jammy2211
    ContributorAuthor

    Library PRs created

    Both labelled pending-release.

    Merge order: Brain #367 first, then re-run the Cortex refresh check (it renders through Brain main), then Cortex #23.

    Local science commits made outside the workspace (neither pushed):

    • subhalo_validation — 8df4bf0 (profile filled)
    • euclid_dr1_prelim — f486cee (refer-back + wiki/project scaffold)
  2. Jammy2211 commented on Sep 8, 2026

    @Jammy2211
    ContributorAuthor

    Shipped

    Both halves are merged.

    PR Merge SHA Head SHA
    PyAutoBrain#367 — Cortex: work briefs enter a project through its declared assistant 29dc4796 33136c3
    PyAutoCortex#23 — projects: required assistant field 02318d7d 133620ed

    Summary

    Cortex projects route through their domain assistant at execution time, one way
    only — the project names the assistant, the assistant never learns about the
    Cortex.

    • PyAutoCortex#23 — every projects.yaml row carries a new required
      assistant: field (none or a bare workspace-relative repo name), validated
      by cortex.py check, which stats nothing so the CI render needs no assistant
      checkout. subhalo_validation and euclid_dr1_prelim declare
      autolens_assistant; every dormant and retired row (including the retired
      inference_programme) declares none. 126 tests pass.
    • PyAutoBrain#367 — every work-shaped /cortex payload (next task, rerun,
      planned) now carries a work brief: the entry protocol for the
      execution-tier subagent — read the assistant's AGENTS.md, then the project's
      wiki/project/ state and journal, then the named skills; finish by writing
      the journal entry and rewriting state.md; return the outcome against the
      witness plus any assistant drift. New _work_brief / _with_brief;
      _planned_payload gains the projects map. Check-in, ruling, gate and retire
      payloads are unchanged and name no assistant. 91 conductor tests, 920 overall.
    • Docs — /cortex gains step 4 "Work on a project" and the rule "the door
      names the assistant, never reads it"; WORKFLOW.md gains the "Cortex project
      work" subsection under the subagent prompt contract.

    Backfill — two local science commits, deliberately unpushed from here

    These live outside the workspace and are not pushed by this close-out:

    • Science/subhalo_validation — 8df4bf0 (wiki/project/profile.md filled in
      from what the workspace already records about the human). Pushable to the
      project's own remote.
    • Science/euclid_dr1_prelim — f486cee (assistant refer-back block in
      AGENTS.md). Its origin is the Euclid pipeline repo, so this stays a local
      commit and is never pushed there.

    Traps

    • Merge order is load-bearing — the Cortex refresh check renders through
      Brain main, so it stays red until the Brain PR merges. Merge Brain first,
      then re-run the Cortex check.
    • Migration for any external projects.yaml — add assistant: none (or the
      assistant's repo name) to every row, after ledger:; check reports it
      missing otherwise.

    Mind record: PyAutoMind/complete/2026/09/cortex-assistant-entry.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions