Skip to content

feat: science-project memory — a fresh chat resumes a project by default #315

Description

@Jammy2211

Overview

Point a fresh agent chat at a science project folder and it should work out where the project got to, how to resume, and what its history is — by default, without being asked "what have we done?". Today it does not: the generated project AGENTS.md has no session-start section, nothing reads the journal on arrival, results_summary.md is consumed at Publish but never templated, results/manifests/ is aspirational (empty after a full real campaign), and profile.md conflates user-level facts with the per-project science goal.

This task makes that true for projects scaffolded by the assistants' start-new-project skill, and lands the change once — via a new sync mode on the PyAutoBrain clone conductor that re-applies the reference assistant's _SHARED_GENERIC diff to its siblings as a reviewable patch, rather than four hand edits.

euclid_assistant is a different design and is explicitly out of scope.

Plan

  • Phase 1 — PyAutoBrain. Add a sync mode to the clone conductor: given a reference assistant and a commit range, take the reference's diff to the _SHARED_GENERIC files and apply it to each sibling as a three-way patch, reporting per-file/per-sibling applied cleanly / hunks rejected / file absent. Dry-run by default; --apply writes. Domain-adapted lines in siblings survive; conflicts surface for a human instead of being resolved silently. Wire into bin/pyauto-brain clone sync, document in the conductor's AGENTS.md, unit-test with a temp-dir fixture.
  • Phase 2 — autolens_assistant (the reference copy). Deliverables A–D: a session-start block in the generated project AGENTS.md (+ a project CLAUDE.md that @-imports it); a state.md head pointer (templated, scaffolded, rewritten-not-appended each session, wired into the end-of-session journal ritual); a templated results_summary.md with a covers_through: stamp; results/manifests/ deleted from the scaffold and from every sentence that promises it; and profile.md split at the user/project seam.
  • Phase 3 — propagate. Run the Phase-1 sync dry-run from autolens_assistant to autocti_assistant, autogalaxy_assistant, autofit_assistant. Apply and PR the two that take the patch cleanly; autofit_assistant (diverged by ~343 lines) is not applied — its dry-run report is posted here for a human.
Detailed implementation plan

Work Classification

Library (three independent repos; no workspace follow-up — the assistants are their own workspaces).

Affected Repositories

  • PyAutoBrain (primary)
  • autolens_assistant
  • autocti_assistant
  • autogalaxy_assistant
  • autofit_assistant (dry-run only — no edits this task)

Branch Survey

Repository Current Branch Dirty?
./PyAutoBrain main clean
./autolens_assistant main clean
./autocti_assistant main clean
./autogalaxy_assistant main clean
./autofit_assistant main clean

Suggested branch: feature/science-project-memory
Worktree root: ~/Code/PyAutoLabs-wt/science-project-memory/

Implementation Steps

Phase 1 — PyAutoBrain/agents/conductors/clone/

  1. _clone.py: add a sync subcommand path. Semantics — not a blind overwrite:
    • Resolve the reference (default autolens_assistant) and its _SHARED_GENERIC file set (reuse REFERENCE_PROFILES[...]["generic"] / match_any, not a second list).
    • Compute the reference's diff over a commit range (--since <rev>, default: the sibling's last recorded sync, else origin/main), restricted to the generic paths.
    • For each sibling, apply per file with git apply --3way --check then --3way, falling back to reporting rejected hunks. Report per file: applied / rejected (<hunk list>) / absent / unchanged.
    • Dry run by default (--check only, nothing written); --apply writes.
  2. bin/pyauto-brain: wire clone sync ... through clone.sh.
  3. agents/conductors/clone/AGENTS.md: document the mode, its semantics, and that a rejected hunk is a human decision, never auto-resolved.
  4. tests/test_clone_sync.py: temp-dir fixture with three fake assistants (one divergent) asserting clean-apply, rejected-hunk reporting, absent-file reporting, and that a dry run writes nothing.

Phase 2 — autolens_assistant

  • skills/start-new-project.md:
    • Generated project AGENTS.md template gains "## Session start — do this first, every session": read wiki/project/profile.md, wiki/project/state.md, and the newest dated wiki/project/YYYY-MM-DD-*.md, then answer.
    • Scaffold emits a project CLAUDE.md that @-imports AGENTS.md (the same two-line pattern as PyAutoLabs/CLAUDE.md).
    • Scaffold tree: add state.md + results_summary.md; remove results/manifests/ and every sentence promising per-run manifests (Phase 2 step 1, the generated AGENTS.md, the generated README.md, the Publish gate) — the journal run table is the run record.
    • Seed the project's profile.md from the assistant clone's own wiki/project/profile.md when present, else from _profile_template.md.
  • wiki/project/_state_template.md (new, ~30 lines): Where we are now / In flight / Open, carried forward / Traps — don't repeat / Journal index.
  • wiki/project/_results_summary_template.md (new): covers_through: <date> at the top.
  • wiki/project/_profile_template.md: user-level sections only (background, interaction mode, HPC access + authorization, automation preference); science goal + data-on-hand move to the state template.
  • wiki/project/README.md: state.md rule — rewritten each session, never appended; the end-of-session ritual becomes "journal entry + rewrite state.md"; an entry is finished when state.md is current.
  • AGENTS.md step 2: read state.md and the newest journal entry too, when present.
  • The clone boundary already covers wiki/project/* and skills/start-new-project* as _SHARED_GENERIC, so check_boundary.py stays green.

Phase 3 — propagation

  • clone sync --reference autolens_assistant dry-run → all three siblings.
  • Apply + PR autocti_assistant and autogalaxy_assistant if hunks land cleanly; in autocti_assistant reconcile the orphan wiki/project/state.md (real content: CTI stack/release-train/heritage facts) against the scaffolded template.
  • autofit_assistant: no apply — post the dry-run report here.

Key Files

  • PyAutoBrain/agents/conductors/clone/_clone.py_SHARED_GENERIC, match_any, REFERENCE_PROFILES; new sync mode
  • PyAutoBrain/agents/conductors/clone/AGENTS.md — operating summary
  • PyAutoBrain/bin/pyauto-brain — conductor dispatch
  • PyAutoBrain/tests/test_clone_sync.py — new
  • autolens_assistant/skills/start-new-project.md — the scaffold (A, C, D)
  • autolens_assistant/wiki/project/{README,_profile_template,_state_template,_results_summary_template}.md
  • autolens_assistant/AGENTS.md — session-start step 2

Original Prompt

Click to expand starting prompt

Science-project memory: a fresh chat pointed at a project folder resumes…

Type: feature
Target: PyAutoBrain
Repos:

  • autocti_assistant
  • autofit_assistant
  • autogalaxy_assistant
  • autolens_assistant
  • PyAutoBrain
    Difficulty: large
    Autonomy: supervised
    Priority: high
    Status: formalised

Science-project memory: a fresh chat pointed at a project folder resumes by default

Type: feature
Difficulty: large
Autonomy: supervised
Priority: high

Make "point a fresh agent chat at a science project folder and it works out where it got to, how to resume, and the project's history" true by default for projects scaffolded by the assistants' start-new-project skill.

Scope: autolens_assistant (reference copy), autofit_assistant, autogalaxy_assistant, autocti_assistant, and the pyautobrain clone conductor. The euclid assistant is a different design and is explicitly out of scope.

User's original request (verbatim): "I want to know how much memory of a project they truly have. Ideally, I can always load a new agent (e.g. claude) chat, and point it to a science project and it can quickly work out where it got to, how to resume and the general history of the science project. I feel like this would need more in the wiki than just the code run, so how much of this type of more persistant memory is avaialble? Is this something we should add, like an indexed tracker of tasks and work analogous to how organ repos and other tasks are easily resumeable?"

Findings verified 2026-08-28 (one real project scaffolded in July, two journal entries, one full run campaign):

  1. The generated project AGENTS.md (from skills/start-new-project.md) has no session-start section; each assistant's own AGENTS.md step 2 reads wiki/project/profile.md only. Nothing reads the journal on arrival, so a fresh chat does not resume unless asked "what have we done?".
  2. autocti_assistant/wiki/project/state.md is an orphan — no AGENTS.md or skill reads or writes it (the only precedent for a head pointer, and it is dead).
  3. results_summary.md is consumed by the Publish phase (gh release create --notes-file wiki/project/results_summary.md, autolens_assistant skills/start-new-project.md:381) but is never templated or created by the scaffold.
  4. results/manifests/ is aspirational: empty after a full real campaign; the journal's run tables do the job.
  5. profile.md conflates user-level facts (role, HPC/SSH access, automation consent) with the per-project science goal; a second project re-elicits everything.
  6. The four copies of skills/start-new-project.md and wiki/project/{README,_profile_template,_template}.md have four distinct hashes (the autofit_assistant copy diverged by 343 lines). The clone conductor lists them as _SHARED_GENERIC (agents/conductors/clone/_clone.py:69-81) but only has a birth mode — nothing re-syncs after birth.

Deliverables, in priority order (A alone gets most of the value):
A. Session-start block in the generated project AGENTS.md, and in the assistants' AGENTS.md step 2: read profile.md, wiki/project/state.md, and the newest dated journal entry before answering. Verify the scaffold emits a CLAUDE.md that @-imports AGENTS.md so Claude Code actually loads it.
B. wiki/project/state.md (keep autocti_assistant's filename) — a small head pointer, rewritten not appended each session: Where we are now / In flight (runs, job IDs, what unblocks each) / Open, carried forward (struck when done) / Traps — don't repeat / one-line journal index. Wire it into the existing end-of-session "want a journal entry?" ritual: an entry is finished when state.md is rewritten. Markdown read by an LLM, not YAML — a science project is one repo with a handful of in-flight runs.
C. Template results_summary.md with a covers_through: <date> stamp so staleness is visible. Drop results/manifests/ from the scaffold and the AGENTS text (delete the trap rather than document it).
D. Split profile.md at the user/project seam: the user half (who you are, HPC access, automation preference) seeded from the assistant clone's profile.md and free to diverge; the project goal moves to state.md.
E. Land via a clone-conductor sync mode that re-applies _SHARED_GENERIC files from autolens_assistant (reference) to the other three, rather than four hand edits. A human reviews the autofit_assistant divergence before any overwrite — some of it may be deliberate domain adaptation.

Out of scope: the library-repo anonymisation of a named science target is filed as its own small maintenance prompt in the data-reduction library.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions