Skip to content

feat(batch): batch status box on both dashboards + rolling science review #341

New issue
New issue
Closed
#342
Closed
feat(batch): batch status box on both dashboards + rolling science review#341
#342

Description

@Jammy2211
Jammy2211
opened on Sep 3, 2026

Overview

Neither dashboard renders any batch state today: _intake.py has zero references to batches/, and _cortex.py's census computes batches/reviews but no renderer consumes them. This task adds a batch status box at the top of both dashboards (Mind and Cortex, markdown and HTML twins), rendered by one shared stdlib-only module, listing every member of the open slot on one line each as in progress / awaiting review / ruled, with a Review this batch button linking to the Pages packet once there is something to review.

It also makes the Cortex (science) board genuinely rolling: a slot stays open across partial reviews so a finished run can be reviewed on the fly while its siblings are still running, while mid-flight members hold no review control. Mind (dev) batch behaviour is unchanged — dispatched at once, reviewed at once, button on collected:.

Epic: two-slot-batching, phase 6. Parent: draft/feature/pyautomind/two_slot_batching_epic.md.

Plan

  • Add a new shared status module PyAutoBrain/agents/conductors/batch/_status.py (stdlib + _theme only, so it runs in the Cortex Pages job with no pip install) that computes the open slot and per-member state, and renders it identically for HTML and markdown.
  • Teach _batch.py to keep a Cortex slot open across partial reviews — later reviews land as batches/reviews/<slot>-r<N>.md, collect merges them (ruled sticky), and the packet refreshes without re-splicing ruled members. The dev refusal on an existing review file stays.
  • Teach PyAutoCortex/scripts/cortex.py to accept the numbered review files in batch_problems / rule --batch.
  • Render the box on the Cortex dashboard (_cortex.py) from a census adapter joining the batch record to the phase-file states and live notes.
  • Render the box on the Mind dashboard (_intake.py), above "Start here" as the original prompt specifies.
  • Trigger PyAutoMind/.github/workflows/dashboard_refresh.yml on batches/** (the Cortex workflow already does).
  • Write the partial-review rule into doctrine (Brain batch AGENTS.md, the Cortex batches/ docs, one sentence in the Mind batches/AGENTS.md).
Detailed implementation plan

Work Classification

Library (organ code) → start_library → ship_library. Worktree root: ~/Code/PyAutoLabs-wt/batch-status-box/

Affected Repositories

  • PyAutoBrain (primary) — the shared status module, _batch.py rolling logic, both renderers, doctrine
  • PyAutoCortex — scripts/cortex.py numbered-review acceptance, batches/ doctrine
  • PyAutoMind — dashboard_refresh.yml paths, batches/AGENTS.md

Branch Survey

Repository Current Branch Dirty?
./PyAutoBrain main clean
./PyAutoCortex main clean
./PyAutoMind main 1 modified (this prompt)

No worktree claim conflict: worktree_check_conflict batch-status-box PyAutoBrain PyAutoCortex PyAutoMind exits 0; no existing worktree root claims any of the three repos.

Suggested branch: feature/batch-status-box

Implementation Steps

1 — new PyAutoBrain/agents/conductors/batch/_status.py (stdlib + _theme only)

  • read_record(text) moves here from _cortex.py:273 (the read-only, lossless reader); _cortex.py imports it back. _batch.py's line-indexed read_record (:439) stays.
  • Shared vocabulary moves here and is re-exported by _batch.py: PENDING_RE, RULING_WORDS, CORTEX_BOARD_STATES.
  • pick_slot(rows): the open record with the latest dispatched: (never newest_slot's lexical order).
  • dev_status(slot, keys, members, review_exists, pages): open iff no reviewed-at: and no review file. Member: PENDING_RE outcome ⇒ in progress; ruling/health word ⇒ awaiting review (detail = head of the outcome text); empty ⇒ not delivered. reviewable iff collected: stamped. Dev behaviour unchanged.
  • cortex_status(slot, keys, members, states, live, pages): states = phase-file state: per slug (census join), live = _live_note text. Members named in carried: are excluded from the open test. Open iff any non-carried member is in a board state. Member: submitted/running ⇒ in progress (+ wall/budget/last-refresh); pulled ⇒ pulled, awaiting collect; awaiting-ruling ⇒ awaiting review; accepted/rerun/dropped ⇒ ruled. reviewable iff ≥1 awaiting-ruling.
  • review_url(pages, slot) = <pages>packets/<slot>.html.
  • render_html(st) / render_md(st) written once here so both organs' --check stay byte-consistent. HTML: <div class="verdict warn|ok"> (existing _theme.py class), title line Batch <slot> — N in progress · M awaiting review, one line per member with pills(...), then <a class="go" href=…>Review this batch →</a> (new a.go rule added to _theme.py beside .boards a; disabled text when no packet exists). st is None ⇒ fixed fixture "No batch in flight." Nothing time-derived in the box.
  • Test: new PyAutoBrain/tests/test_batch_status.py (dev in-flight / collected ⇒ button / reviewed ⇒ None; cortex mixed board; all-ruled ⇒ closed; carried member ⇒ closed; pick_slot by dispatched:; md/html parity of slugs + URL).

2 — PyAutoBrain/agents/conductors/batch/_batch.py

  • Import the vocabulary from _status via spec_from_file_location (pattern at :707) — _cortex.py must stay stdlib/Mind-free, and _batch.py imports _sizing at module scope, so the shared module cannot import _batch.
  • _apply_packet (:2143): refuse on an existing review file only for dev.
  • collect (:1213): read every partial review, reviews/<slot>.md + reviews/<slot>-r*.md, merged (later wins per member, ruled sticky) so _on_the_board keeps skipping ruled members.
  • organ_for (:2314): for cortex, review_path = the next free reviews/<slot>[-r<N>].md. packet.js reads REVIEW_PATH, so the page needs no change.
  • record_update_cortex (:2725): closed = not open per _status (not "review exists"); append a further - review: line per partial file.
  • Tests in test_batch_kinds.py: a second partial review refreshes the packet with earlier sections untouched; a ruled member is not re-spliced; the offered review path is the next free one. test_batch_collect.py: the dev refusal still fires.

3 — PyAutoCortex/scripts/cortex.py

  • batch_problems (:823): strip -r\d+$ from the review stem for the record/member lookup; title accepts either stem. rule --batch <slot> reads all <slot>[-r*].md files.
  • Test in PyAutoCortex/tests/: <slot>-r2.md beside <slot>.md checks clean; a numbered file naming a non-member still fails.

4 — _cortex.py renderers

  • census (:401) gains c["batch"] via an adapter: records → pick_slot → cortex_status with states from the census rows and _live_note (:629) details.
  • Insert render_html(c["batch"]) at :838 (after the fresh banner, above h2("awaiting")) and render_md at :694 (same position in the markdown twin).
  • Tests in test_cortex_conductor.py: the box lists every member of the open slot; the button appears as soon as one member is awaiting-ruling; nothing open ⇒ "No batch in flight"; the --check determinism test still passes.

5 — _intake.py renderers

  • _batch_status(mind) beside _pages_url (:1641); census gains c["batch"].
  • Insert the box after the fresh block (html :2406–2409) and after the Last-updated line (md :2098), above "Start here" as the existing prompt specifies.
  • Tests in test_intake_dashboard.py: _mind(...) gains batches=; in-flight text, button-on-collected, empty Mind ⇒ fixture, md/html parity.

6 — PyAutoMind/.github/workflows/dashboard_refresh.yml

  • Add batches/** to the push and pull_request paths (matching the Cortex workflow's list).

7 — doctrine

  • PyAutoBrain/agents/conductors/batch/AGENTS.md ("Two kinds, two records", "rolling board"): the partial-review rule + -r<N> naming; a Cortex slot closes when nothing is left on the board.
  • PyAutoCortex/batches/AGENTS.md, batches/reviews/AGENTS.md, batches/packets/TEMPLATE.md: the same rule, the numbered-file grammar, submit URL = the next free review path.
  • PyAutoMind/batches/AGENTS.md: one sentence that the dev rule is unchanged.

Merge order

PyAutoCortex → PyAutoBrain → PyAutoMind. Cortex first because the Brain's numbered-review path must not land before cortex.py check accepts it; Brain before Mind because the Mind/Cortex dashboard checks render against Brain main.

Key Files

  • PyAutoBrain/agents/conductors/batch/_status.py — new shared, stdlib-only status + render module
  • PyAutoBrain/agents/conductors/batch/_batch.py — rolling collect, partial reviews, organ_for, record_update_cortex
  • PyAutoBrain/agents/conductors/cortex/_cortex.py — census adapter + both renderers
  • PyAutoBrain/agents/conductors/intake/_intake.py — _batch_status(mind) + both renderers
  • PyAutoBrain/board/_theme.py — new a.go rule beside .boards a (:534); the .verdict ok/warn classes are at :492
  • PyAutoCortex/scripts/cortex.py — batch_problems, rule --batch
  • PyAutoMind/.github/workflows/dashboard_refresh.yml — batches/** triggers

Out of scope (noted, not done)

  • Conductor-opened slots for awaiting-ruling/carried members — separate formalised prompt draft/feature/pyautobrain/the_batch_conductor_s_plan_kind_cortex.md.
  • The two existing hand-written Cortex packets have no sentinels; the first conductor refresh of a hand-written packet degrades to member splices (already handled in packet_html).

Verification

  • cd PyAutoBrain && python -m pytest tests/test_batch_status.py tests/test_batch_kinds.py tests/test_batch_collect.py tests/test_cortex_conductor.py tests/test_intake_dashboard.py (Cortex-kind tests need PYAUTO_CORTEX set).
  • cd PyAutoCortex && python -m pytest tests/.
  • pyauto-brain cortex dashboard --check and the Mind dashboard --check on the worktrees (needs PYAUTO_CORTEX for the Mind badge) — both render the box, and re-running is idempotent.
  • Render the Mind dashboard against the real PyAutoMind/batches/ (both slots reviewed ⇒ "No batch in flight") and the Cortex dashboard against PyAutoCortex/batches/ (2026-09-02-pm is closed, its carried member excluded ⇒ fixture).
  • Simulate a rolling slot in a tmp Cortex tree: two members, one awaiting-ruling ⇒ button; submit a partial review file; collect again with the second member pulled ⇒ the packet gains the member, the first section byte-identical, cortex.py check clean.
  • Ship each PR via ship_library, CI green, /prm in the merge order above.

Original Prompt

Click to expand starting prompt 
# Batch phase 6 — the batch status box on the Mind and Cortex dashboards

Type: feature
Target: pyautobrain
Repos:
- PyAutoBrain
- PyAutoMind
- PyAutoCortex
Themes:
- dashboard
- mind-workflow
Difficulty: medium
Autonomy: safe
Status: draft
Consequence: judge
Review-minutes: 20
Unattended: ready
Epic: two-slot-batching
Phase: 6
Parent: draft/feature/pyautomind/two_slot_batching_epic.md
Filed: 2026-08-30

**Re-scoped by the human 2026-08-31 (see the epic's "Development through use"
section): not a second published surface.** The batch state is a strip on the
Mind dashboard, placed **above the "Start here" section** — the first thing the
human sees when they open the dashboard. The original standalone-board scope is
recorded in the epic and closed; do not rebuild it.

## What the strip renders

A single compact "Batches" block, generated by the same collect the dashboard
already runs (`_intake.py` — the markdown section builder near the "## Start
here" emit at `agents/conductors/intake/_intake.py:2027` and its HTML twin at
`:2325`; the strip is emitted immediately before both):

- **Latest packet** — link to the newest `batches/packets/<date>-<slot>.html`
  (the Pages URL `https://pyautolabs.github.io/PyAutoMind/packets/…`), with its
  `generated:` stamp.
- **Review state** — whether `batches/reviews/<date>-<slot>.md` exists for that
  packet: `review submitted` or `awaiting your review`.
- **In-flight batch** — if the newest batch record has no `collected:` yet: its
  member count, `review-at:`, and a PENDING note.
- **Archive** — one link to `batches/packets/` (the history).

Pure render over files that already exist; it decides nothing and recomputes
nothing. Every chip routes to the packet, the review file, or the batch record.

## Widened 2026-09-02 — both dashboards, and the science board goes live

Human's request, verbatim:

> I would like each dashboard to have a batch status box at the top, which basically has
> information in if a batch + review job is on going, it list all tasks as one line, and
> states whether they are in progress or awaiting review. When we are ready to review a
> button with the review URL link should be provided on the dashboard.
>
> For the PyautoMind development batch work, it is normal to wait for the full batch to be
> complete and then review in one go. This should be the default and dont change the
> behaviour.
>
> For the PyAutoCortex science batch work, I don't think we should be so constrained. Some
> science runs can take hours, others may take days. Instead, it should basically be that
> the review is live whenever anything can be reviewed, and when a task is completed is it
> added to the existing review board. WE should not be able to review something which is
> mid flight, this is more so that we can review stuff more on the fly if I want. The
> dashboard status thing here is thereofre more important as it shows all live information
> on how progress of science runs are going.

What this adds to the strip above:

- **One shared status computation** (`PyAutoBrain/agents/conductors/batch/_status.py`,
  stdlib only — the Cortex renders in a Pages job with no pip install) that both
  `_intake.py` and `_cortex.py` draw from, markdown and HTML twins alike: the open slot
  (latest `dispatched:` among open records — never lexical), one line per member with
  *in progress* / *awaiting review* / *ruled*, and the Pages packet URL
  (`https://pyautolabs.github.io/<Organ>/packets/<slot>.html`) as a button once there is
  something to review. Nothing open renders the fixed fixture "No batch in flight".
- **Mind rule (unchanged behaviour):** the button appears once `collected:` is stamped;
  before that the box says in flight, review at `review-at:`.
- **Cortex rule (rolling):** the button appears as soon as one member is
  `awaiting-ruling`; `submitted`/`running` members show as in progress with wall vs budget
  and last refresh, and hold no review control. A slot **stays open across partial
  reviews** — a later review lands as `batches/reviews/<slot>-r<N>.md` (append-only),
  `batch collect --kind cortex` keeps refreshing the packet while anything is on the board,
  ruled members are never re-spliced, and the slot closes when nothing is left in a board
  state. `cortex.py check`/`rule` accept the numbered files.
- `PyAutoMind/.github/workflows/dashboard_refresh.yml` triggers on `batches/**` (the
  Cortex workflow already does).

Merge order: PyAutoCortex (check accepts `-r<N>`) → PyAutoBrain → PyAutoMind.

## Not in scope

Review actions (merge/tweak/reject live on the packet page and in the
orchestrator chat), gate-leg rendering, usage-window readings, any scheduled
pre-slot publish. If the strip ever needs its own collect, that is the
`batch collect` verb's job, not the dashboard's.

## Done when

- Both dashboards (markdown and HTML) open with the status box, correct with
  nothing open (fixture), with an in-flight batch, and with a reviewable one
  (button present); the Cortex box shows a mixed running/awaiting board with the
  button live and running members without review control.
- A second partial Cortex review file refreshes the packet without touching
  earlier member sections, and `cortex.py check` stays clean.
- Regen is idempotent and the existing dashboard tests pass.

Activity

  1. Jammy2211 commented on Sep 3, 2026

    @Jammy2211
    Jammy2211
    on Sep 3, 2026
    ContributorAuthor

    PyAutoCortex leg (numbered partial review files) is up: PyAutoLabs/PyAutoCortex#7. Merge order: Cortex → Brain → Mind.

  2. Jammy2211 commented on Sep 3, 2026

    @Jammy2211
    Jammy2211
    on Sep 3, 2026
    ContributorAuthor

    Library PRs created

    No workspace impact (organ code only). Next: CI green, then /prm in the order above.

  3. Jammy2211
    added 2 commits that reference this issue on Sep 3, 2026

    test(batch): genericise fixture facts for the tenant firewall (#341)

    8d65b66

    Merge pull request #342 from PyAutoLabs/feature/batch-status-box

    0761e03
  4. Jammy2211
    closed this as completedin #342on Sep 3, 2026

  5. Jammy2211 commented on Sep 3, 2026

    @Jammy2211
    Jammy2211
    on Sep 3, 2026
    ContributorAuthor

    Shipped

    Behaviour: Mind (dev) batch stays collected-once, reviewed-once with the button on collected:. Cortex (science) board is rolling — the button appears as soon as one member is awaiting-ruling, running members show progress and hold no review control, and the slot closes only when nothing is left in a board state. Nothing open renders "No batch in flight."

    Session notes: Brain CI went red twice for reasons outside the code — the rolling-review test needs Cortex#7 on the CI checkout (merge-order dependency), then the tenant firewall flagged org facts in test fixtures (genericised in 8d65b66). Heart read RED at ship for library release validation, acked in the Mind registry as unrelated to these organ PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions