Skip to content

feat: organ skeleton — registry, pinned reader + receipts, board, workflows (phase 2) #1

Closed

Description

@Jammy2211

Overview

Phase 2 of the profiling-organ-birth epic: the PyAutoPulse organ skeleton — registry, commit-pinned reader with receipts, cross-project board, CLI, hermetic tests, Pages/refresh/lint workflows and prose — modelled on the PyAutoEyes skeleton (PyAutoEyes#2 + PyAutoBrain#428) and the spec PyAutoBrain/docs/research/profiling_inference_organs.md (#444). Reads the live profiling-summary@1 file at autolens_profiling/dashboard/summary.json (159 records, producer_revision 227b5c91). Phase 0 (organ row) merged 2026-10-02 as PyAutoMind#463. Human decisions 2026-10-02: ship as one task, two PRs (PyAutoPulse first, then PyAutoBrain); the Brain boards: pulse entry lands here (deferred from phase 0). Heart at the door: YELLOW 85 (another session's dirty Cortex checkout; .github table; no rehearsal).

Plan

  • Build the pulse/ package on the Eyes skeleton: registry loader, summary reader + validator, ingest with one-commit resolution and receipts, board renderer, CLI dispatcher bin/pyauto-pulse (check, board, census, fetch [--instance K]).
  • registry.yaml with the lens row only; repo location and GitHub identity resolved from the Mind body map (repos.yaml), never a second catalogue (spec lines 104-108; deviation from Eyes, which carries path/github itself).
  • Ingest: resolve the instance branch to one commit, read summary_path at that SHA, validate the exchange contract, write receipts/<instance>.json; failures keep the last good snapshot labelled cached with its original evidence time and the new fetch error; --path local reads report checkout revision + dirty state.
  • Board: one row per instance (scope, evidence time, last fetch, coverage, status, links) + per-instance detail of the producer's own comparisons[] (with provenance and the /profiling triage … manual-handoff action descriptor) and limitations[]; transport integrity, evidence freshness (valid_until null → "freshness policy unspecified" + age) and scientific qualification kept visibly separate; valid empty feed → "no measurements"; no league table, no recomputed ratio, no Heart verdict. state.json = cockpit feed organ: pulse, repo: PyAutoPulse, validated by PyAutoBrain/board/_state.py.
  • Hermetic tests over tests/fixtures/ for the reader's acceptance cases + a second-producer fixture with a different cell grammar behind the same contract.
  • Workflows: lint.yml, pages_dashboard.yml, dashboard_refresh.yml (push main / PR / nightly / repository_dispatch: pulse-refresh), shared concurrency group pulse-main-writers.
  • Brain: config/policy.yaml test_witness row pyautopulse: PyAutoPulse/tests and boards: pulse: PyAutoPulse (after hands); remove WITNESS_EXEMPT["PyAutoPulse"]; update the two board-order assertions in tests/test_board_theme.py.
  • Prose: AGENTS.md (boundaries, owned state, the spec's never-list), REFERENCE.md (registry schema, the v1 contract as read, receipt format), README.md.
Detailed implementation plan

Affected Repositories

  • PyAutoPulse (primary; new code)
  • PyAutoBrain (witness row, boards entry, test updates)

Branch Survey

Repository Current Branch Dirty?
organs/PyAutoPulse main clean
organs/PyAutoBrain main clean

worktree_check_conflict exit 0. Suggested branch: feature/pyautopulse-organ-skeleton — worktree ~/Code/PyAutoLabs-wt/pyautopulse-organ-skeleton.

Implementation Steps (PyAutoPulse)

  1. Layout (mirror organs/PyAutoEyes): pulse/__init__.py (ORGAN_ROOT), pulse/registry.py, pulse/summary.py, pulse/ingest.py, pulse/board.py, pulse/cli.py; bin/pyauto-pulse (11-line shim as bin/pyauto-eyes); ruff.toml copied from Eyes; .gitignore add _mind/, receipts/*.tmp; badge.json (shields endpoint, label pulse).
  2. registry.yaml: schema: 1; instances: [{instance: lens, repo: autolens_profiling, summary_path: dashboard/summary.json, supported_schema: "profiling-summary@1", dashboard_url: https://pyautolabs.github.io/autolens_profiling/, library_refs: [PyAutoLens, PyAutoGalaxy, PyAutoArray, PyAutoFit], cortex_project: null}]. registry.py: FIELDS, Instance dataclass, validate/load/get; body-map resolution: locate PyAutoMind/repos.yaml via $PYAUTO_MIND, then $PYAUTO_ROOT/ORGAN_ROOT.parent(.parent) (Eyes workspace_roots() pattern), read path/github for repo; the refresh workflow checks Mind out to _mind and passes --mind _mind as Eyes does. Disjoint-scope check across instances.
  3. summary.py: SUPPORTED = {("profiling-summary", 1)}; validate(doc) -> list[str] per autolens_profiling/dashboard/README.md (Envelope / Records / Comparisons): required envelope keys (schema, version, project, scope, generated_at, evidence_updated_at, producer_revision, comparison_policy, coverage, records, comparisons, limitations), ISO-8601 dates, valid_until null-or-date ≥ evidence_updated_at, record ids unique, axis/unit per record consistent with the axis table (runtime→s; a memory unit in a seconds series is refused), finite numerics in measurement, provenance keys present with has_provenance/qualified booleans (unknown provenance → the record is shown unqualified, never dropped silently), evidence.path repo-relative with no ../absolute/URL, each comparison's baseline/candidate resolve to record ids and share axis and the hardware identity (device,backend,precision,host) — a compile-vs-runtime or changed-hardware pair is refused/grouped separately, never merged. Coverage block shape checked. Parsing never imports scientific libraries.
  4. ingest.py: resolve_commit(github, branch="main") via https://api.github.com/repos/<owner>/<repo>/commits/<branch> (stdlib urllib, TIMEOUT=20, USER_AGENT, optional GH_TOKEN header); fetch_at(github, sha, path) from https://raw.githubusercontent.com/<owner>/<repo>/<sha>/<path>; ingest(instance, *, offline, local_path=None) → Snapshot{instance, source: remote|local, commit, dirty, fetched_at, doc|None, errors[]}; write_receipt(receipts/<instance>.json) = {instance, repo, github, branch, resolved_commit, summary_path, fetched_at, outcome: ok|invalid|unavailable|unsupported, schema, version, record_count, errors[], cached_from (prior receipt's fetched_at when outcome≠ok)}; on failure the previous receipts/<instance>.json + its cached snapshots/<instance>.json (committed last-good copy, like Eyes' marker-carried values) are kept and surfaced as cached with the ORIGINAL evidence_updated_at/fetched_at plus the new error. Local --path read: git rev-parse HEAD + git status --porcelain → commit, dirty: true/false, labelled "local checkout".
  5. board.py: collect(registry, snapshots) → rows; render_markdown/render_html/render_badge/render_state/_carried_updated (deterministic re-render as Eyes 623) / write. Row columns: instance, scope, evidence (evidence_updated_at + reason), last fetch + commit (short SHA, linked), coverage (observed.series/points/releases/cells, excluded count), integrity (ok/invalid/unavailable/unsupported/cached), freshness (valid_until missing → "freshness policy unspecified · age Nd"), qualification (qualified count / total from records), links (project Pages, repo, receipt). Detail per instance: comparisons[] table (key, policy id, axis/metric, baseline→candidate, ratio, status, qualified, reasons) with each row's evidence paths linked to the project repo at the resolved commit and a /profiling triage <comparison_key> action descriptor (kind: prompt, safety: scientific_judgement, per _state.py ITEM_OPTIONAL actions); limitations[] listed verbatim. State items: red = instance unavailable/invalid/unsupported; yellow = cached or freshness unspecified beyond N days (N configurable, default 90 — presentation only, no judgement of the science); info = census line. headline e.g. 1 project · 159 records · 0 cached.
  6. cli.py: check (registry valid, body-map resolves, each instance ingests — --offline uses committed snapshots — validates, receipt written, dashboard markers match receipt commits, _check_state via Brain _state.py with the Eyes importlib fallback → prints check: OK), board [--offline] [--from INSTANCE=PATH] [--mind PATH], census (one line per instance: records/comparisons/limitations/qualified), fetch [--instance K] (ingest + receipt only).
  7. Tests (tests/conftest.py with no_network autouse monkeypatching urlopen, fake_mind writing a minimal repos.yaml, fake_brain with _state.py copy or import; fixtures under tests/fixtures/): valid_empty.json (records=[] → "no measurements", check OK); unavailable (urlopen raises → cached snapshot kept, evidence time unchanged, error shown); unsupported_schema.json (version: 2 → outcome unsupported); duplicate_id.json; stale_cached (two renders, assert evidence_updated_at and fetched_at not re-aged); unknown_provenance.json (has_provenance: false → shown unqualified, count correct); axis_mismatch.json (comparison pairing a compile record with a runtime record → refused with reason); changed_hardware.json (same cell on local_cpu_fp64 vs a100_fp32 → separate groups, no combined ratio); second_producer.json (project autogalaxy_profiling_fixture, different section/cell vocabulary, scope release-runtime) → identical reader code path, no branching on project name (assert via a registry with two instances rendering two rows). Real-feed smoke: tests/fixtures/lens_summary_v1.json = a trimmed copy (≤20 records, 5 comparisons) of the live file, validates clean. test_cli.py: check prints check: OK; board writes three files; state.json passes validate_state.
  8. Workflows (copy Eyes, rename): lint.yml (ruff check/format, pytest, bin/pyauto-pulse check --offline, lychee excluding raw.githubusercontent, pyautolabs.github.io/PyAutoPulse, GitHub issue/pull/commit URLs); pages_dashboard.yml (push main paths dashboard.html, badge.json, state.json; concurrency pages; configure-pages@v5 enablement: true; stage _site/index.html+badge.json+state.json; checkout PyAutoBrain → python3 PyAutoBrain/board/_state.py _site/state.json; upload/deploy); dashboard_refresh.yml (repository_dispatch: types: [pulse-refresh], cron 55 3 * * *, workflow_dispatch, push main paths registry.yaml/pulse/**, PR; permissions contents+actions write; concurrency pulse-main-writers no-cancel; checkout Mind → _mind; 3-attempt fetch/reset/bin/pyauto-pulse board --mind _mind/commit pulse: refresh dashboard (${GITHUB_EVENT_NAME}) [skip ci]/push/gh workflow run pages_dashboard.yml --ref main).
  9. Prose: AGENTS.md (keep the four repos_sync marker blocks; add: what Pulse owns — registry, read contract, receipts, board; the never-list verbatim from spec 169-173 + 158-159; the three-notions rule; "the Brain's profiling conductor is the only judge"; how to add an instance = registry row + fixture); REFERENCE.md (registry schema table, profiling-summary@1 as the organ reads it with the validation list above, receipt JSON schema, snapshot/cached semantics, CLI); README.md (what/why, Pages URL, how to run check/board, links to the spec + autolens_profiling dashboard README). First commit of the real board: run bin/pyauto-pulse board online once and commit dashboard.md/.html, state.json, badge.json, receipts/lens.json, snapshots/lens.json.

Implementation Steps (PyAutoBrain)

  1. config/policy.yaml: test_witness add pyautopulse: PyAutoPulse/tests after the eyes row (line ~128); board.boards add pulse: PyAutoPulse after hands: (one name: Repo per line — the Mind footer regex). tests/test_policy_seams.py: delete the "PyAutoPulse" entry from WITNESS_EXEMPT. tests/test_board_theme.py:205-206,228-229: insert pulse after hands in the expected order. board/_theme.py ORGANS/MARKS: add a pulse style entry (colour + mark) so the footer chip is not the umbrella fallback. Full pytest tests green. Brain PR merges AFTER the PyAutoPulse PR (witness dir must exist on main).

Key Files

  • organs/PyAutoEyes/eyes/{registry,manifest,board,cli}.py, tests/conftest.py, .github/workflows/*.yml — the model
  • organs/PyAutoBrain/docs/research/profiling_inference_organs.md:89-113,152-173,202-227,250-255 — registry, contract, dashboard rules, acceptance cases
  • lens/autolens_profiling/dashboard/README.md + summary.json — the v1 contract and live feed
  • organs/PyAutoBrain/board/_state.py — state validator (validate_state, build_state)
  • organs/PyAutoBrain/config/policy.yaml:106-130,164-178 — witness + boards
  • lens/autolens_visualization/.github/workflows/render.yml:85-99 — dispatch sender pattern (follow-up prompt for autolens_profiling, out of scope)

Testing

ruff check . && ruff format --check . && pytest -q tests && bin/pyauto-pulse check in PyAutoPulse; pytest tests in Brain; python3 PyAutoBrain/board/_state.py state.json exit 0.

Out of scope

Brain board card + cockpit identity transition (phase 3); autolens_profiling dispatch sender (separate one-line prompt); second real producer (phase 4); inference organ.

Human acts

Enable GitHub Pages on PyAutoLabs/PyAutoPulse (source: GitHub Actions) before/at merge — first Pages run fails otherwise (PyAutoEyes#2 precedent). Merge via /prm, PyAutoPulse first.

Original Prompt

Click to expand starting prompt

Profiling organ phase 2 — the organ skeleton: registry, reader, board, workflows

Type: feature
Target: PyAutoPulse
Repos:

  • PyAutoPulse
  • PyAutoBrain
    Themes:
  • profiling
    Difficulty: large
    Autonomy: supervised
    Priority: high
    Status: draft
    Consequence: judge
    Witness: bin/pyauto-pulse check prints check: OK against the lens registry row (summary resolves at one commit, validates, receipt written); hermetic pytest green over the acceptance cases; bin/pyauto-pulse board writes dashboard.md/.html + state.json (validated by PyAutoBrain/board/_state.py) with one autolens_profiling row linking to its Pages page; lint/pages/refresh workflows green
    Review-minutes: 20
    Unattended: ready
    Filed: 2026-10-02
    Epic: profiling-organ-birth
    Phase: 2

Blocked on: phase 0 shipped (the repo exists and is named) and phase 1 shipped (the profiling-summary v1 file it reads is published).

Phase 2 of the profiling-organ-birth epic. Human decision 2026-10-02: the
organ is PyAutoPulse (organ key pulse); the repo is created in phase 0. Modelled on the Eyes skeleton
(complete/2026/09/eyes-organ-skeleton.md, PyAutoEyes#2): the organ reads,
validates and links; it renders no measurement, judges nothing and copies no
result trees.

Task

  1. registry.yaml — version + one row per instance, fields per the spec
    ("Proposed project registry"): instance: lens, repo: autolens_profiling
    (a Mind body-map identity — location and GitHub come from repos.yaml, no
    second catalogue), summary_path: dashboard/summary.json,
    supported_schema: profiling-summary@1, dashboard_url, library_refs: [PyAutoLens, PyAutoGalaxy, PyAutoArray, PyAutoFit], cortex_project when
    one exists. Lens row only.
  2. Reader (<pkg>/registry.py, <pkg>/summary.py, <pkg>/ingest.py;
    PyYAML + stdlib, no scientific imports): resolve the instance's branch to
    one commit, read summary_path at that commit (raw GitHub URL at the
    SHA), validate the exchange contract (known schema/version, required
    fields, unique ids, finite numerics, dates, safe evidence paths), and write
    a receipt (receipts/<instance>.json: resolved commit, fetch time,
    outcome, error). Unsupported version / invalid feed → the instance is
    reported failed and the last good snapshot is kept, labelled cached with
    its original evidence time and the new fetch error (never re-aged by a
    render). A local --path read reports checkout revision + dirty state and
    is labelled as such.
  3. Board (<pkg>/board.py → dashboard.md, dashboard.html,
    state.json): one row per registered project with scope, evidence time,
    last fetch, coverage, status and links; a detail section per instance
    showing the producer's own comparisons[] (drift candidates with their
    provenance and the /profiling triage … prompt as a manual-handoff
    action descriptor) and limitations[]. Keep the three notions separate on
    the page: transport/schema integrity, evidence freshness (valid_until
    missing → "freshness policy unspecified", shown with the age), scientific
    qualification. A valid empty feed reads "no measurements". No league table
    across cells; no recomputed ratio; no Heart verdict. state.json is the
    organ's own cockpit feed (organ: pulse, repo: PyAutoPulse), status summarising
    the organ's monitoring scope only.
  4. bin/pyauto-pulse dispatcher (Heart/Eyes pattern): check, board,
    census, fetch [--instance K].
  5. Tests (hermetic, fixtures under tests/fixtures/): the spec's acceptance
    cases that belong to the reader — valid empty producer; one project
    unavailable (cached snapshot + error shown); unsupported schema; duplicate
    record id; stale cached snapshot keeps its evidence time; unknown
    provenance; compile/runtime axis mismatch refused; changed hardware shown
    as a separate group, never merged. Plus a second-producer fixture with
    a different cell grammar behind the same contract, proving the reader has
    no project-specific branching (a fixture, not a manufactured repo).
  6. Workflows: lint.yml (ruff + pytest + check), pages_dashboard.yml
    (publish dashboard.html as index with state.json beside it, validated by
    PyAutoBrain/board/_state.py), dashboard_refresh.yml (push to main, PR,
    nightly, repository_dispatch from autolens_profiling's
    pages_dashboard.yml — the Eyes/Cortex pattern; the sender side is a
    one-line follow-up in the project repo, filed separately), shared
    concurrency group for the main-writers.
  7. Prose: AGENTS.md (organ boundaries, the growth-rule state it owns, the
    "never" list from the spec's "Profiling domain contract"), REFERENCE.md
    (registry schema, the profiling-summary v1 contract as the organ reads it,
    receipt format), README.md. Brain: config/policy.yaml test_witness
    row for the new repo (as Brain#428 did for Eyes).

Out of scope

The Brain board card and the cockpit identity transition (phase 3); any change
to autolens_profiling beyond the dispatch sender follow-up; a second real
producer (phase 4); the inference organ.

Ship policy

Supervised: plan on the issue, end at PR-open, merge human (/prm). Human
enables GitHub Pages (source: GitHub Actions) on the new repo — the first
Pages run fails until then, as PyAutoEyes#2 did.

Activity

  1. Jammy2211 commented on Oct 2, 2026

    @Jammy2211
    ContributorAuthor

    Library PRs Created

    Merge PyAutoPulse first, then PyAutoBrain (witness row needs PyAutoPulse/tests on main). Human: enable GitHub Pages on PyAutoLabs/PyAutoPulse (source: GitHub Actions) before merge. Follow-ups filed separately: pulse-refresh dispatch sender in autolens_profiling; phase 3 Brain board card + cockpit transition.

  2. Jammy2211 commented on Oct 2, 2026

    @Jammy2211
    ContributorAuthor

    Shipped

    Phase 2 of profiling-organ-birth merged 2026-10-02: the PyAutoPulse organ skeleton.

    Merged PRs

    • PyAutoPulse#2 — registry.yaml (lens row; location/GitHub from the Mind body map), pulse/ package (registry, summary validator, commit-pinned ingest + receipts/snapshots, board, CLI), bin/pyauto-pulse, 76 hermetic tests + fixtures, lint.yml / pages_dashboard.yml / dashboard_refresh.yml, AGENTS/REFERENCE/README, first live render (lens @ autolens_profiling d9f385f0, 159 records)
    • PyAutoBrain#449 — test_witness row pyautopulse, boards: pulse, WITNESS_EXEMPT["PyAutoPulse"] removed, board-order tests, Pulse chip palette/mark

    CI notes. Pulse lint failed twice on lychee: github.com answers unauthenticated blob page fetches with 503 (every blob page, incl. cpython's; Eyes' main lint hit the same at 10:57). Fixed on the branch by passing GITHUB_TOKEN to lychee (--github-token, documented by lychee) — commit b2d4a595; lint then green. Brain Tests green on 3.12 and 3.13. Heart YELLOW 85 at ship (Cortex checkout drift owned by another session; .github table; no rehearsal), acknowledged by the human. Pages enabled by the human before merge.

    Follow-ups

    • autolens_profiling/.github/workflows/pages_dashboard.yml: add the repository_dispatch pulse-refresh sender (copy autolens_visualization/render.yml:85-99).
    • PyAutoEyes lint.yml: the same lychee --github-token one-liner.
    • PyAutoBrain/bin/ensure_workspace_labels.sh: add PyAutoPulse to its repo list (label was created by hand with a different description).
    • Phase 3: draft/feature/pyautopulse/profiling_organ_p3_brain_board_cockpit_transition.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions