Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>PyAutoCortex Dashboard</title>
<!-- generated by `pyauto-brain cortex dashboard --apply` on 2026-10-19 β€” regenerate, do not hand-edit -->
<!-- generated by `pyauto-brain cortex dashboard --apply` on 2026-10-21 β€” regenerate, do not hand-edit -->
<style>:root{color-scheme:light dark;--bg:#fff;--fg:#1f2328;--muted:#59636e;
--board-max:77.5rem;--board-gutter:1rem;--board-measure:65ch;
--line:#d8dee4;--btn:#f6f8fa;--ok:#1a7f37;--warn:#9a6700;--bad:#d1242f;
Expand Down Expand Up @@ -334,7 +334,7 @@
- Open autofit_inference: https://github.com/PyAutoLabs/autofit_inference
- Open euclid_detailed: https://github.com/PyAutoLabs/euclid_detailed
- Open cowls: https://github.com/PyAutoLabs/cowls
- Open cowls_ansofi: https://github.com/PyAutoLabs/cowls_ansofi</textarea></details><div class="orchestration-freshness" data-freshness="grey"><details data-freshness-stamp data-refreshed-at="2026-10-19T19:05:59Z"><summary><span data-freshness-label>Last updated 2026-10-19 19:05:59 UTC</span></summary><time datetime="2026-10-19T19:05:59Z">2026-10-19 19:05:59 UTC</time></details><a data-refresh-link href="https://github.com/PyAutoLabs/PyAutoCortex/actions/workflows/dashboard_refresh.yml" title="Open the dashboard refresh controls">↻ Update</a></div></div><p class="orchestration-status" role="status" aria-live="polite"></p></section>
- Open cowls_ansofi: https://github.com/PyAutoLabs/cowls_ansofi</textarea></details><div class="orchestration-freshness" data-freshness="grey"><details data-freshness-stamp data-refreshed-at="2026-10-21T19:12:01Z"><summary><span data-freshness-label>Last updated 2026-10-21 19:12:01 UTC</span></summary><time datetime="2026-10-21T19:12:01Z">2026-10-21 19:12:01 UTC</time></details><a data-refresh-link href="https://github.com/PyAutoLabs/PyAutoCortex/actions/workflows/dashboard_refresh.yml" title="Open the dashboard refresh controls">↻ Update</a></div></div><p class="orchestration-status" role="status" aria-live="polite"></p></section>
<nav class="board-nav" aria-label="Board sections"><a class="board-nav-card" href="#summary"><span class="board-nav-count">16</span><span class="board-nav-label">Running</span></a><a class="board-nav-card" href="#summary"><span class="board-nav-count">47</span><span class="board-nav-label">Open</span></a><a class="board-nav-card" href="#projects"><span class="board-nav-count">12</span><span class="board-nav-label">Projects</span></a><a class="board-nav-card" href="#orchestration-cortex"><span class="board-nav-label">Check in</span></a></nav>

<p class="muted mdsrc"><a href="https://github.com/PyAutoLabs/PyAutoCortex/blob/main/dashboard.md" class="board-source-link" title="Markdown version" aria-label="Markdown version"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" focusable="false"><path d="M14 3H6a1 1 0 0 0-1 1v16a1 1 0 0 0 1 1h12a1 1 0 0 0 1-1V8Z"/><path d="M14 3v5h5M8 12h8M8 16h6"/></svg></a></p>
Expand All @@ -348,7 +348,7 @@
<tr><td><b>ep_toy_gaussian</b></td><td>0</td><td>0</td><td>2026-10-14</td></tr>
<tr><td><b>slope_hierarchy_scale</b></td><td>0</td><td>0</td><td>2026-10-14</td></tr>
<tr><td><b>ic50_workspace</b></td><td>0</td><td>0</td><td>2026-10-14</td></tr>
<tr><td><b>autolens_inference</b></td><td>0</td><td>0</td><td>2026-10-19</td></tr>
<tr><td><b>autolens_inference</b></td><td>0</td><td>0</td><td>2026-10-21</td></tr>
<tr><td><b>autofit_inference</b></td><td>0</td><td>0</td><td>2026-10-16</td></tr>
<tr><td><b>euclid_detailed</b></td><td>0</td><td>0</td><td>2026-10-15</td></tr>
<tr><td><b>cowls</b></td><td>0</td><td>0</td><td>2026-10-15</td></tr>
Expand Down Expand Up @@ -612,7 +612,7 @@ <h4>Runs (0)</h4>
</div>
</details>
<details class="project" id="project-autolens_inference">
<summary><h3>autolens_inference</h3><span class="purpose">Do the PyAutoLens inference backends agree on the HST SLaM posterior</span><span class="meta">no runs recorded Β· updated 2026-10-19</span></summary>
<summary><h3>autolens_inference</h3><span class="purpose">Do the PyAutoLens inference backends agree on the HST SLaM posterior</span><span class="meta">no runs recorded Β· updated 2026-10-21</span></summary>
<div class="project-body">
<dl class="where">
<dt>Registered:</dt><dd>not recorded</dd>
Expand All @@ -625,13 +625,13 @@ <h4>Current position</h4>
</div>
<h4>Runs (0)</h4>
<p class="muted">nothing on the cluster</p>
<details class="sub"><summary>Recent log β€” 5 of 45 entries</summary>
<details class="sub"><summary>Recent log β€” 5 of 46 entries</summary>
<ul class="log">
<li>2026-10-21 β€” <i>note</i> β€” <details class="full"><summary><span class="clip">autolens_inference#43 (likelihood-lifecycle phase 0C, merged 2026-10-21 as 07880c2) moved the rows: results/slam/imaging/hst/&lt;variant&gt;/&lt;config&gt;/stages_seed&lt;n&gt;.* are now results/imaging/&lt;rectangular|delaunay&gt;/hst/validation/&lt;default|light_f…</span><span class="less">show less</span></summary><p>autolens_inference#43 (likelihood-lifecycle phase 0C, merged 2026-10-21 as 07880c2) moved the rows: results/slam/imaging/hst/&lt;variant&gt;/&lt;config&gt;/stages_seed&lt;n&gt;.* are now results/imaging/&lt;rectangular|delaunay&gt;/hst/validation/&lt;default|light_fixed&gt;/&lt;setup_id&gt;/seed_&lt;n&gt;/stages.*; results/searches/point_source/&lt;sampler&gt;/... are results/point_source_source/source_plane/simple/samplers/default/&lt;sampler&gt;/explicit/&lt;setup_id&gt;/seed_&lt;n&gt;/; results/archive/&lt;dated&gt; is archive/YYYY/MM/&lt;dated&gt;/. Old β†’ new map: catalogue/migration_2026_10.json. Entries above this line cite the old paths. #44 (bdc8b2d) exports dashboard/walltime_validation.json (walltime-validation@1, 56 records) to PyAutoPulse.</p></details></li>
<li>2026-10-19 β€” <i>note</i> β€” <details class="full"><summary><span class="clip">412539 (jax_cpu dense) elapsed from sacct, read 2026-10-19 ~17:25Z by the session architect: 412539_0 1-05:59:53 (ended 2026-10-18T23:46:01), 412539_1 1-02:57:03 (ended 2026-10-18T20:43:11). This supplements the 2026-10-19 finished entry,…</span><span class="less">show less</span></summary><p>412539 (jax_cpu dense) elapsed from sacct, read 2026-10-19 ~17:25Z by the session architect: 412539_0 1-05:59:53 (ended 2026-10-18T23:46:01), 412539_1 1-02:57:03 (ended 2026-10-18T20:43:11). This supplements the 2026-10-19 finished entry, which carries no wall. Its rows have still not been read.</p></details></li>
<li>2026-10-19 β€” <i>run</i> β€” <details class="full"><summary><span class="clip">412543 failed β€” wall 0:47: hst-imaging-mass-samplers wave 1 (cleanslate_2026_10_17_wave1, submit_hst_cleanslate_mass_samplers_jax_gpu_dense): 24-arm slice nautilus/emcee/nuts/smc x sampler seeds 0-2 x both light modes on the base-seed-0 pr…</span><span class="less">show less</span></summary><p>412543 failed β€” wall 0:47: hst-imaging-mass-samplers wave 1 (cleanslate_2026_10_17_wave1, submit_hst_cleanslate_mass_samplers_jax_gpu_dense): 24-arm slice nautilus/emcee/nuts/smc x sampler seeds 0-2 x both light modes on the base-seed-0 prepared problems of hpc_a100_jax_gpu_dense_fp64, array 0-23%4, 6 h containment, --dependency afterok:412535_0, pin 2026.10.17.1; human-authorised 2026-10-17 β€” sacct read 2026-10-19 ~17:25Z by the session architect: tasks _0.._17 all FAILED on 2026-10-17 (ends 18:51:38-20:32:26Z, elapsed 00:04:15-00:47:10; wall given is the longest task); _18 CANCELLED after 00:04:00; _[19-23] CANCELLED unstarted 2026-10-17T20:01:03Z. Failure classes as in the 2026-10-17 note (NUTS/SMC A100 OOM; nautilus/emcee row-writer TypeError). Its output and the 8 failed NUTS/SMC rows were moved aside on 2026-10-19 to aside/autolens_inference_cleanslate_wave1_attempt1/ (RAL and laptop)</p></details></li>
<li>2026-10-19 β€” <i>note</i> β€” <details class="full"><summary><span class="clip">HUMAN DECISION 2026-10-19, verbatim: "I want us to check the on going inference campaign on RAL, but I am ready for us to update folder structures on RAL and locally and clean things up so cancel on going jobs and we'll work on the clean u…</span><span class="less">show less</span></summary><p>HUMAN DECISION 2026-10-19, verbatim: "I want us to check the on going inference campaign on RAL, but I am ready for us to update folder structures on RAL and locally and clean things up so cancel on going jobs and we'll work on the clean up". Scheduler read 15:49:05Z: only 412541 and 412565 running; both scancelled ~15:51Z, queue empty; 412539 had COMPLETED. On "yes, go", autolens_inference#36 (flatten, closes #35) merged 15:53:33Z as c3ca4de; the RAL checkout fast-forwarded b91ce9c -&gt; c3ca4de and the laptop to c3ca4de, after a full hpc/sync pull. flatten_cleanslate.py --apply: RAL 28 moves, 55 rewrites; laptop mirror 28 moves, 38 rewrites; both EXIT 0. New layout: output/autolens/imaging/hst/slam/&lt;variant&gt;/&lt;config&gt;/seed_&lt;n&gt;/&lt;stage&gt;/, output/autolens/prepared/&lt;config&gt;/seed_&lt;n&gt;/&lt;variant&gt;/, rows at results/slam/imaging/hst/&lt;variant&gt;/&lt;config&gt;/ (no __cleanslate_2026_10_17 suffix). RAL and laptop each hold 1974 files with identical paths and sizes; hpc/sync status dry run transfers 0. Human, verbatim: "check some older chats, think we dont need prepared, experiments and experiment_sources to exist anymore? Can we archive them somewhere separate to makr things easier to follow. Then do the apply". The wave-1 attempt-1 material (output/experiments 341M, output/experiment_sources 76M, results/searches/experiments with 8 failed NUTS/SMC rows) moved to /mnt/ral/jnightin/inference/aside/autolens_inference_cleanslate_wave1_attempt1/ on RAL and /mnt/c/Users/Jammy/Science/inference/aside/autolens_inference_cleanslate_wave1_attempt1/ on the laptop. prepared/ KEPT as the agent's proposal, not yet confirmed by the human: wave.py needs its manifests, and so does the sparse fixed-light repro. Partial stage dirs of the cancelled legs remain; they are not failures. Execution completion only. No compute submitted.</p></details></li>
<li>2026-10-19 β€” <i>run</i> β€” <details class="full"><summary><span class="clip">412565 failed β€” wall 44:52: hst-imaging-mass-samplers clean slate: numba_cpu sparse SLaM base (64gb, 8 CPUs), --lens-light both, seeds 0-1, from autolens_inference main b91ce9c (PR #32), pin 2026.10.17.1, output root output/cleanslate_2026…</span><span class="less">show less</span></summary><p>412565 failed β€” wall 44:52: hst-imaging-mass-samplers clean slate: numba_cpu sparse SLaM base (64gb, 8 CPUs), --lens-light both, seeds 0-1, from autolens_inference main b91ce9c (PR #32), pin 2026.10.17.1, output root output/cleanslate_2026_10_17, results tag cleanslate_2026_10_17, prepared export; --stages-from hpc_a100_jax_cpu_dense_fp64:source_lp shares source_lp[1] from the jax_cpu dense chain with an in-job wait (hpc/wait_for_stage.sh); rows mark that stage shared and exclude its wall from the numba chain wall; submitted 2026-10-17T18:58Z by the architect; human decisions 2026-10-17 (8 CPUs, shared stage, submit now) β€” CANCELLED by scancel ~15:51Z on the human's decision, not a run failure (sacct CANCELLED, elapsed 1-20:52:11). Complete at cancel (.completed markers): seeds 0 and 1 slam_base light[1], source_pix[1], source_pix[2], mass_total[1] (source_lp[1] shared from jax_cpu dense via --stages-from). Incomplete: slam_base_lightfixed mass_total[1] on both seeds (running). Partial stage dirs left on disk under output/autolens/imaging/hst/slam/&lt;variant&gt;/hpc_a100_numba_cpu_sparse_fp64/seed_&lt;n&gt;/ (flat layout, autolens_inference#36)</p></details></li>
<li>2026-10-19 β€” <i>run</i> β€” <details class="full"><summary><span class="clip">412541 failed β€” wall 47:04: hst-imaging-mass-samplers clean slate: hpc_a100_jax_cpu_sparse_fp64 (submit_hst_cleanslate_jax_cpu_sparse), --lens-light both, seeds 0-1, pin 2026.10.17.1 (PYAUTO_LIB_BASE=/mnt/ral/jnightin/inference/pins/2026.1…</span><span class="less">show less</span></summary><p>412541 failed β€” wall 47:04: hst-imaging-mass-samplers clean slate: hpc_a100_jax_cpu_sparse_fp64 (submit_hst_cleanslate_jax_cpu_sparse), --lens-light both, seeds 0-1, pin 2026.10.17.1 (PYAUTO_LIB_BASE=/mnt/ral/jnightin/inference/pins/2026.10.17.1), output root output/cleanslate_2026_10_17, results tag cleanslate_2026_10_17, prepared export output/prepared/cleanslate_2026_10_17/hpc_a100_jax_cpu_sparse_fp64/seed_&lt;n&gt;/&lt;variant&gt;/; human-authorised 2026-10-17 β€” CANCELLED by scancel ~15:51Z on the human's decision, not a run failure (sacct CANCELLED, elapsed 1-23:04:43). Complete at cancel (.completed markers): seed 0 slam_base light[1], source_lp[1], source_pix[1], source_pix[2], mass_total[1]; seed 1 source_lp[1], source_pix[1]. Incomplete: seed 0 slam_base_lightfixed mass_total[1] (running), seed 1 source_pix[2] (running) onward and its lightfixed leg. Partial stage dirs left on disk under output/autolens/imaging/hst/slam/&lt;variant&gt;/hpc_a100_jax_cpu_sparse_fp64/seed_&lt;n&gt;/ (flat layout, autolens_inference#36)</p></details></li>
</ul>
<p class="muted"><a href="https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/autolens_inference.md">full log</a></p>
<p class="muted">active Β· both Β· <a href="https://github.com/PyAutoLabs/autolens_inference/issues/4">autolens_inference#4</a></p>
Expand Down
Loading
Loading