One ledger per science project: what is on the cluster right now, what the human said they would do next, and the last few things they wrote down. Nothing here is inferred from a result — the board hands you the check-in and the resume prompt; what a run meant is yours to log.
+/cortex — anywhere: `pyauto-brain cortex checkin --apply --push` — stamp, re-render and push the board since the last check-in (2026-09-25T18:10Z), then read me the by-project summary. Record nothing about results — I will tell you what to log." aria-label="Copy the Claude command">📋
check in since last time
+
Last check-in
Summary
Project
Running
Open
Last update
subhalo_validation
0
5
2026-09-07
-
euclid_dr1_prelim
1
3
2026-09-17
euclid_dr1
1
2
2026-09-23
analytic_gaussian
0
0
2026-09-10
ep_toy_gaussian
0
0
2026-09-24
slope_hierarchy_scale
0
3
2026-09-24
ic50_workspace
0
4
2026-09-09
autolens_inference
6
0
2026-09-24
-
euclid_sersics
0
1
2026-09-14
Projects
@@ -272,29 +270,6 @@
subhalo_validation — Do the SLaM subhalo pipelines false-detect on lenses
resume subhalo_validation
-
euclid_dr1_prelim — Refit the ten DR1-prelim tiles and rebuild the catalogue on RAL
342629 reran the ten tiles on refreshed libraries; eight finished and confirmed all four 09-10 defects fixed, tiles 1 and 3 are still in vis_lp. The SED chain ran on GPU for the eight (342648) and the CPU verification on tile 9 (342668_9) passed in 9:24, faster than the GPU — the CPU route is the one the main experiment uses. Next: pull, bundle with output_sed, compare magnitudes (µJy fluxes) against the 20260623 catalogue; then tiles 1/3 SED on the CPU script. The parity clause needs restating: rerun scatter, not ordering, is what fails it.
342648_[0,2,4-9] — open — gpu — 2026-09-11 — ordered_rerun_catalogue_parity: SED chain `hpc/batch_gpu/submit_sersic_waveband_8tiles` (a copy of `submit_sersic_waveband` differing only in `--array=0,2,4-9`…
-
342668_9 — open — ral — 2026-09-11 — ordered_rerun_catalogue_parity: CPU verification of the SED chain via hpc/batch_cpu/submit_sersic_waveband (pipeline PR #70, sbatch --array=9): tile 102008848,…
-
342796_[1,3,8] — open — ral — 2026-09-12 — ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveb…
-
-
Last 5
-
-
2026-09-17 — result — Witt–Wynne SIEP projection now a catalogue product (pipeline PR #86 merged 2026-09-17, stage 7 of the inspection bundle). Ran catalogue/scripts/witt_wynne.py on the ten pulled dr1_prelim_grade_ab tiles: 10/10 rows, 0 skips, source centre r…
-
2026-09-12 — run — 342796_[1,3,8] submitted: ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveband (md5 65db7d8b… on RAL = the clone's commit 88dda9d…
-
2026-09-11 — note — **CPU verification PASSED and was faster than the GPU.** `342668_9` (ral, 8 CPU, JAX on the CPU backend) COMPLETED in **9:24** wall, exit 0, all eight `sersic_lens_model/{vis,nir_y,nir_j,nir_h,decam_g,decam_r,decam_i,decam_z}` searches wri…
-
2026-09-11 — note — **342629 scored on the eight tiles that finished; all four 09-10 defects confirmed fixed; parity fails on rerun scatter, not ordering.** `sacct`: tasks 0,2,4,5,6,7,8,9 COMPLETED (walls 5:45–9:42, MaxRSS 7.6–10.9 GB); tasks 1 (`Tile10200729…
-
2026-09-11 — run — 342668_9 submitted: ordered_rerun_catalogue_parity — CPU verification of the SED chain via hpc/batch_cpu/submit_sersic_waveband (pipeline PR #70, sbatch --array=9): tile 102008848, JAX on the CPU backend, 8 CPU; same seeded vis_lp zip as t…
The four Sersic-stage variants are done on the real sample (342808: 99 of 100 lenses with all four tags; task 72 lost its later variants to the walltime) and the simulated sample's vis_lp stage is done too (342911: 100 of 100). 343043 is now running the same four variants on the 100 mocks — the recovery test, where each fit's n can be read against the truth.json it was simulated from (49 of the 100 were simulated at n = 3.0 under the 4.95 prior-edge rule). Next: when 343043 lands, pull it with the 342911 results, scrape all four variant tags on both samples with catalogue/scripts/lens_sersic.py --unique_tag sersic_lens_model_<variant>, and put the recovered n against truth side by side with the real sample's n distributions. The human rules on whether any variant clears the n = 5 pile-up.
-
Runs
-
-
343043_[0-99] — open — ral — 2026-09-14 — euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array ta…
-
-
Last 5
-
-
2026-09-14 — run — 343043_[0-99] submitted: euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks on the simulated sample dataset/dr1_sep1_sersics_si…
-
2026-09-14 — run — 342911_[0-99] finished — wall 1:59: euclid_sersics_sim_vis_lp: the vis_lp stage (initial_lens_model.py --stage=vis_lp, MGE lens + MGE source, 15 free parameters, identical to the real sample's stage 1) on the new simulated sample dataset/d…
-
2026-09-14 — run — 342808_[0-99] finished — wall 18:00: euclid_sersics_variants: the Sersic stage rerun in four variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks (one lens per task), array 0-99, 4 cpu…
-
2026-09-13 — note — the simulated sample behind 342911: scripts/simulator.py --from-result rebuilds each of the 100 lenses from its own sersic_lens_model_baseline/vis result of run 342808 (Sersic lens light + Sersic source), keeping that tile's real PSF stamp…
-
2026-09-13 — run — 342911_[0-99] submitted: euclid_sersics_sim_vis_lp: the vis_lp stage (initial_lens_model.py --stage=vis_lp, MGE lens + MGE source, 15 free parameters, identical to the real sample's stage 1) on the new simulated sample dataset/dr1_sep1_ser…
euclid — The original Euclid DR1 grade-AB catalogue project, superseded by euclid_dr1_prelim
dormant · both · no issue yet · projects/euclid.md · local /mnt/c/Users/Jammy/Science/euclid · RAL /mnt/ral/jnightin/euclid_strong_lens_modeling_pipeline
Now
@@ -456,9 +411,11 @@
euclid — The original Euclid DR1 grade-AB catalogue project, superseded by
inference_programme — The first inference programme, retired and restarted as autolens_inference — retired 2026-09-07: restarting from scratch: the project setup was unsatisfactory and it informed how the Cortex manages projects; RAL outputs stashed at /mnt/ral/jnightin/inference_programme_retired_2026-09-07
+
euclid_dr1_prelim — Refit the ten DR1-prelim tiles and rebuild the catalogue on RAL — retired 2026-09-25: retire euclid_dr1_prelim; I will move it to z_vault manually but I don't want it to eat up download time
slope_hierarchy — Hierarchical slope recovery at N=5, wrapped up and succeeded by slope_hierarchy_scale — retired 2026-09-08: wrapped up 2026-07-22 at N=5; succeeded by slope_hierarchy_scale; tree vaulted under z_projects_complete/
+
euclid_sersics — Why does the lens-light Sersic index pile up at the n=5 prior edge — retired 2026-09-25: retire euclid_sersics as well
No ledger
diff --git a/dashboard.md b/dashboard.md
index cbf4768..eef1df0 100644
--- a/dashboard.md
+++ b/dashboard.md
@@ -10,34 +10,32 @@ One ledger per science project: what is on the cluster right now, what the human
| Where | Count |
|-------|------:|
-| [Running](#projects) | 8 |
-| [Open](#projects) | 18 |
-| [Projects](#projects) | 9 |
+| [Running](#projects) | 7 |
+| [Open](#projects) | 14 |
+| [Projects](#projects) | 7 |
📋 check in since last time
```
/cortex — on the laptop: `pyauto-brain cortex pull` — pull every active science project through its own sync CLI and show me where each run stands.
-/cortex — anywhere: `pyauto-brain cortex checkin --apply --push` — stamp, re-render and push the board since the last check-in (2026-09-19T10:31Z), then read me the by-project summary. Record nothing about results — I will tell you what to log.
+/cortex — anywhere: `pyauto-brain cortex checkin --apply --push` — stamp, re-render and push the board since the last check-in (2026-09-25T18:10Z), then read me the by-project summary. Record nothing about results — I will tell you what to log.
```
-### Last check-in: 2026-09-19T10:31Z
+### Last check-in: 2026-09-25T18:10Z
## Summary
| Project | Running | Open | Last update |
|---|---:|---:|---|
| subhalo_validation | 0 | 5 | 2026-09-07 |
-| euclid_dr1_prelim | 1 | 3 | 2026-09-17 |
| euclid_dr1 | 1 | 2 | 2026-09-23 |
| analytic_gaussian | 0 | 0 | 2026-09-10 |
| ep_toy_gaussian | 0 | 0 | 2026-09-24 |
| slope_hierarchy_scale | 0 | 3 | 2026-09-24 |
| ic50_workspace | 0 | 4 | 2026-09-09 |
| autolens_inference | 6 | 0 | 2026-09-24 |
-| euclid_sersics | 0 | 1 | 2026-09-14 |
## Projects
@@ -76,40 +74,6 @@ Next: pull when the numba chains land and read the evidence_increase of each; no
-### euclid_dr1_prelim — Refit the ten DR1-prelim tiles and rebuild the catalogue on RAL
-
-active · both · [PyAutoCortex#35](https://github.com/PyAutoLabs/PyAutoCortex/issues/35) · [projects/euclid_dr1_prelim.md](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_dr1_prelim.md) · local `/mnt/c/Users/Jammy/Science/euclid_dr1_prelim` · RAL `/mnt/ral/jnightin/euclid_dr1_prelim`
-
-**Now**
-
-342629 reran the ten tiles on refreshed libraries; eight finished and confirmed all four 09-10 defects fixed, tiles 1 and 3 are still in vis_lp. The SED chain ran on GPU for the eight (342648) and the CPU verification on tile 9 (342668_9) passed in 9:24, faster than the GPU — the CPU route is the one the main experiment uses.
-Next: pull, bundle with output_sed, compare magnitudes (µJy fluxes) against the 20260623 catalogue; then tiles 1/3 SED on the CPU script. The parity clause needs restating: rerun scatter, not ordering, is what fails it.
-
-**Runs**
-
-- 342629_[0-9] — running — ral — 2026-09-10 — ordered_rerun_catalogue_parity: two-stage vis_lp(n_live=750)+vis_pix(n_live=300), array 0-9, ten dr1_prelim tiles; libraries refreshed BEFORE submit (PyAutoNer…
-- 342648_[0,2,4-9] — open — gpu — 2026-09-11 — ordered_rerun_catalogue_parity: SED chain `hpc/batch_gpu/submit_sersic_waveband_8tiles` (a copy of `submit_sersic_waveband` differing only in `--array=0,2,4-9`…
-- 342668_9 — open — ral — 2026-09-11 — ordered_rerun_catalogue_parity: CPU verification of the SED chain via hpc/batch_cpu/submit_sersic_waveband (pipeline PR #70, sbatch --array=9): tile 102008848,…
-- 342796_[1,3,8] — open — ral — 2026-09-12 — ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveb…
-
-**Last 5**
-
-- 2026-09-17 — *result* — Witt–Wynne SIEP projection now a catalogue product (pipeline PR #86 merged 2026-09-17, stage 7 of the inspection bundle). Ran catalogue/scripts/witt_wynne.py on the ten pulled dr1_prelim_grade_ab tiles: 10/10 rows, 0 skips, source centre r…
-- 2026-09-12 — *run* — 342796_[1,3,8] submitted: ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveband (md5 65db7d8b… on RAL = the clone's commit 88dda9d…
-- 2026-09-11 — *note* — **CPU verification PASSED and was faster than the GPU.** `342668_9` (ral, 8 CPU, JAX on the CPU backend) COMPLETED in **9:24** wall, exit 0, all eight `sersic_lens_model/{vis,nir_y,nir_j,nir_h,decam_g,decam_r,decam_i,decam_z}` searches wri…
-- 2026-09-11 — *note* — **342629 scored on the eight tiles that finished; all four 09-10 defects confirmed fixed; parity fails on rerun scatter, not ordering.** `sacct`: tasks 0,2,4,5,6,7,8,9 COMPLETED (walls 5:45–9:42, MaxRSS 7.6–10.9 GB); tasks 1 (`Tile10200729…
-- 2026-09-11 — *run* — 342668_9 submitted: ordered_rerun_catalogue_parity — CPU verification of the SED chain via hpc/batch_cpu/submit_sersic_waveband (pipeline PR #70, sbatch --array=9): tile 102008848, JAX on the CPU backend, 8 CPU; same seeded vis_lp zip as t…
-
-[full log](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_dr1_prelim.md)
-
-📋 resume euclid_dr1_prelim
-
-```
-/cortex — resume euclid_dr1_prelim: read PyAutoCortex projects/euclid_dr1_prelim.md (Now, Runs, Log) and then /mnt/c/Users/Jammy/Science/euclid_dr1_prelim/wiki/project/state.md and the assistant autolens_assistant's AGENTS.md; tell me where I left off and what I said I would do next. Submit nothing and log nothing until I say.
-```
-
-
-
### euclid_dr1 — Does the sep1 DR1 delivery reproduce the ten prelim lenses by eye
active · ral · no issue yet · [projects/euclid_dr1.md](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_dr1.md) · local `/mnt/c/Users/Jammy/Science/euclid_dr1` · RAL `/mnt/ral/jnightin/euclid_dr1`
@@ -303,37 +267,6 @@ All pre-likelihood-speedup SLaM HST rows (slam_base A100 dense/sparse, the 34269
-### euclid_sersics — Why does the lens-light Sersic index pile up at the n=5 prior edge
-
-active · ral · [euclid_strong_lens_modeling_pipeline#74](https://github.com/PyAutoLabs/euclid_strong_lens_modeling_pipeline/issues/74) · [projects/euclid_sersics.md](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_sersics.md) · local `/mnt/c/Users/Jammy/Science/euclid_sersics` · RAL `/mnt/ral/jnightin/euclid_sersics`
-
-**Now**
-
-The four Sersic-stage variants are done on the real sample (342808: 99 of 100 lenses with all four tags; task 72 lost its later variants to the walltime) and the simulated sample's vis_lp stage is done too (342911: 100 of 100). 343043 is now running the same four variants on the 100 mocks — the recovery test, where each fit's n can be read against the truth.json it was simulated from (49 of the 100 were simulated at n = 3.0 under the 4.95 prior-edge rule).
-Next: when 343043 lands, pull it with the 342911 results, scrape all four variant tags on both samples with catalogue/scripts/lens_sersic.py --unique_tag sersic_lens_model_, and put the recovered n against truth side by side with the real sample's n distributions. The human rules on whether any variant clears the n = 5 pile-up.
-
-**Runs**
-
-- 343043_[0-99] — open — ral — 2026-09-14 — euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array ta…
-
-**Last 5**
-
-- 2026-09-14 — *run* — 343043_[0-99] submitted: euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks on the simulated sample dataset/dr1_sep1_sersics_si…
-- 2026-09-14 — *run* — 342911_[0-99] finished — wall 1:59: euclid_sersics_sim_vis_lp: the vis_lp stage (initial_lens_model.py --stage=vis_lp, MGE lens + MGE source, 15 free parameters, identical to the real sample's stage 1) on the new simulated sample dataset/d…
-- 2026-09-14 — *run* — 342808_[0-99] finished — wall 18:00: euclid_sersics_variants: the Sersic stage rerun in four variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks (one lens per task), array 0-99, 4 cpu…
-- 2026-09-13 — *note* — the simulated sample behind 342911: scripts/simulator.py --from-result rebuilds each of the 100 lenses from its own sersic_lens_model_baseline/vis result of run 342808 (Sersic lens light + Sersic source), keeping that tile's real PSF stamp…
-- 2026-09-13 — *run* — 342911_[0-99] submitted: euclid_sersics_sim_vis_lp: the vis_lp stage (initial_lens_model.py --stage=vis_lp, MGE lens + MGE source, 15 free parameters, identical to the real sample's stage 1) on the new simulated sample dataset/dr1_sep1_ser…
-
-[full log](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_sersics.md)
-
-📋 resume euclid_sersics
-
-```
-/cortex — resume euclid_sersics: read PyAutoCortex projects/euclid_sersics.md (Now, Runs, Log) and then /mnt/c/Users/Jammy/Science/euclid_sersics/wiki/project/state.md and the assistant autolens_assistant's AGENTS.md; tell me where I left off and what I said I would do next. Submit nothing and log nothing until I say.
-```
-
-
-
### euclid — The original Euclid DR1 grade-AB catalogue project, superseded by euclid_dr1_prelim
dormant · both · no issue yet · [projects/euclid.md](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid.md) · local `/mnt/c/Users/Jammy/Science/euclid` · RAL `/mnt/ral/jnightin/euclid_strong_lens_modeling_pipeline`
@@ -354,10 +287,12 @@ Dormant since 2026-09-07: superseded by euclid_dr1_prelim. Three questions were
[full log](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid.md)
-2 retired
+4 retired
- [inference_programme](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/inference_programme.md) — The first inference programme, retired and restarted as autolens_inference — retired 2026-09-07: restarting from scratch: the project setup was unsatisfactory and it informed how the Cortex manages projects; RAL outputs stashed at /mnt/ral/jnightin/inference_programme_retired_2026-09-07
+- [euclid_dr1_prelim](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_dr1_prelim.md) — Refit the ten DR1-prelim tiles and rebuild the catalogue on RAL — retired 2026-09-25: retire euclid_dr1_prelim; I will move it to z_vault manually but I don't want it to eat up download time
- [slope_hierarchy](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/slope_hierarchy.md) — Hierarchical slope recovery at N=5, wrapped up and succeeded by slope_hierarchy_scale — retired 2026-09-08: wrapped up 2026-07-22 at N=5; succeeded by slope_hierarchy_scale; tree vaulted under z_projects_complete/
+- [euclid_sersics](https://github.com/PyAutoLabs/PyAutoCortex/blob/main/projects/euclid_sersics.md) — Why does the lens-light Sersic index pile up at the n=5 prior edge — retired 2026-09-25: retire euclid_sersics as well
diff --git a/projects.yaml b/projects.yaml
index 1f54d62..3d0d535 100644
--- a/projects.yaml
+++ b/projects.yaml
@@ -98,8 +98,8 @@ euclid_dr1_prelim:
assistant: autolens_assistant
witness_file: results/**/*.json
partition: both
- status: active
- note: "created by euclid-dr1-prep phase 4 (a Cortex task after migration); every path is the planned one; active 2026-09-07: phase 4 launched (jobs 342301_[0-9])"
+ status: retired
+ note: "retired 2026-09-25: retire euclid_dr1_prelim; I will move it to z_vault manually but I don't want it to eat up download time"
euclid_dr1:
remote: none
@@ -286,8 +286,8 @@ euclid_sersics:
assistant: autolens_assistant
witness_file: catalogue/**/*.csv
partition: ral
- status: active
- note: "born 2026-09-11: why the lens-light Sersic index piles up at the prior edge n = 5 in the June Euclid catalogue (61 % of 2983 fits above 4.5; 67 of the chosen 100); site copy of PyAutoLabs/euclid_strong_lens_modeling_pipeline, sample dataset/dr1_sep1_sersics (100 unanimous-success lenses on the sep1 Segmentation.zip data, join table sample/lens_map.csv); remote none until the human creates one"
+ status: retired
+ note: "retired 2026-09-25: retire euclid_sersics as well"
analytic_gaussian:
remote: none
diff --git a/projects/euclid_dr1_prelim.md b/projects/euclid_dr1_prelim.md
index 004bdf5..ddde724 100644
--- a/projects/euclid_dr1_prelim.md
+++ b/projects/euclid_dr1_prelim.md
@@ -10,22 +10,14 @@ Next: pull, bundle with output_sed, compare magnitudes (µJy fluxes) against the
## Runs
-- 342629_[0-9] — running — ral — 2026-09-10 — ordered_rerun_catalogue_parity: two-stage vis_lp(n_live=750)+vis_pix(n_live=300), array 0-9, ten dr1_prelim tiles; libraries refreshed BEFORE submit (PyAutoNerves 0e7163bc2, PyAutoFit e354dbb6b, PyAutoArray 667deed3a, PyAutoGalaxy 6640a7494, PyAutoLens 0da06de63) carrying PyAutoFit#1598/#1600/#1602 and PyAutoLens#734; pipeline main 2a8b4db (PR#65 + #67);
- 342398 output archived to output_v2_pre_refresh on laptop and RAL; purpose: confirmation rerun — vis_lp latents written (12 keys on 102005065), catalogue rebuild with 0 blank cells, same-library rerun scatter vs 342398 for the restated parity witness
-- 342648_[0,2,4-9] — open — gpu — 2026-09-11 — ordered_rerun_catalogue_parity: SED chain `hpc/batch_gpu/submit_sersic_waveband_8tiles` (a copy of `submit_sersic_waveband` differing only in `--array=0,2,4-9`, because `hpc/sync submit` passes no sbatch arguments), `PYAUTO_OUTPUT_DIR=output_sed`: Sersic lens model on VIS then every non-VIS waveband fit (nir_y/j/h on all eight tiles, plus decam_g/r/i/z on the seven non-102005065 tiles), for the eight tiles 342629 completed; tiles 1 and 3 omitted because 342629_1/_3 are still fitting `vis_lp` and have no result to seed from.
- `output_sed///initial_lens_model/vis_lp/.zip` seeded by `cp -p` from `output/` for all eight tiles before submit so the upstream stage short-circuits; `output/` untouched (16 zips before and after).
- No library refresh on RAL, no push. Purpose: produce `sersic_lens_model/{vis,}` so `catalogue/scripts/magnitudes.py` and `multi_wavelength.py` can run and the per-band magnitudes can be compared against `dr1_prelim_grade_ab_catalogue_csvs_20260623`.
- Progress at 15:40 on 2026-09-11: tasks 0, 2, 4, 5, 6 COMPLETED on `euclid-ral-gpu-2` in 13:34–20:15 each (13:43, 15:08, 15:42, 20:15, 13:34), task 7 RUNNING, task 8 PENDING. **Task `_9` (`Tile102008848RA0601376380877DECNEG0634605061157`) was cancelled while still PENDING on 2026-09-11**, on the human's authorisation, so that tile could be run instead on the CPU route as the verification run
- recorded below; it is the only array index removed and the other seven are unaffected
-- 342668_9 — open — ral — 2026-09-11 — ordered_rerun_catalogue_parity: CPU verification of the SED chain via hpc/batch_cpu/submit_sersic_waveband (pipeline PR #70, sbatch --array=9): tile 102008848, JAX on the CPU backend, 8 CPU; same seeded vis_lp zip as the GPU tiles; purpose: verify the CPU setup the main experiment will use, and give a GPU-vs-CPU wall-time and result comparison.
- Started 15:37:49 BST on `euclid-ral-compute-10-2`, 8 CPUs, partition `ral`. First-log evidence, all four pre-registered PASS criteria met inside 30 s: `JAX backend: cpu` (the script's cpu-backend guard);
- `15:37:59 … Starting non-linear search with JAX (CPU: cpu)`; `15:38:02 vis_lp … output path … initial_lens_model/vis_lp/ac98229c0d67998215f1885157f8c5f4` — the seeded hash exactly — followed by `Fit Already Completed: skipping non-linear search`; then the Sersic VIS search `15:38:14 vis … Starting new Nautilus non-linear search` and `15:38:18 … JAX jit compilation of vectorized (vmap) likelihood function complete in 1.5 seconds`.
- The submitted script is PR #70's byte-for-byte apart from the four site lines this clone's other `batch_cpu/` scripts already carry (`PROJECT_PATH` / `PYAUTO_HPC_BASE` exports, `sample=dr1_prelim_grade_ab`, the ten-tile dataset list, the real `--mail-user`); the PR file is the generic pipeline version (`sample=q1_walsmley`, one-entry dataset list, no exports), so a byte-identical copy would have resolved array index 9 to an empty dataset.
- Every element under test — the `#SBATCH` resource directives, `PYAUTO_OUTPUT_DIR=output_sed`, the cpu-backend guard, the `env` line and `--number_of_cores=$THREADS` — is unchanged from PR #70
-- 342796_[1,3,8] — open — ral — 2026-09-12 — ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveband (md5 65db7d8b… on RAL = the clone's commit 88dda9d). Tiles 1 (Tile102007299…) and 3 (Tile102007903…) had no SED chain at all because 342629_1/_3 only finished vis_lp/vis_pix on 2026-09-12 06:58 and 08:50 (walls 1-09:11:56 and 1-11:04:44); their initial_lens_model/vis_lp zips were seeded into output_sed by cp -p (md5-identical, sources intact) so stage 1 short-circuits. Tile 8 (Tile102008532…) is a resume: 342648_8 TIMEOUT at 8:00:19 on the GPU with vis, nir_y, nir_j and nir_h written but decam_g hung after its jit compile completed at 15:58:39 and produced zero Nautilus iterations in the remaining 7h45m (no checkpoint.hdf5; only pre-fit files in the search dir), leaving decam_r/i/z unrun. Nothing deleted on RAL — the completed band zips short-circuit and the empty decam_g dir restarts as a new search. Tile 0 (Tile102005065…) was NOT resubmitted: its 5 zips are a complete chain, because band membership is discovered from the cut-out FITS and that tile carries no DECam imaging (342648_0 exited 0 straight after nir_h).
## Log
+- 2026-09-25 — note — retired: retire euclid_dr1_prelim; I will move it to z_vault manually but I don't want it to eat up download time
+- 2026-09-25 — run — 342796_[1,3,8] finished: ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveband (md5 65db7d8b… on RAL = the clone's commit 88dda9d). Tiles 1 (Tile102007299…) and 3 (Tile102007903…) had no SED chain at all because 342629_1/_3 only finished vis_lp/vis_pix on 2026-09-12 06:58 and 08:50 (walls 1-09:11:56 and 1-11:04:44); their initial_lens_model/vis_lp zips were seeded into output_sed by cp -p (md5-identical, sources intact) so stage 1 short-circuits. Tile 8 (Tile102008532…) is a resume: 342648_8 TIMEOUT at 8:00:19 on the GPU with vis, nir_y, nir_j and nir_h written but decam_g hung after its jit compile completed at 15:58:39 and produced zero Nautilus iterations in the remaining 7h45m (no checkpoint.hdf5; only pre-fit files in the search dir), leaving decam_r/i/z unrun. Nothing deleted on RAL — the completed band zips short-circuit and the empty decam_g dir restarts as a new search. Tile 0 (Tile102005065…) was NOT resubmitted: its 5 zips are a complete chain, because band membership is discovered from the cut-out FITS and that tile carries no DECam imaging (342648_0 exited 0 straight after nir_h). — ended — the human's word at retirement, 2026-09-25; not read from the cluster
+- 2026-09-25 — run — 342668_9 finished: ordered_rerun_catalogue_parity: CPU verification of the SED chain via hpc/batch_cpu/submit_sersic_waveband (pipeline PR #70, sbatch --array=9): tile 102008848, JAX on the CPU backend, 8 CPU; same seeded vis_lp zip as the GPU tiles; purpose: verify the CPU setup the main experiment will use, and give a GPU-vs-CPU wall-time and result comparison. — ended — the human's word at retirement, 2026-09-25; not read from the cluster
+- 2026-09-25 — run — 342648_[0,2,4-9] finished: ordered_rerun_catalogue_parity: SED chain `hpc/batch_gpu/submit_sersic_waveband_8tiles` (a copy of `submit_sersic_waveband` differing only in `--array=0,2,4-9`, because `hpc/sync submit` passes no sbatch arguments), `PYAUTO_OUTPUT_DIR=output_sed`: Sersic lens model on VIS then every non-VIS waveband fit (nir_y/j/h on all eight tiles, plus decam_g/r/i/z on the seven non-102005065 tiles), for the eight tiles 342629 completed; tiles 1 and 3 omitted because 342629_1/_3 are still fitting `vis_lp` and have no result to seed from. — ended — the human's word at retirement, 2026-09-25; not read from the cluster
+- 2026-09-25 — run — 342629_[0-9] finished: ordered_rerun_catalogue_parity: two-stage vis_lp(n_live=750)+vis_pix(n_live=300), array 0-9, ten dr1_prelim tiles; libraries refreshed BEFORE submit (PyAutoNerves 0e7163bc2, PyAutoFit e354dbb6b, PyAutoArray 667deed3a, PyAutoGalaxy 6640a7494, PyAutoLens 0da06de63) carrying PyAutoFit#1598/#1600/#1602 and PyAutoLens#734; pipeline main 2a8b4db (PR#65 + #67); — ended — the human's word at retirement, 2026-09-25; not read from the cluster
- 2026-09-17 — result — Witt–Wynne SIEP projection now a catalogue product (pipeline PR #86 merged 2026-09-17, stage 7 of the inspection bundle). Ran catalogue/scripts/witt_wynne.py on the ten pulled dr1_prelim_grade_ab tiles: 10/10 rows, 0 skips, source centre recomputed from the fit (pre-source-key wcs.json). Verdicts: 9 tiles 2-image, 1 tile 4-image; PointSolver on the full SIE+shear models agrees on 9/10 (Tile102007903 says 2 where the full model gives 4). Positions differ from the full model by 10–40 % of b; caustic-matched projection is the default because the vector sum disagrees in PA by 15–84° on 6/10 tiles. Redshifts are placeholders (z_l 0.5, z_s 1.0) so lags are fiducial. Files: inspect/dr1_prelim_grade_ab/witt_wynne.csv + per-tile witt_wynne.in. Recorded by the agent at /prm close-out of #84; human to confirm wording.
- 2026-09-12 — run — 342796_[1,3,8] submitted: ordered_rerun_catalogue_parity: CPU SED chain for the three tiles still missing one, via a direct ssh sbatch --array=1,3,8 on hpc/batch_cpu/submit_sersic_waveband (md5 65db7d8b… on RAL = the clone's commit 88dda9d). Tiles 1 (Tile102007299…) and 3 (Tile102007903…) had no SED chain at all because 342629_1/_3 only finished vis_lp/vis_pix on 2026-09-12 06:58 and 08:50 (walls 1-09:11:56 and 1-11:04:44); their initial_lens_model/vis_lp zips were seeded into output_sed by cp -p (md5-identical, sources intact) so stage 1 short-circuits. Tile 8 (Tile102008532…) is a resume: 342648_8 TIMEOUT at 8:00:19 on the GPU with vis, nir_y, nir_j and nir_h written but decam_g hung after its jit compile completed at 15:58:39 and produced zero Nautilus iterations in the remaining 7h45m (no checkpoint.hdf5; only pre-fit files in the search dir), leaving decam_r/i/z unrun. Nothing deleted on RAL — the completed band zips short-circuit and the empty decam_g dir restarts as a new search. Tile 0 (Tile102005065…) was NOT resubmitted: its 5 zips are a complete chain, because band membership is discovered from the cut-out FITS and that tile carries no DECam imaging (342648_0 exited 0 straight after nir_h).
- 2026-09-11 — note — **CPU verification PASSED and was faster than the GPU.** `342668_9` (ral, 8 CPU, JAX on the CPU backend) COMPLETED in **9:24** wall, exit 0, all eight `sersic_lens_model/{vis,nir_y,nir_j,nir_h,decam_g,decam_r,decam_i,decam_z}` searches written for `Tile102008848…`; the GPU tiles of `342648` took 13:34–20:56 each (`_0,_2,_4-7` COMPLETED).
diff --git a/projects/euclid_sersics.md b/projects/euclid_sersics.md
index e155c4e..03d8513 100644
--- a/projects/euclid_sersics.md
+++ b/projects/euclid_sersics.md
@@ -10,10 +10,11 @@ Next: when 343043 lands, pull it with the 342911 results, scrape all four varian
## Runs
-- 343043_[0-99] — open — ral — 2026-09-14 — euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks on the simulated sample dataset/dr1_sep1_sersics_sim, array 0-99, 4 cpu / 14 GB / 18 h on partition ral, submitted 2026-09-14 ~16:50 BST; each variant restores 342911's vis_lp result (task 0 confirmed 'Fit Already Completed' at the expected hash) and fits only the Sersic stage, so the recovered n can be read against each mock's truth.json — the recovery test the sim sample was built for; submitter hpc/batch_cpu/submit_sersic_variants_sim
## Log
+- 2026-09-25 — note — retired: retire euclid_sersics as well
+- 2026-09-25 — run — 343043_[0-99] finished: euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks on the simulated sample dataset/dr1_sep1_sersics_sim, array 0-99, 4 cpu / 14 GB / 18 h on partition ral, submitted 2026-09-14 ~16:50 BST; each variant restores 342911's vis_lp result (task 0 confirmed 'Fit Already Completed' at the expected hash) and fits only the Sersic stage, so the recovered n can be read against each mock's truth.json — the recovery test the sim sample was built for; submitter hpc/batch_cpu/submit_sersic_variants_sim — ended — the human's word at retirement, 2026-09-25; not read from the cluster
- 2026-09-14 — run — 343043_[0-99] submitted: euclid_sersics_sim_variants: the same four Sersic-stage variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks on the simulated sample dataset/dr1_sep1_sersics_sim, array 0-99, 4 cpu / 14 GB / 18 h on partition ral, submitted 2026-09-14 ~16:50 BST; each variant restores 342911's vis_lp result (task 0 confirmed 'Fit Already Completed' at the expected hash) and fits only the Sersic stage, so the recovered n can be read against each mock's truth.json — the recovery test the sim sample was built for; submitter hpc/batch_cpu/submit_sersic_variants_sim
- 2026-09-14 — run — 342911_[0-99] finished — wall 1:59: euclid_sersics_sim_vis_lp: the vis_lp stage (initial_lens_model.py --stage=vis_lp, MGE lens + MGE source, 15 free parameters, identical to the real sample's stage 1) on the new simulated sample dataset/dr1_sep1_sersics_sim, array 0-99, one lens per task, 8 cpu / 64 GB / 36 h on partition ral, submitted ~14:56 BST; the sample is the 100 real lenses resimulated from their own 342808 sersic_lens_model_baseline/vis results, so the recovered n can be read against a known truth — 100 of 100 tasks COMPLETED, max wall 1:59:33; every mock has its vis_lp result at output/dr1_sep1_sersics_sim//initial_lens_model/vis_lp/.zip on the cluster (not yet pulled to the local clone).
- 2026-09-14 — run — 342808_[0-99] finished — wall 18:00: euclid_sersics_variants: the Sersic stage rerun in four variants — baseline, wide_n, central_noise, sersic_point — run sequentially inside each of 100 array tasks (one lens per task), array 0-99, 4 cpu / 14 GB / 18 h on partition ral, submitted ~12:10 BST; output tags sersic_lens_model_{baseline,wide_n,central_noise,sersic_point}; all four reuse 342696's vis_lp result; pipeline code euclid_strong_lens_modeling_pipeline#75 (open, CI green) — 99 of 100 tasks COMPLETED, per-variant walls about 10 to 45 minutes; task 72 (Tile102027146RA0860828972205DECNEG0426060761921, the 2.5 h baseline lens) was RESTARTED by SLURM and then hit the 18 h walltime, so that one lens has a partial variant set. Outputs are pulled to the local clone and all four variant tags are on disk for the other 99 lenses.