Skip to content

perf(web): keep the launches and each workspace's counts until their jobs or runs change - #232

Merged
harenome merged 1 commit into
ThinkFlowLab:mainfrom
harenome:perf/workspace-launch-counts
Nov 19, 2026
Merged

harenome merged 1 commit into
ThinkFlowLab:mainfrom
harenome:perf/workspace-launch-counts

Conversation

@harenome

@harenome harenome commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #230

What

  • workspace.heads(roots) is a cheap change key. For each root and each folder in it, it takes _head, the key load_job already uses: the stat of the folder, of its result.json and of its config.json. These are what find_job_dirs and a finished job's own records change by. It costs about three stats per folder, with no listing per job, no realpath and no JSON.
  • runs._every() keeps its last result in _EVERY. The key is each record's id, state and jobs folder, plus the heads of those folders. It keeps the result only when every job it read is finished.
  • workspaces._counts(ws, recs) keeps each closed workspace's counts in _COUNTS. The key is the workspace's roots, its job_roots, the heads of its job roots and recs, the records' ids and states, built once per request. It keeps them only when all its jobs are finished. The open workspace is computed on each load.

Both memos live in memory, so the first load after a start costs what it does today. Nothing is written to disk.

Touches

backend (nullius/workspace.py) · routes (web/runs.py, web/workspaces.py) · tests

Verified

A synthetic profile in a rootless Podman container: 100 workspaces, 2000 Harbor jobs, 200 run records. Warm is the median of requests 2 to 5; two runs each, a fresh process per page.

Page main, warm branch, warm main, first branch, first
Workspaces 0.67 / 0.63 s 0.21 / 0.21 s 6.5 / 5.4 s 5.5 / 5.1 s
Launches 0.47 / 0.45 s 0.33 / 0.33 s 2.3 / 2.4 s 2.2 / 2.4 s
  • Same output: /workspaces, /launches and /launches?ws=ws000 were saved from main and from the branch, each loaded twice so the second comes from the memo. After normalising the CSRF tokens, the asset digests and the interface digest, all six files are identical.
  • python3 app/tests/parallel.py: 3061 tests in 155 modules, OK.
  • pip install -r app/requirements.txt, then python3 app/tests/parallel.py app/ui_tests: 664 tests in 55 modules, OK.
  • New tests:
    • test_workspace.py: heads moves when a job is added, removed or has its records rewritten, and not for a change inside a trial.
    • test_run_routes.py, Workspaces: the memo is used, then the counts follow a job added, a job removed, a job's config rewritten in place, a launch added to the workspace and a run record removed.
    • test_run_routes.py, Launches: the list follows a run that ends, a job added, a job removed and a job rewritten in place.
    • With heads stubbed to return (), both route tests fail.

Not checked: a profile with runs in progress, which takes the full path as today.

Risk

  • A finished job whose only change is inside a trial folder keeps its old counts until its own records, its folder or a run record change, or the process restarts. The same holds for a root whose link is pointed elsewhere. A shortcut: comment on heads says when to key on each job's signature instead. The counts read no trial.
  • A job left with n_running_trials above 0 by a crash is never kept, so that page costs what it does today.
  • To undo, revert the commit.

…jobs or runs change

Both the Workspaces and the Launches pages re-read every run record's jobs
(find_job_dirs and parse_job over each) through runs._every on every load,
and Workspaces walked and grouped every workspace's jobs again for its run
and check counts.

workspace.heads stats each root and the head of each folder in it (the
folder, its result.json and config.json): what find_job_dirs and an ended
job's own records change by. runs._every keeps its result, and the
Workspaces page each closed workspace's counts, while that, the run
records' ids and states (and the workspace's roots) are unchanged and every
job it read had ended. Anything else is recomputed as before.
@harenome
harenome force-pushed the perf/workspace-launch-counts branch from 527598d to 333e852 Compare November 19, 2026 13:33
@harenome
harenome merged commit 32488c6 into ThinkFlowLab:main Nov 19, 2026
2 checks passed
@harenome
harenome deleted the perf/workspace-launch-counts branch November 19, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workspaces and Launches recount every workspace and reread every run's jobs on each load

1 participant