Lazily prepare execution-server function catalog

benchmark/results/perf_lazy_catalog_20260929/README.md

@@ -0,0 +1,28 @@
+# Lazy execution-server function catalog: official30 1w_1t (2026-09-29)
+
+This report accompanies the lazy catalog initialization change for [OpenHCS issue #165](https://github.com/OpenHCSDev/openhcs/issues/165). Both fresh-server runs used the same CPU-only integration checkout and native CP 4.2.8.1 reference. The control CSV is the completed direct-docstring-parser run; the new run changes only execution-server catalog initialization and its import preflight. Each observation owns a fresh ZMQ server, so `total_seconds` includes that server's startup and shutdown. The 1w_1t preset uses one well and one OpenHCS thread.
+
+All 30 cases succeeded on the final code path. Every case had a lower total, with a paired median saving of 1.171 seconds. All 110 files under the produced `results` directories were byte-identical to the control outputs.
+
+| Sum over 30 observations (seconds) | Control fresh | Lazy catalog fresh | Difference |
+| --- | ---: | ---: | ---: |
+| Server compilation jobs | 58.632 | 83.759 | +25.126 |
+| Server execution jobs | 93.115 | 89.219 | -3.896 |
+| Other measured time (`total - compile - execute`) | 160.110 | 101.289 | -58.822 |
+| Per-observation total | 311.858 | 274.266 | **-37.592** |
+
+The patch moves some registry work into compilation; the execution-job difference is run variation, not a claimed effect of this initialization change. A separately timed full CLI invocation took 296.684 seconds. That wall time includes source preparation outside the per-observation timer and has no matched control wall measurement. The native CP total-phase reference sums to 1,319.357 seconds, making its 5× reference line 263.871 seconds; the new per-observation sum remains 10.395 seconds above that line. The native total-phase and OpenHCS well-throughput timers have different preparation boundaries, so the line is a reference rather than a like-for-like end-to-end ratio.
+
+`fresh_total_breakdown.png` shows the measured time shift and native reference. `lazy_native_execution_speedup.png` and `lazy_native_execution_cdf.png` show execution-only ratios relative to native CP; the patch does not target those ratios. The completed multi-well native-relative scaling figures predate this startup-only change and remain at `/home/ts/code/projects/openhcs-benchmark-runs/official30-four-modes-native-relative-fresh-v2-20260928`.
+
+Validation: 104 focused tests passed after the final import-guard change. A live client connected in 1.795 seconds and fetched the complete 266-function catalog on demand in a further 1.616 seconds. The benchmark's ordinary ZMQ outcome and progress evidence path was used throughout.
+
+Fresh benchmark command:
+
+```bash
+OPENHCS_CPU_ONLY=true .venv/bin/python scripts/benchmark_cppipe_well_throughput.py \
+  --manifest benchmark/manifests/official30_portable_axis1.json \
+  --output-dir /path/to/output \
+  --native-summary-csv benchmark/results/perf_lazy_catalog_20260929/native_cp_summary.csv \
+  --mode 1w_1t
+```

benchmark/results/perf_lazy_catalog_20260929/control_fresh.csv

@@ -0,0 +1,31 @@
+case_name,mode_name,worker_count,well_count,compile_seconds,prepare_seconds,execute_seconds,total_seconds,wells_per_second,successful_wells,native_single_sample_execution_seconds,projected_native_execution_seconds,projected_execution_speedup,peak_memory_mb,status,memory_limit_mb,error_message,execution_route,server_lifecycle,run_input_sha256
+ExampleColocalization,1w_1t,1,1,2.2928268909454346,0.0,3.9878079891204834,11.62109541380778,0.25076433036098894,1,37.035618890076876,37.035618890076876,9.287212170474922,1344.6640625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleCometAssay,1w_1t,1,1,2.0185365676879883,0.0,2.8911311626434326,9.938480501063168,0.3458853797161092,1,13.304211451206356,13.304211451206356,4.601732229623919,1263.62109375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleFly,1w_1t,1,1,2.2488327026367188,0.0,2.1445205211639404,10.066884044092149,0.46630470080894776,1,8.094948755111545,8.094948755111545,3.774712657316053,946.58203125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleFlyURL,1w_1t,1,1,1.9677801132202148,0.0,1.019911527633667,8.41873428830877,0.9804772011158023,1,4.900778113398701,4.900778113398701,4.80510120791474,883.3203125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleHuman,1w_1t,1,1,1.923978567123413,0.0,1.0841801166534424,8.417618672829121,0.9223559670940262,1,4.420192994643003,4.420192994643003,4.0769913843161865,852.3828125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleIlluminationCorrection_Example1_AllMethod,1w_1t,1,1,1.5901720523834229,0.0,4.088348627090454,10.868056402076036,0.24459753587884892,1,946.6214662808925,946.6214662808925,231.54127806232916,1269.7265625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleIlluminationCorrection_Example1_EachMethod,1w_1t,1,1,1.5006680488586426,0.0,0.17162799835205078,6.682493566069752,5.826555163504015,1,4.54475614009425,4.54475614009425,26.48027235493273,790.26953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleIlluminationCorrection_Example2,1w_1t,1,1,1.881812334060669,0.0,2.101905107498169,9.374385974835604,0.47575887057540306,1,48.04419008316472,48.04419008316472,22.85744961167643,995.20703125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleIlluminationCorrection_Example3,1w_1t,1,1,1.8129961490631104,0.0,0.7289443016052246,7.743487971834838,1.3718469268473292,1,2.1520505752414465,2.1520505752414465,2.9522839680650055,827.015625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleImagingFlowCytometryObjectsInGrid,1w_1t,1,1,2.560279607772827,0.0,23.013134479522705,30.937172566074878,0.04345344615657676,1,61.60158259188756,61.60158259188756,2.6768010523165024,2518.06640625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleNeighbors,1w_1t,1,1,1.74615478515625,0.0,1.3547496795654297,8.286614634096622,0.7381437435148723,1,4.5466188420541584,4.5466188420541584,3.3560582524091105,901.8671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExamplePercentPositive,1w_1t,1,1,1.7290244102478027,0.0,0.42984962463378906,7.382856552023441,2.326394959288266,1,2.5050158216618,2.5050158216618,5.827656180451365,842.484375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleSpeckles,1w_1t,1,1,1.637129306793213,0.0,0.822502613067627,7.375865648966283,1.2158016085448948,1,3.179383432958275,3.179383432958275,3.86549949197166,976.84765625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleTrackObjects,1w_1t,1,1,1.716566562652588,0.0,3.287581205368042,10.32921796804294,0.3041749960022815,1,9.587221414782107,9.587221414782107,2.916193035514335,1670.55078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleTumor,1w_1t,1,1,1.6034469604492188,0.0,0.8798391819000244,7.372601582668722,1.1365713423224535,1,4.360487373080105,4.360487373080105,4.956004986801763,1072.33203125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleUntangleAndStraightenWorms,1w_1t,1,1,2.0078978538513184,0.0,1.2335312366485596,8.449991872068495,0.8106807272403965,1,3.4416790287941694,3.4416790287941694,2.790102857990879,888.4375,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleUntangleWorms,1w_1t,1,1,2.0958409309387207,0.0,2.062998056411743,10.002663507126272,0.48473143098318805,1,4.618807300925255,4.618807300925255,2.238881072413095,934.63671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleUntangleWormsBrightField,1w_1t,1,1,2.1711883544921875,0.0,3.5632073879241943,11.496679156087339,0.2806460278986362,1,7.377188352867961,7.377188352867961,2.0703786082924758,1045.79296875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleVitra,1w_1t,1,1,1.8910613059997559,0.0,1.2376892566680908,8.538714677095413,0.8079572433972968,1,4.122247557155788,4.122247557155788,3.330599772880831,960.13671875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleWoundHealing,1w_1t,1,1,1.5049092769622803,0.0,2.042593240737915,8.341959646902978,0.4895737340434634,1,4.702593941241503,4.702593941241503,2.3022664755037696,1272.14453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleYeastColonies,1w_1t,1,1,2.1904988288879395,0.0,2.0774261951446533,10.033609306905419,0.48136487464016453,1,12.096297012176365,12.096297012176365,5.8227324948764725,1121.2578125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+ExampleYeastPatches,1w_1t,1,1,2.182415246963501,0.0,2.6008615493774414,10.576405676081777,0.3844879787005065,1,5.990533472970128,5.990533472970128,2.30328810636001,1177.07421875,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_3d_monolayer,1w_1t,1,1,2.876032829284668,0.0,10.151128053665161,18.547642110381275,0.09851121911903579,1,14.058660756796598,14.058660756796598,1.3849358103329792,1453.578125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_advanced_segmentation_final,1w_1t,1,1,3.3548684120178223,0.0,10.546051740646362,20.040677363052964,0.09482221636992562,1,36.253466514870524,36.253466514870524,3.437634046032906,1147.62890625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_quality_control,1w_1t,1,1,1.3814642429351807,0.0,0.6194014549255371,7.224075731821358,1.6144618196291087,1,4.787729676347226,4.787729676347226,7.729606765167826,855.1953125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_beginner_segmentation_final,1w_1t,1,1,2.4305648803710938,0.0,6.888291358947754,14.83460831688717,0.14517388244633697,1,16.074153264053166,16.074153264053166,2.333547236380058,1046.05078125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_pixel_based_classification,1w_1t,1,1,1.6967191696166992,0.0,0.8674452304840088,7.88223431725055,1.1528105347262438,1,4.208680049050599,4.208680049050599,4.851810697837696,1035.89453125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_translocation_final,1w_1t,1,1,1.854893445968628,0.0,0.8097820281982422,7.803559121210128,1.2349002141045178,1,3.004684013314545,3.004684013314545,3.710484931358554,931.90625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp4_supplement_combine_objects,1w_1t,1,1,1.3613243103027344,0.0,0.14962530136108398,6.642139645293355,6.683361643407789,1,1.8334413380362093,1.8334413380362093,12.253551514069457,825.0390625,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818
+cp_tutorial_translocation_start,1w_1t,1,1,1.402390956878662,0.0,0.25930023193359375,6.627473204862326,3.856533380410157,1,2.1760300607420504,2.1760300607420504,8.39193256602766,866.1328125,success,,,ordinary-zmq-outcomes-v1,fresh-per-observation,dbed67c7d17c4e678119e0e634ed4eb2a7c6b0d51b7b581196590fed56cc8818